Sorry for the poor title, but hopefully the description makes it clearer.
At the momemnt I have one main application which is built together with other libraries (like libpng, libvorbis,etc.). I'm trying to add libtheoraplayer to the main application, but I keep running into problems:
1) Linking to a pre-built library of llibtheoraplayer and including the appropriate header files gives me an error saying that pushMemoryManager cannot be found (part of our custom memory manager)
2) Building the library together with the main application results in a linker error "error LNK2005: "void * __cdecl operator new(unsigned int,void *)" (??2#YAPAXIPAX#Z) already defined in win32Mem.obj"
I'm not quite sure where to start debugging this. The main application does not support STL, and I started by changing references to STL in the libtheoraplayer to our own STL replacements, but I'm still getting error 2 as above after finding the offending files and changing them.
Any wild ideas?
Silence the linker error (convert to warning) with /FORCE and pray that there are no cross-library allocations ending up using different allocators.
It's rather difficult to replace allocation functions on Windows (NT or CE), because:
Dynamic symbols are loaded from specific libraries, so you will replace which allocator is used in your code, but not which is used in libraries you link dynamically. If you don't link dynamically and silence the linker error with /FORCE, it will do the right thing and override the allocators from standard library just fine.
With dynamic linking if code living in one library allocates and code living in another library frees, there will be mismatch in allocators used and thus probably crash. Unfortunately it may easily happen even if both functions are defined by the same library, but one of them is a template (so the instance lives in the library refering to it) and the other is not (so it lives in the library defining it).
They have some non-standard entry points to the allocator and sometimes use them from other code. We used to use duma library and actually managed to override both C (malloc/realloc/free) and C++ (operator new and operator delete) allocators, but hit a wall when we started using iostreams, where they allocate with the standard function, but free with something like __debug_free or the other way 'round.
On a side-note, the same is absolutely trivial on Linux, because in GNU libc malloc, realloc and free call the real allocator through pointers that you can easily override (operator new and operator delete just call malloc and free respectively basically everywhere).
Related
I am writing a XLL (using XLW library) that calls a DLL function. This DLL function will get a vector reference, modify the vector and return it by argument.
I have a VS10 solution with several c++ projects, some DLLs and a XLL that will call DLL functions from excel. I compiled everything using VS10 compiler, with _HAS_ITERATOR_DEBUGGING=0 and _CRT_SECURE_NO_WARNINGS and used same runtime library (/MDd) for all projects.
I also had to rebuild the XLW library to comply with _HAS_ITERATOR_DEBUGGING=0 that I have to use in my projects.
When calling the xll_function I was getting Heap Corruption errors and couldn't figure out why.
After I tried resizing my vector before calling the dll function the error was gonne. That is, I can call the function and get the right vector returned by argument and no heap corruptions.
Could someone shed some light on this?
As I am new to using DLLs I'm not sure if this should happen or if I am doing something wrong.
As you can see in the code below, the dll function will try to resize forwards and that is the point that I think is generating the heap errors.
I'm trying to understand why this happens and how this resizing and allocation works for dlls. Maybe I can't resize a vector allocated in another heap.
** Code below - the first function is a static method in a class from a dll project and the second function is exported to the XLL.
void dll_function(double quote, const std::vector<double>& drift, const std::vector<double>& divs, std::vector<double>& forwards)
{
size_t size = drift.size();
forwards.resize(size);
for( size_t t = 0; t < size; t++)
{
forwards[t] = (quote - divs[t]) * drift[t];
}
}
MyArray xll_function(double quote, const MyArray& drift, const MyArray& divs)
{
// Resizing the vector before passing to function
std::vector<double> forwards(drift.size());
dll_function(quote, drift, divs, forwards);
return forwards;
}
To pass references to std::vector or other C++ collections across DLL boundaries, you need to do following.
Use same C++ compiler for both modules, and same version of the compiler.
In project settings, set up same value to the setting General / Platform Toolset.
In project settings, set up C/C++ / Code Generation / Runtime Library value to “Multi-threaded DLL (/MD)”, or Multi-threaded Debug DLL (/MDd) for debug config. If one of the projects have a dependency which requires static CRT setting, sorry you’re out of luck, it won’t work.
Use same configuration in both sides: if you’ve built debug version of the DLL, don’t link with release version of the consuming EXE. Also don’t change preprocessor defines like _ITERATOR_DEBUG_LEVEL or _SCL_SECURE_NO_WARNINGS, or if you did, change them to the same value for both projects.
The reason for these complications, C++ doesn’t have standardized ABI. The memory layout of std::vector and other classes changes based on many things. Operators new and delete are also in C++ standard library, i.e. you can’t allocate memory with C++ in one module, free in different one.
If you can’t satisfy these conditions, there’re several workarounds, here’s a nice summary: https://www.codeproject.com/Articles/28969/HowTo-Export-C-classes-from-a-DLL
i think is very stupid, but I can't understand,
for example, I want use Windows API like GetWindowsDirectory, GetSystemInfo and etc... I can use Api directly or calling through GetProcAddress :
Method 1
here I can calling APIs with LoadLibrary and GetProcAddress :
#include <windows.h>
typedef UINT (WINAPI *GET_WIN_DIR)(LPWSTR lpBuffer, UINT size);
TCHAR infoBuffer[MAX_PATH + 1];
HINSTANSE dllLoad = LoadLibrary("Kernel32.dll");
GET_WIN_DIR function = (GET_WIN_DIR )GetProcAddress(dllLoad, "GetWindowsDirectoryW");
int result = function2(infoBuffer, MAX_PATH + 1);
Method 2
here I can calling directly APIs like GetWindowsDirectory :
#include <windows.h>
TCHAR infoBuffer[MAX_PATH + 1];
GetWindowsDirectory(infoBuffer, MAX_PATH);
I have 2 question :
What is the difference between the two methods above?
is it load Library impact on executable file?(.exe)(I did test, but it'snot changed)
Microsoft calls
Method 1 ... Explicit linking
Method 2 ... Implicit linking
From MSDN Linking an Executable to a DLL:
Implicit linking is sometimes referred to as static load or load-time dynamic linking. Explicit linking is sometimes referred to as dynamic load or run-time dynamic linking.
With implicit linking, the executable using the DLL links to an import library (.lib file) provided by the maker of the DLL. The operating system loads the DLL when the executable using it is loaded. The client executable calls the DLL's exported functions just as if the functions were contained within the executable.
With explicit linking, the executable using the DLL must make function calls to explicitly load and unload the DLL and to access the DLL's exported functions. The client executable must call the exported functions through a function pointer.
An executable can use the same DLL with either linking method. Furthermore, these mechanisms are not mutually exclusive, as one executable can implicitly link to a DLL and another can attach to it explicitly.
In our projects we use implicit linking in any common case.
We use the explicit linking exceptionally in two situations:
for plug-in DLLs which are loaded explicitly at run-time
in special cases where the implicit linked function is not the right one.
The 2nd case may happen if we use DLLs which themselves link to distinct versions of other DLLs (e.g. from Microsoft). This is, of course, a bit critical. Actually, we try to prevent the 2nd case.
No, I don't think it's stupid at all. If you don't understand, ask. That's what this site is for. Maybe you'll get downvoted, who knows, but not by me. Goes with the territory. No pain, no gain, ask me how I know.
Anyway, the main purpose of what #Scheff calls 'explicit linking' is twofold:
If you're not sure whether the the DLL you want to use to is going to be present on the machine at runtime (although you can also use /DELAYLOAD for this which is a lot more convenient).
If you're not sure if the function you want to call is present in (for example) all versions of Windows on which you want your application to run.
Regard point 1, an example of this might be reading or writing WMA files. Some older versions of Windows did not include WMA support by default (we're going back quite a long way here) and if you implicitly link to WMA.DLL then your application won't start up if it's not present. Using explicit linking (or /DELAYLOAD) lets you check for this at runtime and put up a polite message if it's missing while still allowing the rest of your app to function as normal.
As for point 2, you might, for example, want to make use of the LoadIconWithScaleDown() function because it generally produces a nicer scaled icon than LoadIcon(). However, if you just blindly call it then, again, your app wont run on XP because XP doesn't support it, so you would instead call it conditionally, via GetProcAddress(), if it's available and fall back to LoadIcon() if not.
Okay, so to round off, what's the deal with /DELAYLOAD? Well, this is a linker switch that lets you tell the linker which DLL's are optional for your app. Once you've done that, then you can do something like this:
if (LoadIconWithScaleDown)
LoadIconWithScaleDown (...);
else
LoadIcon (...);
And that is pretty neat.
So I hope you can now see that this question is really about the utility of explicit linking versus the inconvenience involved (all of which goes way anyway with /DELAYLOAD). What goes on under the covers is, for me, less interesting.
And yes, the end result, in terms of the way the program behaves, is the same. Explicit linking or delay loading might involve a small (read: tiny) performance overhead but I really wouldn't worry about that, and delay loading involves a few potential 'gotchas' (which won't affect most normal mortals) as detailed here.
I have the problem with passing by reference std::string to function in dll.
This is function call:
CAFC AFCArchive;
std::string sSSS = std::string("data\\gtasa.afc");
AFCER_PRINT_RET(AFCArchive.OpenArchive(sSSS.c_str()));
//AFCER_PRINT_RET(AFCArchive.OpenArchive(sSSS));
//AFCER_PRINT_RET(AFCArchive.OpenArchive("data\\gtasa.afc"));
This is function header:
#define AFCLIBDLL_API __declspec(dllimport)
AFCLIBDLL_API EAFCErrors CAFC::OpenArchive(std::string const &_sFileName);
I try to debug pass-by-step through calling the function and look at _sFileName value inside function.
_sFileName in function sets any value(for example, t4gs..\n\t).
I try to detect any heap corruption, but compiler says, that there is no error.
DLL has been compiled in debug settings. .exe programm compiled in debug too.
What's wrong?? Help..!
P.S. I used Visual Studio 2013. WinApp.
EDIT
I have change header of func to this code:
AFCLIBDLL_API EAFCErrors CAFC::CreateArchive(char const *const _pArchiveName)
{
std::string _sArchiveName(_pArchiveName);
...
I really don't know, how to fix this bug...
About heap: it is allocated in virtual memory of our process, right? In this case, shared virtual memory is common.
The issue has little to do with STL, and everything to do with passing objects across application boundaries.
1) The DLL and the EXE must be compiled with the same project settings. You must do this so that the struct alignment and packing are the same, the members and member functions do not have different behavior, and even more subtle, the low-level implementation of a reference and reference parameters is exactly the same.
2) The DLL and the EXE must use the same runtime heap. To do this, you must use the DLL version of the runtime library.
You would have encountered the same problem if you created a class that does similar things (in terms of memory management) as std::string.
Probably the reason for the memory corruption is that the object in question (std::string in this case) allocates and manages dynamically allocated memory. If the application uses one heap, and the DLL uses another heap, how is that going to work if you instantiated the std::string in say, the DLL, but the application is resizing the string (meaning a memory allocation could occur)?
C++ classes like std::string can be used across module boundaries, but doing so places significant constraints on the modules. Simply put, both modules must use the same instance of the runtime.
So, for instance, if you compile one module with VS2013, then you must do so for the other module. What's more, you must link to the dynamic runtime rather than statically linking the runtime. The latter results in distinct runtime instances in each module.
And it looks like you are exporting member functions. That also requires a common shared runtime. And you should use __declspec(dllexport) on the entire class rather than individual members.
If you control both modules, then it is easy enough to meet these requirements. If you wish to let other parties produce one or other of the modules, then you are imposing a significant constraint on those other parties. If that is a problem, then consider using more portable interop. For example, instead of std::string use const char*.
Now, it's possible that you are already using a single shared instance of the dynamic runtime. In which case the error will be more prosaic. Perhaps the calling conventions do not match. Given the sparse level of detail in your question, it's hard to say anything with certainty.
I encountered similar problem.
I resolved it synchronizing Configuration Properties -> C / C++ settings.
If you want debug mode:
Set _DEBUG definition in Preprocessor Definitions in both projects.
Set /MDd in Code Generation -> Runtime Library in both projects.
If you want release mode:
Remove _DEBUG definition in Preprocessor Definitions in both projects.
Set /MD in Code Generation -> Runtime Library in both projects.
Both projects I mean exe and dll project.
It works for me especially if I don't want to change any settings of dll but only adjust to them.
Out of curiosity, what exactly happens when an application compiled with the MSVCR is loaded, resp. how does the loader of Windows actually initialize the CRT? For what I have gathered so far, when the program as well as all the imported libraries are loaded into memory and all relocations are done, the CRT startup code (_CRT_INIT()?) initializes all global initializers in the .CRT$XC* sections and calls the user defined main() function. I hope this is correct so far.
But let's assume, for the sake of explanation, a program that is not using the MSVCR (e.g. an application built with Cygwin GCC or other compilers) tries to load a library at runtime, requiring the CRT, using a custom loader/runtime linker, so no LoadLibrary() involved. How would the loader/linker has to handle CRT initialization? Would it have to manually initialize all "objects" in said sections, does it have to do something else to make the internal wiring of the library work properly, or would it have to just call _CRT_INIT() (which unpractically is defined in the runtime itself and not exported anywhere as far as I am aware). Would this mix-up even work in any way, assuming that the non-CRT application and the CRT-library would not pass any objects, exceptions and things the like between them?
I would be very interested in finding out, because I can't quite make out what the CRT has an effect on the actual loading process...
Any information is very appreciated, thanks!
The entrypoint for an executable image is selected with the /ENTRY linker option. The defaults it uses are documented well in the MSDN Library article. They are the CRT entrypoint.
If you want to replace the CRT then either pick the same name or use the /ENTRY option explicitly when you link. You'll also need /NODEFAULTLIB to prevent it from linking the regular .lib
Each library compiled against the C++ runtime is calling _DllMainCRTStartup when it's loaded. _DllMainCRTStartup calls _CRT_INIT, which initializes the C/C++ run-time library and invokes C++ constructors on static, non-local variables.
The PE format contains an optional header that has a slot called 'addressofentrypoint', this slot calls a function that will call _DllMainCRTStartup which fires the initialization chain.
after _DllMainCRTStartup finishes the initialization phase it will call your own implemented DllMain() function.
When you learn about programming, someone will tell you that "the first thing that happens is that the code runs in main. But that's a bit like when you learn about atoms in School, they are fairly well organized and operate acorrding to strict rules. If you later go to a Nuclear/Particle Physics class at university, those simple/strict rules are much more detailed and don't always apply, etc.
When you link a C or C++ program the CRT contains some code something like this:
start()
{
CRT_init();
...
Global_Object_Constructors();
...
exit(main());
}
So the initialization is done by the C runtime library itself, BEFORE it calls your main.
A DLL has a DllMain that is executed by LoadLibrary() - this is responsible for initializing/creating global objects in the DLL, and if you don't use LoadLibrary() [e.g. loading the DLL into memory yourself] then you would have to ensure that objects are created and initialized.
I have a native DLL that is a plug-in to a different application (one that I have essentially zero control of). Everything works just great until I link with an additional .lib file (links my DLL to another DLL named ABQSMABasCoreUtils.dll). This file contains some additional API from the parent application that I would like to utilize. I haven't even written any code to use any of the functions exported but just linking in this new DLL is causing problems. Specifically, I get the following error when I attempt to run the program:
The application failed to initialize properly (0xc0000025). Click on OK to terminate the application.
I believe I have read somewhere that this is typically due to a DllMain function returning FALSE. Also, the following message is written to the standard output:
ERROR: Memory allocation attempted before component initialization
I am almost 100% sure this error message is coming from the application and is not some type of Windows error.
Looking into this a little more (aka flailing around and flipping every switch I know of) I linked with /MAP turned on and found this in the resulting .map file:
0001:000af220 ??3#YAXPEAX#Z 00000001800b0220 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af226 ??2#YAPEAX_K#Z 00000001800b0226 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af22c ??_U#YAPEAX_K#Z 00000001800b022c f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
0001:000af232 ??_V#YAXPEAX#Z 00000001800b0232 f ABQSMABasCoreUtils_import:ABQSMABasCoreUtils.dll
If I undecorate those names using "undname" they give the following (same order):
void __cdecl operator delete(void * __ptr64)
void * __ptr64 __cdecl operator new(unsigned __int64)
void * __ptr64 __cdecl operator new[](unsigned __int64)
void __cdecl operator delete[](void * __ptr64)
I am not sure I understand how anything from ABQSMABasCoreUtils.dll can exist within this .map file or why my DLL is even attempting to load ABQSMABasCoreUtils.dll if I don't have any code that references this DLL. Can anyone help me put this information together and find out why this isn't working? For what it's worth I have confirmed via "dumpbin" that the parent application imports ABQSMABasCoreUtils.dll, so it is being loaded no matter what. I have also tried delay loading this DLL in my DLL but that did not change the results.
EDIT
I have double checked and all files involved are 64 bit.
I just had exactly the same problem. This is an issue with the Abaqus API rather than with the loading of DLLS.
I think it is because the Abaqus API overrides the new and delete functions (as you seem to have noticed). If you call new or delete in your program before initializing the Abaqus API, such as by calling odb_initializeAPI(); then you get the
ERROR: Memory allocation attempted before component initialization
error message and the program crashes.
In my program, calling odb_initializeAPI(); before the first new resolved the problem.
Well, sure you'll reference the imports of that library. Hard to write a C++ program without using the new or delete operator. Dealing with 3rd party software that thinks it needs to override the CRT version of those operators is hard enough, impossible when it won't allow you to call them until it thinks the time is right. Abandon all hope or seek help from the vendor.
One of the possible reason of an error during loading of ABQSMABasCoreUtils.dll is that some dependency module (inclusive delayed load DLLs) could not be found. Use Dependency Walker (see http://www.dependencywalker.com/) to examine all dependencies of ABQSMABasCoreUtils.dll.
I have two suggestions:
Verify that you can load ABQSMABasCoreUtils.dll with respect of LoadLibrary. You don't need call any function from ABQSMABasCoreUtils.dll. Usage of LoadLibrary I don't see as the end solution. It' s only a diagnostic test. With the test you can verify either you have some general problem of loading ABQSMABasCoreUtils.dll in your program or you have some kind of process initialization problem.
If loading of ABQSMABasCoreUtils.dll with respect of LoadLibrary will failed, then use profiling feature of Dependency Walker to protocol of all calls done during loading of ABQSMABasCoreUtils.dll. One other way would be usage of Process Monitor (see http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) to trace what file and registry operations will be done during loading of ABQSMABasCoreUtils.dll.
If LoadLibrary is not failed, then you have really an initialization problem of DLLs. Typically the problem exist if a DLL inside of DllMain try use a function from another DLL which is not yet initialized (not yet returns from DllMain). Before one start diagnostic of this problem, we should try to exclude a more simple problems with LoadLibrary.
The ABQSMABasCoreUtils.dll looks like it's importing 64-bit functions. Is your dll also 64-bit? If not, then that's the problem - you cannot mix DLLs compiled for different architectures in the same process.