I am testing a little sound library called clunk (http://sourceforge.net/projects/clunk/).
I built that library for visual studio 11 and linked it in my visual studio project. When I try the test.cpp I am getting an assertion thrown by msvcr110d.dll.
Does it have to do with my runtime librarie settings: It is "Multithreaded-Debug-DLL (/MDd)" ?
In cmakelist.txt in clunk I added following line of code:
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MDd")
I am still getting the message that there are problems with pointer allocation. Why that?
You're probably allocating memory on one side of an application/library boundary and freeing it on the other. That's difficult to get right and likely best avoided.
You must ensure that memory is returned to the very same allocator that allocated it. Here are a few patterns to avoid this problem:
Instead of the library allocating memory for a returned structure, have the application do it. Then the application can free the structure.
Let the library allocate memory for a structure, but instead of the application freeing it, have the application call a special free function. So if there's a 'getFoo' function in the library that returns an allocated structure, have a 'freeFoo' function that releases that structure. This ensures the library returns the structure to its own allocator.
Have the library use statically allocated structures that are valid until some particular next call into the library.
Give the library a 'setAlloctor' function and pass it a pointer to malloc and free from the application. This way, the library will always use the application's allocator.
Give the library a getAllocator function that returns pointers to the malloc and free functions the library is using. This way, the application can get memory from the library's allocator (for the library to possibly free) or return memory to the library's allocator (that the library allocated).
Take a look at the code that's generating the assertion and see if it can be modified to use one of these patterns. It's possible, for example, that you're just calling delete on a pointer to an object you got from the library when you should be using a special destructor function provided by the library.
Related
Is it safe to call delete on a base-class pointer to a heap object allocated by a dynamically loaded library? Both that lib and the client were built by the same compiler (GCC).
It is safe to delete the pointer if and only if
The destructor of the base class is virtual.
The pointer was returned by new (not new[], not malloc, not mmap, ...)
You are the sole owner of the pointer. In other words: If no other piece of code (within or out of the dll) is going to use or destroy the pointed object.
As far as the standard is concerned, the library must have linked with the same version of runtime library that define the allocation functions. Same requirement applies to separate object files that are linked together statically.
Technically multiple different implementation of memory allocation functions violates the one definition rule, but a C++ implementation that extends the language with standard libraries may possibly extend the language to allow multiple different implementations for those functions. If that is the case for the implementation that you use, then it might not be safe to deallocate memory allocated by the shared library unless you can prove that same versions were used for both the library and the code that deletes the object.
Whether it is safe or not, it is a bad idea to provide an API that returns resources (such as dynamic memory), but not API that takes care of releasing those resources.
It depends on the version of the compiler and flags that dynamic library was compiled with.
The library's call to new grabs the object from the heap controlled by the library.
Your call to delete places the object back to the heap controlled by your executable.
This may or may not be the same heap managed by the same library.
The safe way is to wrap the delete in the library call, like fopen/fclose.
Yes, if it was allocated with new, and if the library (or any other code) doesn't try to use the object after the delete.
I'm using an exe which dynamically loads a DLL. A function in the DLL allocates memory on the heap and passes a pointer to that memory to the exe.
A senior says that it is bad practice to do so. He says that if I ever have to share memory between the exe and the DLL, the exe has to allocate memory and pass a pointer to that to the DLL, and not vice versa. Is this true? Why?
EDIT: In my case, I planned to allocate and deallocate memory inside the DLL itself.
Here are some reasons for having the caller supply a pointer:
Symmetric ownership semantics. This is already explained by several other answers.
Avoids mismatching the allocator and deallocator. As mentioned in Aesthete's answer, if the DLL allocates a pointer and returns it, the caller must call the corresponding deallocator to free it. This is not necessarily trivial: the DLL might be statically linked against one version of, say, malloc/free while the .exe is linked against a different version of malloc/free. (For example, the DLL could be using release versions while the .exe is using specialized debug versions.)
Flexibility. If the DLL is meant for general use, having the caller allocate the memory gives the caller more options. Suppose the caller doesn't want to use malloc and instead wants memory to be allocated from some specific memory pool. Maybe it's a case where the caller could provide a pointer to memory allocated on the stack. If the DLL allocated the memory itself, the caller does not have any of these options.
(The second and third points also mostly can be addressed by having the .exe supply an allocator/deallocator for the DLL code to use.)
One of the basic idea behind the design patterns is ownership. The idea is - one who creates a resource (and thereby holds it in the pointer) should be responsible for deleting the resource. This will ensure the sanctity of the design and in longer life of the projects, its developer can see lesser bugs.
So now it in your case, the DLL can be attached by any executable and he can try to delete the resource, which may cause future problem. So I think it has been suggested for vice-versa and I would say it as a sound advice.
I have seen this issue before, and it is caused by the DLL and exe linking differently to the CRT (static, dynamic MT etc).
I you're going to pass a pointer to memory between DLL and executable, they should both provide some sort of Free() functionality to free memory from their respective heaps.
Generally, the heap (The One Heap) belongs to the process, and it does not matter where you allocate from, so this will work just fine, except when it doesn't.
Therefore, the claim that it is "bad practice" is valid. There are few things that are worse than something that works fine, except when it doesn't.
The worst part about it is that when everything blows up, it's not immediately obvious what's wrong, and you can easily run into an issue without knowing. It may be something as easy as linking the a particular version of the CRT into your DLL. Or someone in your team might have created a separate heap for some reason. Or, some other reason that isn't immediately obvious which caused another heap being created.
What makes the free-from-wrong-heap situation so vicious is that you don't generally know what will happen, or when (or if someone will notice).
You may get a NULL return value from a heap function or an exception. Your application might be prepared for either of them or it might not be (be honest, you always check return values, don't you?). It might crash immediately upon freeing, or you might just silently leak memory and run out of address space (or memory) minutes or hours later, and nobody will know why. Or, something else.
Thus, what you see may not at all be in (apparent) correlation to what caused the problem.
I only want to point out that passing an allocator to the DLL to allocate memory is perfectly safe and is the standard model used by the C++ std library itself. In this case, the allocation is done by the DLL via a method passed from the caller, avoiding to pass pointers, and avoiding the issue of linking to different implementations of malloc().
the exe and the dll may have different heaps. when either try to free memory allocated by the other the free fails and there is a leak.
only if both, the exe and the dll use CRT dynamically with the same version of the CRT they use the same heap.
so it is a very good advice to do allocation and free in the same binary.
As #kumar_m_kiran already pointed out, it's about ownership issue, however I want to note, that #aleguna is correct too, therefore, imho, correct rule is "Either allocate and deallocate same memory in DLL or in EXE, but not in both".
I'd argue that it is generally bad practice to hand around raw pointers. In case of a dll, it should return a smart-pointer that has a custom deleter that appropriately handles the cleanup (options are std::unique_ptr, std::shared_ptr or boost::shared_ptr in order of preference).
In fact, it is probably safer to use std::weak_ptr or boost::weak_ptr - these require you to check each time you want to access the resource, because the dll might have been unloaded in the meantime.
It's not necessarily bad practice but it's dangerous practice (and probably bad). You need to carefully think about who is responsible for freeing the memory. The exe will probably not be able to (or should not be able to) free the DLL's memory directly itself so presumably will be passing this pointer back to the DLL at some later date. So now we're passing a pointer back and forth between an EXE and a DLL which isn't nice.
I'd say no, it's not "bad practice". From my experience, you just need to be careful that you release that pointer in the right memory space. I've built a graphics engine that allocated assets in multiple DLLs (each DLL represented a mini game); using a shared_ptr (at the time it was Boost, however, I'm sure the C++11 (or newer) std::shared_ptr supports the same semantics). I would supply a function that would delete the memory in the right space. The main concern, at this point, is that you ensure you free your shared_ptrs before you free the DLL. I can't recall now, but we might have used a list of shared_ptrs owned by the DLL/DLL wrapper, and all other uses of the pointer were through a weak_ptr TO that shared_ptr.
I came across below note in the book Windows via C-C++ by Jeffrey Richter and Christophe Nasarre.
Examine the following code:
V
OID EXEFunc() {
PVOID pv = DLLFunc();
// Access the storage pointed to by pv...
// Assumes that pv is in EXE's C/C++ run-time heap
free(pv);
}
PVOID DLLFunc() {
// Allocate block from DLL's C/C++ run-time heap
return(malloc(100));
}
So, what do you think? Does the preceding code work correctly? Is the block allocated by the
DLL's function freed by the EXE's function? The answer is: maybe. The code shown does not
give you enough information. If both the EXE and the DLL link to the DLL C/C++ run-time
library, the code works just fine. However, if one or both of the modules link to the static C/C++
run-time library, the call to free fails.
I'm not able to understand why call to free would fail here when linking modules with static C runtime.
Can someone explain why free fails?
Found similar question here:
Memory Allocation in Static vs Dynamic Linking of C Runtime
But I've same doubt here as MrPhilTx:
Wouldn't all of the heaps be in the same address space?
Thanks!
When both your DLL and EXE are staticly linked to the C runtime the two runtimes simply don't know about each other. So both the EXE and DLL get their own copy of the runtime, their own heap and heap metadata. Neither side knows about the others Metadata and there is no safe way to update the data when you free memory. You end up with inconstant metadata and things will eventually fail (and if your very lucky, it will fail right away).
What this means is that you end up with at least two heaps in your process, and each heap has it's own rules and metadata. There is no way for the EXE to know the exact way the DLL allocates memory so there is no way for it to free it.
As for why you can get away with sharing a heap when everything is dynamically linked, that's easy, there is only one copy of the C Runtime DLL in the process, so if every DLL links against it they will all be calling the same code with the same metadata.
You can't allocate memory from one allocator and free it with another. Different allocators use different internal implementations and the result of giving a memory block to an allocator that didn't allocate it is unpredictable.
So unless you know for a fact that two sections of code are using the same allocator, you can't allocate memory in one section of code and free it in the other. The usual solution is to ensure the same unit both allocates and frees the memory. In your example, the DLL could offer a "free" function that the main code could call into instead of calling its own free function which frees to its own allocator.
So do this instead:
OID EXEFunc() {
PVOID pv = DLLFunc();
// Access the storage pointed to by pv...
// Assumes that pv is in EXE's C/C++ run-time heap
DLLFreeFunc(pv);
}
...
PVOID DLLFunc() {
// Allocate block from DLL's C/C++ run-time heap
return(malloc(100));
}
DLLFreeFunc(PVOID x) {
free(x);
}
On Linux, a program uses the brk and sbrk system calls to request extra data pages from the kernel. sbrk returns an address pointing to the data segment that can be used by your program.
malloc and free use the data segment returned by brk and sbrk by turning it into a heap. The heaps is a large block of memory in the current porcess's space which small blocks of memory can be requested and returned as required. It is important to note that many calls to malloc and free will make no system calls.
Now when malloc and free want to make use of the heap they need to get a pointer to the heap. This pointer is stored in a separate data segment called static data and is allocated when the application loads. In order to insure that different DLLs (or shared libraries on linux) to no clash with eachother, each DLL has its own static data sections.
Now let us assume that both the dll and the executable are statically linked to their own libraries. In such a case the dll and the executable will have pointers to a different heaps and is such event both dll and executable must free their own memory.
However on linux both the dll and the executable will access malloc and free through a common DLL (libc.so on linux). In such a case, since both the dll and executable are effectively accessing libc's heap, the executable can safely free memory allocated by the dll.
In any event it is good practice for the dll to provide its own free function. This if nothing else documents that the pointer returned by DLLFunc needs to be freed.
I imagine this is true on Windows as well.
The code critically depends on the implementation of malloc and free. A good implementation has no issues, a poor one will indeed fail. It's definitely easier to create a working DLL implementation of malloc and free, but it's far from impossible to do so in a static library.
A trivial example would be a static library which forwards the calls directly to GlobalAlloc and GlobalFree.
I know that if your DLL static links against a different version of the runtime then it creates its own heap. It will also if it is instructed to make a heap. Under these circumstances, it is unsafe for the DLL to delete what the exe allocated. In what cases does this NOT apply (as in, it is safe for the DLL to delete what the exe allocated)? Is it safe if both the exe and the DLL static link against the same runtime library?
Thanks
basically is there a way where whoever allocates it could just do addEvent(new DerivedEvent(), FunctorDestroyClass());
I may be reading more into your question than is there, but if you are wanting to know how you can allocate and free memory across DLL boundaries, then you might use something like the following:
#define DLLMemAlloc( size ) HeapAlloc( GetProcessHeap(), 0, size )
#define DLLMemFree( mem ) HeapFree( GetProcessHeap(), 0, mem )
That might be safer (a partial attempt at future-proofing). Relying on various build options to guarantee the safety of allocating and freeing across boundaries might lead to problems.
And (also not part of the question), you might re-think whether it is really necessary to be able to do this. It seems like there may be a design flaw if one DLL has to allocate something that another DLL (or executable) has to free.
DLL will get it's own memory manager if you link run-time library statically. You have 3 options: link run-time dynamically, always allocate and deallocate in the same place (either DLL or executable, providing forwarding if necessary), or use 3rd party memory allocator that takes care of this problem.
As in topic.. I've found something like this in one application. In main C application we have declaration:
void* buff = NULL;
and later there is a call:
ReadData(&buff);
SaveToFile(buff);
SaveToFile() is a C function from main function.
ReadData(void* * ) is a C++ function from external dll. In this function memory for buffer is allocated by malloc function and filed with data.
So here is my question: is it correct?
All modules in a running process share the same address space (doesn't care whether you're Windows or Linux or whatever actually, it's a common principle). However, be careful: reading or writing from module A to a buffer owned by module B is fine - but freeing the buffer is probably bad.
On Windows, it depends on the runtime library the application is linked against. If it is not the DLL runtime ('multithreaded dll'), every module maintains its own copy of the heap manager. Thus, the module that allocated a memory area must also be responsible for destroying it because only its own heap manager knows about it. If you follow this guideline, you won't run into problems (linking against the DLL runtime avoids the problem because all modules deal with the same heap manager residing somewhere in msvXXXnnn.dll, but gives rise to other issues).
Edit:
ReadData(void* * ) is a C++ function from external dll. In this function memory for buffer is allocated by malloc function and filed with data.
That might run into the aforementioned allocator issue. Either add another function to that DLL (FreeData) which is explicitly responsible for freeing up the buffer (as proposed by Neil Butterworth) and just calls its own free(). Or you add a DLL function to query the size of the buffer, allocate it upfront and pass it to ReadData (that's the cleanest choice imo).
If both the DLL and the main executable were linked with the same C runtime, this is OK and you can call free() on the pointer to release it. However, a better idea is in the DLL to provide a function FreeData( void * ) which releases the data. In this way all memory management is done in the context of the DLL.
It's safe. However, you should always check:
if the same allocator is used for both allocation and deallocation
who is responsible for freeing (so there are no surprises)
watch out for any kind of automatic memory magement (if it's plain C/C++ then it's no problem).
It depends on the intention of the design and users the library is directed to. A better way is to take a buffer of some fixed size and fill it and return. But, you should be careful while freeing the buffer. It is better to call the free function (if any) provided by the third party DLL itself rather than calling free from your main.
In case of windows, if your third party DLL is using a different heap and if your application is using a different heap, it might lead to undefined behaviour. For Ex: if your third party DLL is build using VC8 and your application is built using VC6, then if you free the memory allocated by your external DLL, it will lead to problems.
Yes, this is correct. Memory in a process is equally accessible by all modules (EXE and DLLs) in that process.
Yes, there is no problem with this.