I know that if your DLL static links against a different version of the runtime then it creates its own heap. It will also if it is instructed to make a heap. Under these circumstances, it is unsafe for the DLL to delete what the exe allocated. In what cases does this NOT apply (as in, it is safe for the DLL to delete what the exe allocated)? Is it safe if both the exe and the DLL static link against the same runtime library?
Thanks
basically is there a way where whoever allocates it could just do addEvent(new DerivedEvent(), FunctorDestroyClass());
I may be reading more into your question than is there, but if you are wanting to know how you can allocate and free memory across DLL boundaries, then you might use something like the following:
#define DLLMemAlloc( size ) HeapAlloc( GetProcessHeap(), 0, size )
#define DLLMemFree( mem ) HeapFree( GetProcessHeap(), 0, mem )
That might be safer (a partial attempt at future-proofing). Relying on various build options to guarantee the safety of allocating and freeing across boundaries might lead to problems.
And (also not part of the question), you might re-think whether it is really necessary to be able to do this. It seems like there may be a design flaw if one DLL has to allocate something that another DLL (or executable) has to free.
DLL will get it's own memory manager if you link run-time library statically. You have 3 options: link run-time dynamically, always allocate and deallocate in the same place (either DLL or executable, providing forwarding if necessary), or use 3rd party memory allocator that takes care of this problem.
Related
I am testing a little sound library called clunk (http://sourceforge.net/projects/clunk/).
I built that library for visual studio 11 and linked it in my visual studio project. When I try the test.cpp I am getting an assertion thrown by msvcr110d.dll.
Does it have to do with my runtime librarie settings: It is "Multithreaded-Debug-DLL (/MDd)" ?
In cmakelist.txt in clunk I added following line of code:
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MDd")
I am still getting the message that there are problems with pointer allocation. Why that?
You're probably allocating memory on one side of an application/library boundary and freeing it on the other. That's difficult to get right and likely best avoided.
You must ensure that memory is returned to the very same allocator that allocated it. Here are a few patterns to avoid this problem:
Instead of the library allocating memory for a returned structure, have the application do it. Then the application can free the structure.
Let the library allocate memory for a structure, but instead of the application freeing it, have the application call a special free function. So if there's a 'getFoo' function in the library that returns an allocated structure, have a 'freeFoo' function that releases that structure. This ensures the library returns the structure to its own allocator.
Have the library use statically allocated structures that are valid until some particular next call into the library.
Give the library a 'setAlloctor' function and pass it a pointer to malloc and free from the application. This way, the library will always use the application's allocator.
Give the library a getAllocator function that returns pointers to the malloc and free functions the library is using. This way, the application can get memory from the library's allocator (for the library to possibly free) or return memory to the library's allocator (that the library allocated).
Take a look at the code that's generating the assertion and see if it can be modified to use one of these patterns. It's possible, for example, that you're just calling delete on a pointer to an object you got from the library when you should be using a special destructor function provided by the library.
I'm using an exe which dynamically loads a DLL. A function in the DLL allocates memory on the heap and passes a pointer to that memory to the exe.
A senior says that it is bad practice to do so. He says that if I ever have to share memory between the exe and the DLL, the exe has to allocate memory and pass a pointer to that to the DLL, and not vice versa. Is this true? Why?
EDIT: In my case, I planned to allocate and deallocate memory inside the DLL itself.
Here are some reasons for having the caller supply a pointer:
Symmetric ownership semantics. This is already explained by several other answers.
Avoids mismatching the allocator and deallocator. As mentioned in Aesthete's answer, if the DLL allocates a pointer and returns it, the caller must call the corresponding deallocator to free it. This is not necessarily trivial: the DLL might be statically linked against one version of, say, malloc/free while the .exe is linked against a different version of malloc/free. (For example, the DLL could be using release versions while the .exe is using specialized debug versions.)
Flexibility. If the DLL is meant for general use, having the caller allocate the memory gives the caller more options. Suppose the caller doesn't want to use malloc and instead wants memory to be allocated from some specific memory pool. Maybe it's a case where the caller could provide a pointer to memory allocated on the stack. If the DLL allocated the memory itself, the caller does not have any of these options.
(The second and third points also mostly can be addressed by having the .exe supply an allocator/deallocator for the DLL code to use.)
One of the basic idea behind the design patterns is ownership. The idea is - one who creates a resource (and thereby holds it in the pointer) should be responsible for deleting the resource. This will ensure the sanctity of the design and in longer life of the projects, its developer can see lesser bugs.
So now it in your case, the DLL can be attached by any executable and he can try to delete the resource, which may cause future problem. So I think it has been suggested for vice-versa and I would say it as a sound advice.
I have seen this issue before, and it is caused by the DLL and exe linking differently to the CRT (static, dynamic MT etc).
I you're going to pass a pointer to memory between DLL and executable, they should both provide some sort of Free() functionality to free memory from their respective heaps.
Generally, the heap (The One Heap) belongs to the process, and it does not matter where you allocate from, so this will work just fine, except when it doesn't.
Therefore, the claim that it is "bad practice" is valid. There are few things that are worse than something that works fine, except when it doesn't.
The worst part about it is that when everything blows up, it's not immediately obvious what's wrong, and you can easily run into an issue without knowing. It may be something as easy as linking the a particular version of the CRT into your DLL. Or someone in your team might have created a separate heap for some reason. Or, some other reason that isn't immediately obvious which caused another heap being created.
What makes the free-from-wrong-heap situation so vicious is that you don't generally know what will happen, or when (or if someone will notice).
You may get a NULL return value from a heap function or an exception. Your application might be prepared for either of them or it might not be (be honest, you always check return values, don't you?). It might crash immediately upon freeing, or you might just silently leak memory and run out of address space (or memory) minutes or hours later, and nobody will know why. Or, something else.
Thus, what you see may not at all be in (apparent) correlation to what caused the problem.
I only want to point out that passing an allocator to the DLL to allocate memory is perfectly safe and is the standard model used by the C++ std library itself. In this case, the allocation is done by the DLL via a method passed from the caller, avoiding to pass pointers, and avoiding the issue of linking to different implementations of malloc().
the exe and the dll may have different heaps. when either try to free memory allocated by the other the free fails and there is a leak.
only if both, the exe and the dll use CRT dynamically with the same version of the CRT they use the same heap.
so it is a very good advice to do allocation and free in the same binary.
As #kumar_m_kiran already pointed out, it's about ownership issue, however I want to note, that #aleguna is correct too, therefore, imho, correct rule is "Either allocate and deallocate same memory in DLL or in EXE, but not in both".
I'd argue that it is generally bad practice to hand around raw pointers. In case of a dll, it should return a smart-pointer that has a custom deleter that appropriately handles the cleanup (options are std::unique_ptr, std::shared_ptr or boost::shared_ptr in order of preference).
In fact, it is probably safer to use std::weak_ptr or boost::weak_ptr - these require you to check each time you want to access the resource, because the dll might have been unloaded in the meantime.
It's not necessarily bad practice but it's dangerous practice (and probably bad). You need to carefully think about who is responsible for freeing the memory. The exe will probably not be able to (or should not be able to) free the DLL's memory directly itself so presumably will be passing this pointer back to the DLL at some later date. So now we're passing a pointer back and forth between an EXE and a DLL which isn't nice.
I'd say no, it's not "bad practice". From my experience, you just need to be careful that you release that pointer in the right memory space. I've built a graphics engine that allocated assets in multiple DLLs (each DLL represented a mini game); using a shared_ptr (at the time it was Boost, however, I'm sure the C++11 (or newer) std::shared_ptr supports the same semantics). I would supply a function that would delete the memory in the right space. The main concern, at this point, is that you ensure you free your shared_ptrs before you free the DLL. I can't recall now, but we might have used a list of shared_ptrs owned by the DLL/DLL wrapper, and all other uses of the pointer were through a weak_ptr TO that shared_ptr.
I came across below note in the book Windows via C-C++ by Jeffrey Richter and Christophe Nasarre.
Examine the following code:
V
OID EXEFunc() {
PVOID pv = DLLFunc();
// Access the storage pointed to by pv...
// Assumes that pv is in EXE's C/C++ run-time heap
free(pv);
}
PVOID DLLFunc() {
// Allocate block from DLL's C/C++ run-time heap
return(malloc(100));
}
So, what do you think? Does the preceding code work correctly? Is the block allocated by the
DLL's function freed by the EXE's function? The answer is: maybe. The code shown does not
give you enough information. If both the EXE and the DLL link to the DLL C/C++ run-time
library, the code works just fine. However, if one or both of the modules link to the static C/C++
run-time library, the call to free fails.
I'm not able to understand why call to free would fail here when linking modules with static C runtime.
Can someone explain why free fails?
Found similar question here:
Memory Allocation in Static vs Dynamic Linking of C Runtime
But I've same doubt here as MrPhilTx:
Wouldn't all of the heaps be in the same address space?
Thanks!
When both your DLL and EXE are staticly linked to the C runtime the two runtimes simply don't know about each other. So both the EXE and DLL get their own copy of the runtime, their own heap and heap metadata. Neither side knows about the others Metadata and there is no safe way to update the data when you free memory. You end up with inconstant metadata and things will eventually fail (and if your very lucky, it will fail right away).
What this means is that you end up with at least two heaps in your process, and each heap has it's own rules and metadata. There is no way for the EXE to know the exact way the DLL allocates memory so there is no way for it to free it.
As for why you can get away with sharing a heap when everything is dynamically linked, that's easy, there is only one copy of the C Runtime DLL in the process, so if every DLL links against it they will all be calling the same code with the same metadata.
You can't allocate memory from one allocator and free it with another. Different allocators use different internal implementations and the result of giving a memory block to an allocator that didn't allocate it is unpredictable.
So unless you know for a fact that two sections of code are using the same allocator, you can't allocate memory in one section of code and free it in the other. The usual solution is to ensure the same unit both allocates and frees the memory. In your example, the DLL could offer a "free" function that the main code could call into instead of calling its own free function which frees to its own allocator.
So do this instead:
OID EXEFunc() {
PVOID pv = DLLFunc();
// Access the storage pointed to by pv...
// Assumes that pv is in EXE's C/C++ run-time heap
DLLFreeFunc(pv);
}
...
PVOID DLLFunc() {
// Allocate block from DLL's C/C++ run-time heap
return(malloc(100));
}
DLLFreeFunc(PVOID x) {
free(x);
}
On Linux, a program uses the brk and sbrk system calls to request extra data pages from the kernel. sbrk returns an address pointing to the data segment that can be used by your program.
malloc and free use the data segment returned by brk and sbrk by turning it into a heap. The heaps is a large block of memory in the current porcess's space which small blocks of memory can be requested and returned as required. It is important to note that many calls to malloc and free will make no system calls.
Now when malloc and free want to make use of the heap they need to get a pointer to the heap. This pointer is stored in a separate data segment called static data and is allocated when the application loads. In order to insure that different DLLs (or shared libraries on linux) to no clash with eachother, each DLL has its own static data sections.
Now let us assume that both the dll and the executable are statically linked to their own libraries. In such a case the dll and the executable will have pointers to a different heaps and is such event both dll and executable must free their own memory.
However on linux both the dll and the executable will access malloc and free through a common DLL (libc.so on linux). In such a case, since both the dll and executable are effectively accessing libc's heap, the executable can safely free memory allocated by the dll.
In any event it is good practice for the dll to provide its own free function. This if nothing else documents that the pointer returned by DLLFunc needs to be freed.
I imagine this is true on Windows as well.
The code critically depends on the implementation of malloc and free. A good implementation has no issues, a poor one will indeed fail. It's definitely easier to create a working DLL implementation of malloc and free, but it's far from impossible to do so in a static library.
A trivial example would be a static library which forwards the calls directly to GlobalAlloc and GlobalFree.
As in topic.. I've found something like this in one application. In main C application we have declaration:
void* buff = NULL;
and later there is a call:
ReadData(&buff);
SaveToFile(buff);
SaveToFile() is a C function from main function.
ReadData(void* * ) is a C++ function from external dll. In this function memory for buffer is allocated by malloc function and filed with data.
So here is my question: is it correct?
All modules in a running process share the same address space (doesn't care whether you're Windows or Linux or whatever actually, it's a common principle). However, be careful: reading or writing from module A to a buffer owned by module B is fine - but freeing the buffer is probably bad.
On Windows, it depends on the runtime library the application is linked against. If it is not the DLL runtime ('multithreaded dll'), every module maintains its own copy of the heap manager. Thus, the module that allocated a memory area must also be responsible for destroying it because only its own heap manager knows about it. If you follow this guideline, you won't run into problems (linking against the DLL runtime avoids the problem because all modules deal with the same heap manager residing somewhere in msvXXXnnn.dll, but gives rise to other issues).
Edit:
ReadData(void* * ) is a C++ function from external dll. In this function memory for buffer is allocated by malloc function and filed with data.
That might run into the aforementioned allocator issue. Either add another function to that DLL (FreeData) which is explicitly responsible for freeing up the buffer (as proposed by Neil Butterworth) and just calls its own free(). Or you add a DLL function to query the size of the buffer, allocate it upfront and pass it to ReadData (that's the cleanest choice imo).
If both the DLL and the main executable were linked with the same C runtime, this is OK and you can call free() on the pointer to release it. However, a better idea is in the DLL to provide a function FreeData( void * ) which releases the data. In this way all memory management is done in the context of the DLL.
It's safe. However, you should always check:
if the same allocator is used for both allocation and deallocation
who is responsible for freeing (so there are no surprises)
watch out for any kind of automatic memory magement (if it's plain C/C++ then it's no problem).
It depends on the intention of the design and users the library is directed to. A better way is to take a buffer of some fixed size and fill it and return. But, you should be careful while freeing the buffer. It is better to call the free function (if any) provided by the third party DLL itself rather than calling free from your main.
In case of windows, if your third party DLL is using a different heap and if your application is using a different heap, it might lead to undefined behaviour. For Ex: if your third party DLL is build using VC8 and your application is built using VC6, then if you free the memory allocated by your external DLL, it will lead to problems.
Yes, this is correct. Memory in a process is equally accessible by all modules (EXE and DLLs) in that process.
Yes, there is no problem with this.
Consider this case:
dll = LoadDLL()
dll->do()
...
void do() {
char *a = malloc(1024);
}
...
UnloadDLL(dll);
At this point, will the 1k allocated in the call to malloc() be available to the host process again?
The DLL is statically linking to the CRT.
Memory used by a process as tracked by the OS is applicable to the full process and not specific to a DLL.
Memory is given to the program in chunks by the OS, called heaps
The heap managers (malloc / new etc) further divide up the chunks and hands it out to requesting code.
Only when a new heap is allocated does the OS detect an increase in memory.
When a DLL is statically linked to the C Run time library (CRT), a private copy of CRT with the CRT functions that the DLL's code invokes is compiled and put into the DLL's binary. Malloc is also inclued in this.
This private copy of malloc will be invoked whenever the code present inside the statically linked DLL tries to allocate memory.
Consequently, a private heap visible only to this copy of malloc, is acquired from the OS by this malloc and it allocates the memory requested by the code within this private heap.
When the DLL unloads, it unloads its private heap, and this leak goes unnoticed as the entire heap is returned back to the OS.
However If the DLL is dynamically linked, the memory is allocated by a single shared version of malloc, global to all code that is linked in the shared mode.
Memory allocated by this global malloc, comes out of a heap which is also the heap used for all other code that is linked in the dynamic aka shared mode and hence is common. Any leaks from this heap therefore becomes a leak which affects the whole process.
Edit - Added descriptions of the linking scenario.
You can't tell. This depends on the implementation of your static and dynamic CRT. It may even depend on the size of the allocation, as there are CRTs that forward large allocations to the OS, but implement their own heap for small allocations.
The problem with a CRT that leaks is of course that it leaks. The problem with a CRT that does not leak is that the executable might reasonable expect to use the memory, as malloc'ed memory should remain usable until free is called.
From MSDN Potential Errors Passing CRT Objects Across DLL Boundaries
Each copy of the CRT library has a
separate and distinct state. As such,
CRT objects such as file handles,
environment variables, and locales are
only valid for the copy of the CRT
where these objects are allocated or
set. When a DLL and its users use
different copies of the CRT library,
you cannot pass these CRT objects
across the DLL boundary and expect
them to be picked up correctly on the
other side.
Also, because each copy of the CRT
library has its own heap manager,
allocating memory in one CRT library
and passing the pointer across a DLL
boundary to be freed by a different
copy of the CRT library is a potential
cause for heap corruption.
Hope this helps.
Actually, the marked answer is incorrect. That right there is a leak. While it is technically feasible for each dll to implement its own heap, and free it on shutdown, most "runtime" heaps - static or dynamic - are wrappers around the Win32 process heap API.
Unless one has taken specific care to guarantee that this is not the case, the dll will leak the allocation per load,do,unload cycle.
One could do a test and see if there are memory leaks. You run a simple test 30 times allocating 1 MB each time. You should figure that out quite quickly.
One thing is for sure. If you allocated memory in the DLL you should also free that memory there (in the DLL).
For example you should have something like this (simple but intuitive pseudocode):
dll = DllLoad();
ptr = dll->alloc();
dll->free(ptr);
DllUnload(dll);
This must be done because the DLL has a different heap than the original process (that loads the dll).
No, you do not leak.
If you mix dll models (static, dynamic) then you can end up with a memory error if you allocate memory in a dll, that you free in a different one (or freed in the exe)
This means that the heap created by the statically-linked CRT is not the same heap as a different dll's CRT.
If you'd linked with the dynamic version of the CRT, then you'd have a leak as the heap is shared amongst all dynamically-linked CRTs. It means you should always design your apps to use the dynamic CRTs, or ensure you never manage memory across a dll boundary (ie if you allocate memory in a dll, always provide a routine to free it in the same dll)