I want performant run-time memory metrics so I wrote a memory tracker based on overloading new & delete. It basically lets walk your allocations in the heap and analyze everything about them - fragmentation, size, time, number, callstack, etc. But, it has 2 fatal flaws: It can't track memory allocated in other DLLs and when ownership of objects is passed to DLLs or vice versa crashes ensue. And some smaller flaws: If a user uses malloc instead of new it's untracked; or if a user makes a class defined new/delete.
How can I eliminate these flaws? I think I must be going about this fundamentally incorrectly by overloading new/delete, is there a better way?
The right way to implement this is to use detours and a separate tool that runs in its own process. The procedure is roughly the following:
Create memory allocation in a remote process.
Place there code of a small loader that will load your dll.
Call CreateRemoteThread API that will run your loader.
From inside of the loaded dll establish detours (hooks, interceptors) on the alloc/dealloc functions.
Process the calls, track activity.
If you implement your tool this way, it will be not important from what DLL or directly from exe the memory allocation routines are called. Plus you can track activities from any process, not necessarily that you compiled yourself.
MS Windows allows checking contents of the virtual address space of the remote process. You can summarize use of virtual address space that was collected this way in a histogram, like the following:
From this picture you can see how many virtual allocation of what size are existing in your target process.
The picture above shows an overview of the virtual address space usage in 32-bit MSVC DevEnv. Blue stripe means a commited piece of emory, magenta stripe - reserved. Green is unoccupied part of the address space.
You can see that lower addresses are pretty fragmented, while the middle area - not. Blue lines at high addresses - various dlls that are loaded into the process.
You should find out the common memory management routines that are called by new/delete and malloc/free, and intercept those. It is usually malloc/free in the end, but check to make sure.
On UNIX, I would use LD_PRELOAD with some library that re-implemented those routines. On Windows, you have to hack a little bit, but this link seems to give a good description of the process. It basically suggests that you use Detours from Microsoft Research.
Passing ownership of objects between modules is fundamentally flawed. It showed up with your custom allocator, but there are plenty of other cases that will fail also:
compiler upgrades, and recompiling only some DLLs
mixing compilers from different vendors
statically linking the runtime library
Just to name a few. Free every object from the same module that allocated it (often by exporting a deletion function, such as IUnknown::Release()).
Related
It is well known that the freeing of heap memory must be done with the same allocator as the one used to allocate it. This is something to take into account when exchanging heap allocated objects across DLL boundaries.
One solution is to provide a destructor for each object, like in a C API: if a DLL allows creating object A it will have to provide a function A_free or something similar 1.
Another related solution is to wrap all allocations into shared_ptr because they store a link to the deallocator 2.
Another solution is to "inject" a top-level allocator into all loaded DLLs (recursively) 3.
Another solution is to just to not exchange heap allocated objects but instead use some kind of protocol 4.
Yet another solution is to be absolutely sure that the DLLs will share the same heap, which should (will?) happen if they share compatible compilations options (compiler, flags, runtime, etc.) 5 6.
This seems quite difficult to guarantee, especially if one would like to use a package manager and not build everything at once.
Is there a way to check at runtime that the heaps are actually the same between multiple DLLs, preferably in a cross-platform way?
For reliability and ease of debugging this seems better than hoping that the application will immediately crash and not corrupt stuff silently.
I think the biggest problem here is your definition of "heap". That assumes there is a unique definition.
The problem is that Windows has HeapAlloc, while C++ typically uses "heap" as the memory allocated by ::operator new. These two can be the same, distinct, a subset, or partially overlapping.
With two DLL's, both might be written in C++ and use ::operator new, but they both could have linked their own unique versions. So there might be multiple answers to the observation in the previous paragraph.
Now let's assume for an example that one DLL has ::operator new forward directly to HeapAlloc, but the other counts allocations before calling HeapAlloc. Clearly the two can't be mixed formally, because the count kept by the second allocator would be wrong. But the code is so simple that both news are probably inlined. So at assembly level you just have calls to HeapAlloc.
There's no way you can detect this at runtime, even if you would disassemble the code on the fly (!) - the inlined counter increment instruction is not distinguishable from the surrounding code.
I would like to allocate a set amount of memory for the program upon initialization so that other programs cannot steal memory from it. Essentially, I would like to create a Heap for my program (without having to program a heap module all for myself).
If this is not possible, can you please refer me to a heap module that I can import into my project?
Using C++17.
Edit: More specifically, I am trying to for example specify that it is only allowed to malloc 4MB of data for example. If it tries to allocate anymore, it should throw an error.
What you ask is not possible with the features provided by ISO C++.
However, on most common platforms, reserving physical RAM is possible using platform-specific extensions. For example, Linux provides the function mlock and Microsoft Windows provides the function VirtualLock. But, in order to use these functions, you must either
know which memory pages the default allocator is using for memory allocation, which can get messy and complicated, or
use your own implementation of a memory allocator, so that it can itself call mlock/VirtualLock whenever it receives memory from the operating system.
Your own implementation of a memory allocator could be as simple as forwarding all memory allocation request to the operating system's kernel, for example using mmap on Linux or VirtualAlloc on Windows. However, this has the disadvantage that the granularity of all memory allocation requests is the size of a memory page, which on most systems is at least 4096 bytes. This means that even very small memory allocation requests of a few bytes will actually take 4096 bytes of memory. This would be a big waste of memory. Also, in your question, you stated that you wanted to preallocate a certain amount of memory when you start your application, so that you can use that memory later to satisfy smaller memory allocation requests. This cannot be done using the method described above.
Therefore, you may want to consider using a "proper" memory allocator implementation, which is able to satisfy several smaller allocation requests using a single memory page. See this list on Wikipedia for a list of common implementations.
That said, what you describe may be an XY problem, depending on what operating system you are using. For example, in contrast to Windows, Linux will typically overcommit memory. This means that the Linux kernel will allow applications to allocate more memory than is actually available, on the assumption that most applications will not use all the memory they request. Therefore, a call to std::malloc or new will seldom fail on Linux (but it is still possible, depending on the configuration). Instead, under low memory conditions, the Linux OOM killer (out of memory killer) will start killing processes that are taking up large amounts of memory, in order to free up memory and to keep the system running.
For this reason, the methods described above are likely to work on Microsoft Windows, but on Linux, they could be counterproductive, as they would make your process more likely to fall prey to the OOM killer.
However, even if you are able to accomplish what you want using the methods described above, I generally don't recommend that you use these methods, as this behavior is unfair towards the other processes in the system. Generally, you should leave the task of deciding which process gets (fast) physical memory and which process gets (slow) swap space to the operating system, as the operating system can do a better job of fairly distributing its resources among its processes.
If you want to force actual allocation of memory pages to your process, there's no way around managing your own memory.
In C++, the canonical way to do this would be to write an implementation for operator new() and operator delete() (the global ones!) which are responsible to perform the actual memory allocation. The function signatures are:
void* operator new (size_t size);
void operator delete (void *pointer);
and you'll need to include the #include <new> header.
Your implementation can do its work via one of three possible routes:
It allocates the memory using the C function malloc(), and immediately touches each memory page by writing a value to it. This forces the system kernel to actually back the memory region with real memory.
It allocates the memory using malloc(), and proceeds to call mlockall(). This is the nuclear option for when you absolutely must avoid all paging, including paging of code segments and shared libraries.
It asks the kernel directly for some chunks of memory using mmap() and proceeds to lock them into RAM via mlock(). The effect is similar to the previous option, but it is targeted only at the memory you allocated for your operator new() implementation.
The first method works independent of the OS kernel, the other two assume a Linux kernel.
With GCC, you can perform the memory allocation before main() is called by using the __attribute__((constructor)).
Writing such a memory allocator is not rocket science. It's not even a lot of code if done right. I once wrote an operator new()/operator delete() implementation that fits into 170 lines, including all my special features, comments, empty lines, and the license declaration. It's really not that hard.
I would like to allocate a set amount of memory for the program upon initialization so that other programs cannot steal memory from it.
Why would you want to do that?
it is not your business to decide if your program is more important than others !
Imagine your program running in parallel with some printing utility driving the printer. This is a common occurrence: I have downloaded some long PDF document (e.g. several hundred pages, like the C++ standard n3337), and I want to print it on paper to study it in a train, an airplane, at home and annotate it with a pencil and paper. The printing is likely to last more than an hour, and require computing resources (e.g. on Linux some CUPS printer driver converting PDF to PCL). During the printing, I could use your program.
If I am a user of your program, you have decided (at my place) that printing that document is less important for me than using your program (while the printer is slowly spitting pages).
Leave the allocation and management of memory to the operating system of your user.
There are of course important exceptions to that common sense rule. A typical medical robot used in neurosurgery has some embedded software with constraints different of a web server software. See also this draft report. For Linux, read Advanced Linux Programming then syscalls(2).
More specifically, I am trying to for example specify that it is only allowed to malloc 4MB of data for example.
This is really simple. Some OSes provide the ability to limit resources (on Linux, see setrlimit(2)...). Write your own malloc routine, above operating system specific primitives such as (on Linux) mmap(2). See also this, this and that answers (all focused on Linux; adapt them to your particular operating system). You probably can find open source implementations of malloc (on github or gitlab) for your particular operating system. For Linux, look here, then study the source code of glibc or musl-libc. In C++, study the source code of GCC or Clang (probably ::operator new is using malloc)
I am using _CrtDumpMemoryLeaks to identify memory leaks in our software. We are using a third party library in a multi-threaded application. This library does have memory leaks and therefore in our tests we want to identify those that ours and discard those we do not have any control over.
We use continuous integration so new functions/algorithms/bug fixes get added all the time.
So the question is - is there a safe way of identifying those leaks that are ours and those that are the third parties library. We though about using allocation numbers but is that safe?
In a big application I worked on the global new and delete operators were overwritten (eg. see How to properly replace global new & delete operators) and used private heaps (eg. HeapCreate). Third party libraries would use the process heap and thus the allocation would be clearly separated.
Frankly I don't think you can get far with allocation numbers. Using explicit separate heaps for app/libraries (and maybe even have separate per-component heaps within your own app) would be much more manageable. Consider that you can add your own app specific header to each allocated block and thus enable very fancy memory tracking. For example capture the allocation entire call-stack would be possible, for debugging. Enable per-component accounting. Etc etc.
You might be able to do this using Mirosoft's heap debugging library without using any third-party solutions. Based on what I learned from a previous question here, you should just make sure that all memory allocated in your code is allocated through a call to _malloc_dbg where the second argument is set to _CLIENT_BLOCK. Then you can set a callback function with _CrtSetDumpClient, and that callback will only receive information about the client blocks that were allocated, not the other ones.
You can easily use the preprocessor to convert all the calls to malloc and free to actually call their debugging versions (e.g. _malloc_dbg); just look at how it's done in crtdbg.h which comes with Visual Studio.
The tricky part for me would be figuring out how to override the new and delete operators to call debugging functions like _malloc_dbg. It might be hard to find a solution where only the news and deletes in your own code are affected, and not in the third-party library.
You may want to use DebugDiag Tool provided by Microsoft. For complete information about the tool
we can refer : http://www.microsoft.com/en-sg/download/details.aspx?id=40336
DebugDiag can be used for identifying various issue. We can follow the steps to track down the
leaks(ours and third party module):
Configure the DebugDiag under Rule Type "Native(non .NET) Memory and Handle Leak".
Now Re-run the application for sometime and capture the dump files. We can also configure
the DebugDiag to capture the dump file after specified interval.
Now we can open/analyze the captured dump file using DebugDiag under the "Performance Analyzers".
Once analysis is complete, DebugDiag would automatically generate the report and would give you the
modules/DLL information where leak is possible(with probability). After this we get the information about the modules from DebugDiag tool, we can concentrate on that particular module by doing static code analysis. If modules belongs to third party DLL, we can share the DebugDiag report to them. In addition to this, if you run/attach your application with appropriate PDB file, DebugDiag also provides the call stack from where chances of memory leak is possible.
These information were very useful in the past while debugging memory leak on windows based application. Hopefully above information would be useful.
The answer would REALLY depend on the actual implementation of the third partly library. Does it only leak a consistent number of items, or does that depend on, for example, the number of threads, what functions are used within the library, or some such? When are the allocations made?
Even then if it's a consistent number of leaks regardless of library usage, I'd be hesitant to use this the allocation number. By all means, give it a try. If all the allocations are made very early on, and they don't depend on any of "your" code, then it could work - and it is a REALLY simple thing. But try adding for example a static std::vector<int>(100) to see if memory allocations in static variables are affecting the allocation number... If it does, this method is probably doomed (unless you have very strict rules on static objects).
Using a separate heap (with new/delete operators replaced) would be the correct solution, as this can probably be expanded to gather other statistics too [like number of allocations made, to detect parts of the code that makes excessive allocations - of course, this has to be analysed based on what the code actually does].
The newer Doug Lea malloc's include the mspace abstraction. An mspace is a separate heap. In our couple 100K NCSL application, we use a dozen different mspace's for different parts of the code. We use allocators to have STL containers allocate memory from the right mspace.
Some of the benefits
3rd party code does not use mspaces, so their allocation (and leaks) do not mix with ours
We can look at the memory usage of each mspace to see which piece of code might have memory leaks
Any memory corruption is contained within one mspace thus limiting the amount of code we need to look at for debugging.
I was reading that all a process's memory is released by the OS when the process terminates (by any means) so negating the need to call every dtor in turn.
Now my question is how does the memory of a DLL or SO relate to clean up of alloc'd memory?
I ask because I will probably end up using a Java and/or C# to call into a C++ DLL with some static C style functions which will allocate the C++ objects on the heap. Sorry if I got carried away with the heap vs stack thread, I feel I have lost sight of the concept of _the_ heap (ie only one).
Any other potential pitfalls for memory leaks when using libraries?
The library becomes part of the process when it is loaded. Regarding tidy up of memory, handles, resources etc., the system doesn't distinguish whether they were created in the executable image or the library.
There is nothing for you to worry about. The operating system's loader takes care of this.
In general, shared libraries will be made visible to your process's address space via memory mapping (all done by the loader), and the OS keeps track of how many processes still need a given shared library. State data that is needed separately per process is typically handled by copy-on-write, so there's no danger that your crypto library might accidentally be using another process's key :-) In short, don't worry.
Edit. Perhaps you're wondering what happens if your library function calls malloc() and doesn't clean up. Well, the library's code becomes part of your process, so it is really your process that requests the memory, and so when your process terminates, the OS cleans up as usual.
I'm writing a 32-bit .NET program with a 2 stage input process:
It uses native C++ via C++/CLI to parse an indefinite number files into corresponding SQLite databases (all with the same schema). The allocations by C++ 'new' will typically consume up to 1GB of the virtual address space (out of 2GB available; I'm aware of the 3GB extension but that'll just delay the issue).
It uses complex SQL queries (run from C#) to merge the databases into a single database. I set the cache_size to 1GB for the merged database so that the merging part has minimal page faults.
My problem is that the cache in stage 2 does not re-use the 1GB of memory allocated by 'new' and properly released by 'delete' in stage 1. I know there's no leak because immediately after leaving stage 1, 'private bytes' drops down to a low amount like I'd expect. 'Virtual size' however remains at about the peak of what the C++ used.
This non-sharing between the C++ and SQLite cache causes me to run out of virtual address space. How can I resolve this, preferably in a fairly standards-compliant way? I really would like to release the memory allocated by C++ back to the OS.
This is not something you can control effectively from the C++ level of abstraction (in other words you cannot know for sure if memory that your program released to the C++ runtime is going to be released to the OS or not). Using special allocation policies and non-standard extensions to try to handle the issue is probably not working anyway because you cannot control how the external libraries you use deal with memory (e.g. if the have cached data).
A possible solution would be moving the C++ part to an external process that terminates once the SQLite databases have been created. Having an external process will introduce some annoyiance (e.g. it's a bit harder to keep a "live" control on what happens), but also opens up more possibilities like parallel processing even if libraries are not supporting multithreading or using multiple machines over a network.
Since you're interoperating with C++/CLI, you're presumably using Microsoft's compiler.
If that's the case, then you probably want to look up _heapmin. After you exit from your "stage 1", call it, and it'll release blocks of memory held by the C++ heap manager back to the OS, if the complete block that was allocated from the OS is now free.
On Linux, we used google malloc (http://code.google.com/p/google-perftools/). It has a function to release the free memory to the OS: MallocExtension::instance()->ReleaseFreeMemory().
In theory, gcmalloc works on Windows, but I never personally used it there.
You could allocate it off the GC from C#, pin it, use it, and then allow it to return, thus freeing it and letting the GC compact it and re-use the memory.