I'm trying to use glowcode to track a C++ mem-leak issue in our app.
But first of the first, glowcode cannot track all the memory allocated from our app. It only tracks 300MB memory usage when TaskManager shows we're using 700MB;
I doubted that the lost memory is allocated by some 3rd-party dll(s), but it's hard to find it out.
Do you have any similar issues or any ideas about that? Thanks in advance.
EDIT:
WOW! VMMap is a great tool, its timeline feature fits me well.
Well, it seems that VMMap is quite suited for the job, the timeline functionality is really a killer feature. I have sorted out our memleak problem with it.
Thanks #sergmat!
As for Glowcode, it looks like that it only tracks heap-allocation, not VirtualAlloc-ed memory (which is shown in VMMap as `Private Data'), so that's why the figures did not match with Task-Manager.
For tracking stacktrace, VMMap takes quite a lot of memory for itself compared with Glowcode. But for a 8GB memory machine, it won't be a big issue.
Related
I have a C++ process running in Solaris which creates 3 threads to do some tasks.
These threads execute in loops and it runs as long as the process is running.
But, I see that the memory usage of the process grows continuously and the process core dumps once the memory usage exceeds 4GB.
Can someone give me some pointers on what could be the issue behind memory usage growth?
What can I do to prevent process from core dumping because of memory exhaustion?
Will thread restart help?
Any pointers welcome.
No, restarting a thread would not help.
It seems like you have a memory leak in your application.
In my experience there are two types of memory leaks:
real memory leaks that you can see when the application exits
'false' memory leaks, like a big list that increases during the lifetime of your application but which is correctly cleaned up at the end
For the first type, there are tools which can report the memory that has not been freed by your application when it exits. I don't know about Solaris but there are numerous tools under Windows which can do that. For Unix, I think that Valgrind does this.
For the second type, there are also tools under Windows that can take snapshots of the memory of your application. Simply take two snapshots with an interval of a few minutes or hours (depending on your application) and let them compare by the tool. There are probably simlar tools like this on Solaris.
Using these tools will probably require your application to take much more memory, since the tool needs to store the call stack of every memory allocation. Because of this it will also run much slower. However, you will only see this effect when you are actively using this tool, so there is no effect in real-life production code.
So, just look for this kind of tools under Solaris. I quickly Googled for it and found this link: http://prefetch.net/blog/index.php/2006/02/19/finding-memory-leaks-on-solaris-systems/. This could be a starting point.
EDIT: Some additional information: are you looking at the right kind of memory? Even if you only allocated 3GB in total, the total virtual address space may still reach 4GB because of memory fragmentation. Unfortunately, there is nothing you can do about this (except using another memory allocation strategy).
We have a big multi-threaded C++ application running on Linux. We see that occupied by the application memory grows fast and believe there are some leaks. We have tried every tool we have (valgrind, DynLeak, Purify) but did not find anything. Since this application can run on Windows, we have also tried Bounds Checker. Did not help, too.
We need a new tool that can help. I've looked at Google Perfomrance Tools, MMGR by Paul Nettle, MemCheck Deluxe. None of them impressed me.
Is there anywhere a good tool for this task?
The definition of a memory leak in C/C++ is very specific: it is memory that has been allocated and then the pointer was overwritten or otherwise lost. Valgrind generally detects such cases out of the box, but things are not always that simple.
Your application could very well be still using that memory. In that case you might have what a Java programmer would consider a leak, e.g. entering data in a structure and rarely (or never) removing entries.
You might be measuring the memory usage of your memory incorrectly. On Linux memory usage measurements are not as straight-forward as they seem. How have you measured your memory usage?
You should consider using the application hooks (Valgrind calls them client requests) of whatever memory analysis tool your are using, to avoid the issue with reports only being issued at program termination. Using those hooks might help you pin-point the location of your leak.
You should try using a heap profiler, such as massif from Valgrind, to look for memory allocation locations with inordinate amounts of allocated memory.
Make sure you are not using a custom allocator or garbage collector in your application. As far as I know, no memory analysis tool will work with a custom allocator without user interference.
If your memory leak is massive enough to be detectable within an acceptable amount of application run-time, you could try a binary search of old revisions through your version control system to identify the commit that introduced the problem. At least Mercurial
and Git offer built-in support for this task.
If by "did not help" you mean it did not report memory leaks, it is quite possible you don't have one and just use more and more memory that is still referenced by pointers and can be deleted.
To help you debug the problem, perhaps in your logging, you should also write memory size, number of objects (their type) and a few other stats which are useful to you. At least until you become more familiar with the tools you mentioned.
my program is running and creating variables, I need to know what's the total of Bytes these variables take.
I don't want to know how much is the physical memory space that the system gives my program to be executed, I know I can open the processes manager and find out.
I neither want to write into my code some sizeof and agregations so I can know the total size of the variable pool (let say the code is too complex to be modify like that).
Finally I'm using Microsoft VC++ 2010 Express, I just want to know if there is a workspace which monitor that kind of information.
Thanks in advance.
Check this out: Memory Performance Information . There are few metrics of a running process you might be interested in, you will primarily want private bytes, and this data is available both programmatically or through tools like Performance Monitor. You can also enumerate heaps of the process with GetProcessHeaps (and even HeapWalk if you need details) and check heap allocation sizes directly.
Valgrind Massif profiler is a great tool (see here ) but only for Unix/Linux I think. In your case, on Windows I think Insure++ or softwareverify are good choices (they are commercial tools).
A free alternative is Google's tcmalloc which provides a heap profiler here
I have a C++ Application, when I observe Task Manager, it shows that applicaiton's memory usage increases gradually.
I manually check my source code, and I used Visual Leak Detector for Visual C++ to find memory leak, but I couldn't find any.
Is it 100% that there is a memory leak, and I couldn't find it or is there any possibility that Task Manager misguide me?
It isn't. It has several options for memory statistics (use View + Columns) and the version matters but the default view shows the working set. How much of the virtual memory your program uses is actually in RAM. That's a statistical number that can change very quickly. Just minimize the main window of your app for example.
The VM size it can show isn't great either. That number includes free heap blocks. Getting actual memory in use is very tricky, read the small print in the SDK article for HeapWalk.
It is useless for leak detection, unless you leak gobs of it.
I use Process Explorer as replacement for Task Manager. It shows history graphs for CPU/mem usage
I use Extended Task manager
http://www.warecase.com/products.asp
This is useful for debugging purpose especially to check if a thread exists or not and other such cases. It can provide lots of information if you have pdb for your process or application.
Probably you can use DevPartner for identifying memory leaks. It is very useful.
Suppose you have a fairly large (~2.2 MLOC), fairly old (started more than 10 years ago) Windows desktop application in C/C++. About 10% of modules are external and don't have sources, only debug symbols.
How would you go about reducing application's memory footprint in half? At least, what would you do to find out where memory is consumed?
Override malloc()/free() and new()/delete() with wrappers that keep track of how big the allocations are and (by recording the callstack and later resolving it against the symbol table) where they are made from. On shutdown, have your wrapper display any memory still allocated.
This should enable you both to work out where the largest allocations are and to catch any leaks.
this is description/skeleton of memory tracing application I used to reduce memory consumption of our game by 20%. It helped me to track many allocations done by external modules.
It's not an easy task. Begin by chasing down any memory leaks you cand find (a good tool would be Rational Purify). Skim the source code and try to optimize data structures and/or algorithms.
Sorry if this may sound pessimistic, but cutting down memory usage by 50% doesn't sound realistic.
There is a chance is you can find some significant inefficiencies very fast. First you should check what is the memory used for. A tool which I have found very handy for this is Memory Validator
Once you have this "memory usage map", you can check for Low Hanging Fruit. Are there any data structures consuming a lot of memory which could be represented in a more compact form? This is often possible, esp. when the data access is well encapsulated and when you have a spare CPU power you can dedicate to compressing / decompressing them on each access.
I don't think your question is well posed.
The size of source code is not directly related to the memory footprint. Sure, the compiled code will occupy some memory but the application might will have memory requirements on it's own. Both static (the variables declared in the code) and dynamic (the object the application creates).
I would suggest you to profile program execution and study the code carefully.
First places to start for me would be:
Does the application do a lot of preallocation memory to be used later? Does this memory often sit around unused, never handed out? Consider switching to newing/deleting (or better use a smart_ptr) as needed.
Does the code use a static array such as
Object arrayOfObjs[MAX_THAT_WILL_EVER_BE_USED];
and hand out objs in this array? If so, consider manually managing this memory.
One of the tools for memory usage analysis is LeakDiag, available for free download from Microsoft. It apparently allows to hook all user-mode allocators down to VirtualAlloc and to dump process allocation snapshots to XML at any time. These snapshots then can be used to determine which call stacks allocate most memory and which call stacks are leaking. It lacks pretty frontend for snapshot analysis (unless you can get LDParser/LDGrapher via Microsoft Premier Support), but all the data is there.
One more thing to note is that you may have false leak positives from BSTR allocator due to caching, see "Hey, why am I leaking all my BSTR's?"