I have a C++ Application, when I observe Task Manager, it shows that applicaiton's memory usage increases gradually.
I manually check my source code, and I used Visual Leak Detector for Visual C++ to find memory leak, but I couldn't find any.
Is it 100% that there is a memory leak, and I couldn't find it or is there any possibility that Task Manager misguide me?
It isn't. It has several options for memory statistics (use View + Columns) and the version matters but the default view shows the working set. How much of the virtual memory your program uses is actually in RAM. That's a statistical number that can change very quickly. Just minimize the main window of your app for example.
The VM size it can show isn't great either. That number includes free heap blocks. Getting actual memory in use is very tricky, read the small print in the SDK article for HeapWalk.
It is useless for leak detection, unless you leak gobs of it.
I use Process Explorer as replacement for Task Manager. It shows history graphs for CPU/mem usage
I use Extended Task manager
http://www.warecase.com/products.asp
This is useful for debugging purpose especially to check if a thread exists or not and other such cases. It can provide lots of information if you have pdb for your process or application.
Probably you can use DevPartner for identifying memory leaks. It is very useful.
Related
I am analysing high memory consumption problem in our software. I have a core file corresponding to this high memory consumption(this core file is generated by killing our application which generates core file). But I am not able to view the actual memory consumption using this core file. I used Totalview and gdb...using these two I am not getting a snapshot of the total memory consumed by my process and which library is eating up all the memory.
This memory consumption is hitting us over 10 to 20 days of time and hence I am trying to find out what has caused this high memory consumption.
Can valgrind help me in analysing this core file?
Any input/suggestion is highly appreciated.
#suresh,
Hi, I'm the product manager for TotalView at Rogue Wave Software.
Can you describe the scenario a bit more? Is the program running along with "normal memory consumption" for a long time and then suddenly the memory consumption goes through the roof? Or is the program slowly and steadily consuming memory till it exhausts available resources?
When it is crashing is it crashing because it literally runs out of memory or are you killing it because it has started swapping and being unresponsive?
In general I'd recommend running it under MemoryScape (in TotalView or the Standalone version) and when it starts to show unexpected memory consumption you want to pause it and run a memory leak report. It is likely that this will point right to the problem.
It is possible that the memory use isn't a "classical" leak because you still have pointers referencing the data -- but you are simply over-allocating. In this case you won't see anything on the leak report, but you may be able to pick out which allocation is "gone bad" by watching which allocations are growing. There are a number of ways to do this.
Pause periodically and note how the heap usage breaks down in the Heap Status Source Code report. For example you may note that the number of allocations associated with a specific source code file just keeps increasing.
If you are using TotalView you can use the "set heap baseline" functionality when the program seems be behaving well then filter against this baseline. Again you may want to use the source code report (though the graphical or backtrace reports support filtering too).
Or you can use the "export memory data" feature to store an image of what the "normal" heap status is. This creates a binary heap status file. Then let the program run till you get into the state where your program has high memory consumption. At that point you pause your live app, load the stored heap data file and you can do a comparison.
Wow, this is getting long. One final thought. You said you are getting core files. Under the debugger you should get a chance to examine the running program before it gets cleaned up. If this doesn't happen let me know. If you really want to work via core files (for example, this is happening in a production environment and you don't want to run the debugger there) let us know -- there are techniques where we can instrument the application using the HIA and then enable you to do offline analysis of your heap status.
Good luck!
Chris Gotbrath
Principal Product Manager for TotalView and ThreadSpotter, Rogue Wave Software
email: first . last at roguewave . com
We have a big multi-threaded C++ application running on Linux. We see that occupied by the application memory grows fast and believe there are some leaks. We have tried every tool we have (valgrind, DynLeak, Purify) but did not find anything. Since this application can run on Windows, we have also tried Bounds Checker. Did not help, too.
We need a new tool that can help. I've looked at Google Perfomrance Tools, MMGR by Paul Nettle, MemCheck Deluxe. None of them impressed me.
Is there anywhere a good tool for this task?
The definition of a memory leak in C/C++ is very specific: it is memory that has been allocated and then the pointer was overwritten or otherwise lost. Valgrind generally detects such cases out of the box, but things are not always that simple.
Your application could very well be still using that memory. In that case you might have what a Java programmer would consider a leak, e.g. entering data in a structure and rarely (or never) removing entries.
You might be measuring the memory usage of your memory incorrectly. On Linux memory usage measurements are not as straight-forward as they seem. How have you measured your memory usage?
You should consider using the application hooks (Valgrind calls them client requests) of whatever memory analysis tool your are using, to avoid the issue with reports only being issued at program termination. Using those hooks might help you pin-point the location of your leak.
You should try using a heap profiler, such as massif from Valgrind, to look for memory allocation locations with inordinate amounts of allocated memory.
Make sure you are not using a custom allocator or garbage collector in your application. As far as I know, no memory analysis tool will work with a custom allocator without user interference.
If your memory leak is massive enough to be detectable within an acceptable amount of application run-time, you could try a binary search of old revisions through your version control system to identify the commit that introduced the problem. At least Mercurial
and Git offer built-in support for this task.
If by "did not help" you mean it did not report memory leaks, it is quite possible you don't have one and just use more and more memory that is still referenced by pointers and can be deleted.
To help you debug the problem, perhaps in your logging, you should also write memory size, number of objects (their type) and a few other stats which are useful to you. At least until you become more familiar with the tools you mentioned.
What tools exist to help finding memory leaks for handles?
I have a file.exe and an inproc-server dll, which is using file.exe. I have about 10 memory leaks of handles evertyime it completes its operation.
Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653) is one of the SysInternals tools that you can use to show the handles that a program has allocated. You have to configure the lower pane view to show handles in order to see them. It tells you the type of handle, which may help in finding the source of the leak. It won't tell you an allocation path, but it is still useful when other tools also can't detect the leaks.
I've also used DevPartnerStudio successfully to find a lot of leaks. (http://www.microfocus.com/products/micro-focus-developer/devpartner/index.aspx). It does have a tendency to report false leaks, so you have to play with the settings to make sure you're getting accurate results. It is a very good product and I always suggest buying it when doing C++ development on Windows.
I thought I would ask the experts - see if you can help me :o)
My son has written C++ code for Collision Detection using Brute Force and Octree algorithms.
He has used Debug etc - and to collect stats on mem usage he has used windows & task manager - which have given him all the end results he has needed so far. The results are not yet as they were expect to be (that Octree would use more memory overall).
His tutor has suggested he checks memory once each is "initialised" and then plot at points through the test.
He was pointed in the direction of Valgrind .... but it looked uite complicated and becaus ehe has autism, he is worried that it might affect his programmes :o)
Anyone suggest a simple way to grab the information on Memory if not also Frame Rate and CPU usage???
Any help gratefully received, as I know nothing so can't help him at all, except for typing this on here - as it's "social" environment he can't deal with it.
Thanks
Rosalyn
For the memory leaks:
If you're on Windows, Visual C++ by Microsoft (the Express version is free) has a nice tool for debugging and is easy to setup with instructions can be found here; otherwise if you're on Linux, Valgrind is one of the standards. I have used the Visual C++ tool often and it's a nice verification that you have no memory leaks. Also, you can use it to enabled your programs to break on allocation numbers that you get from the memory leak log so it quickly points you to when and where the memory is getting assigned that leaks. Again, it's easy to implement (just a few header files and then a single function call where you want to dump the leaks at).
I have found the best way to implement the VC++ tool is to make the call to dump the memory leaks to the output window right before main returns a value. That way, you can catch the leaks of absolutely everything in your program. This works very well and I have used it for some advanced software.
For the framerate and CPU usage:
I usually use my own tools for benchmarking since they're not difficult to code once you learn the functions to call; this would usually require OS API calls, but I think Boost has that available and is cross-platform. There might be other tools out there that can track the process in the OS to get benchmarking data as well, but I'm not certain if they would be free or not.
It looks like you're running under a windows system. This isn't a programming solution, and you may have already tried it (so feel free to ignore), but if not, you should take a look at performance monitor (it's one of the tools that ships with windows). It'll let you track all sorts of useful stats about individual proceses and the system as a whole (cpu/commit size etc). It plots the results for you as a graph as the program is running and you can save the results off for future viewing.
On Windows 7, you get to it from here:
Control Panel\All Control Panel Items\Performance Information and Tools\Advanced Tools
Then Open Performance Monitor.
For older versions of windows, it used to be one of the administrative tools options.
An application I am working with is exhibiting the following behaviour:
During a particular high-memory operation, the memory usage of the process under Task Manager (Mem Usage stat) reaches a peak of approximately 2.5GB (Note: A registry key has been set to allow this, as usually there is a maximum of 2GB for a process under 32-bit Windows)
After the operation is complete, the process size slowly starts decreasing at a rate of 1MB per second.
I am trying to figure out the easiest way to quickly determine who is freeing this memory, and where it is being free'd.
I am having trouble attaching a memory profiler to my code, and I don't particularly want to override the new/delete operators to track the allocations/deallocations (IOW, I want to do this without re-compiling my code).
Can anyone offer any useful suggestions of how I could do this via the Visual Studio debugger?
Update
I should also mention that it's a multi-threaded application, so pausing the application and analysing the call stack through the debugger is not the most desirable option. I considered freezing different threads one at a time to see if the memory stops reducing, but I'm fairly certain this will cause the application to crash.
Ahh! You're looking at the wrong counter!
Mem Usage doesn't tell you that memory is being freed. Only that the working set is being purged! This could mean some other application needs memory, or the VMM decided to mark some of your process's pages as Stand By for some other process to quickly use. It does not mean that VirtualFree, HeapFree or any other free function is being called.
Look at the commit size (VM Size, Private Bytes, etc).
But if you still want to know when memory is being decommitted or freed or what-have-you, then break on some free calls. E.g. (for Visual C++)
{,,kernel32.dll}HeapFree
or
{,,msvcr80.dll}free
etc.
Or just a regular function breakpoint on the above. Just make sure it resolves the address.
cdb/WinDbg let you do it via
bp kernel32!HeapFree
bp msvcrt!free
etc.
Names may vary depending on which CRT version you use and how you link against it (via /MT or /MD and its variants)
You might find this article useful:
http://www.gamasutra.com/view/feature/1430/monitoring_your_pcs_memory_usage_.php?print=1
basically what I had in mind was hooking the low level allocation functions.
A couple different ideas:
The C runtime has a set of memory debugging functions; you'd need to recompile though. You could get a snapshot at computation completion and later, and use _CrtMemDifference to see what changed.
Or, you can attach to the process in your debugger, and cause it to dump a core before and after the memory freeing. Using NTSD, you can see what heaps are around, and the sizes of things. (You'll need a lot of disk space, and a fair amount of patience.) There's a setting (I think you get it through gflags, but I don't remember) that causes it to save a piece of the call stack as part of the dump; using that you can figure out what kind of object is being deallocated. Unfortunately, it only stores 4 or 5 stack frames, so you'll likely have to do something more clever as the next step to figure out where it's being freed. Either look at the code ("oh yeah, there's only one place where that can happen") or put in breakpoints on those destructors, or add tracing to the allocations and deallocations.
If your memory manager wipes free'd data to a known value (usually something like 0xfeeefeee), you can set a data breakpoint on a particular instance of something you're interested in. When it gets free'd, the breakpoint will trigger when the memory gets wiped.
I recommend you to check UMDH tool that comes with Debugging Tools For Windows (You can find usage and samples in the debugging tools help). You can snap shot running process's heap allocations with stack trace and compare them.
You could try Memory Validator to monitor the allocations and deallocations. Memory Validator has a couple of features that will help you identify where data is being deallocated:
Hotspots view. This can show you a tree of all allocations and deallocations or just all allocations or just all deallocations. It presents the data as a percentage of memory activity (based on amount of memory (de)allocated at a given location).
Analysis view. You can perform queries asking for data in a given address range. You can restrict these queries to any of alloc, realloc, dealloc behaviours.
Objects view. You can view allocations by type and see the maximum number of objects of each type (plus lots of other stats). Right click on a type to get a context menu, choose show all deallocations - will show deallocation locations for that type on Analysis tab.
I think the Hotspots view may give you the insight you need.