Let's say I have created a program in C/C++ and have a source code.
I'd like to know the total memory during the program execution.
Someone has mentioned something about "malloc" and "hook"
Is there any other way to trace the spaced used?
If you are running Linux or something Unix-based, you could most likely use Valgrind. Valgrind runs the program and intercepts all of its memory allocations and prints the stats once it exits. It's a very useful tool for checking for memory leaks and memory usage. If you're running Windows, I haven't a clue.
You can monitor memory use with the "top" command in linux or taskmgr in windows.
In linux-like systems, you can use info from
/proc/self
to find out total amount of memory used by your program during the runtime. It also contains many other info about the process, see
man 5 proc
for details.
Related
When using Valgrind Massif in LINUX, I tried to see the massif.out.pid file in real time but it produces this file after Massif finished executing. I want to place breakpoints in the code to watch the effect certain instructions have on the heap as there are only a few points of interest. I know I could rewrite some of the code to manage this but this means influencing the subject code in a way that won't be the final result as well that their is numerous programs I want to do this on. Is there a way to watch points in the code while watching the profiler profile the heap at the same time, so I can distinguish between points of execution?
Apparently there is a tool in Linux that can give real time data, I haven't tried it out but found it after additional google searching. Its called heaptrack and apparently it can track the heap while the program is running:
http://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux
Write sleep(0.5) in the code where you want to see the memory heap usage.
then execute application with heaptrack ./app
and then open the result file with heaptrack_gui app.heaptrack.gz
go to consumed tap and check the memory usage by time line.
Maybe it's a bit late, but massif-visualizer is the tool you looking for :
massif-visualizer
I am trying to improve the performance of my C++ program and I found that converting memory allocations (mallocs) into object pool is giving great results.
The problem is detecting the places from which malloc is called, since the code base is quite large. I can't use simple gdb with break points because there are many timers and signal handlers running in parallel.
Is there a way in gdb using which I can print the entire stack trace whenever malloc is called without having to do it manually each time.
OR
Can you suggest any other tool which will help me do the same.
You can script gdb using Python.
You can also implement your own malloc function and link with that. The return address will be on the stack, which will give you the caller.
The valgrind suite of tools contains massif which you can use for precisely this purpose:
valgrind --tool=massif ./mybinary
This collects details of all allocations including stack traces that you can examine after the program has finished executing. Please see the massif documentation for more details on the output: http://valgrind.org/docs/manual/ms-manual.html. Hope that helps.
P.S. Also checkout the TCMalloc library - it already possibly already does what you want, although you can do better depending on your specific application. The best thing is that no source code changes are needed - you simply replace the malloc function from glibc using a linker directive.
How to get the total memory, used memory, free memory from C++ code on Linux system?
Run your program through valgrind. For a program called foo, for example:
valgrind foo
It'll run the program in a harness that keeps track of memory use and print out that information after the program terminates.
If you don't have valgrind installed for some reason, you should be able to find it in your distro's package repository.
As commented by Chris Stratton, you can -on Linux- query a lot of system information in /proc/ so read proc(5); which contain textual pseudo-files (a bit like pipes) to be read sequentially. These are not real disk files so are read very quickly. You'll need to open and close them at every measurement.
From inside a process, you can query its address space in virtual memory using /proc/self/maps -and /proc/self/smaps; outside of that process, for another process of pid 1234, use /proc/1234/maps & /proc/1234/smaps; you can get system wide memory information thru /proc/meminfo
So try the following commands in a terminal:
cat /proc/meminfo
cat /proc/$$/maps
cat /proc/$$/smaps
cat /proc/self/maps
to understand more what they could give you.
Be aware that malloc and free (used by new and delete) are often requesting space (from the kernel) using syscalls like mmap(2) but are managing previously free-d memory to reuse it, so often don't release memory back to the kernel with munmap. Read wikipage on C memory management. In other words, the used heap is not accessible outside the process (since some unused, but reusable for future malloc-s, memory remains mmap-ed). See also mallinfo(3) and malloc_stats(3).
As Justin Lardinois answered, use valgrind to detect memory leaks.
Advanced Linux Programming is a good book to read. It has several related chapters.
I have a tool to generate snapshots of my C program as it's running, in the form of corefiles.
Is it possible to bring these corefiles back to life as executable programs? And if so, how do i do it? Are there any libraries that do this already?
I'm working on *nix systems.
Yes it is, in theory.
If your snapshot is a full dump of the whole memory of your program, the stack, the heap, along with the PC, it is possible to put this dump back in memory and launch the process of execution again. But you will have to do this in kernel land I think.
GDB will do that for you. I have used it for programs that have crashed, creating a core, but this should be no different, in theory.
In some programming contests, problems have a memory limit (like 64MB or 256MB). How can I understand the memory used by my program (written in C++) with bash commands? Is there any way to limit the memory used by the program? The program should terminate if it uses more memory than the limit.
The command top will give you a list of all running processes and the current memory and swap or if you prefer the GUI you can use the System Monitor Application.
As for locking down memory usage you can always use the ulimit -v to set the maximum virtual address range for a process. This will cause malloc and its buddies to fail if they try to get more memory than that set limit.
Depending on how much work you want to put into it you can look at getrusage(), getrlimit(), and setrlimit(). For testing purposes you can call them at the beginning of your program or perhaps set them up in a parent process and fork your contest program off as a child. Then dispense with them when you submit your program for contest consideration.
Also, for process 1234, you could look into /proc/1234/maps or /proc/1234/smaps or run pmap 1234, all these commands display the memory map of that process of pid 1234.
Try to run cat /proc/self/maps to get an example (the memory map of the process running that cat command).
The memory map of a process is initialized by execve(2) and changed by the mmap(2) syscall (etc...)