I want to test my C++ code for memory leaks with Valgrind (memcheck) x86.
But the software gets cross-compiled and is running on ARM.
In order to do some automated testing I decided to emulate my ARM hardware via QEMU.
And I also decided to use the cpputest unit test ARM binaries to ensure a deterministic behaviour and search for memory leaks within the scope the unit test covers.
All in all, I have an ARM binary which should be emulated via QEMU user mode.
My call looks like that:
./valgrind --smc-check=all qemu-arm-static -L ... arm-ptest-binary
My C++ code looks like that. It has a memory leak of 20 byte and the valgrind call do not find this leak when using it with QEMU.
After I insert a memory allocation and no freeing mechanism I'd have expected an memory leak
int test_func ()
{
int *foo;
foo = new int [5];
printf("test_func called!\n");
return 1;
}
Valgrind output:
==19300== HEAP SUMMARY:
==19300== in use at exit: 1,103,129 bytes in 2,316 blocks
==19300== total heap usage: 4,259 allocs, 1,943 frees, 1,866,916 bytes allocated
==19300==
==19300== LEAK SUMMARY:
==19300== definitely lost: 0 bytes in 0 blocks
==19300== indirectly lost: 0 bytes in 0 blocks
==19300== possibly lost: 304 bytes in 1 blocks
==19300== still reachable: 1,102,825 bytes in 2,315 blocks
==19300== suppressed: 0 bytes in 0 blocks
[...]
When I run this program on ARM hardware the valgrind-arm finds the leak with the exact same binary.
Does anyone of you have an idea why Valgrind does not find the memory leak in combination with QEMU user mode?
Thanks in advance
You are running Valgrind on QEMU itself, which will cause valgrind to report memory leaks in QEMU's own code, but valgrind does not have sufficient visibility into what the guest program running under QEMU is doing to be able to report leaks in the guest. In particular, Valgrind works by intercepting calls to malloc, free, operator new, etc -- it will be doing this for the host QEMU process's (x86) allocation and free calls, but has no way to intercept the (arm) calls your guest process makes.
You might look at running an entire guest OS under QEMU's system emulation mode, and then running the Arm Valgrind inside that on your guest program.
Related
I am writing some code that uses MPI and I was keeping noticing some memory leaks when running it with valgrind. While trying to identify where the problem was, I ended up with this simple (and totally useless) main:
#include "/usr/include/mpi/mpi.h"
int main(int argc,char** argv)
{
MPI_Init(&argc, &argv);
MPI_Finalize();
return 0;
}
As you can see, this code doesn't do anything and shouldn't create any problem. However, when I run the code with valgrind (both in the serial and parallel case), I get the following summary:
==28271== HEAP SUMMARY:
==28271== in use at exit: 190,826 bytes in 2,745 blocks
==28271== total heap usage: 11,214 allocs, 8,469 frees, 16,487,977 bytes allocated
==28271==
==28271== LEAK SUMMARY:
==28271== definitely lost: 5,950 bytes in 55 blocks
==28271== indirectly lost: 3,562 bytes in 32 blocks
==28271== possibly lost: 0 bytes in 0 blocks
==28271== still reachable: 181,314 bytes in 2,658 blocks
==28271== suppressed: 0 bytes in 0 blocks
I don't understand why there are these leaks. Maybe it's just me not able to read the valgrind output or to use MPI initialization/finalization correctly...
I am using OMPI 1.4.1-3 under ubuntu on a 64 bit architecture, if this can help.
Thanks a lot for your time!
The OpenMPI FAQ addresses issues with valgrind. This refers initalization issues and memory leaks during finalization - which should have no practical negative impact.
There are many situations, where Open MPI purposefully does not
initialize and subsequently communicates memory, e.g., by calling
writev. Furthermore, several cases are known, where memory is not
properly freed upon MPI_Finalize.
This certainly does not help distinguishing real errors from false
positives. Valgrind provides functionality to suppress errors and
warnings from certain function contexts.
In an attempt to ease debugging using Valgrind, starting with v1.5,
Open MPI provides a so-called Valgrind-suppression file, that can be
passed on the command line:
mpirun -np 2 valgrind
--suppressions=$PREFIX/share/openmpi/openmpi-valgrind.supp
You're not doing anything wrong. Memcheck false positives with valgrind are common, the best you can do is suppress them.
This page of the manual speaks more about these false positives. A quote near the end:
The wrappers should reduce Memcheck's false-error rate on MPI
applications. Because the wrapping is done at the MPI interface, there
will still potentially be a large number of errors reported in the MPI
implementation below the interface. The best you can do is try to
suppress them.
I am running valgrind on the macos x 10.8. Valgrind says on startup
"==11312== WARNING: Support on MacOS 10.8 is experimental and mostly broken.
==11312== WARNING: Expect incorrect results, assertions and crashes.
==11312== WARNING: In particular, Memcheck on 32-bit programs will fail to
==11312== WARNING: detect any errors associated with heap-allocated data."
Valgrind is giving this leak summary:
"LEAK SUMMARY:
==11312== definitely lost: 0 bytes in 0 blocks
==11312== indirectly lost: 48 bytes in 2 blocks
==11312== possibly lost: 0 bytes in 0 blocks
==11312== still reachable: 45,857 bytes in 270 blocks
==11312== suppressed: 16,805 bytes in 87 blocks"
According to valgrinds faq, http://valgrind.org/docs/manual/faq.html#faq.deflost, "indirectly lost" means your program is leaking memory in a pointer-based structure. (E.g. if the root node of a binary tree is "definitely lost", all the children will be "indirectly lost".) If you fix the "definitely lost" leaks, the "indirectly lost" leaks should go away.
I dont have any definitely lost leaks or even possibly lost leaks to fix. What am I supposed to fix? Could this report be a bug due to the experimental nature of valgrind in 10.8?
I believe i am compiling this as a 64 bit program since the compiler is a 64 bit compiler.
I feel weird answering my own question.
Yes the report by valgrind on mac is incorrect. According to valgrind on linux all heap blocks were freed so no leaks are possible.
I really hope valgrind fixes the issues with mac since I mainly am developing on mac now.
Valgrind has been updated. Use (if you use homebrew):
brew unlink valgrind
brew install valgrind
And, lo and behold:
==23998== LEAK SUMMARY:
==23998== definitely lost: 0 bytes in 0 blocks
==23998== indirectly lost: 0 bytes in 0 blocks
==23998== possibly lost: 0 bytes in 0 blocks
==23998== still reachable: 76,800 bytes in 2 blocks
==23998== suppressed: 58,420 bytes in 359 blocks
I have the following C++ file, pwd01.cpp:
#include <pwd.h>
#include <iostream>
int main() {
passwd* pwd = getpwuid(getuid());
}
I compile this with the following command:
g++ pwd01.cpp -Wall -o pwd01
On Ubuntu 12.04.1 LTS / gcc version 4.6.3, valgrind reports a leak (see below). When I compile the same code with the same command on Mac OS 10.6.8 / gcc version 4.2.1, valgrind reports no leaks.
I am aware that I do not need to free passwd* ( should I free pointer returned by getpwuid() in Linux? ); so what am I missing?
valgrind ./pwd01
==10618== Memcheck, a memory error detector
==10618== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==10618== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==10618== Command: ./pwd01
==10618==
==10618==
==10618== HEAP SUMMARY:
==10618== in use at exit: 300 bytes in 11 blocks
==10618== total heap usage: 68 allocs, 57 frees, 10,130 bytes allocated
==10618==
==10618== LEAK SUMMARY:
==10618== definitely lost: 60 bytes in 1 blocks
==10618== indirectly lost: 240 bytes in 10 blocks
==10618== possibly lost: 0 bytes in 0 blocks
==10618== still reachable: 0 bytes in 0 blocks
==10618== suppressed: 0 bytes in 0 blocks
==10618== Rerun with --leak-check=full to see details of leaked memory
==10618==
==10618== For counts of detected and suppressed errors, rerun with: -v
==10618== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
Does not seem to be a "real" leak, i.e., if called several times, the leak doesn't compound; probably it holds a static pointer to a memory area, if it is NULL (the first time) it allocates those 60 bytes, and then doesn't free them up.
The MacOS X version either uses a truly static area, or its valgrind has got better suppressors.
Just run the getpwuid a couple hundred times in a loop to ensure it really leaks only 60 bytes (and not 1200), just to be sure.
UPDATE
I have finally tracked the leak to several structures inside nssswitch.c and getXXent.c, of different sizes and persuasions. While the code seems to make many more allocations than really necessary, needing malloc locks, this shouldn't be usually appreciable performance-wise, and I for one surely do not intend to second-guess the maintainers of glibc!
It may not be getpwuid() itself that is causing that (false) positive. It could be any number of other things that the C library initializes at start up time, but then doesn't tear down at process termination (because the process is going away, along with all the mapped memory that belongs to it, some things don't really need to be destructed/unallocated). As another answer said, run some additional tests, especially as you build more code beyond the simple example you provided, and make sure the numbers are stable, and not directly attributable to your own code. There's not much you can do about the library code directly, except submit a bug report (I'm assuming you're not one of the C library developers, anyway).
I'm doing a little bit of memory profiling to my software and after running standard memory leak check with valgrind's following command
valgrind --tool=memcheck --leak-check=full ./path_to_program
I got following summary:
==12550== LEAK SUMMARY:
==12550== definitely lost: 597,170 bytes in 7 blocks
==12550== indirectly lost: 120 bytes in 10 blocks
==12550== possibly lost: 770,281 bytes in 1,455 blocks
==12550== still reachable: 181,189 bytes in 2,319 blocks
==12550== suppressed: 0 bytes in 0 blocks
==12550== Reachable blocks (those to which a pointer was found) are not shown.
==12550== To see them, rerun with: --leak-check=full --show-reachable=yes
==12550==
==12550== For counts of detected and suppressed errors, rerun with: -v
==12550== ERROR SUMMARY: 325 errors from 325 contexts (suppressed: 176 from 11)
It doesn't look quite good to me, so my question is
Why isn't my program exploding if it has all these leaks?
And also what is the difference between:
definitely lost
indirectly lost
possibly lost
still reachable
and how can I try to fix them?
I suggest visiting the Valgrind FAQ:
With Memcheck's memory leak detector, what's the difference between
"definitely lost", "indirectly lost", "possibly lost", "still
reachable", and "suppressed"?
The details are in the Memcheck section
of the user manual.
In short:
"definitely lost" means your program is leaking memory -- fix those
leaks!
"indirectly lost" means your program is leaking memory in a
pointer-based structure. (E.g. if the root node of a binary tree is
"definitely lost", all the children will be "indirectly lost".) If you
fix the "definitely lost" leaks, the "indirectly lost" leaks should go
away.
"possibly lost" means your program is leaking memory, unless you're
doing unusual things with pointers that could cause them to point into
the middle of an allocated block; see the user manual for some
possible causes. Use --show-possibly-lost=no if you don't want to see
these reports.
"still reachable" means your program is probably ok -- it didn't free
some memory it could have. This is quite common and often reasonable.
Don't use --show-reachable=yes if you don't want to see these reports.
"suppressed" means that a leak error has been suppressed. There are
some suppressions in the default suppression files. You can ignore
suppressed errors.
I have run my code through valgrind with these results:
==4492== Memcheck, a memory error detector
==4492== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==4492== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==4492== Command: ./mem
==4492== Parent PID: 4455
==4492==
==4492==
==4492== HEAP SUMMARY:
==4492== in use at exit: 0 bytes in 0 blocks
==4492== total heap usage: 19,595,342 allocs, 19,595,342 frees, 27,194,270 bytes allocated
==4492==
==4492== All heap blocks were freed -- no leaks are possible
==4492==
==4492== For counts of detected and suppressed errors, rerun with: -v
==4492== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
However, while the code is running, I see a small, steady increase in the memory used by the program. How sure can I be with that result?
I run valgrind using:
valgrind --track-origins=yes --leak-check=yes
--tool=memcheck --read-var-info=yes --log-file=error.txt`
and I compile the program using the -g and the -march=core2 tags.
You need to distinguish between memory leaks (memory that was allocated, but you lost all references to) and memory hogs (memory that was allocated, that you keep references to, but forgot to deallocate).
The later one can not be detected by valgrind, since valgrind doesn't know you did not want to use it anymore.
To get some statistics about your programs memory usage, you can use the massif tool of valgrind, which will show you in more detail where your memory gets allocated. This might be helpful in finding memory hogs.
A small increase in memory usage is not necessarily something to worry about - it may be that your program is ramping up and will peak at some point. Without knowing the logic of that application, it's hard to tell. However, it's adamant that all allocated blocks were freed and it's usually pretty good.
You may want to consider letting it run for longer, increasing the work it has to do somehow (again this depends on the application) to see if it ever peaks or continues to rise forever (or until it runs out of virtual memory, anyway).
I'd also look at those last two lines:
==4492== For counts of detected and suppressed errors, rerun with: -v
==4492== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
You may want to run it with -v just to check what those suppressions were. They may be nothing but it doesn't hurt to look into it.
Do you see memory usage increasing with a tool like top? Depending on the behavior of your program, if you allocate and free memory continually you may introduce fragmentation that causes the address space to grow. If you let the process run long enough it may stabilize and stop growing.
valgrind can detect memory leaks, but not poor usage of memory. It's possible that a bug in your code is continually allocating memory for no apparent reason, and that defensive code is then cleaning it all up afterwards anyway.
That said, I wouldn't trust your mechanism for determining your process's memory usage, either. There's a lot that goes on behind the scenes: caching, for one.
I'd call this "inconclusive".