Valgrind tells me this:
==19305== 16 bytes in 1 blocks are definitely lost in loss record 19 of 179
==19305== at 0x402842F: operator new(unsigned int)
==19305== by 0x805273E: Loader::createLevel(int, int, std::string, Player*, int, int, int)
==19305== by 0x80551B0: Loader::loadLevel()
==19305== by 0x80676C2: main (main.cpp:38)
My function Loader:.createLevel has got several new statements. How can I know which one of them is causing the leak (i.e., the line)?
Thanks!
P.S.: I'd gladly post the code but it is too long :/
Pass -g option to gcc or g++ so that your executable have debug symbol in them. Here is example from running valgrind on binary with -g.
==20538== 4 bytes in 1 blocks are definitely lost in loss record 1 of 1
==20538== at 0x4A05809: malloc (vg_replace_malloc.c:149)
==20538== by 0x4004F7: main (test.c:8)
==20538==
==20538== LEAK SUMMARY:
==20538== definitely lost: 4 bytes in 1 blocks.
==20538== possibly lost: 0 bytes in 0 blocks.
==20538== still reachable: 0 bytes in 0 blocks.
==20538== suppressed: 0 bytes in 0 blocks.
==20538== Reachable blocks (those to which a pointer was found) are not shown.
==20538== To see them, rerun with: --show-reachable=yes
gcc -g test.c
This way you can see the line at which the allocation was made.
Related
I have an application that uses OpenMPI and launch it on Windows and Linux. The version for Windows is working fine, however, running on a Linux cause memory allocation error. The problem occurs for certain app arguments, that require more calculations.
To eliminate memory leaks I checked Linux version app by using Valgrind and got some output. After all, I tried to search information about the output and found some posts on stack overflow and GitHub(not enough reputation to attach links). After all, I updated openMPI to 2.0.2 and check app again. New output. Is it memory leaks in OpenMPI or I'm doing something wrong?
A piece of output:
==16210== 4 bytes in 1 blocks are definitely lost in loss record 5 of 327
==16210== at 0x4C2DBB6: malloc (vg_replace_malloc.c:299)
==16210== by 0x5657A59: strdup (strdup.c:42)
==16210== by 0x51128E6: opal_basename (in /home/vshmelev/OMPI_2.0.2/lib/libopen-pal.so.20.2.0)
==16210== by 0x7DDECA9: ???
==16210== by 0x7DDEDD4: ???
==16210== by 0x6FBFF84: ???
==16210== by 0x4E4EA9E: orte_init (in /home/vshmelev/OMPI_2.0.2/lib/libopen-rte.so.20.1.0)
==16210== by 0x4041FD: orterun (orterun.c:818)
==16210== by 0x4034E5: main (main.c:13)
OpenMPI version:Open MPI: 2.0.2
Valgrind version: valgrind-3.12.0
Virtual mashine characteristics: Ubuntu 16.04 LTS x64
In case of using MPICH, the Valgrind output is:
==87863== HEAP SUMMARY:
==87863== in use at exit: 131,120 bytes in 2 blocks
==87863== total heap usage: 2,577 allocs, 2,575 frees, 279,908 bytes allocated
==87863==
==87863== 131,120 bytes in 2 blocks are still reachable in loss record 1 of 1
==87863== at 0x4C2DBB6: malloc (vg_replace_malloc.c:299)
==87863== by 0x425803: alloc_fwd_hash (sock.c:332)
==87863== by 0x425803: HYDU_sock_forward_stdio (sock.c:376)
==87863== by 0x432A99: HYDT_bscu_stdio_cb (bscu_cb.c:19)
==87863== by 0x42D9BF: HYDT_dmxu_poll_wait_for_event (demux_poll.c:75)
==87863== by 0x42889F: HYDT_bscu_wait_for_completion (bscu_wait.c:60)
==87863== by 0x42863C: HYDT_bsci_wait_for_completion (bsci_wait.c:21)
==87863== by 0x40B123: HYD_pmci_wait_for_completion (pmiserv_pmci.c:217)
==87863== by 0x4035C5: main (mpiexec.c:343)
==87863==
==87863== LEAK SUMMARY:
==87863== definitely lost: 0 bytes in 0 blocks
==87863== indirectly lost: 0 bytes in 0 blocks
==87863== possibly lost: 0 bytes in 0 blocks
==87863== still reachable: 131,120 bytes in 2 blocks
==87863== suppressed: 0 bytes in 0 blocks
==87863==
==87863== For counts of detected and suppressed errors, rerun with: -v
==87863== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
These outputs point to some memory leak in the MPI library, not your application code. You can safely ignore them.
More specifically, these leaks are coming from the launchers. ORTE is the runtime environment for OpenMPI responsible for launching and managing MPI processes. Hydra is the launcher and process manager for MPICH.
The term "definitely lost" means the main function of your program at line 13 (As far as i see in the output) is leaking memory directly or calls some other function (orterun) which causes memory leak . you must fix those leaks or provide some more of your code.
take a look here before everything.
Here is my test program:
#include "opencv2/videoio.hpp"
int main(int argc, char** argv) {
cv::VideoCapture videoCapture(argv[1]);
cv::Mat frame;
videoCapture.read(frame);
return 0;
}
I run this program like this:
valgrind --leak-check=yes ./GyroRecord ./walks6/w63/39840012.avi > valgrind_output 2>&1
So that the entire output is saved in the valgrind_output file.
The contents of valgrind_output can be checked here.
But, if the link dies in the future, this is the summary:
==9677== LEAK SUMMARY:
==9677== definitely lost: 0 bytes in 0 blocks
==9677== indirectly lost: 0 bytes in 0 blocks
==9677== possibly lost: 1,352 bytes in 18 blocks
==9677== still reachable: 166,408 bytes in 1,296 blocks
==9677== of which reachable via heuristic:
==9677== newarray : 1,536 bytes in 16 blocks
==9677== suppressed: 0 bytes in 0 blocks
==9677== Reachable blocks (those to which a pointer was found) are not shown.
==9677== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==9677==
==9677== For counts of detected and suppressed errors, rerun with: -v
==9677== ERROR SUMMARY: 18 errors from 18 contexts (suppressed: 0 from 0)
I would like to reduce "possibly lost" bytes to 0. Is that possible? Or will I always have some "possibly lost" bytes when using OpenCV?
OpenCV comes with suppression files (with the extension .supp) for valgrind that can be used to hide messages about resources allocated (often early in he program's execution) that will be kept allocated until the program dies and the OS has to clean up the mess.
The suppression files are placed in /usr/share/OpenCV (on my system):
Example:
valgrind --leak-check=yes --suppressions=/usr/share/OpenCV/valgrind.supp --suppressions=/usr/share/OpenCV/valgrind_3rdparty.supp ./GyroRecord ./walks6/w63/39840012.avi
Using these helped me a lot when running valgrind on an OpenCV project.
This is what i get after executing my program with Valgrind:
1 jscherman#jscherman:~/ClionProjects/algo2-t4-tries$ g++ Set.hpp tests.cpp DiccString.hpp && valgrind --leak-check=yes --show-leak-kinds=all ./a.out
2 ==6823== Memcheck, a memory error detector
3 ==6823== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
4 ==6823== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
5 ==6823== Command: ./a.out
6 ==6823==
7 test_empty_dicc...ok
8 test_copy_constructor...ok
9 test_define_defined...ok
10 test_get..ok
11 test_remove...ok
12 test_remove_tiny...ok
13 test_keys...ok
14 ==6823==
15 ==6823== HEAP SUMMARY:
16 ==6823== in use at exit: 72,704 bytes in 1 blocks
17 ==6823== total heap usage: 282 allocs, 281 frees, 275,300 bytes allocated
18 ==6823==
19 ==6823== 72,704 bytes in 1 blocks are still reachable in loss record 1 of 1
20 ==6823== at 0x4C2DC10: malloc (vg_replace_malloc.c:299)
21 ==6823== by 0x4EC3EFF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
22 ==6823== by 0x40104E9: call_init.part.0 (dl-init.c:72)
23 ==6823== by 0x40105FA: call_init (dl-init.c:30)
24 ==6823== by 0x40105FA: _dl_init (dl-init.c:120)
25 ==6823== by 0x4000CF9: ??? (in /lib/x86_64-linux-gnu/ld-2.23.so)
26 ==6823==
27 ==6823== LEAK SUMMARY:
28 ==6823== definitely lost: 0 bytes in 0 blocks
29 ==6823== indirectly lost: 0 bytes in 0 blocks
30 ==6823== possibly lost: 0 bytes in 0 blocks
31 ==6823== still reachable: 72,704 bytes in 1 blocks
32 ==6823== suppressed: 0 bytes in 0 blocks
33 ==6823==
34 ==6823== For counts of detected and suppressed errors, rerun with: -v
35 ==6823== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
It seems like there are no leaks as last line of the output says. Yet, we have also this line:
17 ==6823== total heap usage: 282 allocs, 281 frees, 275,300 bytes allocated
How is that i don't have any errors but there is still an allocation that hasn't been freed? Is there something wrong with my program or maybe something being done by Valgrind behind the scenes?
The backtrace reported by valgrind shows that the memory allocation in question was made in the initialization function of one of the shared libraries loaded by the application, apparently the C++ library itself.
It is quite common for shared libraries to make one-time allocations for various bits and pieces of data, but not bother to explicitly deallocate them, when they get unloaded.
This does not comprise a memory leak in your own code.
valgrind comes with a list of known allocations of this nature, it's called a "suppression list", for the explicit purpose of suppressing reports about these known one-off allocations.
But, occasionally, these suppression lists do miss an allocation, or two.
I am trying to install torque 6.0.2 on Debian 8.5 on a Intel Xeon E5-2620v4. However, when i try start pbs_server i returned a segment fault, with gdb:
#1 0x0000000000440ab6 in container::item_container<pbsnode*>::unlock (this=0xb5d900 <allnodes>) at ../../src/include/container.hpp:537
#2 0x00000000004b787f in mom_hierarchy_handler::nextNode (this=0x4e610c0 <hierarchy_handler>, iter=0x7fffffff98b8) at mom_hierarchy_handler.cpp:122
#3 0x00000000004b7a7d in mom_hierarchy_handler::make_default_hierarchy (this=0x4e610c0 <hierarchy_handler>) at mom_hierarchy_handler.cpp:149
#4 0x00000000004b898d in mom_hierarchy_handler::loadHierarchy (this=0x4e610c0 <hierarchy_handler>) at mom_hierarchy_handler.cpp:433
#5 0x00000000004b8ae8 in mom_hierarchy_handler::initialLoadHierarchy (this=0x4e610c0 <hierarchy_handler>) at mom_hierarchy_handler.cpp:472
#6 0x000000000045262a in pbsd_init (type=1) at pbsd_init.c:2299
#7 0x00000000004591ff in main (argc=2, argv=0x7fffffffdec8) at pbsd_main.c:1883
dmesg:
traps: pbs_server[22249] general protection ip:7f9c08a7a2c8 sp:7ffe520b5238 error:0 in libpthread-2.19.so[7f9c08a69000+18000]
valgrind:
==22381== Memcheck, a memory error detector
==22381== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==22381== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==22381== Command: pbs_server
==22381==
==22381==
==22381== HEAP SUMMARY:
==22381== in use at exit: 18,051 bytes in 53 blocks
==22381== total heap usage: 169 allocs, 116 frees, 42,410 bytes allocated
==22381==
==22382==
==22382== HEAP SUMMARY:
==22382== in use at exit: 19,755 bytes in 56 blocks
==22382== total heap usage: 172 allocs, 116 frees, 44,114 bytes allocated
==22382==
==22381== LEAK SUMMARY:
==22381== definitely lost: 0 bytes in 0 blocks
==22381== indirectly lost: 0 bytes in 0 blocks
==22381== possibly lost: 0 bytes in 0 blocks
==22381== still reachable: 18,051 bytes in 53 blocks
==22381== suppressed: 0 bytes in 0 blocks
==22381== Rerun with --leak-check=full to see details of leaked memory
==22381==
==22381== For counts of detected and suppressed errors, rerun with: -v
==22381== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==22383==
==22383== Process terminating with default action of signal 11 (SIGSEGV)
==22383== General Protection Fault
==22383== at 0x72192CB: __lll_unlock_elision (elision-unlock.c:33)
==22383== by 0x4E7E1A: unlock_node(pbsnode*, char const*, char const*, int) (u_lock_ctl.c:268)
==22383== by 0x4B7A66: mom_hierarchy_handler::make_default_hierarchy() (mom_hierarchy_handler.cpp:164)
==22383== by 0x4B898C: mom_hierarchy_handler::loadHierarchy() (mom_hierarchy_handler.cpp:433)
==22383== by 0x4B8AE7: mom_hierarchy_handler::initialLoadHierarchy() (mom_hierarchy_handler.cpp:472)
==22383== by 0x452629: pbsd_init(int) (pbsd_init.c:2299)
==22383== by 0x4591FE: main (pbsd_main.c:1883)
==22382== LEAK SUMMARY:
==22382== definitely lost: 0 bytes in 0 blocks
==22382== indirectly lost: 0 bytes in 0 blocks
==22382== possibly lost: 0 bytes in 0 blocks
==22382== still reachable: 19,755 bytes in 56 blocks
==22382== suppressed: 0 bytes in 0 blocks
==22382== Rerun with --leak-check=full to see details of leaked memory
==22382==
==22382== For counts of detected and suppressed errors, rerun with: -v
==22382== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==22383==
==22383== HEAP SUMMARY:
==22383== in use at exit: 325,348 bytes in 186 blocks
==22383== total heap usage: 297 allocs, 111 frees, 442,971 bytes allocated
==22383==
==22383== LEAK SUMMARY:
==22383== definitely lost: 134 bytes in 6 blocks
==22383== indirectly lost: 28 bytes in 3 blocks
==22383== possibly lost: 524 bytes in 17 blocks
==22383== still reachable: 324,662 bytes in 160 blocks
==22383== suppressed: 0 bytes in 0 blocks
==22383== Rerun with --leak-check=full to see details of leaked memory
==22383==
==22383== For counts of detected and suppressed errors, rerun with: -v
==22383== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
~
No other software have this behavior, i tested the machine by 2 days with full load without problens. Already try to update the processors microcode. Please, anybody have this behavior with torque 6.0.2 or some othe scenarios ?
Best regards.
This is no microcode fault. It is an outright lock balance issue in whatever software you're running (and not in glibc/libpthreads).
Don't try to unlock an already unlocked lock. That's forbidden behavior, and the reason for the trap.
For performance reasons, glibc doesn't bother to test for it and segfault, so a lot of broken code got away with it for a long time. The hardware implementations of lock elision, OTOH, do raise traps (Intel TSX, IBM Power 8, S390/X...), so this kind of breakage is going to become apparent everywhere, very fast.
As per question, I'm running into some memory leaking by getnameinfo. I'm using Ubuntu 12.04 (Linux scv 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux) with gcc version 4.6.3.
I'm linking my server executable with g++ and so far valgrind hasn't reported an issue. I've then added a simple call to getnameinfo to print out what are the network name and port of connecting clients.
And I get the following:
==4425==
==4425== HEAP SUMMARY:
==4425== in use at exit: 10 bytes in 1 blocks
==4425== total heap usage: 4,508 allocs, 4,507 frees, 134,939,153 bytes allocated
==4425==
==4425== 10 bytes in 1 blocks are definitely lost in loss record 1 of 1
==4425== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4425== by 0x50D7D71: strdup (strdup.c:43)
==4425== by 0x1484B861: ???
==4425== by 0x515B871: gethostbyaddr_r##GLIBC_2.2.5 (getXXbyYY_r.c:256)
==4425== by 0x5161D06: getnameinfo (getnameinfo.c:223)
==4425== by 0x404175: solsrv_run (solsrv.c:381)
==4425== by 0x404DAC: main (main.c:167)
==4425==
==4425== LEAK SUMMARY:
==4425== definitely lost: 10 bytes in 1 blocks
==4425== indirectly lost: 0 bytes in 0 blocks
==4425== possibly lost: 0 bytes in 0 blocks
==4425== still reachable: 0 bytes in 0 blocks
==4425== suppressed: 0 bytes in 0 blocks
==4425==
==4425== For counts of detected and suppressed errors, rerun with: -v
==4425== ERROR SUMMARY: 12 errors from 11 contexts (suppressed: 2 from 2)
What am I doing wrong?
Code is simply as follows:
struct sockaddr addr;
socklen_t addr_sz = sizeof(addr);
char host[NI_MAXHOST],
serv[NI_MAXSERV];
int infd = accept(srv_fd, (struct sockaddr*)&addr, &addr_sz);
if (infd == -1) {
... manage error on accept ...
}
if(getnameinfo((struct sockaddr *)&addr, addr_sz, host, NI_MAXHOST, serv, NI_MAXSERV, NI_NUMERICSERV)) {
strncpy(host, "<unknown host>", NI_MAXHOST-1);
strncpy(serv, "<unknown port>", NI_MAXSERV-1);
}
And there you have the leak...
I can confirm, that the leak is happening: for 6 clients connected valgrind found 60 bytes leaked (I guess the clients were connecting from the same host so if it's related to host name the growth is linear as expected).
Any idea?
Cheers
Eventually found the real leak.
When connecting to the server socket use name.local instead of localhost and/or the fully qualified name.
getnameinfo() will then leak.
I can reproduce the bug on 12.04, 12.10 both x64 and x86.
If I connect specifying .local on the name it leaks.
Cheers