I have the following sample source code which allow to send a HTTP request with libcurl.
I noticed massive memory usage. debug with valgrind show memory leak. Even that I used curl_easy_cleanup(). I used also curl_easy_reset(), curl_global_cleanup() but they don't have any effect at all and the memory leak persist.
I need to know how to free allocated memory by libcurl.
Here after a basic simple source code that initiate and clean libcurl.
int main()
{
CURL* pEasy = NULL;
pEasy = curl_easy_init();
if (pEasy != NULL)
{
curl_easy_setopt(pEasy, CURLOPT_USERNAME, "test");
curl_easy_setopt(pEasy, CURLOPT_PASSWORD, "test");
curl_easy_setopt(pEasy, CURLOPT_URL, "http:/192.168.22.217:3000");
curl_easy_perform(pEasy);
curl_easy_cleanup(pEasy);
}
}
Valgrind show memory leak of this program.
Find below the valgrind output
sudo valgrind --leak-check=full --show-leak-kinds=all ./mem
==7215== Memcheck, a memory error detector
==7215== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==7215== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==7215== Command: ./mem
==7215==
==7215==
==7215== HEAP SUMMARY:
==7215== in use at exit: 6,957 bytes in 67 blocks
==7215== total heap usage: 1,445 allocs, 1,378 frees, 199,287 bytes allocated
==7215==
==7215== 3 bytes in 1 blocks are still reachable in loss record 1 of 60
==7215== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7215== by 0x5136489: strdup (strdup.c:42)
==7215== by 0x5E65B14: PR_NewLogModule (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x5E707A9: ??? (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x4E8FB38: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E67344: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E674F7: curl_easy_init (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x400872: main (in /home/mem_test/mem)
==7215==
==7215== 3 bytes in 1 blocks are still reachable in loss record 2 of 60
==7215== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7215== by 0x5136489: strdup (strdup.c:42)
==7215== by 0x5E65B14: PR_NewLogModule (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x5E7081B: ??? (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x4E8FB38: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E67344: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E674F7: curl_easy_init (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x400872: main (in /home/mem_test/mem)
==7215==
==7215== 4 bytes in 1 blocks are still reachable in loss record 3 of 60
==7215== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7215== by 0x5136489: strdup (strdup.c:42)
==7215== by 0x5E65B14: PR_NewLogModule (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x5E707BC: ??? (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x4E8FB38: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E67344: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E674F7: curl_easy_init (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x400872: main (in /home/mem_test/mem)
==7215==
......................
==7215==
==7215== 1,344 bytes in 8 blocks are still reachable in loss record 60 of 60
==7215== at 0x4C2FB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7215== by 0x5E79749: PR_NewMonitor (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x5E6995D: ??? (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x5E70899: ??? (in /usr/lib/x86_64-linux-gnu/libnspr4.so)
==7215== by 0x4E8FB38: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E67344: ??? (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x4E674F7: curl_easy_init (in /usr/lib/x86_64-linux-gnu/libcurl-nss.so.4.4.0)
==7215== by 0x400872: main (in /home/mem_test/mem)
==7215==
==7215== LEAK SUMMARY:
==7215== definitely lost: 0 bytes in 0 blocks
==7215== indirectly lost: 0 bytes in 0 blocks
==7215== possibly lost: 0 bytes in 0 blocks
==7215== still reachable: 6,957 bytes in 67 blocks
==7215== suppressed: 0 bytes in 0 blocks
==7215==
==7215== For counts of detected and suppressed errors, rerun with: -v
==7215== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
How to clean properly the curl memories?
Doing a curl_global_init(0) before curl_easy_init() and doing a curl curl_global_cleanup() after curl_easy_cleanup() seems to fix the issue
==21041== Memcheck, a memory error detector
==21041== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==21041== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==21041== Command: ./a.out
==21041==
==21041==
==21041== HEAP SUMMARY:
==21041== in use at exit: 0 bytes in 0 blocks
==21041== total heap usage: 1,452 allocs, 1,452 frees, 418,773 bytes allocated
==21041==
==21041== All heap blocks were freed -- no leaks are possible
==21041==
==21041== For counts of detected and suppressed errors, rerun with: -v
==21041== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Related
I must edit the complex code that is written with Fortran. Actually, I do not have good knowledge about Fortran. So in this post, I hope receive your help.
In this code, someone declare variables by header file-commonkb.h :
common/cntrl2/SWNU,BB6(500,500)
real*8 SWNU,BB6
when I run:
valgrind --tool=memcheck --leak-check=full --log-file=log ./main
I have the error:
==4026== Memcheck, a memory error detector
==4026== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4026== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==4026== Command: ./main
==4026== Parent PID: 3252
==4026==
==4026== Warning: set address range perms: large range [0x324000, 0x28499000) (defined)
==4026==
==4026== HEAP SUMMARY:
==4026== in use at exit: 1,072 bytes in 4 blocks
==4026== total heap usage: 10,875 allocs, 10,871 frees, 733,306 bytes allocated
==4026==
==4026== 536 (24 direct, 512 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 4
==4026== at 0x290C8B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4026== by 0x292EF7F4: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x29474D25: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x2946C3B9: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x292EE867: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x284A9732: call_init (dl-init.c:72)
==4026== by 0x284A9732: _dl_init (dl-init.c:119)
==4026== by 0x2849A0C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4026==
==4026== 536 (24 direct, 512 indirect) bytes in 1 blocks are definitely lost in loss record 4 of 4
==4026== at 0x290C8B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4026== by 0x292EF7F4: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x29474D25: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x2946C48B: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x292EE867: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0)
==4026== by 0x284A9732: call_init (dl-init.c:72)
==4026== by 0x284A9732: _dl_init (dl-init.c:119)
==4026== by 0x2849A0C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4026==
==4026== LEAK SUMMARY:
==4026== definitely lost: 48 bytes in 2 blocks
==4026== indirectly lost: 1,024 bytes in 2 blocks
==4026== possibly lost: 0 bytes in 0 blocks
==4026== still reachable: 0 bytes in 0 blocks
==4026== suppressed: 0 bytes in 0 blocks
==4026==
==4026== For counts of detected and suppressed errors, rerun with: -v
==4026== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
How can I solve this problem?
Thank you for your help.
Valgring doesn`t detect memory errors.
I am using valgrind 3.11, gcc 5.4.0 under ubuntu and
have an incorrect code in my program, like in the sample.
I analyzed this program, using valgrind. But valgrind doesn't report about any errors.
#include <string.h>
int main(){
int a[3];
memcpy(a,"aaabbbcccdddeeefffggghhh", 24);
return 0;
}
What's wrong with valgrind?
valgrind doesn't know a, not its size then, while you stay in the stack it cannot detect the error
To compare, having that :
#include <string.h>
int main(){
int * a = new int[3];
memcpy(a,"aaabbbcccdddeeefffggghhh", 24);
return 0;
}
valgrind can detect the error because it knows the size of the allocated block :
pi#raspberrypi:/tmp $ valgrind ./a.out
==16164== Memcheck, a memory error detector
==16164== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16164== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==16164== Command: ./a.out
==16164==
==16164== Invalid write of size 8
==16164== at 0x4865F44: ??? (in /usr/lib/arm-linux-gnueabihf/libarmmem.so)
==16164== Address 0x4bc9f60 is 8 bytes inside a block of size 12 alloc'd
==16164== at 0x48485F0: operator new[](unsigned int) (vg_replace_malloc.c:417)
==16164== by 0x105A7: main (v.cc:3)
==16164==
==16164== Invalid write of size 8
==16164== at 0x4865F54: ??? (in /usr/lib/arm-linux-gnueabihf/libarmmem.so)
==16164== Address 0x4bc9f68 is 4 bytes after a block of size 12 alloc'd
==16164== at 0x48485F0: operator new[](unsigned int) (vg_replace_malloc.c:417)
==16164== by 0x105A7: main (v.cc:3)
==16164==
==16164==
==16164== HEAP SUMMARY:
==16164== in use at exit: 12 bytes in 1 blocks
==16164== total heap usage: 2 allocs, 1 frees, 20,236 bytes allocated
==16164==
==16164== LEAK SUMMARY:
==16164== definitely lost: 12 bytes in 1 blocks
==16164== indirectly lost: 0 bytes in 0 blocks
==16164== possibly lost: 0 bytes in 0 blocks
==16164== still reachable: 0 bytes in 0 blocks
==16164== suppressed: 0 bytes in 0 blocks
==16164== Rerun with --leak-check=full to see details of leaked memory
==16164==
==16164== For counts of detected and suppressed errors, rerun with: -v
==16164== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 6 from 3)
I'm actually developping a game in C++ and trying to do the AI with a script langage. To do so, i choosed Python2 with Python/C api.
My AI is actually working but there is a big problem : when I run valgrind on my program, there is a lot of error and memory leaks. So, I would know if this happened because of my code or by the API ?
Here is a summary of my class AI :
IA::IA()
{
setenv("PYTHONPATH",".",1);
Py_Initialize();
PyRun_SimpleString("import sys");
pName = PyBytes_FromString((char*)"Test");
pModule = PyImport_Import(pName);
pDict = PyModule_GetDict(pModule);
pFunc = PyDict_GetItemString(pDict, "push_f");
}
IA::~IA()
{
Py_DECREF(pValue);
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
}
void IA::LaunchIA(float x, float y, float z)
{
PyObject *toSend;
toSend = Py_BuildValue("(OOO)", TlistMob, TlistPlayer, pDPosIA);
pResult = PyObject_CallObject(pFunc, toSend);
PyErr_Print();
printf("return = %f\n", (float)PyInt_AsLong(pResult));
}
My (very) simple Python code :
def push_f(MobList, PlayerList, pos):
return 0
And the valgrind error (x1000) :
==11602== Memcheck, a memory error detector
==11602== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11602== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==11602== Command: ./a.out
==11602==
==11602== Invalid read of size 4
==11602== at 0x4FCE173: PyObject_Free (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F02FC2: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4FBDE9A: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F85BAD: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F872FF: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F88559: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EFF697: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F4B1E2: PyObject_Call (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x5021446: PyEval_CallObjectWithKeywords (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF45C5: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x502201B: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF0B88: PyEval_EvalCode (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== Address 0x693c020 is 2,560 bytes inside a block of size 2,731 free'd
==11602== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11602== by 0x4F81D28: PyMarshal_ReadLastObjectFromFile (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F85A22: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F872FF: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F88559: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EFF697: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F4B1E2: PyObject_Call (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x5021446: PyEval_CallObjectWithKeywords (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF45C5: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x502201B: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF0B88: PyEval_EvalCode (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F851B3: PyImport_ExecCodeModuleEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== Block was alloc'd at
==11602== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11602== by 0x4F81CDF: PyMarshal_ReadLastObjectFromFile (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F85A22: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F872FF: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F88559: PyImport_ImportModuleLevel (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EFF697: ??? (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F4B1E2: PyObject_Call (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x5021446: PyEval_CallObjectWithKeywords (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF45C5: PyEval_EvalFrameEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x502201B: PyEval_EvalCodeEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4EF0B88: PyEval_EvalCode (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602== by 0x4F851B3: PyImport_ExecCodeModuleEx (in /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0)
==11602==
==11602==
==11602== HEAP SUMMARY:
==11602== in use at exit: 491,741 bytes in 204 blocks
==11602== total heap usage: 3,301 allocs, 3,097 frees, 3,567,424 bytes allocated
==11602==
==11602== LEAK SUMMARY:
==11602== definitely lost: 0 bytes in 0 blocks
==11602== indirectly lost: 0 bytes in 0 blocks
==11602== possibly lost: 1,072 bytes in 2 blocks
==11602== still reachable: 490,669 bytes in 202 blocks
==11602== suppressed: 0 bytes in 0 blocks
==11602== Rerun with --leak-check=full to see details of leaked memory
==11602==
==11602== For counts of detected and suppressed errors, rerun with: -v
==11602== Use --track-origins=yes to see where uninitialised values come from
==11602== ERROR SUMMARY: 497 errors from 25 contexts (suppressed: 0 from 0)
In my main, you need to know that I'm creating only one IA object.
Am I doing something wrong ? Or is it just the API ?
(This is not a duplicate because I run valgrind on my C++ executable and not Python, my c++ is running the script)
Thanks in advance !!!
I initially suggested this as a duplicate. I don't think that's the case any more, but it still gives useful advice on removing false positives from the Valgrind output for Python programs.
Beyond that there are a number of specific issues with your program:
No error checking - the Python C API typically uses a NULL return value to indicate an error. There are clearly various ways of writing error checking code, but I'd be tempted to go for something like
IA::IA() :
pModule(NULL), pDict(NULL), pFunc(NULL), pName(NULL) // initially null initialize everything
{
setenv("PYTHONPATH",".",1);
Py_Initialize();
// I don't think you actually use this line, so maybe remove it
if (PyRun_SimpleString("import sys") == -1) goto error;
pName = PyBytes_FromString((char*)"Test");
if (!pName) goto error;
pModule = PyImport_Import(pName);
if (!pModule) goto error;
pDict = PyModule_GetDict(pModule);
if (!pDict) goto error;
pFunc = PyDict_GetItemString(pDict, "push_f");
if (!pFunc) goto error;
return; // completed OK
error:
Py_XDECREF(pName); // XDECREF is OK with NULL...
Py_XDECREF(pModule);
Py_XDECREF(pDict);
Py_XDECREF(pFunc);
PyErr_Print();
throw std::runtime_error(""); // ??? - possibly get and use the error string
// see https://stackoverflow.com/a/1418703/4657412
}
I'm aware people are suspicious of goto but in this case it's a reasonably clean way of jumping to an error handling block. You can structure it differently if you like.
The destructor doesn't decref pFunc, which looks like a memory leak.
IA::LaunchIA similarly is lacking working error checking.
IA::LaunchIA never decrefs toSend, pResult, TlistMob, TlistPlayer, pDPosIA. Some of these are related to the incompleteness of the code you show, but if they aren't decrefed then you're leaking memory.
You call PyErr_Print() without checking for an error. The documentation says:
Call this function only when the error indicator is set. (Otherwise it will cause a fatal error!)
I suspect that this might be your biggest problem.
Those are all the issues I can see. Without a minimal complete example it's impossible to actually check what you're seeing. Given you're using C++ you might be well advised to use/make a decent, object oriented wrapper for PyObject* to avoid having to worry about reference counting yourself - Boost Python has one.
This is what i get after executing my program with Valgrind:
1 jscherman#jscherman:~/ClionProjects/algo2-t4-tries$ g++ Set.hpp tests.cpp DiccString.hpp && valgrind --leak-check=yes --show-leak-kinds=all ./a.out
2 ==6823== Memcheck, a memory error detector
3 ==6823== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
4 ==6823== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
5 ==6823== Command: ./a.out
6 ==6823==
7 test_empty_dicc...ok
8 test_copy_constructor...ok
9 test_define_defined...ok
10 test_get..ok
11 test_remove...ok
12 test_remove_tiny...ok
13 test_keys...ok
14 ==6823==
15 ==6823== HEAP SUMMARY:
16 ==6823== in use at exit: 72,704 bytes in 1 blocks
17 ==6823== total heap usage: 282 allocs, 281 frees, 275,300 bytes allocated
18 ==6823==
19 ==6823== 72,704 bytes in 1 blocks are still reachable in loss record 1 of 1
20 ==6823== at 0x4C2DC10: malloc (vg_replace_malloc.c:299)
21 ==6823== by 0x4EC3EFF: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
22 ==6823== by 0x40104E9: call_init.part.0 (dl-init.c:72)
23 ==6823== by 0x40105FA: call_init (dl-init.c:30)
24 ==6823== by 0x40105FA: _dl_init (dl-init.c:120)
25 ==6823== by 0x4000CF9: ??? (in /lib/x86_64-linux-gnu/ld-2.23.so)
26 ==6823==
27 ==6823== LEAK SUMMARY:
28 ==6823== definitely lost: 0 bytes in 0 blocks
29 ==6823== indirectly lost: 0 bytes in 0 blocks
30 ==6823== possibly lost: 0 bytes in 0 blocks
31 ==6823== still reachable: 72,704 bytes in 1 blocks
32 ==6823== suppressed: 0 bytes in 0 blocks
33 ==6823==
34 ==6823== For counts of detected and suppressed errors, rerun with: -v
35 ==6823== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
It seems like there are no leaks as last line of the output says. Yet, we have also this line:
17 ==6823== total heap usage: 282 allocs, 281 frees, 275,300 bytes allocated
How is that i don't have any errors but there is still an allocation that hasn't been freed? Is there something wrong with my program or maybe something being done by Valgrind behind the scenes?
The backtrace reported by valgrind shows that the memory allocation in question was made in the initialization function of one of the shared libraries loaded by the application, apparently the C++ library itself.
It is quite common for shared libraries to make one-time allocations for various bits and pieces of data, but not bother to explicitly deallocate them, when they get unloaded.
This does not comprise a memory leak in your own code.
valgrind comes with a list of known allocations of this nature, it's called a "suppression list", for the explicit purpose of suppressing reports about these known one-off allocations.
But, occasionally, these suppression lists do miss an allocation, or two.
I have written a simple curl get wrapper to access http, https content. If I run my testcases with valgrind, I can see some still reachable sectors. Yes I know they are not as evil as lost references or definitely losts. But i want to keep my project clean.
If I disable SSL with curl_global_init(CURL_GLOBAL_NOTHING) no memleaks are detected. But then there is also no https support. So I guess its a libcrypt, libssl problem? What can I do to initialize and cleanup a https curl call correctly without having valgrind notifications?
jami#jami-mbp:rcc$ valgrind --leak-check=full --show-reachable=yes ./tests/testsuite
==7171== Memcheck, a memory error detector
==7171== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==7171== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==7171== Command: ./tests/testsuite
==7171==
==7171== Conditional jump or move depends on uninitialised value(s)
==7171== at 0x703784B: ASN1_STRING_set (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702552C: ASN1_mbstring_ncopy (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x7025753: ASN1_mbstring_copy (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x7026614: ASN1_STRING_to_UTF8 (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x7027A42: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x7027FA6: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702E4E2: ASN1_item_ex_d2i (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702F09F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702F2E7: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702EB50: ASN1_item_ex_d2i (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702F09F: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x702F2E7: ??? (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171==
OK (3)
==7171==
==7171== HEAP SUMMARY:
==7171== in use at exit: 64 bytes in 2 blocks
==7171== total heap usage: 10,535 allocs, 10,533 frees, 898,726 bytes allocated
==7171==
==7171== 32 bytes in 1 blocks are still reachable in loss record 1 of 2
==7171== at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7171== by 0x6F821FF: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x700A82E: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x6D012A9: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x6D031F8: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x6D08618: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x549A1E2: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0)
==7171== by 0x54A1B89: curl_global_init (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0)
==7171== by 0x559240: RCC::IO::HttpClient::get(std::string) (HttpClient.cpp:53)
==7171== by 0x559FFE: HttpClientTest::testBasicRequest() (HttpClientTest.cpp:55)
==7171== by 0x55BB11: CppUnit::TestCaller<HttpClientTest>::runTest() (TestCaller.h:166)
==7171== by 0x56FA081: CppUnit::TestCaseMethodFunctor::operator()() const (in /usr/lib/libcppunit-1.12.so.1.0.0)
==7171==
==7171== 32 bytes in 1 blocks are still reachable in loss record 2 of 2
==7171== at 0x4C2B3F8: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7171== by 0x6F821FF: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x700A84C: sk_new (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==7171== by 0x6D012A9: ??? (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x6D031F8: SSL_COMP_get_compression_methods (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x6D08618: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
==7171== by 0x549A1E2: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0)
==7171== by 0x54A1B89: curl_global_init (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0)
==7171== by 0x559240: RCC::IO::HttpClient::get(std::string) (HttpClient.cpp:53)
==7171== by 0x559FFE: HttpClientTest::testBasicRequest() (HttpClientTest.cpp:55)
==7171== by 0x55BB11: CppUnit::TestCaller<HttpClientTest>::runTest() (TestCaller.h:166)
==7171== by 0x56FA081: CppUnit::TestCaseMethodFunctor::operator()() const (in /usr/lib/libcppunit-1.12.so.1.0.0)
==7171==
==7171== LEAK SUMMARY:
==7171== definitely lost: 0 bytes in 0 blocks
==7171== indirectly lost: 0 bytes in 0 blocks
==7171== possibly lost: 0 bytes in 0 blocks
==7171== still reachable: 64 bytes in 2 blocks
==7171== suppressed: 0 bytes in 0 blocks
==7171==
==7171== For counts of detected and suppressed errors, rerun with: -v
==7171== Use --track-origins=yes to see where uninitialised values come from
==7171== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 2 from 2)
The curl wrapper (essential part of it):
static size_t WriteCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
std::string HttpClient::get(std::string url)
{
curl_global_init(CURL_GLOBAL_ALL);
CURL *curl = NULL;
CURLcode result;
std::string readBuffer = "";
curl = curl_easy_init();
if (curl == 0)
throw std::runtime_error("Unable to create CURL instance");
if (useProxy)
curl_easy_setopt(curl, CURLOPT_PROXY, proxyHost.c_str());
if (useAuth)
authenticate(curl);
if (followRedirect)
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
LOG_DEBUG("http client fetch " + url);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_POST, false);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
result = curl_easy_perform(curl);
if (result != CURLE_OK) {
curl_easy_cleanup(curl);
throw std::runtime_error(curl_easy_strerror(result));
}
curl_easy_cleanup(curl);
curl_global_cleanup();
return readBuffer;
}
Run the tests with:
valgrind --leak-check=full --show-reachable=yes -v ./tests/testsuite
Ok, I accept the good answer from Joachim Pileborg (a comment under the main question). OpenSSL use some data per process which survives the lifecycle of my program. So valgrind detect it as still reachable and my OpenSSL initialization/finalization seems to be correct. Thank you Joachim