This is a related post to this one as it deals with the same program, but i now implemented it iterative and not recursive anymore, but I still get SIGSEGV (but later) while running the program. I did some other changes to my program to narrow it down, and I figured that changing a vector of objects, to a vector of ptr to the objects on the heap does give me some extra rounds (about 200), but still crashes. I suggest that somehow my memory to save variables within the program gets exhausted, but as I dump the stacksize of the program:
rlimit rlim;
getrlimit(RLIMIT_STACK,&rlim);
std::cout << "rlim_cur ist:" << rlim.rlim_cur << std::endl;
std::cout << "rlim_max ist:" << rlim.rlim_max << std::endl;
the output is:
rlim_cur ist:8388608
rlim_max ist:18446744073709551615
and this seems to be pretty big and not used up, are there any other relevant limits to dump
to narrow my problem down further and hopefully solve it?
Here the dump of my debugger:
Program received signal SIGSEGV, Segmentation fault.
0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98
98 return left_over_capacity;
(gdb) backtrace
#0 0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98
#1 0x000000000040b9ab in Town::compare_by_capacity (eins=0x0, zwei=0x0) at ./solver/Darstellung.cpp:135
#2 0x00000000004124c7 in std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__a=..., __b=..., __c=...,
__comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:108
#3 0x0000000000411250 in std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=...,
__comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2260
#4 0x000000000040f111 in std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, long, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __depth_limit=21,
__comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2302
#5 0x000000000040de63 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>)
at /usr/include/c++/4.5/bits/stl_algo.h:5250
#6 0x000000000040ce5a in Solution_Stack::get_towns_by_capacity (this=0x7fffffffe010) at ./solver/Darstellung.cpp:331
#7 0x000000000040a6cf in solver::treat_towns_with_zero_capacity (ptr=0x7fffffffe010) at ./solver/Solver.cpp:184
#8 0x0000000000409ff2 in solver::solve_problem (ptr=0x7fffffffe010) at ./solver/Solver.cpp:94
#9 0x000000000041475f in main (argc=3, argv=0x7fffffffe208) at ./main/Main.cpp:50
The new dump after adding a check-function to trace down zero values:
#0 0x000000000040b2a0 in Town::get_cur_capacity (this=0x40) at ./solver/Darstellung.cpp:98
#1 0x000000000040b9e9 in Town::compare_by_index (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:139
#2 0x000000000040bad1 in Town::compare_by_index_inv (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:153
#3 0x00000000004127ea in std::__unguarded_partition<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, Town*, bool (*)(Town const*, Town const*)> (
__first=..., __last=..., __pivot=#0x631ef0, __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>)
at /usr/include/c++/4.5/bits/stl_algo.h:2229
#4 0x0000000000411444 in std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (
__first=..., __last=..., __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2261
#5 0x000000000040f2c5 in std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, long, bool (*)(Town const*, Town const*)> (
__first=..., __last=..., __depth_limit=7, __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>)
at /usr/include/c++/4.5/bits/stl_algo.h:2302
#6 0x000000000040e017 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=...,
__comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:5250
#7 0x000000000040d1e6 in Solution_Stack::get_partners_of_by_index_inv (this=0x7fffffffe010, id=523) at ./solver/Darstellung.cpp:371
#8 0x000000000040a4d7 in solver::treat_towns_considering_their_index (ptr=0x7fffffffe010) at ./solver/Solver.cpp:165
#9 0x000000000040a016 in solver::solve_problem (ptr=0x7fffffffe010) at ./solver/Solver.cpp:100
#10 0x0000000000414913 in main (argc=3, argv=0x7fffffffe208) at ./main/Main.cpp:50
Valgrind gave me this as a dump, just at the place where the problem used to occure anyways. I is quite confusing to me, due to its size:
==16150== Invalid read of size 4
==16150== at 0x40B2A0: Town::get_cur_capacity() const (Darstellung.cpp:98)
==16150== by 0x40B9AA: Town::compare_by_capacity(Town const*, Town const*) (Darstellung.cpp:135)
==16150== by 0x4124C6: void std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:108)
==16150== by 0x41124F: __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > > std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:2260)
==16150== by 0x40F110: void std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)) (stl_algo.h:2302)
==16150== by 0x40DE62: void std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:5250)
==16150== by 0x40CE59: Solution_Stack::get_towns_by_capacity() (Darstellung.cpp:331)
==16150== by 0x40A6CE: solver::treat_towns_with_zero_capacity(Solution_Stack*) (Solver.cpp:184)
==16150== by 0x409FF1: solver::solve_problem(Solution_Stack*) (Solver.cpp:94)
==16150== by 0x41475E: main (Main.cpp:50)
==16150== Address 0x8 is not stack'd, malloc'd or (recently) free'd
==16150==
==16150==
==16150== Process terminating with default action of signal 11 (SIGSEGV)
==16150== Access not within mapped region at address 0x8
==16150== at 0x40B2A0: Town::get_cur_capacity() const (Darstellung.cpp:98)
==16150== by 0x40B9AA: Town::compare_by_capacity(Town const*, Town const*) (Darstellung.cpp:135)
==16150== by 0x4124C6: void std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:108)
==16150== by 0x41124F: __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > > std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:2260)
==16150== by 0x40F110: void std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)) (stl_algo.h:2302)
==16150== by 0x40DE62: void std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:5250)
==16150== by 0x40CE59: Solution_Stack::get_towns_by_capacity() (Darstellung.cpp:331)
==16150== by 0x40A6CE: solver::treat_towns_with_zero_capacity(Solution_Stack*) (Solver.cpp:184)
==16150== by 0x409FF1: solver::solve_problem(Solution_Stack*) (Solver.cpp:94)
==16150== by 0x41475E: main (Main.cpp:50)
==16150== If you believe this happened as a result of a stack
==16150== overflow in your program's main thread (unlikely but
==16150== possible), you can try to increase the size of the
==16150== main thread stack using the --main-stacksize= flag.
==16150== The main thread stack size used in this run was 8388608.
==16150==
==16150== HEAP SUMMARY:
==16150== in use at exit: 771,174 bytes in 19,239 blocks
==16150== total heap usage: 9,821,251 allocs, 9,802,012 frees, 384,861,557 bytes allocated
==16150==
==16150== 50,678 bytes in 1,491 blocks are possibly lost in loss record 28 of 35
==16150== at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150== by 0x4ECBE6C: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150== by 0x4ECC08D: std::string::_M_mutate(unsigned long, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150== by 0x4ECC730: std::string::erase(unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150== by 0x407FB6: utility::split_helper(std::string, std::string) (Tools.cpp:28)
==16150== by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150== by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150== by 0x40539C: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:27)
==16150== by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150== by 0x414699: main (Main.cpp:34)
==16150==
==16150== 62,606 (11,928 direct, 50,678 indirect) bytes in 1,491 blocks are definitely lost in loss record 30 of 35
==16150== at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150== by 0x407F68: utility::split_helper(std::string, std::string) (Tools.cpp:24)
==16150== by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150== by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150== by 0x40539C: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:27)
==16150== by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150== by 0x414699: main (Main.cpp:34)
==16150==
==16150== 94,406 (18,440 direct, 75,966 indirect) bytes in 2,305 blocks are definitely lost in loss record 32 of 35
==16150== at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150== by 0x407F68: utility::split_helper(std::string, std::string) (Tools.cpp:24)
==16150== by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150== by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150== by 0x40573F: parser::get_finished_cities(std::vector<std::string, std::allocator<std::string> >, std::vector<City*, std::allocator<City*> >) (Parser.cpp:42)
==16150== by 0x40511A: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150== by 0x414699: main (Main.cpp:34)
==16150==
==16150== 178,720 (131,208 direct, 47,512 indirect) bytes in 1,491 blocks are definitely lost in loss record 35 of 35
==16150== at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150== by 0x40541B: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:28)
==16150== by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150== by 0x414699: main (Main.cpp:34)
==16150==
==16150== LEAK SUMMARY:
==16150== definitely lost: 161,576 bytes in 5,287 blocks
==16150== indirectly lost: 174,156 bytes in 5,287 blocks
==16150== possibly lost: 50,678 bytes in 1,491 blocks
==16150== still reachable: 384,764 bytes in 7,174 blocks
==16150== suppressed: 0 bytes in 0 blocks
==16150== Reachable blocks (those to which a pointer was found) are not shown.
==16150== To see them, rerun with: --leak-check=full --show-reachable=yes
==16150==
==16150== For counts of detected and suppressed errors, rerun with: -v
==16150== Use --track-origins=yes to see where uninitialised values come from
==16150== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 4 from 4)
#0 0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98
It looks like you're invoking get_cur_capacity on a NULL pointer. You're getting here from a comparator:
#1 0x000000000040b9ab in Town::compare_by_capacity (eins=0x0, zwei=0x0) at ./solver/Darstellung.cpp:135
which is comparing NULL to NULL. You further get here from a sort:
#5 0x000000000040de63 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>)
at /usr/include/c++/4.5/bits/stl_algo.h:5250
This is sorting std::vector<Town*>, and is called from:
#6 0x000000000040ce5a in Solution_Stack::get_towns_by_capacity (this=0x7fffffffe010) at ./solver/Darstellung.cpp:331
Most likely the vector you're sorting contains NULL pointers, and your compare_by_capacity function is not prepared to handle this eventuality. Either make sure the vector has no NULLs, or have compare_by_capacity explicitly check for NULL and do something sensible (eg, sort it before anything other than another NULL).
#1 0x000000000040b9e9 in Town::compare_by_index (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:139
This 0x40 looks like you have either uninitialized memory or corrupted memory. How are you resizing this vector?
It's really quite difficult to try to diagnose this without actually seeing any of your code.
If your code is compiled with debugging symbols and without optimization, then we can probably believe the Town::get_cur_capacity (this=0x0) line which means you got a null pointer in your vector somewhere. The code that put it in may have run at a totally different point in time.
If nulls are actually intended to be allowed in your vector then your sort predicate compare_by_capacity needs to be prepared to handle that eventuality.
In this case you may need to review the code that populates your vector, and valgrind may help you track down if there are memory problems in your code.
Related
I'm trying to install the mongo-c-driver and mongo-cxx-driver in a Docker image, but when I run my C++ program it throws a Segmentation fault. Any ideas? Valgrind says it's mongo's fault.
I have some IOT devices and in my own computer they run just fine.
I'm using:
ubuntu:focal as base image
mongo-c-driver version 1.16.2 (also tried with 1.17 and 1.15)
mongo-cxx-driver version 3.5 (also tried with 3.6)
MongoDB server version 4.2.8
g++ version (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Code:
std::string MongoDB::insertDocument(std::string database,
std::string collection, bsoncxx::document::view_or_value document)
{
try
{
mongocxx::uri uri(_connection_string);
mongocxx::client conn(uri); // seg fault occurs here
mongocxx::database db = conn[database];
mongocxx::collection coll = db[collection];
auto inserted = coll.insert_one(document.view());
bsoncxx::oid oid = inserted->inserted_id().get_oid().value;
return oid.to_string();
}
catch (const mongocxx::exception & e)
{
std::cerr << "WARNING: Unable to connect to MongoDB" << std::endl;
return "UNKNOWN";
}
}
Variable values:
_connection_string = "mongodb://burns"
database = "hopping"
collection = "executions"
document =
{
"created_at" : "2020.09.29 10:48:51",
"hostname" : "XXXXX",
"interface" : "wlp3s0mon",
"schedule" : [ 1 ],
"slot_size" : 200,
"type" : "MOCK_NODE"
}
Log:
==213== Invalid read of size 8
==213== at 0x5695C0A: mongoc_counter_clients_active_add (mongoc-counters.defs:37)
==213== by 0x5695C0A: mongoc_counter_clients_active_inc (mongoc-counters.defs:37)
==213== by 0x5695C0A: _mongoc_client_new_from_uri (mongoc-client.c:1101)
==213== by 0x5318074: mongocxx::v_noabi::client::client(mongocxx::v_noabi::uri const&, mongocxx::v_noabi::options::client const&) (in /usr/local/lib/libmongocxx.so.3.5.0)
==213== by 0x4B691D0: MongoDB::insertDocument(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bsoncxx::v_noabi::view_or_value<bsoncxx::v_noabi::document::view, bsoncxx::v_noabi::document::value>) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x4B69714: MongoDB::insertJSONDocument(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x4B61AD0: DataLogger::setExecutionParameters(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x1C232B: main (in /root/mock_node/build/mock_node)
==213== Address 0x38 is not stack'd, malloc'd or (recently) free'd
==213==
==213==
==213== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==213== Access not within mapped region at address 0x38
==213== at 0x5695C0A: mongoc_counter_clients_active_add (mongoc-counters.defs:37)
==213== by 0x5695C0A: mongoc_counter_clients_active_inc (mongoc-counters.defs:37)
==213== by 0x5695C0A: _mongoc_client_new_from_uri (mongoc-client.c:1101)
==213== by 0x5318074: mongocxx::v_noabi::client::client(mongocxx::v_noabi::uri const&, mongocxx::v_noabi::options::client const&) (in /usr/local/lib/libmongocxx.so.3.5.0)
==213== by 0x4B691D0: MongoDB::insertDocument(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bsoncxx::v_noabi::view_or_value<bsoncxx::v_noabi::document::view, bsoncxx::v_noabi::document::value>) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x4B69714: MongoDB::insertJSONDocument(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x4B61AD0: DataLogger::setExecutionParameters(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /usr/local/lib/influx_logging/libinflux_logging.so)
==213== by 0x1C232B: main (in /root/mock_node/build/mock_node)
==213== If you believe this happened as a result of a stack
==213== overflow in your program's main thread (unlikely but
==213== possible), you can try to increase the size of the
==213== main thread stack using the --main-stacksize= flag.
==213== The main thread stack size used in this run was 8388608.
==213==
==213== HEAP SUMMARY:
==213== in use at exit: 123,339 bytes in 262 blocks
==213== total heap usage: 3,719 allocs, 3,457 frees, 367,254 bytes allocated
==213==
==213== LEAK SUMMARY:
==213== definitely lost: 400 bytes in 1 blocks
==213== indirectly lost: 6,752 bytes in 20 blocks
==213== possibly lost: 576 bytes in 2 blocks
==213== still reachable: 115,611 bytes in 239 blocks
==213== suppressed: 0 bytes in 0 blocks
==213== Rerun with --leak-check=full to see details of leaked memory
==213==
==213== For lists of detected and suppressed errors, rerun with: -s
==213== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
Ps: mock_node is my executable and influxdb-logging is a library of mine.
Thank you very much,
I have no idea what could be causing this...
From Tutorial for mongocxx:
IMPORTANT: Before making any connections, you need to create one and only one instance of mongocxx::instance. This instance must exist for the entirety of your program.
So, you can add this as a static member of your MongoDB class:
header
class MongoDB {
//...
private:
static mongocxx::instance inst;
};
impl
mongocxx::instance MongoDB::inst{};
Here's an example where I connect to the the default mongodb server (mongodb://localhost:27017), use the database dbted and the collection movie, both of which previously created using the mongo cli.
I added a method to get a mongocxx::cursor to all documents in the collection.
#include <bsoncxx/builder/stream/document.hpp>
#include <bsoncxx/json.hpp>
#include <mongocxx/client.hpp>
#include <mongocxx/exception/exception.hpp>
#include <mongocxx/instance.hpp>
#include <mongocxx/uri.hpp>
#include <iostream>
class MongoDB {
public:
MongoDB() :
client{mongocxx::uri{"mongodb://localhost:27017"}},
db{client["dbted"]},
coll{db["movie"]}
{}
mongocxx::cursor all() { return coll.find({}); }
private
mongocxx::client client;
mongocxx::database db;
mongocxx::collection coll;
static mongocxx::instance inst;
};
mongocxx::instance MongoDB::inst{};
int main() {
try {
MongoDB db;
std::cout << "Connected successfully\n";
for(auto doc : db.all()) {
std::cout << bsoncxx::to_json(doc) << '\n';
}
} catch(const std::exception& ex) {
std::cerr << "Exception: " << ex.what() << std::endl;
return 1;
}
}
I already read similar posts on Stackoverflow but they do not solve my problem.
My problem
I get the following error message from Valgrind when I run the debug version of the executable with Valgrind.
==16631== Invalid read of size 8
==16631== at 0x217890: std::__detail::__variant::_Uninitialized<long long, true>::_Uninitialized<long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:222)
==16631== by 0x2178C9: std::__detail::__variant::_Variadic_union<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variadic_union<long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:342)
==16631== by 0x2178FD: std::__detail::__variant::_Variant_storage<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:399)
==16631== by 0x2156F0: std::__detail::__variant::_Copy_ctor_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:504)
==16631== by 0x215716: std::__detail::__variant::_Move_ctor_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:532)
==16631== by 0x21573C: std::__detail::__variant::_Copy_assign_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:592)
==16631== by 0x215762: std::__detail::__variant::_Move_assign_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:649)
==16631== by 0x215791: std::__detail::__variant::_Variant_base<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_base<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:701)
==16631== by 0x213F29: std::variant<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::variant<0ul, long long const&, long long, void>(std::in_place_index_t<0ul>, long long const&) (variant:1398)
==16631== by 0x213F6B: std::variant<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::variant<long long const&, void, void, long long, void>(long long const&) (variant:1369)
==16631== by 0x20B8ED: tgbot::Endpoints::deleteMessage(long long const&, int const&) const (Endpoints.cpp:1957)
==16631== by 0x2B17F6: ChatCleaner::run(std::shared_ptr<tgbot::Bot> const&) (ChatCleaner.cpp:40)
==16631== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==16631==
==16631==
==16631== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==16631== Access not within mapped region at address 0x0
==16631== at 0x217890: std::__detail::__variant::_Uninitialized<long long, true>::_Uninitialized<long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:222)
==16631== by 0x2178C9: std::__detail::__variant::_Variadic_union<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variadic_union<long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:342)
==16631== by 0x2178FD: std::__detail::__variant::_Variant_storage<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:399)
==16631== by 0x2156F0: std::__detail::__variant::_Copy_ctor_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:504)
==16631== by 0x215716: std::__detail::__variant::_Move_ctor_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:532)
==16631== by 0x21573C: std::__detail::__variant::_Copy_assign_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:592)
==16631== by 0x215762: std::__detail::__variant::_Move_assign_base<false, long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_storage<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:649)
==16631== by 0x215791: std::__detail::__variant::_Variant_base<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::_Variant_base<0ul, long long const&>(std::in_place_index_t<0ul>, long long const&) (variant:701)
==16631== by 0x213F29: std::variant<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::variant<0ul, long long const&, long long, void>(std::in_place_index_t<0ul>, long long const&) (variant:1398)
==16631== by 0x213F6B: std::variant<long long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<tools::InputFile> >::variant<long long const&, void, void, long long, void>(long long const&) (variant:1369)
==16631== by 0x20B8ED: tgbot::Endpoints::deleteMessage(long long const&, int const&) const (Endpoints.cpp:1957)
==16631== by 0x2B17F6: ChatCleaner::run(std::shared_ptr<tgbot::Bot> const&) (ChatCleaner.cpp:40)
==16631== If you believe this happened as a result of a stack
==16631== overflow in your program's main thread (unlikely but
==16631== possible), you can try to increase the size of the
==16631== main thread stack using the --main-stacksize= flag.
==16631== The main thread stack size used in this run was 8388608.
==16631==
==16631== HEAP SUMMARY:
==16631== in use at exit: 317,149 bytes in 4,488 blocks
==16631== total heap usage: 1,171,900 allocs, 1,167,412 frees, 359,098,054 bytes allocated
==16631==
==16631== LEAK SUMMARY:
==16631== definitely lost: 0 bytes in 0 blocks
==16631== indirectly lost: 0 bytes in 0 blocks
==16631== possibly lost: 0 bytes in 0 blocks
==16631== still reachable: 317,045 bytes in 4,487 blocks
==16631== suppressed: 104 bytes in 1 blocks
==16631== Reachable blocks (those to which a pointer was found) are not shown.
==16631== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==16631==
==16631== For lists of detected and suppressed errors, rerun with: -s
==16631== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Speicherzugriffsfehler (Speicherabzug geschrieben)
The suitable source code
"m_msgs_to_delete" is a static vector in the class ChatCleaner.
32void ChatCleaner::run(const tgbot::Bot::ptr &bot) noexcept
33{
34 if(!m_msgs_to_delete.empty())
35 {
36 //Reverse to have the oldest message at the end.
37 std::reverse(m_msgs_to_delete.begin(), m_msgs_to_delete.end());
38
39 //Delete the message.
40 bot->get_endpnts()->deleteMessage(m_msgs_to_delete.at(m_msgs_to_delete.size() - 1)->chat->id, m_msgs_to_delete.at(m_msgs_to_delete.size() - 1)->message_id);
41 m_msgs_to_delete.pop_back();
42
43 //Reverse again.
44 std::reverse(m_msgs_to_delete.begin(), m_msgs_to_delete.end());
45
46 m_last_time = tools::Tools::get_time();
47 m_msgs_to_delete.shrink_to_fit();
48 }
49}
1953 bool Endpoints::deleteMessage(const long long &chat_id, const int &message_id) const noexcept
{
//HTTP arguments
std::vector<tools::HttpArg> http_args;
1957 http_args.push_back(tools::HttpArg("chat_id", chat_id));
http_args.push_back(tools::HttpArg("message_id", message_id));
tools::HttpClient http_client("https://api.telegram.org/bot" + m_token + "/deleteMessage", http_args);
std::string json = http_client.send_post_req_multipart().m_body;
1963 rapidjson::Document doc;
doc.Parse(json.c_str());
if(doc.IsObject())
if(doc.HasMember("result"))
if(doc["result"].IsBool())
1969 return doc["result"].GetBool();
else
tools::Tools::write_err_log(Messages::field_does_not_contain_bool("result"));
else
tools::Tools::write_err_log_tmp(Messages::field_non_existent("result"));
else
tools::Tools::write_err_log(Messages::server_resp_not_json_object);
1976
1977 return false;
1978 }
From what I see the error occurs in line 40 of the snippet above as that is an excerpt from the file ChatCleaner.cpp. This line accesses a vector which can totally be the error as users with similar problems on Stackoverflow also had problems with vector or array accessing. Unfortunately, I cannot see what I did wrong. This error error happens absolutely arbitrarily and I cannot reproduce it.
Additional notes that might help
The code is from a my chat bot for Telegram which runs 24/7.
The error can happen a few minutes after starting the software or after days or weeks.
The vector contains std::shared_ptr<> of sent messages which shall be deleted after a while.
I already checked whether the vector content may be broken which was not the case when the error occurred. The chat and message ID were totally fine. I highly assume that is just a memory accessing problem but I cannot figure out the soure of the problem.
Question
What is the cause of the problem?
I finally found the source of the problem with the help of the hints #cdhowie gave. The member variable "chat" of the Message object caused the problem since as you can see here
if(doc.HasMember("chat"))
{
if(doc["chat"].IsObject())
chat = std::make_shared<Chat>(tools::Tools::get_json_as_string(doc["chat"]));
else
tools::Tools::write_err_log(Messages::field_does_not_contain_json_obj("chat"));
}
this part of the Message constructor only creates a chat instance if the passed HTTP response body contains the proper field. Therefore, this member access m_msgs_to_delete.at(m_msgs_to_delete.size() - 1)->chat->id in line 40 of ChatCleaner.cpp called a null pointer as "chat" does not exist on the heap in this case.
PS: The object doc above comes from the library rapidjson. The definition is rapidjson::Document doc;. Its purpose is parsing JSON objects and making data accessible. The whole source code is from my Telegram chat bot.
I somehow fixed a memory issue in my code but am not quite sure what actually happens and I would like to see if anyone could possibly give me a reasonable explanation.
The code involves doing some computation with a class ANN_Force, initially when setting up parameter values_of_biased_nodes, I use following code:
values_of_biased_nodes = bias;
and that gives me following memory error:
Program received signal SIGSEGV, Segmentation fault.
__memmove_avx_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:138
138 ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S: No such file
or directory.
#0 __memmove_avx_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:138
#1 0x00007ffff7a0c903 in std::vector<double, std::allocator<double> >::operator=(std::vector<double, std::allocator<double> > const&) () from /home/kengyangyao/.openmm/lib/libOpenMM.so
#2 0x00007ffff7a5bdcf in std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::operator=(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&) () from /home/kengyangyao/.openmm/lib/libOpenMM.so
#3 0x00007ffff76a3122 in OpenMM::ANN_Force::set_values_of_biased_nodes (this=0x6480e0, bias=...)
at /home/kengyangyao/Dropbox/temp_Linux/ANN_Force/openmmapi/src/ANN_Force.cpp:57
#4 0x000000000040cd4a in test_forward_and_backward_prop () at test_ANN_package.cpp:110
#5 0x000000000041468b in main (argc=1, argv=0x7fffffffdae8) at test_ANN_package.cpp:598
When I added a line:
values_of_biased_nodes.resize(bias.size());
values_of_biased_nodes = bias;
The problem disappeared but I have no idea why. My guess would be without resize, values_of_biased_nodes = bias; simply moves whatever in bias directly to values_of_biased_nodes without checking if the memory near values_of_biased_nodes is large enough to hold the values. I wonder if anyone could help me figure it out? Also I noticed that in some machines it works without resize, is it due to different memory management strategies, or simply because those machines have large enough memory?
Thank you!
UPDATED: I ran it with valgrind, here is output:
==13494== Memcheck, a memory error detector
==13494== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==13494== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==13494== Command: ./test_ANN_package
==13494==
/home/kengyangyao/.openmm/lib/plugins
running test_forward_and_backward_prop
==13494== Conditional jump or move depends on uninitialised value(s)
==13494== at 0x4F9689E: std::vector<double, std::allocator<double> >::operator=(std::vector<double, std::allocator<double> > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x4FE5DCE: std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::operator=(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x5374121: OpenMM::ANN_Force::set_values_of_biased_nodes(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >) (ANN_Force.cpp:57)
==13494== by 0x40CD49: test_forward_and_backward_prop() (test_ANN_package.cpp:110)
==13494== by 0x41468A: main (test_ANN_package.cpp:598)
==13494== Uninitialised value was created by a heap allocation
==13494== at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13494== by 0x41BACF: __gnu_cxx::new_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::allocate(unsigned long, void const*) (new_allocator.h:104)
==13494== by 0x41ACA5: std::allocator_traits<std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::allocate(std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&, unsigned long) (alloc_traits.h:491)
==13494== by 0x419809: std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_allocate(unsigned long) (stl_vector.h:170)
==13494== by 0x41904A: std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_create_storage(unsigned long) (stl_vector.h:185)
==13494== by 0x41728C: std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_Vector_base(unsigned long, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&) (stl_vector.h:136)
==13494== by 0x4155AB: std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector(unsigned long, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&) (stl_vector.h:278)
==13494== by 0x415072: OpenMM::ANN_Force::ANN_Force() (in /home/kengyangyao/Dropbox/temp_Linux/ANN_Force/openmmapi/tests/test_ANN_package)
==13494== by 0x40C8AA: test_forward_and_backward_prop() (test_ANN_package.cpp:103)
==13494== by 0x41468A: main (test_ANN_package.cpp:598)
==13494==
==13494== Invalid write of size 1
==13494== at 0x4C3245C: memcpy#GLIBC_2.2.5 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13494== by 0x4F96902: std::vector<double, std::allocator<double> >::operator=(std::vector<double, std::allocator<double> > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x4FE5DCE: std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::operator=(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x5374121: OpenMM::ANN_Force::set_values_of_biased_nodes(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >) (ANN_Force.cpp:57)
==13494== by 0x40CD49: test_forward_and_backward_prop() (test_ANN_package.cpp:110)
==13494== by 0x41468A: main (test_ANN_package.cpp:598)
==13494== Address 0x3f947ae147ae147b is not stack'd, malloc'd or (recently) free'd
==13494==
==13494==
==13494== Process terminating with default action of signal 11 (SIGSEGV)
==13494== General Protection Fault
==13494== at 0x4C3245C: memcpy#GLIBC_2.2.5 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13494== by 0x4F96902: std::vector<double, std::allocator<double> >::operator=(std::vector<double, std::allocator<double> > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x4FE5DCE: std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >::operator=(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&) (in /home/kengyangyao/.openmm/lib/libOpenMM.so)
==13494== by 0x5374121: OpenMM::ANN_Force::set_values_of_biased_nodes(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >) (ANN_Force.cpp:57)
==13494== by 0x40CD49: test_forward_and_backward_prop() (test_ANN_package.cpp:110)
==13494== by 0x41468A: main (test_ANN_package.cpp:598)
==13494==
==13494== HEAP SUMMARY:
==13494== in use at exit: 2,016,954 bytes in 7,600 blocks
==13494== total heap usage: 9,419 allocs, 1,819 frees, 2,218,864 bytes allocated
==13494==
==13494== LEAK SUMMARY:
==13494== definitely lost: 0 bytes in 0 blocks
==13494== indirectly lost: 0 bytes in 0 blocks
==13494== possibly lost: 0 bytes in 0 blocks
==13494== still reachable: 2,016,954 bytes in 7,600 blocks
==13494== of which reachable via heuristic:
==13494== stdstring : 196 bytes in 6 blocks
==13494== suppressed: 0 bytes in 0 blocks
==13494== Reachable blocks (those to which a pointer was found) are not shown.
==13494== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==13494==
==13494== For counts of detected and suppressed errors, rerun with: -v
==13494== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
[1] 13494 segmentation fault (core dumped) valgrind --leak-check=yes --track-origins=yes ./test_ANN_package
seems that some memory is not correctly handled?
Assigning one vector to another does not require that you resize the target vector before performing the assignment; that would be a terrible design. I suspect you have other memory issues in play and this just reveals them. Have you tried running this under valgrind?
After looking at your valgrind output, it's clear that you have memory corruption but it's difficult to say exactly what's going on without seeing the offending lines of source code. I recommend running under gdb and breaking on the line that generates the first valgrind error. If you can figure the problem out on your own, then fix it and rerun it under valgrind, repeating as necessary until you fix all the errors. If you cannot figure out the problem, please post the offending lines +- 10 lines and I'll take a look.
I'm linking a library against opencv libraries. I noticed that when I run the unit test executable (using GTest) for that library, that valgrind reports a memory leak/conditional jump or move depends on uninitialised value, even when the test doesn't do anything at all (just empty test method).
I traced that issue back to a single line in my CMakeLists.txt - when I remove the linkage against opencv, the memory leak goes away. This is the releavant snippet:
find_package(OpenCV REQUIRED)
set(libImageSources src/SourceImageFile.cpp)
add_library(image SHARED ${libImageSources})
# removing this line fixes the leak:
target_link_libraries(image ${OpenCV_LIBS})
Otherwise when runnig an empty unit test, I get an error like this:
==18681== Memcheck, a memory error detector
==18681== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18681== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==18681== Command: ./image_test /Users/max/Documents/playground/cpp/image-server/tests
==18681==
==18681== Conditional jump or move depends on uninitialised value(s)
==18681== at 0x7FFF5FC24A27: bcmp (in /usr/lib/dyld)
==18681== by 0x7FFF5FC11904: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC16B7A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC10A6E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08468: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08188: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07EED: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07642: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08705: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==18681==
==18681== Use of uninitialised value of size 8
==18681== at 0x7FFF5FC24A3F: bcmp (in /usr/lib/dyld)
==18681== by 0x7FFF5FC11904: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC16B7A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC10A6E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08468: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08188: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07EED: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07642: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08705: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==18681==
==18681== Use of uninitialised value of size 8
==18681== at 0x7FFF5FC24A44: bcmp (in /usr/lib/dyld)
==18681== by 0x7FFF5FC11904: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC16B7A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC10A6E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08468: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08188: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07EED: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07642: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08705: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==18681==
==18681== Conditional jump or move depends on uninitialised value(s)
==18681== at 0x7FFF5FC11907: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC16B7A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC10A6E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08468: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08188: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07EED: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC07642: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC08705: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==18681== by 0x7FFF5FC0E85D: ImageLoader::recursiveLoadLibraries(ImageLoader::LinkContext const&, bool, ImageLoader::RPathChain const&) (in /usr/lib/dyld)
==18681==
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from ImageProcessingConfigurationService
[ RUN ] ImageProcessingConfigurationService.evaluateConfigurationFile
[ OK ] ImageProcessingConfigurationService.evaluateConfigurationFile (13 ms)
[----------] 1 test from ImageProcessingConfigurationService (28 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (114 ms total)
[ PASSED ] 1 test.
==18681==
==18681== HEAP SUMMARY:
==18681== in use at exit: 1,346,030 bytes in 1,453 blocks
==18681== total heap usage: 2,926 allocs, 1,473 frees, 3,045,052 bytes allocated
==18681==
==18681== LEAK SUMMARY:
==18681== definitely lost: 10,580 bytes in 137 blocks
==18681== indirectly lost: 10,032 bytes in 152 blocks
==18681== possibly lost: 9,568 bytes in 183 blocks
==18681== still reachable: 43,598 bytes in 520 blocks
==18681== suppressed: 1,272,252 bytes in 461 blocks
==18681== Rerun with --leak-check=full to see details of leaked memory
==18681==
==18681== For counts of detected and suppressed errors, rerun with: -v
==18681== Use --track-origins=yes to see where uninitialised values come from
==18681== ERROR SUMMARY: 36 errors from 4 contexts (suppressed: 0 from 0)
Compiler is clang++ Apple LLVM version 7.3.0 (clang-703.0.31), OS X El Capitan (10.11.1).
Anyone else has had an issue like that? It seems stuff works nontheless, but the error creates noise that I'd rather ignore, I'm not sure if I can fix the root cause.
The issue seems like a false positive reported by valgrind which I can ignore as it's not caused by code.
I tried (at the time of writing) the latest valgrind suppressions file for macOS which didn't work for me, the issue was still reported.
I let valgrind dump the suppressions needed to ignore this error, using a command like:
valgrind --leak-check=full --show-reachable=yes --error-limit=no --gen-suppressions=all --log-file=l.log ./image_test
And then used a tool to parse the log output (the awk script didn't work for me as it contained a syntax error, I used the perl version) into a suppressions file:
cat l.log | ./parse_valgrind_suppressions.pl > s.supp
Now running the test with the generated suppresions file ignores the memory leak:
valgrind --suppressions=./s.supp ./image_test
I am new to valgrind, so this could be missintepreting the output.
Valgrind seems to think bytes are being lost in the following lines:
if ( mysqlpp::StoreQueryResult result = query.store() ){
for ( size_t i = 0; i < result.num_rows(); ++i ){
hash = std::string( result[i]["hash"].data(), result[i]["hash"].length() );
label = std::string( result[i]["label"].data(), result[i]["label"].length() );
keywords = std::string( result[i]["keywords"].data(), result[i]["keywords"].length() );
task_raw = std::string( result[i]["task"].data(), result[i]["task"].length() );
solution_raw = std::string( result[i]["solution"].data(), result[i]["solution"].length() );
In fact it complains every time there is an std::string being constructed this way^^. I cannot figure out if this is a false-positive, if the way I'm copying is not the right way, or if the result is being leaked somehow.
Interestingly enough, it complains about another piece of code that doesn't use mysql++, but again a string constructor is used:
for ( auto & f : frames ){
for ( unsigned int i = 0; i < f.getTask().size(); i++ ){
if ( i + 1 < f.getTask().size() ){
auto t = f.getTask()[i].value();
auto n = f.getTask()[i+1].value();
pairs.push_back( std::string( t + n ) );
The line where a string is being constructed and pushed back, are possibly lost. Then a detailed output of malloc, operator new, std::string, etc are given.
The same story appears again, when copying a string from a std::istringstream:
std::copy( std::istream_iterator<std::string>(iss), std::istream_iterator<std::string>(), std::back_inserter<std::vector<std::string> >(tokens));
Are those false-positives? Does it have to do with using C++0x? Or are those legit leaks that may be originating from somewhere I have not yet detected to be leaking?
EDIT
Below is a sample of the reoccurring valgrind warning:
==18141== 9,555 bytes in 147 blocks are possibly lost in loss record 250 of 256
==18141== at 0x4C28BED: malloc (vg_replace_malloc.c:263)
==18141== by 0x7CA74C: operator new(unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4208: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /trunk/Build/icarus)
==18141== by 0x7B5904: char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) (in trunk/Build/icarus)
==18141== by 0x7B5A7C: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) (in trunk/Build/icarus)
==18141== by 0x610D65: Frame::mySQLloadFrameFromExample(std::string const&) (Frame.cpp:80)
==18141== by 0x6103FB: Frame::Frame(std::string const&) (Frame.cpp:5)
==18141== by 0x6054DB: Domain::mySQLloadFrames() (Domain.cpp:90)
==18141== by 0x60489D: Domain::Domain(std::string) (Domain.cpp:7)
==18141== by 0x628876: DomainController::loadDomainData() (DomainController.cpp:33)
==18141== by 0x628582: DomainController::DomainController() (DomainController.cpp:5)
==18141== by 0x627002: Controller::Controller() (Controller.cpp:26)
They are almost all identical, with the only difference being the actual line in code (i.e: the string constructor line).
The second type of warning is:
==18141== 3,562 bytes in 137 blocks are possibly lost in loss record 242 of 256
==18141== at 0x4C28BED: malloc (vg_replace_malloc.c:263)
==18141== by 0x7CA74C: operator new(unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4208: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in trunk/Build/icarus)
==18141== by 0x7B4EFA: std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4FDF: std::string::reserve(unsigned long) (in trunk/Build/icarus)
==18141== by 0x784F2F: std::basic_istream<char, std::char_traits<char> >& std::operator>><char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (in trunk/Build/icarus)
==18141== by 0x61208E: std::istream_iterator<std::string, char, std::char_traits<char>, long>::_M_read() (stream_iterator.h:121)
==18141== by 0x613405: std::istream_iterator<std::string, char, std::char_traits<char>, long>::operator++() (stream_iterator.h:95)
==18141== by 0x6131BC: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move<false, false, std::input_iterator_tag>::__copy_m<std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:298)
==18141== by 0x612D09: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move_a<false, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:384)
==18141== by 0x6121FD: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move_a2<false, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:422)
==18141== by 0x611C6A: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::copy<std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:454)
Which seems to be related with the std::istringstream code mentioned above.