data race between ip::tcp::socket::async_connect & io_service::run ? - c++

I'm getting helgrind errors using asio::ip::tcp::socket::async_connect while at the same time io_service::run is running in a thread pool. The reports is:
Possible data race during read of size 8 at 0x507bb50 by thread #6
==13316== at 0xDDF1CC: boost::asio::detail::epoll_reactor::descriptor_state::perform_io(unsigned int) (epoll_reactor.ipp:610)
==13316== by 0xDDF355: boost::asio::detail::epoll_reactor::descriptor_state::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) (epoll_reactor.ipp:648)
==13316== by 0xD7CF93: boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) (task_io_service_operation.hpp:37)
==13316== by 0xE17469: boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock&, boost::asio::detail::task_io_service::thread_info&, boost::asio::detail::op_queue&, boost::system::error_code const&) (task_io_service.ipp:394)
==13316== by 0xE1713B: boost::asio::detail::task_io_service::run(boost::system::error_code&) (task_io_service.ipp:146)
==13316== by 0xE176FC: boost::asio::io_service::run() (io_service.ipp:59)
==13316== by 0xF97421: Common::ServiceRunner::ServiceThread() (ServiceRunner.cpp:67)
....
This conflicts with a previous write of size 8 by thread #9
==13316== at 0xDDE5EF: boost::asio::detail::epoll_reactor::register_descriptor(int, boost::asio::detail::epoll_reactor::descriptor_state*&) (epoll_reactor.ipp:156)
==13316== by 0xE75C34: boost::asio::detail::reactive_socket_service_base::do_open(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, int, int, int, boost::system::error_code&) (reactive_socket_service_base.ipp:152)
==13316== by 0xE7B6E2: boost::asio::detail::reactive_socket_service::open(boost::asio::detail::reactive_socket_service::implementation_type&, boost::asio::ip::tcp const&, boost::system::error_code&) (reactive_socket_service.hpp:106)
==13316== by 0xE79FF7: boost::asio::stream_socket_service::open(boost::asio::detail::reactive_socket_service::implementation_type&, boost::asio::ip::tcp const&, boost::system::error_code&) (stream_socket_service.hpp:127)
==13316== by 0xE78904: void boost::asio::basic_socket >::async_connect >, boost::_bi::list3 >, boost::arg<1> (*)(), boost::_bi::value > > > > >(boost::asio::ip::basic_endpoint const&, boost::asio::detail::wrapped_handler >, boost::_bi::list3 >, boost::arg<1> (*)(), boost::_bi::value > > > > const&) (basic_socket.hpp:712)
==13316== by 0xE7376B: Common::Comms::TCPClient::HandleResolve(boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator) (TCPClient.cpp:297)
*I found that async_connect is not real problem, but opening a socket is, when io_service::run is running.*
thanks in advance for your help

Related

Conflicts in Multi-Threaded Xerces-c Application

I am writing a multi-threaded XML parser using Xercesc and I have followed the guide in https://xerces.apache.org/xerces-c/faq-parse-3.html#faq-6 by declaring the XMLPlatformUtils::Initialize() and XMLPlatformUtils::Terminate(); in the main thread while creating new parser and handler that can parse different XML files concurrently. The threads are created this way
std::vector<std::thread> thread_vector;
for (std::string i: input_folder){
SAXParser* parser = new SAXParser();
parser->setValidationScheme(SAXParser::ValSchemes::Val_Never);
parser->setDoSchema(false);
parser->setValidationSchemaFullChecking(false);
parser->cacheGrammarFromParse(false);
parser->setDoNamespaces(true); // optional
SAXHandler* handler= new SAXHandler();
parser->setDocumentHandler(handler);
parser->setErrorHandler(handler);
thread_vector.emplace_back(std::thread(parseFile,i,outputDir,handler,parser));
// parseFile(i,outputDir,handler,parser);
}
std::for_each(thread_vector.begin(),thread_vector.end(),std::mem_fn(&std::thread::join));
Inside the handle function I output some of the parsed items to an output file using std::ofstream and also use XMLString::transcode() to transcode XML strings. Although the result of the code running in single thread and multi-thread is similar, I find errors when I'm running it on valgrind with --tool=drd. Here is some of the error
==1045== Thread 7:
==1045== Conflicting load by thread 7 at 0x00126098 size 8
==1045== at 0x4D5E6E3: std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x111DA8: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x112A77: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*> > >::_M_run() (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4D01DE3: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x48424BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x4E30608: start_thread (pthread_create.c:477)
==1045== by 0x4F6A132: clone (clone.S:95)
==1045== Allocation context: BSS section of /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser
==1045== Other segment start (thread 2)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 2)
==1045== at 0x484687C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x49D48AF: xercesc_3_2::XMLMutexLock::~XMLMutexLock() (Mutexes.cpp:84)
==1045== by 0x4B26C76: xercesc_3_2::ICULCPTranscoder::transcode(char16_t const*, xercesc_3_2::MemoryManager*) (ICUTransService.cpp:949)
==1045== by 0x4B2810E: xercesc_3_2::PosixFileMgr::fileOpen(char16_t const*, bool, xercesc_3_2::MemoryManager*) (PosixFileMgr.cpp:59)
==1045== by 0x49D2D8F: xercesc_3_2::BinFileInputStream::BinFileInputStream(char16_t const*, xercesc_3_2::MemoryManager*) (BinFileInputStream.cpp:40)
==1045== by 0x4A2C2E2: xercesc_3_2::LocalFileInputSource::makeStream() const (LocalFileInputSource.cpp:162)
==1045== by 0x4A5C237: xercesc_3_2::ReaderMgr::createReader(xercesc_3_2::InputSource const&, bool, xercesc_3_2::XMLReader::RefFrom, xercesc_3_2::XMLReader::Types, xercesc_3_2::XMLReader::Sources, bool, unsigned long) (ReaderMgr.cpp:379)
==1045== by 0x4A586CE: xercesc_3_2::IGXMLScanner::scanReset(xercesc_3_2::InputSource const&) (IGXMLScanner2.cpp:1287)
==1045== by 0x4A79C0D: xercesc_3_2::XMLScanner::scanFirst(xercesc_3_2::InputSource const&, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:538)
==1045== by 0x4A7AE49: xercesc_3_2::XMLScanner::scanFirst(char16_t const*, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:514)
==1045== by 0x4A7B1C9: xercesc_3_2::XMLScanner::scanFirst(char const*, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:523)
==1045== by 0x111B65: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== Other segment start (thread 3)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 3)
==1045== at 0x484687C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x49D48AF: xercesc_3_2::XMLMutexLock::~XMLMutexLock() (Mutexes.cpp:84)
==1045== by 0x4B26C76: xercesc_3_2::ICULCPTranscoder::transcode(char16_t const*, xercesc_3_2::MemoryManager*) (ICUTransService.cpp:949)
==1045== by 0x11439B: SAXHandler::transcode[abi:cxx11](char16_t const*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x11F38D: SAXHandler::endElement(char16_t const*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4AA4BC4: xercesc_3_2::SAXParser::endElement(xercesc_3_2::XMLElementDecl const&, unsigned int, bool, char16_t const*) (SAXParser.cpp:865)
==1045== by 0x4A4C588: xercesc_3_2::IGXMLScanner::scanEndTag(bool&) (IGXMLScanner.cpp:1178)
==1045== by 0x4A509FB: xercesc_3_2::IGXMLScanner::scanNext(xercesc_3_2::XMLPScanToken&) (IGXMLScanner.cpp:377)
==1045== by 0x111B86: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x112A77: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*> > >::_M_run() (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4D01DE3: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x48424BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== Other segment start (thread 4)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 4)
==1045== at 0x484687C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x49D48AF: xercesc_3_2::XMLMutexLock::~XMLMutexLock() (Mutexes.cpp:84)
==1045== by 0x4B26C76: xercesc_3_2::ICULCPTranscoder::transcode(char16_t const*, xercesc_3_2::MemoryManager*) (ICUTransService.cpp:949)
==1045== by 0x11439B: SAXHandler::transcode[abi:cxx11](char16_t const*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x1173D0: SAXHandler::startElement(char16_t const*, xercesc_3_2::AttributeList&) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4AA48C8: xercesc_3_2::SAXParser::startElement(xercesc_3_2::XMLElementDecl const&, unsigned int, char16_t const*, xercesc_3_2::RefVectorOf<xercesc_3_2::XMLAttr> const&, unsigned long, bool, bool) (SAXParser.cpp:982)
==1045== by 0x4A4EED7: xercesc_3_2::IGXMLScanner::scanStartTagNS(bool&) (IGXMLScanner.cpp:2632)
==1045== by 0x4A509DA: xercesc_3_2::IGXMLScanner::scanNext(xercesc_3_2::XMLPScanToken&) (IGXMLScanner.cpp:386)
==1045== by 0x111B86: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x112A77: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*> > >::_M_run() (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4D01DE3: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x48424BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== Other segment start (thread 5)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 5)
==1045== at 0x484687C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x49D48AF: xercesc_3_2::XMLMutexLock::~XMLMutexLock() (Mutexes.cpp:84)
==1045== by 0x4B26C76: xercesc_3_2::ICULCPTranscoder::transcode(char16_t const*, xercesc_3_2::MemoryManager*) (ICUTransService.cpp:949)
==1045== by 0x4B2810E: xercesc_3_2::PosixFileMgr::fileOpen(char16_t const*, bool, xercesc_3_2::MemoryManager*) (PosixFileMgr.cpp:59)
==1045== by 0x49D2D8F: xercesc_3_2::BinFileInputStream::BinFileInputStream(char16_t const*, xercesc_3_2::MemoryManager*) (BinFileInputStream.cpp:40)
==1045== by 0x4A2C2E2: xercesc_3_2::LocalFileInputSource::makeStream() const (LocalFileInputSource.cpp:162)
==1045== by 0x4A5C237: xercesc_3_2::ReaderMgr::createReader(xercesc_3_2::InputSource const&, bool, xercesc_3_2::XMLReader::RefFrom, xercesc_3_2::XMLReader::Types, xercesc_3_2::XMLReader::Sources, bool, unsigned long) (ReaderMgr.cpp:379)
==1045== by 0x4A586CE: xercesc_3_2::IGXMLScanner::scanReset(xercesc_3_2::InputSource const&) (IGXMLScanner2.cpp:1287)
==1045== by 0x4A79C0D: xercesc_3_2::XMLScanner::scanFirst(xercesc_3_2::InputSource const&, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:538)
==1045== by 0x4A7AE49: xercesc_3_2::XMLScanner::scanFirst(char16_t const*, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:514)
==1045== by 0x4A7B1C9: xercesc_3_2::XMLScanner::scanFirst(char const*, xercesc_3_2::XMLPScanToken&) (XMLScanner.cpp:523)
==1045== by 0x111B65: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== Other segment start (thread 6)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 6)
==1045== at 0x4F590AF: __libc_write (write.c:26)
==1045== by 0x4F590AF: write (write.c:24)
==1045== by 0x4ED9E8C: _IO_file_write##GLIBC_2.2.5 (fileops.c:1181)
==1045== by 0x4EDB950: new_do_write (fileops.c:449)
==1045== by 0x4EDB950: _IO_new_do_write (fileops.c:426)
==1045== by 0x4EDB950: _IO_do_write##GLIBC_2.2.5 (fileops.c:423)
==1045== by 0x4EDBE92: _IO_file_overflow##GLIBC_2.2.5 (fileops.c:784)
==1045== by 0x4ED6972: putc (putc.c:31)
==1045== by 0x4D5E258: std::ostream::put(char) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x111B0D: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x112A77: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*> > >::_M_run() (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4D01DE3: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x48424BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x4E30608: start_thread (pthread_create.c:477)
==1045== by 0x4F6A132: clone (clone.S:95)
==1045== Other segment start (thread 6)
==1045== at 0x4F6A125: clone (clone.S:78)
==1045== Other segment end (thread 6)
==1045== at 0x4F590AF: __libc_write (write.c:26)
==1045== by 0x4F590AF: write (write.c:24)
==1045== by 0x4ED9E8C: _IO_file_write##GLIBC_2.2.5 (fileops.c:1181)
==1045== by 0x4EDB950: new_do_write (fileops.c:449)
==1045== by 0x4EDB950: _IO_new_do_write (fileops.c:426)
==1045== by 0x4EDB950: _IO_do_write##GLIBC_2.2.5 (fileops.c:423)
==1045== by 0x4EDBE92: _IO_file_overflow##GLIBC_2.2.5 (fileops.c:784)
==1045== by 0x4ED6972: putc (putc.c:31)
==1045== by 0x4D5E258: std::ostream::put(char) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x111B0D: parseFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*) (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x112A77: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SAXHandler*, xercesc_3_2::SAXParser*> > >::_M_run() (in /mnt/c/Users/dennnn/Documents/Projects/ParserCPP/parser)
==1045== by 0x4D01DE3: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==1045== by 0x48424BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
==1045== by 0x4E30608: start_thread (pthread_create.c:477)
==1045== by 0x4F6A132: clone (clone.S:95)
I wonder why an object that is declared inside a threaded object can still be accessed by threads from another object (which I thought should be independent from each other and in this case is the XMLString::transcode() and std::ofstream) and cause data race as well as how to approach this problem. Thank you.
UPDATE 24/06/2022
The ICULCPTranscoder::transcode or the object used to transcode the XMLCh* is declared in the XMLPlatformUtils::Initialize() and is shared among the threads so it will use mutex lock to operate. Also, it might be that the conflicting objects belong to the same cache line.

How can I debug Boost by viewing the file after precompilation?

I'm getting a very weird bug when defining a test suite with boost like this:
BOOST_AUTO_TEST_SUITE(zerocoin_implementation_tests)
The error looks like this:
terminate called after throwing an instance of 'std::length_error'
what(): basic_string::_M_create
Here's the relevant backtrace:
#5 0x00007ffff5ce6fe8 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff5ce2875 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x00007ffff5d7c949 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#8 0x00007ffff70afe15 in boost::unit_test::test_unit::test_unit(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::test_unit_type) () from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
#9 0x00007ffff70b0456 in boost::unit_test::test_suite::test_suite(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long) () from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
#10 0x00007ffff70b0612 in boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::decorator::collector&) ()
from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
From what I can tell, this has to do with Boost trying to create a max length string. I'd like to see exactly what it is doing. What's the best way of expanding boost macros to see the pre-compiled version?
Side Note
Weirdly, if I change the line very slightly to:
BOOST_AUTO_TEST_SUITE(zerocsoin_implementation_tests)
I get the following error:
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
And backtrace:
#6 0x00007ffff5ce7594 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x00007ffff70afe15 in boost::unit_test::test_unit::test_unit(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::test_unit_type) () from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
#8 0x00007ffff70b0456 in boost::unit_test::test_suite::test_suite(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long) () from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
#9 0x00007ffff70b0612 in boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::decorator::collector&) ()
from /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1
The source code for the file (and the rest of the project) can be found here: https://github.com/phoreproject/Phore/blob/segwit/src/test/zerocoin_implementation_tests.cpp
Diff that probably caused the bug: https://github.com/phoreproject/phore/compare/master...segwit#diff-bb4f094cc636d668944ed6af9b72c0d9

Two approaches:
Exception Breakpoints
Just start the test in the debugger and catch the exception.
In gdb you could do
(gdb) catch throw
Catchpoint 2 (throw)
which act like a general breakpoint. Visual Studio has a Manage Exeptions dialog.¹
Boost Test Breakpoints
For debugging Boost Test I like to set a break at the test_method member of the specific test case class I want to break at. E.g. with a test_runner that has a few nested suites like:
./test_runner --list_content
import*
utility*
xml*
xml_utilities*
child_text_test*
loggable_xml_path_test*
And we run these 3 tests like:
./test_runner -t import/utility/xml
Running 3 test cases...
*** No errors detected
To debug them with gdb I'd do
gdb ./test_runner
start -t import/utility/xml
Which stops at main, then I type:
break import::utility::xml
Auto completion helps, so to get the exact names, you can just pick from the completions:
xml
xml::as_element(xmlpp::Node const&)
xml::attr_value(xmlpp::Element const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
xml::attr_value(xmlpp::Node const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
xml::child_text[abi:cxx11](xmlpp::Element const&, char const*)
xml::child_text_test
xml::child_text_test::test_method()
xml::child_text_test_invoker()
xml::child_text_test_registrar62
xml::end_suite94_registrar94
xml::first_child(xmlpp::Element const&, char const*)
xml::get_content[abi:cxx11](xmlpp::Element const&)
xml::get_content[abi:cxx11](xmlpp::Node const*)
xml::is_attr_value(xmlpp::Node const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
xml::loggable_xml_path[abi:cxx11](xmlpp::Node const&)
xml::loggable_xml_path_test
xml::loggable_xml_path_test::test_method()
xml::loggable_xml_path_test_invoker()
xml::loggable_xml_path_test_registrar77
xml::trace_xml(xmlpp::Element const&, LogSource::LogTx)
xml::trace_xml_formatted(xmlpp::Element const&, LogSource::LogTx)
xml::xml_registrar20
xml::xml_utilities
xml::xml_utilities::test_method()
xml::xml_utilities_invoker()
xml::xml_utilities_registrar22
Pick the ones named test_method(), e.g.
break import::utility::xml::child_text_test::test_method()
Breakpoint 2 at 0x730762: file /path/src/import/utility/xml_tests.cpp, line 62.
Now you can continue execution and the debugger will automatic pause execution at the start of your unit test.
¹ see also
Make Visual Studio break on User (std::exception) Exceptions?
How do I make VC++'s debugger break on exceptions?

Best practice tracking uninitialised values using Valgrind C++

I'm working on a open source project called Palabos and am currently having difficulty tracking uninitialised values due to the high volume of files.
Valgrind is running with the following options:
--track-origins=yes --expensive-definedness-checks=yes
Is there a better way to identify which variable is the culprit?
==27210== Conditional jump or move depends on uninitialised value(s)
==27210== at 0x100096EEF: plb::contained(long, long, long, plb::Box3D const&) (geometry3D.h:240)
==27210== by 0x100096EC3: plb::contained(plb::Dot3D, plb::Box3D const&) (geometry3D.h:292)
==27210== by 0x100096A4E: plb::DetectBorderLineFunctional3D<int>::process(plb::Box3D, plb::ScalarField3D<int>&) (voxelizer.hh:769)
==27210== by 0x100081F3F: plb::BoxProcessingFunctional3D_S<int>::processGenericBlocks(plb::Box3D, std::__1::vector<plb::AtomicBlock3D*, std::__1::allocator<plb::AtomicBlock3D*> >) (dataProcessingFunctional3D.hh:62)
==27210== by 0x1001153CD: plb::BoxProcessor3D::process() (dataProcessingFunctional3D.cpp:115)
==27210== by 0x100111F33: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, std::__1::vector<plb::AtomicBlock3D*, std::__1::allocator<plb::AtomicBlock3D*> >) (atomicBlockOperations3D.cpp:39)
==27210== by 0x100170CB0: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, std::__1::vector<plb::MultiBlock3D*, std::__1::allocator<plb::MultiBlock3D*> >) (multiBlockOperations3D.cpp:334)
==27210== by 0x100170E12: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, plb::MultiBlock3D&) (multiBlockOperations3D.cpp:348)
==27210== by 0x100081D45: void plb::applyProcessingFunctional<int>(plb::BoxProcessingFunctional3D_S<int>*, plb::Box3D, plb::MultiScalarField3D<int>&) (multiDataProcessorWrapper3D.hh:231)
==27210== by 0x10007CEE5: void plb::detectBorderLine<int>(plb::MultiScalarField3D<int>&, plb::Box3D const&, long) (voxelizer.hh:703)
==27210== by 0x10007C69C: std::__1::unique_ptr<plb::MultiScalarField3D<int>, std::__1::default_delete<plb::MultiScalarField3D<int> > > plb::voxelize<double>(plb::TriangularSurfaceMesh<double> const&, plb::Box3D const&, long) (voxelizer.hh:157)
==27210== by 0x10007A893: std::__1::unique_ptr<plb::MultiScalarField3D<int>, std::__1::default_delete<plb::MultiScalarField3D<int> > > plb::voxelize<double>(plb::TriangularSurfaceMesh<double> const&, long, long) (voxelizer.hh:117)
==27210== Uninitialised value was created by a heap allocation
==27210== at 0x1003BBA5B: malloc (in /usr/local/Cellar/valgrind/HEAD/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==27210== by 0x1005037DD: operator new(unsigned long) (in /usr/lib/libc++.1.dylib)
==27210== by 0x100040B6F: std::__1::__split_buffer<plb::Dot3D, std::__1::allocator<plb::Dot3D>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<plb::Dot3D>&) (new:168)
==27210== by 0x10004046C: std::__1::__split_buffer<plb::Dot3D, std::__1::allocator<plb::Dot3D>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<plb::Dot3D>&) (__split_buffer:305)
==27210== by 0x10008CF40: void std::__1::vector<plb::Dot3D, std::__1::allocator<plb::Dot3D> >::__push_back_slow_path<plb::Dot3D const&>(plb::Dot3D const&) (vector:1571)
==27210== by 0x1000966FB: plb::DetectBorderLineFunctional3D<int>::process(plb::Box3D, plb::ScalarField3D<int>&) (vector:1592)
==27210== by 0x100081F3F: plb::BoxProcessingFunctional3D_S<int>::processGenericBlocks(plb::Box3D, std::__1::vector<plb::AtomicBlock3D*, std::__1::allocator<plb::AtomicBlock3D*> >) (dataProcessingFunctional3D.hh:62)
==27210== by 0x1001153CD: plb::BoxProcessor3D::process() (dataProcessingFunctional3D.cpp:115)
==27210== by 0x100111F33: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, std::__1::vector<plb::AtomicBlock3D*, std::__1::allocator<plb::AtomicBlock3D*> >) (atomicBlockOperations3D.cpp:39)
==27210== by 0x100170CB0: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, std::__1::vector<plb::MultiBlock3D*, std::__1::allocator<plb::MultiBlock3D*> >) (multiBlockOperations3D.cpp:334)
==27210== by 0x100170E12: plb::executeDataProcessor(plb::DataProcessorGenerator3D const&, plb::MultiBlock3D&) (multiBlockOperations3D.cpp:348)
==27210== by 0x100081D45: void plb::applyProcessingFunctional<int>(plb::BoxProcessingFunctional3D_S<int>*, plb::Box3D, plb::MultiScalarField3D<int>&) (multiDataProcessorWrapper3D.hh:231)
==27210==

c++ boost causing crashes

I wrote some multithreaded application with c++11 features and boost, but I got annoying problem.
code:
boost::asio::io_service io_service;
Client c(io_service, server->hostname.c_str(), server->port.c_str());
io_service.run(); //here is problem , it causes crashes
bt:
#0 0x414910 std::vector<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp>, std::allocator<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> > >::operator[](this=0x0, __n=0) (/usr/include/c++/4.8/bits/stl_vector.h:771)
#1 0x4131d4 boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>::dereference(this=0x7fffdb7fd250) (/usr/include/boost/asio/ip/basic_resolver_iterator.hpp:183)
#2 0x411f24 boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>::operator*(this=0x7fffdb7fd250) (/usr/include/boost/asio/ip/basic_resolver_iterator.hpp:123)
#3 0x40f8c6 Client::handle_resolve(this=0x7fffdb7fd800, err=..., endpoint_iterator=...) (*******Client.h:48)
#4 0x41c21f boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >::operator() (this=0x7fffdb7fd450, p=0x7fffdb7fd800, a1=..., a2=...) (/usr/include/boost/bind/mem_fn_template.hpp:280)
#5 0x41bd3b boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/bind/bind.hpp:392)
#6 0x41b74e boost::_bi::bind_t<void, boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >, boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/bind/bind_template.hpp:102)
#7 0x41adaf boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >, boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/asio/detail/bind_handler.hpp:127)
#8 0x419fe1 boost::asio::asio_handler_invoke<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >, boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/asio/handler_invoke_hook.hpp:64)
#9 0x4191d8 boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >, boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37)
#10 0x41776c boost::asio::detail::resolve_op<boost::asio::ip::tcp, boost::_bi::bind_t<void, boost::_mfi::mf2<void, Client, boost::system::error_code const&, boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp> >, boost::_bi::list3<boost::_bi::value<Client*>, boost::arg<1> (*)() (/usr/include/boost/asio/detail/resolve_op.hpp:112)
#11 0x409df4 boost::asio::detail::task_io_service_operation::complete(this=0x7fffc8000db0, owner=..., ec=..., bytes_transferred=0) (/usr/include/boost/asio/detail/task_io_service_operation.hpp:37)
#12 0x40c4f7 boost::asio::detail::task_io_service::do_run_one(this=0x7fffc8000910, lock=..., this_thread=..., ec=...) (/usr/include/boost/asio/detail/impl/task_io_service.ipp:384)
#13 0x40bf27 boost::asio::detail::task_io_service::run(this=0x7fffc8000910, ec=...) (/usr/include/boost/asio/detail/impl/task_io_service.ipp:153)
#14 0x40c8a7 boost::asio::io_service::run(this=0x7fffdb7fdbc0) (/usr/include/boost/asio/impl/io_service.ipp:59)
#15 0x41d9a3 some_function(server=0x67e0b0) (*********Server.cpp:17)
Client.h
void handle_resolve(const boost::system::error_code& err,
tcp::resolver::iterator endpoint_iterator)
{
tcp::endpoint endpoint = *endpoint_iterator; //48 line
ip_ = endpoint.address().to_string();
if (!err)
{
boost::asio::async_connect(socket_, endpoint_iterator,
boost::bind(&Client::handle_connect, this,
boost::asio::placeholders::error));
}
Just edited and added client part.

Looking at the stack trace, the problem is in Client::handle_resolve, line 48 of Client.h.
My best guess is that you're dereferencing an iterator without checking whether it's valid first. Perhaps you could post that function, if the error isn't obvious when you look at it.
UPDATE: My guess was correct: you're dereferencing the iterator before checking whether the operation succeeded. If it didn't, then the iterator won't be dereferencable.
Move the first two lines inside the if (!err) block.

Since I don't know io_service in detail, I have to guess a little bit. Which platform are you using? On boost 1.41 we had the problem that in QNX the automatic multithread capability was not detected correctly and therefore BOOST_ASIO_HAS_THREADS may not be set. This leads to a null_mutex being used instead of POSIX or Windows mutexes. Since std::vector itself is not thread safe, this could lead to the race condition with the segfault.

Valgrind complains bytes are being lost at std::string constructor

I am new to valgrind, so this could be missintepreting the output.
Valgrind seems to think bytes are being lost in the following lines:
if ( mysqlpp::StoreQueryResult result = query.store() ){
for ( size_t i = 0; i < result.num_rows(); ++i ){
hash = std::string( result[i]["hash"].data(), result[i]["hash"].length() );
label = std::string( result[i]["label"].data(), result[i]["label"].length() );
keywords = std::string( result[i]["keywords"].data(), result[i]["keywords"].length() );
task_raw = std::string( result[i]["task"].data(), result[i]["task"].length() );
solution_raw = std::string( result[i]["solution"].data(), result[i]["solution"].length() );
In fact it complains every time there is an std::string being constructed this way^^. I cannot figure out if this is a false-positive, if the way I'm copying is not the right way, or if the result is being leaked somehow.
Interestingly enough, it complains about another piece of code that doesn't use mysql++, but again a string constructor is used:
for ( auto & f : frames ){
for ( unsigned int i = 0; i < f.getTask().size(); i++ ){
if ( i + 1 < f.getTask().size() ){
auto t = f.getTask()[i].value();
auto n = f.getTask()[i+1].value();
pairs.push_back( std::string( t + n ) );
The line where a string is being constructed and pushed back, are possibly lost. Then a detailed output of malloc, operator new, std::string, etc are given.
The same story appears again, when copying a string from a std::istringstream:
std::copy( std::istream_iterator<std::string>(iss), std::istream_iterator<std::string>(), std::back_inserter<std::vector<std::string> >(tokens));
Are those false-positives? Does it have to do with using C++0x? Or are those legit leaks that may be originating from somewhere I have not yet detected to be leaking?
EDIT
Below is a sample of the reoccurring valgrind warning:
==18141== 9,555 bytes in 147 blocks are possibly lost in loss record 250 of 256
==18141== at 0x4C28BED: malloc (vg_replace_malloc.c:263)
==18141== by 0x7CA74C: operator new(unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4208: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /trunk/Build/icarus)
==18141== by 0x7B5904: char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) (in trunk/Build/icarus)
==18141== by 0x7B5A7C: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) (in trunk/Build/icarus)
==18141== by 0x610D65: Frame::mySQLloadFrameFromExample(std::string const&) (Frame.cpp:80)
==18141== by 0x6103FB: Frame::Frame(std::string const&) (Frame.cpp:5)
==18141== by 0x6054DB: Domain::mySQLloadFrames() (Domain.cpp:90)
==18141== by 0x60489D: Domain::Domain(std::string) (Domain.cpp:7)
==18141== by 0x628876: DomainController::loadDomainData() (DomainController.cpp:33)
==18141== by 0x628582: DomainController::DomainController() (DomainController.cpp:5)
==18141== by 0x627002: Controller::Controller() (Controller.cpp:26)
They are almost all identical, with the only difference being the actual line in code (i.e: the string constructor line).
The second type of warning is:
==18141== 3,562 bytes in 137 blocks are possibly lost in loss record 242 of 256
==18141== at 0x4C28BED: malloc (vg_replace_malloc.c:263)
==18141== by 0x7CA74C: operator new(unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4208: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in trunk/Build/icarus)
==18141== by 0x7B4EFA: std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) (in trunk/Build/icarus)
==18141== by 0x7B4FDF: std::string::reserve(unsigned long) (in trunk/Build/icarus)
==18141== by 0x784F2F: std::basic_istream<char, std::char_traits<char> >& std::operator>><char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (in trunk/Build/icarus)
==18141== by 0x61208E: std::istream_iterator<std::string, char, std::char_traits<char>, long>::_M_read() (stream_iterator.h:121)
==18141== by 0x613405: std::istream_iterator<std::string, char, std::char_traits<char>, long>::operator++() (stream_iterator.h:95)
==18141== by 0x6131BC: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move<false, false, std::input_iterator_tag>::__copy_m<std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:298)
==18141== by 0x612D09: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move_a<false, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:384)
==18141== by 0x6121FD: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::__copy_move_a2<false, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:422)
==18141== by 0x611C6A: std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > std::copy<std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > > >(std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::istream_iterator<std::string, char, std::char_traits<char>, long>, std::back_insert_iterator<std::vector<std::string, std::allocator<std::string> > >) (stl_algobase.h:454)
Which seems to be related with the std::istringstream code mentioned above.