unable to create a C++ ruby extension - c++

I have problems creating a ruby extension to export a C++ library I wrote to ruby under OSX. This simple example:
#include <boost/regex.hpp>
extern "C" void Init_bayeux()
{
boost::regex expression("^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?");
}
results in a bad_cast exception being thrown:
#0 0x00000001014663bd in __cxa_throw ()
#1 0x00000001014cf6b2 in __cxa_bad_cast ()
#2 0x00000001014986f9 in std::use_facet<std::collate<char> > ()
#3 0x0000000101135a4f in boost::re_detail::cpp_regex_traits_base<char>::imbue (this=0x7fff5fbfe4d0, l=#0x7fff5fbfe520) at cpp_regex_traits.hpp:218
#4 0x0000000101138d42 in cpp_regex_traits_base (this=0x7fff5fbfe4d0, l=#0x7fff5fbfe520) at cpp_regex_traits.hpp:173
#5 0x000000010113eda6 in boost::re_detail::create_cpp_regex_traits<char> (l=#0x7fff5fbfe520) at cpp_regex_traits.hpp:859
#6 0x0000000101149bee in cpp_regex_traits (this=0x101600200) at cpp_regex_traits.hpp:880
#7 0x0000000101142758 in regex_traits (this=0x101600200) at regex_traits.hpp:75
#8 0x000000010113d68c in regex_traits_wrapper (this=0x101600200) at regex_traits.hpp:169
#9 0x000000010113bae1 in regex_data (this=0x101600060) at basic_regex.hpp:166
#10 0x000000010113981e in basic_regex_implementation (this=0x101600060) at basic_regex.hpp:202
#11 0x0000000101136e1a in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::do_assign (this=0x7fff5fbfe710, p1=0x100540ae0 "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?", p2=0x100540b19 "", f=0) at basic_regex.hpp:652
#12 0x0000000100540a66 in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign (this=0x7fff5fbfe710, p1=0x100540ae0 "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?", p2=0x100540b19 "", f=0) at basic_regex.hpp:379
#13 0x0000000100540a13 in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign (this=0x7fff5fbfe710, p=0x100540ae0 "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?", f=0) at basic_regex.hpp:364
#14 0x000000010054096e in basic_regex (this=0x7fff5fbfe710, p=0x100540ae0 "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?", f=0) at basic_regex.hpp:333
#15 0x00000001005407e2 in Init_bayeux () at bayeux.cpp:10
#16 0x0000000100004593 in dln_load (file=0x1008bc000 "/Users/todi/sioux/lib/debug/rack/bayeux.bundle") at dln.c:1293
I compile the extension with:
g++ ./source/rack/bayeux.cpp -o /Users/todi/sioux/obj/debug/rack/bayeux.o -Wall -pedantic -Wno-parentheses -Wno-sign-compare -fno-common -c -pipe -I/Users/todi/sioux/source -ggdb -O0
And finally link the dynamic library with:
g++ -o /Users/todi/sioux/lib/debug/rack/bayeux.bundle -bundle -ggdb /Users/todi/sioux/obj/debug/rack/bayeux.o -L/Users/todi/sioux/lib/debug -lrack -lboost_regex-mt-d -lruby
I've searched the web and tried all kind of link and compiler switches. If I build a executable there is no such problem. Does someone else had such a problem and found a solution?
I've further investigated this and found that the function causing the exception looks like this:
std::locale loc = std::locale("C");
std::use_facet< std::collate<char> >( loc );
In the source of std::collate<> I found the throw statment:
use_facet(const locale& __loc)
{
const size_t __i = _Facet::id._M_id();
const locale::facet** __facets = __loc._M_impl->_M_facets;
if (__i >= __loc._M_impl->_M_facets_size || !__facets[__i])
__throw_bad_cast();
#ifdef __GXX_RTTI
return dynamic_cast<const _Facet&>(*__facets[__i]);
#else
return static_cast<const _Facet&>(*__facets[__i]);
#endif
}
Does this makes any sense to you?
Update: I've tried Jan's suggestion:
Todis-MacBook-Pro:rack todi$ g++ -shared -fpic -o bayeux.bundle bayeux.cpp
Todis-MacBook-Pro:rack todi$ ruby -I. -rbayeux -e 'puts :ok'
terminate called after throwing an instance of 'std::bad_cast'
what(): std::bad_cast
Abort trap
versions:
Todis-MacBook-Pro:rack todi$ ruby -v
ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-darwin10.6.0]
Todis-MacBook-Pro:rack todi$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin10/4.5.2/lto-wrapper
Target: x86_64-apple-darwin10
Configured with: ../gcc-4.5.2/configure --prefix=/opt/local --build=x86_64-apple-darwin10 --enable-languages=c,c++,objc,obj-c++,fortran,java --libdir=/opt/local/lib/gcc45 --includedir=/opt/local/include/gcc45 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-4.5 --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-4.5 --with-gxx-include-dir=/opt/local/include/gcc45/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --enable-stage1-checking --disable-multilib --enable-fully-dynamic-string
Thread model: posix
gcc version 4.5.2 (GCC)
Update:
It's not the bound-check in use_facet() that throws, but the next line, that actually does a dynamic cast. This example boils it down to maybe something with RTTI:
#define private public
#include <locale>
#include <iostream>
#include <typeinfo>
extern "C" void Init_bayeux()
{
std::locale loc = std::locale("C");
printf( "size: %i\n", loc._M_impl->_M_facets_size );
printf( "id: %i\n", std::collate< char >::id._M_id() );
const std::locale::facet& fac = *loc._M_impl->_M_facets[ std::collate< char >::id._M_id() ];
printf( "name: %s\n", typeid( fac ).name());
printf( "name: %s\n", typeid( std::collate<char> ).name());
const std::type_info& a = typeid( fac );
const std::type_info& b = typeid( std::collate<char> );
printf( "equal: %i\n", !a.before( b ) && !b.before( a ) );
dynamic_cast< const std::collate< char >& >( fac );
}
I've used printf() because usage of cout also fails. The output of the code above is:
size: 28
id: 5
name: St7collateIcE
name: St7collateIcE
equal: 1
terminate called after throwing an instance of 'std::bad_cast'
what(): std::bad_cast
Abort trap
Build with:
g++ -shared -fpic -o bayeux.bundle bayeux.cpp && ruby -I. -rbayeux -e 'puts :ok'
Update:
If I rename Init_bayeux to main() and link it to an executable, the output is the same, but no call to terminate.
Update:
When I write a little program to load the shared library and to execute Init_bayeux(), again, no exception is thrown:
#include <dlfcn.h>
int main()
{
void* handle = dlopen("bayeux.bundle", RTLD_LAZY|RTLD_GLOBAL );
void(*f)(void) = (void(*)(void)) dlsym( handle, "Init_bayeux" ) ;
f();
}
So it looks to me, that it might be a problem with how the ruby.exe was build. Does that make sense?
Update:
I had a look at the addresses containing the names of the two type_info objects. Same content, but different addresses. I added the -flat_namespace switch to the link command. Now the dynamic_cast works. The original Problem with the boost regex library still exists, but I think this might be solvable by linking boost statically into the shared library or by rebuilding the boost libraries with the -flat_namespace switch.
Update:
Now I'm back to the very first example with the boost regex expression, build with this command:
g++ -shared -flat_namespace -fPIC -o bayeux.bundle /Users/todi/boost_1_49_0/stage/lib/libboost_regex.a bayeux.cpp
But when loading the extension into the ruby interpreter, initializing of static symbols fails:
ruby(59384,0x7fff712b8cc0) malloc: *** error for object 0x7fff70b19500: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Program received signal SIGABRT, Aborted.
0x00007fff8a6ab0b6 in __kill ()
(gdb) bt
#0 0x00007fff8a6ab0b6 in __kill ()
#1 0x00007fff8a74b9f6 in abort ()
#2 0x00007fff8a663195 in free ()
#3 0x0000000100541023 in boost::re_detail::cpp_regex_traits_char_layer<char>::init (this=0x10060be50) at basic_string.h:237
#4 0x0000000100543904 in boost::object_cache<boost::re_detail::cpp_regex_traits_base<char>, boost::re_detail::cpp_regex_traits_implementation<char> >::do_get (k=#0x7fff5fbfddd0) at cpp_regex_traits.hpp:366
#5 0x000000010056005b in create_cpp_regex_traits<char> (l=<value temporarily unavailable, due to optimizations>) at pending/object_cache.hpp:69
#6 0x0000000100544c33 in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::do_assign (this=0x7fff5fbfe090, p1=0x100567158 "^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?", p2=0x100567191 "", f=0) at cpp_regex_traits.hpp:880
#7 0x0000000100566280 in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign ()
#8 0x000000010056622d in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign ()
#9 0x0000000100566188 in boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::basic_regex ()
#10 0x0000000100566025 in Init_bayeux ()
#11 0x0000000100003a23 in dln_load (file=0x10201a000 "/Users/todi/sioux/source/rack/bayeux.bundle") at dln.c:1293
#12 0x000000010016569d in vm_pop_frame [inlined] () at /Users/todi/.rvm/src/ruby-1.9.2-p320/vm_insnhelper.c:1465
#13 0x000000010016569d in rb_vm_call_cfunc (recv=4303980440, func=0x100042520 <load_ext>, arg=4303803000, blockptr=0x1, filename=<value temporarily unavailable, due to optimizations>, filepath=<value temporarily unavailable, due to optimizations>) at vm.c:1467
#14 0x0000000100043382 in rb_require_safe (fname=4303904640, safe=0) at load.c:602
#15 0x000000010017cbf3 in vm_call_cfunc [inlined] () at /Users/todi/.rvm/src/ruby-1.9.2-p320/vm_insnhelper.c:402
#16 0x000000010017cbf3 in vm_call_method (th=0x1003016b0, cfp=0x1004ffef8, num=1, blockptr=0x1, flag=8, id=<value temporarily unavailable, due to optimizations>, me=0x10182cfa0, recv=4303980440) at vm_insnhelper.c:528
...
Again, this doesn't fail, when I load the shared library by the little c program from above.
Update:
Now I link the first example static:
g++ -shared -fPIC -flat_namespace -nodefaultlibs -o bayeux.bundle -static -lstdc++ -lpthread -lgcc_eh -lboost_regex-mt bayeux.cpp
With the same error:
ruby(15197,0x7fff708aecc0) malloc: *** error for object 0x7fff7027e500: pointer being freed was not allocated
otool -L confirmed that every library is linked static:
bayeux.bundle:
bayeux.bundle (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
debug:
If I link against the boost debug version, then it works like expected.

For the records: I've now build boost and my application with the very same compiler (version 4.2.1 [official apple version]). No problems so far. Why it will not work as expected when the ruby extension links all libraries statically is a miracle to me. Thank to all who put time into this issue.
Kind regards
Torsten

Related

Why does clang MemorySanitizer produce error in fstream open?

I am trying to integrate Memory Sanitizer into my project. And following code raise an use-of-uninitialized-value error:
#include <fstream>
int main () {
std::ifstream ifs;
ifs.open ("test.txt", std::ifstream::in);
char c = ifs.get(); // this line doesn't matter
if (!ifs.good()) {
return -1;
}
ifs.close();
return 0;
}
$ clang -fsanitize=memory -fPIE -pie -fno-omit-frame-pointer -g -O3 main.cpp -lstdc++
$ ./a.out
==43028==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x55986e5d76ae in std::basic_ifstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/fstream
#1 0x55986e5d76ae in main /home/promanjuk/learn/sanityzers/main.cpp:5:7
#2 0x7f6389ffdd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#3 0x7f6389ffde3f in __libc_start_main csu/../csu/libc-start.c:392:3
#4 0x55986e551334 in _start (/home/promanjuk/learn/sanityzers/a.out+0x21334) (BuildId: 597886c0634c006622192d0fa4c5ab7f169b7625)
SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/fstream in std::basic_ifstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode)
Exiting
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
GodBolt behavior is the same (from clang 5.0 to 15.0, older versions produce a linker error).
Any ideas?
Thanks.
Code example I take from https://cplusplus.com/reference/fstream/ifstream/open/
So, this I've tried to run with/without
char c = ifs.get();
Tried different version of the clang.

Segfault in TSAN in Boost CI - Need the help of an expert

I am debugging A segfault reported by TSAN in the CI of Boost.Beast.
I strongly believe it to be a false positive, but I don't know what to look for in order to suppress it.
It seems to me from the stack trace that the code is correctly instrumented.
The code passes all other tests, inclding valgrind, ubsan, etc.
I'm hoping some kind expert may put me out of my misery.
Here is the output:
====== BEGIN OUTPUT ======
beast.http.read
ThreadSanitizer:DEADLYSIGNAL
==132842==ERROR: ThreadSanitizer: SEGV on unknown address 0x7ff5d9cff000 (pc 0x7ff5dceba0d0 bp 0x000000000000 sp 0x7ff5d9c3d910 T132844)
==132842==The signal is caused by a READ memory access.
#0 __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1, 20>::Put(__sanitizer::StackTrace, bool*) <null> (libtsan.so.2+0xba0d0)
#1 __tsan::CurrentStackId(__tsan::ThreadState*, unsigned long) <null> (libtsan.so.2+0x8c48f)
#2 __sanitizer::DD::MutexInit(__sanitizer::DDCallback*, __sanitizer::DDMutex*) <null> (libtsan.so.2+0xac534)
#3 __tsan::DDMutexInit(__tsan::ThreadState*, unsigned long, __tsan::SyncVar*) <null> (libtsan.so.2+0x9a3f8)
#4 __tsan::MetaMap::GetSync(__tsan::ThreadState*, unsigned long, unsigned long, bool, bool) <null> (libtsan.so.2+0xa85dc)
#5 __tsan_atomic32_fetch_add <null> (libtsan.so.2+0x783e9)
#6 __gnu_cxx::__exchange_and_add(int volatile*, int) /usr/include/c++/12/ext/atomicity.h:66 (read+0x4188f8)
#7 __gnu_cxx::__exchange_and_add_dispatch(int*, int) /usr/include/c++/12/ext/atomicity.h:101 (read+0x4188f8)
#8 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release_last_use() /usr/include/c++/12/bits/shared_ptr_base.h:187 (read+0x4188f8)
#9 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/12/bits/shared_ptr_base.h:361 (read+0x40c592)
#10 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/12/bits/shared_ptr_base.h:1071 (read+0x418fee)
#11 std::__shared_ptr<boost::asio::detail::strand_executor_service::strand_impl, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/12/bits/shared_ptr_base.h:1524 (read+0x420f83)
#12 std::shared_ptr<boost::asio::detail::strand_executor_service::strand_impl>::~shared_ptr() /usr/include/c++/12/bits/shared_ptr.h:175 (read+0x420faf)
#13 boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>::~invoker() <null> (read+0x471f5f)
#14 boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) <null> (read+0x48b0ac)
#15 boost::asio::detail::scheduler_operation::complete(void*, boost::system::error_code const&, unsigned long) boost/asio/detail/scheduler_operation.hpp:40 (read+0x4fa38e)
#16 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) boost/asio/detail/impl/scheduler.ipp:492 (read+0x4e8835)
#17 boost::asio::detail::scheduler::run(boost::system::error_code&) boost/asio/detail/impl/scheduler.ipp:210 (read+0x4e74fb)
#18 boost::asio::io_context::run() boost/asio/impl/io_context.ipp:63 (read+0x4dc122)
#19 boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}::operator()() const <null> (read+0x412363)
#20 void std::__invoke_impl<void, boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>(std::__invoke_other, boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}&&) <null> (read+0x4a1f92)
#21 std::__invoke_result<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>::type std::__invoke<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>(boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}&&) <null> (read+0x49f9dc)
#22 void std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (read+0x49c90a)
#23 std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> >::operator()() <null> (read+0x49941e)
#24 std::thread::_State_impl<std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> > >::_M_run() <null> (read+0x494c2a)
#25 execute_native_thread_routine <null> (libstdc++.so.6+0xdbb72)
#26 __tsan_thread_start_func <null> (libtsan.so.2+0x393ef)
#27 start_thread <null> (libc.so.6+0x8ce2c)
#28 clone3 <null> (libc.so.6+0x1121af)
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV (/lib64/libtsan.so.2+0xba0d0) in __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1, 20>::Put(__sanitizer::StackTrace, bool*)
==132842==ABORTING
The code being tested is latest master branch.
Command line to reproduce:
$ ./b2 toolset=gcc thread-sanitizer=norecover link=static variant=debug libs/beast/test -q -d+2 -j1
My compiler info:
$ gcc --version
gcc (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1)
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
My OS is Fedora 36. But we see this happen on Ubuntu as well.
Using the b2 line I could repro the SEGV on linux using master (00293a6adb5 from the superproject).
I started with a - to me - more convenient setup based on CMake. I modified the CMake to use thread,undefined sanitizers instead of address,undefined for VARIANT=ubasan.
Interestingly, it doesn't segfault. It does however seem to have a legit TSAN violation in basic_stream.cpp, where the effective flags are:
{
"directory": "/backup/cloudbackup/custom_ex/superboost/libs/beast/build/test/beast/core",
"command": "/home/sehe/bin/g++-10 -DBOOST_ALL_STATIC_LINK=1 -DBOOST_ASIO_DISABLE_BOOST_ARRAY=1 -DBOOST_ASIO_DISABLE_BOOST_BIND=1 -DBOOST_ASIO_DISABLE_BOOST_DATE_TIME=1 -DBOOST_ASIO_DISABLE_BOOST_REGEX=1 -DBOOST_ASIO_NO_DEPRECATED=1 -DBOOST_ASIO_SEPARATE_COMPILATION=1 -DBOOST_BEAST_ALLOW_DEPRECATED -DBOOST_BEAST_SEPARATE_COMPILATION=1 -DBOOST_BEAST_TESTS -DBOOST_COROUTINES_NO_DEPRECATION_WARNING=1 -I/backup/cloudbackup/custom_ex/superboost/libs/beast/include -I/backup/cloudbackup/custom_ex/superboost/libs/beast/. -I/backup/cloudbackup/custom_ex/superboost/libs/beast/test/./extern -I/backup/cloudbackup/custom_ex/superboost/libs/beast/test/./extras/include -isystem /backup/cloudbackup/custom_ex/superboost -std=c++11 -Wall -Wextra -Wpedantic -Wno-unused-parameter -DBOOST_BEAST_NO_SLOW_TESTS=1 -msse4.2 -funsigned-char -fno-omit-frame-pointer -fsanitize=thread,undefined -O2 -g -DNDEBUG -pthread -o CMakeFiles/tests-beast-core.dir/basic_stream.cpp.o -c /backup/cloudbackup/custom_ex/superboost/libs/beast/test/beast/core/basic_stream.cpp",
"file": "/backup/cloudbackup/custom_ex/superboost/libs/beast/test/beast/core/basic_stream.cpp"
},
Breaking it down for readability:
g++-10
-DBOOST_ALL_STATIC_LINK=1
-DBOOST_ASIO_DISABLE_BOOST_ARRAY=1
-DBOOST_ASIO_DISABLE_BOOST_BIND=1
-DBOOST_ASIO_DISABLE_BOOST_DATE_TIME=1
-DBOOST_ASIO_DISABLE_BOOST_REGEX=1
-DBOOST_ASIO_NO_DEPRECATED=1
-DBOOST_ASIO_SEPARATE_COMPILATION=1
-DBOOST_BEAST_ALLOW_DEPRECATED
-DBOOST_BEAST_SEPARATE_COMPILATION=1
-DBOOST_BEAST_TESTS
-DBOOST_COROUTINES_NO_DEPRECATION_WARNING=1
-I/superboost/libs/beast/include
-I/superboost/libs/beast/.
-I/superboost/libs/beast/test/./extern
-I/superboost/libs/beast/test/./extras/include
-isystem /superboost
-std=c++11
-Wall
-Wextra
-Wpedantic
-Wno-unused-parameter
-DBOOST_BEAST_NO_SLOW_TESTS=1
-msse4.2
-funsigned-char
-fno-omit-frame-pointer
-fsanitize=thread,undefined
-O2 -g
-DNDEBUG
-pthread
-o CMakeFiles/tests-beast-core.dir/basic_stream.cpp.o
-c /superboost/libs/beast/test/beast/core/basic_stream.cpp
The reported diagnostic: https://paste.ubuntu.com/p/6SKjmZ9wFT/ (lines truncated for SO):
beast.core.basic_stream
==================
WARNING: ThreadSanitizer: data race (pid=24051)
Write of size 8 at 0x7b0400000030 by thread T1:
#0 pipe ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1726 (libtsan.so.0+0x3e574)
#1 __sanitizer::IsAccessibleMemoryRange(unsigned long, unsigned long) ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:281 (libu...
#2 operator() /superboost/libs/beast/test/beast/core/basic_stream.cpp:228 (tests-beast-core+0x4f0271)
#3 operator() /superboost/boost/asio/detail/bind_handler.hpp:171 (tests-beast-core+0x4f0271)
#4 invoke<boost::asio::detail::binder1<boost::beast::(anonymous namespace)::test_server::test_server(boost::beast::string_view, boost::asio::ip::tcp::end...
#5 complete<boost::asio::detail::binder1<boost::beast::(anonymous namespace)::test_server::test_server(boost::beast::string_view, boost::asio::ip::tcp::e...
#6 do_complete /superboost/boost/asio/detail/reactive_socket_accept_op.hpp:150 (tests-beast-core+0x4f0271)
#7 boost::asio::detail::scheduler_operation::complete(void*, boost::system::error_code const&, unsigned long) /superboost/boost/asio/detail/scheduler_ope...
#8 boost::asio::detail::epoll_reactor::descriptor_state::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, ...
#9 boost::asio::detail::scheduler_operation::complete(void*, boost::system::error_code const&, unsigned long) /superboost/boost/asio/detail/scheduler_ope...
#10 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info...
#11 boost::asio::detail::scheduler::run(boost::system::error_code&) /superboost/boost/asio/detail/impl/scheduler.ipp:210 (tests-beast-core+0x90d424)
#12 boost::asio::io_context::run() /superboost/boost/asio/impl/io_context.ipp:63 (tests-beast-core+0x91bf39)
#13 operator() /superboost/libs/beast/test/beast/core/basic_stream.cpp:234 (tests-beast-core+0x4cf1f9)
#14 __invoke_impl<void, boost::beast::(anonymous namespace)::test_server::test_server(boost::beast::string_view, boost::asio::ip::tcp::endpoint, std::ost...
#15 __invoke<boost::beast::(anonymous namespace)::test_server::test_server(boost::beast::string_view, boost::asio::ip::tcp::endpoint, std::ostream&)::<la...
#16 _M_invoke<0> /usr/include/c++/10/thread:264 (tests-beast-core+0x4cf1f9)
#17 operator() /usr/include/c++/10/thread:271 (tests-beast-core+0x4cf1f9)
#18 _M_run /usr/include/c++/10/thread:215 (tests-beast-core+0x4cf1f9)
#19 execute_native_thread_routine ../../../../../src/libstdc++-v3/src/c++11/thread.cc:82 (libstdc++.so.6+0xd44bf)
Previous write of size 8 at 0x7b0400000030 by main thread:
#0 pipe ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1726 (libtsan.so.0+0x3e574)
#1 __sanitizer::IsAccessibleMemoryRange(unsigned long, unsigned long) ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:281 (libu...
#2 void boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> >::initiate_async_...
#3 void boost::asio::detail::completion_handler_async_result<boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_typ...
#4 boost::asio::constraint<boost::asio::detail::async_result_has_initiate_memfn<boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context:...
#5 decltype ((async_initiate<boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul>, bo...
#6 boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul>, boost::beast::unlimited_rate...
#7 boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul>, boost::beast::unlimited_rate...
#8 boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul>, boost::beast::unlimited_rate...
#9 void boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul>, boost::beast::unlimited...
#10 void boost::asio::detail::completion_handler_async_result<boost::beast::basic_stream_test::handler, void (boost::system::error_code, unsigned long)>:...
#11 boost::asio::constraint<boost::asio::detail::async_result_has_initiate_memfn<boost::beast::basic_stream_test::handler, void (boost::system::error_cod...
#12 boost::asio::async_result<std::decay<boost::beast::basic_stream_test::handler>::type, void (boost::system::error_code, unsigned long)>::return_type b...
#13 boost::beast::basic_stream_test::testRead() /superboost/libs/beast/test/beast/core/basic_stream.cpp:488 (tests-beast-core+0x678b01)
#14 boost::beast::basic_stream_test::run() /superboost/libs/beast/test/beast/core/basic_stream.cpp:1401 (tests-beast-core+0x70b193)
#15 void boost::beast::unit_test::suite::run<void>(boost::beast::unit_test::runner&) /superboost/libs/beast/include/boost/beast/_experimental/unit_test/s...
#16 void boost::beast::unit_test::suite::operator()<void>(boost::beast::unit_test::runner&) /superboost/libs/beast/include/boost/beast/_experimental/unit...
#17 boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha...
#18 void std::__invoke_impl<void, boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx11::basic_string<char, std::char_tr...
#19 std::enable_if<std::__and_<std::is_void<void>, std::__is_invocable<boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__c...
#20 std::_Function_handler<void (boost::beast::unit_test::runner&), boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx1...
#21 std::function<void (boost::beast::unit_test::runner&)>::operator()(boost::beast::unit_test::runner&) const /usr/include/c++/10/bits/std_function.h:62...
#22 boost::beast::unit_test::suite_info::run(boost::beast::unit_test::runner&) const /superboost/libs/beast/include/boost/beast/_experimental/unit_test/s...
#23 bool boost::beast::unit_test::runner::run<void>(boost::beast::unit_test::suite_info const&) /superboost/libs/beast/include/boost/beast/_experimental/...
#24 bool boost::beast::unit_test::runner::run_each<boost::beast::unit_test::suite_list>(boost::beast::unit_test::suite_list const&) /superboost/libs/beas...
#25 main /superboost/libs/beast/include/boost/beast/_experimental/unit_test/main.ipp:82 (tests-beast-core+0x4cc1d3)
Thread T1 (tid=24054, running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5fe84)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /build/gcc-11-YRKbe7/gcc-11-...
#2 boost::beast::basic_stream_test::testRead() /superboost/libs/beast/test/beast/core/basic_stream.cpp:484 (tests-beast-core+0x67891b)
#3 boost::beast::basic_stream_test::run() /superboost/libs/beast/test/beast/core/basic_stream.cpp:1401 (tests-beast-core+0x70b193)
#4 void boost::beast::unit_test::suite::run<void>(boost::beast::unit_test::runner&) /superboost/libs/beast/include/boost/beast/_experimental/unit_test/su...
#5 void boost::beast::unit_test::suite::operator()<void>(boost::beast::unit_test::runner&) /superboost/libs/beast/include/boost/beast/_experimental/unit_...
#6 boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char...
#7 void std::__invoke_impl<void, boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx11::basic_string<char, std::char_tra...
#8 std::enable_if<std::__and_<std::is_void<void>, std::__is_invocable<boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cx...
#9 std::_Function_handler<void (boost::beast::unit_test::runner&), boost::beast::unit_test::make_suite_info<boost::beast::basic_stream_test>(std::__cxx11...
#10 std::function<void (boost::beast::unit_test::runner&)>::operator()(boost::beast::unit_test::runner&) const /usr/include/c++/10/bits/std_function.h:62...
#11 boost::beast::unit_test::suite_info::run(boost::beast::unit_test::runner&) const /superboost/libs/beast/include/boost/beast/_experimental/unit_test/s...
#12 bool boost::beast::unit_test::runner::run<void>(boost::beast::unit_test::suite_info const&) /superboost/libs/beast/include/boost/beast/_experimental/...
#13 bool boost::beast::unit_test::runner::run_each<boost::beast::unit_test::suite_list>(boost::beast::unit_test::suite_list const&) /superboost/libs/beas...
#14 main /superboost/libs/beast/include/boost/beast/_experimental/unit_test/main.ipp:82 (tests-beast-core+0x4cc1d3)
SUMMARY: ThreadSanitizer: data race ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:281 in __sanitizer::IsAccessibleMemoryRange(uns...
==================
559ms, 1 suite, 1 case, 217 tests total, 0 failures
ThreadSanitizer: reported 1 warnings
Given this observation, I thought to see whether excluding the offending TU (basic_stream.cpp) from the b2 build removes the SEGV. No such luck.
On the contrary, the SEGV manfifest with the following TUs:
module
TU
test/beast/core
buffered_read_stream.cpp
test/beast/http
read.cpp
test/beast/http
write.cpp
test/beast/websocket
close.cpp
test/beast/websocket
handshake.cpp
test/beast/websocket
ping.cpp
test/beast/websocket
read2.cpp
test/beast/websocket
write.cpp
test/doc
http_examples.cpp
Dropping these TUs from their respective test/**/Jamfile allows all remaining tests to pass TSAN under b2. Now, I did some soul searching and e.g. unique include diving using a script like:
#!/bin/bash -i
alias PP='find build/ -name *.cpp.i'
PP -delete
make -C build/test/beast/core buffered_read_stream.i
make -C build/test/beast/http read.i
make -C build/test/beast/http write.i
make -C build/test/beast/websocket close.i
make -C build/test/beast/websocket handshake.i
make -C build/test/beast/websocket ping.i
make -C build/test/beast/websocket read2.i
make -C build/test/beast/websocket write.i
make -C build/test/doc http_examples.i
PP | nl
n=$(PP | wc -l)
set -x
export LANG=C
PP -exec sort -b {} \+ | uniq -dc | grep -wP "^\s*$n # 1" | grep -P '1( 3 4)?$' | nl
Which uncovers a common subset of includes of 854 includes. 40 of the beast headers are in that consistent set, but 208 are asio headers.
Questions from here:
why is SEGV not happening in the CMake build?
are there headers in the common subset that do not appear in the TUs that don't trip TSAN up?
is a recent change in Asio relevant?
Choosing to address these 3., 1., 2. (optimizing for return-on-effort)
3. Is a recent Asio change involved? [YES]
Doing the b2 test with only Asio reverted to 1.79.0 (e929e5cf Merge asio from 'develop') passes all the tests cleanly.
Just to check that no compiler flags were harmed in the process e.g. buffered_read_stream.cpp showed the same exact commands:
gcc.compile.c++ bin.v2/libs/beast/test/beast/core/buffered_read_stream.test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/buffered_read_stream.o
"g++-10" -fvisibility-inlines-hidden -fsanitize=thread -fno-sanitize-recover=thread -fno-omit-frame-pointer -m64 -pthread -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_DISABLE_BOOST_ARRAY=1 -DBOOST_ASIO_DISABLE_BOOST_BIND=1 -DBOOST_ASIO_DISABLE_BOOST_DATE_TIME=1 -DBOOST_ASIO_DISABLE_BOOST_REGEX=1 -DBOOST_ASIO_NO_DEPRECATED=1 -DBOOST_ASIO_SEPARATE_COMPILATION -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_BEAST_ALLOW_DEPRECATED -DBOOST_BEAST_SEPARATE_COMPILATION -DBOOST_BEAST_TESTS -DBOOST_COROUTINES_NO_DEPRECATION_WARNING=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -D_GNU_SOURCE=1 -D_XOPEN_SOURCE=600 -I"." -I"libs/beast" -I"libs/beast/test/extern" -I"libs/beast/test/extras/include" -c -o "bin.v2/libs/beast/test/beast/core/buffered_read_stream.test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/buffered_read_stream.o" "libs/beast/test/beast/core/buffered_read_stream.cpp"
gcc.link bin.v2/libs/beast/test/beast/core/buffered_read_stream.test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/buffered_read_stream
"g++-10" -o "bin.v2/libs/beast/test/beast/core/buffered_read_stream.test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/buffered_read_stream" -Wl,--start-group "bin.v2/libs/beast/test/beast/core/buffered_read_stream.test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/buffered_read_stream.o" "bin.v2/libs/beast/test/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/lib-test.a" "bin.v2/libs/beast/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/lib-asio.a" "bin.v2/libs/beast/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/lib-beast.a" "bin.v2/libs/coroutine/build/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/libboost_coroutine.a" "bin.v2/libs/context/build/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/libboost_context.a" "bin.v2/libs/filesystem/build/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/libboost_filesystem.a" "bin.v2/libs/atomic/build/gcc-10.0/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/libboost_atomic.a" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -m64 -pthread -g -fvisibility=hidden -fvisibility-inlines-hidden -fsanitize=thread -fno-sanitize-recover=thread -fno-omit-frame-pointer
So, now we know that something inside the 208 Asio headers must be involved.
1. Why is SEGV not happening in the CMake build?
Surely, here we should be able to spot difference in compilation flags? Ever-so-slightly redacted to remove spelling differences (left = CMake, right = b2):
I used the process of elimination, figured out that the culprit is -fno-inline -O0. Somehow it leads to recursive TSAN errors:
beast.core.buffered_read_stream
ThreadSanitizer:DEADLYSIGNAL
==2827==ERROR: ThreadSanitizer: SEGV on unknown address 0x60000212fff8 (pc 0x7f319a938bfc bp 0x7f3196fbe1c0 sp 0x7f3196fbe1a8 T2829)
==2827==The signal is caused by a WRITE memory access.
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.
Observations without -fno-inline:
from -O2 the symptoms go away
at -O1 the symptoms go away if NDEBUG is defined
at -O0 the symptom is there regardless
This suggests that a NDEBUG-sensitive piece of code could be involved. This might be a lead to guide minimization.
Comparing preprocessed sources may highlight specific possible causes. My main suspicions are the revamped spawned_thread_base in asio/spawn.hpp, source-locations and or cancellation slots.
As a courtesy, here's the preprocessed-buffered-stream-reader.zip containing 4 files (/home/sehe/{with,without}-NDEBUG.i.asio1.{79,80}.0).
Thanks everyone who chipped in on this.
I was was able to get some help from the legendary Chris Kohlhoff, author of the Asio library.
Quoting:
It seems that thread-sanitizer does not correctly handle coroutine/fiber stacks that migrate between threads. This would either be considered a bug in thread-sanitizer, or perhaps a feature request for it.
The issue is in the thread sanitiser library itself.
The workaround in this case is to restructure the code so that the initiation of the fiber takes place on the same thread as the one in which it makes progress.
Here's the old (correct but not TSAN-compatible) code:
template<class F0, class... FN>
inline
void
enable_yield_to::
spawn(F0&& f, FN&&... fn)
{
asio::spawn(ioc_,
[&](yield_context yield)
{
f(yield);
std::lock_guard<std::mutex> lock{m_};
if(--running_ == 0)
cv_.notify_all();
}
, boost::coroutines::attributes(2 * 1024 * 1024));
spawn(fn...);
}
And here's the code with the workaround:
template<class F0, class... FN>
inline
void
enable_yield_to::
spawn(F0&& f, FN&&... fn)
{
// dispatch of spawn is a workaround for
// https://github.com/boostorg/beast/issues/2499
asio::dispatch(ioc_,
[&]
{
asio::spawn(ioc_,
[&](yield_context yield)
{
f(yield);
std::lock_guard<std::mutex> lock{m_};
if(--running_ == 0)
cv_.notify_all();
}
, boost::coroutines::attributes(2 * 1024 * 1024));
});
spawn(fn...);
}

Why the packaged task is terminating after throwing system error? [duplicate]

I got a strange core dump which I copied from a part of code in http://en.cppreference.com/w/cpp/thread/packaged_task,
#include <future>
#include <iostream>
#include <cmath>
void task_lambda() {
std::packaged_task<int(int,int)> task([](int a, int b) {
return std::pow(a, b);
});
std::future<int> result = task.get_future();
task(2, 9);
std::cout << "task_lambda:\t" << result.get() << '\n';
}
int main() {
task_lambda();
}
I got this
terminate called after throwing an instance of 'std::system_error'
what(): Unknown error -1
[1] 28373 abort (core dumped) ./a.out
The call stack is like below:
#0 0x00007ffff71a2428 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff71a402a in __GI_abort () at abort.c:89
#2 0x00007ffff7ae484d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7ae26b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7ae2701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7ae2919 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff7b0b7fe in std::__throw_system_error(int) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x0000000000404961 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__once=..., __f=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x1246d>)
at /usr/include/c++/5/mutex:746
#8 0x0000000000403eb2 in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (
this=0x61ec30, __res=..., __ignore_failure=false) at /usr/include/c++/5/future:387
#9 0x0000000000402b76 in std::__future_base::_Task_state<task_lambda()::<lambda(int, int)>, std::allocator<int>, int(int, int)>::_M_run(<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17680>, <unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17685>) (this=0x61ec30, __args#0=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17680>,
__args#1=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17685>) at /usr/include/c++/5/future:1403
#10 0x00000000004051c1 in std::packaged_task<int (int, int)>::operator()(int, int) (this=0x7fffffffdca0, __args#0=2, __args#1=9) at /usr/include/c++/5/future:1547
#11 0x0000000000401c7d in task_lambda () at aa.cc:12
#12 0x0000000000401d1b in main () at aa.cc:19
Then I added some sample code into my program, it became
#include <iostream>
#include <cmath>
#include <future>
#include <thread>
int f(int x, int y) { return std::pow(x,y); }
void task_thread() {
std::packaged_task<int(int,int)> task(f);
std::future<int> result = task.get_future();
std::thread task_td(std::move(task), 2, 10);
task_td.join();
std::cout << "task_thread:\t" << result.get() << '\n';
}
void task_lambda() {
std::packaged_task<int(int,int)> task([](int a, int b) {
return std::pow(a, b);
});
std::future<int> result = task.get_future();
task(2, 9);
std::cout << "task_lambda:\t" << result.get() << '\n';
}
int main() {
task_lambda();
}
the error was gone. How can I correct the program by adding a function even though I never call it?
gcc version
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
Promgram compiled with command:
g++ -std=c++11 aa.cc -lpthread
With #fedepad's help, I got correct output by replace lpthread with pthread. But I still confused how second code work by add a dummy function!
I tried your first code snippet using the following version of g++:
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
and compiled with the following
g++ -o test_threads test_threads.cpp -std=c++11 -pthread
and I can run the program with no problems, and getting the following output:
$ ./test_threads
task_lambda: 512
If I then use the -lpthread as you did with the following
g++ -o test_threads test_threads.cpp -std=c++11 -lpthread
I get
$ ./test_threads
terminate called after throwing an instance of 'std::system_error'
what(): Unknown error -1
[1] 7890 abort ./test_threads
So please use -pthread as a flag and not -lpthread.
This behavior is also mentioned in the following
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59830
There's a difference between -pthread and -lpthread.
Looking at the man page for g++
-pthread
Adds support for multithreading with the pthreads library. This option sets flags for both the preprocessor and linker.
To have a look to what flags are activated for both one can check with the following:
g++ -dumpspecs | grep pthread
g++ -dumpspecs | grep lpthread
As one can clearly see, there are some preprocessor macros that are not activated if one is using -lpthread.

Boost property_tree error: conversion of data to type "j" failed when getting an element in a .ini file

I have a .ini file contains below data
[SYSTEM]
num_of_vps = 1
And I have this code to read an element in .ini file. (uint defined as typedef unsigned int uint)
boost::property_tree::ptree pt;
boost::property_tree::ini_parser::read_ini(iniFilePath, pt);
hwCount = pt.get<uint>("SYSTEM.num_of_vps");
I created a from files including above code and called it in a wrapper function in main.cc file. Then I am getting below error
terminate called after throwing an instance of 'boost_1_68_0::exception_detail::clone_impl<boost_1_68_0::exception_detail::error_info_injector<boost_1_68_0::property_tree::ptree_bad_data> >'
what(): conversion of data to type "j" failed
stack trace
#12 0x00002aaab613fcd5 in abort () from /lib64/libc.so.6
#13 0x00002aaab9b29315 in __gnu_cxx::__verbose_terminate_handler () at ../../../../src/gcc-7.3.0/libstdc++-v3/libsupc++/vterminate.cc:95
#14 0x00002aaab9a9e8f6 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../src/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
#15 0x00002aaab9a9e941 in std::terminate () at ../../../../src/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
#16 0x00002aaab9a9ea74 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x2aaab9e1ff60 <typeinfo for boost_1_68_0::exception_detail::clone_impl<boost_1_68_0::exception_detail::error_info_injector<boost_1_68_0::property_tree::ptree_bad_data> >>, dest=0x2aaab99bef18 <boost_1_68_0::exception_detail::clone_impl<boost_1_68_0::exception_detail::error_info_injector<boost_1_68_0::property_tree::ptree_bad_data> >::~clone_impl()>) at ../../../../src/gcc-7.3.0/libstdc++-v3/libsupc++/eh_throw.cc:93
#17 0x00002aaab99bec82 in boost_1_68_0::throw_exception<boost_1_68_0::exception_detail::error_info_injector<boost_1_68_0::property_tree::ptree_bad_data> > (e=...) at throw_exception.hpp:72
#18 0x00002aaab99be576 in boost_1_68_0::exception_detail::throw_exception_<boost_1_68_0::property_tree::ptree_bad_data> (x=..., current_function=0x2aaab9b45fc0 <boost_1_68_0::enable_if<boost_1_68_0::property_tree::detail::is_translator<boost_1_68_0::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, unsigned int> >, unsigned int>::type boost_1_68_0::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<unsigned int, boost_1_68_0::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, unsigned int> >(boost_1_68_0::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, unsigned int>) const::__PRETTY_FUNCTION__> "typename boost_1_68_0::enable_if<boost_1_68_0::property_tree::detail::is_translator<Translator>, Type>::type boost_1_68_0::property_tree::basic_ptree<Key, Data, KeyCompare>::get_value(T"..., file=0x2aaab9b45830 "property_tree/detail/ptree_implementation.hpp", line=675) at throw_exception.hpp:89
#19 0x00002aaab99be01e in boost_1_68_0::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<unsigned int, boost_1_68_0::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, unsigned int> > (this=0xbc86c8, tr=...) at property_tree/detail/ptree_implementation.hpp:673
#20 0x00002aaab99bd6a5 in boost_1_68_0::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<unsigned int> (this=0xbc86c8) at property_tree/detail/ptree_implementation.hpp:683
#21 0x00002aaab99bc545 in boost_1_68_0::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get<unsigned int> (this=0x7fffffff9470, path=...) at property_tree/detail/ptree_implementation.hpp:754
#22 0x00002aaab99bba83 in MyRT::DUTConfigFile::readIniFile (this=0xbc5d50, iniFilePath=...) at DUTConfigFile.cpp:231
#23 0x00002aaab99ba8d2 in MyRT::DUTConfigFile::DUTConfigFile (this=0xbc5d50, iniFilePath=..., configFilePath=...) at DUTConfigFile.cpp:26
#24 0x00002aaab99c0839 in setupMyConfigs () at SimXLInterface.cpp:83
#31 0x0000000000408847 in main ()
I tried gdb and it is throwing an exception when converting string to uint using istringstream. Below are the two types of the transition
template<class K, class D, class C>
template<class Type> inline
Type basic_ptree<K, D, C>::get_value() const
{
return get_value<Type>(
typename translator_between<data_type, Type>::type());
}
(gdb) p typeid(Type).name()
$2 = 0x2aaab5c33c91 <typeinfo name for unsigned int> "j"
(gdb) p typeid(data_type).name()
could not find typeinfo symbol for 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >'
Below function in basic_ios.h throws a __throw_bad_cast() exception (here this->_M_num_get = NULL)
const __num_get_type& __ng = __check_facet(this->_M_num_get);
Internal build automation Makefile is addding -D_GLIBCXX_USE_CXX11_ABI=0 to object file creation by default
g++ -I/usr/myboost/boost/boost -Wall -std=c++14 -fopenmp -m64 -msse2 -fPIC \
-D_GLIBCXX_USE_CXX11_ABI=0 -g -o *.o -fPIC -c *.cpp
.so building command
g++ -m64 -msse2 -pthread -shared -static-libstdc++ -static-libgcc \
-Wl,-znoexecstack -g -o runtime.so *.o -ldl -lrt -lz -fopenmp -lcrypto
If I change it to read a std::string type and later convert it to uint, it is working without an exception
std::string strHwCount = pt.get<std::string>("SYSTEM.num_of_vps");
hwCount = static_cast<uint>(std::stoul(strHwCount));
I am using boost-1.68 and gcc-7.3. Dose this have to do with the ABI macro that I'm using? Is there a way to resolve this without above workaround
UPDATE
First of all I am migrating project from cmake to a internal custom build automation (some enhanced version of Makefile). In cmake this worked fine. Based on sehe's answer I did some further analysis.
In a small standalone example this error is not reproducible (using both local string and actual .ini file read).
When going through gdb, In file boost/property_tree/stream_translator.hpp, I could find the read value from the file (changed to num_of_vps = 4 in file) in the both string and istringstream.
optional<Type> basic_ptree<K, D, C>
::get_value_optional(Translator tr) const
{
return tr.get_value(data());
}
(gdb) p data()
$1 = "4"
boost_1_68_0::optional<E> get_value(const internal_type &v) {
std::basic_istringstream<Ch, Traits, Alloc> iss(v);
iss.imbue(m_loc);
E e;
customized::extract(iss, e);
....
}
(gdb) p v
$1 = "4"
(gdb) p iss.str()
$2 = "4"
Inside above customized::extract(iss, e) function there is a string to unsigned int conversion using a basic_istream
static void extract(std::basic_istream<Ch, Traits>& s, E& e) {
s >> e;
if(!s.eof()) { s >> std::ws; }
}
Inside this >> operator there is a _M_extract function and inside that __check_facet function throws the exception (by cheking NULL of this->_M_num_get )
basic_istream<_CharT, _Traits>::
_M_extract(_ValueT& __v) {
__try {
const __num_get_type& __ng = __check_facet(this->_M_num_get);
...
inline const _Facet&
__check_facet(const _Facet* __f) {
if (!__f)
__throw_bad_cast();
return *__f;
}
stack trace above scenario
#0 std::istream::_M_extract<unsigned int>(unsigned int&) (__f=0x0) at gcc-7.3.0/objdir/x86_64-centos-linux/libstdc++-v3/include/bits/basic_ios.h:49
#1 std::istream::operator>>(unsigned int&) (this=0x7fffffff9980, __n=#0x7fffffff997c: 0)
#2 boost_1_68_0::property_tree::customize_stream::extract(std::istream&, unsigned int&) (s=..., e=#0x7fffffff997c: 0)
#3 boost_1_68_0::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, unsigned int>::get_value(std::string const&) (this=0x7fffffff9b58, v="4")
I could see the same behavior in the another place that we are using std::stringstream . Variable ss shows empty even with << operator adding strings
std::stringstream ss;
ss << std::setw(8) << std::setfill('0') << std::hex << firmId;
I had a concern about that the ABI flag causing this, but I was able to remove it and the issue is still there. I searched about this kind of istream issues but couldn't find anything useful.
Your input is corrupt. Likely it's using an encoding or code-points that you didn't expect, but don't stand out for a human reader.
Live On Coliru
#include <boost/property_tree/ini_parser.hpp>
#include <iostream>
int main() {
std::istringstream iss(R"([SYSTEM]
num_of_vps = 1)");
boost::property_tree::ptree pt;
read_ini(iss, pt);
uint hwCount = pt.get<uint>("SYSTEM.num_of_vps");
std::cout << "hwCount: " << hwCount << "\n";
}
Prints
hwCount: 1
However, if you change the input to be e.g.
std::istringstream iss(R"([SYSTEM]
num_of_vps = 1)");
(Note that the space after = is now < > 160, \240, U+00A0 NO-BREAK SPACE, ^KNS, ) it prints:
Live On Coliru
Prints
terminate called after throwing an instance of 'boost::wrapexcept<boost::property_tree::ptree_bad_data>'
what(): conversion of data to type "j" failed
With a tweak to the gcc command I was able to run the executable with the .so with all sstream usages. Previously the gcc linking command was
g++ -m64 -msse2 -pthread -shared -static-libstdc++ -static-libgcc \
-Wl,-znoexecstack -g -o runtime.so *.o -ldl -lrt -lz -fopenmp -lcrypto
Removing -static-libstdc++ and dynamically linking libstdc++.so worked
g++ -m64 -msse2 -pthread -shared -static-libgcc -Wl,-znoexecstack -o runtime.so *.o \
-fPIC -ldl -lrt -lz -fopenmp -lcrypto -Wl,-rpath,/depot/gcc-7.3.0/lib64

C++: Core dump with packaged_task

I got a strange core dump which I copied from a part of code in http://en.cppreference.com/w/cpp/thread/packaged_task,
#include <future>
#include <iostream>
#include <cmath>
void task_lambda() {
std::packaged_task<int(int,int)> task([](int a, int b) {
return std::pow(a, b);
});
std::future<int> result = task.get_future();
task(2, 9);
std::cout << "task_lambda:\t" << result.get() << '\n';
}
int main() {
task_lambda();
}
I got this
terminate called after throwing an instance of 'std::system_error'
what(): Unknown error -1
[1] 28373 abort (core dumped) ./a.out
The call stack is like below:
#0 0x00007ffff71a2428 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff71a402a in __GI_abort () at abort.c:89
#2 0x00007ffff7ae484d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7ae26b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7ae2701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7ae2919 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff7b0b7fe in std::__throw_system_error(int) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x0000000000404961 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__once=..., __f=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x1246d>)
at /usr/include/c++/5/mutex:746
#8 0x0000000000403eb2 in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (
this=0x61ec30, __res=..., __ignore_failure=false) at /usr/include/c++/5/future:387
#9 0x0000000000402b76 in std::__future_base::_Task_state<task_lambda()::<lambda(int, int)>, std::allocator<int>, int(int, int)>::_M_run(<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17680>, <unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17685>) (this=0x61ec30, __args#0=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17680>,
__args#1=<unknown type in /home/ace/test/a.out, CU 0x0, DIE 0x17685>) at /usr/include/c++/5/future:1403
#10 0x00000000004051c1 in std::packaged_task<int (int, int)>::operator()(int, int) (this=0x7fffffffdca0, __args#0=2, __args#1=9) at /usr/include/c++/5/future:1547
#11 0x0000000000401c7d in task_lambda () at aa.cc:12
#12 0x0000000000401d1b in main () at aa.cc:19
Then I added some sample code into my program, it became
#include <iostream>
#include <cmath>
#include <future>
#include <thread>
int f(int x, int y) { return std::pow(x,y); }
void task_thread() {
std::packaged_task<int(int,int)> task(f);
std::future<int> result = task.get_future();
std::thread task_td(std::move(task), 2, 10);
task_td.join();
std::cout << "task_thread:\t" << result.get() << '\n';
}
void task_lambda() {
std::packaged_task<int(int,int)> task([](int a, int b) {
return std::pow(a, b);
});
std::future<int> result = task.get_future();
task(2, 9);
std::cout << "task_lambda:\t" << result.get() << '\n';
}
int main() {
task_lambda();
}
the error was gone. How can I correct the program by adding a function even though I never call it?
gcc version
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
Promgram compiled with command:
g++ -std=c++11 aa.cc -lpthread
With #fedepad's help, I got correct output by replace lpthread with pthread. But I still confused how second code work by add a dummy function!
I tried your first code snippet using the following version of g++:
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
and compiled with the following
g++ -o test_threads test_threads.cpp -std=c++11 -pthread
and I can run the program with no problems, and getting the following output:
$ ./test_threads
task_lambda: 512
If I then use the -lpthread as you did with the following
g++ -o test_threads test_threads.cpp -std=c++11 -lpthread
I get
$ ./test_threads
terminate called after throwing an instance of 'std::system_error'
what(): Unknown error -1
[1] 7890 abort ./test_threads
So please use -pthread as a flag and not -lpthread.
This behavior is also mentioned in the following
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59830
There's a difference between -pthread and -lpthread.
Looking at the man page for g++
-pthread
Adds support for multithreading with the pthreads library. This option sets flags for both the preprocessor and linker.
To have a look to what flags are activated for both one can check with the following:
g++ -dumpspecs | grep pthread
g++ -dumpspecs | grep lpthread
As one can clearly see, there are some preprocessor macros that are not activated if one is using -lpthread.