How to dynamically build and link boost::regex with hidden inlines? - c++

As part of my (OSX) project, I am building boost::regex as a dynamic library. When I build and link without -fvisibility-inlines-hidden, everything works as expected.
To work around a bug in another external dependency, I need to turn this compiler switch on, however. Doing so results in lots of linker errors like this one
Undefined symbols for architecture x86_64:
"boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*,
std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<
char const*, std::string> > >, boost::regex_traits<char,
boost::cpp_regex_traits<char> > >::find()", referenced from:
bool boost::regex_search<__gnu_cxx::__normal_iterator<char const*,
std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<
char const*, std::string> > >, char, boost::regex_traits<char,
boost::cpp_regex_traits<char> > >(__gnu_cxx::__normal_iterator<char const*,
std::string>, __gnu_cxx::__normal_iterator<char const*, std::string>,
boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>,
std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*,
std::string> > > >&, boost::basic_regex<char, boost::regex_traits<char,
boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags,
__gnu_cxx::__normal_iterator<char const*, std::string>) in Regex.o
when I try to link boost to my project (the file "Regex.o" is part of the boost dylib). As I understand Apple's documentation, limiting inline visibility should just cause every dynamic lib to get its own instance of the inlined function, but should never cause linker errors.
It seems there is a related compiler bug for virtual methods, but neither perl_matcher::find() nor regex_search() are virtual methods.
Any suggestions/ideas?
$ clang --version
Apple clang version 3.1 (tags/Apple/clang-318.0.58) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.4.0
Thread model: posix

It turns out that this is a compiler bug in clang 3.1. Boost declares the template inline member function perl_matcher<...>::find() (and others) as external; only the instance linked into the boost dylib is not marked as external. Because it is inline, clang 3.1 hides this instance from the dylib's export table with -fvisibility-inlines-hidden.
It seems that this bug is fixed in the current trunk of clang (3.2.x). Until then, defining BOOST_REGEX_NO_EXTERNAL_TEMPLATES works around this bug (if it is acceptable to possibly have multiple instances of these inline functions in your binary). Otherwise, patching boost::regex to explicitly export the symbols defined in boost/regex/v4/instances.hpp using __attribute__ ((visibility("default"))) when BOOST_REGEX_INSTANTIATE is defined should also do the job (see the #define template hack at the top of instances.hpp).

Related

Undefined Boost reference

I'm trying to link against a 3rd party library that uses Boost. I've linked against the correct boost library (libboost_program_options.a), but still not finding it.
The error msg (formatted a bit for clarity):
undefined reference to `boost::program_options::validate(boost::any&,
std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*,
int)'
nm --demangle libboost_program_options.a | grep validate
boost::program_options::validate(boost::any&, std::vector<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::allocator<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > > const&,
bool*,
int)
boost::program_options::validate(boost::any&,
std::vector<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::allocator<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > > const&,
std::string*,
int)
boost::program_options::validate(boost::any&,
std::vector<std::string, std::allocator<std::string> > const&,
bool*,
int)
boost::program_options::validate(boost::any&,
std::vector<std::string, std::allocator<std::string> > const&,
std::string*,
int)
The second record looks similar, but apparently not close enough. Any idea how I can compile Boost to get a signature that matches what's in the library? I have a request out to the library owner to see what version of Boost they're using and such, but haven't heard back yet.
This is on a CentOS 7 box, which uses g++ version 4.8.5. But the library I'm tryng to link against uses C++11 heavily and was compiled with g++ v 6.1, so I installed devtoolset-6 which gives me a g++ 6 environment (g++ version 6.3.1)
I downloaded and built Boost from scratch (v1.65.1) so that it's built with the same compiler, rather then the system version.
Edit...
I think John Zwinck is on the right track, but I can't get the boost library to compile to the new ABI.
The validate() functions are found in value_semantic.cpp
Stripping the build down to the basics, and adding the flags discussed:
g++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI -c -o test.o libs/program_options/src/value_semantic.cpp
nm --demangle test.o | grep validate
00000000000008b6 T boost::program_options::validate(boost::any&, std::vector<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::allocator<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > > const&, bool*, int)
0000000000000c02 T boost::program_options::validate(boost::any&, std::vector<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::allocator<std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > > const&, std::string*, int)
00000000000005f2 T boost::program_options::validate(boost::any&, std::vector<std::string, std::allocator<std::string> > const&, bool*, int)
0000000000000b9a T boost::program_options::validate(boost::any&, std::vector<std::string, std::allocator<std::string> > const&, std::string*, int)
Does the _GLIBCXX_USE_CXX11_ABI macro only work with gcc 5.1?
It appears you have run afoul of the GCC Dual ABI for std::string in C++11: https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html
Your program is trying to link against an API using
std::__cxx11::basic_string<char>*
But your Boost library has
std::basic_string<char>*
This implies that your Boost library was built with GCC older than 5.1, or with the new ABI turned off. In either case, you can compile your own code with the new ABI turned off by adding this compiler flag:
-D_GLIBCXX_USE_CXX11_ABI=0
By compiling your program with that, it will be possible to use the system-provided (old ABI) Boost libraries. But then the question will be which C++ ABI your vendor compiled their code with (ask them, or look for __cxx11 in their library).
As explained at https://stackoverflow.com/a/52611576/981959 you can't use the new cxx11 ABI in the devtoolset version of GCC (because the whole point of the devtoolset GCC is to remain compatible with the system libstdc++'s ABI).
Your options are to either:
get a new version of the third-party library that uses the gcc4-compatible ABI, so you can link to it using the devtoolset compiler; or
build and install a newer GCC yourself, use that instead of the devtoolset GCC, and ensure that the new libstdc++.so gets used at runtime.
Ok, I think I've figured out why I can't get g++ to generate the cxx11 signature (new ABI).
If I run g++ with the -v option:
g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-6/root/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-6/root/usr --mandir=/opt/rh/devtoolset-6/root/usr/share/man --infodir=/opt/rh/devtoolset-6/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --enable-plugin --with-linker-hash-style=gnu --enable-initfini-array --disable-libgcj --with-default-libstdcxx-abi=gcc4-compatible --with-isl=/builddir/build/BUILD/gcc-6.3.1-20170216/obj-x86_64-redhat-linux/isl-install --enable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 6.3.1 20170216 (Red Hat 6.3.1-3) (GCC)
Note the following under the "configured with" field
--with-default-libstdcxx-abi=gcc4-compatible
What I believe this means is that the devtoolset-6 version of g++ is built with the old style ABI "baked in", and it therefore doesn't respond to the _GLIBCXX_USE_CXX11_ABI macro.
According to this link: https://gcc.gnu.org/onlinedocs/libstdc++/manual/configure.html
--with-default-libstdcxx-abi=OPTION
Set the default value for the _GLIBCXX_USE_CXX11_ABI macro (see Macros). The default is OPTION=new which sets the macro to 1, use OPTION=gcc4-compatible to set it to 0. This option does not change the library ABI.
It would seem to only change the default value of _GLIBCXX_USE_CXX11_ABI, but in my attempts setting _GLIBCXX_USE_CXX11_ABI doesn't have any effect.
Not positive of this, but it's my working theory for now. Any additional insight is appreciated.

undefined reference to boost::filesystem::path_traits::convert

I am trying to compile a program using cmake, and am seeing the following linker error:
/home/quant/bin/boost_1_61_0/stage/lib/libboost_log_setup.so:
undefined reference to
boost::filesystem::path_traits::convert(wchar_t const*, wchar_t
const*, std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >&, std::codecvt<wchar_t, char, __mbstate_t>
const&)' /home/quant/bin/boost_1_61_0/stage/lib/libboost_log.so:
undefined reference to
boost::filesystem::path_traits::dispatch(boost::filesystem::directory_entry
const&, std::__cxx11::basic_string,
std::allocator >&)'
The linker command that ninja generated looks like this:
g++ -pthread -DBOOST_ALL_DYN_LINK
utility/test/CMakeFiles/utilityTest.dir/loadCSVTests.cpp.o
utility/test/CMakeFiles/utilityTest.dir/main.cpp.o
utility/test/CMakeFiles/utilityTest.dir/randomDeviceTests.cpp.o -o
utility/test/utilityTest -rdynamic
/home/quant/bin/boost_1_61_0/stage/lib/libboost_thread.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_program_options.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_serialization.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_unit_test_framework.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_system.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_log.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_log_setup.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_filesystem.so
utility/lib/libutilityLib.a utility/testLib/libutilityTestLib.a
utility/lib/libutilityLib.a
/home/quant/bin/boost_1_61_0/stage/lib/libboost_thread.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_program_options.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_serialization.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_unit_test_framework.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_system.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_log.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_log_setup.so
/home/quant/bin/boost_1_61_0/stage/lib/libboost_filesystem.so
-Wl,-rpath,/home/quant/bin/boost_1_61_0/stage/lib
As you can see, I am linking against boost_filesystem and boost_system, so it's not the same problem as referenced on this SO post (and the many others like it).
I am using boost 1.61, which I compiled with gcc 5.3 (the same compiler as the one I'm compiling my program with).
What am I doing wrong?
I had a similar issue, this could be because of a new ABI which is introduced from gcc 5.1.
https://github.com/openframeworks/openFrameworks/issues/4203
I fixed mine by adding "add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)" to CMakeLists.txt

Compiling boost::program_options on Mac OS X with g++ 4.8 (Mac Ports)

I'm trying to compile one of the boost::program_options examples, http://svn.boost.org/svn/boost/trunk/libs/program_options/example/first.cpp, using gcc 4.8 (installed via MacPorts). However, I keep getting errors:
Undefined symbols for architecture x86_64:
"boost::program_options::to_internal(std::basic_string, std::allocator > const&)", referenced from:
std::vector, std::allocator >, std::allocator, std::allocator > > > boost::program_options::to_internal, std::allocator > >(std::vector, std::allocator >, std::allocator, std::allocator > > > const&) in ccEWnIGV.o
"boost::program_options::options_description::options_description(std::basic_string, std::allocator > const&, unsigned int, unsigned int)", referenced from:
_main in ccEWnIGV.o
"boost::program_options::invalid_option_value::invalid_option_value(std::basic_string, std::allocator > const&)", referenced from:
void boost::program_options::validate(boost::any&, std::vector, std::allocator >, std::allocator, std::allocator > > > const&, double*, long) in ccEWnIGV.o
"boost::program_options::error_with_option_name::error_with_option_name(std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, int)", referenced from:
boost::program_options::validation_error::validation_error(boost::program_options::validation_error::kind_t, std::basic_string, std::allocator > const&, std::basic_string, std::allocator > const&, int) in ccEWnIGV.o
"boost::program_options::detail::cmdline::set_additional_parser(boost::function1, std::allocator >, std::basic_string, std::allocator > >, std::basic_string, std::allocator > const&>)", referenced from:
boost::program_options::basic_command_line_parser::extra_parser(boost::function1, std::allocator >, std::basic_string, std::allocator > >, std::basic_string, std::allocator > const&>) in ccEWnIGV.o
"boost::program_options::detail::cmdline::cmdline(std::vector, std::allocator >, std::allocator, std::allocator > > > const&)", referenced from:
boost::program_options::basic_command_line_parser::basic_command_line_parser(int, char const* const*) in ccEWnIGV.o
"boost::program_options::operator >&, boost::program_options::options_description const&)", referenced from:
_main in ccEWnIGV.o
"boost::program_options::abstract_variables_map::operator[](std::basic_string, std::allocator > const&) const", referenced from:
boost::program_options::variables_map::operator[](std::basic_string, std::allocator > const&) const in ccEWnIGV.o
"boost::program_options::error_with_option_name::substitute_placeholders(std::basic_string, std::allocator > const&) const", referenced from:
vtable for boost::program_options::invalid_option_value in ccEWnIGV.o
vtable for boost::program_options::validation_error in ccEWnIGV.o
etc...
The boost libraries were installed via MacPorts, placing the header files in /opt/local/include and library files in /opt/local/lib. The compile command used was:
$ g++ -I/opt/local/include first.cpp -L/opt/local/lib -lboost_program_options-mt
Compiling header-only boost functions work fine using g++ 4.8 i.e factorial, hermite etc.
What I find strange is that this example compiles if I use the llvm-g++ compiler:
$ llvm-g++ -I/opt/local/include first.cpp -L/opt/local/lib -lboost_program_options-mt
$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin12/4.8.1/lto-wrapper
Target: x86_64-apple-darwin12
Thread model: posix
gcc version 4.8.1 (MacPorts gcc48 4.8.1_3)
\
$ llvm-g++ -v
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
\
Why doesn't this example compile with gcc 4.8? Any ideas? Is there a way to fix this?
Update: The example also compiles using clang++
I suspect you're using Boost compiled with Clang, which is incompatible with GCC.
This is because C++ implementation differs between them, and Clang's STL classes are put in a namespace different from GCC's using inline namespace for the sake differentiating Clang from more popular GCC; e.g. std::basic_string is std::__1::basic_string in Clang by inline namespace).
Try a command below, If you see "libc++" in output, yes, your Boost was built with Clang (or "libstdc++" for GCC):
$ otool -L /opt/local/lib/libboost_program_options-mt.dylib
/opt/local/lib/libboost_program_options-mt.dylib:
/opt/local/lib/libboost_program_options-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
If Boost was built with Clang, I recommend you to rebuild Boost with GCC-4.8 provided by MacPorts:
sudo port upgrade --force boost configure.compiler=macports-gcc-4.8
You can find other available compilers in https://trac.macports.org/wiki/UsingTheRightCompiler#configure-compiler .
Other useful link:
http://wiki.inkscape.org/wiki/index.php/CompilingMacOsX#Compiling_with_clang_3.3_and_libc.2B.2B_.28c.2B.2B11_enabled.29_.282013.29
As Shigerello already mentioned, the problem lies in boost compiled with clang and boost compiled with gcc being incompatible.
A simpler variation to get boost compiled with GCC-4.8 is to just use the gcc48 variant of boost:
sudo port install boost +gcc48

clang mac link errors with gtest Undefined symbols for architecture x86_64

Not sure what issue I am having here.
Mac clang 3.1 cmake
gtest
few files
using few C++11 features
I have downloaded and installed XCode build tools
CMAKE_CXX_FLAGS = -Wall -std=c++0x -stdlib=libc++ -v
builds fine with output...
[100%] Building CXX object CMakeFiles/soupbintcptest.dir/soupmessages_tests.cpp.o
clang version 3.1 (tags/RELEASE_31/final)
Target: x86_64-apple-darwin11.3.0
Thread model: posix
Linking dumps a slew of errors. ABI errors? I know there is this C++11 namespace mangling thing happening.
Undefined symbols for architecture x86_64:
They are all things in the std:: namespace that will not link like:
"std::basic_ostream<char, std::char_traits<char> >& std::operator<< <char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, std::_Setprecision)", referenced from:
testing::AssertionResult testing::internal::FloatingPointLE<float>(char const*, char const*, float, float) in libgtest.a(gtest-all.cc.o)
testing::AssertionResult testing::internal::FloatingPointLE<double>(char const*, char const*, double, double) in libgtest.a(gtest-all.cc.o)
testing::Message::Message() in libgtest.a(gtest-all.cc.o)
or
"std::cerr", referenced from:
testing::internal::GTestLog::GetStream() in libgtest.a(gtest-all.cc.o)
"std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()", referenced from:
testing::UnitTest::AddTestPartResult(testing::TestPartResult::Type, char const*, int, testing::internal::String const&, testing::internal::String const&) in libgtest.a(gtest-all.cc.o)
testing::internal::SingleFailureChecker::~SingleFailureChecker() in libgtest.a(gtest-all.cc.o)
testing::internal::StringStreamToString(std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*) in libgtest.a(gtest-all.cc.o)
testing::TestInfo::TestInfo(char const*, char const*, char const*, char const*, void const*, testing::internal::TestFactoryBase*) in libgtest.a(gtest-all.cc.o)
testing::TestInfo::~TestInfo() in libgtest.a(gtest-all.cc.o)
testing::internal::ReportInvalidTestCaseType(char const*, char const*, int) in libgtest.a(gtest-all.cc.o)
testing::internal::XmlUnitTestResultPrinter::PrintXmlUnitTest(__sFILE*, testing::UnitTest const&) in libgtest.a(gtest-all.cc.o)
...
I believe I have built gtest with the same clang++ version. Not sure what else is going on here to drive these issues.
I've seen similar errors to yours when the linker is pulling in a different c++ library to the one that matches what the compiler is building for. In your case, you are only modifying the compiler flags, but the linker doesn't know your choice and is likely linking to the wrong C++ library.
If you are able to use CMake 3.2 or later, then rather than modifying the C++ compiler flags directly, I'd recommend you just tell CMake you want to use C++11 and then let it work out the appropriate compiler and linker flags for you. This can be done by adding the following before your project() call:
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
This and related CMake features are discussed in detail in this article.
There is nothing C++11 about name mangling, it's been going on since the very first C++ implementations. However, do not forget that you must have built gtest for the same stdlib switch, because it can't find the Standard libraries.

Boost Regex throwing an error

I have the following error when I try to compile my code in g+ compiler using eclipse
In function `ZSt19__iterator_categoryIPKSsENSt15iterator_traitsIT_E17iterator_categoryERKS3_':
C:/Program Files (x86)/mingw/bin/../lib/gcc/mingw32/3.4.5/../../../../include/c++/3.4.5/bits/stl_algobase.h:(.text$_ZN5boost11basic_regexIcNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE6assignEPKcS7_j[boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign(char const*, char const*, unsigned int)]+0x22): undefined reference to `boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*, unsigned int)'
collect2: ld returned 1 exit status
Build error occurred, build is stopped
All I have done is this statement
boost::regex re("\s+"); along with the header #inlucde
Could you kindly tell me how to proceed ?
It appears that you're not linking to the correct library. Most Boost libraries are header-only, so you don't need to do anything about them at link time. Boost::regex, however, is one of the few that requires that you link with a library along with giving the compiler the proper headers.
After you fix that, you'll want to re-check the escape in your string -- right now your passing "\s", which shouldn't be allowed (at a guess, you probably want "\s+" instead).