g++ undefined reference error when linking to third party shared libraries - c++

I'm trying to link a c++ program that uses several shared libraries from 3rd parties. Mainly, these libraries come from a github project called MBSim and I downloaded the latest daily build with all its binaries, libraries and headers, which I installed on /usr/local/mbsim-env.
The most important libraries called are libmbsim, libopenmbvcppinterface, libfmatvec and libboost_filesystem (the last one comes with the MBSim distro).
I set up a simple code to test it and it compiles like a charm using
g++ main.cpp -m64 -g3 -std=c++11 -Wall
-Wfatal-errors -Werror -Wno-unknown-pragmas -fopenmp
`pkg-config --cflags mbsim` -I. -c -o main.o
The pkg-config part calls, as you can wonder, the include directories and flags:
-DHAVE_ANSICSIGNAL -DHAVE_OPENMBVCPPINTERFACE -DHAVE_BOOST_FILE_LOCK
-I/usr/local/mbsim-env/include
-I/usr/include/x86_64-linux-gnu
-I/usr/include/x86_64-linux-gnu/c++/5
-I/usr/local/include
My problems appear when I try to link the objects with the pre-compiled libraries with:
g++ system.o main.o -o teste -L/usr/local/mbsim-env/lib
-lmbsim -lopenmbvcppinterface -lboost_system
-lfmatvec -lm
-Wl,-rpath,/usr/local/mbsim-env/lib
Edit: On the above command, I've tried to use pkg-config --libs as well. The results remain the same.
First of all, the linker issues a warning that I'm linking against an older boost library:
/usr/bin/ld: warning: libboost_system.so.1.53.0,
needed by /usr/local/mbsim-env/lib/libmbsim.so,
may conflict with libboost_system.so.1.61.0
I'm aware of that, but I intentionally want to link against the old one because that's the one that was used to compile the MBSim libraries.
After that, I got several undefined reference warnings for almost every method that I call from MBSim:
system.cpp:59: undefined reference to
MBSim::RigidBody::RigidBody(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&)'
It seems to me that this error means that the target libraries don't have the RigidBody method implemented. Well, I know they do.
My first guess was that maybe the linker was looking at the wrong library path, so I set LD_LIBRARY_PATH=/usr/local/mbsim-env/lib and added the -rpath to the same folder. That didn't help at all.
Some googling showed me that the problem could be compiling in 64bits and linking it against 32bits libs. I believe that's not the case: I've done everything in Ubuntu 16.04 64bits and the MBSim libraries are also 64bits.
Could somebody point me out of this dead end?

Could it be that your 3rd party shared libraries are compiled using an older version of GCC? There was a new ABI introduced in GCC 5. It's probably enabled by default on your platform. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html.
You could try switching to the older ABI, but that might require your other libraries to have been compiled with the old ABI also. You might also be able to switch ABIs for the 3rd party libraries explicitly.

undefined reference to MBSim::RigidBody::RigidBody(std::__cxx11::basic_string<std::char_traits<char>, std::allocator<char> > const&)
This is a reference to MBSim::RigidBody::RigidBody(const string&), compiled with g++ -std=c++11 and g++ version 5 or later.
As any1 probably correctly guessed, the binary for libmbsim.so that you downloaded was built with g++ 4.x, and defines the same function with this signature:
MBSim::RigidBody::RigidBody(std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&)
Note lack of __cxx11 namespace. You can easily confirm or disprove this:
nm -C libmbsim.so | grep 'MBSim::RigidBody::RigidBody'
If this mismatch is indeed the cause, simply rebuild this project from source with your compiler. That's what open source is for.

Related

Undefined symbols when using user operator in tensorflow-gpu>=1.15

everybody. I wrote some user operators to extend tensorflow and tried to use CMake to compile the code to different shared libraries to fit different versions of tensorflow.
It works fine with tensorflow-gpu<=1.14 but not with 1.15 and 2.0. I got the following error when loading the library.
tensorflow.python.framework.errors_impl.NotFoundError: build/lib/libtensorflow_ctext.so: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
I tried nm build/lib/libtensorflow_ctext.so on 1.14 version and 2.0 version, both shared libraries have this undefined symbol in the middle.
U _ZN10tensorflow12OpDefBuilder4AttrENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
It seems that the program is going to find this symbol in the linked Tensorflow framework library libtensorflow_framework.so. I searched libtensorflow_framework.so.2 for similar symbols and found several of them.
0000000000cacc50 T _ZN10tensorflow12OpDefBuilder10DeprecatedEiSs
0000000000cace00 T _ZN10tensorflow12OpDefBuilder10SetShapeFnESt8functionIFNS_6StatusEPNS_15shape_inference16InferenceContextEEE
0000000000cacb20 T _ZN10tensorflow12OpDefBuilder13ControlOutputESs
0000000000cac980 T _ZN10tensorflow12OpDefBuilder13SetIsStatefulEv
0000000000cac970 T _ZN10tensorflow12OpDefBuilder14SetIsAggregateEv
0000000000cac960 T _ZN10tensorflow12OpDefBuilder16SetIsCommutativeEv
0000000000cac990 T _ZN10tensorflow12OpDefBuilder27SetAllowsUninitializedInputEv
0000000000cacb50 T _ZN10tensorflow12OpDefBuilder3DocESs
0000000000caca90 T _ZN10tensorflow12OpDefBuilder4AttrESs
0000000000cacac0 T _ZN10tensorflow12OpDefBuilder5InputESs
0000000000cacaf0 T _ZN10tensorflow12OpDefBuilder6OutputESs
0000000000cac830 T _ZN10tensorflow12OpDefBuilderC1ESs
0000000000cac830 T _ZN10tensorflow12OpDefBuilderC2ESs
0000000000c702d0 W _ZN10tensorflow12OpDefBuilderD1Ev
0000000000c702d0 W _ZN10tensorflow12OpDefBuilderD2Ev
The symbol _ZN10tensorflow12OpDefBuilder4AttrESs looks very similar but different in the last several letters. I don't really know what those "ESs"s and "ENSt7"s stand for.
Hints on how I could debug it are very appreciated. Here is the command to build my shared library (generated by cmake)
g++ -fPIC -shared -Wl,-soname,libtensorflow_ctext.so -o lib/libtensorflow_ctext.so src/CMakeFiles/bp_par_2d.dir/bp_par_2d.cc.o src/CMakeFiles/bp_par_2d_sv.dir/bp_par_2d_sv.cc.o src/CMakeFiles/fp_par_2d.dir/fp_par_2d.cc.o src/CMakeFiles/filter.dir/filter.cc.o cuda/CMakeFiles/bp_par_2d_cu.dir/bp_par_2d.cu.o cuda/CMakeFiles/bp_par_2d_sv_cu.dir/bp_par_2d_sv.cu.o cuda/CMakeFiles/fp_par_2d_cu.dir/fp_par_2d.cu.o cuda/CMakeFiles/filter_cu.dir/filter.cu.o tensorflow/CMakeFiles/bp_par_2d_ops.dir/bp_par_2d_ops.cu.o tensorflow/CMakeFiles/bp_par_2d_sv_ops.dir/bp_par_2d_sv_ops.cu.o tensorflow/CMakeFiles/fp_par_2d_ops.dir/fp_par_2d_ops.cu.o tensorflow/CMakeFiles/ramp_filter_ops.dir/ramp_filter_ops.cu.o CMakeFiles/tensorflow_ctext.dir/cmake_device_link.o -L/usr/lib/x86_64-linux-gnu/stubs -Wl,-rpath,/home/ltl/anaconda3/envs/tf_test/lib/python3.7/site-packages/tensorflow_core /home/ltl/anaconda3/envs/tf_test/lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so.2 -lcudadevrt -lcudart_static -lrt -lpthread -ldl
Well, this problem is solved.
I used nm -C instruction to look inside the .so files and found that in Tensorflow>=1.15.0, the function is defined as
0000000000caca90 T tensorflow::OpDefBuilder::Attr(std::string)
while in Tensorflow<=1.14.0, the function is defined as
0000000000c96ed0 T tensorflow::OpDefBuilder::Attr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
So, they use different settings on _GLIBCXX_USE_CXX11_ABI when compiling the shared library.
In order to be consistant and avoid those undefined symbol problems, I need to define -D_GLIBCXX_USE_CXX11_ABI=1 for early versions of Tensorflow and define -D_GLIBCXX_USE_CXX11_ABI=0 for later versions.

Linking error in boost program_options

I'm trying to build a 3rd party C++ library from source, and it depends on Boost. In the very last step when building, I got errors like this:
[ 90%] Linking CXX executable Shannon_RNASeq_Cpp
CMakeFiles/Shannon_RNASeq_Cpp.dir/src/main.cpp.o: In function `command_line_for_find_rep(int, char**, Shannon_C_setting&,
std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >&, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >&)':
/home/lambda/Shannon_Cpp_RNA_seq/src/main.cpp:320: undefined reference to `boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)'
I searched other questions on StackOverflow (e.g. Boost Program Options link error), and initially thought that it was caused by the different compilers used to build the non-header part of Boost, so I downloaded the source of Boost 1.68.0 and built it in my personal directory using the same compiler I used to build the 3rd party library (the compiler is gcc 8.2.0), and from the error message, I can tell that C++11 standard was used (you can see cxx11 in the error message), so it can't be the case that compiler not supporting C++11 is causing the trouble, as supposed previously (e.g. undefined reference to boost::program_options in xubuntu). To build Boost with a custom compiler (the default compiler of the server is terribly outdated), I followed the instructions here.
Some further search reveals that this is a linking problem. The 3rd party library must be built by CMake, and I followed the instruction in an answer to a previous question (How to link C++ program with Boost using CMake), by adding this line to CMakeLists.txt in the root directory of the 3rd party library:
target_link_libraries(Shannon_RNASeq_Cpp ${Boost_PROGRAM_OPTIONS_LIBRARY_RELEASE})
However, the same problem persists. I checked the file CMakeCache.txt, and confirmed that the desired compiler is used and Boost_PROGRAM_OPTIONS_LIBRARY_RELEASE points to the libboost_program_options.so in the directory where I put Boost 1.68.0 (the server has other older versions of Boost). I then used make VERBOSE=1 to check the bash commands invoked while building the library. The command doesn't sound wrong; the Boost program_options library has been linked; here's the command from make VERBOSE=1:
/usr/bin/cmake3 -E cmake_link_script CMakeFiles/Shannon_RNASeq_Cpp.dir/link.txt --verbose=1
It's referring to something in link.txt, and here's what I found there:
/home/lambda/mylibs/bin/c++ -g -rdynamic CMakeFiles/Shannon_RNASeq_Cpp.dir/src/main.cpp.o
CMakeFiles/Shannon_RNASeq_Cpp.dir/src/run_tasks.cpp.o -o
Shannon_RNASeq_Cpp lib_shannon/libmulti_graph_handler.a
lib_shannon/libcontig_graph_handler.a lib_shannon/libprimary_lib.a
-lboost_program_options lib_shannon/libsparse_flow_handler.a
-lglpk lib_shannon/libseq_graph_handler.a -lpthread
-lboost_system -Wl,-Bstatic -lmetis -Wl,-Bdynamic -lboost_filesystem
-lboost_program_options
So yes, Boost program_options has been linked (-lboost_program_options). I wonder if this is something wrong with program_options, since all errors that occurred at this stage are from program_options; I also linked to 2 other Boost compiled libraries, namely filesystem and system, and they do not appear in any error message I saw here.
I solved this problem myself (see my answer below), but new to CMake (and my only experiences with C++ is Rcpp), I don't know why it worked. Can someone please explain why switching to a static library worked? Also, when CMake linked to the default version of Boost on the server, it also by default used the .sos rather than the .as, even though the .as are available. Is there any way to tell CMake to use the .as rather than the .sos? Are there other solutions?
What I did was to change all the .sos in CMakeCache.txt into .as, like I changed libboost_program_options.so into libboost_program_options.a, and this error went away. Then the command from make VERBOSE=1 became:
/home/lambda/mylibs/bin/c++ -g -rdynamic CMakeFiles/Shannon_RNASeq_Cpp.dir/src/main.cpp.o
CMakeFiles/Shannon_RNASeq_Cpp.dir/src/run_tasks.cpp.o
-o Shannon_RNASeq_Cpp lib_shannon/libmulti_graph_handler.a
lib_shannon/libcontig_graph_handler.a lib_shannon/libprimary_lib.a
-Wl,-Bstatic -lboost_program_options lib_shannon/libsparse_flow_handler.a
-Wl,-Bdynamic -lglpk lib_shannon/libseq_graph_handler.a -lpthread
-Wl,-Bstatic -lboost_system -lmetis -lboost_filesystem -lboost_program_options -Wl,-Bdynamic

Static linking to libcrypto++, with g++

I am trying to compile a program on my system with Debian Wheezy and g++4.7. I want it to be able to run on another system with Debian Squeeze (and no recent g++). I can't compile the program on the Squeeze, because I use certain C++11 features the old g++ does not support, as well as a new Boost version and libcrypto++9.
As far as I understand the usual way to get around this problem is to static link the libraries not supported at the other system, in my case libstdc, boost and crypto++.
My (linking) compiler call right now is
g++-4.7 .obj/btcmirco.o -Wl,-Bstatic -lboost_program_options -lboost_system -lcrypto++ -Wl,-Bdynamic -lcurl -static-libgcc -std=c++11 -o MyProgram
However I seem to have missed something, because it throws a lot of undefined reference errors. It works fine if I dynamic link to crypto++ (and only static link libstdc and boost).
Can anyone tell me whats wrong, or if there is a fundamental error in my approach?
The linker errors I get are (shorted):
`.text._ZN8CryptoPP22BufferedTransformationD2Ev' referenced in section `.text._ZN8CryptoPP22BufferedTransformationD1Ev[_ZN8CryptoPP22BufferedTransformationD1Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o): defined in discarded section `.text._ZN8CryptoPP22BufferedTransformationD2Ev[_ZN8CryptoPP22BufferedTransformationD5Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o)
`.text._ZN8CryptoPP25MessageAuthenticationCodeD2Ev' referenced in section `.text._ZN8CryptoPP25MessageAuthenticationCodeD1Ev[_ZN8CryptoPP25MessageAuthenticationCodeD1Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o): defined in discarded section `.text._ZN8CryptoPP25MessageAuthenticationCodeD2Ev[_ZN8CryptoPP25MessageAuthenticationCodeD5Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o)
I experienced the same problem and this has to do with the fact that you are trying to mix code generated by g++-4.7 (your program) with code generated by a previous version of g++ (cryptopp library).
The reason behind this is that when you execute compile the library executing make command, it uses the default version of g++ set up for your system, usually the one that comes with the OS.
In order to solve the issue what you should do is compile cryptopp library with g++-4.7.
For that, compile the library by executing make CXX=g++-4.7. The resulting static library shouldn't give you the error when being linked with your code.

Error when static linking g++

I have a problem, i want to compile my application with static linking of mysql connector.
My command line:
g++ -o newserver stdafx.cpp ... -lboost_system -lboost_thread
-lpthread -lmysqlcppconn -static /usr/lib/libmysqlcppconn-static.a -std=c++0x
But i have error:
/usr/bin/ld: cannot find -lmysqlcppconn
/tmp/ccxpOfdZ.o: In function `IsEqualsDns(unsigned long, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)':
Server.cpp:(.text+0x356e): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
collect2: ld returned 1 exit status
How can i fix this?
Thanks!
Where is the library libsqlcppconn.a or libsqucppconn.so
(static or dynamic)? The compiler is looking for it, and
doesn't find it.
Presumably, this is the same library as
/usr/lib/mysqlcppconn-static.a. If so, just drop the
-lmysqlcppconn. Or just use -lmysqlcppconn-static (no
spaces), and forget about the /usr/lib/libmysqlconn-static.a.
With a name like that, there shouldn't be a corresponding .so,
which means that g++ will link it statically, even without the
-static. You only need the -static if there is both
a libmysqlconn-static.so and a libmysqlconn-static.a in the
same directory.
With regards to the second error (which is just a warning, but
will cause problems if you try to run the linked program on
other machines, or even after an upgrade of your machine): if
you use -static anywhere in your command line (as you
currently do), then it applies to all files linked afterwards.
Including the system libraries, which you don't want to link
statically. My guess is that the -static isn't necessary (see
above); if it is, place it immediately before the library you
want to link statically, and place a -dynamic immediately
after (so that any following libraries, including the system
libraries, will be dynamically linked).
You could try g++ -static YOUR ARGUMENTS.
If you are coming from a Windows platform, linking against Boost can give a few surprises. The typicall Boost installation (e.g. after ./b2 install) will make both dynamic and static libraries and put them in the same directory. Typically, the two library forms only differ in their extension (.so or .a).
Windows supports auto-linking, which basically means that library files contain some flags in their first few bytes indicating whether they are for dynamic or for static linking. On Linux platforms, this is not the case and the linker gets confused which file to load (since you don't provide the extension of the library name). Therefore, you need to tell your linker which form of linking you want.

undefined reference to boost::system::system_category() when compiling

I'm trying to compile a program on Ubuntu 11.10 that uses the Boost libraries. I have the 1.46-dev Boost libraries from the Ubuntu Repository installed, but I get an error when compiling the program.
undefined reference to boost::system::system_category()
What is it that I do wrong?
The boost library you are using depends on the boost_system library. (Not all of them do.)
Assuming you use gcc, try adding -lboost_system to your compiler command line in order to link against that library.
Linking with a library that defines the missing symbol (-lboost_system) is the obvious solution, but in the particular case of Boost.System, a misfeature in the original design makes it use boost::system::generic_category() and boost::system::system_category() needlessly. Compiling with the flag -DBOOST_SYSTEM_NO_DEPRECATED disables that code and lets a number of programs compile without requiring -lboost_system (that link is of course still needed if you explicitly use some of the library's features).
Starting from Boost 1.66 and this commit, this behavior is now the default, so hopefully fewer and fewer users should need this answer.
As noticed by #AndrewMarshall, an alternative is to define BOOST_ERROR_CODE_HEADER_ONLY which enables a header-only version of the code. This was discouraged by Boost as it can break some functionality. However, since 1.69, header-only seems to have become the default, supposedly making this question obsolete.
Another workaround for those who don't need the entire shebang: use the switch
-DBOOST_ERROR_CODE_HEADER_ONLY.
If you use CMake, it's add_definitions(-DBOOST_ERROR_CODE_HEADER_ONLY).
The above error is a linker error... the linker a program that takes one or more objects generated by a compiler and combines them into a single executable program.
You must add -lboost_system to you linker flags which indicates to the linker that it must look for symbols like boost::system::system_category() in the library libboost_system.so.
If you have main.cpp, either:
g++ main.cpp -o main -lboost_system
OR
g++ -c -o main.o main.cpp
g++ main.o -lboost_system
When using CMAKE and find_package, make sure it is :
find_package(Boost COMPONENTS system ...)
and not
find_package(boost COMPONENTS system ...)
Some people may have lost hours for that ...
I got the same Problem:
g++ -mconsole -Wl,--export-all-symbols -LC:/Programme/CPP-Entwicklung/MinGW-4.5.2/lib -LD:/bfs_ENTW_deb/lib -static-libgcc -static-libstdc++ -LC:/Programme/CPP-Entwicklung/boost_1_47_0/stage/lib \
D:/bfs_ENTW_deb/obj/test/main_filesystem.obj \
-o D:/bfs_ENTW_deb/bin/filesystem.exe -lboost_system-mgw45-mt-1_47 -lboost_filesystem-mgw45-mt-1_47
D:/bfs_ENTW_deb/obj/test/main_filesystem.obj:main_filesystem.cpp:(.text+0x54):
undefined reference to `boost::system::generic_category()
Solution was to use the debug-version of the system-lib:
g++ -mconsole -Wl,--export-all-symbols -LC:/Programme/CPP-Entwicklung/MinGW-4.5.2/lib -LD:/bfs_ENTW_deb/lib -static-libgcc -static-libstdc++ -LC:/Programme/CPP-Entwicklung/boost_1_47_0/stage/lib \
D:/bfs_ENTW_deb/obj/test/main_filesystem.obj \
-o D:/bfs_ENTW_deb/bin/filesystem.exe -lboost_system-mgw45-mt-d-1_47 -lboost_filesystem-mgw45-mt-1_47
But why?
When I had this, problem, the cause was the ordering of the libraries. To fix it, I put libboost_system last:
g++ mingw/timer1.o -o mingw/timer1.exe -L/usr/local/boost_1_61_0/stage/lib \
-lboost_timer-mgw53-mt-1_61 \
-lboost_chrono-mgw53-mt-1_61 \
-lboost_system-mgw53-mt-1_61
This was on mingw with gcc 5.3 and boost 1.61.0 with a simple timer example.
in my case, adding -lboost_system was not enough, it still could not find it in my custom build environment. I had to use the advice at Get rid of "gcc - /usr/bin/ld: warning lib not found" and change my ./configure command to:
./configure CXXFLAGS="-I$HOME/include" LDFLAGS="-L$HOME/lib -Wl,-rpath-link,$HOME/lib" --with-boost-libdir=$HOME/lib --prefix=$HOME
for more details see Boost 1.51 : "error: could not link against boost_thread !"
...and in case you wanted to link your main statically, in your Jamfile add the following to requirements:
<link>static
<library>/boost/system//boost_system
and perhaps also:
<linkflags>-static-libgcc
<linkflags>-static-libstdc++