Getting undefined reference error but nm shows symbol present - c++

I am building a large application using libkml. I am using the cmake port of libkml from here: https://github.com/rashadkm/libkml
I am getting a stranged undefined reference to symbol error even thought the symbol appears to be referenced and defined.
This is the make command:
/usr/bin/c++ -fPIC -Werror=return-type -Werror=return-type -Wall
-Werror=parentheses -Werror=uninitialized -Werror=missing-braces
-fPIC -O0 -Wall -fPIC -fvisibility=hidden -fno-strict-aliasing
-Wno-long-long -m64 -g -D_DEBUG --coverage -Wl,-Bsymbolic -Wl,--
no-undefined -shared -o GPS2KML.plb CMakeFiles/GPS2KML.dir
/gps.cpp.o CMakeFiles/GPS2KML.dir/kml.cpp.o CMakeFiles/GPS2KML.dir
/stdafx.cpp.o /trunk/src/filter/GPS2KML/external/libkml/lib/cmake
/libkml/../../libkmlconvenience.so.1.3.1 /trunk/src/filter/GPS2KML
/external/libkml/lib/cmake/libkml/../../libkmlengine.so.1.3.1
/trunk/src/filter/GPS2KML/external/libkml/lib/cmake/libkml/../..
/libkmldom.so.1.3.1 /trunk/src/filter/GPS2KML/external/libkml
/lib/cmake/libkml/../../libkmlbase.so.1.3.1 -lminizip -luriparser
-lexpat
The make output:
CMakeFiles/GPS2KML.dir/kml.cpp.o: In function `cKML::~cKML()':
/trunk/src/filter/GPS2KML/src/kml.cpp:55: undefined reference to `*kmldom::SerializePretty(boost::intrusive_ptr<kmldom::Element> const&)*'
collect2: error: ld returned 1 exit status
Now if i do this:
daniyal#daniyal-Inspiron-5521:/$ nm --demangle --extern-only --defined-only ../trunk/src/filter/GPS2KML/external/libkml/lib/libkmldom.so | grep SerializePretty
It clearly shows:
000000000013c9aa T kmldom::SerializePretty[abi:cxx11](boost::intrusive_ptr<kmldom::Element> const&)
Now I fail to understand what is the problem. I have check the existing questions on stackoverflow regarding this, I found four solutions in the existing questions:
In some cases the demangled symbol name did not match with the symbol that was causing error. This is clearly not my case.
Use -llibrary at the end of the command after putting the name of the .o file. So that when the linker encounters the library it has an undefined symbol present in that library. This is clearly not a solution for me as I have the libraries present at the end of the command.
In some cases the symbol is present in the shared library but does not have external linkage or is not defined. This can be confirmed by using nm with --extern-only and --defined-only. And hence this is also not a solution for me.
Edit:
Additional Info:
This is the cmake file I am using:
find_package(LibKML REQUIRED)
include_directories(${LIBKML_INCLUDE_DIRS})
add_filter(${PROJECT}
gps.h
gps.cpp
kml.h
kml.cpp
#can2gps.h
#can2gps.cpp
stdafx.h
stdafx.cpp )
target_link_libraries (${PROJECT} ${LIBKML_LIBRARIES})
add_filter roughly translates to this macro:
add_library(${NAME} MODULE ${ARGN} ${${NAME}_MOC} ${${NAME}_UI} ${${NAME}_QRC})
target_link_libraries(${NAME} ${BUILD_LIBS} ${QT_LIBRARIES} ${ADTF_OPENGL_LIBRARY} ${ADTF_ADDITIONAL_UTILS_LIBS})
set_target_properties(${NAME}
PROPERTIES
SUFFIX ".plb"
)
if(UNIX)
set_target_properties(${NAME}
PROPERTIES
PREFIX ""
)

It looks like you have an ABI mismatch issue. ABI is "Application Binary Interface", basically the specification for exactly how arguments make it onto the stack (or are put in registers) and various other things like that.
Try making sure your code is compiled with the -std=c++11 (or -std=gnu++11 if you use any GNU extensions) flag. It looks like that's how libkml was compiled. C++11 has a bunch of new features that require an ABI compatibility break with pre-C++11. C++14 and C++1z are less drastic changes, but they may also break ABI compatibility, I'm not sure. In this case though, the demangled symbol is clear, libkml wants at least C++11.

Related

udata.cpp: undefined reference to `icudt71_dat'

I am getting an odd ICU related linking error in the now project when building on Ubuntu 22.04.
/usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x23
/home/bkey1/vcpkg/installed/x64-linux/debug/lib/libicuuc.a(udata.ao): in function `openCommonData(char const*, int, UErrorCode*)':
udata.cpp:(.text+0x23f7): undefined reference to `icudt71_dat'
/usr/bin/ld: udata.cpp:(.text+0x2458): undefined reference to `icudt71_dat'
The link command is as follows.
usr/bin/cmake -E cmake_link_script CMakeFiles/now.dir/link.txt --verbose=1
/usr/bin/c++ -std=c++2a -Wall -Wextra -Wfloat-equal -Wno-long-long -Wpedantic -funsigned-char -D_GNU_SOURCE=1 -rdynamic CMakeFiles/now.dir/GetStardate.cpp.o CMakeFiles/now.dir/GetTime.cpp.o CMakeFiles/now.dir/GetTimePlatformPOSIX.cpp.o CMakeFiles/now.dir/GetTimePlatformWin32.cpp.o CMakeFiles/now.dir/ISO8601_time.cpp.o CMakeFiles/now.dir/InitLocale.cpp.o CMakeFiles/now.dir/executable_path.cpp.o CMakeFiles/now.dir/now.cpp.o CMakeFiles/now.dir/nowStrings.cpp.o -o now /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_chrono.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_filesystem.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_locale.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_log.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_program_options.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_regex.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_system.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_thread.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_date_time.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_log_setup.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libboost_atomic.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libicudata.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libicui18n.a /home/bkey1/vcpkg/installed/x64-linux/debug/lib/libicuuc.a
First I would like to stress, that there is very little information provided, so the answers such as my own will most likely need to guess what is happening. On the other hand I understand your situation: you cannot include info, that you don't know is relevant to the topic.
Answer:
I would like to draw your attention to the fact, that the symbol icudt71_dat does not appear anywhere in the code directly, but is generated using a macro. (Check .../source/common/unicode/utypes.h) So if the linker complains about not having found such a symbol it most probably means you are linking against a different version of the library than you have a header for. Now I don't know how specifically this could have happened, I would have to see your system, include path, link path etc. However I strongly suggest to revisit both include and link paths. You could perhaps recompile the library, verify, there is no other version recompile and start again. If the header corresponds to the source, it should work.
Aparently you missing icu-devtools package. Install that with command
sudo apt get install icu-devtools

Getting error "can not be used when making a shared object; recompile with -fPIC" although fpic is used

I am currently building a shared library (lib1.so) out of a cmake environment.
lib1.so depends on an external static lib libLASlib.a (which I am able to recompile if necessary).
Everything works wonder on windows so far, but it's another story when switching to linux:
/usr/bin/ld: lib/LASlib/libLASlib.a(lasreader.cpp.o): relocation R_X86_64_PC32 against symbol `_ZN9LASreader35read_point_filtered_and_transformedEv' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
So I tried recompiling the libLASlib with -fPIC -> same error
Due to my environment I could not verify the fpic was effectivelly added to the gcc command line.
Here is what I tried to confirm there was no issue with the fPIC:
readelf --dynamic libLASlib.a | grep lasreader.cpp.o -A2
File: libLASlib.a(lasreader.cpp.o)
There is no dynamic section in this file.
For the record not a single cpp.o was found with a dynamic section
I have tried just to see what it would give if i changed liblas from a static to a shared library -> no error
Any thoughs?
Many thanks!
You need to compile lasreader.cpp with -fPIC. Something like this:
g++ -c -fPIC -o lasreader.cpp.o lasreader.cpp
The fPIC was indeed not applied
Conan doesn't seem to forward the fPIC option
I edited the CMAKELIST and added
set_property(TARGET LASlib PROPERTY POSITION_INDEPENDENT_CODE ON)
And it eventualy passed

Clang and undefined symbols when building a library

I'm working on a C++ framework, and there's a few issues when I compile it on OSX with Clang.
First of, I'm using some other libraries, such as openssl, and clang complains that some symbols aren't solved when I build the library. They shouldn't be: these libraries will be linked with the final binary, it shouldn't happen on an intermediary.
Then, there's also a few methods and variables that are supposed to be implemented in the "client" binary... with GCC, no problems, but Clang also complains that these symbols can't be solved during compilation.
How come ? What should I do ?
Here's my CMakeLists.txt in case that can be useful:
cmake_minimum_required(VERSION 2.8)
project(crails_project)
set(CMAKE_CXX_FLAGS "-std=c++0x -Wall -Wno-deprecated-declarations -pedantic -DASYNC_SERVER -DSERVER_DEBUG -DUSE_MONGODB_SESSION_STORE")
find_package(cppnetlib REQUIRED)
include_directories(include /usr/local/include ${CPPNETLIB_INCLUDE_DIRS} .)
file(GLOB crails_core
src/*.cpp)
file(GLOB crails_sql
src/sql/*.cpp)
file(GLOB crails_mongodb
src/mongodb/*.cpp)
add_library(crails-core SHARED ${crails_core})
add_library(crails-sql SHARED ${crails_sql})
add_library(crails-mongodb SHARED ${crails_mongodb})
This is the command that crashes:
/usr/bin/c++ -std=c++0x -Wall -Wno-deprecated-declarations -pedantic -DASYNC_SERVER -DSERVER_DEBUG -DUSE_MONGODB_SESSION_STORE -dynamiclib -Wl,-headerpad_max_install_names -o libcrails-core.dylib -install_name /Users/michael/Personal/crails/build/libcrails-core.dylib CMakeFiles/crails-core.dir/src/assets.cpp.o CMakeFiles/crails-core.dir/src/cgi2params.cpp.o CMakeFiles/crails-core.dir/src/cipher.cpp.o [...]
And here are the two kinds of error I get:
Undefined symbols for architecture x86_64:
"_BIO_ctrl", referenced from:
Cipher::encode_base64(unsigned char*, unsigned int) const in cipher.cpp.o
And the second one:
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
"vtable for boost::detail::thread_data_base", referenced from:
boost::detail::thread_data_base::thread_data_base() in server.cpp.o
I don't recommend to enable global dynamic lookup:
-undefined dynamic_lookup
which will mark all undefined symbols as having to be looked up at runtime.
Much more safe way to resolve it for specific symbols:
-Wl,-U,symbol_name, which only does so for the given symbol (note: you have to prepend an underscore to the symbol name)
You could also use weak dynamic linking:
extern int SayHello() __attribute__((weak));
Solved it ! Clang needs to receive the option -undefined dynamic_lookup to ignore missing symbols when compiling a library.
Add this to the CMakeFile.txt to produce the expected effect:
if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")
endif()
According to one of the commenters you have to use -lcrypto to prevent the first error.
The second error seems to be due to an ABI incompatibility of clang and gcc. Rebuild boost with clang++ and libc++. See SO posts Is clang++ ABI same as g++?, Why can't clang with libc++ in c++0x mode link this boost::program_options example? and How to compile/link Boost with clang++/libc++?.
In case you run into linker troubles with other libraries you should also try to rebuild those with clang++.
Edit:
In order to instruct the OS X linker to allow unresolved symbols you should add -undefined dynamic_lookup to the linker options. See also SO post Error when making dynamic lib from .o

How to cope with older library installed in `/usr/lib` by sysadmin

I have recently got an account on a supercomputer grid, and I'm trying to compile my code in theri system. Problem is that program won't link with following errors:
/mnt/opt/tools/slc6/binutils/2.22/bin/ld: warning: libboost_system.so.1.55.0, needed by /mnt/home/jbzdak/tools/boost_1_55//lib/libboost_thread.so, may conflict with libboost_system.so.5
/mnt/opt/tools/slc6/binutils/2.22/bin/ld: /mnt/home/jbzdak/tools/boost_1_55//lib/libboost_thread.so: undefined reference to symbol '_ZN5boost6system15system_categoryEv'
/mnt/opt/tools/slc6/binutils/2.22/bin/ld: note: '_ZN5boost6system15system_categoryEv' is defined in DSO /mnt/home/jbzdak/tools/boost_1_55//lib/libboost_system.so.1.55.0 so try adding it to the linker command line
/mnt/home/jbzdak/tools/boost_1_55//lib/libboost_system.so.1.55.0: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
Which is due to the fact that my program needs boost 1.55, and only 1.39 is instlled on the system in /usr/lib64. I have installed my version of boost in local folder, but somehow still system one is loaded first.
Here is excerpt from flags passed to the compiler:
-std=gnu++11 -Werror -Wall -lboost_thread -lboost_filesystem -lboost_system -lboost_iostreams -g -DG4OPTIMISE -Iinclude
-W -Wall -ansi -pedantic -Wno-non-virtual-dtor -Wno-long-long -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -pipe -O2
full listing of flags is here (they should be irrevelant).
Here are revelant config variables:
LIBRARY_PATH /mnt/home/jbzdak/tools/boost_1_55/lib:
CPLUS_INCLUDE_PATH /mnt/home/jbzdak/tools/boost_1_55/include:/mnt/home/jbzdak/tools/geant4.9.6.3/compile/include/Geant4
LD_LIBRARY_PATH /mnt/home/jbzdak/tools/boost_1_55/lib:/mnt/opt/tools/slc6/gcc/4.8.3/lib64: ...
Directory /mnt/home/jbzdak/tools/boost_1_55 contains installed boost library.
I use GCC 4.8.3 with ld 2.22.
I have very little experience with linker errors hence the question. Is there any way to exclude boost libraries in /usr/lib64, or make the linker use locally installed libraries, and and ignore the system one?
I said in a comment:
There's no -L/alternative/location/of/boost/lib shown, so the compiler (linker) doesn't know it needs to look somewhere else for your modern Boost library. You may need -Wl,rpath,/alternative/location/of/boost/lib as well.
And the question was asked:
Why didn't LD_LIBRARY_PATH solve the issue?
Because LD_LIBRARY_PATH is a run-time variable rather than a link-time variable. It affects where the /lib/ld.so.1 (or equivalent) dynamic loader looks for libraries when you run a program, not where the linker looks to find its libraries.
After some additional debugging and asking another question, I found out the root cause of problem. Any -L parameter has precedence over LIBRARY_PATH and somehow -L/usr/lib64 was added (hence it had precedence over my version).
To check what options are sent to gcc pass -v parameter.

Linking errors with "-Wl,--no-undefined -Wl,--no-allow-shlib-undefined"

Using the flags "-Wl,--no-undefined -Wl,--no-allow-shlib-undefined" with GCC leads to the following compilation errors on the Travis CI image but not on my machine (both are Ubuntu 12.04 64-bits):
Linking CXX shared library libmocap.so
cd /tmp/_travis/build/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/mocap.dir/link.txt --verbose=1
/usr/bin/g++ -fPIC --coverage -Werror -pedantic -Wno-long-long -Wall -Wextra -Wcast-align -Wcast-qual -Wformat -Wwrite-strings -Wconversion -fvisibility=hidden -Wl,--no-undefined -Wl,--no-allow-shlib-undefined -Wl,--as-needed -shared -Wl,-soname,libmocap.so.0.0.0 -o libmocap.so.UNKNOWN CMakeFiles/mocap.dir/abstract-marker.cc.o CMakeFiles/mocap.dir/abstract-virtual-marker.cc.o CMakeFiles/mocap.dir/color.cc.o CMakeFiles/mocap.dir/link.cc.o CMakeFiles/mocap.dir/marker-set-factory.cc.o CMakeFiles/mocap.dir/marker-set.cc.o CMakeFiles/mocap.dir/marker-trajectory-factory.cc.o CMakeFiles/mocap.dir/marker-trajectory.cc.o CMakeFiles/mocap.dir/marker.cc.o CMakeFiles/mocap.dir/mars-marker-set-factory.cc.o CMakeFiles/mocap.dir/math.cc.o CMakeFiles/mocap.dir/pose.cc.o CMakeFiles/mocap.dir/segment.cc.o CMakeFiles/mocap.dir/string.cc.o CMakeFiles/mocap.dir/trc-marker-trajectory-factory.cc.o CMakeFiles/mocap.dir/virtual-marker-one-point-measured.cc.o CMakeFiles/mocap.dir/virtual-marker-relative-to-bone.cc.o CMakeFiles/mocap.dir/virtual-marker-three-points-measured.cc.o CMakeFiles/mocap.dir/virtual-marker-three-points-ratio.cc.o CMakeFiles/mocap.dir/virtual-marker-two-points-measured.cc.o CMakeFiles/mocap.dir/virtual-marker-two-points-ratio.cc.o
/lib/x86_64-linux-gnu/libc.so.6: undefined reference to `_dl_argv#GLIBC_PRIVATE'
/lib/x86_64-linux-gnu/libc.so.6: undefined reference to `_rtld_global_ro#GLIBC_PRIVATE'
/usr/lib/gcc/x86_64-linux-gnu/4.6/libstdc++.so: undefined reference to `__tls_get_addr#GLIBC_2.3'
/lib/x86_64-linux-gnu/libc.so.6: undefined reference to `_rtld_global#GLIBC_PRIVATE'
/lib/x86_64-linux-gnu/libc.so.6: undefined reference to `__libc_enable_secure#GLIBC_PRIVATE'
collect2: ld returned 1 exit status
make[2]: *** [src/libmocap.so.UNKNOWN] Error 1
Strangely, this does not occur on my machine and hence is a bit difficult to reproduce.
What is the right way to link against the libc and libstdc++ when these flags are enabled?
(as you might guess from the output, I use CMake to generate the compilation command)
The default for the 2nd argument is "--allow-shlib-undefined".
Probably if you choose that option the code will build.
This 2nd argument deals with build time checking where enabling this means checking that the library that you are linking against in turn has its dependencies wired up at build time.. And that is not necessarily the case.
The first argument makes sure that you have not forgotten to state a dependency to a runtime library (which may also be a dependency that a runtime library has to another runtime library).For example if you are calling a function where the implementation is in an example runtime library "libfunc.so" and that library in turn will call a function in another runtime library "libext.so" then by declaring a dependency to both "func" and "ext" the libfunc.so will be generated to internally include a dependency reference to libext.
If you would leave out "--no undefined" and forget to add dependency declarations then the build would still succeed, trusting that your runtime linker would resolve dependencies at runtime. And since the build was successful you might trust that everything will be ok not knowing that the build has deferred responsibility to the runtime linker. But most often the runtime linker is not designed to search for unresolved references but expects to find such dependencies stated in the runtime library. And if no such reference is there you will get a runtime error. Runtime errors are typically much more costly than resolving a compile time error.