I am trying to build a shared library. My aim is to pull all dependencies as static libraries into a single shared library. My understanding is that this can be done with the -Wl,--whole-archiveflag. Here is a snippet of my CMake script responsible for configuring the shared library.
# shared library
add_library(semsim SHARED "${SEMSIM_HEADERS}" "${SEMSIM_SOURCES}") # created the shared library
#fPIC for linux shared library strs
set_property(TARGET semsim PROPERTY POSITION_INDEPENDENT_CODE ON) #turn fPIC on
target_compile_options(semsim PRIVATE -Wl,--whole-archive) # enable pulling static libraries into shared library
The error I'm getting whilst trying to compile (verbose mode is on) is:
/usr/bin/c++ -fPIC -std=c++14 -g -shared -Wl,-soname,libsemsim.so -o libsemsim.so CMakeFiles/semsim.dir/CurlGet.cpp.o CMakeFiles/semsim.dir/RDFNode.cpp.o CMakeFiles/semsim.dir/Subject.cpp.o CMakeFiles/semsim.dir/Predicate.cpp.o CMakeFiles/semsim.dir/Resource.cpp.o CMakeFiles/semsim.dir/Triple.cpp.o CMakeFiles/semsim.dir/SemsimUtils.cpp.o CMakeFiles/semsim.dir/MetaID.cpp.o CMakeFiles/semsim.dir/XmlAssistant.cpp.o CMakeFiles/semsim.dir/Reader.cpp.o CMakeFiles/semsim.dir/Editor.cpp.o CMakeFiles/semsim.dir/Writer.cpp.o CMakeFiles/semsim.dir/RDF.cpp.o CMakeFiles/semsim.dir/Participant.cpp.o CMakeFiles/semsim.dir/PhysicalEntity.cpp.o CMakeFiles/semsim.dir/PhysicalPhenomenon.cpp.o CMakeFiles/semsim.dir/PhysicalProcess.cpp.o CMakeFiles/semsim.dir/PhysicalPropertyResource.cpp.o CMakeFiles/semsim.dir/PhysicalForce.cpp.o CMakeFiles/semsim.dir/Query.cpp.o CMakeFiles/semsim.dir/SemsimCombineArchive.cpp.o CMakeFiles/semsim.dir/Triples.cpp.o ../../third_party/libCombine-0.2.3/INSTALL/lib/libcombine-static.a ../../third_party/zipper/INSTALL/lib/libZipper-static.a ../../third_party/zlib-1.2.11/INSTALL/lib/libz.a /usr/local/lib/libbz2.a /usr/local/lib/libxml2.a -ldl -lbz2 -lz -lcurl -lxslt
/usr/bin/ld: /usr/local/lib/libxml2.a(uri.o): relocation R_X86_64_PC32 against symbol `xmlFree' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
However, according to this question, /usr/local/lib/lxml2.a was already compiled fPIC:
(base) ciaran#DESKTOP:/usr/local/lib$ ls
cmake libcharset.so.1.0.0 libraptor2.a librasqal.la librdf.so libxml2.so.2
libbz2.a libcurlcpp.a libraptor2.la librasqal.so librdf.so.0 libxml2.so.2.9.10
libcharset.a libiconv.la libraptor2.so librasqal.so.3 librdf.so.0.0.0 pkgconfig
libcharset.la libiconv.so libraptor2.so.0 librasqal.so.3.0.0 libxml2.a python2.7
libcharset.so libiconv.so.2 libraptor2.so.0.0.0 librdf.a libxml2.la python3.6
libcharset.so.1 libiconv.so.2.6.1 librasqal.a librdf.la libxml2.so xml2Conf.sh
(base) ciaran#DESKTOP:/usr/local/lib$ readelf -d libxml2.a | grep TEXT
(base) ciaran#DESKTOP:/usr/local/lib$
This leads me to believe that perhaps I've misunderstood the error message - could somebody shed some light on what's going on here?
You haven't misunderstood the error message, or at least not significantly.
However, according to this question, /usr/local/lib/lxml2.a was already compiled fPIC
You conclude this because the command:
/usr/local/lib$ readelf -d libxml2.a | grep TEXT
outputs nothing, which shows that you chose the second most popular answer to
that question, although at this writing it is not nearly as up-voted as the most popular answer
I guess you did that because the most popular answer tells you how to test an object file for
PIC-ness, and you want to a test a library, as the second most popular answer does. Or
perhaps because you tried the most popular answer first and it indicated that your libxml2.a
was PIC.
But the second most popular answer tests a shared library, and you use it to test
the object files in a static library. A shared library is very different from
an object file, or a static library of such, and the question to which this answer was given asks how to test
the PIC-ness of an object file. So this answer does not answer the question: really
it suggests how to test whether a file that has a name like a shared library actually is
a shared library.
The most popular answer does answer the question, and if was a correct answer then
it would be the right one for you too, because what the linker has diagnosed:
/usr/local/lib/libxml2.a(uri.o): relocation R_X86_64_PC32 against symbol `xmlFree' \
can not be used when making a shared object; recompile with -fPIC
is that the object file uri.o archived in libxml2.a is not PIC.
But the most popular answer, notwithstanding its popularity, is invalid.
The PIC-ness test that it proposes - with no supporting argument - can
misidentify non-PIC object files as PIC (as perhaps you observed).
None of the answers to that question
is a correct answer for you, and it doesn't really matter why. The system linker itself
is the ultimate authority on whether an object file is or is not position-independent.
Any one-liner test other than attempting to link the object file into a DSO is just an attempt
to second-guess the linker: if its verdict differs from
the linker's it implies that the test is broken, not the linker.
And you already know the linker's verdict. It has tried to link libxml2.a(uri.o)
into a DSO and found that it can't, because the it contains a non-position-independent
relocation record.
The /usr/local/lib/libxml2.a you've got was built on the traditional
default assumption that the object files archived therein need not be compiled to
position-independent code (with -fPIC) because the static library would only
be input to the linkage of non-position-independent executables. If you want to
link some PI binary, then you'll link it with the shared libary libxml2.so,
which is PI by definition. Your libxml2.a was furthermore built with a compiler
that does not emit PI object code by default. It is possible that your current
compiler still has that increasingly antiquated trait, but you don't need to find
out.
You need to replace your local libxml2 install with one in which the object
files in libxml2 have been compiled with fPIC. If you already know how to do that you can skip the rest of this and get on with it.
If you have somewhere got the source package of libxml2 that was built
and installed and you want to stick with that revision, then cd into its root directory and run:
make uninstall
Otherwise, remove the installation by deleting, as root, all files and symlinks matching
/usr/local/lib/libxml2 and the directory /usr/local/include/libxml2
If you want to stick with a source package you've already got (where you ran make uninstall),
then, in its root directory, run:
make distclean
to restore to its pristine state.
If you don't have a source package you want to stick with, then clone or download-and-extract the latest from https://gitlab.gnome.org/GNOME/libxml2/,
then cd into the root directory and run:
./autogen.sh
to generate the build-system.
Whatever you've done thus far, in the package root directory then run:
./configure CFLAGS=-fPIC [any other non-default configuration options]
If that completes successfully, then run:
make
If that completes successfully, then as root run:
make install
If that completes successfully, then /usr/local/lib/libxml2.a will
be recreated containing PI object files and you will be able to link your
shared library against it.
If you are unsure about [any other non-default configuration options] for
the ./configure command, then run:
./configure -h
beforehand for help, and/or seek advice.
Related
I am trying to cross-compile for the Raspberry Pi with LLVM using LLD as the linker with the gnu stdlib implementation. I get a load of the following warnings, which refer to relocations inside read-only sections, all contained within the standard library implementation (seems to happen for c and cpp).
ld.lld: error: can't create dynamic relocation R_ARM_ABS32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /home/ted/cross.llvm.raspbian/sysroot/usr/lib/gcc/arm-linux-gnueabihf/6.3.0/../../../arm-linux-gnueabihf/libm.a(s_atan.o)
>>> referenced by s_atan.o:(atanMp.constprop.0) in archive /home/ted/cross.llvm.raspbian/sysroot/usr/lib/gcc/arm-linux-gnueabihf/6.3.0/../../../arm-linux-gnueabihf/libm.a
While the diagnostic tells me to use -z,notext this is apparently bad. Is there another way to fix the issue, or am I using incorrect flags for the cross-compilation?
Side note: for the cross-compilation I use
clang++ --target=arm-linux-gnueabihf --sysroot=./sysroot -fuse-ld=lld --verbose test.cpp -o test
./sysroot contains /usr and /lib from the Raspberry.
The problem is that by default clang compiler tries to link dynamic libraries. And your dynamic library is (most probably) a symlink to absolute path.
To me it is like that:
$ ls -l /Volumes/AlphaRacing/emlid-rootfs/lib/arm-linux-gnueabihf/libm*
# ...
/Volumes/AlphaRacing/emlid-rootfs/lib/arm-linux-gnueabihf/libm.so -> /lib/arm-linux-gnueabihf/libm.so.6
As long as you cross-compile, you the real path of symlink source is different.
So you have two options:
Compile with -static flag.
Fix those symlinks to use relative paths. But beware, there are plenty of them.
Thanks to this post, it helped me to figure out this problem.
I'm new to programming c++ with g++ and have big problems to get wsdlpull-library to work. The version is 1.24.
I followed the installation steps from http://wsdlpull.sourceforge.net/.:
./configure --prefix=/home/jesse/Dropbox/Programmering/C++/test --disable-opt --enable-examples
make
make install
I then copied print.cpp file from the example to the "prefix"-directory. The next step would be "add the $(prefix)/include in your include path and add $(prefix)/libs and -lwsdl -lschema -lxmlpull to your LDFLAGS". The first problem is that there's no folder "libs" but "lib" and the include-folder has another folder inside named wsdlpull. Therefore I instead entered the following command:
g++ -I include/wsdlpull -L lib -lwsdl -lschema -lxmlpull -o print print.cpp
This resulted in the following error:
/usr/bin/ld: cannot find -lwsdl
/usr/bin/ld: cannot find -lschema
/usr/bin/ld: cannot find -lxmlpull
collect2: error: ld returned 1 exit status
I have no idea what this means, why does it look in /usr/bin/ld when I point to "lib"?
When you use -L lib, it means that, lib dir is available in pwd where you are doing make. Check whether you have said lib is really available or not.
To make things simple, you can give the absolute path of the lib folder.
i.e -L<path to lib>/lib. This would help the make to see the lib without any confusions.
Next, you might want to explicitly specify whether libwsdl is a static or shared library. Although this is optional.
Or, the best solution is to place the libraries in system libs location i.e /lib or /usr/lib or /use/local/lib and run ldconfig such that the linker/loader knows where the newly added libraries exists. In such case, you can simple issue -lwsdl without specifying the -L flag.
For more info, please refer ldconfig.
I have downloaded some C++ which I want to compile from source. After running cmake ../src and then make from the command line, whilst in the build directory, I get the following error:
/usr/bin/ld: /usr/local/lib/libBulletCollision.a(btDbvtBroadphase.o):
relocation R_X86_64_32S against `_ZTV16btDbvtBroadphase' can not be used when making a shared object;
recompile with -fPIC
I believe that this is something to do with trying to create a shared library, but not providing the necessary memory to do so (although I don't fully understand this...). Anyway, following the advice I've seen around, I tried the following instead:
make CXXFLAGS='-fPIC'
However, this gives me exactly the same error as before. Any ideas as to why this did not solve the problem?
I believe Mark is correct when he says the error is about libBulletCollision.a. You are building a .so built with -fPIC but linking against a .a that did not use -fPIC. You will either need to change and build a static library or rebuild and install a new dependent libBulletCollision.a using -fPIC.
By the way, you should add the -fPIC flag to your build in a more permanent way in the cmake configure step rather in this transient way overriding CXXFLAGS when running make.
Also using make VERBOSE=1 with cmake generated makefile builds is quite helpful when debugging builds.
I was trying to reinstall my ffmpeg, following this guide, on my ARM Ubuntu machine. Unfortunately, when I compile a program which uses this lib I get the following failure:
/usr/bin/ld: /usr/local/lib/libavcodec.a(amrnbdec.o): relocation R_ARM_MOVW_ABS_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libavcodec.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
Now I would like to recompile it with -fPIC like the compiler is suggesting but I have no idea how. Any help is appreciated.
Briefly, the error means that you can't use a static library to be linked w/ a dynamic one.
The correct way is to have a libavcodec compiled into a .so instead of .a, so the other .so library you are trying to build will link well.
The shortest way to do so is to add --enable-shared at ./configure options. Or even you may try to disable shared (or static) libraries at all... you choose what is suitable for you!
Have a look at this page.
you can try globally adding the flag using: export CXXFLAGS="$CXXFLAGS -fPIC"
I had this problem when building FFMPEG static libraries (e.g. libavcodec.a) for Android x86_64 target platform (using Android NDK clang). When statically linking with my library the problem occured although all FFMPEG C -> object files (*.o) were compiled with -fPIC compile option:
x86_64/libavcodec.a(h264_qpel_10bit.o):
requires dynamic R_X86_64_PC32 reloc against 'ff_pw_1023'
which may overflow at runtime; recompile with -fPIC
The problem occured only for libavcodec.a and libswscale.a.
Source of this problem is that FFMPEG has assembler optimizations for x86* platforms e.g. the reported problem cause is in libavcodec/h264_qpel_10bit.asm -> h264_qpel_10bit.o.
When producing X86-64 bit static library (e.g. libavcodec.a) it looks like assembler files (e.g. libavcodec/h264_qpel_10bit.asm) uses some x86 (32bit) assembler commands which are incompatible when statically linking with x86-64 bit target library since they don't support required relocation type.
Possible solutions:
compile all ffmpeg files with no assembler optimizations (for ffmpeg this is configure option: --disable-asm)
produce dynamic libraries (e.g. libavcodec.so) and link them in your final library dynamically
I chose 1) and it solved the problem.
Reference: https://tecnocode.co.uk/2014/10/01/dynamic-relocs-runtime-overflows-and-fpic/
After the configure step you probably have a makefile. Inside this makefile look for CFLAGS (or similar). puf -fPIC at the end and run make again. In other words -fPIC is a compiler option that has to be passed to the compiler somewhere.
If you're building a shared library but need to link with static libavcodec add linker flags:
-Wl,-Bsymbolic
In case of cmake:
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,-Bsymbolic")
I hit this same issue trying to install Dashcast on Centos 7. The fix was adding -fPIC at the end of each of the CFLAGS in the x264 Makefile. Then I had to run make distclean for both x264 and ffmpeg and rebuild.
In addirion to the good answers here, specifically Robert Lujo's.
I want to say in my case I've been deliberately trying to statically compile a version of ffmpeg. All the required dependencies and what else heretofore required, I've done static compilation.
When I ran ./configure for the ffmpeg process I didnt notice --enable-shared was on the commandline. Removing it and running ./configure is only then I was able to compile correctly (All 56 mbs of an ffmpeg binary). Check that out as well if your intention is static compilation
I'm building ffmpeg 5.1.2 on CentOS7 with gcc4.8.5.
As mentioned in ${ffmpegRoot}/doc/platform.texi:
1)configure with option
"--enable-pic"
2)add the following option to your project LDFLAGS
"-Wl,-Bsymbolic"
Before compiling make sure that "rules.mk" file is included properly in Makefile or include it explicitly by:
"source rules.mk"
I'm trying to create a shared object (.so) that will make it so, by including one shared object with -lboost, I implicitly include all the boost libraries. Here's what I tried:
#!/bin/sh
BOOST_LIBS="-lboost_date_time-gcc43-mt -lboost_filesystem-gcc43-mt"
#truncated for brevity
g++ $BOOST_LIBS -shared -Wl,-soname,libboost.so.1 -o libboost.so.1.0
ln -si libboost.so.1.0 libboost.so.1
ln -si libboost.so.1 libboost.so
After placing all 3 created files (libboost.so libboost.so.1 libboost.so.1.0) in the same directory as all the boost libraries, I tried compiling a test program with it (which depends on -lboost_date_time-gcc43-mt):
g++ -lboost test.cpp
Doing this, I got the same undefined reference message as not having -lboost. Having -lboost_date_time-gcc43-mt works, but that's too wordy :) How do I get -lboost to automatically bring in the other shared libraries?
You don't. Not really, anyway.
The linker is stripping out all of the symbol dependencies because the .so doesn't use them.
You can get around this, perhaps, by writing a linker script that declares all of the symbols you need as EXTERN() dependencies. But this implies that you'll need to list all of the mangled names for the symbols you need. Not at all worth the effort, IMO.
I don't have a solution for creating a dummy '.so', but I do have something that will simplify your life... I highly suggest that you try using cross-platform make (CMake). In CMake, linking against those libraries is easy:
FIND_PACKAGE(Boost 1.37 COMPONENTS date_time filesystem REQUIRED)
ADD_EXECUTABLE(myexecutable ${myexecutable_SRCS})
TARGET_LINK_LIBRARIES(myexecutable ${Boost_LIBRARIES})
The commands above, if placed in a "CMakeLists.txt" file, is all you need to:
Verify that Boost 1.37 or later is installed, with the "date_time" and "filesystem" libraries installed.
Create an executable named "myexecutable" from the sources listed in the corresponding variable.
Link the executable "myexecutable" against the boost "date_time" and "filesystem" libraries.
See also: Why the KDE project switched to CMake.
Actually, making one .so depend on all boost .so files is quite possible (but might not actually help you). I've just tried this:
$ export BOOST_ROOT=/home/ghost/Work/Boost/boost-svn
$ g++ -shared -Wl,-soname,libboost.so -o libboost.so $BOOST_ROOT/stage/lib/libboost_program_options.so
$ g++ -L . -I $BOOST_ROOT first.cpp -lboost -Wl,-R$BOOST_ROOT/stage/lib
$ LD_LIBRARY_PATH=.:$BOOST_ROOT/stage/lib ./a.out
And it did work. However, note that dancing with -R and LD_LIBRARY_PATH. I don't know an way how you can include the path to Boost .so inside your libboost.so so that they are used both for linking and actually running the application. I can include rpath inside libboost.so just fine, but it's ignored when resolving symbols for the application.