I am trying to create a shared library (really a Python module) that links against a static library. Both libraries are part of the same project and built using cmake.
Now, the shared library is built like this:
add_library(MyLibPython SHARED ${PYTHON_WRAPPERS_SRC})
set_target_properties(MyLibPython PROPERTIES PREFIX "")
target_link_libraries(MyLibPython MyLibStatic ${LIBS})
This builds without error, but when I try to import the Python module, I get:
ImportError:
lib/python/MyLibPython.so: undefined symbol: _Zone_of_my_MyLibStatic_functions
I also have a number of executables (unit tests) that are built in a similar way, and they work perfectly.
I should add, this is using gcc on Linux.
Check your linker command line. Is it passing something like -Wl,--as-needed? If so, it might not be including everything required by the static library.
I don't think your technique is portable in general. Can you get a shared library to link against? I think that there are some platforms where everything that goes into a shared library needs to be compiled as PIC.
Anyway, to link an entire archive with GNU ld (look up man ld):
gcc -o foo foo.o bar.o baz.o -Wl,--whole-archive libfoo.a -Wl,--no-whole-archive [rest-of-linker-args]
Related
I've got a project that used to be a giant set of source files that all got compiled and then linked as one executable. As a first step in making the project more modular, I am breaking up the build into several smaller chunks and making them static libraries. There's a hierarchy, so Exe1 will link against static libs Lib2A and Lib2B. Lib2A will depend on static Lib3A, lib3B, lib3C, etc. The numbers here show their layer in the hierarchy.
The problem is that I need to use --whole-archive when linking or else some symbols from the underlying libraries are not found.
When I add the below for the linking of Exe1:
target_link_libraries(Exe1 -Wl,--whole-archive Lib2A Lib2B -Wl,--no-whole-archive)
I end up with an actual link stage command like:
g++ -o Exe1 -Wl,--whole-archive libLib2A.a libLib2B.a -Wl,--no-whole-archive libLib3A.a libLib3B.a libLib3C.a
Inevitably, symbols from some of the layer 3 static libraries get lost and I get missing symbol errors.
I expected that because Lib2A has Lib3* libraries as dependencies, that they would also be "inside" the --whole-archive part of the linker command, but they show up outside.
I've tried many different combinations (e.g. putting the --whole-archive stuff at lower layers), but haven't come across an approach that works using CMake. What am I doing wrong?
Thanks
For 3.12 and newer versions of CMake, I would use object libraries.
The workaround I found for versions earlier than that was to create an intermediate static library that used some property magic to place all linkage dependencies inside the --whole-archive section. For me, the top-level static library was called 'source'. It contained actually nothing itself, but had linkage dependencies on a bunch of other static libraries. I created 'source-combined' as follows:
add_library(source-combined STATIC "")
set_target_properties(source-combined PROPERTIES LINKER_LANGUAGE CXX)
target_link_libraries(source-combined PUBLIC
-Wl,--whole-archive
$<TARGET_PROPERTY:source,INTERFACE_LINK_LIBRARIES>
-Wl,--no-whole-archive
)
Now when I create an executable or a shared library by linking against this souce-combined library, I get the --whole-archive and --no-whole-archive as bookends around the entire set of static libraries that were the link dependencies of 'source'. It took forever to stumble across this technique, so I'm sharing it.
The following worked for me. Consider two libraries:
my_platform
my_clib
We want the whole archive of my_clib, and my_platform links to it.
add_library(my_platform INTERFACE) # this could also be a regular library
add_library(my_clib STATIC)
target_sources(my_clib
PRIVATE
gcc_newlib_nano.c
gcc_newlib_nano_cpp.cc
)
# Link my_clib and any other libs
target_link_libraries(my_platform
INTERFACE
my_clib
)
# Ensure the whole archive is linked
target_link_options(my_platform
INTERFACE
-Wl,--whole-archive ${CMAKE_CURRENT_BINARY_DIR}/libmy_clib.a -Wl,--no-whole-archive
)
As an alternative to the above answer, I needed to get something quick and dirty to see if the effort to add whole archive target flags (or convert the code base to object libraries...) was the right solution. By following the CMake Source Code for the default link command, I modified my project's command to:
set(CMAKE_CXX_LINK_EXECUTABLE "<CMAKE_CXX_COMPILER> <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> -Wl,--start-group -Wl,--whole-archive <LINK_LIBRARIES> -Wl,--no-whole-archive -Wl,--end-group")
It worked a treat! While not the greatest solution, it will at least get some results quickly.
If you need to use the linker option --whole-archive, then you definably should use object libraries:
# Lib2A/CMakeLists.txt
add_library(Lib2A OBJECT ${Lib2A_SRC})
# Lib2B/CMakeLists.txt
add_library(Lib2B OBJECT ${Lib2B_SRC})
It is portable and does not require use the linker option --whole-archive.
I make a program using a dynamic library, libexample.so. The dynamic library depends on another dynamic library, libtool.so.
It looks linker succeeded linking the libexample.so because of message from gcc.
Building target: libexample.so
Invoking: GCC C++ Linker
g++ -L/home/takehiro/Documents/documents/code/lib/tool -shared -o "libexample.so" ./classes/example.o ./classes/example_template.o ./classes/example_test.o ./classes/impl.o -ltool
Finished building target: libexample.so
cp libexample.so /home/takehiro/Documents/documents/code/lib/example
However, it failed to link it with the libtool.so.
ldd /home/takehiro/Documents/documents/code/lib/example/libexample.so
...
libtool.so => not found
...
I checked existence of the libtool.so under /home/takehiro/Documents/documents/code/lib/tool which is pointed by -L optoin at above linker by
ls /home/takehiro/Documents/documents/code/lib/tool
libtool.so
This is the first time to use a dynamic library depending another dynamic library. So I am so confused. Is it normal or malfunction? Why it cannot link them?
Does someone have a suggestion or a solution to me? I am very glad it.
Thank you very much.
All that the -L option does is tell the linker where the shared library is, at link time.
This does not have any effect on where the runtime loader searches for shared libraries. That's why the shared library fails to be loaded at runtime.
You also need to pass the -rpath option to the linker, when you're linking your shared library, in order to set the RPATH attribute on the shared library, that specifies where its dependencies are to be searched for. Something like
g++ -L/home/takehiro/Documents/documents/code/lib/tool \
-Wl,-rpath=/home/takehiro/Documents/documents/code/lib/tool \
... remaining options
I want to start with a simple linking usage to explain my problem. Lets assume that there is a library z which could be compiled to shared library libz.dll(D:/libs/z/shared/libz.dll) or to static library libz.a (D:/libs/z/static/libz.a).
Let I want to link against it, then I do this:
gcc -o main.exe main.o -LD:/libs/z/static -lz
According to this documentation, gcc would search for libz.a, which is
archive files whose members are object files
I also can do the following:
gcc -o main.exe main.o -LD:/libs/z/shared -lz
It is not mentioned in the documentation above that -l flag will search for lib<name>.so.
What will happen if I libz.a and libz.dll will be in the same directory? How the library will be linked with a program? Why I need the flags -Wl,-Bstatic and -Wl,-Bdynamic if -l searches both for shared and static libraries?
Why some developers provide .a files with .dll files for the same modules, if I compile a shared library distribution?
For example, Qt provides .dll files in bin directory with .a files in lib directory. Is it the same library, but built like shared and static, respectively? Or .a files are some kind of dummy libraries which provide linking with shared libraries, where there are real library implementations?
Another example is OpenGL library on Windows. Why every compiler must provide the static OpenGL lib like libopengl32.a in MingW?
What are files with .dll.a and .la extensions used for?
P.S. There are a lot of questions here, but I think each one depends on the previous one and there is no need to split them into several questions.
Please, have a look at ld and WIN32 (cygwin/mingw). Especially, the direct linking to a dll section for more information on the behavior of -l flag on Windows ports of LD. Extract:
For instance, when ld is called with the argument -lxxx it will attempt to find, in the first directory of its search path,
libxxx.dll.a
xxx.dll.a
libxxx.a
cygxxx.dll (*)
libxxx.dll
xxx.dll
before moving on to the next directory in the search path.
(*) Actually, this is not cygxxx.dll but in fact is <prefix>xxx.dll, where <prefix> is set by the ld option -dll-search-prefix=<prefix>. In the case of cygwin, the standard gcc spec file includes -dll-search-prefix=cyg, so in effect we actually search for cygxxx.dll.
NOTE: If you have ever built Boost with MinGW, you probably recall that the naming of Boost libraries exactly obeys the pattern described in the link above.
In the past there were issues in MinGW with direct linking to *.dll, so it was advised to create a static library lib*.a with exported symbols from *.dll and link against it instead. The link to this MinGW wiki page is now dead, so I assume that it should be fine to link directly against *.dll now. Furthermore, I did it myself several times with the latest MinGW-w64 distribution, and had no issues, yet.
You need link flags -Wl,-Bstatic and -Wl,-Bdynamic because sometimes you want to force static linking, for example, when the dynamic library with the same name is also present in a search path:
gcc object1.o object2.o -lMyLib2 -Wl,-Bstatic -lMyLib1 -Wl,-Bdynamic -o output
The above snippet guarantees that the default linking priority of -l flag is overridden for MyLib1, i.e. even if MyLib1.dll is present in the search path, LD will choose libMyLib1.a to link against. Notice that for MyLib2 LD will again prefer the dynamic version.
NOTE: If MyLib2 depends on MyLib1, then MyLib1 is dynamically linked too, regardless of -Wl,-Bstatic (i.e. it is ignored in this case). To prevent this you would have to link MyLib2 statically too.
I have a linux based OS with a lot of system libraries compiled as static libraries.
How can I use such libraries in my application, and link them to my final binary?
You use them as you do use shared libraries, except that you link against statically. An introduction to GCC - shared libraries and static libraries article will get you started.
I've trouble to understand. If you are linking with something like
g++ -o myprog myprog.o obj1.o obj2.o -L/path/to/lib -L/path2/to/lib -llib1 -llib2 -lib3
the linker called through the gcc or g++ wrapper will do "the right thing(tm)", if liblib1.so exist in the library path (/path/to/lib, /path2/to/lib plus a set of system specific directories where system libraries probably are), it will be linked dynamically, if not liblib1.a will be linked statically. The only thing to be aware of, is that if there are mutual dependencies in static libaries (lib1 needs lib2 and lib2 needs lib1), you may need to repeat them several times or use the --start-group and --end-group options of ld to mark libraries which needs to be considered together.
I use log4cxx logging library. I need to link with its static version to avoid additional binary dependencies. I use it in my dynamic library. Default build of log4cxx produces static library but I cannot link with it because it was compiled w/o -fPIC flag. So I changed log4cxx bulding as:
CPPFLAGS="-fPIC -static" ./configure
make
As a result I received a liblog4cxx.a that I can link with my .so library. Linking was done by Cmake, something like:
target_link_libraries(my_dynamic_lib log4cxx)
link_directories(relative_path_to_dir_where_liblog4cxx.a_lives)
Everything looked fine until runtime. I cannot load my_dynamic_lib.so because of undefined symbol "logger"
Please explain me what's wrong and how to resolve this problem.
thanks
You can verify whether the shared library contains the symbol by using
nm -g my_dynamic_lib.so | grep logger
If it is shown with symbol type U it means it's undefined.
Normally a shared library will not resolve all the symbols it needs until run-time, so it is possible (and perfectly normal) to link a shared library with missing symbols.
If you put -llog4cxx at the start of the linker command line for my_dynamic_lib.so then it won't link to any of the code in there, and will leave the logger symbol unresolved until run-time. To force it to use the symbols in the static library make sure you list the static library after the objects that need it:
g++ -fPIC -shared -o my_dynamic_lib.so obj1.o obj2.o -llog4cxx ...
I don't know how to do that with cmake, but it looks as though your CMakefile only links to log4cxx when linking the main executable, not the dynamic library.
Usually you would link liblog4cxx.a with your executable and not with your my_dynamic_lib.so. I don't think you can link like in your example unless you can provide documentation that says otherwise.