I am trying to use --whole-archive option, but it seems to not work with -Xlinker. How can I add this option by using -Xlinker ?
Here is the whole story:
I write a main program A, and several shared libraries B1,B2,etc. main program will use dlopen to load these shared libraries at runtime. I compile main program A linking with ALL the possible static libraries used by main program (S1,S2...etc) or shared libraries (S3,S4...etc). then I compile shared library without static library.
When calling dlopen, the program fail because of undefined symbol in shared library. this symbol does exist in static library S3, but I can not find it in symbol table of main program.
So I add --whole-archive option when compiling main program, but the symbol still not exists in final binary. how can I add this symbol to main program while it is not direct used in main program.
So I add --whole-archive option when compiling main program
You still have not answered the question: what command line you used. Adding linker options when compiling in generally pointless: they only matter when linking.
The correct link line for the situation you described is (assuming you are using GCC on a UNIX/ELF system):
gcc -rdynamic -o exename main.c \
-Wl,--whole-archive -lS1 -lS2 -Wl,--no-whole-archive
Explanation of important points:
The -rdynamic is needed so that symbols defined in the main
executable are visible to dlopened shared libraries.
Libraries should follow sources on the link line (order of libraries and sources matters).
Libraries which you want to link in entirely should be surrounded by --whole-archive and --no-whole-archive options.
Related
I have a static library which I do not have the source code, and need its symbols to be called dynamically from the LuaJIT FFI.
As it is static, I can't load it dynamically, so I'm trying to embed it in a shared library and then load the shared library at runtime.
Problem is that exported symbols of the static library are present in the symbols table of the shared lib, but are undefined.
nm libUSBDevices.a
shows a lot of lines, among which the symbols that interest me:
00001d80 T _ZN9USBDevice16FlightControllerC1EPKc
00001e30 T _ZN9USBDevice16FlightControllerD1Ev
00000140 T _ZN9USBDevice7AxisFctC1Ev
00000180 T _ZN9USBDevice7AxisFctclEv
Then I compiled the shared library using these two g++ commands :
g++ -m32 -c -Wall -Werror -fpic USBDevicesLoader.cpp -llibUSBDevices.a
which outputs USBDevicesLoader.o (USBDevicesLoader.cpp contains some exported functions which call symbols inside the static library, those ones are correctly present in the .so)
g++ -m32 -shared -o libUSBDevicesLoader.so USBDevicesLoader.o
This outputs the shared lib, but when loaded at runtime, it shows this:
[...] symbol lookup error: /home/me/USBDevices-loader/libUSBDevicesLoader.so: undefined symbol: _ZN9USBDevice16FlightControllerC1EPKc
And when I run nm on the shared lib, it shows the symbols as undefined:
U _ZN9USBDevice16FlightControllerC1EPKc
U _ZN9USBDevice7AxisFctclEv
I suggest the problem is somewhere in the compilation commands, I also tried to build the shared lib directly out of the .a without compiling the cpp first (just replace USBDevicesLoader.o in the second command by the .a, skip the first command) but the problem stays the same.
So, is there a way to embed all symbols of a static library (without having the source) in a dynamic lib which can then be loaded and used at runtime? Thanks
You can use the --whole-archive option to achieve this as seen here and in the docs:
--whole-archive: For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared library, forcing every object to be included in the resulting shared library. This option may be used more than once.
As an example:
g++ -shared -o libnew.so -Wl,--whole-archive libmylib_static.a -Wl,--no-whole-archive
You can then link to the shared libnew.so library as you would normally do.
I want to start with a simple linking usage to explain my problem. Lets assume that there is a library z which could be compiled to shared library libz.dll(D:/libs/z/shared/libz.dll) or to static library libz.a (D:/libs/z/static/libz.a).
Let I want to link against it, then I do this:
gcc -o main.exe main.o -LD:/libs/z/static -lz
According to this documentation, gcc would search for libz.a, which is
archive files whose members are object files
I also can do the following:
gcc -o main.exe main.o -LD:/libs/z/shared -lz
It is not mentioned in the documentation above that -l flag will search for lib<name>.so.
What will happen if I libz.a and libz.dll will be in the same directory? How the library will be linked with a program? Why I need the flags -Wl,-Bstatic and -Wl,-Bdynamic if -l searches both for shared and static libraries?
Why some developers provide .a files with .dll files for the same modules, if I compile a shared library distribution?
For example, Qt provides .dll files in bin directory with .a files in lib directory. Is it the same library, but built like shared and static, respectively? Or .a files are some kind of dummy libraries which provide linking with shared libraries, where there are real library implementations?
Another example is OpenGL library on Windows. Why every compiler must provide the static OpenGL lib like libopengl32.a in MingW?
What are files with .dll.a and .la extensions used for?
P.S. There are a lot of questions here, but I think each one depends on the previous one and there is no need to split them into several questions.
Please, have a look at ld and WIN32 (cygwin/mingw). Especially, the direct linking to a dll section for more information on the behavior of -l flag on Windows ports of LD. Extract:
For instance, when ld is called with the argument -lxxx it will attempt to find, in the first directory of its search path,
libxxx.dll.a
xxx.dll.a
libxxx.a
cygxxx.dll (*)
libxxx.dll
xxx.dll
before moving on to the next directory in the search path.
(*) Actually, this is not cygxxx.dll but in fact is <prefix>xxx.dll, where <prefix> is set by the ld option -dll-search-prefix=<prefix>. In the case of cygwin, the standard gcc spec file includes -dll-search-prefix=cyg, so in effect we actually search for cygxxx.dll.
NOTE: If you have ever built Boost with MinGW, you probably recall that the naming of Boost libraries exactly obeys the pattern described in the link above.
In the past there were issues in MinGW with direct linking to *.dll, so it was advised to create a static library lib*.a with exported symbols from *.dll and link against it instead. The link to this MinGW wiki page is now dead, so I assume that it should be fine to link directly against *.dll now. Furthermore, I did it myself several times with the latest MinGW-w64 distribution, and had no issues, yet.
You need link flags -Wl,-Bstatic and -Wl,-Bdynamic because sometimes you want to force static linking, for example, when the dynamic library with the same name is also present in a search path:
gcc object1.o object2.o -lMyLib2 -Wl,-Bstatic -lMyLib1 -Wl,-Bdynamic -o output
The above snippet guarantees that the default linking priority of -l flag is overridden for MyLib1, i.e. even if MyLib1.dll is present in the search path, LD will choose libMyLib1.a to link against. Notice that for MyLib2 LD will again prefer the dynamic version.
NOTE: If MyLib2 depends on MyLib1, then MyLib1 is dynamically linked too, regardless of -Wl,-Bstatic (i.e. it is ignored in this case). To prevent this you would have to link MyLib2 statically too.
I have a linux based OS with a lot of system libraries compiled as static libraries.
How can I use such libraries in my application, and link them to my final binary?
You use them as you do use shared libraries, except that you link against statically. An introduction to GCC - shared libraries and static libraries article will get you started.
I've trouble to understand. If you are linking with something like
g++ -o myprog myprog.o obj1.o obj2.o -L/path/to/lib -L/path2/to/lib -llib1 -llib2 -lib3
the linker called through the gcc or g++ wrapper will do "the right thing(tm)", if liblib1.so exist in the library path (/path/to/lib, /path2/to/lib plus a set of system specific directories where system libraries probably are), it will be linked dynamically, if not liblib1.a will be linked statically. The only thing to be aware of, is that if there are mutual dependencies in static libaries (lib1 needs lib2 and lib2 needs lib1), you may need to repeat them several times or use the --start-group and --end-group options of ld to mark libraries which needs to be considered together.
I use log4cxx logging library. I need to link with its static version to avoid additional binary dependencies. I use it in my dynamic library. Default build of log4cxx produces static library but I cannot link with it because it was compiled w/o -fPIC flag. So I changed log4cxx bulding as:
CPPFLAGS="-fPIC -static" ./configure
make
As a result I received a liblog4cxx.a that I can link with my .so library. Linking was done by Cmake, something like:
target_link_libraries(my_dynamic_lib log4cxx)
link_directories(relative_path_to_dir_where_liblog4cxx.a_lives)
Everything looked fine until runtime. I cannot load my_dynamic_lib.so because of undefined symbol "logger"
Please explain me what's wrong and how to resolve this problem.
thanks
You can verify whether the shared library contains the symbol by using
nm -g my_dynamic_lib.so | grep logger
If it is shown with symbol type U it means it's undefined.
Normally a shared library will not resolve all the symbols it needs until run-time, so it is possible (and perfectly normal) to link a shared library with missing symbols.
If you put -llog4cxx at the start of the linker command line for my_dynamic_lib.so then it won't link to any of the code in there, and will leave the logger symbol unresolved until run-time. To force it to use the symbols in the static library make sure you list the static library after the objects that need it:
g++ -fPIC -shared -o my_dynamic_lib.so obj1.o obj2.o -llog4cxx ...
I don't know how to do that with cmake, but it looks as though your CMakefile only links to log4cxx when linking the main executable, not the dynamic library.
Usually you would link liblog4cxx.a with your executable and not with your my_dynamic_lib.so. I don't think you can link like in your example unless you can provide documentation that says otherwise.
Say I had a library called libfoo which contained a class, a few static variables, possibly something with 'C' linkage, and a few other functions.
Now I have a main program which looks like this:
int main() {
return 5+5;
}
When I compile and link this, I link against libfoo.
Will this have any effect? Will my executable increase in size? If so, why? Do the static variables or their addresses get copied into my executable?
Apologies if there is a similar question to this or if I'm being particularly stupid in any way.
It won't do anything in a modern linker, because it knows the executable doesn't actually use libfoo's symbols. With gcc 4.4.1 and ld 2.20 on my system:
g++ linker_test.cpp -static -liberty -lm -lz -lXp -lXpm -o linker_test_unnecessary
g++ linker_test.cpp -static -o linker_test_none
ls -l linker_test_unnecessary linker_test_none
They are both 626094 bytes. Note this also applies to dynamic linking, though the size they both are is much lower.
A library contains previously compiled object code - basically a static library is an archive of .o or .obj files.
The linker looks at your object code and sees if there are any unresolved names and if so looks for these in the library, if it finds them it includes the object file that contains them and repeats this.
Thus only the parts of the static library that are needed are included in your executable.
Thus in your case nothing from libfoo will be added to you executable