I know this question has been asked a few times, but none of the solutions have worked for me. I have a statically linked library that I would like to use with a JNI layer with versions pre Java 8. Based on my reading of "how to link static library into dynamic library in gcc", it seems possible. Here is my command line:
/usr/bin/g++ -shared -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -DNDEBUG -fPIC -Wl,--whole-archive target/vw_jni.a -o target/vw_jni.lib
This is based on writing a JNI layer to the Vowpal Wabbit library.
At this point in the build process I have statically created a file called target/vw_jni.a through static linking
target/vw_jni.a: In function `_fini':
(.fini+0x0): multiple definition of `_fini'
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crti.o:(.fini+0x0): first defined here
target/vw_jni.a: In function `data_start':
(.data+0x8): multiple definition of `__dso_handle'
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtbeginS.o:(.data.rel.local+0x0): first defined here
target/vw_jni.a: In function `_init':
(.init+0x0): multiple definition of `_init'
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crti.o:(.init+0x0): first defined here
/usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS): In function `__libc_csu_init':
(.text+0x0): multiple definition of `__libc_csu_init'
target/vw_jni.a:(.text+0x1cea20): first defined here
/usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS): In function `__libc_csu_fini':
(.text+0x70): multiple definition of `__libc_csu_fini'
target/vw_jni.a:(.text+0x1ceab0): first defined here
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtendS.o:(.tm_clone_table+0x0): multiple definition of `__TMC_END__'
target/vw_jni.a:(.data+0x2630): first defined here
/usr/bin/ld: target/vw_jni.a: .preinit_array section is not allowed in DSO
/usr/bin/ld: failed to set dynamic section sizes: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
I'm not sure what this means and when I search around for it find results such as "C program no longer compiles in Ubuntu" that seem to suggest that I forgot a -o flag, but I know I have not.
Is what I'm trying to do possible?
What am I doing wrong?
I'm doing this on a Docker instance obtained through docker pull ubuntu:14.04
UPDATE:
I'm able to get rid of a few of the errors with the following command line
/usr/bin/g++ -shared -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -DNDEBUG -fPIC -nostdlib -Wl,--whole-archive target/vw_jni.a -o target/vw_jni.lib
This generates the following output
/usr/bin/ld: warning: Cannot create .note.gnu.build-id section, --build-id ignored.
/usr/bin/ld: target/vw_jni.a: .preinit_array section is not allowed in DSO
/usr/bin/ld: failed to set dynamic section sizes: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
The reason I think this works is that by excluding the standard lib I don't have the redefinitions. I'm not sure where to go from here though.
I think the problem here is with the command line you're using. The flags --whole-archive and --no-whole-archive apply to all subsequent arguments on the command line, which includes a number of standard libraries and object files (e.g., -lc, crt0.o, libc_nonshared.a, etc.) that are automatically appended behind the scenes.
If there's no --no-whole-archive switch directly after the filenames of static libraries you intend to bring in, this will also apply --whole-archive to the built-in archives that come after your last argument, e.g., it will try to also bring in every object from libc.a, libm.a, etc. which will at the very least fail with a "multiple definition" error. Try adding --no-whole-archive switch to your command line directly after target/vw_jni.a so you end up with something like:
/usr/bin/g++ -shared -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -O3 \
-fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -DNDEBUG -fPIC \
-Wl,--whole-archive target/vw_jni.a -Wl,--no-whole-archive \
-o target/vw_jni.lib
This is practically not wise, because if you put the object files extracted from some static library these object files still remain position dependent code.
Shared libraries should very preferably contain position independent code (because the dynamic linker ld-linux.so is mmap-ing segments inside the .so at some random address, e.g. because of ASLR), otherwise the dynamic linker would have to process a big lot of relocations (so dynamic linking becomes very inefficient).
So even if you succeed to convert your static library into a shared one, it won't be wise to do so.
So keep your static library as is, or recompile their source code with -fPIC to build a shared object.
Related
I'm having difficulty linking my STM32 project using CMake. The link command that is generated is:
/Users/jeremy/gcc-arm-none-eabi-6-2017-q2-update/bin/arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=hard -fdata-sections -ffunction-sections -g -Og -gdwarf-2 -MMD -MP -std=c++11 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -specs=nano.specs -T/Users/jeremy/stm32l432kc_freertos_template/STM32L432KCUx_FLASH.ld -Wl,-Map=target.map,--cref -Wl,--gc-sections
< ... lots of .o files here ...>
-o stm32l432kc_freertos -lc -lm -lnosys
Unfortunately I get two sets of errors. The first is:
arm-none-eabi/bin/ld: warning: cannot find entry symbol arch_paths_first; defaulting to 0000000008000190
which indicates that there is no entry symbol, but in the LD file the first line of code is: ENTRY(Reset_Handler). The symbol Reset_Handler is defined in a linked file startup_stm32l432xx.s.
The second set of errors relate to the stdlib:
/Users/jeremy/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/lib/thumb/v7e-m/fpv4-sp/hard/libg_nano.a(lib_a-signalr.o): In function `_kill_r':
signalr.c:(.text._kill_r+0xe): undefined reference to `_kill'
/Users/jeremy/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/lib/thumb/v7e-m/fpv4-sp/hard/libg_nano.a(lib_a-signalr.o): In function `_getpid_r':
signalr.c:(.text._getpid_r+0x0): undefined reference to `_getpid'
Which are supposed to be solved by linking -lnosys, but the linker appears to be ignoring that.
Essentially, the linker appears to be ignoring some of the directives in the LD file and ignoring some of the flags I have passed. I realise it is probably something I am doing wrong, but I can't see what it is.
If I add -specs=nosys.specs the latter two errors go away, but this shouldn't be necessary? Can someone please help me understand what is going wrong here?
Looks like CMake on OSX is adding the -Wl,-search_paths_first which is unsupported for the gcc arm toolchain. Fix is to add this to the CMakelists.txt file:
if ( APPLE )
string ( REPLACE "-Wl,-search_paths_first" "" CMAKE_C_LINK_FLAGS ${CMAKE_C_LINK_FLAGS} )
string ( REPLACE "-Wl,-search_paths_first" "" CMAKE_CXX_LINK_FLAGS ${CMAKE_CXX_LINK_FLAGS} )
endif ()
Fix is taken from here: https://github.com/kiibohd/controller/blob/master/Lib/CMake/build.cmake
Still have no idea about the -lnosys being ignored though.
I'm trying to compile my project and I use the lib ncurse. And I've got some errors when compiler links files.
Here is my flags line in Makefile:
-W -Wall -Werror -Wextra -lncurses
I've included ncurses.h
Some layouts :
prompt$> dpkg -S curses.h
libslang2-dev:amd64: /usr/include/slcurses.h
libncurses5-dev: /usr/include/ncurses.h
libncurses5-dev: /usr/include/curses.h
prompt$> dpkg -L libncurses5-dev | grep .so
/usr/lib/x86_64-linux-gnu/libncurses.so
/usr/lib/x86_64-linux-gnu/libcurses.so
/usr/lib/x86_64-linux-gnu/libmenu.so
/usr/lib/x86_64-linux-gnu/libform.so
/usr/lib/x86_64-linux-gnu/libpanel.s
And here are my erros :
gcc -W -Wall -Werror -Wextra -I./Includes/. -lncurses -o Sources/NCurses/ncurses_init.o -c Sources/NCurses/ncurses_init.c
./Sources/NCurses/ncurses_init.o: In function `ncruses_destroy':
ncurses_init.c:(.text+0x5): undefined reference to `endwin'
./Sources/NCurses/ncurses_init.o: In function `ncurses_write_line':
ncurses_init.c:(.text+0xc5): undefined reference to `mvwprintw'
./Sources/NCurses/ncurses_init.o: In function `ncurses_init':
ncurses_init.c:(.text+0xee): undefined reference to `initscr'
collect2: error: ld returned 1 exit status
Thanks a lot
You need to change your makefile so that the -lncurses directive comes after your object code on the gcc command line, i.e. it needs to generate the command:
gcc -W -Wall -Werror -Wextra -I./Includes/. -o Sources/NCurses/ncurses_init.o -c Sources/NCurses/ncurses_init.c -lncurses
This is because object files and libraries are linked in order in a single pass.
In C++ , I fixed it just by linking the ncurses library .
Here is the command :
g++ main.cpp -lncurses
I got flags to correct order by using LDLIBS variable:
ifndef PKG_CONFIG
PKG_CONFIG=pkg-config
endif
CFLAGS+=-std=c99 -pedantic -Wall
LDLIBS=$(shell $(PKG_CONFIG) --libs ncurses)
man gcc | grep -A10 "\-l library"
-l library
Search the library named library when linking. (The second alternative with the library as a separate argument is only for POSIX
compliance and is not recommended.)
It makes a difference where in the command you write this option; the linker searches and processes libraries and object files
in the order they are specified. Thus, foo.o -lz bar.o searches
library z after file foo.o but
before bar.o. If bar.o refers to functions in z, those functions may not be loaded.
I'm trying to build some example c++ code that use boost library. I use this as reference example of static linking.
And everything is fine when I build with dynamic libs.
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/main.o src/main.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/ThreadExample.o src/ThreadExample.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/Utils.o src/Utils.cpp
g++ src/main.o src/ThreadExample.o src/Utils.o -lboost_thread -lboost_filesystem -lboost_system -lboost_timer -o ThreadExampleBinary
But when I use static libs I get lots of undefined reference errors:
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/main.o src/main.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/ThreadExample.o src/ThreadExample.cpp
g++ -Wall -std=c++0x -O3 -Wfatal-errors -I/usr/include/boost/include -c -o src/Utils.o src/Utils.cpp
g++ -static src/main.o src/ThreadExample.o src/Utils.o -lboost_thread -lboost_filesystem -lboost_system -lboost_timer -o ThreadExampleBinary
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::start()':
(.text+0x7fd): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::stop()':
(.text+0x94c): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::elapsed() const':
(.text+0xa59): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::cpu_timer::resume()':
(.text+0xb60): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o): In function `boost::timer::auto_cpu_timer::auto_cpu_timer(std::ostream&, short)':
(.text+0xca5): undefined reference to `boost::chrono::steady_clock::now()'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libboost_timer.a(cpu_timer.o):(.text+0xd4e): more undefined references to `boost::chrono::steady_clock::now()' follow
collect2: error: ld returned 1 exit status
make: *** [ThreadExampleBinary] Error 1
Seems this can be fixed adding additional -lboost_chrono library.
But why it works in dinamic setting?
With static linking, you have to also statically link to any libraries depended on by the libraries you are linking to.
The difference is that shared libraries have an entry in the ELF header, called NEEDED that lists other shared libraries that are to be included when you link in this one.
You can see them with this command:
$ objdump -p /usr/lib/libboost_timer.so | grep NEEDED
NEEDED libboost_chrono.so.1.60.0
NEEDED libboost_system.so.1.60.0
NEEDED librt.so.1
NEEDED libstdc++.so.6
NEEDED libgcc_s.so.1
NEEDED libc.so.6
But for static libraries there is no such system, as they are simply collection of object files.
It is worth noting that the NEEDED entry in the shared objects are entirely optional, and if they are not available, then they will behave exactly like the static ones. But most shared libraries include them.
Many libraries use the pkg-config infrastructure to provide the full command line needed, but AFAIK boost is not one of them.
How to automate this process? Well, you do not. You just include what is needed and follow the linker errors to discover further needs.
You can find which static library includes a symbol with something like:
$ nm --print-file-name --defined-only --demangle /usr/lib/*.a 2> /dev/null | \
grep -q 'boost::chrono::steady_clock::now()'
/usr/lib/libboost_chrono.a:chrono.o:0000000000000090 T boost::chrono::steady_clock::now()
But why it works in dinamic setting?
Most of my make files have the following notes (from when I found the answer back when I needed it ... sorry, I don't know where I found it.)
Note - when a build using '-l' finds the .so version of that library (so - shared object) and the same .a archive is also there, g++ prefers the .so over the .a.
However, you can still achieve static link by fully specifying the path to .a.
Example:
$(CC) $(CC_FLAGS) $< /usr/local/lib/libboost_chrono.a -o $# ...
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sometimes, archive code refers to symbols in yet another archive.
i.e. -lyyy_i686 uses some function from -lxxx_i686, and xxx was listed first in the build command. It is possible that a symbol might remain unresolved and the linker fails.
When this happens, try adding xxx a second time to the end of the archive list
from: -lxxx_i686 -lyyy_i686 -lrt -pthread
becomes -lxxx_i686 -lyyy_i686 -lrt -pthread -lxxx_i686
^^^^^^^^^_____________________________^^^^^^^^^^
The preceeding presumes only .a libraries are findable (no xxx.so)
Another alternative, you can command the linker to search an archive multiple times
-lyyy_i686 -lrt -pthread -(-lxxx_i686-)
tell gcc to link this library as many times as needed __^^__________^^
Again, only .a is findable
Finally, if you choose to link with the .so, note that this does not pull in the entire lib into the current build target. Instead, at runtime, the .so is loaded into memory, if it is not already there. This happens after the program starts. For most applications, this is considered a 'small' start-up performance hit as the program adjusts its memory map (automagically behind the scenes). I believe I once found that the app itself can control when to load the .so.
The use of .so pulls in the entire library to memory (if not already installed). Thus, there would not be any missing symbols, in my xxx vs yyy scenario above, the .so pulls all the symbols, whether used or not.
And once again, when trying to load xxx using "-lxxx_i686", the linker prefers to pull in libxxx_i686.so even when the libxxx_i686.a is in the same directory.
I have study makefile projec. I am using the gnu gcc and as linker, I use the gcc with the following argument:
linkerCmdLine=$(exe) -Wl,--start-group -nostdlib -Map=test_mapfile.map -l:libc.a -l:libgcc.a -l:libm.a -Wl,--end-group -o $(output)
During the linking process, the linker does not use all of this flags:
The flag -Map=test_mapfile.map is not used.
Why??
There is a project settings file and in this one is also stored the linker settings.
\u201ClinkerCmdLine\=$(exe) -Wl, --start-group -nostdlib -Map\=./99_output/PUZ_GPF.map --gc-sections -l\:libc.a -l\:libgcc.a -l\:libm.a -Wl,--end-group -o $(output)\u201D
How must I store the settings correct?
Edited:
With the the map file works fine, but I have actually problem with the linker file. I use only the following flag:
"linkerCmdLine=$(exe) -Wl, --script=C:/workspace/test_project/test_linker_file.ld -static -o $(output)”
I get this error in the console window:
arm-none-eabi-gcc.exe: error: unrecognized command line option '--script=C:/workspace/test_project/test_linker_file.ld'
I don't understand why, because the map file and the .ld file belong to the linker flags.
The -Wl, prefix only passes the current argument, with commas separating linker arguments. So in order to pass -z defs, you would use -Wl,-z,defs.
This mechanism only works for true flags, i.e. when the position in the command line is irrelevant. The -l parameters are not flags, and the gcc compiler driver collects them into a separate list and passes them in a single block.
So this technique doesn't work for the exact commandline you want. On the other hand, the -nostdlib inhibits the -lc -lgcc -lm that would be the default.
Normally there is a provision in the gcc specs (the script for the compiler driver) to use --start-group and --end-group around the standard libraries when linking statically (which appears to be what you want to do).
So I think that
linkerCmdLine=$(exe) -Wl,-Map=test_mapfile.map -static -o $(output)
should work for your case.
I'm writing a small C program that uses librt. I'm quite surprised that the program won't compile if I place the link flag at the start instead of at the end:
At the moment, to compile the program I do:
gcc -o prog prog.c -lrt -std=gnu99
If I were to do the following, it will fail to find the functions in librt:
gcc -std=gnu99 -lrt -o prog prog.c
Yet, this works with other libraries. I found the issue when attempting to use a simple Makefile. make actually compiled prog.c without liking first (using -c flag) and then did the linking.
This is the Makefile:
CC = gcc
CFLAGS = -std=gnu99
LIBS= -lrt
LDFLAGS := -lrt
prog: prog.o
$(CC) -o prog prog.c -lrt -std=gnu99
The output I would get when typing make would be:
gcc -std=gnu99 -c -o prog.o prog.c
gcc -lrt prog.o -o prog
prog.o: In function `main':
prog.c:(.text+0xe6): undefined reference to `clock_gettime'
prog.c:(.text+0x2fc): undefined reference to `clock_gettime'
collect2: ld returned 1 exit status
make: *** [buff] Error 1
I have now crafted a Makefile that puts the linking at the end of the gcc line, however I'm puzzled why it doesn't work if the linking flag is at the start.
I would appreciate if anybody can explain this to me. Thanks.
As the linker processes each module (be it a library or a object file), it attempts to resolve each undefined symbol while potentially adding to its list of undefined symbols. When it gets to the end of the list of modules, it either has resolved all undefined symbols and is successful or it reports undefined symbols.
In your case, when it processed librt, it had no undefined symbols. Processing proc resulted in clock_gettime being an undefined symbol. gcc will not go back and look in librt for the undefined symbols.
For that reason, you should always have your code first, followed by your libraries, followed by platform provided libraries.
Hope this helps.
From the ld (the GNU linker) documentation (http://sourceware.org/binutils/docs/ld/Options.html#Options):
The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.
So if you specify the library too early, the linker will scan it, but not find anything of interest. Then the linker moves on to the object file produced by the compiler and finds references that need to be resolved, but it has already scanned the library and won't bother looking there again.