In my use case, I have YAML-CPP, SQLite3, and my 'data.cpp' file that I want to all be combined into the same dynamic library, 'libdata.so'.
I first compiled yaml-cpp (as an archive):
mkdir -p "build"
cd "build"
cmake ..
make -j5
to get 'libyaml-cpp.a'.
I then compile sqlite3:
gcc -c -o libsqlite3.a sqlite3.c -lpthread -ldl
to get 'libsqlite3.a'. I know that this a C-based file, and there are differences between it and C++, but I've read that it shouldn't make too much difference here. I also know that I'm using -lpthread -ldl which is for dynamic loading, but I'm not sure how to get around it.
My question is: Can I compile my 'data.cpp' file with YAMP-CPP and SQLite3 such that they all exist in the same 'libdata.so' output file (where the linker will use the YAML-CPP and SQLite3 functions contained in 'libdata.so' when they're called by 'data.cpp')?
I have tried:
g++ -c -fPIC -o libdata.so \
-Wl,--whole-archive libsqlite3.a \
-Wl,--whole-archive libyaml-cpp.a \
-ldl -lpthread \
data.cpp
(for the sake of the snippet, all files reside in the same directory)
UPDATE
I added the suggestion from botje to the line and it helped in part. After more research, I found a few more pieces that progressed further:
gcc -DSQLITE_OMIT_LOAD_EXTENSION -c -fPIC -lpthread -o libsqlite3.a sqlite3.c
mkdir -p "build"
cd "build"
env CFLAGS='-fPIC' CXXFLAGS='-fPIC' cmake ..
make -j$(CORES)
cd ..
cp "build/libyaml-cpp.a" ./
g++ -shared -fPIC -o libdata.so \
-L./ \
-Wl,-Bdynamic data.cpp \
-Wl,-Bstatic -lsqlite3 -lyaml-cpp \
-Wl,-Bdynamic -lpthread
g++ -L./ -ldata -o tester tester.cpp
The library now compiles, however, when I try to link against it with 'tester.cpp', I get the error:
/usr/bin/ld: libdata.so: undefined reference to YAML::detail...
I'm guessing this may be a flag ordering issue, but I'm not sure what order it should be then. Placing the flags for SQLite3 and YAML-CPP before the data.cpp argument fails to compile the shared library.
After some more research, here's the method that worked for me (with extra verbosity):
# Compile SQLite3:
# - Disable the plugin loader (removes the libdl dependency)
# - Compile only (-c)
# - Use Position Independent Code (-fPIC)
# - Add the PThread library
# - After compilation, archive object (for completeness)
gcc -DSQLITE_OMIT_LOAD_EXTENSION -c -fPIC -pthread -o sqlite3.o sqlite3.c
# Compile YAML-CPP
# - Create (and enter) a build directory
# - Run CMAKE with -fPIC enabled
# - Run MAKE
# - Exit and copy archive from build directory
mkdir -p "build"
cd "build"
env CFLAGS='-fPIC' CXXFLAGS='-fPIC' cmake ..
make -j$(CORES)
cd ..
cp build/libyaml-cpp.a libyaml-cpp.a
# Compile Shared Library
# - Ensure shared (-shared) (also prevents looking for a 'main')
# - Use Position Independent Code (-fPIC)
# - Use current directory for locating libraries
# - Set target CPP file
# - STATICALLY link from SQLite3 and YAML-CPP archives
# - DYNAMICALLY link from PThread library (used by SQLite3 for thread-safe access)
g++ -shared -fPIC -o libdata.so data.cpp \
-L./ \
-Wl,-Bstatic -l:sqlite3.o -lyaml-cpp \
-Wl,-Bdynamic -pthread
# Compile Test Program
# - Specify current directory for includes and libraries
# - Link dynamically to 'libdata.so'
g++ -I./ -L./ -ldata -o tester tester.cpp
The last issue I encountered ended up being a missing include directory for YAML-CPP.
A couple of notes for credit:
#Botje: For pointing out that I need -shared and not -c in the compilation of a shared library. (libdata.so)
#Maxim Egorushkin: For linking to a very useful document on the matter.
One thing to note as well, is that when linking against a C library in a C++ program, you may need to use 'extern "C"' (as elaborated in the linked page). This is especially important when using the SQLite3 library.
Note that linking .a files into .so is rather unusual. People do that, but for wrong reasons.
When you link a .so, provide individual .o files compiled with -fPIC. Don't pack those .o files into .a first, that doesn't make much sense.
Why? Because .a file is merely a bunch of .o files. There is no point in making a .a file from a bunch of .o files just to turn that then into .so file.
To make a static library one builds .o files and packs them into .a. In fact, static library is a wrong name, technically, .a file is an archive (of .o files). Archives cannot link to other libraries they need because .o file cannot carry dependencies, neither can .a files.
To make a shared library one builds .o files with -fPIC option and links them into .so, along with any required libraries (static or shared). This is the .so file that carries dependency information on other .so files, .a archives are linked in.
When you build a .a that means you trade sharing code (in form of .so) for maximum execution efficiency (in the form of linking parts of .a into your executable directly). That means you build .o files without -fPIC option (it introduces extra access overhead) and bundle them into .a. Note, that .a file cannot refer to other libraries it needs (unlike .so), it is just a bunch of .o files. Static library .a is almost just a form to refer to multiple .o file. For local builds you should use thin archives that don't copy .o into .a rather refer to .o.
Also note, that when you link .a archive, only those .o files from the archive get linked into your executable (or shared library) that resolve currently unresolved symbols (unless --whole-archive). That means, if you have a global/namespace scope object with a constructor and link that into .so then it links in everything from the supplied object files and your global object constructor runs as expected. However, if you link in .a, the linker only pulls in those symbols/object-files that resolve currently undefined symbols, so that if your global object isn't referred to (possibly indirectly) from a file with main function, it won't be linked in and its constructor won't run.
For your purpose of building one .so from multiple 3rd-party libraries, you should compile those libraries' object files with -fPIC but not link them into .a. Then you link all those .o files into one .so file with all the libraries required by those comprising .o (either statically or dynamically).
With regards to -lpthread this is sadly a very common misconception perpetuated by POSIX standard wording being out of date.
In the old days there were two implementations of Pthreads API on Linux (and probably other systems): LinuxThreads and NPTL. POSIX standard merely says that if you want POSIX-compliant behaviour then link NPTL, not LinuxThreads and that is what that -lpthread linker option for. They fail to explain this reasoning or remove that sentence because it is woefully out of date.
Nowadays, modern Linux, and probably other systems, provide only the POSIX-compliant version. Hence, that -lpthread flag is obsolete, serves no purpose and isn't sufficient to build correct multi-threaded programs.
When you build multi-threaded programs you need to follow the documentation of your compiler. gcc and clang require using -pthread flag for both compiling and linking.
I apologize that this is redundant, but none of the answers available seem to be able to solve my problem. I am attempting to compile an executable using a shared object library. The shared object library is called libsession.so and is found in the same directory that I am compiling the executable. To compile and link, I use the following command
g++ test_main.cpp -o program -std=c++11 -I ../src/base -L. -lsession
Unforutanely, I get the cannot find -lsession error when linking. If I change the command to directly reference the shared library as follows
g++ test_main.cpp -o program -std=c++11 -I ../src/base libsession.so
then the executable compiles/links and all is well. Does anyone have any thoughts as to what I may be doing wrong?
The only difference between using an '-l' option and specifying a file
name is that '-l' surrounds library with 'lib' and `.a' and searches
several directories.
https://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_3.html#SEC16
I want to use a shared library (resides in a custom directory) into an executable.
I've created this makefile
all: SayHello
SayHello: compiledObjects/SayHello.o myLib/libNames.so
g++ compiledObjects/SayHello.o -o SayHello -Icommons -LmyLib -lNames
compiledObjects/SayHello.o: SayHello.cpp
g++ -c SayHello.cpp -o compiledObjects/SayHello.o
myLib/libNames.so: commons/Names.cpp commons/Names.h
g++ -shared -fPIC commons/Names.cpp -o myLib/libNames.so
That create correctly the executable and shared library infact I can Execute the program using this command
LD_LIBRARY_PATH=/custom/path/to/lib/myLib/libNames.so
./SayHello
How can I execute ./SayHello without specify LD_LIBRARY_PATH?
I'm not using any IDE and I'm on linux.
Use the -rpath option to link your executable. See the ld(1) manual page for more information.
P.S. Your makefile appears to have a bug. If you successfully make your program, and immediately run make again, looks like your makefile will attempt to recompile the program again, even though nothing has changed.
After all, the whole purpose of a makefile is to avoid doing unneeded compilations.
The SayHello.o build target should be compiledObjects/SayHello.o.
You need to tell g++ to pass the -rpath option to the linker using -Wl,-rpath. Also, you need to specify a path to the -rpath option.
Putting it all together your last build step should look like this:
SayHello: compiledObjects/SayHello.o myLib/libNames.so
g++ compiledObjects/SayHello.o -o SayHello -Icommons -LmyLib -lNames -Wl,-rpath=/custom/path/to/lib/myLib/
Relative RPATH:
If you want to specify an RPATH relative to your binary you should use
$ORIGIN as a placeholder: -rpath='$ORIGIN/rel/path'.
I am making a simple hello world program to learn about linking shared libraries in linux. I have managed to compile the main program into an executable with the shared library using the following:
g++ -fPIC -c lab2_hello_main.cpp <--create position independent objects
g++ -fPIC -c lab2_hello_sub.cpp
g++ -fPIC -shared -Wl,-soname=libfuncs.so.1.0 *.o -o libfuncs.so.1.0 -lc <--make the shared library
ln -s libfuncs.so.1.0 libfuncs.so <-- soft links for compiling and running
ln -s libfuncs.so.1.0 libfuncs.so.1
g++ -o hello_dyn lab2_hello_main.cpp -L/mypath -lfuncs <-- Linking the library to main
When I do an ldd on hello_dyn I get an output stating that the library can't be found:
"libfuncs.so.1.0 => not found"
The other libraries it looks for automatically are fine.
Anyone know why this might be?
Your shared library's location is not in the linker's search path. You can confirm this by adding the directory in which your library is located to the LD_LIBRARY_PATH environment variable and then run ldd again. See the ld.so(8) man page for details.
I'm trying to compile an executable file which i want also to use as shared library. When i'm clearly compile and linking it as "executable" - everything fine - file could start and work correctly. At this phase i cant correctly linking other libraries with it (tons of redefinitions in log). When i'm trying to add options -Fpic -shared - program copiles successfully, but starting with segmentation fault. How can i make it executable and "sharedlibrary" at the same time?
A single file cannot be a shared library and an executable at the same time. But you can link your object files twice to make both. It'd go something like this:
g++ -c -o module.o module.cpp # create an object that has no main()
g++ -shared -fPIC -o libmodule.so module.o # build shared library
g++ -o program module.o main.cpp # build executable
Or instead, the last line could link the shared library (in which case you'll need the library present when you run the executable):
g++ -o program -l module main.cpp