By default, libboostpython.a is compiled without -fPIC. But I have to make a python extension and it is a dynamic library with -fPIC that links to static libraries.
How can I compile a static library (libboostpython.a) with -fPIC from boost.python?
There are a couple options you could use:
Compile boost from source and pass extra compiler options to bjam. E.g. bjam ... cxxflags='-fPIC'. That would compile every boost source file as position independent code.
Use boost in the form of shared libraries. In this case you probably want to ship boost shared libraries along with your application to make sure the appropriate version of boost is used. You can link your executable with '-Wl,-rpath,$ORIGIN' flag, so that when the dynamic linker searches for shared libraries required by your executable it looks for them in the directory where the executable is. See man ld.so for more details on $ORIGIN.
Note that if you already run bjam once you need to clear the targets first
it is helpful also to print the commands by applying -d+2:
./bjam clean &&
./bjam -d+2 link=static cxxflags="-fPIC" install
Related
I have built the boost 1.55 serialzation library with the following command:
b2 --build-dir=build toolset=gcc --with-serialization --layout=tagged link=static threading=multi stage
and got libboost_serialization-mt.a and libboost_wserialization-mt.a in my stage/lib directory - fine. Then I added boost_serialization to my C::B project's linker library list and compiled the boost serialization example and it ran fine from the command line. I then built the dynamic and single thread variants additionally using
b2 --build-dir=build toolset=gcc --with-serialization --layout=tagged link=static,shared threading=multi,single stage
and got more libraries in my stage/lib directory, as expected. What puzzles me is that there is a .so file for every library, even those that should be static. Why is it there? What is it needed for?
When I now compile the project, the executable complains:
error while loading shared libraries: libboost_serialization.so.1.55.0: cannot open shared object file: No such file or directory
The library is definitely there and I might just need to add the path to it to LD_LIBRARY_PATH, but I want to link statically for now. How can I do that?
I also don't quite understand the library naming: I have some libboost_wserialization... libraries in my lib folder and the w prefix to serialization is not described in the library naming section of the current boost getting started page.
Your answers gave me a better understanding of what was going on - now I know where the boost_wserialization libraries came from. I turned out that after doing the second build, all present libraries were shared, and the static libraries were overwritten. That's why I got confused by the "extra" .so files for those libraries which previously were indeed static.
Ok, first question:
Why is there a boost_serialization and boost_wserialization library?
the wserialization library is wchar_t oriented. Put into a separate library because it may not actually be needed.
Why are there multiple shared/static libraries?
The reason you're seeing all those extra shared libraries is because you're invoking b2 with link=static,shared, which instructs boost to build the shared libraries as well as the static libraries. Additionally, adding in the thread=multi causes the building of the mt libraries, which are libraries that should be used when linking to multi-threaded applications.
Why am I getting the run-time link error about libboost_serialization.so.1.55.0?
By default, most unix/linux systems will prefer the use of shared libraries over static libraries when linking, so when you try to link it will prefer to use the shared libraries over the static ones. If you want to force the link of the static library rather than the shared one, you tell the compile-time linker to do that, using:
-Wl,-Bstatic -lboost_serialization -Wl,-Bdynamic
This will cause the linker to look for the static variant of the boost_serialization library, rather than the dynamic one.
Now, because you're using code::blocks, I'd have to look up how to specify these flags on a case-by-case basis, but the most sensible thing to do is to clean the boost build using ./b2 clean and then rebuild, specifying only link=static, then you should end up with only .a files, which will produce stand-alone executables again.
If you want to specify this option for code::blocks, you would need to put them into the Build Settings -> Linker settings -> Other Linker Options field for the project. Simply specifying the library in the libraries field will not work for this case. Additionally, forgetting to pass in the -Wl,-Bdynamic option will cause it to try to link in static versions of some platform libraries, which can lead to build failure if the library in question is not present.
If you want to avoid having to set LD_LIBRARY_PATH to run the binary, you can add the option -Wl,-rpath,/path/to/boost/libraries, to the linker flags which will cause the compiled program to search that directory when trying to resolve the location of libraries.
What puzzles me is that there is a .so file for every library, even
those that should be static. Why is it there? What is it needed for?
You apparently are using someone else's make file. I wrote my own. My build command does not create a ".so" (shared object library). It only creates the ".a" (archive library). The linker knows how to use either.
See man ar. The utility ar builds archives.
See man ld. The utility ld can build shared objects.
You might look for these utility invocations in your build sequence, or ask someone where they are and comment out the use of ld, as you most likely do not need both (and building both will extend your build time unnecessarily). Alternatively, you might temporarily rename the ld command, and try your build. When it can not find the ld command, you might get a useful hint as to where the ld is invoked.
In my make file, the commands look like the following. The comment char is a # at beginning of line. (The string expansions $(AR) and $(LD) allow the use of non-standard utilities.)
$(TARGET_ARCHIVE): $(OBJ)
#echo R10: $(TARGET_ARCHIVE) :from: $(OBJ)
$(AR) crs $(TARGET_ARCHIVE) $(OBJ)
# $(TARGET_OLB) : $(OBJ)
# #echo R00: $(TARGET_OLB) :from: $(OBJ)
# $(LD) -o $(TARGET_OLB) -r $(OBJ)
The archive (.a), when used, is linked directly to and included in your executable. When the executable is loaded, all the referenced symbols of the .a are already in it. (un-referenced symbols and code are not linked in)
The shared object (.so) is not directly linked, but rather your executable gets a handle (or perhaps a file name) to the .so. It is my belief that when your executable is loaded, the .so is not immediately loaded. The .so does not load until the first time your executable references a symbol that is in the .so. At that loading, your app will experience a delay, but probably this late loading is reasonable for most applications.
It is also possible that the .so is already loaded in system memory before you activated your process. In that case, when your executable first references a symbol in the .so, some system code will 'map' the existing in-memory .so to your application -- probably faster than loading it, but I suppose the big benefit is that a .so that is used / referenced by many processes need only be loaded once, saving memory space. The loaded .so has all of its symbols, even if your app does not need all of them.
In either case, your executable will be smaller with .so's, bigger with .a's, but the .so's have some small performance hit for each .so that needs to be loaded or mapped in. With 4 GB in my desktop, the desktop has never felt 'crowded'. It's swap has never been used (afaik). So I generally use .a's.
NOTE: When the linker has access to both an archive (.a) and a shared object (.so) file, the linker will use the .so (and ignore the .a). Probably you can override that preference, but I have not tried. I find it easier to simply move the archive (.a) into a separate (from the .so's) directory, and inform the linker via the -L build option.
First I built the Boost libraries that require building by going to /usr/local/boost_1_49_0/ and running the bootstrap.sh. This went off alright.
Step (1) created all the .so and .a files in /usr/local/boost_1_49_0/stage/lib
I tested linking against a library, say lboost_regex and #include <boost/regex> in my source code. This also went off alright.
Finally trying out the example on asio, I tried:
g++ -I/usr/local/boost_1_49_0 MAIN.cpp -o MAIN -L/usr/local/boost_1_49_0/stage/lib -lboost_thread -lboost_system -lpthread
(4) compiled alright. But when I run the program with ./MAIN, I get the following error:
./MAIN: error while loading shared libraries: libboost_system.so.1.49.0: cannot open shared object file: No such file or directory
The -L option only sets a compile-time library search path; if you want a shared library to be found at runtime then its directory must be known at runtime.
One way to set this with g++ is to pass -rpath to the linker, via the compiler; in your case you could say -Wl,-rpath -Wl,/usr/local/boost_1_49_0/stage/lib. (This embeds the directory in the executable.)
Another way is to install the libraries in a place that the linker searches by default (e.g. /usr/local/lib might be such a place, depending on how the system is configured).
Yet another way is to set an environment variable such as LD_LIBRARY_PATH (Linux or Solaris) or DYLD_LIBRARY_PATH (Mac OS X), to tell the linker where to search when launching executables from the shell where the variable is set.
Are you sure the shared library is in a place where the loader can find it? Either put it in a system wide directory or the same directory as the executable.
Here's a link with more information about the loader.
I have both static and dynamic versions of Boost installed in /usr/local/lib, i.e. both libboost_system.dylib and libboost_system.a exist.
In my qmake project file I've added the Boost libraries to the linker like so: LIBS += -lboost_system
Can I tell qmake to prefer the static versions without explicitly stating the filename, so that I can reduce the amount of qmake code to get static linking on all platforms?
I have not used qmake, but there is a way to make gcc link to a static lib by put -static flags to linker. All libs that you want to link statically should put after this flag.
I searched for qmake's document, and found that you can modify the link flags by change the variable QMAKE_LFLAGS, or may directly add this flag to LIBS
It seems that this is not really possible, and specifying absolute paths to libraries is a much better solution (i.e. how CMake does it by default).
If I pass a lib name with -l option, then GCC will link to the best lib from /usr/lib. I think the dynamic version is chosen over the static version when both are present.
But there are also debug versions of the same lib. Debug versions are supposed to be compiled with optimisations off and extra runtime checks on. I checked it with the package manager (Synaptic on Ubuntu) that dbg versions are indeed installed on my computer, but not entirely sure where are those libs and how to link to them.
Any hints are greatly appreciated. Thanks.
If you are aware of the name of the debug library, you can directly use it in compilation instead of going with -l option. For example :
gcc -o <output_file> *.c <libdebug_file>.so
how about moving those libraries to a different path rather than standard ones and then compile using g++ with the -l (library-name option)
I have ran bjam.exe --build-dir="C:\build-boost" --build-type=minimal msvc stage
and now I have libraries .lib with these headers, for example
libboost_serialization-vc100-mt
libboost_serialization-vc100-mt-1_45
libboost_serialization-vc100-mt-gd
libboost_serialization-vc100-mt-gd-1_45
I believe these should be static libraries for debug and release version. When I run the compiler with Multi-threaded Debug (/MTd) it gives an error LNK1104: cannot open file 'libboost_serialization-vc100-mt-sgd-1_45.lib' It is looking for one with -sgd
where am i going wrong?
Something that is kind of confusing is there are two 'static' options for building boost with MSVC.
B2.exe takes the option link=static which tells boost that you want to link it (boost) statically. If you are compiling your VC project with /MT or /MTd you will also need to use the runtime-link=static option to tell boost that you will be linking to the VC runtime libraries statically.
It is the second runtime-link=static which puts the -s in the .lib name.
My command line for building boost was
b2.exe --toolset=msvc variant=release link=static threading=multi runtime-link=static stage
You have the dynamic versions. The static ones are delimited by having the "s" in the name. Make sure you specified link=static on the bjam command line. If not, you'll have to rebuild to make the static versions.
See Boost getting started windows section 6.3 naming and section 6.1 on Unix naming
For static libraries there should be a s in there e.g. -sgd so you have dynamic libraries
here is how i break it down
libboost_serialization-vc100-mt-sgd-1_45.lib
lib- if boost library starts with lib then its a static library , shared library do not start with lib prefix. Also static library will have a '-s' in the name.
mt- multi-threaded , obtained by specifying threading=multi when you ran bjam or b2.This is the default threading.
g- use debug libraries for building the code
d- build a debug version of your code
So your compiler is searching for multi-threaded static debug library(mt-sgd) as you ran with /MTd(Creates a debug multithreaded executable file using LIBCMTD.lib). I guess by default it must be searching for static library. If you want a dynamic library, insert these lines in your code or define a macro
#define BOOST_ALL_DYN_LINK
Please check this document:
http://www.boost.org/doc/libs/1_45_0/more/getting_started/windows.html#library-naming
There you can find the meanings of all letters and how you can build the boost accordingly also...