I have a C++ source tree developed under Ubuntu 12.04 using clang++ 3.2 that builds some libraries, then compiles some applications with these libraries and the usual collection of other various system libraries. Two puzzles. Client reports that code fails to build with undefined reference to clock_gettime(). Sure enough, I did not include the obligatory "-lrt" in the build logic (scons).
First puzzle: This compiles, links and executes correctly and without complaint on my system even though I do not specify "-lrt" anywhere! How does this symbol get correctly resolved? I suspect it is because the application links against a dynamic library that itself requires librt but I don't understand the logic behind why this would happen?
Second puzzle: Assuming a satisfactory explanation of how clock_gettime() gets resolved without "-lrt", why would this happen on my system but not on client's very similar setup?
"... a riddle, wrapped in a mystery, inside an enigma" --- Winston Churchill
Suggestions for tools to reveal what is really going on here would be most welcomed.
From SUSv4 (Utilities/c99):
-l rt
This option shall make available all interfaces referenced in <aio.h>, <mqueue.h>, <sched.h>, <semaphore.h>, and <spawn.h>, interfaces marked as optional in <sys/mman.h>, interfaces marked as ADV (Advisory Information) in <fcntl.h>, and interfaces beginning with the prefix clock_ and time_ in <time.h>. An implementation may search this library in the absence of this option.
I presume the above suffices to at least justify why this behavior is allowed by POSIX.
Most likely the involved systems are different in this regard. For example, clock_gettime() might be implemented in libc for you, but in librt for your client. Don't take chances: use a portable -l rt and forget about the issue.
A more obvious example is -l xnet, which behaves similarly according to POSIX, but, at least on Gentoo, Debian and Ubuntu Linux systems, compiling with -l xnet actually yields an error. (libxnet purportedly contains the implementation for the UNIX sockets interface.)
If you want to further investigate the issue, try ldd in GNU/Linux systems. ldd should display the dynamic dependencies for your binary. I'd bet that clock_gettime() is simply implemented in libc.so.
Related
I have a C++ based dynamic library that I have built for the big 3 OSs that relies heavily on boost. Currently, I am compiling it for the raspberry pi. It took me a while to find the magic words to get the library to even build (-frepo as a compiler flag was the key, but I confess that I am not certain why this is the case).
Now, when I try to link to the library, I get an 'undefined reference' error to every boost call that my library makes, i.e.:
//`libmylib.so`: undeifined reference to `boost::shared_ptr<boost::detail::thread_data_base>::shared_ptr()'
When I build libmylib.so, I also build a custom version of boost as libboost.a. This all compiles and links fine on other OSs and non-ARM architectures so I tried putting -lboost as one of the flags, but I still get the same plethora of undefined reference errors form libmylib.so.
Needless to say, all my paths are correct.
It seems like linking behaves a bit differently on the raspberry pi than it does on other linux systems. For example, I built a static library (libmythread.a) that uses libpthread. When I link to that libmythread.a, I also get undefined reference errors unless I also use -lpthread in the build recipe. On my Thinkpad running Fedora, I would never have to do this since I included -lpthread in the compilation of the static library libmythread.a.
I would love to find a tutorial or guide that explains these discrepancies. I would also love to overcome them!
I also tried the same build on a conventional linux machine and everything linked fine, no problem. At least I know that my build process is OK. This does open up the possibility, though, that the -frepo flag is doing something funny that I don't understand and that this could be the root of the problem.
Solved. In the end, the trouble stemmed from the -frepo flag. This was necessary to compile a file called legacy_abi.cpp that is part of my library to allow third party developers using older and more exotic OSs/compilers. This isn't needed on the Pi, so I just removed it from the offending file from the build, dropped the -frepo flag and happy happy.
One final note, aptitude (for Pi, anyway) only supplies boost up to 1.49 (as far as I can tell). My project requires boost >= 1.50. This is an inherited project, so I'm still discovering all its little idiosyncracies.
I'm using Code::Blocks IDE(v13.12) with GNU GCC Compiler.
I want to the linker to link static versions of required runtime libraries for my programs, how may I do this?
I already know that my executable size will increase. Would you please tell me other the downsides?
What about doing this in Visual C++ Express?
Since nobody else has come up with an answer yet, I will give it a try. Unfortunately, I don't know that Code::Blocks IDE so my answer will only be partial.
1 How to Create a Statically Linked Executable with GCC
This is not IDE specific but holds for GCC (and many other compilers) in general. Assume you have a simplistic “hello, world” program in main.cpp (no external dependencies except for the standard library and runtime library). You'd compile and statically link it via:
Compile main.cpp to main.o (the output file name is implicit):
$ g++ -c -Wall main.cpp
The -c tells GCC to stop after the compilation step (not run the linker). The -Wall turns on most diagnostic messages. If novice programmers would use it more often and pay more attention to it, many questions on this site would not have been asked. ;-)
Link main.o (could list more than one object file) statically pulling in the standard and runtime library and put the executable in the file main:
$ g++ -o main main.o -static
Without using the -o main switch, GCC would have put the final executable in the not so well-named file a.out (which once eventually stood for “assembly output”).
Especially at the beginning, I strongly recommend doing such things “by hand” as it will help get a better understanding of the build tool-chain.
As a matter of fact, the above two commands could have been combined into just one:
$ g++ -Wall -o main main.cpp -static
Any reasonable IDE should have options for specifying such compiler / linker flags.
2 Pros and Cons of Static Linking
Reasons for static linking:
You have a single file that can be copied to any machine with a compatible architecture and operating system and it will just work, no matter what version of what library is installed.
You can execute the program in an environment where the shared libraries are not available. For example, putting a statically linked CGI executable into a chroot() jail might help reduce the attack surface on a web server.
Since no dynamic linking is needed, program startup might be faster. (I'm sure there are situations where the opposite is true, especially if the shared library was already loaded for another process.)
Since the linker can hard-code function addresses, function calls might be faster.
On systems that have more than one version of a common library (LAPACK, for example) installed, static linking can help make sure that a specific version is always used without worrying about setting the LD_LIBRARY_PATH correctly. Obviously, this is also a disadvantage since now you cannot select the library any more without recompiling. If you always wanted the same version, why would you have installed more than one in the first place?
Reasons against static linking:
As you have already mentioned, the size of the executable might grow dramatically. This depends of course heavily on what libraries you link in.
The operating system might be smart enough to load the text section of a shared library into the RAM only once if several processes need the library at the same time. By linking statically, you void this advantage and the system might run short of memory more quickly.
Your program no longer profits from library upgrades. Instead of simply replacing one shared library with a (hopefully ABI compatible) newer release, a system administrator will have to recompile and reinstall every program that uses it. This is the most severe drawback in my opinion.
Consider for example the OpenSSL library. When the Heartbleed bug was discovered and fixed earlier this year, system administrators could install a patched version of OpenSSL and restart all services to fix the vulnerability within a day as soon as the patch was out. That is, if their services were linking dynamically against OpenSSL. For those that have been linked statically, it would have taken weeks until the last one was fixed and I'm pretty sure that there is still proprietary “all in one” software out in the wild that did not see a fix up to the present day.
Your users cannot replace a shared library on the fly. For example, the torsocks script (and associated library) allows users to replace (via setting LD_PRELOAD appropriately) the networking system library by one that routes their traffic through the Tor network. And this even works for programs whose developers never even thought of that possibility. (Whether this is secure and a good idea is subject of an unrelated debate.) An other common use-case is debugging or “hardening” applications by replacing malloc and the like with specialized versions.
In my opinion, the disadvantages of static linking outweigh the advantages in all but very special cases. As a rule of thumb: link dynamically if you can and statically if you have to.
A Addendum
As Alf has pointed out (see comments), there is a special GCC option to selectively link in the C++ standard library statically but not link the whole program statically. From the GCC manual:
-static-libstdc++
When the g++ program is used to link a C++ program, it normally automatically links against libstdc++. If libstdc++ is available as a shared library, and the -static option is not used, then this links against the shared version of libstdc++. That is normally fine. However, it is sometimes useful to freeze the version of libstdc++ used by the program without going all the way to a fully static link. The -static-libstdc++ option directs the g++ driver to link libstdc++ statically, without necessarily linking other libraries statically.
In Visual C++, the /MT option does a static link and the /MD option does a dynamic link. (see http://msdn.microsoft.com/en-us/library/2kzt1wy3.aspx)
I'd recommend using /MD and redistributing the C++ runtime, which is freely available from Microsoft. Once the C++ runtime is installed, than any program requiring the run time will continue to work. You would need to pass the proper option to tell the compiler which runtime to use. There is a good explanation here, Should I compile with /MD or /MT?
On Linux, I'd recommend redistributing libstdc++ instead of a static link. If their system libstdc++ works, I'd let the user just use that. System libraries, such as libpthread and libgcc should just use the system default. This requires compiling the program on a system with symbols compatible with all linux versions you are distributing for.
On Mac OS X, just redistribute the app with dynamic linking to libstdc++. Anyone using the same OS version should be able to use your program.
How can i compile my app linking statically glibc library, but only the code needed for my app? (Not all lib)
Now my compile command:
g++ -o newserver test.cpp ... -lboost_system -lboost_thread -std=c++0x
Thanks!
That's what -static does (as described in another answer): unneeded modules won't get linked into your program. But your expectations on the amount of stuff which is needed (in a sense that we can't convince linker to the contrary) may be too optimistic.
If you trying to do it for portability (running an executable on other machines with older glibc or something like that), there is one easy test question to see if you're going to get what you want:
Did you think of the problem with libnss, and are you sure it is not going to bite you?
If your answer is yes, maybe it makes sense to go on. If the answer is no, or the question seems too obscure and there is no answer, just quit your expirements with statically linked glibc: it has more chance to hurt than help.
Add -static to the compile line. It will only add what your application needs [and of course, any functions the functions you application calls, and any functions those functions call, including a bunch of startup code and some other bits and pieces], so it will be around 800K (for a simple "hello world" program) on an x86 machine. Other architectures vary. Since boost probably also calls the standard library at least a little bit, it's likely that you will have more than 800K added to your appliciation. But it only applies functions used by any of the code in the final binary, not the entire library [about 2MB as a shared library].
If you ONLY want link glibc, you will need to modify the linking line to your compile to:
-Wl,-Bstatic -libc -Wl,-Bdynamic. This will prevent any other library from being linked statically [you sometimes need to have more than one of these statements, as sometimes something pulled in by another library requires "more" from glibc to be pulled in - don't worry, it won't bring in anything more than the linker thinks is necessary].
I wrote a small application on Redhat Linux 6 using g++ 4.4.6. After compilation, I received an error
/usr/bin/ld: cannot find -lcrypto
I did a search for the crypto library and find them here,
[root#STL-DUNKEL01 bin]# find / -name libcrypto*
/usr/lib64/libcrypto.so.0.9.8e
/usr/lib64/libcrypto.so.10
/usr/lib64/libcrypto.so.6
/usr/lib64/libcrypto.so.1.0.0
My question is whether the compilation error is caused by /usr/bin/ld not having /usr/lib64/ in the search path? If yes, how can I add it?
Thanks.
No, you have likely incorrectly diagnosed the cause.
You need a libcrypto.so to link against. This is usually a symlink to one of the actual libraries, whose soname (libcrypto.so.??) will be embedded into the binary. Only that library is needed at runtime, but the symlink is necessary to compile.
See Diego E. Pettenò: Linkers and names for more details.
You have to add -L/usr/lib64 when calling gcc or ld.
Note, you can specify LD_LIBRARY_PATH as well/instead, but it is considered harmful to do so. (The link mentions Solaris specifically, but the issues apply to other OSs as well.)
Quote:
LD_LIBRARY_PATH is used in preference to any run time or default system linker path. If (God forbid) you had it set to something like /dcs/spod/baduser/lib, if there was a hacked version of libc in that directory (for example) your account could be compromised. It is for this reason that set-uid programs completely ignore LD_LIBRARY_PATH.
When code is compiled and depends on this to work, it can cause confusion where different versions of a library are installed in different directories, for example there is a libtiff in /usr/openwin/lib and /usr/local/lib. In this case, the former library is an older one used by some programs that come with Solaris.
Sometimes when using precompiled binaries they may have been built with 3rd party libraries in specific locations; ideally code should either ship with the libraries and install into a certain location or link the code as a pre-installation step. Solaris 7 introduces $ORIGIN which allows for a relative library location to be specified at run time (see the Solaris Linker and Libraries Guide). The alternative is to set LD_LIBRARY_PATH on a per-program basis, either as a wrapper program to the real program or a shell alias. Note however, that LD_LIBRARY_PATH may be inherited by programs called by the wrapped one ...
Add the directory to /etc/ld.so.conf
then run "sudo ldconfig" to make the changes take effect.
You can provide the directories to search for the libraries in as a parameter to gcc like so -L<directory_to_search_in>. And note that there can be multiple parameters to -L. Also, are you trying to build a 32-bit application or a 64-bit one?
There is a possibility to compile BOOST libraries in the so-called thread-aware mode. If so you will see "...-mt..." appeared in the library name. I can't understand what it gives me and when do I need to use such mode? Does it give me any benefits?
More than that I'm really confused by having BOOST Threads library compiled in NO-thread-aware regime (with no -mt in the name). It does not make any sense for me. Looks self-contradictory :/
Thanks a lot for any help!
Because you did not specify how you have built, and on what platform, I'll explain the whole story. Both on Linux and Windows, Boost.Thread library is built in MT mode. On Windows, by default, you get -mt suffix for it. On Linux, by default in 1.42, you get no suffix. The reason you get no suffix on Linux is that pretty much no other library uses such convention, and it's much less important on Linux anyway.
Does this clarify things?
There is an option to put "-mt" suffix back (bjam --layout=tagged)
--layout=<layout> Determines whether to choose library names
and header locations such that multiple
versions of Boost or multiple compilers can
be used on the same system.
versioned - Names of boost binaries
include the Boost version number, name and
version of the compiler and encoded build
properties. Boost headers are installed in a
subdirectory of <HDRDIR> whose name contains
the Boost version number.
tagged -- Names of boost binaries include the
encoded build properties such as variant and
threading, but do not including compiler name
and version, or Boost version. This option is
useful if you build several variants of Boost,
using the same compiler.
system - Binaries names do not include the
Boost version number or the name and version
number of the compiler. Boost headers are
installed directly into <HDRDIR>. This option
is intended for system integrators who are
building distribution packages.
The default value is 'versioned' on Windows, and
'system' on Unix.
MT enables multithreaded support in the boost libraries meaning you are safe to use them in your multithreaded programs (at least from the library's internal code point of view).
And indeed building the threads library in the "no threads" mode does not make any sense but I was under the impression that that specific build target is disabled.
Check these out
http://sodium.resophonic.com/boost-cmake/current-docs/build_variants.html
http://www.boost.org/doc/libs/1_41_0/more/getting_started/windows.html#library-naming
You can build Boost with multi-threading support or not (threading=multi|single). Boost.Thread force the build of the library by setting threading=multi in its Jamfile (the bjam equivalent of a Makefile).
So independently of whether you request threading support or not, Boost.Thread always provide it. Hence you can find both names.
Since, under Linux, the -mt version is aliased/bound to the regular version, it makes no difference. In a vanilla modern system, both are simply included for ease of compilation.
I'm not a Boost guru, but I assume it is this:
In a MT environment, any global or shared data may have more than one thread trying to access it at the same time, which can lead to data corruption. An MT-aware object will use synchronisation (Critical Sections, Mutexes, etc.) to ensure that only one thread can access data at a time.
There might be functions in the Boost thread library that still work in single-threaded programs. Alternatively, the functions may resolve to no-ops (harmless do-nothing functions) so that the same program can be compiled with MT (and the boost functions work) or Single threaded (and the boost functions do nothing) without having to change the code.