I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
Related
I have a C++ project and I want to compile it into a single binary file which contains all the shared libraries, third-parties, and so on.
I want to move the compiled file into different servers without installing any dependencies. That's why I need an all-in-one/bundled binary file.
For example, I've tried to compile this:
g++ sample.cpp -o sample -Wl,-Bstatic -l:libyaml-cpp.so
But I got this error, while I really have libyaml-cpp.so file in the /usr/lib directory:
/usr/bin/ld: attempted static link of dynamic object `/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/../../../../lib/libyaml-cpp.so'
collect2: error: ld returned 1 exit status
What's the best way to fix the issue and generally have a compiled file with all dependencies in GNU/Linux platform?
Normally you would use g++ sample.cpp -o sample -static -lyaml-cpp.
But it seems Arch tends to not include static libraries in their packages. Your options are:
Build yaml-cpp yourself to get a static library.
Distribute the dynamic library along with your executable.
To avoid having to install it in system directories, you'll can do one of:
Set LD_LIBRARY_PATH env variable when running your program, to the location of the .so.
Add -Wl,-rpath='$ORIGIN' to the linker flags, then the resulting binary will automatically try to load libraries from its directory, before checking system directories.
You can also modify rpath in existing executables and shared libraries using patchelf. This is useful when you want to include shared libraries that other shared libraries depend on, and the latter weren't compiled with -Wl,-rpath='$ORIGIN'.
You're explicitly demanding linkage against a shared object (.so)!
-l:libyaml-cpp.so
You say your linker can't find -lyaml-cpp, so you seem to be missing the necessary files; that should usually not be the case; the installation of the development files for libyaml-cpp should have included the necessary files.
Generally speaking, there are two routes to go to ensure your program only consists of a single (executable) file:
Static Linking
You can compile a single binary, provided you have access to static libraries for all your dependencies (typically named something like lib*.a). Static libraries behave very similar to ordinary object files when linking the program. Note that your question explicitly refers to a dynamic library (lib*.so), which is the other kind of library that is compiled to be loaded from a file.
Be aware that the glibc does not like to be statically linked, and may not support it at all.
Embed Shared Object as Resource
Alternatively, you can embed shared objects (and other resources) into your executable and unpack it for execution. This can be done in a variety of ways (think self-extracting zip archives, files stored in global variables as byte arrays, this question may get you started).
You will then have to dump the shared libraries into files (e.g., in tempfs) and dlopen them.
I have a C++ project called testlib.pro (using Qtcreator) which will create a static library libtest.a.
The project is also included staticlib.a (example) and staticlib1.a. i.e using someother static libraries im creating one static library. After creating static library, Im creating C wrapper (testApi.c) to use the C++ code using the static library. I am compiling the testApi.c using the below option
gcc -o demo testApi.c -L ./testlib -ltest
But it is giving linker errors which stats that it requires the static libraries which i used to link the libtest.a. So i recompile the program with below comment and it works fine
gcc -o demo testApi.c -L ./testlib -ltest -lstatuclib -lstaticlib1
My understanding is If I ship the libtest.a to someother machine and try to compile testApi.c file it may requires staticlib.a and staticlib1.a in that machine. But I would like to use only newly created static library libtest.a. Am i missing any?
NOTE: I have included staticlib.a, staticlib1.a using -l option in my testlib.pro
If your library uses other static libraries then they will also need to be provided at link time.
There is an ugly way around it (on Linux). You can unpack existing static library (or several of them) and repack them into a new library. So you could, if you felt particularly frisky, unpack those libraries that yours depends on, then pack their contents along with your own stuff into a new library. Ugly, confusing, possibly causing all sorts of other problems, but if that is the way you want to go...
The static library concept is archaic in nature. When a program had a lot of modules, it was sometimes impossible to put them all in the command line for the linker to add them to the program. Also, for libraries, as the .o modules where all being included in the final executable, there had to be some mechanism to allow the linker select only the needed modules and not to include all of them in the final executable, or the executables will grow a lot including modules that the program will not use. Both things where solved with the introduction of dynamic shared objects, so using .a files is somewhat deprecated today and it is only used for statically linking programs.
Anyway, the algorithm to select the object modules in the linker is not recursive, so when it opens a .a library to search for dependent files to be included in the final executable, it searches only for .o (and probably .so, but I have not tested this), and it will ignore any .a file it finds in there. Many systems include an index file in the archive that has a mapping between provided identifiers, and the name of the module that provides them, so in one pass the compiler knows which archived objects need to be extracted. That index file should be appended (and rebuilt) in case a library (with its own index) where included in the file, so this justifies not using recursion at all in the library search.
The solution for this problem, is to link all those libraries you need to make the final executable, or as you have already been told, to extract the .o files in the library and put them in another library. There is still a third solution, that is: The linker allows you to specify a file that has options (and you can specify library names, and .o files you want it to scan) and it will read that file to check the set of libraries you want it to scan.
Another point is that the linker never includes a library as such. A library is just an archive (like a .tar or .zip file) in which the linker explores and extracts the files it needs, so there's no need to make the search algorithm recursive at all. And there's no difference between an archived file in a library and that same file out of the archive.
i'm trying to link shared library to another shared library(protobuf) with -rpath option, the problem is that the lib is in another direcory in compile time than in runtime, and -rpath option requires an existing in compile time path. (so i get an "No such file or directory" error)
Is there any workaround for that?
I would rather not to use LD_LIBRARY_PATH variable to solve this problem.
-rpath specifies the runtime library search path.
Here is a quoate from the link Program Library HOWTO
"
During development, there's the potential problem of modifying a library that's also used by many other programs -- and you don't want the other programs to use the developmental''library, only a particular application that you're testing against it. One link option you might use is ld'srpath'' option, which specifies the runtime library search path of that particular program being compiled. From gcc, you can invoke the rpath option by specifying it this way:
-Wl,-rpath,$(DEFAULT_LIB_INSTALL_PATH)
If you use this option when building the library client program, you don't need to bother with LD_LIBRARY_PATH (described next) other than to ensure it's not conflicting, or using other techniques to hide the library. "
I found an answer here Can I change 'rpath' in an already compiled binary?
Changing rpath after compilation solves my problem.
[edited]
There is another, better approach: how to link to shared lib from shared lib with relative path
For some reason I'm never able to use external libraries in any language. I'm looking for instructions/explanations of how to use external libraries, as well as how they work. When I search online, I get fragments that never seem to apply to whatever library I download and try and use. I work on both a mac and a pc, and C++ examples are fine. I use eclipse IDE with the C++ plug in. If there are instructions that apply to all libraries that would be great.
Say you have a class Unuseful defined as follows:
File Unuseful.h:
class Unuseful {
public:
void printUnusefulStatement();
};
File Unuseful.cpp:
#include "unuseful.h"
#include <iostream>
void Unuseful::printUnusefulStatement()
{
std::cout << "Hello world!" << std::endl;
}
Now, you have another class that needs printing unuseful statements:
Unuseful u;
u.printUnusefulStatement();
This means that you want to use an external library containing the specific implementation (printUnusefulStatement) that you want to include in your code.
You may use this library in two ways:
By providing the source code to the compiler
By providing a binary file (which had been previously compiled for your architecture), to the linker
Case 1: using a library at compile time
This is the simplest case.
You have the source code of the library you have to use and you simply have to compile it together with your existing code (say main.cpp file).
Typically you are the author and user of the library (a class that accomplishes a task you need).
Compiling with this command:
g++ main.cpp unuseful.cpp
allows you to use the implementation you need in your main.cpp file.
Case 2: linking a library
More often than Case 1, you don't have the source code of the library you want to use. You only have the header file (Unuseful.h, to continue with the example) and a static or shared library (probably[*] libunuseful.a and libunuseful.so files, respectively).
The static library is an archive of object files (*.o) that are linked inside your final executables, the shared libraries instead are loaded dynamically - at run time (look at this page for a better understanding of the difference).
Static libraries are created by simply archiving the *.o files with the ar program:
# Create the object files (only one here)
g++ -c unuseful.cpp
# Create the archive (insert the lib prefix)
ar rcs libunuseful.a unuseful.o
Shared libraries are created with the g++ -shared option:
# Create the object file with Position Independent Code[**]
g++ -fPIC -c unuseful.cpp
# Crate the shared library (insert the lib prefix)
g++ -shared -o libunuseful.so unuseful.o
Let's suppose now you have the Unuseful.h file and the shared library (libunuseful.so file) and you have a main.cpp file that instantiates a Unuseful object and calls the printUnusefulStatement method.
If you try to compile this file (g++ main.cpp) the linker will complain because it cannot find the printUnusefulStatement symbol.
It's time to use the library:
g++ main.cpp -L. -lunuseful
The -L option tells the linker where to search for library files and the -l flag tells the linker the name of the libraries to be used (without the lib prefix).
Now the executable (a.out, because I didn't specify a different name) is created, and you have used a library to implement a functionality you needed (printUnusefulStatement).
Since the shared library is loaded at run-time, the execution of the a.out executable may fail because the system is not able to find the library.
Typically this can be solved by appropriately setting an environment variable indicating which paths to use to search for dynamic libraries:
# Set the LD_LIBRARY_PATH [*]
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
Done, now your executable has been compiled and it will be able to run and load the library it needs.
Conclusion
This is a rapid overview on libraries which I hope can help you understand how they are used and provided to others.
There are many many aspects that should be investigated in more detail, if you are interested: g++ options when creating shared libraries, ar options, environment variables, the shared libraries format and so on.
[*]: In a Unix environment
[**]: If supported for the target machine, emit position-independent code, suitable for dynamic linking and avoiding any limit on the size of the global offset table. This option makes a difference on the m68k, PowerPC and SPARC. Position-independent code requires special support, and therefore works only on certain machines. [From the g++ man page]
Here's where you start
http://en.wikipedia.org/wiki/Library_(computing)
Basically, a 'library' is a collection of compiled functions and class declarations.
On a Mac there are also "frameworks" which are somewhat similar to Pascal's units and contain both the declarations and the compiled code.
In managed languages like Java or C# there are packages and assemblies. Both are closely related to libraries.
To use libraries in C or C++ you've got to have a .lib-file (or .a-file for most POSIX or GCC toolchain based compilers) and the prototypes of the functions which are compiled into the .lib file. Depending on your development environment (for Eclipse you are most likely using the GCC compiler and GNU toolchain with LD linker), you just specify the library files (.lib or .a) as the input to the linker. Most of the time the library is accompanied with header files which contain the definitions of function prototypes.
Even if you did not know about the linker, which is strange enough, the libraries are still used in your program implicitly - the std::cout is in the libstdc++ or the C Run-Time Library.
As an example of a huge library and a useful set of C++ classes you might want to look at Boost.
To write GUI on Windows you can use the WinAPI which is described in MSDN.
To write GUI on Mac you can use Carbon API which is somewhat similar to WinAPI, but is now deprecated. The only way to write "legit" GUI for MacOS is to use Cocoa and Objective-C.
To write cross-platform GUI you can use a lot of libraries: Qt, wxWidgets, GTK among them.
The last, but not the least. C++ is not the best language for GUI.
The best way to use external C++ libraries is make use of a C++ package manager, go and learn one of these:
conan
vcpkg
hunter
cppan
build2
Some of them involve using CMake, you can find a well written tutorial on it here.
.
I have a software library and I used to create .a files, so that people can install them and link against them: g++ foo.o -L/path/to -llibrary
But now I often encounter third-party libraries where only .so files are available (instead of .a), and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
What are the differences between these solutions? Should I prefer creating .so files for the users of my library?
Typically, libfoo.a is a static library, and libfoo.so is a shared library. You can use the same -L/-l linker options against either a static or shared. Or you can name the full path to the lib with static or shared. Often libraries are built both static and shared to provide application developers the choice of which they want.
All the code needed from a static lib is part of the final executable. This obviously makes it bigger, but it also means it's self-contained. Once it is compiled, you can run your app without the lib.
Code from a shared lib is not part of the executable. There are just some hooks in place to make the executable aware of the name of the lib it needs. In order to run your app, the shared lib has to be present in the lib search path (e.g. $LD_LIBRARY_PATH).
If you have two apps that share the same code, they can each link against a shared lib to keep the binary size down. If you want to upgrade parts of the app without rebuilding the whole thing, shared libs are good for that too.
Good overview of static, shared dynamic and loadable libraries at
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
Some features that aren't really called out from comments I've seen so far.
Static linkage (.a/.lib)
Sharing memory between these compilation units is generally ok because they should(?will) all be using the same runtime.
Static linkage means you avoid 'dll hell' but the cost is recompilation to make use of any change at all. static linkage into Shared libraries (.so) can lead to strange results if you have more than 1 such shared library used by the final executable - global variables may exist multiple times and which one is used and when they are initialised can cause an entirely different hell.
The library will be part of the shipped product but obfuscated and not directly usable.
Shared/Dynamic libraries (.so/.dll)
Sharing memory between these compilation units can be hazardous as they may choose to use different runtime. This can mean you provide different Shared/Dynamic libraries based on the debug/release or single/multi threaded or...
Shared libraries (.so) are less prone to 'dll hell' then Dynamic libraries (.dll) as they include options for quite specific versioning.
Compiling against a .so will capture version information internal to the file (hard to fake) so that you get quite specific .so usage. Compiling against the .lib/.dll only gives a basic file name, any versioning is done managed by the developer (using naming or manually loading the library and checking version details by hand)
The library will have to ship with the final product (somebody else can pick it up and use it)
But now I often encounter third-party libraries where only .so files are available [...] and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
JFYI, if you link to a shared library which does not have a SONAME set (compare with readelf -a liblibrary.so), you will end up putting the specified path of liblibrary.so into your target object (executable or another shared library), and which is usually undesired, for users have their own ideas of where to put a program and its associated files. The preferred way is to use -L/path/to -llibrary, perhaps together with -Wl,-rpath,/whatever/path/to if this is the final path (such pathing decisions are made by Linux distributions for example).
Should I prefer creating .so files for the users of my library?
If you distribute source code, the user will make the particular choice.