I've recently learnt that static linking and implicit linking are basically the same thing, just different nomenclature. My understanding is that when you statically (implicitely) link to a binary, you are by definition linking against a *.lib (windows) or *.a (linux) file, often using target_link_libraries in cmake. On the other hand when you explicitely link (using LoadLibrary on windows) you are by definition linking to a *.dll file (or *.so on linux) (and there is no corresponding cmake command because all the work is done inside the actual code).
However, in multiple places I've read people referring to statically/implicitely linking to a dll file, which has confused me. Clearly there is a hole in my knowledge somewhere and I was hoping somebody here could plug it.
Edit
Its been pointed out that this question refers mainly to windows, which it does. However, I am currently trying to produce cross platform code so I am still interested on how (or if) these concepts generalise to other platforms.
There are actually 3 different kinds of linking, not 2.
For UNIX:
Link against archive (aka static) library:
gcc main.o libfoo.a
link against dynamic (aka shared) library:
gcc main.o libfoo.so
Link against libdl, which allows you to dlopen arbitrary other shared libraries (which don't need to exist at the time of the link):
gcc main.o -ldl
Both 2 and 3 involve dynamic linker (and are using shared libraries), but to a different extent.
An equivalent exists on Windows: when you link against foo.lib, you are using either 1 or 2, depending on whether foo.lib contains actual code, or refers to foo.dll.
When you use LoadLibrary, you are in case 3.
Related
We have a given set of legacy C++ libraries compiled on an old release system with the very old compiler gcc 3.2 using libstdc++.so.5.0.0. An up-to-date CentOS7 will run an executable compiled on the old OS using the compatibility library libstdc++.so.5.0.7, while the current C++ version with a different library ABI is libstdc++.so.6.
Our new version of the executable is built on the new OS and cannot be built on the old platform any more. But currently recompiling the legacy libs on CentOS7 is also not an option: the numerical results will be slightly different. So I would like to use them as they are with their libstdc++.so.5, while the rest of my binary uses c++ version 6.
this post suggest using a wrapper c-interface library which is linked with the static version libstdc++.a.5.
this post mainly suggests the same.
This method seems possible in our case since the interface is only C-style and does not use C++ objects. And the C-ABI seems to work as a kind of barrier between the two versions of the c++ lib.
However, I have yet to find someone providing the static C++.5 libs. Also I think I have once heard talk about using two shared lib versions of libstdc++ in parallel, in one application. So is there also a solution keeping the shared C++.5 libs? I have some weak memory that they also used a C-library as border. Would it also work with dlopen()?
My current situation is like this:
Compiled on old OS with old gcc-3.2, libstdc++.so.5, 32-bit mode:
libA1.so using C++
libA2.so calling libB.so, both use C++, libA2 explicitely linked against
libstdc++ with linker option -lstdc++, libB not but probably it gets its
symbols from libA
libwrapper.so only using C not C++, linked with/using libA1 and libA2.
Compiled on new OS with new gcc-4.8.5:
executable (32-bit) calling libwrapper.so by using dlopen() system call,
so indirectly calling C++ libs libA1.so and libA2.
executable also uses a shared C++ lib libC.so (but interface is C style)
executable also uses a shared C++ lib libB.so (but interface is C style), which is also needed by libA1 and libA2.
So the executable must use the new libstdc++.so.6 for libC, but will indirectly call libA1, libA2, libB which must use the old libstdc++.so.5.
Do I have to limit the symbols exported by the wrapper lib exactly to my API, not exposing any c++ symbols like in pobedims comment in 1, using a version-script flag for the linker ?
Or maybe I just link the legacy lib on the old OS not with -lstdc++, but with the full name /usr/lib/libstdc++.so.5 ? And libB will continue to get the correct symbols from libA2 ?
My executable links/depends on ...
libB.so (C++)
libC.so (C++) which needs libstdc++.6
libwrapper.so (with C interface), which is loaded using the dynamic load dlopen(), and which itself is linked to the "worker" libA1 and libA2 (which also present a C interface):
libA1.so using C++.5
libA2.so using C++.5
libB.so using C++.5
The executable is linked with -rdynamic, maybe that is a problem? How can I be sure that no symbols in libA1, A2, B are resolved using symbols from the executable with same name?
Edit later / Resolution:
Note1: Trying on the old platform to link using archive version libstdc++.a.5 leads to an error which is only fixed in later version of gcc >= 3.4:
Error: ld: libwrapper.so: undefined versioned symbol name _ZSt10time_put_w##GLIBCPP_3.2
ld: failed to set dynamic section sizes: Bad value
so this is not an option.
Note2: LibB was a problem since it is expected by the working libs and by the executable itself. If the executable uses the new version of libB it introduces libstdc++.so.6, of which the symbols are passed to the worker libs, which crash upon start. I have changed the version of libB used to the one compiled on the old system, simply by exchanging it on the runtime system. This is possible because of the simple C interface being binary compatible. Thus my executable (via libB) and the worker libs libA1 and libA2 all use the same libstdc++.so.5. My (C) executable is built directly linking libstdc++.so.5. Now my executable can be started and works.
Note3: I found some numerical differences between the reference results and the new result. Using LD_PRELOAD mechanism has worked to resolve these:
LD_PRELOAD=libstdc++.so.5 my_executable
This seems to work nicely. I think by using LD_PRELOAD I make sure that libstdc++.so.5 has precendence in supplying the symbols resolved from libstdc++. What I did not mention before: we used the Intel compiler on the old platform, and I used the math lib libimf.so from it like this: LD_PRELOAD="libimf.so libstdc++.so.5" to provide both libs. As a result the numerical differences vanished, which seem to be caused by the implementation of common math functions.
I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
I have mostly worked in Windows, and recently I started working in Linux. I have a doubt. I have used Visual Studio as IDE in Windows and used Makefile in Linux.
There are two types of libraries in Windows (VC++), static library (.lib) and DLL. It is quite obvious (isn't it?) if I link with lib file I am using static linking else dynamic linking.
Now when I use g++ compiler, why I need to explicitly mention -Bstatic/-static or Bdynamic/-dynamic flags. Because if file is .a file then I must be using static -linking and if file is .so I am using dynamic linking.
There are times when you want to "force" the compiler to do what it normally wouldn't. In particularly -static is useful when you are building against a library that may not be installed [or not have the same version installed] on another machine where you want your code to run.
the -Bdynamic is useful if you want to have ONE library linked statically, but not EVERY library that your code uses.
e.g. gcc -o myprog myprog.o -Wl,-Bstatic -lspecial -Wl,-Bdynamic
will link myprog using static linking for libspecial (which may be something that isn't widely distributed, for example something you have built yourself)
For general, local development, you don't need either.
In the GNU toolchain, and other Unix compilers, you generally don't specify the full path to a library; you give the linker a flag such as
-lfoo
and let it figure out whether it should link with libfoo.a, libfoo.so, and where these files are located. When only a static library is available, it will link with that.
So, you don't really need to specify -Bstatic except in a few specific cases. Dynamic linking is the default and if you don't want a custom library to be linked dynamically, then just don't build a .so out of it.
Besides, you can explicitly link with a static library by giving the full filename to the linker
gcc -o some_binary main.o libfoo.a
to get a dynamically linked binary with libfoo statically linked in.
I think you've got it backwards. Whether I'm linking with
a dynamic library or with a static one, under Windows, I need
a .lib file; they just aren't generated in the same manner,
and don't contain the same thing. Under Unix, when I link with
a shared object, I link directly against the .so file, and not
against some additional file generated at the same time.
The options -Bstatic and -Bdynamic only come into play when
both a .so and a .a are present, in the same directory; they
tell the linker which one to choose for libraries specified
using the -l option. In Windows, this situation cannot occur;
since you need a .lib for the .dll, you cannot have a .lib
to be statically linked in the same directory.
Because if file is .a file then I must be using static -linking and if file is .so I am using dynamic linking.
These filename "extensions" are only human conventions — there is no reason that they have to be .a and .so respectively. Quite rightly does GCC not force us into that convention.
For some reason I'm never able to use external libraries in any language. I'm looking for instructions/explanations of how to use external libraries, as well as how they work. When I search online, I get fragments that never seem to apply to whatever library I download and try and use. I work on both a mac and a pc, and C++ examples are fine. I use eclipse IDE with the C++ plug in. If there are instructions that apply to all libraries that would be great.
Say you have a class Unuseful defined as follows:
File Unuseful.h:
class Unuseful {
public:
void printUnusefulStatement();
};
File Unuseful.cpp:
#include "unuseful.h"
#include <iostream>
void Unuseful::printUnusefulStatement()
{
std::cout << "Hello world!" << std::endl;
}
Now, you have another class that needs printing unuseful statements:
Unuseful u;
u.printUnusefulStatement();
This means that you want to use an external library containing the specific implementation (printUnusefulStatement) that you want to include in your code.
You may use this library in two ways:
By providing the source code to the compiler
By providing a binary file (which had been previously compiled for your architecture), to the linker
Case 1: using a library at compile time
This is the simplest case.
You have the source code of the library you have to use and you simply have to compile it together with your existing code (say main.cpp file).
Typically you are the author and user of the library (a class that accomplishes a task you need).
Compiling with this command:
g++ main.cpp unuseful.cpp
allows you to use the implementation you need in your main.cpp file.
Case 2: linking a library
More often than Case 1, you don't have the source code of the library you want to use. You only have the header file (Unuseful.h, to continue with the example) and a static or shared library (probably[*] libunuseful.a and libunuseful.so files, respectively).
The static library is an archive of object files (*.o) that are linked inside your final executables, the shared libraries instead are loaded dynamically - at run time (look at this page for a better understanding of the difference).
Static libraries are created by simply archiving the *.o files with the ar program:
# Create the object files (only one here)
g++ -c unuseful.cpp
# Create the archive (insert the lib prefix)
ar rcs libunuseful.a unuseful.o
Shared libraries are created with the g++ -shared option:
# Create the object file with Position Independent Code[**]
g++ -fPIC -c unuseful.cpp
# Crate the shared library (insert the lib prefix)
g++ -shared -o libunuseful.so unuseful.o
Let's suppose now you have the Unuseful.h file and the shared library (libunuseful.so file) and you have a main.cpp file that instantiates a Unuseful object and calls the printUnusefulStatement method.
If you try to compile this file (g++ main.cpp) the linker will complain because it cannot find the printUnusefulStatement symbol.
It's time to use the library:
g++ main.cpp -L. -lunuseful
The -L option tells the linker where to search for library files and the -l flag tells the linker the name of the libraries to be used (without the lib prefix).
Now the executable (a.out, because I didn't specify a different name) is created, and you have used a library to implement a functionality you needed (printUnusefulStatement).
Since the shared library is loaded at run-time, the execution of the a.out executable may fail because the system is not able to find the library.
Typically this can be solved by appropriately setting an environment variable indicating which paths to use to search for dynamic libraries:
# Set the LD_LIBRARY_PATH [*]
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
Done, now your executable has been compiled and it will be able to run and load the library it needs.
Conclusion
This is a rapid overview on libraries which I hope can help you understand how they are used and provided to others.
There are many many aspects that should be investigated in more detail, if you are interested: g++ options when creating shared libraries, ar options, environment variables, the shared libraries format and so on.
[*]: In a Unix environment
[**]: If supported for the target machine, emit position-independent code, suitable for dynamic linking and avoiding any limit on the size of the global offset table. This option makes a difference on the m68k, PowerPC and SPARC. Position-independent code requires special support, and therefore works only on certain machines. [From the g++ man page]
Here's where you start
http://en.wikipedia.org/wiki/Library_(computing)
Basically, a 'library' is a collection of compiled functions and class declarations.
On a Mac there are also "frameworks" which are somewhat similar to Pascal's units and contain both the declarations and the compiled code.
In managed languages like Java or C# there are packages and assemblies. Both are closely related to libraries.
To use libraries in C or C++ you've got to have a .lib-file (or .a-file for most POSIX or GCC toolchain based compilers) and the prototypes of the functions which are compiled into the .lib file. Depending on your development environment (for Eclipse you are most likely using the GCC compiler and GNU toolchain with LD linker), you just specify the library files (.lib or .a) as the input to the linker. Most of the time the library is accompanied with header files which contain the definitions of function prototypes.
Even if you did not know about the linker, which is strange enough, the libraries are still used in your program implicitly - the std::cout is in the libstdc++ or the C Run-Time Library.
As an example of a huge library and a useful set of C++ classes you might want to look at Boost.
To write GUI on Windows you can use the WinAPI which is described in MSDN.
To write GUI on Mac you can use Carbon API which is somewhat similar to WinAPI, but is now deprecated. The only way to write "legit" GUI for MacOS is to use Cocoa and Objective-C.
To write cross-platform GUI you can use a lot of libraries: Qt, wxWidgets, GTK among them.
The last, but not the least. C++ is not the best language for GUI.
The best way to use external C++ libraries is make use of a C++ package manager, go and learn one of these:
conan
vcpkg
hunter
cppan
build2
Some of them involve using CMake, you can find a well written tutorial on it here.
.
In my quest to learn C++, I have come across dynamic and static libraries.
I generally get the gist of them: compiled code to include into other programs.
However, I would like to know a few things about them:
Is writing them any different than a normal C++ program, minus the main() function?
How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?
Once I get it to its format, how do I include it in another program?
Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?
What is the difference (technically and practically) between a dynamic and static library?
How would I use third party libraries in my code (I'm staring at .dylib and .a files for the MySql C++ Connector)
Everything I have found relating to libraries seems to be targeting those who already know how to use them. I, however, don't. (But would like to!)
Thanks!
(I should also note I'm using Mac OS X, and although would prefer to remain IDE-neutral or command-line oriented, I use QtCreator/Netbeans)
Is writing them any different than a normal C++ program, minus the main() function?
No.
How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?
Pass the -dynamiclib flag when you're compiling. (The name of the result is still by default a.out. On Mac OS X you should name your dynamic libraries as lib***.dylib, and on Linux, lib***.so (shared objects))
Once I get it to its format, how do I include it in another program?
First, make a header file so the the other program can #include to know what functions can be used in your dylib.
Second, link to your dylib. If your dylib is named as libblah.dylib, you pass the -lblah flag to gcc.
Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?
/usr/lib or /usr/local/lib.
What is the difference (technically and practically) between a dynamic and static library?
Basically, for a static lib, the whole library is embedded into the file it "links" to.
How would I use third party libraries in my code (I'm staring at .dylib and .a files for the MySql C++ Connector)
See the 3rd answer.
Is writing them any different than a normal C++ program, minus the main() function?
Except for the obvious difference that a library provides services for other programs to use, usually (*) there isn't a difference.
* in gcc classes/functions are exported by default - this isn't the case in VC++, there you have to explicitly export using __declspec(export).
How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?
This depends on your compiler. In Visual Studio you specify this in your project configuration. In gcc to create a static library you compile your code normally and then package it in an archive using ar. To create a shared you compile first (with the -fpic flag to enable position independent code generation, a requirement for shared libraries), then use the -shared flag on the object files. More info can be found in the man pages.
Once I get it to its format, how do I include it in another program?
Again this is a little compiler-dependant. In VS, if it's a shared library, when including the class/function you wish to use it should be marked with a __declspec(import) (this is usually done with ifdefs) and you have to specify the .lib file of the shared library for linkage. For a static library you only have to specify the .lib file (no export/import needed since the code will end up in your executable).
In gcc you only need to specify the library which you link against using -llibrary_name.
In both cases you will need to provide your client some header files with the functions/classes that are intended for public use.
Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?
If it's your own library then it's up to you. Usually you can specify the linker additional folders to look in. We have a lib folder in our source tree where all .lib (or .a/.so) files end up and we add that folder to the additional folder to look in.
If you're shipping a library on UNIX the common place is usually /usr/lib (or /usr/local/lib), this is also where gcc searches in by default.
What is the difference (technically and practically) between a dynamic and static library?
When you link a program to static libraries the code of the libraries ends up in your executable. Practically this makes your executable larger and makes it harder to update/fix a static library for obvious reasons (requires a new version of your executable).
Shared libraries are separate from your executable and are referenced by your program and (usually) loaded at runtime when needed.
It's also possible to load shared libraries without linking to them. It requires more work since you have to manually load the shared library and any symbol you wish to use. On Windows this is done using LoadLibrary/GetProcAddress and on POSIX systems using dlsym/dlopen.
How would I use third party libraries in my code?
This is usually accomplished by including the necessary header files and linking with the appropriate library.
A simple example to link with a static library foo would look like this: gcc main.cpp -o main.o -L/folder/where/foo.a/is/at -lfoo.
Most open source projects have a readme that gives more detailed instructions, I'd suggest to take a look at it if there is one.
Is writing [libraries] any different than a normal C++ program, minus the main() function?
That depends on your definition of "different." From the language's point of view, you write a file or collection of files, don't put in a main() and you tell the compiler to generate a library instead of an executable.
However, designing libraries is much harder because you have no control over the code that calls you. Libraries need to be more robust against failure than normal code. You can't necessarily delete pointers somebody passes to your function. You can't tell what macros will mess with your code. You also can't accidentally pollute the global namespace (eg., don't put using namespace std at the beginning of your header files).
How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?
That depends on the compiler. In Visual C++ this is a project config setting. In gcc (going from memory) it's something like gcc -c foo.c -shared.
Once I get it to its format, how do I include it in another program?
That depends on your compiler and linker. You make sure the header files are available via a project setting or environment variable, and you make sure the binaries are available via a different project setting or compiler variable.
Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?
That depends on the operating system. In UNIX you're going to put things in places like /usr/lib, /usr/local/lib. On Windows people used to put DLLs in places like C:\WINDOWS but that's no longer allowed. Instead you put it in your program directory.
What is the difference (technically and practically) between a dynamic and static library?
Static libraries are the easier, original model. At compile time the linker puts all the functions from the library into your executable. You can ship the executable without the library, because the library is baked in.
Dynamic libraries (also called shared libraries) involve the compiler putting enough information in the executable that at runtime the linker will be able to find the correct libraries and call the methods in there. The libraries are shared across the whole system among the programs that use them. Using dynamic linking (dlsym(), et. al.) adds a few details to the picture.
How would I use third party libraries in my code (I'm staring at .dylib and .a files for the MySql C++ Connector)
That's going to depend on your platform, and unfortunately I can't tell you much about .dylib files. .a files are static libraries, and you simply need to add them to your final call to gcc (gcc main.c foo.a -o main if you know where foo.a is, or gcc main.c -lfoo -o main if the system knows where foo.a, foo.la, or foo.so are). Generally you make sure the compiler can find the library and leave the linker to do the rest.
The difference between a static and dynamic library is that the linking is done at compile time for static libraries, embedding the executable code into your binary, while for dynamic libraries linking is done dynamically at program start. The advantages are that the libraris can be separately distributed, updated and the code (memory) can be shared among several programs.
To use a library you simply provide -l to g++ for a lib.a or lib.so
I'm writing this to be more pragmatic than technically correct. It's enough to give you the general idea of what you're after.
Is writing them any different than a normal C++ program, minus the main() function?
For a static library, there's really not much difference.
For a dynamic library, the most likely difference you'll need to be aware of is that you may need to export the symbols you want to be available outside your library. Basically everything you don't export is invisible to users of your library. Exactly how you export, and whether you even need to by default, depends on your compiler.
For a dynamic library you also need to have all symbols resolved, which means the library can't depend on a function or variable that comes from outside the library. If my library uses a function called foo(), I need to include foo() in my library by writing it myself or by linking to another library that supplies it. I can't use foo() and just assume the user of my library will supply it. The linker won't know how to call a foo() that doesn't yet exist.
How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?
It's similar to how you turn test.cpp into test.exe - compile and link. You pass options to the compiler to tell it whether to create an executable, a static library, or a dynamic library.
Once I get it to its format, how do I include it in another program?
In your source code, you include header files necessary to use the library, much as you would include a header file for code that's not in a library. You'll also need to include the library on your link line, telling the linker where to find the library. For many systems, creating a dynamic library generates two files, the shared library and a link library. It's the link library that you include on the link line.
Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?
There is an environment variable that tells the linker where to look for libraries. The name of that variable is different from one system to another. You can also tell the linker about additional places to look.
What is the difference (technically and practically) between a dynamic and static library?
A static library gets copied into the thing it is linked to. An executable will include a copy of the static library and can be run on another machine without also copying the static library.
A dynamic library stays in a separate file. The executable loads that separate file when it runs. You have to distribute a copy of the dynamic library with your program or it won't run. You can also replace the dynamic library with a new version, and as long as the new library has the same interface it will still run with the old executable. It also may save space if several executables use the same dynamic library. In fact dynamic libraries are often called shared libraries.
How would I use third party libraries in my code
Same as you would use one you created yourself, as described above.