Compile a static library link with standard library (static) - c++

I'm trying to compile a static library (let's call it library.a). This library consumes resources of standard libraries. There is some way that the library can statically link a standard library.
I have proven something like:
g++ -c library -static-libstdc++ -o library.o
ar rcs library.o library.a
But if I do so there is no link to the standard libraries.
Then I have proved this way:
g++ library -static-stdlib -o library.o
ar rcs library.o library.a
But ask me to add a main function.
Is there any possibility of creating a static library by statically linking also standard libraries (std :: string, std :: vector, std :: cin, etc ...).
Thanks :)

Is there any possibility of creating a static library by statically linking also standard libraries
No, because you cannot link anything to a static library. You can link things only into
a file that is produced by the linker. Otherwise no linking is involved in its production.
The linker can produce two kinds of files: programs and shared libraries.
Programs and shared libraries are similar: they're both composed of executable code
that the program loader can load into a process. They are not similar to a static
library. A static library is produced by the archiving program ar and is just a
bag of object files - an archive - from which the linker can extract the ones it
needs to complete the linkage of a program or shared library.
A command like:
g++ -c library.cpp -static-libstdc++ -o library.o
just compiles library.cpp into library.o and ignores the linkage option -static-libstdc++
because -c means Just compile. Don't link
Your real problem only comes to light in one of your later comments1:
The problem is that I am doing a wrapper of a code in C++ to be able to use it in C,
in C I can not use the standard C++ libraries. The only way is to include inside
the library the functions that I use from the standard library.
Now since a shared library is something that the linker can produce, you can statically
link libstdc++ into a shared library. So, instead of making your C wrapper library
a static library, you could make it a shared libary.
It seems you already know how to wrap C++ code in a C API. So let's write such
a C wrapper that reverses the first N characters from an input buffer to an output buffer,
using the standard C++ library to do all the work:
reverse.h
#ifndef REVERSE_H
#define REVERSE_H
#ifdef __cplusplus
extern "C" {
#endif
void reverse(char const * in, char * out, unsigned len);
#ifdef __cplusplus
} // extern "C"
#endif
reverse.cpp
#include "reverse.h"
#include <string>
#include <algorithm>
extern "C" {
void reverse(char const * in, char * out, unsigned len)
{
std::string s{in,len};
std::reverse(s.begin(),s.end());
std::copy(s.begin(),s.end(),out);
}
} // extern "C"
Then a C program that calls reverse:
main.c
#include <stdio.h>
#include <reverse.h>
int main()
{
char in[] = "dlrow olleH";
reverse(in,in,sizeof(in) - 1);
puts(in);
return 0;
}
Now we'll build our shared wrapper library.
Compile our one library source file:
$ g++ -fPIC -Wall -Wextra -std=c++11 -c reverse.cpp
Notice -fPIC. All object files that we're going to link in a shared library
must be Position Independent Code. And as we're compiling one source file -
-c reverse.cpp - we can skip the -o option and accept the default, -o reverse.o
Then link our shared library:
$ g++ -shared -o libreverse.so reverse.o -static-libstdc++
Now the shared library is ./libreverse.so and it has no runtime dependency
on libstdc++:
$ ldd libreverse.so
linux-vdso.so.1 => (0x00007ffca98c9000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa178862000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa178498000)
/lib64/ld-linux-x86-64.so.2 (0x000055d17658c000)
Next compile our C program source file:
$ gcc -Wall -Wextra -I. -c main.c
And finally link the program with libreverse:
$ gcc -o prog main.o -L. -lreverse
This program ./prog has a runtime dependency on the shared library libreverse.so.
You can't statically link with libreverse.so because it's not a static library.
But libreverse.so has been statically linked with libstdc++.a, and has
a C API.
If libreverse.so was a serious library what we'd do now is is install it
one of the runtime loader's standard
search directories, so that programs could load it automatically at runtime. We
could do that like:
$ sudo cp libreverse.so /usr/local/lib/
$ sudo ldconfig
But since libreverse.so is just a toy library we won't modify our system for
it. Instead we'll just run ./prog like:
$ export LD_LIBRARY_PATH=. # For now, tell the loader to look for libraries here
$ ./prog
Hello world
So you can make a wrapper library with a C API and C++ internals, with libstdc++
statically linked, by making the wrapper library a shared library.
But why bother?
It seems that you want to bother because you believe that "in C I can not use the standard C++ libraries".
That is a misconception. Here's how you can build the same program with a static
wrapper library
First make your static wrapper library
$ rm libreverse.so # Better delete the old shared one just to avoid confusion.
$ g++ -Wall -Wextra -std=c++11 -c reverse.cpp
$ ar rcs libreverse.a reverse.o
Then compile your C source:
$ gcc -Wall -Wextra -I. -c main.c
At this point, you have an object file main.o, a static library libreverse.a
that contains (only) reverse.o, and to make prog you simply need to link main.o
and libreverse.a(reverse.o) together with the standard C library and the standard C++
library. There is no question of C not allowing you to do this. You finished
with C when you compiled main.c. These object files and libraries will be
linked by your system linker, which will not know or care what language any of
them were compiled from.
So you could invoke the linker via g++, like:
$ g++ -o prog main.o -L. -lreverse
and once more you have a program that does this:
$ ./prog
Hello world
Or you could invoke the linker via gcc, like:
$ gcc -o prog main.o -L. -lreverse -lstdc++
which is just the same linkage, with just the same result:
$ ./prog
Hello world
As you see, the one difference between linking via g++ rather than gcc is that
g++ automatically adds both the standard C library and the standard C++ library
to the linker's inputs, and gcc doesn't add the C++ library, so you have to
do it yourself.
For that matter, if you happened to have GNU Fortran installed on your computer
you could do the same linkage with it:
$ $ gfortran -o prog main.o -L. -lreverse -lstdc++
$ ./prog
Hello world
although nothing in prog was written in Fortran.
C doesn't stop you from linking libstdc++ with anything. A possible,
but unlikely, rational motive you might have for wishing to statically
link libstdc++ into a library with a C API is that you want somebody else to
be able to link programs with this library on a system that doesn't have libstdc++ or doesn't
have one that is ABI compatible with yours. If that is your motivation, then
make your library a shared one as shown above. Otherwise, just link libstdc++
with any program that depends on it.
[1] On Stackoverflow, always state your real problem in your question.
See The XY Problem

Related

Is it possible to link Linux static C++ library at runtime?

I know that the question is strange because we all know that a static .a library can be linked only at compile time.
I have a confidential code that I cannot share, but my question is what can let a code compiles and links against a static library successfully, but at runtime it complains about a missing symbol that is present in the .a lib which was linked with the code in the first place ?
What I can share is a little:
add_library(${NAME} STATIC ${NAME_SOURCES})
then this library is added to a global variable called LIBS that has all libraries needed to link to final binary.
I found the static library and I did an objdump on it and found the missing symbol.
So, it compiled the static lib then it compiled the final binary using that library, so why it complains about not finding it at runtime ?
There are many ways to cause this behaviour, here's one.
Suppose you have a shared library libA.so and a static library libB.a. Both export the same symbol foo.
// foo.c
void foo() {}
gcc -fPIC -shared -o libA.so foo.c
gcc -c foo.c && ar r libB.a foo.o
You link your program against both libraries:
// main.c
extern void foo();
int main() { foo(); }
gcc -o myexe main.c -L. -lA -lB -Wl,-rpath=. # dangerous, do not really do this
foo is resolved against libA.so.
Now suppose that at run time you have a different version of libA.so, one that does not export foo. Perhaps you have several versions of the library lying around, or maybe there is a software update.
// foo.c v2.0
void nofoo() {}
gcc -fPIC -shared -o libA.so foo.c
When you try to run myexe, the system will complain about
./myexe: symbol lookup error: ./myexe: undefined symbol: foo
myexe is linked againsg libB.a which defines foo, but as you can see, this doesn't help one little bit.

Using a shared library in another shared library

I am creating a shared library from a class from an example I got here C++ Dynamic Shared Library on Linux. I would like to call another shared library from the shared library created and then use it in the main program. So I have the myclass.so library and I want to call another library say anotherclass.so from the myclass.so library and then use this myclass.so library in the main program. Any idea on how I can do this please.
There is more than one way in which multiple shared libraries may be added to
the linkage of a program, if you are building all the libraries, and the program,
yourself.
The elementary way is simply to explicitly add all of the libraries to the
the linkage of the program, and this is the usual way if you are building only the
program and linking libraries built by some other party.
If an object file foo.o in your linkage depends on a library libA.so, then
foo.o should precede libA.so in the linkage sequence. Likewise if libA.so
depends on libB.so then libA.so should precede libB.so. Here's an illustration.
We'll make a shared library libsquare.so from the files:
square.h
#ifndef SQUARE_H
#define SQUARE_H
double square(double d);
#endif
and
square.cpp
#include <square.h>
#include <cmath>
double square(double d)
{
return pow(d,2);
}
Notice that the function square calls pow, which is declared in the
Standard header <cmath> and defined in the math library, libm.
Compile the source file square.cpp to a position-independent object file
square.o:
$ g++ -Wall -fPIC -I. -c square.cpp
Then link square.o into a shared library libsquare.so:
$ g++ -shared -o libsquare.so square.o
Next we'll make another shared library libcube.so from these files:
cube.h
#ifndef CUBE_H
#define CUBE_H
double cube(double d);
#endif
and
cube.cpp
#include <cube.h>
#include <square.h>
double cube(double d)
{
return square(d) * d;
}
See that the function cube calls square, so libcube.so is going to
depend on libsquare.so. Build the library as before:
$ g++ -Wall -fPIC -I. -c cube.cpp
$ g++ -shared -o libcube.so cube.o
We haven't bothered to link libsquare with libcube, even though libcube
depends on libsquare, and even though we could have, since we're building libcube.
For that matter, we didn't bother to link libm with libsquare. By default the
linker will let us link a shared library containing undefined references, and it
is perfectly normal. It won't let us link a program with undefined references.
Finally let's make a program, using these libraries, from this file:
main.cpp
#include <cube.h>
#include <iostream>
int main()
{
std::cout << cube(3) << std::endl;
return 0;
}
First, compile that source file to main.o:
$ g++ -Wall -I. -c main.cpp
Then link main.o with all three required libraries, making sure to list
the linker inputs in dependency order: main.o, libcube.so, libsquare.so, libm.so:
$ g++ -o prog main.o -L. -lcube -lsquare -lm
libm is a system library so there's no need to tell the linker where to look for
it. But libcube and libsquare aren't, so we need to tell the linker to look for
them in the current directory (.), because that's where they are. -L. does that.
We've successfully linked ./prog, but:
$ ./prog
./prog: error while loading shared libraries: libcube.so: cannot open shared object file: No such file or directory
It doesn't run. That's because the runtime loader doesn't know where to find libcube.so (or libsquare.so, though it didn't get that far).
Normally, when we build shared libraries we then install them in one of the loader's default
search directories (the same ones as the linker's default search directories), where they're available to any program, so this wouldn't happen. But I'm not
going to install these toy libraries on my system, so as a workaround I'll prompt the loader where to look
for them by setting the LD_LIBRARY_PATH in my shell.
$ export LD_LIBRARY_PATH=.
$ ./prog
27
Good. 3 cubed = 27.
Another and better way to link a program with shared libraries that aren't located
in standard system library directories is to link the program using the linker's
-rpath=DIR option. This will write some information into the executable to tell
the loader that it should search for required shared libraries in DIR before it tries
the default places.
Let's relink ./prog that way (first deleting the LD_LIBRARY_PATH from the shell so that it's not effective any more):
$ unset LD_LIBRARY_PATH
$ g++ -o prog main.o -L. -lcube -lsquare -lm -Wl,-rpath=.
And rerun:
$ ./prog
27
To use -rpath with g++, prefix it with -Wl, because it's an option for linker, ld,
that the g++ frontend doesn't recognise: -Wl tells g++ just to pass the
option straight through to ld.
I would like to add some points to the response of #Mike.
As you do not link libcube library with libsquare you are creating a sort of "incomplete library". When I say incomplete, I meant that when you link your application you must link it with both libcube and libsquare even though it does not use any symbol directly from libsquare.
It is better to link libcube directly with libsquare. This link will create the library with a NEEDED entry like:
readelf -d libcube.so
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libsquare.so]
Then when you link your application you can do:
g++ -o prog main.o -L. -lcube
Although, this will not link because the linker tries to locate the NEEDED library libsquare. You must precise its path by adding -Wl,-rpath-link=. to the linking command:
g++ -o prog main.o -L. -lcube -Wl,-rpath-link=.
Note: For runtime, you must still set LD_LIBRARY_PATH or link with rpath as mentioned by #Mike.
In your library if you are using any other shared library so simply your library user is also dependent on that library. While creating library you can use -l so the linker have notion for shared library and it will link when required.
But when you deliver your library as its dependent on some other library you need to export that too along with your and provide some environment variable or linker flag to load it from specified path (Your exported package). That will not lead any discrepancy other wise if its some standard library function user might get definition from his system's some other library and will lead in disastrous situation.
Simply use the library like you'd use it in any other application. You don't have to link to anotherclass.so, just to myclass.so.
However, you will have to make both libraries (myclass.so and anotherclass.so) available for your later application's runtime. If one of them is missing you'll get runtime errors just like it is with any other application.

Creating a static CUDA library to be linked with a C++ program

I am attempting to link a CUDA kernel with a C++ autotools project however cannot seem to pass the linking stage.
I have a file GPUFloydWarshall.cu that contains the kernel and a wrapper C function that I would like place into a library libgpu.a. This will be consistent with the remainder of the project. Is this at all possible?
Secondly, the library would then need to be linked to around ten other libraries for the main executable which at the moment using mpicxx.
Currently I am using/generating the below commands to compile and create the libgpu.a library
nvcc -rdc=true -c -o temp.o GPUFloydWarshall.cu
nvcc -dlink -o GPUFloydWarshall.o temp.o -L/usr/local/cuda/lib64 -lcuda -lcudart
rm -f libgpu.a
ar cru libgpu.a GPUFloydWarshall.o
ranlib libgpu.a
When this is all linked into the main executable I get the following error
problem/libproblem.a(libproblem_a-UTRP.o): In function `UTRP::evaluate(Solution&)':
UTRP.cpp:(.text+0x1220): undefined reference to `gpu_fw(double*, int)'
Th gpu_fw function is my wrapper function.
Is this at all possible?
Yes, it's possible. And creating a (non-CUDA) wrapper function around it makes it even easier. You can make your life easier still if you rely on C++ linking throughout (you mention a wrapper C function). mpicxx is a C++ compiler/linker alias, and cuda files (.cu) follow C++ compiler/linker behavior by default. Here's a very simple question that discusses building cuda code (encapsulated in a wrapper function) into a static library.
Secondly, the library would then need to be linked to around ten other libraries for the main executable which at the moment using mpicxx.
Once you have a C/C++ (non-CUDA) wrapper exposed in your library, linking should be no different than ordinary linking of ordinary libraries. You may still need to pass the cuda runtime libraries and any other cuda libraries you may be using in the link step, but this is the same conceptually as any other libraries your project may depend on.
EDIT:
It's not clear you need to use device linking for what you want to do. (But it's acceptable, it just complicates things a bit.) Anyway, your construction of the library is not quite correct, now that you have shown the command sequence. The device link command produces a device-linkable object, that does not include all necessary host pieces. To get everything in one place, we want to add both GPUFloydWarshall.o (which has the device-linked pieces) AND temp.o (which has the host code pieces) to the library.
Here's a fully worked example:
$ cat GPUFloydWarshall.cu
#include <stdio.h>
__global__ void mykernel(){
printf("hello\n");
}
void gpu_fw(){
mykernel<<<1,1>>>();
cudaDeviceSynchronize();
}
$ cat main.cpp
#include <stdio.h>
void gpu_fw();
int main(){
gpu_fw();
}
$ nvcc -rdc=true -c -o temp.o GPUFloydWarshall.cu
$ nvcc -dlink -o GPUFloydWarshall.o temp.o -lcudart
$ rm -f libgpu.a
$ ar cru libgpu.a GPUFloydWarshall.o temp.o
$ ranlib libgpu.a
$ g++ main.cpp -L. -lgpu -o main -L/usr/local/cuda/lib64 -lcudart
$ ./main
hello
$

How to force gcc to link an unused static library

I have a program and a static library:
// main.cpp
int main() {}
// mylib.cpp
#include <iostream>
struct S {
S() { std::cout << "Hello World\n";}
};
S s;
I want to link the static library (libmylib.a) to the program object (main.o), although the latter does not use any symbol of the former directly.
The following commands do not seem to the job with g++ 4.7. They will run without any errors or warnings, but apparently libmylib.a will not be linked:
g++ -o program main.o -Wl,--no-as-needed /path/to/libmylib.a
or
g++ -o program main.o -L/path/to/ -Wl,--no-as-needed -lmylib
Do you have any better ideas?
Use --whole-archive linker option.
Libraries that come after it in the command line will not have unreferenced symbols discarded. You can resume normal linking behaviour by adding --no-whole-archive after these libraries.
In your example, the command will be:
g++ -o program main.o -Wl,--whole-archive /path/to/libmylib.a
In general, it will be:
g++ -o program main.o \
-Wl,--whole-archive -lmylib \
-Wl,--no-whole-archive -llib1 -llib2
The original suggestion was "close":
How to force gcc to link unreferenced, static C++ objects from a library
Try this: -Wl,--whole-archive -lyourlib
I like the other answers better, but here is another "solution".
Use the ar command to extract all the .o files from the archive.
cd mylib ; ar x /path/to/libmylib.a
Then add all those .o files to the linker command
g++ -o program main.o mylib/*.o
If there is a specific function in the static library that is stripped by the linker as unused, but you really need it (one common example is JNI_OnLoad() function), you can force the linker to keep it (and naturally, all code that is called from this function). Add -u JNI_OnLoad to your link command.

g++: In what order should static and dynamic libraries be linked?

Let's say we got a main executable called "my_app" and it uses several other libraries: 3 libraries are linked statically, and other 3 are linked dynamically.
In which order should they be linked against "my_app"?
But in which order should these be linked?
Let's say we got libSA (as in Static A) which depends on libSB, and libSC which depends on libSB:
libSA -> libSB -> libSC
and three dynamic libraries:libDA -> libDB -> libDC (libDA is the basic, libDC is the highest)
in which order should these be linked? the basic one first or last?
g++ ... -g libSA libSB libSC -lDA -lDB -lDC -o my_app
seems like the currect order, but is that so? what if there are dependencies between any dynamic library to a static one, or the other way?
In the static case, it doesn't really matter, because you don't actually link static libraries - all you do is pack some object files together in one archive. All you have to is compile your object files, and you can create static libraries right away.
The situation with dynamic libraries is more convoluted, there are two aspects:
A shared library works exactly the same way as static library (except for shared segments, if they are present), which means, you can just do the same - just link your shared library as soon as you have the object files. This means for example symbols from libDA will appear as undefined in libDB
You can specify the libraries to link to on the command line when linking shared objects. This has the same effect as 1., but, marks libDB as needing libDA.
The difference is that if you use the former way, you have to specify all three libraries (-lDA, -lDB, -lDC) on the command line when linking the executable. If you use the latter, you just specify -lDC and it will pull the others automatically at link time. Note that link time is just before your program runs (which means you can get different versions of symbols, even from different libraries).
This all applies to UNIX; Windows DLL work quite differently.
Edit after clarification of the question:
Quote from the ld info manual.
The linker will search an archive only
once, at the location where it is
specified on the command line. If the
archive defines a symbol which was
undefined in some object which
appeared before the archive on the
command line, the linker will include
the appropriate file(s) from the
archive. However, an undefined symbol
in an object appearing later on the
command line will not cause the linker
to search the archive again.
See the `-(' option for a way to force
the linker to search archives multiple
times.
You may list the same archive multiple
times on the command line.
This type of archive searching is
standard for Unix linkers. However, if
you are using `ld' on AIX, note that
it is different from the behaviour of
the AIX linker.
That means:
Any static library or object that depends on other library should be placed before it in the command line. If static libraries depend on each other circularly, you can eg. use the -( command line option, or place the libraries on the command line twice (-lDA -lDB -lDA). The order of dynamic libraries doesn't matter.
This is the sort of question that's best solved by a trivial example. Really! Take 2 minutes, code up a simple example, and try it out! You'll learn something, and it's faster than asking.
For example, given files:
a1.cc
#include <stdio.h>
void a1() { printf("a1\n"); }
a2.cc
#include <stdio.h>
extern void a1();
void a2() { printf("a2\n"); a1(); }
a3.cc
#include <stdio.h>
extern void a2();
void a3() { printf("a3\n"); a2(); }
aa.cc
extern void a3();
int main()
{
a3();
}
Running:
g++ -Wall -g -c a1.cc
g++ -Wall -g -c a2.cc
g++ -Wall -g -c a3.cc
ar -r liba1.a a1.o
ar -r liba2.a a2.o
ar -r liba3.a a3.o
g++ -Wall -g aa.cc -o aa -la1 -la2 -la3 -L.
Shows:
./liba3.a(a3.o)(.text+0x14): In function `a3()':
/tmp/z/a3.C:4: undefined reference to `a2()'
Whereas:
g++ -Wall -g -c a1.C
g++ -Wall -g -c a2.C
g++ -Wall -g -c a3.C
ar -r liba1.a a1.o
ar -r liba2.a a2.o
ar -r liba3.a a3.o
g++ -Wall -g aa.C -o aa -la3 -la2 -la1 -L.
Succeeds. (Just the -la3 -la2 -la1 parameter order is changed.)
PS:
nm --demangle liba*.a
liba1.a:
a1.o:
U __gxx_personality_v0
U printf
0000000000000000 T a1()
liba2.a:
a2.o:
U __gxx_personality_v0
U printf
U a1()
0000000000000000 T a2()
liba3.a:
a3.o:
U __gxx_personality_v0
U printf
U a2()
0000000000000000 T a3()
From man nm:
If lowercase, the symbol is local; if uppercase, the symbol is global (external).
"T" The symbol is in the text (code) section.
"U" The symbol is undefined.
I worked in a project with a bunch of internal libraries that unfortunately depended on each other (and it got worse over time). We ended up "solving" this by setting up SCons to specify all libs twice when linking:
g++ ... -la1 -la2 -la3 -la1 -la2 -la3 ...
The dependencies for linking a library or executable have to be present at link-time, so you cannot link libXC before libXB is present. It doesn't matter if statically or dynamically.
Start with the most basic one, which has no (or just outside of your project) dependencies.
It's good practice to keep libraries independent of each other to avoid link order issues.