How to use static library(.a file) instead of a group of object files(.o) to create a shared library - c++

I have a shared library and I use it to create an executable binary. I can only control the build process of the library and the executable binary and not the source files involved. As expected, the source files in executable binary refer to a lot of functions from the library.
Currently the shared library is built using the objectfiles(.o) directly.
g++ -shared ${OBJECT_FILES} -o ${SHARED_LIBRARY}
I want to publish the object files too by grouping them in a static library (.a file or achive).
To save on space, I delete all the .o files on creation of the archive file. So now, the build command is
g++ -shared ${ACRCHIVE_FILE} -o ${SHARED_LIBRARY}
The library builds fine.
But when I try to build the executable binary by linking to this shared library, the symbols refereed to by the binary are not defined and it fails to link. (undefined reference to Context::Get())
By my understanding it should not matter if we create the shared library using .o files directly or an archive consisting of all the .o files, but evidently either it is not possible or I may be missing something.

Shared libraries libfoo.so should contain PIC code and static libraries libfoo.a contain ordinary non PIC code. So you cannot create a shared library from a static one.
Shared libraries want Position Independent Code because their segment(s) is mmap(2)-ed at nearly arbitrary and variable addresses - without MAP_FIXED ... See also ASLR.
You could in principle build a static library from PIC object files, but nobody does that in practice.
You might (if you really insist) make an ELF shared object made of non-PIC code, but the result would have very bad performance; the dynamic linker would have a lot of relocations, so most segments would be unshared and the dynamic linking would be very slow.
To compile foo.cc for a shared library into a PIC object file foo.pic.o :
g++ -Wall -c -O foo.cc -fPIC -o foo.pic.o
To compile it for a static library int an ordinary non-PIC object file foo.o :
g++ -Wall -c -O foo.cc -o foo.o
To make a shared library of foo.pic.o and bar.pic.o into libfoobar.so linking in some libdep.so shared library:
g++ -shared foo.pic.o bar.pic.o -ldep -o libfoobar.so
You often want to add more linking options when making a shared library like above, e.g. -Wl,-rpath,... or -Wl,-soname,....
BTW, it is not possible to link an archive libfoobar.a (even if it is made of PIC files) into a libfoobar.so because no name is undefined and requires linking some object files from libfoobar.a (maybe you could try to undefine some symbol symb with -u symb but I don't recommend doing that).
To make a static library of foo.o and bar.o into libfoobar.a :
ar cv libfoobar.a foo.o bar.o
Notice that ranlib (creating an index of the archive) is no more necessary since GNU ar does its job.
Read also ld.so(8), ldd(1), ld(1), ar(1), objdump(1), readelf(1), dlopen(3) (often, you need to link the main program with -rdynamic if it loads dlopen-ed plugins at runtime, to enable the plugin to find some symbols in the main program).
NB: I would just build a shared library and not care about static linking at all. Notice that on some systems, PIC has a slight cost (slightly bigger and/or slower code). AFAIK, PIC overhead is less costly on x86-64 than on 32 bits Linux x86 code. On some architectures and ABIs PIC may have negligible overhead (or perhaps even be more efficient than non position independent code).
References: Drepper's paper: How to Write Shared Libraries & Levine's book: Linkers and Loaders & Program Library HowTo

Related

How to embed a static library in a shared library (Linux)?

I have a static library which I do not have the source code, and need its symbols to be called dynamically from the LuaJIT FFI.
As it is static, I can't load it dynamically, so I'm trying to embed it in a shared library and then load the shared library at runtime.
Problem is that exported symbols of the static library are present in the symbols table of the shared lib, but are undefined.
nm libUSBDevices.a
shows a lot of lines, among which the symbols that interest me:
00001d80 T _ZN9USBDevice16FlightControllerC1EPKc
00001e30 T _ZN9USBDevice16FlightControllerD1Ev
00000140 T _ZN9USBDevice7AxisFctC1Ev
00000180 T _ZN9USBDevice7AxisFctclEv
Then I compiled the shared library using these two g++ commands :
g++ -m32 -c -Wall -Werror -fpic USBDevicesLoader.cpp -llibUSBDevices.a
which outputs USBDevicesLoader.o (USBDevicesLoader.cpp contains some exported functions which call symbols inside the static library, those ones are correctly present in the .so)
g++ -m32 -shared -o libUSBDevicesLoader.so USBDevicesLoader.o
This outputs the shared lib, but when loaded at runtime, it shows this:
[...] symbol lookup error: /home/me/USBDevices-loader/libUSBDevicesLoader.so: undefined symbol: _ZN9USBDevice16FlightControllerC1EPKc
And when I run nm on the shared lib, it shows the symbols as undefined:
U _ZN9USBDevice16FlightControllerC1EPKc
U _ZN9USBDevice7AxisFctclEv
I suggest the problem is somewhere in the compilation commands, I also tried to build the shared lib directly out of the .a without compiling the cpp first (just replace USBDevicesLoader.o in the second command by the .a, skip the first command) but the problem stays the same.
So, is there a way to embed all symbols of a static library (without having the source) in a dynamic lib which can then be loaded and used at runtime? Thanks
You can use the --whole-archive option to achieve this as seen here and in the docs:
--whole-archive: For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared library, forcing every object to be included in the resulting shared library. This option may be used more than once.
As an example:
g++ -shared -o libnew.so -Wl,--whole-archive libmylib_static.a -Wl,--no-whole-archive
You can then link to the shared libnew.so library as you would normally do.

How to make 64 shared 64-bit linux compatible library (*.so), for C++ code

My requirement is to work on some interface .h files. Right now I have .h and .cpp/.cc files in my project.
I need to compile it into shared 64-bit linux compatible library (*.so), using NetBeans/ Eclipse on Linux Fedora.
Since the GCC C++ ABI conventions did slightly change (in particular because of C++ standard libraries evolution, or name mangling convention) from one GCC version to the next (e.g. from g++-4.4 to g++-4.6) your shared library may be dependent upon the version of g++ used to build it
(In practice, the changes are often small inside g++, so you might be non affected)
If you want a symbol to be publicly accessible with dlsym you should preferably declare it extern "C" in your header files (otherwise you should mangle its name).
Regarding how to make a shared library, read documentation like Program Library Howto.
See also this question
And I suggest building your shared libraries with ordinary command-line tools (eg Makefile-s). Don't depend upon a complex IDE like NetBeans/ Eclipse to build them (they are invoking command-line utilities anyway).
If you are compiling a library from the 3 C++ source files called a.cc, b.cc, and c.cc respectively;
g++ -fpic -Wall -c a.cc
g++ -fpic -Wall -c b.cc
g++ -fpic -Wall -c c.cc
g++ -shared -Wl,-soname,libmylib.so.0 -o libmylib.so.0.0.0 a.o b.o c.o
Then you install the library using ldconfig, see man 8 ldconfig
you can then compile the program that uses the libary as follows (but be sure to prefix extern "C" before the class declarations in the header files included in the source code using the library.)
g++ -o myprog main.cc -lmylib
I have tried these compile options with my own sample code, and have been successful.
Basically What is covered in Shared Libraries applies to C++, just replace gcc with g++.
The theory behind all of this is;
Libraries are loaded dynamically when the program is first loaded, as can be confirmed by doing a system call trace on a running program, e.g. strace -o trace.txt ls which will dump a list of the system calls that the program made during execution into a file called trace.txt. At the top of the file you will see that the program (in this case ls) had indeed mmapped all the library's into memory.
Since libraries are loaded dynamically, it is unknown at link time where the library code will exist in the program's virtual address space during run time. Therefore library code must be compiled using position independent code - Hence the -fpic option which tells the translation stage to generate assembly code that has been coded with position independent code in mind. If you tell gcc/g++ to stop after the translation stage, with the -S (upper case S) option, and then look at resulting '.s' file, once with the -fpic option, and once without, you will see the difference (i.e. the dynamic code has #GOTPCREL and #PLT, at least on x86_64).
The linker, of course must be told to link all the ELF relocatatable object types into executable code suitable for use as a Linux shared library.

g++ trying (failing) to link statically to libstdc++ for shared object

I'm trying to create a shared object using a number of .O files created with the -fPIC command. When I run g++ with the -shared argument it appears to be trying to statically link to the libstdc++.a library, which of course fails. I'm trying to figure out why it's automatically trying to link statically when I'm not using the -static-stdc++ argument.
when I try creating the shared object I get the error ...libstdc++.a(ios) relocate R_x86_64_325 against 'vtable for std::ios_base': cannot be used when making a shared object
I ran G++ with the -V argument and received and can see LD receives the argument -lstdc++.
When linking together a single shared object, you need to do this from existing .o files. You can not do this from existing .so files; this would link those .so files to your .so file, but not into your .so file. So gcc seeks out and finds an archive of .o files (.a) and tries to link them. But since those are not compiled for relocation (no -fPIC), these can not be used to create .so files.
Your options are:
dynamically link your .so to the libstdc++ (and thus make it depending on the .so file that is installed in the system)
build .o files for libstdc++ and compile them with -fPIC then compile from those your .so file (here it does not matter if you use the .o files directly or an ar archive)
For the first (that I would recommend) option the following will suffice (it is from a makefile that I use for creating malloc/free intercepting .so files)
gcc -shared -lstdc++ -o your.so yourfiles.o
I'll bet it's finding the static library first in its library search path, or ONLY finding the static library. Make sure that the appropriate version of the shared version is installed and can be found. You can probably truss your g++ run to hunt down the order in which it's opening libraries.

Making a shared library from existing object files

I have a project in my IDE. I need to make a shared library of it to use in extensions. I don't want to make a copy of this project with shared-library settings. Is there any way to build a shared library using the object files (.o) from my already existing project? As I understand, I can write a makefile for this.
I assume you're on some sort of Unix and are probably using the GNU toolchain. In that case, to create a proper shared library, you'd need to compile your code using the position-independent code flags (-fpic or -fPIC) before you can create a shared library. Unless your .o files are already compiled with those flags, chances are you won't end up with a working shared lib.
If they already are compiled for position independent code, the usual g++ -shared ... should do the trick.
g++ -shared -fPIC -o myshared.so *.o

How do I source/link external functions in C or C++?

EDIT: I suppose I should clarify, in case it matters. I am on a AIX Unix box, so I am using VAC compilers - no gnu compilers.
End edit
I am pretty rusty in C/C++, so forgive me if this is a simple question.
I would like to take common functions out of a few of my C programs and put them in shared libraries or shared objects. If I was doing this in perl I would put my subs in a perl module and use that module when needed.
For the sake of an example, let's say I have this function:
int giveInteger()
{
return 1034;
}
Obviously this is not a real world example, but if I wanted to share that function, how would I proceed?
I'm pretty sure I have 2 options:
Put my shared function in a file, and have it compile with my main program at compile time. If I ever make changes to my shared function, I would have to recompile my main program.
Put my shared function in a file, and compile it as a shared library (if I have my terms correct), and have my main program link to that shared library. Any changes I make to my shared library (after compiling it) would be integrated into my main program at runtime without re-compiling my main program.
Am I correct on that thinking?
If so, how can I complish either/both of those methods? I've searched a lot and I seem to find information how how I could have my own program link to someone else's shared library, but not how to create my own shared functions and compile them in a way I can use them in my own program.
Thanks so much!
Brian
EDIT: Conclusion
Thanks everyone for your help! I thought I would add to this post what is working for me (for dynamic shared libraries on AIX) so that others can benefit:
I compile my shared functions:
xlc -c sharedFunctions.c -o sharedFunctions.o
Then make it a shared object:
xlc -qmkshrobj -qexpfile=exportlist sharedFunctions.o
xlc -G -o libsharedFunctions.so sharedFunctions.o -bE:exportlist
Then link it another program:
xlc -brtl -o mainProgram mainProgram.c -L. -lsharedFunctions
And another comment helped me find this link, which also helped:
http://publib.boulder.ibm.com/infocenter/comphelp/v7v91/topic/com.ibm.vacpp7a.doc/proguide/ref/compile_library.htm
Thanks again to all who helped me out!
Yeah you are correct. The first is called a static library, while the second is called a shared library, because the code is not bound to the executable at compile time, but everytime again when your program is loaded.
Static library
Compile your library's code as follows:
gcc -c *.c
The -c tells the program not to link the object file, but just leaves you with object files for each .c file that was compiled. Now, archive them into one static library:
ar rcs libmystuff.a *.o
man ar will tell you what the rcs options mean. Now, libmystuff.a is a archive file (you can open it with some zip-file viewers) which contain those object files, together with an index of symbols for each object file. You can link it to your program:
gcc *.c libmystuff.a -o myprogram
Now, your program is ready. Note that the order of where the static libraries appear in the command matter. See my Link order answer.
Shared library
For a shared library, you will create your library with
gcc -shared -o libmystuff.so *.c
That's all it takes, libmystuff.so is now a shared object file. If you want to link a program to it, you have to put it into a directory that is listed in the /etc/ld.so.conf file, or that is given by the -L switch to GCC, or listed in the LD_LIBRARY_PATH variable. When linking, you cut the lib prefix and .so suffix from the library name you tell gcc.
gcc -L. -lmystuff *.c -o myprogram
Internally, gcc will just pass your arguments to the GNU linker. You can see what arguments it pass using the -### option: Gcc will print the exact arguments given to each sub process.
For details about the linking process (how some stuff is done internally), view my Linux GCC linker answer.
You've got a third option. In general, your C++ compiler should be able to link C routines. The necessary options may vary from compiler to compiler, so R your fine M, but basically, you should be able to compile with g++ as here:
$ g++ -o myapp myapp.cpp myfunc.c giveint.c
... or compile separately
$ gcc -c myfunc.c
$ gcc -c giveint.c
$ g++ -c myapp.cpp
$ g++ -o myapp myapp.o myfunc.o
You also need to include your declaration of the functions; you do that in C++ as
extern "C" {
int myfunc(int,int);
int giveInterger(void);
}
You need to distinguish between recompiling and relinking.
If you put giveInteger() into a separate (archive) library, and then modify it later, you'll (obviously) need to recompile the source file in which it is defined, and relink all programs that use it; but you will not need to recompile such programs [1].
For a shared library, you'll need to recompile and relink the library; but you will not have to relink or recompile any of the programs which use it.
Building C++ shared libraries on AIX used to be complicated; you needed to use makeC++SharedLib shell script. But with VAC 5.0 and 6.0 it became quite easy. I believe all you need to do is [2]:
xlC -G -o shr.o giveInteger.cc
xlC -o myapp main.cc shr.o
[1] If you write correct Makefile (which is recommended practice), all of this will happen automatically when you type make.
[2] There is a certain feature of AIX which may complicate matters: by default shared libraries are loaded into memory, and "stick" there until subsequent reboot. So you may rebuild the shr.o, rerun the program, and observe "old" version of the library being executed. To prevent this, a common practice is to make shr.o world-unreadable:
chmod 0750 shr.o