Undefined symbol when loading a shared library - c++

In my program I need to load a shared library dynamically with dlopen(). Both the program and the shared library are successfully cross-compiled for an ARM architecture with the cross-compiler installed on my x86. However, whenever the program tries to load the library at run time on ARM, it fails giving this error:
undefined symbol: _dl_hwcap
I cannot find the culprit of this error.
Let me give details on how the shared library (libmyplugin.so) is built on x86 first. I use the g++ cross-compiler as below:
/home/me/arm/gcc-arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ -march=armv7-a -mfloat-abi=hard -c -s -fPIC -o build/module1.o module1.cpp
/home/me/arm/gcc-arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ -march=armv7-a -mfloat-abi=hard -c -s -fPIC -o build/module2.o module2.cpp
/home/me/arm/gcc-arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++ -o dist/libmyplugin.so build/module1.o build/module2.o --sysroot /home/me/arm/sysroot/ -Wl,--no-as-needed -ldl -lX11 -lXext /home/me/arm/libstatic.a -shared -s -fPIC
Please pay attention to the following notes:
module1.cpp and module2.cpp are my source code files.
libstatic.a is a big archive of object .o files implementing the stuff directly invoked/referenced by module1.cpp and module2.cpp. These object files have been compiled by others for the same ARM architecture as mine, with the same compiler flags, but using a slightly more updated g++ compiler (v4.9 instead of my v4.8.3). Unfortunately, I have no control on the building of these objects.
--sysroot /home/me/arm/sysroot/ represents the remote filesystem of my ARM OS from which the local g++ cross-compiler can take the native libraries while linking.
-Wl,--no-as-needed -ldl -lX11 -lXext: these flags are required to force the dynamic loader to load the X11 libraries present on the system when my shared library is loaded by the program. In particular, --no-as-needed is required because the X11 libraries are NOT directly referenced by module1.o and module2.o; on the contrary the X11 libraries are referenced by the static library only.
Note that all the above setup works on x86. It's just that I don't understand what is the reason of the _dl_hwcap symbol not resolved when the program tried to load the library on ARM.
Do you have any idea how to investigate this issue?

There are a myriad of things that could be problematic, but here are four avenues of exploration. I am concentrating on the -shared in your link line, but the last item addresses that as well.
(A nice HOWTO on shared libraries is here:
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
a) Check your environment variable LD_LIBRARY_PATH. Since you aren't using RPATH to the linker (RPATH embeds a full path to the .so so you can find it at runtime), then the only way the linker can find your code is to search the LD_LIBRARY_PATH.
Make sure the .so or .0 you want is in the path.
b) Use the UNIX utility 'nm' to search .so (shared objects) and .a files for that symbol. For example, 'nm -D /usr/lib64/libpython2.6.so' will show all dynamic symbols
in the libpython.so, and you can look for symbols of interest:
For example, Is 'initgc' defined or used in libpython?
% nm -D /usr/lib64/libpython2.6.so | grep initgc
000003404300cf0 T initgc
The 'T' means TEXT or, yes, it is defined there. See if you can find the symbol in the module of interest using grep and nm. (A 'U' means undefined, which means it is defined in another module).
c) Another useful tool is 'ldd'. It shows all dynamic libraries that the library you are looking on depends on. For example:
% ldd /usr/lib64/libpython2.6.so
linux-vdso.so.1 => (0x00007fffa49ff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00000033f0200000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000033f0600000)
libutil.so.1 => /lib64/libutil.so.1 (0x00000033fea00000)
libm.so.6 => /lib64/libm.so.6 (0x00000033f0a00000)
libc.so.6 => /lib64/libc.so.6 (0x00000033efe00000)
/lib64/ld-linux-x86-64.so.2 (0x00000033efa00000)
If it can't find a library (because it's not on the LD_LIBRARY_PATH or wasn't specified in the RPATH), the library will turn up empty.
d) I am a little worried from your link line of seeing a '.a' file with a -shared option. Some compilers/linkers cannot use a '.a' (archive) file to create a '.so' file. '.so' files usually have to made from other '.so' files or '.o' files that have been compiled with -fPIC.
I would recommend (if you can), recompile /home/me/arm/libstatic.a so that it's a .so. If you can't do, you might have to make your final output a '.a' file as well. (In other words, get rid of the -shared command line option).
In summary: Check your LD_LIBRARY_PATH, use nm and ldd to look around at your .a and .so files, but I think the end result is that you may not be able to combine .so and .a files.
I hope this helps.

I think this symbol may be in the "ld-lsb" library needed by "Xext". On my system the library is a symlink "/lib64/ld-lsb-x86-64.so -> ld-linux-x86-64.so.2", but I am sure that is not the same on the arm. Maybe give it a whirl on your linker line?

Related

g++ compiling: can I link to a directory with symlinked binaries?

The below compiles:
g++ -L../../lib -o my_prog my_prog.cpp -ltest1 -ltest2
where ../../lib contains symlinks to libtest1.so and libtest2.so
But I am getting an error when I run the program: "error while loading shared libraries: libtest1.so: cannot open shared object file: No such file or directory" and am not sure if symlinking is the culprit.
Option -L is for linker ld to find .a and .so during linking.
Option -Wl,-rpath= is for the dynamic linker ld.so to find .so when the application is run. You need to use -Wl,-rpath= when a required shared library is not in (standard system) directories specified in /etc/ld.so.conf.
Use $ORIGIN dynamic linker variable to make rpath relative to the path of your executable:
g++ -L../../lib -Wl,-rpath='${ORIGIN}/../../lib' -o my_prog my_prog.cpp -ltest1 -ltest2
Be careful to make sure ${ORIGIN} is not expanded by the shell or your makefile (this is why it is in single quotes).
${ORIGIN}:
$ORIGIN and rpath
ld.so understands the string $ORIGIN (or equivalently ${ORIGIN}) in an rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory containing the application executable. Thus, an application located in somedir/app could be compiled with gcc -Wl,-rpath,'$ORIGIN/../lib' so that it finds an associated shared library in somedir/lib no matter where somedir is located in the directory hierarchy. This facilitates the creation of "turn-key" applications that do not need to be installed into special directories, but can instead be unpacked into any directory and still find their own shared libraries.
What happens at runtime is related to the rpath.
You may want (not really recommended, see this) to set your LD_LIBRARY_PATH appropriately before running your executable, or better yet you want to set the executable's rpath when linking it (e.g. by passing -Wl,--rpath $(realpath ../../lib/) to the g++ command doing the link.
Read Drepper's How to write shared libraries paper and the Program Library HowTo

g++ not find .so files

I am trying to generate a c++ library using the g++ compiler. My library has another C library as dependency and I have compiled it in order to obtain the .so files.
I have the following structure:
src:
include/linux:
libcustom.a
libcustom.la
libcustom.so
libcustom.so.0
libcustom.so.0.0.0
Now, when I have all the .o files of my cpp classes, and I want to link the library, I execute the following command:
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o mylibrary.so File1.o File2.o File3.o -L./include/linux -lc++ -lutil -lm -lcustom -Wl,-rpath='$ORIGIN/include/linux' -L/usr/lib/R/lib -lR
But it throws me the error:
libcustom.so.0: cannot open shared object file: No such file or directory
I am executing the command from the src directory.
I know it could be fixed editing the LD_LIBRARY_PATH, but the idea it is someone can use my library without the need of configuring anything, so I am trying to do that with the c++'s -rpath flag.
Any idea how can I fix it, or the reason for the error?
The error message you got seems to come from the run-time loader ld.so instead of the linker ld (I know the names are confusing). You have to distinguish between finding so's at link-time and at run-time. The -L flag you give at link-time has nothing to do with localizing the library at run-time.
Your rpath=./include/linux value is not correct, because dot is not recognized by the ld as relative path. Relative searching path should be given like
-Wl,-rpath='$ORIGIN/include/linux'
where the $ORIGIN represents the folder where your executable (not mylibrary.so) locates. Make sure to use single quote and not double quote because the string $ORIGIN should be passed to the linker literally and hard coded into the executable file.
More details goes
how to link to shared lib from shared lib with relative path
ld: Using -rpath,$ORIGIN inside a shared library (recursive)

cpp: usr/bin/ld: cannot find -l<nameOfTheLibrary>

I created a cpp project, which used a lib file named: libblpapi3_64.so
This file comes from a library which I download it from Internet.
My project runs without any error. So I update it to bitbucket.
Then my colleague downloads it and runs it at his own computer. But he gets an error:
usr/bin/ld: cannot find -lblpapi3_64.
In fact, I have copied it into my project repository. I mean I created a file named lib under my project and all lib files that I used are in it.
There are also other lib files such as liblog4cpp.a, but they are all good. Only the libblpapi3_64.so gets the error.
Is it because it's a .so file not .a file? Or there is other reason?
Btw, the file name of libblpapi3_64.so is green and others files(.a) is white. I think it's not a link file, it's the original file.
Briefly:
ld does not know about where your project libs are located. You have to place it into ld's known directories or specify the full path of your library by -L parameter to the linker.
To be able to build your program you need to have your library in /bin/ld search paths and your colleague too. Why? See detailed answer.
Detailed:
At first, we should understand what tools do what:
The compiler produces simple object files with unresolved symbols (it does not care about symbols so much at it's running time).
The linker combines a number of object and archive files, relocates their data and ties up symbol references into a single file: an executable or a library.
Let's start with some example. For example, you have a project which consists of 3 files: main.c, func.h and func.c.
main.c
#include "func.h"
int main() {
func();
return 0;
}
func.h
void func();
func.c
#include "func.h"
void func() { }
So, when you compile your source code (main.c) into an object file (main.o) it can't be run yet because it has unresolved symbols. Let's start from the beginning of producing an executable workflow (without details):
The preprocessor after its job produces the following main.c.preprocessed:
void func();
int main() {
func();
return 0;
}
and the following func.c.preprocessed:
void func();
void func() { }
As you may see in main.c.preprocessed, there are no connections to your func.c file and to the void func()'s implementation, the compiler simply does not know about it, it compiles all the source files separately. So, to be able to compile this project you have to compile both source files by using something like cc -c main.c -o main.o and cc -c func.c -o func.o, this will produce 2 object files, main.o and func.o. func.o has all it's symbols resolved because it has only one function which body is written right inside the func.c but main.o does not have func symbol resolved yet because it does not know where it is implemented.
Let's look what is inside func.o:
$ nm func.o
0000000000000000 T func
Simply, it contains a symbol which is in text code section so this is our func function.
And let's look inside main.o:
$ nm main.o
U func
0000000000000000 T main
Our main.o has an implemented and resolved static function main and we are able to see it in the object file. But we also see func symbol which marked as unresolved U, and thus we are unable to see its address offset.
For fixing that problem, we have to use the linker. It will take all the object files and resolve all these symbols (void func(); in our example). If the linker somehow is unable to do that it throws a error like unresolved external symbol: void func(). This may happen if you don't give the func.o object file to the linker. So, let's give all the object files we have to the linker:
ld main.o func.o -o test
The linker will go through main.o, then through func.o, try to resolve symbols and if it goes okay - put it's output to the test file. If we look at the produced output we will see all symbols are resolved:
$ nm test
0000000000601000 R __bss_start
0000000000601000 R _edata
0000000000601000 R _end
00000000004000b0 T func
00000000004000b7 T main
Here our job is done. Let's look the situation with dynamic(shared) libraries. Let's make a shared library from our func.c source file:
gcc -c func.c -o func.o
gcc -shared -fPIC -Wl,-soname,libfunc.so.1 -o libfunc.so.1.5.0 func.o
Voila, we have it. Now, let's put it into known dynamic linker library path, /usr/lib/:
sudo mv libfunc.so.1.5.0 /usr/lib/ # to make program be able to run
sudo ln -s libfunc.so.1.5.0 /usr/lib/libfunc.so.1 #creating symlink for the program to run
sudo ln -s libfunc.so.1 /usr/lib/libfunc.so # to make compilation possible
And let's make our project depend on that shared library by leaving func() symbol unresolved after compilation and static linkage process, creating an executable and linking it (dynamically) to our shared library (libfunc):
cc main.c -lfunc
Now if we look for the symbol in its symbols table we still have our symbol unresolved:
$ nm a.out | grep fun
U func
But this is not a problem anymore because func symbol will be resolved by dynamic loader before each program start. Okay, now let's back to the theory.
Libraries, in fact, are just the object files which are placed into a single archive by using ar tool with a single symbols table which is created by ranlib tool.
Compiler, when compiling object files, does not resolve symbols. These symbols will be replaced to addresses by a linker. So resolving symbols can be done by two things: the linker and dynamic loader:
The linker: ld, does 2 jobs:
a) For static libs or simple object files, this linker changes external symbols in the object files to the addresses of the real entities. For example, if we use C++ name mangling linker will change _ZNK3MapI10StringName3RefI8GDScriptE10ComparatorIS0_E16DefaultAllocatorE3hasERKS0_ to 0x07f4123f0.
b) For dynamic libs it only checks if the symbols can be resolved (you try to link with correct library) at all but does not replace the symbols by address. If symbols can't be resolved (for example they are not implemented in the shared library you are linking to) - it throws undefined reference to error and breaks up the building process because you try to use these symbols but linker can't find such symbol in it's object files which it is processing at this time. Otherwise, this linker adds some information to the ELF executable which is:
i. .interp section - request for an interpreter - dynamic loader to be called before executing, so this section just contains a path to the dynamic loader. If you look at your executable which depends on shared library (libfunc) for example you will see the interp section $ readelf -l a.out:
INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
ii. .dynamic section - a list of shared libraries which interpreter will be looking for before executing. You may see them by ldd or readelf:
$ ldd a.out
linux-vdso.so.1 => (0x00007ffd577dc000)
libfunc.so.1 => /usr/lib/libfunc.so.1 (0x00007fc629eca000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fefe148a000)
/lib64/ld-linux-x86-64.so.2 (0x000055747925e000)
$ readelf -d a.out
Dynamic section at offset 0xe18 contains 25 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libfunc.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Note that ldd also finds all the libraries in your filesystem while readelf only shows what libraries does your program need. So, all of these libraries will be searched by dynamic loader (next paragraph).
The linker works at build time.
Dynamic loader: ld.so or ld-linux. It finds and loads all the shared libraries needed by a program (if they were not loaded before), resolves the symbols by replacing them to real addresses right before the start of the program, prepares the program to run, and then runs it. It works after the build and before running the program. Less speaking, dynamic linking means resolving symbols in your executable before each program start.
Actually, when you run an ELF executable with .interp section (it needs to load some shared libraries) the OS (Linux) runs an interpreter at first but not your program. Otherwise you have an undefined behavior - you have symbols in your program but they are not defined by addresses which usually means that the program will be unable to work properly.
You may also run dynamic loader by yourself but it is unnecessary (binary is /lib/ld-linux.so.2 for 32-bit architecture elf and /lib64/ld-linux-x86-64.so.2 for 64-bit architecture elf).
Why does the linker claim that /usr/bin/ld: cannot find -lblpapi3_64 in your case? Because it tries to find all the libraries in it's known paths. Why does it search the library if it will be loaded during runtime? Because it needs to check if all the needed symbols can be resolved by this library and to put it's name into the .dynamic section for dynamic loader. Actually, the .interp section exists in almost every c/c++ elf because the libc and libstdc++ libraries are both shared, and compiler by default links any project dynamically to them. You may link them statically as well but this will enlarge the total executable size. So, if the shared library can't be found your symbols will remain unresolved and you will be UNABLE to run your application, thus it can't produce an executable. You may get the list of directories where libraries are usually searched by:
Passing a command to the linker in compiler arguments.
By parsing ld --verbose's output.
By parsing ldconfig's output.
Some of these methods are explained here.
Dynamic loader tries to find all the libraries by using:
DT_RPATH dynamic section of an ELF file.
DT_RUNPATH section of the executable.
LD_LIBRARY_PATH environment variable.
/etc/ld.so.cache - own cache file which contains a compiled list of candidate libraries previously found in the augmented library path.
Default paths: In the default path /lib, and then /usr/lib. If the binary was linked with -z nodeflib linker option, this step is skipped.
ld-linux search algorithm
Also, note please, that if we are talking about shared libraries, they are not named .so but in .so.version format instead. When you build your application the linker will look for .so file (which is usually a symlink to .so.version) but when you run your application the dynamic loader looks for .so.version file instead. For example, let's say we have a library test which version is 1.1.1 according to semver. In the filesystem it will look like:
/usr/lib/libtest.so -> /usr/lib/libtest.so.1.1.1
/usr/lib/libtest.so.1 -> /usr/lib/libtest.so.1.1.1
/usr/lib/libtest.so.1.1 -> /usr/lib/libtest.so.1.1.1
/usr/lib/libtest.so.1.1.1
So, to be able to compile you must have all of versioned files (libtest.so.1, libtest.so.1.1 and libtest.so.1.1.1) and a libtest.so file but for running your app you must have only 3 versioned library files listed first. This also explains why do Debian or rpm packages have devel-packages separately: normal one (which consists only of the files needed by already compiled applications for running them) which has 3 versioned library files and a devel package which has only symlink file for making it possible to compile the project.
Resume
After all of that:
You, your colleague and EACH user of your application code must have all the libraries in their system linker paths to be able to compile (build your application). Otherwise, they have to change Makefile (or compile command) to add the shared library location directory by adding -L<somePathToTheSharedLibrary> as argument.
After successful build you also need your library again to be able to run the program. Your library will be searched by dynamic loader (ld-linux) so it needs to be in it's paths (see above) or in system linker paths. In most of linux program distributions, for example, games from steam, there is a shell-script which sets the LD_LIBRARY_PATH variable which points to all shared libraries needed by the game.
You could look at our Rblapi package which uses this very library too.
Your basic question of "how do I make a library visible" really has two answers:
Use ld.so. The easiest way is to copy blpapi3_64.so to /usr/local/lib. If you then call ldconfig to update the cache you should be all set. You can test this via ldconfig -p | grep blpapi which should show it.
Use an rpath instruction when building your application; this basically encodes the path and makes you independent of ld.so.

Missing symbols from static library in linked executable

I have a problem with static library symbols missed in linked executable.
Here is the description of my problem:
I have static library built from several object files. These object files provides sevaral groups of symbols, associated with:
A set of C functions and structs (several object files). Let's call corresponding symbols level1 symbols.
A set of C++ wrapper classes for this C functions and structs (another object file). Let's call corresponding symbols level2 symbols.
A set of C++ wrapper classes inhereted from level 2 classes (another object file) with extended functionality. Let's call corresponding symbols level3 symbols.
Undeground C code uses several other project and external libs. Some of them are static, so currently this lib is static too.
Library is linked with executable file. This file used directly only level2 symbols from lib. But some dynamic libraries loaded by this executable during execution needs level3 symbols.
The problem is, that level3 symbols for some reason are missed into this executable(nm approved).
Library itself contains all groups of symbols. ALso there is another executable linked with this library and it also contains all groups of symbols. The main difference between these executables is that second executable (where all symbols are presented) uses leve3 symbols directly.
The whole project is built with CMake in debug configuration (this means "-g" option is presented in g++ commands). The underlying OS is GNU/Linux x86_64. g++ version is 4.4.
I've checked several similiar questions on StackOverflow but I haven't found any acceptable solution.
I've already tried several linking options to solve the problem (--export-dynamic, --whole_archive) but neither helps.
I'll be glad to see any ideas to solve this problem or, at least, possible reasons of this strange behaviour.
This is command line used to build executable. Command was generated by CMake. I only add --whole_archive option, then delete executable and rerun command. I also hope you will excuse me for replacing all project specific names with "???".
exec_name - name of executable we are talking about
lib_name - name of library we are talking about
/usr/bin/c++ - symlink to g++ v4.4 executable
/usr/bin/c++ -Wextra -g -fPIC CMakeFiles/exec_dir.dir/main.cpp.o CMakeFiles/exec_dir.dir/options.cpp.o CMakeFiles/exec_dir.dir/runtime.cpp.o CMakeFiles/exec_dir.dir/plugins.cpp.o CMakeFiles/exec_dir.dir/CServer.cpp.o -o exec_name -rdynamic ../lib/???/lib???.a --whole-archive ../../lib/???/???/lib_name.a ../lib/???/lib???.so ../../lib/???/???/lib???.a ../../???/???/lib???.a ../../lib/???/lib???.a -ldl -lboost_filesystem -lboost_signals -lboost_system -lboost_thread ../../lib/???/lib???.so /usr/local/ssl/lib64/libcrypto.so -ldl -luuid -lodbc ../lib/log/lib???.so ../lib/config/lib???a -lpthread ../../???/???/lib???.a -Wl,-rpath,/home/beduin/???/build/deb/???/lib/???:/home/beduin/???/build/deb/lib/???:/usr/local/ssl/lib64
Use -rdynamic -Wl,-whole-archive <all your libs> -Wl,-no-whole-archive <boost, pthread and so on> - one of your libs aren't within --whole-archive

Simple shared library

Is the STD library a shared library or what is it ? out of curiosity .
Are there any books describe in detail the shared , static libraries development ?
Are there any tutorial ?
p.s (i'm using netbeans , eclipse, anjuta) and the tutorials aren't useful as I'm trying to understand what's actually going on.
On my platform (Ubuntu Maverick) it is:
g++ test.cpp
ldd a.out
linux-vdso.so.1 => (0x00007fffee1ff000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f18755fd000)
libm.so.6 => /lib/libm.so.6 (0x00007f187537a000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f1875163000)
libc.so.6 => /lib/libc.so.6 (0x00007f1874de0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1875920000)
Note libstdc++.so.6 above.
With cmake creating a shared library is very easy.
1.
Install cmake 2.6 or later.
2.
Create a file test.cpp with the code for your library.
3.
Create a file CMakeLists.txt:
cmake_minimum_required(VERSION 2.6)
project(TEST)
add_library(test SHARED test.cpp)
4.
Run cmake to create a makefile:
cmake -G "Unix Makefiles"
5.
Run make to build your shared library.
With CMake you can also generate an Eclipse CDT project using the following command
cmake -G "Eclipse CDT4 - Unix Makefiles"
You can also find an interesting article on the topic with further references here.
1.) Is the STD library a shared library or what is it?
I have no idea. Could be either. Probably both. Does it matter? Unless you are dealing with something really exotic like a stand-alone statically linked binary for system rebuilding, as long as the compiler/system knows how to link it in, you are unlikely to be concerned with it.
In a nutshell, code can be in static libraries, in which case it's linked into the final (compiled/generated) executable and those binaries can become quite large. Or it can be in a shared library, in which case the library is dynamically loaded and multiple applications can (theoretically) share one common memory image. Unless you are doing something that is quite large, and that will be shared across multiple applications, I'd question the wisdom of going with shared libraries. The additional headaches, especially debugging headaches, are rarely worth it. And without multiple concurrently running applications, there's no savings...
To make a static library, I'd compile a bunch of files into object files... Than use ar and randlib. E.g.:
g++ -c foo1.C -o foo1.o
g++ -c foo2.C -o foo2.o
ar -rv libfoo.a foo1.o foo2.o
ranlib libfoo.a
Subsequently, I'd just link that library in:
g++ testfoo.C -o testfoo -L. -lfoo
Note that if you are using multiple libraries, the ordering of -lbar1 -lbar2 on that (g++ testfoo.C) command line is important! It determines which libraries can call functions/methods in other libraries. Circular dependencies are BAD!
With respect to foo1.o foo2.o files to ar, the ordering makes no difference.
Dynamic libraries...
Some time ago, under an ancient fedora core 3 system, I was playing around with shared libraries under linux. Back then, I would compile my shared library, say fooLibrary.c, with:
g++ -shared -Wl,-soname,libfooLibrary.so.1 -o libfooLibrary.so.1.0 -fPIC fooLibrary.c -ldl
At that time I was playing with LD_PRELOAD, so I had a little script to run my program that did:
export LD_PRELOAD=libfooLibrary.so ; export LD_LIBRARY_PATH=. ; ./myTestProgram
(Note that I did NOT want LD_PRELOAD set when running commands like g++, ls, cd, etc as I was intercepting system calls.)
(FYI: strace is also fun to play with... You should also check out ldd and nm.)
You may want to look at things like dlopen() and dlsym() -- for manually accessing dynamic libraries...
Oh, and the environment variable LD_LIBRARY_PATH adds directories to the default searchpath for dynamic libraries...
(With respect to debugging, let me just mention that when I intercepted malloc(), I found that somewhere inside dlopen()/dlsym() were calls to malloc(). Meaning that I needed to use malloc() before I could manually load the library that provided the real malloc(). Fun times debugging that one...)
PS One more thought: You may want to review the command-line options to gcc/g++. There's a lot of useful info in there...
http://gcc.gnu.org/onlinedocs/gcc-4.5.1/gcc/index.html#toc_Invoking-GCC