GCC/LD Only link against direct dependencies - c++

I have scenario where I have a binary that depends on library A which in turn depends on library B.
I have built library A against library B, but none of library B:s symbols leak out of library A, everything is contained in the cpp-file.
Now I only want to link the binary against library A, since all of the symbols found in the binary can be satisfied by library A. Is this possible?
In the real application, library B is an implementation of a network protocol and I have a lot of binaries who link against the intermediate library. And I don't want the binaries to be aware of the different network protocols used.
Platform: Linux / GCC
Code:
liba/liba.h:
#ifndef LIBA_H
#define LIBA_H
int getANumber();
#endif
liba/liba.cpp:
#include "liba.h"
#include "../libb/libb.h"
int getANumber(){ return getBNumber(); }
libb/libb.h:
#ifndef LIBB_H
#define LIBB_H
int getBNumber();
#endif
libb/libb.cpp:
#include "libb.h"
int getBNumber(){ return 42; }
main.cpp:
#include "liba/liba.h"
#include <iostream>
int main(int argc, char** argv) {
std::cout << getANumber() << std::endl;
return 0;
}
commands:
~/libb/ $ g++ -shared -o libb.so libb.cpp
~/liba/ $ g++ -shared -o liba.so liba.cpp -L../libb -lb
~/ $ g++ -o main main.cpp -Lliba -la # fails
~/ # These two work, but I don't want to specify libb here.
~/ $ g++ -o main main.cpp -Lliba -la -Wl,-rpath-link,libb
~/ $ LD_LIBRARY_PATH=libb g++ -o main main.cpp -Lliba -la
What is the best way to solve this? Do I have to create it as a plugin?
Best regards,
Daniel

If you want to CHANGE which library you use at runtime, then you can't link against it directly, but use a "manual" loading. In other words, call dlopen and dlsym
This probably also means that you need a slightly different architecture in "liba", since each function in "libb" becomes a function pointer, so something along these lines:
int (*getBNumber)() = NULL;
void initialize()
{
void *handle = dlopen("libb", RTLD_NOW);
getBNumber = (int (*)())dlsym(handle, "getBNumber");
}
int getANumber(){ return getBNumber(); }
You'll need to set something up to call initialize at some point - or have a if (!initialized) initialize(); in each function.

Related

g++/clang override linker for specific functions

Is there a simple way to specify alternative functions (to the linker) to be used rather than the standard ones?
I have a wrapper around open/close/read/write system functions. I can test the functionality of the good path for these functions relatively easily.
But testing the potential errors is harder. To do this I need to get open/close/read/write to generate specific error codes for each test. Is there a way to link in alternative versions of these functions that I can then program to set the appropriate errno value before returning?
The linker option --wrap is for this purpose.
Some code that calls open:-
main.c
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#define N 32
int main(int argc, char *argv[])
{
char buf[N] = {0};
int fd = open(argv[1],O_RDONLY);
read(fd,buf,N - 1);
puts(buf);
close(fd);
return 0;
}
For simplicity it's a program, but it needn't be.
Using the true open:
$ gcc -Wall -c main.c
$ gcc -o prog main.o
$ echo "Hello world" > hw.txt
$ ./prog hw.txt
Hello world
Your alternative open must be called __wrap_open, and
must refer to the real open, if it needs to, as __real_open:
dbg_open.c
#include <stdio.h>
extern int __real_open(const char *path, int oflag);
int __wrap_open(const char *path, int oflag)
{
printf("In tester %s\n",__FUNCTION__);
return __real_open(path,oflag);
}
There's no need to recompile main.c to replace open with __wrap_open in prog; just
reuse main.o in a different linkage of prog
$ gcc -Wall -c dbg_open.c
$ gcc -o prog main.o dbg_open.o -Wl,--wrap=open
$ ./prog hw.txt
In tester __wrap_open
Hello world
If your __wrap_foo alternatives needed to knock out C++ functions
foo then you would need to obtain the mangling of foo to specify in the linkage
option --wrap=mangled-foo. But since you want to knock out system calls
you're spared that complication.

Singleton across compilation units: linking library vs linking objects

I apologize if the title is not fully self-explanatory. I'm trying to understand why my singleton factory pattern is not working properly, and I ran into a bizarre difference when using library vs linking single objects files.
Here's a simplified version of the code:
main.cpp
#include <iostream>
#include "bar.hpp"
int main (int /*argc*/, char** /*argv*/)
{
A::get().print();
return 0;
}
bar.hpp
#ifndef BAR_HPP
#define BAR_HPP
#include <iostream>
class A
{
public:
static A& get ()
{
static A a;
return a;
}
bool set(const int i)
{
m_i = i;
print();
return true;
}
void print ()
{
std::cout << "print: " << m_i << "\n";
}
private:
int m_i;
A () : m_i(0) {}
};
#endif // BAR_HPP
baz.hpp
#ifndef BAZ_HPP
#define BAZ_HPP
#include "bar.hpp"
namespace
{
static bool check = A::get().set(2);
}
#endif // BAZ_HPP
baz.cpp
#include "baz.hpp"
Now, I build my "project" in two ways:
Makefile:
all:
g++ -std=c++11 -c baz.cpp
g++ -std=c++11 -o test main.cpp baz.o
lib:
g++ -std=c++11 -c baz.cpp
ar rvs mylib.a baz.o
g++ -std=c++11 -o test main.cpp mylib.a
Here are the outputs I get:
$ make all
$ ./test
print: 2
print: 2
$ make lib
$ ./test
print: 0
In the first case the call to A::get().set(2) in baz.hpp takes place, and the same instantiation of A is then used in the main function, which therefore prints 2. In the second case, the call to A::get().set(2) in baz.hpp never takes place, and in the main function the value set by the constructor (that is, 0) is printed.
So finally I can ask my question: why is the behavior different in the two cases? I would expect that either both print 0 once or print 2 twice. I always assumed that a library was just a compact way to ship object files, and that the behavior of linking mylib.a should be the same as that of linking baz.o directly. Why isn't that the case?
Edit: the reason, as explained by Richard, is that no symbols defined in baz.cpp are required in main.cpp, so baz.o is not extracted from the library and linked. This raises another question: is there a workaround to ensure that the instruction A::get().set(2) is executed? I would like to avoid making the singleton a global object, but I'm not sure it's possible. I would also like to avoid to include baz.hpp in the main, since there may be many bazxyz.hpp and that would require main.cpp to know in advance all of them, defying the whole purpose of the factory-like registration process...
If this is to be a static library, then some module somewhere is going to have to address something in each implementation file of the objects that are going to register themselves with the factory.
A reasonable place for this would be in bar.cpp (which is a file you don't yet have). It would contain some or all of the implementation of A plus some means of calling the registration functions the widgets you're going to create.
Self-discovery only works if the object files are linked into the executable. This gives the c++ startup sequence a chance to know about and construct all objects with global linkage.

dlopen a dynamic library from a static library, when the dynamic library uses symbols of the static one

This question is closely related to dlopen a dynamic library from a static library linux C++, but contains a further complication (and uses C++ instead of C):
I have an application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so). In addition, the dynamic libraries call functions defined in the static one.
Is there a way to compile this without linking the dynamic libraries against the static one or vice versa?
Here comes what I tried so far, slightly modifying the example from the related question:
app.cpp:
#include "staticlib.hpp"
#include <iostream>
int main()
{
std::cout << "and the magic number is: " << doSomethingDynamicish() << std::endl;
return 0;
}
staticlib.hpp:
#ifndef __STATICLIB_H__
#define __STATICLIB_H__
int doSomethingDynamicish();
int doSomethingBoring();
#endif
staticlib.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingDynamicish()
{
void* handle = dlopen("./libdynlib.so",RTLD_NOW);
if(!handle)
{
std::cout << "could not dlopen: " << dlerror() << std::endl;
return 0;
}
typedef int(*dynamicfnc)();
dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
const char* err = dlerror();
if(err)
{
std::cout << "could not dlsym: " <<err << std::endl;
return 0;
}
return func();
}
staticlib2.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingBoring()
{
std::cout << "This function is so boring." << std::endl;
return 0;
}
dynlib.cpp:
#include "staticlib.hpp"
extern "C" int GetMeANumber()
{
doSomethingBoring();
return 1337;
}
and build:
g++ -c -o staticlib.o staticlib.cpp
g++ -c -o staticlib2.o staticlib2.cpp
ar rv libstaticlib.a staticlib.o staticlib2.o
ranlib libstaticlib.a
g++ -rdynamic -o app app.cpp libstaticlib.a -ldl
g++ -fPIC -shared -o libdynlib.so dynlib.cpp
When I run it with ./app I get
could not dlopen: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv
and the magic number is: 0
From the dlopen manual page:
If the executable was linked with the flag "-rdynamic" (or, synonymously, "--export-dynamic"), then the global symbols in the executable will also be used to resolve references in a dynamically loaded library.
That means that for the application to export its symbols for use in the dynamic library, you have to link your application with the -rdynamic flag.
Besides the problem described above, there is another problem and that has to do with the static library: The problem is namely that since the doSomethingBoring function is not called in your main program, the object file staticlib2.o from the static library is not linked.
The answer can be found in e.g. this old question, which tells you to add the --whole-archive linker flag:
g++ -rdynamic -o app app.cpp -L. \
-Wl,--whole-archive -lstaticlib \
-Wl,--no-whole-archive -ldl

Linking with multiple versions of a library

I have an application that statically links with version X of a library, libfoo, from thirdparty vendor, VENDOR1. It also links with a dynamic (shared) library, libbar, from a different thirdparty vendor, VENDOR2, that statically links version Y of libfoo from VENDOR1.
So libbar.so contains version Y of libfoo.a and my executable contains version X of libfoo.a
libbar only uses libfoo internally and there are no libfoo objects passed from my app to libbar.
There are no errors at build time but at runtime the app seg faults. The reason seems to be that version X uses structures that have a different size they version Y and the runtime linker seems to be mixing up which get used by which.
Both VENDOR1 & VENDOR2 are closed source so I cannot rebuild them.
Is there a way to build/link my app such that it always resolves to version X and libbar alway resolves to version Y and the two never mix?
Thanks for all the responses. I have a solution that seem to be working.
Here's the problem in detail with an example.
In main.c we have:
#include <stdio.h>
extern int foo();
int bar()
{
printf("bar in main.c called\n");
return 0;
}
int main()
{
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
In foo.c we have:
extern int bar();
int foo()
{
int x = bar();
return x;
}
In bar.c we have:
#include <stdio.h>
int bar()
{
printf("bar in bar.c called\n");
return 2;
}
Compile bar.c and foo.c:
$ gcc -fPIC -c bar.c
$ gcc -fPIC -c foo.c
Add bar.o to a static library:
$ ar r libbar.a bar.o
Now create a shared library using foo.o and link with static libbar.a
$ gcc -shared -o libfoo.so foo.o -L. -lbar
Compile main.c and link with shared library libfoo.so
$ gcc -o main main.c -L. -lfoo
Set LD_LIBRARY_PATH to find libfoo.so and run main:
$ setenv LD_LIBRARY_PATH `pwd`
$ ./main
bar in main.c called
result from foo is 0
bar in main.c called
result from bar is 0
Notice that the version of bar in main.c is called, not the version linked into the shared library.
In main2.c we have:
#include <stdio.h>
#include <dlfcn.h>
int bar()
{
printf("bar in main2.c called\n");
return 0;
}
int main()
{
int x;
int (*foo)();
void *handle = dlopen("libfoo.so", RTLD_GLOBAL|RTLD_LAZY);
foo = dlsym(handle, "foo");
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
Compile and run main2.c (notice we dont need to explicitly link with libfoo.so):
$ gcc -o main2 main2.c -ldl
$ ./main2
bar in bar.c called
result from foo is 2
bar in main2.c called
result from bar is 0
Now foo in the shared library calls bar in the shared library and main calls bar in main.c
I don't think this behaviour is intuitive and it is more work to use dlopen/dlsym, but it does resolve my problem.
Thanks again for the comments.
Try a partial link so that you have an object file "partial.o" with libbar and libfoo-Y. Use objcopy with "--localize-symbols " to make the symbols in partial.o from libfoo-Y local. You should be able to generate by running nm on libfoo-Y and massaging the output. Then take the modified partial.o and link it to your app.
I've done something similar with gcc toolchain on vxWorks where dynamic libs are not a complication but two versions of the same lib needed to link cleanly into a monolithic app.
Sorry no. My understanding of the way that Linux (and possibly most *nixes) is that that is not possible. The only 'solution' for your problem I can think of, is if you create a proxy app, which exposes what you need from libbar in the form of some IPC. You can then make that proxy load the correct version using LD_LIBRARY_PATH or something simmilar.

Dynamic loaded libraries and shared global symbols

Since I observed some strange behavior of global variables in my dynamically loaded libraries, I wrote the following test.
At first we need a statically linked library: The header test.hpp
#ifndef __BASE_HPP
#define __BASE_HPP
#include <iostream>
class test {
private:
int value;
public:
test(int value) : value(value) {
std::cout << "test::test(int) : value = " << value << std::endl;
}
~test() {
std::cout << "test::~test() : value = " << value << std::endl;
}
int get_value() const { return value; }
void set_value(int new_value) { value = new_value; }
};
extern test global_test;
#endif // __BASE_HPP
and the source test.cpp
#include "base.hpp"
test global_test = test(1);
Then I wrote a dynamically loaded library: library.cpp
#include "base.hpp"
extern "C" {
test* get_global_test() { return &global_test; }
}
and a client program loading this library: client.cpp
#include <iostream>
#include <dlfcn.h>
#include "base.hpp"
typedef test* get_global_test_t();
int main() {
global_test.set_value(2); // global_test from libbase.a
std::cout << "client: " << global_test.get_value() << std::endl;
void* handle = dlopen("./liblibrary.so", RTLD_LAZY);
if (handle == NULL) {
std::cout << dlerror() << std::endl;
return 1;
}
get_global_test_t* get_global_test = NULL;
void* func = dlsym(handle, "get_global_test");
if (func == NULL) {
std::cout << dlerror() << std::endl;
return 1;
} else get_global_test = reinterpret_cast<get_global_test_t*>(func);
test* t = get_global_test(); // global_test from liblibrary.so
std::cout << "liblibrary.so: " << t->get_value() << std::endl;
std::cout << "client: " << global_test.get_value() << std::endl;
dlclose(handle);
return 0;
}
Now I compile the statically loaded library with
g++ -Wall -g -c base.cpp
ar rcs libbase.a base.o
the dynamically loaded library
g++ -Wall -g -fPIC -shared library.cpp libbase.a -o liblibrary.so
and the client
g++ -Wall -g -ldl client.cpp libbase.a -o client
Now I observe: The client and the dynamically loaded library possess a different version of the variable global_test. But in my project I'm using cmake. The build script looks like this:
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
PROJECT(globaltest)
ADD_LIBRARY(base STATIC base.cpp)
ADD_LIBRARY(library MODULE library.cpp)
TARGET_LINK_LIBRARIES(library base)
ADD_EXECUTABLE(client client.cpp)
TARGET_LINK_LIBRARIES(client base dl)
analyzing the created makefiles I found that cmake builds the client with
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
This ends up in a slightly different but fatal behavior: The global_test of the client and the dynamically loaded library are the same but will be destroyed two times at the end of the program.
Am I using cmake in a wrong way? Is it possible that the client and the dynamically loaded library use the same global_test but without this double destruction problem?
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
CMake adds -rdynamic option allowing loaded library to resolve symbols in the loading executable... So you can see that this is what you don't want. Without this option it just misses this symbol by accident.
But... You should not do any stuff like that there. Your libraries and executable should
not share symbols unless they are really should be shared.
Always think of dynamic linking as static linking.
If using shared libraries you must define the stuff you want to export with macro like here. See DLL_PUBLIC macro definition in there.
By default, the linker won't combine a global variable (a 'D') in the base executable with one in a shared library. The base executable is special. There might be an obscure way to do this with one of those obscure control files that ld reads, but I sort of doubt it.
--export-dynamic will cause a.out 'D' symbols to be available to shared libs.
However, consider the process. Step 1: you create a DSO from a .o with a 'U' and a .a with a 'D'. So, the linker incorporates the symbol in the DSO. Step 2, you create the executable with a 'U' in one of the .o files, and 'D' in both a .a and the DSO. It will try to resolve using the left-to-right rule.
Variables, as opposed to functions, pose certain difficulties for the linker across modules in any case. A better practice is to avoid global var references across module boundaries, and use function calls. However, that would still fail for you if you put the same function in both the base executable and a shared lib.
My first question is if there is any particular reason for which you both statically and dynamically (via dlopen) link the same code?
For your problem: -rdynamic will export the symbols from your program and what probably is happening is that dynamic linker resolves all references to your global variable to the first symbol it encounters in symbol tables. Which one is that I don't know.
EDIT: given your purpose I would link your program that way:
g++ -Wall -g -ldl client.cpp -llibrary -L. -o client
You may need to fix the order.
I would advise to use a dlopen(... RTLD_LAZY|RTLD_GLOBAL); to merge global symbol tables.
I would propose to compile any .a static library which you plan to link to a dinamic library, with -fvisibility=hidden parameter, so:
g++ -Wall -fvisibility=hidden -g -c base.cpp