Linking with multiple versions of a library - c++

I have an application that statically links with version X of a library, libfoo, from thirdparty vendor, VENDOR1. It also links with a dynamic (shared) library, libbar, from a different thirdparty vendor, VENDOR2, that statically links version Y of libfoo from VENDOR1.
So libbar.so contains version Y of libfoo.a and my executable contains version X of libfoo.a
libbar only uses libfoo internally and there are no libfoo objects passed from my app to libbar.
There are no errors at build time but at runtime the app seg faults. The reason seems to be that version X uses structures that have a different size they version Y and the runtime linker seems to be mixing up which get used by which.
Both VENDOR1 & VENDOR2 are closed source so I cannot rebuild them.
Is there a way to build/link my app such that it always resolves to version X and libbar alway resolves to version Y and the two never mix?

Thanks for all the responses. I have a solution that seem to be working.
Here's the problem in detail with an example.
In main.c we have:
#include <stdio.h>
extern int foo();
int bar()
{
printf("bar in main.c called\n");
return 0;
}
int main()
{
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
In foo.c we have:
extern int bar();
int foo()
{
int x = bar();
return x;
}
In bar.c we have:
#include <stdio.h>
int bar()
{
printf("bar in bar.c called\n");
return 2;
}
Compile bar.c and foo.c:
$ gcc -fPIC -c bar.c
$ gcc -fPIC -c foo.c
Add bar.o to a static library:
$ ar r libbar.a bar.o
Now create a shared library using foo.o and link with static libbar.a
$ gcc -shared -o libfoo.so foo.o -L. -lbar
Compile main.c and link with shared library libfoo.so
$ gcc -o main main.c -L. -lfoo
Set LD_LIBRARY_PATH to find libfoo.so and run main:
$ setenv LD_LIBRARY_PATH `pwd`
$ ./main
bar in main.c called
result from foo is 0
bar in main.c called
result from bar is 0
Notice that the version of bar in main.c is called, not the version linked into the shared library.
In main2.c we have:
#include <stdio.h>
#include <dlfcn.h>
int bar()
{
printf("bar in main2.c called\n");
return 0;
}
int main()
{
int x;
int (*foo)();
void *handle = dlopen("libfoo.so", RTLD_GLOBAL|RTLD_LAZY);
foo = dlsym(handle, "foo");
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
Compile and run main2.c (notice we dont need to explicitly link with libfoo.so):
$ gcc -o main2 main2.c -ldl
$ ./main2
bar in bar.c called
result from foo is 2
bar in main2.c called
result from bar is 0
Now foo in the shared library calls bar in the shared library and main calls bar in main.c
I don't think this behaviour is intuitive and it is more work to use dlopen/dlsym, but it does resolve my problem.
Thanks again for the comments.

Try a partial link so that you have an object file "partial.o" with libbar and libfoo-Y. Use objcopy with "--localize-symbols " to make the symbols in partial.o from libfoo-Y local. You should be able to generate by running nm on libfoo-Y and massaging the output. Then take the modified partial.o and link it to your app.
I've done something similar with gcc toolchain on vxWorks where dynamic libs are not a complication but two versions of the same lib needed to link cleanly into a monolithic app.

Sorry no. My understanding of the way that Linux (and possibly most *nixes) is that that is not possible. The only 'solution' for your problem I can think of, is if you create a proxy app, which exposes what you need from libbar in the form of some IPC. You can then make that proxy load the correct version using LD_LIBRARY_PATH or something simmilar.

Related

Linking on different version of shared libraries

I have two versions of a a shared library:
library version 2:
simple.h
#pragma once
int first(int x);
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first##LIBSIMPLE_2.0");
int first_2_0(int x)
{
int y;
printf("lib: %d\n", y);
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
linker version script file:
LIBSIMPLE_1.0{
global:
first;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.2.0.0
And library version 3:
simple.h
#pragma once
#ifdef SIMPLELIB_VERSION_3_0
int first(int x, int normfactor);
#else
int first(int x);
#endif //SIMPLELIB_VERSION_3_0
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first#LIBSIMPLE_2.0");
int first_2_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
__asm__(".symver first_3_0,first##LIBSIMPLE_3.0");
int first_3_0(int x, int normfactor)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * normfactor;
}
Linker version script file:
LIBSIMPLE_1.0{
global:
first; second;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
LIBSIMPLE_3.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.3.0.0
So i end up with having two different libraries. Next i create a simple application, that eventually i want to link to a library version 3, so in it i use function first() that takes two arguments:
main.c
#include <stdio.h>
#include "simple.h"
int main(int argc, char* argv[])
{
int nFirst = first(1, 10);
printf("First(1) = %d\n", nFirst);
}
I compile app with next commands:
gcc -g -Wall -DSIMPLELIB_VERSION_3_0 -c main.c
And then, by accident, instead of linking to a library version 3, i linked against library version 2. I expected linking to fail, but it went through, and application was working.
gcc main.o -Wl,-L. -lsimple.2.0.0 -Wl,-R. -o demo
So my questions are:
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
p.s. Sorry for a big amount of code. I was trying to make it clear.
Thanks
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
Yes, since plain C does not have function overload there is no need for mangling and as a consequence only function name will be used as a symbol for linking. In the end your application code wants to link with function and your library code exports function and this is enough to keep linker happy (even though it is not valid from binary interface perspective).
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
Yes, this problem should not occur in C++ because of name mangling. However, this is true only if you have both your application code and library code in C++ or if you bridge your C and C++ code the right way.
It is hard to say (without full listing) what happened in your case when you used g++ but from the looks of it you ended up having application code in C++ and library code still in C. If that is the case your application code will now want to link with mangled function while your library code still exports unmangled function.
To verify this you can inspect your object file with something like:
nm main.o
... and see exactly what kind of symbol does it want. If you will get something like this:
...
U _Z3functionii
...
... instead of:
...
U function
...
... then that is the case.
To "fix" this and make your C++ application code link with unmangled function from library code you'll need to declare your function prototype as extern "C".

Static variable is initialized twice

Consider I have a static variable in a compilation unit which ends up in a static library libA. I then have another compilation unit accessing this variable which ends up in a shared library libB.so (so libA must be linked into libB). Finally I have a main function also accessing the static variable from A directly and having a dependency to libB (so I link against libA and libB).
I then observe, that the static variable is initialized twice, i.e. its constructor is run twice! This doesn't seem to be right. Shouldn't the linker recognize both variables to be the same and optimize them as one?
To make my confusion perfect, I see it is run twice with the same address! So maybe the linker did recognize it, but did not remove the second call in the static_initialization_and_destruction code?
Here's a showcase:
ClassA.hpp:
#ifndef CLASSA_HPP
#define CLASSA_HPP
class ClassA
{
public:
ClassA();
~ClassA();
static ClassA staticA;
void test();
};
#endif // CLASSA_HPP
ClassA.cpp:
#include <cstdio>
#include "ClassA.hpp"
ClassA ClassA::staticA;
ClassA::ClassA()
{
printf("ClassA::ClassA() this=%p\n", this);
}
ClassA::~ClassA()
{
printf("ClassA::~ClassA() this=%p\n", this);
}
void ClassA::test()
{
printf("ClassA::test() this=%p\n", this);
}
ClassB.hpp:
#ifndef CLASSB_HPP
#define CLASSB_HPP
class ClassB
{
public:
ClassB();
~ClassB();
void test();
};
#endif // CLASSB_HPP
ClassB.cpp:
#include <cstdio>
#include "ClassA.hpp"
#include "ClassB.hpp"
ClassB::ClassB()
{
printf("ClassB::ClassB() this=%p\n", this);
}
ClassB::~ClassB()
{
printf("ClassB::~ClassB() this=%p\n", this);
}
void ClassB::test()
{
printf("ClassB::test() this=%p\n", this);
printf("ClassB::test: call staticA.test()\n");
ClassA::staticA.test();
}
Test.cpp:
#include <cstdio>
#include "ClassA.hpp"
#include "ClassB.hpp"
int main(int argc, char * argv[])
{
printf("main()\n");
ClassA::staticA.test();
ClassB b;
b.test();
printf("main: END\n");
return 0;
}
I then compile and link as follows:
g++ -c ClassA.cpp
ar rvs libA.a ClassA.o
g++ -c ClassB.cpp
g++ -shared -o libB.so ClassB.o libA.a
g++ -c Test.cpp
g++ -o test Test.cpp libA.a libB.so
Output is:
ClassA::ClassA() this=0x804a040
ClassA::ClassA() this=0x804a040
main()
ClassA::test() this=0x804a040
ClassB::ClassB() this=0xbfcb064f
ClassB::test() this=0xbfcb064f
ClassB::test: call staticA.test()
ClassA::test() this=0x804a040
main: END
ClassB::~ClassB() this=0xbfcb064f
ClassA::~ClassA() this=0x804a040
ClassA::~ClassA() this=0x804a040
Can somebody please explain what is going on here? What is the linker doing? How can the same variable be initialized twice?
You are including libA.a into libB.so. By doing this, both libB.so and libA.a contain ClassA.o, which defines the static member.
In the link order you specified, the linker pulls in ClassA.o from the static library libA.a, so ClassA.o initialization code is run before main(). When the first function in the dynamic libB.so is accessed, all initializers for libB.so are run. Since libB.so includes ClassA.o, ClassA.o's static initializer must be run (again).
Possible fixes:
Don't put ClassA.o into both libA.a and libB.so.
g++ -shared -o libB.so ClassB.o
Don't use both libraries; libA.a is not needed.
g++ -o test Test.cpp libB.so
Applying either of the above fixes the problem:
ClassA::ClassA() this=0x600e58
main()
ClassA::test() this=0x600e58
ClassB::ClassB() this=0x7fff1a69f0cf
ClassB::test() this=0x7fff1a69f0cf
ClassB::test: call staticA.test()
ClassA::test() this=0x600e58
main: END
ClassB::~ClassB() this=0x7fff1a69f0cf
ClassA::~ClassA() this=0x600e58
Can somebody please explain what is going on here?
It's complicated.
First, the way that you linked your main executable and the shared library causes two instances of staticA (and all the other code from ClassA.cpp) to be present: one in the main executable, and another in libB.so.
You can confirm this by running
nm -AD ./test ./libB.so | grep staticA
It is then not very surprising that the ClassA constructor for the two instances runs two times, but it is still surprising that the this pointer is the same (and corresponds to staticA in the main executable).
That is happening because the runtime loader (unsuccessfully) tries to emulate the behavior of linking with archive libraries, and binds all references to staticA to the first globally-exported instance it observes (the one in test).
So what can you do to fix this? That depends on what staticA actually represents.
If it is some kind of singleton, that should only exist once in any program, then the easy solution is make it so that there is only a single instance of staticA. And a way to do that is to require that any program that uses libB.so also links against libA.a, and not link libB.so against libA.a. That will eliminate the instance of sttaicA inside libB.so. You've claimed that "libA must be linked into libB", but that claim is false.
Alternatively, if you build libA.so instead of libA.a, then you can link libB.so against libA.so (so libB.so is self-contained). If the main application also links against libA.so, that wouldn't be a problem: there will only be one instance of staticA inside libA.so, not matter how many times that library is used.
On the other hand, if staticA represents some kind of internal implementation detail, and you are ok with having two instances of it (so long as they don't interfere with each other), then the solution is to mark all of ClassA symbols with hidden visibility, as this answer suggests.
Update:
why the linker does not eliminate the second instance of staticA from the executable.
Because the linker does what you told it to do. If you change your link command line to:
g++ -o test Test.cpp libB.so libA.a
then the linker should not link ClassA into the main executable. To understand why the order of libraries on command line matters, read this.

dlopen a dynamic library from a static library, when the dynamic library uses symbols of the static one

This question is closely related to dlopen a dynamic library from a static library linux C++, but contains a further complication (and uses C++ instead of C):
I have an application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so). In addition, the dynamic libraries call functions defined in the static one.
Is there a way to compile this without linking the dynamic libraries against the static one or vice versa?
Here comes what I tried so far, slightly modifying the example from the related question:
app.cpp:
#include "staticlib.hpp"
#include <iostream>
int main()
{
std::cout << "and the magic number is: " << doSomethingDynamicish() << std::endl;
return 0;
}
staticlib.hpp:
#ifndef __STATICLIB_H__
#define __STATICLIB_H__
int doSomethingDynamicish();
int doSomethingBoring();
#endif
staticlib.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingDynamicish()
{
void* handle = dlopen("./libdynlib.so",RTLD_NOW);
if(!handle)
{
std::cout << "could not dlopen: " << dlerror() << std::endl;
return 0;
}
typedef int(*dynamicfnc)();
dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
const char* err = dlerror();
if(err)
{
std::cout << "could not dlsym: " <<err << std::endl;
return 0;
}
return func();
}
staticlib2.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingBoring()
{
std::cout << "This function is so boring." << std::endl;
return 0;
}
dynlib.cpp:
#include "staticlib.hpp"
extern "C" int GetMeANumber()
{
doSomethingBoring();
return 1337;
}
and build:
g++ -c -o staticlib.o staticlib.cpp
g++ -c -o staticlib2.o staticlib2.cpp
ar rv libstaticlib.a staticlib.o staticlib2.o
ranlib libstaticlib.a
g++ -rdynamic -o app app.cpp libstaticlib.a -ldl
g++ -fPIC -shared -o libdynlib.so dynlib.cpp
When I run it with ./app I get
could not dlopen: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv
and the magic number is: 0
From the dlopen manual page:
If the executable was linked with the flag "-rdynamic" (or, synonymously, "--export-dynamic"), then the global symbols in the executable will also be used to resolve references in a dynamically loaded library.
That means that for the application to export its symbols for use in the dynamic library, you have to link your application with the -rdynamic flag.
Besides the problem described above, there is another problem and that has to do with the static library: The problem is namely that since the doSomethingBoring function is not called in your main program, the object file staticlib2.o from the static library is not linked.
The answer can be found in e.g. this old question, which tells you to add the --whole-archive linker flag:
g++ -rdynamic -o app app.cpp -L. \
-Wl,--whole-archive -lstaticlib \
-Wl,--no-whole-archive -ldl

GCC/LD Only link against direct dependencies

I have scenario where I have a binary that depends on library A which in turn depends on library B.
I have built library A against library B, but none of library B:s symbols leak out of library A, everything is contained in the cpp-file.
Now I only want to link the binary against library A, since all of the symbols found in the binary can be satisfied by library A. Is this possible?
In the real application, library B is an implementation of a network protocol and I have a lot of binaries who link against the intermediate library. And I don't want the binaries to be aware of the different network protocols used.
Platform: Linux / GCC
Code:
liba/liba.h:
#ifndef LIBA_H
#define LIBA_H
int getANumber();
#endif
liba/liba.cpp:
#include "liba.h"
#include "../libb/libb.h"
int getANumber(){ return getBNumber(); }
libb/libb.h:
#ifndef LIBB_H
#define LIBB_H
int getBNumber();
#endif
libb/libb.cpp:
#include "libb.h"
int getBNumber(){ return 42; }
main.cpp:
#include "liba/liba.h"
#include <iostream>
int main(int argc, char** argv) {
std::cout << getANumber() << std::endl;
return 0;
}
commands:
~/libb/ $ g++ -shared -o libb.so libb.cpp
~/liba/ $ g++ -shared -o liba.so liba.cpp -L../libb -lb
~/ $ g++ -o main main.cpp -Lliba -la # fails
~/ # These two work, but I don't want to specify libb here.
~/ $ g++ -o main main.cpp -Lliba -la -Wl,-rpath-link,libb
~/ $ LD_LIBRARY_PATH=libb g++ -o main main.cpp -Lliba -la
What is the best way to solve this? Do I have to create it as a plugin?
Best regards,
Daniel
If you want to CHANGE which library you use at runtime, then you can't link against it directly, but use a "manual" loading. In other words, call dlopen and dlsym
This probably also means that you need a slightly different architecture in "liba", since each function in "libb" becomes a function pointer, so something along these lines:
int (*getBNumber)() = NULL;
void initialize()
{
void *handle = dlopen("libb", RTLD_NOW);
getBNumber = (int (*)())dlsym(handle, "getBNumber");
}
int getANumber(){ return getBNumber(); }
You'll need to set something up to call initialize at some point - or have a if (!initialized) initialize(); in each function.

Dynamic loaded libraries and shared global symbols

Since I observed some strange behavior of global variables in my dynamically loaded libraries, I wrote the following test.
At first we need a statically linked library: The header test.hpp
#ifndef __BASE_HPP
#define __BASE_HPP
#include <iostream>
class test {
private:
int value;
public:
test(int value) : value(value) {
std::cout << "test::test(int) : value = " << value << std::endl;
}
~test() {
std::cout << "test::~test() : value = " << value << std::endl;
}
int get_value() const { return value; }
void set_value(int new_value) { value = new_value; }
};
extern test global_test;
#endif // __BASE_HPP
and the source test.cpp
#include "base.hpp"
test global_test = test(1);
Then I wrote a dynamically loaded library: library.cpp
#include "base.hpp"
extern "C" {
test* get_global_test() { return &global_test; }
}
and a client program loading this library: client.cpp
#include <iostream>
#include <dlfcn.h>
#include "base.hpp"
typedef test* get_global_test_t();
int main() {
global_test.set_value(2); // global_test from libbase.a
std::cout << "client: " << global_test.get_value() << std::endl;
void* handle = dlopen("./liblibrary.so", RTLD_LAZY);
if (handle == NULL) {
std::cout << dlerror() << std::endl;
return 1;
}
get_global_test_t* get_global_test = NULL;
void* func = dlsym(handle, "get_global_test");
if (func == NULL) {
std::cout << dlerror() << std::endl;
return 1;
} else get_global_test = reinterpret_cast<get_global_test_t*>(func);
test* t = get_global_test(); // global_test from liblibrary.so
std::cout << "liblibrary.so: " << t->get_value() << std::endl;
std::cout << "client: " << global_test.get_value() << std::endl;
dlclose(handle);
return 0;
}
Now I compile the statically loaded library with
g++ -Wall -g -c base.cpp
ar rcs libbase.a base.o
the dynamically loaded library
g++ -Wall -g -fPIC -shared library.cpp libbase.a -o liblibrary.so
and the client
g++ -Wall -g -ldl client.cpp libbase.a -o client
Now I observe: The client and the dynamically loaded library possess a different version of the variable global_test. But in my project I'm using cmake. The build script looks like this:
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
PROJECT(globaltest)
ADD_LIBRARY(base STATIC base.cpp)
ADD_LIBRARY(library MODULE library.cpp)
TARGET_LINK_LIBRARIES(library base)
ADD_EXECUTABLE(client client.cpp)
TARGET_LINK_LIBRARIES(client base dl)
analyzing the created makefiles I found that cmake builds the client with
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
This ends up in a slightly different but fatal behavior: The global_test of the client and the dynamically loaded library are the same but will be destroyed two times at the end of the program.
Am I using cmake in a wrong way? Is it possible that the client and the dynamically loaded library use the same global_test but without this double destruction problem?
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
CMake adds -rdynamic option allowing loaded library to resolve symbols in the loading executable... So you can see that this is what you don't want. Without this option it just misses this symbol by accident.
But... You should not do any stuff like that there. Your libraries and executable should
not share symbols unless they are really should be shared.
Always think of dynamic linking as static linking.
If using shared libraries you must define the stuff you want to export with macro like here. See DLL_PUBLIC macro definition in there.
By default, the linker won't combine a global variable (a 'D') in the base executable with one in a shared library. The base executable is special. There might be an obscure way to do this with one of those obscure control files that ld reads, but I sort of doubt it.
--export-dynamic will cause a.out 'D' symbols to be available to shared libs.
However, consider the process. Step 1: you create a DSO from a .o with a 'U' and a .a with a 'D'. So, the linker incorporates the symbol in the DSO. Step 2, you create the executable with a 'U' in one of the .o files, and 'D' in both a .a and the DSO. It will try to resolve using the left-to-right rule.
Variables, as opposed to functions, pose certain difficulties for the linker across modules in any case. A better practice is to avoid global var references across module boundaries, and use function calls. However, that would still fail for you if you put the same function in both the base executable and a shared lib.
My first question is if there is any particular reason for which you both statically and dynamically (via dlopen) link the same code?
For your problem: -rdynamic will export the symbols from your program and what probably is happening is that dynamic linker resolves all references to your global variable to the first symbol it encounters in symbol tables. Which one is that I don't know.
EDIT: given your purpose I would link your program that way:
g++ -Wall -g -ldl client.cpp -llibrary -L. -o client
You may need to fix the order.
I would advise to use a dlopen(... RTLD_LAZY|RTLD_GLOBAL); to merge global symbol tables.
I would propose to compile any .a static library which you plan to link to a dinamic library, with -fvisibility=hidden parameter, so:
g++ -Wall -fvisibility=hidden -g -c base.cpp