I have two versions of a a shared library:
library version 2:
simple.h
#pragma once
int first(int x);
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first##LIBSIMPLE_2.0");
int first_2_0(int x)
{
int y;
printf("lib: %d\n", y);
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
linker version script file:
LIBSIMPLE_1.0{
global:
first;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.2.0.0
And library version 3:
simple.h
#pragma once
#ifdef SIMPLELIB_VERSION_3_0
int first(int x, int normfactor);
#else
int first(int x);
#endif //SIMPLELIB_VERSION_3_0
simple.c
#include "simple.h"
#include <stdio.h>
__asm__(".symver first_1_0,first#LIBSIMPLE_1.0");
int first_1_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return x + 1;
}
__asm__(".symver first_2_0,first#LIBSIMPLE_2.0");
int first_2_0(int x)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * 1000;
}
__asm__(".symver first_3_0,first##LIBSIMPLE_3.0");
int first_3_0(int x, int normfactor)
{
printf("lib: %s\n", __FUNCTION__);
return (x + 1) * normfactor;
}
Linker version script file:
LIBSIMPLE_1.0{
global:
first; second;
local:
*;
};
LIBSIMPLE_2.0{
global:
first;
local:
*;
};
LIBSIMPLE_3.0{
global:
first;
local:
*;
};
gcc -Wall -g -O0 -fPIC -c simple.c
gcc -shared simple.o -Wl,--version-script,script -o libsimple.so.3.0.0
So i end up with having two different libraries. Next i create a simple application, that eventually i want to link to a library version 3, so in it i use function first() that takes two arguments:
main.c
#include <stdio.h>
#include "simple.h"
int main(int argc, char* argv[])
{
int nFirst = first(1, 10);
printf("First(1) = %d\n", nFirst);
}
I compile app with next commands:
gcc -g -Wall -DSIMPLELIB_VERSION_3_0 -c main.c
And then, by accident, instead of linking to a library version 3, i linked against library version 2. I expected linking to fail, but it went through, and application was working.
gcc main.o -Wl,-L. -lsimple.2.0.0 -Wl,-R. -o demo
So my questions are:
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
p.s. Sorry for a big amount of code. I was trying to make it clear.
Thanks
Is it because library exports symbol with name 'function', and application tries to link to the same symbol name, and that is why linker didn't complain, and just linked against library version 2 ?
Yes, since plain C does not have function overload there is no need for mangling and as a consequence only function name will be used as a symbol for linking. In the end your application code wants to link with function and your library code exports function and this is enough to keep linker happy (even though it is not valid from binary interface perspective).
I thought since c++ mangles symbol names, such thing wouldn't happen, and linker wouldn't link to a library version 2. So i tried all the same, but instead of a gcc compiler, i tried to use g++. Everything went well, until i tried to link application to a library, and i received unresolved links error. Can not figure out why.
Yes, this problem should not occur in C++ because of name mangling. However, this is true only if you have both your application code and library code in C++ or if you bridge your C and C++ code the right way.
It is hard to say (without full listing) what happened in your case when you used g++ but from the looks of it you ended up having application code in C++ and library code still in C. If that is the case your application code will now want to link with mangled function while your library code still exports unmangled function.
To verify this you can inspect your object file with something like:
nm main.o
... and see exactly what kind of symbol does it want. If you will get something like this:
...
U _Z3functionii
...
... instead of:
...
U function
...
... then that is the case.
To "fix" this and make your C++ application code link with unmangled function from library code you'll need to declare your function prototype as extern "C".
Related
I've been following Apple's Dynamic Library Programming Topics
docs to create and use a runtime-loaded library using dlopen() / dlsym().
It seems I'm getting a failure to find the desired symbol on my Mid 2012 MacBook Air, running macOS Mojave.
Library Source Code
// adder.h
int add(int x);
and
// adder.cpp
#include "adder.h"
int add(int x) {
return (x + 1);
}
Compiled with clang -dynamiclib adder.cpp -o libAdd.A.dylib
Main Source
// main.cpp
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>
#include "adder.h"
int main() {
void* adder_handle = dlopen("libAdd.A.dylib", RTLD_LOCAL|RTLD_LAZY);
if (!adder_handle) {
printf("[%s] Unable to load library: %s\n\n", __FILE__, dlerror());
exit(EXIT_FAILURE);
}
while(true) {
void* voidptr = dlsym(adder_handle, "add");
int (*add)(int) = (int (*)(int))voidptr;
if (!add) {
printf("[%s] Unable to get symbol: %s\n\n", __FILE__, dlerror());
exit(EXIT_FAILURE);
}
printf("%d\n", add(0));
}
dlclose(adder_handle);
return 0;
}
Compiled with clang main.cpp -o main
I've also set the DYLD_LIBRARY_PATH environment variable to ensure the library can be found. Everything compiles ok.
Nevertheless, when I run the main executable, I get the error:
[main.cpp] Unable to get symbol: dlsym(0x7fb180500000, add): symbol not found
Running nm -gC libAdd.A.dylib outputs:
0000000000000fa0 T add(int)
U dyld_stub_binder
Any ideas on what could be wrong, or what I need to do to debug this issue?
Thanks!
C++ actually mangles the functionname which results in a different symbolname.
Your are able to spot these mangled symbol names using nm -g <yourlib.dylib>
You can change this behavior by wrapping your method into
extern "C" {
int add(int x);
}
This question already has answers here:
C++ static initialization order
(6 answers)
Closed 6 years ago.
I have following code
File hello.cc
static A dummyl;
A:: A() {
fun();
}
void A::fun() {
int y = 10;
int z = 20;
int x = y + z;
}
File hello.h
class A {
public:
A a;
void fun();
};
File main.cc
#include <dlfcn.h>
#include "hello.h"
typedef void (*pf)();
int main() {
void *lib;
pf greet;
const char * err;
printf("\n Before dlopen\n");
lib = dlopen("libhello.so", RTLD_NOW | RTLD_GLOBAL);
if (!lib) {
exit(1);
}
A *a = new A ;
a->fun();
dlerror(); /*first clear any previous error; redundant in this case but a useful habit*/
dlclose(lib);
return 0;
}
Build phases:
g++ -fPIC -c hello.cc
g++ -shared -o libhello.so hello.o
g++ -o myprog main.cc -ldl -L. -lhello
Since my shared library in real case is libQtCore.so , I need to link it as using -lQtCore in linker because I cannot use the symbols directly and since there are many of functions then libQtCore, it will not practically advisable to use dlysym for each function of libQtCore.so
Since I link, my static global variables gets initialized before dlopen. Is there any flag for linker g++ that tells compiler to load the shared library only after _dlopen _?
Is there any flag for linker g++ that tells compiler to load the shared library only after _dlopen _?
Yes. On Windows. It's known as delay loading. It doesn't exist on Linux/OS X. It's possible to implement it, but it'd require modifications to binutils and ld.so. See this article for some background on that.
But you don't need to care about any of it. Really.
You are facing a well known problem. So well known that it even has a name: The Static Initialization Order Fiasco.
The solution to all your woes is trivial: don't use static global variables the way you do.
To fix it, use Q_GLOBAL_STATIC, or reimplement something like it yourself. That way the value will be constructed at the time it's first used, not prematurely. That's all there's to it.
Side note: Your recent questions suffer badly by being an XY Problem. You're trying to come up with all sorts of Rube Goldberg-esque solutions to a rather simple issue that it took you a week+ to finally divulge. Ask not about a potential solution, but about the underlying issue is you're attempting to solve. All the library loading stuff is completely and utterly unrelated to your problem, and you don't need to concern yourself with it at all.
I have encountered on CentOS 6.5. As I have searched online that static variable behaves differently on Windows and on Linux when using dynamic library. That is, Windows would cause duplication of variables and Linux would not, like this one:
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
However, when I wrote a small program to validate this, I found that Linux also causes duplication. Here is my small program, including four files:
(1) A.h
#ifndef A_H
#define A_H
#include <cstdio>
static int b;
extern "C" class A {
public:
int mem;
A() {
printf("A's address: %p\n", this);
printf("B's address: %p\n", &b);
}
void print() {
printf("%p: %d\n", this, mem);
}
~A() {
printf("DELETE A!!!!! %p\n", this);
}
};
extern A a;
#endif
(2) A.cpp
#include "A.h"
A a;
(3) d.cpp
#include "A.h"
extern "C" void exec() {
a.print();
}
(4) main.cpp
#include "A.h"
#include <dlfcn.h>
typedef void (*fptr) ();
int main() {
a.mem = 22;
a.print();
void *handle;
handle = dlopen("d.so", RTLD_LAZY);
fptr exec = reinterpret_cast<fptr>(dlsym(handle, "exec"));
(*exec)();
dlclose(handle);
return 0;
}
Here is how I compile and run my program:
g++ d.cpp A.cpp -shared -rdynamic -o d.so -ldl -I. -fPIC -g -std=c++1y
g++ main.cpp A.cpp -ldl -I. -g -std=c++1y
./a.out
Both the dynamic part d.cpp and the static part main.cpp use the variables a and b declared in A.cpp and A.h. And here is the result of the program on my machine:
A's address: 0x600f8c
B's address: 0x600f90
0x600f8c: 22
A's address: 0x7fb8fe859e4c
B's address: 0x7fb8fe859e50
0x7fb8fe859e4c: 0
DELETE A!!!!! 0x7fb8fe859e4c
DELETE A!!!!! 0x600f8c
This surprises me a lot, because the addresses of global variable a and static variable b should be the same in the dynamic part and the static part. And it seems that modification on a in static part does not effect the a in dynamic part. Would anyone please answer my question, or help find out some mistakes in the program (if any)?
By the way, to be honest, on another project I am working on, I find that addresses of global variables are the same in dynamic library and in static library. But that project is too big and I cannot provide a small program to reproduce the behavior.
Thanks a lot !
The first command you showed builds a shared object d.so. Based on the context of your question, I surmise that you also intended to link with d.so, but your second command seems to be missing that part. I'm assuming that it's a typo, as this is the only explanation for the program output you showed -- that A.cpp is both linked to directly, and is also built into your d.so library.
Given that, quoting from the article you linked:
Object code routines used by both should not be duplicated in each.
This is especially true for code which use static variables such as
singleton classes. A static variable is global and thus can only be
represented once. Including it twice will provide unexpected results.
But that's exactly the rule you seem to be breaking, you're representing the statically-scoped instance of the A class twice, in your d.so, and in your main application executable.
So, that seems to be the indicated outcome: "unexpected results".
As mentioned in other SO answers I am using the wrapping mechanism of GNU ld to intercept calls to malloc on Linux (see here for an example).
The used linker flag is -Wl,--wrap=malloc and the respective void __wrap_malloc(size_t) is also defined.
This works fine in a test application where all compilation unit are linked into the same binary.
Now, I need to modify a dynamically linked library which is loaded into a main program via dlopen().
Linking the library succeeds, but loading it into the main program fails with undefined symbol: __real_malloc.
Running nm on the library shows that __wrap_malloc is defined but __real_malloc is not.
But, according to man ld and this SO answer, malloc should get replaced with __wrap_malloc and __real_malloc should point to malloc when using this technique.
In the test application, I see that __real_malloc is undefined in the compiled object files but is resolved after getting linked into the executable.
So, why is the symbol resolved in the test application but not in the dynamic library?
In both cases a final link step is performed which should resolve this symbol.
Or is it required to add another library during the link step of the dynamic library in order to get __real_malloc resolved?
Just in case, it is not possible to modify the target application which loads the dynamic library via dlopen.
It should work and only requires slight changes to the code in the question you linked.
testapp.c
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
typedef void f_t(void);
int main()
{
void* h = dlopen("./malloc_wrapper.so", RTLD_NOW);
if (h == NULL)
{
puts(dlerror());
exit(1);
}
f_t* f = (f_t*)dlsym(h, "test");
if (f == NULL)
{
puts(dlerror());
exit(1);
}
(*f)();
return 0;
}
malloc_wrapper.c
#include <stdlib.h> /* added */
#include <stdio.h>
void *__real_malloc (size_t);
/* This function wraps the real malloc */
void *__wrap_malloc(size_t size)
{
void *lptr = __real_malloc(size);
printf("Malloc: %lu bytes #%p\n", size, lptr);
return lptr;
}
void test(void) /* added */
{
free(malloc(1024));
}
Compile and run.
gcc -Wl,-wrap,malloc -shared -fpic malloc_wrapper.c -o malloc_wrapper.so
gcc testapp.c -o testapp -ldl
./testapp
Malloc: 1024 bytes #0x1d44680
Compiling malloc_wrapper.so like this reproduces the error you describe:
gcc -shared -fpic malloc_wrapper.c -o malloc_wrapper.so
./testapp
./malloc_wrapper.so: undefined symbol: __real_malloc
Perhaps you were using wrap when compiling and linking the executable instead of the shared object?
I have an application that statically links with version X of a library, libfoo, from thirdparty vendor, VENDOR1. It also links with a dynamic (shared) library, libbar, from a different thirdparty vendor, VENDOR2, that statically links version Y of libfoo from VENDOR1.
So libbar.so contains version Y of libfoo.a and my executable contains version X of libfoo.a
libbar only uses libfoo internally and there are no libfoo objects passed from my app to libbar.
There are no errors at build time but at runtime the app seg faults. The reason seems to be that version X uses structures that have a different size they version Y and the runtime linker seems to be mixing up which get used by which.
Both VENDOR1 & VENDOR2 are closed source so I cannot rebuild them.
Is there a way to build/link my app such that it always resolves to version X and libbar alway resolves to version Y and the two never mix?
Thanks for all the responses. I have a solution that seem to be working.
Here's the problem in detail with an example.
In main.c we have:
#include <stdio.h>
extern int foo();
int bar()
{
printf("bar in main.c called\n");
return 0;
}
int main()
{
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
In foo.c we have:
extern int bar();
int foo()
{
int x = bar();
return x;
}
In bar.c we have:
#include <stdio.h>
int bar()
{
printf("bar in bar.c called\n");
return 2;
}
Compile bar.c and foo.c:
$ gcc -fPIC -c bar.c
$ gcc -fPIC -c foo.c
Add bar.o to a static library:
$ ar r libbar.a bar.o
Now create a shared library using foo.o and link with static libbar.a
$ gcc -shared -o libfoo.so foo.o -L. -lbar
Compile main.c and link with shared library libfoo.so
$ gcc -o main main.c -L. -lfoo
Set LD_LIBRARY_PATH to find libfoo.so and run main:
$ setenv LD_LIBRARY_PATH `pwd`
$ ./main
bar in main.c called
result from foo is 0
bar in main.c called
result from bar is 0
Notice that the version of bar in main.c is called, not the version linked into the shared library.
In main2.c we have:
#include <stdio.h>
#include <dlfcn.h>
int bar()
{
printf("bar in main2.c called\n");
return 0;
}
int main()
{
int x;
int (*foo)();
void *handle = dlopen("libfoo.so", RTLD_GLOBAL|RTLD_LAZY);
foo = dlsym(handle, "foo");
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
Compile and run main2.c (notice we dont need to explicitly link with libfoo.so):
$ gcc -o main2 main2.c -ldl
$ ./main2
bar in bar.c called
result from foo is 2
bar in main2.c called
result from bar is 0
Now foo in the shared library calls bar in the shared library and main calls bar in main.c
I don't think this behaviour is intuitive and it is more work to use dlopen/dlsym, but it does resolve my problem.
Thanks again for the comments.
Try a partial link so that you have an object file "partial.o" with libbar and libfoo-Y. Use objcopy with "--localize-symbols " to make the symbols in partial.o from libfoo-Y local. You should be able to generate by running nm on libfoo-Y and massaging the output. Then take the modified partial.o and link it to your app.
I've done something similar with gcc toolchain on vxWorks where dynamic libs are not a complication but two versions of the same lib needed to link cleanly into a monolithic app.
Sorry no. My understanding of the way that Linux (and possibly most *nixes) is that that is not possible. The only 'solution' for your problem I can think of, is if you create a proxy app, which exposes what you need from libbar in the form of some IPC. You can then make that proxy load the correct version using LD_LIBRARY_PATH or something simmilar.