I've a linux application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so)
If I compile the static library as dynamic and link it to the application, the dlopen it will work as expected, but if I use it as described above it won't.
Can't a static library uses the dlopen function to load shared libraries?
Thanks.
There should be no problem with what you're trying to do:
app.c:
#include "staticlib.h"
#include "stdio.h"
int main()
{
printf("and the magic number is: %d\n",doSomethingDynamicish());
return 0;
}
staticlib.h:
#ifndef __STATICLIB_H__
#define __STATICLIB_H__
int doSomethingDynamicish();
#endif
staticlib.c:
#include "staticlib.h"
#include "dlfcn.h"
#include "stdio.h"
int doSomethingDynamicish()
{
void* handle = dlopen("./libdynlib.so",RTLD_NOW);
if(!handle)
{
printf("could not dlopen: %s\n",dlerror());
return 0;
}
typedef int(*dynamicfnc)();
dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
const char* err = dlerror();
if(err)
{
printf("could not dlsym: %s\n",err);
return 0;
}
return func();
}
dynlib.c:
int GetMeANumber()
{
return 1337;
}
and build:
gcc -c -o staticlib.o staticlib.c
ar rcs libstaticlib.a staticlib.o
gcc -o app app.c libstaticlib.a -ldl
gcc -shared -o libdynlib.so dynlib.c
First line builds the lib
second line packs it into a static lib
third builds the test app, linking in the newly created static, plus the linux dynamic linking library(libdl)
fourth line builds the soon-to-be-dynamically-loaded shared lib.
output:
./app
and the magic number is: 1337
Related
I'm planning to implement my own malloc/free and I ran into some problems while trying to link my shared library with my executable.
Right now, I can get it to work with LD_PRELOAD, but not by linking the .so to the executable, although I can get similiar libraries, like tcmalloc, to work properly just by linking them to my executable, and would like to do the same.
I'm building everything with cmake, this is the CMakeLists of my shared library:
cmake_minimum_required(VERSION 2.8)
project(allocator)
add_library(allocator SHARED exports.cpp)
target_link_libraries(allocator dl)
target_compile_features(allocator PRIVATE cxx_range_for)
and this is exports.cpp:
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif
#include <stdio.h>
#include <dlfcn.h>
typedef void * (*MallocType)(size_t);
typedef void (*FreeType)(void *);
static bool g_initialized = false;
static MallocType real_malloc = nullptr;
static FreeType real_free = nullptr;
static void alloc_init(void)
{
real_malloc = (MallocType) dlsym(RTLD_NEXT, "malloc");
if (!real_malloc)
{
fprintf(stderr, "Error in `dlsym`: %s\n", dlerror());
}
real_free = (FreeType) dlsym(RTLD_NEXT, "free");
if (!real_free)
{
fprintf(stderr, "Error in `dlsym`: %s\n", dlerror());
}
g_initialized = true;
}
extern "C" void * malloc(size_t size)
{
if (!g_initialized)
{
alloc_init();
}
printf("Allocate %u.\n", size);
return real_malloc(size);
}
extern "C" void free(void *ptr)
{
if (!g_initialized)
{
alloc_init();
}
printf("Free %p.\n", ptr);
real_free(ptr);
}
As I said, trying to link the resulting .so to an executable doesn't really link the library (there's no entry in ldd, and libc malloc is called). I was wondering what am I doing wrong.
Edit:
I've also tried compiling with
g++ -o liballocator.so -shared exports.cpp -std=c++11 -fPIC -ldl
g++ -o test launcher.cpp memusage.cpp app.cpp -ldl -L. -lallocator -std=c++11
CMake isn't your tool of choice here. CMake creates makefiles or IDE project files for C source, and has a kind of working assumption that all the code is doing conventional things in conventional ways. That is no longer true if you have undertaken to provide your own malloc.
Most C compilers can be coaxed into linking a user-supplied version of malloc, often by playing about with the order of the link flags. But it is an error-prone process, since there might be indirect calls or submodules bound early. You can instantly solve all those problems by renaming malloc() mymalloc(), but then of course you must rewrite the client code.
This question is closely related to dlopen a dynamic library from a static library linux C++, but contains a further complication (and uses C++ instead of C):
I have an application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so). In addition, the dynamic libraries call functions defined in the static one.
Is there a way to compile this without linking the dynamic libraries against the static one or vice versa?
Here comes what I tried so far, slightly modifying the example from the related question:
app.cpp:
#include "staticlib.hpp"
#include <iostream>
int main()
{
std::cout << "and the magic number is: " << doSomethingDynamicish() << std::endl;
return 0;
}
staticlib.hpp:
#ifndef __STATICLIB_H__
#define __STATICLIB_H__
int doSomethingDynamicish();
int doSomethingBoring();
#endif
staticlib.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingDynamicish()
{
void* handle = dlopen("./libdynlib.so",RTLD_NOW);
if(!handle)
{
std::cout << "could not dlopen: " << dlerror() << std::endl;
return 0;
}
typedef int(*dynamicfnc)();
dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
const char* err = dlerror();
if(err)
{
std::cout << "could not dlsym: " <<err << std::endl;
return 0;
}
return func();
}
staticlib2.cpp:
#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingBoring()
{
std::cout << "This function is so boring." << std::endl;
return 0;
}
dynlib.cpp:
#include "staticlib.hpp"
extern "C" int GetMeANumber()
{
doSomethingBoring();
return 1337;
}
and build:
g++ -c -o staticlib.o staticlib.cpp
g++ -c -o staticlib2.o staticlib2.cpp
ar rv libstaticlib.a staticlib.o staticlib2.o
ranlib libstaticlib.a
g++ -rdynamic -o app app.cpp libstaticlib.a -ldl
g++ -fPIC -shared -o libdynlib.so dynlib.cpp
When I run it with ./app I get
could not dlopen: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv
and the magic number is: 0
From the dlopen manual page:
If the executable was linked with the flag "-rdynamic" (or, synonymously, "--export-dynamic"), then the global symbols in the executable will also be used to resolve references in a dynamically loaded library.
That means that for the application to export its symbols for use in the dynamic library, you have to link your application with the -rdynamic flag.
Besides the problem described above, there is another problem and that has to do with the static library: The problem is namely that since the doSomethingBoring function is not called in your main program, the object file staticlib2.o from the static library is not linked.
The answer can be found in e.g. this old question, which tells you to add the --whole-archive linker flag:
g++ -rdynamic -o app app.cpp -L. \
-Wl,--whole-archive -lstaticlib \
-Wl,--no-whole-archive -ldl
As mentioned in other SO answers I am using the wrapping mechanism of GNU ld to intercept calls to malloc on Linux (see here for an example).
The used linker flag is -Wl,--wrap=malloc and the respective void __wrap_malloc(size_t) is also defined.
This works fine in a test application where all compilation unit are linked into the same binary.
Now, I need to modify a dynamically linked library which is loaded into a main program via dlopen().
Linking the library succeeds, but loading it into the main program fails with undefined symbol: __real_malloc.
Running nm on the library shows that __wrap_malloc is defined but __real_malloc is not.
But, according to man ld and this SO answer, malloc should get replaced with __wrap_malloc and __real_malloc should point to malloc when using this technique.
In the test application, I see that __real_malloc is undefined in the compiled object files but is resolved after getting linked into the executable.
So, why is the symbol resolved in the test application but not in the dynamic library?
In both cases a final link step is performed which should resolve this symbol.
Or is it required to add another library during the link step of the dynamic library in order to get __real_malloc resolved?
Just in case, it is not possible to modify the target application which loads the dynamic library via dlopen.
It should work and only requires slight changes to the code in the question you linked.
testapp.c
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
typedef void f_t(void);
int main()
{
void* h = dlopen("./malloc_wrapper.so", RTLD_NOW);
if (h == NULL)
{
puts(dlerror());
exit(1);
}
f_t* f = (f_t*)dlsym(h, "test");
if (f == NULL)
{
puts(dlerror());
exit(1);
}
(*f)();
return 0;
}
malloc_wrapper.c
#include <stdlib.h> /* added */
#include <stdio.h>
void *__real_malloc (size_t);
/* This function wraps the real malloc */
void *__wrap_malloc(size_t size)
{
void *lptr = __real_malloc(size);
printf("Malloc: %lu bytes #%p\n", size, lptr);
return lptr;
}
void test(void) /* added */
{
free(malloc(1024));
}
Compile and run.
gcc -Wl,-wrap,malloc -shared -fpic malloc_wrapper.c -o malloc_wrapper.so
gcc testapp.c -o testapp -ldl
./testapp
Malloc: 1024 bytes #0x1d44680
Compiling malloc_wrapper.so like this reproduces the error you describe:
gcc -shared -fpic malloc_wrapper.c -o malloc_wrapper.so
./testapp
./malloc_wrapper.so: undefined symbol: __real_malloc
Perhaps you were using wrap when compiling and linking the executable instead of the shared object?
I have an application that statically links with version X of a library, libfoo, from thirdparty vendor, VENDOR1. It also links with a dynamic (shared) library, libbar, from a different thirdparty vendor, VENDOR2, that statically links version Y of libfoo from VENDOR1.
So libbar.so contains version Y of libfoo.a and my executable contains version X of libfoo.a
libbar only uses libfoo internally and there are no libfoo objects passed from my app to libbar.
There are no errors at build time but at runtime the app seg faults. The reason seems to be that version X uses structures that have a different size they version Y and the runtime linker seems to be mixing up which get used by which.
Both VENDOR1 & VENDOR2 are closed source so I cannot rebuild them.
Is there a way to build/link my app such that it always resolves to version X and libbar alway resolves to version Y and the two never mix?
Thanks for all the responses. I have a solution that seem to be working.
Here's the problem in detail with an example.
In main.c we have:
#include <stdio.h>
extern int foo();
int bar()
{
printf("bar in main.c called\n");
return 0;
}
int main()
{
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
In foo.c we have:
extern int bar();
int foo()
{
int x = bar();
return x;
}
In bar.c we have:
#include <stdio.h>
int bar()
{
printf("bar in bar.c called\n");
return 2;
}
Compile bar.c and foo.c:
$ gcc -fPIC -c bar.c
$ gcc -fPIC -c foo.c
Add bar.o to a static library:
$ ar r libbar.a bar.o
Now create a shared library using foo.o and link with static libbar.a
$ gcc -shared -o libfoo.so foo.o -L. -lbar
Compile main.c and link with shared library libfoo.so
$ gcc -o main main.c -L. -lfoo
Set LD_LIBRARY_PATH to find libfoo.so and run main:
$ setenv LD_LIBRARY_PATH `pwd`
$ ./main
bar in main.c called
result from foo is 0
bar in main.c called
result from bar is 0
Notice that the version of bar in main.c is called, not the version linked into the shared library.
In main2.c we have:
#include <stdio.h>
#include <dlfcn.h>
int bar()
{
printf("bar in main2.c called\n");
return 0;
}
int main()
{
int x;
int (*foo)();
void *handle = dlopen("libfoo.so", RTLD_GLOBAL|RTLD_LAZY);
foo = dlsym(handle, "foo");
printf("result from foo is %d\n", foo());
printf("result from bar is %d\n", bar());
}
Compile and run main2.c (notice we dont need to explicitly link with libfoo.so):
$ gcc -o main2 main2.c -ldl
$ ./main2
bar in bar.c called
result from foo is 2
bar in main2.c called
result from bar is 0
Now foo in the shared library calls bar in the shared library and main calls bar in main.c
I don't think this behaviour is intuitive and it is more work to use dlopen/dlsym, but it does resolve my problem.
Thanks again for the comments.
Try a partial link so that you have an object file "partial.o" with libbar and libfoo-Y. Use objcopy with "--localize-symbols " to make the symbols in partial.o from libfoo-Y local. You should be able to generate by running nm on libfoo-Y and massaging the output. Then take the modified partial.o and link it to your app.
I've done something similar with gcc toolchain on vxWorks where dynamic libs are not a complication but two versions of the same lib needed to link cleanly into a monolithic app.
Sorry no. My understanding of the way that Linux (and possibly most *nixes) is that that is not possible. The only 'solution' for your problem I can think of, is if you create a proxy app, which exposes what you need from libbar in the form of some IPC. You can then make that proxy load the correct version using LD_LIBRARY_PATH or something simmilar.
Since I observed some strange behavior of global variables in my dynamically loaded libraries, I wrote the following test.
At first we need a statically linked library: The header test.hpp
#ifndef __BASE_HPP
#define __BASE_HPP
#include <iostream>
class test {
private:
int value;
public:
test(int value) : value(value) {
std::cout << "test::test(int) : value = " << value << std::endl;
}
~test() {
std::cout << "test::~test() : value = " << value << std::endl;
}
int get_value() const { return value; }
void set_value(int new_value) { value = new_value; }
};
extern test global_test;
#endif // __BASE_HPP
and the source test.cpp
#include "base.hpp"
test global_test = test(1);
Then I wrote a dynamically loaded library: library.cpp
#include "base.hpp"
extern "C" {
test* get_global_test() { return &global_test; }
}
and a client program loading this library: client.cpp
#include <iostream>
#include <dlfcn.h>
#include "base.hpp"
typedef test* get_global_test_t();
int main() {
global_test.set_value(2); // global_test from libbase.a
std::cout << "client: " << global_test.get_value() << std::endl;
void* handle = dlopen("./liblibrary.so", RTLD_LAZY);
if (handle == NULL) {
std::cout << dlerror() << std::endl;
return 1;
}
get_global_test_t* get_global_test = NULL;
void* func = dlsym(handle, "get_global_test");
if (func == NULL) {
std::cout << dlerror() << std::endl;
return 1;
} else get_global_test = reinterpret_cast<get_global_test_t*>(func);
test* t = get_global_test(); // global_test from liblibrary.so
std::cout << "liblibrary.so: " << t->get_value() << std::endl;
std::cout << "client: " << global_test.get_value() << std::endl;
dlclose(handle);
return 0;
}
Now I compile the statically loaded library with
g++ -Wall -g -c base.cpp
ar rcs libbase.a base.o
the dynamically loaded library
g++ -Wall -g -fPIC -shared library.cpp libbase.a -o liblibrary.so
and the client
g++ -Wall -g -ldl client.cpp libbase.a -o client
Now I observe: The client and the dynamically loaded library possess a different version of the variable global_test. But in my project I'm using cmake. The build script looks like this:
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
PROJECT(globaltest)
ADD_LIBRARY(base STATIC base.cpp)
ADD_LIBRARY(library MODULE library.cpp)
TARGET_LINK_LIBRARIES(library base)
ADD_EXECUTABLE(client client.cpp)
TARGET_LINK_LIBRARIES(client base dl)
analyzing the created makefiles I found that cmake builds the client with
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
This ends up in a slightly different but fatal behavior: The global_test of the client and the dynamically loaded library are the same but will be destroyed two times at the end of the program.
Am I using cmake in a wrong way? Is it possible that the client and the dynamically loaded library use the same global_test but without this double destruction problem?
g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client
CMake adds -rdynamic option allowing loaded library to resolve symbols in the loading executable... So you can see that this is what you don't want. Without this option it just misses this symbol by accident.
But... You should not do any stuff like that there. Your libraries and executable should
not share symbols unless they are really should be shared.
Always think of dynamic linking as static linking.
If using shared libraries you must define the stuff you want to export with macro like here. See DLL_PUBLIC macro definition in there.
By default, the linker won't combine a global variable (a 'D') in the base executable with one in a shared library. The base executable is special. There might be an obscure way to do this with one of those obscure control files that ld reads, but I sort of doubt it.
--export-dynamic will cause a.out 'D' symbols to be available to shared libs.
However, consider the process. Step 1: you create a DSO from a .o with a 'U' and a .a with a 'D'. So, the linker incorporates the symbol in the DSO. Step 2, you create the executable with a 'U' in one of the .o files, and 'D' in both a .a and the DSO. It will try to resolve using the left-to-right rule.
Variables, as opposed to functions, pose certain difficulties for the linker across modules in any case. A better practice is to avoid global var references across module boundaries, and use function calls. However, that would still fail for you if you put the same function in both the base executable and a shared lib.
My first question is if there is any particular reason for which you both statically and dynamically (via dlopen) link the same code?
For your problem: -rdynamic will export the symbols from your program and what probably is happening is that dynamic linker resolves all references to your global variable to the first symbol it encounters in symbol tables. Which one is that I don't know.
EDIT: given your purpose I would link your program that way:
g++ -Wall -g -ldl client.cpp -llibrary -L. -o client
You may need to fix the order.
I would advise to use a dlopen(... RTLD_LAZY|RTLD_GLOBAL); to merge global symbol tables.
I would propose to compile any .a static library which you plan to link to a dinamic library, with -fvisibility=hidden parameter, so:
g++ -Wall -fvisibility=hidden -g -c base.cpp