GNU linker: Adapt to change of name mangling algorithm - c++

I am trying to re-compile an existing C++ application.
Unfortunately, I must rely on a proprietary library I only have a pre-compiled static archive of.
I use g++ version 7.3.0 and ld version 2.30.
Whatever GCC version it was compiled with, it is ancient.
The header file defines the method:
class foo {
int bar(int & i);
}
As nm lib.a shows, the library archive contains the corresponding exported function:
T bar__4fooRi
nm app.o shows my recent compiler employing a different kind of name mangling:
U _ZN4foo9barERi
Hence the linker cannot resolve the symbols provided by the library.
Is there any option to chose the name mangling algorithm?
Can I introduce a map or define the mangled names explicitly?

#Botje's suggestion lead me to writing a linker script like this (the spaces in the PROVIDE stanza are significant):
EXTERN(bar__4fooRi);
PROVIDE(_ZN4foo9barERi = bar__4fooRi);
As far as I understood, this will regard bar__4fooRi as an externally defined symbol (which it is). If _ZN4foo9barERi is searched for, but not defined, bar__4fooRi will take its place.
I am calling the linker from the GNU toolchain like this (mind the order – the script needs to be after the dependant object but before the defining library):
g++ -o application application.o script.ld -lfoo
It looks like this could work.
At least in theory.
The linker now regards other parts of the library, which in turn depends on other unresolvable symbols including (but not limited to) __throw, __cp_pop_exception, and __builtin_delete. I have no idea where these functions are defined nowadays. Joxean Koret shows some locations in this blog post based on guesswork (__builtin_new probably is malloc) – but I am not that confident.
These findings lead me to the conclusion that the library relies on a different style of exception handling and probably memory management, too.
EDIT: The result may be purely academical due to ABI changes as pointed out by #eukaryota, a linker script can indeed be used to "alias" symbols. Here is a complete minimal example:
foo.h:
class Foo {
public:
int bar(int);
};
foo.cpp:
#include "foo.h"
int Foo::bar(int i) {
return i+21;
}
main.cpp:
class Foo {
public:
int baa(int); // use in-place "header" to simulate different name mangling algorithm
};
int main(int, char**) {
Foo f;
return f.baa(21);
}
script.ld:
EXTERN(_ZN3Foo3barEi);
PROVIDE(_ZN3Foo3baaEi = _ZN3Foo3barEi); /* declare "alias" */
Build process:
g++ -o libfoo.o -c foo.c
ar rvs libfoo.a libfoo.o # simulate building a library
g++ -o app main.o -L. script.ld -lfoo
app is compiled, can be executed and returns expected result.

Related

Wrapping symbols during linking on OS X

I'm trying to wrap one symbol by another during link. As I understand this is easily done with ld --wrap option, but on OS X it's not available. There is '-idefinition:indirection', here how I'm trying to cheat main so it will print 42:
a.cpp:
int foo() {
return 1;
}
b.cpp:
int wrap_foo() {
return 42;
}
main.cpp:
#include <cstdio>
int foo();
int wrap_foo();
int main() {
printf("%d\n", foo());
}
How I build and link them:
$ gcc -c *.cpp
$ gcc -Wl,-i__Z3foov:__Z8wrap_foov *.o
duplicate symbol __Z3foov in:
a.o
b.o
ld: 1 duplicate symbol for architecture x86_64
Is it even possible to achieve what I want?
EDIT:
In my task I don't have control over a.cpp and main.cpp (have only objects for them), but I can write any code and do whatever I need with the objects before link.
I've found following quistion before which related to similar task (and describes it better)
GNU gcc/ld - wrapping a call to symbol with caller and callee defined in the same object file
Rereading the answers I see that the case is same, symbol already present and I will have a collision.
How to remove foo from a.o without touching source code?
Does this have to be done at link versus runtime? Note entirely sure what you're trying to do, so just asking.
The problem with using that linker command is that it creates a new symbol, which is why you get the duplicate symbol.
-alias symbol_name alternate_symbol_name
Create an alias named alternate_symbol_name for the symbol
symbol_name. By default the alias symbol has global visibil-
ity. This option was previous the -idef:indir option.
If you don't have a.cpp which defines foo (ie. you are omitting foo), you can do something like this:
tmp$ gcc -Wl,-alias,__Z8wrap_foov,__Z3foov b.o main.o
tmp$ ./a.out
42
What you could always do is have a.cpp have another foo like original_foo. This way if you wanted that implementation, you could use alias to map that to foo. Not perfect, but it would achieve roughly what you want. You're already going through the effort of adjusting it at link, so it would seem potentially acceptable.

Why doesn't the linker complain of duplicate symbols?

I have a dummy.hpp
#ifndef DUMMY
#define DUMMY
void dummy();
#endif
and a dummy.cpp
#include <iostream>
void dummy() {
std::cerr << "dummy" << std::endl;
}
and a main.cpp which use dummy()
#include "dummy.hpp"
int main(){
dummy();
return 0;
}
Then I compiled dummy.cpp to three libraries, libdummy1.a, libdummy2.a, libdummy.so:
g++ -c -fPIC dummy.cpp
ar rvs libdummy1.a dummy.o
ar rvs libdummy2.a dummy.o
g++ -shared -fPIC -o libdummy.so dummy.cpp
When I try compile main and link the dummy libs
g++ -o main main.cpp -L. -ldummy1 -ldummy2
There is no duplicate symbol error produced by linker. Why does this happen when I link two identical libraries statically?
When I try
g++ -o main main.cpp -L. -ldummy1 -ldummy
There is also no duplicate symbol error, Why?
The loader seems always to choose dynamic libs and not the code compiled in the .o files.
Does it mean the same symbol is always loaded from the .so file if it is both in a .a and a .so file?
Does it mean symbols in the static symbol table in static library never conflict with those in the dynamic symbol table in a .so file?
There's no error in either Scenario 1 (dual static libraries) or Scenario 2 (static and shared libraries) because the linker takes the first object file from a static library, or the first shared library, that it encounters that provides a definition of a symbol it has not yet got a definition for. It simply ignores any later definitions of the same symbol because it already has a good one. In general, the linker only takes what it needs from a library. With static libraries, that's strictly true. With shared libraries, all the symbols in the shared library are available if it satisfied any missing symbol; with some linkers, the symbols of the shared library may be available regardless, but other versions only record the use a shared library if that shared library provides at least one definition.
It's also why you need to link libraries after object files. You could add dummy.o to your linking commands and as long as that appears before the libraries, there'll be no trouble. Add the dummy.o file after libraries and you'll get doubly-defined symbol errors.
The only time you run into problems with this double definitions is if there's an object file in Library 1 that defines both dummy and extra, and there's an object file in Library 2 that defines both dummy and alternative, and the code needs the definitions of both extra and alternative — then you have duplicate definitions of dummy that cause trouble. Indeed, the object files could be in a single library and would cause trouble.
Consider:
/* file1.h */
extern void dummy();
extern int extra(int);
/* file1.cpp */
#include "file1.h"
#include <iostream>
void dummy() { std::cerr << "dummy() from " << __FILE__ << '\n'; }
int extra(int i) { return i + 37; }
/* file2.h */
extern void dummy();
extern int alternative(int);
/* file2.cpp */
#include "file2.h"
#include <iostream>
void dummy() { std::cerr << "dummy() from " << __FILE__ << '\n'; }
int alternative(int i) { return -i; }
/* main.cpp */
#include "file1.h"
#include "file2.h"
int main()
{
return extra(alternative(54));
}
You won't be able to link the object files from the three source files shown because of the double-definition of dummy, even though the main code does not call dummy().
Regarding:
The loader seems always to choose dynamic libs and not compiled in the .o files.
No; the linker always attempts to load object files unconditionally. It scans libraries as it encounters them on the command line, collecting definitions it needs. If the object files precede the libraries, there's not a problem unless two of the object files define the same symbol (does 'one definition rule' ring any bells?). If some of the object files follow libaries, you can run into conflicts if libraries define symbols that the later object files define. Note that when it starts out, the linker is looking for a definition of main. It collects the defined symbols and referenced symbols from each object file it is told about, and keeps adding code (from libraries) until all the referenced symbols are defined.
Does it means the same symbol is always loaded from .so file, if it is both in .a and .so file?
No; it depends which was encountered first. If the .a was encountered first, the .o file is effectively copied from the library into the executable (and the symbol in the shared library is ignored because there's already a definition for it in the executable). If the .so was encountered first, the definition in the .a is ignored because the linker is no longer looking for a definition of that symbol — it's already got one.
Does it mean that symbols in static symbol table in a static library are never in conflict with those in dynamic symbol table in .so file?
You can have conflicts, but the first definition encountered resolves the symbol for the linker. It only runs into conflicts if the code that satisfies the reference causes a conflict by defining other symbols that are needed.
If I link 2 shared libs, can I get conflicts and the link phase failed?
As I noted in a comment:
My immediate reaction is "Yes, you can". It would depend on the content of the two shared libraries, but you could run into problems, I believe. […cogitation…] How would you show this problem? … It's not as easy as it seems at first sight. What is required to demonstrate such a problem? … Or am I overthinking this? … […time to go play with some sample code…]
After some experimentation, my provisional, empirical answer is "No, you can't" (or "No, on at least some systems, you don't run into a conflict"). I'm glad I prevaricated.
Taking the code shown above (2 headers, 3 source files), and running with GCC 5.3.0 on Mac OS X 10.10.5 (Yosemite), I can run:
$ g++ -O -c main.cpp
$ g++ -O -c file1.cpp
$ g++ -O -c file2.cpp
$ g++ -shared -o libfile2.so file2.o
$ g++ -shared -o libfile1.so file1.o
$ g++ -o test2 main.o -L. -lfile1 -lfile2
$ ./test2
$ echo $?
239
$ otool -L test2
test2:
libfile2.so (compatibility version 0.0.0, current version 0.0.0)
libfile1.so (compatibility version 0.0.0, current version 0.0.0)
/opt/gcc/v5.3.0/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.21.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/opt/gcc/v5.3.0/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
$
It is aconventional to use .so as the extension on Mac OS X (it's usually .dylib), but it seems to work.
Then I revised the code in the .cpp files so that extra() calls dummy() before the return, and so does alternative() and main(). After recompiling and rebuilding the shared libraries, I ran the programs. The first line of output is from the dummy() called by main(). Then you get the other two lines produced by alternative() and extra() in that order because the calling sequence for return extra(alternative(54)); demands that.
$ g++ -o test2 main.o -L. -lfile1 -lfile2
$ ./test2
dummy() from file1.cpp
dummy() from file2.cpp
dummy() from file1.cpp
$ g++ -o test2 main.o -L. -lfile2 -lfile1
$ ./test2
dummy() from file2.cpp
dummy() from file2.cpp
dummy() from file1.cpp
$
Note that the function called by main() is the first one that appears in the libraries it is linked with. But (on Mac OS X 10.10.5 at least) the linker does not run into a conflict. Note, though, that the code in each shared object calls 'its own' version of dummy() — there is disagreement between the two shared libraries about which function is dummy(). (It would be interesting to have the dummy() function in separate object files in the shared libraries; then which version of dummy() gets called?) But in the extremely simple scenario shown, the main() function manages to call just one of the dummy() functions. (Note that I'd not be surprised to find differences between platforms for this behaviour. I've identified where I tested the code. Please let me know if you find different behaviour on some platform.)

Static *template* class member across dynamic library

Edit: the comments below the accepted answer show that it might be an issue with the Android dynamic loader.
I have a header for a template class with a static member. At runtime the address of the static member is used in the library and in the client code. The template is implicitly instantiated both in the library and in the client code. It works fine on Linux and OSX, the symbol is duplicated but marked as "uniqued" as shown by nm (see below).
However when I compile for ARM (Android), the symbol is marked weak in both the DSO and the executable. The loader does not unify and the symbol is effectively duplicated at runtime!
I read these:
two instances of a static member, how could that be?
Static template data members storage
and especially this answer:
https://stackoverflow.com/a/2505528/2077394
and:
http://gcc.gnu.org/wiki/Visibility
but I am still a little bit puzzled. I understand that the attributes for visibility helps to optimize, but I thought it should work by default. I know the C++ standard does not care about shared library, but does it means that using shared libraries breaks the standard? (or at least this implementation is not C++ standard conform?)
Bonus: how can I fix it? (and not using template is not an acceptable answer:))
Header:
template<class T>
struct TemplatedClassWithStatic {
static int value;
};
template<class T>
int TemplatedClassWithStatic<T>::value = 0;
shared.cpp:
#include "TemplateWithStatic.hpp"
int *addressFromShared() {
return &TemplatedClassWithStatic<int>::value;
}
main.cpp:
#include "TemplateWithStatic.hpp"
#include <cstdio>
int *addressFromShared();
int main() {
printf("%p %p\n", addressFromShared(), &TemplatedClassWithStatic<int>::value);
}
And building, looking at the symbols definitions:
producing .so:
g++-4.8 -shared src/shared.cpp -o libshared.so -I include/ -fPIC
compiling and linking main:
g++-4.8 src/main.cpp -I include/ -lshared -L.
symbols are marked as "unique":
nm -C -A *.so a.out | grep 'TemplatedClassWithStatic<int>::value'
libshared.so:0000000000200a70 u TemplatedClassWithStatic<int>::value
a.out:00000000006012b0 u TemplatedClassWithStatic<int>::value
producing .so
~/project/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-g++ -o libshared.so src/shared.cpp -I include/ --sysroot=/Users/amini/project/android-ndk-r9/platforms/android-14/arch-arm/ -shared
compiling and linking main
~/project/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-g++ src/main.cpp libshared.so -I include/ --sysroot=${HOME}/project/android-ndk-r9/platforms/android-14/arch-arm/ -I ~/project/android-ndk-r9/sources/cxx-stl/gnu-libstdc++/4.8/include -I ~/project/android-ndk-r9/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi-v7a/include -I ~/project/android-ndk-r9/sources/cxx-stl/gnu-libstdc++/4.8/include/backward -I ~/project/android-ndk-r9/platforms/android-14/arch-arm/usr/include ~/project/android-ndk-r9/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi-v7a/libgnustl_static.a -lgcc
symbols are weak!
nm -C -A *.so a.out | grep 'TemplatedClassWithStatic<int>::value'
libshared.so:00002004 V TemplatedClassWithStatic<int>::value
a.out:00068000 V TemplatedClassWithStatic<int>::value
Edit, note for the context: I was playing with OOLua, a library helping binding C++ to Lua and my unittests were failing when I started to target Android. I don't "own" the code and I would rather modifying it deeply.
Edit, to run it on Android:
adb push libshared.so data/local/tmp/
adb push a.out data/local/tmp/
adb shell "cd data/local/tmp/ ; LD_LIBRARY_PATH=./ ./a.out"
0xb6fd7004 0xb004
Android does not support unique symbols. It is a GNU extension of ELF format that only works with GLIBC 2.11 and above. Android does not use GLIBC at all, it employs a different C runtime called Bionic.
(update) If weak symbols don't work for you (end update) I'm afraid you would have to modify the code such that it does not rely on static data.
There may be some compiler/linker settings that you can tweak to enable this (have you looked at the -fvisibility flag?).
Possibly a GCC attribute modifier may be worth trying (explicitly set __attribute__ ((visibility ("default"))) on the variable).
Failing that, the only workarounds I could suggest are: (all are somewhat ugly):
Explicitly instantiate all forms of the template that are created in the shared library and provide the initializers in its implementation (not in the header). This may or may not work.
Like (1) but use a shim function as a myers singleton for the shared variable (example below).
Allocate a variable in a map for the class based upon rtti (which might also fail across a shared library boundary).
e.g.
template<class T>
struct TemplatedClassWithStatic {
static int& getValue() { return TemplatedClassWithStatic_getValue((T const*)0); }
};
// types used by the shared library.. can be forward declarations here but you run the risk of violating ODR.
int& TemplatedClassWithStatic_getValue(TypeA*);
int& TemplatedClassWithStatic_getValue(TypeB*);
int& TemplatedClassWithStatic_getValue(TypeC*);
shared.cpp
int& TemplatedClassWithStatic_getValue(TypeA*) {
static int v = 0;
return v;
}
int& TemplatedClassWithStatic_getValue(TypeB*) {
static int v = 0;
return v;
}
int& TemplatedClassWithStatic_getValue(TypeC*) {
static int v = 0;
return v;
}
The executable would also have to provide implementations for any types that it uses to instantiate the template.

Link dynamic shared library in Linux - Undefined reference to function

I know there are many questions related to shared libraries on Linux but maybe because I'm tired of having a hard day trying to create a simple dynamic library on Linux (on Windows it would have taken less than 10 minutes) I can't find what happens in this case.
So, I am trying to create a library to be linked at build-time and used at run-time (not a static library, not a library to be embedded into the executable, in other words). For now it contains a simple function. These are my files:
1.
// gugulibrary.cpp
// This is where my function is doing its job
#include "gugulibrary.h"
namespace GuGu {
void SayHello() {
puts("Hello!");
}
}
2.
// gugulibrary.h
// This is where I declare my shared functions
#include <stdio.h>
namespace Gugu {
void SayHello();
}
3.
// guguapp.cpp
// This is the executable using the library
#include "gugulibrary.h"
int main() {
GuGu::SayHello();
return 0;
}
This is how I try to build my project (and I think this is what is wrong):
gcc -Wall -s -O2 -fPIC -c gugulibrary.cpp -o gugulibrary.o
ld -shared -o bin/libGugu.so gugulibrary.o
gcc -Wall -s -O2 guguapp.cpp -o bin/GuGu -ldl
export LD_LIBRARY_PATH=bin
This is saved as a .sh file which I click and execute in a terminal. The error I get when trying to link the library is this:
/tmp/ccG05CQD.o: In function `main':
guguapp.cpp:(.text.startup+0x7): undefined reference to `SayHello'
collect2: ld returned 1 exit status
And this is where I am lost. I want the library to sit in the same folder as the executable for now and maybe I need some symbols/definitions file or something, which I don't know how to create.
Thanks for your help!
In your C++ file, GuGu::SayHello is declared as a C++ symbol. In your header, you are wrapping it in an extern "C" block. This is actually undefined, as you aren't allowed to use C++ syntax (namespace) in that context. But my guess is that what the compiler is doing is ignoring the namespace and generating a C symbol name of "SayHello". Obviously such a function was never defined by your library. Take out the extern "C" bits, because your API as defined cannot be used from C anyway.
You are inconsistent with your GuGu, there are also Gugu's running around, this needs to be made consistent, then it works (At least on my computer are some Gugu's now)

How to link a C++ shared library with gcc

I have seen a GCC link with a C++ shared library, but I am not able to reproduce it on my own. So first I create a C++ library with a testfunction:
g++ -shared -o libtest.so test.c
Then I have a test main function which calls the library function and compile it like this
gcc -o prog.out main.c -L. -ltest
Then i receive the error
undefined reference to 'testfunc'
which i think is caused by different refernce in the library ... C names the function testfunc and C++ names the function [some stuff]__testfunc[maybe again some stuff].
I have also tried to use
gcc -o prog.out main.c -l:libtest.so
but this results in the same error.
Therefore, my question is: How is it possible to link a c++ library with gcc to a c file?
Update: I know i can use extern "C", but that's not the way it is solved. Maybe there are some parameters for the linker instead?
Update2: Just thought it could also be possible that the first part is just compiled with c++ and linked with gcc. Also tried this:
g++ -c testlib.c -o testlib.o
gcc -shared -o libtest.so testlib.o
gcc -o prog.out -l:libtest.so
still doesn't work. Is there something wrong with the flags?
Yes, the problem has nothing to do with shared libraries (I think...) and everything to do with name mangling.
In your header, you must declare the function like this:
#ifdef __cplusplus
extern "C" {
#endif
void testfunc(void);
#ifdef __cplusplus
}
#endif
This will cause testfunc to have the same symbol and calling conventions for both C and C++.
On the system I'm using right now, the C symbol name will be _testfunc and the C++ symbol name (assuming you don't use extern "C") will be __Z8testfuncv, which encodes information about the parameter types so overloading will work correctly. For example, void testfunc(int x) becomes __Z8testfunci, which doesn't collide with __Z8testfuncv.
When you use g++ it compiles ALL source as C++. This means all function use the C++ ABI (this also including name mangling). When you use gcc it compiles *.c files using the C ABI (no name mangling).
Thus the same function compiles with the two different compilers will generate different functions (in a lot of ways). That's because they are different languages.
To force g++ to compile a function using the C ABI prefix it with extern "C"
extern "C" void testfunc(char*);
Alternatively use the block version
extern "C" {
<multiple Functions>
}
To be honest I never compile anything with gcc anymore (unless there is some hard requirement to do so (in which case I usually fix the code so it works in C++)). If you compile all files with g++ just makes the processes simpler.
If you are sure it's not because of name mangling. Then it means gcc could not find the library try giving the full path of the library unless the .so file is in standard location. If you are not sure then recheck for any conflict in variable (function) name. Use namespaces to group classes and define the functions inside the classes to avoid naming conflict