Let's say I have libA.so with version 1.1 and 1.2, both have the same symbols defined.
what happens if myApp need a symbol from libA but mistakenly linked both versions:
ld -o myApp -Lpath -lA_1_1 -lA_1_2
Am I right it will use symbols from the first one as long as it can find it?
Am I right it will use symbols from the first one as long as it can find it?
Yes, in general. And that might not be what you want.
If a function foo() in libA_1_2 calls another public function in the library, bar(), then it will use the symbol from libA_1_1, which might do the wrong thing (for example if v1.2 of the library was changed so that foo() expects bar() to deallocate some memory, but the v1.1 version of bar() doesn't do that).
Related
I am building my project in C++. I have these two libraries: shared_lib1 and static_lib2. Both have a function with the same name as func(). Only func() in static_lib2 has an assert statement. shared_lib1 is a shared library and static_lib2 is a static library.
and lib3 has dependency on shared_lib1.
I can't change any of these two libs: shared_lib1, static_lib2, because they are third-party libraries. lib3 is available either as a dynamic library or a static library.
I use lib3 as a shared library: shared_lib3. When this shared_lib3 was built, ldd command shows its dependency on shared_lib1. So I think shared_lib3 should use func() in shared_lib1.
In my project, it links to both shared_lib3 and static_lib2, and src1.cpp calls a function func3() in shared_lib3, which is supposed to call func() in shared_lib1, but the program aborted with a core dump and the backtrace shows it aborted because of the assert statement in func() from static_lib2.
So I think shared_lib3 should use func() in shared_lib1.
That is not how shared libraries work on UNIX. See this answer.
When you link static_lib2 into the main executable, the func() defined there gets to the head of the queue, and will be used to resolve all calls to func() regardless of where these calls come from. This is one of the major differences between UNIX shared libraries and windows DLLs.
There are a few things you may try to fix this symbol collision problem.
If you are using Linux or another ELF platform, you may be able to "hide" the func() symbol in the main executable. See man objcopy and --localize-symbol. This will work if nothing calls func() outside the .o file (part of static_lib2) which defines that function.
If that doesn't work, you could try to turn the func() definition into a HIDDEN symbol (which will prevent it from being exported from the main executable). See this answer and the answer it references.
If that doesn't work, objcopy can also rename the func() symbol to a different name -- see --redefine-sym flag.
In Linux when I load shared library using dlopen() from executable (or shared library) I expect that undefined symbols in this library will be automatically resolved, of course as long as executable (or shared library) defines these symbols.
For example, I have library A with these header and source files:
#pragma once
int funcA();
#include "A/header.h"
int funcA() {}
also I have library B with this source file:
#include "A/header.h"
void funcB() {
funcA();
}
for library B I specify path to header of library A, but I don't link library A to library B.
In this case, if I load library B from library A by calling dlopen(), then undefined symbol funcA in library B will be resolved, so library B will be able to call funcA.
Is it also true for Windows, or I have to manually find addresses for all symbols I need?
After researching already answered questions on Stack Overflow:
External symbol resolving in a dll
Compile to .dll with some undefined references with MinGW
I realized that if I want to make something similar work on Windows, I have to create some import library for my shared library A.
At first I thought it's needed only for MSVC, but looks like MinGW needs import library too, because it's how things work on Windows. Correct me if I miss something.
For me it's big no-no, so probably I will change a way how I work with shared libraries to explicitly retrieve every symbol I need via additional interface. Fortunately, there are not so many of them.
This code:
void undefined_fcn();
void defined_fcn() {}
struct api_t {
void (*first)();
void (*second)();
};
api_t api = {undefined_fcn, defined_fcn};
defines a global variable api with a pointer to a non-existent function. However, it compiles, and to my surprise, links with absolutely no complaints from GCC, even with all those -Wall -Wextra -Werror -pedantic flags.
This code is part of a shared library. Only when I load the library, at run-time, it finally fails. How do I check, at library link-time, that I did't forget to define any function?
Update: this question mentions the same problem, and the answer is the same: -Wl,--no-undefined. (by the way, I guess this could even be marked as duplicate). However, according to the accepted answer below, you should be careful when using -Wl,--no-undefined.
This code is part of a shared library.
That's the key. The whole purpose of having a shared library is to have an "incomplete" shared object, with undefined symbols that must be resolved when the main executable loads it and all other shared libraries it gets linked with. At that time, the runtime loader attempts to resolve all undefined symbols; and all undefined symbols must be resolved, otherwise the executable will not start.
You stated you're using gcc, so you are likely using GNU ld. For the reason stated above, ld will link a shared library with undefined symbols, but will fail to link an executable unless all undefined symbols are resolved against the shared libraries the executable gets linked with. So, at runtime, the expected behavior is that the runtime loader is expected to successfully resolve all symbols too; so the only situation when the runtime loader fails to start the executable will indicate a fatal runtime environment failure (such as a shared library getting replaced with an incompatible version).
There are some options that can be used to override this behavior. The --no-undefined option instructs ld to report a link failure for undefined symbols when linking a shared libraries, just like executables. When invoking ld indirectly via gcc this becomes -Wl,--no-undefined.
However, you are likely to discover that this is going to be a losing proposition. You better hope that none of the code in your shared library uses any class in the standard C++ or C library. Because, guess what? -- those references will be undefined symbols, and you will fail to link your shared library!
In other words, this is a necessary evil that you need to deal with.
You can't have the compiler tell you whether you forgot to define the function in that implementation file. And the reason is when you define a function it is implicitly marked extern in C++. And you cannot tell what is in a shared library until after it is linked (the compiler's linker does not know if the reference is defined)
If you are not familiar with what extern means. Things marked extern signal external linkage, so if you have a variable that is extern the compiler doesn't require a definition for that variable to be in the translation unit that uses it. The definition can be in another implementation file and the reference is resolved at link time (when you link with a translation unit that defines the variable). The same applies for functions, which are essentially variables of a function type.
To get the behavior you want make the function static which tells the compiler that the function is not extern and is a part of the current translation unit, in which case it must be defined -Wundefined-internal picks up on this (-Wundefined-internal is a part of -Werror so just compile with that)
It appears GCC linker doesn't care for one variable being defined in two files. I suspect this is the cause of trouble a 3rd party library is causing us.
Take this:
File a.cpp contains:
int foo;
//do things with it.
File b.cpp contains:
int foo;
//do other things with it.
File c.cpp contains:
extern int foo;
//do other things with it.
They are all compiled by gcc to .o files, then linked as shared object.
gcc -fPIC -c a.cpp
gcc -fPIC -c b.cpp
gcc -fPIC -c c.cpp
ld *.o -shared -soname,mylib -o mylib
The linker doesn't complain at all, but the resulting binary misbehaves. We suspect there are at least a few conflicts of this kind and would like to locate them. What kind of linker options would let us detect them?
(interestingly, if the variables are initialized (int foo=0) in both files, it produces an error).
Hold on now -- are you using foo for two different purposes in the two files? That would certainly lead to run-time errors. If foo needs to be global, then it should be defined in just one module -- the linker may accept it, but you will still only get one copy of foo. If it doesn't need to be global, it should be declared 'static int foo;'
This is a serious design bug in gcc/ld, it does not occur using MSVC. It won't happen linking programs, only shared libraries. When you link a program, the linker ensures all external references are satisfied, at least at link time. When you link a shared library it does not. Instead, external references for which there are no definitions are just left to dangle, the argument (given in ld man page) being that the symbol has to be resolved at load-time dynamically anyhow, so there's no point checking it. It's also hard if you use the stupid feature of shared libraries grabbing symbols from executables.
Your program will not misbehave. If you specify that symbols in a library must be satisfied on loading, you will get a load-time error, if you specify lazy linkage, the error will still occur, but only on the first use of the symbol (AFAIK!)
Some older OS, I believe BSD for example, allowed unsatisfied external pointers to be left as NULL so that you could write an "in program" check to see if the symbol was linked or not. Linux ld at least does not support this AFAIK.
There is a linker switch to force satisfaction of external references for shared libraries, but it is hard to use correctly in portable builds because it requires you to explicitly link the startup library for your processor.
I consider this a very serious design bug, and tried to file a bug report. In my own product we were happy to be able to build under Cygwin because underneath it uses MSVC linker which does not permit this behaviour, quite a lot of bugs were found in my code this way.
It seems compiler option -fno-common forces all the variables to be initialized, so it triggers errors upon linking.
This is my 2nd post on this site in my effort to understand the compilation/linking process with gcc. When I try to make an executable, symbols need to be resolved at link time, but when I try to make a shared library, symbols are not resolved at link time of this library. They will perhaps be resolved when I am trying to make an executable using this shared library. Hands-on:
bash$ cat printhello.c
#include <stdio.h>
//#include "look.h"
void PrintHello()
{
look();
printf("Hello World\n");
}
bash$ cat printbye.c
#include <stdio.h>
//#include "look.h"
void PrintBye()
{
look();
printf("Bye bye\n");
}
bash$ cat look.h
void look();
bash$ cat look.c
#include <stdio.h>
void look()
{
printf("Looking\n");
}
bash$ gcc printhello.c printbye.c
/usr/lib/gcc/i386-redhat-linux/4.1.2/../../../crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
/tmp/cck21S0u.o: In function `PrintHello':
printhello.c:(.text+0x7): undefined reference to `look'
/tmp/ccNWbCnd.o: In function `PrintBye':
printbye.c:(.text+0x7): undefined reference to `look'
collect2: ld returned 1 exit status
bash$ gcc -Wall -shared -o libgreet printhello.c printbye.c
printhello.c: In function 'PrintHello':
printhello.c:6: warning: implicit declaration of function 'look'
printbye.c: In function 'PrintBye':
printbye.c:5: warning: implicit declaration of function 'look'
So my question is why are symbols not resolved when I am linking a shared library. This work(Resolving symbols of its downstream) will need to be done when I will use this library to make an executable, but that means we need to know what this library depends on when using this library, but isn't it not undesirable?
Thanks,
Jagrati
Does adding -z defs when building the library do what you want? If not, check the ld man pages, there are quite a few options on the handling of undefined symbols.
Since you didn't give the -c (compile only) option, you requested gcc to compile the two source files and link them with the standard library (libc) and the c run-time startup (crt0, typically) to produce a running program. crt0 tries to enter your program by calling main(), which is the undefined symbol the linker can't find. It can't find it because you don't have a main() in either of your .c files, right?
So, on to your actual question, "Why symbols of a shared library are not resolved at link time?" The answer is, what do you mean by "link time?" By defintion, a dynamically linked program isn't "linked" until it starts (or maybe not even then, depending on your system.)
On a Linux system, you can see which dynamic libraries a program depends on with the ldd command (on Mac OS use 'otool -L'). The output of ldd will tell you which dynamic libraries a program depends on, which where found in the library search path, and which ones cannot be found (if any).
When you dynamic program starts, the dynamic linker that was linked into it locates and loads the dynamic libraries the program depends on, and "fixes" the references to the external symbols. If any of these fail, your program will fail to start. One all of the formerly unresolved symbols have been resolved, the dynamic linker returns and the C runtime will call your main() function. (It's somewhat different on Mac OS, but similar in effect, the linking happens after your program is started.)
I think the linker option -Bsymbolic is what you're looking for.
The linked has no way of knowing, in ELF at least, where the symbols are (i.e. in which libraries). In OS X, on the other hand, you need to link libraries the way you described. In the end, it is a question of design. One is more flexible, the other, more rigorous.
Even when you build a shared library it must resolve all the dependencies.
Thus when a shared library is loaded at compile time it knows what other shared libraries to load at runtime so that it can resolve other dependencies.
1) Build a shared (look.<sharedLib>) library with look()
2) Build a shared (hg.<sharedLib>) library with hello() bye() link against look.<sharedLib>
3) Build Application with main() that links against hg.<sharedlib>
At runtime the application will then load hg.<sharedlib> which will intern load the shared library look.<sharedlib>
A executable requires a entry point. But a shared library can be built without the entry point and later the executable can be compiled with this shared library.