I have the following problem. I have a shared library, which is just a bunch of translation units linked together so when I compile that shared library I won't get any linker error (undefined references, even though I might have).
The shared library gets loaded dynamically from an executable which also contains the exports which my shared library is using (The references used in my library are resolved at runtime).
The main problem is that I want the undefined reference warnings so I can fix them statically instead of waiting the application to crash.
I read somewhere that I can pass "-Wl,--no-undefined" to gcc so I can get these errors back, indeed it worked but it also gave me all the undefined references of the executable's exports. I want to filter these warnings just to the scope of my translation units.
Is this possible? If not, how can I define reference to a executable which has exports for a shared library.
you can try linking the library & main program with -Wl,-z,now. that should make the runtime ldso resolve all references immediately and throw an error if none are found.
otherwise, i'm not seeing an option off hand in the linker manual to say "allow this ELF to satisfy symbols, but don't actually list it as a DT_NEEDED".
you could try using -Wl,--no-undefined and parsing the output with a script so you can filter out symbols you know will be satisfied by the main program.
another option might be to label all the symbols you know the main program provides with __attribute__((weak)) and then still use -Wl,--no-undefined. the weak symbols won't be reported as an error.
Related
This code:
void undefined_fcn();
void defined_fcn() {}
struct api_t {
void (*first)();
void (*second)();
};
api_t api = {undefined_fcn, defined_fcn};
defines a global variable api with a pointer to a non-existent function. However, it compiles, and to my surprise, links with absolutely no complaints from GCC, even with all those -Wall -Wextra -Werror -pedantic flags.
This code is part of a shared library. Only when I load the library, at run-time, it finally fails. How do I check, at library link-time, that I did't forget to define any function?
Update: this question mentions the same problem, and the answer is the same: -Wl,--no-undefined. (by the way, I guess this could even be marked as duplicate). However, according to the accepted answer below, you should be careful when using -Wl,--no-undefined.
This code is part of a shared library.
That's the key. The whole purpose of having a shared library is to have an "incomplete" shared object, with undefined symbols that must be resolved when the main executable loads it and all other shared libraries it gets linked with. At that time, the runtime loader attempts to resolve all undefined symbols; and all undefined symbols must be resolved, otherwise the executable will not start.
You stated you're using gcc, so you are likely using GNU ld. For the reason stated above, ld will link a shared library with undefined symbols, but will fail to link an executable unless all undefined symbols are resolved against the shared libraries the executable gets linked with. So, at runtime, the expected behavior is that the runtime loader is expected to successfully resolve all symbols too; so the only situation when the runtime loader fails to start the executable will indicate a fatal runtime environment failure (such as a shared library getting replaced with an incompatible version).
There are some options that can be used to override this behavior. The --no-undefined option instructs ld to report a link failure for undefined symbols when linking a shared libraries, just like executables. When invoking ld indirectly via gcc this becomes -Wl,--no-undefined.
However, you are likely to discover that this is going to be a losing proposition. You better hope that none of the code in your shared library uses any class in the standard C++ or C library. Because, guess what? -- those references will be undefined symbols, and you will fail to link your shared library!
In other words, this is a necessary evil that you need to deal with.
You can't have the compiler tell you whether you forgot to define the function in that implementation file. And the reason is when you define a function it is implicitly marked extern in C++. And you cannot tell what is in a shared library until after it is linked (the compiler's linker does not know if the reference is defined)
If you are not familiar with what extern means. Things marked extern signal external linkage, so if you have a variable that is extern the compiler doesn't require a definition for that variable to be in the translation unit that uses it. The definition can be in another implementation file and the reference is resolved at link time (when you link with a translation unit that defines the variable). The same applies for functions, which are essentially variables of a function type.
To get the behavior you want make the function static which tells the compiler that the function is not extern and is a part of the current translation unit, in which case it must be defined -Wundefined-internal picks up on this (-Wundefined-internal is a part of -Werror so just compile with that)
What will happen if executable and shared library contain functions with the same name? For example EXE has definition like this:
extern int fund()
{
return 0;
}
and shared library has same definition:
extern int fund()
{
return 1;
}
what function will be called from executable and from shared library:
1 - for Windows?
2 - for unix-base?
PS: When I define AfxWinMain in my MFC application, on startup it will be called instead of the AfxWinMain in the MFC DLL. I need some theory why is it so?
You have answered the question in the heading yourselves already.
Non-shared library dependencies are resolved at link time, not at load time. Once the linker has satisfied that external reference towards a static library, it will stay that way and neither the Windows nor the Unix loader will try to resolve it anymore (the symbol is normally not even "visible" in the binary after the link stage).
When linking against libraries (regardless of static or dynamic), the linker stops searching for a symbol to resolve as soon as it has found a reference that satisfies the requirement and will not look any further in any other (or the same) library for that symbol. That is why you can supply multiple definitions for the same function in libraries (as opposed to object files, those are guaranteed to be searched exhaustively and thus will be checked for duplicate symbols).
Only symbols that need to be resolved at load time are marked as "external shared" and are resolved by the loader at runtime.
I don't see a fundamental difference in this respect between unixoid OSs and Windows.
I have a large Qt project under Ubuntu. Just found that G++ lets me compile AND link code where I'm calling an declared but undefined method. It crashes at runtime at that call.
I couldn't reproduce this behavior with a test project, although I enforced the same g++ command line.
The questions are:
why does it let me do that?
How can I make the linker generate an error?
Edits (based on the comments):
I know it's not optimized away, as it crashes at runtime when I call that method.
I declared and called another identical method with a dummy name - I think something along the lines of gfdsgfdhgasfdhgfa() will do :) - same thing.
The app crashes when the undefined method is called. Sorry for missing this important detail.
The undefined method is not a slot.
Yes, I'm clearing the build dir. I'm using qmake.
Just found there's an utility called nm. If I'm running it with the -u (show undefined only) option on the output .so I can see this method in the list. Why is GCC assuming it's external?
It looks like by default GCC (not only G++) assumes all undefined symbols are externals. Visual Studio doesn't.
Relevant question: Force GCC to notify about undefined references in shared libraries
--allow-shlib-undefined
--no-allow-shlib-undefined
Allows (the default) or disallows undefined symbols in shared
libraries (It is meant, in shared libraries _linked_against_, not the
one we're creating!--Pavel Shved). This switch is similar to --no-un-
defined except that it determines the behaviour when the undefined
symbols are in a shared library rather than a regular object file. It
does not affect how undefined symbols in regular object files are
handled.
The reason that --allow-shlib-undefined is the default is that the
shared library being specified at link time may not be the same as
the one that is available at load time, so the symbols might actually
be resolvable at load time. Plus there are some systems, (eg BeOS)
where undefined symbols in shared libraries is normal. (The kernel
patches them at load time to select which function is most appropri-
ate for the current architecture. This is used for example to dynam-
ically select an appropriate memset function). Apparently it is also
normal for HPPA shared libraries to have undefined symbols.
Description :
a. Class X contains a static private data member ptr and static public function member getptr()/setptr().
In X.cpp, the ptr is set to NULL.
b. libXYZ.so (shared object) contains the object of class X (i.e libXYZ.so contains X.o).
c. libVWX.so (shared object) contains the object of class X (i.e libVWX.so contains X.o).
d. Executable a.exe contains X.cpp as part of translation units and finally is linked to libXYZ.so, libVWX.so
PS:
1. There are no user namespaces involved in any of the classes.
2. The libraries and executable contain many other classes also.
3. no dlopen() has been done. All libraries are linked during compile time using -L and -l flags.
Problem Statement:
When compiling and linking a.exe with other libraries (i.e libXYZ.so and libVWX.so), I expected a linker error (conflict/occurance of same symbol multiple times) but did not get one.
When the program was executed - the behavior was strange in SUSE 10 Linux and HP-UX 11 IA64.
In Linux, when execution flow was pushed across all the objects in different libraries, the effect was registered in only one copy of X.
In HPUX, when execution flow was pushed across all the objects in different libraries, the effect was registered in 3 differnt copies of X (2 belonging to each libraries and 1 for executable)
PS : I mean during running the program, the flow did passed thourgh multiple objects belonging to a.exe, libXYZ.so and libVWX.so) which interacted with static pointer belonging to X.
Question:
Is Expecting linker error not correct? Since two compilers passed through compilation silently, May be there is a standard rule in case of this type of scenario which I am missing. If so, Please let me know the same.
How does the compiler (gcc in Linux and aCC in HPUX) decide how many copies of X to keep in the final executable and refer them in such scenarios.
Is there any flag supported by gcc and aCC which will warn/stop compilation to the users in these kind of scenarios?
Thanks for your help in advance.
I'm not too sure that I've completely understood the scenario. However,
the default behavior on loading dynamic objects under Linux (and other
Unices) is to make all symbols in the library available, and to only use
the first encountered. Thus, if you both libXYZ.so and libVWX.so
contain a symbol X::ourData, it is not an error; if you load them in
that order, libVWX.so will use the X::ourData from libXYZ.so,
instead of its own. Logically, this is a lot like a template definition
in a header: the compiler chooses one, more or less by chance, and if
any of the definitions is not the same as all of the others, it's
undefined behavior. This behavior can be
overridden by passing the flag RTLD_LOCAL to dlopen.
With regards to your questions:
The linker is simply implementing the default behavior of dlopen (that which you get when the system loads the library implicitely). Thus, no error (but the logical equivalent of undefined behavior if any of the definitions isn't the same).
The compiler doesn't decide. The decision is made when the .so is loaded, depending on whether you specify RTLD_GLOBAL or RTLD_LOCAL when calling dlopen. When the runtime calls dlopen implicitly, to resolve a dependency, it will use RTLD_GLOBAL if this occurs when loading the main executable, and what ever was used to load the library when the dependency comes from a library. (This means, of course, that RTLD_GLOBAL will propagate until you invoke dlopen explicitly.)
The function is "public static", so I assume it's OOP-meaning of "static" (does not need instance), not C meaning of static (file-static; local to compilation unit). Therefore the functions are extern.
Now in Linux you have explicit right to override library symbols, both using another library or in the executable. All extern symbols in libraries are resolved using the global offset table, even the one the library actually defines itself. And while functions defined in the executable are normally not resolved like this, but the linker notices the symbols will get to the symbol table from the libraries and puts the reference to the executable-defined one there. So the libraries will see the symbol defined in the executable, if you generated it.
This is explicit feature, designed so you can do things like replace memory allocation functions or wrap filesystem operations. HP-UX probably does not have the feature, so each library ends up calling it's own implementation, while any other object that would have the symbol undefined will see one of them.
There is a difference beetween "extern" symbols (which is the default in c++) and "shared libary extern". By default symbols are only "extern" which means the scope of one "link unit" e.g. an executable or a library.
So the expected behaviour would be: no compiler error and every module works with its own copy.
That leads to problems of course in case of inline compiling etc...etc...
To declare a symbol "shared library extern" you have to use a ".def" file or an compiler declaration.
e.g. in visual c++ that would be "_declspec(dllexport)" and "_declspec(dllimport)".
I do not know the declarations for gcc at the moment but I am sure someone does :-)
This is my 2nd post on this site in my effort to understand the compilation/linking process with gcc. When I try to make an executable, symbols need to be resolved at link time, but when I try to make a shared library, symbols are not resolved at link time of this library. They will perhaps be resolved when I am trying to make an executable using this shared library. Hands-on:
bash$ cat printhello.c
#include <stdio.h>
//#include "look.h"
void PrintHello()
{
look();
printf("Hello World\n");
}
bash$ cat printbye.c
#include <stdio.h>
//#include "look.h"
void PrintBye()
{
look();
printf("Bye bye\n");
}
bash$ cat look.h
void look();
bash$ cat look.c
#include <stdio.h>
void look()
{
printf("Looking\n");
}
bash$ gcc printhello.c printbye.c
/usr/lib/gcc/i386-redhat-linux/4.1.2/../../../crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
/tmp/cck21S0u.o: In function `PrintHello':
printhello.c:(.text+0x7): undefined reference to `look'
/tmp/ccNWbCnd.o: In function `PrintBye':
printbye.c:(.text+0x7): undefined reference to `look'
collect2: ld returned 1 exit status
bash$ gcc -Wall -shared -o libgreet printhello.c printbye.c
printhello.c: In function 'PrintHello':
printhello.c:6: warning: implicit declaration of function 'look'
printbye.c: In function 'PrintBye':
printbye.c:5: warning: implicit declaration of function 'look'
So my question is why are symbols not resolved when I am linking a shared library. This work(Resolving symbols of its downstream) will need to be done when I will use this library to make an executable, but that means we need to know what this library depends on when using this library, but isn't it not undesirable?
Thanks,
Jagrati
Does adding -z defs when building the library do what you want? If not, check the ld man pages, there are quite a few options on the handling of undefined symbols.
Since you didn't give the -c (compile only) option, you requested gcc to compile the two source files and link them with the standard library (libc) and the c run-time startup (crt0, typically) to produce a running program. crt0 tries to enter your program by calling main(), which is the undefined symbol the linker can't find. It can't find it because you don't have a main() in either of your .c files, right?
So, on to your actual question, "Why symbols of a shared library are not resolved at link time?" The answer is, what do you mean by "link time?" By defintion, a dynamically linked program isn't "linked" until it starts (or maybe not even then, depending on your system.)
On a Linux system, you can see which dynamic libraries a program depends on with the ldd command (on Mac OS use 'otool -L'). The output of ldd will tell you which dynamic libraries a program depends on, which where found in the library search path, and which ones cannot be found (if any).
When you dynamic program starts, the dynamic linker that was linked into it locates and loads the dynamic libraries the program depends on, and "fixes" the references to the external symbols. If any of these fail, your program will fail to start. One all of the formerly unresolved symbols have been resolved, the dynamic linker returns and the C runtime will call your main() function. (It's somewhat different on Mac OS, but similar in effect, the linking happens after your program is started.)
I think the linker option -Bsymbolic is what you're looking for.
The linked has no way of knowing, in ELF at least, where the symbols are (i.e. in which libraries). In OS X, on the other hand, you need to link libraries the way you described. In the end, it is a question of design. One is more flexible, the other, more rigorous.
Even when you build a shared library it must resolve all the dependencies.
Thus when a shared library is loaded at compile time it knows what other shared libraries to load at runtime so that it can resolve other dependencies.
1) Build a shared (look.<sharedLib>) library with look()
2) Build a shared (hg.<sharedLib>) library with hello() bye() link against look.<sharedLib>
3) Build Application with main() that links against hg.<sharedlib>
At runtime the application will then load hg.<sharedlib> which will intern load the shared library look.<sharedlib>
A executable requires a entry point. But a shared library can be built without the entry point and later the executable can be compiled with this shared library.