I compiled a shared library with gcc and linked it to my main. The main class should initialize a logger class, which should be visible inside the shared library, but it looks as if the shared library has it's own instance of it.
The include file looks like this:
extern Log gLog;
In main it is declared.
Log gLog(new StreamWriter());
When I try to link it, I get linker errors undefined symbol _gLog in the shared library. I thought that it might be because it is a class instance, so I changed it to a pointer, but I get the same. To make it worse, I figured I could create a small dummy module where I create the same global variable in the shared library and then call a function to initialize it. But for this function I also get a linker error because it is not visible in main.
In the shared library:
Log *gLogger;
int initLibrary(Log *pLogger)
{
gLogger = pLogger;
}
And in main:
Log gLog(new StreamWriter());
int initLibrary(Log *pLogger);
main()
{
initLibrary(&gLog);
}
Again I get an undefined symbol in the linker, this time for my initLibrary function.
For now I solve the problem by creating a dummy class, which works. However, I would like to know how to properly define symbols across shared library boundaries, as my understanding seems to be wrong about it.
When using google I found some threads here Using a global variable in a shared library and Global variables, shared libraries and -fPIC effect as examples (there are several others as well with this problem). However I tried to recompile everything with -fpic, also the main module and it still doesn't work. The -rdynamic option is unknown so I don't know where this comes from.
I can use classes from the shared library and vice versa, so this affects only global symbols. So what am I doing wrong that the main code and the shared library can not see symbols from each other?
The right approach is to create the instance of the Logger inside the shared library, using either a global variable (better if encapsulated in a namespace) or a Singleton class. Then let your main program use it.
You need to declare you variables with default visibility to make it visible to other shared libraries or the main program. It appears that you are compiling with -fvisibility=hidden, so that a symbol in the library does not resolve to a definition in the main program or other libraries and vice versa. With the visibility attribute of GCC, you can reverse this effect.
In a nutshell
scope visibility of a declaration across entities of a single object file (global, local, ..)
linkage visibility of a declaration across entities of multiple object files (external, internal)
visibility visibility of a declaration across entities of different shared libraries (default, hidden).
Another possibility is that you are mixing C and C++ code and messing up with the language linkage.
I finally found the problem why this didn't work for global symbols(no problem with classes).
I'm compiling under cygwin, and apparently the shared object is compiled as a DLL. I tried to look into the library and notized that it is an EXE format and not ELF. So I tried to use the Microsoft DLL syntax for exporting symbols and suddenly it worked. Adding a __declspec(dllexport) to the symbol did the trick.
I expected that I would have to use __declspec(dllimport) in the main project for importing the symbols, but this doesn't work. Not sure if I missunderstood this parameter or if the cygwin version of gcc does some magic work.
So when you compile a shared library under cygwin it must look like this if you want to export symbols.
Shared library foo.h:
__declspec(dllexport) int foo(int a);
Shared library foo.cpp:
int foo(int a)
{
....
}
In the executable foo.h:
int foo(int a);
In the executable main.cpp:
main()
{
foo(1);
}
The shared library must be compiled with the -fpic switch and linked with -shared.
Related
Porting a C++ library, under CMake, to Windows.
Option 1: Decorate the API with __declspec(dllexport).
Option 2: set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON).
We go for option 2, since it seems much easier, and leaves our sources intact.
Works like a charm: our application compiles and gets linked with the library dll.
However, it ensures only the export of classes and functions. What needs to be done to export global variables? Like in:
lib_source.cpp:
int my_secret;
app.cpp:
extern int my_secret;
For global data symbols, __declspec(dllimport) must still be used when compiling against the code in the .dll.
src: https://cmake.org/cmake/help/latest/prop_tgt/WINDOWS_EXPORT_ALL_SYMBOLS.html
Here’s the code:
struct S
{
S()
{
__debugbreak();
}
};
static const S g_s;
Obviously, I want some code to run at startup.
This only works for some source files, that have symbols referenced by outside code. For source files in a static library that don’t have any symbols referenced from outside, looks like the compiler or linker drop the complete compilation unit, so the global object is not constructed.
Is there a way to force construction of static objects, or otherwise force running of the code at startup?
I’ve double checked compilation settings for these source files, they are identical, and they are in the same static library project.
The static library is used by a DLL. Global objects are expected to be constructed before DLL_PROCESS_ATTACH call.
You will need to link "everything" from this library using linker options such as
-Wl--whole-archive -lmylib -Wl--no-whole-archive (gcc)
or
/INCLUDE symbol (vc)
or
/WHOLEARCHIVE:mylib (vc)
However relying on dynamic initialization stage and / or making assumptions about state of global objects during program startup / dll loading makes your program doomed. So you should consider explicit initialization instead.
This code:
void undefined_fcn();
void defined_fcn() {}
struct api_t {
void (*first)();
void (*second)();
};
api_t api = {undefined_fcn, defined_fcn};
defines a global variable api with a pointer to a non-existent function. However, it compiles, and to my surprise, links with absolutely no complaints from GCC, even with all those -Wall -Wextra -Werror -pedantic flags.
This code is part of a shared library. Only when I load the library, at run-time, it finally fails. How do I check, at library link-time, that I did't forget to define any function?
Update: this question mentions the same problem, and the answer is the same: -Wl,--no-undefined. (by the way, I guess this could even be marked as duplicate). However, according to the accepted answer below, you should be careful when using -Wl,--no-undefined.
This code is part of a shared library.
That's the key. The whole purpose of having a shared library is to have an "incomplete" shared object, with undefined symbols that must be resolved when the main executable loads it and all other shared libraries it gets linked with. At that time, the runtime loader attempts to resolve all undefined symbols; and all undefined symbols must be resolved, otherwise the executable will not start.
You stated you're using gcc, so you are likely using GNU ld. For the reason stated above, ld will link a shared library with undefined symbols, but will fail to link an executable unless all undefined symbols are resolved against the shared libraries the executable gets linked with. So, at runtime, the expected behavior is that the runtime loader is expected to successfully resolve all symbols too; so the only situation when the runtime loader fails to start the executable will indicate a fatal runtime environment failure (such as a shared library getting replaced with an incompatible version).
There are some options that can be used to override this behavior. The --no-undefined option instructs ld to report a link failure for undefined symbols when linking a shared libraries, just like executables. When invoking ld indirectly via gcc this becomes -Wl,--no-undefined.
However, you are likely to discover that this is going to be a losing proposition. You better hope that none of the code in your shared library uses any class in the standard C++ or C library. Because, guess what? -- those references will be undefined symbols, and you will fail to link your shared library!
In other words, this is a necessary evil that you need to deal with.
You can't have the compiler tell you whether you forgot to define the function in that implementation file. And the reason is when you define a function it is implicitly marked extern in C++. And you cannot tell what is in a shared library until after it is linked (the compiler's linker does not know if the reference is defined)
If you are not familiar with what extern means. Things marked extern signal external linkage, so if you have a variable that is extern the compiler doesn't require a definition for that variable to be in the translation unit that uses it. The definition can be in another implementation file and the reference is resolved at link time (when you link with a translation unit that defines the variable). The same applies for functions, which are essentially variables of a function type.
To get the behavior you want make the function static which tells the compiler that the function is not extern and is a part of the current translation unit, in which case it must be defined -Wundefined-internal picks up on this (-Wundefined-internal is a part of -Werror so just compile with that)
What will happen if executable and shared library contain functions with the same name? For example EXE has definition like this:
extern int fund()
{
return 0;
}
and shared library has same definition:
extern int fund()
{
return 1;
}
what function will be called from executable and from shared library:
1 - for Windows?
2 - for unix-base?
PS: When I define AfxWinMain in my MFC application, on startup it will be called instead of the AfxWinMain in the MFC DLL. I need some theory why is it so?
You have answered the question in the heading yourselves already.
Non-shared library dependencies are resolved at link time, not at load time. Once the linker has satisfied that external reference towards a static library, it will stay that way and neither the Windows nor the Unix loader will try to resolve it anymore (the symbol is normally not even "visible" in the binary after the link stage).
When linking against libraries (regardless of static or dynamic), the linker stops searching for a symbol to resolve as soon as it has found a reference that satisfies the requirement and will not look any further in any other (or the same) library for that symbol. That is why you can supply multiple definitions for the same function in libraries (as opposed to object files, those are guaranteed to be searched exhaustively and thus will be checked for duplicate symbols).
Only symbols that need to be resolved at load time are marked as "external shared" and are resolved by the loader at runtime.
I don't see a fundamental difference in this respect between unixoid OSs and Windows.
Description :
a. Class X contains a static private data member ptr and static public function member getptr()/setptr().
In X.cpp, the ptr is set to NULL.
b. libXYZ.so (shared object) contains the object of class X (i.e libXYZ.so contains X.o).
c. libVWX.so (shared object) contains the object of class X (i.e libVWX.so contains X.o).
d. Executable a.exe contains X.cpp as part of translation units and finally is linked to libXYZ.so, libVWX.so
PS:
1. There are no user namespaces involved in any of the classes.
2. The libraries and executable contain many other classes also.
3. no dlopen() has been done. All libraries are linked during compile time using -L and -l flags.
Problem Statement:
When compiling and linking a.exe with other libraries (i.e libXYZ.so and libVWX.so), I expected a linker error (conflict/occurance of same symbol multiple times) but did not get one.
When the program was executed - the behavior was strange in SUSE 10 Linux and HP-UX 11 IA64.
In Linux, when execution flow was pushed across all the objects in different libraries, the effect was registered in only one copy of X.
In HPUX, when execution flow was pushed across all the objects in different libraries, the effect was registered in 3 differnt copies of X (2 belonging to each libraries and 1 for executable)
PS : I mean during running the program, the flow did passed thourgh multiple objects belonging to a.exe, libXYZ.so and libVWX.so) which interacted with static pointer belonging to X.
Question:
Is Expecting linker error not correct? Since two compilers passed through compilation silently, May be there is a standard rule in case of this type of scenario which I am missing. If so, Please let me know the same.
How does the compiler (gcc in Linux and aCC in HPUX) decide how many copies of X to keep in the final executable and refer them in such scenarios.
Is there any flag supported by gcc and aCC which will warn/stop compilation to the users in these kind of scenarios?
Thanks for your help in advance.
I'm not too sure that I've completely understood the scenario. However,
the default behavior on loading dynamic objects under Linux (and other
Unices) is to make all symbols in the library available, and to only use
the first encountered. Thus, if you both libXYZ.so and libVWX.so
contain a symbol X::ourData, it is not an error; if you load them in
that order, libVWX.so will use the X::ourData from libXYZ.so,
instead of its own. Logically, this is a lot like a template definition
in a header: the compiler chooses one, more or less by chance, and if
any of the definitions is not the same as all of the others, it's
undefined behavior. This behavior can be
overridden by passing the flag RTLD_LOCAL to dlopen.
With regards to your questions:
The linker is simply implementing the default behavior of dlopen (that which you get when the system loads the library implicitely). Thus, no error (but the logical equivalent of undefined behavior if any of the definitions isn't the same).
The compiler doesn't decide. The decision is made when the .so is loaded, depending on whether you specify RTLD_GLOBAL or RTLD_LOCAL when calling dlopen. When the runtime calls dlopen implicitly, to resolve a dependency, it will use RTLD_GLOBAL if this occurs when loading the main executable, and what ever was used to load the library when the dependency comes from a library. (This means, of course, that RTLD_GLOBAL will propagate until you invoke dlopen explicitly.)
The function is "public static", so I assume it's OOP-meaning of "static" (does not need instance), not C meaning of static (file-static; local to compilation unit). Therefore the functions are extern.
Now in Linux you have explicit right to override library symbols, both using another library or in the executable. All extern symbols in libraries are resolved using the global offset table, even the one the library actually defines itself. And while functions defined in the executable are normally not resolved like this, but the linker notices the symbols will get to the symbol table from the libraries and puts the reference to the executable-defined one there. So the libraries will see the symbol defined in the executable, if you generated it.
This is explicit feature, designed so you can do things like replace memory allocation functions or wrap filesystem operations. HP-UX probably does not have the feature, so each library ends up calling it's own implementation, while any other object that would have the symbol undefined will see one of them.
There is a difference beetween "extern" symbols (which is the default in c++) and "shared libary extern". By default symbols are only "extern" which means the scope of one "link unit" e.g. an executable or a library.
So the expected behaviour would be: no compiler error and every module works with its own copy.
That leads to problems of course in case of inline compiling etc...etc...
To declare a symbol "shared library extern" you have to use a ".def" file or an compiler declaration.
e.g. in visual c++ that would be "_declspec(dllexport)" and "_declspec(dllimport)".
I do not know the declarations for gcc at the moment but I am sure someone does :-)