Template class with static members across multiple DLL/DSO - c++

There were many questions about C++ template classes which contain static member variables, as well as about exporting them from dynamic libraries or shared objects. But this one is a bit deeper: what to do if there are multiple shared objects, each of them having its own set of instantiations but possibly using instantiations from another shared object?
Consider the following example code:
/* file: common.h */
#include <stdio.h>
#define PRINT fprintf (stderr, "(template %d) %d -> %d\n", parameter, data, new_data)
template <int parameter>
class SharedClass
{
static int data;
public:
static void Set(int new_data) { PRINT; data = new_data; }
};
template <int parameter>
int SharedClass<parameter>::data = parameter;
/* file: library1.h */
extern template class SharedClass<1>;
void Library1Function();
/* file: library1.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
template class SharedClass<1>;
void Library1Function()
{
SharedClass<1>::Set (100);
SharedClass<2>::Set (200);
}
/* file: library2.h */
extern template class SharedClass<2>;
void Library2Function();
/* file: library2.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
template class SharedClass<2>;
void Library2Function()
{
SharedClass<1>::Set (1000);
SharedClass<2>::Set (2000);
}
/* file: main.cpp */
#include "common.h"
#include "library1.h"
#include "library2.h"
int main()
{
Library1Function();
Library2Function();
SharedClass<1>::Set (-1);
SharedClass<2>::Set (-2);
}
Let's then assume we build the two libraries and an application using GCC:
$ g++ -fPIC -fvisibility=default -shared library1.cpp -o lib1.so
$ g++ -fPIC -fvisibility=default -shared library2.cpp -o lib2.so
$ g++ -fvisibility=default main.cpp -o main -Wl,-rpath=. -L. -l1 -l2
And then run the executable, we'll get the following result:
$ ./main
(template 1) 1 -> 100
(template 2) 2 -> 200
(template 1) 100 -> 1000
(template 2) 200 -> 2000
(template 1) 1000 -> -1
(template 2) 2000 -> -2
Which means that both libraries and the executable access the same per-template static storage.
If we run "nm -C" on the binaries, we'll see that each static member is defined only once and in the corresponding library:
$ nm -C -A *.so main | grep ::data
lib1.so:0000000000001c30 u SharedClass<1>::data
lib2.so:0000000000001c30 u SharedClass<2>::data
But I've got some questions.
Why, if we remove the extern template class ... from both headers, we'll see that the static members are present in each binary, but the test application will continue to work properly?
$ nm -C -A *.so main | grep ::data
lib1.so:0000000000001c90 u SharedClass<1>::data
lib1.so:0000000000001c94 u SharedClass<2>::data
lib2.so:0000000000001c94 u SharedClass<1>::data
lib2.so:0000000000001c90 u SharedClass<2>::data
main:0000000000401e48 u SharedClass<1>::data
main:0000000000401e4c u SharedClass<2>::data
Is it possible to build this under MSVC?
Or, more specifically, how to deal with __declspec(dllexport) and __declspec(dllimport) to make some instantiations exported, and some - imported?
And, finally: is this an example of undefined behavior?

To answer point 1: When the dynamic linker resolves symbols, it uses a list of modules to link against. The first module loaded is checked first, then the second, and so on.
IIRC, when the data member is used in main, lib1.so, and lib2.so, this is still treated as a dynamic symbol reference, even though the member is declared in the same module. So when the linker goes to resolve the symbols when you run the program, all three modules wind up using the data member implementation in just one of the three modules: whichever was loaded first. The other two pairs are still loaded into memory, but are unused.
(Try std::cout << &(SharedClass<n>::data) << std::endl in all three modules; the address printed should be the same for all six cases.)
To answer point 3, I don't believe this behavior is undefined at all. What happens exactly depends on your system's dynamic linker, but I don't know of any linker that wouldn't handle this situation in exactly the same way.
I can't speak to point 2 since I don't have a whole lot of experience with MSVC.

Related

Singleton across compilation units: linking library vs linking objects

I apologize if the title is not fully self-explanatory. I'm trying to understand why my singleton factory pattern is not working properly, and I ran into a bizarre difference when using library vs linking single objects files.
Here's a simplified version of the code:
main.cpp
#include <iostream>
#include "bar.hpp"
int main (int /*argc*/, char** /*argv*/)
{
A::get().print();
return 0;
}
bar.hpp
#ifndef BAR_HPP
#define BAR_HPP
#include <iostream>
class A
{
public:
static A& get ()
{
static A a;
return a;
}
bool set(const int i)
{
m_i = i;
print();
return true;
}
void print ()
{
std::cout << "print: " << m_i << "\n";
}
private:
int m_i;
A () : m_i(0) {}
};
#endif // BAR_HPP
baz.hpp
#ifndef BAZ_HPP
#define BAZ_HPP
#include "bar.hpp"
namespace
{
static bool check = A::get().set(2);
}
#endif // BAZ_HPP
baz.cpp
#include "baz.hpp"
Now, I build my "project" in two ways:
Makefile:
all:
g++ -std=c++11 -c baz.cpp
g++ -std=c++11 -o test main.cpp baz.o
lib:
g++ -std=c++11 -c baz.cpp
ar rvs mylib.a baz.o
g++ -std=c++11 -o test main.cpp mylib.a
Here are the outputs I get:
$ make all
$ ./test
print: 2
print: 2
$ make lib
$ ./test
print: 0
In the first case the call to A::get().set(2) in baz.hpp takes place, and the same instantiation of A is then used in the main function, which therefore prints 2. In the second case, the call to A::get().set(2) in baz.hpp never takes place, and in the main function the value set by the constructor (that is, 0) is printed.
So finally I can ask my question: why is the behavior different in the two cases? I would expect that either both print 0 once or print 2 twice. I always assumed that a library was just a compact way to ship object files, and that the behavior of linking mylib.a should be the same as that of linking baz.o directly. Why isn't that the case?
Edit: the reason, as explained by Richard, is that no symbols defined in baz.cpp are required in main.cpp, so baz.o is not extracted from the library and linked. This raises another question: is there a workaround to ensure that the instruction A::get().set(2) is executed? I would like to avoid making the singleton a global object, but I'm not sure it's possible. I would also like to avoid to include baz.hpp in the main, since there may be many bazxyz.hpp and that would require main.cpp to know in advance all of them, defying the whole purpose of the factory-like registration process...
If this is to be a static library, then some module somewhere is going to have to address something in each implementation file of the objects that are going to register themselves with the factory.
A reasonable place for this would be in bar.cpp (which is a file you don't yet have). It would contain some or all of the implementation of A plus some means of calling the registration functions the widgets you're going to create.
Self-discovery only works if the object files are linked into the executable. This gives the c++ startup sequence a chance to know about and construct all objects with global linkage.

Is there any linker flag that tells to defer the loading of dynamic library after dlopen [duplicate]

This question already has answers here:
C++ static initialization order
(6 answers)
Closed 6 years ago.
I have following code
File hello.cc
static A dummyl;
A:: A() {
fun();
}
void A::fun() {
int y = 10;
int z = 20;
int x = y + z;
}
File hello.h
class A {
public:
A a;
void fun();
};
File main.cc
#include <dlfcn.h>
#include "hello.h"
typedef void (*pf)();
int main() {
void *lib;
pf greet;
const char * err;
printf("\n Before dlopen\n");
lib = dlopen("libhello.so", RTLD_NOW | RTLD_GLOBAL);
if (!lib) {
exit(1);
}
A *a = new A ;
a->fun();
dlerror(); /*first clear any previous error; redundant in this case but a useful habit*/
dlclose(lib);
return 0;
}
Build phases:
g++ -fPIC -c hello.cc
g++ -shared -o libhello.so hello.o
g++ -o myprog main.cc -ldl -L. -lhello
Since my shared library in real case is libQtCore.so , I need to link it as using -lQtCore in linker because I cannot use the symbols directly and since there are many of functions then libQtCore, it will not practically advisable to use dlysym for each function of libQtCore.so
Since I link, my static global variables gets initialized before dlopen. Is there any flag for linker g++ that tells compiler to load the shared library only after _dlopen _?
Is there any flag for linker g++ that tells compiler to load the shared library only after _dlopen _?
Yes. On Windows. It's known as delay loading. It doesn't exist on Linux/OS X. It's possible to implement it, but it'd require modifications to binutils and ld.so. See this article for some background on that.
But you don't need to care about any of it. Really.
You are facing a well known problem. So well known that it even has a name: The Static Initialization Order Fiasco.
The solution to all your woes is trivial: don't use static global variables the way you do.
To fix it, use Q_GLOBAL_STATIC, or reimplement something like it yourself. That way the value will be constructed at the time it's first used, not prematurely. That's all there's to it.
Side note: Your recent questions suffer badly by being an XY Problem. You're trying to come up with all sorts of Rube Goldberg-esque solutions to a rather simple issue that it took you a week+ to finally divulge. Ask not about a potential solution, but about the underlying issue is you're attempting to solve. All the library loading stuff is completely and utterly unrelated to your problem, and you don't need to concern yourself with it at all.

Xcode does not share static variable with shared library

I've been working on a couple established C++ projects that use static variables from a shared library to store parameters. When compiled with g++ or clang++, the static variable is shared (has the same memory location) throughout the entire program. However, when compiled with Xcode, the main function static variable has a different memory location than the shared library static variable. Is there a way to get Xcode to compile/run the code the same as g++ or clang++, while still being able to debug with Xcode?
Please see example below:
main.cpp:
#include <iostream>
#include "Params.hpp"
int main(int argc, const char * argv[]) {
Params param = Params();
param.addParams();
std::vector<int> vi = Params::ParamsObj();
vi.push_back(10);
for(std::vector<int>::iterator it = vi.begin(); it != vi.end(); ++it) {
std::cout << "i = " << *it << std::endl;
}
return 0;
}
Params.hpp:
#ifndef Params_hpp
#define Params_hpp
#include <vector>
class Params{
typedef std::vector<int> ParamVector;
public:
static ParamVector& ParamsObj() {
static ParamVector m;
return m;
}
void addParams();
};
#endif /* Params_hpp */
Params.cpp:
#include "Params.hpp"
void Params::addParams(){
Params::ParamsObj().push_back(5);
}
Makefile:
clang:
clang++ -dynamiclib Params.cpp -o libshared_clang.dylib
clang++ main.cpp -o main_clang ./libshared_clang.dylib
gpp:
g++-mp-4.9 -Wall -shared -fPIC -o libshared_gpp.so Params.cpp
g++-mp-4.9 -Wall -o main_gpp main.cpp ./libshared_gpp.so
Output from both g++ and clang++ is:
i = 5
i = 10
While Xcode only outputs i = 10.
If I don't use a shared library and compile everything into one binary, Xcode will properly output both print statements.
My current solution is to add the project's main function into its own shared library and then create an Xcode specific file which merely calls the main function in the newly created shared library. However, I was hoping for a solution that didn't require changing the underlying project's code.
I'm pretty sure that if you turn on optimalization for gcc/clang (which you did not in your example), they will produce the same behavior as your compilation with XCode (which isn't a compiler, but an IDE).
Your problem is that the ParamsObj() function is inline (defining it in the class body adds an implicit inline keyword to it), allowing the compiler to just "paste" it into the main method instead of calling it.
With dll boundaries, this might result in the allocation of multiple static variables, if the function is used in multiple libraries (in your case, it's used in the dll, and inlined into the main executable).
Refactor the ParamsObj() method into a declaration and a separate definition in the corresponding C++ file, and you'll get the same behavior everywhere, printing both numbers.

Both static variables and global variables show different addresses in dynamic library and static library on Linux?

I have encountered on CentOS 6.5. As I have searched online that static variable behaves differently on Windows and on Linux when using dynamic library. That is, Windows would cause duplication of variables and Linux would not, like this one:
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
However, when I wrote a small program to validate this, I found that Linux also causes duplication. Here is my small program, including four files:
(1) A.h
#ifndef A_H
#define A_H
#include <cstdio>
static int b;
extern "C" class A {
public:
int mem;
A() {
printf("A's address: %p\n", this);
printf("B's address: %p\n", &b);
}
void print() {
printf("%p: %d\n", this, mem);
}
~A() {
printf("DELETE A!!!!! %p\n", this);
}
};
extern A a;
#endif
(2) A.cpp
#include "A.h"
A a;
(3) d.cpp
#include "A.h"
extern "C" void exec() {
a.print();
}
(4) main.cpp
#include "A.h"
#include <dlfcn.h>
typedef void (*fptr) ();
int main() {
a.mem = 22;
a.print();
void *handle;
handle = dlopen("d.so", RTLD_LAZY);
fptr exec = reinterpret_cast<fptr>(dlsym(handle, "exec"));
(*exec)();
dlclose(handle);
return 0;
}
Here is how I compile and run my program:
g++ d.cpp A.cpp -shared -rdynamic -o d.so -ldl -I. -fPIC -g -std=c++1y
g++ main.cpp A.cpp -ldl -I. -g -std=c++1y
./a.out
Both the dynamic part d.cpp and the static part main.cpp use the variables a and b declared in A.cpp and A.h. And here is the result of the program on my machine:
A's address: 0x600f8c
B's address: 0x600f90
0x600f8c: 22
A's address: 0x7fb8fe859e4c
B's address: 0x7fb8fe859e50
0x7fb8fe859e4c: 0
DELETE A!!!!! 0x7fb8fe859e4c
DELETE A!!!!! 0x600f8c
This surprises me a lot, because the addresses of global variable a and static variable b should be the same in the dynamic part and the static part. And it seems that modification on a in static part does not effect the a in dynamic part. Would anyone please answer my question, or help find out some mistakes in the program (if any)?
By the way, to be honest, on another project I am working on, I find that addresses of global variables are the same in dynamic library and in static library. But that project is too big and I cannot provide a small program to reproduce the behavior.
Thanks a lot !
The first command you showed builds a shared object d.so. Based on the context of your question, I surmise that you also intended to link with d.so, but your second command seems to be missing that part. I'm assuming that it's a typo, as this is the only explanation for the program output you showed -- that A.cpp is both linked to directly, and is also built into your d.so library.
Given that, quoting from the article you linked:
Object code routines used by both should not be duplicated in each.
This is especially true for code which use static variables such as
singleton classes. A static variable is global and thus can only be
represented once. Including it twice will provide unexpected results.
But that's exactly the rule you seem to be breaking, you're representing the statically-scoped instance of the A class twice, in your d.so, and in your main application executable.
So, that seems to be the indicated outcome: "unexpected results".

initialisation of static object when linking against a static library

What are the rules for initialisation of static object declared in another shared library? For instance, consider the following:
file X.hpp:
struct X {
X ();
static X const s_x;
};
struct Y {
Y (X const &) {}
};
file X.cpp:
#include "X.hpp"
#include <iostream>
X::X ()
{
std::cout << "side effect";
}
X const X::s_x;
I compiled X.cpp in a static library libX.a, and I tried to link the following executable against it (file main.cpp):
#include "X.hpp"
int main ()
{
(void)X::s_x; // (1)
X x = s_x; // (2)
Y y = s_x; // (3)
}
with only (1) or (2), nothing happens. But if I add (3), the static object is initialised (i.e. "side effect" is printed). (I use gcc 4.6.1).
Is there any way to predict what will happen here?
I don't understand how the instruction (2) does not force the X::s_x object to be default-constructed, whereas (3) does.
EDIT: build commands:
g++ -c X.cpp
g++ -c main.cpp
ar rcs libX.a X.o
g++ -o test main.o -L. -lX
By default on many platforms, if your program doesn't reference any symbols from a given object file in a static library, the whole object file (including static initializers) will be dropped. So the linker is ignoring X.o in libX.a because it looks like it is unused.
There are a few solutions here:
Don't depend on the side-effects of static initializers. This is the most portable/simple solution.
Introduce some fake dependency on each file by referencing a dummy symbol in a way the compiler will not see through (like storing the address into a externally-visible global).
Use some platform-specific trick to retain the objects in question. For example, on Linux you can use -Wl,-whole-archive a.o b.a -Wl,-no-whole-archive.