I have a library loaded in runtime. The library contains const static variable e.g. std::map of objects.
I load and unload this library multiple times in runtime using multiple threads.
And GCC thread sanitizer complains that threads have data races regarding this object. E.g. while one thread removes the object, second one creates new one - they conflict with the std::map. I wonder if shared library loaded multiple times have the same addresses for global variables, functions?
Have can I make the code thread safe in this case?
Where can I read about it?
Related
Suppose I have shared library with this function where "i" is some global variable.
int foo() {
return i++;
}
When I call this function from multiple processes the value of "i" in each process is independent on the other processes.
This behavior is quite expected.
I was just wondering how is usually this behavior implemented by the linker? From my understanding the code is shared between processes, so the variable has to have the same virtual address in all address spaces of every program that uses this library. That condition seems quite difficult to accomplish to me so I guess I am missing something here and it is done differently.
Can I get some more detailed info on this subject?
The dynamic linking process at run time (much the same as the static linking process), allocates separate data (and bss) segments for each process, and maps those into the process address space. Only the text segments are shared between processes. This way, each process gets its own copy of static data.
the code is shared between processes, so the variable has to have the
same virtual address in all address spaces of every program that uses
this library
The code is not shared the way you think. Yes the dynamic shared object is loaded only once but the memory references or the stack or the heap that code in the so uses is not shared. Only the section that contains the code is shared.
Each process has it's own unique address space, so when a process access the variable it can have different values then the other process. If the process should share the same memory, they would have to specifically set this up. A shared library is not enough for that.
I have a library (L) that is dynamically loaded by a program (P) using dlopen. L implements a plugin interface and so calls back on it's parent to obtain some functionality.
Inside P is a singleton object that dynamically creates a thread pool object A.
I need access to A from L.
However, because the singleton works by using a static variable, when L is loaded it ends up creatng it's own instance which is some cases would be fine but I want the instance that was created in P. Is there a way around this?
You should not have a static A in L. Let P pass the address of A to L, i.e., L.init(&A).
File scope names declared static have internal linkage. Internal linkage means that they are invisible to other translation units, even in a "classic" linking model without any dynamic libraries. Given that statics are not visible even to other translation units in the same executable, it is not reasonable to expect them to be visible from attached dynamic libraries.
You have to think of a way to achieve the necessary linkage using external, dynamic symbols. Perhaps the singleton simply cannot have an internal name, but must have an external name.
L is creating its own instance of the object not just because the object is static, but because you have linked into L the thread pool module which defines that singleton and the thread pool functions. This can happen even with objects that have external names, depending on how the library is linked.
You must pick a single object in which the thread pool service will reside, and then make sure it only resides there. Doesn't your project have a utility library where you can stick in this sort of thing?
You can adhere to the model that it is the program executable P which provides the thread pool API. This is really the same thing. The program P is another dynamic object and effectively serves as the library for the thread pool module, which it provides to itself and to other shared objects.
Regardless of where that thread pool module lives, make sure that you are not statically linking copies of that module into other objects: it lives just in one place.
If the thread pool singleton's external name is part of that API (everyone knows its documented name and uses it directly, passing that global pool to the API functions) then that name should be made external, and declared in the header file.
If the singleton is to be private, then you have to think of some way of hiding it, like making it implicit in the function calls (there is only one thread pool, and that is that) or else abstracting the access to it somewhat (provide an ensure_thread_pool) function which creates a thread pool if one does not exist, or else returns the previously created one, in a thread safe way.
Think: why do not, for instance, stdin and stdout have this problem? Why doesn't every library instantiate its own stdout stream and call its own fprintf function on that stream? Why, obviously, because these things live in one place: the C library. Copies of them do not live in other places; other places just use them by reference via the dynamic symbols.
I have a static library which contains singletons. I need to load a separate instance of those singletons in the same process for testing purposes.
So I have created a DLL which links the same static library, and then the main process loads that DLL.
As soon as the DLL tries to load, I get access violations when trying to access the static instance pointers in the singletons.
Some posts that I have read say that it's impossible and that I need a second process, while others say that each DLL gets it's own copies of all the static variables in the static library it links, which suggests that this should work..
Is what I am trying to do possible?
Most of the time a singleton is really meant to be only one - your request is unusual.
I know that linking a static library into a DLL can result in multiple instances of static variables, because I've seen it myself. Each DLL or EXE gets its own copy of the static library via the linker, and thus its own copy of the static variables.
The access violations may come from problems with initialization order. The best way to control that is to make sure the static variables are within a function that initializes them just-in-time, rather than global variables.
Sorry if this question has been answered before; however all of the questions that are similar seem to be related to global or static variables in a DLL and sharing of those.
Is it possible to have one instance of a dll shared between two separate applications?
I have two applications (appA, appB) and a DLL (theDLL).
I am seeing if it is possible for appA to call a function in theDLL which then creates a variable on the heap and stores that in theDLL. At a later time, I would like to have appB connect to theDLL and be able to access that variable created earlier. Again, sorry if this answer is the same as static and global variables in dlls.
Here is some psuedo code:
(theDLL)
class VariableHolder
{
public:
void StoreVariable(int x)
{
mInt = new int(x);
}
int GetVariable()
{
return mInt;
}
private:
int mInt;
}
(appA)
int main()
{
...
(assuming we got access to a VariableHolder singleton created in theDLL)
theVarialbeHolder.StoreVariable(5);
...
}
(appB)
int main()
{
...
(assuming we got access to a VariableHolder singleton created in theDLL)
if (theVarialbeHolder.GetVariable() == 5)
{
cout << "Hurray, how did you know?";
}
...
}
This exactly is not possible - as the address spaces for the two processes are different (because they're virtual, having been created by the kernel), so a valid pointer in one won't work within the other. However, you can use shared memory to transport raw scalar data (strings, integers) between processes - here's how.
Yes, this is possible using shared memory. It doesn't need to use a shared DLL though.
Depending on the operating, the approaches are somewhat different:
On Windows, a shared file is used on mapped into memory (see Creating Named Shared Memory).
On Linux and Unix, there are direct functions to create shared memory areas, e.g. System V IPC. Just google for it.
Shared libraries on almost any modern operating system are implemented by shared read-only executable and data pages, mapped simultaneously into the address space of any process that uses the given library. On Windows though (in contrast to most Unix system) this sharing can also be extended to read-write data segments in DLLs, so it is possible to have global variables in a DLL, shared among all images that have the DLL loaded. To achieve this, there is a two-step process involved. First you tell the compiler to put the shared variables in a new named data section:
#pragma data_seg (".myshared")
int iscalar = 0;
int iarray[10] = { 0 };
#pragma data_seg ()
It is important to have all those variables statically intialised otherwise they will end up into the .bss section instead. Then you have to tell the linker that you'd like to have the .myshared section with shared read-write attributes using the /SECTION:.myshared,RWS option.
This mechanism is much simpler than creating and binding to named shared memory objects but it only allows to share statically allocated global variables - you cannot use it to share data on the heap as the heap is private to the process. For anything more complex you should use shared memory mappings, i.e. as shown on the MSDN page, linked in the answer from H2CO3.
This is not possible. The DLL can be shared in the 2 process but the data isn't. It's the code or program image (i.e. the logic or instructions) that is shared and not the data. Every Dll is mapped into the virtual address space of the process that loads it so the data either is on the data section of the process or on stack if it is local to the function. When a process is executing the address of the other process data is not visible.
You need to do some reading on virtual memory and how memory management unit(MMU) works. The OS, CPU, MMU works together to make it possible. The reliable way to do this is inter process communication. You can use shared memory where each process has a copy of data in form of virtual address but it is eventually mapped to same location into the real memory i.e the real address. The OS makes it possible.
This as #H2CO3 pointed out, is not possible because of different address spaces.
However, from your problem, it looks like you need either a surrogate process around that DLL or a Service and then different processes can connect to that surrogate process/exe and use the shared memory.
You must use shared memory (as was written above).
I recommend to use boost interprocess library. See documentation about shared memory - Shared memory between processes
I have a singleton class, and I will compile it as a library static(lib) or dynamic(dll).
Is it guaranteed that calls to same file in a machine always refer to same and unique instance in both cases?
Edit:
What if a process loads two different library and different functions from these libraries want to use instance of singleton class. Now, they are accessing same instance, am I right?
Thanks,
No, different processes will not share a singleton from a common library. Different processes use different memory spaces, so each will instantiate its own singleton.
You'll need to use some sort of shared memory or interprocess-communication to share data between processes.
If a single process loads libraries A and B, and both those libraries use a singleton from library C, then they will be using the same instance.
You don't call files. You load code into your process from the file, the code contains initialisation instructions that construct your singleton. The singleton is thus unique within your process, regardless of what other code within the process uses it. The singleton is not unique from one process to another, even if they use the same shared library.