I am writing a server side application in C/C++ which consists of 1 main daemon and several child processes.
I want the child processes to be extremely lightweight so that they can be spawned/killed without too much overhead (over and above that imposed by the OS).
I am building the main daemon and the children apps to make extensive use of shared libraries. In fact, the main daemon loads up all the shared libraries required by the child applications, and sets up the required (shared) memory structures etc.
My underlying assumption is that since the shared libraries (some of which are huge) are already loaded by the main daemon, the child applications will be able to launch quickly and simply attach to the loaded libraries - without having to load the shared libs, and thus resulting in a slightly fast time to be spawned - is this assumption correct?
[[Added]]
I am working on Ubuntu 10.0.4 LTS
The code segment of your shared libraries will be shared by all processes, no particular restriction w.r.t who loaded/spawned. However, it may take variable time depending upon how many symbols are used in the process, as those will be resolved during load.
But if you are forking, there isn't much to do so it will be fast with respect to launching new binary.
Related
I recently read this question How to statically link to TBB? and I still don't really understand the problems with using tbb as a statically linked library (which is possible with their makefile if you do make extra_inc=big_iron.inc tbb)
The answer seems to say that the problem is that there can be multiple singletons in a single program, all (most?) implementations of a singletons don't let that happen. I don't understand the reasoning behind this.
Is the problem that when you fork() another process the singleton becomes two separate singletons in two separate processes? Is that what they mean by "program"? Also if thats the case why can they not mmap() shared memory and use that as the communication medium?
Also doesn't dynamically linking only mean that the library itself is shared in memory, i.e. the code segment?
Thanks!
No, the singleton explanation refers to a single process, not the multiple processes case (though, it has some of the same issues with oversubscription and load balancing).
Dynamic linker makes sure there is only one global data section exists for the library and calls global constructors exactly once implementing singleton.
With statically linked TBB library, one can end up with multiple instances of TBB thread pool working in the same process simultaneously, which come from different components of an application. This causes the issue of over-subscription or even worse if somehow a memory or some object being allocated and registered in one instance of the scheduler gets used in another instance of the scheduler. This is especially easy to achieve because of thread-local storage that is heavily used by TBB scheduler. Each instance of the scheduler would use separate TLS breaking rules of nested parallelism up to deadlock and enabling memory leaks and segfaults because tasks allocated in one scheduler might end up being returned to another scheduler. Thus, this situation might not be obvious for developers who don't even intend to pass objects between module boundaries.
Sometimes, such a situation happens even with dynamic linkage when e.g. TBB shared library is renamed for one of application components. TBB team is working to solve this issue.
Soft MetaTrader 5. It's trading terminal. It's "indicator" windows are little cpp-like programs. They can load pure cpp dlls. Every "indicator" works in separate thread.
I need to create shared memory stuff which can be accessable from every "indicator". Also for shared memory could be loaded in every indicator it must be in particular dll.
I found info about boost interprocesses.
I am newbee with boost and multithreading.
So I wonder am I right way?
Create dll with shared memory functionality and interface to access it from indicator.
Load dll in several "indicators".
Access it from several "indicators" in real-time?
Could you also advice other ways?
Global variables in shared libraries are not shared across the library user processes. That segment of data is created for every process which loads the library, only the read-only code segment is actually shared.
You need to use a library for shared memory, such as boost::interprocess shared_memory_object or POSIX Shared Memory, or Qt's QSharedMemory. That is however in case you need inter-process communication.
There is nothing special you need to do in order for multiple threads to access shared memory in the same process, aside from using a mutex to prevent data races.
I'm working on a project that involves legacy systems written in Progress 9.1d. These systems need to use a shared library I programmed in C language.
I was told by the Progress folks that the aplication works via something called an "appserver". This appserver has something they call "agents" and when a user executes the Progress application, the appserver instantiates (I suppose it´s called that way) an agent to attend the petition. There are a limited number of agents and when the limit is surpassed, the petitions are queued.
So, each of these agents executes the Progress code that uses my shared library. My fear is that there could be data collisions between them. The shared library does not has global nor static variables. All the data the shared library function uses is created inside it, all variables are local.
The shared library and the Progress appserver are in a same UNIX server HP-UX 11.1
I'm guessing that each new agent has it's own copy of the data of the Progress aplication, but if it does, I don't know if the same happens with the shared library stuff...
If someone has expereince using shared libraries with Progress, are there some measures to take for concurrency?
So far our tests have been without problems.
Any comment would be appreciated, thanks.
Each app server instance is an individual UNIX process. So your worries about shared data shouldn't come up.
Shared libraries can work and can be called by Progress, even on such an ancient and obsolete release as 9.1D -- but Progress is aggressively single-threaded so if your shared lib uses threads in any way it may fail.
Who is responsible for calling the shared library from the 4GL code? You? Or the Progress developers? In either case this might be helpful:
http://dbappraise.com/ppt/shlib.pptx
In my c++ code (my_app) I need to launch external app (app_ext) that dynamically loads my library (dll,so) written in fortran (lib_fort). From this library (lib_fort) I need to call back to some method from my_app, synchronously.
So its like that:
(my_app) --launches--> (app_ext) --loads--> (lib_fort) --"calls"--> (my_app)
app_ext is not developed by me.
Do you have any suggestions how to do it, and what's most important, do it efficiently??
Edit:
Clarification. Launching external app (app_ext) and loading my library from it (lib_fort) will happen only once per whole program execution. So that part doesn't need to be ultra-efficient. Communication between lib_fort and my_app is performance critical. Lib_fort needs to "call" my_app millions of times.
The whole point is about efficient inter-process communication.
My_app role after launching app_ext is to wait and serve "calls" from lib_fort. The tricky part is that solution needs to work both for distributed and shared memory environment, i.e. both my_app and app_ext+lib_fort on single host (1) and my_app and app_ext+lib_fort on different machines (2).
In (1) scenario I was thinking about MPI, but I'm not sure if it is possible to communicate with MPI between two different applications (in contrast to single, multi-process, MPI application).
In (2) scenario probably some kind of inter-process communication using shared memory? (or maybe also MPI?)
OK, the real issue is how to communicate between processes. (Forget MPI, that's for a different kind of problem.) You may be talking about COM (Component Object Model) or RPC (Remote Procedure Call) or pipes, but underneath it's going to be using sockets. IME the simplest and most efficient thing is to open the socket connections yourself and converse over those. That will be the rate-limiter and there really isn't anything faster.
I have an old C++ library which has been designed for use in single-threaded environmens.
The library exposes the interfaces for initialization, which change the internal data structures of the library, and usage, which only reads data and makes calculations.
My objective is to use this library in a Windows multithreaded application, with different threads calling instances of the dll initialized with different data.
Assuming that rewriting the dll to allow multithreading would be prohibitive, is there some way to let multiple instances of a DLL exist in the same process, with separate memory spaces, or to obtain a similar result by other means?
If the DLL contains static resources, then those would be shared among all instances created.
One possible way would be to create a single instance and restrict access to it using some kind of lock mechanism. This may reduce performance depending on usage, but without modifying internal structure of DLL, it may be difficult to work with multiple instance.
The sharing of static resources between all threads attached to a single DLL within a process conspires against you here.
However, there is a trick to achieve this. So long as DLLs have different names, then the system regards them as being different and so separate instances of code and data are created.
The way to achieve this is, for each thread, copy the DLL to a temporary file and load from there with LoadLibrary. You have to use explicit linking (GetProcAddress) rather than lib files but that's really the only way.