I want to keep some global objects in an Apache C++ module persistent across Apache child process invocations. How do I do this?
You must use some form of storage external to the Apache processes.
Basic choices:
A database.
Shared memory (OS dependent).
Another process and use an IPC mechanism (eg. a socket)
A file.
Which one is appropriate depends on your requirements, and you might combine them. For example, "a database" is actually implemented as another process that makes things persistent in a file and it deals with concurrency issues in a known way.
In general, a database is probably the first thing to try and only go to other alternatives if you have specific issues that can be solved by taking a different approach.
Related
I am hoping this isn't too vague a question...
I am writing a python app that will have multiple instances of itself to allow load balancing and redundancy to take place.
Each instance will need to be able to read and write to the backend database, which raises the issue of two 'gateways' trying to update the same item.
Can anyone recommend an approach (not primarily looking for code solution) to this?
Thank you in advance ;)
You have mainly two ways to avoid loosing consistency on your data set:
either use some lock mechanisms between the python processes,
or use transactions when accessing the database.
Lock mechanisms must be based on inter-process communications. Depending on your underlying operating system, you may have access to shared memory (think about test-and-set primitive, to use shared memory for such a goal), semaphores and others tools. See Python IPC, for instance: https://docs.python.org/2/library/ipc.html
Transactions depend on your database. For instance, with a traditional SQL database, you can often configure the behaviours according to your needs: auto-commit, optimistic concurrency, etc...
I have two projects:
Embedded one, written in C++, which uses a lot of static/global variables.
Second one, running on PC and using the same source codes as embedded one uses.
It works very well.
But now second project should run more than one instances of embedded project. Furthermore each instance should have its own copy of static/global variables, and I should be able to interact with each instance in one program scope. I don't know how to do this with all that static/global variables.
Is there any simple way to solve my problem?
There are several ways you can solve this:
Spawn multiple processes (each with their own set of globals) and setup channels of communication between them and the main program.
Get rid of the global variables. The easiest way to do this would be to dump them all in a class (as non-static members) and use instances of that class to access each set of variables.
Either way, it's not a small problem to solve if you have a large number of globals.
Run two separate processes and use some form of IPC to communicate between the the processes. In Windows IPC mechanisms available include:
Clipboard
COM
Data Copy
DDE
File Mapping
Mailslots
Pipes
RPC
Windows Sockets
See here for details of each of these. Similar mechanisms are available in other operating systems.
A perhaps simpler alternative is to run each instance in a separate thread and place the globals in thread local storage.
In all cases however, you should avoid nit just "a lot" but any global variables. It is generally indicative of poor design. See this article for why globals are bad, and ways to avoid them.
As the other answers state the best solution is to get rid of the globals, but I understand that this is not always feasible.
I ran into the exact same problem with our code base.
The solution I used was to build each instance as a separate DLL.
Then load I loaded each DLL with LoadLibrary() at runtime.
In this way you can get everything to run in a single process and have multiple version of the same globals and singletons.
And then you don't need to use any IPC but can pass data between the instances with a simple function call. It also makes the debugging easier, because you can see everything in one debugger.
NOTE: I made it on Windows, but I assume the something similar is possible on Unix.
I have an old C++ library which has been designed for use in single-threaded environmens.
The library exposes the interfaces for initialization, which change the internal data structures of the library, and usage, which only reads data and makes calculations.
My objective is to use this library in a Windows multithreaded application, with different threads calling instances of the dll initialized with different data.
Assuming that rewriting the dll to allow multithreading would be prohibitive, is there some way to let multiple instances of a DLL exist in the same process, with separate memory spaces, or to obtain a similar result by other means?
If the DLL contains static resources, then those would be shared among all instances created.
One possible way would be to create a single instance and restrict access to it using some kind of lock mechanism. This may reduce performance depending on usage, but without modifying internal structure of DLL, it may be difficult to work with multiple instance.
The sharing of static resources between all threads attached to a single DLL within a process conspires against you here.
However, there is a trick to achieve this. So long as DLLs have different names, then the system regards them as being different and so separate instances of code and data are created.
The way to achieve this is, for each thread, copy the DLL to a temporary file and load from there with LoadLibrary. You have to use explicit linking (GetProcAddress) rather than lib files but that's really the only way.
I have an application (A) that needs to launch another application (B). I need to pass data between the applications. I can think of two approaches. The first is to open a socket. The second is to share data via a dll.
The opening socket approach is straight forward.
The dll approach I have some questions? I can load plug-in dlls into B. I want to create a dll that A can use to pass data to B. When loading dlls, is only one instance of the dll loaded? If so, does this mean that data can be shared between applications that load the dll?
What is the better choice?
Are there other ways of doing this?
You can't effectively share data via a DLL. Other ways:
disk files
pipes
shared memory
messages
RPC
CORBA
COM
etc.
The simplest method (assuming Windows since you mention a DLL) is probably to use CreateProcess and open a pipe to the child process, as described in simplified form here: http://msdn.microsoft.com/en-us/library/ms682499.aspx
Named Pipes can be an alternative, especially if you aren't in control of the lifetime of all of the processes. http://msdn.microsoft.com/en-us/library/aa365590.aspx
For simple cases, mailslots may be a sufficient alternative.
http://msdn.microsoft.com/en-us/library/aa365574.aspx#base.using_a_mailslot_for_ipc
Here's a longer list of various Interprocess Communication techniques for Windows.
http://msdn.microsoft.com/en-us/library/aa365574.aspx
For something happening locally, using sockets seems sort of overkill. Plus you have to implement your own security mechanism to prevent spoofing attacks, rather than depending on the integrated security mechanism of most of the other IPC methods.
Its always good to explore alternative possible solutions, but I personally believe that using sockets as a transport layer for data between applications is not only future proof, but scalable as well. Using sockets will eliminate the need for you to write copious amounts of OS specific code, which could proclude you from porting your application in the future to non-Windows operating systems.
I would suggest sockets.
You can have a shared cache (example a windows service or hidden process) that can be listening - returning data to all subscribers. This using a Observer pattern approach.
I would agree somewhat with Juan Zamora M except that the service providing the data should have an API that can be requested when needed not pushed when changed via listeners.
This might help. Sharing Files and Memory
I have a Django app, which spawns a thread to communicate with another server, using Pyro.
Unfortunately, it seems like under fastcgi, multiple versions of this thread are fired off, and a dictionary that should be globally constant within my program, isn't. (Sometimes it has the values I expect, sometimes not)
What's the best way to ensure that there's one and only one copy of a dictionary in a django / fastcgi app?
I strongly recommend against relying on global anything in django. The problem is that, just as you seem to be encountering, the type of deployment will determine how (or whether or not) this global state is shared. To be a style nazi, that's a completely different level of abstraction from the code, which is relying on some guarantee of consistent global state.
I'm not experienced with fastcgi, but my understanding is that it, like many other frameworks, has a pre-forked and a threaded mode. In pre-forked mode, you have separate processes, not threads, running your python code. This spells nightmare for shared global state.
Barring some fragile workaround, which ought to be possible and which someone may or may not suggest, the only persistence you can really rely on is in the database, and, to a lesser extent, whatever caching mechanism you choose. You could use the low-level api to cache and retrieve keys and values.