I have a very simple interface which needs to communicate between processes. It's currently implemented in a very simple manner (all single proc):
bool GetFoo(struct Foo *outFoo);
bool GetBar(struct Bar *getBar);
Such as:
Foo foo;
if (!GetFoo(&foo))
{
ReportError();
}
GetFoo fills out the "Foo" data structure with pure data (that is, no pointers - it's purely blitable data).
I need to convert this in-process function call to be between two processes on the same machine (in this case it's always the same machine). Is there a commonly followed idiom for cross-process calls in C++ on Windows? Is there some kind of intra-process communication supported by Windows? Should I be using shared memory instead?
One note: I don't want to take a dependency on anything other than the Windows APIs if at all possible.
You have many choices, but in my personal experience the most popular/easy to use ones are: sockets & pipes.
See here for all IPC options available for Windows.
I'm not sure what the most common is -- to truly answer that we'd have to have some kind of polling. That said, the most flexible way would probably be to expose the methods via DCOM.
A common method would be RPC, it can be implemented in various ways for instance as Billy mentioned using COM` (or DCOM if the processes are residing on different machines).
Although you may want to think about not doing direct RPC calls and instead have a named pipe between your processes which is used for the communication.
There are a number of ways to communicate between processes on the same computer in Windows. Which one works best depends on the relationship between the two processes. Is one process expected to start the other? In that case an out-of-process COM server would probably work best, since you can restart the other process if it is not already running.
If performance is critical, then shared memory will give you the most control the speed of passing the data between your processes.
One thing to think about is the failure semantics of running multiple processes. What does the calling process do if the callee is not there?
Related
My application needs to store and exchange some (one) values between different processes. Also behind a start of the application this value is needed (but not behind a system reboot).
I can write and read this value in a file and synchronize the access. The file could lay in a ramfs. The solution would work but I have the feeling I use the wrong method.
Is there a better lightweight solution for this? Do I miss an straightforward approach?
I was thinking about named pipes (mkfifo) but there needs always and active writer and reader?
You are asking about inter-process communication. There are a number of methods for communicating between processes:
Low-level shared memory
Low-level named pipes
Low-level sockets
Remote procedure call mechanisms (DCOM, Corba, ONC RPC, etc.)
REST api
Distributed system frameworks
Which of these will work best for you depends on the complexity of the messages exchanged between processes, the complexity of the overall system, the need for portability, etc.
Shared memory is a very low-level approach and can feel the easiest solution "because it's just bytes in memory addressed by a pointer". However, it's inherently low-level nature makes it also tedious to use. There is no universally agreed upon C++ interface to these facilities, so you are left with low-level C style APIs for accessing and configuring shared memory between processes. There are differences between platforms (POSIX does it one way; Windows does it another).
Boost.Interprocess gives you a portable way to access shared memory mechanisms and aims to make using them simpler.
In C you can create multi process application using fork() and you can then communicate using a FIFO pipe. I have learned that C++ only supports multi threaded applications and if you want a multi-process application you have to rely on fork().
But in C++ type checking is crucial so I can't just pipe objects through a pipe without any risks. You could cast to void* and ask sizeof and the send everything through a pipe to typecast it back to the original object.
So why does this feel so wrong? Is multi-process architecture not used in C++ or are there libraries or better ways of doing things. I have googled a little and the only thing you find is multithreaded C++ or multi-process C.
The reason for wanting more processes is that I want my application to be as robust as possible. If my web service crashes I want my main process to restart it. There is no way of doing this multithreaded since you never know if 1 thread hasn't corrupted memory in another thread so if you get an error in one thread you have to restart out of safety.
I assume by multiprocess programs you mean concurrently running two separate instances of a program with some shared data between them. Communicating between processes can be tricky. I have always gotten by with the pipe-typecasting method you described, or by using a local socket system to send data back and forth, however there do exist libraries for higher-level communication between processes, see boost.interprocess I would give that a look, and see if it can fit your needs.
As far as I understand your question right your main problem is passing data from one process to another via some kind of serial connection.
If this is the case and you are just passing flat data structures up and down the line there should be no problem using the method you already described. Just shot the whole bunch of bits down the line and cast to the corresponding type on the other end of the line. As far as you're only communicating with processes running on the same machine with executables generated by the same compiler and with same compiler switches there won't be any problem with byte-order, member alignment or whatever.
If on the other hand your data is more complex containing some kind of references to other objects lists of dynamic length or the like then you'll soon get into big trouble using the simple cast to void* and pump the data bit by bit strategy. So you'll definately need a more sophisticated approach at "serialization" and "deserialization" of your objects.
These ("serialization" and "deserialization") are the two terms you might want to investigate further on to find the approach that best fits your problem.
As you'll soon find out these are problems to which "solutions" are invented over and over again with such a zoo of standards like XDR used by sun RPC and ASN.1 to name just a few that it is hard to tell which one will best fit to your use case.
If you're going with C++ you might want to take a look at the solution the boost has to offer (see: http://www.boost.org/doc/libs/1_39_0/libs/serialization/doc/index.html)
Yet again - if it's just flat data structures you're passing back and forth don't bother with any overhead of that kind and just shoot the data down the line bit by bit.
I was wondering why in the new C++11 they added threads and not processes.
Couldn't have they done a wrapper around platform specific functions?
Any suggestion about the most portable way to do multiprocessing? fork()? OpenMP?
If you could use Qt, QProcess class could be an elegant platform independent solution.
If you want to do this portably I'd suggest you avoid calling fork() directly and instead write your own library function that can be mapped on to a combination of fork() and exec() on systems where that's available. If you're careful you can make your function have the same or similar semantics as CreateProcess() on Win32.
UNIX systems tend to have a quite different approach to processes and process management compared to Windows based systems so it's non-trivial to make all but the simplest wrappers portable.
Of course if you have C++11 or Boost available I'd just stick with that. If you don't have any globals (which is a good thing generally anyway) and don't set up and shared data any other way then the practical differences between threads and processes on modern systems is slim. All the threads you create can make progress independently of each other in the same way the processes can.
Failing that you could look at An MPI implementation if message passing suits your task, or a batch scheduler system.
I am using Boost Interprocess.
It does not provide the possibility to create new processes, but once they are there, it allows them to communicate.
In this particular case I can create the processes I need from a shell script.
I need to call a function (an LLVM JIT to be specific) from a C++ application. This call might fail or even signal abort() or exit(). How can I avoid or at least reduce effects on my host application? Someone suggested using fork(), however I need a solution for both windows and posix. Even if I would use fork() ... would it be possible for the two processes to communicate (pass some pointers around)?
You basically have to isolate the call that might fail spectacularly, so yes, you probably have to create a separate process for it. I'd actually be tempted to create a small executable just containing this particular call and the necessary supporting functionality and call that from your main executable. This gets you around the lack of fork() on Windows and allows you to use the same mechanisms to communicate.
You can't pass pointers around between processes as they're not sharing the same address space. What I would do is have the spawned process reading data from stdin and write to stdout with the controlling process piping data into the child's stdin and reading from the child's stdout. Basically the way a Unix (command line) filter works. Another alternative if you're passing around a lot of data would be to write/read to/from a file on disk (better, a RAM disk) and communicate that way, but unless you're talking a lot of data, that's overkill.
As Eugen pointed out in the comments, you can also use shared memory if you want to pass pointers around or another inter-process communication mechanism depending on how much data you need to pass around. That said, choose the simplest possible method as nested executables like these aren't that easy to debug in the first place.
I have an application (A) that needs to launch another application (B). I need to pass data between the applications. I can think of two approaches. The first is to open a socket. The second is to share data via a dll.
The opening socket approach is straight forward.
The dll approach I have some questions? I can load plug-in dlls into B. I want to create a dll that A can use to pass data to B. When loading dlls, is only one instance of the dll loaded? If so, does this mean that data can be shared between applications that load the dll?
What is the better choice?
Are there other ways of doing this?
You can't effectively share data via a DLL. Other ways:
disk files
pipes
shared memory
messages
RPC
CORBA
COM
etc.
The simplest method (assuming Windows since you mention a DLL) is probably to use CreateProcess and open a pipe to the child process, as described in simplified form here: http://msdn.microsoft.com/en-us/library/ms682499.aspx
Named Pipes can be an alternative, especially if you aren't in control of the lifetime of all of the processes. http://msdn.microsoft.com/en-us/library/aa365590.aspx
For simple cases, mailslots may be a sufficient alternative.
http://msdn.microsoft.com/en-us/library/aa365574.aspx#base.using_a_mailslot_for_ipc
Here's a longer list of various Interprocess Communication techniques for Windows.
http://msdn.microsoft.com/en-us/library/aa365574.aspx
For something happening locally, using sockets seems sort of overkill. Plus you have to implement your own security mechanism to prevent spoofing attacks, rather than depending on the integrated security mechanism of most of the other IPC methods.
Its always good to explore alternative possible solutions, but I personally believe that using sockets as a transport layer for data between applications is not only future proof, but scalable as well. Using sockets will eliminate the need for you to write copious amounts of OS specific code, which could proclude you from porting your application in the future to non-Windows operating systems.
I would suggest sockets.
You can have a shared cache (example a windows service or hidden process) that can be listening - returning data to all subscribers. This using a Observer pattern approach.
I would agree somewhat with Juan Zamora M except that the service providing the data should have an API that can be requested when needed not pushed when changed via listeners.
This might help. Sharing Files and Memory