multi-process C++ fifo - c++

In C you can create multi process application using fork() and you can then communicate using a FIFO pipe. I have learned that C++ only supports multi threaded applications and if you want a multi-process application you have to rely on fork().
But in C++ type checking is crucial so I can't just pipe objects through a pipe without any risks. You could cast to void* and ask sizeof and the send everything through a pipe to typecast it back to the original object.
So why does this feel so wrong? Is multi-process architecture not used in C++ or are there libraries or better ways of doing things. I have googled a little and the only thing you find is multithreaded C++ or multi-process C.
The reason for wanting more processes is that I want my application to be as robust as possible. If my web service crashes I want my main process to restart it. There is no way of doing this multithreaded since you never know if 1 thread hasn't corrupted memory in another thread so if you get an error in one thread you have to restart out of safety.

I assume by multiprocess programs you mean concurrently running two separate instances of a program with some shared data between them. Communicating between processes can be tricky. I have always gotten by with the pipe-typecasting method you described, or by using a local socket system to send data back and forth, however there do exist libraries for higher-level communication between processes, see boost.interprocess I would give that a look, and see if it can fit your needs.

As far as I understand your question right your main problem is passing data from one process to another via some kind of serial connection.
If this is the case and you are just passing flat data structures up and down the line there should be no problem using the method you already described. Just shot the whole bunch of bits down the line and cast to the corresponding type on the other end of the line. As far as you're only communicating with processes running on the same machine with executables generated by the same compiler and with same compiler switches there won't be any problem with byte-order, member alignment or whatever.
If on the other hand your data is more complex containing some kind of references to other objects lists of dynamic length or the like then you'll soon get into big trouble using the simple cast to void* and pump the data bit by bit strategy. So you'll definately need a more sophisticated approach at "serialization" and "deserialization" of your objects.
These ("serialization" and "deserialization") are the two terms you might want to investigate further on to find the approach that best fits your problem.
As you'll soon find out these are problems to which "solutions" are invented over and over again with such a zoo of standards like XDR used by sun RPC and ASN.1 to name just a few that it is hard to tell which one will best fit to your use case.
If you're going with C++ you might want to take a look at the solution the boost has to offer (see: http://www.boost.org/doc/libs/1_39_0/libs/serialization/doc/index.html)
Yet again - if it's just flat data structures you're passing back and forth don't bother with any overhead of that kind and just shoot the data down the line bit by bit.

Related

Boost: is there an interprocess::message_queue-like mechanism for thread-only communication?

The boost::interprocess::message_queue mechanism seems primarily designed for just that: interprocess communication.
The problem is that it serializes the objects in the message:
"A message queue just copies raw bytes between processes and does not send objects."
This makes it completely unsuitable for fast and repeated interthread communication with large composite objects being passed.
I want to create a message with a ref/shared_ptr/pointer to a known and previously-created object and safely pass it from one thread to the next.
You CAN use asio::io_service and post with bind completions, but that's rather klunky AND requires that the thread in question be using asio, which seems a bit odd.
I've already written my own, sadly based on asio::io_service, but would prefer to switch over to a boost-supported general mechansim.
You need a mechanism, that designed for interprocess communication because separate processes has separate address space and you cannot simply pass pointers except very spacial cases. For thread communication you can use standard containers like std::stack, std::queue and std::priority_queue to communicate between threads, you just need to provide proper synchronization through mutexes. Or you can use lock-free containers, which also provided by boost. What else would you need for interthread communication?
Whilst I'm no expert in Boost per se, there is a fundamental difficulty in communicating between processes and threads via a pipe, message queue, etc, especially if it is assumed that a program's data is classes containing dynamically allocated memory (which is pretty much the case for things written with Boost; a string is not a simple object like it is in C...).
Copying of Data in Classes
Message queues and pipes are indeed just a way of passing a collection of bytes from one thread/process to another thread/process. Generally when you use them you're looking for the destination thread to end up with a copy of the original data, not just a copy of the references to the data (which would be pointing back at the original data).
With a simple C struct containing no pointers at all it's easy; a copy of the struct contains all the data, no problem. But a C++ class with complex data types like strings is now a structure containing references / pointers to allocated memory. Copy that structure and you haven't actually copied the data in the allocated memory.
That's where serialisation comes in. For interprocess communications where both processes can't ordinarily share the same memory serialisation serves as a way of parcelling up the structure to be sent plus all the data it refers to into a stream of bytes that can be unpacked at the other end. For threads it's no different if you don't want the two threads accessing the same memory at the same time. Serialisation is a convenient way of saving yourself having to navigating through a class to see exactly what needs to be copied.
Efficiency
I don't know what Boost uses for serialisation, but clearly serialising to XML would be painfully inefficient. A binary serialisation like ASN.1 BER would be much faster.
Also, copying data through pipes, message queues is no longer as inefficient as it used to be. Traditionally programmers don't do it because of the perceived waste of time spent copying the data repeatedly just to share it with another thread. With a single core machine that involves a lot of slow and wasteful memory accesses.
However, if one considers what "memory access" is in these days of QPI, Hypertransport, and so forth, it's not so very different to just copying the data in the first place. In both cases it involves data being sent over a serial bus from one core's memory controller to another core's cache.
Today's CPUs are really NUMA machines with memory access protocols layered on top of serial networks to fake an SMP environment. Programming in the style of copying messages through pipes, message queues, etc. is definitely edging towards saying that one is content with the idea of NUMA, and that really you don't need SMP at all.
Also, if you do all your inter-thread communications as message queues, they're not so very different to pipes, and pipes aren't so different to network sockets (at least that's the case on Not-Windows). So if you write your code carefully you can end up with a program that can be redeployed across a distributed network of computers or across a number of threads within a single process. That's a nice way of getting scalability because you're not changing the shape or feel of your program in any significant way when you scale up.
Fringe Benefits
Depending on the serialisation technology used there can be some fringe benefits. With ASN.1 you specify a message schema in which you set out the valid ranges of the message's contents. You can say, for example, that a message contains an integer, and it can have values between 0 and 10. The encoders and decoders generated by decent ASN.1 tools will automatically check that the data you're sending or receiving meets that constraint, and returns errors if not.
I would be surprised if other serialisers like Google Protocol Buffers didn't do a similar constraints check for you.
The benefit is that if you have a bug in your program and you try and send an out of spec message, the serialiser will automatically spot that for you. That can save a ton of time in debugging. Also it is something you definitely don't get if you share a memory buffer and protect it with a semaphore instead of using a message queue.
CSP
Communicating Sequential Processes and the Actor model are based on sending copies of data through message queues, pipes, etc. just like you're doing. CSP in particular is worth paying attention to because it's a good way of avoiding a lot of the pitfalls of multi-threaded software that can lurk undetected in source code.
There are some CSP implementations you can just use. There's JCSP, a class library for Java, and C++CSP, built on top of Boost to do CSP for C++. They're both from the University of Kent.
C++CSP looks quite interesting. It has a template class called csp::mobile, which is kind of like a Boost smart pointer. If you send one of these from one thread to another via a channel (CSP's word for a message queue) you're sending the reference, not the data. However, the template records which thread 'owns' the data. So a thread receiving a mobile now owns the data (which hasn't actually moved), and the thread that sent it can no longer access it. So you get the benefits of CSP without the overhead of copying the data.
It also looks like C++CSP is able to do channels over TCP; that's a very attractive feature, up scaling is a really simple possibility. JCSP works over network connections too.

Execute External function belonging to other process

I would need help with a problem. I have 2 processes running, one the Watchdog and the other a simple test process. I need process 2 to call code from the Watchdog, the reason I do this is to reduce the size of process 2. For example process 2 must call a function called "IsSafe" from the watchdog. The IsSafe function relies on other code belonging to Watchdog process and it will not be viable to rewrite this code for process 2. I have thought of ideas, please could you advise on which is the best solution and or give advice.
Idea One
Use Named pipes to communicate between processes and pass parameters and return values around.
Idea Two
Use Share Memory to share parameters and return values
Idea Three
Use windows messages, I honestly think this will not work
Idea Four
Somehow create a executable portion of shared memory and execute this code with a far jmp.
Please could you advise.
RPC was invented long ago. Then COM on top of that. In my opinion best forget your idea, but if you must, use COM.
By the way, to communicate between processes on the same Windows machine without COM, use mailslots.
Seems you forgot about them in you list.
Cheers & hth.,
Although putting the code in the process which needs to call it is good advice in general, in the particular case of a watchdog (also debugger and any other form of error handler) using separate processes is correct. You don't want the watchdog to fail due to an error in the main code, so it needs to be a separate process.
A named pipe would be ideal in this scenario, the TransactNamedPipe function is designed just for this.
A DLL is the standard implementation of idea 4. It's loaded in both address spaces, but shared in physical RAM. You don't need special tricks; it works everywhere and Windows will deal with any security issues for you.
It's also portable to most other Operating Systems, although they're generally called something else, e.g. .so on Linux.
All you really need is some IPC. For a lightweight and easy solution, simply define an application specific message with WM_APP and have a mapping from the wParam/lParam for parameters. If you find you need more than 8 bytes, you could use WM_COPYDATA instead.

Is it possible to cause multi processes hibernate (core dump?)?

I have a software (c++) that runs few processes (each process is a major system itself).
The processes have communication with each other via xml-rpc or boost asio
I want to be able to freeze or stop all processes at a given moment and be able to raise the system (all processes) later to the same state as before hibernating.
How can I do that in c++?
Would it be feasible due to the fact that the processes communicates with each other?
The big picture is that you need to get the system to a stable consistent state, then persist that state in some re-creatable form.
You can in principle write such code, the degree of difficulty depends on your application. You will need to figure out things such as:
How the processes agree that they are in a consistent state. You may need to define some new "Get ready to hibernate" and "I'm ready" messages.
For each process you need to figure out how to persist and recover it's state. Depending upon the complexity of any live data structures that may be quite tricky. On the other hand, if your processes are stateless then this could be really easy.
You'll need to devise a scheme for managing the sets of hibernated data, how you determine a consistent set across all the processes.
I see this as significant coding effort, the degree of difficulty will depend on the complexity of your application and the quality of its implementation. In a well structured application such major "replumbing" exercises often go surprisingly simply.
Unless you're an OS - no, it won't be possible.
What you need to do instead is to make sure that each process can do it for itself (i.e.: write a functionality that allows saving and restoring the states for each of the processes), and also to accommodate for inconsistencies in the communication (for example - to ensure ACK on the messages, and resend if saved state without receiving ACK).
It's feasible if done right, but it's easier said than done, of course, and assumes you can actually change the processes.
Well,
the other answers are fine. There is another rather "exotic" way which may solve this quickly, but it may be overkill or not suitable. But who knows ? So just in case...
I suggest to run your program into a virtual machine (I mean for example a linux with vmware) and pause/wake up this virtual machine at will.
If you are using an inter-process communication method which is not disrupted by this kind of operation, it may work and save you a lot of time.
Good luck.

How to make a function call bulletproof?

I need to call a function (an LLVM JIT to be specific) from a C++ application. This call might fail or even signal abort() or exit(). How can I avoid or at least reduce effects on my host application? Someone suggested using fork(), however I need a solution for both windows and posix. Even if I would use fork() ... would it be possible for the two processes to communicate (pass some pointers around)?
You basically have to isolate the call that might fail spectacularly, so yes, you probably have to create a separate process for it. I'd actually be tempted to create a small executable just containing this particular call and the necessary supporting functionality and call that from your main executable. This gets you around the lack of fork() on Windows and allows you to use the same mechanisms to communicate.
You can't pass pointers around between processes as they're not sharing the same address space. What I would do is have the spawned process reading data from stdin and write to stdout with the controlling process piping data into the child's stdin and reading from the child's stdout. Basically the way a Unix (command line) filter works. Another alternative if you're passing around a lot of data would be to write/read to/from a file on disk (better, a RAM disk) and communicate that way, but unless you're talking a lot of data, that's overkill.
As Eugen pointed out in the comments, you can also use shared memory if you want to pass pointers around or another inter-process communication mechanism depending on how much data you need to pass around. That said, choose the simplest possible method as nested executables like these aren't that easy to debug in the first place.

Fast Cross Platform Inter Process Communication in C++

I'm looking for a way to get two programs to efficiently transmit a large amount of data to each other, which needs to work on Linux and Windows, in C++. The context here is a P2P network program that acts as a node on the network and runs continuously, and other applications (which could be games hence the need for a fast solution) will use this to communicate with other nodes in the network. If there's a better solution for this I would be interested.
boost::asio is a cross platform library handling asynchronous io over sockets. You can combine this with using for instance Google Protocol Buffers for your actual messages.
Boost also provides you with boost::interprocess for interprocess communication on the same machine, but asio lets you do your communication asynchronously and you can easily have the same handlers for both local and remote connections.
I have been using ICE by ZeroC (www.zeroc.com), and it has been fantastic. Super easy to use, and it's not only cross platform, but has support for many languages as well (python, java, etc) and even an embedded version of the library.
Well, if we can assume the two processes are running on the same machine, then the fastest way for them to transfer large quantities of data back and forth is by keeping the data inside a shared memory region; with that setup, the data is never copied at all, since both processes can access it directly. (If you wanted to go even further, you could combine the two programs into one program, with each former 'process' now running as a thread inside the same process space instead. In that case they would be automatically sharing 100% of their memory with each other)
Of course, just having a shared memory area isn't sufficient in most cases: you would also need some sort of synchronization mechanism so that the processes can read and update the shared data safely, without tripping over each other. The way I would do that would be to create two double-ended queues in the shared memory region (one for each process to send with). Either use a lockless FIFO-queue class, or give each double-ended queue a semaphore/mutex that you can use to serialize pushing data items into the queue and popping data items out of the queue. (Note that the data items you'd be putting into the queues would only be pointers to the actual data buffers, not the data itself... otherwise you'd be back to copying large amounts of data around, which you want to avoid. It's a good idea to use shared_ptrs instead of plain C pointers, so that "old" data will be automatically freed when the receiving process is done using it). Once you have that, the only other thing you'd need is a way for process A to notify process B when it has just put an item into the queue for B to receive (and vice versa)... I typically do that by writing a byte into a pipe that the other process is select()-ing on, to cause the other process to wake up and check its queue, but there are other ways to do it as well.
This is a hard problem.
The bottleneck is the internet, and that your clients might be on NAT.
If you are not talking internet, or if you explicitly don't have clients behind carrier grade evil NATs, you need to say.
Because it boils down to: use TCP. Suck it up.
I would strongly suggest Protocol Buffers on top of TCP or UDP sockets.
So, while the other answers cover part of the problem (socket libraries), they're not telling you about the NAT issue. Rather than have your users tinker with their routers, it's better to use some techniques that should get you through a vaguely sane router with no extra configuration. You need to use all of these to get the best compatibility.
First, ICE library here is a NAT traversal technique that works with STUN and/or TURN servers out in the network. You may have to provide some infrastructure for this to work, although there are some public STUN servers.
Second, use both UPnP and NAT-PMP. One library here, for example.
Third, use IPv6. Teredo, which is one way of running IPv6 over IPv4, often works when none of the above do, and who knows, your users may have working IPv6 by some other means. Very little code to implement this, and increasingly important. I find about half of Bittorrent data arrives over IPv6, for example.