Remote proxy with shared memory in C++ - c++

Suppose I have a daemon that is sharing it's internal state to various applications via shared memory. Processes can send IPC messages to the daemon on a named pipe to perform various operations. In this scenario, I would like to create a C++ wrapper class for clients that acts as a kind of "Remote Proxy" to hide some of the gory details (synchronization, message passing, etc) from clients and make it easier to isolate code for unit tests.
I have three questions:
Generally, is this a good idea/approach?
Do you have any tips or gotchas for synchronization in this setup, or is it enough to use a standard reader-writer mutex setup?
Are there any frameworks that I should consider?
The target in question is an embedded linux system with a 2.18 kernel, therefore there are limitations on memory and compiler features.

Herb Sutter had an article Sharing Is the Root of All Contention that I broadly agree with; if you are using a shared memory architecture, you are exposing yourself to quite a bit of potential threading problems.
A client/server model can make things drastically simpler, where clients write to the named server pipe, and the server writes back on a unique client pipe (or use sockets). It would also make unit testing simpler (since you don't have to worry about testing shared memory), could avoid mutexing, etc.

There's Boost.Interprocess library, though I can't comment on its suitability for embedded systems.

Related

Process Isolation in Rust

I want to implement a server for a protocol. For security reasons the parser should be isolated in its own thread from the rest of the program and only a bidirectional channel should be held open for communication.
The parser thread should lose any possibility to modify the other thread's memory and lose its power to do syscalls (using seccomp).
Is there an easy way to achieve this behavior for the parser thread in Rust?
If you're concerned about issues beyond what Rust's strong safety and type system can protect against (e.g. bugs in those, or in third-party libraries etc.) then you really want separate processes rather than just threads; even if you use seccomp on an untrusted thread, at the OS/CPU level it still has full write access to other threads' memory in the same process.
Either way you'll need to write code designed to run in seccomp carefully (for example allocating extra heap memory might not work) - but the good news is that Rust is a great language for having that control!
There's a reasonably useful discussion on seccomp in Rust which has some suggestions.
The best bet looks like gaol from the Servo project, which is a more general process sandbox (including seccomp). There are also some other lower level seccomp wrappers like this one.
I haven't tried any of this yet, so I'd be interested to hear any other viewpoints/experience.

Can libraries replace local socketing in C/C++?

I'm trying to develop a specific DB server in C++, and I have two questions:
Is it possible to have a dynamic library take care of the communication between client programs instead of using sockets? This way, serializing is avoided and all querying can be made using native C/C++ library calls, while the server listens to the library for incoming requests
Does any known database work like that, and if yes what are the pros and cons of such an approach?
As far as I can see, having native calls to the DB server through the library removes overhead from serializing and socket system calls (even though it adds calls to a dynamic library). Also, I'm not sure how memory can be shared with libraries, but if it can then it could be very beneficial to "almost" share memory with the server as a client.
(I am focusing on Linux and POSIX, but the principles would be the same on other OSes like Windows, Android, MacOSX)
The communication between a database client and the database server is very likely to happen on socket(7)s or some similar byte stream, like pipe(7)s or fifo(7)s. Using shared memory (shm_overview(7)...) for that communication is unusual, and you still need some synchronization mechanism (e.g. semaphores sem_overview(7)...).
There are some libraries (above sockets) to facilitate such communications, e.g.
0mq.
Some database libraries exist that work without communicating to some database sever, in particular sqlite, which manage the database storage directly (in your client process). You might have some issues if several processes are accessing the same database concurrently (so ACID properties might not be guaranteed, at least if using sqlite without care).
Notice that local inter-process communications are quite efficient on Linux. It is not unusual to have a bandwidth of several hundreds of megabytes per second on a local pipe (use rather large buffers, e.g. of 64 Kbytes or a megabyte, for read(2) & write(2)...)
In practice, in a database, indexing and disk access are more likely to be the bottleneck than client <-> server communication, at least on the same local host. If the server is a remote host, network communication is probably the bottleneck (at least on common gigabit/sec ethernet).
Read also this, in particular the table in Answers section.
Perhaps gdbm, redis, mongodb, postgresql might be relevant for your issues.
Yes, if you DB clients are on the same machine that your DB server is on, they could communicate directly using techniques like shared memory IPC. However, this is typically not useful, because:
A database with all its clients on a single machine is rare.
A database with even one client on the same machine other than an administrative interface is not typical.
Systems like Linux already have optimizations built in for localhost socket communication, so it doesn't go via the network at all--only through the kernel.
A database whose performance is limited by socket IPC due to syscalls could easily overcome this by simply using a third-party kernel bypass solution for network communication, which does not require any special code at all--just plug in a kernel-bypass TCP stack--you can do this with many existing databases.

Exchange and store values between different processes

My application needs to store and exchange some (one) values between different processes. Also behind a start of the application this value is needed (but not behind a system reboot).
I can write and read this value in a file and synchronize the access. The file could lay in a ramfs. The solution would work but I have the feeling I use the wrong method.
Is there a better lightweight solution for this? Do I miss an straightforward approach?
I was thinking about named pipes (mkfifo) but there needs always and active writer and reader?
You are asking about inter-process communication. There are a number of methods for communicating between processes:
Low-level shared memory
Low-level named pipes
Low-level sockets
Remote procedure call mechanisms (DCOM, Corba, ONC RPC, etc.)
REST api
Distributed system frameworks
Which of these will work best for you depends on the complexity of the messages exchanged between processes, the complexity of the overall system, the need for portability, etc.
Shared memory is a very low-level approach and can feel the easiest solution "because it's just bytes in memory addressed by a pointer". However, it's inherently low-level nature makes it also tedious to use. There is no universally agreed upon C++ interface to these facilities, so you are left with low-level C style APIs for accessing and configuring shared memory between processes. There are differences between platforms (POSIX does it one way; Windows does it another).
Boost.Interprocess gives you a portable way to access shared memory mechanisms and aims to make using them simpler.

Functionality implementation: Processes or Threads division?

I'm working on an application (C++ combined with Qt for graphic part) to be run on an embedded Linux platform. I need know how to divide the application in different "cores" each one taking care of a different part of the application in such a way to improve stability, efficiency and security of the application itself.
My doubt is: is it more convenient to divide functionalities into threads or to fork different processes?
Let me provide a functional view of the application: there are different user interfaces each one allowing users to do more or less the same things (don't mind about data consistency, I've already solved this problem). Each of these interfaces must act as a stand-alone (like different terminal of the same system). I want all of them to send and receive messages from the same "core" which will take care of updating application data or do other proper stuff.
What's the best way to implement the division between the inner "core" and a user interface?
For sure I'm missing some knowledge but so far I came up with two alternatives:
1 - fork a child from father "core" and let the child execute a specific UI program (I have no practical experience of doing this so how, in this case, can I make father and child communicate (baring in mind that child is a new process)?)
2 - create different threads for each core and UI.
I need this division because the application is required to be as stable as possible and capable of restarting a UI in the case of a crash. Keep in mind also that the overall application wont have infinite memory and resources available.
Thanks in advance for your help, regards.
There are a several reasons why going down the separate process route might is a good choice in an embedded system:
Decoupling of component: running components as seperate processes is the ultimate decoupling. Often useful when projects become very large
Security and privilege management: Quite likely in an embedded system that some components need elevated privilege in order to control devices, whereas others are potential security hazards (for instance network facing components) you want to run with as little as little privilege as possible. Other likely scenarios are components that need real-time threading or to be able to mmap() a lot of system memory. Overallocation of either will lock your system up in a way it won't recover from.
Reliably: You can potentially respawn parts of the system if they fail leaving the remainder running
Building such an arrangement is actually easier than others here are suggesting - Qt has really good support for dbus - which nicely takes care of your IPC, and is used extensive in the Linux desktop for system management functionality.
As for the scenario you describe, you might want to daemonise the 'core' of the application and then allow client connections over dbus from UI components.
Running the UI in a different thread won't give you much in the way of additional stability -- the other thread can trash your heap of the engine, and even if you terminate the thread any resources it has won't be recycled.
You can improve stability a bit by having a really strong wall of abstraction between the Engine and the UI. So this isn't completely futile.
Multiple processes require lots of hoops to jump through -- you need a method of IPC (interprocess communication).
Note that IPC and to a lesser extent walls of abstraction can add to the overhead of your program.
An important question to ask is "how much data has to pass between the UI and the Engine?" -- if it is little enough data (like "start the task" from UI to engine, and "this task is 50% done" from engine to UI), IPC is less of a hassle. If you are an interactive painting application with real-time full-screen updates of an image, IPC is more annoying and less practical.
Now, a quick google on Qt and IPC tells me that there is a Qt extension for embedded linux that allows the Qt signals and slots to pass messages between processes: Qt COmmunications Protocol (QCOP). One issue I have had with frameworks like this is that it can easily lead to entanglements between the client and server state that can compromise stability on the other end of the communications pipe, compared to relatively simple protocols.

Most suitable asynchronous socket model for an instant messenger client?

I'm working on an instant messenger client in C++ (Win32) and I'm experimenting with different asynchronous socket models. So far I've been using WSAAsyncSelect for receiving notifications via my main window. However, I've been experiencing some unexpected results with Winsock spawning additionally 5-6 threads (in addition to the initial thread created when calling WSAAsyncSelect) for one single socket.
I have plans to revamp the client to support additional protocols via DLL:s, and I'm afraid that my current solution won't be suitable based on my experiences with WSAAsyncSelect in addition to me being negative towards mixing network with UI code (in the message loop).
I'm looking for advice on what a suitable asynchronous socket model could be for a multi-protocol IM client which needs to be able to handle roughly 10-20+ connections (depending on amount of protocols and protocol design etc.), while not using an excessive amount of threads -- I am very interested in performance and keeping the resource usage down.
I've been looking on IO Completion Ports, but from what I've gathered, it seems overkill. I'd very much appreciate some input on what a suitable socket solution could be!
Thanks in advance! :-)
There are four basic ways to handle multiple concurrent sockets.
Multiplexing, that is using select() to poll the sockets.
AsyncSelect which is basically what you're doing with WSAAsyncSelect.
Worker Threads, creating a single thread for each connection.
IO Completion Ports, or IOCP. dp mentions them above, but basically they are an OS specific way to handle asynchronous I/O, which has very good performance, but it is a little more confusing.
Which you choose often depends on where you plan to go. If you plan to port the application to other platforms, you may want to choose #1 or #3, since select is not terribly different from other models used on other OS's, and most other OS's also have the concept of threads (though they may operate differently). IOCP is typically windows specific (although Linux now has some async I/O functions as well).
If your app is Windows only, then you basically want to choose the best model for what you're doing. This would likely be either #3 or #4. #4 is the most efficient, as it calls back into your application (similar, but with better peformance and fewer issues to WSAsyncSelect).
The big thing you have to deal with when using threads (either IOCP or WorkerThreads) is marshaling the data back to a thread that can update the UI, since you can't call UI functions on worker threads. Ultimately, this will involve some messaging back and forth in most cases.
If you were developing this in Managed code, i'd tell you to look at Jeffrey Richter's AysncEnumerator, but you've chose C++ which has it's pros and cons. Lots of people have written various network libraries for C++, maybe you should spend some time researching some of them.
consider to use the ASIO library you can find in boost (www.boost.org).
Just use synchronous models. Modern operating systems handle multiple threads quite well. Async IO is really needed in rare situations, mostly on servers.
In some ways IO Completion Ports (IOCP) are overkill but to be honest I find the model for asynchronous sockets easier to use than the alternatives (select, non-blocking sockets, Overlapped IO, etc.).
The IOCP API could be clearer but once you get past it it's actually easier to use I think. Back when, the biggest obstacle was platform support (it needed an NT based OS -- i.e., Windows 9x did not support IOCP). With that restriction long gone, I'd consider it.
If you do decide to use IOCP (which, IMHO, is the best option if you're writing for Windows) then I've got some free code available which takes away a lot of the work that you need to do.
Latest version of the code and links to the original articles are available from here.
And my views on how my framework compares to Boost::ASIO can be found here: http://www.lenholgate.com/blog/2008/09/how-does-the-socket-server-framework-compare-to-boostasio.html.