Simple IPC between C++ and Python (cross platform) - c++

I have a C++ process running in the background that will be generating 'events' infrequently that a Python process running on the same box will need to pick up.
The code on the C side needs to be as lightweight as possible.
The Python side is read-only.
The implementation must be cross-platform.
The data being sent is very simple.
What are my options?
Thanks

zeromq -- and nothing else. encode the messages as strings.
However, If you want to get serialiazation from a library use protobuf it will generate classes for Python and C++. You use the SerializeToString() and ParseFromString() functions on either end, and then pipe the strings via ZeroMq.
Problem solved, as I doubt any other solution is faster, and neither will any other solution be as easy to wire-up and simple to understand.
If want to use specific system primitives for rpc such as named pipes on Windows and Unix Domain Sockets on unix then you should look at Boost::ASIO. However, unless you have (a) a networking background, and (b) a very good understanding of C++, this will be very time consuming

Use zeromq, it's about as simple as you can get.

Google's protobuf is a great library for RPC between programs. It generates bindings for Python and C++.
If you need a distributed messaging system, you could also use something like RabbitMQ, zeromq, or ActiveMQ. See this question for a discussion on the message queue libraries.

Another option is to just call your C code from your Python code using the ctypes module rather than running the two programs separately.

How complex is your data? If it is simple I would serialize it as a string. If it was moderately complex I would use JSON. TCP is a good cross-platform IPC transport. Since you say that this IPC is rare the performance isn't very important, and TCP+JSON will be fine.

You can use Google GRPC for this

I will say you create a DLL that will manage the communication between the two. The python will load DLL and call method like getData() and the DLL will in turn communicate with process and get the data.
That should not be hard.
Also you can use XML file or SQLite database or any database to query data. The daemon will update DB and Python will keep querying. There might be a filed for indicating if the data in DB is already updated by daemon and then Python will query.
Of course it depends on performance and accuracy factors!

Related

Web Services using C++

I am building a server-client application that involves heavy signal processing (e.g. FFT). I have a working application written in C++/Qt, where everything (signal processing and other calculations) is done in client and server just sends raw data. Now I feel it would be easier to implement these features on the server. So, that maintenance becomes easier.
As I am doing signal processing, I think I should stick to C++ for performance. But I am open to new ideas.
Constraints:
I need type checking so javascript is out of discussion.
Scaling includes adding more server and each server will have at the max
10-12 users. So, Hardware cost is important. I cannot use x number of
i7 processors.
No option of using cloud services.
So, right now my question is as follows:
How can I create web services using C++ for Linux server? (Although cross platform is not important, I would appreciate if I can achieve it.)
EDIT [02:09:2015]
Right now, I think the choice is between poco and C++ Rest SDK. I feel I should go for C++ Rest SDK. Mainly because it has only those features that I need. And Also it is supported by microsoft and uses boost internally. So, I feel in future, this might be well integreated with standard.
You could use cross-platform Poco library to implement HTTP server, it is really straightforward with this framework, and they have a lot of examples. You can also use JSON serialization (like rapidjson library) to implement REST service on top of HTTP - this way your Web service will be accesable by most of the modern Web frameworks.
You might want to take a look at the C++ Rest SDK, an open source, cross platform API from Microsoft.
Like #nogard suggested, I also recommend POCO for now. It's the most serious and feature-full solution. Given you mentioned Qt, I suggest you to take a look at Tufão.
EDIT:
I forgot to mention one comparison of mine on the C++ HTTP server frameworks.
If you directly handle HTTP requests, you might loose the functionality what Web Servers does well what it was build to do. I had a similar issue, what I did was wrap up my Qt c++ code inside a PHP extension. In your case you can do the same. Wrap your logic inside what ever technology you are about to use, doesn't matter it's PHP, net , Java or anything else.

Simple HTTP Server lib

What is a good choice for a simple http Server lib? It doesn't need high performance. I rather look for something simple for some REST/JSON communication ("API").
It must be able though to work in a multithreaded environment and must be able to handle large POST request.
Any suggestions? I already tried cpp-netlib but this seems to be much too complicated for such an easy task...
Edit: I am looking for something really light-weight and simple. E.g. like Sinatra in the Ruby world. Poco is for me another example of a too heavy-weight library.
The first one that comes to mind is Poco Library ( http://pocoproject.org/ )
Cross platform, stable, well documented. While the library itself offers more than you probably need you can build and omit the portions you aren't planning on using to reduce bloat.
They have a fully featured Net library that includes several salient classes and utilities.
Here is a pdf of slides from that library, of particular interest is the HTTPServer class:
http://pocoproject.org/slides/200-Network.pdf
Not sure about large POST data, but I've previously used mongoose: https://github.com/cesanta/mongoose/.
If the LGPL license is unwanted there is a MIT fork from when the project was MIT that also add a C++ API https://github.com/bel2125/civetweb
I would encourage you to start with http server samples in boost.asio. They are so simple and easy to understand, that you should be able to easily extend them as needed.
However, if you want to jump onto something more polished than just sample code, I know of 3 http servers in C++ which you may like to try:
"x0 - HTTP Web Server Framework" to me personally this one seems most promising, because it's lightweight and simple
"highpower / xiva" is a simple http server framework for delivering notifications to browsers
"Pion, a project of Atomic Labs" is a part of elaborate framework for handling large amounts of data
Pretty late answer; but hope this helps.
For your interest of a server that can handle REST, here is the easiest HTTP Server library to use (in my opinion): https://github.com/yhirose/cpp-httplib.
For JSON parsing, you may search for another library to use it in conjunction.
Personally, I'd go for Arachnida but that might be because I wrote it.

Program interacting with web

How or what do I need to know programming wise in order to interact with the web using c++. For instance i want to wrote a program that automatically sends invites to players on yahoo chess. How would i go about doing this?
You'll need to understand the basics of TCP/IP and HTTP, possibly UDP, and the protocols involved with Yahoo's chess systems or posses a tool to work around them (A brief search leads me to believe there are few if any). You'll probably need a network API, I'd suggest looking at:
QtNetwork Module
Boost.Asio
Where Qt is easy to use, Asio is more powerful, and more 'C++' in nature. Qt has some nice webkit components, and I've used it to build a small web server, which was a lot of fun. You can accomplish quite a lot with it.
This page says they've added a captcha system to prevent certain people from interacting with their systems. I'm not familiar with Yahoo games and what the result of this has on what you'd want to do, however it suggests to me they'd rather you didn't write code to interact with their systems.
For this you need to use network APIs and use server side script like PHP/ASP to communicate with the web and C using message queue.

Communicate between AIR(Flex) and C++ Applications

I need to be able to communicate between two applications that reside on the same machine. One is using Flex and the other is in C++. I would like to be able to call functions and pass arguments to each other. What is the best way to communicate between them? I was thinking about using sockets.
As for now yes, you'll need to use sockets.
AIR 2.0 will provide access to native processes, but that will require a native (per OS) installer. More info: http://www.mikechambers.com/blog/2009/09/22/fotb-slides-advanced-desktop-development-with-adobe-air/
As for AIR 1.x, Mike Chambers posted a sample library called CommandProxy which does the trick. It provides code for AS3 and .NET which could easily be ported to C++ if you have a decent XML parser.
The basic idea is that each compnent can send each other XML "packets" across the wire where each one has an ID to ensure the request/response can be tied together. However, both the AS3 and .NET code don't account for slow or fast TCP/IP speeds where two XML chunks could be sent on one packet or one XML chunk split across two. Any ways, it does the trick.
Also, you could look for a C++ AMF (ActionScript Message Format) library so that you could serialize AS3 objects over the wire which would have a more "integrated" feel.

What is the best way for distributed processes to communicate asynchronously?

I'm developing an application in which distributed components talk to one another over a network, in an asynchronous, pub/sub kind of way.
For this, I like the idea of sending XML over sockets - it's asynchronous, I don't need a server of any kind, and it can work locally or over a network. I would have to roll my own pub/sub mechanism, which probably won't be too onerous - I could use the IP address as the id of the subscriber. The implementation language will be C++. I've used this method before with good results. But, I wonder if there is a better way? Any advice welcome.
Edit: I haven't made a decision yet, but I'm interested in cparcode's comment on protobuf:
"I never understood the popularity of using XML for network comms. Any way, I realize that the protocol is only part of your problem but on that subject, there's Google's protobuf too: http://code.google.com/p/protobuf/ – caparcode (Mar 11 at 1:01)"
Instead of using XML it might be best to use an existing message passing framework. Have a look at libt2n or d-bus
But it you have used your own socket implementation in the past with good results i don't see a reason why you should change. If it ain't broke don't fix it :)
Instead of just straight sockets, you might consider a solution like AMQP.
When you're talking about using socket connections for a pub/sub interface, that usually means point-to-point communication, which isn't always a scalable solution. AMQP really addresses this sort of pub/sub problem. It's free, it's open-source, and it works.
Given that they've already solved the pub/sub problem, you might want to leverage off their work instead of doing it yourself.
I recommend OpenAMQ.
That being said, it really depends on what sort of environment you're working in. AMQP requires an AMQP broker (an application responsible for routing messages) to be running somewhere in the system.
Take a look at ActiveMQ. It's a JMS provider and also has bindings in lots of other languages, works on several transports, and has a selection of protocols that it uses for its messages.
RabbitMQ is also an AMQP broker, that supports other protocols like XMPP, HTTP, STOMP, SMTP as well, using adaptors.
Especially if you have done it before, and it still meets your needs, stick with it.
How about embedding a web server in your app? EasyHTTPD is open source, C++, and pretty simple to use. The asynchronous part is easy enough. You either spin up a new thread on the sender or the receiver.
If you're looking for XML, and cross-platform to boot, you may consider an XML-RPC mechanism like SOAP (I've used gSoap for communication between a C server on linux with a C++ client on win32, and it worked fine once you figure it all out).
If you are not heavily inclined towards XML and you want something simple and efficient then consider looking at YAMI library. It is rather simple to use and the given link contains enough documentation, rationale and examples to get started.