I'm developing an application in which distributed components talk to one another over a network, in an asynchronous, pub/sub kind of way.
For this, I like the idea of sending XML over sockets - it's asynchronous, I don't need a server of any kind, and it can work locally or over a network. I would have to roll my own pub/sub mechanism, which probably won't be too onerous - I could use the IP address as the id of the subscriber. The implementation language will be C++. I've used this method before with good results. But, I wonder if there is a better way? Any advice welcome.
Edit: I haven't made a decision yet, but I'm interested in cparcode's comment on protobuf:
"I never understood the popularity of using XML for network comms. Any way, I realize that the protocol is only part of your problem but on that subject, there's Google's protobuf too: http://code.google.com/p/protobuf/ – caparcode (Mar 11 at 1:01)"
Instead of using XML it might be best to use an existing message passing framework. Have a look at libt2n or d-bus
But it you have used your own socket implementation in the past with good results i don't see a reason why you should change. If it ain't broke don't fix it :)
Instead of just straight sockets, you might consider a solution like AMQP.
When you're talking about using socket connections for a pub/sub interface, that usually means point-to-point communication, which isn't always a scalable solution. AMQP really addresses this sort of pub/sub problem. It's free, it's open-source, and it works.
Given that they've already solved the pub/sub problem, you might want to leverage off their work instead of doing it yourself.
I recommend OpenAMQ.
That being said, it really depends on what sort of environment you're working in. AMQP requires an AMQP broker (an application responsible for routing messages) to be running somewhere in the system.
Take a look at ActiveMQ. It's a JMS provider and also has bindings in lots of other languages, works on several transports, and has a selection of protocols that it uses for its messages.
RabbitMQ is also an AMQP broker, that supports other protocols like XMPP, HTTP, STOMP, SMTP as well, using adaptors.
Especially if you have done it before, and it still meets your needs, stick with it.
How about embedding a web server in your app? EasyHTTPD is open source, C++, and pretty simple to use. The asynchronous part is easy enough. You either spin up a new thread on the sender or the receiver.
If you're looking for XML, and cross-platform to boot, you may consider an XML-RPC mechanism like SOAP (I've used gSoap for communication between a C server on linux with a C++ client on win32, and it worked fine once you figure it all out).
If you are not heavily inclined towards XML and you want something simple and efficient then consider looking at YAMI library. It is rather simple to use and the given link contains enough documentation, rationale and examples to get started.
Related
I am building a server-client application that involves heavy signal processing (e.g. FFT). I have a working application written in C++/Qt, where everything (signal processing and other calculations) is done in client and server just sends raw data. Now I feel it would be easier to implement these features on the server. So, that maintenance becomes easier.
As I am doing signal processing, I think I should stick to C++ for performance. But I am open to new ideas.
Constraints:
I need type checking so javascript is out of discussion.
Scaling includes adding more server and each server will have at the max
10-12 users. So, Hardware cost is important. I cannot use x number of
i7 processors.
No option of using cloud services.
So, right now my question is as follows:
How can I create web services using C++ for Linux server? (Although cross platform is not important, I would appreciate if I can achieve it.)
EDIT [02:09:2015]
Right now, I think the choice is between poco and C++ Rest SDK. I feel I should go for C++ Rest SDK. Mainly because it has only those features that I need. And Also it is supported by microsoft and uses boost internally. So, I feel in future, this might be well integreated with standard.
You could use cross-platform Poco library to implement HTTP server, it is really straightforward with this framework, and they have a lot of examples. You can also use JSON serialization (like rapidjson library) to implement REST service on top of HTTP - this way your Web service will be accesable by most of the modern Web frameworks.
You might want to take a look at the C++ Rest SDK, an open source, cross platform API from Microsoft.
Like #nogard suggested, I also recommend POCO for now. It's the most serious and feature-full solution. Given you mentioned Qt, I suggest you to take a look at Tufão.
EDIT:
I forgot to mention one comparison of mine on the C++ HTTP server frameworks.
If you directly handle HTTP requests, you might loose the functionality what Web Servers does well what it was build to do. I had a similar issue, what I did was wrap up my Qt c++ code inside a PHP extension. In your case you can do the same. Wrap your logic inside what ever technology you are about to use, doesn't matter it's PHP, net , Java or anything else.
What is a good choice for a simple http Server lib? It doesn't need high performance. I rather look for something simple for some REST/JSON communication ("API").
It must be able though to work in a multithreaded environment and must be able to handle large POST request.
Any suggestions? I already tried cpp-netlib but this seems to be much too complicated for such an easy task...
Edit: I am looking for something really light-weight and simple. E.g. like Sinatra in the Ruby world. Poco is for me another example of a too heavy-weight library.
The first one that comes to mind is Poco Library ( http://pocoproject.org/ )
Cross platform, stable, well documented. While the library itself offers more than you probably need you can build and omit the portions you aren't planning on using to reduce bloat.
They have a fully featured Net library that includes several salient classes and utilities.
Here is a pdf of slides from that library, of particular interest is the HTTPServer class:
http://pocoproject.org/slides/200-Network.pdf
Not sure about large POST data, but I've previously used mongoose: https://github.com/cesanta/mongoose/.
If the LGPL license is unwanted there is a MIT fork from when the project was MIT that also add a C++ API https://github.com/bel2125/civetweb
I would encourage you to start with http server samples in boost.asio. They are so simple and easy to understand, that you should be able to easily extend them as needed.
However, if you want to jump onto something more polished than just sample code, I know of 3 http servers in C++ which you may like to try:
"x0 - HTTP Web Server Framework" to me personally this one seems most promising, because it's lightweight and simple
"highpower / xiva" is a simple http server framework for delivering notifications to browsers
"Pion, a project of Atomic Labs" is a part of elaborate framework for handling large amounts of data
Pretty late answer; but hope this helps.
For your interest of a server that can handle REST, here is the easiest HTTP Server library to use (in my opinion): https://github.com/yhirose/cpp-httplib.
For JSON parsing, you may search for another library to use it in conjunction.
Personally, I'd go for Arachnida but that might be because I wrote it.
How or what do I need to know programming wise in order to interact with the web using c++. For instance i want to wrote a program that automatically sends invites to players on yahoo chess. How would i go about doing this?
You'll need to understand the basics of TCP/IP and HTTP, possibly UDP, and the protocols involved with Yahoo's chess systems or posses a tool to work around them (A brief search leads me to believe there are few if any). You'll probably need a network API, I'd suggest looking at:
QtNetwork Module
Boost.Asio
Where Qt is easy to use, Asio is more powerful, and more 'C++' in nature. Qt has some nice webkit components, and I've used it to build a small web server, which was a lot of fun. You can accomplish quite a lot with it.
This page says they've added a captcha system to prevent certain people from interacting with their systems. I'm not familiar with Yahoo games and what the result of this has on what you'd want to do, however it suggests to me they'd rather you didn't write code to interact with their systems.
For this you need to use network APIs and use server side script like PHP/ASP to communicate with the web and C using message queue.
I have a C++ process running in the background that will be generating 'events' infrequently that a Python process running on the same box will need to pick up.
The code on the C side needs to be as lightweight as possible.
The Python side is read-only.
The implementation must be cross-platform.
The data being sent is very simple.
What are my options?
Thanks
zeromq -- and nothing else. encode the messages as strings.
However, If you want to get serialiazation from a library use protobuf it will generate classes for Python and C++. You use the SerializeToString() and ParseFromString() functions on either end, and then pipe the strings via ZeroMq.
Problem solved, as I doubt any other solution is faster, and neither will any other solution be as easy to wire-up and simple to understand.
If want to use specific system primitives for rpc such as named pipes on Windows and Unix Domain Sockets on unix then you should look at Boost::ASIO. However, unless you have (a) a networking background, and (b) a very good understanding of C++, this will be very time consuming
Use zeromq, it's about as simple as you can get.
Google's protobuf is a great library for RPC between programs. It generates bindings for Python and C++.
If you need a distributed messaging system, you could also use something like RabbitMQ, zeromq, or ActiveMQ. See this question for a discussion on the message queue libraries.
Another option is to just call your C code from your Python code using the ctypes module rather than running the two programs separately.
How complex is your data? If it is simple I would serialize it as a string. If it was moderately complex I would use JSON. TCP is a good cross-platform IPC transport. Since you say that this IPC is rare the performance isn't very important, and TCP+JSON will be fine.
You can use Google GRPC for this
I will say you create a DLL that will manage the communication between the two. The python will load DLL and call method like getData() and the DLL will in turn communicate with process and get the data.
That should not be hard.
Also you can use XML file or SQLite database or any database to query data. The daemon will update DB and Python will keep querying. There might be a filed for indicating if the data in DB is already updated by daemon and then Python will query.
Of course it depends on performance and accuracy factors!
Basically my question is the exact same one as this:
Simple client/server, TCP/IP encrypting the message stream, SSL
The difference is that I need this for pure C++, not .NET. I cannot use 3rd party libraries, so unless it's a Windows system component (like the above) I need something with source so I can get the generel idea and build it myself.
Thanks :)
Quoting the other question for reference:
"Writing a little TCP/IP client server
app. Basically it creates a server,
and then you can create several
different clients and set up a bit of
a chat session. What I am wondering is
there is any way to incorporate, using
standard .net libraries some form of
encryption?
m_mainSocket = new
Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
Is there any way of speficying tcp
using rsa?
Or would you (me that is) have to
write some custom libaries to do key
exchange and then encrypt the
subsequent chat messages? I have done
that before for uni but that was in
java but I know it would'nt be hard to
convert them. Just trying not to have
to reinvent the wheel...
Or what about utilising a ssl?
Thanks, Ron."
Have you considered using the ASIO library? think-async dot com/Asio/
There is an example specifically for an SSL based client/server. http://think-async.com/Asio/asio-1.4.1/doc/asio/examples.html#asio.examples.ssl
Its as "pure c++" as you can get.
You can always look at OpenSSL which is open source, but that would be like implement SSL yourself. I would suggest wrapping OpenSSL and use it. Or use the SSL tunnel application available in OpenSSL.
Writing your own encryption code is "not recommended". It's easy enough to make a simple mistake when using one of these libraries, let alone when you try to write one yourself.
What you really want to use is OpenSSL with Boost.ASIO on top of it. If you can't do that then your next best alternative is to use the Internet Explorer COM object. This isn't quite as flexible, but might work out fine depending on what your exact needs are. You can also explore the Win32 API. Last I looked there weren't enough crypto APIs widely available to do this. The final way of dealing with this is to wrap the .NET APIs so that you can make use of them from native C++.
Only if none of that works out for you should you even consider writing this yourself. You will make mistakes and your application will be less secure as a result. So, before you start trying to write your own crypto code you could also try to look at tunnelling SOCKS over SSH and use somebody else's SSH implementation. The next thing I would look at is to buy in the code rather than write it yourself. The code won't be as good as open source offerings as it will be less used so will have more security problems, but it will still be better than anything you would write on your first outing doing this.
Only if you've exhausted all of these options should you think about writing this yourself. Once you think about it you should try all of the other options again to make sure that you didn't miss getting one of them to work for you the first time around.
If you do still write your own implementation then throw it away and use one of the other options before putting it into production use as there will be mistakes that compromise the security to the extent where you probably may as well not have bothered.
Sorry to sound down on all of this, but getting these things right is really hard and not something you can do by just taking a quick look at somebody else's implementation.