I've been playing around with Boost::Asio and checking out it's capability when it comes to n independent users collaborating to compute a function on shared inputs.
At one point I need to get each user to send to (resp. receive from) all the other users some data.
I guess effectively I'm looking for "active" establishment by each user opposed to the common client-server examples seen all over the internet.
I've got working code that allows a user to send data to another user. I'm having trouble seeing how I should get all users to be simultaneously sending and receiving from each other. I also can't seem to generalise multiple async_connects at once.
Are there any suggestions as to how this sort of protocol should be implemented? Should I be looking at using threads to manage both sending and receiving?
If you on Unix-based OS, such as Linux, I would start with much less complex solution using C programming language (or C++ if you need that), but without using Boost library. Just use plain old select syscall instead. See manpage and tutorial manpage.
Only after that I'd switch to something fancy like Boost.
See also this tutorial about BSD sockets programming.
Related
I need to couple two codes ( one is in Fortran77 and the other in Fortran90 ) which have to be controlled by a daemon and being able to pass information between them.
I have been searching and two possible options are PVM or MPI. The problem is that I need to compile them separately, any ideas?
MPI is well adapted to the SPMD paradigm (Single Program / Multiple data). If you want to couple 2 different binaries, MPI is probably not the best tool. Inter-process communication is more like what you want to do. In Linux, if you stay on the same machine, you can use named pipes (see man mkfifo) and you can transfer your data using Fortran I/O calls. Another possibility if you want to communicate between different machines is to use ZeroMQ for example, and there exists a Fortran binding.
The simplest way is using POSIX sockets - but you will need to do data serialization/deserialization and it is pretty slow in general. So I would not recommend using sockets.
Technically MPI can work. If you can use MPI 2.0 compliant library then you can use the client-server mechanism implemented there. Look at the documentation for MPI_Open_port and MPI_Comm_connect. The first one will give you the port name that you will need to pass this name to the client somehow. One option is to use name publishing but it may not work with any MPI library. The other option is to share it using some other mechanism (socket connection, file system or anything else).
But, in fact, I still do not see the reason why you should compile these two apps separately (unless there is a licensing issue) - you can just compile them into one package (I anticipate some code change but it is minor) and then run them as one app.
I'm writing a distributed system wherein each node interfaces with local applications via some RESTful API, supports extensions and runtime customization, etc. It's a bit like an Enterprise Service Bus over the wide area, but with a lot else going on which is not related to the question at hand.
I've read a little about both MPI and Asio - originally I was set on Asio, but then I found MPI, and now again I'm thinking Asio is the better solution for me. MPI seems to provide a lot I don't need and a higher level of abstraction than I want - for example, I only need point-to-point communication, and it is important to me to be in control of when and what data is transmitted. (e.g. I have already designed a packet structure that I would conform to ideally)
So my primary question: it is worth it to start from a lower level with Asio, or should I try to graft MPI onto what i'm looking for? Further, are there 'skeleton applications' available which use MPI or Asio which would aid development? (Actually I am 100% new to C++.. ;) Or, does it make sense to use them in tandem?
For more perspective, maybe it's worth noting that I already have implemented the bulk of this project in Perl using Perl Object Environment, which itself is just an asynchronous event system with a ton of networking libraries.
Also, if it makes a difference, I would ideally use threads for this.
Honestly though I have not used Boost at all yet, as I hinted above, so any input is appreciated.
I should start by saying that I know nothing about Asio, but from the 2 minute scan of the website and the description of your problem, it sounds like while either would probably work for you, Asio might be simpler. MPI is really designed less for general purpose network communication and more for running applications where the set of processes and applications is a little more static. While it provides a client/server style interface if desired, it's not the main focus of the library.
MPI is also more difficult to use if you already have a packet structure designed. MPI is great for abstracting away the need to worry about packets, location of processes, etc. but if you've already taken all of that into account in you application, you have already done the hard work.
There have been at least one other discussion of Asio vs. MPI that you can take a look at (For distributed applications, which to use, ASIO vs. MPI?) to get more opinions too.
I want one application on a linux host to call procedures from applications on Win7x64 in a VM. I guess that the VM runs on the linux host does not matter since it should use sockets. How do approach this plan? Are there any libraries for that in the internet?
Edit:
Well I took am look at all of them. XMLRPC seems to be some kind of predecessor of SOAP. Prototbuf seem to me like having the focus on easy serialization of large objects. So my decision falls on SOAP. But now I am searching for a nice-to-use c++ binding. I read a few pages of the apache axis2 manual but its everything else but nice-to-use, and anyway in C.
Another question in my specific case is: Isn't it easier to send just send some textual numbers plus parameters to identify the procedures to call and reply in textual form, as I know the signature of the called procedures?
Edit2:
As SOAP is just a standard and RPC is just curious, (Imagined something like function pointers over IP :D) ,imho none of them all is a solution. But ZeroMQ is indeed a progress.
There is no magic to RPC. I would suggest having a look at a combination of ZeroMQ and Google protobuf. ZeroMQ is a very easy to use messaging system (your communication layer). You would use the REQ/REP pattern. Google protobuf is used to describe and serialize/deserialize your messages. Both libraries are cross-platform and even cross language (ruby, python, c++, etc. etc.)
Have you taken a look at SOAP? It has pros and cons but may meet your needs.
The fact that your Windows box is VM shouldn't make a difference providing that it's up and running, and so long as the system you choose treats the servers as logically separate sockets normally would, your solution will remain flexible.
One decent alternative is XMLRPC, which runs on top of HTTP. It's simpler than SOAP, at least. I've used it for Java-Python communication, and it was not much code then, but don't know any C++ libraries to recommend, so I'll just say http://en.wikipedia.org/wiki/XML-RPC and its C++ section to that.
Edit after edited question: For a really simple solution, just use plain HTTP, mapping request path to function and giving parameters with GET or POST. Return value in HTTP response data as simply as you can: plain text for number or string, for more complex return value a binary blob if binary compatibility can be ensured (same CPU, same word size, plain struct, using compiler struct packing options if needed), or as json (or even XML, but then you are starting to reinvent SOAP/XMLRPC...).
I want to stream some data from my program for other subscribers (other programs). These programs can use these data as streaming event.
What I want?
How it is done generally?
Any libraries or papers pointing to technique and pros/cons?
Security related Ideas?
I will dig more information on my own even if I get a small hint.
Example :
Program A : Object A changed =======> Program B : Report Change in Object A
Two things are generally used: sockets/pipes which are just your basic byte streams, and message passing which is a bit more complex, made for parallel use and horizontal scalability.
I am not sure if it is a bit advanced, but have a look at boost::asio
http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/overview/core/basics.html
Best I found:
Google Protobuf
Facebook Thrift
Pros:
Helps with establishing the formats for streaming
Fast
Easy to build
Cons:
List item
Other top-level design issues (bandwidth control, cancelation) have to go on top.
Broadly it sounds like you're trying to perform inter-process communication, aka IPC.
In the tags to your question you refer to Windows. This link provides a broad list of the IPC options within Windows.
The tag list for your question also says platform-independent. Either a socket based solution or a Pipe based solution will mostly standard across a large number of platforms that you're likely to develop for. You can either use sockets directly or use one of the numerous cross-platform wrappers, eg. ZeroMQ and Boost, to hide some of the detail.
It's not clear from the question whether Program A and Program B are running on the same machine? If not then using sockets is a better approach.
at work we have been using JGroups and I have to say that I really like it, however, at home I primarily build with C/C++.
I'm wondering if anyone knows of any good/solid ports of JGroups to C++ or any other equivalent software library for C++. I'm interested in hearing pros/cons of various libraries out there.
Though this isn't a requirement, I am most familiar with SDL's SDL_net so a nice wrapper class around SDL_net would be the most favorable to me.
I don't know of any ports of JGroups to C++. Some folks run JGroups as a daemon and then talk to it from other languages via a protocol. I recently created a STOMP protocol, which allows for messages sent by clients written in different languages to be processed by JGroups, and for those clients to receive messages as well.
Re: native stacks, you could take a look at Spread (www.spread.org).
You might want to check out 0mq. Its pretty fast and easy to use.