How does ZeroC ICE compare to 0MQ? I know that 0MQ/Crossroads and DDS are very similar, but cant seem to figure out where ICE comes in.
I need to quickly implement a system that offloads real-time market-data from C++ to C#, as a first phase of my project. The next phase will be to implement an Event Based architecture with an underlying Pub/Sub design.
I am willing to use TCP.. but the the system is currently running on a single 24 core server.. so an IPC option would be nice. From what I understand ICE is only TCP, while DDS and 0mq have an IPC option.
Currently ,I am leaning towards using Protobuf with either ICE or Crossroads IO. Got turned off from the OpenSplice DDS website. Ive done lots research on the various options, was originally considering OpenMPI + boost:mpi, but there does not seem to be MPI for .NET.
My question is:
How does ICE compare to 0MQ? I cant wrap my head around this. Was unable to find anything online that compares the two.
thanks in advance.
........
More about my project:
Currently using CMAKE C++ on Windows, but the plan is to move to CentOS at some point. An additional desired feature is to store the tic data and all the messages in a "NoSql" database such as Hbase/Hadoop or HDF5. Do any of these middleware/messaging/pub-sub libraries have any database integration?
Some thoughts about ZeroC:
Very fast; Able to have multiple endpoints; Able to load balance on the endpoints; Able to reconnect to a different endpoint in case one of the node goes down. This is transparent to the end user; Has good tool chain (IceGrid, IceStorm, IceBox, etc); Distributed, high availability, multiple failover, etc
Apart from that, I have used it for hot swapping code modules (something similar to Erlang) by having the client create the proxy with multiple endpoints, and later on bring down each endpoint for a quick upgrade one by one. With the transparent retry to a different endpoint, I could have the system up and running the whole time i did an upgrade. Not sure if this is an advertised feature or an unadvertised side-effect :)
Overall, it is very easy to scale out your servers if need be using ZeroC Ice.
I know ZeroMQ provides a fantastic set of tools and messaging patterns and I would keep using it for my pet projects. However, The problem that i see is that it is very easy to go overboard and lose track of all your distributed components. This is a must have in a distributed environment. How will you know where your clients/server are when you need to upgrade? If one of components down the chain does not receive a message, how to identify where the issue is? the publisher? the client? or any one of the bridges (REP/REQ, XREP/XREQ, etc) in between?
Overall, ZeroC provides a much better toolset and ecosystem for enterprise solutions.
And it is open source :)
Jaybny,
ZMQ:
If you want real good performance and the only job for Phase 1 of your job is to move data from C++ to C#, then Zmq is the best option.
Having a pub/sub model for event driven architecture is also something that Zmq can help you with, with its in-built messaging pattern.
Zmq also supports your IPC requirements in this case. Eg: you can have one instance of your application that consumes 24 cores by multithreading and communicating via IPC.
ZeroC Ice:
Ice is a RPC framework very much like CORBA.
Eg.
Socket/ZMQ - You send message over the wire. Read it at the other end, parse the message, do some action, etc.
ZeroC Ice - Create a contract between client and server. Contract is nothing but a template of a class. Now the client calls a proxy method of that class, and the server implements/actions it and returns the value. Thus, int result = mathClass.Add(10,20) is what the client calls. The method, parameters, etc is marshalled and sent to the server, server implements the Add method, returns the result, and the client gets 30 as the result. Thus on the client side, the api is nothing but a proxy for a servant running on a remote host.
Conclusion:
ZeroC ICE has some nice enterprisy features which are really good. However, for your project requirements, ZMQ is the right tool.
Hope this helps.
For me.. the correct answer was Crossroads I/O . It does everything I need.. but still unable to pub/sub when using protobufs... im sure ZeroC ICE is great for distributed IPC, but 0MQ/Crossroads, gives you the added flexibility to use Inter-Thread-Communication.
Note: on windows, 0mq does not have IPC.
So, all in all, the crossroads fork of 0mq is the best. but you will have to roll your own windows/ipc (or use tcp::127..) , and publisher side topic filtering features for pub/sub.
nanomsg, from the guy who wrote crossroads and 0mq (i think).
http://nanomsg.org/
Related
For my scientific collaboration, I wrote a C++ server/client (terminal server for linux, and cross-platform GUI client using Qt) that is mainly meant to transfer data from multiple clients around the world to store it on one server for analysis (pretty much like LIGO and Virgo that caught gravitational waves). For the communication low-level protocol I used boost::asio::ssl.
The "odd" part: In my programs, I created my own half-duplex messaging protocol between the server and the client from scratch. The messaging was in string form containing the version of the protocol, endianness of the computer, length of the message, type of the message (login/file/error/etc...), an MD5 hash for verification of the completeness of the transferred data. I got highly criticized on Stackoverflow chat when I said I did this. The part that got specially criticized is: Writing my own messaging protocol.
Now I created this, and I know there should be better protocols that are already written that I shouldn't rewrite it from scratch. However, I wanted to learn how to do this myself, and I did, and the program works, and my collaboration is satisfied, and the sky is blue with birds singing.
My question: If I am to rewrite this program again, what kind of libraries should I use? I'm looking for a protocol, using which I can send messages/data and get the server to respond with messages/data, including username/password to authenticate the user before any communication is transferred? How would you have done it?
PS: Please consider this question coming from a beginner in writing network and internet wide programs. And please don't hesitate to ask for more details.
i know this question maybe seems duplicate, but i think the new versions of RPC frameworks better to compare again, after all i m a newbie in RPC and HLA
my requirements:
Real-time pub/sub messaging architecture, i have 12 nodes connect each other and i want each process of my application run multiple times in different VM servers on each node
each process must know about its replicated process too, if memory in one VM goes up a replicated process must help this process in parallel
log of error occurred for each one of processes for tracing problem, and number of lost messages
i need supporting RTI and HLA for my simulation objects
why DDS is more used for critical systems like Military or Air Traffic management? is opensplice dds is that much good or OMG supported and created by military and DARPA guys too :D ?
do these frameworks provide such an options (for DDS opensource dds based on TAO ACE)?
what is my another options (like thrift)?
good compare of these frameworks? thanks a lot.
I am implementing a messaging system using C++ and Qt. After much thought, I have determined that multicasting or a multicast-style technique will work best to solve my problem. However, I have learned about UDP's unreliability and believe it to be unacceptable.
My requirements are as follows:
Messages are to be sent in a binary serialized form.
From any given node on the network, I must be able to send messages to the other nodes.
Message delivery must be insured.
I have heard of OpenPGM and NORM as alternatives for UDP. If anyone has experience with either of these, could you please share?
I am also open to the possibility of implementing "reliable" multicast by myself, in the application layer, but I would prefer not to if there is a library that already implements this.
I am using C++ and Qt, therefore .NET or Java-based solutions are not acceptable unless they are open-source and I may port them to C++.
Thank you very much.
EDIT 20120816T1853 MDT: An additional question: would either PGM or NORM have to be implemented at the hardware/IP level? Or can they be overlayed on top of existing protocols?
We have implemented our own reliable multicast protocol over UDP called RSP, since we needed something cross-platform and at the time couldn't find a good solution between Linux and Windows. The Windows PGM implementation disconnects slow clients which leave the send window, whereas our implementation throttles the sender similar to TCP. Afaik OpenPGM can be configured to do the same.
The open source NORM at http://cs.itd.nrl.navy.mil/work/norm can be built as a library and has C++ API with Python and Java language bindings. If you ping the developer (me) via the mailing list, I can help get you started.
There is a large RFC division of reliable multicast protocols, and many implementations out there. It's a long time since I looked at this but there are TRAM, LRMP, ...
I am penning down the features that a remote logging
library might need when built from scratch.
I looked up this: http://www.aggsoft.com/serial-data-logger.htm
I wish to know that what differences can be between a
remote logging library and a remote logger software.
Few things that I thought of:
1. The library can be used in C++ programs to log error messages on the fly.
2. The library will require programming knowledge on the end user's part.
3. The software cannot be used "inside" a C++ program, so we won't be able to log the error messages on the fly? Not sure about this one.
I would like to know that besides logging error messages, what are the things for which it makes sense to use the remote logging library? Sharing big files? Anything else than these two things?
Secondly which is better in what way out of a library and a software - in the current case?
As I mentioned in the my comments to your question, I would think that a logging library would provide some sort of an API/SDK, whereas remote software would not. The same would hold true if its sending messages via TCP/UDP or a serial port. The difference between the 2 options would be how much coding you would have to do. That is, how much would you have to reinvent the wheel?
IMHO, nearly all debug environment/tools support redirect the console output the serial port (using print, or other API). It usually not a a task of Application programmer.
There are other methods for "remote logging":
1) syslog, syslog-ng 's remote service
2) save log local, fetch using ftp
I read some questions here but couldn't really find the specific problem I'am faced with here...
I need to implement a "DeviceCache" in a particular project which caches all device-names found in /proc/net/dev .
The Language is C/++
So I thought about a seperate thread looking every X seconds in the directory mentioned above but was encouraged to find a more direct way.
How can I register a method of my process to the device manager of linux?
Is there a similar way like events/signals?
I looked in other sites but couldn't find any helpful code... Im relatively new to Linux-programming but willing to learn new things :)
Based on your comments, what you really want is to track which network interfaces are operational at any given time.
The only true way to determine if a network interface is up is to test it - after all, the router on the other end may be down. You could send pings out periodically, for example.
However, if you just want to know if the media goes down (ie, the network cable is unplugged), take a look at these SO questions:
Linux carrier detection notification
Get notified about network interface change on Linux
If you just want to be notified of the actual hardware-level registration of interfaces (eg, when a USB NIC is plugged in), you can use udev events if your platform has udev; otherwise, I believe there's another netlink category for hardware addition/removal events.