ZeroMQ XPUB/XSUB Serious Flaw? - c++

It seems as though the XPUB/XSUB socket types have a serious flaw that is difficult to work around:
This is my implementation of that center node:
#include <zmq.hpp>
int main()
{
zmq::context_t context(1);
//Incoming publications come here
zmq::socket_t sub(context, ZMQ_XSUB);
sub.bind("ipc://subscriber.ipc");
//Outgoing publications go out through here.
zmq::socket_t pub(context, ZMQ_XPUB);
pub.bind("ipc://publisher.ipc");
zmq::proxy(sub, pub, nullptr);
return 0;
}
The problem is, of course, slow joiner syndrome. If I connect a new publisher to XSUB and publish some messages, they disappear into the void:
#include "zhelpers.hpp"
int main () {
// Prepare our context and publisher
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUB);
publisher.connect("ipc://subscriber.ipc");
s_sendmore (publisher, "B");
s_send (publisher, "Disappears into the void!!");
return 0;
}
However, if I sleep(1) after connecting to XSUB, it magically works:
#include "zhelpers.hpp"
int main () {
// Prepare our context and publisher
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUB);
publisher.connect("ipc://subscriber.ipc");
sleep(1);
s_sendmore (publisher, "B");
s_send (publisher, "Magically works!!");
return 0;
}
The Guide claims there is a simple solution to this "slow joiners" syndrome, but then never delivers a working synchronized XSUB/XPUB implementation. After much searching it looks like most people are just sleeping, which is really bad.
Why hasn't this ever been fixed? Are there any known workarounds? All my google queries just point me back to the guide...

I found one workaround here, and that is to use PUSH/PULL on the publisher side, and PUB/SUB on the subscriber side. The new topology looks like this:
This is the code you need for the center node:
#include <zmq.hpp>
int main()
{
zmq::context_t context(1);
//Incoming publications come here
zmq::socket_t sub(context, ZMQ_PULL);
sub.bind("ipc://subscriber.ipc");
//Outgoing publications go out through here.
zmq::socket_t pub(context, ZMQ_PUB);
pub.bind("ipc://publisher.ipc");
zmq::proxy(sub, pub, nullptr);
return 0;
}
And then for publishers:
#include "zhelpers.hpp"
int main () {
// Prepare our context and publisher
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUSH);
publisher.connect("ipc://subscriber.ipc");
s_sendmore (publisher, "B");
s_send (publisher, "No sleep!");
return 0;
}
This solution seems to work fairly well, and I hope people chime in if they see any downsides to it. If I come across a better answer, I'll post here.

The downside is your publishers are always publishing. In the XSUB/XPUB case, subscriptions are forwarded to your publishers so that they can restrict what they are sending to the proxy. That results in less network traffic and less work for the proxy. In the PULL/PUB case, the publishers never see the subscription information. A worst case scenario would be a subscriber's subscription means they only want data from one publisher. All publishers would still be sending their data to the proxy and the proxy would filter out everything but the one publisher's messages.

Related

C++ ZMQ Pub and Sub not connecting

I am currently working on a project that requires me to connect two terminals via ZMQ sockets, and my current solution does so via the PUB-SUB Socket functionality. However, when I run the programs, while the publisher sends the messages, the subscriber never receives any of the messages. I've tried changing the IP address between them, and trying to "brute force send" message between the sub and the pub, but to no avail.
Reduced form of the code:
Server.cpp:
#include <zmq.h>
const char* C_TO_S = "tcp://127.0.0.1:5557";
const char* S_TO_C = "tcp://127.0.0.1:5558";
int main() {
zmq::context_t context(1);
zmq::socket_t pub(context, ZMQ_PUB);
zmq::socket_t sub(context, ZMQ_SUB);
int sndhwm = 0;
sub.connect(C_TO_S);
pub.bind(S_TO_C);
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
//cout << C_TO_S << endl;
while(true) {
zmq::message_t rx_msg;
sub.recv(&rx_msg);
cout << "b\n";
// other code goes here
}
}
Client.cpp:
#incldue <zmq.h>
const char* C_TO_S = "tcp://127.0.0.1:5557";
const char* S_TO_C = "tcp://127.0.0.1:5558";
void network_thread() {
zmq::context_t context(1);
zmq::socket_t pub(context, ZMQ_PUB);
zmq::socket_t sub(context, ZMQ_SUB);
int sndhwm = 0;
sub.connect(S_TO_C);
pub.connect(C_TO_S);
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
while (true) {
cout << pub.send("a", strlen("a"), 0);
cout << "AA\n";
}
// Other code that doesn't matter
}
The main in Client.cpp calls network_thread in a separate thread, and then spams the publisher to send the message "a" to the server. However, the server does not get any message from the client. If the server got any message, it would print out "b", however it never does that. I also know that the publisher is sending messages because it prints out "1" upon execution.
Also, assume that the client subscriber and the server publisher has a purpose. While they don't work atm either, a fix to the other set should translate into a fix of those.
I have tried changing the port, spamming send messages, etc. Nothing resulted in the server receiving any messages.
Any help would be appreciated, thank you.
You set a message filter option on the SUB socket. This means that you will only receive messages that begin with the bytes set by the filter.
This code:
int sndhwm = 0;
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
Sets the filter to sizeof(sndhwm) bytes with value 0x00. But your message does not begin with this number of 0x00 bytes. Hence the message is ignored by the SUB socket.
You should remove the setsockopt call.
If your intent was to clear the message filter, you can do this with:
sub.setsockopt(ZMQ_SUBSCRIBE, nullptr, 0);

ZMQ_DEALER with ZMQ_REP

I'm sorry for this kinda stupid question, but I didn't find any other answer. How can I send a message from ZMQ_DEALER to ZMQ_REP?
There is server code:
std::string ans;
zmq::context_t context;
zmq::socket_t socket(context, ZMQ_DEALER);
int port = bind_socket(socket);
std::cout<<port<<std::endl;
std::cout<<"sending\n";
send_message(socket,"test");
std::cout<<"SUCCESS\n";
std::cout<<"trying to get msg from client...\n";
ans=receive_message(socket);
std::cout<<"TOTAL SUCCESS\n";
std::cout<<ans<<std::endl;
close(port);
and there is client code:
zmq::context_t context;
zmq::socket_t socket(context, ZMQ_REP);
std::string recv;
recv=receive_message(socket);
std::cout<<" total successss\n";
send_message(socket,"success");
std::cout<<recv<<std::endl;
Client can't receive message from server. I tried to find something in official ZeroMQ book, and I found this:
"When a ZMQ_DEALER socket is connected to a ZMQ_REP socket each message sent must consist of an empty message part, the delimiter, followed by one or more body parts."
As demanded by the ZeroMQ documentation, the sender has to take care of following the API requirement.
This should meet the need:
#include <string>
#include <zmq.hpp>
int main()
{
zmq::context_t aCTX;
// ----------------------------------------------------------------- Context() instance
zmq::socket_t aDemoSOCKET( aCTX, zmq::socket_type::dealer);
// ----------------------------------------------------------------- ZeroMQ Archetype
aDemoSOCKET.bind( "inproc://DEMO" );
// ----------------------------------------------------------------- ZeroMQ I/F
const std::string_view BLANK_FRAME_MSG = "";
const std::string_view PAYLOAD_FRAME_MSG = "Hello, world ...";
...
aDemoSOCKET.send( zmq::buffer( BLANK_FRAME_MSG ), zmq::send_flags::sndmore ); //[*]
aDemoSOCKET.send( zmq::buffer( PAYLOAD_FRAME_MSG ), zmq::send_flags::dontwait );
...
}
The API-requested empty frame is the trick [*], enforced by the flags-parameter in the c-API. There is no more magic behind this. If in doubts, feel free to seek further in many real-world helpful answers here

Why a zmq REQ-REP not working?

I have 1 master ( using REQ ) and 2 slaves ( A, B ) using REP. The master sends a request to one of the slaves and expects a response from him.
The message is being sent to the wrong slave, even if I set the address in the ZMQ envelope. How to specify the slave address? I think I am setting it correctly in master, but it's not working and sending the requests from the master in round robin fashion.
master.cpp
#include "zhelpers.hpp"
#include <string>
int main (int argc, char *argv[])
{
zmq::context_t context(1);
zmq::socket_t requester(context, ZMQ_REQ);
requester.setsockopt(ZMQ_IDENTITY,"M");
requester.bind("tcp://*:5559");
for( int request = 0 ; request < 10 ; request++) {
std::string cmd;
std::cin>>cmd;
s_sendmore (requester, "B");
s_sendmore (requester, "");
s_send (requester, cmd);
s_dump(requester);
}
}
slaveA.cpp
#include "zhelpers.hpp"
int main (int argc, char *argv[])
{
zmq::context_t context(1);
zmq::socket_t responder(context, ZMQ_REQ);
responder.setsockopt(ZMQ_IDENTITY, "A", 1);
responder.connect("tcp://localhost:5559");
while(1)
{
s_dump(responder);
sleep (1);//
// s_sendmore (responder, "M"); //Should I set this ??
// s_sendmore (responder, "");
s_send (responder, "FromSlaveA");
}
}
slaveB.cpp
#include "zhelpers.hpp"
int main (int argc, char *argv[])
{
zmq::context_t context(1);
zmq::socket_t responder(context, ZMQ_REP);
responder.setsockopt(ZMQ_IDENTITY, "B", 1);
responder.connect("tcp://localhost:5559");
while(1)
{
s_dump(responder);
sleep (1);
s_send (responder, "FromSlaveB");
}
}
What is wrong?
OS: Ubuntu 16.04, ZMQ version 4.X.X
Update 1:
Changed the slaveA socket to REP but still master is sending the message to slaveA and SlaveB in a round robin fashion. Now, I think am I setting the message envelop correctly to slaveB ? But when I print the envelope, I get this at slave's that proves I set the envelop to B correctly, isn't ?
[001]B
[000]
[005]jjjjj
Your slaveA.cpp ought use the proper ZeroMQ access-point archetype:
zmq::socket_t responder( context, ZMQ_REP ); // !ZMQ_REQ);
Post Festum: Documentation to be read twice before generating issues
ZMQ_IDENTITY: Set socket identity
The ZMQ_IDENTITY option shall set the identity of the specified socket when connecting to a ROUTER socket. The identity should be from 1 to 255 bytes long and may contain any values.
If two clients use the same identity when connecting to a ROUTER, the results shall depend on the ZMQ_ROUTER_HANDOVER option setting. If that is not set ( or set to the default of zero ), the ROUTER socket shall reject clients trying to connect with an already-used identity. If that option is set to 1, the ROUTER socket shall hand-over the connection to the new client and disconnect the existing one.
Option value type binary data
Option value unit N/A
Default value NULL
Applicable socket types ZMQ_REQ, ZMQ_REP, ZMQ_ROUTER, ZMQ_DEALER.
Sorry,this sort of .setsockopt() settings is:
1) this feature is excellently documented having all relevant details of operation & possible concepts of use
2) meaningful only for objects that are not present in your code at all
3) decide on a functional behaviour that is not relevant to your code at all
4) this feature is absolutely irrelevant to the hard-wired round-robin delivery of the objects that are present in your code, for the outgoing traffic.
So, read the documentation, it is "enough" and a fair way to professionally avoid generating issues.
Q.E.D.

ZMQ Publish/Subscribe pattern publisher connects to subscribers

I am new to ZMQ and already did the tutorials about the publish subscribe pattern. But for my application they don't quite apply. I have two types of applications. Application one can create connections to multiple "Applications two"s over network and send them data.
I tried implementing this with the Publish/Subscribe pattern, but instead of the subscriber connecting to the publisher the publisher connects to the subscribers.
Publisher:
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_PUB);
socket.connect("tcp://localhost:5555");
std::string text = "Hello World";
zmq::message_t message(text.size());
memcpy(message.data(), text.c_str(), text.size());
socket.send(message);
Subscriber:
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_SUB);
socket.bind("tcp://*:5555");
const char* filter = "Hello ";
socket.setsockopt(ZMQ_SUBSCRIBE, filter, strlen(filter));
zmq::message_t request;
socket.recv(&request);
std::string message = std::string(static_cast<char*>(request.data()), request.size());
std::cout << "Message received!" << std::endl;
std::cout << message << std::endl;
The publisher finnishes without error, but the subscriber is stuck in the recv(). And yes I am starting them in the right order (subscriber first)
I found the solution myself:
The problem is, that the publisher send the message, before the subscriber was ready to receive.
A simple
zmq_sleep(1)
before the
socket.send(message);
did the job.

ZeroMQ C++ req-rep termination error

Here is ZeroMq C++ code for simple request - reply where both exchange messages alternatively.
But error occures when messages are only sent continuously ....
Reply code :
include zmq.hpp
include string
include iostream
include unistd.h
include ctime
int main () {
// Prepare our context and socket
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REP);
socket.bind ("tcp://*:5557");
zmq::message_t reply (5);
memcpy ((void *) reply.data (), "World", 5);
zmq::message_t request;
// Wait for next request from client
socket.recv (&request);
///////////////// """observe_line_1"""
std::cout << "Received Hello" << std::endl;
while (true)
{
// Send reply back to client
socket.send (reply);
}
return 0;
}
Request code :
include zmq.hpp
include string
include iostream
include ctime
int main ()
{
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REQ);
socket.connect ("tcp://localhost:5557");
zmq::message_t request (6);
memcpy ((void *) request.data (), "Hello", 5);
zmq::message_t reply;
socket.send(request);
//////////////////////////////////////////"""observe_line_2"""
for (int request_nbr = 0; request_nbr != 1000; request_nbr++)
{
socket.recv (&reply);
std::cout << "Received World " << request_nbr << std::endl;
}
return 0;
}
Output
After executing Reqply then executing request then following error occurs
terminate called after throwing an instance of 'zmq::error_t'
what(): Operation cannot be accomplished in current state
Aborted
Point to be observed
when "observe_line_1" is inside the for loop below it and "observe_line_2" is inside while loop below it and executed then the code is executed succesfully without any errors ????
Sending multiple replies for one request is not allowed by the REQ-REP model.
The REQ-REP socket pair is in lockstep. The client issues zmq_send() and then zmq_recv(), in a loop (or once if that's all it needs). Doing any other sequence (e.g., sending two messages in a row) will result in a return code of -1 from the send or recv call. Similarly, the service issues zmq_recv() and then zmq_send() in that order, as often as it needs to.
reference, alternative socket types, maybe this can help
Actually you can send more than one message in row, as long as you send data with flags=zmq::SNDMORE.