Do ZMQ_ROUTER and ZMQ_DEALER need delimiter empty string? - c++

I have question about ZeroMQ Dealer and Router sockets. I have an architecture like this below:
Dealer1 and Dealer2 has ids and I set it with this:
m_socket->setsockopt(ZMQ_IDENTITY, socketId.data(), socketId.size());
I am sending messages from Dealer1 socket to Dealer3 via Router socket. It is a zmq::proxy
zmq::context_t context(1);
zmq::socket_t frontEnd(context, ZMQ_ROUTER);
frontEnd.bind(socketAddress);
zmq::socket_t backend(context, ZMQ_DEALER);
backend.bind("inproc://mainSocket");
zmq::proxy(frontEnd, backend, nullptr);
When I send message to Dealer3 from Dealer1 like this, everything is okey:
std::string ReceiveReplyString()
{
zmq::message_t message;
int ret = m_socket->recv(&message);
if (ret)
{
return std::string(static_cast<char*>(message.data()), message.size());
}
else
{
throw zmq::error_t();
}
}
const std::string emptyString = "";
zmq::message_t request(dataString.data(), dataString.size());
int ret = m_socket->send(request);
if (ret)
{
m_replyString = ReceiveReplyString();
//Sending empty string as delimeter
zmq::message_t delimeter(emptyString.size());
memcpy(delimeter.data(), emptyString.data(), emptyString.size());
m_socket->send(delimeter);
}
else
{
throw zmq::error_t();
}
}
I can send multiple messages but If I send messages without empty string after sending real message, I can not send or get any message after first one.

Related

C++ ZMQ Pub and Sub not connecting

I am currently working on a project that requires me to connect two terminals via ZMQ sockets, and my current solution does so via the PUB-SUB Socket functionality. However, when I run the programs, while the publisher sends the messages, the subscriber never receives any of the messages. I've tried changing the IP address between them, and trying to "brute force send" message between the sub and the pub, but to no avail.
Reduced form of the code:
Server.cpp:
#include <zmq.h>
const char* C_TO_S = "tcp://127.0.0.1:5557";
const char* S_TO_C = "tcp://127.0.0.1:5558";
int main() {
zmq::context_t context(1);
zmq::socket_t pub(context, ZMQ_PUB);
zmq::socket_t sub(context, ZMQ_SUB);
int sndhwm = 0;
sub.connect(C_TO_S);
pub.bind(S_TO_C);
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
//cout << C_TO_S << endl;
while(true) {
zmq::message_t rx_msg;
sub.recv(&rx_msg);
cout << "b\n";
// other code goes here
}
}
Client.cpp:
#incldue <zmq.h>
const char* C_TO_S = "tcp://127.0.0.1:5557";
const char* S_TO_C = "tcp://127.0.0.1:5558";
void network_thread() {
zmq::context_t context(1);
zmq::socket_t pub(context, ZMQ_PUB);
zmq::socket_t sub(context, ZMQ_SUB);
int sndhwm = 0;
sub.connect(S_TO_C);
pub.connect(C_TO_S);
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
while (true) {
cout << pub.send("a", strlen("a"), 0);
cout << "AA\n";
}
// Other code that doesn't matter
}
The main in Client.cpp calls network_thread in a separate thread, and then spams the publisher to send the message "a" to the server. However, the server does not get any message from the client. If the server got any message, it would print out "b", however it never does that. I also know that the publisher is sending messages because it prints out "1" upon execution.
Also, assume that the client subscriber and the server publisher has a purpose. While they don't work atm either, a fix to the other set should translate into a fix of those.
I have tried changing the port, spamming send messages, etc. Nothing resulted in the server receiving any messages.
Any help would be appreciated, thank you.
You set a message filter option on the SUB socket. This means that you will only receive messages that begin with the bytes set by the filter.
This code:
int sndhwm = 0;
sub.setsockopt(ZMQ_SUBSCRIBE, &sndhwm, sizeof(sndhwm));
Sets the filter to sizeof(sndhwm) bytes with value 0x00. But your message does not begin with this number of 0x00 bytes. Hence the message is ignored by the SUB socket.
You should remove the setsockopt call.
If your intent was to clear the message filter, you can do this with:
sub.setsockopt(ZMQ_SUBSCRIBE, nullptr, 0);

REQ socket drop message silently if two message coming together very fast

This is a reproducible message silently dropped issue with a very small changes to the lbbroker.cpp.
https://github.com/booksbyus/zguide/blob/master/examples/C%2B%2B/lbbroker.cpp
In the lbbroker.cpp file, a worker will be pop out from work_queue when handling a request from client thread. If we modify the line 163 from std::string worker_addr = worker_queue.front(); to static std::string worker_addr = worker_queue.front();, as the following picture shows. All request will be forwarded to one worker, then we can see some messages will be dropped by the REQ socket the worker. Is it normal? REQ socket cannot receive other message when the worker logic is running ?
The client thread:
static void * client_thread(void *arg) {
zmq::context_t context(1);
zmq::socket_t client(context, ZMQ_REQ);
#if (defined (WIN32))
s_set_id(client, (intptr_t)arg);
client.connect("tcp://localhost:5672"); // frontend
#else
s_set_id(client); // Set a printable identity
client.connect("ipc://frontend.ipc");
#endif
// Send request, get reply
s_send(client, "HELLO");
std::string reply = s_recv(client);
std::cout << "Client: " << reply << std::endl;
return (NULL);
}
The worker thread:
// Worker using REQ socket to do LRU routing
//
static void *
worker_thread(void *arg) {
zmq::context_t context(1);
zmq::socket_t worker(context, ZMQ_REQ);
#if (defined (WIN32))
s_set_id(worker, (intptr_t)arg);
worker.connect("tcp://localhost:5673"); // backend
#else
s_set_id(worker);
worker.connect("ipc://backend.ipc");
#endif
// Tell backend we're ready for work
s_send(worker, "READY");
while (1) {
// Read and save all frames until we get an empty frame
// In this example there is only 1 but it could be more
std::string address = s_recv(worker);
{
std::string empty = s_recv(worker);
assert(empty.size() == 0);
}
// Get request, send reply
std::string request = s_recv(worker);
std::cout << "Worker: " << request << std::endl;
s_sendmore(worker, address);
s_sendmore(worker, "");
s_send(worker, "OK");
}
return (NULL);
}
The main function(start client and worker threads):
Note: std::string worker_addr = worker_queue.front(); is changed to static std::string worker_addr = worker_queue.front(); to let all requests from different clients send to one work thread, then some message will be dropped by the REQ socket in work thread.
int main(int argc, char *argv[])
{
// Prepare our context and sockets
zmq::context_t context(1);
zmq::socket_t frontend(context, ZMQ_ROUTER);
zmq::socket_t backend(context, ZMQ_ROUTER);
#if (defined (WIN32))
frontend.bind("tcp://*:5672"); // frontend
backend.bind("tcp://*:5673"); // backend
#else
frontend.bind("ipc://frontend.ipc");
backend.bind("ipc://backend.ipc");
#endif
int client_nbr;
for (client_nbr = 0; client_nbr < 10; client_nbr++) {
pthread_t client;
pthread_create(&client, NULL, client_thread, (void *)(intptr_t)client_nbr);
}
int worker_nbr;
for (worker_nbr = 0; worker_nbr < 3; worker_nbr++) {
pthread_t worker;
pthread_create(&worker, NULL, worker_thread, (void *)(intptr_t)worker_nbr);
}
// Logic of LRU loop
// - Poll backend always, frontend only if 1+ worker ready
// - If worker replies, queue worker as ready and forward reply
// to client if necessary
// - If client requests, pop next worker and send request to it
//
// A very simple queue structure with known max size
std::queue<std::string> worker_queue;
while (1) {
// Initialize poll set
zmq::pollitem_t items[] = {
// Always poll for worker activity on backend
{ backend, 0, ZMQ_POLLIN, 0 },
// Poll front-end only if we have available workers
{ frontend, 0, ZMQ_POLLIN, 0 }
};
if (worker_queue.size())
zmq::poll(&items[0], 2, -1);
else
zmq::poll(&items[0], 1, -1);
// Handle worker activity on backend
if (items[0].revents & ZMQ_POLLIN) {
// Queue worker address for LRU routing
worker_queue.push(s_recv(backend));
{
// Second frame is empty
std::string empty = s_recv(backend);
assert(empty.size() == 0);
}
// Third frame is READY or else a client reply address
std::string client_addr = s_recv(backend);
// If client reply, send rest back to frontend
if (client_addr.compare("READY") != 0) {
{
std::string empty = s_recv(backend);
assert(empty.size() == 0);
}
std::string reply = s_recv(backend);
s_sendmore(frontend, client_addr);
s_sendmore(frontend, "");
s_send(frontend, reply);
if (--client_nbr == 0)
break;
}
}
if (items[1].revents & ZMQ_POLLIN) {
// Now get next client request, route to LRU worker
// Client request is [address][empty][request]
std::string client_addr = s_recv(frontend);
{
std::string empty = s_recv(frontend);
assert(empty.size() == 0);
}
std::string request = s_recv(frontend);
static std::string worker_addr = worker_queue.front();
worker_queue.pop();
s_sendmore(backend, worker_addr);
s_sendmore(backend, "");
s_sendmore(backend, client_addr);
s_sendmore(backend, "");
s_send(backend, request);
}
}
return 0;
}

How can I get to the message data in ZMQ using C++?

Within C++ ZMQ, I'm having trouble printing ( or otherwise analyzing ) data, which is pushed to consumers.In Python and scala it works great, but C++ the output is either null or some un-decodable junk.From official helper example:
//I added the printf line:
static std::string s_recv (zmq::socket_t & socket) {
zmq::message_t message;
socket.recv(&message);
printf ("[-] Message: %s\n", (char*)message.data());
return std::string(static_cast<char*>(message.data()), message.size());
}
The following is the context of where s_recv is called from:
int main () { // Official ZMQ Hello World example
zmq::context_t context (1); // Prepare our context and socket
zmq::socket_t socket ( context,
ZMQ_REP // REP-side of a REQ/REP pattern
);
socket.bind ( "tcp://*:5555" );
while (true) {
zmq::message_t request;
// Wait for next request from client
// socket.recv (&request); // commented out to treat as string
s_recv(socket); // added for same reason as above
std::cout << "Received Hello" << std::endl;
sleep(1); // Do some 'work'
zmq::message_t reply (5); // Send reply back to client
memcpy ((void *) reply.data (), "World", 5);
socket.send (reply);
}
return 0;
}
The output:
[-] Message: #
In my non-hello world code, I get output resembling some kind of encoding error, my terminal throws a bunch of "?".
How can I get to the messages in ZMQ using C++?
Edit for more info:
In my non-hello world code, I use PUSH / PULL, not a REQ / REP.
Sample pushing code, same problem with both code bases:
ifstream file(filePath.c_str(), ios::in|ios::binary);
file.read(&chunk_data[0], chunk_size);
memcpy((void*)message.data(), &chunk_data[0], this_chunk_size);
socket.send(message);
Output and binary garbage:
[-] Message: �_�z�

ZMQ C++ Req to Router issues

I am using ZeroMQ for my network layer and so far everything works except when it comes to ROUTER sockets. In particular I receive the expected message on the ROUTER but when I try send an answer back to my REQ socket the message is never received.
Here is a relatively simple test I wrote which tries to send a "HELLO" message to the ROUTER and expects a message to come back.
here the client code :
try
{
zmq::context_t myContext;
zmq::socket_t reqSocket(myContext, ZMQ_REQ);
reqSocket.setsockopt(ZMQ_IDENTITY, "REQ", 3);
reqSocket.connect(gpRouterAddress);
//request delimiter
zmq::message_t zmqMsgReqDelimiter(1);
memcpy ((void *) zmqMsgReqDelimiter.data(), "\0", 1);
reqSocket.send(zmqMsgReqDelimiter, ZMQ_SNDMORE);
//some message
zmq::message_t reqMsg(5);
memcpy ((void *) reqMsg.data(), "HELLO", 5);
reqSocket.send(reqMsg, 0);
int rcvMore = 0;
size_t sizeInt = sizeof(int);
bool bRcvMore = true;
std::vector<std::string> history;
while(bRcvMore)
{
zmq::message_t zmqMsg;
reqSocket.recv(&zmqMsg, rcvMore);
const char* pMsgStr = static_cast<char*>(zmqMsg.data());
history.push_back(pMsgStr);
reqSocket.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);
bRcvMore = (rcvMore == 1);
}
}
catch (zmq::error_t error)
{
std::string errorStr = error.what();
}
and here is my Router code (can run in a different thread , in which case theContext would be the same as "myContext" from the code above) or a entirely different application :
try
{
zmq::context_t theContext;
zmq::socket_t router (theContext, ZMQ_ROUTER);
int value = 1;
router.setsockopt(ZMQ_ROUTER_MANDATORY, &value, sizeof(int));
router.setsockopt(ZMQ_IDENTITY, "ROUT", 4);
router.bind(gpRouterAddress);
zmq::message_t zmqMsgInternalAddress;
router.recv(&zmqMsgInternalAddress, 0);
const char* pAddressStr = static_cast<char*>(zmqMsgInternalAddress.data());
zmq::message_t zmqMsgDelimiter;
router.recv(&zmqMsgDelimiter, ZMQ_RCVMORE);
const char* pDelimiterStr = static_cast<char*>(zmqMsgDelimiter.data());
int rcvMore = 0;
size_t sizeInt = sizeof(int);
bool bRcvMore = true;
router.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);
bRcvMore = (rcvMore == 1);
std::vector<std::string> history;
while(bRcvMore)
{
zmq::message_t zmqMsg;
router.recv(&zmqMsg, rcvMore);
const char* pMsgStr = static_cast<char*>(zmqMsg.data());
history.push_back(pMsgStr);
router.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);
bRcvMore = (rcvMore == 1);
}
//reply address
size_t len = strlen(pAddressStr) - 1; //if I don't subtract 1 char here, an exception will be raised
zmq::message_t replyAddress(len);
memcpy ((void *) replyAddress.data(), pAddressStr, len);
router.send(replyAddress, ZMQ_SNDMORE);
//reply delimiter
zmq::message_t zmqMsgReplyDelimiter(1);
memcpy ((void *) zmqMsgReplyDelimiter.data(), "\0", 1);
router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);
//some message
zmq::message_t replyMsg(5);
memcpy ((void *) replyMsg.data(), "WORLD", 5);
router.send(replyMsg, 0);
}
catch (zmq::error_t error)
{
std::string errorStr = error.what();
}
I receive the "HELLO" message on the router and I can step through the ROUTER send and everything seems to be sent okay (i.e. no exception is being raised) but I never receive a message on the REQ socket which will keep waiting indefinitely.
According to the ZeroMQ Guide I should expect the ROUTER to receive the following messages :
The REQ socket sends
empty
HELLO
the ROUTER receives
REQ
empty
HELLO
but I receive
REQ
some binary message
empty
HELLO
and I send
REQ
empty
WORLD
which I would expect to arrive at REQ as
empty
WORLD
If I connect to a REP socket instead (using a simple REQ-REP topology everything works fine).
Can anyone see what I am missing/doing wrong?
I found the issue.
Basically the error was in how I sent the delimiter
zmq::message_t zmqMsgReplyDelimiter(1);
memcpy ((void *) zmqMsgReplyDelimiter.data(), "\0", 1);
router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);
it should just be
zmq::message_t zmqMsgReplyDelimiter(0);
router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);

ZeroMQ C++ req-rep termination error

Here is ZeroMq C++ code for simple request - reply where both exchange messages alternatively.
But error occures when messages are only sent continuously ....
Reply code :
include zmq.hpp
include string
include iostream
include unistd.h
include ctime
int main () {
// Prepare our context and socket
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REP);
socket.bind ("tcp://*:5557");
zmq::message_t reply (5);
memcpy ((void *) reply.data (), "World", 5);
zmq::message_t request;
// Wait for next request from client
socket.recv (&request);
///////////////// """observe_line_1"""
std::cout << "Received Hello" << std::endl;
while (true)
{
// Send reply back to client
socket.send (reply);
}
return 0;
}
Request code :
include zmq.hpp
include string
include iostream
include ctime
int main ()
{
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REQ);
socket.connect ("tcp://localhost:5557");
zmq::message_t request (6);
memcpy ((void *) request.data (), "Hello", 5);
zmq::message_t reply;
socket.send(request);
//////////////////////////////////////////"""observe_line_2"""
for (int request_nbr = 0; request_nbr != 1000; request_nbr++)
{
socket.recv (&reply);
std::cout << "Received World " << request_nbr << std::endl;
}
return 0;
}
Output
After executing Reqply then executing request then following error occurs
terminate called after throwing an instance of 'zmq::error_t'
what(): Operation cannot be accomplished in current state
Aborted
Point to be observed
when "observe_line_1" is inside the for loop below it and "observe_line_2" is inside while loop below it and executed then the code is executed succesfully without any errors ????
Sending multiple replies for one request is not allowed by the REQ-REP model.
The REQ-REP socket pair is in lockstep. The client issues zmq_send() and then zmq_recv(), in a loop (or once if that's all it needs). Doing any other sequence (e.g., sending two messages in a row) will result in a return code of -1 from the send or recv call. Similarly, the service issues zmq_recv() and then zmq_send() in that order, as often as it needs to.
reference, alternative socket types, maybe this can help
Actually you can send more than one message in row, as long as you send data with flags=zmq::SNDMORE.