Not able to parse Jsoncpp object received in 0MQ socket - c++

I am building a server and client in c++ which use ZeroMQ PAIR sockets to communicate. Since I want to call some RPC from client to server, I am using JSON::Value structure to encode the function name and arguments, so that the server can parse it and call the appropriate function.
I checked out gRPC for doing the same but I felt it as an overkill as it requires too much effort to get it right.
As a draft, I am created a sample client and server apps. I am able to send the data from client to server, but on the server side I am receiving parse error. Could anyone suggest me what I might be doing wrong?
client.cpp
#include <zmq.hpp>
#include <string>
#include <iostream>
#include <jsoncpp/json/value.h>
#include <jsoncpp/json/reader.h>
#include <jsoncpp/json/writer.h>
int main ()
{
// Prepare our context and socket
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_PAIR);
/**
* Json object value parser
*/
Json::Value out;
Json::Value res;
Json::StreamWriterBuilder builder;
out["MESSAGE"] = "Anaconda";
out["NEWS"] = "Something is wrong";
std::cout << "Connecting to hello world server…" << std::endl;
socket.connect ("tcp://localhost:5555");
zmq::message_t request(out.size());
std::string str = Json::writeString(builder, out);
std::cout<<str<<std::endl;
memcpy (request.data(),&str, out.size());
socket.send (request);
// Get the reply.
zmq::message_t reply;
socket.recv (&reply);
return 0;
}
server.cpp
int main () {
// Prepare our context and socket
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_PAIR);
socket.bind ("tcp://*:5555");
/**
* Json object value parser
*/
Json::Reader mReader = {};
Json::Value res;
while (true) {
zmq::message_t request;
// Wait for next request from client
socket.recv (&request);
std::string str = std::string(static_cast<char*>(request.data()), request.size());
std::cout <<str<<std::endl;
auto ok = mReader.parse(str, res);
if (!ok) {
std::cout <<"ConHash:: Error while parsing: %s "<< mReader.getFormattedErrorMessages().c_str() << std::endl;
return false;
} else {
std::cout<<"Successfully parsed !!" <<std::endl;
}
// Do some 'work'
}
return 0;
}
Also could you please advice me is there any other better way to make RPC from client to server?

You are using the size of the Json::Value out object as the size to send over the socket. This is incorrect, you need to use the size of the newly created string.
Change the code to:
std::string str = Json::writeString(builder, out);
zmq::message_t request(str.c_str(), str.size());
socket.send (request);

Related

Serializing and Deserializing in a client-server architecture

I'm using cereal to serialize and deserialize data in c++, and I came upon a problem. I'm sending data using a socket to the client side, so i send a stringstream with the serialized files in a JSON format.
The problem is that I can't deserialize it with simple data types, nor with complex ones, on the client side as it fails a rapidjson check and it tells me it's not an object
What would be the proper way of deserializing data considering that i can't create an instance of a class from the server side ?
Here is a simple example in which i try to send the username of a user to the client side
The send function :
void DBUser::SendUser()
{
std::stringstream os;
{
cereal::JSONOutputArchive archive_out(os);
archive_out(CEREAL_NVP(m_user));
}
ServerSocket* instance= ServerSocket::GetInstance();
instance->SendData(os.str());
}
In case it is needed, here is the function used to send the stringstream to the client side
void ServerSocket::SendData(std::string message)
{
try {
asio::error_code ignored_error;
asio::write(*socket, asio::buffer(message), ignored_error);
}
catch (std::exception& e)
{
std::cerr << e.what() << std::endl;
}
}
And here is the code when i try to deserialize:
std::array<char, 5000> buf;
asio::error_code error;
size_t len = socket.read_some(asio::buffer(buf), error);
std::string testString;
std::stringstream is(buf.data());
{
cereal::JSONInputArchive archive_in(is);
archive_in(testString);
}
std::cout << "Message from server: ";
std::cout << testString;
std::cout << std::endl;
You need to null terminate the received socket data. Can you try null terminating it like buf[len] = '\0'; after this size_t len = socket.read_some(asio::buffer(buf), error); statement.

c++ : How to send a struct from client to server through socket and serialize it?

I developed a c++ program for communication between client and server through socket. They are sending to each other string messages.
On the other hand, I developed a c++ program for serializing data into human readable form.
My problem is, how to modify those programs in order to send a structure from client to server and then the server serialize it and save it into a file ?
This is the server program :
#include <iostream>
#include <boost/asio.hpp>
using namespace boost::asio;
using ip::tcp;
using std::string;
using std::cout;
using std::endl;
string read_(tcp::socket & socket) {
boost::asio::streambuf buf;
boost::asio::read_until( socket, buf, "\n" );
string data = boost::asio::buffer_cast<const char*>(buf.data());
return data;
}
void send_(tcp::socket & socket, const string& message) {
const string msg = message + "\n";
boost::asio::write( socket, boost::asio::buffer(message) );
}
int main() {
boost::asio::io_service io_service;
//listen for new connection
tcp::acceptor acceptor_(io_service, tcp::endpoint(tcp::v4(), 1234 ));
//socket creation
tcp::socket socket_(io_service);
//waiting for connection
acceptor_.accept(socket_);
//read operation
string message = read_(socket_);
cout << message << endl;
//write operation
send_(socket_, "Hello From Server!");
cout << "Servent sent Hello message to Client!" << endl;
return 0;
}
This is the client program :
#include <iostream>
#include <boost/asio.hpp>
using namespace boost::asio;
using ip::tcp;
using std::string;
using std::cout;
using std::endl;
int main() {
boost::asio::io_service io_service;
//socket creation
tcp::socket socket(io_service);
//connection
socket.connect( tcp::endpoint( boost::asio::ip::address::from_string("127.0.0.1"), 1234 ));
// request/message from client
const string msg = "Hello from Client!\n";
boost::system::error_code error;
boost::asio::write( socket, boost::asio::buffer(msg), error );
if( !error ) {
cout << "Client sent hello message!" << endl;
}
else {
cout << "send failed: " << error.message() << endl;
}
// getting response from server
boost::asio::streambuf receive_buffer;
boost::asio::read(socket, receive_buffer, boost::asio::transfer_all(), error);
if( error && error != boost::asio::error::eof ) {
cout << "receive failed: " << error.message() << endl;
}
else {
const char* data = boost::asio::buffer_cast<const char*>(receive_buffer.data());
cout << data << endl;
}
return 0;
}
This is how serialize strcuture :
const char* PERSON_FORMAT_OUT = "(%s, %d, %c)\n";
//const char* PERSON_FORMAT_IN = "(%[^,], %d, %c)\n";
typedef struct Person {
char name[20];
int age;
char gender;
} Person;
int main (int argc, char* argv[]) {
Person p1 = {
.name = "Andrew",
.age = 22,
.gender = 'M'
};
//Person p2;
FILE* file;
fopen_s(&file, "people.dat", "w+");
fseek(file, 0, SEEK_SET);
fprintf_s(file, PERSON_FORMAT_OUT, p1.name, p1.age, p1.gender);
//fscanf_s(file, PERSON_FORMAT_IN, p2.name, 20, &p2.age, &p2.gender);
return 0;
}
If you intend to have common structures definitions so that both client and server can exchange information using the same format, I would recommend using Protocol Buffers (https://developers.google.com/protocol-buffers). You can generate a .proto file with the data definitions and share it between client/server code. Another option is to use gRPC (https://grpc.io/).
You can send any characters over the socket, so you could even define your own communication protocol with no standard structure as you did above. Another approach would be to define your data structure in JSON/XML and use a third-party library such as https://github.com/nlohmann/json to create and parse the structures. You could then convert these structures into STL strings or string streams and transport those over your socket connection.
The main drawback of going this way, and not the protobuf or grpc way, is that any time you need to change the structure definition you will need to update both your client and server code. You also get free goodies from using these, as they are more flexible, maintainable and scale better if the communication interface feature set grows.
Use Protobuf https://developers.google.com/protocol-buffers/docs/cpptutorial
Use ZeroMQ or any message queue rather than using plain socket.https://zeromq.org/
Best things for protocol is to use google protobufs
We solve this using:
FastBinaryEncoding - for defining data model and serialization
CppServer - for transport layer (tcp, ssl, http, https, WebSockets)
There is a relatively new kid on the block that does excellent job of defining multi platform/language structures and manage its serialization Flatbuffers. In contrast with protobuf, it does not have packing/unpacking step. Generally, it saves one memory copy and trade off is extra code needed on construction of object. If you care about performance, gRPC and flatbuffers is a probably a best choice.

ZMQ Hello world does not work

I try to run simple ZMQ application ( ROUTER/DEALER ).
I just send a request from DEALER to ROUTER, send it back. But DEALER cannot receive it.
I run it in one process ( ROUTER has its own thread ).
#include <zmq.hpp>
#include <string>
#include <iostream>
#include <thread>
void router()
{
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_ROUTER);
socket.bind("tcp://*:5561");
while(1)
{
// Wait for next request from client
zmq::message_t reply;
socket.recv (&reply);
std::cout << "Router: Received request" << std::endl;
// Send reply back to client
std::string string= "example";
zmq::message_t message(string.size());
memcpy (message.data(), string.data(), string.size());
std::cout << "Router: Sending" << std::endl;
socket.send (message);
}
}
int main ()
{
std::thread t{&router};
// Prepare our context and socket
zmq::context_t context (2);
zmq::socket_t socket (context, ZMQ_DEALER);
std::cout << "Dealer: Connecting to hello world server…" << std::endl;
socket.connect ("tcp://127.0.0.1:5561");
for (int i = 0; i != 10; i++)
{
zmq::message_t request (5);
memcpy (request.data (), "Hello", 5);
std::cout << "Dealer: Sending Hello " << i << "…" << std::endl;
socket.send (request);
zmq::message_t reply;
socket.recv (&reply);
std::cout << "Dealer: Received " << i << std::endl;
}
return 0;
}
I have an output:
Dealer: Connecting to hello world server…
Dealer: Sending Hello 0…
Router: Received request
Router: Sending
Router: Received request
Router: Sending
From ZMQ's documentation on socket :
When receiving messages a ZMQ_ROUTER socket shall prepend a message
part containing the identity of the originating peer to the message
before passing it to the application. Messages received are
fair-queued from among all connected peers. When sending messages a
ZMQ_ROUTER socket shall remove the first part of the message and use
it to determine the identity of the peer the message shall be routed
to.
So modify your code to something like this :
#include <zmq.hpp>
#include <string>
#include <iostream>
#include <thread>
#include <unistd.h>
void router()
{
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_ROUTER);
socket.bind("tcp://*:5561");
while(1) // Fix that infinite loop or your thread won't join
{
// Wait for next request from client
zmq::message_t id;
socket.recv (&id);
zmq::message_t reply;
socket.recv (&reply);
std::cout << std::string(static_cast<char*>(reply.data()),reply.size()) << std::endl;
std::cout << "Router: Received request" << std::endl;
// Send reply back to client
zmq::message_t copy_id;
copy_id.copy(&id);
std::string string= "example";
zmq::message_t message(string.size());
memcpy (message.data(), string.data(), string.size());
std::cout << "Router: Sending" << std::endl;
socket.send(id, ZMQ_SNDMORE);
socket.send(message);
}
sleep(1);
socket.setsockopt(ZMQ_LINGER, 0);
socket.close();
context.close();
}
int main ()
{
std::thread t{&router};
// Prepare our context and socket
zmq::context_t context (2);
zmq::socket_t socket (context, ZMQ_DEALER);
std::cout << "Dealer: Connecting to hello world server…" << std::endl;
socket.connect ("tcp://127.0.0.1:5561");
for (int i = 0; i != 10; i++)
{
zmq::message_t request (5);
memcpy (request.data (), "Hello", 5);
std::cout << "Dealer: Sending Hello " << i << "…" << std::endl;
socket.send(request);
zmq::message_t reply;
socket.recv(&reply);
std::cout << "Dealer: Received " << i << std::endl;
}
socket.setsockopt(ZMQ_LINGER, 0);
socket.close();
context.close();
t.join();
return 0;
}

Receiving 0 size messages while using ZMQ and protobuf

I am trying to set up a basic communication system between client and server using ZMQ. I am using protobuf for the message format.
My problem is when I send the message from client the message size is 34 but the message size received on the server is 0.
Following is my code;
Client.cpp
tutorial::Person person;
person.set_id(1234);
person.set_name("john");
person.set_email("john#mxyz.com");
person.set_phonenumber("12345678");
zmq::context_t context (1); // Prepare our context and socket
zmq::socket_t socket (context, ZMQ_PAIR);
std::cout << "Connecting to server…" << std::endl;
int linger = 0;// ms
socket.setsockopt(ZMQ_LINGER, &linger, sizeof(linger));
socket.connect ("tcp://127.0.0.1:20000");
std::string msg_str;
person.SerializeToString(&msg_str);
std::cout << "Size of message string is "<< msg_str.size()<<std::endl;
zmq::message_t request (msg_str.size());
memcpy ((void *) request.data (), msg_str.c_str(), msg_str.size());
std::cout << "Sending Person data ..." << std::endl;
socket.send (request);
socket.close();
google::protobuf::ShutdownProtobufLibrary();
return 0;
Server.cpp :
zmq::context_t context(1); // Prepare our context and socket
zmq::socket_t socket(context, ZMQ_PAIR);
int linger = 0; // ms
socket.setsockopt(ZMQ_LINGER, &linger, sizeof(linger));
socket.bind("tcp://127.0.0.1:20000");
while (true)
{
zmq::message_t request;
int recieved = socket.recv(&request);
std::string msg(static_cast<char*>(request.data()),request.size());
std::cout<<"Size of message recieved is "<< msg.size()<<std::endl;
tutorial::Person person;
person.ParseFromString(msg);
std::string text_str1;
google::protobuf::TextFormat::PrintToString(person, &text_str1);
}
socket.close();
Output of Client is :
Size of message string is 34
Output of server is :
Size of message received is 0
I have tried tried switching to ParseToArray also but it did not help.
Any kind of help is appreciated.
I have solved it with another way. Actually earlier I installed libzmq lib , but later when I switched to libzmq3-dev it worked for me.
I had the exact same problem; what I found is that the zmq::message_t had to go out of scope OR rebuild needed to be called.
For example, if the publisher looked like this.
zmq::message_t msg(3);
memcpy(msg.data(), "abc", 3);
while (1 == 1) {
pubSocket.send(msg);
sleep(1);
}
The subscriber would receive messages of zero length. If the code was changed to
while (1 == 1) {
zmq::message_t msg(3);
memcpy(msg.data(), "abc", 3);
pubSocket.send(msg);
sleep(1);
}
OR
zmq::message_t msg(3);
memcpy(msg.data(), "abc", 3);
while (1 == 1) {
pubSocket.send(msg);
msg.rebuild(3);
sleep(1);
}
Then the subscriber would receive a 3-byte message.

Zeromq: How to access tcp message in c++

I am a new-by to ZeroMQ and make my way through the C++ hello-world example of the echo client-server pattern (Request-Reply). The server looks like:
//
// Hello World server in C++
// Binds REP socket to tcp://*:5555
// Expects "Hello" from client, replies with "World"
//
#include <zmq.hpp>
#include <string>
#include <iostream>
#include <unistd.h>
int main () {
// Prepare our context and socket
zmq::context_t context (1);
zmq::socket_t socket (context, ZMQ_REP);
socket.bind ("tcp://*:5555");
while (true) {
zmq::message_t request;
// Wait for next request from client
socket.recv (&request);
std::cout << "Received Hello" << std::endl;
// Do some 'work'
sleep (1);
// Send reply back to client
zmq::message_t reply (5);
memcpy ((void *) reply.data (), "World", 5);
socket.send (reply);
}
return 0;
}
Now my question: How can I access / read the real data that socket.recv() ? Trying:
std::cout << request << std::endl;
resulted in an error message:
error: no match for ‘operator<<’ in ‘std::operator<< [with _Traits =
std::char_traits<char>](((std::basic_ostream<char, std::char_traits<char> >&)
(& std::cout)), ((const char*)"Received Hello")) << request’
The same goes for the client side that is sending the message. I don't find a way to display the real message...
The hello world example goes only half way and outputs the hard-coded values:
std::cout << "Received Hello" << std::endl;
Printing the actual response can be done as follows:
zmq::message_t reply;
socket.recv (&reply);
std::string rpl = std::string(static_cast<char*>(reply.data()), reply.size());
std::cout << rpl << std::endl;
There are some other useful examples in zhelpers.hpp.
I found that the following does what I want:
zmq::message_t request (msglen);
memcpy ( (void *) request.data(), myMessage, msglen);
char * requestmsg = new char [msglen];
memcpy (requestmsg, request.data(), request.size());
requestsocket.send (request);
std::cout << "Sending " << requestmsg << std::endl;
where msglen is of type int and myMessage is const char * tyoe.
In this way, the server receives a human readable message.
Hope this is not against any zeromq rules...
While I think we need to go through the guide before we can write elegant ZeroMQ codes. I found lines of simple codes from the HELLO WORLD example for extracting data received from the socket and sending back response:
zmq::message_t request;
socket.recv (&request);
std::cout << "Received request: [" << (char*) request.data() << "]" << std::endl;
// Do some 'work'
Sleep (1);
// Send reply back to client
zmq::message_t reply (6);
memcpy ((void *) reply.data (), "World", 6);
socket.send (reply);
However, this solution does not specify the length of received data, following Nikolai Koudelia's way above, I make a string for received data:
std::cout << "Received request: [" << std::string(static_cast<char*>(request.data()), request.size()) << "]" << std::endl;