C++ Boost Serialization: Input Stream Error - c++

Question

Hi fellow C++ developers,
I'm trying to send a C++ class over the network with zmq and boost::serialization.
The concept is to serialize the class PlayCommand on the client. Then send it to the server with zmq. And the deserialize it on the server.
This works fine in the rest of the application. For some reason I get input stream errors while deserializing the PlayCommand on the server from time to time. I can not figure out why it is sometimes throwing this exception and sometimes not.
It seems to be a time sensitive problem. Do I have to wait at some point to let boost do its thing ?
std::shared_ptr<PlayCommand> _exe(dynamic_cast<PlayCommand*>(_cmd.get()));
zmq::context_t _ctx(1);
zmq::socket_t _skt(_ctx, ZMQ_PUB);
_skt.connect("tcp://0.0.0.0:" + this->kinect_daemon_com_port);
std::stringstream _type_stream;
std::stringstream _exe_stream;
boost::archive::text_oarchive _type_archive(_type_stream);
boost::archive::text_oarchive _exe_archive(_exe_stream);
_type_archive << _type;
_exe_archive << *_exe.get();
std::string _type_msg_str = _type_stream.str();
std::string _exe_msg_str = _exe_stream.str();
zmq::message_t _type_msg(_type_msg_str.length());
zmq::message_t _exe_msg(_exe_msg_str.length());
memcpy(_type_msg.data(), _type_msg_str.data(), _type_msg_str.length());
memcpy(_exe_msg.data(), _exe_msg_str.data(), _exe_msg_str.length());
_skt.send(_type_msg, ZMQ_SNDMORE);
_skt.send(_exe_msg, 0);
void ZMQMessageResolver::resolve_message(std::shared_ptr<Event> _event, unsigned _unique_thread_id)
{
std::cout << "ZMQMessageResolver::resolve_message(std::shared_ptr<Event> _event, unsigned _unique_thread_id)" << std::endl;
std::shared_ptr<ZMQMessageEvent> _zmq_event = std::static_pointer_cast<ZMQMessageEvent>(_event);
//(static_cast<ZMQMessageEvent*>(_event.get()));
ZMQMessageType _type;
PlayCommand _cmd;
auto _messages = _zmq_event->get_data();
auto _type_string = std::string(static_cast<char*>(_messages->front()->data()), _messages->front()->size());
auto _cmd_string = std::string(static_cast<char*>(_messages->back()->data()), _messages->back()->size());
std::stringstream _type_stream{_type_string};
std::istringstream _cmd_stream{_cmd_string};
boost::archive::text_iarchive _type_archive{_type_stream};
boost::archive::text_iarchive _cmd_archive{_cmd_stream};
std::cout << "1" << std::endl;
_type_archive >> _type;
std::cout << "2" << std::endl;
_cmd_archive & _cmd;
std::cout << "3" << std::endl;
std::shared_ptr<ThreadEvent> _thread_event = std::make_shared<ThreadEvent>(_zmq_event->get_event_message());
_cmd.execute(_thread_event);
std::lock_guard<std::mutex> _lock{*this->thread_mutex};
this->finished_threads.push_back(_unique_thread_id);
}
The complete project is on github: rgbd-calib and rgbd-calib-py.
The important files are /framework/ZMQMessageResolver.cpp in rgbd-calib and /src/KinectDaemon.cpp in rgbd-calib-py.
I would appreciate any help.
First insights
I checked for shared zmq::socket_t instances. I could not find any so thread safety should be a non issue.
I found out that other developers are also experiencing problems with ZMQ multi part messages. Maybe that could be an issue in my case as well. Maybe someone as experiences with those. Do I have to take any safety measures when sending and receiving multi part messages ?

Answer 1

If it's timing sensitive, no doubt it's unrelated to boost: the boost code shown is completely synchronous and local. If you don't always receive the full streams you get this error. Likewise if there's a protocol error interpreting the received data you might get corrupt data.
Both cases would easily lead to "input stream error".
I have no experience with 0MQ so I don't know whether the code as shown could receive incomplete messages, but I'd look into that.
A minor note is that it's rather strange to have one stringstream and the other istringstream. There might be differences in seek behaviour.

Answer 2

Let me add a few cents on ZeroMQ part of the story:
Fact #1:ZeroMQ does never deliver a piece of trash it delivers either a complete message( as it was sent ) or nothing at all
this principal design feature helps to sort out one of the claimed potential issues.
If an application indeed receives a ZeroMQ message delivered, one can be sure of its being of a shape and sound copy of what has been dispatched from the remote process. It is simply the same. Pullstop.
Fact #2: ZeroMQ architects and evangelists do in every chapter since the very beginning of the ZeroMQ API v2.xx warn to never share
which seems from the code depicted above unclear.
If one instantiates a ZeroMQ-socket AccessPoint ( of a SUB-type in the above context ), the thread owning such AccessPoint resource is the only thread that may manipulate with such resource and never "let" any other touch this toy. Never. While there might be some recent talks and efforts to re-design the ZeroMQ core, so as to add a thread-safety ex-post, I remain skeptical as per these moves, being principally sure that a non-blocking high-performance + low-latency motivated designs in distributed computing should never share a common piece, right because of the costs of overheads and lost principal safety ( which is not easy to be got just ex-post bought back by any inter-thread signalling / locking / blocking ).
You may review the code, so as to confirm or deny any kind of sharing ZeroMQ instances ( a Context being another, separate subject in this ) and for cases, where shared pieces were detected, your team ought re-design the code so as to avoid it.
Yes. Avoid sharing and ZeroMQ tools will serve you as a hell.