I'm having difficulties figuring out correct way of using boost serialization/asio to send objects over network. The message class is as simple as possible. It's not C++ friendly nor suitable for my needs, I just keep it simple temporarily to test asio/ser:
class message {
friend class boost::serialization::access;
public:
message(){}
int type;
int sender;
int assignment;
int nogood;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & type;
ar & sender;
ar & assignment;
ar & nogood;
}
};
On the client side when agent decides to send a message, sends it to server over it's tcp connection:
message m;
// do something to generate message
boost::asio::streambuf bufx;
std::ostream os( &bufx );
boost::archive::binary_oarchive ar( os );
ar & m;
boost::asio::write( socket, bufx);
Server side code:
boost::asio::streambuf bufx;
std::istream is(&bufx);
boost::archive::binary_iarchive ia(is); // <--- Exception: invalid signature
size_t rcx = asio::read(socket,bufx);
message m;
ia >> m;
I got same exception.
And This official example helps me.
If you are still in trouble, try this.
size_t n = sock.receive(bufs);
// received data is "committed" from output sequence to input sequence
b.commit(n);
std::istream is(&b);
std::string s;
is >> s;
In my case, I use async_read. Actually, I modified the example.
boost::asio::streambuf inbound_;
boost::asio::streambuf::mutable_buffers_type bufs = inbound_.prepare(inbound_data_size);
void (connection::*f)(
const boost::system::error_code&, std::size_t,
T&, boost::tuple<Handler>)
= &connection::handle_read_data<T, Handler>;
boost::asio::async_read(socket_, boost::asio::buffer(bufs),
boost::bind(f, this,
boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred, boost::ref(t), handler));
And then at the handler
/// Handle a completed read of message data.
template <typename T, typename Handler>
void handle_read_data(const boost::system::error_code& e, std::size_t bytes_transferred,
T& t, boost::tuple<Handler> handler)
{
if (e)
{
boost::get<0>(handler)(e);
}
else
{
// Extract the data structure from the data just received.
try
{
inbound_.commit(bytes_transferred);
std::istream archive_stream(&inbound_);
boost::archive::binary_iarchive archive(archive_stream);
archive >> t;
}
catch (std::exception& err)
{
// Unable to decode data.
boost::system::error_code error(boost::asio::error::invalid_argument);
boost::get<0>(handler)(error);
return;
}
// Inform caller that data has been received ok.
boost::get<0>(handler)(e);
}
}
In your server-side code, your streambuf is empty when you create the binary archive. If the archive constructor is looking for a magic number at the beginning of the archive then it won't find it. Try filling the streambuf with the call to boost::asio::read() before constructing the stream and archive.
Related
I've been debugging this for 2 days now. I've tried simpler code bases which supposedly reproduce the bug... But they work just fine. I'm not sure if the bug is in how I'm misusing the boost asio functions, or some other C++ feature.
I'm writing a TCP server, where the message structure is defined as:
uint64_t size // Size of the message, in bytes
uint64_t timestamp // Some timestamp
char *text // Some ASCII data
The code
The Header class, which represents the size and timestamp, is declared as:
class Header {
private:
uint64_t buf[2];
void *data() const {
return (void *) &buf[0];
}
public:
Header() {
buf[0] = 0UL;
buf[1] = 0UL;
}
// Reads the header from the socket
void readHeader(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
std::size_t getHeaderSize() const {
return sizeof(buf);
}
uint64_t getDatagramSize() const {
return buf[0];
}
uint64_t getTimestamp() const {
return buf[1];
}
};
The class ImuDatagram holds the whole message, and has methods to read the datagram from the socket.
class ImuDatagram {
public:
Header header;
std::string theJson;
void readDatagram(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
void readBody(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
};
The methods are defined as follows:
void Header::readHeader(boost::asio::ip::tcp::socket &socket,
std::function<void (void)> cb)
{
boost::asio::mutable_buffer asio_buf = boost::asio::buffer(data(), getHeaderSize());
boost::asio::async_read(socket, asio_buf,
[this, cb](const boost::system::error_code &ec, std::size_t bytes_transferred){
// TAG1
// bytes_transferred: 16 (as expected)
// getDatagramSize() == 116 (as expected)
// getTimestamp() == ... (as expected)
cb();
});
}
void ImuDatagram::readBody(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb) {
uint64_t dgSize = header.getDatagramSize();
uint64_t jsonSize = dgSize - header.getHeaderSize();
// TAG2
// This will throw std::bad_alloc.
theJson.reserve(jsonSize);
boost::asio::async_read(socket, boost::asio::buffer(theJson, jsonSize),
[this, &socket, cb](const boost::system::error_code &ec, std::size_t bytes_transferred) {
// Error checks omitted
cb();
});
}
void ImuDatagram::readDatagram(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb) {
header.readHeader(socket, [this, &socket, cb](){
readBody(socket, cb);
});
}
Finally, the main that gets everything going is:
tcp::acceptor imuAcceptor(ioc, tcp::endpoint(tcp::v4(), 8081));
imuAcceptor.async_accept(imuSocket, [this](const boost::system::error_code& ec)
{
// Error checks omitted
ImuDatagram imu;
imu.readDatagram(socket, [this, &imu, &socket]() {
// Do work
});
}
Problem Description
Consider TAG1 in Header::readHeader. 16 bytes were read (the size of the header's buffer), as expected. I confirmed that theJson's buffer wasn't overrun. Nothing wrong yet.
We then move to TAG2 in ImuDatagram::readBody. As indicated, this throws std::bad_alloc.
As part of my debugging, I inserted a protection byte after the header to see if it would get written. At TAG1, it is not written. At TAG2, it was written. At first, I thought async_read was writing past the buffer. But it isn't, as verified with the protection byte I added.
Can someone please tell me where this (obscure to me) bug is? I wrote a simpler version of this using STDIN instead of a socket, and it worked just fine. My suspicion is that the problem doesn't come from the socket though.
Thank you.
I am currently developing some software in C++ where I am sending and receiving custom data packets. I want to parse and manage these packets in a well structured manner. Obviously I am first receiving the header and after that the body of the data. The main problem is that I don't like creating a Packet-Object with only the header information and later on adding the body data. What is an elegant way of parsing and storing custom data packets?
Here is a rough sketch of what such a custom data packet could look like:
+-------+---------+---------+----------+------+
| Magic | Command | Options | Bodysize | Body |
+-------+---------+---------+----------+------+
(Lets assume Magic is 4 bytes, Command 1 byte, Options 2 bytes, Bodysize 4 bytes and the body itself is variable in length.)
How would I parse this without using any third party libraries?
Normally I'd say something like this could be done to store packet data:
#include <array>
class Packet {
public:
explicit Packet(std::array<char, 10> headerbytes);
void set_body(std::vector<char> data);
std::vector<char> get_body();
int8_t get_command();
int16_t get_options();
bool is_valid();
private:
bool valid;
int8_t _command;
int16_t _options;
int32_t body_size;
std::vector<char> _data;
};
The problem is that I provide the header-information first and than add the body data in a hacky way later on. The packet object has a point of time where it is accessible in an incomplete state.
I first receive the header and after the header was received another receive call is made to read the body.
Would it make sense to have a parser instance that populates information into the packet object only make it accessible once it holds all needed information? Would it make sense to have a separate class for the header and the body? What would be the best design choice?
I am developing with C++ and for the sending and receiving of data over sockets the boost library is used.
If you don’t want to tie the data reading into one complete constructor (for understandable reasons of separation of concerns), this is a good application for non-polymorphic inheritance:
struct Header {
static constexpr SIZE=10;
Header(std::array<char,SIZE>);
std::int8_t get_command() const {return command;}
std::int16_t get_options() const {return options;}
std::int32_t body_size() const {return length;}
private:
std::int8_t command;
std::int16_t options;
std::int32_t length;
};
struct Packet : private Header {
using Body=std::vector<char>;
Packet(const Header &h,Body b) : Header(h),body(std::move(b))
{if(body.size()!=body_size()) throw …;}
using Header::get_command;
using Header::get_options;
const Body& get_body() const {return body;}
private:
Body body;
};
// For some suitable Stream class:
Header read1(Stream &s)
{return {s.read<Header::SIZE>()};}
Packet read2(const Header &h,Stream &s)
{return {h,s.read(h.body_size())};}
Packet read(Stream &s)
{return read2(read1(s),s);}
Note that the private inheritance prevents undefined behavior from deleting a Packet via a Header*, as well as the surely-unintended
const Packet p=read(s);
const Packet q=read2(p,s); // same header?!
Composition would of course work as well, but might result in more adapter code in a full implementation.
If you were really optimizing, you could make a HeaderOnly without the body size and derive Header and Packet from that.
For this case I would use the pipeline design pattern creating 3 packet processor classes:
Command (handles magic bytes too)
Options
Body (handles body size too)
all derived from one base class.
typedef unsigned char byte;
namespace Packet
{
namespace Processor
{
namespace Field
{
class Item
{
public:
/// Returns true when the field was fully processed, false otherwise.
virtual bool operator () (const byte*& begin, const byte* const end) = 0;
};
class Command: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
class Options: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
class Body: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
}
class Manager
{
public:
/// Called every time new data is received
void operator () (const byte* begin, const byte* const end)
{
while((*fields[index])(begin, end))
{
incrementIndex();
}
}
protected:
void incrementIndex();
Field::Command command;
Field::Options options;
Field::Body body;
Field::Item* const fields[3] = { &command, &options, &body };
byte index;
};
}
}
You can use exceptions to prevent creation of incomplete packet objects.
I'd use char pointers instead of vectors for performance.
// not intended to be inherited
class Packet final {
public:
Packet(const char* data, unsigned int data_len) {
if(data_len < header_len) {
throw std::invalid_argument("data too small");
}
const char* dataIter = data;
if(!check_validity(dataIter)) {
throw std::invalid_argument("invalid magic word");
}
dataIter += sizeof(magic);
memcpy(&command, dataIter, sizeof(command)); // can use cast & assignment, too
dataIter += sizeof(command);
memcpy(&options, dataIter, sizeof(options)); // can use cast & assignment, too
dataIter += sizeof(options);
memcpy(&body_size, dataIter, sizeof(body_size)); // can use cast & assignment, too
dataIter += sizeof(body_size);
if( data_len < body_size+header_len) {
throw std::invalid_argument("data body too small");
}
body = new char[body_size];
memcpy(body, dataIter, body_size);
}
~Packet() {
delete[] body;
}
int8_t get_command() const {
return command;
}
int16_t get_options() const {
return options;
}
int32_t get_body_size() const {
return body_size;
}
const char* get_body() const {
return body;
}
private:
// assumes len enough, may add param in_len for robustness
static bool check_validity(const char* in_magic) {
return ( 0 == memcmp(magic, in_magic, sizeof(magic)) );
}
constexpr static char magic[] = {'a','b','c','d'};
int8_t command;
int16_t options;
int32_t body_size;
char* body;
constexpr static unsigned int header_len = sizeof(magic) + sizeof(command)
+ sizeof(options) + sizeof(body_size);
};
Note: this is my first post in SO, so please let me know if something's wrong with the post, thanks.
I'm guessing you are trying Object-oriented networking. If so, the best solution for such parsing would be Flatbuffers or Cap’n Proto C++ code generator. By defining a schema, you will get state machine code that will parse the packets in an efficient and safe way.
My session enables shared_from_this to make it possible to call boost::async_write and boost::asio::ip::tcp::socket::async_read before an object destruction. The problem is that when I want to chain some async_write and async_read functions, to create a state machine, the code gets ugly:
template <typename ArrType, std::size_t N> inline decltype(auto) boost_buf(ArrType (&arr)[N])
{
return boost::asio::buffer(arr, N);
}
...
// Inside session class
async_write(socket, boost_buf(header), [self = shared_from_this()](boost_err_code ec,
std::size_t /* len */) {
if (!ec)
self->socket.async_read_some(boost_buf(self->data), [self = std::move(self)](boost_err_code ec,
std::size_t /* len */) {
if (!ec)
async_write(
self->socket, boost_buf(respons1),
[self = std::move(self)](boost_err_code ec, std::size_t /* len */) {
if (!ec)
self->socket.async_read_some( ...
Is there a way to transform this to something like that:
a_write(header, ()[] {
a_read_into(buf, ()[] {
a_write(response1, ...
I could use bind function and for each lambda create a private method, but I would like to have the state machine visible within one scope which fits in my screen. I don't care about the content of the response.
I've been trying to figure out a_write and a_read_into to look like this:
template <typename F>
void a_write( tcp::socket &socket,
const std::string &data,
std::shared_ptr<mbarx_echo_mock_session> &&session,
F && f );
but the problem arises when chaining the second function.
I have the following class definition:
// SocketTypeT may be e.g. 'boost::asio::ip::tcp::socket'
template<class SocketTypeT>
class Socket : public SocketTypeT, public boost::enable_shared_from_this< Socket<SocketTypeT> > {
[...]
Within this class I have the following method 'writeAsync':
void writeAsync(const std::string& strData) {
boost::asio::async_write(*this, boost::asio::buffer(strData),
boost::bind(&Socket::handle_write_async,
shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
And finally the handler (also a class member function) used in 'writeAsync':
void handle_write_async(const boost::system::error_code& ec, std::size_t cntBytesSent) {
cout << "handle_write_async" << endl;
if (m_pSocketAsyncObserver) {
m_pSocketAsyncObserver->handleAsyncWrite(connectionClosed, cntBytesSent, ec);
}
}
Problem:
The data is successfully transmitted to the server, however 'handle_write_async' gets never called. What might be the reason for this?
For continuous execution of run you need to supply io_service::work object. Please read this question
I'm trying to bind a function to boost::asio::async_write, but I got a semantic error in write.hpp
class Client{
public:
Client(const int &frame_,char* buf,const int& size_){
frame=frame_;
b=buf;
size=size_;
}
void doNothing(){
//?
}
void handle(const boost::system::error_code& error,std::size_t bytes_transferred ){
//Should handle the socket being closed properly and the data in the buffer being released
cout<<"Bytes sent: "<<bytes_transferred<<endl;
cout<<error<<endl;
}
void runClient()
{
try
{
tcp::resolver resolver(io_service);
tcp::resolver::query query(tcp::v4(), host, port);
tcp::resolver::iterator iterator = resolver.resolve(query);
s=new boost::asio::ip::tcp::socket(io_service);
boost::asio::connect(*s, iterator);
std::cout << "Sending png: frame"<<frame<<" Size: "<<size<<"Bytes ... "<<endl;
int number_to_send = size; // Put your value
int converted_number = htonl(number_to_send);
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),&Client::doNothing);
boost::asio::async_write(*s, boost::asio::buffer(b, size),
boost::bind(&Client::handle,
this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
std::cout << "Done!"<<endl;
}
catch (std::exception& e)
{
std::cerr << "Exception: " << e.what() << "\n";
}
}
private:
enum { max_length = 1024 };
string port="1112";
string host="157.193.215.48";
char * b;
int size;
int frame;
boost::asio::io_service io_service;
boost::asio::ip::tcp::socket* s;
};
This code has no syntax errors, but it does give this error in write.hpp when compiling:
/usr/local/include/boost/asio/impl/write.hpp:615:3:
Called object type 'void (Client::*)()' is not a function or function pointer.
Affected code:
template <typename AsyncWriteStream, typename ConstBufferSequence,
typename WriteHandler>
inline BOOST_ASIO_INITFN_RESULT_TYPE(WriteHandler,
void (boost::system::error_code, std::size_t))
async_write(AsyncWriteStream& s, const ConstBufferSequence& buffers,
BOOST_ASIO_MOVE_ARG(WriteHandler) handler)
{
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a WriteHandler.
BOOST_ASIO_WRITE_HANDLER_CHECK(WriteHandler, handler) type_check;
detail::async_result_init<
WriteHandler, void (boost::system::error_code, std::size_t)> init(
BOOST_ASIO_MOVE_CAST(WriteHandler)(handler));
detail::write_op<AsyncWriteStream, ConstBufferSequence,
detail::transfer_all_t, BOOST_ASIO_HANDLER_TYPE(
WriteHandler, void (boost::system::error_code, std::size_t))>(
s, buffers, transfer_all(), init.handler)(
boost::system::error_code(), 0, 1);
return init.result.get();
}
Affected line:
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a WriteHandler.
BOOST_ASIO_WRITE_HANDLER_CHECK(WriteHandler, handler) type_check;
It's obvious that there's something wrong with my handlers, both of them. I just can't find out what and I'm getting really annoyed by it.
in your first async write, you send a pointer to a non static member function without having binded it to its this pointer like you did in the second write.
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),&Client::doNothing);
should be
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),boost::bind(&Client::doNothing,this));