I've been debugging this for 2 days now. I've tried simpler code bases which supposedly reproduce the bug... But they work just fine. I'm not sure if the bug is in how I'm misusing the boost asio functions, or some other C++ feature.
I'm writing a TCP server, where the message structure is defined as:
uint64_t size // Size of the message, in bytes
uint64_t timestamp // Some timestamp
char *text // Some ASCII data
The code
The Header class, which represents the size and timestamp, is declared as:
class Header {
private:
uint64_t buf[2];
void *data() const {
return (void *) &buf[0];
}
public:
Header() {
buf[0] = 0UL;
buf[1] = 0UL;
}
// Reads the header from the socket
void readHeader(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
std::size_t getHeaderSize() const {
return sizeof(buf);
}
uint64_t getDatagramSize() const {
return buf[0];
}
uint64_t getTimestamp() const {
return buf[1];
}
};
The class ImuDatagram holds the whole message, and has methods to read the datagram from the socket.
class ImuDatagram {
public:
Header header;
std::string theJson;
void readDatagram(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
void readBody(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb);
};
The methods are defined as follows:
void Header::readHeader(boost::asio::ip::tcp::socket &socket,
std::function<void (void)> cb)
{
boost::asio::mutable_buffer asio_buf = boost::asio::buffer(data(), getHeaderSize());
boost::asio::async_read(socket, asio_buf,
[this, cb](const boost::system::error_code &ec, std::size_t bytes_transferred){
// TAG1
// bytes_transferred: 16 (as expected)
// getDatagramSize() == 116 (as expected)
// getTimestamp() == ... (as expected)
cb();
});
}
void ImuDatagram::readBody(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb) {
uint64_t dgSize = header.getDatagramSize();
uint64_t jsonSize = dgSize - header.getHeaderSize();
// TAG2
// This will throw std::bad_alloc.
theJson.reserve(jsonSize);
boost::asio::async_read(socket, boost::asio::buffer(theJson, jsonSize),
[this, &socket, cb](const boost::system::error_code &ec, std::size_t bytes_transferred) {
// Error checks omitted
cb();
});
}
void ImuDatagram::readDatagram(boost::asio::ip::tcp::socket &socket, std::function<void (void)> cb) {
header.readHeader(socket, [this, &socket, cb](){
readBody(socket, cb);
});
}
Finally, the main that gets everything going is:
tcp::acceptor imuAcceptor(ioc, tcp::endpoint(tcp::v4(), 8081));
imuAcceptor.async_accept(imuSocket, [this](const boost::system::error_code& ec)
{
// Error checks omitted
ImuDatagram imu;
imu.readDatagram(socket, [this, &imu, &socket]() {
// Do work
});
}
Problem Description
Consider TAG1 in Header::readHeader. 16 bytes were read (the size of the header's buffer), as expected. I confirmed that theJson's buffer wasn't overrun. Nothing wrong yet.
We then move to TAG2 in ImuDatagram::readBody. As indicated, this throws std::bad_alloc.
As part of my debugging, I inserted a protection byte after the header to see if it would get written. At TAG1, it is not written. At TAG2, it was written. At first, I thought async_read was writing past the buffer. But it isn't, as verified with the protection byte I added.
Can someone please tell me where this (obscure to me) bug is? I wrote a simpler version of this using STDIN instead of a socket, and it worked just fine. My suspicion is that the problem doesn't come from the socket though.
Thank you.
Related
My session enables shared_from_this to make it possible to call boost::async_write and boost::asio::ip::tcp::socket::async_read before an object destruction. The problem is that when I want to chain some async_write and async_read functions, to create a state machine, the code gets ugly:
template <typename ArrType, std::size_t N> inline decltype(auto) boost_buf(ArrType (&arr)[N])
{
return boost::asio::buffer(arr, N);
}
...
// Inside session class
async_write(socket, boost_buf(header), [self = shared_from_this()](boost_err_code ec,
std::size_t /* len */) {
if (!ec)
self->socket.async_read_some(boost_buf(self->data), [self = std::move(self)](boost_err_code ec,
std::size_t /* len */) {
if (!ec)
async_write(
self->socket, boost_buf(respons1),
[self = std::move(self)](boost_err_code ec, std::size_t /* len */) {
if (!ec)
self->socket.async_read_some( ...
Is there a way to transform this to something like that:
a_write(header, ()[] {
a_read_into(buf, ()[] {
a_write(response1, ...
I could use bind function and for each lambda create a private method, but I would like to have the state machine visible within one scope which fits in my screen. I don't care about the content of the response.
I've been trying to figure out a_write and a_read_into to look like this:
template <typename F>
void a_write( tcp::socket &socket,
const std::string &data,
std::shared_ptr<mbarx_echo_mock_session> &&session,
F && f );
but the problem arises when chaining the second function.
I need to establish up to three different TCP connections to different servers. All three connections requiring different protocols, different handshakes and different heartbeats. Studying http://www.boost.org/doc/libs/1_61_0/doc/html/boost_asio/example/cpp11/chat/chat_client.cpp, reading stuff here and following Chris Kohlhoffs advices I tried to implement it as below.
The problem is that with this architecture I'm getting a bad_weak_pointer exception at calling shared_from_this() in doConnect() no matter what I'm doing.
Importent These are just snippets of a not running code, which can contain bugs! Importent
I'm having a base class which is containing some basic methods.
Connection.h
class Connection : public std::enable_shared_from_this<Connection>
{
public:
//! Ctor
inline Connection();
//! Dtor
inline virtual ~Connection();
inline void setReconnectTime(const long &reconnectAfterMilisec)
{
m_reconnectTime = boost::posix_time::milliseconds(reconnectAfterMilisec);
}
inline void setHandshakePeriod(const long &periodInMilisec)
{
m_handshakePeriod = boost::posix_time::milliseconds(periodInMilisec);
}
virtual void doConnect() = 0;
virtual void stop() = 0;
//... and some view more...
}
I have then my three classes which are derived from the base class. Here just one (and also the core part) to depict the approach.
ConnectionA.h
//queues which containing also the age of the messages
typedef std::deque<std::pair<handshakeMsg, boost::posix_time::ptime>> handskMsg_queue;
typedef std::deque<std::pair<errorcodeMsg, boost::posix_time::ptime>> ecMsg_queue;
typedef std::deque<std::pair<A_Msg, boost::posix_time::ptime>> A_Msg_queue;
class ConnectionA : public Connection
{
public:
ConnectionA();
ConnectionA(const std::string& IP, const int &port);
ConnectionA& operator=(const ConnectionA &other);
virtual ~ConnectionA();
virtual void stop() override;
virtual void doConnect() override;
void doPost(std::string &message);
void doHandshake();
void sendErrorCode(const int &ec);
std::shared_ptr<boost::asio::io_service>m_ioS;
private:
std::shared_ptr<tcp::socket> m_socket;
std::shared_ptr<boost::asio::deadline_timer> m_deadlineTimer; // for reconnetions
std::shared_ptr<boost::asio::deadline_timer> m_handshakeTimer; // for heartbeats
void deadlineTimer_handler(const boost::system::error_code& error);
void handshakeTimer_handler(const boost::system::error_code& error);
void doRead();
void doWrite();
std::string m_IP;
int m_port;
handskMsg_queue m_handskMsgQueue;
ecMsg_queue m_ecMsgQueue;
A_Msg_queue m_AMsgQueue;
}
ConnectionA.cpp
ConnectionA::ConnectionA(const std::string &IP, const int &port)
: m_ioS()
, m_socket()
, m_deadlineTimer()
, m_handshakeTimer()
, m_IP(IP)
, m_port(port)
, m_handskMsgQueue(10)
, m_ecMsgQueue(10)
, m_AMsgQueue(10)
{
m_ioS = std::make_shared<boost::asio::io_service>();
m_socket = std::make_shared<tcp::socket>(*m_ioS);
m_deadlineTimer = std::make_shared<boost::asio::deadline_timer>(*m_ioS);
m_handshakeTimer = std::make_shared<boost::asio::deadline_timer> (*m_ioS);
m_deadlineTimer->async_wait(boost::bind(&ConnectionA::deadlineTimer_handler, this, boost::asio::placeholders::error));
m_handshakeTimer->async_wait(boost::bind(&ConnectionA::handshakeTimer_handler, this, boost::asio::placeholders::error));
}
ConnectionA::~ConnectionA()
{}
void ConnectionA::stop()
{
m_ioS->post([this]() { m_socket->close(); });
m_deadlineTimer->cancel();
m_handshakeTimer->cancel();
}
void ConnectionA::doConnect()
{
if (m_socket->is_open()){
return;
}
tcp::resolver resolver(*m_ioS);
std::string portAsString = std::to_string(m_port);
auto endpoint_iter = resolver.resolve({ m_IP.c_str(), portAsString.c_str() });
m_deadlineTimer->expires_from_now(m_reconnectTime);
// this gives me a bad_weak_pointer exception!!!
auto self = std::static_pointer_cast<ConnectionA>(static_cast<ConnectionA*>(this)->shared_from_this());
boost::asio::async_connect(*m_socket, endpoint_iter, [this, self](boost::system::error_code ec, tcp::resolver::iterator){
if (!ec)
{
doHandshake();
doRead();
}
else {
// don't know if async_connect can fail but set the socket to open
if (m_socket->is_open()){
m_socket->close();
}
}
});
}
void ConnectionA::doRead()
{
auto self(shared_from_this());
boost::asio::async_read(*m_socket,
boost::asio::buffer(m_readBuf, m_readBufSize),
[this, self](boost::system::error_code ec, std::size_t){
if(!ec){
// check server answer for errors
}
doRead();
}
else {
stop();
}
});
}
void ConnectionA::doPost(std::string &message)
{
A_Msg newMsg (message);
auto self(shared_from_this());
m_ioS->post([this, self, newMsg](){
bool writeInProgress = false;
if (!m_A_MsgQueue.empty()){
writeInProgress = true;
}
boost::posix_time::ptime currentTime = time_traits_t::now();
m_AMsgQueue.push_back(std::make_pair(newMsg,currentTime));
if (!writeInProgress)
{
doWrite();
}
});
}
void ConnectionA::doWrite()
{
while (!m_AMsgQueue.empty())
{
if (m_AMsgQueue.front().second + m_maxMsgAge < time_traits_t::now()){
m_AMsgQueue.pop_front();
continue;
}
if (!m_socket->is_open()){
continue;
}
auto self(shared_from_this());
boost::asio::async_write(*m_socket,
boost::asio::buffer(m_AMsgQueue.front().first.data(),
m_AMsgQueue.front().first.A_lenght),
[this, self](boost::system::error_code ec, std::size_t /*length*/)
{
if (!ec) // successful
{
m_handshakeTimer->expires_from_now(m_handshakePeriod); // reset timer
m_AMsgQueue.pop_front();
doWrite();
}
else {
if (m_socket->is_open()){
m_socket->close();
}
}
});
}
}
void ConnectionA::deadlineTimer_handler(const boost::system::error_code& error){
if (m_stopped){
return;
}
m_deadlineTimer->async_wait(boost::bind(&ConnectionA::deadlineTimer_handler, this, boost::asio::placeholders::error));
if (!error && !m_socket->is_open()) // timer expired and no connection was established
{
doConnect();
}
else if (!error && m_socket->is_open()){ // timer expired and connection was established
m_deadlineTimer->expires_at(boost::posix_time::pos_infin); // to reactivate timer call doConnect()
}
}
And finally there is also another class which encapsulate these classes make it more comfortable to use:
TcpConnect.h
class CTcpConnect
{
public:
/*! Ctor
*/
CTcpConnect();
//! Dtor
~CTcpConnect();
void initConnectionA(std::string &IP, const int &port);
void initConnectionB(std::string &IP, const int &port);
void initConnectionC(std::string &IP, const int &port);
void postMessageA(std::string &message);
void run();
void stop();
private:
ConnectionA m_AConnection;
ConnectionB m_BConnection;
ConnectionC m_CConnection;
}
TcpConnect.cpp
CTcpConnect::CTcpConnect()
: m_AConnection()
, m_BConnection()
, m_CConnection()
{}
CTcpConnect::~CTcpConnect()
{}
void CTcpConnect::run(){
[this](){ m_AConnection.m_ioS->run(); };
[this](){ m_BConnection.m_ioS->run(); };
[this](){ m_CConnection.m_ioS->run(); };
}
void CTcpConnect::stop(){
m_AConnection.stop();
m_BConnection.stop();
m_CConnection.stop();
}
void CTcpConnect::initConnectionA(std::string &IP, const int &port)
{
m_AConnection = ConnectionA(IP, port);
m_AConnection.setMaxMsgAge(30000);
//... set some view parameter more
m_AConnection.doConnect();
}
// initConnectionB & initConnectionC are quite the same
void CTcpConnect::postMessageA(std::string &message)
{
m_AConnection.doWrite(message);
}
In the beginning I tried also to have only one io_service (for my approach this would be fine), but holding the service just as reference gave some headache, because my implementation requires also a default constructor for the connections. Now each connection has its own io-service.
Any ideas how I can bring this code to run?
Feel free to make suggestion for other architectures. If you could came up this some snippets would be even the better. I'm struggling with this implementation for weeks already. I'm grateful for every hint.
BTW I'm using boost 1.61 with VS12.
This is the problem:
m_AConnection = ConnectionA(IP, port);
That is, ConnectionA derives from Connection which derives from enable_shared_from_this. That means that ConnectionA must be instantiated as a shared pointer for shared_from_this to work.
Try this:
void CTcpConnect::initConnectionA(std::string &IP, const int &port)
{
m_AConnection = std::make_shared<ConnectionA>(IP, port);
m_AConnection->setMaxMsgAge(30000);
//... set some view parameter more
m_AConnection->doConnect();
}
EDIT1:
You are right. That was the issue. Now I realised that the way I'm calling io-service.run() is total crap.
It is very uncommon to use more than one io_service, and extremely uncommon to use one per connection :)
However, do you know if I need the cast then calling shared_from_this()? I noticed the asynch_connect() works fine with and without the cast.
Many Asio examples use shared_from_this() for convenience, I for example don't use it in my projects at all. There are certain rules that you need to be careful when working with Asio. For example, one is that the reading and writing buffers must not be destructed before the corresponding callback is executed, if the lambda function captures a shared pointer to the object that holds the buffers, this condition holds trivially.
You could for example do something like this as well:
auto data = std::make_shared<std::vector<uint8_t>>(10);
async_read(socket,
boost::asio::const_buffer(*data),
[data](boost::system::error_code, size_t) {});
It would be valid but would have the performance drawback that you'd be allocating a new data inside std::vector on each read.
Another reason why shared_from_this() is useful can be seen when you look at some some of your lambdas, they often have the form:
[this, self,...](...) {...}
That is, you very often want to use this inside them. If you did not capture self as well, you'd need to use other measures to make sure this has not been destroyed when the handler is invoked.
I'm trying to bind a function to boost::asio::async_write, but I got a semantic error in write.hpp
class Client{
public:
Client(const int &frame_,char* buf,const int& size_){
frame=frame_;
b=buf;
size=size_;
}
void doNothing(){
//?
}
void handle(const boost::system::error_code& error,std::size_t bytes_transferred ){
//Should handle the socket being closed properly and the data in the buffer being released
cout<<"Bytes sent: "<<bytes_transferred<<endl;
cout<<error<<endl;
}
void runClient()
{
try
{
tcp::resolver resolver(io_service);
tcp::resolver::query query(tcp::v4(), host, port);
tcp::resolver::iterator iterator = resolver.resolve(query);
s=new boost::asio::ip::tcp::socket(io_service);
boost::asio::connect(*s, iterator);
std::cout << "Sending png: frame"<<frame<<" Size: "<<size<<"Bytes ... "<<endl;
int number_to_send = size; // Put your value
int converted_number = htonl(number_to_send);
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),&Client::doNothing);
boost::asio::async_write(*s, boost::asio::buffer(b, size),
boost::bind(&Client::handle,
this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
std::cout << "Done!"<<endl;
}
catch (std::exception& e)
{
std::cerr << "Exception: " << e.what() << "\n";
}
}
private:
enum { max_length = 1024 };
string port="1112";
string host="157.193.215.48";
char * b;
int size;
int frame;
boost::asio::io_service io_service;
boost::asio::ip::tcp::socket* s;
};
This code has no syntax errors, but it does give this error in write.hpp when compiling:
/usr/local/include/boost/asio/impl/write.hpp:615:3:
Called object type 'void (Client::*)()' is not a function or function pointer.
Affected code:
template <typename AsyncWriteStream, typename ConstBufferSequence,
typename WriteHandler>
inline BOOST_ASIO_INITFN_RESULT_TYPE(WriteHandler,
void (boost::system::error_code, std::size_t))
async_write(AsyncWriteStream& s, const ConstBufferSequence& buffers,
BOOST_ASIO_MOVE_ARG(WriteHandler) handler)
{
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a WriteHandler.
BOOST_ASIO_WRITE_HANDLER_CHECK(WriteHandler, handler) type_check;
detail::async_result_init<
WriteHandler, void (boost::system::error_code, std::size_t)> init(
BOOST_ASIO_MOVE_CAST(WriteHandler)(handler));
detail::write_op<AsyncWriteStream, ConstBufferSequence,
detail::transfer_all_t, BOOST_ASIO_HANDLER_TYPE(
WriteHandler, void (boost::system::error_code, std::size_t))>(
s, buffers, transfer_all(), init.handler)(
boost::system::error_code(), 0, 1);
return init.result.get();
}
Affected line:
// If you get an error on the following line it means that your handler does
// not meet the documented type requirements for a WriteHandler.
BOOST_ASIO_WRITE_HANDLER_CHECK(WriteHandler, handler) type_check;
It's obvious that there's something wrong with my handlers, both of them. I just can't find out what and I'm getting really annoyed by it.
in your first async write, you send a pointer to a non static member function without having binded it to its this pointer like you did in the second write.
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),&Client::doNothing);
should be
boost::asio::async_write(*s,boost::asio::buffer(&converted_number, sizeof(converted_number)),boost::bind(&Client::doNothing,this));
I'm having difficulties figuring out correct way of using boost serialization/asio to send objects over network. The message class is as simple as possible. It's not C++ friendly nor suitable for my needs, I just keep it simple temporarily to test asio/ser:
class message {
friend class boost::serialization::access;
public:
message(){}
int type;
int sender;
int assignment;
int nogood;
template<class Archive>
void serialize(Archive & ar, const unsigned int version)
{
ar & type;
ar & sender;
ar & assignment;
ar & nogood;
}
};
On the client side when agent decides to send a message, sends it to server over it's tcp connection:
message m;
// do something to generate message
boost::asio::streambuf bufx;
std::ostream os( &bufx );
boost::archive::binary_oarchive ar( os );
ar & m;
boost::asio::write( socket, bufx);
Server side code:
boost::asio::streambuf bufx;
std::istream is(&bufx);
boost::archive::binary_iarchive ia(is); // <--- Exception: invalid signature
size_t rcx = asio::read(socket,bufx);
message m;
ia >> m;
I got same exception.
And This official example helps me.
If you are still in trouble, try this.
size_t n = sock.receive(bufs);
// received data is "committed" from output sequence to input sequence
b.commit(n);
std::istream is(&b);
std::string s;
is >> s;
In my case, I use async_read. Actually, I modified the example.
boost::asio::streambuf inbound_;
boost::asio::streambuf::mutable_buffers_type bufs = inbound_.prepare(inbound_data_size);
void (connection::*f)(
const boost::system::error_code&, std::size_t,
T&, boost::tuple<Handler>)
= &connection::handle_read_data<T, Handler>;
boost::asio::async_read(socket_, boost::asio::buffer(bufs),
boost::bind(f, this,
boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred, boost::ref(t), handler));
And then at the handler
/// Handle a completed read of message data.
template <typename T, typename Handler>
void handle_read_data(const boost::system::error_code& e, std::size_t bytes_transferred,
T& t, boost::tuple<Handler> handler)
{
if (e)
{
boost::get<0>(handler)(e);
}
else
{
// Extract the data structure from the data just received.
try
{
inbound_.commit(bytes_transferred);
std::istream archive_stream(&inbound_);
boost::archive::binary_iarchive archive(archive_stream);
archive >> t;
}
catch (std::exception& err)
{
// Unable to decode data.
boost::system::error_code error(boost::asio::error::invalid_argument);
boost::get<0>(handler)(error);
return;
}
// Inform caller that data has been received ok.
boost::get<0>(handler)(e);
}
}
In your server-side code, your streambuf is empty when you create the binary archive. If the archive constructor is looking for a magic number at the beginning of the archive then it won't find it. Try filling the streambuf with the call to boost::asio::read() before constructing the stream and archive.
This is a boost::asio udp echo demo based on a boost asio example.
The meat of this version using C++ lambda is less than half the size of the boost example one, but gcc tells me that received is not visible in recv_from.
It pains me to have to write this in a more verbose manner. Can some C++ guru help me with a trick to define mutually recursive lambdas?
class server {
public:
server(io_service& io_service, short port)
: socket_(io_service, udp::endpoint(udp::v4(), port)) {
auto recv_from = [&,received]() {
socket_.async_receive_from(buffer(data_, max_length), sender_endpoint_,
received);
};
auto received = [&,recv_from](const error_code& error, size_t bytes_transferred) {
if (!error && bytes_transferred > 0) {
socket_.async_send_to(buffer(data_, bytes_transferred), sender_endpoint_,
[&](const error_code&, size_t) {
recv_from();
});
} else {
recv_from(); // loop
}
};
recv_from();
}
private:
udp::socket socket_;
udp::endpoint sender_endpoint_;
enum { max_length = 1024 };
char data_[max_length];
};
Edit, solution: I needed to add this:
std::function<void(const error_code&, size_t)> received;
to make it easy for the type-inference engine (I'm spoiled having programmed Haskell)
Edit2: There are lifetime issues so that won't work.
Answering my own question:
There are actually no less than three problems with my code.
I have been careful to copy the received and recv_from into the corresponding closures so that they would be available when the constructor goes out of scope.
Unfortunately, the closures go out of scope at the same time as the constructor. Thus the [&, xxx] copying of xxx makes no sense.
The type of at least(?) one of the lambdas must be fixed to please the type inference engine.
But that doesn't solve issue #1. To fix the lifetime issue, I should have stored the closure objects in the server object.
So I think this is close to what I need to do:
class server {
public:
server(io_service& io_service, short port)
: socket_(io_service, udp::endpoint(udp::v4(), port)) {
recv_from = [&]() {
socket_.async_receive_from(buffer(data_, max_length), sender_endpoint_,
received);
};
received = [&](const error_code& error, size_t bytes_transferred) {
if (!error && bytes_transferred > 0) {
socket_.async_send_to(buffer(data_, bytes_transferred), sender_endpoint_,
[&](const error_code&, size_t) {
recv_from();
});
} else {
recv_from(); // loop
}
};
recv_from();
}
private:
udp::socket socket_;
udp::endpoint sender_endpoint_;
std::function<void(const error_code&, size_t)> received;
std::function<void()> recv_from;
enum { max_length = 1024 };
char data_[max_length];
};