boost::asio problem passing dynamically sized data to async handler - c++

I am processing custom tcp data packet with boost. Since all operations are asynchronously a handler must be called to process the data. The main problem is that I don't know how to pass the data to the handler when the size is not known at compiletime?
For example, say you receive the header bytes, parse them which tells you the length of the body:
int length = header.body_size();
I somehow need to allocate an array with the size of the body and then call the handler (which is a class member, not a static function) and pass the data to it. How do I do that properly?
I tried different things such as but always ended up getting a segfault or I had to provide a fixed size for the body buffer which is not what I want. An attempt I made can be found below.
After receiving the header information:
char data[header.body_size()];
boost::asio::async_read(_socket, boost::asio::buffer(data, header.body_size()),
boost::bind(&TCPClient::handle_read_body, this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred, data));
The handler:
void TCPClient::handle_read_body(const boost::system::error_code &error, std::size_t bytes_transferred,
const char *buffer) {
Logger::log_info("Reading body. Body size: " + std::to_string(bytes_transferred));
}
This example throws a segfault.
How can I allocate a buffer for the body after knowing the size?
And how can I then call the handler and passing over the error_code, the bytes_transferred and the body data?
An example snippet would be really appreciated since the boost-chat examples that do this are not very clear to me.

char data[header.body_size()]; is not standard in C++ and will become invalid once it goes out of scope while async_read requires buffer to remain alive until completion callback is invoked. So you should probably add a field to TCPClient holding a list of data buffers (probably of std::vector kind) pending to be received.

All you need to do is to create buffer onto heap instead of stack. In place of VLA - char [sizeAtRuntime] you can use std::string or std::vector with std::shared_ptr. By using string/vector you can set buffer to have any size and by using shared_ptr you can prolong lifetime of your buffer.
Version with bind:
void foo()
{
std::shared_ptr<std::vector<char>> buf = std::make_shared<std::vector<char>>(); // buf is local
buf->resize( header.body_size() );
// ditto with std::string
boost::asio::async_read(_socket, boost::asio::buffer(*buf),
boost::bind(&TCPClient::handle_read_body,
this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
buf)); // buf is passed by value
}
void handle_read_body(const boost::system::error_code&,
size_t,
std::shared_ptr<std::vector<char>>)
{
}
in above example buf is created onto stack and points to vector onto heap, because bind takes its arguments by value, so buf is copied and reference counter is increased - it means your buffer still exists when async_read ends and foo ends.
You can achive the same behaviour with lambda, then buf should be captured by value:
void foo()
{
std::shared_ptr<std::vector<char>> buf = std::make_shared<std::vector<char>>(); // buf is local
buf->resize( header.body_size() );
// ditto with std::string
boost::asio::async_read(_socket, boost::asio::buffer(*buf),
boost::bind(&TCPClient::handle_read_body, this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred, buf)); // buf is passed by value
boost::asio::async_read(_socket, boost::asio::buffer(*buf),
[buf](const boost::system::error_code& , size_t)
^^^ capture buf by value, increates reference counter of shared_ptr
{
});
}

Related

boost::asio::async_read_until Don´t truncate the input message

i am trying to truncate a input message when the program read a specific character, for this i have the next code:
This is the ActiveSocketClientConnection.h
class ActiveSocketClientConnection : public boost::enable_shared_from_this<ActiveSocketClientConnection>{
private:
boost::shared_ptr<tcp::socket> socket_;
boost::asio::streambuf data_;
...
public:
...
}
This is the ActiveSocketClientConnection.cpp
void ActiveSocketClientConnection::handleConnect(const boost::system::error_code& error){
std::string sETX;
sETX.push_back(0x3A); //0x3A = :
boost::asio::async_read_until(
*socket_.get(),
data_,
sETX.c_str(),
boost::bind(&ActiveSocketClientConnection::handleReadBody,
this,
boost::asio::placeholders::error
)
);
}
void ActiveSocketClientConnection::handleReadBody( boost::system::error_code error){
size_t t = data_.size();
unsigned char* output = (unsigned char*)malloc(t);
memcpy(output, boost::asio::buffer_cast<const void*>(data_.data()), t);
data_.consume(t);
...
}
If i pass the message (for example) AA:A with a socket connection. The function async_read_until save all the message in data_, don´t truncate the message where the caracter : is present.
Someone could say me what i am doing wrong?
Thank you.
First, you have undefined behaviour. You call async_read_until with std::string_view as a delimiter. But this view is created based on std::string which is local inside your handle function. async_read_until ends immediately, string as local is destroyed and you have dangling pointer inside string view (std::string_view doesn't make deep copy of string, it is just a pair: a pointer to data and its size).
As solution just call overload taking char:
boost::asio::async_read_until(
*socket_.get(),
data_,
0x3A, // <- added
boost::bind(&ActiveSocketClientConnection::handleReadBody,
this,
boost::asio::placeholders::error
)
);
Official boost reference states:
After a successful async_read_until operation, the dynamic buffer
sequence may contain additional data beyond the delimiter. An
application will typically leave that data in the dynamic buffer
sequence for a subsequent async_read_until operation to examine.
So you have to parse data looking for first occurence of delimiter and extract proper subbuffer of read data.

use member function of socket class or common function in Boost::asio?

I am learning Boost::asio socket; I saw some examples, where they use the member function of socket class to read and receive messages, or use the boost::asio common function which passes the socket as the first param.
So I am wondering what the difference is between the two approaches? thanks!
//one kind
tcp::socket sock;
sock.async_write_some(***);
//another kind
boost::asio::async_write(socket,***);
async_write as static function guarantees that all data in buffer
is written before this function returns.
async_write_some as function member guarantees that at least one byte
is written from buffer before this function ends.
So if you want to use async_write_some you need to provide more code
to handle the situation when not all data from buffer was written.
Suppose you have string with 10 bytes, it is your buffer and you want to ensure
all buffer is send:
// Pseudocode:
string str;
// add 10 bytes to str
SIZE = 10;
int writtenBytes = 0;
socket.async_write_some (buffer(str.c_str(),SIZE), makeCallback());
void callback (
error_code ec,
size_t transferredBytes)
{
// check errors
// [1]
writtenBytes += transferredBytes;
if (writtenBytes == SIZE)
return;
// call async_write_some
socket.async_write_some (buffer(str.c_str()+writtenBytes,SIZE-writtenBytes),makeCallback());
}
in callback [1] you need to check how many bytes were written,
if result is different from SIZE you need to call async_write_some again
to send the remainder of data and so on, your callback may be invoked many times.
The use of async_write is simpler:
string str; // add 10 bytes
async_write (buffer(str.c_str(),SIZE),makeCallback());
void callback() {
// check errors
[1]
}
if no errors occured in [1] you know that all data was sent.

Will my buffer used by all connections?

I have a main which creates an io_service and passes them to an instance of TcpServer.
TcpServer has a member std::array<char, 8192> m_buffer. It has 4 methods: the constructor, startAccept, handleAccept and handleRead.
The constructor only initializes some members and calls startAccept.
startAccept creates a shared pointer of TcpConnection which extends std::enable_shared_from_this<TcpConnection. After that start accept calls m_acceptor.async_accept and binds the accept to the handleAccept method mentioned before.
And this is my handleAccept method. It calls async_read_some with the boost::asio::buffer which uses the member variable declared in TcpServer.
void TcpServer::handleAccept(std::shared_ptr<TcpConnection> newConnection, const boost::system::error_code &error)
{
if (!error) {
//newConnection->start();
std::cout << "Accepting new connection" << std::endl;
newConnection->getSocket().async_read_some(
boost::asio::buffer(m_buffer),
boost::bind(&TcpServer::handleRead, this, newConnection, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)
);
}
startAccept();
}
I am not sure, but if there are multiple connections, all of them will use the same buffer object, right? And they will probably overwrite it, won't they?
Yes, all connections will use same buffer, that is defined in TcpServer. You actually should store buffer in connection, rather than in server.
boost::asio::buffer will use that overload. So, data from read will be stored to your m_buffer. You should store your buffer in connection, or use some synchronization (i.e. some boolean flag, like is_in_read, but that is bad idea).

How to stop boost::asio async reads from getting mixed up?

I am using boost::asio::ip::tcp::socket to receive data. I need an interface which allows me to specify a buffer and call a completion handler once this buffer is filled asynchronously.
When reading from sockets, we can use the async_read_some method.
However, the async_read_some method may read less than the requested number of bytes, so it must call itself with the rest of the buffer if this happens. Here is my current approach:
template<typename CompletionHandler>
void read(boost::asio::ip::tcp::socket* sock, char* target, size_t size, CompletionHandler completionHandler){
struct ReadHandler {
boost::asio::ip::tcp::socket* sock;
char* target;
size_t size;
CompletionHandler completionHandler;
ReadHandler(ip::tcp::socket* sock, char* target, size_t size, CompletionHandler completionHandler)
: sock(sock),target(target),size(size),completionHandler(completionHandler){}
// either request the remaining bytes or call the completion handler
void operator()(const boost::system::error_code& error, std::size_t bytes_transferred){
if(error){
return;
}
if(bytes_transferred < size){
// Read incomplete
char* newTarg =target+bytes_transferred;
size_t newSize = size-bytes_transferred;
sock->async_read_some(boost::asio::buffer(newTarg, newSize), ReadHandler(sock,newTarg,newSize,completionHandler));
return;
} else {
// Read complete, call handler
completionHandler();
}
}
};
// start first read
sock->async_read_some(boost::asio::buffer(target, size), ReadHandler(this,target,size,completionHandler));
}
So basically, we call async_read_some until the whole buffer is filled, then we call the completion handler. So far so good. However, I think that things get mixed up once I call this method more than once before the first call finishes a receive:
void thisMayFail(boost::asio::ip::tcp::socket* sock){
char* buffer1 = new char[128];
char* buffer2 = new char[128];
read(sock, buffer1, 128,[](){std::cout << "Buffer 1 filled";});
read(sock, buffer2, 128,[](){std::cout << "Buffer 2 filled";});
}
of course, the first 128 received bytes should go into the first buffer and the second 128 should go into the second. But in my understanding, it may be the case that this does not happen here:
Suppose the first async_read_some returns only 70 bytes, then it would issue a second async_read_some with the remaining 58 bytes. However, this read will be queued behind the second 128 byte read(!), so the first buffer will receive the first 70 bytes, the next 128 will go into the second buffer and the final 50 go into the first. I.e., in this case the second buffer would even be filled before the first is filled completely. This may not happen.
How to solve this? I know there is the async_read method, but its documentation says it is simply implemented by calling async_read_some multiple times, so it is basically the same as my read implementation and will not fix the problem.
You simply can't have two asynchronous read operations active at the same time: that's undefined behaviour.
You can
use the free function async_read_until or async_read functions, which already have the higher-level semantics and loop callling the socket's async_read_some until a condition is matched or the buffer is full.
use asynchronous operation chaining to sequence the next async read after the first. In short, you initiate the second boost::asio::async_read* call in the completion handler of the first.
Note:
Gives you the opportunity to act on transport errors first too.
together the free function interface will both raise the abstraction level of the code and solve the problem (the problem was initiating two simultaneous read operations)
use a strand in case you run multiple IO service threads; See Why do I need strand per connection when using boost::asio?

C++ Linux Google Protobuf + boost::asio Cannot Parse

I am trying to send a Google Protobuf message over a boost::asio socket via TCP. I recognize that TCP is a streaming protocol and thus I am performing length-prefixing on the messages before they go through the socket. I have the code working, but it only appears to work some of the time, even though I'm repeating the same calls and not changing the environment. On occasion I will receive the following error:
[libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "xxx" because it is missing required fields: Name, ApplicationType, MessageType
The reason is easy to understand, but I cannot single out why this only occurs sometimes and parses just fine the majority of the time. It is very easy to duplicate the error by just having a single client talking to the server and simply restarting the processes.
Below are the socket code snippets.
const int TCP_HEADER_SIZE = 8;
Sender:
bool Write(const google::protobuf::MessageLite& proto) {
char header[TCP_HEADER_SIZE];
int size = proto.ByteSize();
char data[TCP_HEADER_SIZE + size];
sprintf(data, "%i", size);
proto.SerializeToArray(data+TCP_HEADER_SIZE, size);
boost::asio::async_write(Socket,
boost::asio::buffer(data, TCP_HEADER_SIZE + size),
boost::bind(&TCPSender::WriteHandler,
this, _1, _2));
}
Receiver:
std::array<char, TCP_HEADER_SIZE> Header;
std::array<char, 8192> Bytes;
void ReadHandler(const boost::system::error_code &ec,
std::size_t bytes_transferred) {
if(!ec) {
int msgsize = atoi(Header.data());
if(msgsize > 0) {
boost::asio::read(Socket, boost::asio::buffer(Bytes,static_cast<std::size_t>(msgsize)));
ReadFunc(Bytes.data(), msgsize);
}
boost::asio::async_read(Socket, boost::asio::buffer(Header, TCP_HEADER_SIZE),
boost::bind(&TCPReceiver::ReadHandler, this, _1, _2));
}
else {
std::cerr << "Server::ReadHandler::" << ec.message() << '\n';
}
}
ReadFunc:
void HandleIncomingData(const char *data, const std::size_t size) {
xxx::messaging::CMSMessage proto;
proto.ParseFromArray(data, static_cast<int>(size));
}
I should mention that I need this to be as fast as possible, so any optimizations would be very much appreciated as well.
The program invokes undefined behavior as it fails to meet a lifetime requirement for boost::asio::async_write()'s buffers parameter:
[...] ownership of the underlying memory blocks is retained by the caller, which must guarantee that they remain valid until the handler is called.
Within the Write() function, boost::asio::async_write() will return immediately, and potentially cause data to go out of scope before the asynchronous write operation has completed. To resolve this, consider expanding the life of the underlying buffer, such as by associating the buffer with the operation and performing cleanup in the handler, or making the buffer a data member on TCPSender.