boost asio async_read() seems to be skipping some nulls - c++

Question

I'm going a bit crazy with a simple boost asio TCP conversation.
I have a server and a client. I use length-prefixed messges. The client sends "one" and the server responds with "two". So this is what I see happen:
The client sends, and the server receives, 00 00 00 03 6F 6E 65 (== 0x0003 one).
The server responds by sending 00 00 00 03 74 77 6F (== 0x0003 two).
Now here is where it is very strange (code below). If the client reads four bytes, I expect it to get 00 00 00 03. If it reads seven, I expect to see 00 00 00 03 74 77 6F. (In fact, it will read four (the length header), then three (the body).)
But what I actually see is that, while if I read seven at once I do see 00 00 00 03 74 77 6F, if I only ask for four, I see 74 77 6F 03. This doesn't make any sense to me.
Here is the code I'm using to receive it (minus some print statements and such):
const int kTcpHeaderSize = 4;
const int kTcpMessageSize = 2048;
std::array<char, kTcpMessageSize + kTcpHeaderSize> receive_buffer_;
void TcpConnection::ReceiveHeader() {
boost::asio::async_read(
socket_, boost::asio::buffer(receive_buffer_, kTcpHeaderSize),
[this](boost::system::error_code error_code,
std::size_t received_length) {
if (error_code) {
LOG_WARNING << "Header read error: " << error_code;
socket_.close(); // TODO: Recover better.
return;
}
if (received_length != kTcpHeaderSize) {
LOG_ERROR << "Header length " << received_length
<< " != " << kTcpHeaderSize;
socket_.close(); // TODO: Recover better.
return;
}
uint32_t read_length_network;
memcpy(&read_length_network, receive_buffer_.data(),
kTcpHeaderSize);
uint32_t read_length = ntohl(read_length_network);
// Error: read_length is in the billions.
ReceiveBody(read_length);
});
}
Note that kTcpHeaderSize is 4. If I change it to 7 (which makes no sense, but just for the experiment) I see the stream of 7 bytes I expect. When it is 4, I see a stream that is not the first four bytes of what I expect.
Any pointers what I am doing wrong?

Answer 1

From what I can see in your code it should work according to the async_read documentation:
The asynchronous operation will continue until one of the following conditions is true:
The supplied buffers are full. That is, the bytes transferred is equal to the sum of the buffer sizes.
An error occurred.
However see the remark at the bottom:
This overload is equivalent to calling:
boost::asio::async_read(
s, buffers,
boost::asio::transfer_all(),
handler);
It looks like the transfer_all condition might be the only thing checked.
Try using the transfer_exactly condition and if it does work report an issue on https://github.com/boostorg/asio/issues.

Answer 2

The suggestion by #sergiopm to use transfer_all was good, and I'm pretty sure it helped. The other issue involved buffer lifetimes in the asynchronous send/receive functions. I got a bit confused, apparently, about how long certain things would live and how long I needed them to live, and so I was overwriting things from time to time. That may have been more important than transfer_all, but I'm still happy to give #sergiopm credit for helping getting me on my way.
The intent has just been to have a simple tcp client or server that I can declare, hand it a callback, and then go on my way knowing that I can only pay attention to those callbacks.
I'm pretty sure something like this must exist (thousands of times over). Do feel free to comment below, both for me and for those who come after, if you think there are better libraries than asio for this task (i.e., that would involve substantially less code on my part). The principle constraint is that, due to multiple languages and services, we need to own the wire protocol. Otherwise we get into things like "does library X have a module for language Y?".
As an aside, it's interesting to me that essentially every example I've found does length-prefix encoding rather than beginning/end of packet encoding. Length prefix is really easy to implement but, unless I'm quite mistaken, suffers from re-sync hell: if a stream is interrupted ("I'm going to send you 100 bytes, here are the first 50 but then I died") it's not clear to me that there aren't scenarios where I'm unable to resync properly.
Anyway, I learned a lot along the way, I recommend the exercise.