I'm developing a client-server app, both sides of which use boost::asio.
I'm trying to send a large package of data over TCP (356 kb)
On server side, I write like:
boost::asio::async_write(Msocket,
boost::asio::buffer(sendBuffer,dataLen),
boost::bind(&ServerSession::onDataWrite,
this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_trasferred));
The onDataWrite is simple:
void ServerSession::onDataWrite(const boost::system::error_code& error, const std::size_t bytesSent) {
if (error) {
std::cout << "Error " << error << " while sending data" << std::endl;
}
}
On client side:
int readSize = ...; // defined from msg header, in this case equals 300 kbytes.
boost::asio::async_read(*Msocket,
boost::asio::buffer(recvBuffer, 50*1024*1024),
boost::asio::transfer_exactly(readSize),
boost::bind(&ClientSession::onDataRead,
this, boost::asio::placeholders::error,
boost::asio::placeholders::bytes_trasferred,
readSize));
And onDataRead is:
void ClientSession::onDataRead(const boost::system::error_code& error, const std::size_t bytesRecvd, const int readSize) {
if (error || bytesRecvd != readSize) {
std::cout << "Error " << error << " while getting data, expect " << readSize <<", got " << bytesRecvd << std::endl;
}
}
During write, server side prints
Error system:10014 while sending data
And client prints
Error system:0 while getting data, expect 393216, got 131064
While 131064 = 128kb - 8 bytes of header.
It looks like this 128-kb issue is caused by send/receive buffer overflow. But I though Boost will take care about those buffers itself, transparently for me.
What do I misunderstand?
Related
i am trying to generate a class for reading from a specific serial device.
For the start process it is necessary to send a char '1', then i have to wait for a response (254 and 255).
Within a period of 10 milliseconds i must sent the next command to the device, but this time the command length is 5 char.
When the communication hasn´t been send in the correct time, the device will run into a timeout and is sending me 255,255,255,2,4.
So i need different sizes of reading and the most importing thing for me is a timeout for the communication, cause otherwise the system will stop working by missing some values.
Therefore i have tried to generate a class using boost::asio::async_read.
It is working in the correct way, i can define the timeout,also the size of bytes to be read. When the device isn´t sending the correct size, the routine is going to be left.
But only the first time, when i try it a second time, the device isn´t sending me something. I have tried to use .open again, but it isn´t solving the issue. Also deactivating the close-function isn´t solving the issue, then the routine is running into an error.
Can someone give me a small tip for my issue. Maybe i am to blind to see my problem.... Bernd
ConnectionWithTimeout::ConnectionWithTimeout(int timeout_)
: timer_(io_service_, boost::posix_time::milliseconds(timeout_))
, serial_port_(io_service_) {
}
void ConnectionWithTimeout::ReadNumberOfChars(int numberOfCharactersToRead_)
{
buffer_.resize(numberOfCharactersToRead_);
for (int i = 0; i < numberOfCharactersToRead_; ++i) {
std::cout << "Clear Buffer[" << i << "]" << std::endl;
buffer_[i] = 0;
}
timer_.async_wait(boost::bind(&::ConnectionWithTimeout::Stop, this));
//async read from serial port
boost::asio::async_read(serial_port_, boost::asio::buffer(buffer_),
boost::bind(&ConnectionWithTimeout::ReadHandle, this,
boost::asio::placeholders::error));
io_service_.run();
}
void ConnectionWithTimeout::Stop() {
std::cout << "Connection is being closed." << std::endl;
serial_port_.close();
std::cout << "Connection has been closed." << std::endl;
}
void ConnectionWithTimeout::ReadHandle(const boost::system::error_code& ec) {
if (ec) {
std::cout << "The amount of data is to low: " << ec << std::endl;
for (std::vector<char>::iterator it = buffer_.begin();
it != buffer_.end(); ++it)
{
std::cout << int(*it) << std::endl;
}
}
else {
std::cout << "The amount of data is correct: " << ec << std::endl;
for (std::vector<char>::iterator it = buffer_.begin(); it !=
buffer_.end(); ++it)
{
std::cout << int(*it) << std::endl;
}
}
}
I have this requirement where my app have to connect to another app via sockets and will have to maintain persistent connection for quiet long time. My app will be a TCP client and the other is a TCP server. My app will send commands and the server will respond accordingly.
The problem am facing right now is how to read the whole data from server a string and return for app which will issue the next command. Reading synchronously (with asio::read) looked like a good option up until I observed socket hanging up until I terminate the server. Looking at the documentation I found that the library is correctly working.
his function is used to read a certain number of bytes of data from a stream. The call will block until one of the following conditions is true:
1. The supplied buffers are full. That is, the bytes transferred is equal to the sum of the buffer sizes.
2. An error occurred.
The problem is I don't know correct buffer size as the response from the server varies. So If I put a too small buffer it returns fine but missing some data. If I put too big it will hang forever until server quits.
So I thought I would do the async reading. It works only once and I don't know how to make it fetch data until whole data it read.
here is the relevant async code
#define ASIO_STANDALONE 1
#include <iostream>
#include <asio.hpp>
int main()
{
asio::io_context context;
size_t reply_length;
size_t length = 1024;
std::vector<char> buffer;
//create socket
asio::ip::tcp::socket socket(context);
socket.connect(asio::ip::tcp::endpoint(asio::ip::address::from_string("127.0.0.1"), 8088));
std::string dataOut = "list --files"; //some command to write
std::error_code error;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
buffer.resize(length);
asio::async_read(socket, asio::buffer(buffer), [&buffer, &context](const asio::error_code &ec, std::size_t bytes_transferred) {
std::copy(buffer.begin(), buffer.end(), std::ostream_iterator<char>(std::cout, ""));
std::cout << "\nRead total of:" << bytes_transferred << "\n";
context.run();
});
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
context.run();
}
Searching didn't help much solving my issue.
So my question is, how can I read all the data in a persistent socket with asio? Am not using boost.
You need to loop async_read calls. If you don't want your client to hang on read operation you can define the smallest possible buffer i.e. 1 byte.
Define function which takes socket, buffer and two additional parameters according to async_read's handler signature, and this function calls itself with async_read to make the loop of async_read calls - it reads until some error occures:
void onRead (
asio::ip::tcp::socket& socket,
std::array<char,1>& buf,
const system::error_code& ec,
std::size_t bytes)
{
if (ec)
{
if (ec == asio::error::eof && bytes == 1)
std::cout << buf[0];
return;
}
std::cout << buf[0];
asio::async_read(socket,asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1, // error code
std::placeholders::_2)); // transferred bytes
}
and the changes in main:
std::array<char,1> buf;
asio::write(socket, asio::buffer(dataOut), error);
if (!error)
{
std::cout << "Receiving...!" << std::endl;
asio::async_read(socket, asio::buffer(buf),
std::bind(onRead, std::ref(socket), std::ref(buf),
std::placeholders::_1,
std::placeholders::_2));
context.run();
}
else
{
std::cout << "send failed: " << error.message() << std::endl;
}
(I am using Boost, so you should replace system::error_code on asio::error_code).
I am learning about boost and was messing around with its server and client communication to make a simple chat server, where anything that a client sends, is just displayed on the server. The server itself doesn't send anything and starts the receiving part. It is pretty straight-forward.
Server code:
#include <boost\asio\placeholders.hpp>
#include <boost\bind.hpp>
#include <boost\asio\ip\tcp.hpp>
#include <boost\asio\io_context.hpp>
#include <iostream>
class Server
{
private :
boost::asio::ip::tcp::socket server_socket;
boost::asio::ip::tcp::endpoint server_endpoint;
boost::asio::ip::tcp::acceptor acceptor;
std::string msg;
public :
Server(boost::asio::io_context &io) :
server_socket(io),
server_endpoint(boost::asio::ip::make_address("127.0.0.1"), 27015),
acceptor(io, server_endpoint)
{
acceptor.async_accept(server_socket,
boost::bind(&Server::async_acceptor_handler, this,
boost::asio::placeholders::error));
}
void async_acceptor_handler(const boost::system::error_code &ec)
{
if (!ec)
{
std::cout << "One client connected...\n";
server_socket.async_read_some(boost::asio::buffer(msg),
boost::bind(&Server::async_read_some_handler, this,
boost::asio::placeholders::error));
}
else
{
std::cout << "async_acceptor failed with error code : " << ec.value() << std::endl;
std::cout << "Error description : " << ec.message() << std::endl;
}
}
void async_read_some_handler(const boost::system::error_code &ec)
{
if (!ec)
{
std::cout << msg << std::endl;
server_socket.async_read_some(boost::asio::buffer(msg),
boost::bind(&Server::async_read_some_handler, this,
boost::asio::placeholders::error));
}
else
{
std::cout << "async_acceptor failed with error code : " << ec.value() << std::endl;
std::cout << "Error description : " << ec.message() << std::endl;
}
}
};
int main()
{
boost::asio::io_context io;
Server s(io);
io.run();
return 0;
}
In the client part, it is again a pretty straight-forward code, simply connects to the server and starts taking input from user and sends to server.
Client code:
#include <boost\asio\placeholders.hpp>
#include <boost\bind.hpp>
#include <boost\asio\ip\tcp.hpp>
#include <boost\asio\io_context.hpp>
#include <iostream>
class Client
{
private :
boost::asio::ip::tcp::socket client_socket;
boost::asio::ip::tcp::endpoint server_endpoint;
std::string msg;
public :
Client(boost::asio::io_context &iocontext) :
client_socket(iocontext),
server_endpoint(boost::asio::ip::make_address("127.0.0.1"), 27015)
{
//connect to server endpoint
client_socket.async_connect(server_endpoint,
boost::bind(&Client::async_connect_handler, this,
boost::asio::placeholders::error));
}
void async_connect_handler(const boost::system::error_code &ec)
{
if (!ec)
{
std::cout << "Connected to chat server...\n";
//wait for user input
std::cin >> msg;
std::cout << "\rC : " << msg << std::endl;
client_socket.async_write_some(boost::asio::buffer(msg),
boost::bind(&Client::async_write_some_handler, this,
boost::asio::placeholders::error));
}
else
{
std::cout << "async_connect failed with error code : " << ec.value() << std::endl;
std::cout << "Error description : " << ec.message() << std::endl;
}
}
void async_write_some_handler(const boost::system::error_code &ec)
{
//wait for user input
std::cin >> msg;
std::cout << "\rC : " << msg << std::endl;
client_socket.async_write_some(boost::asio::buffer(msg),
boost::bind(&Client::async_write_some_handler, this,
boost::asio::placeholders::error));
}
};
int main()
{
boost::asio::io_context io;
Client c(io);
io.run();
return 0;
}
Now the problem:
It works fine, and connects to the server too. I get the proper "Connected to chat server..." in client and "One client connected..." in server. The problem arises after that :
In the server console, after the "One client" message, it just starts printing nothing and goes on and on.
The messages sent by the client are never showed in the server console.
Problem 1 can be a issue on my part as I am yet to check the wait functions and other calls which make the server wait. If you can guide me on that, it will be more than amazing. But the major problem is the part 2 of the problem, since, I have no idea why the server is always receiving nothing from client.
PS: This is an incomplete code and I plan to play a bit more with it, so, if there are some major flaws, please tell me so... :)
PPS: Before you say check other questions similar to this, I went through all the similar questions. For ex: this and this, but this are not relevant.
What is the size of string msg in the server side? It is 0. So the server reads always 0 bytes.
When you want to read to string and you call buffer::asio::buffer string must have some size, for example 10. It means you want to read 10 bytes into msg. You can call msg.resize(10) (before reading operation is initiated), then some data will be read into msg by async_read_some (it could be 1,2 bytes, whatever - it is how async_read_some works, but the maximum read characters is 10). But it is poor solution.
You are sending text, so you may consider using read data into streambuf instead of string, when you don't know how many bytes can come from the client side. Then you can call async_read_until with delimiter - it can be for example new line character.
Another solution is to use dynamic buffer. Where data is appened into string and you don't care about the initial size of string buffer. But dynamic buffer doesn't work with member functions of socket like async_read_some, it could be used with async_read as free function.
I would like to create client/server communication programs pair using Boost ASIO + SSL. So I started off with the examples provided by boost, and I learned how that works, and I'm almost ready to develop my communication protocol, except that there's one problem.
So starting from this example, I'm modifying the handle_read() callback function after the handshake. The following is my code. My only modification is: Add another callback function called startComm(), which will start the communication.
void handle_read(const boost::system::error_code& error,
size_t bytes_transferred)
{
if (!error)
{
std::cout << "Reply: ";
std::cout.write(reply_, bytes_transferred);
std::cout << "\n";
boost::asio::async_write(socket_,
boost::asio::buffer(std::string("Now?")),
boost::bind(&SSLClient::startComm, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
else
{
std::cout << "Read failed: " << error.message() << "\n";
}
}
void startComm(const boost::system::error_code& error,
size_t bytes_transferred)
{
if (!error)
{
std::cout << "Reply: ";
std::cout.write(reply_, bytes_transferred); //problem here, bytes transferred should contain the number of received chars not number of written chars
std::cout << "\n";
}
else
{
std::cout << "Read failed: " << error.message() << "\n";
}
}
In the async_write() above, there's an argument boost::asio::placeholders::bytes_transferred which parametrizes my callback function to provide the number of bytes that were sent to the server. Now I would like to know the number of bytes the server responded with. How can I do that in my simple example?
Thanks. If you require any additional details, please ask.
The write call sends data.
Since it doesn't, at all, receive data the number of bytes received is by definition 0.
If you want to receive data, use (async_)read and it will tell you the number of bytes received.
These call backs use the same placeholder (bytes_transferred) but it carries different meaning depending on the direction of the transfer that has been completed.
Here's a solution that technically does what you want: define an extra parameter of startComm and bind it (not with a placeholder).
void handle_read(const boost::system::error_code &error, size_t bytes_received) {
if (!error) {
std::cout << "Reply: ";
std::cout.write(reply_, bytes_received);
std::cout << "\n";
boost::asio::async_write(socket_, boost::asio::buffer(std::string("Now?")),
boost::bind(&SSLClient::startComm,
this,
boost::asio::placeholders::error,
bytes_received,
boost::asio::placeholders::bytes_transferred));
} else {
std::cout << "Read failed: " << error.message() << "\n";
}
}
void startComm(const boost::system::error_code &error, size_t had_received, size_t bytes_sent) {
if (!error) {
std::cout << "Reply: ";
std::cout.write(reply_, had_received);
std::cout << "\n";
} else {
std::cout << "Write failed: " << error.message() << "\n";
}
}
Note that I still think you might mistakenly expect async_write to receive a reply, which (obviously?) isn't the case
I have studied the existing examples:
Sending Protobuf Messages with boost::asio
Reading Protobuf objects using boost::asio::read_async
Google Protocol Buffers: parseDelimitedFrom and writeDelimitedTo for C++
Are there C++ equivalents for the Protocol Buffers delimited I/O functions in Java?
Sending Protobuf Messages with boost::asio
but I still can not figure out how to pass Google Protobuf messages using the Boost::asio API. In particular I have no clear understanding of the following problems:
Interaction between boost::asio::streambuf and google::protobuf::io objects (and the necessity of the applying of the last ones)
Correct implementation of the message streaming (due to the lack of writeDelimitedTo and parseDelimitedFrom methods in C++ API)
Here is my implementation based on boost::asio v. 1.39 ssl_client from examples.
class client
{
public:
client(boost::asio::io_service& io_service, boost::asio::ssl::context& context,
boost::asio::ip::tcp::resolver::iterator endpoint_iterator)
: socket_(io_service, context),
request_stream(&b),
raw_output(&request_stream),
coded_output(&raw_output)
{
...
}
void handle_connect(const boost::system::error_code& error,
boost::asio::ip::tcp::resolver::iterator endpoint_iterator)
{
...
}
//Debugging function
void print_buffers_condition(const char *step)
{
std::cout << "\nBuffer conditions after " << step << std::endl;
std::cout << "boost::asio::streambuf\t\tb: " << b.size() << std::endl;
std::cout << "google::protobuf::io::OstreamOutputStream raw_output: " << raw_output.ByteCount() << std::endl;
std::cout << "google::protobuf::io::CodedOutputStream coded_output: " << coded_output.ByteCount() << std::endl;
std::cout << std::endl;
}
//Sending test message after SSL Handshake
void handle_handshake(const boost::system::error_code& error)
{
std::cout << "-----------------------------SENDING-----------------------------" << std::endl;
print_buffers_condition("handle handshake");
if (!error)
{
SearchRequest msg;
msg.set_query("qwerty");
msg.set_code(12345);
std::cout << "Debugged" << std::endl;
msg.PrintDebugString();
//Writing the length of the message before and serializing
print_buffers_condition("before serialising");
coded_output.WriteVarint32(msg.ByteSize());
if (!msg.SerializeToCodedStream(&coded_output))
{
std::cout << "serailizing error" << std::endl;
}
else
{
std::cout << "serializing success" << std::endl;
}
//Sending
buffers_condition("before async write");
boost::asio::async_write(socket_,
b,
boost::bind(&client::handle_write, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
buffers_condition("after async write");
}
else
{
std::cout << "Handshake failed: " << error << "\n";
}
}
void handle_write(const boost::system::error_code& error,
size_t bytes_transferred)
{
std::cout << " bytes_trransferred: " << bytes_transferred << std::endl;
if (!error)
{
std::cout << "No error" << std::endl;
...
}
else
{
std::cout << "Write failed: " << error << "\n";
}
}
void handle_read(const boost::system::error_code& error,
size_t bytes_transferred)
{
...
}
private:
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket_;
boost::asio::streambuf b;
std::ostream request_stream;
google::protobuf::io::OstreamOutputStream raw_output;
google::protobuf::io::CodedOutputStream coded_output;
};
This code is operational, so after creating the message we fall into the void handle_write(const boost::system::error_code& error, size_t bytes_transferred) function. Printing the bytes_transferred_ value returns 0: server (implemented on the base of examples too) recieves nothing.
The usage of the debugging function void print_buffers_condition(const char *step) hints at loss of message during its transmission through a stack of different buffering objects:
$ ./client 127.0.0.1 5000
-----------------------------SENDING-----------------------------
Buffer conditions after handle handshake
boost::asio::streambuf b: 0
google::protobuf::io::OstreamOutputStream raw_output: 8192
google::protobuf::io::CodedOutputStream coded_output: 0
Debugged:
query: "qwerty"
code: 12345
Buffer conditions after before serialization
boost::asio::streambuf b: 0
google::protobuf::io::OstreamOutputStream raw_output: 8192
google::protobuf::io::CodedOutputStream coded_output: 0
serializing success
Buffer conditions after before async write
boost::asio::streambuf b: 0
google::protobuf::io::OstreamOutputStream raw_output: 8192
google::protobuf::io::CodedOutputStream coded_output: 13
Buffer conditions after after async write
boost::asio::streambuf b: 0
google::protobuf::io::OstreamOutputStream raw_output: 8192
google::protobuf::io::CodedOutputStream coded_output: 13
bytes_trransferred: 0
I have no idea how to do it in a proper way.
OS is RHEL 6.4.
Thank you.
I'm not familiar with asio, but it looks to me like the problem is that you aren't flushing your buffers. The data is stuck in CodedOutputStream and never finds its way into asio.
CodedOutputStream should be allocated on the stack, such that it is destroyed as soon as you're done writing the message. The destructor will flush the buffer. Note that CodedOutputStream is cheap to allocate so there's no performance problem with putting it on the stack (in fact, it's probably better that way).
OstreamOutputStream can similarly be allocated on the stack, but it heap-allocates a buffer which you might want to reuse. If you choose to reuse the same object, make sure to call Flush() to flush the buffer after the CodedOutputStream is destroyed.
Incidentally, OstreamOutputStream is not particularly efficient, because it has to do its own layer of buffering on top of what ostream is already doing. You may want to serialize to a string (str = message.SerializeAsString() or message.SerializeToString(&str)) and then write that directly to the socket (if asio allows this), as it will probably avoid a redundant copy.