I'm pretty new to boost. I needed a cross platform low level C++ network API, so I chose asio. Now, I've successfully connected and written to a socket, but since I'm using the asynchronous read/write, I need a way to keep track of the requests (to have some kind of IDs, if you will). I've looked at the documentation/reference, and I found no way to pass user data to my handler, the only option I can think of is creating a special class that acts as a callback and keeps track of it's id, then pass it to the socket as a callback. Is there a better way? Or is the best way to do it?
The async_xxx functions are templated on the type of the completion handler. The handler does not have to be a plain "callback", and it can be anything that exposes the right operator() signature.
You should thus be able to do something like this:
// Warning: Not tested
struct MyReadHandler
{
MyReadHandler(Whatever ContextInformation) : m_Context(ContextInformation){}
void
operator()(const boost::system::error_code& error, std::size_t bytes_transferred)
{
// Use m_Context
// ...
}
Whatever m_Context;
};
boost::asio::async_read(socket, buffer, MyReadHander(the_context));
Alternatively, you could also have your handler as a plain function and bind it at the call site, as described in the asio tutorial. The example above would then be:
void
HandleRead(
const boost::system::error_code& error,
std::size_t bytes_transferred
Whatever context
)
{
//...
}
boost::asio::async_read(socket, buffer, boost::bind(&HandleRead,
boost::asio::placeholders::error_code,
boost::asio::placeholders::bytes_transferred,
the_context
));
Related
Background
I'm new to using Boost::Asio library and am having trouble getting the behaviour I want. I am trying to implement some network communication for custom hardware solution. The communication protocol stack we are using relies heavily on Boost::Asio async methods and I don't believe it is entirely thread safe.
I have successfully implemented sending but encountered a problem when trying to setup the await for receiving. Most boost::asio examples I have found rely on socket behaviour to implement async await with socket_.async_read_some() or other similar functions. However this doesn't work for us as our hardware solution requires calling driver function directly rather than utilising sockets.
The application uses an io_service that is passed into boost::asio::generic::raw_protocol::socket as well as other classes.
Example code from protocol stack using sockets
This is the example code from the protocol stack. do_receive() is called in the constructor of RawSocketLink.
void RawSocketLink::do_receive()
{
namespace sph = std::placeholders;
socket_.async_receive_from(
boost::asio::buffer(receive_buffer_), receive_endpoint_,
std::bind(&RawSocketLink::on_read, this, sph::_1, sph::_2));
}
void RawSocketLink::on_read(const boost::system::error_code& ec, std::size_t read_bytes)
{
if (!ec) {
// Do something with received data...
do_receive();
}
}
Our previous receive code without the protocol stack
Prior to implementing the stack we had been using the threading library to create separate threads for send and recieve. The receive method is shown below. Mostly it relies on calling the receive_data() function from the hardware drivers and waiting for it to return. This is a blocking call but is required to return data.
void NetworkAdapter::Receive() {
uint8_t temp_rx_buffer[2048];
rc_t rc;
socket_t *socket_ptr;
receive_params_t rx_params;
size_t rx_buffer_size;
char str[100];
socket_ptr = network_if[0];
while (1) {
rx_buffer_size = sizeof(temp_rx_buffer);
// Wait until receive_data returns then process
rc = receive_data(socket_ptr,
temp_rx_buffer,
&rx_buffer_size,
&rx_params,
WAIT_FOREVER);
if (rc_error(rc)) {
(void)fprintf(stderr, "Receive failed");
continue;
}
// Do something with received packet ....
}
return;
}
Note that the socket_t pointer in this code is not the same thing as a TCP/UDP socket for Boost::Asio.
Current implement of async receive
This is my current code and where I need help. I'm not sure how to use boost::asio method to wait for receive_data to return. We are trying to replicate the behaviour of socket.async_read_from(). The NetworkAdapter has access to the io_service.
void NetworkAdapter::do_receive() {
rc_t rc;
socket_t *socket_ptr;
receive_params_t rx_params;
size_t rx_buffer_size;
socket_ptr = network_if[0];
rx_buffer_size = receive_buffer_.size();
// What do I put here to await for this to return asynchronously?
rc = receive_data(socket_ptr, receive_buffer_.data(), &rx_buffer_size, &rx_params, ATLK_WAIT_FOREVER);
on_read(rc, rx_buffer_size, rx_params);
}
void NetworkAdapter::on_read(const rc_t &rc, std::size_t read_bytes, const receive_params_t &rx_params) {
if (!rc) {
// Do something with received data...
} else {
LOG(ERROR) << "Packet receieve failure";
}
do_receive();
}
Summary
How do I use boost::asio async/await functions to await a function return? In particular I want to replicate the behaviour of socket.async_receive_from() but with a function rather than a socket.
*Some function names and types have been changed due to data protection requirements.
N4045 Library Foundations for Asynchronous Operations, Revision 2
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4045.pdf
On page 24 there is an example on how to implement an asio async API in terms of callback-based os API.
// the async version of your operation, implementing all kinds of async paradigm in terms of callback async paradigm
template <class CompletionToken>
auto async_my_operation(/* any parameters needed by the sync version of your operation */, CompletionToken&& token)
{
// if CompletionToken is a callback function object, async_my_operation returns void, the callback's signature should be void(/* return type of the sync version of your operation, */error_code)
// if CompletionToken is boost::asio::use_future, async_my_operation returns future</* return type of the sync version of your operation */>
// if CompletionToken is ..., ...
// you are not inventing new async paradigms so you don't have to specialize async_completion or handler_type, you should focus on implement the os_api below
async_completion<CompletionToken, void(/* return type of the sync version of your operation, */error_code)/* signature of callback in the callback case */> completion(token);
typedef handler_type_t<CompletionToken, void(error_code)> Handler;
unique_ptr<wait_op<Handler>> op(new wait_op<Handler>(move(completion.handler))); // async_my_operation initates your async operation and exits, so you have to store completion.handler on the heap, the completion.handler will be invoked later on a thread pool (e.g. threads blocked in IOCP if you are using os api, threads in io_context::run() if you are using asio (sockets accept an io_context during construction, so they know to use which io_context to run completion.handler))
// most os api accepts a void* and a void(*)(result_t, void*) as its C callback function, this is type erasure: the void* points to (some struct that at least contains) the C++ callback function object (can be any type you want), the void(*)(result_t, void*) points to a C callback function to cast the void* to a pointer to C++ callback function object and call it
os_api(/* arguments, at least including:*/ op.get(), &wait_callback<Handler>);
return completion.result.get();
}
// store the handler on the heap
template <class Handler>
struct wait_op {
Handler handler_;
explicit wait_op(Handler handler) : handler_(move(handler)) {}
};
// os post a message into your process's message queue, you have several threads blocking in a os api (such as IOCP) or asio api (such as io_context::run()) that continuously takes message out from the queue and then call the C callback function, the C callback function calls your C++ callback function
template <class Handler>
void wait_callback(result_t result, void* param)
{
unique_ptr<wait_op<Handler>> op(static_cast<wait_op<Handler>*>(param));
op‐>handler_(/* turn raw result into C++ classes before passing it to C++ code */, error_code{});
}
//trivial implementation, you should consult the socket object to get the io_context it uses
void os_api(/* arguments needed by your operation */, void* p_callback_data, void(*p_callback_function)(result_t, void*))
{
std::thread([](){
get the result, blocks
the_io_context_of_the_socket_object.post([](){ (*p_callback_function)(result, p_callback_data); });
}).detach();
}
boost.asio has changed from async_completion and handler_type to async_result, so the above code is outdated.
Requirements on asynchronous operations - 1.75.0
https://www.boost.org/doc/libs/1_75_0/doc/html/boost_asio/reference/asynchronous_operations.html
I have a program (client + server) that works with no issue with this write:
boost::asio::write(this->socket_, boost::asio::buffer(message.substr(count,length_to_send)));
where socket_ is boost::asio::ssl::stream<boost::asio::ip::tcp::socket> and message is an std::string.
I would like to make this better and non-blocking, so I created a function that could replace this, it's called like follows:
write_async_sync(socket_,message.substr(count,length_to_send));
The purpose of this function is:
To make the call async, intrinsically
To keep the interface unchanged
The function I implemented simply uses promise/future to simulate sync behavior, which I will modify later (after it works) to be cancellable:
std::size_t
SSLClient::write_async_sync(boost::asio::ssl::stream<boost::asio::ip::tcp::socket>& socket,
const std::string& message_to_send)
{
boost::system::error_code write_error;
std::promise<std::size_t> write_promise;
auto write_future = write_promise.get_future();
boost::asio::async_write(socket,
boost::asio::buffer(message_to_send),
[this,&write_promise,&write_error,&message_to_send]
(const boost::system::error_code& error,
std::size_t size_written)
{
logger.write("HANDLING WRITING");
if(!error)
{
write_error = error;
write_promise.set_value(size_written);
}
else
{
write_promise.set_exception(std::make_exception_ptr(std::runtime_error(error.message())));
}
});
std::size_t size_written = write_future.get();
return size_written;
}
The problem: I'm unable to get the async functionality to work. The sync one works fine, but async simply freezes and never enters the lambda part (the writing never happens). What am I doing wrong?
Edit: I realized that using poll_one() makes the function execute and it proceeds, but I don't understand it. This is how I'm calling run() for io_service (before starting the client):
io_service_work = std::make_shared<boost::asio::io_service::work>(io_service);
io_service_thread.reset(new std::thread([this](){io_service.run();}));
where basically these are shared_ptr. Is this wrong? Does this way necessitate using poll_one()?
Re. EDIT:
You have the io_service::run() correctly. This tells me you are blocking on the future inside a (completion) handler. That, obviously, prevents run() from progressing the event loop.
The question asked by #florgeng was NOT whether you have an io_service instance.
The question is whether you are calling run() (or poll()) on it suitably for async operations to proceed.
Besides, you can already use future<> builtin:
http://www.boost.org/doc/libs/1_64_0/doc/html/boost_asio/overview/cpp2011/futures.html
Example: http://www.boost.org/doc/libs/1_64_0/doc/html/boost_asio/example/cpp11/futures/daytime_client.cpp
std::future<std::size_t> recv_length = socket.async_receive_from(
boost::asio::buffer(recv_buf),
sender_endpoint,
boost::asio::use_future);
Are nested boost::bind permissible, and if so what am I doing wrong? I can nest lambda in bind successfully, but not bind in bind.
First example
The simple case
I can manage the standard use boost::bind to pass a complex completion handler invocation where a simple one taking only error code is needed:
socket->receive(buffer, boost::bind(...));
Nested case
but if I want to encapsulate a combination of boost asio operations (e.g. multi-stage async_connect and async_ssl_handshake).
My outer operation will be something like:
connect_and_ssl(socket, boost::bind(...));
and my first stage definition will pass the outer handler on to the second completion in another bind, so that the outer handler can be invoked at the end:
template<typename Socket, typename Handler>
void connect_and_ssl(Socket socket, Handler handler)
{
socket.async_connect(endpoint,
boost::bind(&w::handle_connect, this, socket, handler, boost::asio::placeholders::error));
};
template<typename Socket, typename Handler>
void handle_connect(Socket socket, Handler handler, const boost::system::error_code& ec) {
socket->async_handshake(handler);
}
however handler which is a boost::bind really does not like being part of another boost bind. I get a whole screen full of errors, about not being able to determine the type, and others.
Lambdas work
But I find that I can easily use lambdas instead:
template<typename Socket, typename Handler>
void connect_and_ssl(Socket socket, Handler handler)
{
socket.async_connect(endpoint,
[=](const boost::system::error_code& ec) { handle_connect(socket, handler, ec); } );
};
why? Lambdas are so much easier to write, and understand, but do they make possible something that was impossible with nested binds, or was I just expressing the binds wrongly?
Second example
Simple case
although this will compile:
m_ssl_socket->async_read_some(buffer, m_strand->wrap(handler));
Nested case
when converting to be also invoked from a strand:
m_strand->post(boost::bind(&boost::asio::ssl::stream<boost::asio::ip::tcp::socket&>::async_read_some, m_ssl_socket, buffer, m_strand->wap(handler)));
it will no longer compile - no doubt due to the strand->wrap being inside a boost::bind
Lambda
However the lamda version compiles and runs fine:
m_strand->post([=](){m_ssl_socket->async_read_some(buffer, m_strand->wrap(handler)); } );
I can't work it out, but I'm very glad for lamdas.
Nested bind requires protect.
Boost Bind has it.
In C++11 you have to define one yourself (e.g. using reference_wrapper).
I want to use std::functions for callback parameters in a wrapper class.
The class wraps a library that allows asynchronous TCP/IP operations (actually boost::asio but neither boost::asio nor TCP/IP should matter here, only that it has asynchronous operations).
The library functions allow me to pass another callback function object that is asynchronously called when the requested operation is finished.
Depending on the result of the asynchronous operation I want to invoke the callback specified by my client or start further operations.
The following code tries to sketch what I intend.
using ConnectHandler = std::function<void(boost::system::error_code ec)>;
class MyConnection
{
public:
void Connect(ConnectHandler handler); // (1)
}
void MyConnection::Connect(ConnectHandler handler)
{
SomeLibrary::async_connect(...,
[handler](boost::system::error_code ec, ...) // (2)
{
//Depending on ec start a nested read/write operation.
//One way or another it finally invokes the callback
handler(ec); // (3)
});
}
The client code would look something like this
MyConnection conn;
conn.Connect([](boost::system::error_code ec)
{
//check ec and start read/write operation
});
My question is:
what is the best way to declare my Connect method in (1), f.e
void Connect(ConnectHandler handler);
void Connect(const ConnectHandler& handler);
void Connect(ConnectHandler&& handler);
and depending on that how do I correctly capture the callback handler in the lambda capture clause in (2) such that I can call it in (3)?
A side note:
the clients instance of MyConnection will never go out of scope until all asynchronous operations have completed!
std::function are cheap to move, so taking it by value is acceptable. Taking by && is mostly pointless, as at best is saves a move. And it forces the caller to move, not copy, and maybe the caller wants to copy?
They are not cheap to copy, so you could consider capturing by move in your callable object.
In C++14, this is as simple as:
[handler=std::move(handler)]
as a capture list (generalized capture expressions).
In C++11 you need to write a custom object to do this.
struct custom_work {
ConnectHandler handler;
void operator()(boost::system::error_code ec, ...) const {
//Depending on ec start a nested read/write operation.
//One way or another it finally invokes the callback
handler(ec); // (3)
}
};
then
SomeLibrary::async_connect(...,
some_work{std::move(handler)}
);
which has the disadvantage of moving the code from inline to out of line.
bool Connection::Receive(){
std::vector<uint8_t> buf(1000);
boost::asio::async_read(socket_,boost::asio::buffer(buf,1000),
boost::bind(&Connection::handler, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
int rcvlen=buf.size();
ByteBuffer b((std::shared_ptr<uint8_t>)buf.data(),rcvlen);
if(rcvlen <= 0){
buf.clear();
return false;
}
OnReceived(b);
buf.clear();
return true;
}
The method works fine but only when I make a breakpoint inside it. Is there an issue with timing as it waits to receive? Without the breakpoint, nothing is received.
You are trying to read from the receive buffer immediately after starting the asynchronous operation, without waiting for it to complete, that is why it works when you set a breakpoint.
The code after your async_read belongs into Connection::handler, since that is the callback you told async_read to invoke after receiving some data.
What you usually want is a start_read and a handle_read_some function:
void connection::start_read()
{
socket_->async_read_some(boost::asio::buffer(read_buffer_),
boost::bind(&connection::handle_read_some, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
void connection::handle_read_some(const boost::system::error_code& error, size_t bytes_transferred)
{
if (!error)
{
// Use the data here!
start_read();
}
}
Note the shared_from_this, it's important if you want the lifetime of your connection to be automatically taken care of by the number of outstanding I/O requests. Make sure to derive your class from boost::enable_shared_from_this<connection> and to only create it with make_shared<connection>.
To enforce this, your constructor should be private and you can add a friend declaration (C++0x version; if your compiler does not support this, you will have to insert the correct number of arguments yourself):
template<typename T, typename... Arg> friend boost::shared_ptr<T> boost::make_shared(const Arg&...);
Also make sure your receive buffer is still alive by the time the callback is invoked, preferably by using a statically sized buffer member variable of your connection class.