I'm trying to implement some network application using Boost.Asio. I have a problem with multiple layers of callbacks. In other languages that natively support async/await syntax, I can write my logic like this
void do_send(args...) {
if (!endpoint_resolved) {
await resolve_async(...); // results are stored in member variables
}
if (!connected) {
await connect_async(...);
}
await send_async(...);
await receive_async(...);
}
Right now I have to write it using multiple layers of callbacks
void do_send(args...) {
if (!endpoint_resolved) {
resolve_async(..., [captures...](args...) {
if (!connected) {
connect_async(..., [captures...](args...) {
send_async(..., [captures...](args...) {
receive_async(..., [captures...](args...) {
// do something
}); // receive_async
}); // send_async
}); // connect_async
}
});
}
}
This is cumbersome and error-prone. An alternative is to use std::bind to bind member functions as callbacks, but this does not solve the problem because either way I have to write complicated logic in the callbacks to determine what to do next.
I'm wondering if there are better solutions. Ideally I would like to write code in a synchronous way while I can await asynchronously on any I/O operations.
I've also checked std::async, std::future, etc. But they don't seem to fit into my situation.
Boost.Asio's stackful coroutines would provide a good solution. Stackful coroutines allow for asynchronous code to be written in a manner that reads synchronous. One can create a stackful coroutine via the spawn function. Within the coroutine, passing the yield_context as a handler to an asyncornous operation will start the operation and suspend the coroutine. The coroutine will be resumed automatically when the asynchronous operation completes. Here is the example from the documentation:
boost::asio::spawn(my_strand, do_echo);
// ...
void do_echo(boost::asio::yield_context yield)
{
try
{
char data[128];
for (;;)
{
std::size_t length =
my_socket.async_read_some(
boost::asio::buffer(data), yield);
boost::asio::async_write(my_socket,
boost::asio::buffer(data, length), yield);
}
}
catch (std::exception& e)
{
// ...
}
}
Related
I'm using asio for async io, but there are some times where I'd like to "escape" the async world and get my data back into the regular synchronous world.
For instance, consider that I have a std::deque<string> _data that is being used in my async process (in a single thread always running in the background), and were I've created async function to read / write from it.
What is the "natural" way to read from this deque in a synchronous way from another thread ?
So far I've used atomics to do this but this feels a bit "wrong".
For example:
std::string getDataSync()
{
std::atomic<int> signal = 0;
std::string str;
asio::post(io_context, [this, &signal, &str] {
str = _data.front();
_data.pop_front();
signal = 1;
});
while(signal == 0) { }
return str;
}
Is it ok to do this?
Does asio provide anything cleaner to do this kind of operations?
Thanks
If you want to synchronize two threads, then you have to use sychronize primitives (like std::atomic). Asio doesn't provide more advanced primitives, but the STL (and boost) is full of it. For your simple example, you might want to use std::future and std::promise to move the top item of the deque to another thread.
Here is a small example. I assume that you don't want to access the deque directly from the other thread, just the top item. I also assume that you are running boost::asio::run in another thread.
inline constexpr std::string pop_from_queue() { return "hello world"; }
int main() {
auto context = boost::asio::io_context{};
auto promise = std::promise<std::string>{};
auto result = promise.get_future();
boost::asio::post(context,
[&promise] { promise.set_value(pop_from_queue()); });
auto thread = std::thread{[&context] { context.run(); }};
std::cout << result.get(); // blocking
thread.join();
}
Background
I'm new to using Boost::Asio library and am having trouble getting the behaviour I want. I am trying to implement some network communication for custom hardware solution. The communication protocol stack we are using relies heavily on Boost::Asio async methods and I don't believe it is entirely thread safe.
I have successfully implemented sending but encountered a problem when trying to setup the await for receiving. Most boost::asio examples I have found rely on socket behaviour to implement async await with socket_.async_read_some() or other similar functions. However this doesn't work for us as our hardware solution requires calling driver function directly rather than utilising sockets.
The application uses an io_service that is passed into boost::asio::generic::raw_protocol::socket as well as other classes.
Example code from protocol stack using sockets
This is the example code from the protocol stack. do_receive() is called in the constructor of RawSocketLink.
void RawSocketLink::do_receive()
{
namespace sph = std::placeholders;
socket_.async_receive_from(
boost::asio::buffer(receive_buffer_), receive_endpoint_,
std::bind(&RawSocketLink::on_read, this, sph::_1, sph::_2));
}
void RawSocketLink::on_read(const boost::system::error_code& ec, std::size_t read_bytes)
{
if (!ec) {
// Do something with received data...
do_receive();
}
}
Our previous receive code without the protocol stack
Prior to implementing the stack we had been using the threading library to create separate threads for send and recieve. The receive method is shown below. Mostly it relies on calling the receive_data() function from the hardware drivers and waiting for it to return. This is a blocking call but is required to return data.
void NetworkAdapter::Receive() {
uint8_t temp_rx_buffer[2048];
rc_t rc;
socket_t *socket_ptr;
receive_params_t rx_params;
size_t rx_buffer_size;
char str[100];
socket_ptr = network_if[0];
while (1) {
rx_buffer_size = sizeof(temp_rx_buffer);
// Wait until receive_data returns then process
rc = receive_data(socket_ptr,
temp_rx_buffer,
&rx_buffer_size,
&rx_params,
WAIT_FOREVER);
if (rc_error(rc)) {
(void)fprintf(stderr, "Receive failed");
continue;
}
// Do something with received packet ....
}
return;
}
Note that the socket_t pointer in this code is not the same thing as a TCP/UDP socket for Boost::Asio.
Current implement of async receive
This is my current code and where I need help. I'm not sure how to use boost::asio method to wait for receive_data to return. We are trying to replicate the behaviour of socket.async_read_from(). The NetworkAdapter has access to the io_service.
void NetworkAdapter::do_receive() {
rc_t rc;
socket_t *socket_ptr;
receive_params_t rx_params;
size_t rx_buffer_size;
socket_ptr = network_if[0];
rx_buffer_size = receive_buffer_.size();
// What do I put here to await for this to return asynchronously?
rc = receive_data(socket_ptr, receive_buffer_.data(), &rx_buffer_size, &rx_params, ATLK_WAIT_FOREVER);
on_read(rc, rx_buffer_size, rx_params);
}
void NetworkAdapter::on_read(const rc_t &rc, std::size_t read_bytes, const receive_params_t &rx_params) {
if (!rc) {
// Do something with received data...
} else {
LOG(ERROR) << "Packet receieve failure";
}
do_receive();
}
Summary
How do I use boost::asio async/await functions to await a function return? In particular I want to replicate the behaviour of socket.async_receive_from() but with a function rather than a socket.
*Some function names and types have been changed due to data protection requirements.
N4045 Library Foundations for Asynchronous Operations, Revision 2
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4045.pdf
On page 24 there is an example on how to implement an asio async API in terms of callback-based os API.
// the async version of your operation, implementing all kinds of async paradigm in terms of callback async paradigm
template <class CompletionToken>
auto async_my_operation(/* any parameters needed by the sync version of your operation */, CompletionToken&& token)
{
// if CompletionToken is a callback function object, async_my_operation returns void, the callback's signature should be void(/* return type of the sync version of your operation, */error_code)
// if CompletionToken is boost::asio::use_future, async_my_operation returns future</* return type of the sync version of your operation */>
// if CompletionToken is ..., ...
// you are not inventing new async paradigms so you don't have to specialize async_completion or handler_type, you should focus on implement the os_api below
async_completion<CompletionToken, void(/* return type of the sync version of your operation, */error_code)/* signature of callback in the callback case */> completion(token);
typedef handler_type_t<CompletionToken, void(error_code)> Handler;
unique_ptr<wait_op<Handler>> op(new wait_op<Handler>(move(completion.handler))); // async_my_operation initates your async operation and exits, so you have to store completion.handler on the heap, the completion.handler will be invoked later on a thread pool (e.g. threads blocked in IOCP if you are using os api, threads in io_context::run() if you are using asio (sockets accept an io_context during construction, so they know to use which io_context to run completion.handler))
// most os api accepts a void* and a void(*)(result_t, void*) as its C callback function, this is type erasure: the void* points to (some struct that at least contains) the C++ callback function object (can be any type you want), the void(*)(result_t, void*) points to a C callback function to cast the void* to a pointer to C++ callback function object and call it
os_api(/* arguments, at least including:*/ op.get(), &wait_callback<Handler>);
return completion.result.get();
}
// store the handler on the heap
template <class Handler>
struct wait_op {
Handler handler_;
explicit wait_op(Handler handler) : handler_(move(handler)) {}
};
// os post a message into your process's message queue, you have several threads blocking in a os api (such as IOCP) or asio api (such as io_context::run()) that continuously takes message out from the queue and then call the C callback function, the C callback function calls your C++ callback function
template <class Handler>
void wait_callback(result_t result, void* param)
{
unique_ptr<wait_op<Handler>> op(static_cast<wait_op<Handler>*>(param));
op‐>handler_(/* turn raw result into C++ classes before passing it to C++ code */, error_code{});
}
//trivial implementation, you should consult the socket object to get the io_context it uses
void os_api(/* arguments needed by your operation */, void* p_callback_data, void(*p_callback_function)(result_t, void*))
{
std::thread([](){
get the result, blocks
the_io_context_of_the_socket_object.post([](){ (*p_callback_function)(result, p_callback_data); });
}).detach();
}
boost.asio has changed from async_completion and handler_type to async_result, so the above code is outdated.
Requirements on asynchronous operations - 1.75.0
https://www.boost.org/doc/libs/1_75_0/doc/html/boost_asio/reference/asynchronous_operations.html
I am new to C++. I found that the following programming style is quite interesting to me. I wrote a simplified version here.
#include <iostream>
using namespace std;
class MyClass {
public :
MyClass(int id_) : id(id_) {
cout<<"I am a constructor"<<endl;
}
bool error = false;
void run() {
//do something ...
if (!error) {
read();
}
}
void read() {
//do something ...
if (!error) {
write();
}
}
void write() {
//do something ...
if (!error) {
read();
}
}
private :
int id;
};
int main() {
MyClass mc(1);
mc.run();
return 0;
}
The example here is compilable, but I didn't run it because I must go into an infinite loop. But, I hope to use this as a reference. The read() and write() are calling each other. I first encountered this programming style in boost.asio. When the server received a message in do_read(), it calls do_write() to echo the client, then it calls do_read() again at the end of the do_write().
I have two questions regarding this type of coding.
Will this cause stack overflow? Because the functions are keeping calling themselves and the function ends only an error occurs.
What is the advantage of it? Why can't I use a function to loop them orderly and break the loop whenever it encounters an error.
bool replied = true;
while (!error) {
if (replied) read();
else {
write();
replied = !replied;
}
}
Your simplified version leaves out the most important aspect: the write() and read() calls are asynchronous.
Therefore, the functions don't actually cause recursion, see this recent answer: Do "C++ boost::asio Recursive timer callback" accumulate callstack?
The "unusual" thing about async_read(...) and async_write(...) is that the functions return before the IO operation has actually been performed, let alone completed. The actual execution is done on a different schedule¹.
To signal compleion back to the "caller" the async calls typically take a completion handler, which gets called with the result of the IO operation.
In that completion handler, it's typical to see either the end of the communication channel, or the next IO operation being scheduled. This is known as asynchronous call chaining and is very prominently present in many languages that support asynchronous operations ²
It takes some getting used to, but ultimately you get used to the pattern.
With this in mind, revisit one of the boost samples and see if the penny drops:
Documentation sample Chat Client
void handle_connect(const boost::system::error_code& error)
{
if (!error)
{
boost::asio::async_read(socket_,
boost::asio::buffer(read_msg_.data(), chat_message::header_length),
boost::bind(&chat_client::handle_read_header, this,
boost::asio::placeholders::error));
}
}
void handle_read_header(const boost::system::error_code& error)
{
if (!error && read_msg_.decode_header())
{
boost::asio::async_read(socket_,
boost::asio::buffer(read_msg_.body(), read_msg_.body_length()),
boost::bind(&chat_client::handle_read_body, this,
boost::asio::placeholders::error));
}
else
{
do_close();
}
}
void handle_read_body(const boost::system::error_code& error)
{
if (!error)
{
std::cout.write(read_msg_.body(), read_msg_.body_length());
std::cout << "\n";
boost::asio::async_read(socket_,
boost::asio::buffer(read_msg_.data(), chat_message::header_length),
boost::bind(&chat_client::handle_read_header, this,
boost::asio::placeholders::error));
}
else
{
do_close();
}
}
void do_write(chat_message msg)
{
bool write_in_progress = !write_msgs_.empty();
write_msgs_.push_back(msg);
if (!write_in_progress)
{
boost::asio::async_write(socket_,
boost::asio::buffer(write_msgs_.front().data(),
write_msgs_.front().length()),
boost::bind(&chat_client::handle_write, this,
boost::asio::placeholders::error));
}
}
void handle_write(const boost::system::error_code& error)
{
if (!error)
{
write_msgs_.pop_front();
if (!write_msgs_.empty())
{
boost::asio::async_write(socket_,
boost::asio::buffer(write_msgs_.front().data(),
write_msgs_.front().length()),
boost::bind(&chat_client::handle_write, this,
boost::asio::placeholders::error));
}
}
else
{
do_close();
}
}
void do_close()
{
socket_.close();
}
Benefit Of Asynchronous Operations
Asynchronous IO are useful for a more event-based model of IO. Also they remove the first "ceiling" when scaling to large volumes of IO operations. In traditional, imperative code patterns many clients/connections would require many threads in order to be able to serve them simultaneously. In practice, though, threads fail to scale (since a typical server has a smallish number of logical CPUs) and it would mean that IO operations block each other ³.
With asynchronous IO you can often do all IO operations on a single thread, greatly improving efficiency - and thereby some aspects of the program design (because fewer threading issues need to be involved).
¹ Many choices exist, but imagine that io_service::run() is running on a separate thread, that would lead to the IO operations being actually executed, potentially resumed when required and completed on that thread
² I'd say javascript is infamous for this pattern
³ A classical example is when a remote procedure call keeps a thread occupied while waiting for e.g. a database query to complete
This is my opinion:
Regarding recursion
One way to cause a stack overflow is to have a function calling itself recursively, overflowing the call stack. A set of functions calling each other in a circular manner would be equivalent to that, so yes, your intuition is correct.
An iterative version of the algorithm, such as the loop you describe, could prevent that.
Now, another thing that can prevent a stack overflow is the presence of code that could be optimized for tail recursion. Tail recursion optimization requires a compiler implementing this feature. Most major compilers implement it. The Boost.Asio function you mention seems to be benefiting from this optimization.
Regarding code design
Now, C++ implements many programming paradigms. These paradigms are also implemented by many other programming languages. The programming paradigms relevant to what you are discussing would be:
Structured programming
Object oriented programming
From a structured programming point of view, you should try to emphasize code reuse as much as possible by diving the code in subroutines that minimize redundant code.
From an object oriented point of view, you should model classes in a way that encapsulates their logic as much as possible.
The logic you present so far seems encapsulated enough, however, you may need to review if the methods write and read should remain public, or if they should be private instead. Minimizing the number of public methods helps achieving a higher level of encapsulation.
I'm building a network service with boost::asio and I'm unsure about the thread safety.
io_service.run() is called only once from a thread dedicated for the io_service work
send_message() on the other hand can be called either by the code inside the second io_service handlers mentioned later, or by the mainThread upon user interaction. And that is why I'm getting nervous.
std::deque<message> out_queue;
// send_message will be called by two different threads
void send_message(MsgPtr msg){
while (out_queue->size() >= 20){
Sleep(50);
}
io_service_.post([this, msg]() { deliver(msg); });
}
// from my understanding, deliver will only be called by the thread which called io_service.run()
void deliver(const MsgPtr){
bool write_in_progress = !out_queue.empty();
out_queue.push_back(msg);
if (!write_in_progress)
{
write();
}
}
void write()
{
auto self(shared_from_this());
asio::async_write(socket_,
asio::buffer(out_queue.front().header(),
message::header_length), [this, self](asio::error_code ec, std::size_t/)
{
if (!ec)
{
asio::async_write(socket_,
asio::buffer(out_queue.front().data(),
out_queue.front().paddedPayload_size()),
[this, self](asio::error_code ec, std::size_t /*length*/)
{
if (!ec)
{
out_queue.pop_front();
if (!out_queue.empty())
{
write();
}
}
});
}
});
}
Is this scenario safe?
A similar second scenario: When the network thread receives a message, it posts them into another asio::io_service which is also run by its own dedicated thread. This io_service uses an std::unordered_map to store callback functions etc.
std::unordered_map<int, eventSink> eventSinkMap_;
//...
// called by the main thread (GUI), writes a callback function object to the map
int IOReactor::registerEventSink(std::function<void(int, std::shared_ptr<message>)> fn, QObject* window, std::string endpointId){
util::ScopedLock lock(&sync_);
eventSink es;
es.id = generateRandomId();
// ....
std::pair<int, eventSink> eventSinkPair(es.id, es);
eventSinkMap_.insert(eventSinkPair);
return es.id;
}
// called by the second thread, the network service thread when a message was received
void IOReactor::onMessageReceived(std::shared_ptr<message> msg, ConPtr con)
{
reactor_io_service_.post([=](){ handleReceive(msg, con); });
}
// should be called only by the one thread running the reactor_io_service.run()
// read and write access to the map
void IOReactor::handleReceive(std::shared_ptr<message> msg, ConPtr con){
util::ScopedLock lock(&sync_);
auto es = eventSinkMap_.find(msg.requestId);
if (es != eventSinkMap_.end())
{
auto fn = es->second.handler;
auto ctx = es->second.context;
QMetaObject::invokeMethod(ctx, "runInMainThread", Qt::QueuedConnection, Q_ARG(std::function<void(int, std::shared_ptr<msg::IMessage>)>, fn), Q_ARG(int, CallBackResult::SUCCESS), Q_ARG(std::shared_ptr<msg::IMessage>, msg));
eventSinkMap_.erase(es);
}
first of all: Do I even need to use a lock here?
Ofc both methods access the map, but they are not accessing the same elements (the receiveHandler cannot try to access or read an element that has not yet been registered/inserted into the map). Is that threadsafe?
First of all, a lot of context is missing (where is onMessageReceived invoked, and what is ConPtr? and you have too many questions. I'll give you some specific pointers that will help you though.
You should be nervous here:
void send_message(MsgPtr msg){
while (out_queue->size() >= 20){
Sleep(50);
}
io_service_.post([this, msg]() { deliver(msg); });
}
The check out_queue->size() >= 20 requires synchronization unless out_queue is thread safe.
The call to io_service_.post is safe, because io_service is thread safe. Since you have one dedicated IO thread, this means that deliver() will run on that thread. Right now, you need synchronization there too.
I strongly suggest using a proper thread-safe queue there.
Q. first of all: Do I even need to use a lock here?
Yes you need to lock to do the map lookup (otherwise you get a data race with the main thread inserting sinks).
You do not need to lock during the invocation (in fact, that seems like a very unwise idea that could lead to performance issue or lockups). The reference remains valid due to Iterator invalidation rules.
The deletion of course requires a lock again. I'd revise the code to do deletion and removal at once, and invoke the sink only after releasing the lock. NOTE You will have to think about exceptions here (in your code when there is an exception during invocation, the sink doesn't get removed (ever?). This might be important to you.
Live Demo
void handleReceive(std::shared_ptr<message> msg, ConPtr con){
util::ScopedLock lock(&sync_);
auto es = eventSinkMap_.find(msg->requestId);
if (es != eventSinkMap_.end())
{
auto fn = es->second.handler;
auto ctx = es->second.context;
eventSinkMap_.erase(es); // invalidates es
lock.unlock();
// invoke in whatever way you require
fn(static_cast<int>(CallBackResult::SUCCESS), std::static_pointer_cast<msg::IMessage>(msg));
}
}
I find myself writing code that basically looks like this:
using boost::system::error_code;
socket.async_connect(endpoint, [&](error_code Error)
{
if (Error)
{
print_error(Error);
return;
}
// Read header
socket.async_read(socket, somebuffer, [&](error_code Error, std::size_t N)
{
if (Error)
{
print_error(Error);
return;
}
// Read actual data
socket.async_read(socket, somebuffer, [&](error_code Error, std::size_t N)
{
// Same here...
});
});
};
So basically I'm nesting callbacks in callbacks in callbacks, while the logic is simple and "linear".
Is there a more elegant way of writing this, so that the code is both local and in-order?
One elegant solution is to use coroutines. Boost.Asio supports both stackless coroutines, which introduce a small set of pseudo-keywords, and stackful coroutines, which use Boost.Coroutine.
Stackless Coroutines
Stackless coroutines introduce a set of pseudo-keywords preprocessor macros, that implement a switch statement using a technique similar to Duff's Device. The documentation covers each of the keywords in detail.
The original problem (connect->read header->read body) might look something like the following when implemented with stackless coroutines:
struct session
: boost::asio::coroutine
{
boost::asio::ip::tcp::socket socket_;
std::vector<char> buffer_;
// ...
void operator()(boost::system::error_code ec = boost::system::error_code(),
std::size_t length = 0)
{
// In this example we keep the error handling code in one place by
// hoisting it outside the coroutine. An alternative approach would be to
// check the value of ec after each yield for an asynchronous operation.
if (ec)
{
print_error(ec);
return;
}
// On reentering a coroutine, control jumps to the location of the last
// yield or fork. The argument to the "reenter" pseudo-keyword can be a
// pointer or reference to an object of type coroutine.
reenter (this)
{
// Asynchronously connect. When control resumes at the following line,
// the error and length parameters reflect the result of
// the asynchronous operation.
yield socket_.async_connect(endpoint_, *this);
// Loop until an error or shutdown occurs.
while (!shutdown_)
{
// Read header data. When control resumes at the following line,
// the error and length parameters reflect the result of
// the asynchronous operation.
buffer_.resize(fixed_header_size);
yield socket_.async_read(boost::asio::buffer(buffer_), *this);
// Received data. Extract the size of the body from the header.
std::size_t body_size = parse_header(buffer_, length);
// If there is no body size, then leave coroutine, as an invalid
// header was received.
if (!body_size) return;
// Read body data. When control resumes at the following line,
// the error and length parameters reflect the result of
// the asynchronous operation.
buffer_.resize(body_size);
yield socket_.async_read(boost::asio::buffer(buffer_), *this);
// Invoke the user callback to handle the body.
body_handler_(buffer_, length);
}
// Initiate graceful connection closure.
socket_.shutdown(tcp::socket::shutdown_both, ec);
} // end reenter
}
}
Stackful Coroutines
Stackful coroutines are created using the spawn() function. The original problem may look something like the following when implemented with stackful coroutines:
boost::asio::spawn(io_service, [&](boost::asio::yield_context yield)
{
boost::system::error_code ec;
boost::asio::ip::tcp::socket socket(io_service);
// Asynchronously connect and suspend the coroutine. The coroutine will
// be resumed automatically when the operation completes.
socket.async_connect(endpoint, yield[ec]);
if (ec)
{
print_error(ec);
return;
}
// Loop until an error or shutdown occurs.
std::vector<char> buffer;
while (!shutdown)
{
// Read header data.
buffer.resize(fixed_header_size);
std::size_t bytes_transferred = socket.async_read(
boost::asio::buffer(buffer), yield[ec]);
if (ec)
{
print_error(ec);
return;
}
// Extract the size of the body from the header.
std::size_t body_size = parse_header(buffer, bytes_transferred);
// If there is no body size, then leave coroutine, as an invalid header
// was received.
if (!body_size) return;
// Read body data.
buffer.resize(body_size);
bytes_transferred =
socket.async_read(boost::asio::buffer(buffer), yield[ec]);
if (ec)
{
print_error(ec);
return;
}
// Invoke the user callback to handle the body.
body_handler_(buffer, length);
}
// Initiate graceful connection closure.
socket.shutdown(tcp::socket::shutdown_both, ec);
});