The problem can be a bit complex. I will try to explain the best possible the situation and what tools I imagined to solve my problems.
I am writing a socket application that may write into a socket and expects a response. The protocol enable that in an easy way: each request has a "command id" that will be forwarded back into the response so we can have code that react to that particular request.
For simplicity, we will assume all communication is done using json in the socket.
First, let's assume this session type:
using json = /* assume any json lib */;
struct socket_session {
auto write(json data) -> boost::awaitable<void>;
auto read() -> boost::awaitable<json>;
private:
boost::asio::ip::tcp::socket socket;
};
Usually, I would go with a callback system that go (very) roughly like this.
using command_it_t = std::uint32_t;
// global incrementing command id
command_it_t command_id = 0;
// All callbacks associated with commands
std::unordered_map<command_id_t, std::function<void(json)>> callbacks;
void write_command_to_socket(
boost::io_context& ioc,
socket_session& session,
json command,
std::function<void(json)> callback
) {
boost::co_spawn(ioc, session->write(command), asio::detached);
callbacks.emplace(command_id++, callback);
}
// ... somewhere in the read loop, we call this:
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if (auto const it = callbacks.find(command_id_t{command_id}); it != callbacks.end()) {
// We found the callback for this command, call it!
auto const& [id, callback] = *it;
callback(response["payload"]);
callbacks.erase(it);
}
}
}
It would be used like this:
write_command_to_socket(ioc, session, json_request, [](json response) {
// do stuff
});
As I began using coroutine more and more for asynchronous code, I noticed that it's a golden opportunity to use them in that kind of system.
Instead of sending a callback to the write function, it would return a boost::awaitable<json>, that would contain the response payload, I imagined it a bit like this:
auto const json_response = co_await write_command_to_socket(session, json_request);
Okay, here's the problem
So the first step to do that was to transform my code like this:
void write_command_to_socket(socket_session& session, json command) {
co_await session->write(command);
co_return /* response data from the read loop?? */
}
I noticed that I don't have any mean to await on the response, as it is on another async loop. I was able to imagine a system that looked like I wanted, but I have no idea how to translate my own mental model to asio with coroutines.
// Type from my mental model: an async promise
template<typename T>
struct promise {
auto get_value() -> boost::awaitable<T>;
auto write_value(T value);
};
// Instead of callbacks, my mental model needs promises structured in a similar way:
std::unordered_map<command_id_t, promise<json>> promises;
void write_command_to_socket(socket_session& session, json command) {
auto const [it, inserted] = promises.emplace(session_id++, promise<json>{});
auto const [id, promise] = *it;
co_await session->write(command);
// Here we awaits until the reader loop sets the value
auto const response_json = co_await promise.get_value();
co_return response_json;
}
// ... somewhere in the read loop
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if(auto const it = promises.find(command_id_t{command_id}); it != promises.end()) {
auto const& [id, promise] = *it;
// Effectively calls the write_command_to_socket coroutine to continue
promise.write_value(response["payload"]);
promise.erase(it);
}
}
}
As far as I know, the "promise type" I written here as an example don't exist in boost. Without that type, I really struggle how my command system can exist. Would I need to write my own coroutine type for that kind of system? Is there a way I can just get away using boost's coroutine types?
With asio, as I said, the "promise type" don't exist. Asio instead uses continuation handlers, which are kind of callbacks that may actually call a callback or resume a coroutine.
To create such continuation handler, one must first initiate an async operation. The async operation can be resumed by another if you want, or composed of many async operation. This is done with the asio::async_initiate function, which takes some parameter reguarding the form of the continuation:
// the completion token type can be a callback,
// could be `asio::use_awaitable_t const&` or even `asio::detached_t const&`
return asio::async_initiate<CompletionToken, void(json)>(
[self = shared_from_this()](auto&& handler) {
// HERE! `handler` is a callable that resumes the coroutine!
// We can register it somewhere
callbacks.emplace(command_id, std::forward<decltype(handler)>(handler));
}
);
To resume the async operation, you simply have to call the continuation handler:
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if (auto const it = callbacks.find(command_id_t{command_id}); it != callbacks.end()) {
// We found the continuation handler for this command, call it!
// It resumes the coroutine with the json as its result
auto const& [id, callback] = *it;
callback(response["payload"]);
callbacks.erase(it);
}
}
}
Here's the rest of the system, how it would look like (very roughtly):
using command_it_t = std::uint32_t;
// global incrementing command id
command_it_t command_id = 0;
// All callbacks associated with commands
std::unordered_map<command_id_t, moveable_function<void(json)>> callbacks;
void write_command_to_socket(
boost::io_context& ioc,
socket_session session,
json command
) -> boost::asio::awaitable<json> {
return asio::async_initiate<boost::asio::use_awaitable_t<> const&, void(json)>(
[&ioc, session](auto&& handler) {
callbacks.emplace(command_id, std::forward<decltype(handler)>(handler));
boost::asio::co_spawn(ioc, session.write(command), asio::detached);
}
);
}
Related
We started using the modern C++20 coroutines on our project recently. There is a list of coroutines referred to as Tasks in the Executor, which steps through them one by one resuming them. All of this is done on a single thread. Sometimes coroutines need not to be resumed until some predicate is satisfied. In some cases it may be satisfied by another coroutine, which makes suspending for later execution just fine.
Here are the types in use:
struct Task : std::coroutine_handle<task_promise_t> {
using promise_type = task_promise_t;
};
struct task_promise_t {
Task get_return_object() { return {Task::from_promise(*this)}; }
std::suspend_always initial_suspend() noexcept { return {}; }
std::suspend_always final_suspend() noexcept { return {}; }
void return_void() {}
void unhandled_exception() {}
};
struct Executor {
/* snip */
void enqueue_task(Task &&task) { tasks.push_back(task); }
void tick() {
while (!tasks.empty())
this->step();
}
void step() {
Task task = std::move(tasks.front());
tasks.pop_front();
task.resume();
if (!task.done())
tasks.push_back(task);
}
std::deque<Task> tasks;
/* snip */
}
Example of how I expect it to be used:
auto exec = Executor();
static bool global_predicate = false;
exec.enqueue_task([](Executor* exec) -> Task {
co_await WaitFor(/* bool(void) */ []() -> bool { return global_predicate; });
/* prerequisite satisfied, other logic goes here */
std::cout << "Hello, world" << std::endl;
}(&exec));
exec.step(); // no output, predicate false
exec.step(); // no output, predicate false
global_predicate = true;
exec.step(); // predicate true, "Hello, world!", coroutine is also done
I did manage to get the implementation going, this seems to work fine.
static bool example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
But I can't a good way to generalize and abstract it into it's own class. How would one go about it? I would expect to see that functionality in the standard library, but seeing how customizable the coroutines are I doubt there is a way to implement a one-size-fits-all solution.
The "await" style of coroutines is intended for doing asynchronous processing in a way that mirrors the synchronous equivalent. In sinchronous code, you might write:
int func(float f)
{
auto value = compute_stuff(f);
auto val2 = compute_more_stuff(value, 23);
return val2 + value;
}
If one or both of these functions is asychronous, you would rewrite it as follows (assuming the presence of appropriate co_await machinery):
task<int> func(float f)
{
auto value = compute_stuff(f);
auto val2 = co_await async_compute_more_stuff(value, 23);
co_return val2 + value;
}
It's structurally the same code except that in one case, func will halt halfway through until async_compute_more_stuff has finished its computation, then be resumed and return its value through Task<int>. The async nature of the code is as implicit as possible; it largely looks like synchronous code.
If you already have some extant async process, and you just want a function to get called when that process concludes, and there is no direct relationship between them, you don't need a coroutine. This code:
static atomic<bool> example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
Is not meaningfully different from this:
static atomic<bool> example_global_predicate;
register_polling_task([]() -> bool
{
if(!example_global_predicate)
return false;
/* example_global_predicate is now true, do stuff */
return true;
});
register_polling_task represents some global construct which will at regular intervals call your function until it returns true, at which point it assumes that it has done its job and removes the task. Your coroutine version might hide this global construct, but it still needs to be there because somebody has to wake the coroutine up.
Overall, this is not an async circumstance where using coroutines buys you anything in particular.
However, it could still be theoretically useful to attach coroutine resumption to a polling task. The most reasonable way to do this is to put the polling in a task outside of a coroutine. That is, coroutines shouldn't poll for the global state; that's someone else's job.
A coroutine would do something like co_await PollingTask(). This hands the coroutine_handle off to the system that polls the global state. When that global state enters the correct state, it will resume that handle. And when executing the co_await expression, it should also check the state then, so that if the state is already signaled, it should just not halt the coroutine's execution.
PollingTask() would return an awaitable that has all of this machinery built into it.
I have bidirectional streaming async grpc client that use ClientAsyncReaderWriter for communication with server. RPC code looks like:
rpc Process (stream Request) returns (stream Response)
For simplicity Request and Response are bytes arrays (byte[]). I send several chunks of data to server, and when server accumulate enough data, server process this data and send back the response and continue accumulating data for next responses. After several responses, the server send final response and close connection.
For async client I using CompletionQueue. Code looks like:
...
CompletionQueue cq;
std::unique_ptr<Stub> stub;
grpc::ClientContext context;
std::unique_ptr<grpc::ClientAsyncReaderWriter<Request,Response>> responder = stub->AsyncProcess(&context, &cq, handler);
// thread for completition queue
std::thread t(
[]{
void *handler = nullptr;
bool ok = false;
while (cq_.Next(&handler, &ok)) {
if (can_read) {
// how do you know that it is read data available
// Do read
} else {
// do write
...
Request request = prepare_request();
responder_->Write(request, handler);
}
}
}
);
...
// wait
What is the proper way to async reading? Can I try to read if it no data available? Is it blocking call?
Sequencing Read() calls
Can I try to read if it no data available?
Yep, and it's going to be case more often than not. Read() will do nothing until data is available, and only then put its passed tag into the completion queue. (see below for details)
Is it blocking call?
Nope. Read() and Write() return immediately. However, you can only have one of each in flight at any given moment. If you try to send a second one before the previous has completed, it (the second one) will fail.
What is the proper way to async reading?
Each time a Read() is done, start a new one. For that, you need to be able to tell when a Read() is done. This is where tags come in!
When you call Read(&msg, tag), or Write(request, tag),you are telling grpc to put tag in the completion queue associated with that responder once that operation has completed. grpc doesn't care what the tag is, it just hands it off.
So the general strategy you will want to go for is:
As soon as you are ready to start receiving messages:
call responder->Read() once with some tag that you will recognize as a "read done".
Whenever cq_.Next() gives you back that tag, and ok == true:
consume the message
Queue up a new responder->Read() with that same tag.
Obviously, you'll also want to do something similar for your calls to Write().
But since you still want to be able to lookup the handler instance from a given tag, you'll need a way to pack a reference to the handler as well as information about which operation is being finished in a single tag.
Completion queues
Lookup the handler instance from a given tag? Why?
The true raison d'ĂȘtre of completion queues is unfortunately not evident from the examples. They allow multiple asynchronous rpcs to share the same thread. Unless your application only ever makes a single rpc call, the handling thread should not be associated with a specific responder. Instead, that thread should be a general-purpose worker that dispatches events to the correct handler based on the content of the tag.
The official examples tend to do that by using pointer to the handler object as the tag. That works when there's a specific sequence of events to expect since you can easily predict what a handler is reacting to. You often can't do that with async bidirectional streams, since any given completion event could be a Read() or a Write() finishing.
Example
Here's a general outline of what I personally consider to be a clean way to go about all that:
// Base class for async bidir RPCs handlers.
// This is so that the handling thread is not associated with a specific rpc method.
class RpcHandler {
// This will be used as the "tag" argument to the various grpc calls.
struct TagData {
enum class Type {
start_done,
read_done,
write_done,
// add more as needed...
};
RpcHandler* handler;
Type evt;
};
struct TagSet {
TagSet(RpcHandler* self)
: start_done{self, TagData::Type::start_done},
read_done{self, TagData::Type::read_done},
write_done{self, TagData::Type::write_done} {}
TagData start_done;
TagData read_done;
TagData write_done;
};
public:
RpcHandler() : tags(this) {}
virtual ~RpcHandler() = default;
// The actual tag objects we'll be passing
TagSet tags;
virtual void on_ready() = 0;
virtual void on_recv() = 0;
virtual void on_write_done() = 0;
static void handling_thread_main(grpc::CompletionQueue* cq) {
void* raw_tag = nullptr;
bool ok = false;
while (cq->Next(&raw_tag, &ok)) {
TagData* tag = reinterpret_cast<TagData*>(raw_tag);
if(!ok) {
// Handle error
}
else {
switch (tag->evt) {
case TagData::Type::start_done:
tag->handler->on_ready();
break;
case TagData::Type::read_done:
tag->handler->on_recv();
break;
case TagData::Type::write_done:
tag->handler->on_write_done();
break;
}
}
}
}
};
void do_something_with_response(Response const&);
class MyHandler final : public RpcHandler {
public:
using responder_ptr =
std::unique_ptr<grpc::ClientAsyncReaderWriter<Request, Response>>;
MyHandler(responder_ptr responder) : responder_(std::move(responder)) {
// This lock is needed because StartCall() can
// cause the handler thread to access the object.
std::lock_guard lock(mutex_);
responder_->StartCall(&tags.start_done);
}
~MyHandler() {
// TODO: finish/abort the streaming rpc as appropriate.
}
void send(const Request& msg) {
std::lock_guard lock(mutex_);
if (!sending_) {
sending_ = true;
responder_->Write(msg, &tags.write_done);
} else {
// TODO: add some form of synchronous wait, or outright failure
// if the queue starts to get too big.
queued_msgs_.push(msg);
}
}
private:
// When the rpc is ready, queue the first read
void on_ready() override {
std::lock_guard l(mutex_); // To synchronize with the constructor
responder_->Read(&incoming_, &tags.read_done);
};
// When a message arrives, use it, and start reading the next one
void on_recv() override {
// incoming_ never leaves the handling thread, so no need to lock
// ------ If handling is cheap and stays in the handling thread.
do_something_with_response(incoming_);
responder_->Read(&incoming_, &tags.read_done);
// ------ If responses is expensive or involves another thread.
// Response msg = std::move(incoming_);
// responder_->Read(&incoming_, &tags.read_done);
// do_something_with_response(msg);
};
// When has been sent, send the next one is there is any
void on_write_done() override {
std::lock_guard lock(mutex_);
if (!queued_msgs_.empty()) {
responder_->Write(queued_msgs_.front(), &tags.write_done);
queued_msgs_.pop();
} else {
sending_ = false;
}
};
responder_ptr responder_;
// Only ever touched by the handler thread post-construction.
Response incoming_;
bool sending_ = false;
std::queue<Request> queued_msgs_;
std::mutex mutex_; // grpc might be thread-safe, MyHandler isn't...
};
int main() {
// Start the thread as soon as you have a completion queue.
auto cq = std::make_unique<grpc::CompletionQueue>();
std::thread t(RpcHandler::handling_thread_main, cq.get());
// Multiple concurent RPCs sharing the same handling thread:
MyHandler handler1(serviceA->MethodA(&context, cq.get()));
MyHandler handler2(serviceA->MethodA(&context, cq.get()));
MyHandlerB handler3(serviceA->MethodB(&context, cq.get()));
MyHandlerC handler4(serviceB->MethodC(&context, cq.get()));
}
If you have a keen eye, you will notice that the code above stores a bunch (1 per event type) of redundant this pointers in the handler. It's generally not a big deal, but it is possible to do without them via multiple inheritance and downcasting, but that's starting to be somewhat beyond the scope of this question.
I'm trying to make an async tcp client(it's gonna not waits for result of a request before sending another request).
A request method looks like:
std::future<void> AsyncClient::SomeRequestMethod(sometype& parameter)
{
return std::async(
std::launch::async,
[&]()
{
// Gonna send a json. ';' at the end of a json separates the requests.
const std::string requestJson = Serializer::ArraySumRequest(numbers) + ';';
boost::system::error_code err;
write(requestJson, err);
// Other stuff.
write method calls boost::asio::write like this:
void AsyncClient::write(const std::string& strToWrite, boost::system::error_code& err)
{
// m_writeMutex is a class member I use to synchronize writing.
std::lock_guard<std::mutex> lock(m_writeMutex);
boost::asio::write(m_socket,
boost::asio::buffer(strToWrite), err);
}
But looks like still multiple threads do write concurrently as what I receive in server is like:
{"Key":"Val{"Key":Value};ue"};
What should I do?
You did put the lock guard around writing to the asio. There is, as you can see, no guarantee the other end will have them processed with the same guard.
You should rather put the guard where you need it, on writing the json, out of the asio:
void AsyncClient::write(const std::string& strToWrite, boost::system::error_code& err)
{
// m_writeMutex is a class member I use to synchronize writing.
// std::lock_guard<std::mutex> lock(m_writeMutex);
boost::asio::write(m_socket,
boost::asio::buffer(strToWrite), err);
}
return std::async(
std::launch::async,
[&]()
{
std::lock_guard<std::mutex> lock(m_writeMutex); // <--- here
// Gonna send a json. ';' at the end of a json separates the requests.
const std::string requestJson = Serializer::ArraySumRequest(numbers) + ';';
boost::system::error_code err;
write(requestJson, err);
// Other stuff.
How can I accomplish this scenario purely asynchronously:
Let's assume there is one thread working asynchronously which creates new thread when some conditions were fullfiled:
class listener {
...
void on_message(data_type data) {
if(some_specific_data_found(data)) {
do_some_work_in_new_thread(new_thread, data, callback_on_end);
}
}
...
void callback_on_end(result_type result) {
do_some_work_in_this_thread(result);
}
...
}
Newly created thread looks like this:
void new_thread(data_type data) {
auto result = do_some_work_here();
push_result(result); // This result should be accessible in
// callback_on_end function.
}
I know that I can achieve similar solution using futures, but I don't want to call any blocking function like get() or wait().
mutex::lock / unlock
http://en.cppreference.com/w/cpp/thread/mutex
you can unlock it when the first thread returns.
I used boost::future::then like this:
boost::future f =
boost::async(boost::launch::async, // start task in new thread
boost::bind(task_code, task_args));
boost::future end_callback = f.then([&](auto i) {
auto result = i.get(); // i is of type future; get() returns object
// of type result_type_of_task_code
// get() is nonblocking
do_something(result);
});
I create a new object and set a data and a callback something like this:
class DownloadData
{
std::function<void(int, bool)> m_callback;
int m_data;
public:
void sendHttpRequest()
{
// send request with data
}
private:
void getHttpResponse(int responseCode)
{
if (responseCode == 0)
{
// save data
m_callback(responseCode, true);
delete this;
return;
}
// some processing here
if (responseCode == 1 && some other condition here)
{
m_callback(responseCode, false);
delete this;
return;
}
}
}
Now the usage - I create a new object:
if (isNeededToDownloadTheFile)
{
DownloadData* p = new DownloadData(15, [](){});
p->sendHttpRequest();
}
But as you can see https://isocpp.org/wiki/faq/freestore-mgmt#delete-this it is highly not desirable to make a suicide. Is there a good design pattern or an approach for this?
You could put them in a vector or list, have getHttpResponse() set a flag instead of delete this when it's completed, and then have another part of the code occasionally traverse the list looking for completed requests.
That would also allow you to implement a timeout. If the request hasn't returned in a day, it's probably not going to and you should delete that object.
If you want to put the delete out of that function, the only way is to store the object somehow. However, this raises the ownership questions: who is the owner of the asynchronous http request that's supposed to call a callback?
In this scenario, doing the GCs job actually makes the code pretty clear. However, if you wanted to make it more adaptable to C++, I'd probably settle on a promise-like interface, similar to std::async. That way the synchronous code path makes it way easier to store the promise objects.
You asked for a code example, so there goes:
Typical approach would look like this:
{
DownloadData* p = new DownloadData(15, [](auto data){
print(data)
});
p->sendHttpRequest();
}
Once the data is available, it can be printed. However, you can look at the problem "from the other end":
{
Future<MyData> f = DownloadData(15).getFuture();
// now you can either
// a) synchronously wait for the future
// b) return it for further processing
return f;
}
f will hold the actual value once the request actually processes. That way you can push it as if it was a regular value all the way up to the place where that value is actually needed, and wait for it there. Of course, if you consume it asynchronously, you might as well spawn another asynchronous action for that.
The implementation of the Future is something that's outside of the scope of this answer, I think, but then again numerous resources are available online. The concept of Promises and Futures isn't something specific to C++.
If the caller keeps a reference to the downloading object then it can erase it when the download signals it has ended:
class DownloadData
{
// true until download stops (atomic to prevent race)
std::atomic_bool m_downloading;
int m_data;
std::function<void(int, bool)> m_callback;
public:
DownloadData(int data, std::function<void(int, bool)> callback)
: m_downloading(true), m_data(data), m_callback(callback) {}
void sendHttpRequest()
{
// send request with data
}
// called asynchronously to detect dead downloads
bool ended() const { return !m_downloading; }
private:
void getHttpResponse(int responseCode)
{
if (responseCode == 0)
{
// save data
m_callback(responseCode, true);
m_downloading = false; // signal end
return;
}
// some processing here
if(responseCode == 1)
{
m_callback(responseCode, false);
m_downloading = false; // signal end
return;
}
}
};
Then from the caller's side:
std::vector<std::unique_ptr<DownloadData>> downloads;
// ... other code ...
if (isNeededToDownloadTheFile)
{
// clean current downloads by deleting all those
// whose download is ended
downloads.erase(std::remove_if(downloads.begin(), downloads.end(),
[](std::unique_ptr<DownloadData> const& d)
{
return d->ended();
}), downloads.end());
// store this away to keep it alive until its download ends
downloads.push_back(std::make_unique<DownloadData>(15, [](int, bool){}));
downloads.back()->sendHttpRequest();
}
// ... etc ...