Awaiting a predicate with C++20 coroutines - c++

We started using the modern C++20 coroutines on our project recently. There is a list of coroutines referred to as Tasks in the Executor, which steps through them one by one resuming them. All of this is done on a single thread. Sometimes coroutines need not to be resumed until some predicate is satisfied. In some cases it may be satisfied by another coroutine, which makes suspending for later execution just fine.
Here are the types in use:
struct Task : std::coroutine_handle<task_promise_t> {
using promise_type = task_promise_t;
};
struct task_promise_t {
Task get_return_object() { return {Task::from_promise(*this)}; }
std::suspend_always initial_suspend() noexcept { return {}; }
std::suspend_always final_suspend() noexcept { return {}; }
void return_void() {}
void unhandled_exception() {}
};
struct Executor {
/* snip */
void enqueue_task(Task &&task) { tasks.push_back(task); }
void tick() {
while (!tasks.empty())
this->step();
}
void step() {
Task task = std::move(tasks.front());
tasks.pop_front();
task.resume();
if (!task.done())
tasks.push_back(task);
}
std::deque<Task> tasks;
/* snip */
}
Example of how I expect it to be used:
auto exec = Executor();
static bool global_predicate = false;
exec.enqueue_task([](Executor* exec) -> Task {
co_await WaitFor(/* bool(void) */ []() -> bool { return global_predicate; });
/* prerequisite satisfied, other logic goes here */
std::cout << "Hello, world" << std::endl;
}(&exec));
exec.step(); // no output, predicate false
exec.step(); // no output, predicate false
global_predicate = true;
exec.step(); // predicate true, "Hello, world!", coroutine is also done
I did manage to get the implementation going, this seems to work fine.
static bool example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
But I can't a good way to generalize and abstract it into it's own class. How would one go about it? I would expect to see that functionality in the standard library, but seeing how customizable the coroutines are I doubt there is a way to implement a one-size-fits-all solution.

The "await" style of coroutines is intended for doing asynchronous processing in a way that mirrors the synchronous equivalent. In sinchronous code, you might write:
int func(float f)
{
auto value = compute_stuff(f);
auto val2 = compute_more_stuff(value, 23);
return val2 + value;
}
If one or both of these functions is asychronous, you would rewrite it as follows (assuming the presence of appropriate co_await machinery):
task<int> func(float f)
{
auto value = compute_stuff(f);
auto val2 = co_await async_compute_more_stuff(value, 23);
co_return val2 + value;
}
It's structurally the same code except that in one case, func will halt halfway through until async_compute_more_stuff has finished its computation, then be resumed and return its value through Task<int>. The async nature of the code is as implicit as possible; it largely looks like synchronous code.
If you already have some extant async process, and you just want a function to get called when that process concludes, and there is no direct relationship between them, you don't need a coroutine. This code:
static atomic<bool> example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
Is not meaningfully different from this:
static atomic<bool> example_global_predicate;
register_polling_task([]() -> bool
{
if(!example_global_predicate)
return false;
/* example_global_predicate is now true, do stuff */
return true;
});
register_polling_task represents some global construct which will at regular intervals call your function until it returns true, at which point it assumes that it has done its job and removes the task. Your coroutine version might hide this global construct, but it still needs to be there because somebody has to wake the coroutine up.
Overall, this is not an async circumstance where using coroutines buys you anything in particular.
However, it could still be theoretically useful to attach coroutine resumption to a polling task. The most reasonable way to do this is to put the polling in a task outside of a coroutine. That is, coroutines shouldn't poll for the global state; that's someone else's job.
A coroutine would do something like co_await PollingTask(). This hands the coroutine_handle off to the system that polls the global state. When that global state enters the correct state, it will resume that handle. And when executing the co_await expression, it should also check the state then, so that if the state is already signaled, it should just not halt the coroutine's execution.
PollingTask() would return an awaitable that has all of this machinery built into it.

Related

Boost awaitable: write into a socket and await particular response

The problem can be a bit complex. I will try to explain the best possible the situation and what tools I imagined to solve my problems.
I am writing a socket application that may write into a socket and expects a response. The protocol enable that in an easy way: each request has a "command id" that will be forwarded back into the response so we can have code that react to that particular request.
For simplicity, we will assume all communication is done using json in the socket.
First, let's assume this session type:
using json = /* assume any json lib */;
struct socket_session {
auto write(json data) -> boost::awaitable<void>;
auto read() -> boost::awaitable<json>;
private:
boost::asio::ip::tcp::socket socket;
};
Usually, I would go with a callback system that go (very) roughly like this.
using command_it_t = std::uint32_t;
// global incrementing command id
command_it_t command_id = 0;
// All callbacks associated with commands
std::unordered_map<command_id_t, std::function<void(json)>> callbacks;
void write_command_to_socket(
boost::io_context& ioc,
socket_session& session,
json command,
std::function<void(json)> callback
) {
boost::co_spawn(ioc, session->write(command), asio::detached);
callbacks.emplace(command_id++, callback);
}
// ... somewhere in the read loop, we call this:
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if (auto const it = callbacks.find(command_id_t{command_id}); it != callbacks.end()) {
// We found the callback for this command, call it!
auto const& [id, callback] = *it;
callback(response["payload"]);
callbacks.erase(it);
}
}
}
It would be used like this:
write_command_to_socket(ioc, session, json_request, [](json response) {
// do stuff
});
As I began using coroutine more and more for asynchronous code, I noticed that it's a golden opportunity to use them in that kind of system.
Instead of sending a callback to the write function, it would return a boost::awaitable<json>, that would contain the response payload, I imagined it a bit like this:
auto const json_response = co_await write_command_to_socket(session, json_request);
Okay, here's the problem
So the first step to do that was to transform my code like this:
void write_command_to_socket(socket_session& session, json command) {
co_await session->write(command);
co_return /* response data from the read loop?? */
}
I noticed that I don't have any mean to await on the response, as it is on another async loop. I was able to imagine a system that looked like I wanted, but I have no idea how to translate my own mental model to asio with coroutines.
// Type from my mental model: an async promise
template<typename T>
struct promise {
auto get_value() -> boost::awaitable<T>;
auto write_value(T value);
};
// Instead of callbacks, my mental model needs promises structured in a similar way:
std::unordered_map<command_id_t, promise<json>> promises;
void write_command_to_socket(socket_session& session, json command) {
auto const [it, inserted] = promises.emplace(session_id++, promise<json>{});
auto const [id, promise] = *it;
co_await session->write(command);
// Here we awaits until the reader loop sets the value
auto const response_json = co_await promise.get_value();
co_return response_json;
}
// ... somewhere in the read loop
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if(auto const it = promises.find(command_id_t{command_id}); it != promises.end()) {
auto const& [id, promise] = *it;
// Effectively calls the write_command_to_socket coroutine to continue
promise.write_value(response["payload"]);
promise.erase(it);
}
}
}
As far as I know, the "promise type" I written here as an example don't exist in boost. Without that type, I really struggle how my command system can exist. Would I need to write my own coroutine type for that kind of system? Is there a way I can just get away using boost's coroutine types?
With asio, as I said, the "promise type" don't exist. Asio instead uses continuation handlers, which are kind of callbacks that may actually call a callback or resume a coroutine.
To create such continuation handler, one must first initiate an async operation. The async operation can be resumed by another if you want, or composed of many async operation. This is done with the asio::async_initiate function, which takes some parameter reguarding the form of the continuation:
// the completion token type can be a callback,
// could be `asio::use_awaitable_t const&` or even `asio::detached_t const&`
return asio::async_initiate<CompletionToken, void(json)>(
[self = shared_from_this()](auto&& handler) {
// HERE! `handler` is a callable that resumes the coroutine!
// We can register it somewhere
callbacks.emplace(command_id, std::forward<decltype(handler)>(handler));
}
);
To resume the async operation, you simply have to call the continuation handler:
void call_command(json response) {
if (auto const& command_id = response["command"]; command_id.is_integer()) {
if (auto const it = callbacks.find(command_id_t{command_id}); it != callbacks.end()) {
// We found the continuation handler for this command, call it!
// It resumes the coroutine with the json as its result
auto const& [id, callback] = *it;
callback(response["payload"]);
callbacks.erase(it);
}
}
}
Here's the rest of the system, how it would look like (very roughtly):
using command_it_t = std::uint32_t;
// global incrementing command id
command_it_t command_id = 0;
// All callbacks associated with commands
std::unordered_map<command_id_t, moveable_function<void(json)>> callbacks;
void write_command_to_socket(
boost::io_context& ioc,
socket_session session,
json command
) -> boost::asio::awaitable<json> {
return asio::async_initiate<boost::asio::use_awaitable_t<> const&, void(json)>(
[&ioc, session](auto&& handler) {
callbacks.emplace(command_id, std::forward<decltype(handler)>(handler));
boost::asio::co_spawn(ioc, session.write(command), asio::detached);
}
);
}

Most efficient way to get coroutine handle from within a C++ coroutine

I'm fairly new to C++ coroutines. When my app shuts down, I'd like to be able to give any outstanding co-routines a hint that I'd prefer they finished early. One solution/idea I came up with was to put a bool on the promise, and then I need a way to check the bool from within the co-routine.
I pieced together a fairly simple solution (using co_await to get the handle), but this feels like a kluge -- the compiler ought to already have access to the handle from within the frame. I'm wondering if there is a standard intrinsic that allows access to the handle or the promise? Or if anyone has come up with a cleaner, better solution?
struct co_gethandle_awaitable
{
co_gethandle_awaitable()
: m_hWaiter(nullptr)
{
}
bool await_ready() const noexcept
{
return m_hWaiter != nullptr;
}
void await_suspend(std::coroutine_handle<> hWaiter) noexcept
{
m_hWaiter = hWaiter;
m_hWaiter.resume();
}
void await_resume() noexcept
{
}
std::coroutine_handle<> m_hWaiter;
};
coroutine_ref<void> MyFunc()
{
// get my coroutine handle. Ideally this would be a helper function
// but then it would have to be awaitable, which would add further cost.
co_gethandle_awaitable obj;
co_await obj;
std::coroutine_handle<> hMe = obj.m_hWaiter;
// loop.
{
// do work.
if (is_cancelled(hMe))
break;
}
co_return;
}

Resume ASIO Stackless Coroutine

Having played a little with the current implementation of Coroutine TS in Clang, I stumbled upon the asio stackless coroutine implementation. They are described to be Portable Stackless Coroutines in One* Header.
Dealing mostly with asynchronous code I wanted to try them as well.
The coroutine block inside the main function shall await the result asynchronously set by the thread spawned in function foo. However I am uncertain on how to let execution continue at the point <1> (after the yield expression) once the thread set the value.
Using the Coroutine TS I would call the coroutine_handle, however boost::asio::coroutine seems not to be callable.
Is this even possible using boost::asio::coroutine?
#include <thread>
#include <chrono>
#include <boost/asio/coroutine.hpp>
#include <boost/asio/yield.hpp>
#include <cstdio>
using namespace std::chrono_literals;
using coroutine = boost::asio::coroutine;
void foo(coroutine & coro, int & result) {
std::thread([&](){
std::this_thread::sleep_for(1s);
result = 3;
// how to resume at <1>?
}).detach();
}
int main(int, const char**) {
coroutine coro;
int result;
reenter(coro) {
// Wait for result
yield foo(coro, result);
// <1>
std::printf("%d\n", result);
}
std::thread([](){
std::this_thread::sleep_for(2s);
}).join();
return 0;
}
Thanks for your help
First off, stackless coroutines are better described as resumable functions. The problem you're currently having is using main. If you extract your logic to a separate functor it would be possible:
class task; // Forward declare both because they should know about each other
void foo(task &task, int &result);
// Common practice is to subclass coro
class task : coroutine {
// All reused variables should not be local or they will be
// re-initialized
int result;
void start() {
// In order to actually begin, we need to "invoke ourselves"
(*this)();
}
// Actual task implementation
void operator()() {
// Reenter actually manages the jumps defined by yield
// If it's executed for the first time, it will just run from the start
// If it reenters (aka, yield has caused it to stop and we re-execute)
// it will jump to the right place for you
reenter(this) {
// Yield will store the current location, when reenter
// is ran a second time, it will jump past yield for you
yield foo(*this, result);
std::printf("%d\n", result)
}
}
}
// Our longer task
void foo(task & t, int & result) {
std::thread([&](){
std::this_thread::sleep_for(1s);
result = 3;
// The result is done, reenter the task which will go to just after yield
// Keep in mind this will now run on the current thread
t();
}).detach();
}
int main(int, const char**) {
task t;
// This will start the task
t.start();
std::thread([](){
std::this_thread::sleep_for(2s);
}).join();
return 0;
}
Note that it's not possible to yield from sub functions. This is a limitation of stackless coroutines.
How it works:
yield stores a unique identifier to jump to inside the coroutine
yield will run the expression you put behind it, should be an async call or little benefit would be given
after running, it will break out of the reenter block.
Now "start" is done, and you start another thread to wait for. Meanwhile, foo's thread finishes its sleep and call your task again. Now:
the reenter block will read the state of your coroutine, to find it has to jump past the foo call
your task will resume, print the result and drop out of the function, returning to the foo thread.
foo thread is now done and main is likely still waiting for the 2nd thread.

Async constructor in C++11

Sometimes I need to create objects whose constructors take very long time to execute.
This leads to responsiveness problems in UI applications.
So I was wondering if it could be sensible to write a constructor designed to be called asynchronously, by passing a callback to it which will alert me when the object is available.
Below is a sample code:
class C
{
public:
// Standard ctor
C()
{
init();
}
// Designed for async ctor
C(std::function<void(void)> callback)
{
init();
callback();
}
private:
void init() // Should be replaced by delegating costructor (not yet supported by my compiler)
{
std::chrono::seconds s(2);
std::this_thread::sleep_for(s);
std::cout << "Object created" << std::endl;
}
};
int main(int argc, char* argv[])
{
auto msgQueue = std::queue<char>();
std::mutex m;
std::condition_variable cv;
auto notified = false;
// Some parallel task
auto f = []()
{
return 42;
};
// Callback to be called when the ctor ends
auto callback = [&m,&cv,&notified,&msgQueue]()
{
std::cout << "The object you were waiting for is now available" << std::endl;
// Notify that the ctor has ended
std::unique_lock<std::mutex> _(m);
msgQueue.push('x');
notified = true;
cv.notify_one();
};
// Start first task
auto ans = std::async(std::launch::async, f);
// Start second task (ctor)
std::async(std::launch::async, [&callback](){ auto c = C(callback); });
std::cout << "The answer is " << ans.get() << std::endl;
// Mimic typical UI message queue
auto done = false;
while(!done)
{
std::unique_lock<std::mutex> lock(m);
while(!notified)
{
cv.wait(lock);
}
while(!msgQueue.empty())
{
auto msg = msgQueue.front();
msgQueue.pop();
if(msg == 'x')
{
done = true;
}
}
}
std::cout << "Press a key to exit..." << std::endl;
getchar();
return 0;
}
Do you see any drawback in this design? Or do you know if there is a better approach?
EDIT
Following the hints of JoergB's answer, I tried to write a factory which will bear the responsibility to create an object in a sync or async way:
template <typename T, typename... Args>
class FutureFactory
{
public:
typedef std::unique_ptr<T> pT;
typedef std::future<pT> future_pT;
typedef std::function<void(pT)> callback_pT;
public:
static pT create_sync(Args... params)
{
return pT(new T(params...));
}
static future_pT create_async_byFuture(Args... params)
{
return std::async(std::launch::async, &FutureFactory<T, Args...>::create_sync, params...);
}
static void create_async_byCallback(callback_pT cb, Args... params)
{
std::async(std::launch::async, &FutureFactory<T, Args...>::manage_async_byCallback, cb, params...);
}
private:
FutureFactory(){}
static void manage_async_byCallback(callback_pT cb, Args... params)
{
auto ptr = FutureFactory<T, Args...>::create_sync(params...);
cb(std::move(ptr));
}
};
Your design seems very intrusive. I don't see a reason why the class would have to be aware of the callback.
Something like:
future<unique_ptr<C>> constructedObject = async(launchopt, [&callback]() {
unique_ptr<C> obj(new C());
callback();
return C;
})
or simply
future<unique_ptr<C>> constructedObject = async(launchopt, [&cv]() {
unique_ptr<C> ptr(new C());
cv.notify_all(); // or _one();
return ptr;
})
or just (without a future but a callback taking an argument):
async(launchopt, [&callback]() {
unique_ptr<C> ptr(new C());
callback(ptr);
})
should do just as well, shouldn't it? These also make sure that the callback is only ever called when a complete object is constructed (when deriving from C).
It shouldn't be too much effort to make any of these into a generic async_construct template.
Encapsulate your problem. Don't think about asynchronous constructors, just asynchronous methods which encapsulate your object creation.
It looks like you should be using std::future rather than constructing a message queue. std::future is a template class that holds a value and can retrieve the value blocking, timeout or polling:
std::future<int> fut = ans;
fut.wait();
auto result = fut.get();
I will suggest a hack using thread and signal handler.
1) Spawn a thread to do the task of the constructor. Lets call it child thread. This thread will intialise the values in your class.
2) After the constructor is completed, child thread uses the kill system call to send a signal to the parent thread. (Hint : SIGUSR1). The main thread on receiving the ASYNCHRONOUS handler call will know that the required object has been created.
Ofcourse, you can use fields like object-id to differentiate between multiple objects in creation.
My advice...
Think carefully about why you need to do such a long operation in a constructor.
I find often it is better to split the creation of an object into three parts
a) allocation
b) construction
c) initialization
For small objects it makes sense to do all three in one "new" operation. However, heavy weight objects, you really want to separate the stages. Figure out how much resource you need and allocate it. Construct the object in the memory into a valid, but empty state.
Then... do your long load operation into the already valid, but empty object.
I think I got this pattern a long time ago from reading a book (Scott Myers perhaps?) but I highly recommend it, it solves all sorts of problems. For example, if your object is a graphic object, you figure out how much memory it needs. If it fails, show the user an error as soon as possible. If not mark the object as not read yet. Then you can show it on screen, the user can also manipulate it, etc.
Initialize the object with an asynchronous file load, when it completes, set a flag in the object that says "loaded". When your update function sees it is loaded, it can draw the graphic.
It also REALLY helps with problems like construction order, where object A needs object B. You suddenly find you need to make A before B, oh no!! Simple, make an empty B, and pass it as a reference, as long as A is clever enough to know that be is empty, and wait to it is not before it uses it, all is well.
And... Not forgetting.. You can do the opposite on destruction.
Mark your object as empty first, so nothing new uses it (de-initialisation)
Free the resources, (destruction)
Then free the memory (deallocation)
The same benefits apply.
Having partially initialized objects could lead to bugs or unnecessarily complicated code, since you would have to check whether they're initialized or not.
I'd recommend using separate threads for UI and processing, and then use message queues for communicating between threads. Leave the UI thread for just handling the UI, which will then be more responsive all the time.
Place a message requesting creation of the object into the queue that the worker thread waits on, and then after the object has been created, the worker can put a message into UI queue indicating that the object is now ready.
Here's yet another pattern for consideration. It takes advantage of the fact that calling wait() on a future<> does not invalidate it. So, as long you never call get(), you're safe. This pattern's trade-off is that you incur the onerous overhead of calling wait() whenever a member function gets called.
class C
{
future<void> ready_;
public:
C()
{
ready_ = async([this]
{
this_thread::sleep_for(chrono::seconds(3));
cout << "I'm ready now." << endl;
});
}
// Every member function must start with ready_.wait(), even the destructor.
~C(){ ready_.wait(); }
void foo()
{
ready_.wait();
cout << __FUNCTION__ << endl;
}
};
int main()
{
C c;
c.foo();
return 0;
}

Futures vs. Promises

I'm confusing myself with difference between a future and a promise.
Obviously, they have different methods and stuff, but what is the actual use case?
Is it?:
when I'm managing some async task, I use future to get the value "in future"
when I'm the async task, I use promise as the return type to allow the user get a future from my promise
Future and Promise are the two separate sides of an asynchronous operation.
std::promise is used by the "producer/writer" of the asynchronous operation.
std::future is used by the "consumer/reader" of the asynchronous operation.
The reason it is separated into these two separate "interfaces" is to hide the "write/set" functionality from the "consumer/reader".
auto promise = std::promise<std::string>();
auto producer = std::thread([&]
{
promise.set_value("Hello World");
});
auto future = promise.get_future();
auto consumer = std::thread([&]
{
std::cout << future.get();
});
producer.join();
consumer.join();
One (incomplete) way to implement std::async using std::promise could be:
template<typename F>
auto async(F&& func) -> std::future<decltype(func())>
{
typedef decltype(func()) result_type;
auto promise = std::promise<result_type>();
auto future = promise.get_future();
std::thread(std::bind([=](std::promise<result_type>& promise)
{
try
{
promise.set_value(func()); // Note: Will not work with std::promise<void>. Needs some meta-template programming which is out of scope for this question.
}
catch(...)
{
promise.set_exception(std::current_exception());
}
}, std::move(promise))).detach();
return std::move(future);
}
Using std::packaged_task which is a helper (i.e. it basically does what we were doing above) around std::promise you could do the following which is more complete and possibly faster:
template<typename F>
auto async(F&& func) -> std::future<decltype(func())>
{
auto task = std::packaged_task<decltype(func())()>(std::forward<F>(func));
auto future = task.get_future();
std::thread(std::move(task)).detach();
return std::move(future);
}
Note that this is slightly different from std::async where the returned std::future will when destructed actually block until the thread is finished.