I'm trying to learn async programming in C++. In Python, we have await, with which we can resume a function from that point, but in C++ future waits for the results and halts the next line of code. What if we don't want to get the result, but instead continue to the next line of code? How can I do this?
You can use std::future::wait_for to check if the task has completed execution, e.g.:
if (future.wait_for(100ms) == std::future_status::ready) {
// Result is ready.
} else {
// Do something else.
}
The Concurrency TS includes std::future::is_ready (may be included in C++20), which is non blocking. If it gets included in the standard, usage will be something like:
auto f = std::async(std::launch::async, my_func);
while (!f.is_ready()) {
/* Do other stuff. */
}
auto result = f.get();
/* Do stuff with result. */
Alternatively, the Concurrency TS also includes std::future::then, which I interpret can be used e.g. as:
auto f = std::async(std::launch::async, my_func)
.then([] (auto fut) {
auto result = fut.get();
/* Do stuff when result is ready. */
});
/* Do other stuff before result is ready. */
Also see: How to check if a std::thread is still running?
future waits for the results and halts the next line of code
This is only true when you invoke .get() or when the future is being destroyed. You can run multiple tasks in parallel with std::future:
std::future<int> f = std::async(std::launch::async, foo);
auto res0 = bar();
auto res1 = f.get();
In the example above, bar and foo will run in parallel.
If you want to attach asynchronous continuations to an existing future, currently you cannot do that with std::future.
boost::future supports non-blocking .then(...), .when_all(...), and .when_any(...) continuations. These are proposed for standardization in "Extensions for concurrency".
There's also a "Coroutines" TS that aims to introduce resumable functions and co_await/co_yield.
Unsurprisingly, boost also provides a coroutine library that can be used today to implement resumable functions.
Related
We started using the modern C++20 coroutines on our project recently. There is a list of coroutines referred to as Tasks in the Executor, which steps through them one by one resuming them. All of this is done on a single thread. Sometimes coroutines need not to be resumed until some predicate is satisfied. In some cases it may be satisfied by another coroutine, which makes suspending for later execution just fine.
Here are the types in use:
struct Task : std::coroutine_handle<task_promise_t> {
using promise_type = task_promise_t;
};
struct task_promise_t {
Task get_return_object() { return {Task::from_promise(*this)}; }
std::suspend_always initial_suspend() noexcept { return {}; }
std::suspend_always final_suspend() noexcept { return {}; }
void return_void() {}
void unhandled_exception() {}
};
struct Executor {
/* snip */
void enqueue_task(Task &&task) { tasks.push_back(task); }
void tick() {
while (!tasks.empty())
this->step();
}
void step() {
Task task = std::move(tasks.front());
tasks.pop_front();
task.resume();
if (!task.done())
tasks.push_back(task);
}
std::deque<Task> tasks;
/* snip */
}
Example of how I expect it to be used:
auto exec = Executor();
static bool global_predicate = false;
exec.enqueue_task([](Executor* exec) -> Task {
co_await WaitFor(/* bool(void) */ []() -> bool { return global_predicate; });
/* prerequisite satisfied, other logic goes here */
std::cout << "Hello, world" << std::endl;
}(&exec));
exec.step(); // no output, predicate false
exec.step(); // no output, predicate false
global_predicate = true;
exec.step(); // predicate true, "Hello, world!", coroutine is also done
I did manage to get the implementation going, this seems to work fine.
static bool example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
But I can't a good way to generalize and abstract it into it's own class. How would one go about it? I would expect to see that functionality in the standard library, but seeing how customizable the coroutines are I doubt there is a way to implement a one-size-fits-all solution.
The "await" style of coroutines is intended for doing asynchronous processing in a way that mirrors the synchronous equivalent. In sinchronous code, you might write:
int func(float f)
{
auto value = compute_stuff(f);
auto val2 = compute_more_stuff(value, 23);
return val2 + value;
}
If one or both of these functions is asychronous, you would rewrite it as follows (assuming the presence of appropriate co_await machinery):
task<int> func(float f)
{
auto value = compute_stuff(f);
auto val2 = co_await async_compute_more_stuff(value, 23);
co_return val2 + value;
}
It's structurally the same code except that in one case, func will halt halfway through until async_compute_more_stuff has finished its computation, then be resumed and return its value through Task<int>. The async nature of the code is as implicit as possible; it largely looks like synchronous code.
If you already have some extant async process, and you just want a function to get called when that process concludes, and there is no direct relationship between them, you don't need a coroutine. This code:
static atomic<bool> example_global_predicate;
auto coro = []() -> Task {
while (!example_global_predicate)
co_await std::suspend_always();
/* example_global_predicate is now true, do stuff */
co_return;
}();
Is not meaningfully different from this:
static atomic<bool> example_global_predicate;
register_polling_task([]() -> bool
{
if(!example_global_predicate)
return false;
/* example_global_predicate is now true, do stuff */
return true;
});
register_polling_task represents some global construct which will at regular intervals call your function until it returns true, at which point it assumes that it has done its job and removes the task. Your coroutine version might hide this global construct, but it still needs to be there because somebody has to wake the coroutine up.
Overall, this is not an async circumstance where using coroutines buys you anything in particular.
However, it could still be theoretically useful to attach coroutine resumption to a polling task. The most reasonable way to do this is to put the polling in a task outside of a coroutine. That is, coroutines shouldn't poll for the global state; that's someone else's job.
A coroutine would do something like co_await PollingTask(). This hands the coroutine_handle off to the system that polls the global state. When that global state enters the correct state, it will resume that handle. And when executing the co_await expression, it should also check the state then, so that if the state is already signaled, it should just not halt the coroutine's execution.
PollingTask() would return an awaitable that has all of this machinery built into it.
While working with the threaded model of C++11, I noticed that
std::packaged_task<int(int,int)> task([](int a, int b) { return a + b; });
auto f = task.get_future();
task(2,3);
std::cout << f.get() << '\n';
and
auto f = std::async(std::launch::async,
[](int a, int b) { return a + b; }, 2, 3);
std::cout << f.get() << '\n';
seem to do exactly the same thing. I understand that there could be a major difference if I ran std::async with std::launch::deferred, but is there one in this case?
What is the difference between these two approaches, and more importantly, in what use cases should I use one over the other?
Actually the example you just gave shows the differences if you use a rather long function, such as
//! sleeps for one second and returns 1
auto sleep = [](){
std::this_thread::sleep_for(std::chrono::seconds(1));
return 1;
};
Packaged task
A packaged_task won't start on it's own, you have to invoke it:
std::packaged_task<int()> task(sleep);
auto f = task.get_future();
task(); // invoke the function
// You have to wait until task returns. Since task calls sleep
// you will have to wait at least 1 second.
std::cout << "You can see this after 1 second\n";
// However, f.get() will be available, since task has already finished.
std::cout << f.get() << std::endl;
std::async
On the other hand, std::async with launch::async will try to run the task in a different thread:
auto f = std::async(std::launch::async, sleep);
std::cout << "You can see this immediately!\n";
// However, the value of the future will be available after sleep has finished
// so f.get() can block up to 1 second.
std::cout << f.get() << "This will be shown after a second!\n";
Drawback
But before you try to use async for everything, keep in mind that the returned future has a special shared state, which demands that future::~future blocks:
std::async(do_work1); // ~future blocks
std::async(do_work2); // ~future blocks
/* output: (assuming that do_work* log their progress)
do_work1() started;
do_work1() stopped;
do_work2() started;
do_work2() stopped;
*/
So if you want real asynchronous you need to keep the returned future, or if you don't care for the result if the circumstances change:
{
auto pizza = std::async(get_pizza);
/* ... */
if(need_to_go)
return; // ~future will block
else
eat(pizza.get());
}
For more information on this, see Herb Sutter's article async and ~future, which describes the problem, and Scott Meyer's std::futures from std::async aren't special, which describes the insights. Also do note that this behavior was specified in C++14 and up, but also commonly implemented in C++11.
Further differences
By using std::async you cannot run your task on a specific thread anymore, where std::packaged_task can be moved to other threads.
std::packaged_task<int(int,int)> task(...);
auto f = task.get_future();
std::thread myThread(std::move(task),2,3);
std::cout << f.get() << "\n";
Also, a packaged_task needs to be invoked before you call f.get(), otherwise you program will freeze as the future will never become ready:
std::packaged_task<int(int,int)> task(...);
auto f = task.get_future();
std::cout << f.get() << "\n"; // oops!
task(2,3);
TL;DR
Use std::async if you want some things done and don't really care when they're done, and std::packaged_task if you want to wrap up things in order to move them to other threads or call them later. Or, to quote Christian:
In the end a std::packaged_task is just a lower level feature for implementing std::async (which is why it can do more than std::async if used together with other lower level stuff, like std::thread). Simply spoken a std::packaged_task is a std::function linked to a std::future and std::async wraps and calls a std::packaged_task (possibly in a different thread).
TL;DR
std::packaged_task allows us to get the std::future "bounded" to some callable, and then control when and where this callable will be executed without the need of that future object.
std::async enables the first, but not the second. Namely, it allows us to get the future for some callable, but then, we have no control of its execution without that future object.
Practical example
Here is a practical example of a problem that can be solved with std::packaged_task but not with std::async.
Consider you want to implement a thread pool. It consists of a fixed number of worker threads and a shared queue. But shared queue of what? std::packaged_task is quite suitable here.
template <typename T>
class ThreadPool {
public:
using task_type = std::packaged_task<T()>;
std::future<T> enqueue(task_type task) {
// could be passed by reference as well...
// ...or implemented with perfect forwarding
std::future<T> res = task.get_future();
{ std::lock_guard<std::mutex> lock(mutex_);
tasks_.push(std::move(task));
}
cv_.notify_one();
return res;
}
void worker() {
while (true) { // supposed to be run forever for simplicity
task_type task;
{ std::unique_lock<std::mutex> lock(mutex_);
cv_.wait(lock, [this]{ return !this->tasks_.empty(); });
task = std::move(tasks_.top());
tasks_.pop();
}
task();
}
}
... // constructors, destructor,...
private:
std::vector<std::thread> workers_;
std::queue<task_type> tasks_;
std::mutex mutex_;
std::condition_variable cv_;
};
Such functionality cannot be implemented with std::async. We need to return an std::future from enqueue(). If we called std::async there (even with deferred policy) and return std::future, then we would have no option how to execute the callable in worker(). Note that you cannot create multiple futures for the same shared state (futures are non-copyable).
Packaged Task vs async
p> Packaged task holds a task [function or function object] and future/promise pair. When the task executes a return statement, it causes set_value(..) on the packaged_task's promise.
a> Given Future, promise and package task we can create simple tasks without worrying too much about threads [thread is just something we give to run a task].
However we need to consider how many threads to use or whether a task is best run on the current thread or on another etc.Such descisions can be handled by a thread launcher called async(), that decides whether to create a new a thread or recycle an old one or simply run the task on the current thread. It returns a future .
"The class template std::packaged_task wraps any callable target
(function, lambda expression, bind expression, or another function
object) so that it can be invoked asynchronously. Its return value or
exception thrown is stored in a shared state which can be accessed
through std::future objects."
"The template function async runs the function f asynchronously
(potentially in a separate thread) and returns a std::future that will
eventually hold the result of that function call."
What is the difference between the two statements below in terms of execution?
async([]() { ... });
thread([]() { ... }).detach();
std::async ([]() { ... }); // (1)
std::thread ([]() { ... }).detach (); // (2)
Most often when std::async is being discussed the first thing noted is that it's broken, the name implies something which doesn't hold when the returned value isn't honored (assigned to a variable to be destructed at the end of the current scope).
In this case the brokenness of std::async is exactly what is going to result in a huge difference between (1) and (2); one will block, the other won't.
Why does std::async block in this context?
The return-value of std::async is a std::future which has a blocking destructor that must execute before the code continues.
In an example as the below g won't execute until f has finished, simply because the unused return value of (3) can't be destroyed until all work is done in the relevant statement.
std::async (f); // (3)
std::async (g); // (4)
What is the purpose of std::thread (...).detach ()?
When detaching from a std::thread we are simply saying; "I don't care about this thread handle anymore, please just execute the damn thing."
To continue with an example similar to the previous one (about std::async) the difference is notably clear; both f and g will execute simultaneously.
std::thread (f).detach ();
std::thread (g).detach ();
I know a good answer was given to your question but if we were to change your question a little something interesting would occur.
Imagine you kept the future returned by the async and didn't detach the thread but instead made a variable for it like this,
Asynchronous code
auto fut=std::async([]() { ... });
std::thread th([]() { ... });
Now you have the setup to what makes these 2 constructs different.
th.join()//you're here until the thread function returns
fut.wait_for(std::chrono::seconds(1)); //wait for 1 sec then continue.
A thread is an all or nothing thing when joining it where as an async can be checked and you can go do other stuff.
wait_for actually returns a status so you can do things like this.
int numOfDots = 0;
//While not ready after waiting 1 sec do some stuff and then check again
while(fut.wait_for(std::chrono::seconds(1)) != std::future_status::ready)
{
(numOfDots++)%=20;
//Print status to the user you're still working on it.
std::cout << "Working on it" <<std::string(numOfDots,'.')<<"\r"<<std::flush();
}
std::cout << "Thanks for waiting!\nHere's your answer: " << fut.get() <<std::endl();
async returns a future object, detach does not. All detach does is allow the execution to continue independently. In order to achieve a similar effect as async, you must use join. For example:
{
std::async(std::launch::async, []{ f(); });
std::async(std::launch::async, []{ g(); }); // does not run until f() completes
}
{
thread1.join();
thread2.join();
}
I just read the article 'Futures Done Right', and the main thing that c++11 promises are lacking seems to be that creating composite futures from existing ones
I'm looking right now at the documentation of boost::wait_for_any
but consider the following example:
int calculate_the_answer_to_life_the_universe_and_everything()
{
return 42;
}
int calculate_the_answer_to_death_and_anything_in_between()
{
return 121;
}
boost::packaged_task<int> pt(calculate_the_answer_to_life_the_universe_and_everything);
boost:: future<int> fi=pt.get_future();
boost::packaged_task<int> pt2(calculate_the_answer_to_death_and_anything_in_between);
boost:: future<int> fi2=pt2.get_future();
....
int calculate_the_oscillation_of_barzoom(boost::future<int>& a, boost::future<int>& b)
{
boost::wait_for_all(a,b);
return a.get() + b.get();
}
boost::packaged_task<int> pt_composite(boost::bind(calculate_the_oscillation_of_barzoom, fi , fi2));
boost:: future<int> fi_composite=pt_composite.get_future();
What is wrong with this approach to composability? is this a valid way to achieve composability? do we need some elegant syntactic edulcorant over this pattern?
when_any and when_all are perfectly valid ways to compose futures. They both correspond to parallel composition, where the composite operation waits for either one or all the composed operations.
We also need sequential composition (which is not in Boost.Thread). This could be, for example, a future<T>::then function that allows you to queue up an operation that uses the future's value and runs when the future is ready. It is possible to implement this yourself, but with an efficiency tradeoff. Herb Sutter talks about this in his recent Channel9 video.
N3428 is a draft proposal for adding these features (and more) to the C++ standard library. They are all library features and don't add any new syntax to the language. Additionally, N3328 is a proposal to add syntax for resumable functions (like using async/await in C#) which will use future<T>::then internally.
Points for the use of the word edulcorant. :)
The problem with your sample code is that you package everything up into tasks, but you never schedule those tasks for execution!
int calculate_the_answer_to_life() { ... }
int calculate_the_answer_to_death() { ... }
std::packaged_task<int()> pt(calculate_the_answer_to_life);
std::future<int> fi = pt.get_future();
std::packaged_task<int()> pt2(calculate_the_answer_to_death);
std::future<int> fi2 = pt2.get_future();
int calculate_barzoom(std::future<int>& a, std::future<int>& b)
{
boost::wait_for_all(a, b);
return a.get() + b.get();
}
std::packaged_task<int()> pt_composite([]{ return calculate_barzoom(fi, fi2); });
std::future<int> fi_composite = pt_composite.get_future();
If at this point I write
pt_composite();
int result = fi_composite.get();
my program will block forever. It will never complete, because pt_composite is blocked on calculate_barzoom, which is blocked on wait_for_all, which is blocked on both fi and fi2, neither of which will ever complete until somebody executes pt or pt2 respectively. And nobody will ever execute them, because my program is blocked!
You probably meant me to write something like this:
std::async(pt);
std::async(pt2);
std::async(pt_composite);
int result = fi_composite.get();
This will work. But it's extremely inefficient — we spawn three worker threads (via three calls to async), in order to perform two threads' worth of work. That third thread — the one running pt_composite — will be spawned immediately, and then just sit there asleep until pt and pt2 have finished running. That's better than spinning, but it's significantly worse than not existing: it means that our thread pool has one fewer worker than it ought to have. In a plausible thread-pool implementation with only one thread per CPU core, and a lot of tasks coming in all the time, that means that we've got one CPU core just sitting idle, because the worker thread who was meant to be running on that core is currently blocked inside wait_for_all.
What we want to do is declare our intentions declaratively:
int calculate_the_answer_to_life() { ... }
int calculate_the_answer_to_death() { ... }
std::future<int> fi = std::async(calculate_the_answer_to_life);
std::future<int> fi2 = std::async(calculate_the_answer_to_death);
std::future<int> fi_composite = std::when_all(fi, fi2).then([](auto a, auto b) {
assert(a.is_ready() && b.is_ready());
return a.get() + b.get();
});
int result = fi_composite.get();
and then have the library and the scheduler work together to Do The Right Thing: don't spawn any worker thread that can't immediately proceed with its task. If the end-user has to write even a single line of code that explicitly sleeps, waits, or blocks, some performance is definitely being lost.
In other words: Spawn no worker thread before its time.
Obviously it's possible to do all this in standard C++, without library support; that's how the library itself is implemented! But it's a huge pain to implement from scratch, with many subtle pitfalls; so that's why it's a good thing that library support seems to be coming soon.
The ISO proposal N3428 mentioned in Roshan Shariff's answer has been updated as N3857, and N3865 provides even more convenience functions.
It seems that unless you call std::async a std::future will never be set to any other state than future_status::deferred unless you call get or wait on the future. wait_for & wait_until will continue to not block and return future_status::deferred even if the task has already run and stored the result.
Here's an example:
#include <future>
void main()
{
auto func = []() { return 5; };
auto asyncFuture = std::async(std::launch::async, func);
auto status = asyncFuture.wait_for(std::chrono::seconds(0)); // timeout (1)
auto deferredFuture = std::async(std::launch::deferred, func);
status = deferredFuture.wait_for(std::chrono::seconds(0)); // deferred (2)
std::packaged_task<int()> task(func);
auto packagedTaskFuture = task.get_future();
status = packagedTaskFuture.wait_for(std::chrono::seconds(0)); // deferred (2)
task();
status = packagedTaskFuture.wait_for(std::chrono::seconds(0)); // deferred (2)
packagedTaskFuture.wait();
status = packagedTaskFuture.wait_for(std::chrono::seconds(0)); // ready (0)
}
I don't have the current C++11 standard, but the draft standard in 30.6.9 says that when a packaged_task is run it should store the result in the future's shared state. It's not very clear whether that includes setting the expected wait_until/wait_for behavior or not.
There were previously issues with VS11 behavior in this area with respect to async calls: http://social.msdn.microsoft.com/Forums/hu/parallelcppnative/thread/4394f2c1-0404-40df-869b-f4fc36fc035c
Additionally it seems like other compilers have problems in this area: C++ 11 future_status::deferred not working
Anyone that may know the standard better: Is this expected behavior or is there an issue with the implemenation in VS11?
Updates: I somehow missed the report for this: http://connect.microsoft.com/VisualStudio/feedback/details/761829/c-11-unexpected-behavior-for-std-future-wait-for-and-std-packaged-task
This is an issue with VS2012. It's not the only issue either --- their implementation of the C++11 thread library has several bugs. I wrote about a few on my blog.