I am implementing a threadpool that has a push_back method on callable object. However I am getting error on moving a packaged task into a function object using lambda trick.
class Threadpool {
public:
// ...
::std::deque <::std::function<void()>> _work_queue;
::std::mutex _work_queue_mutex;
::std::condition_variable _worker_signal;
template <typename CallableT>
::std::future<::std::result_of_t<CallableT()>> push_back(CallableT&&);
}
template<typename CallableT>
::std::future<::std::result_of_t<CallableT()>> Threadpool::push_back(CallableT&& callable) {
::std::packaged_task<::std::result_of_t<CallableT()>()> task (::std::move(callable));
auto fu = task.get_future();
{
::std::unique_lock<::std::mutex> locker(_work_queue_mutex);
// COMPILE ERROR
_work_queue.emplace_back([task=::std::move(task)] () { task(); })
}
_worker_signal.notify_one();
return fu;
}
Threadpool pool;
pool.emplace_back( []() { ::std::cout << "hello\n"; } );
The compiler complains about the emplace_back by error: no match for call to '(const std::packaged_task<void()>) ()' _work_queue.emplace_back([task=::std::move(task)]() { task(); }); I don't understand what's going wrong since as far as I know packaged_task is only movable and I am capturing the task by move.
There are two issues with your example.
Indeed, std::packaged_task is only movable, so [task=std::move(task)] is correct. But on top of that std::packaged_task::operator() requires not-const object: https://en.cppreference.com/w/cpp/thread/packaged_task/operator()
So the lambda must be defined as mutable to allow the usage of task():
[task=std::move(task)] () mutable { task(); };
But even so the lambda object is only movable and not copyable, while std::function requires a copyable object: https://en.cppreference.com/w/cpp/utility/functional/function
So one of the solutions, is to wrap the packaged_task in a copyable smart pointer as follows:
#include <mutex>
#include <deque>
#include <functional>
#include <condition_variable>
#include <future>
#include <iostream>
#include <type_traits>
class Threadpool
{
public:
// ...
std::deque <std::function<void()>> _work_queue;
std::mutex _work_queue_mutex;
std::condition_variable _worker_signal;
template <typename CallableT>
std::future<std::result_of_t<CallableT()>> push_back(CallableT&&);
};
template<typename CallableT>
std::future<std::result_of_t<CallableT()>> Threadpool::push_back(CallableT&& callable)
{
auto task = std::make_shared<std::packaged_task<std::result_of_t<CallableT()>()>>( std::move(callable) );
auto fu = task->get_future();
{
std::unique_lock<std::mutex> locker(_work_queue_mutex);
_work_queue.emplace_back([task]() { (*task)(); });
}
_worker_signal.notify_one();
return fu;
};
int main()
{
Threadpool pool;
pool.push_back( []() { std::cout << "hello\n"; } );
}
Demo: https://gcc.godbolt.org/z/aEfvo7Mhz
Related
Considering the following code:
#include <iostream>
#include <thread>
#include <chrono>
int main()
{
std::thread t;
{
auto my_lambda = []{
int idx = 0;
while (true) {
std::this_thread::sleep_for (std::chrono::seconds(1));
std::cout << idx ++ << std::endl;
}
};
t = std::thread(my_lambda);
}
t.join();
return 0;
}
Is it safe that the thread runs a lambda function that goes out of scope?
I saw that the constructor of std::thread takes an universal reference for the input function Function&& f and that lambdas are translated into structs. So if the instance of the struct is instantiated inside the scope, the thread will be running the operator() of a dangling reference.
{
struct lambda_translated { void operator()(){ ... } };
lambda_translated instance;
t = std::thread(instance);
}
However I'm not sure that my reasoning is correct.
Side question: does the behavior change if I declare the lambda as an R-value inside the std::thread constructor:
#include <iostream>
#include <thread>
#include <chrono>
int main()
{
std::thread t;
{
t = std::thread([]{
int idx = 0;
while (true) {
std::this_thread::sleep_for (std::chrono::seconds(1));
std::cout << idx ++ << std::endl;
}
});
}
t.join();
return 0;
}
As a summary of the comments:
The lambda is copied (or moved if declared in-place), so you won't have problems.
You have to worry about the captures: do not capture by reference objects that can go out of the scope, or if you pass objects that can be deleted during thread execution (even if copied, think about a raw pointer to an object).
As an extension, same applies if you use std::bind to pass a method and the object goes out of scope or it is deleted.
When I use mutual parameter,'some mutex',in demo of shared data in multi-threading of C++,the two errors occur,and their codes are
C2672:
'invoke': no matching overloaded function found shared_data
C2893:
Failed to specialize function template 'unknown-type std::invoke(_Callable &&) noexcept(<expr>)'
Then I looked the error codes in Microsoft VS Doc,it reads:"To fix this issue, make the template parameter member accessible where it is evaluated.",So what does the sentence mean?Finally,I guess there must be some wrong with the function invoke .
Here's the code:
#include <list>
#include <mutex>
#include <algorithm>
#include<thread>
std::list<int> some_list;
std::mutex some_mutex;
void add_to_list(int new_value)
{
std::lock_guard<std::mutex> guard(some_mutex);
some_list.push_back(new_value);
}
bool list_contains(int value_to_find)
{
std::lock_guard<std::mutex> guard(some_mutex);
return std::find(some_list.begin(), some_list.end(), value_to_find) != some_list.end();
}
int main() {
using std::thread;
thread t1(add_to_list);
thread t2(list_contains);
t1.join();
t2.join();
return 0;
}
So confused that I want to ask for help.
My comment written out as a program :
#include <cassert>
#include <list>
#include <mutex>
#include <algorithm>
#include <future>
// avoid global variable, group related functions in a class
//std::list<int> some_list;
//std::mutex some_mutex;
class my_async_list final
{
public:
void add(const int new_value) // new_value should not change so const
{
// scoped_lock instead of lock_guard https://stackoverflow.com/questions/43019598/stdlock-guard-or-stdscoped-lock
std::scoped_lock<std::mutex> lock{ m_mtx };
m_list.push_back(new_value);
}
bool contains(int value_to_find) const // contains should not change datastructure so it is a const function
{
std::scoped_lock<std::mutex> lock{ m_mtx };
return std::find(m_list.begin(), m_list.end(), value_to_find) != m_list.end();
}
private:
mutable std::mutex m_mtx; // so it can be used in const functions
std::list<int> m_list;
};
int main()
{
my_async_list my_list;
// use lambda functions to launch async function call
auto add_future = std::async(std::launch::async, [&my_list]
{
my_list.add(1);
});
auto contains_future = std::async(std::launch::async, [&my_list]
{
return my_list.contains(1);
});
add_future.get(); // wait until add is done.
bool contains_one = contains_future.get();
assert(contains_one);
return 0;
}
I am working on a processing framework where callbacks are registered to events and to ensure that no callback is invoked on an object, which has been deleted, I would like to use weak capture rather than capture by reference. It was no problem to make this work using C++14 and shared_from_this(), but how is this correctly achieved using C++17 and weak_from_this().
The example below prints nothing when C++17 is used. I am using g++ 6.3.0-18
#define CXX17 // When this is defined, nothing is printed
#ifdef CXX17
# include <experimental/memory>
# include <experimental/functional>
template <typename T>
using enable_shared_from_this = std::experimental::enable_shared_from_this<T>;
#else
# include <memory>
# include <functional>
template <typename T>
using enable_shared_from_this = std::enable_shared_from_this<T>;
#endif
#include <thread>
#include <mutex>
#include <condition_variable>
#include <atomic>
#include <iostream>
struct A : enable_shared_from_this<A> {
int a;
A() : a(7) {}
auto getptr() {
#ifdef CXX17
return this->weak_from_this();
#else
auto sptr = shared_from_this();
auto wptr = std::weak_ptr<decltype(sptr)::element_type>(sptr);
sptr.reset(); // Drop strong referencing
return wptr;
#endif
}
};
std::condition_variable condition;
std::mutex mutex;
std::atomic<bool> start0{false};
std::atomic<bool> start1{false};
std::shared_ptr<A> g_a;
static void thread_func0() {
auto w_a = g_a->getptr();
std::unique_lock<std::mutex> lock {mutex};
condition.wait(lock, [&]() {
return start0.load();
});
std::this_thread::sleep_for(std::chrono::microseconds(10));
if (auto t = w_a.lock()) {
std::cout << t->a << std::endl;
}
}
static void thread_func1() {
std::unique_lock<std::mutex> lock {mutex};
condition.wait(lock, [&]() {
return start1.load();
});
std::this_thread::sleep_for(std::chrono::microseconds(10000));
g_a = nullptr;
}
int main() {
g_a = std::make_shared<A>();
std::thread thread0(thread_func0);
std::thread thread1(thread_func1);
start0 = true;
start1 = true;
condition.notify_all();
thread0.join();
thread1.join();
return 0;
}
Here's a way more reduced example:
#include <experimental/memory>
#include <iostream>
template <typename T>
using enable_shared_from_this = std::experimental::enable_shared_from_this<T>;
struct A : enable_shared_from_this<A> {
int a;
A() : a(7) {}
};
int main() {
auto sp = std::make_shared<A>();
auto wp = sp->weak_from_this();
if (auto s = wp.lock()) {
std::cout << s->a << std::endl;
}
}
This prints nothing. Why? The reason is ultimately the reason why it's std::enable_shared_from_this and not some other type that you yourself can provide: the shared_ptr class needs to opt-in to this functionality. The new functionality is experimental, so std::shared_ptr was not opting in - so the underlying weak_ptr was never initialized. It just doesn't happen, so wp is always an "empty" weak_ptr here.
On the other hand, std::experimental::shared_ptr does opt-in to this functionality. You need to use the shared_ptr corresponding to your enable_shared_from_this - which is std::experimental::shared_ptr.
There's no std::experimental::make_shared (or at least, as far as I could find), but the opt-in mechanism isn't based on that anyway - it's just based on any shared_ptr construction. So if you change:
auto sp = std::make_shared<A>();
to:
auto sp = std::experimental::shared_ptr<A>(new A);
Then the opt-in mechanism matches the shared_ptr type and does the right thing, you get a valid weak_ptr (a std::experimental::weak_ptr), lock() gives you shared ownership of the underlying A, and the program prints 7.
How it is possible that this code is not working ?
I would like MyThread::run to work with any kind of parameters, and the parameters being passed by reference and not by value.
http://ideone.com/DUJu5M
#include <iostream>
#include <future>
#include <string>
class MyThread {
std::future<void> future;
public:
template<class... Args>
MyThread(Args&&... myArgs) :
future(std::async(std::launch::async, &MyThread::run<Args&&...>, this, std::forward<Args>(myArgs)...))
{}
template<class... Args>
void run(Args&&... myArgs) {}
};
int main() {
std::string x;
MyThread thread(x); // Not working
MyThread thread(10); // Working
return 0;
}
You can use std::ref to pass a reference_wrapper. It will be unpacked automatically by standard library features like std::bind/std::thread or std::async.
int main() {
std::string x;
MyThread thread(std::ref(x)); // Not working
MyThread thread2(10); // Working
return 0;
}
demo
I'm trying to wrap std::packaged_task inside another class in order to be used together with a task scheduler.
At the moment I got it all working except std::future support. To get std::future support I figured out I need to use std::packaged_task for the get_future() function that it provides.
I've been trying whole day all sorts of ways to get this to work, but I seem to be unable to properly declare and initialise the packaged_task using the return value from a std::bind. I have tried to decipher the implementations of all the related libstdc++ functions such as std::async, std::future, std::thread etc but with no luck.
The following code is the implementation of both the not working version and the working one. To get it to work uncomment the two /* --- WORKS*/ and comment the other related line.
#include <vector>
#include <deque>
#include <memory>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <future>
#include <iostream>
#include <chrono>
#include <functional>
#include <windows.h>
class task
{
private:
struct task_implementation_base
{
virtual void execute() = 0;
};
template <class callable>
struct task_implementation : public task_implementation_base
{
task_implementation(callable&& f) : /*m_task(std::forward<callable>(f)) WORKS*/m_task(f) { }
void execute() { m_task(); }
//callable m_task; // WORKS
std::packaged_task<typename result_of<callable>::type> m_task;
};
template <class callable>
std::shared_ptr<task_implementation<callable>> make_routine(callable&& f)
{
return std::make_shared<task_implementation<callable>>(std::forward<callable>(f));
}
public:
template <class callable, class... arguments>
task(callable&& f, arguments&&... args) : m_function(make_routine(std::bind(std::forward<callable>(f), std::forward<arguments>(args)...))) {}
void operator()() { run(); }
void run() { m_function->execute(); }
private:
std::shared_ptr<task_implementation_base> m_function;
};
int testint(int i)
{
std::cout << "test6" << " :: ran from thread " << std::this_thread::get_id() << "\n";
fflush(stdout);
return i+100;
}
void test(const char* text)
{
std::cout << text << " :: ran from thread " << std::this_thread::get_id() << "\n";
fflush(stdout);
}
class testclass
{
public:
void print1() { test("test3"); }
void print2() { test("test4"); }
void print3(const char* text) { test(text); }
};
int main()
{
testclass testclass1;
testclass* testclass2 = new testclass;
task test1(test, "test1");
task test2([]() { test("test2"); });
task test3(&testclass::print1, &testclass1);
task test4(&testclass::print2, &*testclass2);
task test5(&testclass::print3, &*testclass2, "test5");
task test6(&testint, 1);
test1();
test2();
test3();
test4();
test5();
test6();
Sleep(2000);
return 0;
}
I'm thinking the problem is typename result_of<callable>::type. I'm guessing it doesn't properly evaluates to the return type of the callable function.
I'm using c++ (Built by MinGW-builds project) 4.8.0 20121225 (experimental) on a Windows 8 64bit. I'm suspecting the errors are irrelevant since I guess I'm just simply trying to get this work the wrong way but here is a pastebin for the errors anyway: errors
std::packaged_task not only takes the result type of the invoked function as a template argument but also the types of the arguments you are passing to the to be invoked function.
You can define them as follows:
// somewhere
int foo(bool, int);
// somewhere else
std::packaged_task<int(bool, int)> p(foo);
To fix your code you need to add two empty parenthesis pairs. What I explained above also applies to std::result_of.
std::packaged_task<typename std::result_of<callable()>::type()> m_task;
It is only response to main topic question. "How to implement"
Example short implementation:
template <typename Signature> /// <---- 1
class Task;
template <typename Res, typename... ArgTypes>
class Task<Res(ArgTypes...)> /// <---- 2
{
public:
template <typename Function>
explicit Task(Function&& callback)
: _task{std::forward<Function>(callback)}
{ }
void execute(ArgTypes... args) noexcept(false)
{
//...
_task(std::forward<ArgTypes>(args)...);
}
private:
std::packaged_task<Res(ArgTypes...)> _task;
};
Not sure why step 1 & 2 are required but I did the same as in lib implementation. Maybe someone could extend this response.