If I want to edit each element of a vector, I can use for_each() to loop through the elements. The problem now is, how can I separate this task into two threads?
I've tried the way below, declaring a thread with for_each(), but I'm getting errors for that.
For example, I'd like to add 1 to each element of the vector. By using the threads, it seems like I'm missing something that the compiler does not like.
#include <iostream>
#include <algorithm>
#include <vector>
#include <thread>
using namespace std;
int main()
{
std::vector<int> nums; //declare a vector
nums.push_back(1);
nums.push_back(2);
nums.push_back(3);
nums.push_back(4); //push each element to the vector
size_t i = (nums.size()/2); //I want to separate the task into two thread
std::thread t1(std::for_each(nums.begin(),nums.begin()+i,[](int& num){
num++;
}));
std::thread t2(std::for_each(nums.begin()+i,nums.end(),[](int& num){
num++;
}));
t1.join();
t2.join();
return 0;
}
I'm getting these two errors:
Failed to specialize function template 'unknown-type std::invoke(_Callable &&) noexcept(<expr>)'
and
invoke': no matching overloaded function found
If I cannot do the threads in this way, what's the right way?
From C++17 onwards, you don't have to create your own threads in order to parallelise your code. Instead, you can pass the appropriate execution_policy to std::for_each:
std::for_each (std::execution::par_unseq, nums.begin (), nums.end (), [] (int& num) { num++; });
Since there are no data races in your code, this is safe.
Paul's got a better approach, but if you don't have C++17 available, and to explain what went wrong, in
std::thread t1(std::for_each(nums.begin(),nums.begin()+i,[](int& num){
num++;
}));
the
std::for_each(nums.begin(),nums.begin()+i,[](int& num){
num++;
})
is a function call that returns a value almost usable in the thread constructor, making the error message much harder to interpret than it otherwise could have been1. You didn't want to call this function. You wanted the function run in the thread, so you need to pass in the function. And that means you somehow need to provide the parameters to the function to the thread.
The easiest fix that I can see is to insert another lambda that calls for_each with the correct range
std::thread t1([i, &nums]()
{
std::for_each(nums.begin(),
nums.begin()+i,
[](int& num)
{
num++;
});
});
1 Just ran a test with a few different compilers and I'm slightly wrong. The error message if you provide something like int func() that returns a type that clearly cannot be callable is just as messy. Nearly identical in fact.
Related
I have a priority queue and the compare function references a value accessed by multiple threads. So it has to be protected by a mutex. Except I don't know when this compare function is ran. Is it ran when I push a value or when I pop a value? Example code below.
#include <iostream>
#include <queue>
#include <mutex>
using namespace std;
int main()
{
int compare = 7;
mutex compare_m;
auto cmp = [&](int a, int b) {return abs(compare - a)>=abs(compare-b);};
priority_queue<int, vector<int>, decltype(cmp)> x(cmp);
mutex x_m;
//in thread
{
scoped_lock m1(x_m);
//do I need this?
scoped_lock m(compare_m);
x.push(6);
}
//in thread
{
scoped_lock m1(x_m);
//do I need this?
scoped_lock m(compare_m);
x.pop();
}
}
To answer the question, if it is not documented anything can happen (and then we cannot then reason about when comparator is invoked).
If we take a look into cppreference, push is defined in terms of push_heap, which then reorganizes the elements into a heap. Given it then needs to reorganize, we can reason that it invokes the comparator. A similar situation happens with pop, that invokes pop_heap, which again modifies the underlying heap. So again, invoking comparator.
So the above implies you need a critical section on both (however please notice the comments regarding whether it is actually safe to change the behaviour of comparison function while the pq contains elements).
I want to use C++11 std::thread. I am writing a code to parse JSON files and the parsing is not dependent on each other. I have 900 files and to increase the speed of execution I want to create as many threads and make the execution parallel.
My problem is at the moment the function I want to pass to the std::thread is a function of a class and it returns the parsed object as a JSON object from which I create a std::vector.
I googled 'returning values from threads' and I am getting the tutorials of using async, is there no other way other than async? I went through the async tutorials I am not able to understand how it's launching new threads.
Following is my code:
for (auto it = jsonFilesList.begin(); it != jsonFilesList.end(); it++) {
std::ifstream jsonFile;
jsonFile.open(*it);
jf = json::parse(jsonFile);
jsonFile.close();
jsonObjects.push_back(jf);
}
I want to pass the function parse(jsonFile) to my threads numbering the count of total JSON files, and return the json object, which I will use to create a std::vector of jsonObjects.
How can I do this?
Using std::async seems like a good approach, since it return a std::future from which you can get the returned value.
Here is a short example that you could use as an idea to get going.
#include <iostream>
#include <future>
#include <vector>
int doWork(int i) {
return i*2;
}
int main()
{
std::vector<int> results;
std::vector<std::future<int>> work;
for (int i = 0; i < 10; ++i) {
work.emplace_back(std::async(std::launch::async, &doWork, i));
}
// do some other stuff, or nothing
// before getting the results from all the futures
for (auto& future : work) {
results.emplace_back(future.get());
}
for(auto i : results) {
std::cout << i << '\n';
}
return 0;
}
std::future also has member functions like valid, wait, wait_for and wait_until if you want more flexibility in how you retrieve the value.
I have a class called MatrixAlt and i'm trying to multi thread a function to do some work on that matrix.
My general method worked when I just implemented it in a couple of functions. But when I try to bring it into the class methods, I get an error.
The problematic line (or where it highlights anyway) is 4 lines from the end and the error message is in the comments just above it.
#include <vector>
#include <future>
#include <thread>
class MatrixAlt
{
public:
MatrixAlt();
// initilaise the matrix to constant value for each entry
void function01(size_t maxThreads);
void function02(size_t threadIndex);
};
MatrixAlt::MatrixAlt()
{
}
void MatrixAlt::function02(size_t threadIndex)
{
// do some stuff
return;
}
void MatrixAlt::function01(size_t maxThreads)
{
// To control async threads and their results
std::vector<std::future<bool>> threadsIssued;
// now loop through all the threads and orchestrate the work to be done
for (size_t threadIndex = 0; threadIndex < maxThreads; ++threadIndex)
{
// line 42 gives error:
// 'MatrixAlt::function02': non-standard syntax; use '&' to create a pointer to member
// 'std::async': no matching overloaded function found
threadsIssued.push_back(std::async(function02, threadIndex));
}
return;
}
Your first problem is solved like this
threadsIssued.push_back(std::async(&MatrixAlt::function02, this, threadIndex));
You need to specify the exact class::function and take its address and which instance of the class your doing it for, and then the parameters.
The second problem which you haven't see yet is this line
std::vector<std::future<bool>> threadsIssued;
All those futures will be lost in scope exit, like tears in rain. Time to destroy.
Freely after Blade runner.
All those moments will be lost in time, like tears in rain. Time to
die.
Whenever you have a member function in C++, that function takes the object itself as an implicit first argument. So you need to pass the object as well, but even then, it can't be called with the same syntax as a normal function that takes the object.
The simplest way to setup an asynchronous job in C++ is typically just to use lambdas. They've very clear and explicit. So, for example, you could change your call to:
threadsIssued.push_back(std::async([this] (size_t t) { this->function02(t);}, threadIndex));
This lambda is explicitly capturing the this pointer, which tells us that all of the function02 calls will be called on the same object that the calling function01 is called on.
In addition to being correct, and explicit, this also helps highlight an important point: all of the function02 objects will be running with mutable access to the same MatrixAlt object. This is very dangerous, so you need to make sure that function02 is thread safe, one way or another (usually easy if its conceptually const, otherwise perhaps need a mutex, or something else).
I am trying to pass my class instance into threads and the return the processed objects from threads. I've googled about C++ multithreading, and found that std::promising can be helpful.
However, I am stuck at the very beginning. Here is my code:
void callerFunc()
{
//...
std::promise<DataWareHouse> data_chunks;
// DataWareHouse is my customized class
//data_chunks has a vector<vector<double>> member variable
std::thread(&run_thread,data_chunks);
// ............
}
void run_thread(std::promise<DataWareHouse> data_chunks)
{
// ...
vector<vector<double>> results;
// ...
data_chunks.set_value(results);
}
The above code generates an error:
`error C2248: 'std::promise<_Ty>::promise' : cannot access private member declared in class 'std::promise<_Ty>'`
May I know what am I wrong and how to fix it?
Many thanks. :-)
Your first problem is that you are using std::thread -- std::thread is a low level class which you should build higher abstractions up on. Threading is newly standardized in C++ in C++11, and all of the rough parts are not filed off yet.
There are three different patterns for using threading in C++11 that might be useful to you.
First, std::async. Second, std::thread mixed with std::packaged_task. And third, dealing with std::thread and std::promise in the raw.
I'll illustrate the third, which is the lowest level and most dangerous, because that is what you asked for. I would advise looking at the first two options.
#include <future>
#include <vector>
#include <iostream>
typedef std::vector<double> DataWareHouse;
void run_thread(std::promise<DataWareHouse> data_chunks)
{
DataWareHouse results;
results.push_back( 3.14159 );
data_chunks.set_value(results);
}
std::future<DataWareHouse> do_async_work()
{
std::promise<DataWareHouse> data_chunks;
std::future<DataWareHouse> retval = data_chunks.get_future();
// DataWareHouse is my customized class
//data_chunks has a vector<vector<double>> member variable
std::thread t = std::thread(&run_thread,std::move(data_chunks));
t.detach(); // do this or seg fault
return retval;
}
int main() {
std::future<DataWareHouse> result = do_async_work();
DataWareHouse vec = result.get(); // block and get the data
for (double d: vec) {
std::cout << d << "\n";
}
}
Live example
With std::async, you'd have a function returning DataWareHouse, and it would return a std::future<DataWareHouse> directly.
With std::packaged_task<>, it would take your run_thread and turn it into a packaged_task that can be executed, and a std::future extracted from it.
std::promise<> is not copyable, and in calling run_thread() you are implicitly trying to invoke the copy constructor. The error message is telling you that you cannot use the copy constructor since it is marked private.
You need to pass a promise by reference (std::promise<DataWareHouse> &). This is safe if callerFunc() is guaranteed not to return until run_thread() is finished with the object (otherwise you will be using a reference to a destroyed stack-allocated object, and I don't have to explain why that's bad).
You're trying to pass the promise to the thread by value; but you need to pass by reference to get the results back to the caller's promise. std::promise is uncopyable, to prevent this mistake.
std::thread(&run_thread,std::ref(data_chunks));
^^^^^^^^
void run_thread(std::promise<DataWareHouse> & data_chunks)
^
The error is telling you you cannot copy an std::promise, which you do here:
void run_thread(std::promise<DataWareHouse> data_chunks)
and here:
std::thread(&run_thread,data_chunks); // makes copy of data_chunks
You should pass a reference:
void run_thread(std::promise<DataWareHouse>& data_chunks);
// ^
And then pass an std::reference_wrapper to the thread, otherwise it too will attempt to copy the promise. This is easily done with std::ref:
std::thread(&run_thread, std::ref(data_chunks));
// ^^^^^^^^
Obviously data_chunks must be alive until the thread finished running, so you will have to join the thread in callerFunc().
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
C++: Simple return value from std::thread?
Is there anyway to get the return code from a std::thread? I have a function which returns a integer, and I want to be able to get the return code from the function when the thread is done executing.
No, that's not what std::thread is for.
Instead, use async to get a future:
#include <future>
int myfun(double, char, bool);
auto f = std::async(myfun, arg1, arg2, arg3); // f is a std::future<int>
// ...
int res = f.get();
You can use the wait_for member function of f (with zero timeout) to see if the result is ready.
As others have suggested, the facilities in <future> can be used for this. However I object to the answer
No, you can't do this with std::thread
Here is one way to do what you want with std::thread. It is by no means the only way:
#include <thread>
#include <iostream>
int func(int x)
{
return x+1;
}
int main()
{
int i;
std::thread t([&] {i = func(2);});
t.join();
std::cout << i << '\n';
}
This will portably output:
3
Kerrek SB is correct with his answer, but I suggested to add another example (which he suggested should be an answer, so here it is).
I discovered recently that at least in VC11, std::async will not release all the resources of the thread until the end of the application, making possible to get memory leak false positive (if you are monitoring them using, for example Visual Leak Detector).
Here I mean that in most basic applications it is not worth looking at the rest of this answer, but if like me you need to check memory leaks and can't afford to let false positive, like static data not released at the end of the main function. If it's your case, then this might help.
std::async is not guaranteed to run in a separate thread by default, it is only if you use std::launch::async as first parameter. Otherwise the implementation decide what to do, and that's why VC11 implementation will use the new Microsoft Concurrency Runtime task manager to manage the provided function as a task pushed in a task pool, which mean threads are maintained and managed in a transparent way. There are ways to explicitely terminate the task manager but that's too platform specific, making async a poor choice when you want exactly 1) be sure to launch a thread and 2) get a result later and 3) be sure the thread is fully released when you get the result.
The alternative that does exactly that is to use std::packaged_task and std::thread in combination with std::future. The way it is done is almost similar to using std::async, just a bit more verbose (which mean you can generalize it in a custom template function if you want).
#include <packaged_task>
#include <thread>
int myfun(double, char, bool);
std::packaged_task<int(double, char, bool)> task(myfun, arg1, arg2, arg3);
auto f = task.get_future(); // f is a std::future<int>
First we create a task, basically an object containing both the function and the std::promise that will be associated with the future. std::packaged_task works mostly like an augmented version of std::function:
Now we need to execute the thread explicitly:
std::thread thread(std::move(task));
thread.detach();
The move is necessary because std::packaged_task is not copyable. Detaching the thread is only necessary if you only want to synchronize using the future – otherwise you will need to join the thread explicitly. If you don't, when thread's destructor is called, it will just call std::terminate().
// ...
int res = f.get(); // Synchronization and retrieval.
Here's an example using packaged_task:
#include <future>
#include <iostream>
void task_waiter(std::future<int>&& f) {
std::future<int> ft = std::move(f);
int result = ft.get();
std::cout << result << '\n';
}
int the_task() {
return 17;
}
int main() {
std::packaged_task<int()> task(the_task);
std::thread thr(task_waiter, task.get_future());
task();
thr.join();
return 0;
}