I'm trying to pass multiple arguments, one of which is an ofstream object, to multiple threads using the C++11 standard.
I want to pass an ofstream object because I want every thread to write in a different output file.
I'm initializing the threads and output streams in this way:
std::thread execution_threads[NUMBER_OF_THREADS]; // creating multiple empty threads
std::ofstream output_files[NUMBER_OF_THREADS]; // Empty output streams
// Opening and initializing the output files
Each thread executes a function that takes two arguments:
void execution(int thread_id, std::ofstream& output_file)
So I've looked around and I've read that in C++11 when a function func has multiple arguments a,b,c,d there's no need to use a struct and you can pass them just by writing std::thread t(func, a,b,c,d);. So I wrote this loop in order to launch the threads:
for (int i = 0; i < utils::NUMBER_OF_THREADS; i++) {
execution_threads[i] = std::thread(execution, i, output_files[i]);
}
The thing is that this code doesn't compile with this error:
Call to implicitly-deleted copy constructor of
'typename decay<basic_ofstream<char, char_traits<char> > &>::type'
(aka 'std::__1::basic_ofstream<char, std::__1::char_traits<char> >')
While if I use a struct as input in this way, everything works fine:
// Struct definition
struct thread_input {
int thread_id;
std::ofstream& output_file;
};
// This is the function that now has only one argument
void execution(struct thread_input input)
// And this is the loop that launches the threads
for (int i = 0; i < utils::NUMBER_OF_THREADS; i++) {
struct thread_input input = {i, output_files[i]};
execution_threads[i] = std::thread(execution, input);
}
// join() functions for each thread and end of the program
In this way everything works fine it compiles and it runs perfectly. But I really don't get why the compiler tells me that I'm trying to use the deleted copy-constructor if I use the other method.
Thank you for your help.
std::thread stores a copy of its arguments. When you pass it a non-copyable object like a std::ofstream it will complain.
You have two options:
1) Don't store an array of std::ofstream objects; just let your threads store their own streams. In this case, there's no need to copy the stream (just a move, which is fine):
for (int i = 0; i < utils::NUMBER_OF_THREADS; i++) {
execution_threads[i] = std::thread(execution, i, std::ofstream{});
//^^^^^^^^^^^^^^^ anonymous temporary
}
Of course, in this case you could just have the thread construct its own stream (maybe just pass in the filename).
2) Pass a std::reference_wrapper<std::ofstream> to your thread. std::reference_wrapper<T> is an object that holds a reference to T and has an implicit conversion to T&, so you'll just end up copying the reference instead of the stream itself. You can use the std::ref factory to deduce T and reduce typing:
for (int i = 0; i < utils::NUMBER_OF_THREADS; i++) {
execution_threads[i] = std::thread(execution, i, std::ref(output_files[i]));
//^^^^^^^^ std::ref added
}
This leaves you with all of the same ownership and lifetime issues that passing a struct containing a std::ofstream& would have (after all, that's all std::reference_wrapper is). It's up to you to make sure your output_files array survives until all of your threads have finished with it.
Related
I have trouble to pass an argument to my dispatch queue that takes a function pointer as a parameter. I have implemented a Dispatch Queue like this tutorial
typedef std::function<std::string( const std::array<float, kMaxSamples> &)> fp_t;
class DispatchQueue {
public:
DispatchQueue(std::string name, size_t thread_cnt = 1);
~DispatchQueue();
//move
void dispatch(fp_t && item); //Take the typedef defined above
private:
std::string _name;
std::queue<fp_t> _q;
std::vector<std::thread> _threads;
void dispatch_thread_handler(void);
std::mutex _lock;
std::condition_variable _cv;
bool _quit;
};
My std::function takes an std::array as a parameter.
Then, later in my code I add in the queue this particular job to process this argument.
queue->dispatch(std::bind(&AudioRecordEngine::run, mRecordingCallbackImp.getAudioData()));
The dispatch function is defined as:
void DispatchQueue::dispatch(fp_t &&item)
{
std::unique_lock<std::mutex> lock(_lock);
_q.push(item);
// Manual unlocking is done before notifying, to avoid waking up
// the waiting thread only to block again (see notify_one for details)
lock.unlock();
_cv.notify_one();
}`
Maybe it is too complicated for this use case, I probably don't know how to do better.
I would greatly appreciate suggestion and help. I am stuck for quite a while.
Thanks a lot
EDIT:
The problem I am facing is at compilation time:
no viable conversion from '__bind<std::__ndk1::__bind<std::__ndk1::basic_string<char, std::__ndk1::char_traits, std::__ndk1::allocator > (AudioRecordEngine::*)(const std::__ndk1::array<float, 44100> &), std::__ndk1::array<float, 44100> > >' to 'fp_t' (aka 'function<basic_string<char, char_traits, allocator > (const array<float, kMaxSamples> &)>')
It seems that my std::function do not support the argument I am passing. The problem look like I do not use the std::bind properly.
Basically I would like to pass to my dispatch function the function pointer with the given argument.
EDIT 2:
The AudioRecordEngine::run is defined as:
std::string AudioRecordEngine::run(const std::array<float, __NUM_SAMPLES__> & audioData) {
std::thread::id this_id = std::this_thread::get_id();
LOGD("In the thread ID %zu \n", this_id);
//double freq = FFTNativeWrapper::fftEntryPoint(audioData);
//LOGD("In the Thread, FFT analysis == %zu \n", freq);
return "from thread";
}
std::array<float, kMaxSamples> RecordingCallbackImp::getAudioData() {
return mData;
}
Inside the implementation of DispatchQueue, functional objects of type fp_t are invoked as fn(), so fp_t should be std::function<std::string()> or std::function<void()>. There should be no const std::array& parameter.
std::bind should give you something that takes no parameters. Your std::bind is almost correct. When you want to invoke a non-static member function, you need an object. That object, in the form of a pointer or a reference, should be the second argument of std::bind:
AudioRecordEngine engine;
queue->dispatch(std::bind(
&AudioRecordEngine::run,
std::ref(engine),
mRecordingCallbackImp.getAudioData()
));
Be careful about engine life time.
Alternatively, you can use a lambda function instead of std::bind:
AudioRecordEngine engine;
queue->dispatch([&] {
engine.run(mRecordingCallbackImp.getAudioData());
});
This and the previous approaches differ by the moment when getAudioData() is called: in 2, the result of getAudioData() execution is stored inside the functional object returned by std::bind, in 3, getAudioData() is invoked just before run is called.
Minimal example
First of all, I want to say that i already made researches on the subject, but nothing relevant...
(Error creating std::thread on Mac OS X with clang: "attempt to use a deleted function")
(Xcode 7: C++ threads ERROR: Attempting to use a deleted function)
(xcode - "attempt to use a deleted function" - what does that mean?)
And here's my issue...:
clang error:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/thread:347:5: error: attempt to use a deleted function
__invoke(_VSTD::move(_VSTD::get<0>(__t)), _VSTD::move(_VSTD::get<_Indices>(__t))...);
And that's my code:
bool GenAI::loadAIs()
{
bool ret = true;
if (_nbThread > 1)
{
std::vector<std::thread> threads;
for (unsigned int i = 0; i < _nbThread; ++i)
threads.push_back(std::thread(static_cast<void (GenAI::*)(bool &, unsigned int)>(&GenAI::loadAIs), this, ret, i));
for (unsigned int i = 0; i < _nbThread; ++i)
threads[i].join();
}
else
loadAIs(ret, 0);
return ret;
}
// And the prototype of the function that i try to call
void GenAI::loadAIs(bool & ret, unsigned int iThread);
If some one could help me that'd be really helpful ! :)
Regards ;)
To pass reference to thread, you have to use std::reference_wrapper, that you can obtain with std::ref. So your code becomes:
threads.emplace_back(static_cast<void (GenAI::*)(bool &, unsigned int)>(&GenAI::loadAIs),
this,
std::ref(ret),
i));
Note:
bool ret should probably be std::atomic<bool> ret, or should should have one bool by thread. Else you may have concurrent access on ret.
The deleted function that it is complaining about is a deleted copy constructor for const thread.
For the deleted function problem, you can use:
threads.emplace_back(
Instead of:
threads.push_back(
What the commenter also referred to is that the function is creating more than one thread and passing to them a reference to the same boolean return variable.
It will crash if you don't use atomic_bool and even if you do, they will all report back to the same memory location, making the function miss the notification if one of them returns false.
I create the two following objects:
bool Reception::createNProcess()
{
for (int y = 0; y < 3; ++y)
{
Process *pro = new Process; // forks() at construction
Thread *t = new Thread[5];
this->addProcess(pro); // Adds the new process to a vector
if (pro->getPid() == 0)
{
for (int i = 0; i < 5; ++i)
{
pro->addThread(&t[i]); // Adds the new thread to a vector
t[i].startThread();
}
}
}
Where I create 3 processes (that I have encapsulated in Process) and create 5 threads in each of these processes.
But I'm not sure the following line is correct:
Thread *t = new Thread[5];
Because my two functions addProcess and addThread both take a pointer to Process and Thread respectively and yet the compiler asks for a reference to t[i] for addThread and I don't understand why.
void Process::addThread(Thread *t)
{
this->threads_.push_back(t);
}
void Reception::addProcess(Process *p)
{
this->createdPro.push_back(p);
}
createdPro is defined in the Reception class as follows:
std::vector<Process *> createdPro;
and threads_ in the Process class like such:
std::vector<Thread *> threads_;
And the error message (as obvious as it is) is as follows:
error: no matching function for call to ‘Process::addThread(Thread&)’
pro->addThread(t[i]);
process.hpp:29:10: note: candidate: void Process::addThread(Thread*)
void addThread(Thread *);
process.hpp:29:10: note: no known conversion for argument 1 from ‘Thread’ to ‘Thread*’
Even though I defined my Thread as a pointer.
You have defined the member function to take a pointer:
void Process::addThread(Thread *t)
{
...
}
You then invoke this function for &t[i], which is a pointer and should work perfectly:
pro->addThread(&t[i]); // Adds the new thread to a vector
You could also invoke it with t+i and it would still be ok. However your error message tells us something different: the compiler doesn't find a match for pro->addThread(t[i]); (i.e. the & is missing).
Either you made a typo in your question, or you made a typo in your code. Or you have another invocation somewhere where you've forgotten the ampersand: t[i] would of course designate an object (it's equivalent to *(t+i) ) and not a pointer, and cause the error message you have (demo mcve)
I have a situation in which i need to instantiate a vector of boost::threads to solve the following:
I have a class called Instrument to hold Symbol information, which looks something like below:
class Instrument
{
public:
Instrument(StringVector symbols, int i);
virtual ~Instrument();
const Instrument& operator= (const Instrument& inst)
{
return *this;
}
String GetSymbol() { return Symbol_; }
LongToSymbolInfoPairVector GetTS() { return TS_; }
bool OrganiseData(TimeToSymbolsInfoPairVector& input, int i);
static int getRandomNumber(const int low, const int high);
static double getProbability();
bool ConstructNewTimeSeries(const int low, const int high);
bool ReconstructTimeSeries(TimeToSymbolsInfoPairVector& reconstructeddata, int i);
private:
LongToSymbolInfoPairVector TS_;
String Symbol_;
const int checkWindow_;
String start_, end_;
long numberofsecsinaday_;
static std::default_random_engine generator_;
};
This class will have as many objects as the number of symbols. These symbols shall be accessed in another class Analysis for further work, whose constructor accepts the vector of the above Instrument class, as shown below.
class Analysis
{
public:
Analysis(std::vector<Instrument>::iterator start, std::vector<Instrument>::iterator end);
virtual ~Analysis();
bool buildNewTimeSeries(TimeToSymbolsInfoPairVector& reconstructeddata);
bool printData(TimeToSymbolsInfoPairVector& reconstructeddata);
private:
std::vector<Instrument> Instruments_;
};
Now i want to multithread this process so that i can separate out say 7 symbols per thread and spawn out, say, 4 threads.
Following is the updated main.
std::vector<Instrument>::iterator block_start = Instruments.begin();
int first = 0, last = 0;
for (unsigned long i=0; i<MAX_THREADS; i++)
{
std::vector<Instrument>::iterator block_end = block_start;
std::advance(block_end, block_size);
last = (i+1)*block_size;
Analysis* analyzed = new Analysis(block_start, block_end /*first, last*/);
analyzed->setData(output, first, last);
threads.push_back(std::thread(std::bind(&Analysis::buildNewTimeSeries, std::ref(*analyzed))));
block_start = block_end;
first = last;
}
for (int i=0; i<MAX_THREADS; i++)
{
(threads[i]).join();
}
This is evidently incorrect, although i know how to instantiate a thread's constructor to pass a class constructor an argument or a member function an argument, but i seem to be facing an issue when my purpose is:
a) Pass the constructor of class Analysis a subset of vector and
b) Call the buildNewTimeSeries(TimeToSymbolsInfoPairVector& reconstructeddata)
for each of the 4 threads and then later on join them.
Can anyone suggest a neat way of doing this please ?
The best way to go about partitioning a vector of resources (like std::vector in ur case) on to limited number of threads is by using a multi-threaded design paradigm called threadpools. There is no standard thread-pool in c++ and hence you might have to build one yourself(or use open source libraries). You can have a look at one of the many good opensource implementations here:- https://github.com/progschj/ThreadPool
Now, I am not going to be using threadpools, but will just give you a couple of suggestions to help u fix ur problem without modifying ur core functionality/idea.
In main you are dynamically creating vectors using new and are passing on the reference of the vector by dereferencing the pointer. Analysis* analyzed = new. I understand that your idea here, is to use the same vector analysis* in both main and the thread function. In my opinion this is not a good design. There is a better way to do it.
Instead of using std::thread use std::async. std::async creates tasks as opposed to threads. There are numerous advantages using tasks by using async. I do not want to make this a long answer by describing thread/tasks. But, one main advantage of tasks which directly helps you in your case is that it lets you return values(called future) from tasks back to the main function.
No to rewrite your main function async, tweak your code as follows,
Do not dynamically create a vector using new, instead just create a
local vector and just move the vector using std::move to the task
while calling async.
Modify Analysis::buildNewTimeSeries to accept rvalue reference.
Write a constructor for analysis with rvalue vector
The task will then modify this vector locally and then
return this vector to main function.
while calling async store the return value of the async
calls in a vector < future < objectType > >
After launching all the tasks using async, you can call the .get() on each of the element of this future vector.
This .get() method will return the vector modified and returned from
thread.
Merge these returned vectors into the final result vector.
By moving the vector from main to thread and then returning it back, you are allowing only one owner to have exclusive access on the vector. So you can not access a vector from main after it gets moved to thread. This is in contrast to your implementation, where both the main function and the thread function can access the newly created vector that gets passed by reference to the thread.
I've been having trouble with this parallel matrix multiplication code, I keep getting an error when trying to access a data member in my structure.
This is my main function:
struct arg_struct
{
int* arg1;
int* arg2;
int arg3;
int* arg4;
};
int main()
{
pthread_t allthreads[4];
int A [N*N];
int B [N*N];
int C [N*N];
randomMatrix(A);
randomMatrix(B);
printMatrix(A);
printMatrix(B);
struct arg_struct *args = (arg_struct*)malloc(sizeof(struct arg_struct));
args.arg1 = A;
args.arg2 = B;
int x;
for (int i = 0; i < 4; i++)
{
args.arg3 = i;
args.arg4 = C;
x = pthread_create(&allthreads[i], NULL, &matrixMultiplication, (void*)args);
if(x!=0)
exit(1);
}
return 0;
}
and the matrixMultiplication method used from another C file:
void *matrixMultiplication(void* arguments)
{
struct arg_struct* args = (struct arg_struct*) arguments;
int block = args.arg3;
int* A = args.arg1;
int* B = args.arg2;
int* C = args->arg4;
free(args);
int startln = getStartLineFromBlock(block);
int startcol = getStartColumnFromBlock(block);
for (int i = startln; i < startln+(N/2); i++)
{
for (int j = startcol; j < startcol+(N/2); j++)
{
setMatrixValue(C,0,i,j);
for(int k = 0; k < N; k++)
{
C[i*N+j] += (getMatrixValue(A,i,k) * getMatrixValue(B,k,j));
usleep(1);
}
}
}
}
Another error I am getting is when creating the thread: "invalid conversion from ‘void ()(int, int*, int, int*)’ to ‘void* ()(void)’ [-fpermissive]
"
Can anyone please tell me what I'm doing wrong?
First you mix C and C++ very badly, either use plain C or use C++, in C++ you can simply use new and delete.
But the reason of your error is you allocate arg_struct in one place and free it in 4 threads. You should allocate one arg_struct for each thread
Big Boss is right in the sense that he has identified the problem, but to add to/augment the reply he made.
Option 1:
Just create an arg_struct in the loop and set the members, then pass it through:
for(...)
{
struct arg_struct *args = (arg_struct*)malloc(sizeof(struct arg_struct));
args->arg1 = A;
args->arg2 = B; //set up args as now...
...
x = pthread_create(&allthreads[i], NULL, &matrixMultiplication, (void*)args);
....
}
keep the free call in the thread, but now you could then use the passed struct directly rather than creating locals in your thread.
Option 2:
It looks like you want to copy the params from the struct internally to the thread anyway so you don't need to dynamically allocate.
Just create an arg_struct and set the members, then pass it through:
arg_struct args;
//set up args as now...
for(...)
{
...
x = pthread_create(&allthreads[i], NULL, &matrixMultiplication, (void*)&args);
}
Then remove the free call.
However as James pointed out you would need to synchronize in the thread/parent on the structure to make sure that it wasn't changed. That would mean a Mutex or some other mechanism. So probably the move of the allocation to the for loop is easier to begin with.
Part 2:
I'm working on windows (so I can't experiment currently), but pthread_create param 3 is referring to the thread function matrixMultiplication which is defined as void* matrixMultiplication( void* ); - it looks correct to me (signature wise) from the man pages online, void* fn (void* )
I think I'll have to defer to someone else on your second error. Made this post a comunnity wiki entry so answer can be put into this if desired.
It's not clear to me what you are trying to do. You start some threads,
then you return from main (exiting the process) before getting any
results from them.
In this case, I'ld probably not use any dynamic allocation, directly.
(I would use std::vector for the matrices, which would use dynamic
allocation internally.) There's no reason to dynamically allocate the
arg_struct, since it can safely be copied. Of course, you'll have to
wait until each thread has successfully extracted its data before
looping to construct the next thread. This would normally be done using
a conditional: the new thread would unblock the conditional once it has
extracted the arguments from the arg_struct (or even better, you could
use boost::thread, which does this part for you). Alternatively, you
could use an array of arg_struct, but there is absolutely no reason to
allocate them dynamically. (If for some reason you cannot use
std::vector for A, B and C, you will want to allocate these
dynamically, in order to avoid any risk of stack overflow. But
std::vector is a much better solution.)
Finally, of course, you must wait for all of the threads to finish
before leaving main. Otherwise, the threads will continue working on
data that doesn't exist any more. In this case, you should
pthread_join all of the threads before exiting main. Presumably,
too, you want to do something with the results of the multiplication,
but in any case, exiting main before all of the threads have finished
accessing the matrices will cause undefined behavior.