I am trying to learn little bit about Multi-Threading because I need it in my bachelor thesis. I think I understand it on low teoretical level but not on practical.
For the beginning I am trying to create very easy program, where I would like to learn how to work with QThread. My small program test all files in directory if they are .txt files.
There is function which just test if one file contains .txt and its used in next functions.
I know I know this is not perfect test for testing if file is .txt but it really doesnt matter. This is about something else.
void FileTester::testIfTxt(QString &file){
if( file.contains(".txt"))
std::cout << file.toStdString() << " is .txt file" << std::endl;
else
std::cout << file.toStdString() << " is NOT .txt file" << std::endl;
}
Here is function for testing without using multi-threading.
void FileTester::normalTest(){
QDir folderPath("D:\\programming\\C++\\multithreading-qt\\files");
QStringList filesList = folderPath.entryList(QDir::NoDotAndDotDot | QDir::Files);
auto t_start = std::chrono::high_resolution_clock::now();
for(QString& file : filesList){
testIfTxt(file);
}
auto t_end = std::chrono::high_resolution_clock::now();
double elapsed_time_ms = std::chrono::duration<double, std::milli>(t_end-t_start).count();
std::cout << elapsed_time_ms << " ms" << std::endl;
}
Now I would like to do the same but with multithreading. For the beginning we can use 3 threads. First thread would take first file and test, while second thread would take second file and test and third thread would take third file and test. When first thread will end work, than take nex untested file and test etc. .......
I started with function there but I really dont understand, how to use something like this. I started with something like this.
void FileTester::multiThreadTest(){
auto t_start = std::chrono::high_resolution_clock::now();
allMultiThreadTest();
auto t_end = std::chrono::high_resolution_clock::now();
double elapsed_time_ms = std::chrono::duration<double, std::milli>(t_end-t_start).count();
std::cout << elapsed_time_ms << " ms" << std::endl;
}
void FileTester::allMultiThreadTest(){
QThread* thread1 = new QThread();
QThread* thread2 = new QThread();
QThread* thread3 = new QThread();
thread1->start();
thread2->start();
thread3->start();
for(QString& file : filesList){
testIfTxt(file);
}
}
Again this is not code from my bachelor thesis. I just want to learn how to use multithreading on this easy program. Thank you for any help.
The QThread documentation, like most Qt documentation, is very good, but the code examples might be a bit overwhelming.
There are basically two ways to use QThread:
Subclassing QThread
Useful when you want to run some code on a thread, and exit the thread when the code finishes.
Create a subclass of QThread and override its run() function. The code you put in that function will run in the thread, and when the function returns, the thread exits.
To start the thread, simply create an instance of your subclass and call start() on it.
Handling signals on a thread
Useful when you want to keep a thread in the background, and occasionally send it some work to do. This approach needs some boilerplate code because you need an object to emit a signal and another object to receive it with a slot.
Create a subclass of QObject and give it a public slot of your choice.
Create an instance of QThread (no subclass needed here). The default implementation of run() contains an event loop, which simply waits for signals and runs the corresponding slots, just like QApplication::exec() does for your main thread.
Create an instance of your QObject subclass. Move the latter to your new thread using QObject::moveToThread(). The effect of this is, that any slots being called by signals will be run on that thread, rather than the main thread. See thread affinity and queued connections.
Connect the slot to a signal that is emitted by any QObject subclass on your main thread.
To send work to the thread, simply emit that signal.
Note: any arguments that you pass to the signal must be known to Qt's meta-type system. Most built-in Qt types are okay. If you run afoul of this rule, you will get a warning at runtime.
Related
Update
I did as recommended to create a std::vector of threads outside the scope, so I can .join() as soon as the thread has finished it's job, the problem now is that as soon as the thread is joined the program not exactly crashes, because it still runs in the background but the abort window appears. I checked if the thread was joinable and indeed it is when trying to join.
Timer.cpp:
void Timer::Start(int time, void(*lf)()) {
slaveTimer = std::thread(&Timer::RunTimer, this, time, lf);
}
void Timer::RunTimer(int seconds, void(*lf)()) {
auto time = (std::chrono::seconds)seconds;
std::this_thread::sleep_for(time);
lf();
slaveTimer.join(); //Program Crashes
}
Main.cpp
Timer timer1(10, [](){ std::cout << "Hello World" << std::endl; });
Original Post
I was trying to make coroutines with multithreading, the thing is that when I try to make the thread wait for X seconds, i then thread.detach(); but that takes a couple of milliseconds and the screen (because I’m displaying with GL) freezes. One of the possible solutions that I can think of is making the thread detach itself before executing the action, but that doesn’t seem possible, so I was wondering if there is any way to do that or something similar to solve this problem.
You cannot call join from the function which is the body of execution thread. It will give you the error:
Reference
Error Conditions :
resource_deadlock_would_occur if this->get_id() ==
std::this_thread::get_id() (deadlock detected)
you need to add additional method for instance
void Timer::stop() {
slaveTimer.join();
}
and call this method from thread which created timer1 instance
Timer timer1(10, [](){ std::cout << "Hello World" << std::endl; });
timer1.stop();
or join thread in dtor of Timer:
Timer::~Timer() {
slaveTimer.join();
}
I am working on a project where I will be ingesting multiple binary files, decode them, and convert their data into a CSV. I figured the quickest way to do this would be to thread the work. Simply load the files into a queue, have the threads grab a file, work on it, convert it, output it, and then die.
What I wrote actually works great, however, I cannot figure out how to get the GUI to be responsive as I have a progress bar that I would like to update or simply have the user move the GUI to a corner while it processes the data. And I believe this is because std::thread is just hanging up the GUI.
In my code I have the following function once a button is pressed to execute:
void MyExtractor::on_Execute_clicked()
{
QStringList binary = tlmFiles.entryList(QStringList() << "*.bin",QDir::Files);
queue.clear();
threadPool.clear();
if(binary.size() != 0)
{
foreach(QString filename, binary)
{
queue.emplace_back(inputDir + '/' + filename);
}
for (unsigned int i = 0; i < std::thread::hardware_concurrency(); ++i)
{
threadPool.emplace_back(&MyExtractor::initThread,this,std::ref(queue),std::ref(mut));
}
}
else
{
message.setText("No binary files found! Please select another folder!");
message.exec();
}
for (auto &&e : threadPool)
{
e.join();
}
}
And initThread looks like this:
void MyExtractor::initThread(std::deque<QString> &queue, std::mutex &mutex)
{
QString file;
QString toOutput = outputDir;
while(queue.size() > 0)
{
{
std::lock_guard<std::mutex> lock(mutex);
if(!queue.empty())
{
file = queue.front();
queue.pop_front();
}
}
BitExtract *bitExtractor = new BitExtract();
if(file.size() != 0)
{
bitExtractor->extract(file,toOutput);
}
delete bitExtractor;
}
}
I have been reading about QThreads. And from what I think I have been reading, it seems I need to create a separate thread to watch the work, and the other thread to watch the GUI? I am not sure if I have worded that correctly. However, I am not even sure how to go about that since I am using a std::thread to do the conversion, and I am not sure how well QThread will play with this. Any suggestions?
EDIT: I should make it clear that threadPool is a std::vector<std::thread>
As noted by #drescherjm, your problem is here:
for (auto &&e : threadPool)
{
e.join();
}
join() won't return until the thread has completed, which means your GUI thread will be blocked inside that for-loop until all threads have exited, which is what you want to avoid. (it's always desirable for any function in the main/Qt/GUI thread to return as quickly as possible, so that Qt's GUI event loop can remain responsive)
Avoiding that is fairly straightforward -- instead of calling join() right after the threads have been spawned, you should only call join() on a thread after the thread has notified you that it has completed its work and is about to exit. That way join() will never take more than a few milliseconds to return.
As for how to get a std::thread to notify your main/GUI thread that it has finished its task, one simple way to do it is to have your std::thread call QApplication::postEvent() just before it exits, and override the event(QEvent *) virtual method on (whatever object you passed in as the first argument to postEvent()) to handle the posted event-object (note that you can make your own subclass of QEvent that contains whatever data you want to send to the GUI thread) by calling join() on the std::thread, plus whatever cleanup and result-handling code you need to execute after a thread has returned its result.
A strange behavior occured in my application when I'm using QtNetwork. I can easily create the QTcpSever and QTcpSocket instance and everything runs fine, but when it comes to QTcpSocket::write() the following error occurs:
The error
QObject: Cannot create children for a parent that is in a different thread.
(Parent is QNativeSocketEngine(0x7f66980022e0), parent's thread is QThread(0x7f66a0020be0), current thread is QThread(0x7f66a0020e20)
QSocketNotifier: Can only be used with threads started with QThread
What is strange to me: I have no idea what/where this QThread(0x7f66a0020e20) is and how to get influence on it (have a look at the debugging below)
The program
I'm extending my main application (which is a library) with a network support. I put the network services into an extra class.
here the excerpt of the main application/library, where my network support is created:
QThread *thread = new QThread;
wifi = new WirelessNet(0, thread);
wifi->moveToThread(thread);
connect(thread,SIGNAL(started()), wifi,SLOT(initWifi()));
thread->start();
the network class extension:
WirelessNet::WirelessNet(QObject *parent, QThread *comThread): QTcpServer(parent)
{
clientThread = comThread;
}
void WirelessNet::initWifi()
{
listen(QHostAddress::Any, 5220);
connect(this,SIGNAL(newConnection()),this,SLOT(connectionRequest()));
}
void WirelessNet::connectionRequest()
{
client = this->nextPendingConnection();
if(client)
connect(client, SIGNAL(readyRead()), this, SLOT(receiveMessage()));
}
void WirelessNet:sendData(QByteArray msg)
{
if (client)
{
qDebug()<<"FIRST "<< client->thread() << " - " << this->thread() << "\n";
client->write(msg);
client->waitForBytesWritten();
qDebug()<<"LAST " << client->thread() << " - " << this->thread() << "\n";
}
}
(client and clientThread are class members: QTcpSocket*, QThread* respectively)
The debugging
Here is what the console prints out when it comes to the sendData() part:
FIRST QThread(0x7f66a0020be0) - QThread(0x7f66a0020be0)
QObject: Cannot create children for a parent that is in a different thread.
(Parent is QNativeSocketEngine(0x7f66980022e0), parent's thread is QThread(0x7f66a0020be0), current thread is QThread(0x7f66a0020e20)
QSocketNotifier: Can only be used with threads started with QThread
LAST QThread(0x7f66a0020be0) - QThread(0x7f66a0020be0)
Concluding
In other words I have no idea on which object I should apply the moveToThread(). I already tried client->moveToThread(clientThread) aswell as this->moveToThread(clientThread). Unfortunately I don't see any additional objects to check on.
Has anyone an idea ?
You seem to be calling WirelessNet:sendData directly from the main thread. This causes everything inside that function to be run in the main thread as well. Your client lives in the new thread, and it is not thread-safe. It tries to create children, but current thread is different from the thread where client lives in. That is why you get that error message.
You can fix it simply by making WirelessNet:sendData a slot and call it via a signal from the main thread.
My guess is that the constructor of your class is called in the calling thread, while the thread itself runs in the run() method of your class. The solution would be to initialize QTcpServer at the beginning of your run() method, so that initialization and communication through that class is done in the very same thread.
My current problem is with two QT threads. One of them emits a signal for starting an operation in the second thread, and afterwards wait for the result. As soon as the second thread finishes, the first thread should continue with its own operation using results from the second thread.
For letting the first thread sleep I use a QMutex and a QWaitCondition. The first thread emits a signal, and afterwards sleeps at the wait condition. But the problem is now: If the second thread somehow managed to be faster than the first thread, and emits the wakeAll()-call before the first thread enters the wait-condition, I get stuck. I could implement a waiting time, but then I am unflexible again, and if the second threads needs more time than the first thread is waiting, I have a problem again.
This problem has already been addressed here: http://woboq.com/blog/qwaitcondition-solving-unavoidable-race.html, but they decided to leave this problem unsolved. So, is there a possibility to avoid this race-condition?
Addition: I don't want to convert this function into a function of the first thread, because this specific function should be accessable from several threads at once without leading to a race condition. I.e. Thread1 should call the function in Thread2, wait til it is finished, Thread3 also wants to call the function, but is not allowed to do that, it has to wait till finish, too. If the function has finished, Thread3 can access it. (Same goes for more than only two threads).
Example function:
This function should emit the signal and afterwards wait for the wake signal:
void Spectrometer_Control::moveStepper(int steps, bool dir)
{
emit stepperMoving();
qDebug() << "From Spectrometer_Control: Stepper should move in direction " + QString::number(dir) + " from position " + QString::number(MonoPos);
int newTarget = MonoPos + ((dir == true)?(steps):(-1 * steps));
qDebug() << "New target: " + QString::number(newTarget);
emit moveStepperToTarget(steps, dir);
qDebug() << "Locking WaitMutex!";
WaitMutex->lock();
qDebug() << "Waiting for signal!";
WaitForEngine->wait(WaitMutex);
WaitMutex->unlock();
qDebug() << "Finally unlocked!";
}
And this function receives the call, and should wake every waiting function up:
void Stepper_Control_Worker::moveStepper(int steps, bool dir)
{
waitMutex->lock();
qDebug() << "Motor moved from down below!";
Stepper_Control_Worker::STP[1]->setValue((dir == true)?BlackLib::high:BlackLib::low);
usleep(50000);
Stepper_Control_Worker::STP[0]->setValue(BlackLib::low);
usleep(50000);
for(int i = 0; i < steps; i++)
{
Stepper_Control_Worker::STP[0]->setValue(BlackLib::high);
usleep(50000);
Stepper_Control_Worker::STP[0]->setValue(BlackLib::low);
}
WaitCond->wakeAll();
waitMutex->unlock();
emit StepperMoved(steps, dir);
}
The second function is a sub-member (not directly, but can only accessed via) of the class "stepper_control". The stepper-controller outer controls can be used by several functions, not only the function moveStepper from Spectrometer_Control, but for making things easier I only added one external function. But after I don't want to get my stepper confused, I wanted to restrict the access as described above.
It's probably safe to let the second thread send a signal back and move the code post wait to that slot.
class Worker1: public QObject{
Q_OBJECT
//...
signals:
void startWorking();
slots:
void answer(QVariant);
};
class Worker2: public QObject{
Q_OBJECT
//...
slots:
void startWorking();
signals:
void answer(QVariant);
};
Otherwise you need to have a variable that the second thread sets while holding the QMutex to signal the first:
thread1:
emit startWorking();
{
QMutexLocker lock(&thread2->mutex);
while(!thread2->finished){//loop guards against spurious wakeups
thread2->cond->wait(&mutex);
}
}
and thread2:
{
QMutexLocker lock(&mutex);
finished=true;
cond->wakeAll();
}
That way if thread2 is faster then thread2->finished is already true by the time thread1 arrives and the mutex protects the variable between testing it and waiting on the QWaitCondition.
Maybe Qt::BlockingQueuedConnection is what you need?
a blocking queued connection is like a queued connection, but the sender thread blocks until the event is picked up by the event loop of the thread the receiver is living in, the slot is invoked, and it returns;
I need to parallelize some tasks in a C++ program and am completely new to parallel programming. I've made some progress through internet searches so far, but am a bit stuck now. I'd like to reuse some threads in a loop, but clearly don't know how to do what I'm trying for.
I am acquiring data from two ADC cards on the computer (acquired in parallel), then I need to perform some operations on the collected data (processed in parallel) while collecting the next batch of data. Here is some pseudocode to illustrate
//Acquire some data, wait for all the data to be acquired before proceeding
std::thread acq1(AcquireData, boardHandle1, memoryAddress1a);
std::thread acq2(AcquireData, boardHandle2, memoryAddress2a);
acq1.join();
acq2.join();
while(user doesn't interrupt)
{
//Process first batch of data while acquiring new data
std::thread proc1(ProcessData,memoryAddress1a);
std::thread proc2(ProcessData,memoryAddress2a);
acq1(AcquireData, boardHandle1, memoryAddress1b);
acq2(AcquireData, boardHandle2, memoryAddress2b);
acq1.join();
acq2.join();
proc1.join();
proc2.join();
/*Proceed in this manner, alternating which memory address
is written to and being processed until the user interrupts the program.*/
}
That's the main gist of it. The next run of the loop would write to the "a" memory addresses while processing the "b" data and continue to alternate (I can get the code to do that, just took it out to prevent cluttering up the problem).
Anyway, the problem (as I'm sure some people can already tell) is that the second time I try to use acq1 and acq2, the compiler (VS2012) says "IntelliSense: call of an object of a class type without appropriate operator() or conversion functions to pointer-to-function type". Likewise, if I put std::thread in front of acq1 and acq2 again, it says " error C2374: 'acq1' : redefinition; multiple initialization".
So the question is, can I reassign threads to a new task when they have completed their previous task? I always wait for the previous use of the thread to end before calling it again, but I don't know how to reassign the thread, and since it's in a loop, I can't make a new thread each time (or if I could, that seems wasteful and unnecessary, but I could be mistaken).
Thanks in advance
The easiest way is to use a waitable queue of std::function objects. Like this:
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <queue>
#include <functional>
#include <chrono>
class ThreadPool
{
public:
ThreadPool (int threads) : shutdown_ (false)
{
// Create the specified number of threads
threads_.reserve (threads);
for (int i = 0; i < threads; ++i)
threads_.emplace_back (std::bind (&ThreadPool::threadEntry, this, i));
}
~ThreadPool ()
{
{
// Unblock any threads and tell them to stop
std::unique_lock <std::mutex> l (lock_);
shutdown_ = true;
condVar_.notify_all();
}
// Wait for all threads to stop
std::cerr << "Joining threads" << std::endl;
for (auto& thread : threads_)
thread.join();
}
void doJob (std::function <void (void)> func)
{
// Place a job on the queu and unblock a thread
std::unique_lock <std::mutex> l (lock_);
jobs_.emplace (std::move (func));
condVar_.notify_one();
}
protected:
void threadEntry (int i)
{
std::function <void (void)> job;
while (1)
{
{
std::unique_lock <std::mutex> l (lock_);
while (! shutdown_ && jobs_.empty())
condVar_.wait (l);
if (jobs_.empty ())
{
// No jobs to do and we are shutting down
std::cerr << "Thread " << i << " terminates" << std::endl;
return;
}
std::cerr << "Thread " << i << " does a job" << std::endl;
job = std::move (jobs_.front ());
jobs_.pop();
}
// Do the job without holding any locks
job ();
}
}
std::mutex lock_;
std::condition_variable condVar_;
bool shutdown_;
std::queue <std::function <void (void)>> jobs_;
std::vector <std::thread> threads_;
};
void silly (int n)
{
// A silly job for demonstration purposes
std::cerr << "Sleeping for " << n << " seconds" << std::endl;
std::this_thread::sleep_for (std::chrono::seconds (n));
}
int main()
{
// Create two threads
ThreadPool p (2);
// Assign them 4 jobs
p.doJob (std::bind (silly, 1));
p.doJob (std::bind (silly, 2));
p.doJob (std::bind (silly, 3));
p.doJob (std::bind (silly, 4));
}
The std::thread class is designed to execute exactly one task (the one you give it in the constructor) and then end. If you want to do more work, you'll need a new thread. As of C++11, that's all we have. Thread pools didn't make it into the standard. (I'm uncertain what C++14 has to say about them.)
Fortunately, you can easily implement the required logic yourself. Here is the large-scale picture:
Start n worker threads that all do the following:
Repeat while there is more work to do:
Grab the next task t (possibly waiting until one becomes ready).
Process t.
Keep inserting new tasks in the processing queue.
Tell the worker threads that there is nothing more to do.
Wait for the worker threads to finish.
The most difficult part here (which is still fairly easy) is properly designing the work queue. Usually, a synchronized linked list (from the STL) will do for this. Synchronized means that any thread that wishes to manipulate the queue must only do so after it has acquired a std::mutex so to avoid race conditions. If a worker thread finds the list empty, it has to wait until there is some work again. You can use a std::condition_variable for this. Each time a new task is inserted into the queue, the inserting thread notifies a thread that waits on the condition variable and will therefore stop blocking and eventually start processing the new task.
The second not-so-trivial part is how to signal to the worker threads that there is no more work to do. Clearly, you can set some global flag but if a worker is blocked waiting at the queue, it won't realize any time soon. One solution could be to notify_all() threads and have them check the flag each time they are notified. Another option is to insert some distinct “toxic” item into the queue. If a worker encounters this item, it quits itself.
Representing a queue of tasks is straight-forward using your self-defined task objects or simply lambdas.
All of the above are C++11 features. If you are stuck with an earlier version, you'll need to resort to third-party libraries that provide multi-threading for your particular platform.
While none of this is rocket science, it is still easy to get wrong the first time. And unfortunately, concurrency-related bugs are among the most difficult to debug. Starting by spending a few hours reading through the relevant sections of a good book or working through a tutorial can quickly pay off.
This
std::thread acq1(...)
is the call of an constructor. constructing a new object called acq1
This
acq1(...)
is the application of the () operator on the existing object aqc1. If there isn't such a operator defined for std::thread the compiler complains.
As far as I know you may not reused std::threads. You construct and start them. Join with them and throw them away,
Well, it depends if you consider moving a reassigning or not. You can move a thread but not make a copy of it.
Below code will create new pair of threads each iteration and move them in place of old threads. I imagine this should work, because new thread objects will be temporaries.
while(user doesn't interrupt)
{
//Process first batch of data while acquiring new data
std::thread proc1(ProcessData,memoryAddress1a);
std::thread proc2(ProcessData,memoryAddress2a);
acq1 = std::thread(AcquireData, boardHandle1, memoryAddress1b);
acq2 = std::thread(AcquireData, boardHandle2, memoryAddress2b);
acq1.join();
acq2.join();
proc1.join();
proc2.join();
/*Proceed in this manner, alternating which memory address
is written to and being processed until the user interrupts the program.*/
}
What's going on is, the object actually does not end it's lifetime at the end of the iteration, because it is declared in the outer scope in regard to the loop. But a new object gets created each time and move takes place. I don't see what can be spared (I might be stupid), so I imagine this it's exactly the same as declaring acqs inside the loop and simply reusing the symbol. All in all ... yea, it's about how you classify a create temporary and move.
Also, this clearly starts a new thread each loop (of course ending the previously assigned thread), it doesn't make a thread wait for new data and magically feed it to the processing pipe. You would need to implement it a differently like. E.g: Worker threads pool and communication over queues.
References: operator=, (ctor).
I think the errors you get are self-explanatory, so I'll skip explaining them.
I think you need a much more simpler answer for running a set of threads more than once, this is the best solution:
do{
std::vector<std::thread> thread_vector;
for (int i=0;i<nworkers;i++)
{
thread_vector.push_back(std::thread(yourFunction,Parameter1,Parameter2, ...));
}
for(std::thread& it: thread_vector)
{
it.join();
}
q++;
} while(q<NTIMES);
You also could make your own Thread class and call its run method like:
class MyThread
{
public:
void run(std::function<void()> func) {
thread_ = std::thread(func);
}
void join() {
if(thread_.joinable())
thread_.join();
}
private:
std::thread thread_;
};
// Application code...
MyThread myThread;
myThread.run(AcquireData);