I am having some trouble resolving a race condition that arises when acquiring a lock. I have a large computation that is triggered asynchronously. I need my previous synchronous task to end before the large task begins. The large task launches and waits on a condition variable and, in an ideal world, the small tasks will notify the condition variable when they are done. My problem comes from the large task starting too soon, and the small tasks trigger a condition variable that has yet to full initialize and thus does not actually trigger, causing the program to be locked.
I have boiled it down to this minimal example.
I would assume this is a common problem. How can I check that my condition variable has actually got hold of a mutex and is locked?
#include <QCoreApplication>
#include <QObject>
#include <QFuture>
#include <QtConcurrent/QtConcurrent>
#include <QFutureWatcher>
#include <QWaitCondition>
class workUnit: public QObject
{
Q_OBJECT
public:
workUnit(QObject *parent)
{
m = new QMutex();
m->lock();
w=new QWaitCondition();
}
QWaitCondition* w;
QMutex* m;
public slots:
void runMe()
{
w->wait(m);
m->unlock();
//perform long computation
}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
workUnit* mUnit=new workUnit(&a);
QFutureWatcher<void> w;
QObject::connect(&w, SIGNAL(finished()), &a, SLOT(quit()));
QFuture<void> f = QtConcurrent::run(mUnit,&workUnit::runMe);
w.setFuture(f);
_sleep(1000);//with this delay it works, without the delay it doesn't
mUnit->w->wakeAll();//This would be triggered by another process
return a.exec();
}
The documentation for QWaitCondition::wait states that:
The mutex must be initially locked by the calling thread.
You should remove m->lock(); from the constructor and put it in the runMe() function before the call to wait.
Related
I'm using QtConcurrent::run (I know other APIs of QtConcurrent has built in support for progress reporting but I can't use them for other reasons). to run an operation inside a different than the main GUI thread. I also need this operation to notify the GUI thread of the progress made. So what I did is that created a separate function for the operation I want which accepts a callback that carries the information about the progress of the operation. This callback then calls the signal on a QObject living in the main thread.
Here is a full working example that shows my structure:
#include <QCoreApplication>
#include <QObject>
#include <QThread>
#include <QtConcurrent/QtConcurrent>
namespace Operations {
template<typename Callback>
void longOperation(Callback progressCallback)
{
qint64 sum = 0;
for(int i = 0; i < 100; ++i){
QThread::msleep(400);
sum += i;
progressCallback(i/100.0);
}
}
}
class Emitter : public QObject
{
Q_OBJECT
public:
Q_INVOKABLE void doSomething()
{
auto progressCallback = [&](qreal p){
emit progress(p);
};
auto lambda = [progressCallback](){
Operations::longOperation(progressCallback);
};
QtConcurrent::run(lambda);
}
signals:
void progress(qreal);
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
Emitter emitter;
QObject::connect(&emitter, &Emitter::progress, [](qreal progress){
qDebug() << "Progress" << progress;
});
emitter.doSomething();
return a.exec();
}
#include "main.moc"
Now my question is using the progressCallback as defined above thread safe? The callback will clearly be triggered from a thread different than the GUI thread, so effectively it's calling emitter.progress() directly on the QObject.
Ok, so I now realised that the code above may not be thread safe. Part of my confusion was what does the emit keyword actually does. It turns out it's actually not much at all. So calling the signal from another thread is not really the best idea.
Instead, one of way of improve the situation is replace the progressCallback with:
auto progressCallback = [&](qreal p){
QMetaObject::invokeMethod(this, [this, p](){ emit progress(p);}, Qt::QueuedConnection);
};
This way the signal is emitted on the thread where the Emitter lives as the lambda slot will be executed "when control returns to the event loop of the receiver's thread" as per the Qt documentation.
I am trying to analyse a segfault that seems to occur when accessing a heap allocated object created by a sender thread and accessed by a receiver thread.
Here is a short version of the code :
#include <QCoreApplication>
#include <QDebug>
#include <QThread>
#include <QTimer>
class Data
{
public:
Data(int data1) : m_data1(data1) {}
int data1() {
return m_data1;
}
private:
int m_data1;
};
class Sender : public QObject
{
Q_OBJECT
public:
Sender(int timeout) : m_timeout(timeout) {}
public slots:
void startSendingDatas() {
QTimer::singleShot(m_timeout, [this]() {
emit datas(new Data(3));
});
}
signals:
void datas(Data *data);
private:
int m_timeout;
};
class Receiver : public QObject
{
Q_OBJECT
public slots:
void onDatas(Data *data) {
qDebug() << "data1 = " << data->data1();
delete data; // is it always safe ?
}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
Sender sender(5000);
Receiver receiver;
QObject::connect(&sender, SIGNAL(datas(Data*)),
&receiver, SLOT(onDatas(Data*)));
QThread worker;
worker.start();
sender.moveToThread(&worker);
// now we call it asynchronously
QMetaObject::invokeMethod(&sender, "startSendingDatas");
return a.exec();
}
#include "main.moc"
Data does not inherits from QObject so deleteLater is not an option here but is it really safe to do that ?
Thank you.
Yes it is 'safe' to do so if you can garantee that the pointer will still be valid when you`ll access it.
In this simple example that seems the case. I see a potential issue in your code that might be the cause of your random crash :
Event driven objects may only be used in a single thread. Specifically, this applies to the timer mechanism and the network module. For example, you cannot start a timer or connect a socket in a thread that is not the object's thread.
from https://doc.qt.io/qt-5/threads-qobject.html paragraph Object reentrancy.
This is what you are doing : sender object is created on the main thread and on so the timer is started on this thread, then it is moved to the worker thread.
Another thing that I am not 100% confident about : you perform the connection before moving the object to the other thread. By default if nothing is said about the connection, when two objects are on the same thread, it is a direct connection, and when object are on different thread, it´s obviously a queued connection. I don´t know to which extend Qt is robust to changing the connection type when an object is moved, but I would rather first move the object and then connect it.
Note: My original post had an important omission: I left out that I had already instantiated the main QApplication instance at the beginning of main. Creating two QApplication instances is what caused the problem. Using the same QApplication instance instead of creating two fixed the issue.
My intention is to run a QApplication before the main application to iterate the available Bluetooth devices, to find a specific one. If the specific one is not found in a certain time limit, the QApplication is terminated. The first stored lambda (startDiscovery) is called as soon as QApplication::exec() is called, but the second stored lambda (cancelDiscovery) is never called! The relevant section is below:
#include <QtBluetooth/QBluetoothDeviceInfo>
#include <QtBluetooth/QBluetoothDeviceDiscoveryAgent>
#include <QtBluetooth/QBluetoothLocalDevice>
#include <QTimer>
#include <QString>
#include <QApplication>
#include <memory>
#define TARGET_BLUETOOTH_DEVICE_NAME "MyBluetoothDevice"
#define BLUETOOTH_DISCOVERY_TIMEOUT 5000 //5 second timeout
int main(int argc, char *argv[])
{
std::shared_ptr<QApplication> mainApplication{std::make_shared<QApplication>(argc, argv)};
//Error checking for no adapters and powered off devices
//omitted for sake of brevity
auto bluetoothAdapters = QBluetoothLocalDevice::allDevices();
std::shared_ptr<QBluetoothLocalDevice> localDevice{std::make_shared<QBluetoothLocalDevice>(bluetoothAdapters.at(0).address())};
std::shared_ptr<QBluetoothDeviceDiscoveryAgent> discoveryAgent{std::make_shared<QBluetoothDeviceDiscoveryAgent>(localDevice.get())};
std::shared_ptr<QBluetoothDeviceInfo> targetDeviceInfo{nullptr};
std::shared_ptr<QApplication> findBluetooth{std::make_shared<QApplication>(argc, argv)};
auto setTargetDeviceInfo = [=](QBluetoothDeviceInfo info) {
if (info.name() == TARGET_BLUETOOTH_DEVICE_NAME) {
targetDeviceInfo = std::make_shared<QBluetoothDeviceInfo>(info);
discoveryAgent->stop();
findBluetooth->exit(0);
}
};
auto cancelDiscovery = [=]() {
discoveryAgent->stop();
findBluetooth->exit(1);
};
auto startDiscovery = [=]() {
discoveryAgent->start();
};
QObject::connect(discoveryAgent.get(), &QBluetoothDeviceDiscoveryAgent::deviceDiscovered, setTargetDeviceInfo);
QTimer::singleShot(0, startDiscovery); //startDiscovery get called fine
QTimer::singleShot(BLUETOOTH_DISCOVERY_TIMEOUT, cancelDiscovery); //cancelDiscovery never gets called!
findBluetooth->exec();
//Now check if targetDeviceInfo is nullptr and run the real application etc...
mainApplication->exec();
}
Answer: discoveryAgent->start(); is basically blocking your main thread. That is why, event which gets posted by QTimer::singleShot(BLUETOOTH_DISCOVERY_TIMEOUT, cancelDiscovery); never gets processed - application executing discoveryAgent->start() and have not opportunity to look into event loop.
My original post had an important omission: I left out that I had already instantiated the main QApplication instance at the beginning of main. Creating two QApplication instances is what caused the problem. Using the same QApplication instance instead of creating two fixed the issue.
I'm using Qt5, QCoreApplication.
In order to allow for readable and easy to maintain code, I need to write a blocking method in a class/thread A that will emit a signal, connected to a slot in a different thread B, and then wait for an answer or a timeout to occur asynchronously in thread B.
I have first been thinking about what felt like a natural solution: let thread B reply with a signal connected to a slot in thread A, and somehow wait for it. It seems QEventLoop can do that for me. But I keep reading contradictory statements: that's the pattern but avoid it if you can :-).
I'm pretty sure I could achieve my purpose by blocking A on a 0 QSemaphore that B would release when ready. The code would probably not be much more complex that way.
What do you experienced Qt developers think?
Is there a good solution or do you find some symptoms of flawed analysis in my description (i.e. do you think I should never ever need to do something like that? :-))?
The key ingredient you can leverage is the Qt::BlockingQueuedConnection.
This connection type lets you pass return value from a slot. You can use it in a signal-slot connection. You can also directly invoke the slot without using a signal through the QMetaMethod::invoke / QMetaObject::invokeMethod mechanism.
#include <QDebug>
#include <QThread>
#include <QCoreApplication>
class Master : public QObject {
Q_OBJECT
public:
Q_SIGNAL bool mySignal();
Q_SLOT void process() {
if (mySignal()) { // this can be a blocking call
qDebug() << "success!";
}
thread()->quit();
}
};
class Slave : public QObject {
Q_OBJECT
public:
Q_SLOT bool mySlot() {
// do whatever processing is needed here
// It's OK to call QCoreApplication::processEvents here
return true;
}
};
int main(int argc, char** argv) {
QCoreApplication app(argc, argv);
QThread masterThread, slaveThread;
Master master;
Slave slave;
master.moveToThread(&masterThread);
slave.moveToThread(&slaveThread);
slave.connect(&master, SIGNAL(mySignal()), SLOT(mySlot()),
Qt::BlockingQueuedConnection);
masterThread.start();
slaveThread.start();
QMetaObject::invokeMethod(&master, "process");
masterThread.wait();
slaveThread.quit();
slaveThread.wait();
return 0;
}
#include "main.moc"
if you just want to emit a signal in your thread, which means your main thread will have a slot to connect you thread signal, it is simple, just emit it.
but if you want a slot in your thread, and receive signal and so something in your thread, you have to use QEventloop in you run method.
usually, I will just use QThread::wait to wait for other thread end.
be careful here, some Qt objects cannot work across the thread like QSql* and QTcpSocket....
I have a very difficult time of understanding how to make a simplest possible working
multithreaded Qt console application.
I have read tons of stuff on how to use QThread class.
Some of them say subclass QThread, others say use worker class wrapper for QThread.
After several tries and retries I could still not make a working multithreaded
Qt console application.
Right now I don't need any fancy Qt Gui.
Can someone please help me to fill the threading parts of the example code ?
It just reads one line at the time from text file and the idea is that each thread (I want to use 4 threads) that is not busy at the moment will print that line to stdout with std::cout as soon as possible. Just print it, no other fancy processing stuff for now to keep this simple for me.
#include <QCoreApplication>
#include <QFile>
#include <iostream>
/* QThread stuff here */
/* Don't know how to make it */
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
/* Create four instances of threads here and
put them to wait readed lines */
QFile file("file.txt");
file.open(QIODevice::ReadOnly | QIODevice::Text);
while(!file.atEnd()) {
/* Read lines here but where should they be saved?
Into a global variable like QList<QByteArray> list ?
So that each thread can read them from there or where ???? */
??? = file.readLine();
}
file.close();
a.exit();
}
Putting the Functionality in a Slot in a QObject
The key points are:
Remember that each QObject has a certain thread() that it "lives" in. Each thread can have an event loop running there. This event loop will deliver the events sent to the objects that "live" in this thread.
Do not derive from QThread. Start stock QThreads. They'll start an even event loop in the default implementation of QThread::run().
Implement your functionality in a slot (or a Q_INVOKABLE) method. The class obviously has to derive from QObject.
The magic happens when you send signals (using signal-slot connection, not directly) to the slot in #3. The connection from the notifier, running in the GUI thread, to the notified objects is done automatically using the Qt::QueuedConnection since the sender and the receiver objects live in different threads.
Sending a signal to such an object results in posting an event to the event queue of the thread the object is in. The event loop's event dispatcher will pick those events and call the appropriate slots. This is the power of Qt - a lot of useful stuff can be done for you.
Note that there is no notion of a "currently busy" thread. The threads execute short slots of the objects that live there. If you want to move threads between a "busy" and "not busy" states, then you'd need extra code for that.
Another way of implementing it would be to derive from QRunnable and use QThreadPool. That's in another answer.
main.cpp
#include <QCoreApplication>
#include <QTextStream>
#include <QThread>
#include <QFile>
#include <cstdio>
class Notified : public QObject {
Q_OBJECT
QTextStream m_out;
public:
Q_SLOT void notify(const QString & text) {
m_out << "(" << this << ") " << text << endl;
}
Notified(QObject *parent = 0) : QObject(parent), m_out(stdout) {}
};
class Notifier : public QObject {
Q_OBJECT
Q_SIGNAL void notification(const QString &);
public:
Notifier(QObject *parent = 0) : QObject(parent) {}
void notifyLines(const QString & filePath) {
QFile file(filePath);
file.open(QIODevice::ReadOnly | QIODevice::Text);
while (! file.atEnd()) {
emit notification(file.readLine());
}
file.close();
}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QObjectList notifieds;
QList<QThread*> threads;
Notifier notifier;
for (int i = 0; i < 4; ++i) {
QThread * thread = new QThread(&a); // thread owned by the application object
Notified * notified = new Notified; // can't have an owner before it's moved to another thread
notified->moveToThread(thread);
thread->start();
notifieds << notified;
threads << thread;
notified->connect(¬ifier, SIGNAL(notification(QString)), SLOT(notify(QString)));
}
notifier.notifyLines("file.txt");
foreach (QThread *thread, threads) {
thread->quit();
thread->wait();
}
foreach (QObject *notified, notifieds) delete notified;
a.exit();
}
#include "main.moc"
For your purposes I would not use QThread at all but the classes from QtConcurrent.
Something simple like (assuming you have C++11):
while(!file.atEnd()) {
QString line = file.readLine();
QtConcurrent::run([line]
{
qDebug() << line;
});
}
Though I'm still not sure what this should give you on a high level.
Below link can be useful for you for the information related to using threads in Qt
http://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/
If you only want file reading to be done in Asynchronous ways Qt is having several alternate techniques like QtConcurrent.
http://qt-project.org/doc/qt-4.8/threads-qtconcurrent.html
Here is some example code to help you for using QtConcurrent
Running a Function in a Separate Thread
extern QString aFunction();
QFuture<void> future = QtConcurrent::run(aFunction);
aFunction should contain the code for reading the file .
You can return the read data in the following way
QFuture<QString> future = QtConcurrent::run(aFunction);
...
QString result = future.result();
Note that the QFuture::result() function blocks and waits for the result to become available. Use QFutureWatcher to get notification when the function has finished execution and the result is available.
Hope this helps. All the above code is taken from Qt documentation.
Putting the functionality in a QRunnable
Perhaps the solution that's closest to your explicit needs would use a QThreadPool. It does what you want: it picks a non-busy thread from its pool, and runs the worker there. If there are no free threads, it'll add the runnable to a run queue that gets drained each time a free thread becomes available.
Note, though, that your explicit wish of having a thread state, namely busy/non-busy, does not really mesh at all with a network penetration system that needs to wait for a reply before trying each new password. You'll want it based on QObjects. I'll modify my other answer to show how you might do it while managing network connections. It's very, very wasteful to waste threads on busy waiting for network answers. You do not want to do that. It will perform poorly.
Your application is I/O bound and could, pretty much, run on one thread without much in the way of undue performance loss. Only if you have a huge network pipe and are testing tens of thousands of accounts at the same time would you need more than one thread. I'm serious.
#include <QCoreApplication>
#include <QTextStream>
#include <QRunnable>
#include <QThreadPool>
#include <QFile>
#include <cstdio>
class Data : public QString {
public:
Data(const QString & str) : QString(str) {}
};
class Worker : public QRunnable {
QTextStream m_out;
Data m_data;
public:
void run() {
// Let's pretend we do something serious with our data here
m_out << "(" << this << ") " << m_data << endl;
}
Worker(const Data & data) : m_out(stdout), m_data(data) {}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QThreadPool * pool = QThreadPool::globalInstance();
QFile file("file.txt");
file.open(QIODevice::ReadOnly | QIODevice::Text);
while (! file.atEnd()) {
const Data data(file.readLine());
Worker * worker = new Worker(data);
pool->start(worker);
}
file.close();
pool->waitForDone();
}