I am prity new on Qt an got some issues with QSharedPointer passing around within signals. I am working with two threads (UI and a worker). The worker sends signals to the UI using signals that contain QSharedPointer of a custom QObject:
class MyObject : QObject {...}
class Window : public QWidget {
Q_OBJECT
public slots:
void onFound(QSharedPointer<MyObject>);
}
class Worker : public QObject {
Q_OBJECT
public signals:
void found(QSharedPointer<MyObject>);
}
I connect the workers found with the windows onFound with Qt::QueuedConnection because they live in different Threads and the communication therefore has to be asynchronous.
Now I observe the folowing behavior, when I pass a the last QSharedPointer refering to my object:
The signal moc casts the reference to my pointer to void* and arcives it.
The function returns resulting in the shared pointer and the corresponding object to be destroyed.
That's not what I expected - though it's reasonable. Are QSharedPointer in general designed to be passed through signals that way? And if so, is there a mecanism to keep a reference while it is queued?
I considered the folowing solutions, but I'm not totally fine with neither of them:
Keep a reference somewhere that keeps reference while it is queued. But where is a reasonable place to do that and when should I let it go.
Make the connection Qt::DirectConnection but then I am still have to switch the thread somehow (same situation as before)
Introduce a new signal/slot with std::function parameter for passing a lambda function to be executed in the target thread and capturing a copy of my shared pointer. (This is my current solution but it's not perty elegant, isn't it?)
Do you have any other suggestions or ideas?
The signal return does not destroy the corresponding object. The QMetaObject::activate call copies the shared pointer. Here's the implementation of send signal:
// SIGNAL 0
void IO::send(const QSharedPointer<Unique> & _t1)
{
void *_a[] = { nullptr, const_cast<void*>(reinterpret_cast<const void*>(&_t1)) };
QMetaObject::activate(this, &staticMetaObject, 0, _a);
}
You're probably experiencing a race: by the time the thread where the signal was emitted resumes execution, the destination thread has already received the object. Thus it appears in the emitting thread that the object is gone - because, by that time, it is. Yet the target object receives the instance just fine. It works fine.
The example below illustrates that it works in both single- and multi-threaded cases, and then reproduces your problem by ensuring that the race is always won by the destination thread:
// https://github.com/KubaO/stackoverflown/tree/master/questions/shared-pointer-queued-49133331
#include <QtCore>
class Unique : public QObject {
Q_OBJECT
int const m_id = []{
static QAtomicInteger<int> ctr;
return ctr.fetchAndAddOrdered(1);
}();
public:
int id() const { return m_id; }
};
class IO : public QObject {
Q_OBJECT
int m_lastId = -1;
public:
Q_SIGNAL void send(const QSharedPointer<Unique> &);
Q_SLOT void receive(const QSharedPointer<Unique> & u) {
m_lastId = u->id();
}
int lastId() const { return m_lastId; }
};
int main(int argc, char ** argv) {
Q_ASSERT(QT_VERSION >= QT_VERSION_CHECK(5,9,0));
QCoreApplication app{argc, argv};
IO src, dst;
QObject::connect(&src, &IO::send, &dst, &IO::receive, Qt::QueuedConnection);
QSharedPointer<Unique> u;
QWeakPointer<Unique> alive;
int id = -1;
// Single-threaded case
alive = (u.reset(new Unique), u);
id = u->id();
Q_ASSERT(dst.lastId() != id); // the destination hasn't seen the object yet
emit src.send(u);
u.reset();
Q_ASSERT(!u); // we gave up ownership of the object
Q_ASSERT(dst.lastId() != id); // the destination mustn't seen the object yet
Q_ASSERT(alive); // the object must be still alive
app.processEvents();
Q_ASSERT(dst.lastId() == id); // the destination must have seen the object now
Q_ASSERT(!alive); // the object should have been destroyed by now
// Multi-threaded setup
struct Thread : QThread { ~Thread() { quit(); wait(); } } worker;
worker.start();
dst.moveToThread(&worker);
QSemaphore s_src, s_dst;
// This thread wins the race
alive = (u.reset(new Unique), u);
id = u->id();
Q_ASSERT(dst.lastId() != id);
QTimer::singleShot(0, &dst, [&]{ s_src.release(); s_dst.acquire(); });
// stop the thread
s_src.acquire(); // wait for thread to be stopped
emit src.send(u);
QTimer::singleShot(0, &dst, [&]{ s_src.release(); });
// resume the main thread when done
u.reset();
Q_ASSERT(!u);
Q_ASSERT(alive); // we won the race: the object must be still alive
s_dst.release(); // get the thread running
s_src.acquire(); // wait for the thread to be done
Q_ASSERT(dst.lastId() == id);
Q_ASSERT(!alive);
// The other thread wins the race
alive = (u.reset(new Unique), u);
id = u->id();
Q_ASSERT(dst.lastId() != id);
emit src.send(u);
QTimer::singleShot(0, &dst, [&]{ s_src.release(); });
// resume the main thread when done
u.reset();
s_src.acquire(); // wait for worker thread to be done
Q_ASSERT(!u);
Q_ASSERT(!alive); // we lost the race: the object must be gone
Q_ASSERT(dst.lastId() == id); // yet the destination has received it!
// Ensure the rendezvous logic didn't mess up
Q_ASSERT(id == 2);
Q_ASSERT(!s_src.available());
Q_ASSERT(!s_dst.available());
}
#include "main.moc"
Related
I am trying to analyse a segfault that seems to occur when accessing a heap allocated object created by a sender thread and accessed by a receiver thread.
Here is a short version of the code :
#include <QCoreApplication>
#include <QDebug>
#include <QThread>
#include <QTimer>
class Data
{
public:
Data(int data1) : m_data1(data1) {}
int data1() {
return m_data1;
}
private:
int m_data1;
};
class Sender : public QObject
{
Q_OBJECT
public:
Sender(int timeout) : m_timeout(timeout) {}
public slots:
void startSendingDatas() {
QTimer::singleShot(m_timeout, [this]() {
emit datas(new Data(3));
});
}
signals:
void datas(Data *data);
private:
int m_timeout;
};
class Receiver : public QObject
{
Q_OBJECT
public slots:
void onDatas(Data *data) {
qDebug() << "data1 = " << data->data1();
delete data; // is it always safe ?
}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
Sender sender(5000);
Receiver receiver;
QObject::connect(&sender, SIGNAL(datas(Data*)),
&receiver, SLOT(onDatas(Data*)));
QThread worker;
worker.start();
sender.moveToThread(&worker);
// now we call it asynchronously
QMetaObject::invokeMethod(&sender, "startSendingDatas");
return a.exec();
}
#include "main.moc"
Data does not inherits from QObject so deleteLater is not an option here but is it really safe to do that ?
Thank you.
Yes it is 'safe' to do so if you can garantee that the pointer will still be valid when you`ll access it.
In this simple example that seems the case. I see a potential issue in your code that might be the cause of your random crash :
Event driven objects may only be used in a single thread. Specifically, this applies to the timer mechanism and the network module. For example, you cannot start a timer or connect a socket in a thread that is not the object's thread.
from https://doc.qt.io/qt-5/threads-qobject.html paragraph Object reentrancy.
This is what you are doing : sender object is created on the main thread and on so the timer is started on this thread, then it is moved to the worker thread.
Another thing that I am not 100% confident about : you perform the connection before moving the object to the other thread. By default if nothing is said about the connection, when two objects are on the same thread, it is a direct connection, and when object are on different thread, it´s obviously a queued connection. I don´t know to which extend Qt is robust to changing the connection type when an object is moved, but I would rather first move the object and then connect it.
My situation is that I have a QWidget-derived class, MyWidget, that will create a QThread-derived class (WorkerThread) to do some uninterruptible, blocking work in its run() method. The results of this are a heap-allocated instance of a QObject-derived class (DataClass) which is then received and processed by MyWidget. MyWidget is a transitory widget, though, and may be deleted while WorkerThread is still running due to user action.
Here's some pseudo-code to illustrate this:
#include <QThread>
#include <QWidget>
class DataClass : public QObject {
Q_OBJECT
public:
// contains some complex data
};
class WorkerThread : public QThread {
Q_OBJECT
public:
virtual void run() {
DataClass *result = new DataClass;
doSomeReallyLongUninterruptibleWork(result);
emit workComplete(result);
}
signals:
void workComplete(DataClass *);
};
class MyWidget : public QWidget {
Q_OBJECT
public:
void doBlockingWork() {
WorkerThread *worker = new WorkerThread;
connect(worker, &WorkerThread::finished, worker, &WorkerThread::deleteLater);
connect(worker, &WorkerThread::workComplete, this, &MyWidget::processData);
worker->start();
}
public slots:
void processData(DataClass *result) {
// Do some stuff
delete result;
// Assuming MyWidget still exists when WorkerThread has finished, no memory has leaked
}
};
Normally the correct "Qt" way to return the results of a worker thread is to have it emit a signal with its arguments being the result of its work, as illustrated above. That's fine for data that can be copied, but since the result is a pointer to a heap-allocated object, I have to be careful to make sure that memory gets freed.
And normally that wouldn't be a problem, because since WorkerThread has finished, I can safely pass the pointer to DataClass to MyWidget, have it process DataClass, and then free it.
The problem is that, as I said earlier, MyWidget is transitory and may be destroyed before WorkerThread is finishing. In this scenario, how can I ensure that the instance of DataClass gets freed one way or the other?
In particular, I'm looking for solutions that have some elegance to them, meaning that it takes advantage of Qt's features and preferably makes it so that WorkerThread maintains its separation from MyWidget so that WorkerThread doesn't need to know anything about it or any other class that might create it. I'm also open to ideas that improve upon the pattern that I'm already using.
Use smart pointer (e.g., QSharedPointer) instead a normal pointer:
DataClass *result = new DataClass;
should be replaced with
QSharedPointer<DataClass> result = QSharedPointer<DataClass>(new DataClass);
Then, you could safely pass it somewhere and do not worry about deleting it. When it is out of the last scope where it can be used, the object will be automatically destroyed.
The worker should push the result to the main thread, to indicate that it's safe to use there (per QObject semantics). The result should be auto-deleted in the main thread after everyone interested has been notified of the completion of the work. It is a minimal change:
void run() override {
auto result = new DataClass;
doSomeReallyLongUninterruptibleWork(result);
result->moveToThread(qApp->thread()); // added
emit workComplete(result);
QObject::connect(this, &QThread::finished, result, &QObject::deleteLater); // added
}
You're guaranteed that deleteLater will be invoked after the last handler of workComplete has finished in the main thread.
A single object in the main thread might wish to retain the results longer. This can be indicated by setting the parent on the result object. The object shouldn't be deleted then:
...
QObject::connect(this, &QThread::finished, result, [result]{
if (!result->parent()) result->deleteLater();
});
If you intend that multiple objects in the main thread retain the results longer, you should be using a QSharedPointer in the workComplete's argument, and you must never set the parent of the results: a non-null parent and a QSharedPointer are mutually incompatible: the former indicates a unique ownership by a parent, the latter indicates a shared ownership.
It is necessary to move the DataClass object to the main thread to avoid a race on DataClass::thead() and to allow deleteLater to work:
Worker Thread: emit workComplete(result)
Main Thread: start using result, result.thread() is the worker instance.
Worker Thread: finishes
Main Thread: result.thread() is now nullptr while the main thread is using it.
This might not be a problem, but usually indicates poor design. As soon as you start using more QObject features of DataClass, it turns the latent bug into a real bug: e.g. deleteLater won't work, timers won't work, etc.
Furthermore, destructing a QObject in any thread other than its thread is not supported. Suppose that you had your original code. The following could happen and leads to undefined behavior:
Worker Thread: emit workComplete(result)
Main Thread: start using result, result.thread() is the worker instance.
Main Thread: delete result. QObject::~QObject is invoked in qApp->thread() but result->thread() is the different, still live instance of the worker thread.
If you wish to catch such issues, add:
DataClass::~DataClass() {
Q_ASSERT(thread() == nullptr || thread() == QThread::currentThread());
...
}
It's OK to destruct a threadless object, but such objects are not fully functional: you can't deleteLater them, their timers don't work, they don't receive events, etc.
The necessity of a parent check prior to deleteLater depends on whether you intend to prolong the existence of the result past the code connected to workComplete.
The "obvious" use of a shared pointer doesn't make it clear which thread can safely access the result iff the result isn't thread-safe. It also does nothing by itself to fix the fact that once the worker finishes, the QObject is half-functional as there's no event loop associated with it. I believe that your intent is that only one thread may own the result, so that its methods don't have to be thread-safe. Luckily, QObject's semantics already express this clearly: the object's thread() is the one authorized to act on the object.
Any recipients of workComplete in the main thread will get to process the results before they vanish. If any object in the main thread wants to take ownership of the result, it can - by setting the parent. Otherwise, as soon the workComplete handlers are done, if none have claimed ownership, the result will get deleted from the main event loop.
Change the QTimer::singleShot(1000, w.data(), [&]{ w.reset(); }) timer to 2500ms to have the widget outlive the worker thread and note the difference in behavior depending on whether it claimed ownership.
Complete example:
// https://github.com/KubaO/stackoverflown/tree/master/questions/worker-shared-37956073
#include <QtCore>
struct DataClass : public QObject {
DataClass() { qDebug() << __FUNCTION__; }
~DataClass() { qDebug() << __FUNCTION__; }
};
void doSomeReallyLongUninterruptibleWork(DataClass*) { QThread::sleep(2); }
class WorkerThread : public QThread {
Q_OBJECT
public:
void run() override {
auto result = new DataClass;
doSomeReallyLongUninterruptibleWork(result);
result->moveToThread(qApp->thread());
emit workComplete(result);
QObject::connect(this, &QThread::finished, result, [result]{
if (!result->parent()) {
qDebug() << "DataClass is unclaimed and will deleteLater";
result->deleteLater();
}
});
}
Q_SIGNAL void workComplete(DataClass*);
};
class MyWidget : public QObject {
void processData(DataClass * result) {
// Do stuff with result
// Retain ownership (optional)
if (true) result->setParent(this);
}
public:
void doBlockingWork() {
auto worker = new WorkerThread;
connect(worker, &WorkerThread::workComplete, this, &MyWidget::processData);
connect(worker, &WorkerThread::finished, worker, &WorkerThread::deleteLater);
worker->start();
}
~MyWidget() { qDebug() << __FUNCTION__; }
};
int main(int argc, char ** argv) {
QCoreApplication app{argc, argv};
QScopedPointer<MyWidget> w{new MyWidget};
w->doBlockingWork();
QTimer::singleShot(1000, w.data(), [&]{ w.reset(); });
QTimer::singleShot(3000, qApp, &QCoreApplication::quit);
return app.exec();
}
#include "main.moc"
You could also forgo the use of an explicit thread, and use QtConcurrent::run instead. There's no clear advantage to that, I'm showing it here just to indicate that either approach is feasible.
#include <QtConcurrent>
struct DataClass : public QObject {
Q_SIGNAL void ready();
Q_OBJECT
};
// Let's not pollute the default pool with long-running stuff
Q_GLOBAL_STATIC(QThreadPool, longPool)
class MyWidget : public QObject {
void processData(DataClass * result) {
// Do stuff with result
// Retain ownership (optional)
if (true) result->setParent(this);
}
public:
void doBlockingWork() {
auto result = new DataClass;
connect(result, &DataClass::ready, this, [=]{ MyWidget::processData(result); });
result->moveToThread(nullptr);
QtConcurrent::run(longPool, [result]{
result->moveToThread(QThread::currentThread());
doSomeReallyLongUninterruptibleWork(result);
result->moveToThread(qApp->thread());
emit result->ready();
QTimer::singleShot(0, result, [result]{
if (!result->parent()) result->deleteLater();
});
});
}
};
I'm very new to using QThread. I'm using QThread to grab images from an Axis Ip Camera. In the following snippet of the code I'm moving the camera class to a new thread:
QThread camThread;
camera = new AxisNetworkCamera();
camera->moveToThread(&camThread);
camThread.start();
connect(camera, SIGNAL(imageUpdate(QImage)), this, SLOT(upDateImage(QImage)));
connect(camera, SIGNAL(cameraDisconnected()), this, SLOT(cameraDisconnected()));
connect(&camThread, &QThread::finished, &camThread, &QThread::deleteLater);
connect(camera, &AxisNetworkCamera::destroyed, &camThread, &QThread::quit);
I'm invoking the function that starts the camera:
QMetaObject::invokeMethod(camera, "deviceStartup", Qt::QueuedConnection, Q_ARG(QString, QString::fromStdString(streamUrl)));
The application runs fine and also closes fine when I close it, but what I'm worried is about a couple of warning messages.
First one is when I start the camera:
Type conversion already registered from type QPair<QByteArray,QByteArray> to type QtMetaTypePrivate::QPairVariantInterfaceImpl
2nd one is when I close the application:
QThreadStorage: Thread 0x7fffb8004da0 exited after QThreadStorage 7 destroyed
Should I be worried about these messages? Do they, specially the second 1, mean any memory mismanagement?
Thanks
The code you've posted makes no sense. QThread is not dynamically allocated so you cannot delete it: the deleteLater call will crash. Probably it never gets executed, since you show no code that would stop the thread anyway. There's also no point in destroying the thread after the camera has been destroyed.
The simplest way to do things safely would be to hold the camera and thread by value in your class, and declare them in proper order so that the thread is destroyed before the camera. At that point, the camera becomes threadless and will be safe to destroy in any thread.
There's also a nicer way to invoke methods in remote threads than using invokeMethod:
class Thread : public QThread {
using QThread::run; // final
public:
Thread(QObject*parent = 0): QThread(parent) {}
~Thread() { quit(); wait(); }
};
// See http://stackoverflow.com/a/21653558/1329652 for details about invoke.
template <typename F> void invoke(QObject * obj, F && fun) {
if (obj->thread == QThread::currentThread())
return fun();
QObject src;
QObject::connect(&src, &QObject::destroyed, obj, std::forward<F>(fun));
}
class MyObject : public QObject {
Q_OBJECT
AxisNetworkCamera camera;
Thread camThread { this };
Q_SLOT void updateImage(const QImage &);
Q_SLOT void cameraDisconnected();
public:
MyObject(QObject * parent = 0) : QObject(parent)
{
connect(&camera, &AxisNetworkCamera::imageUpdate, this, &MyObject::updateImage);
connect(&camera, &AxisNetworkCamera::cameraDisconnected, this, &MyObject::cameraDisconnected);
camera.moveToThread(&camThread);
camThread.start();
}
void startCamera(const QString & streamUrl) {
invoke(&camera, [=]{ camera.deviceStartup(streamUrl); });
}
};
I'm confused about threads and event loops in Qt.
A QThread normally runs exec() in run(). But when you override run(), there will not be an event loop.
This (older) doc states that calling deleteLater() on objects that are created in a thread without an event loop doesn't work:
If no event loop is running, events won't be delivered to the object.
For example, if you create a QTimer object in a thread but never call
exec(), the QTimer will never emit its timeout() signal. Calling
deleteLater() won't work either. (These restrictions apply to the main
thread as well.)
However, look at the following code:
class MyObject : public QObject
{
Q_OBJECT
QString content;
public:
MyObject(QObject *parent = 0);
~MyObject();
};
class MyThread : public QThread
{
Q_OBJECT
public:
explicit MyThread(QObject *parent = 0);
void run();
signals:
public slots:
};
MyObject::MyObject(QObject *parent) :
QObject(parent),
content("foobar")
{}
MyObject::~MyObject()
{
// This code is still executed before I close the program. How?
qDebug() << "Destroying MyObject";
}
MyThread::MyThread(QObject *parent) :
QThread(parent)
{}
void MyThread::run()
{
// Creating a heap object in a thread that does not have
// an event loop (because I reimplemented run()).
MyObject * objectification = new MyObject();
sleep(1);
objectification->deleteLater();
}
So why does the deletelater() call still post an event that is picked up?
As the Qt docs state for deleteLater: -
Since Qt 4.8, if deleteLater() is called on an object that lives in a thread with no running event loop, the object will be destroyed when the thread finishes.
The object is still being deleted when no event loop exists. If you look at the source code for QObject::deleteLater, you'll see that an event is posted:-
void QObject::deleteLater()
{
QCoreApplication::postEvent(this, new QDeferredDeleteEvent());
}
So, let's look at what happens when a thread is deleted. QThreadData's destructor includes this:-
for (int i = 0; i < postEventList.size(); ++i) {
const QPostEvent &pe = postEventList.at(i);
if (pe.event) {
--pe.receiver->d_func()->postedEvents;
pe.event->posted = false;
delete pe.event;
}
}
As we see, although there's no event loop, the event list is still available.
If we look more closely into QThreadPrivate (just taking one platform as an example, in this case unix), you'll see that when the thread finishes, it forwards all deferred deleted messages, so they can continue to be processed:
QCoreApplication::sendPostedEvents(0, QEvent::DeferredDelete);
I have following code:
class A : public QObject
{
Q_OBJECT
public:
A() : QObject()
{
moveToThread(&t);
t.start();
}
~A()
{
t.quit();
t.wait();
}
void doSomething()
{
QMetaObject::invokeMethod(this,"doSomethingSlot");
}
public slots:
void doSomethingSlot()
{
//do something
emit ready();
}
signals:
void ready();
private:
QThread t;
}
The question why from doSomething it must be call via QMetaObject::invokeMethod. I know that there is something with connection type. Could some one explain what is under the hood?
As you haven't specified a Qt::ConnectionType, the method will be invoked as Qt::AutoConnection, which means that it will be invoked synchronously (like a normal function call) if the object's thread affinity is to the current thread, and asynchronously otherwise. "Asynchronously" means that a QEvent is constructed and pushed onto the message queue, and will be processed when the event loop reaches it.
The reason to use QMetaObject::invokeMethod if the recipient object might be in another thread is that attempting to call a slot directly on an object in another thread can lead to corruption or worse if it accesses or modifies non-thread-safe data.
I like this trick:
void A:doSomethingSlot()
{
if (thread()!=QThread::currentThread()) {
QMetaObject::invokeMethod(this,"doSomethingSlot", Qt::QueuedConnection);
return;
}
// this is done always in same thread
...
emit ready();
}