I was thinking about writing method like this:
QString getData() {
QNetworkReply *reply = getReply();
reply->deleteLater();
return QString::fromUtf8(reply->readAll()).trimmed();
}
Is it safe?
If I'm forced to write this like this:
QString getData() {
QNetworkReply *reply = getReply();
QString result = QString::fromUtf8(reply->readAll()).trimmed();
reply->deleteLater();
return result;
}
I'm copping QString twice (am I?, once it's put into result and second when returning it by value), which I wanted to avoid.
From the deleteLater docs:
Schedules this object for deletion.
The object will be deleted when control returns to the event loop. If the event loop is not running when this function is called (e.g. deleteLater() is called on an object before QCoreApplication::exec()), the object will be deleted once the event loop is started.
So what you are doing there is safe. Obviously handing out references or pointers to that object (or its members) that might be persisted is wrong. But if you're returning copies, you're fine.
But what you're doing might or might not do what you want to do. readAll doesn't block, it returns the data currently available. Meaning that a single call to readAll might only read a partial response - unless you've ensured that all data has arrived through other means.
Other things to note, from the docs:
Note that entering and leaving a new event loop (e.g., by opening a modal dialog) will not perform the deferred deletion; for the object to be deleted, the control must return to the event loop from which deleteLater() was called.
So the only thing to worry about when doing this type of thing would be calling functions that somehow re-enters the "current" event loop. But that won't happen if that is done via QCoreApplication::processEvents:
In event you are running a local loop which calls this function continuously, without an event loop, the DeferredDelete events will not be processed.
So that's covered too. The deferred deletion logic is pretty complex, but safe under normal circumstances. If you're digging very deep into the Qt internals (or calling code that might do something fishy there), be defensive. But for normal code flow, deleteLater is safe as long as you don't have dangling references (or pointers) that might persist.
What does deleteLater do? From it's name, I would expect that it registers the object for deletion at some later point in time (end of transaction? end of session?). If so, you can safely use it as long as that later point in time has not occurred. The only issue is knowing when that point occurs, but for things like end of transaction or end of session, you're probably safe—if your function was called within a transaction or session, the transaction or session will not end until you return.
It is safe, but you'd better not use deleteLater at all, because
The object will be deleted when control returns to the event loop. If
the event loop is not running when this function is called (e.g.
deleteLater() is called on an object before QCoreApplication::exec()),
the object will be deleted once the event loop is started.
means that object can be deleted mmm... never. This pretend to work like GC, but it is even worse:
class A: public QObject
{
char x[10000000];
};
void process()
{
A* a = new A();
//delete a;
a->deleteLater();
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
for (int k = 0; k < 1000000; ++k) {
process();
}
return a.exec();
}
At least it is not C++ idiomatic, that uses RAII.
From the other size, copying QString is a cheap operation, because QSring uses copy-on-write ideome.
Related
I am writing a program that uses QtConcurrent to start threads. In my case, I use it to render a QGraphicsView when I use the mouse scroll.
I am starting the threads using the following code:
if (future.isRunning()) {
future.cancel();
}
future = QtConcurrent::run(this,&AeVectorLayer::render, renderparams, pos);
watcher.setFuture(future);
When the thread is finished I catch the signal finished with a QfutureWatcher.
This is my render function:
QList<AeGraphicsItem*> AeVectorLayer::render(Ae::renderParams renderparams, int pos)
{
AeVectorHandler *coso = new AeVectorHandler();
coso->openDataset(AeLayer::absoluteFilePath);
coso->setTransformCoordinates(myEPSG);
QList<AeGraphicsItem*> bla = coso->getItems(renderparams.sceneRect.x(),
renderparams.sceneRect.y(), renderparams.sceneRect.width(),
renderparams.sceneRect.height(), renderparams.zoom, color, this);
for (int i = 0; i < bla.size(); i++)
bla.at(i)->setZValue((qreal)pos);
delete coso;
return bla;
}
As you can see, I have a QList<QGraphicsItem*> in my render function. How can I destroy this list when the future is cancelled? I undestand that in my code I am redefining the future variable but I do not know how to avoid it.
Stop trying to manually manage memory and instead use a smart pointer that fits your use case. Because you use move-unaware QFuture, you will need an std::shared_ptr. Once the QFuture/QFutureWatchers go out of scope, and you hold no more shared_ptr instances, the resource will be deleted. So in your case your render function should return a QList<std::shared_ptr<AeGraphicsItem>>. Be careful when you transfer ownership from the shared_ptrs to e.g. a QGraphicsScene: you must release from the shared_ptr on ownership transfer.
Note that your isRunning check followed by cancel is fundamentally flawed: the future could be running when you call isRunning but be finished by the time you call cancel. If you want to cancel it, just call cancel. Also note that you cannot meaningfully cancel QFutures returned by QtConcurrent::run so what you're doing is very very wrong in and of itself.
My program has a callback function which is called to handle notifications that are received in the form of objects. Because we can handle hundreds a second, this callback function handles the events by spawning a separate thread to handle each one. This is the callback function:
void _OnEvent(LPCTSTR eventID, CNotification cNotificaton) {
if (_pActiveDoc) {
Param_Event* p = new Param_Event;
p->pDoc = _pActiveDoc;
p->lpszEventID = eventID;
p->cNotification = cNotification;
AfxBeginThread(ProcessEvent,p);
}
}
My query comes from the fact that is passed to the callback method is initially created on the stack, and is therefore (according to my understanding) limited to the scope of the calling method:
void CallingMethod(CString strEventID) {
CNotification cNotification;
// Fill in the details of the notification
_OnEvent(strEventID,cNotification);
}
CNotification has a full copy constructor, and since the Param_Event object is created on the heap, my belief was that this would allow the original CNotification object to fall out of scope safely, with the spawned thread working from its own "private" CNotification object that exists until the Param_Event object is deleted with delete. The fact is, however, that we are getting (rare but occasional) crashing, and I am wondering if perhaps my belief here is incorrect: is it possible that the spawned thread is still accessing the original object somehow? If this was the case, this would explain the crashing by the rare occurrence of the object both falling out of scope and being overwritten in memory, thus creating a memory access exception.
Could I be right? Is there anything actually wrong with the method I am using? Would it be safer create the notification object on the heap initially (this would mean changing a lot of our code), or building a new object on the heap to pass to the spawned thread?
For reference, here is my ProcessEvent() method:
Param_TelephoneEvent *p = (Param_TelephoneEvent*)lParam;
p->pDoc->OnTelephoneEvent(p->lpszEventID,p->cNotification);
delete p;
return 0;
All advice welcome. Thanks in advance!
Edit: Copy constructor:
CNotification& CNotification::operator=(const CNotification &rhs)
{
m_eamspeMostRecentEvent = rhs.m_eamspeMostRecentEvent;
m_eamtcsCallStatusAtEvent = rhs.m_eamtcsCallStatusAtEvent;
m_bInbound = rhs.m_bInbound;
strcpy(m_tcExtension , rhs.m_tcExtension);
strcpy(m_tcNumber, rhs.m_tcNumber);
strcpy(m_tcName,rhs.m_tcName);
strcpy(m_tcDDI,rhs.m_tcDDI);
strcpy(m_tcCallID,rhs.m_tcCallID);
strcpy(m_tcInterTelEvent,rhs.m_tcInterTelEvent);
m_dTimestamp = rhs.m_dTimestamp;
m_dStartTime = rhs.m_dStartTime;
m_nCallID = rhs.m_nCallID;
return *this;
}
I have a scenario where an anonymous QObject starts an asynchronous operation by emitting a signal. The receiving slot stores the QObject-pointer and sets a property of this object later. The object could be gone meanwhile.
So, is there a safe way to check if this pointer still valid?
P.S.:
I'm aware of QObject::destroyed signal, which I could connect to the object supposed to call the setProperty of that pointer. But I wonder, if it works easier.
This is a great question, but it is the wrong question.
Is there a way to check if the pointer is valid? Yes. QPointer is designed specifically to do that.
But the answer to this question is useless if the object lives in another thread! You only know whether it's valid at a single point in time - the answer is not valid immediately afterwards.
Absent other mechanisms, it is useless to hold a QPointer to an object in a different thread - it won't help you. Why? Look at this scenario:
Thread A Thread B
1. QPointer returns a non-zero pointer
2. deletes the object
3. Use the now-dangling pointer
I'm aware of QObject::destroyed signal, which I could connect to the object supposed to call the setProperty of that pointer. But I wonder, if it works easier.
The destroyed signals are useless when sent using queued connections - whether within a thread, or across thread boundaries. They are meant to be used within one thread, using direct connections.
By the time the target thread's event loop picks up the slot call, the originating object is long gone. Worse - this is always the case in a single-threaded application. The reason for the problem is the same as with the QPointer: the destroyed signal indicates that the object is no longer valid, but it doesn't mean that it was valid before you received the signal unless you're using a direct connection (and are in the same thread) or you're using a blocking queued connection.
Using the blocking queued connection, the requesting object's thread will block until the async thread finishes reacting to object's deletion. While this certainly "works", it forces the two threads to synchronize on a resource with sparse availability - the front spot in the async thread's event loop. Yes, this is literally what you compete for - a single spot in a queue that can be arbitrarily long. While this might be OK for debugging, it has no place in production code unless it's OK to block either thread to synchronize.
You are trying to work very hard around the fact that you're passing a QObject pointer between threads, and the object's lifetime, from the point of view of the receiving thread, is uncontrolled. That's your problem. You'd solve everything by not passing a raw object pointer. Instead, you could pass a shared smart pointer, or using signal-slot connections: those vanish whenever either end of the connection is destructed. That's what you'd want.
In fact, Qt's own design patterns hint at this. QNetworkReply is a QObject not only because it is a QIODevice, but because it must be to support direct indications of finished requests across thread boundaries. In light of a multitude of requests being processed, connecting to QNetworkAccessManager::finished(QNetworkReply*) can be a premature pessimization. Your object gets notified of a possibly very large number of replies, but it really is only interested in one or very few of them. Thus there must be a way to notify the requester directly that its one and only request is done - and that's what QNetworkReply::finished is for.
So, a simple way to proceed is to make the Request be a QObject with a done signal. When you ready the request, connect the requesting object to that signal. You can also connect a functor, but make sure that the functor executes in the requesting object's context:
// CORRECT
connect(request, &Request::done, requester, [...](...){...});
// WRONG
connect(request, &Request::done, [...](...){...});
The below demonstrates how it could be put together. The requests' lifetimes are managed through the use of a shared (reference-counting) smart pointer. This makes life rather easy. We check that no requests exist at the time main returns.
#include <QtCore>
class Request;
typedef QSharedPointer<Request> RequestPtr;
class Request : public QObject {
Q_OBJECT
public:
static QAtomicInt m_count;
Request() { m_count.ref(); }
~Request() { m_count.deref(); }
int taxIncrease;
Q_SIGNAL void done(RequestPtr);
};
Q_DECLARE_METATYPE(RequestPtr)
QAtomicInt Request::m_count(0);
class Requester : public QObject {
Q_OBJECT
Q_PROPERTY (int catTax READ catTax WRITE setCatTax NOTIFY catTaxChanged)
int m_catTax;
public:
Requester(QObject * parent = 0) : QObject(parent), m_catTax(0) {}
Q_SLOT int catTax() const { return m_catTax; }
Q_SLOT void setCatTax(int t) {
if (t != m_catTax) {
m_catTax = t;
emit catTaxChanged(t);
}
}
Q_SIGNAL void catTaxChanged(int);
Q_SIGNAL void hasRequest(RequestPtr);
void sendNewRequest() {
RequestPtr req(new Request);
req->taxIncrease = 5;
connect(req.data(), &Request::done, this, [this, req]{
setCatTax(catTax() + req->taxIncrease);
qDebug() << objectName() << "has cat tax" << catTax();
QCoreApplication::quit();
});
emit hasRequest(req);
}
};
class Processor : public QObject {
Q_OBJECT
public:
Q_SLOT void process(RequestPtr req) {
QThread::msleep(50); // Pretend to do some work.
req->taxIncrease --; // Figure we don't need so many cats after all...
emit req->done(req);
emit done(req);
}
Q_SIGNAL void done(RequestPtr);
};
struct Thread : public QThread { ~Thread() { quit(); wait(); } };
int main(int argc, char ** argv) {
struct C { ~C() { Q_ASSERT(Request::m_count == 0); } } check;
QCoreApplication app(argc, argv);
qRegisterMetaType<RequestPtr>();
Processor processor;
Thread thread;
processor.moveToThread(&thread);
thread.start();
Requester requester1;
requester1.setObjectName("requester1");
QObject::connect(&requester1, &Requester::hasRequest, &processor, &Processor::process);
requester1.sendNewRequest();
{
Requester requester2;
requester2.setObjectName("requester2");
QObject::connect(&requester2, &Requester::hasRequest, &processor, &Processor::process);
requester2.sendNewRequest();
} // requester2 is destructed here
return app.exec();
}
#include "main.moc"
It is impossible to check is that pointer still valid. So, the only safe way here is to inform receiving part about deleting of that QObject (and in multithreading case: before accessing to object you need to check and block it to be sure, that the object will not be deleted in another thread right after check). The reason of it is simple:
Theoretically it is possible that after deleting of initial object, system will put another object in that memory (so pointer will look like valid).
Or it is possible that object will be deleted, but it's memory will not be overwritten by something else, so it still will look like valid (but it fact it will be invalid).
So, there are no any way to detect is that pointer valid, if you have only pointer. You need something more.
Also it is not safe to just send a signal about deleting of object in multithreading case (or to use QObject::destroyed as you suggested). Why? Because it is possible, that things happens in this order:
QObject send a message "a am going to be deleted",
QObject deleted,
your receiving code uses that pointer (and this is wrong and dangerous),
your receiving code receives message "a am going to be deleted" (too late).
So, in case of only one thread you need QPointer. Else you need something like QSharedPointer or QWeakPointer (both of them are thread-safe) - see answer of Kuba Ober.
I have
struct MyWidget : QWidget {
// non-GUI related stuff:
int data;
int doSth();
};
I need to access a MyWidget instance from another thread (i.e. not the main thread). Is there any way to do that safely? I understand that I cannot access GUI related functions because some backends (e.g. MacOSX/Cocoa) don't support that. However, I only need to access data or doSth() in this example. But from what I have understand, there is simply no way to guarantee the lifetime of the object - i.e. if the parent window with that widget closes, the MyWidget instance gets deleted.
Or is there a way to guarantee the lifetime? I guess QSharedPointer doesn't work because the QWidget does its lifetime handling internally, depending on the parent widget. QPointer of course also doesn't help because it is only weak and there is no locking mechanism.
My current workaround is basically:
int widget_doSth(QPointer<MyWidget> w) {
int ret = -1;
execInMainThread_sync([&]() {
if(w)
ret = w->doSth();
});
return ret;
}
(execInMainThread_sync works by using QMetaMethod::invoke to call a method in the main thread.)
However, that workaround doesn't work anymore for some specific reason (I will explain later why, but that doesn't matter here). Basically, I am not able to execute something in the main thread at that point (for some complicated deadlock reasons).
Another workaround I'm currently thinking about is to add a global mutex which will guard the MyWidget destructor, and in the destructor, I'm cleaning up other weak references to the MyWidget. Then, elsewhere, when I need to ensure the lifetime, I just lock that mutex.
The reason why my current workaround doesn't work anymore (and that is still a simplified version of the real situation):
In MyWidget, the data is actually a PyObject*.
In the main thread, some Python code gets called. (It's not really possible to avoid any Python code calls at all in the main thread in my app.) That Python code ends up doing some import, which is guarded by some Python-import-mutex (Python doesn't allow parallel imports.)
In some other Python thread, some other import is called. That import now locks the Python-import-mutex. And while it's doing its thing, it does some GC cleanup at some point. That GC cleanup calls the traverse function of some object which holds that MyWidget. Thus, it must access the MyWidget. However, execInMainThread_sync (or equivalently working solutions) will deadlock because the main thread currently waits for the Python-import-lock.
Note: The Python global interpreter lock is not really the problem. Of course it gets unlocked before any execInMainThread_sync call. However, I cannot really check for any other potential Python/whatever locks. Esp. I am not allowed to just unlock the Python-import-lock -- it's there for a reason.
One solution you might think of is to really just avoid any Python code at all in the main thread. But that has a lot of drawbacks, e.g. it will be slow, complicated and ugly (the GUI basically only shows data from Python, so there need to be a huge proxy/wrapper around it all). And I think I still need to wait at some points for the Python data, so I just introduce the possible deadlock-situation at some other point.
Also, all the problems would just go away if I could access MyWidget safely from another thread. Introducing a global mutex is the much cleaner and shorter solution, compared to above.
You can use the signal/slot mechanism, but it can be tedious, if the number of GUI controls is large. I'd recommend a single signal and slot to control the gui. Send over a struct with all the info needed for updating the GUI.
void SomeWidget::updateGUISlot(struct Info const& info)
{
firstControl->setText(info.text);
secondControl->setValue(info.value);
}
You don't need to worry about emitting signals, if the recipient is deleted. This detail is handled by Qt. Alternatively, you can wait for your threads to exit, after exiting the GUI threads event loop. You'll need to register the struct with Qt.
EDIT:
From what I've read from your extended question, you're problems are related to communication between threads. Try pipes, (POSIX) message queues, sockets or POSIX signals instead of Qt signals for inter-thread communication.
Personally I don't like designs where GUI stuff (ie: A widget) has non-GUI related stuff... I think you should separate these two from each other. Qt needs to keep the GUI objects always on the main thread, but anything else (QObject derived) can be moved to a thread (QObject::moveToThread).
It seems that what you're explaining has nothing at all to do with widgets, Qt, or anything like that. It's a problem inherent to Python and its threading and the lock structure that doesn't make sense if you're multithreading. Python basically presumes that any object can be accessed from any thread. You'd have the same problem using any other toolkit. There may be a way of telling Python not to do that - I don't know enough about the cpython implementation's details, but that's where you'd need to look.
That GC cleanup calls the traverse function of some object which holds that MyWidget
That's your problem. You must ensure that such cross-thread GC cleanup can't happen. I have no idea how you'd go about it :(
My worry is that you've quietly and subtly shot yourself in the foot by using Python, in spite of everyone claiming that only C/C++ lets you do it at such a grand scale.
My solution:
struct MyWidget : QWidget {
// some non-GUI related stuff:
int someData;
virtual void doSth();
// We reset that in the destructor. When you hold its mutex-lock,
// the ref is either NULL or a valid pointer to this MyWidget.
struct LockedRef {
boost::mutex mutex;
MyWidget* ptr;
LockedRef(MyWidget& w) : ptr(&w) {}
void reset() {
boost::mutex::scoped_lock lock(mutex);
ptr = NULL;
}
};
boost::shared_ptr<LockedRef> selfRef;
struct WeakRef;
struct ScopedRef {
boost::shared_ptr<LockedRef> _ref;
MyWidget* ptr;
bool lock;
ScopedRef(WeakRef& ref);
~ScopedRef();
operator bool() { return ptr; }
MyWidget* operator->() { return ptr; }
};
struct WeakRef {
typedef boost::weak_ptr<LockedRef> Ref;
Ref ref;
WeakRef() {}
WeakRef(MyWidget& w) { ref = w.selfRef; }
ScopedRef scoped() { return ScopedRef(*this); }
};
MyWidget();
~MyWidget();
};
MyWidget::ScopedRef::ScopedRef(WeakRef& ref) : ptr(NULL), lock(true) {
_ref = ref.ref.lock();
if(_ref) {
lock = (QThread::currentThread() == qApp->thread());
if(lock) _ref->mutex.lock();
ptr = _ref->ptr;
}
}
MyWidget::ScopedRef::~ScopedRef() {
if(_ref && lock)
_ref->mutex.unlock();
}
MyWidget::~QtBaseWidget() {
selfRef->reset();
selfRef.reset();
}
MyWidget::MyWidget() {
selfRef = boost::shared_ptr<LockedRef>(new LockedRef(*this));
}
Now, everywhere I need to pass around a MyWidget pointer, I'm using:
MyWidget::WeakRef widget;
And I can use it from another thread like this:
MyWidget::ScopedRef widgetRef(widget);
if(widgetRef)
widgetRef->doSth();
This is safe. As long as ScopedRef exists, MyWidget cannot be deleted. It will block in its destructor. Or it is already deleted and ScopedRef::ptr == NULL.
I have an object that is called from two different threads and after it was called by both it destroys itself by "delete this".
How do I implement this thread-safe? Thread-safe means that the object never destroys itself exactly one time (it must destroys itself after the second callback).
I created some example code:
class IThreadCallBack
{
virtual void CallBack(int) = 0;
};
class M: public IThreadCallBack
{
private:
bool t1_finished, t2_finished;
public:
M(): t1_finished(false), t2_finished(false)
{
startMyThread(this, 1);
startMyThread(this, 2);
}
void CallBack(int id)
{
if (id == 1)
{
t1_finished = true;
}
else
{
t2_finished = true;
}
if (t1_finished && t2_finished)
{
delete this;
}
}
};
int main(int argc, char **argv) {
M* MObj = new M();
while(true);
}
Obviously I can't use a Mutex as member of the object and lock the delete, because this would also delete the Mutex. On the other hand, if I set a "toBeDeleted"-flag inside a mutex-protected area, where the finised-flag is set, I feel unsure if there are situations possible where the object isnt deleted at all.
Note that the thread-implementation makes sure that the callback method is called exactly one time per thread in any case.
Edit / Update:
What if I change Callback(..) to:
void CallBack(int id)
{
mMutex.Obtain()
if (id == 1)
{
t1_finished = true;
}
else
{
t2_finished = true;
}
bool both_finished = (t1_finished && t2_finished);
mMutex.Release();
if (both_finished)
{
delete this;
}
}
Can this considered to be safe? (with mMutex being a member of the m class?)
I think it is, if I don't access any member after releasing the mutex?!
Use Boost's Smart Pointer. It handles this automatically; your object won't have to delete itself, and it is thread safe.
Edit:
From the code you've posted above, I can't really say, need more info. But you could do it like this: each thread has a shared_ptr object and when the callback is called, you call shared_ptr::reset(). The last reset will delete M. Each shared_ptr could be stored with thread local storeage in each thread. So in essence, each thread is responsible for its own shared_ptr.
Instead of using two separate flags, you could consider setting a counter to the number of threads that you're waiting on and then using interlocked decrement.
Then you can be 100% sure that when the thread counter reaches 0, you're done and should clean up.
For more info on interlocked decrement on Windows, on Linux, and on Mac.
I once implemented something like this that avoided the ickiness and confusion of delete this entirely, by operating in the following way:
Start a thread that is responsible for deleting these sorts of shared objects, which waits on a condition
When the shared object is no longer being used, instead of deleting itself, have it insert itself into a thread-safe queue and signal the condition that the deleter thread is waiting on
When the deleter thread wakes up, it deletes everything in the queue
If your program has an event loop, you can avoid the creation of a separate thread for this by creating an event type that means "delete unused shared objects" and have some persistent object respond to this event in the same way that the deleter thread would in the above example.
I can't imagine that this is possible, especially within the class itself. The problem is two fold:
1) There's no way to notify the outside world not to call the object so the outside world has to be responsible for setting the pointer to 0 after calling "CallBack" iff the pointer was deleted.
2) Once two threads enter this function you are, and forgive my french, absolutely fucked. Calling a function on a deleted object is UB, just imagine what deleting an object while someone is in it results in.
I've never seen "delete this" as anything but an abomination. Doesn't mean it isn't sometimes, on VERY rare conditions, necessary. Problem is that people do it way too much and don't think about the consequences of such a design.
I don't think "to be deleted" is going to work well. It might work for two threads, but what about three? You can't protect the part of code that calls delete because you're deleting the protection (as you state) and because of the UB you'll inevitably cause. So the first goes through, sets the flag and aborts....which of the rest is going to call delete on the way out?
The more robust implementation would be to implement reference counting. For each thread you start, increase a counter; for each callback call decrease the counter and if the counter has reached zero, delete the object. You can lock the counter access, or you could use the Interlocked class to protect the counter access, though in that case you need to be careful with potential race between the first thread finishing and the second starting.
Update: And of course, I completely ignored the fact that this is C++. :-) You should use InterlockExchange to update the counter instead of the C# Interlocked class.