Related
Recently I'm thinking a high performance event-driven multi-threads framework using c++11. And it mainly takes c++11 facilities such as std::thread, std::condition_variable, std::mutex, std::shared_ptr etc into consideration. In general, this framework has three basic components: job, worker and streamline, well, it seems to be a real factory. When user construct his business model in server end, he just needs to consider the data and its processor. Once the model is established, user only needs to construct data class inherited job and processor class inherited worker.
For example:
class Data : public job {};
class Processsor : public worker {};
When server get data, it just new a Data object through auto data = std::make_shared<Data>() in the data source callback thread and call the streamline. job_dispatch to transfer the processor and data to other thread. Of course user doesn't have to think to free memory. The streamline. job_dispatch mainly do below stuff:
void evd_thread_pool::job_dispatch(std::shared_ptr<evd_thread_job> job) {
auto task = std::make_shared<evd_task_wrap>(job);
task->worker = streamline.worker;
// worker has been registered in streamline first of all
{
std::unique_lock<std::mutex> lck(streamline.mutex);
streamline.task_list.push_back(std::move(task));
}
streamline.cv.notify_all();
}
The evd_task_wrap used in the job_dispatch defined as:
struct evd_task_wrap {
std::shared_ptr<evd_thread_job> order;
std::shared_ptr<evd_thread_processor> worker;
evd_task_wrap(std::shared_ptr<evd_thread_job>& o)
:order(o) {}
};
Finally the task_wrap will be dispatched into the processing thread through task_list that is a std::list object. And the processing thread mainly do the stuff as:
void evd_factory_impl::thread_proc() {
std::shared_ptr<evd_task_wrap> wrap = nullptr;
while (true) {
{
std::unique_lock<std::mutex> lck(streamline.mutex);
if (streamline.task_list.empty())
streamline.cv.wait(lck,
[&]()->bool{return !streamline.task_list.empty();});
wrap = std::move(streamline.task_list.front());
streamline.task_list.pop_front();
}
if (-1 == wrap->order->get_type())
break;
wrap->worker->process_task(wrap->order);
wrap.reset();
}
}
But I don't know why the process will often crash in the thread_proc function. And the coredump prompt that sometimes the wrap is a empty shared_ptr or segment fault happened in _Sp_counted_ptr_inplace::_M_dispose that is called in wrap.reset(). And I supposed the shared_ptr has the thread synchronous problem in this scenario while I know the control block in shared_ptr is thread-safety. And of course the shared_ptr in job_dispatch and thread_proc is different shared_ptr object even though they point to the same storage. Does anyone has more specific suggestion on how to solve this problem? Or if there exists similar lightweight framework with automatic memory management using c++11
The example of process_task such as:
void log_handle::process_task(std::shared_ptr<crx::evd_thread_job> job) {
auto j = std::dynamic_pointer_cast<log_job>(job);
j->log->Printf(0, j->print_str.c_str());
write(STDOUT_FILENO, j->print_str.c_str(), j->print_str.size());
}
class log_factory {
public:
log_factory(const std::string& name);
virtual ~log_factory();
void print_ts(const char *format, ...) { //here dispatch the job
char log_buf[4096] = {0};
va_list args;
va_start(args, format);
vsprintf(log_buf, format, args);
va_end(args);
auto job = std::make_shared<log_job>(log_buf, &m_log);
m_log_th.job_dispatch(job);
}
public:
E15_Log m_log;
std::shared_ptr<log_handle> m_log_handle;
crx::evd_thread_pool m_log_th;
};
I detected a problem in your code, which may or may not be related:
You use notify_all from your condition variable. That will awaken ALL threads from sleep. It is OK if you wrap your wait in a while loop, like:
while (streamline.task_list.empty())
streamline.cv.wait(lck, [&]()->bool{return !streamline.task_list.empty();});
But since you are using an if, all threads leave the wait. If you dispatch a single product and having several consumer threads, all but one thread will call wrap = std::move(streamline.task_list.front()); while the tasklist is empty and cause UB.
I have a scenario where an anonymous QObject starts an asynchronous operation by emitting a signal. The receiving slot stores the QObject-pointer and sets a property of this object later. The object could be gone meanwhile.
So, is there a safe way to check if this pointer still valid?
P.S.:
I'm aware of QObject::destroyed signal, which I could connect to the object supposed to call the setProperty of that pointer. But I wonder, if it works easier.
This is a great question, but it is the wrong question.
Is there a way to check if the pointer is valid? Yes. QPointer is designed specifically to do that.
But the answer to this question is useless if the object lives in another thread! You only know whether it's valid at a single point in time - the answer is not valid immediately afterwards.
Absent other mechanisms, it is useless to hold a QPointer to an object in a different thread - it won't help you. Why? Look at this scenario:
Thread A Thread B
1. QPointer returns a non-zero pointer
2. deletes the object
3. Use the now-dangling pointer
I'm aware of QObject::destroyed signal, which I could connect to the object supposed to call the setProperty of that pointer. But I wonder, if it works easier.
The destroyed signals are useless when sent using queued connections - whether within a thread, or across thread boundaries. They are meant to be used within one thread, using direct connections.
By the time the target thread's event loop picks up the slot call, the originating object is long gone. Worse - this is always the case in a single-threaded application. The reason for the problem is the same as with the QPointer: the destroyed signal indicates that the object is no longer valid, but it doesn't mean that it was valid before you received the signal unless you're using a direct connection (and are in the same thread) or you're using a blocking queued connection.
Using the blocking queued connection, the requesting object's thread will block until the async thread finishes reacting to object's deletion. While this certainly "works", it forces the two threads to synchronize on a resource with sparse availability - the front spot in the async thread's event loop. Yes, this is literally what you compete for - a single spot in a queue that can be arbitrarily long. While this might be OK for debugging, it has no place in production code unless it's OK to block either thread to synchronize.
You are trying to work very hard around the fact that you're passing a QObject pointer between threads, and the object's lifetime, from the point of view of the receiving thread, is uncontrolled. That's your problem. You'd solve everything by not passing a raw object pointer. Instead, you could pass a shared smart pointer, or using signal-slot connections: those vanish whenever either end of the connection is destructed. That's what you'd want.
In fact, Qt's own design patterns hint at this. QNetworkReply is a QObject not only because it is a QIODevice, but because it must be to support direct indications of finished requests across thread boundaries. In light of a multitude of requests being processed, connecting to QNetworkAccessManager::finished(QNetworkReply*) can be a premature pessimization. Your object gets notified of a possibly very large number of replies, but it really is only interested in one or very few of them. Thus there must be a way to notify the requester directly that its one and only request is done - and that's what QNetworkReply::finished is for.
So, a simple way to proceed is to make the Request be a QObject with a done signal. When you ready the request, connect the requesting object to that signal. You can also connect a functor, but make sure that the functor executes in the requesting object's context:
// CORRECT
connect(request, &Request::done, requester, [...](...){...});
// WRONG
connect(request, &Request::done, [...](...){...});
The below demonstrates how it could be put together. The requests' lifetimes are managed through the use of a shared (reference-counting) smart pointer. This makes life rather easy. We check that no requests exist at the time main returns.
#include <QtCore>
class Request;
typedef QSharedPointer<Request> RequestPtr;
class Request : public QObject {
Q_OBJECT
public:
static QAtomicInt m_count;
Request() { m_count.ref(); }
~Request() { m_count.deref(); }
int taxIncrease;
Q_SIGNAL void done(RequestPtr);
};
Q_DECLARE_METATYPE(RequestPtr)
QAtomicInt Request::m_count(0);
class Requester : public QObject {
Q_OBJECT
Q_PROPERTY (int catTax READ catTax WRITE setCatTax NOTIFY catTaxChanged)
int m_catTax;
public:
Requester(QObject * parent = 0) : QObject(parent), m_catTax(0) {}
Q_SLOT int catTax() const { return m_catTax; }
Q_SLOT void setCatTax(int t) {
if (t != m_catTax) {
m_catTax = t;
emit catTaxChanged(t);
}
}
Q_SIGNAL void catTaxChanged(int);
Q_SIGNAL void hasRequest(RequestPtr);
void sendNewRequest() {
RequestPtr req(new Request);
req->taxIncrease = 5;
connect(req.data(), &Request::done, this, [this, req]{
setCatTax(catTax() + req->taxIncrease);
qDebug() << objectName() << "has cat tax" << catTax();
QCoreApplication::quit();
});
emit hasRequest(req);
}
};
class Processor : public QObject {
Q_OBJECT
public:
Q_SLOT void process(RequestPtr req) {
QThread::msleep(50); // Pretend to do some work.
req->taxIncrease --; // Figure we don't need so many cats after all...
emit req->done(req);
emit done(req);
}
Q_SIGNAL void done(RequestPtr);
};
struct Thread : public QThread { ~Thread() { quit(); wait(); } };
int main(int argc, char ** argv) {
struct C { ~C() { Q_ASSERT(Request::m_count == 0); } } check;
QCoreApplication app(argc, argv);
qRegisterMetaType<RequestPtr>();
Processor processor;
Thread thread;
processor.moveToThread(&thread);
thread.start();
Requester requester1;
requester1.setObjectName("requester1");
QObject::connect(&requester1, &Requester::hasRequest, &processor, &Processor::process);
requester1.sendNewRequest();
{
Requester requester2;
requester2.setObjectName("requester2");
QObject::connect(&requester2, &Requester::hasRequest, &processor, &Processor::process);
requester2.sendNewRequest();
} // requester2 is destructed here
return app.exec();
}
#include "main.moc"
It is impossible to check is that pointer still valid. So, the only safe way here is to inform receiving part about deleting of that QObject (and in multithreading case: before accessing to object you need to check and block it to be sure, that the object will not be deleted in another thread right after check). The reason of it is simple:
Theoretically it is possible that after deleting of initial object, system will put another object in that memory (so pointer will look like valid).
Or it is possible that object will be deleted, but it's memory will not be overwritten by something else, so it still will look like valid (but it fact it will be invalid).
So, there are no any way to detect is that pointer valid, if you have only pointer. You need something more.
Also it is not safe to just send a signal about deleting of object in multithreading case (or to use QObject::destroyed as you suggested). Why? Because it is possible, that things happens in this order:
QObject send a message "a am going to be deleted",
QObject deleted,
your receiving code uses that pointer (and this is wrong and dangerous),
your receiving code receives message "a am going to be deleted" (too late).
So, in case of only one thread you need QPointer. Else you need something like QSharedPointer or QWeakPointer (both of them are thread-safe) - see answer of Kuba Ober.
I need your help with wxWidgets. I have 2 threads (1 wxTimer and 1 wxThread), I need communicate between this 2 threads. I have a class that contains methods to read/write variable in this class. (Share Memory with this object)
My problem is: I instanciate with "new" this class in one thread but I don't know that necessary in second thread. Because if instanciate too, adress of variable are differents and I need communicate so I need even value in variable :/
I know about need wxSemaphore to prevent error when to access same time.
Thanks you for your help !
EDIT: My code
So, I need make a link with my code. Thanks you for all ;)
It's my declaration for my wxTimer in my class: EvtFramePrincipal (IHM)
In .h
EvtFramePrincipal( wxWindow* parent );
#include <wx/timer.h>
wxTimer m_timer;
in .cpp -Constructor EvtFramePrincipal
EvtFramePrincipal::EvtFramePrincipal( wxWindow* parent )
:
FramePrincipal( parent ),m_timer(this)
{
Connect(wxID_ANY,wxEVT_TIMER,wxTimerEventHandler(EvtFramePrincipal::OnTimer),NULL,this);
m_timer.Start(250);
}
So I call OnTimer method every 250ms with this line.
For my second thread start from EvtFramePrincipal (IHM):
in .h EvtFramePrincipal
#include "../Client.h"
Client *ClientIdle;
in .cpp EvtFramePrincipal
ClientIdle= new Client();
ClientIdle->Run();
In .h Client (Thread)
class Client: public wxThread
public:
Client();
virtual void *Entry();
virtual void OnExit();
In .cpp Client (Thread)
Client::Client() : wxThread()
{
}
So here, no probleme, thread are ok ?
Now I need that this class that use like a messenger between my 2 threads.
#ifndef PARTAGE_H
#define PARTAGE_H
#include "wx/string.h"
#include <iostream>
using std::cout;
using std::endl;
class Partage
{
public:
Partage();
virtual ~Partage();
bool Return_Capteur_Aval()
{ return Etat_Capteur_Aval; }
bool Return_Capteur_Amont()
{ return Etat_Capteur_Amont; }
bool Return_Etat_Barriere()
{ return Etat_Barriere; }
bool Return_Ouverture()
{ return Demande_Ouverture; }
bool Return_Fermeture()
{ return Demande_Fermeture; }
bool Return_Appel()
{ return Appel_Gardien; }
void Set_Ouverture(bool Etat)
{ Demande_Ouverture=Etat; }
void Set_Fermeture(bool Etat)
{ Demande_Fermeture=Etat; }
void Set_Capteur_Aval(bool Etat)
{ Etat_Capteur_Aval=Etat; }
void Set_Capteur_Amont(bool Etat)
{ Etat_Capteur_Amont=Etat; }
void Set_Barriere(bool Etat)
{ Etat_Barriere=Etat; }
void Set_Appel(bool Etat)
{ Appel_Gardien=Etat; }
void Set_Code(wxString valeur_code)
{ Code=valeur_code; }
void Set_Badge(wxString numero_badge)
{ Badge=numero_badge; }
void Set_Message(wxString message)
{
Message_Affiche=wxT("");
Message_Affiche=message;
}
wxString Get_Message()
{
return Message_Affiche;
}
wxString Get_Code()
{ return Code; }
wxString Get_Badge()
{ return Badge; }
protected:
private:
bool Etat_Capteur_Aval;
bool Etat_Capteur_Amont;
bool Etat_Barriere;
bool Demande_Ouverture;
bool Demande_Fermeture;
bool Appel_Gardien;
wxString Code;
wxString Badge;
wxString Message_Affiche;
};
#endif // PARTAGE_H
So in my EvtFramePrincipal(wxTimer), I make a new for this class. But in other thread (wxThread), what I need to do to communicate ?
If difficult to understand so sorry :/
Then main thread should create first the shared variable. After it, you can create both threads and pass them a pointer to the shared variable.
So, both of them, know how interact with the shared variable. You need to implement a mutex or wxSemaphore in the methods of the shared variable.
You can use a singleton to get access to a central object.
Alternatively, create the central object before creating the threads and pass the reference to the central object to threads.
Use a mutex in the central object to prevent simultaneous access.
Creating one central object on each thread is not an option.
EDIT 1: Adding more details and examples
Let's start with some assumptions. The OP indicated that
I have 2 threads (1 wxTimer and 1 wxThread)
To tell the truth, I know very little of the wxWidgets framework, but there's always the documentation. So I can see that:
wxTimer provides a Timer that will execute the wxTimer::Notify() method when the timer expires. The documentation doesn't say anything about thread-execution (although there's a note A timer can only be used from the main thread which I'm not sure how to understand). I can guess that we should expect the Notify method will be executed in some event-loop or timer-loop thread or threads.
wxThread provides a model for Thread execution, that runs the wxThread::Entry() method. Running a wxThread object will actually create a thread that runs the Entry method.
So your problem is that you need same object to be accessible in both wxTimer::Notify() and wxThread::Entry() methods.
This object:
It's not one variable but a lot of that store in one class
e.g.
struct SharedData {
// NOTE: This is very simplistic.
// since the information here will be modified/read by
// multiple threads, it should be protected by one or more
// mutexes
// so probably a class with getter/setters will be better suited
// so that access with mutexes can be enforced within the class.
SharedData():var2(0) { }
std::string var1;
int var2;
};
of which you have somewhere an instance of that:
std::shared_ptr<SharedData> myData=std::make_shared<SharedData>();
or perhaps in pointer form or perhaps as a local variable or object attribute
Option 1: a shared reference
You're not really using wxTimer or wxThread, but classes that inherit from them (at least the wxThread::Entry() is pure virtual. In the case of wxTimer you could change the owner to a different wxEvtHandler that will receive the event, but you still need to provide an implementation.
So you can have
class MyTimer: public wxTimer {
public:
void Notify() {
// Your code goes here
// but it can access data through the local reference
}
void setData(const std::shared_ptr<SharedData> &data) {
mLocalReference=data
}
private:
std::shared_ptr<SharedData> mLocalReferece
};
That will need to be set:
MyTimer timer;
timer.setData(myData);
timer.StartOnece(10000); // wake me up in 10 secs.
Similarly for the Thread
class MyThread: public wxThread {
public:
void Entry() {
// Your code goes here
// but it can access data through the local reference
}
void setData(const std::shared_ptr<SharedData> &data) {
mLocalReference=data
}
private:
std::shared_ptr<SharedData> mLocalReferece
};
That will need to be set:
MyThread *thread=new MyThread();
thread->setData(myData);
thread->Run(); // threads starts running.
Option2 Using a singleton.
Sometimes you cannot modify MyThread or MyTimer... or it is too difficult to route the reference to myData to the thread or timer instances... or you're just too lazy or too busy to bother (beware of your technical debt!!!)
We can tweak the SharedData into:
struct SharedData {
std::string var1;
int var2;
static SharedData *instance() {
// NOTE that some mutexes are needed here
// to prevent the case where first initialization
// is executed simultaneously from different threads
// allocating two objects, one of them leaked.
if(!sInstance) {
sInstance=new SharedData();
}
return sInstance
}
private:
SharedData():var2(0) { } // Note we've made the constructor private
static SharedData *sInstance=0;
};
This object (because it only allows the creation of a single object) can be accessed from
either MyTimer::Notify() or MyThread::Entry() with
SharedData::instance()->var1;
Interlude: why Singletons are evil
(or why the easy solution might bite you in the future).
What is so bad about singletons?
Why Singletons are Evil
Singletons Are Evil
My main reasons are:
There's one and only one instance... and you might think that you only need one now, but who knows what the future will hold, you've taken an easy solution for a coding problem that has far reaching consequences architecturally and that might be difficult to revert.
It will not allow doing dependency injection (because the actual class is used in the accessing the object).
Still, I don't think is something to completely avoid. It has its uses, it can solve your problem and it might save your day.
Option 3. Some middle ground.
You could still organize your data around a central repository with methods to access different instances (or different implementations) of the data.
This central repository can be a singleton (it is really is central, common and unique), but is not the shared data, but what is used to retrieve the shared data, e.g. identified by some ID (that might be easier to share between the threads using option 1)
Something like:
CentralRepository::instance()->getDataById(sharedId)->var1;
EDIT 2: Comments after OP posted (more) code ;)
It seems that your object EvtFramePrincipal will execute both the timer call back and it will contain the ClientIdle pointer to a Client object (the thread)... I'd do:
Make the Client class contain a Portage attribute (a pointer or a smart pointer).
Make the EvtFramePrincipal contain a Portage attribute (a pointer or smart pointer). I guess this will have the lifecycle of the whole application, so the Portage object can share that lifecycle too.
Add Mutexes locking to all methods setting and getting in the Portage attribute, since it can be accessed from multiple threads.
After the Client object is instantiated set the reference to the Portage object that the EvtFramePrincipal contains.
Client can access Portage because we've set its reference when it was created. When the Entry method is run in its thread it will be able to access it.
EvtFramePrincipal can access the Portage (because it is one of its attributes), so the event handler for the timer event will be able to access it.
I have
struct MyWidget : QWidget {
// non-GUI related stuff:
int data;
int doSth();
};
I need to access a MyWidget instance from another thread (i.e. not the main thread). Is there any way to do that safely? I understand that I cannot access GUI related functions because some backends (e.g. MacOSX/Cocoa) don't support that. However, I only need to access data or doSth() in this example. But from what I have understand, there is simply no way to guarantee the lifetime of the object - i.e. if the parent window with that widget closes, the MyWidget instance gets deleted.
Or is there a way to guarantee the lifetime? I guess QSharedPointer doesn't work because the QWidget does its lifetime handling internally, depending on the parent widget. QPointer of course also doesn't help because it is only weak and there is no locking mechanism.
My current workaround is basically:
int widget_doSth(QPointer<MyWidget> w) {
int ret = -1;
execInMainThread_sync([&]() {
if(w)
ret = w->doSth();
});
return ret;
}
(execInMainThread_sync works by using QMetaMethod::invoke to call a method in the main thread.)
However, that workaround doesn't work anymore for some specific reason (I will explain later why, but that doesn't matter here). Basically, I am not able to execute something in the main thread at that point (for some complicated deadlock reasons).
Another workaround I'm currently thinking about is to add a global mutex which will guard the MyWidget destructor, and in the destructor, I'm cleaning up other weak references to the MyWidget. Then, elsewhere, when I need to ensure the lifetime, I just lock that mutex.
The reason why my current workaround doesn't work anymore (and that is still a simplified version of the real situation):
In MyWidget, the data is actually a PyObject*.
In the main thread, some Python code gets called. (It's not really possible to avoid any Python code calls at all in the main thread in my app.) That Python code ends up doing some import, which is guarded by some Python-import-mutex (Python doesn't allow parallel imports.)
In some other Python thread, some other import is called. That import now locks the Python-import-mutex. And while it's doing its thing, it does some GC cleanup at some point. That GC cleanup calls the traverse function of some object which holds that MyWidget. Thus, it must access the MyWidget. However, execInMainThread_sync (or equivalently working solutions) will deadlock because the main thread currently waits for the Python-import-lock.
Note: The Python global interpreter lock is not really the problem. Of course it gets unlocked before any execInMainThread_sync call. However, I cannot really check for any other potential Python/whatever locks. Esp. I am not allowed to just unlock the Python-import-lock -- it's there for a reason.
One solution you might think of is to really just avoid any Python code at all in the main thread. But that has a lot of drawbacks, e.g. it will be slow, complicated and ugly (the GUI basically only shows data from Python, so there need to be a huge proxy/wrapper around it all). And I think I still need to wait at some points for the Python data, so I just introduce the possible deadlock-situation at some other point.
Also, all the problems would just go away if I could access MyWidget safely from another thread. Introducing a global mutex is the much cleaner and shorter solution, compared to above.
You can use the signal/slot mechanism, but it can be tedious, if the number of GUI controls is large. I'd recommend a single signal and slot to control the gui. Send over a struct with all the info needed for updating the GUI.
void SomeWidget::updateGUISlot(struct Info const& info)
{
firstControl->setText(info.text);
secondControl->setValue(info.value);
}
You don't need to worry about emitting signals, if the recipient is deleted. This detail is handled by Qt. Alternatively, you can wait for your threads to exit, after exiting the GUI threads event loop. You'll need to register the struct with Qt.
EDIT:
From what I've read from your extended question, you're problems are related to communication between threads. Try pipes, (POSIX) message queues, sockets or POSIX signals instead of Qt signals for inter-thread communication.
Personally I don't like designs where GUI stuff (ie: A widget) has non-GUI related stuff... I think you should separate these two from each other. Qt needs to keep the GUI objects always on the main thread, but anything else (QObject derived) can be moved to a thread (QObject::moveToThread).
It seems that what you're explaining has nothing at all to do with widgets, Qt, or anything like that. It's a problem inherent to Python and its threading and the lock structure that doesn't make sense if you're multithreading. Python basically presumes that any object can be accessed from any thread. You'd have the same problem using any other toolkit. There may be a way of telling Python not to do that - I don't know enough about the cpython implementation's details, but that's where you'd need to look.
That GC cleanup calls the traverse function of some object which holds that MyWidget
That's your problem. You must ensure that such cross-thread GC cleanup can't happen. I have no idea how you'd go about it :(
My worry is that you've quietly and subtly shot yourself in the foot by using Python, in spite of everyone claiming that only C/C++ lets you do it at such a grand scale.
My solution:
struct MyWidget : QWidget {
// some non-GUI related stuff:
int someData;
virtual void doSth();
// We reset that in the destructor. When you hold its mutex-lock,
// the ref is either NULL or a valid pointer to this MyWidget.
struct LockedRef {
boost::mutex mutex;
MyWidget* ptr;
LockedRef(MyWidget& w) : ptr(&w) {}
void reset() {
boost::mutex::scoped_lock lock(mutex);
ptr = NULL;
}
};
boost::shared_ptr<LockedRef> selfRef;
struct WeakRef;
struct ScopedRef {
boost::shared_ptr<LockedRef> _ref;
MyWidget* ptr;
bool lock;
ScopedRef(WeakRef& ref);
~ScopedRef();
operator bool() { return ptr; }
MyWidget* operator->() { return ptr; }
};
struct WeakRef {
typedef boost::weak_ptr<LockedRef> Ref;
Ref ref;
WeakRef() {}
WeakRef(MyWidget& w) { ref = w.selfRef; }
ScopedRef scoped() { return ScopedRef(*this); }
};
MyWidget();
~MyWidget();
};
MyWidget::ScopedRef::ScopedRef(WeakRef& ref) : ptr(NULL), lock(true) {
_ref = ref.ref.lock();
if(_ref) {
lock = (QThread::currentThread() == qApp->thread());
if(lock) _ref->mutex.lock();
ptr = _ref->ptr;
}
}
MyWidget::ScopedRef::~ScopedRef() {
if(_ref && lock)
_ref->mutex.unlock();
}
MyWidget::~QtBaseWidget() {
selfRef->reset();
selfRef.reset();
}
MyWidget::MyWidget() {
selfRef = boost::shared_ptr<LockedRef>(new LockedRef(*this));
}
Now, everywhere I need to pass around a MyWidget pointer, I'm using:
MyWidget::WeakRef widget;
And I can use it from another thread like this:
MyWidget::ScopedRef widgetRef(widget);
if(widgetRef)
widgetRef->doSth();
This is safe. As long as ScopedRef exists, MyWidget cannot be deleted. It will block in its destructor. Or it is already deleted and ScopedRef::ptr == NULL.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What is a good way of dealing with objects and having them talk to each other?
Up until now all my games hobby/student have been small so this problem was generally solved in a rather ugly way, which lead to tight integration and circular dependencies. Which was fine for the size of projects I was doing.
However my projects have been getting bigger in size and complexity and now I want to start re-using code, and making my head a simpler place.
The main problem I have is generally along the lines of Player needs to know about the Map and so does the Enemy, this has usually descended into setting lots of pointers and having lots of dependencies, and this becomes a mess quickly.
I have thought along the lines of a message style system. but I cant really see how this reduces the dependencies, as I would still be sending the pointers everywhere.
PS: I guess this has been discussed before, but I don't know what its called just the need I have.
EDIT: Below I describe a basic event messaging system I have used over and over. And it occurred to me that both school projects are open source and on the web. You can find the second version of this messaging system (and quite a bit more) at http://sourceforge.net/projects/bpfat/ .. Enjoy, and read below for a more thorough description of the system!
I've written a generic messaging system and introduced it into a handful of games that have been released on the PSP as well as some enterprise level application software. The point of the messaging system is to pass only the data around that is needed for processing a message or event, depending on the terminology you want to use, so that objects do not have to know about each other.
A quick rundown of the list of objects used to accomplish this is something along the lines of:
struct TEventMessage
{
int _iMessageID;
}
class IEventMessagingSystem
{
Post(int iMessageId);
Post(int iMessageId, float fData);
Post(int iMessageId, int iData);
// ...
Post(TMessageEvent * pMessage);
Post(int iMessageId, void * pData);
}
typedef float(*IEventMessagingSystem::Callback)(TEventMessage * pMessage);
class CEventMessagingSystem
{
Init ();
DNit ();
Exec (float fElapsedTime);
Post (TEventMessage * oMessage);
Register (int iMessageId, IEventMessagingSystem* pObject, FObjectCallback* fpMethod);
Unregister (int iMessageId, IEventMessagingSystem* pObject, FObjectCallback * fpMethod);
}
#define MSG_Startup (1)
#define MSG_Shutdown (2)
#define MSG_PlaySound (3)
#define MSG_HandlePlayerInput (4)
#define MSG_NetworkMessage (5)
#define MSG_PlayerDied (6)
#define MSG_BeginCombat (7)
#define MSG_EndCombat (8)
And now a bit of an explanation. The first object, TEventMessage, is the base object to represent data sent by the messaging system. By default it will always have the Id of the message being sent so if you want to make sure you have received a message you were expecting you can (Generally I only do that in debug).
Next up is the Interface class that gives a generic object for the messaging system to use for casting while doing callbacks. Additionally this also provides an 'easy to use' interface for Post()ing different data types to the messaging system.
After that we have our Callback typedef, Simply put it expects an object of the type of the interface class and will pass along a TEventMessage pointer... Optionally you can make the parameter const but I've used trickle up processing before for things like stack debugging and such of the messaging system.
Last and at the core is the CEventMessagingSystem object. This object contains an array of callback object stacks (or linked lists or queues or however you want to store the data). The callback objects, not shown above, need to maintain (and are uniquely defined by) a pointer to the object as well as the method to call on that object. When you Register() you add an entry on the object stack under the message id's array position. When you Unregister() you remove that entry.
That is basically it. Now this does have the stipulation that everything needs to know about the IEventMessagingSystem and the TEventMessage object... but this object should Not be changing that often and only passes the parts of information that are vital to the logic dictated by the event being called. This way a player doesn't need to know about the map or the enemy directly for sending events off to it. A managed object can call an API to a larger system also, without needing to know anything about it.
For example: When an enemy dies you want it to play a sound effect. Assuming you have a sound manager that inherits the IEventMessagingSystem interface, you would set up a callback for the messaging system that would accept a TEventMessagePlaySoundEffect or something of that ilk. The Sound Manager would then register this callback when sound effects are enabled (or unregister the callback when you want to mute all sound effects for easy on/off abilities). Next, you would have the enemy object also inherit from the IEventMessagingSystem, put together a TEventMessagePlaySoundEffect object (would need the MSG_PlaySound for its Message ID and then the ID of the sound effect to play, be it an int ID or the name of the sound effect) and simply call Post(&oEventMessagePlaySoundEffect).
Now this is just a very simple design with no implementation. If you have immediate execution then you have no need to buffer the TEventMessage objects (What I used mostly in console games). If you are in a multi-threaded environment then this is a very well defined way for objects and systems running in separate threads to talk to each other, but you will want to preserve the TEventMessage objects so the data is available when processing.
Another alteration is for objects that only ever need to Post() data, you can create a static set of methods in the IEventMessagingSystem so they do not have to inherit from them (That is used for ease of access and callback abilities, not -directly- needed for Post() calls).
For all the people who mention MVC, it is a very good pattern, but you can implement it in so many different manners and at different levels. The current project I am working on professionally is an MVC setup about 3 times over, there is the global MVC of the entire application and then design wise each M V and C also is a self-contained MVC pattern. So what I have tried to do here is explain how to make a C that is generic enough to handle just about any type of M without the need to get into a View...
For example, an object when it 'dies' might want to play a sound effect.. You would make a struct for the Sound System like TEventMessageSoundEffect that inherits from the TEventMessage and adds in a sound effect ID (Be it a preloaded Int, or the name of the sfx file, however they are tracked in your system). Then all the object just needs to put together a TEventMessageSoundEffect object with the appropriate Death noise and call Post(&oEventMessageSoundEffect); object.. Assuming the sound is not muted (what you would want to Unregister the Sound Managers.
EDIT: To clarify this a bit in regards to the comment below:
Any object to send or receive a message just needs to know about the IEventMessagingSystem interface, and this is the only object the EventMessagingSystem needs to know of all the other objects. This is what gives you the detachment. Any object who wants to receive a message simply Register(MSG, Object, Callback)s for it. Then when an object calls Post(MSG,Data) it sends that to the EventMessagingSystem via the interface it knows about, the EMS will then notify each registered object of the event. You could do a MSG_PlayerDied that other systems handle, or the player can call MSG_PlaySound, MSG_Respawn, etc to let things listening for those messages to act upon them. Think of the Post(MSG,Data) as an abstracted API to the different systems within a game engine.
Oh! One other thing that was pointed out to me. The system I describe above fits the Observer pattern in the other answer given. So if you want a more general description to make mine make a bit more sense, that is a short article that gives it a good description.
Hope this helps and Enjoy!
the generic solutions for communication between objects avoiding tight coupling:
Mediator pattern
Observer pattern
Here is a neat event system written for C++11 you can use. It uses templates and smart pointers as well as lambdas for the delegates. It's very flexible. Below you will also find an example. Email me at info#fortmax.se if you have questions about this.
What these classes gives you is a way to send events with arbitrary data attached to them and an easy way to directly bind functions that accept already converted argument types that the system casts and checks for correct conversion prior to calling your delegate.
Basically, every event is derived from IEventData class (you can call it IEvent if you want). Each "frame" you call ProcessEvents() at which point the event system loops through all the delegates and calls the delegates that have been supplied by other systems that have subscribed to each event type. Anyone can pick which events they would like to subscribe to, as each event type has a unique ID. You can also use lambdas to subscribe to events like this: AddListener(MyEvent::ID(), [&](shared_ptr ev){
do your thing }..
Anyway, here is the class with all the implementation:
#pragma once
#include <list>
#include <memory>
#include <map>
#include <vector>
#include <functional>
class IEventData {
public:
typedef size_t id_t;
virtual id_t GetID() = 0;
};
typedef std::shared_ptr<IEventData> IEventDataPtr;
typedef std::function<void(IEventDataPtr&)> EventDelegate;
class IEventManager {
public:
virtual bool AddListener(IEventData::id_t id, EventDelegate proc) = 0;
virtual bool RemoveListener(IEventData::id_t id, EventDelegate proc) = 0;
virtual void QueueEvent(IEventDataPtr ev) = 0;
virtual void ProcessEvents() = 0;
};
#define DECLARE_EVENT(type) \
static IEventData::id_t ID(){ \
return reinterpret_cast<IEventData::id_t>(&ID); \
} \
IEventData::id_t GetID() override { \
return ID(); \
}\
class EventManager : public IEventManager {
public:
typedef std::list<EventDelegate> EventDelegateList;
~EventManager(){
}
//! Adds a listener to the event. The listener should invalidate itself when it needs to be removed.
virtual bool AddListener(IEventData::id_t id, EventDelegate proc) override;
//! Removes the specified delegate from the list
virtual bool RemoveListener(IEventData::id_t id, EventDelegate proc) override;
//! Queues an event to be processed during the next update
virtual void QueueEvent(IEventDataPtr ev) override;
//! Processes all events
virtual void ProcessEvents() override;
private:
std::list<std::shared_ptr<IEventData>> mEventQueue;
std::map<IEventData::id_t, EventDelegateList> mEventListeners;
};
//! Helper class that automatically handles removal of individual event listeners registered using OnEvent() member function upon destruction of an object derived from this class.
class EventListener {
public:
//! Template function that also converts the event into the right data type before calling the event listener.
template<class T>
bool OnEvent(std::function<void(std::shared_ptr<T>)> proc){
return OnEvent(T::ID(), [&, proc](IEventDataPtr data){
auto ev = std::dynamic_pointer_cast<T>(data);
if(ev) proc(ev);
});
}
protected:
typedef std::pair<IEventData::id_t, EventDelegate> _EvPair;
EventListener(std::weak_ptr<IEventManager> mgr):_els_mEventManager(mgr){
}
virtual ~EventListener(){
if(_els_mEventManager.expired()) return;
auto em = _els_mEventManager.lock();
for(auto i : _els_mLocalEvents){
em->RemoveListener(i.first, i.second);
}
}
bool OnEvent(IEventData::id_t id, EventDelegate proc){
if(_els_mEventManager.expired()) return false;
auto em = _els_mEventManager.lock();
if(em->AddListener(id, proc)){
_els_mLocalEvents.push_back(_EvPair(id, proc));
}
}
private:
std::weak_ptr<IEventManager> _els_mEventManager;
std::vector<_EvPair> _els_mLocalEvents;
//std::vector<_DynEvPair> mDynamicLocalEvents;
};
And the Cpp file:
#include "Events.hpp"
using namespace std;
bool EventManager::AddListener(IEventData::id_t id, EventDelegate proc){
auto i = mEventListeners.find(id);
if(i == mEventListeners.end()){
mEventListeners[id] = list<EventDelegate>();
}
auto &list = mEventListeners[id];
for(auto i = list.begin(); i != list.end(); i++){
EventDelegate &func = *i;
if(func.target<EventDelegate>() == proc.target<EventDelegate>())
return false;
}
list.push_back(proc);
}
bool EventManager::RemoveListener(IEventData::id_t id, EventDelegate proc){
auto j = mEventListeners.find(id);
if(j == mEventListeners.end()) return false;
auto &list = j->second;
for(auto i = list.begin(); i != list.end(); ++i){
EventDelegate &func = *i;
if(func.target<EventDelegate>() == proc.target<EventDelegate>()) {
list.erase(i);
return true;
}
}
return false;
}
void EventManager::QueueEvent(IEventDataPtr ev) {
mEventQueue.push_back(ev);
}
void EventManager::ProcessEvents(){
size_t count = mEventQueue.size();
for(auto it = mEventQueue.begin(); it != mEventQueue.end(); ++it){
printf("Processing event..\n");
if(!count) break;
auto &i = *it;
auto listeners = mEventListeners.find(i->GetID());
if(listeners != mEventListeners.end()){
// Call listeners
for(auto l : listeners->second){
l(i);
}
}
// remove event
it = mEventQueue.erase(it);
count--;
}
}
I use an EventListener class for the sake of convenience as base class for any class that would like to listen to events. If you derive your listening class from this class and supply it with your event manager, you can use the very convenient function OnEvent(..) to register your events. And the base class will automatically unsubscribe your derived class from all events when it is destroyed. This is very convenient since forgetting to remove a delegate from event manager when your class is destroyed will almost certainly cause your program to crash.
A neat way to get a unique type id for an event by simply declaring a static function in the class and then casting it's address into an int. Since every class will have this method on different addresses, it can be used for unique identification of class events. You can also cast typename() to an int to get a unique id if you want. There are different ways to do this.
So here is an example on how to use this:
#include <functional>
#include <memory>
#include <stdio.h>
#include <list>
#include <map>
#include "Events.hpp"
#include "Events.cpp"
using namespace std;
class DisplayTextEvent : public IEventData {
public:
DECLARE_EVENT(DisplayTextEvent);
DisplayTextEvent(const string &text){
mStr = text;
}
~DisplayTextEvent(){
printf("Deleted event data\n");
}
const string &GetText(){
return mStr;
}
private:
string mStr;
};
class Emitter {
public:
Emitter(shared_ptr<IEventManager> em){
mEmgr = em;
}
void EmitEvent(){
mEmgr->QueueEvent(shared_ptr<IEventData>(
new DisplayTextEvent("Hello World!")));
}
private:
shared_ptr<IEventManager> mEmgr;
};
class Receiver : public EventListener{
public:
Receiver(shared_ptr<IEventManager> em) : EventListener(em){
mEmgr = em;
OnEvent<DisplayTextEvent>([&](shared_ptr<DisplayTextEvent> data){
printf("It's working: %s\n", data->GetText().c_str());
});
}
~Receiver(){
mEmgr->RemoveListener(DisplayTextEvent::ID(), std::bind(&Receiver::OnExampleEvent, this, placeholders::_1));
}
void OnExampleEvent(IEventDataPtr &data){
auto ev = dynamic_pointer_cast<DisplayTextEvent>(data);
if(!ev) return;
printf("Received event: %s\n", ev->GetText().c_str());
}
private:
shared_ptr<IEventManager> mEmgr;
};
int main(){
auto emgr = shared_ptr<IEventManager>(new EventManager());
Emitter emit(emgr);
{
Receiver receive(emgr);
emit.EmitEvent();
emgr->ProcessEvents();
}
emit.EmitEvent();
emgr->ProcessEvents();
emgr = 0;
return 0;
}
This probably does not only apply to game classes but to classes in the general sense. the MVC (model-view-controller) pattern together with your suggested message pump is all you need.
"Enemy" and "Player" will probably fit into the Model part of MVC, it does not matter much, but the rule of thumb is have all models and views interact via the controller. So, you would want to keep references (better than pointers) to (almost) all other class instances from this 'controller' class, let's name it ControlDispatcher. Add a message pump to it (varies depending on what platform you are coding for), instantiate it firstly (before any other classes and have the other objects part of it) or lastly (and have the other objects stored as references in ControlDispatcher).
Of course, the ControlDispatcher class will probably have to be split down further into more specialized controllers just to keep the code per file at around 700-800 lines (this is the limit for me at least) and it may even have more threads pumping and processing messages depending on your needs.
Cheers
Be careful with "a message style system", it probably depends on implementation, but usually you would loose static type checking, and can then make some errors very difficult to debug. Note that calling object's methods it is already a message-like system.
Probably you are simply missing some levels of abstraction, for example for navigation a Player could use a Navigator instead of knowing all about the Map itself. You also say that this has usually descended into setting lots of pointers, what are those pointers? Probably, you are giving them to a wrong abstraction?.. Making objects know about others directly, without going through interfaces and intermediates, is a straight way to getting a tightly coupled design.
Messaging is definitely a great way to go, but messaging systems can have a lot of differences. If you want to keep your classes nice and clean, write them to be ignorant of a messaging system and instead have them take dependencies on something simple like a 'ILocationService' which can then be implemented to publish/request information from things like the Map class. While you'll end up with more classes, they'll be small, simple and encourage clean design.
Messaging is about more than just decoupling, it also lets you move towards a more asynchronous, concurrent and reactive architecture. Patterns of Enterprise Integration by Gregor Hophe is a great book that talks about good messaging patterns. Erlang OTP or Scala's implementation of the Actor Pattern have provided me with a lot of guidance.
#kellogs suggestion of MVC is valid, and used in a few games, though its much more common in web apps and frameworks. It might be overkill and too much for this.
I would rethink your design, why does the Player need to talk to Enemies? Couldn't they both inherit from an Actor class? Why do Actors need to talk to the Map?
As I read what I wrote it starts to fit into an MVC framework...I have obviously done too much rails work lately. However, I would be willing to bet, they only need to know things like, they are colliding with another Actor, and they have a position, which should be relative to the Map anyhow.
Here is an implementation of Asteroids that I worked on. You're game may be, and probably is, complex.