i have an encoder class with lots of methods . this is a subclass of Qthread. i am new to multi-threading and
trying to understand how this class is
threading its methods
... i understand to thread a method it has to be in a subclass of qthread. and the run of this implements the threaded code for this class. And the thread starts only when a call to start method on the object of this class is made.
Question : firstly what do you infer
from the this run implementation
void Encoder::run(void)
{
VERBOSE(VB_DEBUG, "Encoder::run");
if (WILL_PRINT(VB_DEBUG))
print_stats_timer_id = QObject::startTimer(kEncoderDebugInterval);
health_check_timer_id = QObject::startTimer(kEncoderHealthCheckInterval);
if (init())
exec();
else
VERBOSE(VB_ERROR, "Encoder::run -- failed to initialize encoder");
QObject::killTimer(health_check_timer_id);
if (print_stats_timer_id)
QObject::killTimer(print_stats_timer_id);
cleanup();
}
Question: what does thread context mean in
relation to its methods .
also
Question: what would happen If a method of this
class is called before this class's
thread has started
The class you have written creates a thread and initializes a QObject::timer. It then goes on to call a user defined init() function then the QThread::exec() function.
My guess is that you intended that exec() would be a user defined function where the actual work is to occur. Be aware that QThread::exec() processes the thread's Qt Event Queue.
Also, on some platforms you may get an "Error creating timer from thread" warning message. I've encountered this error on Windows when the code executed fine on Linux
Also, be aware that your timer will never occur if you do not call the QThread::exec() function or QApplication::processEvents() from within your thread.
Thread context in Qt is the same as any other thread concept. That is, all memory is shared between the threaded code (entered at this point in your "run()" function). And any other context which calls into your object. If this object may ever be executing in a thread and accessed from outside of the thread you must protect the shared data.
Because all data is shared between thread contexts (it's a shared memory multiprocessing model) there is no problem with calling functions before/after/during thread execution. Given that:
The object is fully constructed before you call any method. This is not special to threads, necessarily, unless the object is created in a thread.
Any data member is protected with a mutex lock (I eluded to this in #2). QMutexLocker is a handy stack based RAII way of dealing with mutex locks in Qt.
I believe I fully answered your question here, so I'll go ahead and link to RAII and threading articles I have written on another site, just for further reference.
Edit: specificity about threading scenarios:
class MyThreadedClass : public QThread
{
MyThreadClass(const boost::shared_ptr<SomeOtherClass> &t_object)
: m_object(t_object) {}
void doSomething()
{
// Depending on how this method was called (from main, from internal thread)
// will determine which thread this runs on, potentially complicating thread
// safety issues.
m_object->someThing();
}
void run()
{
// I'm now in a thread!
m_object->someFunction(); // oops! The call to someFunction is occurring from
// a thread, this means that SomeOtherClass must be
// threadsafe with mutex guards around shared
// (object level) data.
// do some other stuff
}
};
int main()
{
MyThreadClass thread(someobjectfromsomewhere);
thread.start(); // MyThreadClass is now running
thread.doSomething(); // The call to doSomething occurs from main's thread.
// This means 2 threads are using "thread", main
// and "thread"'s thread.
// The call to thread.doSomething hits Thread.m_object, which means that
// now multiple threads are also accessing m_object ("thread" and "main").
// This can all get very messy very quickly. It's best to tightly control
// how many threads are hitting an object, and how
}
NOTE: It would be a good idea to investigate QFuture, which is designed to handle this kind of asynchronous task, like an encoder, that you are looking at QFuture will avoid some of the potential threading issues of shared data and deadlocks.
Related
I'm creating a logging object which performs the real file writing work on a separate std::thread, and offers an interface to a log command buffer, syncing the caller threads and the one worker thread. Access to the buffer is protected by a mutex, there's an atomic bool for the worker thread exit condition, and I'm using Windows native Events as a signal to wake up the worker thread when new commands arrive. The object's constructor spawns the worker thread so it is immediately available. The worker thread is simply a while loop checking the exit condition, with in the loop a blocking wait for the signal. The object's destructor finally just sets the exit condition, signals the thread to wake up and joins it to ensure it's down before the object is fully destroyed.
Seems simple enough, and when using such an object somewhere in a function it works nicely. However, when declaring such an object as a global variable to have it usable for everyone it stops working. I'm on Windows, using Visual Studio 2017 with the 2015 tool chain. My project is a DLL plugin for another application.
The things I tried so far:
Start the thread in the constructor of the global object. This however makes the main thread hang immediately when my DLL is loaded. Pausing the app in the debugger reveals we're in the std lib, at a point where the main thread should have launched the worker thread and is now stuck waiting for a condition variable, presumably one that is signaled by the worker thread once it is launched?
Delay-construct the thread on demand when we first use the global object from somewhere else. This way constructing it goes nicely without a hang. However, when signalling the worker thread to exit from the destructor, the signal is sent, but the join on the worker thread now hangs. Pausing the app in the debugger reveals our main thread is the only one still alive, and the worker thread is already gone? A breakpoint placed in the worker thread function right before the close brace reveals it is never hit; the thread must be getting killed?
I also tried to start the thread via a std::future, starting it up async, and that one launches perfectly fine from the constructor in global objects. However, when the future tries to join the thread in the destructor, it hangs as well; here again no worker thread to be detected anymore while no breakpoint gets hit in it.
What could be going on? I can't imagine it's because the thread construction and destruction takes place outside main() so to speak; these std primitives should really be available at such moments, right? Or is this Windows specific and is the code running in the context of DllMain's DLL_PROCESS_ATTACH / DLL_THREAD_ATTACH events, where starting up threads might wreak havoc due to thread local storage not yet being up and running or such? (would it?)
EDIT -- added code sample
The following is an abbreviation/simplification of my code; it probably doesn't even compile but it gets the point across I hope :)
class LogWriter {
public:
LogWriter() :
m_mayLive(true) {
m_writerThread = std::thread(&C_LogWriter::HandleLogWrites, this); // or in initializer list above, same result
};
~LogWriter() {
m_mayLive = false;
m_doSomething.signal();
if (m_writerThread.joinable()) {
m_writerThread.join();
}
};
void AddToLog(const std::string& line) { // multithreaded client facing interface
{
Locker locker; // Locker = own RAII locker class
Lock(locker); // using a mutex here behind the scenes
m_outstandingLines.push_back(line);
}
m_doSomething.signal();
}
private:
std::list<std::string> m_outstandingLines; // buffer between worker thread and the rest of the world
std::atomic<bool> m_mayLive; // worker thread exit signal
juce::WaitableEvent m_doSomething; // signal to wake up worker thread; no std -- we're using other libs as well
std::thread m_writerThread;
int HandleLogWrites() {
do {
m_doSomething.wait(); // wait for input; no busy loop please
C_Locker locker; // access our line buffer; auto-released at end of loop iteration
Lock(locker);
while (!m_outstandingLines.empty()) {
WriteLineToLog(m_outstandingLines.front());
m_outstandingLines.pop_front();
if (!m_outstandingLines.empty()) {
locker.Unlock(); // don't hog; give caller threads some room to add lines to the buffer in between
std::this_thread::sleep_for(std::chrono::milliseconds(10));
Lock(locker);
}
};
} while (m_mayLive); // atmoic bool; no need to mutex it
WriteLineToLog("LogWriter shut down"); // doesn't show in the logs; breakpoints here also aren't being hit
return 0;
}
void WriteLineToLog(const std::string& line) {
... fopen, fprintf the line, flush, close ...
}
void Lock(C_Locker& locker) {
static LocalLock lock; // LocalLock is similar to std::mutex, though we're using other libs here
locker.Lock(&lock);
}
};
class Logger {
public:
Logger();
~Logger();
void operator() (const char* text, ...) { // behave like printf
std::string newLine;
... vsnprintf -> std::string ...
m_writer.AddToLog(newLine);
}
private:
LogWriter m_writer;
};
extern Logger g_logger; // so everyone can use g_logger("x = %d\n", x);
// no need to make it a Meyer Singleton; we have no other global objects interfering
Since you're writing a DLL in C++, you have to understand how "globals" in DLL's work. The compiler sticks their initialization in DllMain, before anything else that you would do there. But there are some strict rules what you can do in DllMain, as it runs under loader lock. The short summary is that you can't call anything in another DLL because that DLL cannot be loaded while your DllMain is running. Calling CreateThread is definitely not allowed, not even if wrapped inside a std::thread::thread constructor.
The problem with the destructor is quite possibly because your DLL has exited (can't tell without code). The DLL unloads before the EXE, and their respective globals are also cleaned up in that order. Any attempt to log from a destructor in an EXE will fail for obvious reasons.
There is no simple solution here. Andrei Alexandrescu's "Modern C++ Design" has a reasonable solution for logging in the non-DLL case, but you'll need to harden that for use in a DLL. An alternative is to check in your logging functions if your logger still exists. You can use a named mutex for that. If your log function fails in OpenMutex, then either the logger does not exist yet or it no longer exists.
Think I've encountered that destruction issue with DLLs to use with Unity.
The only solution I found back then was to essentially give up true global variables that would need cleanup.
Instead I put them in a separate class which is instantiated only a single time into a global pointer by some custom launch function. Then my DLL got a "quit()" function also called by the user of the DLL. The quit function correctly destroys the instance carrying the global variables.
Probably not the smoothest solution and you have a pointer-indirection on every access to the global variables, but it turned out to be comfortable for serializing the state of the global variables as well.
I am doing some work with threading on an embedded platform. This platform provides a Thread class, and it has a start method that takes a function pointer, like this:
void do_in_parallel() {
// Some stuff to do in a new thread
}
Thread my_thread;
my_thread.start(do_in_parallel);
The problem is there is no way to pass parameters in.1 I want to solve this by creating an abstract class, call it Thread2, that extends Thread (or it could just have a Thread as instance data).
Thread2 would have a pure virtual function void run() and the goal was to pass that to Thread::start(void*()), except I soon learned that member function pointers have a different type and can't be used like this. I could make run() static, but then I still can't have more than one instance, defeating the whole purpose (not to mention you can't have a virtual static function).
Are there any workarounds that wouldn't involve changing the original Thread class (considering it's a library that I'm stuck with as-is)?
1. Global variables are a usable workaround in many cases, except when instantiating more than one thread from the same function pointer. I can't come up with a way to avoid race conditions in that case.
Write a global thread pool.
It maintains a queue of tasks. These tasks can have state.
Whe you add a task to the queue, you can choose to also request it get a thread immediately. Or you can wait for threads in the pool to be finished what they are doing.
The threads in the pool are created by the provided Thread class, and they get their marching instructions from the pool. For the most part, they should pop tasks, do them, then wait on another task being ready.
If waiting isn't permitted, you could still have some global thread manager that stores state for the threads.
The pool/manager returns the equivalent of a future<T> augmented with whatever features you want. Code that provides tasks interacts with the task through that object instead of the embedded Thread type.
A simple wrapper can be written if locking is permitted
void start(Thread& t, void (*fn)(void*), void* p)
{
static std::mutex mtx; // or any other mutex
static void* sp;
static void (*sfn)(void*);
mtx.lock();
sp = p;
sfn = fn;
t.start([]{
auto p = sp;
auto fn = sfn;
mtx.unlock();
fn(p);
});
}
This is obviously not going to scale well, all thread creations goes through the same lock, but its likely enough.
Note this is exception-unsafe, but I assume that is fine in embedded systems.
With the wrapper in place
template<typename C>
void start(Thread& t, C& c)
{
start(t, [](void* p){
(*(C*)p)();
}, &c);
}
Which allows any callable to be used. This particular implementation places the responsibility of managing the callable's lifetime on the caller.
You can create your own threaded dispatching mechanism (producer-consumer queue) built around the platform specific thread.
I assume that you have the equivalent facilities of mutex and conditional variables/signalling mechanism for the target platform.
Create a thread safe queue that can accept function objects.
The run method creates a thread and waits on the queue.
The calling thread can call post()/invoke() method that simply insert a function object to the queue.
The function object can have the necessary arguments passed to the caller thread.
I have 2 Threads in a Qt5Application:
Thread A: contains a bunch of QObject derived class objects
Thread B: worker in this Thread has all the pointers to the objects in A
Thread A might be very busy at times and Thread B is only there to delegate Signals and manage some other stuff. It never writes to any of these objects, but I need to check some getter functions which return booleans from the objects in A.
in ThreadB:
if (objInThrA->isFinished()) { ... }
The isFinished() returns a boolean.
If Thread A is really busy in a function and I call these isFinished functions in Thread B, will my Thread B get stalled until Thread A is finished with its work, or will this work?
Qt signals and slots can be used between different threads. But there are two rules:
Last argument of connect() should be either Qt::QueuedConection or Qt::BlockingQueuedConnection (or defaults to Qt::AutoConnection, which is the same as Qt::QueuedConnection if objects belong to different threads). QueuedConnection means that emitter does not wait signal processing to be completed, BlockingQueuedConnection means it does.
QObject-derived classes are not suitable for passing between threads. They should be safely copied before that (see QMetaType docs).
You should never access members or directly call functions of an object which in another thread. The only safe way is to access them through signal/slot mechanism. You can have a signal in ThreadB and connect it to a slot in ThreadA which returns the value:
connect(objInThrB, SIGNAL(getFinished()), objInThrA, SLOT(isFinished()), Qt::BlockingQueuedConnection);
This way when you emit the signal in thread B like:
bool ret = getFinished();
the slot in thread A would be called when control returns to the event loop of thread A and the value would be returned. Thread B waits for the slot to be called, that's because the connection type is BlockingQueuedConnection. So be aware not to block the application main thread using this kind of blocking connection.
Ok, so I tested it myself.
In Thread B:
connect(this,SIGNAL(runWork()),objInThrA,SLOT(doWork()));
emit runWork();
QThread::sleep(2);
qDebug() << objInThrA->isFinished();
in Thread A:
qDebug() << "start A sleep";
QThread::sleep(10);
qDebug() << "end A sleep";
OUTPUT:
start A sleep
false
end A sleep
It works, however Im still unsure if I use it this way its correctly done and defined behavior.
The short answer is no.
When you run as described, you're directly calling methods from an object which is not in your thread (the object is in Thread A but you're calling one of its methods from Thread B). Directly calling a method is not something Qt has modified from standard C++, so it doesn't operate through signals, slots, or the event loop and is ignorant of your threading model. That means that if you call the method in Thread B, it runs in Thread B.
Calling methods on an object from another thread can be dangerous if you aren't careful about it because it introduces problems with concurrency. If Thread A is in the middle of updating the _mFinished data member when Thread B calls getFinished() on it, it may get a partially written value. In your specific example it happens to be that there is no "partially written" state for the boolean and you'll probably be fine.
Resolving concurrency problems is done through atomic operations on the elements which will be shared between multiple threads. Either you can use variables that are guaranteed to be atomic read and write operations, like booleans usually are, or you can use locking mechanisms to protect those variables. Mutexes and Semaphores are the most common locking mechanism and allow locking from multiple threads to limit access to the variables while they're being read and written. This is done in such a way as to avoid them being partially written with a new value when a read of the variable occurs.
I'd recommend reading up on mutexes and semaphores (generic multithreaded data structures) if you're doing work that involves more than one thread since they're critical to understanding threading in general. Additionally Qt has nice Qt wrapped versions of them that keep them simple to use and avoid some easy pitfalls in maintaining their usage (e.g. QMutexLocker).
I have a gtkmm application and I'm trying to put some long running tasks into separate threads so they don't lock the GUI. Here's a tutorial I've based my design on:
http://www.velvetcache.org/2008/09/30/gtkmmglibmm-thread-example
I use Glib::Dispatcher signals to notify the GUI thread when the work is done or something needs to be updated, however I'm not sure how to pass the data between the worker thread and GUI thread. So far I've been passing a pointer to the class which creates the worker to the worker and then modifying public members of the class, but something tells me it's not the most correct to do it. Here's an example:
class Some_GUI_class
{
public:
std::string thread_message;
private:
Worker_class* worker;
void start_worker()
{
if (worker != NULL) return;
worker = new Worker_class(this);
worker->sig_message.connect(sigc::mem_fun(*this, &Some_GUI_class::display_message_from_thread);
worker.start();
}
void display_message_from_thread()
{
some_label->set_text(thread_message);
}
}
class Worker_class
{
public:
Worker_class(Some_GUI_class* gui_class) : gui_class(gui_class)
{}
void start()
{
thread = Glib::Thread::create(sigc::mem_fun(*this, &Worker_class::run), true);
}
Glib::Dispather sig_message;
protected:
Glib::Thread* thread;
Glib::Mutex mutex;
Some_GUI_class* gui_class;
void run()
{
// ...
gui_class->thread_message = "Message from a thread!";
sig_message();
}
}
This essentialy works, but I guess if the GUI thread wanted to modify thread_message at the same time there would be a problem? Is it safe to do it like this then as long as I'm sure the variables are only modified by a single thread or is there a better way?
You have a race condition. Even if your gui thread doesn't modify thread_message, allowing the GUI thread to read it while another thread is modifying it is not going to give you long term happiness. This is because std::string is not itself protected from multiple threads accessing it, and has multiple internal fields. If one thread is in the process of modifying one of its internal fields, while another is reading them, the internal state will not be consistent from the point of view of the second.
You can use a mutex in the GUI class to protect access to the variables which might be accessed by another thread. Lock and unlock the mutex in get/set routines, and use those routines for all other accesses to ensure that only one thread gets to access or modify the variables at one time.
Generally mutex usage is not enough to achieve the desired behaviour. The same worker thread (or another one if you have it) could want to send another message while first one had not been processed by the main thread yet. That is why in addition to mutex you should use message queue (e.g. object of std::deque<std::string> class) instead of just a std::string Some_GUI_class::thread_message variable to avoid this kind of message loss.
Follow up question to:
This question
As described in the linked question, we have an API that uses an event look that polls select() to handle user defined callbacks.
I have a class using this like such:
class example{
public:
example(){
Timer* theTimer1 = Timer::Event::create(timeInterval,&example::FunctionName);
Timer* theTimer2 = Timer::Event::create(timeInterval,&example::FunctionName);
start();
cout<<pthread_self()<<endl;
}
private:
void start(){
while(true){
if(condition)
FunctionName();
sleep(1);
}
}
void FunctionName(){
cout<<pthread_self()<<endl;
//Do stuff
}
};
The idea behind this is that you want FunctionName to be called both if the condition is true or when the timer is up. Not a complex concept. What I am wondering, is if FunctionName will be called both in the start() function and by the callback at the same time? This could cause some memory corruption for me, as they access a non-thread safe piece of shared memory.
My testing tells me that they do run in different threads (corruption only when I use the events), even though: cout<<pthread_self()<<endl; says they have the same thread id.
Can someone explains to me how these callbacks get forked off? What order do they get exectued? What thread do they run in? I assume they are running in the thread that does the select(), but then when do they get the same thread id?
The real answer would depend on the implementation of Timer, but if you're getting callbacks run from the same thread, it's most likely using signals or posix timers. Either way, select() isn't involved at all.
With signals and posix timers, there is very little you can do safely from the signal handler. Only certain specific signal safe calls, such as read() and write() (NOT fread() and fwrite(), or even new and cout) are allowed to be used. Typically what one will do is write() to a pipe or eventfd, then in another thread, or your main event loop running select(), notice this notification and handle it. This allows you to handle the signal in a safe manner.
Your code as written won't compile, much less run. Example::FunctionName needs to be static, and needs to take an object reference to be used as a callback function.
If the timers run in separate threads, it's possible for this function to be called by three different threads.