vtkRenderWindowInteractor event loop and threading - c++

What I am trying to do in an application using vtk for both interacting and rendering is to have two different parts:
1 - A thread with Rendering and vtkRenderWindowInteractor for interaction with mouse.
2 - A thread that call some modifier functions of the data defined in the VTK Thread.
From what I've gotten so far in my research it seems rather complicated and VTK is not thread safe. Now I've stumbled upon this post (http://vtk.1045678.n5.nabble.com/Multi-threaded-VTK-td4514620.html) on the VTK mailing list that suggests using Qt Signals and Slots. A first question would be is that still the good solution?
A second question which is still linked to that and to a problem that I've encountered before is that the start()of the vtkRenderWindowInteractor is blocking. And so far, no matter what I've tried all the modification done by rotation or translation or scaling functions are not done as long as the start() method is called (because I enter a rendering loop).
My question would then be: If I use Qt Signals and Slots will that prevent me from that problem?
Here is the basic code that I have so far for rendering and lauching the vtkRenderWindowInteractor:
std::string filename = BUNNY;
// Read all the data from the file
vtkSmartPointer<vtkXMLPolyDataReader> reader =vtkSmartPointer<vtkXMLPolyDataReader>::New();
reader->SetFileName(filename.c_str());
reader->Update();
inputPolyData = reader->GetOutput();
cout << "File Found and Loaded : " << filename << endl ;
vtkSmartPointer<vtkTransform> translation = vtkSmartPointer<vtkTransform>::New();
translation->Translate(0.3, -0.05, 0);
transformFilter = vtkSmartPointer<vtkTransformPolyDataFilter>::New();
//transformFilter->SetInputConnection(reader->GetOutputPort());
transformFilter->SetInputData(inputPolyData);
transformFilter->SetTransform(translation);
//transformFilter->Update();
vtkSmartPointer<vtkPolyDataMapper> mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
mapper->SetInputConnection(transformFilter->GetOutputPort());
mainActor = vtkSmartPointer<vtkActor>::New();
mainActor->SetMapper(mapper);
ren->AddActor(mainActor);
vtkRenderWindowInteractor *iren = vtkRenderWindowInteractor::New();
iren->SetRenderWindow(win);
vtkInteractorStyleMultiTouchCamera *style =
vtkInteractorStyleMultiTouchCamera::New();
iren->SetInteractorStyle(style);
//Start the event loop
iren->Initialize();
iren->Start();
//defineClipping();
win->PolygonSmoothingOn();
win->Render();
win->Start();
ctxView->Render();
So that I could sum it up by asking: will Qt allow me to have to call transforming functions while the rendering and interacting thread of vtk is running with the blocking start() method of vtkRenderWindowInteractor? If not should I change my code and think about different possibilities for interacting with my objects in VTK?

I've been able to do rotations after calling start(), but in my case from the same thread.
The trick is to use a vtkCommand and set a timer in the vtkRenderWindowInteractor to call that command. That command is basically a callback that will be able to modify your actors.
You can see a working example of this in this thread.
Regarding the multi-threading approach you're using, maybe you could keep the rendering thread waiting in vtkCommand::Execute until the modifying thread is done. If you're able to use C++11 you could use a lot of the new tools available in the STL.

Related

Asynchronous Loading of images in gtkmm

Currently, in my project, we are using gtkmm pixbuf create_from_file or create_from_date which hangs up whole GUI for 1-2 seconds in case of high-resolution images and in case of loading multiple images for a screen it becomes awfully slow. Is it possible to load images asynchronously in gtkmm for the above two functions? I am able to find methods in gtk for loading images asynchronously but not in gtkmm. An example would be helpful since I am unable to find anything related to it.
if(!imageName.empty())
{
//Load image in pixbuf
picPixBuff = Gdk::Pixbuf::create_from_file(imageName);
picPixBuff = picPixBuff->scale_simple(150,35,Gdk::INTERP_BILINEAR);
}
I have gone through this.
Related Question - How to load a widget as a different thread in gtk? (vala)
There are docs for that. That example does exactly what you are asking for.
These are the functions that do the magic. The example is easy to follow.
// notify() is called from ExampleWorker::do_work(). It is executed in the worker
// thread. It triggers a call to on_notification_from_worker_thread(), which is
// executed in the GUI thread.
void ExampleWindow::notify()
{
m_Dispatcher.emit();
}
void ExampleWindow::on_notification_from_worker_thread()
{
if (m_WorkerThread && m_Worker.has_stopped())
{
// Work is done.
if (m_WorkerThread->joinable())
m_WorkerThread->join();
delete m_WorkerThread;
m_WorkerThread = nullptr;
update_start_stop_buttons();
}
update_widgets();
}

OpenGL multithreading

Ok I am trying to switch my Game Engine to multithreading. I have done the research on how to make it work to use OpenGL in multithreaded application. I have no problem with rendering or switching contexts. Let my piece of code explain the problem :) :
for (it = (*mIt).Renderables.begin(); it != (*mIt).Renderables.end(); it++)
{
//Set State Modeling matrix
CORE_RENDERER->state.ModelMatrix = (*it).matrix;
CORE_RENDERER->state.ActiveSubmesh = (*it).submesh;
//Internal Model Uniforms
THREAD_POOL->service.post([&]
{
for (unsigned int i = 0; i < CORE_RENDERER->state.ActiveShaderProgram->InternalModelUniforms.size(); i++)
{
CORE_RENDERER->state.ActiveShaderProgram->InternalModelUniforms[i]->Set( CORE_RENDERER->state.ModelMatrix);
}
CORE_RENDERER->state.ActiveSubmesh->_Render();
});
//Sleep(10);
}
I'll quickly explain what are the elements in the code to make my problem more clear. Renderables is a simple std::vector of elements with _Render() function which works perfectly. CORE_RENDERER->state is a struct holding information about current render state such as current material properties as well as current submesh ModelMatrix. So Matrix and Submesh are stored to state struct (I KNOW THIS IS SLOW, I'll probably change that in time :) ) The next piece of code is sent to THREAD_POOL->service which is actually boost::asio::io_service and has only one thread so it acts like a queue of rendering commands. The idea is that the main thread provides information about what to render and do frustum culling and other tests while an auxilary thread does actual rendering. This works fine, except there is a slight problem:
The code that is sent to thread pool starts to execute, but before all InternalModelUniforms are set and submesh is rendered the next iteration of Renderables is executed and both ModelMatrix and ActiveSubmesh are changed. The program doesn't crash but both informations change and some meshes are rendered some matrices are right others not which results in flickering image. Objects apear on frame and the next frame they are gone. The problem is only fixed if I enable that Sleep(10) function which makes sure that the code is executed before next iteration which obviously kills the idea of gaining preformance. What is the best possible solution for this? How can I send commands to the queue each with unique built in data? Maybe I need to implement my own queue for commands and a single thread without io_service?
I will continue my research as I know there is a way. The idea is right cause I get preformance boost as not a single if/else statement is processed by the rendering thread :) Any help or tips will really help!
Thanks!
Update:
After struggling for few nights I have created a very primitive model of communication between main thread and an Aux Thread. I created a class that represents a base command to be executed by aux thread:
class _ThreadCommand
{
public:
_ThreadCommand() {}
~_ThreadCommand() {}
virtual void _Execute() = 0;
virtual _ThreadCommand* Clone() = 0;
};
These commands that are childs of this class have _Execute() function to do whatever operation needs to be done. The main thread upon rendering fills a boost::ptr_vector of these commands While aux thread keeps on checking if there are any commands to process. When commands are found it copies entire vector to it's own vector inside _AuxThread and clears the original one. Commands are then executed by calling _Execute functions on each:
void _AuxThread()
{
//List of Thread commands
boost::ptr_vector<_ThreadCommand> _cmd;
//Infinitive loop
while(CORE_ENGINE->isRunning())
{
boost::lock_guard<boost::mutex> _lock(_auxMutex);
if (CORE_ENGINE->_ThreadCommands.size() > 0)
{
boost::lock_guard<boost::mutex> _auxLock(_cmdMutex);
for (unsigned int i = 0; i < CORE_ENGINE->_ThreadCommands.size(); i++)
{
_cmd.push_back(CORE_ENGINE->_ThreadCommands[i].Clone());
}
//Clear commands
CORE_ENGINE->_ThreadCommands.clear();
//Execute Commands
for (unsigned int i = 0; i < _cmd.size(); i++)
{
//Execute
_cmd[i]._Execute();
}
//Empty _cmd
_cmd.clear();
}
}
//Notify main thread that we have finished
CORE_ENGINE->_ShutdownCondition->notify_one();
}
I know that this is a really bad way to do it. Preformance is quite slower which I'm quite sure is because of all the copying and mutex locks. But at least the renderer works. You can get the idea of what I want to achieve but as I said I am very new to multithreading. What is the best solution for this scenario? Should I return back to ThreadPool system with asio::io_service? How can I feed commands to AuxThread with all values that must be sent to renderer to preform tasks in correct way?
First, a warning. Your "slight problem" is not slight at all. It is race condition, which is undefined behavior in C++, which, in turn, implies that anything could happen, including:
Everything renders fine
Image flickers
Nothing renders at all
It crashes on the last Saturday of every month. Or working fine on your computer and crashing on everyone's else.
Seriously, do not ever rely on UB, especially when writing library/framework/game engine.
Now about your question.
Lets leave aside any practical benefits of your approach and fix it first.
Actually, OpenGL implementation uses something very similar under the hood. Commands are executed asynchronously by the driver thread. I recommend you to read about their implementation to get some ideas on how to improve your design.
What you need to do, is to somehow "capture" the state at the time you post a rendering command. Simplest possible thing - copy the CORE_RENDERER->state into closure and use this copy to do the rendering. If state is large enough, it can be costly, though.
Alternative solution (and OpenGL goes that way) is to make every change in the state a command also, so
CORE_RENDERER->state.ModelMatrix = (*it).matrix;
CORE_RENDERER->state.ActiveSubmesh = (*it).submesh;
translates into
Matrix matrix = (*it).matrix;
Submesh submesh = (*it).submesh;
THREAD_POOL->service.post([&,matrix,submesh]{
CORE_RENDERER->state.ModelMatrix = matrix;
CORE_RENDERER->state.ActiveSubmesh = submesh;
});
Notice, however, that now you can't simply read CORE_RENDERER->state.ModelMatrix from your main thread, as it is changing in a different thread. You must first ensure that command queue is empty.

Read a file in background to update Qjsonvalue

I need to update the content of a field on my QWidget via a JSON file (updated in real time). I've read about functions readLine() and readAll() of QFile, but when I try a loop like :
while(true):
jsfile.readLine()
creation of objects, update of values, display etc ...
I lost the focus on my window. But I want to keep the control of the application with my buttons and obviously to watch the evolution of the JSON values.
I have thought that Qt manages itself the events and keeps the focus on the current window, but like I've said, it's not the case.
Is there a good solution (multi threads maybe) to use my window while the application reads the file (with new informations in real time)?
(With the constraint "real time" I can't read the whole file every time and I've no choice about the format of this file)
Update
I tried the thread method.
So, I choose to create my thread instance into the main (with my main window) and connect here. But, when I run the program, I've this error :
no matching member function for call to 'connect'
Reader reader;
QObject::connect(controler, SIGNAL(ready()),
reader, SLOT(received()));
According to this error, I've thought that the reason was main don't inherits of Object, and so, I've move the connection ans the creation of thread instance into my main window.
Reader reader;
QObject::connect(reader, SIGNAL(newobject(QJsonObject)),
this, SLOT(displayJSON(QJsonObject)));
With this one, I've the same error while I've already connect lot of widget into this class without any error.
What can be the problem ?
Update 2
I've a solution when I give as argument my main window (controler) in reader's constructor and connect into this one but, if possible, I would an explanation for the previous problem.
The current problem that I have is that signals are emit well but slots are executed after the end the application (so after the end of the thread's execution and not during)
This isn't really the subject of this topic so we can close this one.
You can use QThread (Qt documentation: QThread) class to create a thread, which will read your file. The main thread will execute your GUI application and it will be available during file reading.
You can find a simple example in documentation for creating your thread:
class WorkerThread : public QThread
{
Q_OBJECT
void run() Q_DECL_OVERRIDE {
QString result;
/* ... here is the expensive or blocking operation ... */
emit resultReady(result);
}
signals:
void resultReady(const QString &s);
};
void MyObject::startWorkInAThread()
{
WorkerThread *workerThread = new WorkerThread(this);
connect(workerThread, &WorkerThread::resultReady, this, &MyObject::handleResults);
connect(workerThread, &WorkerThread::finished, workerThread, &QObject::deleteLater);
workerThread->start();
}
You can modify this example for your purpose. For example, WorkerThread for your task may be something like this:
class WorkerThread : public QThread
{
Q_OBJECT
void run() Q_DECL_OVERRIDE {
while(!stopFlag)
{
// read JSON file to QByteArray. Use QFile and QTextStream
// use QJsonDocument to read JSON content
// find what is new in JSON
emit signalSomethingNew(/*parameters*/);
QThread::currentThread()->msleep(/*timeout*/);
}
}
signals:
void signalSomethingNew(/*parameters*/);
};
At the end you must implement slot on your QWidget for signalSomethingNew(/*parameters*/) and make connection:
connect(yourThread, &WorkingThread::signalSomethingNew, youWidget, &YouWidget::yourSlot);
For working with JSON data: QJsonDocument
I'm interpreting your question as "my application is unresponsive whilst doing work" rather than "my focus jumped to another window" - please comment if you meant something different.
You have a choice of options:
Create and run a background QThread to do the work. Have it emit signals (connected to your widgets using Qt::QueuedConnection - the default) when it has results to display.
This is a good solution when the worker has a lot of computation to do, or needs all the input to be read before it can start. It works very well when the target system has processors available with no other work to do.
Use a QSocketNotifier to signal your GUI thread when some of the input becomes available (note that the name is misleading - it actually works on all kinds of file descriptor, not just sockets).
This is appropriate when the algorithm is simple and incremental - i.e. if a small chunk of input can be read and processed quickly.
Incorporate periodic calls to processEvents() in your algorithm:
auto *const dispatcher = QThread::currentThread()->eventDispatcher;
while (line = json.readLine()) {
doSomethingWith(line);
if (dispatcher)
dispatcher->processEvents();
}
This won't work unless you can modify the algorithm like this - if the loop is in somebody else's (closed) code, then you'll need one of the other solutions.

How to use threads in Qt GUI

I have an application that continuously reads an image from a camera and displays this to a user. The user can adjust different sliders such as exposure and threshold to modify the image in real-time. I also do a bunch of calculations on this image afterwards which sometimes makes the GUI unresponsive, so I decided to use threads to divide the workload.
However, I can't get it to work properly, sometimes I get segmentation faults and a bunch of "assertion ` GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed" warnings when the slider values are changed or if I try to save the image (save button in my GUI), and the GUI sometimes stops updating the image or freezes so you can't move the sliders or push any buttons.
What I tried to do was to use the standard std::thread in C++, and connect it to the start button through a slot.
QObject::connect(btnStart, SIGNAL(clicked()), this, SLOT(RunStartThread()));
void MainMenu::RunStartThread(){
std::thread t1;
t1= std::thread(&MainMenu::Start,this);
t1.detach();
}
void MainMenu::Start() {
run = true;
window->mngr->ReadCalibration();
window->mngr->InitializeCameras();
while (run) {
window->mngr->CaptureImage();
window->mngr->ProcessImages();
UpdateLabels();
}
window->mngr->Stop();
}
When the user changes the slider values they change variables in my manager (mngr above) that captureImage and ProcessImages uses. I tried using a std::mutex lock/unlock when a variable was to be accessed, but it did not change anything. I've tried to find examples of how to do this online, but have yet to find something that has a continuous while-loop.
I'm a newbie when it comes to threads, so just tell me if I'm approaching this in the wrong way.
First for all for inter thread communication use singnals and slots. By default Qt connections do nice thread hopping between threads which lets to avoid complex synchronization.
Secondly you have three ways of using threads: QThread, QRunnable, QtConcurrent::Run (my favorite, since requires minimum amount of code).
In case QThread please do not subclass it! It is common design mistake.
Example:
SomeClass::~SomeClass()
{
SignalStop();
future.result();
}
void SomeClass::RunStartThread(){
future = QtConcurrent::run(this, &SomeClass::DoOnThread);
}
void SomeClass::DoOnThread()
{
while (ShouldContinueToRun()) {
QImage im1 = CaptureImage();
emit ImageCaptured(im1);
QImage im2 = ProcessImages(im1);
emit ImageProcessed(im2);
}
emit JobCompleted();
}
Please note that QObject::connect has last argument which defines how invocation of slot is performed if different thread is involved. See documentation of enumeration used for this argument.
So by default Qt detects if thread hopping is needed or not. Reading carefully about QObject::moveToThread should also help to understand the problem (note you can't move object to different thread if it has a parent).

QtConcurrent mapped and progress report

I'm using QtConcurrent to do some heavy background image processing and I want to display the image while parts of it are being updated progressively.
Each line of the image is computed separately and is passed a functor.
To compute the full image I then have a sequence of item that I pass to QtConcurrent mapped and each line emits a signal when it is done computing
Here is the instantiation of the class Worker:
//living in the main(gui) thread !
Worker::Worker(VideoEngine* engine):_engine(engine){
_watcher = new QFutureWatcher<bool>;
_watcher->setPendingResultsLimit(200);
connect(_watcher, SIGNAL(resultReadyAt(int)), this, SLOT(onProgressUpdate(int)));
connect(_watcher, SIGNAL(finished()), engine, SLOT(engineLoop()));
}
Here is the slot to report progress:
void Worker::onProgressUpdate(int i){
if(i < (int)_rows.size() && i%10==0){
cout << " index = " << i << " y = "<< _rows[i] << endl;
_engine->checkAndDisplayProgress(_rows[i],i);
}
}
Now the usage:
void Worker::_computeTreeForFrame(.../*unrelevant args*/){
....
....
_watcher->setFuture(
QtConcurrent::mapped(_sequence,
boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
}
}
All the signals are emitted but the slot onProgressUpdate gets called only when Qtconcurrent::mapped is done with all the items in the sequence.
When executing it has a huge delay while the sequence is processing and then all slots are executed sequentially afterwards.
I have tried all types of signal/slots connection and none of them changed this behaviour.
Any clue ?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
EDIT after Shf suggestion
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The call was made until now in the main(gui) thread.
I changed the call to :
_computeFrameWatcher->setFuture(QtConcurrent::run(_worker,&Worker::computeTreeForFrame));
Since _computeTreeForFrame is now executed in another thread, I changed the call to QtConcurrent::mapped to:
_watcher->setFuture(QtConcurrent::mapped(_sequence,
boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
_watcher->waitForFinished();
This results in exactly the same behaviour as before.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
EDIT after Marek R suggestion
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Ok so I made so tests and here is what I observed:
QtConcurrent::map :
Doesn't emit the signal resultReadyAt(int)
QtConcurrent::mapped
Emits resultReadyAt(int) only when finished
It doesn't matter if the call to the map function is done in a separate thread the same behaviour is encountered.
I also gave a try to the signal progressValueChanged(int) as the Qt progressDialog example suggests.
The signal progressValueChanged(int) gets emitted only for 2 lines in the image (the first and last).
This is really weird as in the Qt progress dialog example it is emitted smoothly.
I changed a bit the Qt example to launch the map function in another thread than the main thread and it still works well in that case.
The issue must arise from somewhere else.
Maybe the GUI event loop is doing something I don't expect ? I have no clue what.
I will now try QtConcurrent::mappedReduced and report with the results :-)
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
EDIT after giving a try to QtConcurrent::mappedReduced
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
It doesn't work and calls the "reduce" function ONLY when the "map" function is done. In other words it does the same than the previous signal/slots mechanism.
I'm running low in possibilities now
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
EDIT I'm back to a solution as close as the Qt progress dialog example
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Something must be wrong if I can't get the same behaviour than the Qt example.
Here's the code now:
//created in the main thread! (gui)
Worker::Worker(VideoEngine* engine):_engine(engine),_watcher(0){
_watcher = new QFutureWatcher<void>;
_watcher->setPendingResultsLimit(200);
connect(_watcher,SIGNAL(progressValueChanged(int)), _engine,
SLOT(onProgressUpdate(int)));
connect(_watcher, SIGNAL(finished()), engine, SLOT(engineLoop()));
}
//executed on the main thread
void Worker::computeTreeForFrame(...){
...
_watcher->setFuture(QtConcurrent::map(_sequence,boost::bind(metaEnginePerRow,_1,output)));
...
}
The call to computeTreeForFrame...
...
_worker->computeTreeForFrame();
...
This call is done in a slot .
It emits the signals for the line 0 and for the last line as told before but doesn't emits anything else.
Shouldn't this do EXACTLY what the Qt example does?
From task description it looks like you should use mappedReduced. Problem is that I don't see a good way to get partial results. One way to overcome this problem is to emit signal form reduce function.
It is possible that this thread may help.
It seems, that QtConcurrent::mapped does not put VideoEngine::metaEnginePerRow in another thread, judging by the documentation. If image is processed in the same thread as GUI, then your slots indeed will be executed after processing, no matter what type of connection you select, just as you've described.
The solution is to either run Worker::_computeTreeForFrame (as i understood, your main processing function) in another thread via QtConcurrent::run or to put your Worker object in another thread probably via QObject::moveToThread(). Then, the connection type you should use is Qt::QueuedConnection (or if you will put Worker in another thread before connection, you can connect even with Qt::AutoConnectionor Qt::UniqueConnection, caller and receiver will be in a different threads, so qt will automaticly chose QueuedConnection`)
EDIT:
I'm not sure, but your _watcher = new QFutureWatcher<bool>; is still created in the main thread and if you call
_watcher->setFuture(QtConcurrent::mapped(_sequence,
boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
_watcher->waitForFinished();
would _watcher set GUI thread to wait, in what it was created or thread, where this command is executed. If _watcher->setFuture(QtConcurrent::mapped(_sequence,
boost::bind(&VideoEngine::metaEnginePerRow,_1,output))); if the end of a function, is _watcher->waitForFinished(); needed at all? Qt will destroy thread right after it's execution and you set your processing function to run, why wait?
And _computeFrameWatcher should be of QFuture<void*> type.
EDIT2:
Ok, before i give up, i suggest you to test QObject::moveToThread:
before you call _worker->computeTreeForFrame(); , put it in another thread:
QThread *workerThread=new QThread();
_worker->moveToThread();
_worker->computeTreeForFrame();
/* connect _worker's finished signal with workerThread::quit and deleteLater slots */
and all connections within _worker should be DirectConnection and all connections between _worker and main (GUI) thread should be connected with QueuedConnection. Also it's probably good to create new thread in _worker constructor and move it to another thread immediately, this way you can destroy thread in _worker's destructor and don't worry about thread problem's in GUI thread