I am developing a 3D app using Qt and OpenGL. The app is composed of a QMainWindow with a QOpenGLWidget as central widget and a QML UI as a dock widget. I realized that the user inputs and the UI depends on the rendering performance: if my app runs with low fps, the user inputs are not all caught and it gets difficult to use the UI.
So I was thinking about doing the rendering in a separate thread. I tried several techniques, like using a QTimer or a QThread, but I always get problems sharing the OpenGL context, resizing or using a QPainter.
I am wondering if doing the rendering in another thread is a good approach.
Any suggestions, advices ?
Thanks.
Typical GUI frameworks are not designed to be used from multiple threads directly, and QT is not an exception from. Trying to do GUI stuff from different threads typically results in problems of some kind.
Those frameworks normally have an internal event queue where events are placed in and then processed one after another, which, if the framework is used correctly, assures that the GUI related stuff is accessed from one single thread only. But they allow to add additional events into the queue.
And here we are at the way to go: Keep the entire GUI in one single thread and do user input processing in the other thread. As soon as user data is processed, feed your GUI with appropriately.
Ways to do so offered by Qt are e. g. invoke function or the event system.
Just don't use QOpenGLWidget. Use a single QML window for everything.
Render your OpenGL things in pre-render or post-render function of the QML by using the QQuickWindow::beforeRendering() or QQuickWindow::afterRendering() signals.
That will be using the rendering thread of the QML, so you won't need to create it. And the use cases and synchronization are explained in the qt docs:
http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html
Related
I'm working on an OpenGL project where there are many scenes. I have successfully implemented the functionality of switching scenes at runtime, so the user can change to another scene by choosing a scene name through the ImGui menu. When that happens, the current scene is deleted to clean up dirty OpenGL internal states, then the new scene will be loaded from a factory pattern. Everything works fine, except that the window will freeze for a few seconds during the transition because unloading/loading scenes takes quite a while to finish.
What I want to do now is to create a loading screen, which will be displayed in between. The task of unloading/loading scenes is scheduled asynchronously using std::async and std::future, so that the caller is non-blocking and my loading screen can show up. However, since I'm creating the new scene in background in another thread, that thread cannot see the OpenGL context in the main thread, as a result, any glxxxx() calls would cause access violation so the new scene cannot be created.
I know that OpenGL is a global state machine which does not support multithreading quite well. I've also read somewhere that it is driver-dependent. Most threads on this topic is old, I wonder if it is still difficult to use multithreading in OpenGL as of 2021. From what I see, loading screen and switching scenes are just very basic functionalities, many applications are able to do so, and I believe there're a bunch of them using OpenGL, why is this problem still not commonly addressed today?
Does anyone know of any external libraries in this regard? Or is there another workaround without using multiple threads?
I am experimenting with Qt for a new layout for an instrument simulation program at work. Our current sim is running everything in a single window (we've used both glut (old) and fltk), it uses glViewport(...) and glScissor(...) to split instrument readouts into their own views, and then it uses some form of "ortho2D" calls to create their own virtual pixel space. The simulator currently updates the instruments and then draws each in their own viewport one by one, all in the same thread.
We want to find a better approach, and we settled on Qt. I am working under a few big constraints:
Each of the instrument panels still need to be in their OpenGL viewport. There are a lot of buttons and a lot of instruments. My tentative solution is to use a QOpenGLWidget for each. I have made progress on this.
The sim is not just a pretty readout, but also simulates many of the instruments as feedback for the instrument designers, so it sometimes has a hefty CPU load. It isn't a full hardware emulator, but it does simulate the logic. I don't think that it's feasible to tell the instruments to update themselves at the beginning of its associated widget's paintEvent(...) method, so I want simulation updates to run in a separate thread.
Our customers may have old computers and thus more recent versions of OpenGL have been ruled out. We are still using glBegin() and glEnd() and everything in between, and the instruments draw a crap ton of variable symbols, so drawing is takes a lot of time and I want to split drawing off into it's own thread. I don't yet know if OpenGL 3 is on the table, which will be necessary (I think) for rendering to off-screen buffers.
Problem: QOpenGLWidget does not have on overrideable "update" method, and it only draws during the widgets' paintEvent(...) and paintGL(...) calls.
Tentative Solution: Split the simulator into three threads:
GUI: Runs user input, paintEvent(...), and paintGL(...).
Simulator: Runs all instrument logic and updates values for symbology.
Drawing: Renders latest symbology to an offscreen buffer (will use a frame buffer object (FBO)).
In this design, cross-thread talking is cyclic and one-way, with the GUI thread providing input, the simulator thread taking that input into account on its next loop, the drawing thread reading the latest symbology and rendering it to the FBO and setting a "next frame available" flag to true (or maybe emitting a signal), and then the paintGL(...) method will take that FBO and spit it out to the widget, thus keeping event processing down and GUI responsiveness up. Continue this cycle.
Bottom line question: I've read here that GUI operations cannot be done in a separate thread, so is my approach even feasible?
If feasible, any other caution or suggestions would be appreciated.
Each OpenGL widget has its own OpenGL context, and these contexts are QObjects and thus can be moved to other threads. As with any otherwise non-threadsafe object, you should only access them from their thread().
Additionally - and this is also portable to QML - you could use worker functors to compute display lists that are then submitted to the render thread to be converted into draw calls. The render thread doesn't do any logic and doesn't compute anything: it takes data (vertex arrays, etc.) and submits it for drawing. The worker functors would be submitted for execution on a thread pool using QtConcurrent::run.
You can thus have a main thread, a render thread (perhaps one per widget, but not necessarily), and functors that run your simulation steps.
In any case, convoluting logic and rendering is a very bad idea. Whether you're doing drawing using QPainter on a raster widget, or using QPainter on an QOpenGLWidget, or using direct OpenGL calls, the thread that does the drawing should not have to compute what's to be drawn.
If you don't want to mess with OpenGL calls, and you can represent most of your work as array-based QPainter calls (e.g. drawRects, drawPolygons), these translate almost directly into OpenGL draw calls and the OpenGL backend will render them just as quickly as if you hand-coded the draw calls. QPainter does all this for you if you use it on a QOpenGLWidget!
I have a QTimer for executing OpenCV code and changing an image in a QLabel every 20 milliseconds, but I want to run this OpenCV code more naturally and not depend on the timer.
Instead, I want to have one main thread that deals with user input and another thread that process images with OpenCV, what I can't find is a thread safe way to change the QLabel image (pixmap) in one thread from another thread, could someone describe this process, maybe give some code examples? I also want to know the pros and cons of using QThread, since it's plataform free, it sounds to be user level thread and not a system level which usually runs smoother.
You can only instantiate and work with QPixmap on the main (GUI) thread of your application (e.g. what is returned by QApplication::instance()->thread())
That's not to say you can't work with a QPainter and graphics objects on other threads. Most things work, with exceptions guided by constraints imposed by the OS.
Successive versions of Qt have managed to find ways to support things that previously didn't work. For instance:
Qt 4.0 added rendering QImages from separate threads
Qt 4.4 added the ability to render text into images
Qt 4.8 added the ability to use QPainter in a separate thread to render to a QGLWidget, QGLPixelBuffer and QGLFrameBufferObject
Where Qt 4.4 introduced QFontDatabase::supportsThreadedFontRendering to check to see if font rendering was supported outside the GUI thread, in Qt5 this is considered obsolete and always returns true
Note: you shouldn't hold your breath for the day that Qt adds support to work with QPixmap on non-GUI threads. The reason they exist is to engage the graphics layer directly; and finding a way to work around it just so you could use something named QPixmap wouldn't do any good, as at that point you'd just be using something equivalent to what already exists as QBitmap.
So all that has to do with the ability to instantiate graphics objects like QFont and QImage on another thread, and use them in QPainter calls to generate a graphical image. But that image can't be tied directly to an active part of the display...you'll always be drawing into some off-screen buffer. It will have to be safely passed back to the main thread...which is the gateway that all the direct-to-widget drawing and mouse events and such must pass through.
A well known example of how one might do this is Qt's Mandelbrot Sample; and I'd suggest getting started with that... making sure you understand it completely.
Note: For a different spin on technique, you might be interested to look at my Thinker-Qt re-imagining of that same sample.
I'm writing a program in Qt, which runs 10 worker threads which calculate the trajectory of an object in space. They also have to draw the path of the object. I have a "Body" class deriving QGraphicsEllipseItem and it has a QPainterPath in it. The "Simulation" class takes a list of obstacles in the world, and the body to simulate and runs until the body collides with something. Simulation runs in a separate thread ( done with moveToThread, not by subclassing QThread). When the body collides, the Simulation emits a signal saying that it finished. When all threads have finished I'd like to draw the paths (I do it by invoking a method in "Body" which enables path drawing in its draw method).
Unfortunately I get ASSERT errors :
ASSERT: "!unindexedItems.contains(item)" in file graphicsview\qgraphicsscenebsptreeindex.cpp, line 364
They happen seemingly randomly. I've tried different connection types, to no result.
I'm starting the threads in a loop.
I'm using Qt 5.0
Generally speaking, with Qt you can't do any GUI operations outside of the GUI thread (i.e. the thread that is executing QApplication::exec(), which is typically the main() thread).
So if you have multiple threads manipulating QGraphicsItems (especially QGraphicsItems that are currently part of a QGraphicsScene), that is likely the cause of your assertion failures. That is, when the Qt GUI thread is doing its window refresh, it is reading data from the various QGraphicsItem objects as part of its calculations, and it expects the QGraphicsItems to remain constant for the duration of the refresh operation. If a QGraphicsItem is changed (by another thread) while the refresh routine is executing, then the calculations made by the main thread can become wrong/corrupted, and that occasionally causes an assertion failure (and/or other unwanted behaviors).
If you really need to use multiple threads, what you'll probably need to do is have the threads do all their calculations on their own private data structures that the Qt GUI thread has no access to. Then when the threads have computed their results, they should send the results back to the Qt GUI thread (via queued connection or QApplication::postEvent()). The GUI thread can then look at the results and use them to update the QGraphicsItems, etc; this will be "safe" because this update can't happen in the middle of a window update.
If that sounds like too much work, then you might consider just doing everything in the GUI thread; it will be much easier and simpler to make everything work reliably that way.
As mentioned by Jeremy, Qt rendering must be done on the main thread.
While you could move it all to the main thread, you've likely chosen to create separate ones for efficiency, especially as collision detection can be processor intensive. The best way to handle this is to split the modelling of the objects and their physics from their rendering, as you would in a Model / View / Controller pattern.
Create representations of the body instances that are not derived from any QGraphicsItem/Objects. These can then do their calculations on separate threads and have signals to graphics objects that are running in the main thread, which updates each body instance's graphic representation, allowing real-time rendering of the trajectories.
I am writing a program in OpenGL and I need some sort of interfacing toolbar. My initial reactions were to use a GUI, then further investigation into C++ I realized that GUI's are dependent on the OS you are using (I am on Windows). Therefore, I decided to use QT to help me.
My Question is if I am taking the best/appropriate approach to this solution. Am I even able to write my OpenGL program and have the GUI I want to create interface with the C++ code to do what I want it to do.
For example, If I create a simple "control panel" with arrows in each direction. And on screen I have a box object created by glut can I interface the arrows to be clicked on and interact with the openGL program to move the box?
Using Qt is coherent for your problem: it provides good integration of OpenGl through the QtOpenGL module.
Derive your display classes from QGLWidget (until Qt 4.8) or from QOpenGLWidget (since Qt 5.4) and implement virtual methods paintGL() etc.
You will have access to the Qt's signal and slot system so that you will be able to catch Gui events and update the OpenGl display.
You can use regular non-OpenGL Qt widgets on top of a QGLWidget so what you describe is do-able.
One thing I came across when doing this was that the regular widgets had to be opaque. The moment they were transparent in any way there was all sorts of garbage underneath them. That was a while ago so maybe the latest version addresses this issue. May have been platform-specific too so YMMV.
Stay in GLUT. There's no need to add Qt for a control panel. You could open a sub window (or second window, whichever works better for your program design), and draw the controls into that window, and use GLUT to handle the mouse interaction.
Furthermore, Qt and GLUT each have their own event loops. To use just Qt's event loop, you'd have to abandon much of the GLUT structure, since the GLUT event loop would not be there to call your callback functions. Qt does have the functionality to have let somebody else event loop call Qt's event processing code, but I don't think there's an easy way to make GLUT hand off the event information.