I'd like to show my Qt Quick content on a virtual screen inside my OpenSceneGraph scene.
The approach I'm using right now is highly inefficient:
Render Qt Quick to the offscreen surface using FBO (FrameBufferObject)
Download pixels with QOpenGLFramebufferObject::toImage()
Upload pixels to the OSG
So it's GPU-CPU-GPU transfer. Source code
A proper solution should somehow utilize existing FBO and be able to transfer data solely inside the GPU.
There are two options exist:
Create FBO on the Qt side and use its texture on the OSG side
Create FBO on the OSG side and feed it to the Qt Quick renderer
The Qt part is OK. And I'm completely lost with the OSG. Could anyone provide me with some pointers?
Finally made it.
General idea:
Render QtQuick to texture using FBO - there are some examples over the internet available.
Use this texture inside OpenSceneGraph
The whole thing includes several tricks.
Context Sharing
To perform certain graphics operations, we must initialize OpenGL global state, also known as context.
When a texture is created, context stores it's id. Ids are not globally unique, so when another texture is created within another context, it may get the same id, but with different resource behind it.
If you just pass your texture's id to another renderer (operating within different context), expecting it to show your texture, you end up showing another texture or black screen or crash.
The remedy is context sharing, which effectively means sharing ids.
OpenSceneGraph and Qt abstractions are not compatible, so you need to tell OSG not to use its own context abstraction. This is done by calling setUpViewerAsEmbeddedInWindow
Code:
OsgWidget::OsgWidget(QWidget* parent, Qt::WindowFlags flags)
: QOpenGLWidget(parent, flags)
, m_osgViewer(new osgViewer::Viewer)
{
setFormat(defaultGraphicsSettings());
// ...
m_osgGraphicsContext = m_osgViewer->setUpViewerAsEmbeddedInWindow(x(), y(), width(), height());
}
// osg::ref_ptr<osgViewer::GraphicsWindowEmbedded> m_osgGraphicsContext;
From now on, the existing QOpenGLContext instance will be used as an OpenGL context for OSG rendering.
You will need to create another context for QtQuick rendering and set them shared:
void Widget::initializeGL()
{
QOpenGLContext* qmlGLContext = new QOpenGLContext(this);
// ...
qmlGLContext->setShareContext(context());
qmlGLContext->create();
}
Remember, there can be only one active context at a time.
Your scheme is:
0. create osg::Texture out of QOpenGLFrameBufferObject::texture()
1. make QtQuick context active
2. render QtQuick to texture
3. make primary (OSG) context active
4. render OSG
5. goto 1
Making of a proper osg::Texture
Since OSG and Qt API are incompatible you barely can link QOpenGLFrameBufferObject to osg::Texture2D as it is.
QOpenGLFrameBufferObject has QOpenGLFrameBufferObject::texture() method which returns opengl texture id, but osg::Texture manages all openGL stuff on its own.
Something like osg::Texture2D(uint textureId); could help us but it just doesn't exist.
Let's make one by ourselves.
osg::Texture is backed by osg::TextureObject which stores OpenGL texture id and some other data as well. If we construct osg::TextureObject with a given texture id and pass it to osg::Texture, the latter will use it as its own.
Code:
void Widget::createOsgTextureFromId(osg::Texture2D* texture, int textureId)
{
osg::Texture::TextureObject* textureObject = new osg::Texture::TextureObject(texture, textureId, GL_TEXTURE_2D);
textureObject->setAllocated();
osg::State* state = m_osgGraphicsContext->getState();
texture->setTextureObject(state->getContextID(), textureObject);
}
Complete demo project here
Related
I want to get the unity context into opengl so I can display a unity render texture in an opengl glfw window. I tried using
oldContext = glfwGetCurrentContext(); but the value of oldContext is just null.
I am trying to use the low-level native unity plugin and Texture.GetNativeTexturePtr
Any help would be greatly appreciated!
OpenGL context cannot be queried like OpenGL state related objects via some glGet* API.Context is not part of OpenGL API,it is a part of the system you're running on and it exists to allow you maintaining of OpenGL state and issue command to the driver. You must access a system specific handle that points to the context via system specific API.On Windows (WinGDI)that's would be
HGLRC wglGetCurrentContext();
On linux see related GLX API. You need to find functions to access GLXContext
I did it once in Unity3D (framebuffer readout plugin). But it used Unity's OpenGL or DirectX context to issue API commands only.
Also,I am not sure you can 'inject' or share a context for a window that doesn't own that context. You see, when you (or Unity) init display it creates context and related GL resources,like the default FBO with all required attachments on its own,and that FBO is mapped to some system resource(device) which takes actually care of presenting those pixels on the screen. Therefore, I am not sure display context can be moved from Window to Window in the same manner that a context can be shared between threads.(But I can be wrong on this one)
You can create your plugin Window on some thread,with its own GL context. Then create and share a texture object between those two. Remember, GL textures are shareable. If you copy contents from Unity's screen FBO into that texture,then you can copy it into your plugin's screen FBO from that texture as well.
Btw,look at this SO question .You can see there vendor specific GL extensions which allow copying data into texture from different contexts without requiring shared context,share lists setup.
Regarding why GLFW returns you nullptr. In your example you use GLFW library.
glfwGetCurrentContext()
But if you look at the source code,you see this:
GLFWAPI GLFWwindow* glfwGetCurrentContext(void)
{
_GLFW_REQUIRE_INIT_OR_RETURN(NULL);
return _glfwPlatformGetTls(&_glfw.contextSlot);
}
Which probably means that it retrieves a pointer to GLFWWindow from its own cache and not from the system.And if you didn't create that context via GLFW,you won't get any valid pointer. So try working directly with your system related API as explained above.
I want to display a scene with different points of view in many QOpenGLWidget that are in the same window (separated with a QSplitter architecture).
The important thing is that I want to store my scene data (geometry and textures) only once on the GPU.
But, each QOpenGLWidget have there own QOpenGLContext.
My idea was to create an independant context that render scene into a FrameBufferObject (not QFrameBufferObject) and use the resulting texture in the concerned QOpenGLWidget.
More technically, my approach was to create a new QOpenGLContext and a QOffscreenSurface.
But when I want to use the resulting QOpenGLFunctions given by my QOpenGLContext, it stop with a segmentation fault even if I've checked that my created QOpenGlContext is valid.
In a more general way, it's hard to me to understand the role of the QOpenGLContext, the surface (QOffscreenSurface in my case) and the makeCurrent function.
My request is to understand what is the good way to do this and why I get a segmentation fault.
I come back with an answer of my problem.
In fact I've used my first initialized QOpenGLWidget's context as global context to avoid creating a new one.
The tricky part is to make the global context current everytime it's needed (adding data on GPU or using the fbo).
The paintGL steps are simple :
- Make the global context current.
- Start fbo recording.
- Render the scene.
- Stop the fbo recording.
- Make the QOpenGLWidget context current.
- Render the fbo result on the screen.
I've checked my memory using gDEBugger and all my memory is in the global context (and store only once).
Objective:
To make some onscreen and offscreen rendering via Qt5 OpenGL framework, such that the resources can be easily shared between both rendering parts. Specifically,
the rendering work is done through the offscreen part (the framebuffer might be larger than the display screen);
the results of the offscreen rendering can be displayed in multiple onscreen parts (say, QOpenGLWidgets) under different settings, e.g. different sizes, for simplicity;
the results of the offscreen rendering can also be extracted from GPU and saved into a QImage or cv::Mat object;
the above tasks can be executed asynchronously (doing the second offscreen rendering, while displaying or extracting the first offscreen result).
Current solution:
Since I don't know how to share resources between both parts, the actual rendering work are done redundantly in both parts in my current solution:
The onscreen part:
A QMainWindow containing multiple QOpenGLWidget (subclass of QOpenGLWidget) objects;
The offscreen part:
A custom class involving members of QOffscreenSurface, QOpenGLContext, and QOpenGLFramebufferObject pointers, as well as a QOpenGLFunctions pointer to invoke OpenGL functions do the actual rendering work, much similar to this link.
The actual renderer:
As the reason above, the actual rendering work is extracted into a seperated class and both parts (onscreen and offscreen) have its handle.
Questions:
There are two QOpenGLContexts:
When doing the offscreen work in a background thread (for asynchronously rendering), it says the QWindow-based QOffscreenSurface are not allowed to exist outside the gui thread;
When doing this in the main (GUI) thread, it says the QOpenGLContext is invalid.
So my questions are:
Should I do the offscreen and onscreen work in the same GUI thread or not?
What is the best way of communicating and sharing resources between the offscreen and onscreen parts?
A brief actual code example doing a simple rendering work (say, draw a triangle via shading language) will be much appreciated.
Assuming that QOpenGLContext *main_ctx is the context that was created by QOpenGLWidget for actual rendering, you can create another context ctx in any thread and make it share textures and buffers with the first one:
ctx = std::make_unique<QOpenGLContext>();
ctx->setFormat(main_ctx->format());
ctx->setShareContext(main_ctx);
ctx->create();
I don't think that QOffscreenSurface must be a QWindow-based.
offscreen_surface = std::make_unique<QOffscreenSurface>();
offscreen_surface->setFormat(ctx->format());
offscreen_surface->create();
ctx->makeCurrent(offscreen_surface);
Then create a QOpenGLFramebufferObject and render into it from the second context (second thread).
Then use its texture in the main context: glBindTexture(GL_TEXTURE_2D, fbo->texture());. Maybe there is a need for some synchronization when doing this.
I want to create opengl 2d library, where textures as well as windows are encapsulated as objects. Is it possible to create dummy static DC and make it current when loading textures? All of the windows would have same PIXELFORMATDESCRIPTOR as the static one. This way, users of the library would not have to create window prior to loading textures or passing windows as parameters to textures.
Is it possible to create dummy static DC and make it current when loading textures?
Sort of. As long as the visual formats of the device contexts are compatible with each other, you can bind a OpenGL render context created for this visual format to any of these device contexts.
So you can perfectly fine create a window, with a DC that's never shown on the screen (always kept hidden, size of 0×0) and use that for background OpenGL operations. You can also create a secondary OpenGL context, have it share its namespace with the primary context, make it current on the hidden window on a separate worker thread, so that you can asynchronously perform OpenGL operations (like loading textures) while the main context is used for other things.
I am currently porting from Qt4 to Qt5. While Qt5 still provides the QGLWidget etc. classes, these are to be replaced by QOpenGLWidget etc.
Currently I am drawing complicated stuff into a framebuffer object once. Then I draw it on the screen (into the GL widget) using drawTexture() method:
target->drawTexture(rect, fb->texture());
Afterwards, I overdraw with other temporary things. So whenever I need to update the temporary stuff, I can just re-use the framebuffer object instead of redrawing the complicated stuff.
With QOpenGLWidget, as well as QOpenGLContext, however, there is no convenient drawTexture() method anymore. Drawing the texture by-hand with OpenGL would be a minimum of 20 lines of complicated stuff. Textbook stuff, I admit.
Is there any elegant/Qt way of getting my fbo contents onto the screen? For example, it is documented that QOpenGLWidget also uses an fbo as a backend. Instead of drawing the texture, we could also blit the FBOs. But how?