Related
I'm trying to add an UI as a plugin to an existing application using OpenGL.
For this I'm rendering the UI into a texture and drawing that texture on top of the 3D scene after the scene was drawn.
The texture is generated as follows:
if (context_ == nullptr)
{
QSurfaceFormat format;
format.setDepthBufferSize( 16 );
format.setStencilBufferSize( 8 );
format.setMajorVersion(3);
format.setMinorVersion(3);
native_context_ = new QOpenGLContext;
native_context_->setNativeHandle( QVariant::fromValue(
QGLXNativeContext( native_context_information_->context, native_context_information_->display )));
if ( !native_context_->create())
{
ROS_ERROR( "OverlayManager: Fatal! Failed to create context!" );
}
context_ = new QOpenGLContext;
context_->setFormat( format );
context_->setShareContext( native_context_ );
if ( !context_->create())
{
ROS_ERROR( "OverlayManager: Fatal! Failed to create context!" );
}
surface_ = new QOffscreenSurface;
surface_->setFormat( format );
surface_->create();
if ( !surface_->isValid()) ROS_ERROR( "Surface invalid!" );
context_->makeCurrent( surface_ );
paint_device_ = new QOpenGLPaintDevice( 1920, 1080 );
{
QOpenGLFramebufferObjectFormat format;
format.setSamples(16);
format.setAttachment( QOpenGLFramebufferObject::CombinedDepthStencil );
fbo_ = new QOpenGLFramebufferObject( 1920, 1080, format );
texture_fbo_ = new QOpenGLFramebufferObject( 1920, 1080 );
}
fbo_->bind();
}
else
{
context_->makeCurrent( surface_ );
fbo_->bind();
}
context_->functions()->glClear(GL_COLOR_BUFFER_BIT);
QPainter painter(paint_device_);
painter.setRenderHint(QPainter::RenderHint::Antialiasing);
painter.setBrush(QBrush(Qt::green));
painter.drawRect(0, 0, 400, 300);
// painter.setPen(Qt::red);
painter.setPen(QPen(QBrush(Qt::red), 4));
painter.setFont(QFont("Arial", 20));
painter.drawText(100, 120, "Hello");
painter.drawText( 10, 80, QString( "Rendertime (ms): %1" ).arg( timer_average_ / 15.0 ));
painter.end();
fbo_->release();
QOpenGLFramebufferObject::blitFramebuffer(texture_fbo_, fbo_);
context_->functions()->glFinish();
// Texture looks fine
context_->doneCurrent();
glXMakeCurrent( native_context_information_->display, native_context_information_->drawable, native_context_information_->context );
// Now it is messed up
The probably more interesting part is the drawing of the texture:
glDisable(GL_DEPTH_TEST);
glDisable(GL_CULL_FACE);
glDisable(GL_TEXTURE_2D);
glDisable(GL_LIGHTING);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glOrtho(0, 1920, 0, 1080, -1, 1);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture_fbo_->texture());
glBegin(GL_QUADS);
glTexCoord2f(0, 0); glVertex2f(0, 0);
glTexCoord2f(0, 1); glVertex2f(0, 1080);
glTexCoord2f(1, 1); glVertex2f(1920, 1080);
glTexCoord2f(1, 0); glVertex2f(1920, 0);
// glClear(GL_DEPTH_BUFFER_BIT); // Wrong but not the issue, see second edit
glEnd();
glPopMatrix();
glDisable(GL_TEXTURE_2D);
glEnable(GL_LIGHTING);
glMatrixMode(GL_MODELVIEW);
I've mainly found this method of drawing textures together with some comments saying this is deprecated but couldn't really find much on full view overlaid textures using more recent methods. If you have any short resource / howto on that, I'd appreciate it but since this is far from my field of expertise I don't really want to invest more than a couple of hours into this just to avoid using deprecated code.
I hope you've got a somewhat clear picture of what I'm trying to achieve.
This actually works well on my desktop using an NVidia Geforce GTX 1080 using Ubuntu 16.04, Qt 5.5.1, OGRE 1.9.0 and OpenGL 4.6 (GLSL 4.6) and a VM on my desktop using Ubuntu 18.04, Qt 5.9.5, OGRE 1.9.0 and OpenGL 2.1 (GLSL 1.2).
However, on my notebook using Ubuntu 16.04, Qt 5.5.1, OGRE 1.9.0 and OpenGL 3 (GLSL 1.3) it doesn't work at all.
Now, the obvious question is: Why is that and how can I fix it?
That's how it looks like on my desktop and VM:
That's how it looks like on my notebook:
The entire source code can be found here.
Edit:
In case it's important, if I move the camera in the bottom example, the white areas also change. Because of that I think they might be leftovers from the scene rendering.
Second Edit:
I've done more debugging and it's not the drawing that's wrong as I initially thought but the context switch.
I've saved the texture to a file before the switch and after it, and the texture looks as intended before and is messed up after the switch.
Now, I just need to figure out why.
Calling glClear inside an glBegin…glEnd block is invalid. Also I'm not entirely sure, what your intention is with that. If you want to prevent it writing to the depth buffer, you'd use glDepthMask for that. Also disabling depth testing disables depth writes.
I've solved it with a workaround for now.
When initializing, I check whether context sharing works by creating a texture in one context, switching the context, reading the content back and comparing them.
If it doesn't, I grab the texture content after the rendering using glGetTexImage( GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixel_data_ );, switch the context and upload it again using glTexImage2D.
Not pretty but it works.
When I render my text using TTF_RenderUTF8_Blended I obtain a solid rectangle on the screen. The color depends on the one I choose, in my case the rectangle is red.
My question
What am I missing? It seems like I'm not getting the proper Alpha values from the surface generated with SDL_DisplayFormatAlpha(TTF_RenderUTF8_Blended( ... )), or am I? Does anyone recognize or know the problem?
Additionnal informations
If I use TTF_RenderUTF8_Solid or TTF_RenderUTF8_Shaded the text is drawn properly, but not blended of course.
I am also drawing other textures on the screen, so I draw the text last to ensure the blending will take into account the current surface.
Edit:SDL_Color g_textColor = {255, 0, 0, 0}; <-- I tried with and without the alpha value, but I get the same result.
I have tried to summarize the code without removing too much details. Variables prefixed with "g_" are global.
Init() function
// This function creates the required texture.
bool Init()
{
// ...
g_pFont = TTF_OpenFont("../arial.ttf", 12);
if(g_pFont == NULL)
return false;
// Write text to surface
g_pText = SDL_DisplayFormatAlpha(TTF_RenderUTF8_Blended(g_pFont, "My first Text!", g_textColor)); //< Doesn't work
// Note that Solid and Shaded Does work properly if I uncomment them.
//g_pText = SDL_DisplayFormatAlpha(TTF_RenderUTF8_Solid(g_pFont, "My first Text!", g_textColor));
//g_pText = SDL_DisplayFormatAlpha(TTF_RenderUTF8_Shaded(g_pFont, "My first Text!", g_textColor, g_bgColor));
if(g_pText == NULL)
return false;
// Prepare the texture for the font
GLenum textFormat;
if(g_pText->format->BytesPerPixel == 4)
{
// alpha
if(g_pText->format->Rmask == 0x000000ff)
textFormat = GL_RGBA;
else
textFormat = GL_BGRA_EXT;
}
// Create the font's texture
glGenTextures(1, &g_FontTextureId);
glBindTexture(GL_TEXTURE_2D, g_FontTextureId);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, g_pText->format->BytesPerPixel, g_pText->w, g_pText->h, 0, textFormat, GL_UNSIGNED_BYTE, g_pText->pixels);
// ...
}
DrawText() function
// this function is called each frame
void DrawText()
{
SDL_Rect sourceRect;
sourceRect.x = 0;
sourceRect.y = 0;
sourceRect.h = 10;
sourceRect.w = 173;
// DestRect is null so the rect is drawn at 0,0
SDL_BlitSurface(g_pText, &sourceRect, g_pSurfaceDisplay, NULL);
glBindTexture(GL_TEXTURE_2D, g_FontTextureId);
glEnable(GL_TEXTURE_2D);
glEnable(GL_BLEND);
glBegin( GL_QUADS );
glTexCoord2f(0.0f, 0.0f);
glVertex2f(0.0f, 0.0f);
glTexCoord2f(0.0f, 1.0f);
glVertex2f(0.0f, 10.0f);
glTexCoord2f(1.0f, 1.0f);
glVertex2f(173.0f, 10.0f);
glTexCoord2f(1.0f, 0.0f);
glVertex2f(173.0f, 0.0f);
glEnd();
glDisable(GL_BLEND);
glDisable(GL_TEXTURE_2D);
}
You've made a fairly common mistake. It's on the OpenGL end of things.
When you render the textured quad in DrawText(), you enable OpenGL's blending capability, but you never specify the blending function (i.e. how it should be blended)!
You need this code to enable regular alpha-blending in OpenGL:
glEnable( GL_BLEND );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
This info used to be on the OpenGL website, but I can't find it now.
That should stop it from coming out solid red. The reasons the others worked is because they're not alpha-blended, they're actually just red-on-black images with no alpha, so the blending function doesn't matter. But the blended one only contains red color, with an alpha channel to make it less-red.
I notice a few other small problems in your program though.
In the DrawText() function, you are blitting the surface using SDL and rendering with OpenGL. You should not use regular SDL blitting when using OpenGL; it doesn't work. So this line should not be there:
SDL_BlitSurface(g_pText, &sourceRect, g_pSurfaceDisplay, NULL);
Also, this line leaks memory:
g_pText = SDL_DisplayFormatAlpha( TTF_RenderUTF8_Blended(...) );
TTF_RenderUTF8_Blended() returns a pointer to SDL_Surface, which must be freed with SDL_FreeSurface(). Since you're passing it into SDL_DisplayFormatAlpha(), you lose track of it, and it never gets freed (hence the memory leak).
The good news is that you don't need SDL_DisplayFormatAlpha here because TTF_RenderUTF8_Blended returns a 32-bit surface with an alpha-channel anyway! So you can rewrite this line as:
g_pText = TTF_RenderUTF8_Blended(g_pFont, "My first Text!", g_textColor);
I am trying to figure out the best way to mask of sections of a texture when they ar drawn. My issue comes in the fact that I seem to have run our of alpha masks!
We are using openGL to draw a custom built 2D game engine. The game is built up off of sprites and simple block textures.
My desired outcome is like this:
A character sprite is drawn in place (using it's alpha color to not just be a box)
An item is drawn into the players hand (also using it's alpha color to draw into the scene without being a box)
The item should appear behind the characters arm/hand, but above the rest of the body.
For the moment the only way I can figure out how to accomplish this, is by drawing them in order (Body, Item, Arm) but I would like to avoid this to make art assets a bit easier to deal with. My idea solution would be to draw the character, then draw the item with an alpha mask that blocks out areas of the texture that should be "under" the arm.
Other solutions that I have seen are like this, where the glBlendFuncSeparate() function is used. I am trying to avoid bringing in extensions, as my current version of OpenGL doesn't support it. Not to say that I am opposed to the idea, but it seems a bit of a handle to brig it in just to draw an alpha mask?
I fully admit that this is a learning process for me, and I am using it as an excuse to really see how OpenGL handles. Any suggestions as to where I should head to get this to draw correctly? Is there a way for OpenGL in the fixed pipeline to take a texture, apply an alpha mask on top of it, and THEN draw it into the buffer? Should I give in and separate my character into several parts of its model?
[UPDATE: 8/12/12]
Tried to add the code suggested by Tim, but I seem to be having an issue. When I enable the stencil buffer, everything just gets blocked out, NOT just what I wanted. Here is my test example code.
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
// Disable writing to any of the color fields
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
glStencilFunc(GL_ALWAYS, 0,0);
// Draw our blocking poly
glBegin(GL_POLYGON);
glVertex2f( 50, 50 );
glVertex2f( 50, 50+128 );
glVertex2f( 50+128, 50+128 );
glEnd();
glStencilFunc(GL_GREATER, 0, -1);
glEnable(GL_STENCIL_TEST);
// Re enable drawing of colors
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
// Enable use of textures
glEnable(GL_TEXTURE_2D);
// Bind desired texture for drawing
glBindTexture(GL_TEXTURE_2D,(&texture)[0]);
// Draw the box with colors
glBegin(GL_QUADS);
glTexCoord2d( 0, 0 ); glVertex2f( 50, 50 );
glTexCoord2d( 0, 1 ); glVertex2f( 50, 50+128 );
glTexCoord2d( 1, 1 ); glVertex2f( 50+128, 50+128 );
glTexCoord2d( 1, 0 ); glVertex2f( 50+128, 50 );
glEnd();
// Swap buffers and display!
SDL_GL_SwapBuffers();
Just to be clear, here is my init code as well to set this system up.
When the code is run with stencil disabled, I get this:
When I use glEnable(GL_STENCIL_TEST), I get this:
I've tried playing around with various options, but I cannot see a clear reason why my stencil buffer is blocking everything.
[Update#2 8/12/12]
We got some working code, Thanks tim! Here is what I ended up running to work correctly.
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
// Disable writing to any of the color fields
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glStencilOp(GL_INCR, GL_INCR, GL_INCR);
glEnable(GL_STENCIL_TEST);
// Draw our blocking poly
glBegin(GL_POLYGON);
glVertex2f( 50, 50 );
glVertex2f( 50, 50+128 );
glVertex2f( 50+128, 50+128 );
glEnd();
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
// Re enable drawing of colors
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
// Enable use of textures
glEnable(GL_TEXTURE_2D);
// Bind desired texture for drawing
glBindTexture(GL_TEXTURE_2D,(&texture)[0]);
// Draw the box with colors
glBegin(GL_QUADS);
glTexCoord2d( 0, 0 ); glVertex2f( 50, 50 );
glTexCoord2d( 0, 1 ); glVertex2f( 50, 50+128 );
glTexCoord2d( 1, 1 ); glVertex2f( 50+128, 50+128 );
glTexCoord2d( 1, 0 ); glVertex2f( 50+128, 50 );
glEnd();
glDisable(GL_STENCIL_TEST);
// Swap buffers and display!
SDL_GL_SwapBuffers();
Here's my idea for the situation where you have one texture and one alpha mask:
Draw the character onto the scene like normal.
Lock the RGB color channels so that it cannot be changed with glColorMask
Setup the stencil buffer with glStencilOp(GL_KEEP, GL_KEEP, GL_INCR); glStencilFunc(GL_ALWAYS, 0,0);
Draw the alpha mask with alpha testing enabled. This will increment the stencil buffer anywhere the alpha test passes (you may have to flip this based on your mask polarity)
At this point, you have a character texture in the framebuffer, and a mask outline in the stencil buffer.
Reenable the color channels with glColorMask
Setup the stencil buffer for the weapon with glStencilFunc(GL_GREATER, 0, -1); This will only draw the weapon texels where the stencil buffer is greater than zero, and reject pixels where the stencil is not updated.
Draw the weapon texture as normal.
Tim was pretty clear in his comment, but I want to present you the solution I find the most intuitive. It's 3D, so hold on... ;)
Basically, you can just use the Z coordinate of your images to create virtual "layers". It then doesnt' matter, in which order you draw them. Just alphatest every image individually, and draw it on correct Z value. If it still isn't enough, you could use separate texture containing "depth" of every pixel, and then use the 2nd texture to perform some sort of depth-testing.
Be sure to call glEnable(GL_DEPTH_TEST); if you want to use this approach.
As I see it, the problem is that you have one texture, but part of it represents the arm and part of it the rest of the character. The issue is that you want to draw the weapon over the character, but draw the arm over both.
This means, while drawing two objects, you want to put them into three different "layers". This fundamentally doesn't make sense, so you're kind of stuck.
Here's an idea though: use a fragment program (i.e., a shader).
I suggest you overload the character's texture's alpha channel to encode both transparency and layer. For example, let's use 0=transparent body, 64=opaque body, 128=transparent arm, 255=opaque arm.
From here, you draw your objects, but conditionally set the depth of your objects into three layers. Basically, you write a fragment program that draws your character into two different layers, the character gets pushed backward while the arm gets pulled forward. When the weapon is drawn, it is drawn without a shader, but it's tested against the characters' pixels' depths. It works something like this (untested, obviously).
Define a shader my_shader, which contains a fragment program:
uniform sampler2D character_texture;
void main(void) {
vec4 sample = texture2D(character_texture,gl_TexCoord[0].st);
int type; //Figure out what type of character texel we're looking at
if (fabs(sample.a-0.00)<0.01) type = 0; //transparent body
else if (fabs(sample.a-0.25)<0.01) type = 1; //opaque body
else if (fabs(sample.a-0.50)<0.01) type = 2; //transparent arm
else if (fabs(sample.a-1.00)<0.01) type = 3; //opaque arm
//Don't draw transparent pixels.
if (type==0 || type==2) discard;
gl_FragColor = vec4(sample.rgb,1.0);
//Normally, you (can) write "gl_FragDepth = gl_FragCoord.z". This
//is how OpenGL will draw your weapon. However, for the character,
//we alter that so that the arm is closer and the body is farther.
//Move body farther
if (type==1) gl_FragDepth = gl_FragCoord.z * 1.1;
//Move arm closer
else if (type==3) gl_FragDepth = gl_FragCoord.z * 0.9;
}
Here's some pseudocode for your draw function:
//...
//Algorithm to draw your character
glUseProgram(my_shader);
glBindTexture(GL_TEXTURE_2D,character.texture.texture_gl_id);
glUniform1i(glGetUniformLocation(my_shader,"character_texture"),1);
character.draw();
glUseProgram(0);
//Draw your weapon
glEnable(GL_DEPTH_TEST);
character.weapon.draw();
glDisable(GL_DEPTH_TEST);
//...
I need to compute the average of two depth textures at each pixel. I expect I could do that with a GLSL fragment shader, but I'd prefer a solution that works on the dumbest possible hardware, so I tried blending. Here's the main code:
/// Initialize color buffer to black
glClearColor( 0.0f, 0.0f, 0.0f, 1.0f );
glClear( GL_COLOR_BUFFER_BIT );
// Turn off unnecessary operations
glDisable( GL_DEPTH_TEST );
glDisable( GL_LIGHTING );
glDisable( GL_CULL_FACE );
glDisable( GL_BLEND );
glDisable( GL_STENCIL_TEST );
glDisable( GL_DITHER );
// Set all matrices to identity
glMatrixMode( GL_TEXTURE );
glLoadIdentity();
glMatrixMode( GL_MODELVIEW );
glLoadIdentity();
glMatrixMode( GL_PROJECTION );
glLoadIdentity();
glTexEnvi( GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE );
RenderAQuadTexturedWith( texture1 );
glEnable( GL_BLEND );
glBlendEquation( GL_FUNC_ADD );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
glColor4f( 1.0f, 1.0f, 1.0f, 0.5f );
RenderAQuadTexturedWith( texture2 );
The problem is that the values I'm getting are off by around 0.002, as compared with what I get by getting the pixels of the two textures and computing the average on the CPU. When a set a breakpoint in OpenGL Profiler (this is on Mac OS X 10.6.8) and eyeball the color buffer, it looks about like what I'd expect. Is there some inherent inaccuracy in blend mode?
I also tried setting the current color to 0.5, 0.5, 0.5 and using glBlendFunc( GL_ONE, GL_ONE ), and the errors were in the opposite direction but about the same magnitude.
EDIT TO ADD: In retrospect, I see my mistake clearly: If I render into a render buffer with 8 bits per color component, and then read pixels from one of those components, then I only have 8 bits of accuracy.
So now I need to figure out a way to extract the results without losing accuracy. Maybe a fragment shader that sets gl_FragDepth?
The GPU does all calculations in float. Check that your CPU reference is also using float and not double.
As I added to my question, the root of the problem was not with depth textures or blend mode, but just that I was rendering into a buffer with 8 bits per color component. It probably would work if I went out of my way to get a higher-precision color buffer. But I went for the alternative of using a fragment program that puts the output in gl_FragDepth, and that seems to work as desired, since I already had a 24-bit depth buffer.
Why are those lines appearing in my shape?
I'm initializing OpenGL like this:
glDisable(GL_TEXTURE_2D);
glDisable(GL_DEPTH_TEST);
glDisable(GL_COLOR_MATERIAL);
glEnable(GL_BLEND);
glEnable(GL_POLYGON_SMOOTH);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glClearColor(0, 0, 0, 0);
And drawing the shape like this:
glClear(GL_COLOR_BUFFER_BIT);
glColor3f(1, 0, 0);
glBegin(GL_POLYGON);
glVertex2f(-5, -5); // bottom left
glVertex2f(5, -5); // bottom right...
glVertex2f(6, 0);
glVertex2f(5, 5);
glVertex2f(-5, 5);
glEnd();
Doesn't matter if it's clockwise or CCW.
I think disabling GL_POLYGON_SMOOTH would fix that, but you'd lose the antialiasing. FSAA would work as an alternative, but it'd be slower.
Edit: looking around, there are a lot of examples out there using glBlendFunc( GL_SRC_ALPHA_SATURATE, GL_ONE );
GL_POLYGON_SMOOTH is an antiquated and slow method of polygon anti-aliasing. It also results in the problem you see above.
Using the Multisample buffer extension is the best way to perform fast anti-aliasing on modern machines. Read more here.