LibGDX texture blending with OpenGL blending function - opengl

In libGdx, i'm trying to create a shaped texture: Take a fully-visible rectangle texture and mask it to obtain a shaped textured, as shown here:
Here I test it on rectangle, but i will want to use it on any shape. I have looked into this tutorial and came with an idea to first draw the texture, and then the mask with blanding function:
batch.setBlendFunction(GL20.GL_ZERO, GL20.GL_SRC_ALPHA);
GL20.GL_ZERO - because i really don't want to paint any pixels from the mask
GL20.GL_SRC_ALPHA - from original texture i want to paint only those pixels, where mask was visible (= white).
Crucial part of the test code:
batch0.enableBlending();
batch0.begin();
batch0.draw(original, 0, 0); //to see the original
batch0.draw(mask, width1, 0); //and the mask
batch0.draw(original, 0, height1); //base for the result
batch0.setBlendFunction(GL20.GL_ZERO, GL20.GL_SRC_ALPHA);
batch0.draw(mask, 0, height1); //draw mask on result
batch0.setBlendFunction(GL20.GL_SRC_ALPHA, GL20.GL_ONE_MINUS_SRC_ALPHA);
batch0.end();
The center ot the texture get's selected well, but instead of transparent color around, i see black:
Why is the result blank and not transparent?
(Full code - Warning: very messy)

What you're trying to do looks like a pretty clever use of blending. But I believe the exact way you apply it is "broken by design". Let's walk through the steps:
You render your background with red and green squares.
You render an opaque texture on top of you background.
You erase parts of the texture you rendered in step 2 by applying a mask.
The problem is that for the parts you erase in step 3, the previous background is not coming back. It really can't, because you wiped it out in step 2. The background of the whole texture area was replaced in step 2, and once it's gone there's no way to bring it back.
Now the question is of course how you can fix this. There are two conventional approaches I can think of:
You can combine the texture and mask by rendering them into an off-sreen framebuffer object (FBO). You perform steps 1 and 2 as you do now, but render into an FBO with a texture attachment. The texture you rendered into is then a texture with alpha values that reflect your mask, and you can use this texture to render into your default framebuffer with standard blending.
You can use a stencil buffer. Masking out parts of rendering is a primary application of stencil buffers, and using stencil would definitely be a very good solution for your use case. I won't elaborate on the details of how exactly to apply stencil buffers to your case in this answer. You should be able to find plenty of examples both online and in books, including in other answers on this site, if you search for "OpenGL stencil". For example this recent question deals with doing something similar using a stencil buffer: OpenGL stencil (Clip Entity).
So those would be the standard solutions. But inspired by the idea in your attempt, I think it's actually possible to get this to work with just blending. The approach that I came up with uses a slightly different sequence and different blend functions. I haven't tried this out, but I think it should work:
You render the background as before.
Render the mask. To prevent it from wiping out the background, disable writing to the color components of the framebuffer, and only write to the alpha component. This leaves the mask in the alpha component of the framebuffer.
Render the texture, using the alpha component from the framebuffer (DST_ALPHA) for blending.
You will need a framebuffer with an alpha component for this to work. Make sure that you request alpha bits for your framebuffer when setting up your context/surface.
The code sequence would look like this:
// Draw background.
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE);
glDisable(GL_BLEND);
// Draw mask.
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glEnable(GL_BLEND);
glBlendFunc(GL_DST_ALPHA, GL_ONE_MINUS_DST_ALPHA);
// Draw texture.

A very late answer, but with the current version this is very easy. You simply draw the mask, set the blending mode to use the source color to the destination and draw the original. You'll only see the original image where the mask is.
//create batch with blending
SpriteBatch maskBatch = new SpriteBatch();
maskBatch.enableBlending();
maskBatch.begin();
//draw the mask
maskBatch.draw(mask);
//store original blending and set correct blending
int src = maskBatch.getBlendSrcFunc();
int dst = maskBatch.getBlendDstFunc();
maskBatch.setBlendFunction(GL20.GL_ZERO, GL20.GL_SRC_COLOR);
//draw original
maskBatch.draw(original);
//reset blending
maskBatch.setBlendFunction(src, dst);
//end batch
maskBatch.end();
If you want more info on the blending options, check How to do blending in LibGDX

Related

How do I make my object transparent but still show the texture?

I'm trying to render a model in OpenGL. I'm on Day 4 of C++ and OpenGL (Yes, I have learned this quickly) and I'm at a bit of a stop with textures.
I'm having a bit of trouble making my texture alpha work. In this image, I have this character from Spiral Knights. As you can see on the top of his head, there's those white portions.
I've got Blending enabled and my blend function set to glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
What I'm assuming here, and this is why I ask this question, is that the texture transparency is working, but the triangles behind the texture are still showing.
How do I make those triangles invisible but still show my texture?
Thanks.
There are two important things to be done when using blending:
You must sort primitives back to front and render in that order (order independent transparency in depth buffer based renderers is still an ongoing research topic).
When using textures to control the alpha channel you must either write a shader that somehow gets the texture's alpha values passed down to the resulting fragment color, or – if you're using the fixed function pipeline – you have to use GL_MODULATE texture env mode, or GL_DECAL with the primitive color alpha value set to 0, or use GL_REPLACE.

OpenGL blending: texture on top overlaps its pixels which should be transparent (alpha = 0)

I am drawing a map texture and, on top of it, a colorbar texture. Both have alpha channel and I am using blending, set as
// Turn on blending
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
However, the following happens:
The texture on top (colorbar) with alpha channel imposes its black pixels, which I don't want to happen. The map texture should appear behind where the colorbar alpha = 0.
Is this related to the blending definitions? How should I change it?
Assuming the texture has an alpha channel and it's transparent in the right places, I suspect the issue is with the rendering order and depth testing.
Lets say you render the scale texture first. It blends correctly with a black background. Then you render the orange texture behind it, but the pixels from the scale texture have a higher depth value and cause the orange texture there to be discarded.
So, make sure you render all your transparent stuff in back to front order, or farthest to nearest.
Without getting into order independent transparency, a common approach to alpha transparency is as follows:
Enable the depth buffer
Render all your opaque geometry
Disable depth writes (glDepthMask)
Enable alpha blending (as in the OP's code)
Render your transparent geometry in farthest to nearest order
For particles you can sometimes get away without sorting and it'll still look OK. Another approach is using the alpha test or using alpha to coverage with a multisample framebuffer.

Stencilling a render onto an unknown curved surface

Wanting to decal multiple irregular textures onto a curved surface (mesh with xyz vertices and uv specified at each). I am loading the mesh from a model file, and don't have any a priori knowledge of the surface... all we know is that it will have a "reasonable" uv mapping. Want to select a few uv regions and apply textures to them. Each region is specified by a bounding poly in uv coordinates. Don't know the equivalent xyz poly in this case, or I think the answer would be simple.
We have this working for flat surfaces and also simple cylindrical surfaces (which we approximate as a series of flat stripes, smoothed by choosing the normal as averages). In both cases we know a unique mapping from uv to xyz so we set up the stencil buffer to limit drawing to the desired uv region by drawing the equivalent xyz poly to the stencil buffer ahead of binding a texture and drawing the real surface.
We are also using rgba transparency within the textures when decaling those onto the surface. Typically each textured region is a small rotated rectangle so we draw the four vertices to the stencil buffer, then use the texture matrix to rotate that, and use the rgba transparency within the texture to ensure only the right part of the texture is applied. This all works nicely.
Would like to reuse our working code, but now apply these textures to an arbitrary curved surface/mesh. We are loading and drawing these models, and can already apply textures to whole faces [ie uv goes from (0,0) to (1,1) ]. Now we want to extend this and apply "placed" textures to regions of each surface.
Thought it might be possible draw the uv poly to the stencil buffer directly, not even knowing the equivalent xyz poly... then all the existing code would work. Perhaps could use some trick like a frame buffer object, and do the initial draw of the stencil poly to that, then using that as the stencil during the "real" draw of the curved surface mesh. Would that be a good approach? Or is there a better way?
Any advice or url links to relevant samples welcome...
PS Have looked at these threads... sort of relevant but not quite the same problem I think...
Binding a stencil render buffer to a frame buffer in opengl
Visualizing the Stencil Buffer to a texture
I am currently looking at some working FBO setup/usage code I have for off-screen shadow mapping, and trying to make it work for this seemingly simpler situation. The bit I'm unclear on is the setup gl calls needed ... I am rather confused about how to set this up. Here's an extract of the hardware shadowing FBO setup with bits chopped out and ?? added... any help on correct sequence here appreciated.
glBindTexture(GL_TEXTURE_2D, tex);
?? not
::glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT32, shadowsize, shadowsize, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_INT, NULL);
?? but a more normal binding approp to drawing RGBA textures
::glBindTexture(GL_TEXTURE_2D, 0);
glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, m_Framebuffer);
// Attach everything, tell fbo there will be a drawbuffer, unlike shadows tex draw
// ?? use GL_COLOR_ATTACHMENT0_EXT
glDrawBuffer(GL_NONE);
// no color buffer dest...
??wrong glReadBuffer(GL_NONE);
// no color buffer src
?? glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, GL_TEXTURE_2D, tex, 0);
//??
Note: tex, m_Frambuffer are ints, correctly allocated textureid and framebuffer, think that bit is ok. My main points of confusion are
Seems that code does glBindTexture, glTexImage2D, glBindTexture release to 0: is it correct to release this early?
glDrawBuffer + glReadBuffer calls required?

OpenGL - mask with multiple textures

I have implemented masking in OpenGL according to the following concept:
The mask is composed of black and white colors.
A foreground texture should only be visible in the white parts of the mask.
A background texture should only be visible in the black parts of the mask.
I can make the white part or the black part work as supposed by using glBlendFunc(), but not the two at the same time, because the foreground layer not only blends onto the mask, but also onto the background layer.
Is there anyone who knows how to accomplish this in the best way? I have been searching the net and read something about fragment shaders. Is this the way to go?
This should work:
glEnable(GL_BLEND);
// Use a simple blendfunc for drawing the background
glBlendFunc(GL_ONE, GL_ZERO);
// Draw entire background without masking
drawQuad(backgroundTexture);
// Next, we want a blendfunc that doesn't change the color of any pixels,
// but rather replaces the framebuffer alpha values with values based
// on the whiteness of the mask. In other words, if a pixel is white in the mask,
// then the corresponding framebuffer pixel's alpha will be set to 1.
glBlendFuncSeparate(GL_ZERO, GL_ONE, GL_SRC_COLOR, GL_ZERO);
// Now "draw" the mask (again, this doesn't produce a visible result, it just
// changes the alpha values in the framebuffer)
drawQuad(maskTexture);
// Finally, we want a blendfunc that makes the foreground visible only in
// areas with high alpha.
glBlendFunc(GL_DST_ALPHA, GL_ONE_MINUS_DST_ALPHA);
drawQuad(foregroundTexture);
This is fairly tricky, so tell me if anything is unclear.
Don't forget to request an alpha buffer when creating the GL context. Otherwise it's possible to get a context without an alpha buffer.
Edit: Here, I made an illustration.
Edit: Since writing this answer, I've learned that there are better ways to do this:
If you're limited to OpenGL's fixed-function pipeline, use texture environments
If you can use shaders, use a fragment shader.
The way described in this answer works and is not particularly worse in performance than these 2 better options, but is less elegant and less flexible.
Stefan Monov's is great answer! But for those who still have issues to get his answer working:
you need to check GLES20.glGetIntegerv(GLES20.GL_ALPHA_BITS, ib) - you need non zero result.
if you got 0 - goto EGLConfig and ensure that you pass alpha bits
EGL14.EGL_RED_SIZE, 8,
EGL14.EGL_GREEN_SIZE, 8,
EGL14.EGL_BLUE_SIZE, 8,
EGL14.EGL_ALPHA_SIZE, 8, <- i havn't this and spent a much of time
EGL14.EGL_DEPTH_SIZE, 16,

opengl - blending with previous contents of framebuffer

I am rendering to a texture through a framebuffer object, and when I draw transparent primitives, the primitives are blended properly with other primitives drawn in that single draw step, but they are not blended properly with the previous contents of the framebuffer.
Is there a way to properly blend the contents of the texture with the new data coming in?
EDIT: More information requsted, I will attempt to explain more clearly;
The blendmode I am using is GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA. (I believe that is typically the standard blendmode)
I am creating an application that tracks mouse movement. It draws lines connecting the previous mouse position to the current mouse position, and as I do not want to draw the lines over again each frame, I figured I would draw to a texture, never clear the texture and then just draw a rectangle with that texture on it to display it.
This all works fine, except that when I draw shapes with alpha less than 1 onto the texture, it does not blend properly with the texture's previous contents. Let's say I have some black lines with alpha = .6 drawn onto the texture. A couple draw cycles later, I then draw a black circle with alpha = .4 over those lines. The lines "underneath" the circle are completely overwritten. Although the circle is not flat black (It blends properly with the white background) there are no "darker lines" underneath the circle as you would expect.
If I draw the lines and the circle in the same frame however, they blend properly. My guess is that the texture just does not blend with it's previous contents. It's like it's only blending with the glclearcolor. (Which, in this case is <1.0f, 1.0f, 1.0f, 1.0f>)
I think there are two possible problems here.
Remember that all of the overlay lines are blended twice here. Once when they are blended into the FBO texture, and again when the FBO texture is blended over the scene.
So the first possibility is that you don't have blending enabled when drawing one line over another in the FBO overlay. When you draw into an RGBA surface with blending off, the current alpha is simply written directly into the FBO overlay's alpha channel. Then later when you blend the whole FBO texture over the scene, that alpha makes your lines translucent. So if you have blending against "the world" but not between overlay elements, it is possible that no blending is happening.
Another related problem: when you blend one line over another in "standard" blend mode (src alpha, 1 - src alpha) in the FBO, the alpha channel of the "blended" part is going to contain a blend of the alphas of the two overlay elements. This is probably not what you want.
For example, if you draw two 50% alpha lines over each other in the overlay, to get the equivalent effect when you blit the FBO, you need the FBO's alpha to be...75%. (That is, 1 - (1-.5) * (1-0.5), which is what would happen if you just drew two 50% alpha lines over your scene. But when you draw the two 50% lines, you'll get 50% alpha in the FBO (a blend of 50% with...50%.
This brings up the final issue: by pre-mixing the lines with each other before you blend them over the world, you are changing the draw order. Whereas you might have had:
blend(blend(blend(background color, model), first line), second line);
now you will have
blend(blend(first line, second line), blend(background color, model)).
In other words, pre-mixing the overlay lines into an FBO changes the order of blending and thus changes the final look in a way you may not want.
First, the simple way to get around this: don't use an FBO. I realize this is a "go redesign your app" kind of answer, but using an FBO is not the cheapest thing, and modern GL cards are very good at drawing lines. So one option would be: instead of blending lines into an FBO, write the line geometry into a vertex buffer object (VBO). Simply extend the VBO a little bit each time. If you are drawing less than, say, 40,000 lines at a time, this will almost certainly be as fast as what you were doing before.
(One tip if you go this route: use glBufferSubData to write the lines in, not glMapBuffer - mapping can be expensive and doesn't work on sub-ranges on many drivers...better to just let the driver copy the few new vertices.)
If that isn't an option (for example, if you draw a mix of shape types or use a mix of GL state, such that "remembering" what you did is a lot more complex than just accumulating vertices) then you may want to change how you draw into the VBO.
Basically what you'll need to do is enable separate blending; initialize the overlay to black + 0% alpha (0,0,0,0) and blend by "standard blending" the RGB but additive blending the alpha channels. This still isn't quite correct for the alpha channel but it's generally a lot closer - without this, over-drawn areas will be too transparent.
Then, when drawing the FBO, use "pre-multiplied" alpha, that is, (one, one-minus-src-alph).
Here's why that last step is needed: when you draw into the FBO, you have already multiplied every draw call by its alpha channel (if blending is on). Since you are drawing over black, a green (0,1,0,0.5) line is now dark green (0,0.5,0,0.5). If alpha is on and you blend normally again, the alpha is reapplied and you'l have 0,0.25,0,0.5.). By simply using the FBO color as is, you avoid the second alpha multiplication.
This is sometimes called "pre-multiplied" alpha because the alpha has already been multiplied into the RGB color. In this case you want it to get correct results, but in other cases, programmers use it for speed. (By pre-multiplying, it removes a mult per pixel when the blend op is performed.)
Hope that helps! Getting blending right when the layers are not mixed in order gets really tricky, and separate blend isn't available on old hardware, so simply drawing the lines every time may be the path of least misery.
Clear the FBO with transparent black (0, 0, 0, 0), draw into it back-to-front with
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
and draw the FBO with
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
to get the exact result.
As Ben Supnik wrote, the FBO contains colour already multiplied with the alpha channel, so instead of doing that again with GL_SRC_ALPHA, it is drawn with GL_ONE. The destination colour is attenuated normally with GL_ONE_MINUS_SRC_ALPHA.
The reason for blending the alpha channel in the buffer this way is different:
The formula to combine transparency is
resultTr = sTr * dTr
(I use s and d because of the parallel to OpenGL's source and destination, but as you can see the order doesn't matter.)
Written with opacities (alpha values) this becomes
1 - rA = (1 - sA) * (1 - dA)
<=> rA = 1 - (1 - sA) * (1 - dA)
= 1 - 1 + sA + dA - sA * dA
= sA + (1 - sA) * dA
which is the same as the blend function (source and destination factors) (GL_ONE, GL_ONE_MINUS_SRC_ALPHA) with the default blend equation GL_FUNC_ADD.
As an aside:
The above answers the specific problem from the question, but if you can easily choose the draw order it may in theory be better to draw premultiplied colour into the buffer front-to-back with
glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE);
and otherwise use the same method.
My reasoning behind this is that the graphics card may be able to skip shader execution for regions that are already solid. I haven't tested this though, so it may make no difference in practice.
As Ben Supnik said, the best way to do this is rendering the entire scene with separate blend functions for color and alpha. If you are using the classic non premultiplied blend function try glBlendFuncSeparateOES(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE) to render your scene to FBO. and glBlendFuncSeparateOES(GL_ONE, GL_ONE_MINUS_SRC_ALPHA) to render the FBO to screen.
It is not 100% accurate, but in most of the cases that will create no unexpected transparency.
Keep in mind that old Hardware and some mobile devices (mostly OpenGL ES 1.x devices, like the original iPhone and 3G) does not support separated blend functions. :(