glDrawElements crashes when not using certain vertex attributes - c++

In my OpenGL program I have two shaders. One renders with textures, and the other renders just solid colors. After compiling and linking a shader, I enable a texture coordinate vertex attribute array depending on weather or not the shader contains the attribute.
//This code is called after the shaders are compiled.
//Get the handles
textureU = glGetUniformLocation(program,"texture");
tintU = glGetUniformLocation(program,"tint");
viewMatrixU = glGetUniformLocation(program,"viewMatrix");
transformMatrixU = glGetUniformLocation(program,"transformMatrix");
positionA = glGetAttribLocation(program,"position");
texcoordA = glGetAttribLocation(program,"texcoord");
//Detect if this shader can handle textures
if(texcoordA < 0 || textureU < 0) hasTexture = false;
else hasTexture = true;
//Enable Attributes
glEnableVertexAttribArray(positionA);
if(hasTexture) glEnableVertexAttribArray(texcoordA);
If I am rendering an item that is textured, each element in verts consists of 5 values (x,y,z,tx,ty), but if the item isn't textures, each element in verts contains only 3 values (x,y,z).
Here is the problem: When the first item rendered in the GL context does not have a texture, glDrawElements segfaults! However, if the first item rendered does have a texture, it works fine, and any untextured items after the textured one work fine (that is, until a new context is created).
This chunk of code renders an item
glBindBuffer(GL_ARRAY_BUFFER,engine->vertBuffer);
glBufferData(GL_ARRAY_BUFFER,sizeof(GLfloat)*item->verts.size(),&item->verts[0],GL_DYNAMIC_DRAW);
item->shader->SetShader();
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER,engine->elementBuffer);
glBufferData(GL_ELEMENT_ARRAY_BUFFER,sizeof(GLuint) * item->indicies.size(),&item->indicies[0],GL_DYNAMIC_DRAW);
if(item->usingTexture)
item->shader->SetTexture(item->texture->handle);
glUniformMatrix4fv(item->shader->transformMatrixU,1,GL_TRUE,&item->matrix.contents[0]);
glUniformMatrix4fv(item->shader->viewMatrixU,1,GL_TRUE,&item->batch->matrix.contents[0]);
glUniform4f(item->shader->tintU,item->color.x,item->color.y,item->color.z,item->color.w);
glDrawElements(GL_TRIANGLES,item->indicies.size(),GL_UNSIGNED_INT,0); //segfault
Here is the function seen above that sets the shader.
glUseProgram(program); currentShader = program;
GLsizei stride = 12;
if(hasTexture) stride = 20;
glVertexAttribPointer(positionA,3,GL_FLOAT,GL_FALSE,stride,0);
if(hasTexture)
glVertexAttribPointer(texcoordA,2,GL_FLOAT,GL_FALSE,stride,(void*)12);
As far as I know, this problem is not apparent on Intel Integrated Graphics, which seem to be quite lenient.
Edit: If it is useful to know, I am using GLFW and GLEW.

Try adding corresponding glDisableVertexAttribArray()s after your draw call:
glEnableVertexAttribArray(positionA);
if(hasTexture) glEnableVertexAttribArray(texcoordA);
// draw
glDisableVertexAttribArray(positionA);
if(hasTexture) glDisableVertexAttribArray(texcoordA);

The problem is that I enabled the vertex attribute arrays after compiling my shader. This is not where I should have been enabling them.
I enabled the texcoord attribute when compiling the shader, but since the first item didn't use it, glDrawElements would segfault.
I fixed this by enabling the attribute when setting the shader.

Related

D3D11: Rendering (depth) to texture results in red square, normal rendering works

I'm currently working on a D3D project and want to implement directional shadow mapping. I set everything up according to the Microsoft Guide, but it just doesn't work.
I've created a 2D texture object, a depth stencil view and a shader resource view and set them up using the following descriptions:
D3D11_TEXTURE2D_DESC shadowMapDesc;
ZeroMemory(&shadowMapDesc, sizeof(D3D11_TEXTURE2D_DESC));
shadowMapDesc.Width = width;
shadowMapDesc.Height = height;
shadowMapDesc.MipLevels = 1;
shadowMapDesc.ArraySize = 1;
shadowMapDesc.Format = DXGI_FORMAT_R24G8_TYPELESS;
shadowMapDesc.SampleDesc.Count = 1;
shadowMapDesc.SampleDesc.Quality = 0;
shadowMapDesc.Usage = D3D11_USAGE_DEFAULT;
shadowMapDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL | D3D11_BIND_SHADER_RESOURCE;
shadowMapDesc.CPUAccessFlags = 0;
shadowMapDesc.MiscFlags = 0;
ID3D11Device& d3ddev = dev.getD3DDevice();
uint32_t *initData = new uint32_t[width * height];
ZeroMemory(initData, sizeof(uint32_t) * width * height);
D3D11_SUBRESOURCE_DATA data;
ZeroMemory(&data, sizeof(D3D11_SUBRESOURCE_DATA));
data.pSysMem = initData;
data.SysMemPitch = sizeof(uint32_t) * width;
data.SysMemSlicePitch = 0;
HRESULT hr = d3ddev.CreateTexture2D(&shadowMapDesc, &data, &texture_);
D3D11_DEPTH_STENCIL_VIEW_DESC depthStencilViewDesc;
ZeroMemory(&depthStencilViewDesc, sizeof(D3D11_DEPTH_STENCIL_VIEW_DESC));
depthStencilViewDesc.Format = DXGI_FORMAT_D24_UNORM_S8_UINT;
depthStencilViewDesc.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
depthStencilViewDesc.Texture2D.MipSlice = 0;
hr = d3ddev.CreateDepthStencilView(texture_, &depthStencilViewDesc, &stencilView_);
D3D11_SHADER_RESOURCE_VIEW_DESC shaderResourceViewDesc;
ZeroMemory(&shaderResourceViewDesc, sizeof(D3D11_SHADER_RESOURCE_VIEW_DESC));
shaderResourceViewDesc.Format = DXGI_FORMAT_R24_UNORM_X8_TYPELESS;
shaderResourceViewDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
shaderResourceViewDesc.Texture2D.MipLevels = 1;
shaderResourceViewDesc.Texture2D.MostDetailedMip = 0;
hr = d3ddev.CreateShaderResourceView(texture_, &shaderResourceViewDesc, &shaderView_);
Between these steps there is additional error checking, but all the create-functions return successfully. I then bind the texture, render my scene and unbind the texture using the following functions:
void D3DDepthTexture2D::bindAsTarget(D3DDevice& dev)
{
dev.getDeviceContext().ClearDepthStencilView(stencilView_, D3D11_CLEAR_DEPTH | D3D11_CLEAR_STENCIL, 1.0f, 0);
// Bind target
dev.getDeviceContext().OMSetRenderTargets(0, 0, stencilView_);
// Set viewport
dev.setViewport(static_cast<float>(width_), static_cast<float>(height_), 0.0f, 0.0f);
}
void D3DDepthTexture2D::unbindAsTarget(D3DDevice& dev, float width, float height)
{
// Unbind target
dev.resetRenderTarget();
// Reset viewport
dev.setViewport(width, height, 0.0f, 0.0f);
}
My render-to-depth-texture routine basically looks like this (removing all the unnecessary details):
camera = buildCameraFromLight(light);
setCameraCBuffer(camera);
bindTexture();
activateShader();
for(Object j : objects) {setTransformationCBuffer(j); renderObject(j);}
deactivateShader();
unbindTexture();
Rendering the scene from the light's perspective to the normal render target (screen) results in the proper image (both the actual image and just rendering the depth values). I use a simple vertex shader that just transforms the vertices and a pixel shader that does nothing at all OR returns the depth values (I tried both, doesn't change anything about the end result since we don't care about the color buffer).
After clearing the texture and rendering to it, I render it onto a quad to my screen, but all I get is a red square - so the depth value is 1.0f, the value I've cleared the texture to. I'm really at a loss for what to do, I tried everything, implemented every possible solution from online tutorials or changed things around on my own, but nothing helps. Here's a list of all the things I already checked:
All FAILED(hr)-calls return false, no error message is printed to the console
I tested whether the geometry gets transformed properly by rendering the geometry and their depth values (z / w) to screen, which worked and looked correct
I tested calculating the depth values in the fragment shader and rendering to a normal render target (basically trying to render my color buffer to texture) instead of a depth stencil texture, but that didn't work either, red square
I tested different formats and format combinations for the shadow map and the views, which either caused the creation to fail or didn't change a thing
I checked whether any call between setting and unsetting my texture as the render target during the render call resetted the depth stencil target to something else - not the case
I debugged my texture-to-screen/quad rendering routine already and it works properly with other textures, so I am in fact seeing what the depth texture looks like
I changed the geometry and camera perspective around to see whether that makes anything visible in the depth texture - it doesn't
I came across this similar StackOverflow problem and checked whether my default depth stencil buffer had the same dimensions, AA settings etc. as my texture - and it does (count 1, quality 0)
I really don't know what's up, I've been trying to debug this for hours and hours. I hope someone here can give me any advice on what I'm doing wrong or what I could try to fix this. I'm using C++11 with Direct3D11.
Note: I can't debug any of this using NSight or any Visual Studio tools since they don't seem to work properly with my system right now and I don't have any administrative rights to fix any of it. I just have to deal with it for now. I hope the given information and code samples are enough to provide a rough idea of what I could also try to make this work.
Thanks in advance.
I got NSight to work and debugged the whole thing with that. Turns out the depth texture was properly created and filled with the depth and stencil data and I just forgot that all the depth information is stored in the first channel - so I ignored the g and b data and used 1.0 for a and it worked. Using the g and b channels somehow made the whole thing red (maybe someone wants to add to this and explain why).
Debugging this got much easier once I could observe the texture that is present in the shader - I should've used a debugging tool like NSight or RenderDoc way earlier. Thanks to #EgorShkorov for the advice.

HLSL multi-pass geometry shader culling issue

I'm working on a shader that runs two passes:
A standard vertex / pixel shader that draws the original object with some alpha blending
A geometry shader that subdivides the mesh and makes it crumble
My issue however is that the geometry of the first pass occludes the geometry of the second pass. Even when the original geometry is fully transparent, it completely culls whatever geometry from the second pass is behind it.
The original geometry is a sphere, which as you can see here occludes the result of the geometry shader
The first pass uses this blending mode:
BlendState SrcAlphaBlendingAdd
{
BlendEnable[0] = TRUE;
SrcBlend = SRC_ALPHA;
DestBlend = INV_SRC_ALPHA;
BlendOp = ADD;
SrcBlendAlpha = ZERO;
DestBlendAlpha = ZERO;
BlendOpAlpha = ADD;
RenderTargetWriteMask[0] = 0x0F;
};
both passes currently use this depthstencilstate:
DepthStencilState Depth {
// Depth test parameters
DepthEnable = true;
DepthWriteMask = all;
DepthFunc = less;
StencilEnable = false;
};
Cullmode is set to NONE for both passes.
Is there a way for both passes to use depthtest, without the first pass occluding the second?
If need be I can separate these passes into two shaders and just render the same object with both shaders in engine, but I would like to figure out if it's possible just using multiple passes.

How pick geometries in OpenGL with multisample framebuffer?

(Edit) I made working geometry picking with framebuffer. My goal is draw huge scene in one draw call, but I need to draw to multisample color texture attachment (GL_COLOR_ATTACHMENT0) and draw to (eddited) non-multisample picking texture attachment (GL_COLOR_ATTACHMENT1). The problem is if I use multisample texture to pick, picking is corrupted because of multi-sampling.
I write geometry ID to fragment shader like this:
//...
// Given geometry id
uniform int in_object_id;
// Drawed to screen (GL_COLOR_ATTACHMENT0)
out vec4 out_frag_color0;
// Drawed to pick texture (GL_COLOR_ATTACHMENT1)
out vec4 out_frag_color1;
// ...
void main() {
out_frag_color0 = ...; // Calculating lighting and other stuff
//...
const int max_byte1 = 256;
const int max_byte2 = 65536;
const float fmax_byte = 255.0;
int a1 = in_object_id % max_byte1;
int a2 = (in_object_id / max_byte1) % max_byte1;
int a3 = (in_object_id / max_byte2) % max_byte1;
//out_frag_color0 = vec4(a3 / fmax_byte, a2 / fmax_byte, a1 / fmax_byte, 1);
out_frag_color1 = vec4(a3 / fmax_byte, a2 / fmax_byte, a1 / fmax_byte, 1);
}
(Point of that code is use RGB space for store geometry ID which is then read back a using for changing color of cube)
This happens when I move cursor by one pixel to left:
Because of alpha value of cube pixel:
Without multisample is works well. But multisampling multiplies my output color and geometry id is then corrupted, so it selects random cube with multiplied value.
(Edit) I can't attach one multisample texture target to color0 and non-multisample texture target to color1, it's not supported. How can I do this in one draw call?
Multisampling is not my friend I am not sure If I understand it well (whole framebuffering). Anyway, this way to pick geometries looks horrible for me (I meant calculating ID to color). Am I doing it well? How can I solve multisample problem? Is there better way?
PS: Sorry for low english. :)
Thanks.
You can't do multisampled and non-multisampled rendering in a single draw call.
As you already found, using two color targets in an FBO, with only one of them being multisampled, is not supported. From the "Framebuffer Completeness" section in the spec:
The value of RENDERBUFFER_SAMPLES is the same for all attached renderbuffers; the value of TEXTURE_SAMPLES is the same for all attached textures; and, if the attached images are a mix of renderbuffers and textures, the value of RENDERBUFFER_SAMPLES matches the value of TEXTURE_SAMPLES.
You also can't render to multiple framebuffers at the same time. There is always one single current framebuffer.
The only reasonable option I can think of is to do picking in a separate pass. Then you can easily switch the framebuffer/attachment to a non-multisampled renderbuffer, and avoid all these issues.
Using a separate pass for picking seems cleaner to me anyway. This also allows you to use a specialized shader for each case, instead of always producing two outputs even if one of them is mostly unused.
I think it is posible...
You have to set the picking texture to multisampled and after rendering the scene, you can render 2 triangles over the screen and inside another fragmentshader you can readout each sample... to do that you have to use the GLSL command:
texelFetch(sampler, pixelposition/*[0-texturesize]*/, /*important*/layernumber);
Then you can render it into a single-sampled texture and read the color via glReadPixel.
I haven't tested it now, but I think it works

Driver error when using multiple shaders

I'm using 3 different shaders:
a tessellation shader to use the tessellation feature of DirectX11 :)
a regular shader to show how it would look without tessellation
and a text shader to display debug-info such as FPS, model count etc.
All of these shaders are initialized at the beginning.
Using the keyboard, I can switch between the tessellation shader and regular shader to render the scene. Additionally, I also want to be able toggle the display of debug-info using the text shader.
Since implementing the tessellation shader the text shader doesn't work anymore. When I activate the DebugText (rendered using the text-shader) my screens go black for a while, and Windows displays the following message:
Display Driver stopped responding and has recovered
This happens with either of the two shaders used to render the scene.
Additionally:
I can start the application using the regular shader to render the scene and then switch to the tessellation shader. If I try to switch back to the regular shader I get the same error as with the text shader.
What am I doing wrong when switching between shaders?
What am I doing wrong when displaying text at the same time?
What file can I post to help you help me? :) thx
P.S. I already checked if my keyinputs interrupt at the wrong time (during render or so..), but that seems to be ok
Testing Procedure
Regular Shader without text shader
Add text shader to Regular Shader by keyinput (works now, I built the text shader back to only vertex and pixel shader) (somthing with the z buffer is stil wrong...)
Remove text shader, then change shader to Tessellation Shader by key input
Then if I add the Text Shader or switch back to the Regular Shader
Switching/Render Shader
Here the code snipet from the Renderer.cpp where I choose the Shader according to the boolean "m_useTessellationShader":
if(m_useTessellationShader)
{
// Render the model using the tesselation shader
ecResult = m_ShaderManager->renderTessellationShader(m_D3D->getDeviceContext(), meshes[lod_level]->getIndexCount(),
worldMatrix, viewMatrix, projectionMatrix, textures, texturecount,
m_Light->getDirection(), m_Light->getAmbientColor(), m_Light->getDiffuseColor(),
(D3DXVECTOR3)m_Camera->getPosition(), TESSELLATION_AMOUNT);
} else {
// todo: loaded model depends on distance to camera
// Render the model using the light shader.
ecResult = m_ShaderManager->renderShader(m_D3D->getDeviceContext(),
meshes[lod_level]->getIndexCount(), lod_level, textures, texturecount,
m_Light->getDirection(), m_Light->getAmbientColor(), m_Light->getDiffuseColor(),
worldMatrix, viewMatrix, projectionMatrix);
}
And here the code snipet from the Mesh.cpp where I choose the Typology according to the boolean "useTessellationShader":
// RenderBuffers is called from the Render function. The purpose of this function is to set the vertex buffer and index buffer as active on the input assembler in the GPU. Once the GPU has an active vertex buffer it can then use the shader to render that buffer.
void Mesh::renderBuffers(ID3D11DeviceContext* deviceContext, bool useTessellationShader)
{
unsigned int stride;
unsigned int offset;
// Set vertex buffer stride and offset.
stride = sizeof(VertexType);
offset = 0;
// Set the vertex buffer to active in the input assembler so it can be rendered.
deviceContext->IASetVertexBuffers(0, 1, &m_vertexBuffer, &stride, &offset);
// Set the index buffer to active in the input assembler so it can be rendered.
deviceContext->IASetIndexBuffer(m_indexBuffer, DXGI_FORMAT_R32_UINT, 0);
// Check which Shader is used to set the appropriate Topology
// Set the type of primitive that should be rendered from this vertex buffer, in this case triangles.
if(useTessellationShader)
{
deviceContext->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_3_CONTROL_POINT_PATCHLIST);
}else{
deviceContext->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
}
return;
}
RenderShader
Could there be a problem using sometimes only vertex and pixel shader and after switching using vertex, hull, domain and pixel shader?
Here a little overview of my architecture:
TextClass: uses font.vs and font.ps
deviceContext->VSSetShader(m_vertexShader, NULL, 0);
deviceContext->PSSetShader(m_pixelShader, NULL, 0);
deviceContext->PSSetSamplers(0, 1, &m_sampleState);
RegularShader: uses vertex.vs and pixel.ps
deviceContext->VSSetShader(m_vertexShader, NULL, 0);
deviceContext->PSSetShader(m_pixelShader, NULL, 0);
deviceContext->PSSetSamplers(0, 1, &m_sampleState);
TessellationShader: uses tessellation.vs, tessellation.hs, tessellation.ds, tessellation.ps
deviceContext->VSSetShader(m_vertexShader, NULL, 0);
deviceContext->HSSetShader(m_hullShader, NULL, 0);
deviceContext->DSSetShader(m_domainShader, NULL, 0);
deviceContext->PSSetShader(m_pixelShader, NULL, 0);
deviceContext->PSSetSamplers(0, 1, &m_sampleState);
ClearState
I'd like to switch between 2 shaders and it seems they have different context parameters, right? In clearstate methode it says it resets following params to NULL:
I found following in my Direct3D Class:
depth-stencil state -> m_deviceContext->OMSetDepthStencilState
rasterizer state -> m_deviceContext->RSSetState(m_rasterState);
blend state -> m_device->CreateBlendState
viewports -> m_deviceContext->RSSetViewports(1, &viewport);
I found following in every Shader Class:
input/output resource slots -> deviceContext->PSSetShaderResources
shaders -> deviceContext->VSSetShader to - deviceContext->PSSetShader
input layouts -> device->CreateInputLayout
sampler state -> device->CreateSamplerState
These two I didn't understand, where can I find them?
predications -> ?
scissor rectangles -> ?
Do I need to store them all localy so I can switch between them, because it doesn't feel right to reinitialize the Direct3d and the Shaders by every switch (key input)?!
Have you checked if the device is being reset by the system. Check the return variable of the Present() Method. When switching shaders abruptly DX tends to reset the device for some odd reason.
If this is the problem, just recreate the device and context and you should be good.
Right now you have
void Direct3D::endScene()
{
// Present the back buffer to the screen since rendering is complete.
if(m_vsync_enabled)
{
// Lock to screen refresh rate.
m_swapChain->Present(1, 0);
}
else
{
// Present as fast as possible.
m_swapChain->Present(0, 0);
}
return;
}
I would suggest doing something like so to catch the return value of Present()
ULONG Direct3D::endScene()
{
int synch = 0;
if(m_vsync_enabled)
synch = 1;
// Present as fast as possible or synch it to 1 vertical blank
return m_swapChain->Present(synch, 0);
}
Of course this is only MY way of doing it, and there are many. Also, I forgot to tell you that the issue I had in the past was also using the Effects library. Have you recompiled it in your system? If not, then do so. Or even better get rid of it, that's what I did when I solved my problem.

GLSL geometry shader: iterate over entire mesh

My goal was to color the vertexes according to their order
EDIT: long time goal: access to preceding and following vertexes to simulate gravity behavior
i've used following code
#version 120
#extension GL_EXT_geometry_shader4 : enable
void main( void ) {
for( int i = 0 ; i < gl_VerticesIn ; i++ ) {
gl_FrontColor = vec4(float(i)/float(gl_VerticesIn),0.0,0.0,1.0);
gl_Position = gl_PositionIn[i];
EmitVertex();
}
}
but all vertexes are drawn black, it seem that i is always evaluated as 0, am i missing something or doing it wrong?
EDIT: figured the meta-problem out: how to feed all me model geometry into single geometry shader call, so the mainloop iterates over all the vertexes, not for every triangle.
You don't let a single geometry shader invocation iterate over all your vertexes, it is called for every original primitive (point, line, triangle, ...).
The solution is much easier: In the vertex shader (that is actually called for every vertex) you can read the special variable gl_VertexID, which contains the vertex's index. That index is either just a counter incremented for every vertex (if using glDrawArrays) and reset by every draw call, or the index from the index array (if using glDrawElements).
EDIT: Regarding the long time goal. Not directly but you might use a texture buffer for that. This basically enables you to get direct linear array-access to a buffer object (in your case the vertex buffer) which you can then just index with this vertex index. But there might also be other ways to accomplish that, which may suffice for another question.