I am using GBM, DRM and EGL to render my scene to a HDMI display on my R-PI 4B. No X server is installed, and the R-PI boots into text mode before launching my application.
First my problem: I notice a lot of flickering throughout all of my rendering. So far I do not render much, I render a buch of text elements and a texture and I can see it flicker with my background without any changes to the texture or the other elements rendered.
I have attached a video here:
https://streamable.com/0pq3d0
It does not seem so bad in the video, but with the naked eye flickering is horrible (the black flashes).
For now I assume it has something to do with how I do my end_frame rendering:
void graphics_device_drm::end_frame() {
auto sync = eglCreateSyncKHR(_display, EGL_SYNC_FENCE_KHR, nullptr);
glFlush();
eglClientWaitSyncKHR(_display, sync, 0, EGL_FOREVER_KHR);
eglDestroySyncKHR(_display, sync);
eglSwapBuffers(_display, _surface);
auto bo = gbm_surface_lock_front_buffer(_gbm_surface);
const auto handle = gbm_bo_get_handle(bo).u32;
const auto pitch = gbm_bo_get_stride(bo);
uint32_t fb;
drmModeAddFB(_device, _width, _height, 24, 32, pitch, handle, &fb);
drmModeSetCrtc(_device, _crtc->crtc_id, fb, 0, 0, &_connector_id, 1, &_mode);
if (_previous_bo) {
drmModeRmFB(_device, _previous_fb);
gbm_surface_release_buffer(_gbm_surface, _previous_bo);
}
_previous_bo = bo;
_previous_fb = fb;
}
It almost seems like it is using only one single buffer for rendering. I dont really understand the DRM and GBM methods, so I assume I am doing something wrong there. Any pointers would be appreciated.
As a matter of fact it apparently was not related to my code and an R-PI/driver issue. Nevertheless the following change in /boot/config.txt did the trick:
# dtoverlay=vc4-fkms-v3d
dtoverlay=vc4-kms-v3d-pi4
The commented line (with fkms) was before and the other line was after. I assume for it to work you also need to have the latest Mesa libraries compiled which I anyway did before. No flickering whatsoever now!
Related
I am trying to write a small 3D game in C++, using DirectX 11. This is absolutely the first time I have attempted to write a game using only a graphics API. I have been following the tutorials on the website Rastertek.com up to Tutorial 9 for ambient lighting.
After implementing movement and collisions for the player, I increased the size of my play area. This is when I noticed my issue: the textures I am using for the walls and floor of my play area are not being rendered the way I expected them to.
Wall from close up
Wall from far away
Maybe you can tell how the lines on the wall appear strangely broken up - I was expecting them to be rendered properly at larger distances (like they are close up).
The thing that seems most weird to me, though, is that the lines can be rendered from far away, but only while moving the camera around the scene and only on certain parts of the wall. Standing still breaks the texture again. I tried capturing this effect on video, but I had no success getting it to show up in the video I took with the GeForce Experience.
I tried playing around with a bunch of the settings that DirectX offers, like the rasterizer or the depth buffer descriptions, I tried to enable and disable VSync, Antialiasing and Multisampling, I tried using Anisotropic filtering instead of linear filtering... But none of it had any effect.
I do not know where to look and what to try next. Am I just going to have to accept that my textures will look terrible at any sort of distance?
You need to generate mip maps for the texture you load. Check the DDSTextureLoader.h/cpp and WICTextureLoader.h/cpp here.
For example, to load the .dds image with mip maps, you would use:
HRESULT DirectX::CreateDDSTextureFromFileEx( ID3D11Device* d3dDevice,
ID3D11DeviceContext* d3dContext,
const wchar_t* fileName,
size_t maxsize,
D3D11_USAGE usage,
unsigned int bindFlags,
unsigned int cpuAccessFlags,
unsigned int miscFlags,
bool forceSRGB,
ID3D11Resource** texture,
ID3D11ShaderResourceView** textureView,
DDS_ALPHA_MODE* alphaMode )
Example of usage:
HRESULT hr = DirectX::CreateDDSTextureFromFileEx(device, context, path.c_str(), 0, D3D11_USAGE_DEFAULT, D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET, 0, D3D11_RESOURCE_MISC_GENERATE_MIPS, 0, reinterpret_cast<ID3D11Resource**>(&pTexture), &pSRV);
THROW_IF_FAILED(hr);
Note the flags D3D11_BIND_RENDER_TARGET and D3D11_RESOURCE_MISC_GENERATE_MIPS used.
I need to take sceenshots at every frame and I need very high performance (I'm using freeGlut). What I figured out is that it can be done like this inside glutIdleFunc(thisCallbackFunction)
GLubyte *data = (GLubyte *)malloc(3 * m_screenWidth * m_screenHeight);
glReadPixels(0, 0, m_screenWidth, m_screenHeight, GL_RGB, GL_UNSIGNED_BYTE, data);
// and I can access pixel values like this: data[3*(x*512 + y) + color] or whatever
It does work indeed but I have a huge issue with it, it's really slow. When my window is 512x512 it runs no faster than 90 frames per second when only cube is being rendered, without these two lines it runs at 6500 FPS! If we compare it to irrlicht graphics engine, there I can do this
// irrlicht code
video::IImage *screenShot = driver->createScreenShot();
const uint8_t *data = (uint8_t*)screenShot->lock();
// I can access pixel values from data in a similar manner here
and 512x512 window runs at 400 FPS even with a huge mesh (Quake 3 Map) loaded! Take into account that I'm using openGL as driver inside irrlicht. To my inexperienced eye it seems like glReadPixels is copying every pixel data from one place to another while (uint8_t*)screenShot->lock() is just copying a pointer to already existent array. Can I do something similar to latter using freeGlut? I expect it to be faster than irrlicht.
Note that irrlicht uses openGL too (well it offers directX and other options as well but in the example I gave above I used openGL and by the way it was the fastest compared to other options)
OpenGL methods are used to manage the rendering pipeline. In its nature, while the graphics card is showing image to the viewer, computations of the next frame are being done. When you call glReadPixels; graphics card wait for the current frame to be done, reads the pixels and then starts computing the next frame. Therefore pipeline becomes stalled and becomes sequential.
If you can hold two buffers and tell to the graphics card to read data into these buffers interchanging each frame; then you can read-back from your buffer 1-frame late but without stalling the pipeline. This is called double buffering. You can also do triple buffering with 2 frame late read-back and so on.
There is a relatively old web page describing the phenomenon and implementation here: http://www.songho.ca/opengl/gl_pbo.html
Also there are a lot of tutorials about framebuffers and rendering into a texture on the web. One of them is here: http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-14-render-to-texture/
I'm making an Augmented Reality App (C++) in Windows 10 (universal Windows app / WinRT) with openCV.
Problem:
I want to have a transparent SwapChainPanel background to make the content behind (webcam stream) visible, but with opaque 3D-models (eg. a cone).
Try:
To my research it seems, that setting the CompositeMode of the SwapChainPanel to "minBlend" should do it - yes, but I still want my 3D Objects to be opaque. In fact, I want my objects to be semitransparent, but always visible. The "minBlend" mode is more for text-highlighting, not really to overlay something with semitransparent models (dark areas are not overlayed, see pictures).
Image: Standard DirectX Cube (oqaque background and model)
Image: DirectX Cube overlayed
Do you have any suggestions? Is it possible?
Background:
I'm making an Augmented Reality Windows 10 App with openCV. For getting the current pixel data of my webcam stream I'm using the Win10 methods mediaCapture->GetPreviewFrameAsync(videoframe) with SoftwareBitmap->LockBuffer to get access to the bytes in the memorybuffer. The bytes are processed within openCV functions and after processing is complete, I'm setting up a WriteableBitmap to show the modified webcam-stream in my Xaml-UI element. Because of already having classes to draw my DirectX objects and modify them with touch input, i want to use DirectX to overlay the webcam preview with my objects.
Sorry for not linking the used methods and the linked Images, I haven't enough reputation
Edit: Maybe an alternative would be to create a texture from my pixel data and set up a fullscreen rectangle on my swapchainpanel which functions as a background. Then every frame I have to update the texture data.
Due to some official Windows Store examples, changing the compositeMode is the only solution to make Xaml content behind a swapChainPanel visible.
But this workaround finally works: I've created a Texture2D and update the containing data every frame:
/*----- Update background texture with new videoframe -----*/
// image = openCV cv::Mat containing the pixeldata of the current frame
if (image.data != nullptr)
{
D3D11_MAPPED_SUBRESOURCE mappedResource;
ZeroMemory(&mappedResource, sizeof(D3D11_MAPPED_SUBRESOURCE));
// Disable GPU access to data.
auto m_d3dContext = m_deviceResources->GetD3DDeviceContext();
m_d3dContext->Map(m_pTexture, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
// Update texture
memcpy(mappedResource.pData, image.data, 4*image.rows*image.cols);
// Reenable GPU access to data.
m_d3dContext->Unmap(m_pTexture, 0);
}
At first I'm drawing a fullscreen rectangle (SpriteBatch with DirectXTK) with this texture. After that I can draw anything else.
You may know that trying to capture DirectX fullscreen applications the GDI way (using BitBlt()) gives a black screenshot.
My question is rather simple but I couldn't find any answer: why? I mean technically, why does it give a black screenshot?
I'm reading a DirectX tutorial here: http://www.directxtutorial.com/Lesson.aspx?lessonid=9-4-1. It's written:
[...] the function BeginScene() [...] does something called locking, where the buffer in the video RAM is 'locked', granting you exclusive access to this memory.
Is this the reason? VRAM is locked so GDI can't access it and it gives a black screenshot?
Or is there another reason? Like DirectX directly "talks" to the graphic card and GDI doesn't get it?
Thank you.
The reason is simple: performance.
The idea is to render a scene as much as possible on the GPU out of lock-step with the CPU. You use the CPU to send the rendering buffers to the GPU (vertex, indices, shaders etc), which is overall really cheap because they're small, then you do whatever you want, physics, multiplayer sync etc. The GPU can just crunch the data and render it on its own.
If you require the scene to be drawn on the window, you have to interrupt the GPU, ask for the rendering buffer bytes (LockRect), ask for the graphics object for the window (more interference with the GPU), render it and free every lock. You just lost any sort of gain you had by rendering on the GPU out of sync with the CPU. Even worse when you think of all the different CPU cores just sitting idle because you're busy "rendering" (more like waiting on buffer transfers).
So what graphics drivers do is they paint the rendering area with a magic color and tell the GPU the position of the scene, and the GPU takes care of overlaying the scene over the displayed screen based on the magic color pixels (sort of a multi-pass pixel shader that takes from the 2nd texture when the 1st texture has a certain color for x,y, but not that slow). You get completely out of sync rendering, but when you ask the OS for its video memory, you get the magic color where the scene is because that's what it actually uses.
Reference: http://en.wikipedia.org/wiki/Hardware_overlay
I believe it is actually due to double buffering. I'm not 100% sure but that was actually the case when I tested screenshots in OpenGL. I would notice that the DC on my window was not the same. It was using two different DC's for this one game.. For other games I wasn't sure what it was doing. The DC was the same but swapbuffers was called so many times that I don't think GDI was even fast enough to screenshot it.. Sometimes I would get half a screenshot and half black..
However, when I hooked into the client, I was able to just ask for the pixels like normal. No GDI or anything. I think there is a reason why we don't use GDI when drawing in games that use DirectX or OpenGL..
You can always look at ways to capture the screen here:http://www.codeproject.com/Articles/5051/Various-methods-for-capturing-the-screen
Anyway, I use the following for grabbing data from DirectX:
HRESULT DXI_Capture(IDirect3DDevice9* Device, const char* FilePath)
{
IDirect3DSurface9* RenderTarget = nullptr;
HRESULT result = Device->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &RenderTarget);
result = D3DXSaveSurfaceToFile(FilePath, D3DXIFF_PNG, RenderTarget, nullptr, nullptr);
SafeRelease(RenderTarget);
return result;
}
Then in my hooked Endscene I call it like so:
HRESULT Direct3DDevice9Proxy::EndScene()
{
DXI_Capture(ptr_Direct3DDevice9, "C:/Ssers/School/Desktop/Screenshot.png");
return ptr_Direct3DDevice9->EndScene();
}
You can either use microsoft detours for hooking EndScene of some external application or you can use a wrapper .dll.
I have an OpenGL game, and I want to save what's shown on the screen to a video.
How can I do that? Is there any library or how-to-do-it?
I don't care about compression, I need the most efficient way so hopefully the FPS won't drop.
EDIT:
It's OpenGL 1.1 and it's working on Mac OSX though I need it to be portable.
There most certainly are great video capture software out there you could use to capture your screen, even when running a full screen OpenGL game.
If you are using new versions of OpenGL, as genpfault has mentioned you can use PBOs. If you are using legacy OpenGL (version 1.x), here's how you can capture the screen:
glFinish(); // Make sure everything is drawn
glReadBuffer(GL_FRONT);
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
glReadPixels(blx, bly, w, h, mode, GL_UNSIGNED_BYTE, GL_BGRA);
where blx and bly are the bottom left coordinates of the part of the screen you want to capture (in your case (0, 0)) and w and h are the width and height of the box to be captured. See the reference for glReadPixels for more info, such as the last parameter.
Writing captured screen (at your desired rate, for example 24 fps) to a video file is a simple matter of choosing the file format you want (for example raw video), write the header of the video and write the images (image by image if raw, or image differences in some other format etc)
Use Pixel Buffer Objects (PBOs).