Create d3d11Texture2D with initial data from camera - c++

I am trying to read input from camera and use it to create background surface in D3D11. I receive memory errors all the time.
My render Target size is: 2364 * 1461
Image I get from the camera is an array of type unsigned char
unsigned char* p = g_pOvrvision->GetCamImage(OVR::OV_CAMEYE_LEFT, (OVR::OvPSQuality)processer_quality);
It returns 640 * 480 * 3 bytes. The code I am working with is below. CreateTexture2D gives a memory error. I tried filling the array with dummy data to fit all 2364 * 1461. This did not work either.
Could you please suggest me solution?
D3D11_TEXTURE2D_DESC desc;
ZeroMemory(&desc, sizeof(desc));
desc.Width = renderTargetSize.w;
desc.Height = renderTargetSize.h;
desc.MipLevels = desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DYNAMIC;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
desc.MiscFlags = 0;
ID3D11Texture2D *pTexture = NULL;
D3D11_SUBRESOURCE_DATA TexInitData;
ZeroMemory(&TexInitData, sizeof(D3D11_SUBRESOURCE_DATA));
TexInitData.pSysMem = texArray;
TexInitData.SysMemPitch = static_cast<UINT>(2364 * 3);
TexInitData.SysMemSlicePitch = static_cast<UINT>(3 * 2364 * 1461 * sizeof(unsigned char));
d3dDevice->CreateTexture2D(&desc, &TexInitData, &d3dEyeTexture);
d3dDevice->CreateShaderResourceView(d3dEyeTexture, nullptr, &d3dEyeTextureShaderResourceView);
d3dDevice->CreateRenderTargetView(d3dEyeTexture, nullptr, &d3dEyeTextureRenderTargetView);

Chuck Walbourn's comment gets you 99% of the way there: there is no three-byte DXGI pixel format, so to properly pack the data you are receiving from the camera you must create a new buffer that will match the DXGI_FORMAT_R8G8B8A8_UNORM format, and copy your camera data to it adding a fourth value (an alpha value) to each pixel of 0xFF.
But as you mentioned your last line of code still fails. I believe this is because you haven't set d3dEyeTexture with the bind flag D3D11_BIND_RENDER_TARGET as well as D3D11_BIND_SHADER_RESOURCE. From what I understand, however, if you're trying to make the camera image of 640x480 the background of your 2364x1461 render target, you'll need to resample the data to fit, which is a pain to do (and slow) on the CPU. My recommendation would be to create a separate, D3D11_BIND_SHADER_RESOURCE flagged, 640x480 texture with your camera data, and draw it to fill the whole screen as the first step in your draw loop. This means creating a separate, dedicated 2364x1461 render target as your back buffer, but that's a common first step in setting up a Direct3D app.

Related

DirectShow: Rendering for YV12

I am trying to understand how to work the rendering for YV12 format. For example, I took a simple sample. See this graph:
The webcam creates frames by size 640x480 in RGB24 or MJPEG . After it the LAV decoder transforms the frames to YV12 and sends them to DS renderer (EVR or VMR9).
The decoder changes the frame width (stride) 640 on 1024. Hence, the output size of frame will be 1.5*1024*640=737280. The normal size for YV12 is 1.5*640*480=460800. I know the stride can be more than the width of real frame (https://learn.microsoft.com/en-us/windows/desktop/medfound/image-stride). My first question - why did the renderer select that value (1024) than another? Can I get it programmatically?
When I replace the LAV decoder with my filter for transformation RGB24/YV12 (https://gist.github.com/thedeemon/8052fb98f8ba154510d7), the renderer shows me a shifted image, though all parameters are the same, as for the first graph:
Why? I noted that VIDEOINFOHEADER2 had the set interlacing flag dwInterlaceFlags. Therefore my next question: Do I have to add interlacing into my filter for normal work of renderer?
My first question - why did the renderer select that value (1024) than another? Can I get it programmatically?
Video renderer is using Direct3D texture as a carrier for the image. When texture is mapped into system memory to enable CPU's write access such extended stride could be applied because of specifics of implementation of video hardware. You get the value 1024 via dynamic media type negotiation as described in Handling Format Changes from the Video Renderer.
Your transformation filter has to handle such updates if you want it to be able to connect to video renderer directly.
You are generally not interested in getting this extended stride value otherwise because the one you get via media type update is the one to be used and you have to accept it.
When I replace the LAV decoder with my filter for transformation RGB24/YV12, the renderer shows me a shifted image, though all parameters are the same, as for the first graph...
Why?
Your filter does not handle stride update right.
...I noted that VIDEOINFOHEADER2 had the set interlacing flag dwInterlaceFlags. Therefore my next question: Do I have to add interlacing into my filter for normal work of renderer?
You don't have interlaced video here. The problem is unrelated to interlaced video.
My solution:
I must right copy a YV12 frame into a video buffer by its three surfaces: Y = 4x4, U = 1x2, V = 1x2. Here is a code for the frame size 640x480:
CVideoGrabberFilter::Transform(IMediaSample *pIn, IMediaSample *pOut)
{
BYTE* pSrcBuf = 0;
pIn->GetPointer(&pSrcBuf);
BYTE* pDstBuf = 0;
pOut->GetPointer(&pDstBuf);
SIZE size;
size.cx = 640;
size.cy = 480;
int nLen = pOut->GetActualDataLength();
BYTE* pDstTmp = new BYTE[nLen];
YV12ConverterFromRGB24(pSrcBufEnd, pDstTmp, size.cx, size.cy);
BYTE* pDst = pDstTmp;
int stride = 1024; //the real video stride for 640x480. For other resolutions you need to use pOut->GetMediaType() for the stride defining.
//Y
for (int y = 0; y < size.cy; ++y)
{
memcpy(pDstBuf, pDst, size.cx);
pDst += size.cx;
pDstBuf += stride;
}
stride /= 2;
size.cy /= 2;
size.cx /= 2;
//U and V
for (int y = 0; y < size.cy; y++ )
{
memcpy(pDstBuf, pDst, size.cx );
pDst += size.cx;
pDstBuf += stride;
memcpy(pDstBuf, pDst, size.cx);
pDst += size.cx;
pDstBuf += stride;
}
delete[] pDstTmp;
}

Data race in passing a texture across threads in DirectX

I'm experiencing a data race in a DirectX application with two threads: a consumer and a producer.
The first thread (producer) is a screen grabber which uses desktop duplication to get a desktop image in a texture. It creates a ID3D11Device and ID3D11DeviceContext on adapter X.
dxgi_dd->AcquireNextFrame(INFINITE, &frame_info, &desktop_resource);
ID3D11Texture2D *desktop_texture;
desktop_resource->QueryInterface(__uuidof(ID3D11Texture2D), (void **)&desktop_texture);
desktop_resource->Release();
D3D11_TEXTURE2D_DESC texture_desc;
memset(&texture_desc, 0, sizeof(texture_desc));
texture_desc.Width = desktop_desc.Width;
texture_desc.Height = desktop_desc.Height;
texture_desc.MipLevels = 1;
texture_desc.ArraySize = 1;
texture_desc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
texture_desc.SampleDesc.Count = 1;
texture_desc.SampleDesc.Quality = 0;
texture_desc.Usage = D3D11_USAGE_DEFAULT;
texture_desc.BindFlags = 0;
texture_desc.CPUAccessFlags = 0;
texture_desc.MiscFlags = 0;
d3d11_device->CreateTexture2D(&texture_desc, NULL, &return_texture);
// Copy it to a return texture
immediate_context->CopyResource(return_texture, desktop_texture);
immediate_context->Flush();
dxgi_dd->ReleaseFrame();
desktop_texture->Release();
return encapsulate(return_texture); // Thread returns the pointer encapsulated in a structure
The second thread (consumer) a ID3D11Device and ID3D11DeviceContext on the same adapter X
ID3D11Texture2D *dx_input_texture;
D3D11_TEXTURE2D_DESC desc;
ZeroMemory(&desc, sizeof(D3D11_TEXTURE2D_DESC));
desc.Width = received_texture.width;
desc.Height = received_texture.height;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.Usage = D3D11_USAGE_DEFAULT;
desc.BindFlags = D3D11_BIND_RENDER_TARGET;
desc.CPUAccessFlags = 0;
d3d11_device->CreateTexture2D(&desc, NULL, &dx_input_texture)
ID3D11Texture2D *frame_texture = (ID3D11Texture2D*)received_texture.pointer_received_from_other_thread;
immediate_context->CopyResource(dx_input_texture, frame_texture);
immediate_context->Flush();
// Use dx_input_texture now
Unfortunately I'm having random problems with this approach (invalid textures or pointers causing other DX libraries expecting a valid texture to fail) which disappear if I for example put a sleep(1000) in the producer thread. This makes me thing it could be a data race thing.
Is there any need of synchronization across the two threads? I'm deliberately skipping texture Release() for now (even though I might run out of GPU memory eventually) to debug this data race.
Your code releases pointer desktop_texture->Release(); and then for some reason return it, this looks like a typo and it probably supposed to be return encapsulate(return_texture);
To process texture using different device you will need to create resource with D3D11_RESOURCE_MISC_SHARED flag, convert it to HANDLE by calling IDXGIResource::GetSharedHandle and then make usable by calling ID3D11Device::OpenSharedResource.

Error runtime update of DXT compressed textures with Directx11

Context:
I'm developing a native C++ Unity 5 plugin that reads in DXT compressed texture data and uploads it to the GPU for further use in Unity. The aim is to create an fast image-sequence player, updating image data on-the-fly. The textures are compressed with an offline console application.
Unity can work with different graphics engines, I'm aiming towards DirectX11 and OpenGL 3.3+.
Problem:
The DirectX runtime texture update code, through a mapped subresource, gives different outputs on different graphics drivers. Updating a texture through such a mapped resource means mapping a pointer to the texture data and memcpy'ing the data from the RAM buffer to the mapped GPU buffer. Doing so, different drivers seem to expect different parameters for the row pitch value when copying bytes. I never had problems on the several Nvidia GPU's I tested on, but AMD and Intel GPU seems to act differently and I get distorted output as shown underneath. Furthermore, I'm working with DXT1 pixel data (0.5bpp) and DXT5 data (1bpp). I can't seem to get the correct pitch parameter for these DXT textures.
Code:
The following initialisation code for generating the d3d11 texture and filling it with initial texture data - e.g. the first frame of an image sequence - works perfect on all drivers. The player pointer points to a custom class that handles all file reads and contains getters for the current loaded DXT compressed frame, it's dimensions, etc...
if (s_DeviceType == kUnityGfxRendererD3D11)
{
HRESULT hr;
DXGI_FORMAT format = (compression_type == DxtCompressionType::DXT_TYPE_DXT1_NO_ALPHA) ? DXGI_FORMAT_BC1_UNORM : DXGI_FORMAT_BC3_UNORM;
// Create texture
D3D11_TEXTURE2D_DESC desc;
desc.Width = w;
desc.Height = h;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = format;
// no anti-aliasing
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DYNAMIC;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
desc.MiscFlags = 0;
// Initial data: first frame
D3D11_SUBRESOURCE_DATA data;
data.pSysMem = player->getBufferPtr();
data.SysMemPitch = 16 * (player->getWidth() / 4);
data.SysMemSlicePitch = 0; // just a 2d texture, no depth
// Init with initial data
hr = g_D3D11Device->CreateTexture2D(&desc, &data, &dxt_d3d_tex);
if (SUCCEEDED(hr) && dxt_d3d_tex != 0)
{
DXT_VERBOSE("Succesfully created D3D Texture.");
DXT_VERBOSE("Creating D3D SRV.");
D3D11_SHADER_RESOURCE_VIEW_DESC SRVDesc;
memset(&SRVDesc, 0, sizeof(SRVDesc));
SRVDesc.Format = format;
SRVDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
SRVDesc.Texture2D.MipLevels = 1;
hr = g_D3D11Device->CreateShaderResourceView(dxt_d3d_tex, &SRVDesc, &textureView);
if (FAILED(hr))
{
dxt_d3d_tex->Release();
return hr;
}
DXT_VERBOSE("Succesfully created D3D SRV.");
}
else
{
DXT_ERROR("Error creating D3D texture.")
}
}
The following update code that runs for each new frame has the error somewhere. Please note the commented line containing method 1 using a simple memcpy without any rowpitch specified which works well on NVIDIA drivers.
You can see further in method 2 that I log the different row pitch values. For instace for a 1920x960 frame I get 1920 for the buffer stride, and 2048 for the runtime stride. This 128 pixels difference probably have to be padded (as can be seen in the example pic below) but I can't figure out how. When I just use the mappedResource.RowPitch without dividing it by 4 (done by the bitshift), Unity crashes.
ID3D11DeviceContext* ctx = NULL;
g_D3D11Device->GetImmediateContext(&ctx);
if (dxt_d3d_tex && bShouldUpload)
{
if (player->gather_stats) before_upload = ns();
D3D11_MAPPED_SUBRESOURCE mappedResource;
ctx->Map(dxt_d3d_tex, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
/* 1: THIS CODE WORKS ON ALL NVIDIA DRIVERS BUT GENERATES DISTORTED OR NO OUTPUT ON AMD/INTEL: */
//memcpy(mappedResource.pData, player->getBufferPtr(), player->getBytesPerFrame());
/* 2: THIS CODE GENERATES OUTPUT BUT SEEMS TO NEED PADDING? */
BYTE* mappedData = reinterpret_cast<BYTE*>(mappedResource.pData);
BYTE* buffer = player->getBufferPtr();
UINT height = player->getHeight();
UINT buffer_stride = player->getBytesPerFrame() / player->getHeight();
UINT runtime_stride = mappedResource.RowPitch >> 2;
DXT_VERBOSE("Buffer stride: %d", buffer_stride);
DXT_VERBOSE("Runtime stride: %d", runtime_stride);
for (UINT i = 0; i < height; ++i)
{
memcpy(mappedData, buffer, buffer_stride);
mappedData += runtime_stride;
buffer += buffer_stride;
}
ctx->Unmap(dxt_d3d_tex, 0);
}
Example pic 1 - distorted ouput when using memcpy to copy whole buffer without using separate row pitch on AMD/INTEL (method 1)
Example pic 2 - better but still erroneous output when using above code with mappedResource.RowPitch on AMD/INTEL (method 2). The blue bars indicate zone of error, and need to disappear so all pixels align well and form one image.
Thanks for any pointers!
Best,
Vincent
The mapped data row pitch is in byte, when you divide by four, it is definitely an issue.
UINT runtime_stride = mappedResource.RowPitch >> 2;
...
mappedData += runtime_stride; // here you are only jumping one quarter of a row
It is the height count with a BC format that is divide by 4.
Also a BC1 format is 8 bytes per 4x4 block, so the line below should by 8 * and not 16 *, but as long as you handle row stride properly on your side, d3d will understand, you just waste half the memory here.
data.SysMemPitch = 16 * (player->getWidth() / 4);

Cannot create a 1D Texture on DirectX11

For some reason, the code below crashes when I try to create the 1d texture.
D3D11_TEXTURE1D_DESC desc;
ZeroMemory(&desc, sizeof(D3D11_TEXTURE1D_DESC));
desc.Width = 64;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_SNORM;
desc.Usage = D3D11_USAGE_STAGING;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE | D3D11_CPU_ACCESS_READ;
HRESULT hr = D3DDev_0001->CreateTexture1D(&desc, NULL, &texture); //crashes here
assert(hr == S_OK);
where D3DDev_0001 is a ID3D11Device. I am able to create 3d and 2d textures, but making a 1d texture causes the program to crash. Can anyone explain why?
A USAGE_STAGING texture can't have any BindFlags since it can't be set on the graphics context for use as an SRV, UAV or RTV. Set BindFlags to 0 if you want a STAGING texture, or set the Usage to D3D11_USAGE_DEFAULT if you just want a 'normal' texture that can be bound to the context.
USAGE_STAGING resources are either for the CPU to fill in with data before being copied to a USAGE_DEFAULT resource, or, they're the destination for GPU copies to get data from the GPU back to the CPU.
The exact cause of this error would have been explained in a message printed by "D3D11's Debug Layer"; use it to find the cause of these errors in the future.

Directx11 Texture2D formats

For some reason, I cannot specify DXGI_FORMAT_R32G32B32_FLOAT format when creating a texture 2d in directx11. I can do it just fine in OpenGL, however. It also works fine when using DXGI_FORMAT_R32G32B32A32_FLOAT. I am using these textures as rendertargets for the gbuffer.
// create gbuffer textures/rendertargets
D3D11_TEXTURE2D_DESC textureDesc;
ZeroMemory(&textureDesc, sizeof(D3D11_TEXTURE2D_DESC));
textureDesc.Width = swapChainDesc.BufferDesc.Width;
textureDesc.Height = swapChainDesc.BufferDesc.Height;
textureDesc.ArraySize = 1;
textureDesc.MipLevels = 1;
textureDesc.Format = DXGI_FORMAT_R32G32B32_FLOAT; <----- dosn't like this; returns E_INVALIDARG
textureDesc.SampleDesc.Count = 1;
textureDesc.Usage = D3D11_USAGE_DEFAULT;
textureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET;
for (uint32_t index = 0; index < GBuffer::GBUFFER_NUM_RENDERTARGETS; index++)
{
DXCALL(device->CreateTexture2D(&textureDesc, NULL, &mGeometryTextures[index]));
DXCALL(device->CreateRenderTargetView(mGeometryTextures[index], NULL, &mRenderTargets[index]));
}
Why cant I use DXGI_FORMAT_R32G32B32_FLOAT when creating a 2d texture in directx 11?
I do not need the extra float in my texture, hence I'd rather have just three elements rather than four.
Not all hardware supports using R32G32B32_FLOAT as a render-target and shader-resource (it's optional). You can verify whether the hardware supports the format for those uses by calling CheckFormatSupport. If it is succeeding on the same hardware with OpenGL, this likely means OpenGL is padding the resource out to the full 4-channel variant behind the scenes.
DXGI_FORMAT_R32G32B32_FLOAT support for render targets is optional: http://msdn.microsoft.com/en-us/library/windows/desktop/ff471325(v=vs.85).aspx#RenderTarget
If you think that this format should be supported by your device then turn on debug output as MooseBoys suggested. This should explain why you're getting E_INVALIDARG.