Rendering frames from a webcam to a DirectX 11 texture - c++

I am having difficulty updating a DirectX 11 texture with the image data from a webcam frame buffer in memory. I've managed to create a texture from a single frame in the buffer but as the buffer is overwritten with the next frame the texture doesn't update. So I'm left with a snap shot image rather than a live stream which I'm after.
I am trying to use the Map/Unmap methods for updating an ID3D11Texture2D resource because that is supposedly more efficient than using the UpdateSubresource method. I haven't managed to get either to work. I'm new to DirectX and I just can't find a good explanation anywhere on how to accomplish this.
Create texture here:
bool CreateCamTexture(ID3D11ShaderResourceView** out_srv, RGBQUAD* ptrimg, int* image_width, int* image_height)
{
ZeroMemory(&desc, sizeof(desc));
desc.Width = *image_width;
desc.Height = *image_height;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.Usage = D3D11_USAGE_DYNAMIC;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
std::cout << ptrimg << std::endl;
subResource.pSysMem = ptrimg;
subResource.SysMemPitch = desc.Width * 4;
subResource.SysMemSlicePitch = 0;
g_pd3dDevice->CreateTexture2D(&desc, &subResource, &pTexture);
ZeroMemory(&srvDesc, sizeof(srvDesc));
srvDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
srvDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
srvDesc.Texture2D.MipLevels = desc.MipLevels;
srvDesc.Texture2D.MostDetailedMip = 0;
g_pd3dDevice->CreateShaderResourceView(pTexture, &srvDesc, out_srv);
if (pTexture != NULL) {
pTexture->Release();
}
else
{
std::cout << "pTexture is NULL ShaderResourceView not created" << std::endl;
}
return true;
}
bool CreateDeviceD3D(HWND hWnd)
{
// Setup swap chain
DXGI_SWAP_CHAIN_DESC sd;
ZeroMemory(&sd, sizeof(sd));
sd.BufferCount = 2;
sd.BufferDesc.Width = 0;
sd.BufferDesc.Height = 0;
sd.BufferDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
sd.BufferDesc.RefreshRate.Numerator = 60;
sd.BufferDesc.RefreshRate.Denominator = 1;
sd.Flags = DXGI_SWAP_CHAIN_FLAG_ALLOW_MODE_SWITCH;
sd.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
sd.OutputWindow = hWnd;
sd.SampleDesc.Count = 1;
sd.SampleDesc.Quality = 0;
sd.Windowed = TRUE;
sd.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
UINT createDeviceFlags = 0;
createDeviceFlags |= D3D11_CREATE_DEVICE_DEBUG;
D3D_FEATURE_LEVEL featureLevel;
const D3D_FEATURE_LEVEL featureLevelArray[2] = { D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_10_0, };
Attempting to Map/Unmap texture:
void UpdateCamTexture() {
D3D11_MAPPED_SUBRESOURCE mappedResource;
ZeroMemory(&mappedResource, sizeof(D3D11_MAPPED_SUBRESOURCE));
g_pd3dDeviceContext->Map(
pTexture,
0, //0,
D3D11_MAP_WRITE_DISCARD,
0,
&mappedResource);
memcpy(mappedResource.pData, listener_instance.pImgData, sizeof(listener_instance.pImgData));
// Reenable GPU access to the vertex buffer data.
g_pd3dDeviceContext->Unmap(pTexture, 0);
std::cout << "texture updated" << std::endl;
}
I don't get an error, the image is just black. I don't have debug layer enabled though.

Calling sizeof on a pointer listener_instance.pImgData is not what you want since it returns the size of a pointer type (8 on x64 architecture) and not size of array pointed by the pointer. Calling memcpy with the image data size in bytes is also not completely correct solution. See here for more details.
I will copy the answer from there just in case it's deleted.
Maximus Minimus's answer:
Check the returned pitch from your map call - you're assuming it's width * 4 (for 32-bit RGBA) but it may not be (particularly if your texture is not a power of 2 or it's width is not a multiple of 4).
You can only memcpy the entire block in one operation if pitch is equal to width * number of bytes in the format. Otherwise you must memcpy one row at a time.
Sample code, excuse C-isms:
assumes that src and dst are 32-bit RGBA data
unsigned *src; // this comes from whatever your input is
unsigned *dst = (unsigned *) msr.pData; // msr is a D3D11_MAPPED_SUBRESOURCE derived from ID3D11DeviceContext::Map
width and height come from ID3D11Texture2D::GetDesc
for (int i = 0; i < height; i++)
{
memcpy (dst, src, width * 4); // copy one row at a time because msr.RowPitch may be != (width * 4)
dst += msr.RowPitch >> 2; // msr.RowPitch is in bytes so for 32-bit data we divide by 4 (or downshift by 2, same thing)
src += width; // assumes pitch of source data is equal to width * 4
}
You can, of course, also include a test for if (msr.RowPitch == width * 4) and do a single memcpy of the entire thing if it's true.

Related

Texture readback showing all 0s for DirectX 11

I am new to working with textures and DirectX and having issues reading back texture data from the GPU.
I am interested in reading back only a specific subset of my source texture. Also, I am trying to read it back at the least detailed miplevel (1x1 texture). Steps I follow:
Copy subregion of source texture into new texture
D3D11_TEXTURE2D_DESC desc = {0};
desc.Width = 1;
desc.Height = 1;
desc.MipLevels = 0;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DEFAULT;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET;
desc.CPUAccessFlags = 0;
desc.MiscFlags = D3D11_RESOURCE_MISC_GENERATE_MIPS;
hr = pD3D11Device->CreateTexture2D(&desc, nullptr, &pSrcTexture);
D3D11_BOX srcRegion = {0};
srcRegion.left = 1000;
srcRegion.right = 1250;
srcRegion.top = 500;
srcRegion.bottom = 750;
srcRegion.front = 0;
srcRegion.back = 1;
pD3D11DeviceContext->CopySubresourceRegion(pSrcTexture, 0, 0, 0, 0, srcResource, 0, &srcRegion);
2. Create shader resource view and generate mipmaps for newly created texture
D3D11_SHADER_RESOURCE_VIEW_DESC srvDesc = {0};
srvDesc.Format = desc.Format;
srvDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
srvDesc.Texture2D.MipLevels = -1;
srvDesc.Texture2D.MostDetailedMip = 0;
ID3D11ShaderResourceView* pShaderResourceView = nullptr;
hr = pD3D11Device->CreateShaderResourceView(pSrcTexture, &srvDesc, &pShaderResourceView);
pD3D11DeviceContext->GenerateMips(pShaderResourceView);
3. Copy into staging texture to be read back by CPU
D3D11_TEXTURE2D_DESC desc2 = {0};
desc2.Width = 1;
desc2.Height = 1;
desc2.MipLevels = 1;
desc2.ArraySize = 1;
desc2.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc2.SampleDesc.Count = 1;
desc2.SampleDesc.Quality = 0;
desc2.Usage = D3D11_USAGE_STAGING;
desc2.BindFlags = 0;
desc2.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
desc2.MiscFlags = 0;
ID3D11Texture2D* pStagingTexture = nullptr;
hr = pD3D11Device->CreateTexture2D(&desc2, nullptr, &pStagingTexture);
pD3D11DeviceContext->CopyResource(pStagingTexture, pSrcTexture);
4. Map the subresource to access the underlying data, unmapping when finished
D3D11_MAPPED_SUBRESOURCE mappedResource = {0};
hr = pD3D11DeviceContext->Map(pStagingTexture, 0, D3D11_MAP_READ, 0, &mappedResource);
FLOAT* pTexels = (FLOAT*)mappedResource.pData;
std::cout << pTextels[0] << pTextels[1] << pTextels[2] << pTextels[3] << std::endl; // all zeros here
pD3D11DeviceContext->Unmap(pStagingTexture, 0);
Please note that none of my hr results are failing. Why is my texture data showing as all zeros?
Any guidance on how to resolve?
CopyResource, CopySubresourceRegion, and GenerateMips do not return a HRESULT, but it may have failed in any of those functions. A good way to determine that is to enable the Direct3D Debug Device to look for debug output. See this blog post and Microsoft Docs.
I suspect the problem is that you when called GenerateMips it didn't do anything because you provided a 1x1 texture as the starting place so it doesn't have any mips. I also don't see how you set up srcResource, but you are trying to copy using CopySubresourceRegion from a 250x250 texture region to a 1x1 texture which is going to fail as well.
You should take a look at DirectXTK and the DDSTextureLoader / WICTextureLoader modules in particular which implement auto-mip generation, and ScreenGrab which does read-back.
One minor note: = {0}; was a way to zero-fill structs back in VS 2013 or earlier. With C++11 conformant compilers (VS 2015 or later), just use = {}; as that does the zero-fill.

trying to copy pixal data from cpu to gpu using map every thing runs fine but the screen comes empty directx

in my TextureHendler class, I create an empty texture using this
D3D11_TEXTURE2D_DESC textureDesc = { 0 };
textureDesc.Width = textureWidth;
textureDesc.Height = textureHeight;
textureDesc.Format = dxgiFormat;
textureDesc.Usage = D3D11_USAGE_DYNAMIC;
textureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
textureDesc.MiscFlags = 0;
textureDesc.MipLevels = 1;
textureDesc.ArraySize = 1;
textureDesc.SampleDesc.Count = 1;
textureDesc.SampleDesc.Quality = 0;
textureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
hr = m_deviceResource->GetD3DDevice()->CreateTexture2D(
&textureDesc,
nullptr,
&texture);
and on runtime, I want to load data from CPU memory using the code below
Platform::Array<unsigned char>^ datapointer = GetPixelBytes();
static constexpr int BYTES_IN_RGBA_PIXEL = 4;
auto rowspan = BYTES_IN_RGBA_PIXEL * width;
D3D11_MAPPED_SUBRESOURCE ms;
auto device_context = m_deviceResource->GetD3DDeviceContext();
device_context->Map(m_texture->GetTexture2d(), 0, D3D11_MAP_WRITE_DISCARD, 0, &ms);
uint8_t* mappedData = reinterpret_cast<uint8_t*>(ms.pData);
for (int i = 0; i < datapointer->Length; i++)
{
mappedData[i] = datapointer[i];
}
device_context->Unmap(m_texture->GetTexture2d(), 0);
everything runs fine but the output screen comes black
Update :
std::shared_ptr<TextureHendler> m_texture;
TextureHendler Holds
public:
ID3D11Texture2D* GetTexture2d() { return texture.Get(); }
private:
Microsoft::WRL::ComPtr<ID3D11Texture2D> texture;
and a load function which content is shown above
here are the sample code https://github.com/AbhishekSharma-SEG/Demo_DXPlayer thanks for the help

Draw Text with FreeType (DirectX 11)

So I would like to draw text with FreeType (on DirectX 11). The problem is that I don't really understand how to create the my_draw_bitmap function (I follow this tutorial).
I get all the glyphs then convert them to bitmap, but I don't see how to convert an FT_Bitmap* to a ID3D11Texture2D* (this would - in theory - allow me to render a text)
Here is my code :
FontLoader.cpp
void FontLoader::RenderText(ID3D11Device* p_device, Text* p_text, Math::Vec2 p_position)
{
const char* text = p_text->GetText();
FT_Face face = p_text->GetFont()->GetFace();
FT_GlyphSlot slot = face->glyph;
Math::Vec2 pen = p_position;
for (unsigned int i = 0; i < strlen(text); ++i)
{
if (FT_Load_Char(face, text[i], FT_LOAD_RENDER))
continue;
// draw to our target surface
CreateTextureFromBitmap(p_device, &slot->bitmap, Math::Vec2((float)slot->bitmap_left, (float)slot->bitmap_top));
// Increment pen position
pen._x += slot->advance.x >> 6;
}
}
ID3D11Texture2D* FontLoader::CreateTextureFromBitmap(ID3D11Device* p_device, FT_Bitmap* p_bitmap, Math::Vec2 p_position)
{
D3D11_TEXTURE2D_DESC textureDesc;
textureDesc.Width = p_bitmap->width;
textureDesc.Height = p_bitmap->pitch;
textureDesc.MipLevels = textureDesc.ArraySize = 1;
textureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
textureDesc.SampleDesc.Count = 1;
textureDesc.Usage = D3D11_USAGE_DYNAMIC;
textureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
textureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
textureDesc.MiscFlags = 0;
ID3D11Texture2D *texture2D = NULL;
// don't know how and when to use p_bitmap
p_device->CreateTexture2D(&textureDesc, NULL, &texture2D);
return texture2D;
}
Thanks a lot !

Direct3D11: Sharing a texture between devices: black texture

I have two D3D11 devices, each with its own context but on the same adapter.
I am trying to share a texture beween the two, but the texture I recieve on the other side is always black.
HRESULT hr;
// Make a shared texture on device_A / context_A
D3D11_TEXTURE2D_DESC desc;
ZeroMemory(&desc, sizeof(desc));
desc.Width = 1024;
desc.Height = 1024;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DEFAULT;
desc.CPUAccessFlags = 0;
desc.MiscFlags = D3D11_RESOURCE_MISC_SHARED;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET;
ID3D11Texture2D* copy_tex;
hr = device_A->CreateTexture2D(&desc, NULL, &copy_tex);
// Test the texture by filling it with some color
D3D11_RENDER_TARGET_VIEW_DESC rtvd = {};
rtvd.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
rtvd.ViewDimension = D3D11_RTV_DIMENSION_TEXTURE2D;
rtvd.Texture2D.MipSlice = 0;
ID3D11RenderTargetView* copy_tex_view = 0;
hr = device_A->CreateRenderTargetView(copy_tex, &rtvd, &copy_tex_view);
FLOAT clear_color[4] = {1, 0, 0, 1};
context_A->ClearRenderTargetView(copy_tex_view, clear_color);
// Now try to share it to device_B:
IDXGIResource* copy_tex_resource = 0;
hr = copy_tex->QueryInterface( __uuidof(IDXGIResource), (void**)&copy_tex_resource );
HANDLE copy_tex_shared_handle = 0;
hr = copy_tex_resource->GetSharedHandle(&copy_tex_shared_handle);
IDXGIResource* copy_tex_resource_mirror = 0;
hr = device_B->OpenSharedResource(copy_tex_shared_handle, __uuidof(ID3D11Texture2D), (void**)&copy_tex_resource_mirror);
ID3D11Texture2D* copy_tex_mirror = 0;
hr = copy_tex_resource_mirror->QueryInterface(__uuidof(ID3D11Texture2D), (void**)(&copy_tex_mirror));
However: the copy_tex_mirror texture is always black.
I don't get any HRESULT error codes, and can even use copy_tex_mirror on device_B / context_B normally, but I can't get the pixel data that I put into it on device_A.
Am I missing something?
Thanks in advance!
How do you know that the texture is always black? :-)
GPU operations are queued up by Direct3D, so when you open the shared resource on device_B, the ClearRenderTargetView() on device_A might not have been carried out yet. According to the MSDN library documentation on ID3D11Device::OpenSharedResource Method:
If a shared texture is updated on one device ID3D11DeviceContext::Flush must be called on that device.
We had a lot of issues such as this when we implemented shared textures between devices at work. If you add D3D9 or OpenGL to the mix, the pitfalls multiply..

fill texture3d slice wise

I try to fill a texture3D slice wise with 2d images.
my result only gives me back the first of the 6 images like you can see in this picture:
to be sure that it is not a render problem like wrong uvw coordinates I also give you the picture of the uvw coordinates:
here is the code of the creation of the texture3d:
if (ETextureType::Texture3D == TextureType)
{
ID3D11Texture3D* pTexture3D = nullptr;
D3D11_TEXTURE3D_DESC TextureDesc;
ZeroMemory(&TextureDesc, sizeof(TextureDesc));
TextureDesc.Width = nWidth;
TextureDesc.Height = nHeight;
TextureDesc.MipLevels = nMipMaps;
TextureDesc.Depth = nDepth;
TextureDesc.Usage = D3D11_USAGE_DEFAULT;
TextureDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
switch (TextureFormat)
{
case ETextureFormat::R8G8B8A8:
{
TextureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
}
break;
case ETextureFormat::R32FG32FB32FA32F:
{
TextureDesc.Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
}
break;
default:
DebugAssertOnce(UNKNOWN_TEXTURE_FORMAT);
}
HRESULT hr = m_pD3D11Device->CreateTexture3D(&TextureDesc, nullptr, &pTexture3D);
if (FAILED(hr))
{
DebugAssertOnce(UNABLE_TO_CREATE_TEXTURE);
return false;
}
if (bCreateShaderResourceView)
{
D3D11_SHADER_RESOURCE_VIEW_DESC SRVDesc;
ZeroMemory(&SRVDesc, sizeof(SRVDesc));
SRVDesc.Format = TextureDesc.Format;
SRVDesc.Texture3D.MipLevels = TextureDesc.MipLevels;
SRVDesc.Texture3D.MostDetailedMip = 0;
SRVDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE3D;
hr = m_pD3D11Device->CreateShaderResourceView(pTexture3D, &SRVDesc, &pShaderResourceView);
if (FAILED(hr))
{
pTexture3D->Release();
DebugAssertOnce(UNABLE_TO_CREATE_SHADER_RESOURCE_VIEW);
return false;
}
}
else if (bCreateRenderTargetView)
{
ID3D11RenderTargetView* pRenderTargetView = nullptr;
hr = m_pD3D11Device->CreateRenderTargetView(pTexture3D, nullptr, &pRenderTargetView);
if (FAILED(hr))
{
pShaderResourceView->Release();
pTexture3D->Release();
DebugAssertOnce(UNABLE_TO_CREATE_RENDERTARGET_VIEW);
return false;
}
pView = pRenderTargetView;
}
*ppTexture = new CTextureDX11(TextureType, pTexture3D, pShaderResourceView, pView);
return true;
}
and also the filling part:
bool CGraphicsDriverDX11::CreateTexture3DFromImageBuffers(CTexture** ppTexture, const std::vector<CImageBuffer*>* pvecImageBuffers)
{
uint32_t nWidth = pvecImageBuffers->front()->GetWidth();
uint32_t nHeight = pvecImageBuffers->front()->GetHeight();
uint32_t nMipMapLevels = 1;
bool bRet = CreateTexture(ppTexture, nWidth, nHeight, ETextureType::Texture3D, ETextureFormat::R8G8B8A8, nMipMapLevels, false, true, false, static_cast<UINT>(pvecImageBuffers->size()));
if (bRet)
{
ID3D11Texture3D* pD3DTexture = static_cast<ID3D11Texture3D*>((*ppTexture)->GetTexture());
for (size_t nImageBuffer = 0; nImageBuffer < pvecImageBuffers->size(); ++nImageBuffer)
{
uint32_t nIndex = D3D11CalcSubresource(static_cast<UINT>(nImageBuffer), 0, 1);
m_pD3D11DeviceContext->UpdateSubresource(pD3DTexture, nIndex, nullptr, pvecImageBuffers->at(nImageBuffer)->GetData(), nWidth * 4, 0);
}
}
return bRet;
}
I tried a lot... for example I changed this code to texture2DArray and it worked fine. so the mistake is not my CImageBuffer class. also the nDepth variable has the correct value... I think I have to use another command for UpdateSubresource or at least change the parameters somehow.
I also didn't find some examples in the internet.
Thank you in advanced
All the depth textures in a slice are side-by-side in a single subresource. You also need to compute how many depth images are present in a slice for a given miplevel.
This gives you the subresource index which contains the entire slice:
D3D11CalcSubresource(level, 0, mipLevels);
This gives you the number of images in a slice for a given miplevel:
std::max(depth >> level, 1);
Each image in the slice has a pitch of D3D11_MAPPED_SUBRESOURCE.RowPitch laid out one after another in the subresource, with the total size in bytes of the slice as D3D11_MAPPED_SUBRESOURCE.DepthPitch.
For example, here is some code from DirectXTex trimmed down a bit to make it easier to read. It is reading the data out of a captured 3D volume texture, but the logic is the same when filling out textures.
if (metadata.IsVolumemap())
{
assert(metadata.arraySize == 1);
size_t height = metadata.height;
size_t depth = metadata.depth;
for (size_t level = 0; level < metadata.mipLevels; ++level)
{
UINT dindex = D3D11CalcSubresource(level, 0, metadata.mipLevels);
D3D11_MAPPED_SUBRESOURCE mapped;
HRESULT hr = pContext->Map(pSource, dindex, D3D11_MAP_READ, 0, &mapped);
if (FAILED(hr))
// error
auto pslice = reinterpret_cast<const uint8_t*>(mapped.pData);
size_t lines = ComputeScanlines(metadata.format, height);
// For uncompressed images, lines == height
for (size_t slice = 0; slice < depth; ++slice)
{
const Image* img = result.GetImage(level, 0, slice);
const uint8_t* sptr = pslice;
uint8_t* dptr = img->pixels;
for (size_t h = 0; h < lines; ++h)
{
size_t msize = std::min<size_t>(img->rowPitch, mapped.RowPitch);
memcpy_s(dptr, img->rowPitch, sptr, msize);
sptr += mapped.RowPitch;
dptr += img->rowPitch;
}
pslice += mapped.DepthPitch;
}
pContext->Unmap(pSource, dindex);
if (height > 1)
height >>= 1;
if (depth > 1)
depth >>= 1;
}
}