How to Render To Texture in DirectX12 & C++? What is the process? - c++

I have been trying to figure out how to render the entire scene to a texture in DX12. I know how to do this in OpenGL, but I'm having trouble figuring it out in DirectX12. Plus, there isn't many resources online on how its done.
(Currently we have a 3D Model rendering in the scene with a texture applied)
Would anyone be able to point me towards some resources that I can use to learn Render Targets and Rendering to a Texture in DX12? or any good websites?
Any help is much appreciated.
Kind regards,
Charlie

OpenGL is more like Direct3D 11, where Direct3D 12 and Vulkan are more alike in terms of design/usage and level of graphics knowledge needed to use them effectively. As such, you may find it easier to start with Direct3D 11 before jumping into Direct3D 12 rendering. The concepts and HLSL programming are all very similar between 11 & 12, so it can be a good place to start.
The biggest thing to know about DirectX 12 is that it makes the application (i.e. the programmer) responsible for many aspects that were handled by the Direct3D 11 Runtime: CPU/GPU synchronization, memory management, resource scheduling, etc. DirectX 12 is intended to give the experienced graphics programmer more control and therefore able to achieve higher-levels of CPU-side performance for the same complexity of rendering. This additional control and responsibility, however, can be overwhelming for someone new to graphics or DirectX. It's much easier in DX12 to write something that 'works on my machine', but won't run or even crashes on other people's machines.
With all that said, some good resources for starting with Direct3D 12:
There is a new 'landing page' for DirectX here with many useful links and resources for DirectX 12 development: https://devblogs.microsoft.com/directx/landing-page/
Official DirectX 12 samples written by the DirectX graphics team are at DirectX-Graphics-Samples.
Public samples written by the Xbox Advanced Technology Group are at Xbox-ATG-Samples. In particular, see the IntroGraphics samples which offer many basic samples in both DX11 & DX12 form.
The DirectX Tool Kit is an open-source C++ library that provides helpers for getting going with Direct3D development. There are both DirectX 11 and DirectX 12 versions. If you learn the DX 11 version first, it's pretty simple to move over to DX 12 from there as it handles a number of the 'house-keeping' tasks for you as you learn the new API.
As for the question of 'rendering-to-texture' in DirectX 12, there are some specific samples to look at:
SimpleMSAA does render-to-texture.
This HDR rendering tutorial for DirectX Tool Kit for DX12 does render-to-texture.
The second one uses this this helper class h / cpp.
class RenderTexture
{
public:
RenderTexture(DXGI_FORMAT format) noexcept;
void SetDevice(_In_ ID3D12Device* device, D3D12_CPU_DESCRIPTOR_HANDLE srvDescriptor, D3D12_CPU_DESCRIPTOR_HANDLE rtvDescriptor);
void SizeResources(size_t width, size_t height);
void ReleaseDevice() noexcept;
void TransitionTo(_In_ ID3D12GraphicsCommandList* commandList, D3D12_RESOURCE_STATES afterState);
void BeginScene(_In_ ID3D12GraphicsCommandList* commandList)
{
TransitionTo(commandList, D3D12_RESOURCE_STATE_RENDER_TARGET);
}
void EndScene(_In_ ID3D12GraphicsCommandList* commandList)
{
TransitionTo(commandList, D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE);
}
void SetClearColor(DirectX::FXMVECTOR color)
{
DirectX::XMStoreFloat4(reinterpret_cast<DirectX::XMFLOAT4*>(m_clearColor), color);
}
ID3D12Resource* GetResource() const noexcept { return m_resource.Get(); }
D3D12_RESOURCE_STATES GetCurrentState() const noexcept { return m_state; }
void SetWindow(const RECT& rect);
DXGI_FORMAT GetFormat() const noexcept { return m_format; }
private:
Microsoft::WRL::ComPtr<ID3D12Device> m_device;
Microsoft::WRL::ComPtr<ID3D12Resource> m_resource;
D3D12_RESOURCE_STATES m_state;
D3D12_CPU_DESCRIPTOR_HANDLE m_srvDescriptor;
D3D12_CPU_DESCRIPTOR_HANDLE m_rtvDescriptor;
float m_clearColor[4];
DXGI_FORMAT m_format;
size_t m_width;
size_t m_height;
};
RenderTexture::RenderTexture(DXGI_FORMAT format) noexcept :
m_state(D3D12_RESOURCE_STATE_COMMON),
m_srvDescriptor{},
m_rtvDescriptor{},
m_clearColor{},
m_format(format),
m_width(0),
m_height(0)
{
}
void RenderTexture::SetDevice(_In_ ID3D12Device* device, D3D12_CPU_DESCRIPTOR_HANDLE srvDescriptor, D3D12_CPU_DESCRIPTOR_HANDLE rtvDescriptor)
{
if (device == m_device.Get()
&& srvDescriptor.ptr == m_srvDescriptor.ptr
&& rtvDescriptor.ptr == m_rtvDescriptor.ptr)
return;
if (m_device)
{
ReleaseDevice();
}
{
D3D12_FEATURE_DATA_FORMAT_SUPPORT formatSupport = { m_format, D3D12_FORMAT_SUPPORT1_NONE, D3D12_FORMAT_SUPPORT2_NONE };
if (FAILED(device->CheckFeatureSupport(D3D12_FEATURE_FORMAT_SUPPORT, &formatSupport, sizeof(formatSupport))))
{
throw std::runtime_error("CheckFeatureSupport");
}
UINT required = D3D12_FORMAT_SUPPORT1_TEXTURE2D | D3D12_FORMAT_SUPPORT1_RENDER_TARGET;
if ((formatSupport.Support1 & required) != required)
{
#ifdef _DEBUG
char buff[128] = {};
sprintf_s(buff, "RenderTexture: Device does not support the requested format (%u)!\n", m_format);
OutputDebugStringA(buff);
#endif
throw std::runtime_error("RenderTexture");
}
}
if (!srvDescriptor.ptr || !rtvDescriptor.ptr)
{
throw std::runtime_error("Invalid descriptors");
}
m_device = device;
m_srvDescriptor = srvDescriptor;
m_rtvDescriptor = rtvDescriptor;
}
void RenderTexture::SizeResources(size_t width, size_t height)
{
if (width == m_width && height == m_height)
return;
if (m_width > UINT32_MAX || m_height > UINT32_MAX)
{
throw std::out_of_range("Invalid width/height");
}
if (!m_device)
return;
m_width = m_height = 0;
auto heapProperties = CD3DX12_HEAP_PROPERTIES(D3D12_HEAP_TYPE_DEFAULT);
D3D12_RESOURCE_DESC desc = CD3DX12_RESOURCE_DESC::Tex2D(m_format,
static_cast<UINT64>(width),
static_cast<UINT>(height),
1, 1, 1, 0, D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET);
D3D12_CLEAR_VALUE clearValue = { m_format, {} };
memcpy(clearValue.Color, m_clearColor, sizeof(clearValue.Color));
m_state = D3D12_RESOURCE_STATE_RENDER_TARGET;
// Create a render target
ThrowIfFailed(
m_device->CreateCommittedResource(&heapProperties, D3D12_HEAP_FLAG_ALLOW_ALL_BUFFERS_AND_TEXTURES,
&desc,
m_state, &clearValue,
IID_GRAPHICS_PPV_ARGS(m_resource.ReleaseAndGetAddressOf()))
);
SetDebugObjectName(m_resource.Get(), L"RenderTexture RT");
// Create RTV.
m_device->CreateRenderTargetView(m_resource.Get(), nullptr, m_rtvDescriptor);
// Create SRV.
m_device->CreateShaderResourceView(m_resource.Get(), nullptr, m_srvDescriptor);
m_width = width;
m_height = height;
}
void RenderTexture::ReleaseDevice() noexcept
{
m_resource.Reset();
m_device.Reset();
m_state = D3D12_RESOURCE_STATE_COMMON;
m_width = m_height = 0;
m_srvDescriptor.ptr = m_rtvDescriptor.ptr = 0;
}
void RenderTexture::TransitionTo(_In_ ID3D12GraphicsCommandList* commandList, D3D12_RESOURCE_STATES afterState)
{
TransitionResource(commandList, m_resource.Get(), m_state, afterState);
m_state = afterState;
}
void RenderTexture::SetWindow(const RECT& output)
{
// Determine the render target size in pixels.
auto width = size_t(std::max<LONG>(output.right - output.left, 1));
auto height = size_t(std::max<LONG>(output.bottom - output.top, 1));
SizeResources(width, height);
}
You'd use it like this:
// Setup
m_scene = std::make_unique<DX::RenderTexture>( /* format that matches your resource and your Pipeline State Objects you will use to render */ );
m_scene->SetClearColor( /* color value you use to clear */ );
m_scene->SetDevice(m_device,
/* CPU descriptor handle for your scene as a SRV texture */,
/* CPU descriptor handle for your scene as a RTV texture */);
m_scene->SetWindow( /* provide viewport size for your render texture */ );
// Reset command list and allocator.
// Transition the backbuffer target into the correct state to allow for
// Clear the render texture
CD3DX12_CPU_DESCRIPTOR_HANDLE rtvDescriptor(
/* CPU descriptor handle for your scene as a RTV texture */
static_cast<INT>(m_backBufferIndex), m_rtvDescriptorSize);
CD3DX12_CPU_DESCRIPTOR_HANDLE dsvDescriptor(m_dsvDescriptorHeap->GetCPUDescriptorHandleForHeapStart());
m_commandList->OMSetRenderTargets(1, &rtvDescriptor, FALSE, &dsvDescriptor);
m_commandList->ClearRenderTargetView(rtvDescriptor, /* clear color */, 0, nullptr);
m_commandList->ClearDepthStencilView(dsvDescriptor, D3D12_CLEAR_FLAG_DEPTH, 1.0f, 0, 0, nullptr);
// Set the viewport and scissor rect.
D3D12_VIEWPORT viewport = { 0.0f, 0.0f, /* width/height of your render texture */, D3D12_MIN_DEPTH, D3D12_MAX_DEPTH };
D3D12_RECT scissorRect = { 0, 0, /* width/height of your render texture */ };
m_commandList->RSSetViewports(1, &viewport);
m_commandList->RSSetScissorRects(1, &scissorRect);
// Tell helper we are starting the render
m_scene->BeginScene(m_commandList);
/* Do rendering to m_commandList */
m_scene->EndScene(m_commandList);
Here we've scheduled the transition to render target resource state, populated all the draw calls, and then inserted a barrier back to the pixel shader resource state. At that point, you can use the descriptor handle to your render-texture's SRV to render. As with all things DirectX 12, nothing happens until you actually close the command-list and submit it for execution.

Related

Applying HLSL Pixel Shaders to Win32 Screen Capture

A little background: I'm attempting to make a Windows (10) application which makes the screen look like an old CRT monitor, scanlines, blur, and all. I'm using this official Microsoft screen capture demo as a starting point: At this stage I can capture a window, and display it back in a new mouse-through window as if it were the original window.
I am attempting to use the CRT-Royale CRT shaders which are generally considered the best CRT shaders; these are available in .cg format. I transpile them with cgc to hlsl, then compile the hlsl files to compiled shader byte code with fxc. I am able to successfully load the compiled shaders and create the pixel shader. I then set the pixel shader in the d3d context. I then attempt to copy the capture surface frame to a pixel shader resource and set the created shaders resource. All of this builds and runs, but I do not see any difference in the output image and am not sure how to proceed. Below is the relevant code. I am not a c++ developer and am making this as a personal project which I plan on open sourcing once I have a primitive working version. Any advice is appreciated, thanks.
SimpleCapture::SimpleCapture(
IDirect3DDevice const& device,
GraphicsCaptureItem const& item)
{
m_item = item;
m_device = device;
// Set up
auto d3dDevice = GetDXGIInterfaceFromObject<ID3D11Device>(m_device);
d3dDevice->GetImmediateContext(m_d3dContext.put());
auto size = m_item.Size();
m_swapChain = CreateDXGISwapChain(
d3dDevice,
static_cast<uint32_t>(size.Width),
static_cast<uint32_t>(size.Height),
static_cast<DXGI_FORMAT>(DirectXPixelFormat::B8G8R8A8UIntNormalized),
2);
// ADDED THIS
HRESULT hr1 = D3DReadFileToBlob(L"crt-royale-first-pass-ps_4_0.fxc", &ps_1_buffer);
HRESULT hr = d3dDevice->CreatePixelShader(
ps_1_buffer->GetBufferPointer(),
ps_1_buffer->GetBufferSize(),
nullptr,
&ps_1
);
m_d3dContext->PSSetShader(
ps_1,
nullptr,
0
);
// END OF ADDED CHANGES
// Create framepool, define pixel format (DXGI_FORMAT_B8G8R8A8_UNORM), and frame size.
m_framePool = Direct3D11CaptureFramePool::Create(
m_device,
DirectXPixelFormat::B8G8R8A8UIntNormalized,
2,
size);
m_session = m_framePool.CreateCaptureSession(m_item);
m_lastSize = size;
m_frameArrived = m_framePool.FrameArrived(auto_revoke, { this, &SimpleCapture::OnFrameArrived });
}
void SimpleCapture::OnFrameArrived(
Direct3D11CaptureFramePool const& sender,
winrt::Windows::Foundation::IInspectable const&)
{
auto newSize = false;
{
auto frame = sender.TryGetNextFrame();
auto frameContentSize = frame.ContentSize();
if (frameContentSize.Width != m_lastSize.Width ||
frameContentSize.Height != m_lastSize.Height)
{
// The thing we have been capturing has changed size.
// We need to resize our swap chain first, then blit the pixels.
// After we do that, retire the frame and then recreate our frame pool.
newSize = true;
m_lastSize = frameContentSize;
m_swapChain->ResizeBuffers(
2,
static_cast<uint32_t>(m_lastSize.Width),
static_cast<uint32_t>(m_lastSize.Height),
static_cast<DXGI_FORMAT>(DirectXPixelFormat::B8G8R8A8UIntNormalized),
0);
}
{
auto frameSurface = GetDXGIInterfaceFromObject<ID3D11Texture2D>(frame.Surface());
com_ptr<ID3D11Texture2D> backBuffer;
check_hresult(m_swapChain->GetBuffer(0, guid_of<ID3D11Texture2D>(), backBuffer.put_void()));
// ADDED THIS
D3D11_TEXTURE2D_DESC txtDesc = {};
txtDesc.MipLevels = txtDesc.ArraySize = 1;
txtDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
txtDesc.SampleDesc.Count = 1;
txtDesc.Usage = D3D11_USAGE_IMMUTABLE;
txtDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
auto d3dDevice = GetDXGIInterfaceFromObject<ID3D11Device>(m_device);
ID3D11Texture2D *tex;
d3dDevice->CreateTexture2D(&txtDesc, NULL,
&tex);
frameSurface.copy_to(&tex);
d3dDevice->CreateShaderResourceView(
tex,
nullptr,
srv_1
);
auto texture = srv_1;
m_d3dContext->PSSetShaderResources(0, 1, texture);
// END OF ADDED CHANGES
m_d3dContext->CopyResource(backBuffer.get(), frameSurface.get());
}
}
DXGI_PRESENT_PARAMETERS presentParameters = { 0 };
m_swapChain->Present1(1, 0, &presentParameters);
... // Truncated
Shaders define how things are drawn. However, you don't draw anything - you just copy, which is why the shader doesn't do anything.
What you should do is to remove the CopyResource call, and instead draw a full screen quad on the back buffer (Which requires you to create a vertex buffer that you can bind, then set the back buffer as render target, and finally call Draw/DrawIndexed to actually render something, which then will invoke the shader).
Also - since I'm not sure whether you already do this and just stripped it from the shown code - functions like CreatePixelShader don't return HRESULTs just for the fun of it - you should check what is actually returned, because DirectX silently returns most errors and expects you to handle them, instead of crashing your program.

SDL2 load certain texture make SDL_RenderFillRect color weird

I made a program that has two different states, one is for menu display-"Menu State", and the other state is for drawing some stuff-"Draw State".
But I came across a weird thing, if i load certain png for texture and copy them to renderer to display , then leave "Menu State" to enter "Draw State". The texture will somehow make the rectangle color not display properly (for example make green go dark).
In my code, switching to a new state(invoke MenuState::onExit()) will erase the texture map(map of texture smart pointer indexing with std::string)
So the texutre loaded doesn't even exist in the "Drawing State".
I couldn't figure out what went wrong. Here is some of my codes
void TextureManager::DrawPixel(int x, int y, int width, int height, SDL_Renderer *pRenderer)
{
SDL_Rect rect;
rect.x = x;
rect.y = y;
rect.w = width;
rect.h = height;
SDL_SetRenderDrawColor(pRenderer, 0, 255, 0, 255);//same color value
SDL_RenderFillRect(pRenderer, &rect);
}
static bool TextureManagerLoadFile(std::string filename, std::string id)
{
return TextureManager::Instance().Load(filename, id, Game::Instance().GetRenderer());
}
bool TextureManager::Load(std::string filename, std::string id, SDL_Renderer *pRenderer)
{
if(m_textureMap.count(id) != 0)
{
return false;
}
SDL_Surface *pTempSurface = IMG_Load(filename.c_str());
SDL_Texture *pTexutre = SDL_CreateTextureFromSurface(pRenderer, pTempSurface);
SDL_FreeSurface(pTempSurface);
if(pTexutre != 0)
{
m_textureMap[id] = std::make_unique<textureData>(pTexutre, 0, 0);
SDL_QueryTexture(pTexutre, NULL, NULL, &m_textureMap[id]->width, &m_textureMap[id]->height);
return true;
}
return false;
}
void TextureManager::ClearFromTextureMap(std::string textureID)
{
m_textureMap.erase(textureID);
}
bool MenuState::onEnter()
{
if(!TextureManagerLoadFile("assets/Main menu/BTN PLAY.png", "play_button"))
{
return false;
}
if(!TextureManagerLoadFile("assets/Main menu/BTN Exit.png", "exit_button"))
//replace different png file here will also affect the outcome
{
return false;
}
if(!TextureManagerLoadFile("assets/Main menu/BTN SETTINGS.png", "setting_button"))
{
return false;
}
int client_w,client_h;
SDL_GetWindowSize(Game::Instance().GetClientWindow(),&client_w, &client_h);
int playBtn_w = TextureManager::Instance().GetTextureWidth("play_button");
int playBtn_h = TextureManager::Instance().GetTuextureHeight("play_button");
int center_x = (client_w - playBtn_w) / 2;
int center_y = (client_h - playBtn_h) / 2;
ParamsLoader pPlayParams(center_x, center_y, playBtn_w, playBtn_h, "play_button");
int settingBtn_w = TextureManager::Instance().GetTextureWidth("setting_button");
int settingBtn_h = TextureManager::Instance().GetTuextureHeight("setting_button");
ParamsLoader pSettingParams(center_x , center_y + (playBtn_h + settingBtn_h) / 2, settingBtn_w, settingBtn_h, "setting_button");
int exitBtn_w = TextureManager::Instance().GetTextureWidth("exit_button");
int exitBtn_h = TextureManager::Instance().GetTuextureHeight("exit_button");
ParamsLoader pExitParams(10, 10, exitBtn_w, exitBtn_h, "exit_button");
m_gameObjects.push_back(std::make_shared<MenuUIObject>(&pPlayParams, s_menuToPlay));
m_gameObjects.push_back(std::make_shared<MenuUIObject>(&pSettingParams, s_menuToPlay));
m_gameObjects.push_back(std::make_shared<MenuUIObject>(&pExitParams, s_menuExit));
//change order of the 3 line code above
//or replace different png for exit button, will make the rectangle color different
std::cout << "Entering Menu State" << std::endl;
return true;
}
bool MenuState::onExit()
{
for(auto i : m_gameObjects)
{
i->Clean();
}
m_gameObjects.clear();
TextureManager::Instance().ClearFromTextureMap("play_button");
TextureManager::Instance().ClearFromTextureMap("exit_button");
TextureManager::Instance().ClearFromTextureMap("setting_button");
std::cout << "Exiting Menu State" << std::endl;
return true;
}
void Game::Render()
{
SDL_SetRenderDrawColor(m_pRenderer, 255, 255, 255, 255);
SDL_RenderClear(m_pRenderer);
m_pGameStateMachine->Render();
SDL_RenderPresent(m_pRenderer);
}
Menu State Figure
Correct Color
Wrong Color
edit :Also, I found out that this weird phenomenon only happens when the renderer was created with 'SDL_RENDERER_ACCELERATED' flag and -1 or 0 driver index, i.e SDL_CreateRenderer(m_pWindow, 1, SDL_RENDERER_ACCELERATED); or SDL_CreateRenderer(m_pWindow, -1, SDL_RENDERER_SOFTWARE);works fine!
I have been plagued by this very same issue. The link provided by ekodes is how I resolved it, as order of operations had no effect for me.
I was able to pull the d3d9Device via SDL_RenderGetD3D9Device(), then SetTextureStageState as described in ekodes d3d blending link.
I was having the same issue. I got a vibrant green color when trying to render a light gray.
The combination of the parameters that are fixing the issue for you pertain to the driver to be used. -1 selects the first driver that meets the criteria, int this case it needs to be accelerated.
Using SDL_GetRendererInfo I was able to see this happens when using the "direct3d" driver.
I found this question talking about blending in direct3d.
I figured it out eventually. In addition to Alpha Blending there is a Color Blending. So DirectX merges color of the last texture with the last primitive.
The question does provide a fix for this in DirectX, however I'm not sure how to apply that it in regards to SDL. I also have not been able to find a solution for this problem in SDL.
I was drawing Green text with SDL_ttf, which uses a texture. Then drawing a gray rectangle for another component elsewhere on the screen.
What's strange is it doesn't seem to happen all the time. However, mine seems to predominantly happen with SDL_ttf. At first I thought it may be a byproduct of TTF_RenderText_Blended however, it happens with the other ones as well. It also does not appear to be affected by the blend mode of the Texture generated by those functions
So in my case, I was able to change the order of the operations to get the correct color.
Alternatively, using the OpenGL driver appeared to fix this as well. Similar to what you mentioned. (This was driver index 1 for me)
I'm not sure this classifies as an "Answer" but hopefully it helps someone out or points them in the right direction.

Sequential off-screen rendering / screen capture without windowing system using OpenSceneGraph

I am working currently on an off-screen renderer so that I can do Mutual Information Registration for real-world scenes. I use OpenSceneGraph to cope with the large data and automatic loading. I am having trouble getting a framebuffer capture within a sequential, single-threaded program.
Well, I have this class (header):
#include <osg/ref_ptr>
#include <osg/Array>
#include <osg/ImageUtils>
#include <osgGA/StateSetManipulator>
#include <osgViewer/Viewer>
#include <osg/GraphicsContext>
#include <osg/Texture2D>
#include <osg/FrameBufferObject>
#include <osgDB/WriteFile>
#include <osg/Referenced>
#include <osg/Vec3>
#include <osg/Image>
#include <osg/State>
#include <string>
#include <chrono>
#include <thread>
#include <assert.h>
#include "ImagingPrimitives.h"
class BoundRenderScene {
public:
BoundRenderScene();
virtual ~BoundRenderScene();
void NextFrame(void);
inline OpenThreads::Mutex* GetMutexObject(void) { return &_mutex; }
inline osg::Image* GetFrame(void)
{
OpenThreads::ScopedLock<OpenThreads::Mutex> lock(_mutex);
return _frame.get();
}
inline void GetFrame(osg::Image* img)
{
OpenThreads::ScopedLock<OpenThreads::Mutex> lock(_mutex);
if(_frame.valid() && (img!=NULL) && img->valid())
{
glReadBuffer(GL_BACK);
img->readPixels(0,0,_camera_configuration->GetSX(),_camera_configuration->GetSY(), GL_RGB,GL_UNSIGNED_BYTE);
uint w = img->s(), h = img->t(), d = img->r(), c = uint(img->getPixelSizeInBits()/8);
/*
* bare testing write op
* osgDB::writeImageFile(const_cast<const osg::Image&>(*img), "/tmp/testimg.png");
*/
}
}
inline void SetCameraConfiguration(CameraConfiguration* configuration) { _camera_configuration = configuration; }
inline void SetCameraMatrix(osg::Matrixd camera_matrix) { _camera_matrix = camera_matrix; }
inline void SetScene(osg::Node* scene) { _scene = scene; }
inline void Initialize(void) {
if(!_initialized)
_init();
else
_re_init();
}
protected:
osgViewer::Viewer _viewer;
osg::Matrixd _camera_matrix;
osg::ref_ptr<osg::Texture2D> _tex;
osg::ref_ptr<osg::FrameBufferObject> _fbo;
mutable osg::ref_ptr<osg::Image> _frame;
osg::ref_ptr<osg::Node> _scene;
osg::ref_ptr<osg::GraphicsContext::Traits> _traits;
osg::ref_ptr<osg::GraphicsContext> _gc;
CameraConfiguration* _camera_configuration;
SnapshotCallback* cb;
std::string _filepath;
private:
void _init(void);
void _re_init(void);
bool _initialized;
mutable OpenThreads::Mutex _mutex;
osg::Matrixd pre_transform;
osg::Matrixd transformation;
};
Also, because many examples within offscreen-rendering and for screen capture work with Post/FinalDrawCallaback's, I copied the callback structure from the "osgdistortion" example, but added the mutex for synchronisation:
struct SnapshotCallback : public osg::Camera::DrawCallback
{
public:
inline SnapshotCallback(OpenThreads::Mutex* mtx_obj, std::string filepath, int width, int height) : _filepath(filepath), _output_to_file(false), _mutex(mtx_obj)
{
_image = new osg::Image();
_image->allocateImage(width, height, 1, GL_RGB, GL_UNSIGNED_BYTE);
if(filepath!="")
_output_to_file = true;
}
inline virtual void operator() (osg::RenderInfo& renderInfo) const
{
OpenThreads::ScopedLock<OpenThreads::Mutex> lock(*_mutex);
osg::Camera* camera = renderInfo.getCurrentCamera();
osg::Viewport* viewport = camera ? camera->getViewport() : 0;
if(viewport && _image.valid())
{
glReadBuffer(GL_BACK);
_image->readPixels(int(viewport->x()),int(viewport->y()),int(viewport->width()),int(viewport->height()), GL_RGB, GL_UNSIGNED_BYTE);
if(_output_to_file)
{
osgDB::writeImageFile(*_image, _filepath);
}
}
}
inline virtual void operator() (const osg::Camera& camera) const
{
OpenThreads::ScopedLock<OpenThreads::Mutex> lock(*_mutex);
osg::Viewport* viewport = camera.getViewport();
if(viewport && _image.valid())
{
glReadBuffer(GL_BACK);
_image->readPixels(int(viewport->x()),int(viewport->y()),int(viewport->width()),int(viewport->height()), GL_RGB, GL_UNSIGNED_BYTE);
if(_output_to_file)
{
osgDB::writeImageFile(*_image, _filepath);
}
}
}
std::string _filepath;
bool _output_to_file;
mutable OpenThreads::Mutex* _mutex;
mutable osg::ref_ptr<osg::Image> _image;
};
I initialize and render the scene as follows:
#include "BoundRenderScene.h"
void BoundRenderScene::_init(void)
{
if(_camera!=NULL)
_viewer.setDone(true);
_traits->x = 0;
_traits->y = 0;
_traits->width = _camera_configuration->GetSX();
_traits->height = _camera_configuration->GetSY();
_traits->red = 8;
_traits->green = 8;
_traits->blue = 8;
_traits->alpha = 0;
_traits->depth = 24;
_traits->windowDecoration = false;
_traits->pbuffer = true;
_traits->doubleBuffer = true;
_traits->sharedContext = 0x0;
if(_gc.get()!=NULL)
{
bool release_success = _gc->releaseContext();
if(!release_success)
std::cerr << "Error releasing Graphics Context.";
}
_gc = osg::GraphicsContext::createGraphicsContext(_traits.get());
_viewer.getCamera()->setGraphicsContext(_gc.get());
_viewer.setThreadingModel(osgViewer::Viewer::SingleThreaded);
_viewer.setUpThreading();
_viewer.realize();
_frame->allocateImage(_camera_configuration->GetSX(), _camera_configuration->GetSY(), 1, GL_RGB, GL_UNSIGNED_BYTE);
_viewer.getCamera()->getOrCreateStateSet();
_viewer.getCamera()->setRenderTargetImplementation(osg::Camera::PIXEL_BUFFER);
cb = new SnapshotCallback(&_mutex,_filepath, _camera_configuration->GetSX(), _camera_configuration->GetSY());
//_viewer.getCamera()->setPostDrawCallback( cb );
//Clear colour "black" for representing "no information" => background elimination in natural image, pls.
_viewer.getCamera()->setClearColor(osg::Vec4f(0.25f, 0.25f, 0.25f, 1.0f));
_viewer.getCamera()->setClearMask(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);
_viewer.getCamera()->setDrawBuffer(GL_BACK);
_viewer.getCamera()->setReadBuffer(GL_BACK);
_viewer.getCamera()->setViewport(0,0,_camera_configuration->GetSX(),_camera_configuration->GetSY());
_viewer.getCamera()->setProjectionMatrix(osg::Matrixd::perspective(osg::RadiansToDegrees(_camera_configuration->GetFoV()), _camera_configuration->GetAspectRatio(), 0.1, 150.0));
//looking in geo-coord system
_viewer.getCamera()->setViewMatrix(osg::Matrixd::lookAt(osg::Vec3d(0.0, 0.0, -1.0), osg::Vec3d(0.0, 0.0, 1.0), osg::Vec3d(0.0, 1.0, 0.0)));
_viewer.getCamera()->attach(osg::Camera::COLOR_BUFFER, _frame.get());
_viewer.getCamera()->setRenderTargetImplementation(osg::Camera::FRAME_BUFFER_OBJECT);
_tex->setTextureSize(_camera_configuration->GetSX(), _camera_configuration->GetSY());
_tex->setInternalFormat(GL_RGB);
_tex->setFilter(osg::Texture::MIN_FILTER, osg::Texture::LINEAR);
_tex->setFilter(osg::Texture::MAG_FILTER, osg::Texture::LINEAR);
_tex->setWrap(osg::Texture::WRAP_S, osg::Texture::CLAMP_TO_EDGE);
_tex->setWrap(osg::Texture::WRAP_T, osg::Texture::CLAMP_TO_EDGE);
_tex->setResizeNonPowerOfTwoHint(false);
_tex->setImage(0,_frame.get());
_fbo->setAttachment(osg::Camera::COLOR_BUFFER, osg::FrameBufferAttachment(_tex.get()));
_viewer.setDone(false);
_viewer.setSceneData(_scene.get());
_viewer.setCameraManipulator(0x0);
}
void BoundRenderScene::NextFrame(void)
{
OpenThreads::ScopedLock<OpenThreads::Mutex> lock(_mutex);
if(_frame.valid() && !_viewer.done())
{
osg::Matrixd inverse_cam = osg::Matrixd::inverse(_camera_matrix);
transformation = inverse_cam * pre_transform;
_viewer.getCamera()->setViewMatrix(transformation);
_viewer.updateTraversal();
_viewer.frame();
}
else
std::cout << "Viewer or Camera invalid." << std::endl;
}
The main workflow looks like this (simplified):
BoundRenderScene renderer;
std::vector<osg::Matrixd> poses;
/*
* setting initial parameters
* fill poses with camera positions to render, for regsitration
*/
renderer._init();
for(uint i = 0; i < poses.size(); i++)
{
renderer.SetCameraMatrix(poses.at(i));
renderer.NextImage();
sleep(0.04); // to get the 25fps frame limit
osg::Image* reg_image = renderer.GetImage();
/*
* Do further processing
*/
}
Now comes the crux: the OpenSceneGraph example "osgprenderer" (included in OSG) does off-screen rendering using an osg::Camera::DrawCallback, as my SnapshotCallback. Unfortunately, the operator()-function in my case never get's called in my scenegraph, so that way of screen capture doesn't work for me. It's also rather inconvenient as the rest of the Mutual Information procedure is a rather sequential pipeline.
Other wrappers (https://github.com/xarray/osgRecipes/blob/master/integrations/osgberkelium/osgberkelium.cpp) use methods similar to my "void GetFrame(osg::Image* img)" method, where the image is actively read using "readPixels". That is very convenient for my workflow, but the method always returns a blank image. It doesn't crash, but it doesn't do it's job either.
The method that does work is "osg: and :Image* GetFrame(void)", which returns the bound/attached FBO image. It is similar to the "osgdistortion" example. It does work for rendering one- to two images, but after some time, rendering and processing get out of sync and the application crashes as follows:
[---FIRST FRAME---]
GraphicsCostEstimator::calibrate(..)
cull_draw() 0x1998ca0
ShaderComposer::~ShaderComposer() 0x35a4d40
Renderer::compile()
OpenGL extension 'GL_ARB_vertex_buffer_object' is supported.
OpenGL extension 'GL_EXT_secondary_color' is supported.
OpenGL extension 'GL_EXT_fog_coord' is supported.
OpenGL extension '' is not supported.
OpenGL extension 'GL_EXT_packed_depth_stencil' is supported.
Setting up osg::Camera::FRAME_BUFFER_OBJECT
end cull_draw() 0x1998ca0
[processing]
[ SECOND FRAME ]
cull_draw() 0x1998ca0
OpenGL extension 'GL_ARB_fragment_program' is supported.
OpenGL extension 'GL_ARB_vertex_program' is supported.
OpenGL extension 'GL_ARB_shader_objects' is supported.
OpenGL extension 'GL_ARB_vertex_shader' is supported.
OpenGL extension 'GL_ARB_fragment_shader' is supported.
OpenGL extension 'GL_ARB_shading_language_100' is supported.
OpenGL extension 'GL_EXT_geometry_shader4' is supported.
OpenGL extension 'GL_EXT_gpu_shader4' is supported.
OpenGL extension 'GL_ARB_tessellation_shader' is supported.
OpenGL extension 'GL_ARB_uniform_buffer_object' is supported.
OpenGL extension 'GL_ARB_get_program_binary' is supported.
OpenGL extension 'GL_ARB_gpu_shader_fp64' is supported.
OpenGL extension 'GL_ARB_shader_atomic_counters' is supported.
glVersion=4.5, isGlslSupported=YES, glslLanguageVersion=4.5
Warning: detected OpenGL error 'invalid operation' at end of SceneView::draw()
end cull_draw() 0x1998ca0
[-FROM 3rd FRAME ONWARDS-]
[workload, matrix setup]
[_viewer.frame()]
cull_draw() 0x1998ca0
Warning: detected OpenGL error 'invalid operation' at start of State::apply()
end cull_draw() 0x1998ca0
[next frame]
[BREAKING]
cull_draw() 0x1998ca0
Warning: detected OpenGL error 'invalid operation' at start of State::apply()
end cull_draw() 0x1998ca0
[more work]
Segmentation fault (core dumped)
So, the question is:
I had a look into the source files from osg for the Viewer-related classes, but I was not able to determine where the error
Warning: detected OpenGL error 'invalid operation' at start of State::apply()
comes from. Any idea where to start looking for it ?
For sequential rendering and screen capture, which method is the best to use within OSG ?
How can I obtain the mutex of the normal osg::Viewer, so to sync the renderer with the rest of py pipeline ? (Renderer is single-threaded)
Any other suggestions from experiences OpenSceneGraph off-screen
renderers and screen captures ?
As deeper research turned out, releasing the graphics context in the class destructor freed the OpenGL pipeline, BUT: it also disallocated stateset-bound textures of the loaded scene/model, although the model itself was not suspended (as given in the question: it is re-used in the following passes). So, in further render passes, the render pipeline wanted to access OSG assets which have been released via releasing the GL context.
in code it changed from:
BoundRenderScene::~BoundRenderScene() {
// TODO Auto-generated destructor stub
_viewer.setDone(true);
_viewer.setReleaseContextAtEndOfFrameHint(true);
_gc->releaseContext();
#ifdef DEBUG
std::cout << "BoundRenderScene deleted." << std::endl;
#endif
}
to:
BoundRenderScene::~BoundRenderScene() {
// TODO Auto-generated destructor stub
_viewer.setDone(true);
_viewer.setReleaseContextAtEndOfFrameHint(true);
#ifdef DEBUG
std::cout << "BoundRenderScene deleted." << std::endl;
#endif
}
This resolved the OpenSceneGraph-internal error messages. Now, in order to solve the frame capture problem itself, I implemented the callback from osgprenderer:
struct SnapshotCallback : public osg::Camera::DrawCallback
{
public:
inline SnapshotCallback(std::string filepath) : _filepath(filepath), _output_to_file(false), _image(NULL)
{
if(filepath!="")
_output_to_file = true;
_image = new osg::Image();
}
inline virtual void operator() (osg::RenderInfo& renderInfo) const
{
osg::Camera* camera = renderInfo.getCurrentCamera();
osg::Viewport* viewport = camera ? camera->getViewport() : 0;
if(viewport)
{
glReadBuffer(camera->getDrawBuffer());
_image->allocateImage(int(viewport->width()), int(viewport->height()), 1, GL_RGB, GL_UNSIGNED_BYTE);
_image->readPixels(int(viewport->x()),int(viewport->y()),int(viewport->width()),int(viewport->height()), GL_RGB, GL_UNSIGNED_BYTE);
if(_output_to_file)
{
osgDB::writeImageFile(*reinterpret_cast<osg::Image*>(_image->clone(osg::CopyOp::DEEP_COPY_ALL)), _filepath);
}
}
}
inline virtual void operator() (const osg::Camera& camera) const
{
osg::Viewport* viewport = camera.getViewport();
if(viewport)
{
glReadBuffer(camera.getDrawBuffer());
_image->allocateImage(int(viewport->width()), int(viewport->height()), 1, GL_RGB, GL_UNSIGNED_BYTE);
_image->readPixels(int(viewport->x()),int(viewport->y()),int(viewport->width()),int(viewport->height()), GL_RGB, GL_UNSIGNED_BYTE);
if(_output_to_file)
{
osgDB::writeImageFile(*reinterpret_cast<osg::Image*>(_image->clone(osg::CopyOp::DEEP_COPY_ALL)), _filepath);
}
}
}
inline osg::Image* GetImage(void)
{
return reinterpret_cast<osg::Image*>(_image->clone(osg::CopyOp::DEEP_COPY_ALL));
}
protected:
std::string _filepath;
bool _output_to_file;
mutable osg::ref_ptr<osg::Image> _image;
};
Now, with the cloned buffer instead of the actual image buffer (idea taken over from osgscreencapture example), I do get the real image without memory errors.
For double-buffered rendering, I though have to somehow render the scene twice for the right buffer to contain the objects' images, but this is for my use case currently less of an issue (I/O-bound rendering, not operation-bound).
so, the main function looks like follows:
BoundRenderScene renderer;
std::vector<osg::Matrixd> poses;
/*
* setting initial parameters
* fill poses with camera positions to render, for registration
*/
renderer._init();
for(uint i = 0; i < poses.size(); i++)
{
renderer.SetCameraMatrix(poses.at(i));
renderer.NextImage();
renderer.NextImage();
osg::Image* reg_image = renderer.GetImage();
/*
* Do further processing
*/
}

Kinect Facetracking C++ start up error

We are working with the Kinect to track faces for a schoolproject. We have set up Visual Studio 2012, and all the test programs are working correctly. However we are trying to run this code and it gives us an error. After many attempts to fix the code, it gives the following error:
"The application was unable to start correctly (0xc000007b).Click OK to close the application.
The good thing is that it's finally running. The bad thing is that the compiler doesn't throw any errors other than this vague error.
We are completely lost and we hope that someone can help us or point us into the right direction. Thanks in advance for helping us.
The code:
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <NuiApi.h>
#include <FaceTrackLib.h>
#include <NuiSensor.h>
using namespace std;
HANDLE rgbStream;
HANDLE depthStream;
INuiSensor* sensor;
#define width 640
#define height 480
bool initKinect() {
// Get a working kinect sensor
int numSensors;
if (NuiGetSensorCount(&numSensors) < 0 || numSensors < 1) return false;
if (NuiCreateSensorByIndex(0, &sensor) < 0) return false;
// Initialize sensor
sensor->NuiInitialize(NUI_INITIALIZE_FLAG_USES_DEPTH | NUI_INITIALIZE_FLAG_USES_COLOR);
sensor->NuiImageStreamOpen(
NUI_IMAGE_TYPE_COLOR, // Depth camera or rgb camera?
NUI_IMAGE_RESOLUTION_640x480, // Image resolution
0, // Image stream flags, e.g. near mode
2, // Number of frames to buffer
NULL, // Event handle
&rgbStream);
// --------------- END CHANGED CODE -----------------
return true;
}
BYTE* dataEnd;
USHORT* dataEndD;
void getKinectDataD(){
NUI_IMAGE_FRAME imageFrame;
NUI_LOCKED_RECT LockedRect;
if (sensor->NuiImageStreamGetNextFrame(rgbStream, 0, &imageFrame) < 0) return;
INuiFrameTexture* texture = imageFrame.pFrameTexture;
texture->LockRect(0, &LockedRect, NULL, 0);
const USHORT* curr = (const USHORT*)LockedRect.pBits;
const USHORT* dataEnding = curr + (width*height);
if (LockedRect.Pitch != 0)
{
const BYTE* curr = (const BYTE*)LockedRect.pBits;
dataEnd = (BYTE*)(curr + (width*height) * 4);
}
while (curr < dataEnding) {
// Get depth in millimeters
USHORT depth = NuiDepthPixelToDepth(*curr++);
dataEndD = (USHORT*)depth;
// Draw a grayscale image of the depth:
// B,G,R are all set to depth%256, alpha set to 1.
}
texture->UnlockRect(0);
sensor->NuiImageStreamReleaseFrame(rgbStream, &imageFrame);
}
// This example assumes that the application provides
// void* cameraFrameBuffer, a buffer for an image, and that there is a method
// to fill the buffer with data from a camera, for example
// cameraObj.ProcessIO(cameraFrameBuffer)
int main(){
initKinect();
// Create an instance of a face tracker
IFTFaceTracker* pFT = FTCreateFaceTracker();
if (!pFT)
{
// Handle errors
}
// Initialize cameras configuration structures.
// IMPORTANT NOTE: resolutions and focal lengths must be accurate, since it affects tracking precision!
// It is better to use enums defined in NuiAPI.h
// Video camera config with width, height, focal length in pixels
// NUI_CAMERA_COLOR_NOMINAL_FOCAL_LENGTH_IN_PIXELS focal length is computed for 640x480 resolution
// If you use different resolutions, multiply this focal length by the scaling factor
FT_CAMERA_CONFIG videoCameraConfig = { 640, 480, NUI_CAMERA_COLOR_NOMINAL_FOCAL_LENGTH_IN_PIXELS };
// Depth camera config with width, height, focal length in pixels
// NUI_CAMERA_COLOR_NOMINAL_FOCAL_LENGTH_IN_PIXELS focal length is computed for 320x240 resolution
// If you use different resolutions, multiply this focal length by the scaling factor
FT_CAMERA_CONFIG depthCameraConfig = { 320, 240, NUI_CAMERA_DEPTH_NOMINAL_FOCAL_LENGTH_IN_PIXELS };
// Initialize the face tracker
HRESULT hr = pFT->Initialize(&videoCameraConfig, &depthCameraConfig, NULL, NULL);
if (FAILED(hr))
{
// Handle errors
}
// Create a face tracking result interface
IFTResult* pFTResult = NULL;
hr = pFT->CreateFTResult(&pFTResult);
if (FAILED(hr))
{
// Handle errors
}
// Prepare image interfaces that hold RGB and depth data
IFTImage* pColorFrame = FTCreateImage();
IFTImage* pDepthFrame = FTCreateImage();
if (!pColorFrame || !pDepthFrame)
{
// Handle errors
}
// Attach created interfaces to the RGB and depth buffers that are filled with
// corresponding RGB and depth frame data from Kinect cameras
pColorFrame->Attach(640, 480, dataEnd, FTIMAGEFORMAT_UINT8_R8G8B8, 640 * 3);
pDepthFrame->Attach(320, 240, dataEndD, FTIMAGEFORMAT_UINT16_D13P3, 320 * 2);
// You can also use Allocate() method in which case IFTImage interfaces own their memory.
// In this case use CopyTo() method to copy buffers
FT_SENSOR_DATA sensorData;
sensorData.ZoomFactor = 1.0f; // Not used must be 1.0
bool isFaceTracked = false;
// Track a face
while (true)
{
// Call Kinect API to fill videoCameraFrameBuffer and depthFrameBuffer with RGB and depth data
getKinectDataD();
// Check if we are already tracking a face
if (!isFaceTracked)
{
// Initiate face tracking.
// This call is more expensive and searches the input frame for a face.
hr = pFT->StartTracking(&sensorData, NULL, NULL, pFTResult);
if (SUCCEEDED(hr))
{
isFaceTracked = true;
}
else
{
// No faces found
isFaceTracked = false;
}
}
else
{
// Continue tracking. It uses a previously known face position.
// This call is less expensive than StartTracking()
hr = pFT->ContinueTracking(&sensorData, NULL, pFTResult);
if (FAILED(hr))
{
// Lost the face
isFaceTracked = false;
}
}
// Do something with pFTResult like visualize the mask, drive your 3D avatar,
// recognize facial expressions
}
// Clean up
pFTResult->Release();
pColorFrame->Release();
pDepthFrame->Release();
pFT->Release();
return 0;
}
We figured it out we used the wrong dll indeed, it runs without errors now. But we ran in to an another problem, we have no clue how to use the pFTResult and retrieve the face angles with use of "getFaceRect". Does somebody know how?

Display Different images per monitor directX 10

I am fairly new to DirectX 10 programming, and I have been trying to do the following with my limited skills (though I have a strong background with OpenGL)
I am trying to display 2 different textured Quads, 1 per monitor. To do so, I understood that I need a single D3D10 Device, multiple (2) swap chains, and single VertexBuffer
While I think I'm able to create all of those, I'm still pretty unsure how to handle all of them. Do I need multiple ID3D10RenderTargetView(s) ? How and where should I Use OMSetRenderTargets(...) ?
Other than MSDN, documentation or explaination of those concepts are rather limited, so any help would be very welcome. Here is some code I have :
Here's the rendering code
for(int i = 0; i < screenNumber; i++){
//clear scene
pD3DDevice->ClearRenderTargetView( pRenderTargetView, D3DXCOLOR(0,1,0,0) );
//fill vertex buffer with vertices
UINT numVertices = 4;
vertex* v = NULL;
//lock vertex buffer for CPU use
pVertexBuffer->Map(D3D10_MAP_WRITE_DISCARD, 0, (void**) &v );
v[0] = vertex( D3DXVECTOR3(-1,-1,0), D3DXVECTOR4(1,0,0,1), D3DXVECTOR2(0.0f, 1.0f) );
v[1] = vertex( D3DXVECTOR3(-1,1,0), D3DXVECTOR4(0,1,0,1), D3DXVECTOR2(0.0f, 0.0f) );
v[2] = vertex( D3DXVECTOR3(1,-1,0), D3DXVECTOR4(0,0,1,1), D3DXVECTOR2(1.0f, 1.0f) );
v[3] = vertex( D3DXVECTOR3(1,1,0), D3DXVECTOR4(1,1,0,1), D3DXVECTOR2(1.0f, 0.0f) );
pVertexBuffer->Unmap();
// Set primitive topology
pD3DDevice->IASetPrimitiveTopology( D3D10_PRIMITIVE_TOPOLOGY_TRIANGLESTRIP );
//set texture
pTextureSR->SetResource( textureSRV[textureIndex] );
//get technique desc
D3D10_TECHNIQUE_DESC techDesc;
pBasicTechnique->GetDesc( &techDesc );
// This is where you actually use the shader code
for( UINT p = 0; p < techDesc.Passes; ++p )
{
//apply technique
pBasicTechnique->GetPassByIndex( p )->Apply( 0 );
//draw
pD3DDevice->Draw( numVertices, 0 );
}
//flip buffers
pSwapChain[i]->Present(0,0);
}
And here's the code for creating rendering targets, which I am not sure is good
for(int i = 0; i < screenNumber; ++i){
//try to get the back buffer
ID3D10Texture2D* pBackBuffer;
if ( FAILED( pSwapChain[1]->GetBuffer(0, __uuidof(ID3D10Texture2D), (LPVOID*) &pBackBuffer) ) ) return fatalError("Could not get back buffer");
//try to create render target view
if ( FAILED( pD3DDevice->CreateRenderTargetView(pBackBuffer, NULL, &pRenderTargetView) ) ) return fatalError("Could not create render target view");
pBackBuffer->Release();
pD3DDevice->OMSetRenderTargets(1, &pRenderTargetView, NULL);
}
return true;
}
I hope I got the gist of what you wish to do - render different content on two different monitors while using a single graphics card (graphics adapter) which maps its output to those monitors. For that, you're going to need one device (for the single graphics card/adapter) and enumerate just how many outputs there are at the user's machine.
So, in total - that means one device, two outputs, two windows and therefore - two swap chains.
Here's a quick result of my little experiment:
A little introduction
With DirectX 10+, this falls into the DXGI (DirectX Graphics Infrastructure) which manages the common low-level logistics involved with DirectX 10+ development which, as you probably know, dumped the old requirement of enumerating feature sets and the like - requiring every DX10+ capable card to share in on all of the features defined by the API. The only thing that varies is the extent and capability of the card (in other words, lousy performance is preferable to the app crashing and burning). This was all within DirectX 9 in the past, but people at Microsoft decided to pull it out and call it DXGI. Now, we can use DXGI functionality to set up our multi monitor environment.
Do I need multiple ID3D10RenderTargetView(s) ?
Yes, you do need multiple render target views, count depends (like the swap chains and windows) on the number of monitors you have. But, to save you from spewing words, let's write it out as simple as possible and additional information where it's needed:
Enumerate all adapters available on the system.
For each adapter, enumerate all outputs available (and active) and create a device to accompany it.
With the enumerated data stored in a suitable structure (think arrays which can quickly relinquish size information), use it to create n windows, swap chains, render target views, depth/stencil textures and their respective views where n is equal to the number of outputs.
With everything created, for each window you are rendering into, you can define special routines which will use the available geometry (and other) data to output your results - which resolves to what each monitor gets in fullscreen (don't forget to adjust the viewport for every window accordingly).
Present your data by iterating over every swap chain which is linked to its respective window and swap buffers with Present()
Now, while this is rich in word count, some code is worth a lot more. This is designed to give you a coarse idea of what goes into implementing a simple multimonitor application. So, assumptions are that there is only one adapter ( a rather bold statement nowadays ) and multiple outputs - and no failsafes. I'll leave the fun part to you. Answer to the second question is downstairs...
Do note there's no memory management involved. We assume everything magically gets cleaned up when it is not needed for illustration purposes. Be a good memory citizen.
Getting the adapter
IDXGIAdapter* adapter = NULL;
void GetAdapter() // applicable for multiple ones with little effort
{
// remember, we assume there's only one adapter (example purposes)
for( int i = 0; DXGI_ERROR_NOT_FOUND != factory->EnumAdapters( i, &adapter ); ++i )
{
// get the description of the adapter, assuming no failure
DXGI_ADAPTER_DESC adapterDesc;
HRESULT hr = adapter->GetDesc( &adapterDesc );
// Getting the outputs active on our adapter
EnumOutputsOnAdapter();
}
Acquiring the outputs on our adapter
std::vector<IDXGIOutput*> outputArray; // contains outputs per adapter
void EnumOutputsOnAdapter()
{
IDXGIOutput* output = NULL;
for(int i = 0; DXGI_ERROR_NOT_FOUND != adapter->EnumOutputs(i, &output); ++i)
{
// get the description
DXGI_OUTPUT_DESC outputDesc;
HRESULT hr = output->GetDesc( &outputDesc );
outputArray.push_back( output );
}
}
Now, I must assume that you're at least aware of the Win32 API considerations, creating window classes, registering with the system, creating windows, etc... Therefore, I will not qualify its creation, only elaborate how it pertains to multiple windows. Also, I will only consider the fullscreen case here, but creating it in windowed mode is more than possible and rather trivial.
Creating the actual windows for our outputs
Since we assume existence of just one adapter, we only consider the enumerated outputs linked to that particular adapter. It would be preferable to organize all window data in neat little structures, but for the purposes of this answer, we'll just shove them into a simple struct and then into yet another std::vector object, and by them I mean handles to respective windows (HWND) and their size (although for our case it's constant).
But still, we have to address the fact that we have one swap chain, one render target view, one depth/stencil view per window. So, why not feed all of that in that little struct which describes each of our windows? Makes sense, right?
struct WindowDataContainer
{
//Direct3D 10 stuff per window data
IDXGISwapChain* swapChain;
ID3D10RenderTargetView* renderTargetView;
ID3D10DepthStencilView* depthStencilView;
// window goodies
HWND hWnd;
int width;
int height;
}
Nice. Well, not really. But still... Moving on! Now to create the windows for outputs:
std::vector<WindowDataContainer*> windowsArray;
void CreateWindowsForOutputs()
{
for( int i = 0; i < outputArray.size(); ++i )
{
IDXGIOutput* output = outputArray.at(i);
DXGI_OUTPUT_DESC outputDesc;
p_Output->GetDesc( &outputDesc );
int x = outputDesc.DesktopCoordinates.left;
int y = outputDesc.DesktopCoordinates.top;
int width = outputDesc.DesktopCoordinates.right - x;
int height = outputDesc.DesktopCoordinates.bottom - y;
// Don't forget to clean this up. And all D3D COM objects.
WindowDataContainer* window = new WindowDataContainer;
window->hWnd = CreateWindow( windowClassName,
windowName,
WS_POPUP,
x,
y,
width,
height,
NULL,
0,
instance,
NULL );
// show the window
ShowWindow( window->hWnd, SW_SHOWDEFAULT );
// set width and height
window->width = width;
window->height = height;
// shove it in the std::vector
windowsArray.push_back( window );
//if first window, associate it with DXGI so it can jump in
// when there is something of interest in the message queue
// think fullscreen mode switches etc. MSDN for more info.
if(i == 0)
factory->MakeWindowAssociation( window->hWnd, 0 );
}
}
Cute, now that's done. Since we only have one adapter and therefore only one device to accompany it, create it as usual. In my case, it's simply a global interface pointer which can be accessed all over the place. We are not going for code of the year here, so why the hell not, eh?
Creating the swap chains, views and the depth/stencil 2D texture
Now, our friendly swap chains... You might be used to actually creating them by invoking the "naked" function D3D10CreateDeviceAndSwapChain(...), but as you know, we've already made our device. We only want one. And multiple swap chains. Well, that's a pickle. Luckily, our DXGIFactory interface has swap chains on its production line which we can receive for free with complementary kegs of rum. Onto the swap chains then, create for every window one:
void CreateSwapChainsAndViews()
{
for( int i = 0; i < windowsArray.size(); i++ )
{
WindowDataContainer* window = windowsArray.at(i);
// get the dxgi device
IDXGIDevice* DXGIDevice = NULL;
device->QueryInterface( IID_IDXGIDevice, ( void** )&DXGIDevice ); // COM stuff, hopefully you are familiar
// create a swap chain
DXGI_SWAP_CHAIN_DESC swapChainDesc;
// fill it in
HRESULT hr = factory->CreateSwapChain( DXGIDevice, &swapChainDesc, &p_Window->swapChain );
DXGIDevice->Release();
DXGIDevice = NULL;
// get the backbuffer
ID3D10Texture2D* backBuffer = NULL;
hr = window->swapChain->GetBuffer( 0, IID_ID3D10Texture2D, ( void** )&backBuffer );
// get the backbuffer desc
D3D10_TEXTURE2D_DESC backBufferDesc;
backBuffer->GetDesc( &backBufferDesc );
// create the render target view
D3D10_RENDER_TARGET_VIEW_DESC RTVDesc;
// fill it in
device->CreateRenderTargetView( backBuffer, &RTVDesc, &window->renderTargetView );
backBuffer->Release();
backBuffer = NULL;
// Create depth stencil texture
ID3D10Texture2D* depthStencil = NULL;
D3D10_TEXTURE2D_DESC descDepth;
// fill it in
device->CreateTexture2D( &descDepth, NULL, &depthStencil );
// Create the depth stencil view
D3D10_DEPTH_STENCIL_VIEW_DESC descDSV;
// fill it in
device->CreateDepthStencilView( depthStencil, &descDSV, &window->depthStencilView );
}
}
We now have everything we need. All that you need to do is define a function which iterates over all windows and draws different stuff appropriately.
How and where should I Use OMSetRenderTargets(...) ?
In the just mentioned function which iterates over all windows and uses the appropriate render target (courtesy of our per-window data container):
void MultiRender( )
{
// Clear them all
for( int i = 0; i < windowsArray.size(); i++ )
{
WindowDataContainer* window = windowsArray.at(i);
// There is the answer to your second question:
device->OMSetRenderTargets( 1, &window->renderTargetView, window->depthStencilView );
// Don't forget to adjust the viewport, in fullscreen it's not important...
D3D10_VIEWPORT Viewport;
Viewport.TopLeftX = 0;
Viewport.TopLeftY = 0;
Viewport.Width = window->width;
Viewport.Height = window->height;
Viewport.MinDepth = 0.0f;
Viewport.MaxDepth = 1.0f;
device->RSSetViewports( 1, &Viewport );
// TO DO: AMAZING STUFF PER WINDOW
}
}
Of course, don't forget to run through all the swap chains and swap buffers per window basis. The code here is just for the purposes of this answer, it requires a bit more work, error checking (failsafes) and contemplation to get it working just the way you like it - in other words - it should give you a simplified overview, not a production solution.
Good luck and happy coding! Sheesh, this is huge.