I have an MFC dialog window where I added a WebBrowser control (that encapsulates the Internet Explorer engine.)
The goal of the following code is to render the contents of the said web browser control into a device context that I can later use for printing:
//MFC code, error checks are omitted for brevity
//'m_browser' = is a web browser control of type `CExplorer1`
IDispatch* pHtmlDoc = m_browser.get_Document();
CComPtr<IHTMLDocument2> pHtmlDocument2;
pHtmlDoc->QueryInterface(IID_IHTMLDocument2, (void**)&pHtmlDocument2));
//Get IViewObject2 for the entire document that we will use to render into a DC
CComPtr<IViewObject2> pViewObject;
pHtmlDocument2->QueryInterface(IID_IViewObject2, (void **)&pViewObject));
CComPtr<IHTMLElement> pBody;
pHtmlDocument2->get_body(&pBody));
CComPtr<IHTMLElement2> pBody2;
pBody->QueryInterface(IID_IHTMLElement2, (void **)&pBody2));
//Get default printer DC
CPrintDialog pd(TRUE, PD_ALLPAGES | PD_USEDEVMODECOPIES | PD_NOPAGENUMS | PD_HIDEPRINTTOFILE | PD_NOSELECTION);
pd.m_pd.Flags |= PD_RETURNDC | PD_RETURNDEFAULT;
pd.DoModal(); //corrected later
HDC hPrintDC = pd.CreatePrinterDC();
//Calc bitmap width based on printer DC specs
//Note that this width will be larger than
//the width of the WebControl window itself due to
//printer's much higher DPI setting...
int n_bitmapWidth = ::GetDeviceCaps(hPrintDC, HORZRES); //Use entire printable area
//Get full size of the document
long n_scrollWidth;
long n_scrollHeight;
pBody2->get_scrollWidth(&n_scrollWidth);
pBody2->get_scrollHeight(&n_scrollHeight);
//Calc proportional size of the bitmap in the DC to render
int nWidth = n_bitmapWidth;
int nHeight = n_bitmapWidth * n_scrollHeight / n_scrollWidth;
//Create memory DC to render into
HDC hDc = ::GetDC(hWnd);
HDC hCompDc = ::CreateCompatibleDC(hDC);
//I'm using a raw DIB section here as I'll need to access
//its bitmap bits directly later in my code...
BITMAPINFOHEADER infoHeader = {0};
infoHeader.biSize = sizeof(infoHeader);
infoHeader.biWidth = nWidth;
infoHeader.biHeight = -nHeight;
infoHeader.biPlanes = 1;
infoHeader.biBitCount = 24;
infoHeader.biCompression = BI_RGB;
BITMAPINFO info;
info.bmiHeader = infoHeader;
//Create a bitmap as DIB section of size `nWidth` by `nHeight` pixels
BYTE* pMemory = 0;
HBITMAP hBitmap = ::CreateDIBSection(hDc, &info, DIB_RGB_COLORS, (void**)&pMemory, 0, 0);
HBITMAP hOldBmp = (HBITMAP)::SelectObject(hCompDc, hBitmap);
RECT rcAll = {0, 0, nWidth, nHeight};
::FillRect(hCompDc, &rcAll, (HBRUSH)::GetStockObject(WHITE_BRUSH));
RECTL rectPrnt = {0, 0, nWidth, nHeight};
//Do the upscaling & render -- note that IE8 fails to render it here!!!!
pViewObject->Draw(DVASPECT_CONTENT, //DVASPECT_DOCPRINT
-1, NULL, NULL, NULL, hCompDc,
&rectPrnt,
NULL,
NULL, 0));
::SelectObject(hCompDc, hOldBmp);
//Now the bitmap in `hCompDc` contains the resulting pixels
//For debugging purposes, save it as .bmp file
BITMAPFILEHEADER fileHeader = {0};
fileHeader.bfType = 0x4d42;
fileHeader.bfSize = 0;
fileHeader.bfReserved1 = 0;
fileHeader.bfReserved2 = 0;
fileHeader.bfOffBits = sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
CFile file(
L"path-to\\test.bmp",
CFile::modeCreate | CFile::modeReadWrite | CFile::shareDenyNone);
file.Write((char*)&fileHeader, sizeof(fileHeader));
file.Write((char*)&infoHeader, sizeof(infoHeader));
int bytes = (((24 * nWidth + 31) & (~31)) / 8) * nHeight;
file.Write(pMemory, bytes);
//Clean up
::DeleteObject(hBitmap);
::DeleteDC(hCompDc);
::ReleaseDC(hWnd, hDc);
::DeleteDC(hPrintDC);
This code works fine if I have the latest IE11 installed on my development machine. But if, for instance, someone has IE8 installed on their Windows 7, the IViewObject::Draw method will render only a small part of the document (equal to the size of the web browser control itself.)
The best way to describe it is to illustrate it with the examples:
Normally rendered test page with IE11 installed:
and here's what happens with IE8 installed:
Does anyone have any idea what am I doing wrong here that IE8 doesn't like?
EDIT1: Did some more digging into the IViewObject::Draw function with WinDbg and then found the source code for it. Here's CServer::Draw() that is IViewObject::Draw, and then CDoc::Draw() that is called internally from CServer::Draw().
First, thanks for the interesting question. While not so practical - not a lot of people use IE8 today - it was not so trivial to solve. I'll describe what is the problem and provide a simplistic but working solution that you can improve.
Before I go into IE8 solution, a couple of points:
The solution with window resize to fit scroll size is not stable if you can encounter large documents. If you don't know what to anticipate, you need to refactor the solution for later explorers as well, to avoid relying on resizing window to scroll size.
Why carry giant bitmap? Metafiles etc. may fit better. Any large enough page at this resolution is going to blow memory on PC with the naive DIB creation. Google page in provided sample renders to 100Mb bitmap file while emf, from which rasterization is done, takes less than 1Mb.
While I don't know exact requirements and limitations of your project, I'm 99% sure that drawing into gigantic DIB is not the best solution. Even EMF, while better, is not the best either. If you need, for example, add a signature and then print, there are better ways to handle this. This, of course, just a side note, not related to the question itself.
IE8 rendering problem
In IE8 renderer there is a bug. Draw() will be clipped at pixel dimensions of actual display area (the visible rectangle you see is the original display area in the scale of rendering context).
So, if you have a scaled target that is larger than the actual size, while scaled, it will be clipped to the size in scaled pixels anyway (so it has much less content than original rectangle).
If it doesn't clip for somebody on genuine IE8, then there are remainders of later IE in the system or there is other non-scratch setup, system update or alike.
Workaround possibilities
Good news it is possible to workaround, bad news workarounds are a bit nasty.
First, it is still possible to workaround with IViewObject. But, because there is arbitrary scaling involved and accessible source rectangle is very small, this solution has some complexities that I think are beyong an SO answer. So I would not dive into this path.
Instead, we can render through another, now outdated API: IHTMLElementRender. It allows to render the page with DrawToDC into arbitrary context. Unfortunately, it is not so simple as it may appear and goes beyond just providing device context.
First, there is similar bug of clipping. It can be handled easier because clipping occurs at large values beyond screen dimensions. Second, when using device context transformations it either will not work or will mess the rendered html, so you can't actually rely on scale or translate. Both problems require relatively non-trivial handling and complicate one another.
The solution
I'll describe and provide sample code for non-optimal but working on most simple pages solution. In general, it is possible to achieve a perfect and more efficient solution, but, again, this goes beyond the scope of an answer. Obviously, it is IE8 only, so you'll need to check browser version and execute different handlers for IE8 vs. IE9 or higher, but you can take some ideas to improve other browsers rendering too.
There are two interrelated workarounds here.
Up-scaling
First, how do we upscale the vector content to the printer quality if we can't transform? The workaround here is to render to a context compatible with printer dc. What will happen is that content will be rendered at printer DPI. Note it will not fit exactly printer width, it will scale to printerDPI/screenDPI.
Later, on rasterization, we downscale to fit the printer width. We render initially to EMF, so there is no much of a quality loss (that will occur on the printer itself anyway). If you need higher quality (I doubt it), there are two possibilities - modify the solution to render for target width (this is not trivial) or work with resulting emf instead of bitmap and let printer to make the downscale fit.
Another note is that you currently use just printer width, but there may be non-printable margins that you need to query from printer and account for them. So it may re-scaled by printer even if you provide bitmap in exact printer dimensions. But again, I doubt this resolution disparity will make any difference for your project.
Clipping
Second to overcome is the clipping. To overcome this limitation, we render content in small chunks so they are not clipped by the renderer. After rendering a chunk, we change the scroll position of the document and render next chunk to the appropriate position in the target DC. This can be optimized to use larger chunks e.g. nearest DPI multiple to 1024 (using window resize), but I didn't implement that (it is just a speed optimization). If you don't make this optimization, ensure that minimum browser window size is not too small.
Note, doing this scroll on an arbitrary fractional scale will be an approximation and is not so simple to implement in generic case. But with regular printer and screen we can make integer chunk steps in multiplies of DPI, e.g. if screen is 96 DPI and printer is 600DPI, we make steps in the same multiple of 96 and 600 on each context and everything is much simpler. However, the remainder from top or bottom after processing all whole chunks will not be in DPI multiplies so we can't scroll as easily there.
In general, we can approximate scroll position in printer space and hope there will be no misfit between final chunks. What I did instead is appending an absolutely positioned div with chunk size at the right bottom of the page.
Note, this can interfere with some pages and change the layout (probably not the case with simple reports). If that is a problem, you'll need to add remainder handling after loops instead of adding an element. Most simple solution in that case is still to pad with div but not with the full chunk size but just to make content width multiple of screen DPI.
Even simpler idea, as I've realized later, is just to resize window to the nearest multiple of DPI and take this window size as a chunk size. You can try that instead of div, this will simplify the code and fix pages that may interfere with the injected div.
The code
This is just a sample.
No error handling. You need to add checks for every COM and API call, etc.
No code style, just quick and dirty.
Not sure all acquired resources are released as needed, do your checks
You must disable page borders on browser control for this sample to work (if you need borders around browser, just render them separately, built-in are not consistent anyway). On IE8 this is not so trivial but there are many answers here or on the web. Anyway, I will include this patch in sample solution project. You can render with borders as well and exclude them, but this will be unnecessary complication for a problem that has simple solution.
The full solution project can be found at this link, I'll post only the relevant code here.
The code below renders the page and saves in c:\temp\test.emf + c:\temp\test.bmp
void convertEmfToBitmap(const RECT& fitRect, HDC hTargetDC, HENHMETAFILE hMetafile, LPCTSTR fileName);
CComPtr<IHTMLDOMNode> appendPadElement(IHTMLDocument2* pDoc, IHTMLElement* pBody, long left, long top, long width, long height);
void removeElement(IHTMLElement* pParent, IHTMLDOMNode* pChild);
void CMFCApplication1Dlg::OnBnClickedButton2()
{
COleVariant varNull;
COleVariant varUrl = L"http://www.google.com/search?q=ie+8+must+die";
m_browser.Navigate2(varUrl, varNull, varNull, varNull, varNull);
}
void CMFCApplication1Dlg::OnBnClickedButton1()
{
//get html interfaces
IDispatch* pHtmlDoc = m_browser.get_Document();
CComPtr<IHTMLDocument2> pHtmlDocument2;
pHtmlDoc->QueryInterface(IID_IHTMLDocument2, (void**)&pHtmlDocument2);
CComPtr<IHTMLElement> pBody;
pHtmlDocument2->get_body(&pBody);
CComPtr<IHTMLElement2> pBody2;
pBody->QueryInterface(IID_IHTMLElement2, (void**)&pBody2);
CComPtr<IHTMLBodyElement> pBodyElement;
pBody->QueryInterface(IID_IHTMLBodyElement, (void**)&pBodyElement);
CComPtr<IHTMLElement> pHtml;
pBody->get_parentElement(&pHtml);
CComPtr<IHTMLElement2> pHtml2;
pHtml->QueryInterface(IID_IHTMLElement2, (void**)&pHtml2);
CComPtr<IHTMLStyle> pHtmlStyle;
pHtml->get_style(&pHtmlStyle);
CComPtr<IHTMLStyle> pBodyStyle;
pBody->get_style(&pBodyStyle);
//get screen info
HDC hWndDc = ::GetDC(m_hWnd);
const int wndLogPx = GetDeviceCaps(hWndDc, LOGPIXELSX);
const int wndLogPy = GetDeviceCaps(hWndDc, LOGPIXELSY);
//keep current values
SIZE keptBrowserSize = { m_browser.get_Width(), m_browser.get_Height() };
SIZE keptScrollPos;
//set reasonable viewport size
//m_browser.put_Width(docSize.cx);
//m_browser.put_Height(docSize.cy*2);
pHtml2->get_scrollLeft(&keptScrollPos.cx);
pHtml2->get_scrollTop(&keptScrollPos.cy);
COleVariant keptOverflow;
pBodyStyle->get_overflow(&keptOverflow.bstrVal);
//setup style and hide scroll bars
pHtmlStyle->put_border(L"0px;");
pHtmlStyle->put_overflow(L"hidden");
pBodyStyle->put_border(L"0px;");
pBodyStyle->put_overflow(L"hidden");
//get document size and visible area in screen pixels
SIZE docSize;
pBody2->get_scrollWidth(&docSize.cx);
pBody2->get_scrollHeight(&docSize.cy);
RECT clientRect = { 0 };
pHtml2->get_clientWidth(&clientRect.right);
pHtml2->get_clientHeight(&clientRect.bottom);
//derive chunk size
const SIZE clientChunkSize = {
clientRect.right - clientRect.right % wndLogPx,
clientRect.bottom - clientRect.bottom % wndLogPy };
//pad with absolutely positioned element to have enough scroll area for all chunks
//alternatively, browser can be resized to chunk multiplies (simplest), to DPI multiplies (more work).
//This pad also can be made smaller, to modulus DPI, but then need more work in the loops below
CComPtr<IHTMLDOMNode> pPadNode =
appendPadElement(pHtmlDocument2, pBody, docSize.cx, docSize.cy, clientChunkSize.cx, clientChunkSize.cy);
//get printer info
CPrintDialog pd(TRUE, PD_ALLPAGES | PD_USEDEVMODECOPIES | PD_NOPAGENUMS | PD_HIDEPRINTTOFILE | PD_NOSELECTION);
pd.m_pd.Flags |= PD_RETURNDC | PD_RETURNDEFAULT;
pd.DoModal();
HDC hPrintDC = pd.CreatePrinterDC();
const int printLogPx = GetDeviceCaps(hPrintDC, LOGPIXELSX);
const int printLogPy = GetDeviceCaps(hPrintDC, LOGPIXELSY);
const int printHorRes = ::GetDeviceCaps(hPrintDC, HORZRES);
const SIZE printChunkSize = { printLogPx * clientChunkSize.cx / wndLogPx, printLogPy * clientChunkSize.cy / wndLogPy };
//browser total unscaled print area in printer pixel space
const RECT printRectPx = { 0, 0, docSize.cx* printLogPx / wndLogPx, docSize.cy*printLogPy / wndLogPy };
//unscaled target EMF size in 0.01 mm with printer resolution
const RECT outRect001Mm = { 0, 0, 2540 * docSize.cx / wndLogPx, 2540 * docSize.cy / wndLogPy };
HDC hMetaDC = CreateEnhMetaFile(hPrintDC, L"c:\\temp\\test.emf", &outRect001Mm, NULL);
::FillRect(hMetaDC, &printRectPx, (HBRUSH)::GetStockObject(BLACK_BRUSH));
//unscaled chunk EMF size in pixels with printer resolution
const RECT chunkRectPx = { 0, 0, printChunkSize.cx, printChunkSize.cy };
//unscaled chunk EMF size in 0.01 mm with printer resolution
const RECT chunkRect001Mm = { 0, 0, 2540 * clientChunkSize.cx / wndLogPx, 2540 * clientChunkSize.cy / wndLogPy };
////////
//render page content to metafile by small chunks
//get renderer interface
CComPtr<IHTMLElementRender> pRender;
pHtml->QueryInterface(IID_IHTMLElementRender, (void**)&pRender);
COleVariant printName = L"EMF";
pRender->SetDocumentPrinter(printName.bstrVal, hMetaDC);
//current positions and target area
RECT chunkDestRectPx = { 0, 0, printChunkSize.cx, printChunkSize.cy };
POINT clientPos = { 0, 0 };
POINT printPos = { 0, 0 };
//loop over chunks left to right top to bottom until scroll area is completely covered
const SIZE lastScroll = { docSize.cx, docSize.cy};
while (clientPos.y < lastScroll.cy)
{
while (clientPos.x < lastScroll.cx)
{
//update horizontal scroll position and set target area
pHtml2->put_scrollLeft(clientPos.x);
chunkDestRectPx.left = printPos.x;
chunkDestRectPx.right = printPos.x + printChunkSize.cx;
//render to new emf, can be optimized to avoid recreation
HDC hChunkDC = CreateEnhMetaFile(hPrintDC, NULL, &chunkRect001Mm, NULL);
::FillRect(hChunkDC, &chunkRectPx, (HBRUSH)::GetStockObject(WHITE_BRUSH));
pRender->DrawToDC(hChunkDC);
HENHMETAFILE hChunkMetafile = CloseEnhMetaFile(hChunkDC);
//copy chunk to the main metafile
PlayEnhMetaFile(hMetaDC, hChunkMetafile, &chunkDestRectPx);
DeleteEnhMetaFile(hChunkMetafile);
//update horizontal positions
clientPos.x += clientChunkSize.cx;
printPos.x += printChunkSize.cx;
}
//reset horizontal positions
clientPos.x = 0;
printPos.x = 0;
//update vertical positions
clientPos.y += clientChunkSize.cy;
printPos.y += printChunkSize.cy;
pHtml2->put_scrollTop(clientPos.y);
chunkDestRectPx.top = printPos.y;
chunkDestRectPx.bottom = printPos.y + printChunkSize.cy;
}
//restore changed values on browser
//if for large pages on slow PC you get content scrolling during rendering and it is a problem,
//you can either hide the browser and show "working" or place on top first chunk content
pBodyStyle->put_overflow(keptOverflow.bstrVal);
pHtml2->put_scrollLeft(keptScrollPos.cx);
pHtml2->put_scrollTop(keptScrollPos.cy);
m_browser.put_Width(keptBrowserSize.cx);
m_browser.put_Height(keptBrowserSize.cy);
removeElement(pBody, pPadNode);
//draw to bitmap and close metafile
HENHMETAFILE hMetafile = CloseEnhMetaFile(hMetaDC);
RECT fitRect = { 0, 0, printHorRes, docSize.cy * printHorRes / docSize.cx };
convertEmfToBitmap(fitRect, hWndDc, hMetafile, L"c:\\temp\\test.bmp");
DeleteEnhMetaFile(hMetafile);
//cleanup - probably more here
::ReleaseDC(m_hWnd, hWndDc);
::DeleteDC(hPrintDC);
//{
// std::stringstream ss;
// ss << "====" << docSize.cx << "x" << docSize.cy << " -> " << fitRect.right << "x" << fitRect.bottom << "" << "\n";
// OutputDebugStringA(ss.str().c_str());
//}
}
///////////////
////some util
void convertEmfToBitmap(const RECT& fitRect, HDC hTargetDC, HENHMETAFILE hMetafile, LPCTSTR fileName)
{
//Create memory DC to render into
HDC hCompDc = ::CreateCompatibleDC(hTargetDC);
//NOTE this
BITMAPINFOHEADER infoHeader = { 0 };
infoHeader.biSize = sizeof(infoHeader);
infoHeader.biWidth = fitRect.right;
infoHeader.biHeight = -fitRect.bottom;
infoHeader.biPlanes = 1;
infoHeader.biBitCount = 24;
infoHeader.biCompression = BI_RGB;
BITMAPINFO info;
info.bmiHeader = infoHeader;
//create bitmap
BYTE* pMemory = 0;
HBITMAP hBitmap = ::CreateDIBSection(hCompDc, &info, DIB_RGB_COLORS, (void**)&pMemory, 0, 0);
HBITMAP hOldBmp = (HBITMAP)::SelectObject(hCompDc, hBitmap);
PlayEnhMetaFile(hCompDc, hMetafile, &fitRect);
BITMAPFILEHEADER fileHeader = { 0 };
fileHeader.bfType = 0x4d42;
fileHeader.bfSize = 0;
fileHeader.bfReserved1 = 0;
fileHeader.bfReserved2 = 0;
fileHeader.bfOffBits = sizeof(BITMAPFILEHEADER)+sizeof(BITMAPINFOHEADER);
CFile file(
fileName,
CFile::modeCreate | CFile::modeReadWrite | CFile::shareDenyNone);
file.Write((char*)&fileHeader, sizeof(fileHeader));
file.Write((char*)&infoHeader, sizeof(infoHeader));
int bytes = (((24 * infoHeader.biWidth + 31) & (~31)) / 8) * abs(infoHeader.biHeight);
file.Write(pMemory, bytes);
::SelectObject(hCompDc, hOldBmp);
//Clean up
if (hBitmap)
::DeleteObject(hBitmap);
::DeleteDC(hCompDc);
}
CComPtr<IHTMLDOMNode> appendPadElement(IHTMLDocument2* pDoc, IHTMLElement* pBody, long left, long top, long width, long height)
{
CComPtr<IHTMLElement> pPadElement;
pDoc->createElement(L"DIV", &pPadElement);
CComPtr<IHTMLStyle> pPadStyle;
pPadElement->get_style(&pPadStyle);
CComPtr<IHTMLStyle2> pPadStyle2;
pPadStyle->QueryInterface(IID_IHTMLStyle2, (void**)&pPadStyle2);
pPadStyle2->put_position(L"absolute");
CComVariant value = width;
pPadStyle->put_width(value);
value = height;
pPadStyle->put_height(value);
pPadStyle->put_posLeft((float)left);
pPadStyle->put_posTop((float)top);
CComPtr<IHTMLDOMNode> pPadNode;
pPadElement->QueryInterface(IID_IHTMLDOMNode, (void**)&pPadNode);
CComPtr<IHTMLDOMNode> pBodyNode;
pBody->QueryInterface(IID_IHTMLDOMNode, (void **)&pBodyNode);
pBodyNode->appendChild(pPadNode, NULL);
return pPadNode;
}
void removeElement(IHTMLElement* pParent, IHTMLDOMNode* pChild)
{
CComPtr<IHTMLDOMNode> pNode;
pParent->QueryInterface(IID_IHTMLDOMNode, (void **)&pNode);
pNode->removeChild(pChild, NULL);
}
Sample page output (4958x7656)
I have taken your code and run it on IE11 when the WebBrowser control is smaller then the page size. It rendered a portion of the page equal to control's size. Not sure why you say IE8 and IE11 are any different.
It seems that common approach to taking full page screenshots is adjusting WebBrowser size before taking screenshot, like this:
const long oldH = m_browser.get_Height();
const long oldW = m_browser.get_Width();
m_browser.put_Height(n_scrollHeight);
m_browser.put_Width(n_scrollWidth);
//Do the upscaling & render -- note that IE8 fails to render it here!!!!
pViewObject->Draw(DVASPECT_CONTENT, //DVASPECT_DOCPRINT
-1, NULL, NULL, NULL, hCompDc,
&rectPrnt,
NULL,
NULL, 0);
m_browser.put_Height(oldH);
m_browser.put_Width(oldW);
This seems to work well, even on large pages such as the one you're currently reading (I have taken screenshot 1920x8477). It works both on my IE11 and on IE8 virtual machine
It has a side effect of resetting scrollbars, but that can be solved, for example by using an invisible copy of WebBrowser for screenshots.
PS: You could have done a better job by providing example code that can be compiled, at least ;)
Context:
I'm developing a native C++ Unity 5 plugin that reads in DXT compressed texture data and uploads it to the GPU for further use in Unity. The aim is to create an fast image-sequence player, updating image data on-the-fly. The textures are compressed with an offline console application.
Unity can work with different graphics engines, I'm aiming towards DirectX11 and OpenGL 3.3+.
Problem:
The DirectX runtime texture update code, through a mapped subresource, gives different outputs on different graphics drivers. Updating a texture through such a mapped resource means mapping a pointer to the texture data and memcpy'ing the data from the RAM buffer to the mapped GPU buffer. Doing so, different drivers seem to expect different parameters for the row pitch value when copying bytes. I never had problems on the several Nvidia GPU's I tested on, but AMD and Intel GPU seems to act differently and I get distorted output as shown underneath. Furthermore, I'm working with DXT1 pixel data (0.5bpp) and DXT5 data (1bpp). I can't seem to get the correct pitch parameter for these DXT textures.
Code:
The following initialisation code for generating the d3d11 texture and filling it with initial texture data - e.g. the first frame of an image sequence - works perfect on all drivers. The player pointer points to a custom class that handles all file reads and contains getters for the current loaded DXT compressed frame, it's dimensions, etc...
if (s_DeviceType == kUnityGfxRendererD3D11)
{
HRESULT hr;
DXGI_FORMAT format = (compression_type == DxtCompressionType::DXT_TYPE_DXT1_NO_ALPHA) ? DXGI_FORMAT_BC1_UNORM : DXGI_FORMAT_BC3_UNORM;
// Create texture
D3D11_TEXTURE2D_DESC desc;
desc.Width = w;
desc.Height = h;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = format;
// no anti-aliasing
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DYNAMIC;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
desc.MiscFlags = 0;
// Initial data: first frame
D3D11_SUBRESOURCE_DATA data;
data.pSysMem = player->getBufferPtr();
data.SysMemPitch = 16 * (player->getWidth() / 4);
data.SysMemSlicePitch = 0; // just a 2d texture, no depth
// Init with initial data
hr = g_D3D11Device->CreateTexture2D(&desc, &data, &dxt_d3d_tex);
if (SUCCEEDED(hr) && dxt_d3d_tex != 0)
{
DXT_VERBOSE("Succesfully created D3D Texture.");
DXT_VERBOSE("Creating D3D SRV.");
D3D11_SHADER_RESOURCE_VIEW_DESC SRVDesc;
memset(&SRVDesc, 0, sizeof(SRVDesc));
SRVDesc.Format = format;
SRVDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
SRVDesc.Texture2D.MipLevels = 1;
hr = g_D3D11Device->CreateShaderResourceView(dxt_d3d_tex, &SRVDesc, &textureView);
if (FAILED(hr))
{
dxt_d3d_tex->Release();
return hr;
}
DXT_VERBOSE("Succesfully created D3D SRV.");
}
else
{
DXT_ERROR("Error creating D3D texture.")
}
}
The following update code that runs for each new frame has the error somewhere. Please note the commented line containing method 1 using a simple memcpy without any rowpitch specified which works well on NVIDIA drivers.
You can see further in method 2 that I log the different row pitch values. For instace for a 1920x960 frame I get 1920 for the buffer stride, and 2048 for the runtime stride. This 128 pixels difference probably have to be padded (as can be seen in the example pic below) but I can't figure out how. When I just use the mappedResource.RowPitch without dividing it by 4 (done by the bitshift), Unity crashes.
ID3D11DeviceContext* ctx = NULL;
g_D3D11Device->GetImmediateContext(&ctx);
if (dxt_d3d_tex && bShouldUpload)
{
if (player->gather_stats) before_upload = ns();
D3D11_MAPPED_SUBRESOURCE mappedResource;
ctx->Map(dxt_d3d_tex, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
/* 1: THIS CODE WORKS ON ALL NVIDIA DRIVERS BUT GENERATES DISTORTED OR NO OUTPUT ON AMD/INTEL: */
//memcpy(mappedResource.pData, player->getBufferPtr(), player->getBytesPerFrame());
/* 2: THIS CODE GENERATES OUTPUT BUT SEEMS TO NEED PADDING? */
BYTE* mappedData = reinterpret_cast<BYTE*>(mappedResource.pData);
BYTE* buffer = player->getBufferPtr();
UINT height = player->getHeight();
UINT buffer_stride = player->getBytesPerFrame() / player->getHeight();
UINT runtime_stride = mappedResource.RowPitch >> 2;
DXT_VERBOSE("Buffer stride: %d", buffer_stride);
DXT_VERBOSE("Runtime stride: %d", runtime_stride);
for (UINT i = 0; i < height; ++i)
{
memcpy(mappedData, buffer, buffer_stride);
mappedData += runtime_stride;
buffer += buffer_stride;
}
ctx->Unmap(dxt_d3d_tex, 0);
}
Example pic 1 - distorted ouput when using memcpy to copy whole buffer without using separate row pitch on AMD/INTEL (method 1)
Example pic 2 - better but still erroneous output when using above code with mappedResource.RowPitch on AMD/INTEL (method 2). The blue bars indicate zone of error, and need to disappear so all pixels align well and form one image.
Thanks for any pointers!
Best,
Vincent
The mapped data row pitch is in byte, when you divide by four, it is definitely an issue.
UINT runtime_stride = mappedResource.RowPitch >> 2;
...
mappedData += runtime_stride; // here you are only jumping one quarter of a row
It is the height count with a BC format that is divide by 4.
Also a BC1 format is 8 bytes per 4x4 block, so the line below should by 8 * and not 16 *, but as long as you handle row stride properly on your side, d3d will understand, you just waste half the memory here.
data.SysMemPitch = 16 * (player->getWidth() / 4);
I am using libpng and libjpeg to read and write images. The code I use was taken almost straight from the examples provided with the two libraries' documentation, and image loading works correctly with both libraries. However, when I go to save an image, something goes wrong, and it seems to write corrupted data somehow. The confusing part is that it writes it in exactly the same incorrect way, using both libraries. Here's an example:
Original:
Blurred picture (as it looks in the program, before saving):
How it saves (png):
The jpeg version saves with identical discoloration, just more compressed.
Here's the png saving code:
void PNGHandler::save(PixelBuffer* buffer, std::string fileName)
{
FILE* filePointer = fopen(fileName.c_str(), "wb");
int width = buffer->getWidth();
int height = buffer->getHeight();
png_structp png = png_create_write_struct(PNG_LIBPNG_VER_STRING, NULL, NULL, NULL);
png_infop info = png_create_info_struct(png);
png_init_io(png, filePointer);
// 8-bit depth, RGBA
png_set_IHDR(png, info, width, height, 8,
PNG_COLOR_TYPE_RGBA, PNG_INTERLACE_NONE,
PNG_COMPRESSION_TYPE_DEFAULT,
PNG_FILTER_TYPE_DEFAULT);
png_write_info(png, info);
// Set up rows for writing from
png_bytep *rowPointers = new png_bytep[height];
for(int y = 0; y < height; y++) {
rowPointers[y] = new png_byte[png_get_rowbytes(png,info)];
}
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
ColorData cd = buffer->getPixel(x, height - y - 1);
rowPointers[y][x*4] = (int)(cd.getRed() * 255);
rowPointers[y][x*4+1] = (int)(cd.getGreen() * 255);
rowPointers[y][x*4+2] = (int)(cd.getBlue() * 255);
rowPointers[y][x*4+3] = (int)(cd.getAlpha() * 255);
}
}
png_write_image(png, rowPointers);
png_write_end(png, info);
delete [] rowPointers;
png_destroy_write_struct(&png, &info);
fclose(filePointer);
}
(I know the error handling isn't great right now, but I'll fix that later)
Additionally, the file always saves this way. That is, I can apply the blur and save, then reload the original and do it again, and performing a diff on the two files reveals they're identical. The PixelBuffer pointer that's passed in is a pointer to the buffer that is being displayed on the screen, so all of the color data should be exactly as it appears.
I know this isn't much information to go on, but if someone can guide me toward what I should look for, I can bring more to the table (it's a large project, so I can't post all the code)
Edit: It's also worth noting that the image looks correct after saving, but once the saved image is loaded in, it displays the discoloration. This points toward a problem in the saving methods to me
Your filter/blur probably overflows/underflows the color values. You should make sure that the values are saturated within 0 and 255 (if values goes under 0, set them to 0, and if values goes above 255, set them to 255)
I'm writing a DX9 renderer and currently working on the ability to play AVI movie files. I've been able to retrieve any specified frame using AVIStreamGetFrame(), which returns a packed DIB, and from there I want to be able to copy that bitmap data to an already existing IDirect3DTexture9 *.
My issue is a lack of understanding of the bitmap file format and knowing how to convert the pixel data given from a BITMAPINFOHEADER to a format that IDirect3DTexture9 can interpret.
I first create my DX9 texture like this:
LPBITMAPINFO bmpInfo = m_pVideoData->GetVideoFormat();
D3DXCreateTexture(LtGEngine::GetInstance()->GetDevice(),
bmpInfo->bmiHeader.biWidth,
bmpInfo->bmiHeader.biHeight,
D3DX_DEFAULT,
0,
D3DFMT_A8R8G8B8, // <- DETERMINE HOW?
D3DPOOL_MANAGED, // <- OR D3DPOOL_SYSTEMMEM?
&m_pD3DTexture);
Questions I have here are listed as comments above. When I get the BITMAPINFO and for instance it reads bmpInfo.bmiHeader.biBitCount = 8 (or 16, etc.) does this mean I need to change the D3DFMT_* accordingly?
Later on when I get a LPBITMAPINFOHEADER for the frame I want to render, I'm lost on what to do with pBits returned from the IDirect3DTexture9::LockRect() function. Here is what I have so far:
// Retrieve a frame from the video data as a BITMAPINFOHEADER
LPBITMAPINFOHEADER pBmpInfoHeader;
m_pVideoData->GetVideoFrame(0, 0, &pBmpInfoHeader);
D3DLOCKED_RECT rect;
if(FAILED(m_pD3DTexture->LockRect(0, &rect, NULL, 0)))
{
m_pD3DTexture->UnlockRect(0);
}
DWORD* pDest = (DWORD*)rect.pBits;
// Now what to copy from pBmpInfoHeader?
Are there any API calls that do this for me that I haven't seen? Or does anyone know of an easier way than this? Thanks for reading/helping.
Got it to work!
Couple notes to consider. My AVI file (a single frame/bitmap in this case) was in a 16 bit format, therefore I had to create my destination texture with D3DFMT_X1R5G5B5. Also, bitmaps are stored upside down, so I had to reverse my pointer and read each row backwards.
Here's the code:
// Retrieve a frame from the video data as a BITMAPINFOHEADER
LPBITMAPINFOHEADER pBmpInfoHeader;
m_pVideoData->GetVideoFrame(0, 0, &pBmpInfoHeader);
// Get dimentions
long nWidth = pBmpInfoHeader->biWidth;
long nHeight = pBmpInfoHeader->biHeight;
// Bitmap width correction (might not be needed...)
if (nWidth % 4 != 0)
nWidth = nWidth + (4 - nWidth%4);
// Get Pixel data (should be after the header in memory)
WORD bitCount = pBmpInfoHeader->biBitCount;
DWORD size = nWidth * nHeight * bitCount/8;
BYTE *pPixelSrc = (BYTE *)pBmpInfoHeader + sizeof(pBmpInfoHeader);
// Lock the texture so we can write this frame's texel data
D3DLOCKED_RECT lock;
if(FAILED(m_pD3DTexture->LockRect(0, &lock, NULL, 0)))
{
m_pD3DTexture->UnlockRect(0);
return;
}
int iNumBytesPerRowSrc = pBmpInfoHeader->biWidth * (pBmpInfoHeader->biBitCount/8);
int iNumBytesPerRowDst = lock.Pitch;
int iNumBytesToCopyPerRow = min(iNumBytesPerRowSrc, iNumBytesPerRowDst);
// Bitmap data is stored upside down
// Start at the end and work backwards
pPixelSrc += (iNumBytesPerRowSrc * nHeight);
// Store a pointer to the texture pixel data and write new data
BYTE* ucTexDst = (BYTE *)lock.pBits;
for(int y = 0; y < nHeight; ++y)
{
pPixelSrc -= iNumBytesPerRowSrc;
memcpy(ucTexDst, pPixelSrc, iNumBytesToCopyPerRow);
ucTexDst += iNumBytesPerRowDst;
}
// Unlock texture so gfx card can resume its business
m_pD3DTexture->UnlockRect(0);