I am trying to understand how to work the rendering for YV12 format. For example, I took a simple sample. See this graph:
The webcam creates frames by size 640x480 in RGB24 or MJPEG . After it the LAV decoder transforms the frames to YV12 and sends them to DS renderer (EVR or VMR9).
The decoder changes the frame width (stride) 640 on 1024. Hence, the output size of frame will be 1.5*1024*640=737280. The normal size for YV12 is 1.5*640*480=460800. I know the stride can be more than the width of real frame (https://learn.microsoft.com/en-us/windows/desktop/medfound/image-stride). My first question - why did the renderer select that value (1024) than another? Can I get it programmatically?
When I replace the LAV decoder with my filter for transformation RGB24/YV12 (https://gist.github.com/thedeemon/8052fb98f8ba154510d7), the renderer shows me a shifted image, though all parameters are the same, as for the first graph:
Why? I noted that VIDEOINFOHEADER2 had the set interlacing flag dwInterlaceFlags. Therefore my next question: Do I have to add interlacing into my filter for normal work of renderer?
My first question - why did the renderer select that value (1024) than another? Can I get it programmatically?
Video renderer is using Direct3D texture as a carrier for the image. When texture is mapped into system memory to enable CPU's write access such extended stride could be applied because of specifics of implementation of video hardware. You get the value 1024 via dynamic media type negotiation as described in Handling Format Changes from the Video Renderer.
Your transformation filter has to handle such updates if you want it to be able to connect to video renderer directly.
You are generally not interested in getting this extended stride value otherwise because the one you get via media type update is the one to be used and you have to accept it.
When I replace the LAV decoder with my filter for transformation RGB24/YV12, the renderer shows me a shifted image, though all parameters are the same, as for the first graph...
Why?
Your filter does not handle stride update right.
...I noted that VIDEOINFOHEADER2 had the set interlacing flag dwInterlaceFlags. Therefore my next question: Do I have to add interlacing into my filter for normal work of renderer?
You don't have interlaced video here. The problem is unrelated to interlaced video.
My solution:
I must right copy a YV12 frame into a video buffer by its three surfaces: Y = 4x4, U = 1x2, V = 1x2. Here is a code for the frame size 640x480:
CVideoGrabberFilter::Transform(IMediaSample *pIn, IMediaSample *pOut)
{
BYTE* pSrcBuf = 0;
pIn->GetPointer(&pSrcBuf);
BYTE* pDstBuf = 0;
pOut->GetPointer(&pDstBuf);
SIZE size;
size.cx = 640;
size.cy = 480;
int nLen = pOut->GetActualDataLength();
BYTE* pDstTmp = new BYTE[nLen];
YV12ConverterFromRGB24(pSrcBufEnd, pDstTmp, size.cx, size.cy);
BYTE* pDst = pDstTmp;
int stride = 1024; //the real video stride for 640x480. For other resolutions you need to use pOut->GetMediaType() for the stride defining.
//Y
for (int y = 0; y < size.cy; ++y)
{
memcpy(pDstBuf, pDst, size.cx);
pDst += size.cx;
pDstBuf += stride;
}
stride /= 2;
size.cy /= 2;
size.cx /= 2;
//U and V
for (int y = 0; y < size.cy; y++ )
{
memcpy(pDstBuf, pDst, size.cx );
pDst += size.cx;
pDstBuf += stride;
memcpy(pDstBuf, pDst, size.cx);
pDst += size.cx;
pDstBuf += stride;
}
delete[] pDstTmp;
}
I have an MFC dialog window where I added a WebBrowser control (that encapsulates the Internet Explorer engine.)
The goal of the following code is to render the contents of the said web browser control into a device context that I can later use for printing:
//MFC code, error checks are omitted for brevity
//'m_browser' = is a web browser control of type `CExplorer1`
IDispatch* pHtmlDoc = m_browser.get_Document();
CComPtr<IHTMLDocument2> pHtmlDocument2;
pHtmlDoc->QueryInterface(IID_IHTMLDocument2, (void**)&pHtmlDocument2));
//Get IViewObject2 for the entire document that we will use to render into a DC
CComPtr<IViewObject2> pViewObject;
pHtmlDocument2->QueryInterface(IID_IViewObject2, (void **)&pViewObject));
CComPtr<IHTMLElement> pBody;
pHtmlDocument2->get_body(&pBody));
CComPtr<IHTMLElement2> pBody2;
pBody->QueryInterface(IID_IHTMLElement2, (void **)&pBody2));
//Get default printer DC
CPrintDialog pd(TRUE, PD_ALLPAGES | PD_USEDEVMODECOPIES | PD_NOPAGENUMS | PD_HIDEPRINTTOFILE | PD_NOSELECTION);
pd.m_pd.Flags |= PD_RETURNDC | PD_RETURNDEFAULT;
pd.DoModal(); //corrected later
HDC hPrintDC = pd.CreatePrinterDC();
//Calc bitmap width based on printer DC specs
//Note that this width will be larger than
//the width of the WebControl window itself due to
//printer's much higher DPI setting...
int n_bitmapWidth = ::GetDeviceCaps(hPrintDC, HORZRES); //Use entire printable area
//Get full size of the document
long n_scrollWidth;
long n_scrollHeight;
pBody2->get_scrollWidth(&n_scrollWidth);
pBody2->get_scrollHeight(&n_scrollHeight);
//Calc proportional size of the bitmap in the DC to render
int nWidth = n_bitmapWidth;
int nHeight = n_bitmapWidth * n_scrollHeight / n_scrollWidth;
//Create memory DC to render into
HDC hDc = ::GetDC(hWnd);
HDC hCompDc = ::CreateCompatibleDC(hDC);
//I'm using a raw DIB section here as I'll need to access
//its bitmap bits directly later in my code...
BITMAPINFOHEADER infoHeader = {0};
infoHeader.biSize = sizeof(infoHeader);
infoHeader.biWidth = nWidth;
infoHeader.biHeight = -nHeight;
infoHeader.biPlanes = 1;
infoHeader.biBitCount = 24;
infoHeader.biCompression = BI_RGB;
BITMAPINFO info;
info.bmiHeader = infoHeader;
//Create a bitmap as DIB section of size `nWidth` by `nHeight` pixels
BYTE* pMemory = 0;
HBITMAP hBitmap = ::CreateDIBSection(hDc, &info, DIB_RGB_COLORS, (void**)&pMemory, 0, 0);
HBITMAP hOldBmp = (HBITMAP)::SelectObject(hCompDc, hBitmap);
RECT rcAll = {0, 0, nWidth, nHeight};
::FillRect(hCompDc, &rcAll, (HBRUSH)::GetStockObject(WHITE_BRUSH));
RECTL rectPrnt = {0, 0, nWidth, nHeight};
//Do the upscaling & render -- note that IE8 fails to render it here!!!!
pViewObject->Draw(DVASPECT_CONTENT, //DVASPECT_DOCPRINT
-1, NULL, NULL, NULL, hCompDc,
&rectPrnt,
NULL,
NULL, 0));
::SelectObject(hCompDc, hOldBmp);
//Now the bitmap in `hCompDc` contains the resulting pixels
//For debugging purposes, save it as .bmp file
BITMAPFILEHEADER fileHeader = {0};
fileHeader.bfType = 0x4d42;
fileHeader.bfSize = 0;
fileHeader.bfReserved1 = 0;
fileHeader.bfReserved2 = 0;
fileHeader.bfOffBits = sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
CFile file(
L"path-to\\test.bmp",
CFile::modeCreate | CFile::modeReadWrite | CFile::shareDenyNone);
file.Write((char*)&fileHeader, sizeof(fileHeader));
file.Write((char*)&infoHeader, sizeof(infoHeader));
int bytes = (((24 * nWidth + 31) & (~31)) / 8) * nHeight;
file.Write(pMemory, bytes);
//Clean up
::DeleteObject(hBitmap);
::DeleteDC(hCompDc);
::ReleaseDC(hWnd, hDc);
::DeleteDC(hPrintDC);
This code works fine if I have the latest IE11 installed on my development machine. But if, for instance, someone has IE8 installed on their Windows 7, the IViewObject::Draw method will render only a small part of the document (equal to the size of the web browser control itself.)
The best way to describe it is to illustrate it with the examples:
Normally rendered test page with IE11 installed:
and here's what happens with IE8 installed:
Does anyone have any idea what am I doing wrong here that IE8 doesn't like?
EDIT1: Did some more digging into the IViewObject::Draw function with WinDbg and then found the source code for it. Here's CServer::Draw() that is IViewObject::Draw, and then CDoc::Draw() that is called internally from CServer::Draw().
First, thanks for the interesting question. While not so practical - not a lot of people use IE8 today - it was not so trivial to solve. I'll describe what is the problem and provide a simplistic but working solution that you can improve.
Before I go into IE8 solution, a couple of points:
The solution with window resize to fit scroll size is not stable if you can encounter large documents. If you don't know what to anticipate, you need to refactor the solution for later explorers as well, to avoid relying on resizing window to scroll size.
Why carry giant bitmap? Metafiles etc. may fit better. Any large enough page at this resolution is going to blow memory on PC with the naive DIB creation. Google page in provided sample renders to 100Mb bitmap file while emf, from which rasterization is done, takes less than 1Mb.
While I don't know exact requirements and limitations of your project, I'm 99% sure that drawing into gigantic DIB is not the best solution. Even EMF, while better, is not the best either. If you need, for example, add a signature and then print, there are better ways to handle this. This, of course, just a side note, not related to the question itself.
IE8 rendering problem
In IE8 renderer there is a bug. Draw() will be clipped at pixel dimensions of actual display area (the visible rectangle you see is the original display area in the scale of rendering context).
So, if you have a scaled target that is larger than the actual size, while scaled, it will be clipped to the size in scaled pixels anyway (so it has much less content than original rectangle).
If it doesn't clip for somebody on genuine IE8, then there are remainders of later IE in the system or there is other non-scratch setup, system update or alike.
Workaround possibilities
Good news it is possible to workaround, bad news workarounds are a bit nasty.
First, it is still possible to workaround with IViewObject. But, because there is arbitrary scaling involved and accessible source rectangle is very small, this solution has some complexities that I think are beyong an SO answer. So I would not dive into this path.
Instead, we can render through another, now outdated API: IHTMLElementRender. It allows to render the page with DrawToDC into arbitrary context. Unfortunately, it is not so simple as it may appear and goes beyond just providing device context.
First, there is similar bug of clipping. It can be handled easier because clipping occurs at large values beyond screen dimensions. Second, when using device context transformations it either will not work or will mess the rendered html, so you can't actually rely on scale or translate. Both problems require relatively non-trivial handling and complicate one another.
The solution
I'll describe and provide sample code for non-optimal but working on most simple pages solution. In general, it is possible to achieve a perfect and more efficient solution, but, again, this goes beyond the scope of an answer. Obviously, it is IE8 only, so you'll need to check browser version and execute different handlers for IE8 vs. IE9 or higher, but you can take some ideas to improve other browsers rendering too.
There are two interrelated workarounds here.
Up-scaling
First, how do we upscale the vector content to the printer quality if we can't transform? The workaround here is to render to a context compatible with printer dc. What will happen is that content will be rendered at printer DPI. Note it will not fit exactly printer width, it will scale to printerDPI/screenDPI.
Later, on rasterization, we downscale to fit the printer width. We render initially to EMF, so there is no much of a quality loss (that will occur on the printer itself anyway). If you need higher quality (I doubt it), there are two possibilities - modify the solution to render for target width (this is not trivial) or work with resulting emf instead of bitmap and let printer to make the downscale fit.
Another note is that you currently use just printer width, but there may be non-printable margins that you need to query from printer and account for them. So it may re-scaled by printer even if you provide bitmap in exact printer dimensions. But again, I doubt this resolution disparity will make any difference for your project.
Clipping
Second to overcome is the clipping. To overcome this limitation, we render content in small chunks so they are not clipped by the renderer. After rendering a chunk, we change the scroll position of the document and render next chunk to the appropriate position in the target DC. This can be optimized to use larger chunks e.g. nearest DPI multiple to 1024 (using window resize), but I didn't implement that (it is just a speed optimization). If you don't make this optimization, ensure that minimum browser window size is not too small.
Note, doing this scroll on an arbitrary fractional scale will be an approximation and is not so simple to implement in generic case. But with regular printer and screen we can make integer chunk steps in multiplies of DPI, e.g. if screen is 96 DPI and printer is 600DPI, we make steps in the same multiple of 96 and 600 on each context and everything is much simpler. However, the remainder from top or bottom after processing all whole chunks will not be in DPI multiplies so we can't scroll as easily there.
In general, we can approximate scroll position in printer space and hope there will be no misfit between final chunks. What I did instead is appending an absolutely positioned div with chunk size at the right bottom of the page.
Note, this can interfere with some pages and change the layout (probably not the case with simple reports). If that is a problem, you'll need to add remainder handling after loops instead of adding an element. Most simple solution in that case is still to pad with div but not with the full chunk size but just to make content width multiple of screen DPI.
Even simpler idea, as I've realized later, is just to resize window to the nearest multiple of DPI and take this window size as a chunk size. You can try that instead of div, this will simplify the code and fix pages that may interfere with the injected div.
The code
This is just a sample.
No error handling. You need to add checks for every COM and API call, etc.
No code style, just quick and dirty.
Not sure all acquired resources are released as needed, do your checks
You must disable page borders on browser control for this sample to work (if you need borders around browser, just render them separately, built-in are not consistent anyway). On IE8 this is not so trivial but there are many answers here or on the web. Anyway, I will include this patch in sample solution project. You can render with borders as well and exclude them, but this will be unnecessary complication for a problem that has simple solution.
The full solution project can be found at this link, I'll post only the relevant code here.
The code below renders the page and saves in c:\temp\test.emf + c:\temp\test.bmp
void convertEmfToBitmap(const RECT& fitRect, HDC hTargetDC, HENHMETAFILE hMetafile, LPCTSTR fileName);
CComPtr<IHTMLDOMNode> appendPadElement(IHTMLDocument2* pDoc, IHTMLElement* pBody, long left, long top, long width, long height);
void removeElement(IHTMLElement* pParent, IHTMLDOMNode* pChild);
void CMFCApplication1Dlg::OnBnClickedButton2()
{
COleVariant varNull;
COleVariant varUrl = L"http://www.google.com/search?q=ie+8+must+die";
m_browser.Navigate2(varUrl, varNull, varNull, varNull, varNull);
}
void CMFCApplication1Dlg::OnBnClickedButton1()
{
//get html interfaces
IDispatch* pHtmlDoc = m_browser.get_Document();
CComPtr<IHTMLDocument2> pHtmlDocument2;
pHtmlDoc->QueryInterface(IID_IHTMLDocument2, (void**)&pHtmlDocument2);
CComPtr<IHTMLElement> pBody;
pHtmlDocument2->get_body(&pBody);
CComPtr<IHTMLElement2> pBody2;
pBody->QueryInterface(IID_IHTMLElement2, (void**)&pBody2);
CComPtr<IHTMLBodyElement> pBodyElement;
pBody->QueryInterface(IID_IHTMLBodyElement, (void**)&pBodyElement);
CComPtr<IHTMLElement> pHtml;
pBody->get_parentElement(&pHtml);
CComPtr<IHTMLElement2> pHtml2;
pHtml->QueryInterface(IID_IHTMLElement2, (void**)&pHtml2);
CComPtr<IHTMLStyle> pHtmlStyle;
pHtml->get_style(&pHtmlStyle);
CComPtr<IHTMLStyle> pBodyStyle;
pBody->get_style(&pBodyStyle);
//get screen info
HDC hWndDc = ::GetDC(m_hWnd);
const int wndLogPx = GetDeviceCaps(hWndDc, LOGPIXELSX);
const int wndLogPy = GetDeviceCaps(hWndDc, LOGPIXELSY);
//keep current values
SIZE keptBrowserSize = { m_browser.get_Width(), m_browser.get_Height() };
SIZE keptScrollPos;
//set reasonable viewport size
//m_browser.put_Width(docSize.cx);
//m_browser.put_Height(docSize.cy*2);
pHtml2->get_scrollLeft(&keptScrollPos.cx);
pHtml2->get_scrollTop(&keptScrollPos.cy);
COleVariant keptOverflow;
pBodyStyle->get_overflow(&keptOverflow.bstrVal);
//setup style and hide scroll bars
pHtmlStyle->put_border(L"0px;");
pHtmlStyle->put_overflow(L"hidden");
pBodyStyle->put_border(L"0px;");
pBodyStyle->put_overflow(L"hidden");
//get document size and visible area in screen pixels
SIZE docSize;
pBody2->get_scrollWidth(&docSize.cx);
pBody2->get_scrollHeight(&docSize.cy);
RECT clientRect = { 0 };
pHtml2->get_clientWidth(&clientRect.right);
pHtml2->get_clientHeight(&clientRect.bottom);
//derive chunk size
const SIZE clientChunkSize = {
clientRect.right - clientRect.right % wndLogPx,
clientRect.bottom - clientRect.bottom % wndLogPy };
//pad with absolutely positioned element to have enough scroll area for all chunks
//alternatively, browser can be resized to chunk multiplies (simplest), to DPI multiplies (more work).
//This pad also can be made smaller, to modulus DPI, but then need more work in the loops below
CComPtr<IHTMLDOMNode> pPadNode =
appendPadElement(pHtmlDocument2, pBody, docSize.cx, docSize.cy, clientChunkSize.cx, clientChunkSize.cy);
//get printer info
CPrintDialog pd(TRUE, PD_ALLPAGES | PD_USEDEVMODECOPIES | PD_NOPAGENUMS | PD_HIDEPRINTTOFILE | PD_NOSELECTION);
pd.m_pd.Flags |= PD_RETURNDC | PD_RETURNDEFAULT;
pd.DoModal();
HDC hPrintDC = pd.CreatePrinterDC();
const int printLogPx = GetDeviceCaps(hPrintDC, LOGPIXELSX);
const int printLogPy = GetDeviceCaps(hPrintDC, LOGPIXELSY);
const int printHorRes = ::GetDeviceCaps(hPrintDC, HORZRES);
const SIZE printChunkSize = { printLogPx * clientChunkSize.cx / wndLogPx, printLogPy * clientChunkSize.cy / wndLogPy };
//browser total unscaled print area in printer pixel space
const RECT printRectPx = { 0, 0, docSize.cx* printLogPx / wndLogPx, docSize.cy*printLogPy / wndLogPy };
//unscaled target EMF size in 0.01 mm with printer resolution
const RECT outRect001Mm = { 0, 0, 2540 * docSize.cx / wndLogPx, 2540 * docSize.cy / wndLogPy };
HDC hMetaDC = CreateEnhMetaFile(hPrintDC, L"c:\\temp\\test.emf", &outRect001Mm, NULL);
::FillRect(hMetaDC, &printRectPx, (HBRUSH)::GetStockObject(BLACK_BRUSH));
//unscaled chunk EMF size in pixels with printer resolution
const RECT chunkRectPx = { 0, 0, printChunkSize.cx, printChunkSize.cy };
//unscaled chunk EMF size in 0.01 mm with printer resolution
const RECT chunkRect001Mm = { 0, 0, 2540 * clientChunkSize.cx / wndLogPx, 2540 * clientChunkSize.cy / wndLogPy };
////////
//render page content to metafile by small chunks
//get renderer interface
CComPtr<IHTMLElementRender> pRender;
pHtml->QueryInterface(IID_IHTMLElementRender, (void**)&pRender);
COleVariant printName = L"EMF";
pRender->SetDocumentPrinter(printName.bstrVal, hMetaDC);
//current positions and target area
RECT chunkDestRectPx = { 0, 0, printChunkSize.cx, printChunkSize.cy };
POINT clientPos = { 0, 0 };
POINT printPos = { 0, 0 };
//loop over chunks left to right top to bottom until scroll area is completely covered
const SIZE lastScroll = { docSize.cx, docSize.cy};
while (clientPos.y < lastScroll.cy)
{
while (clientPos.x < lastScroll.cx)
{
//update horizontal scroll position and set target area
pHtml2->put_scrollLeft(clientPos.x);
chunkDestRectPx.left = printPos.x;
chunkDestRectPx.right = printPos.x + printChunkSize.cx;
//render to new emf, can be optimized to avoid recreation
HDC hChunkDC = CreateEnhMetaFile(hPrintDC, NULL, &chunkRect001Mm, NULL);
::FillRect(hChunkDC, &chunkRectPx, (HBRUSH)::GetStockObject(WHITE_BRUSH));
pRender->DrawToDC(hChunkDC);
HENHMETAFILE hChunkMetafile = CloseEnhMetaFile(hChunkDC);
//copy chunk to the main metafile
PlayEnhMetaFile(hMetaDC, hChunkMetafile, &chunkDestRectPx);
DeleteEnhMetaFile(hChunkMetafile);
//update horizontal positions
clientPos.x += clientChunkSize.cx;
printPos.x += printChunkSize.cx;
}
//reset horizontal positions
clientPos.x = 0;
printPos.x = 0;
//update vertical positions
clientPos.y += clientChunkSize.cy;
printPos.y += printChunkSize.cy;
pHtml2->put_scrollTop(clientPos.y);
chunkDestRectPx.top = printPos.y;
chunkDestRectPx.bottom = printPos.y + printChunkSize.cy;
}
//restore changed values on browser
//if for large pages on slow PC you get content scrolling during rendering and it is a problem,
//you can either hide the browser and show "working" or place on top first chunk content
pBodyStyle->put_overflow(keptOverflow.bstrVal);
pHtml2->put_scrollLeft(keptScrollPos.cx);
pHtml2->put_scrollTop(keptScrollPos.cy);
m_browser.put_Width(keptBrowserSize.cx);
m_browser.put_Height(keptBrowserSize.cy);
removeElement(pBody, pPadNode);
//draw to bitmap and close metafile
HENHMETAFILE hMetafile = CloseEnhMetaFile(hMetaDC);
RECT fitRect = { 0, 0, printHorRes, docSize.cy * printHorRes / docSize.cx };
convertEmfToBitmap(fitRect, hWndDc, hMetafile, L"c:\\temp\\test.bmp");
DeleteEnhMetaFile(hMetafile);
//cleanup - probably more here
::ReleaseDC(m_hWnd, hWndDc);
::DeleteDC(hPrintDC);
//{
// std::stringstream ss;
// ss << "====" << docSize.cx << "x" << docSize.cy << " -> " << fitRect.right << "x" << fitRect.bottom << "" << "\n";
// OutputDebugStringA(ss.str().c_str());
//}
}
///////////////
////some util
void convertEmfToBitmap(const RECT& fitRect, HDC hTargetDC, HENHMETAFILE hMetafile, LPCTSTR fileName)
{
//Create memory DC to render into
HDC hCompDc = ::CreateCompatibleDC(hTargetDC);
//NOTE this
BITMAPINFOHEADER infoHeader = { 0 };
infoHeader.biSize = sizeof(infoHeader);
infoHeader.biWidth = fitRect.right;
infoHeader.biHeight = -fitRect.bottom;
infoHeader.biPlanes = 1;
infoHeader.biBitCount = 24;
infoHeader.biCompression = BI_RGB;
BITMAPINFO info;
info.bmiHeader = infoHeader;
//create bitmap
BYTE* pMemory = 0;
HBITMAP hBitmap = ::CreateDIBSection(hCompDc, &info, DIB_RGB_COLORS, (void**)&pMemory, 0, 0);
HBITMAP hOldBmp = (HBITMAP)::SelectObject(hCompDc, hBitmap);
PlayEnhMetaFile(hCompDc, hMetafile, &fitRect);
BITMAPFILEHEADER fileHeader = { 0 };
fileHeader.bfType = 0x4d42;
fileHeader.bfSize = 0;
fileHeader.bfReserved1 = 0;
fileHeader.bfReserved2 = 0;
fileHeader.bfOffBits = sizeof(BITMAPFILEHEADER)+sizeof(BITMAPINFOHEADER);
CFile file(
fileName,
CFile::modeCreate | CFile::modeReadWrite | CFile::shareDenyNone);
file.Write((char*)&fileHeader, sizeof(fileHeader));
file.Write((char*)&infoHeader, sizeof(infoHeader));
int bytes = (((24 * infoHeader.biWidth + 31) & (~31)) / 8) * abs(infoHeader.biHeight);
file.Write(pMemory, bytes);
::SelectObject(hCompDc, hOldBmp);
//Clean up
if (hBitmap)
::DeleteObject(hBitmap);
::DeleteDC(hCompDc);
}
CComPtr<IHTMLDOMNode> appendPadElement(IHTMLDocument2* pDoc, IHTMLElement* pBody, long left, long top, long width, long height)
{
CComPtr<IHTMLElement> pPadElement;
pDoc->createElement(L"DIV", &pPadElement);
CComPtr<IHTMLStyle> pPadStyle;
pPadElement->get_style(&pPadStyle);
CComPtr<IHTMLStyle2> pPadStyle2;
pPadStyle->QueryInterface(IID_IHTMLStyle2, (void**)&pPadStyle2);
pPadStyle2->put_position(L"absolute");
CComVariant value = width;
pPadStyle->put_width(value);
value = height;
pPadStyle->put_height(value);
pPadStyle->put_posLeft((float)left);
pPadStyle->put_posTop((float)top);
CComPtr<IHTMLDOMNode> pPadNode;
pPadElement->QueryInterface(IID_IHTMLDOMNode, (void**)&pPadNode);
CComPtr<IHTMLDOMNode> pBodyNode;
pBody->QueryInterface(IID_IHTMLDOMNode, (void **)&pBodyNode);
pBodyNode->appendChild(pPadNode, NULL);
return pPadNode;
}
void removeElement(IHTMLElement* pParent, IHTMLDOMNode* pChild)
{
CComPtr<IHTMLDOMNode> pNode;
pParent->QueryInterface(IID_IHTMLDOMNode, (void **)&pNode);
pNode->removeChild(pChild, NULL);
}
Sample page output (4958x7656)
I have taken your code and run it on IE11 when the WebBrowser control is smaller then the page size. It rendered a portion of the page equal to control's size. Not sure why you say IE8 and IE11 are any different.
It seems that common approach to taking full page screenshots is adjusting WebBrowser size before taking screenshot, like this:
const long oldH = m_browser.get_Height();
const long oldW = m_browser.get_Width();
m_browser.put_Height(n_scrollHeight);
m_browser.put_Width(n_scrollWidth);
//Do the upscaling & render -- note that IE8 fails to render it here!!!!
pViewObject->Draw(DVASPECT_CONTENT, //DVASPECT_DOCPRINT
-1, NULL, NULL, NULL, hCompDc,
&rectPrnt,
NULL,
NULL, 0);
m_browser.put_Height(oldH);
m_browser.put_Width(oldW);
This seems to work well, even on large pages such as the one you're currently reading (I have taken screenshot 1920x8477). It works both on my IE11 and on IE8 virtual machine
It has a side effect of resetting scrollbars, but that can be solved, for example by using an invisible copy of WebBrowser for screenshots.
PS: You could have done a better job by providing example code that can be compiled, at least ;)
Context:
I'm developing a native C++ Unity 5 plugin that reads in DXT compressed texture data and uploads it to the GPU for further use in Unity. The aim is to create an fast image-sequence player, updating image data on-the-fly. The textures are compressed with an offline console application.
Unity can work with different graphics engines, I'm aiming towards DirectX11 and OpenGL 3.3+.
Problem:
The DirectX runtime texture update code, through a mapped subresource, gives different outputs on different graphics drivers. Updating a texture through such a mapped resource means mapping a pointer to the texture data and memcpy'ing the data from the RAM buffer to the mapped GPU buffer. Doing so, different drivers seem to expect different parameters for the row pitch value when copying bytes. I never had problems on the several Nvidia GPU's I tested on, but AMD and Intel GPU seems to act differently and I get distorted output as shown underneath. Furthermore, I'm working with DXT1 pixel data (0.5bpp) and DXT5 data (1bpp). I can't seem to get the correct pitch parameter for these DXT textures.
Code:
The following initialisation code for generating the d3d11 texture and filling it with initial texture data - e.g. the first frame of an image sequence - works perfect on all drivers. The player pointer points to a custom class that handles all file reads and contains getters for the current loaded DXT compressed frame, it's dimensions, etc...
if (s_DeviceType == kUnityGfxRendererD3D11)
{
HRESULT hr;
DXGI_FORMAT format = (compression_type == DxtCompressionType::DXT_TYPE_DXT1_NO_ALPHA) ? DXGI_FORMAT_BC1_UNORM : DXGI_FORMAT_BC3_UNORM;
// Create texture
D3D11_TEXTURE2D_DESC desc;
desc.Width = w;
desc.Height = h;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = format;
// no anti-aliasing
desc.SampleDesc.Count = 1;
desc.SampleDesc.Quality = 0;
desc.Usage = D3D11_USAGE_DYNAMIC;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
desc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
desc.MiscFlags = 0;
// Initial data: first frame
D3D11_SUBRESOURCE_DATA data;
data.pSysMem = player->getBufferPtr();
data.SysMemPitch = 16 * (player->getWidth() / 4);
data.SysMemSlicePitch = 0; // just a 2d texture, no depth
// Init with initial data
hr = g_D3D11Device->CreateTexture2D(&desc, &data, &dxt_d3d_tex);
if (SUCCEEDED(hr) && dxt_d3d_tex != 0)
{
DXT_VERBOSE("Succesfully created D3D Texture.");
DXT_VERBOSE("Creating D3D SRV.");
D3D11_SHADER_RESOURCE_VIEW_DESC SRVDesc;
memset(&SRVDesc, 0, sizeof(SRVDesc));
SRVDesc.Format = format;
SRVDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
SRVDesc.Texture2D.MipLevels = 1;
hr = g_D3D11Device->CreateShaderResourceView(dxt_d3d_tex, &SRVDesc, &textureView);
if (FAILED(hr))
{
dxt_d3d_tex->Release();
return hr;
}
DXT_VERBOSE("Succesfully created D3D SRV.");
}
else
{
DXT_ERROR("Error creating D3D texture.")
}
}
The following update code that runs for each new frame has the error somewhere. Please note the commented line containing method 1 using a simple memcpy without any rowpitch specified which works well on NVIDIA drivers.
You can see further in method 2 that I log the different row pitch values. For instace for a 1920x960 frame I get 1920 for the buffer stride, and 2048 for the runtime stride. This 128 pixels difference probably have to be padded (as can be seen in the example pic below) but I can't figure out how. When I just use the mappedResource.RowPitch without dividing it by 4 (done by the bitshift), Unity crashes.
ID3D11DeviceContext* ctx = NULL;
g_D3D11Device->GetImmediateContext(&ctx);
if (dxt_d3d_tex && bShouldUpload)
{
if (player->gather_stats) before_upload = ns();
D3D11_MAPPED_SUBRESOURCE mappedResource;
ctx->Map(dxt_d3d_tex, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
/* 1: THIS CODE WORKS ON ALL NVIDIA DRIVERS BUT GENERATES DISTORTED OR NO OUTPUT ON AMD/INTEL: */
//memcpy(mappedResource.pData, player->getBufferPtr(), player->getBytesPerFrame());
/* 2: THIS CODE GENERATES OUTPUT BUT SEEMS TO NEED PADDING? */
BYTE* mappedData = reinterpret_cast<BYTE*>(mappedResource.pData);
BYTE* buffer = player->getBufferPtr();
UINT height = player->getHeight();
UINT buffer_stride = player->getBytesPerFrame() / player->getHeight();
UINT runtime_stride = mappedResource.RowPitch >> 2;
DXT_VERBOSE("Buffer stride: %d", buffer_stride);
DXT_VERBOSE("Runtime stride: %d", runtime_stride);
for (UINT i = 0; i < height; ++i)
{
memcpy(mappedData, buffer, buffer_stride);
mappedData += runtime_stride;
buffer += buffer_stride;
}
ctx->Unmap(dxt_d3d_tex, 0);
}
Example pic 1 - distorted ouput when using memcpy to copy whole buffer without using separate row pitch on AMD/INTEL (method 1)
Example pic 2 - better but still erroneous output when using above code with mappedResource.RowPitch on AMD/INTEL (method 2). The blue bars indicate zone of error, and need to disappear so all pixels align well and form one image.
Thanks for any pointers!
Best,
Vincent
The mapped data row pitch is in byte, when you divide by four, it is definitely an issue.
UINT runtime_stride = mappedResource.RowPitch >> 2;
...
mappedData += runtime_stride; // here you are only jumping one quarter of a row
It is the height count with a BC format that is divide by 4.
Also a BC1 format is 8 bytes per 4x4 block, so the line below should by 8 * and not 16 *, but as long as you handle row stride properly on your side, d3d will understand, you just waste half the memory here.
data.SysMemPitch = 16 * (player->getWidth() / 4);
Here is my trying
in = cvLoadImage("24bpp_1920x1200_1.bmp", 1);
HRESULT err;
IDirect3DTexture9 * texture = NULL;
///D3DFMT_L8, D3DFMT_R8G8B8
err = D3DXCreateTexture(g_pd3dDevice, in->width, in->height, 1, 0 , D3DFMT_R8G8B8, D3DPOOL_MANAGED, &g_pTexture);
D3DLOCKED_RECT lockRect;
RECT rect;
err = g_pTexture->LockRect(0, &lockRect, NULL, 0);//I have specified the format is RGB, then why does lockRect.Picth = 7680?
memcpy(lockRect.pBits, in->imageData, in->widthStep*in->height);
if(FAILED(g_pTexture->UnlockRect(0)))
{
///
}
It can't display an image in RGB format. But it can display an image in grayscale or in RGBA format.
Otherwise, I want to display high resolution image like the sample of displaying image using d3d in Dx Sdk_june10".\DXSDK\Samples\InstalledSamples\Textures" does.
But, again, it can't display an image in size more than 1920x1200px approximately.
How to do ?
Two things to keep in mind:
(1) You need to check for caps that indicates support for the 24-bpp format. Not every Direct3D 9 device supports it.
(2) You have to respect the pitch returned by LockRect, so you need to do the copy scan-line by scan-line, so you can't do the whole thing in just one memcpy. Something like:
BYTE* sptr = in->imageData;
BYTE* dptr = lockRect.pBits;
for( int y = 0; y < in->height; ++y )
{
memcpy( dptr, sptr, in->widthStep );
sptr += in->widthStep;
dptr += lockRect.Pitch;
}
This assumes in->widthStep is the pitch of the source image in bytes, that in->widthStep is <= lockRect.Pitch, and that the format of the source image is identical to the format of the Direct3D resource.