Unreal Engine 4: save rendered frame to memory - c++

Previously, I was looking to output frames rendered by UE4 to file.
I managed to do this and the details can be found in this StackOverflow post
The function to out put the frame to file is:
FScreenshotRequest::RequestScreenshot(filename, false, false);
Now, instead of writing to file, I would like to write to memory. I don't want to write to file and then read into memory.
I have been digging through the source code and found where screenshots are being made, but am having some trouble.
ViewportClient->ProcessScreenShots(this); is being called on line 1012 of UnrealClient.cpp
Following that, I found that the screenshot is actually being generated here:
bScreenshotSuccessful = GetViewportScreenShot(InViewport, Bitmap);
So, after finding all the bits that I think I need, I tried to recreate it in a custom Actor:
UGameViewportClient* gameViewport = GEngine->GameViewport;
FViewport* InViewport = gameViewport->Viewport;
TArray<FColor> Bitmap;
bool bScreenshotSuccessful = GetViewportScreenShot(InViewport, Bitmap);
if (bScreenshotSuccessful){
FIntVector Size(InViewport->GetSizeXY().X, InViewport->GetSizeXY().Y, 0);
TArray<uint8> CompressedBitmap;
FString ScreenShotName = TEXT("out.png");
FImageUtils::CompressImageArray(Size.X, Size.Y, Bitmap, CompressedBitmap);
FFileHelper::SaveArrayToFile(CompressedBitmap, *ScreenShotName);
}
For some reason, bool bScreenshotSuccessful = GetViewportScreenShot(InViewport, Bitmap); throws an Access violation reading location 0x0000000000000020. exception.
I think the error has something to do with this line:
Viewport->ReadPixels(Bitmap, FReadSurfaceDataFlags(), ViewRect)
I've treid 'googeling' for what an 'Access violation' is, and it seems it is something to do with a null pointer or something like that, but I am still not able to figure this out as I'm rather new to c++.
Question
How can I fix this so that bScreenshotSuccessful is true?
NOTE: I realise that FFileHelper::SaveArrayToFile(CompressedBitmap, *ScreenShotName); attempts to save to file, despite me saying that is not what I want to do, so please ignore that as if I have the bitmap, I can compress and stream it.

You have ViewRect just of three points instead of four and in wrong format. Try this snippet:
FIntRect Rect(0, 0, InViewport->GetSizeXY().X, InViewport->GetSizeXY().Y);
bScreenshotSuccessful = GetViewportScreenShot(InViewport, Bitmap, Rect);

Related

Cannot get an image handle from a bitmap within the resources when using "LoadImageA()" and cannot understand why

I am trying to load an image resource using the LoadImageA() function, yet it doesn't work and I don't understand why.
Here's a bit of my code :
bool isRessource = IS_INTRESOURCE(107);
// Load the resource to the HGLOBAL.
HGLOBAL imageResDataHandle = LoadImageA(
NULL,
MAKEINTRESOURCEA(107),
IMAGE_BITMAP,
0,
0,
LR_SHARED
);
HRESULT hr = (imageResDataHandle ? S_OK : E_FAIL);
The image I want to load is a bitmap saved in the resources, and represented as such within resources.h:
#define IDB_BITMAP1 107
When I execute the code, isRessource is equal to true, yet hr is equal to E_FAIL.
Any idea as to why this is happening? I am using Visual Studio 2019, and I made the image using Gimp.
After making the same image with the same format on another application (I used "Krita") and importing it again, the image finally loads with the same code (I only changed the reference to the resource). I guess that all types of bitmaps made from Gimp won't work in Visual Studio (I tried most formats of bitmaps from Gimp).
The first link searched with LoadImage gimp as a keyword is enough to answer this question.
This is some useful information:
The bitmap exported by GIMP has a broken header. Specifically, the
code seems to not write the RGBA masks, which AFAIK are not optional
in a BITMAPV5HEADER. This misaligns and changes the size of the entire
extended header, incidentally making it look like a BITMAPV4HEADER,
which explains why most programs will still open it fine. Without
having done any testing, I'd guess LoadImage() is more picky about the
values in this extended header; returning NULL is how it indicates
failure.
By the way, when you import a bitmap, the system does not remind you that the format of the image is unknown?
Like:
After testing, use LoadImage to load such an image will return NULL, and GetLastError will also return 0.

Trying to encode a GIF file using giflib

I am given image data and color table I am trying to export it as a single frame GIF using giflib. I looked into the API, but can't get it to work. The program crashes even at the first function:
GifFileType image_out;
int errorCode = 0;
char* fileName = "SomeName.gif";
image_out = *EGifOpenFileName(fileName,true, &errorCode);
It is my understanding that I first need to open a file by specifying it's name and then update it with fileHandle. Then Fill in the screen description, the extension block the image data and add the 3B ending to the file. Then use EGifSpew to export the whole gif. The problem is that I can't even use EGifOpenFileName(); The program crashes at that line.
Can someone help me the API of giflib? This problem is getting really frustrating.
Thanks.
EDIT:
For the purposes of simple encoding I do not want to specify a color table and I just want to encode a single frame GIF.
The prototype is:
GifFileType *EGifOpenFileName(char *GifFileName, bool GifTestExistance, int *ErrorCode)
You should write as
GifFileType* image_out = EGifOpenFileName(fileName,true, &errorCode);
Note GifFileType is not POD type so you should NOT copy like that.

How to apply image effects to a JPEG in a memory buffer using LEADTOOLS 19

I'm making a Windows executable using C++, LEADTOOLS19, and VS2015 to read an image from a server, apply an image effect to it using LEADTOOLS, and display it in a browser.
The server provides me the image as an array of chars containing the JPEG encoding of the image (starting with "ÿØÿà") and the length of this buffer. Most of the LEADTOOLS functions read images from files, but I don't want to have to write it to disk just to read it as a bitmap.
The first thing I tried was the StartFeedLoad function:
//pImageData is the buffer of JPEG data, and imageLength is the
//server-provided size of pImageData in bytes
LBuffer buf((L_VOID *)pImageData, imageLength);
LFile imgFile;
LBitmap bitmap;
imgFile.SetBitmap(&imgbitmap);
// Initialize the file-load process
imgmemfile.StartFeedLoad(8, 0,
LOADFILE_ALLOCATE | LOADFILE_STORE, NULL);
imgmemfile.FeedLoad(&buf);
imgmemfile.StopFeedLoad();
With this code, I get this exception when trying to run StartFeedLoad:
Exception thrown at 0x000007F855BC2662 (ltwvcax.dll) in getimage.exe:
0xC0000005: Access violation reading location 0x0000000000000148.
I tried a few different things before calling StartFeedLoad, and tried changing the parameters I was passing it, but got that exception every time.
With that not working, the next method I tried was to save the buffer as an in-memory file using the LEADTOOLS library LMemoryFile class:
LBuffer buf((L_VOID *)pImageData, imageLength);
LMemoryFile imgmemfile;
BITMAPHANDLE pbit;
//The bitmap the image will be loaded into
LBitmap bitmap;
imgmemfile.SetBitmap(&bitmap);
//Load the buffer to the image
ret = imgmemfile.LoadMemory(buf, 0, ORDER_RGBORGRAY, LOADFILE_ALLOCATE | LOADFILE_STORE, NULL);
At this point, LoadMemory returns WRPERR_INVALID_PARAMETERS: One or more invalid parameters were specified. I've tried different bitsPerPixel values, color orders, and with or without adding another NULL parameter as fileInfo but still get the same error.
I feel like I need to do something else to "prep" the bitmap to load, but I don't know it's size or anything else to initialize it.
Thanks!
EDIT 5/9/16: Added "GetInfo" as indicated by Leadtools:
//Load image
LBuffer buf((L_VOID *)pImageData, imageLength);
//LFile imgmemfile;
FILEINFO fileInfo = FILEINFO();
LMemoryFile imgmemfile;
BITMAPHANDLE pbit;
if (LBase::GetLoadedLibraries() & LT_FIL == 0)
return false;
LBitmap bitmap;
imgmemfile.SetBitmap(&bitmap);
ret = imgmemfile.GetInfo(buf, &fileInfo, sizeof(FILEINFO), 0, NULL);
ret = imgmemfile.LoadMemory(buf, 0, ORDER_RGBORGRAY, LOADFILE_ALLOCATE | LOADFILE_STORE, NULL, &fileInfo);
ret = imgmemfile.Save(&buf, FILE_JPEG, 8, 30, NULL);
The code gets past the additional library check, but GetInfo returns -2041, indicating that LTFIL isn't loaded.
You should use LMemoryFile::GetInfo and LMemoryFile::LoadMemory if you have the whole file in memory at the start. If you don't, then FeedLoad is the way to go. There is an example here: https://www.leadtools.com/help/leadtools/v19/main/clib/lfile__startfeedload.html
You can find a full working example in your LEADTOOLS installation folder: C:\LEADTOOLS 19\Examples\ClassLibrary\MSVC\FeedLoad
The functions that LEADTOOLS Support gave me were correct, but I was still ahving the issue because I was linking to both the Unicode and ANSI versions of the Leadtools C++ class library (Ltwvc_x.lib AND Ltwvc_ax.lib). When I removed the Unicode library (from my ANSI project) everything worked fine.

Using new to allocate memory for unsigned char array fails

I'm trying to load a tga file in c++ code that I got from google searching, but the part that allocates memory fails. The beginning of my "LoadTarga" method includes these variables:
int imageSize;
unsigned char* targaImage;
Later on in the method the imageSize variable gets set to 262144 and I use that number to set the size of the array:
// Calculate the size of the 32 bit image data.
imageSize = width * height * 4;
// Allocate memory for the targa image data.
targaImage = new unsigned char[imageSize];
if (!targaImage)
{
MessageBox(hwnd, L"LoadTarga - failed to allocate memory for the targa image data", L"Error", MB_OK);
return false;
}
The problem is that the body of the if statement executes and I have no idea why the memory allocation failed. As far as I know it should work - I know the code compiles and runs up to this point and I haven't seen anything yet in google that would show a proper alternative.
What should I change in my code to make it allocate memory correctly?
Important Update:
Rob L's comments and suggestions were very useful (though I didn't try _heapchk since I solved the issue before I tried using it)
Trying each of fritzone's ideas meant the program ran past the "if (!targaImage)" point without trouble. The code that sets "targaImage and the if statement checks if memory was allocated correctly has been replaced with this:
try
{
targaImage = new unsigned char[imageSize];
}
catch (std::bad_alloc& ba)
{
std::cerr << "bad_alloc caught: " << ba.what() << '\n';
return false;
}
However I got a new problem with the very next bit of code:
count = (unsigned int)fread(targaImage, 1, imageSize, filePtr);
if (count != imageSize)
{
MessageBox(hwnd, L"LoadTarga - failed to read in the targa image data", L"Error", MB_OK);
return false;
}
Count was giving me a value of "250394" which is different to imageSize's value of "262144". I couldn't figure out why this was and doing a bit of searching (though I must admit, not much searching) on how "fread" works didn't yield info.
I decided to cancel my search and try the answer code on the tutorial site here http://www.rastertek.com/dx11s2tut05.html (scroll to the bottom of the page where it says "Source Code and Data Files" and download the zip. However creating a new project, putting in the source files and image file didn't work as I got a new error. At this point I thought maybe the way I converted the image file from to tga might have been incorrect.
So rather than spend a whole lot of time debugging the answer code I put the image file from the answer into my own project. I noted that the size of mine was MUCH smaller than the answer (245KB compared to 1025 KB) )so maybe if I use the answer code's image my code would run fine. Turns out I was right! Now the image is stretched sideways for some reason but my original query appears to have been solved.
Thanks Rob L and fritzone for your help!
You are NOT using the form of new which returns a null pointer in case of error, so it makes no sense for checking the return value. Instead you should be aware of catching a std::bad_alloc. The null pointer returning new for you has the syntax: new (std::nothrow) unsigned char[imageSize];
Please see: http://www.cplusplus.com/reference/new/operator%20new[]/
Nothing in your sample looks wrong. It is pretty unlikely that a modern Windows system will run out of memory allocating 256k just once. Perhaps your allocator is being called in a loop and allocating more than you think, or the value of imagesize is wrong. Look in the debugger.
Another possibility is that your heap is corrupt. Calling _heapchk() can help diagnose that.
Check the "memory peak working set" in windows tasks manager and ensure how much memory you are really trying to allocate.

converting HBITMAP to byte array

I'm working with some scanner api which returns a HANDLE to an image in BMP format(so it is said so in documentation). I'm trying to somehow get BITMAP from this handle, but for example this code doesn't work:
HANDLE handle = getHandleFromScanner();
BITMAP bitmap;
int u = GetObject(handle, sizeof(BITMAP), &bitmap);
u is 0 here and getLastError() returns 6 which means that handle is invalid. But I cannot get any other handle except through getHandleFromScanner() function.
May be some transformations should be done with this handle? any ideas? What is the proper way to work with bitmap handles? Or any simple api exists? Samples I found via google didn't help me.
Thanks a lot.
Have you tried GetDIBits()? This should work but you'll need the device context as well. You may always want to call GetObjectType() on the handle to see if it is really returning an HBITMAP.
Thanks guys.
The handle appeared to be the image itself, so the following code solved the problem:
char* pImage = NULL;
HANDLE hImage= getHandleFromScanner();
pImage = (char *)GlobalLock(hImage);
// pImage now contains the bytes of the image
If someone will ever need it, the Scanner is Olivetti PR2 plus scanner.