C++ VIDEOHDR to Image File in capVideoStreamCallback - c++

I've been working on a Webcam video recorder and I got interested in trying everything when it comes to this topic but there's this problem that I can't solve.
Everything that you might wonder about can be found here
https://msdn.microsoft.com/en-us/library/windows/desktop/dd757677%28v=vs.85%29.aspx and here
https://msdn.microsoft.com/en-us/library/windows/desktop/dd757694%28v=vs.85%29.aspx
Now, in this code
if (capSetCallbackOnVideoStream(hCapWnd, capVideoStreamCallback))
{
capCaptureSequenceNoFile(hCapWnd); //Capture
}
I make sure that every frame that gets captured is sent to capVideoStreamCallback.
Now what I'm trying to do is transform a frame to an image and save it somewhere, this might be useless but it's interesting and it is surely possible.
Here is my capVideoStreamCallback function (it's commented):
LRESULT CALLBACK capVideoStreamCallback(HWND hWnd, LPVIDEOHDR lpVHdr)
{
BYTE *Image;
BITMAPINFO * TempBitmapInfo = new BITMAPINFO;
ULONG Size;
// First we need to get the full size of the image
Size = capGetVideoFormat(hWnd, TempBitmapInfo, sizeof(BITMAPINFO)); //header size
Size += lpVHdr->dwBytesUsed; //bytes used
Image = new BYTE[Size];
memcpy(Image, TempBitmapInfo, sizeof(BITMAPINFO)); //copy the header to Image
// lpVHdr is LPVIDEOHER passed into callback function.
memcpy(Image + sizeof(BITMAPINFO), lpVHdr->lpData, lpVHdr->dwBytesUsed); //copy the data to Image
//write the image
ofstream output("image.dib", ios::binary);
for (int i = 0; i < Size; i++)
{
output << (BYTE)Image[i];
}
output.close();
return (LRESULT)TRUE;
}
So, the information about every frame that gets sent to capVideoStreamCallback can be found in lpVHdr which is a structure (https://msdn.microsoft.com/en-us/library/windows/desktop/dd757688%28v=vs.85%29.aspx) and what I'm trying to do here is to take that information and transform it to an image.
I first start by getting the full size of the image by retrieving the size of the header and the size of the data and then I dynamically declared a BYTE Array called Image and copied the header and the data to Image using memcpy. I finally used ofstream to write the bytes to a file and that's pretty much it.
The problem is that everything works just fine but the image is somehow corrupted because it cannot be opened.
What is wrong in what I'm doing? It seems so logical but it's not working.
Please share your ideas and thanks for reading.

Here's the answer thanks to Frankie-C from http://codeproject.com who reminded me that I needed a BITMAPFILEHEADER structure at the top of the BITMAP File.
There's also few extra stuff that you need to do to get the image to show up the way it should be such as flipping bytes to get BGR instead of RGB etc, here's a nice tut explaining that: http://tipsandtricks.runicsoft.com/Cpp/BitmapTutorial.htm

Related

Print the pixel array from a bmp image

So, I have a 24 bmp image saved in the memory and I want to print the array of pixels, I am using BITMAPFILEHEADER and BITMAPINFOHEADER for figuring out the header and info header of the bitmap, however when looking up documentation I see that the file header has a member called bfOffBits that should show the starting location of the pixels and I can get the size of the pixel array through info_header->Height and Width.
My issues is that when looking at the wikipedia (https://en.wikipedia.org/wiki/BMP_file_format#Bitmap_file_header) page I need to figure out the padding bytes size and the actual pixel bytes. But my image has a Width of 1152, I get that by printing:
BITMAPINFOHEADER* bitmap_info_header = NULL;
bitmap_info_header = (BITMAPINFOHEADER*)(file_info->file_address + sizeof(BITMAPFILEHEADER));
printf("Width %ld \n", bitmap_info_header->biWidth);
and i'm not sure if that means there is no padding? Also when starting to read the pixels I start from :
typedef struct
{
HANDLE file_handle;
HANDLE mapping_handle;
BYTE* file_address;
DWORD file_size;
}FILE_INFO;
file_info->file_address[bitmap_file_header->bfOffBits]
And I am not sure that is correct because reading through https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapinfoheader I find a "Colour table " that I haven't identified.
I just want to find out if there is padding or not and if I read from the correct address, as I get them mixed up a lot. thank you !

Gray scale images processing in C++

How to read an image in C++ as a 2D array? I need to create a C/C++ program that reads an image (all formats) as a 2D array to show pixel values (0-255), divide the image into blocks and apply different compression methods using pixels blocks (BTC, AMBTC, MMBTC) and saving the new image by hand without using already set libraries (must not use magic++)..
thanks in advance
Here's some 'outline' code using MFC's CImage class that may help you. I've shown how to use the basic Load and Save options, and how to get a 'raw' array of pixel data (note: it's best to convert to 32-bit format, so you can be sure the DWORD pointer you get will really be to a width X height array - other BPP formats can give strange results):
First, load from file (CImage will know or guess the format from the file extension):
CImage original;
original.Load("Yourfile.jpg"); // Use actual file path, obviously
int pw = original.GetWidth(), ph = original.GetHeight(); // Dimensions
CImage working; // Use this to hold our 32-bit image
working.Create(pw, ph, 32);
// Next, copy image from original to working...
HDC hDC = working.GetDC();
original.BitBlt(hDC, 0, 0, SRCCOPY);
working.ReleaseDC();
// Get a DWORD pointer to the pixel data...
BITMAP bmp;
GetObject(working.operator HBITMAP(), sizeof(BITMAP), &bmp);
DWORD* pixbuf = static_cast<DWORD*>(bmp.bmBits);
// We can now access any pixel(x,y) data using: pixbuf[x + y * pw]
You now do all sorts of work on your image buffer, using the pixbuf array, as stated in the comment. For clarity: each DWORD (32-bit unsigned) in the buffer will be the RGBA data (where A is the 'alpha` channel - set to zero) but in reversed order; so, for each DWORD, the bytes will be 0xBBBBGGGGRRRR0000.
When you're done, you can save your modified image as follows:
CImage savepic;
savepic.Create(pw, ph, 24); // Change 24 to whatever BPP you require
hDC = savepic.GetDC();
working.BitBlt(hDC, 0, 0, SRCCOPY); // Copies modified image to output
savepic.ReleaseDC();
savepic.Save("NewFile.jpg"); // CImage understand what format to use base on extension
Of course, in a real-world program, there are error checks that you will need to make (most CImage methods return a status indicator, and GetLastError() can be used), and you would probably be safer copying the 'pixbuf' data to a separate memory zone - but, hopefully, this brief outline will help you get started.
Feel free to ask for further clarification and/or explanation.

how to write images as gif to blob using imagemagick Magick++

vector<Magick::Image> frames;
int delay = 20;
for(auto iter=taskList.begin(); iter != taskList.end(); ++iter){
/* hide some codes here*/
frames.push_back(*img);
}
// write images to file, works fine
Magick::writeImages(frames.begin(), frames.end(), "xxx.gif");
Magick::Blob tmpBlob;
// write images to blob, I then decode the data in blob,
// and write this blob to yyy.gif. The gif file only contains the first frame image.
Magick::writeImages(frames.begin(), frames.end(), &tmpBlob, true);
// the length is far too small
LOG_DEBUG("blob data length: %d", tmpBlob.length());
// read from the blob into a imagelist, and print the size of the list
// the size is 1
vector<Magick::Image> image_list;
Magick::readImages(&image_list, tmpBlob);
LOG_DEBUG("new frames length: %d", image_list.size());
hi, I have a problem when I try to write a list of Image to Blob with ImageMagick(version 7.0.3) Magick++ STL.h writeImages function. It doesn't work correctly, it seems that only one frame was written into the blob. But with the same image list, writing them to gif file works just fine. could anybody help me out?
problem solved. The reason why I failed is that I didn't do img->magick("GIF"), which cause the failure of writing to blob correctly.

Openni2 reading recording (.oni) in the wrong order

I am having issues with reading a recording I made using the Recorder class in openni2 with the asus Xtion PRO LIVE. The problem is that once every ~50 frames a wrong frame is read from the recording, this was tested by storing the generated image (converted to an opencv matrix) as an .png image (using the opencv imwrite function) with an index. Shown below is my code, I also tried the code posted in the "OpenNI2: Reading in .oni Recording" question. This also doesnt work. The videostreams, device, videoFrameRefs and the playback controller are all initialized in an initialization function. I can post this as well if necessary.
openni::VideoStream depthStream;
openni::VideoStream colorStream;
openni::Device device;
openni::VideoFrameRef m_depthFrame;
openni::VideoFrameRef m_colorFrame;
openni::PlaybackControl *controler;
typedef struct _ImagePair {
cv::Mat GrayscaleImage;
cv::Mat DepthImage;
cv::Mat RGBImage;
float AverageDistance;
std::string FileName;
int index;
}ImagePair;
ImagePair SensorData::getNextRawImagePair(){
ImagePair result;
openni::Status rc;
std::cout<<"Getting next raw image pair...";
rc = controler->seek(depthStream, rawPairIndex);
if(!correctStatus(rc,"Seek depth"))return result;
openni::VideoStream** m_streams = new openni::VideoStream*[2];
m_streams[0] = &depthStream;
m_streams[1] = &colorStream;
bool newDepth=false,newColor=false;
while(!newDepth || !newColor){
int changedIndex;
openni::Status rc = openni::OpenNI::waitForAnyStream(m_streams, 2, &changedIndex);
if (rc != openni::STATUS_OK)
{
printf("Wait failed\n");
//return 1;
}
switch (changedIndex)
{
case 0:
rc = depthStream.readFrame(&m_depthFrame);
if(!correctStatus(rc,"Read depth")){
return result;
}
newDepth = true;
break;
case 1:
rc = colorStream.readFrame(&m_colorFrame);
if(!correctStatus(rc,"Read color")){
return result;
}
newColor = true;
break;
default:
printf("Error in wait\n");
}
}
//convert rgb to bgr cv::matrix
cv::Mat cv_image;
const openni::RGB888Pixel* colorImageBuffer = (const openni::RGB888Pixel*)m_colorFrame.getData();
cv_image.create(m_colorFrame.getHeight(), m_colorFrame.getWidth(), CV_8UC3);
memcpy( cv_image.data, colorImageBuffer,3*m_colorFrame.getHeight()*m_colorFrame.getWidth()*sizeof(uint8_t) );
//convert to BGR opencv color order
cv::cvtColor(cv_image,cv_image,cv::COLOR_RGB2BGR);
result.RGBImage = cv_image.clone();
//convert to grayscale
cv::cvtColor(cv_image,cv_image,cv::COLOR_BGR2GRAY);
result.GrayscaleImage =cv_image.clone();
//convert depth to cv::Mat INT16
const openni::DepthPixel* depthImageBuffer = (const openni::DepthPixel*)m_depthFrame.getData();
cv_image.create(m_depthFrame.getHeight(), m_depthFrame.getWidth(), CV_16UC1);
memcpy( cv_image.data, depthImageBuffer,m_depthFrame.getHeight()*m_depthFrame.getWidth()*sizeof(INT16) );
result.DepthImage = cv_image.clone();
result.index = rawPairIndex;
rawPairIndex++;
std::cout<<"done"<<std::endl;
return result;
}
I also tried it without the using waitForAnyStream part but that only made it worse. The file I am trying to load is over 1GB, not sure if that is a problem. The behaviour seems random because the indexes of the wrong images are not always the same.
UPDATE:
I changed the seek function to seek in the color stream instead of the depth stream. I also ensured that each stream is only waited for once in the waitForAnyStream function by setting the corresponding point in m_streams to null.
I found out that it is possible to find the actual frame index for each frame (.getFrameIndex) so I was able to compare the indices. After getting 1000 images there where 18 wrong color images, 17 of which had an index error of 53 and 1 had an error of 46. The depth images had an almost constant index error of 9.
The second thing I found was after adding a sleep of 1 ms (also tried 10ms but the results where the same) before the waitForAnyStream function the returned indices change significantly. The indices dont make large jumps anymore but the color images have a standard offset of 53 and the depth images have an offset of 9.
When I change the seek function to seek in the depth stream then with the delay the color stream has a constant error of 46 and the depth stream has an error of 1. Without the delay the color stream has an error of 45 and the depth stream has no error with occasional spikes of errors of 1.
I also looked at the indices when I dont use seek and waitForAnyStream but just do it as is proposed as an answer in "OpenNI2: Reading in .oni Recording". This shows that when the file is read by just calling m_readframe multiple times then the first color frame has index 59 instead of 0. After checking it turns out that frame 0 does excist and is an earlier frame than frame 59. So just opening the file and using readframe doesnt work at all. There are however no index errors, just like when I added the sleep to my own implementation.
UPDATE2:
I have come to the conclusion that either the openni2 library doesnt work properly or I have installed it incorrectly. This is because I am also having problems setting the Xtion to a resolution of 640x480 for both streams. When I do this the depth image only gets updated once every ~100 seconds. I have written a quick fix for my original problem by just filtering out the frames who's indices are wrong and just continuing with the next image.
I would still like to know how to fix this, so if you know please tell me.
UPDATE 3:
The code I am using for setting the framerate and fps for recording is:
//colorstream
openni::VideoMode color_videoMode = m_colorStream.getVideoMode();
color_videoMode.setResolution(WIDTH,HEIGHT);
color_videoMode.setFps(30);
m_colorStream.setVideoMode(color_videoMode);
//depth stream
openni::VideoMode depth_videoMode = m_depthStream.getVideoMode();
depth_videoMode.setResolution(WIDTH,HEIGHT);
depth_videoMode.setFps(30);
m_depthStream.setVideoMode(depth_videoMode);
I forgot to mention that I also tested the resolution by running the sampleviewer example program (I think, it was a few months ago) and changing the resolution in the xml config file. Here the color images would be shown fine but the depth images would update verry rarely.

ISampleGrabber::BufferCB to IplImage; display in OpenCV shows garbled image - C++

I'm using DirectShow to access a video stream, and then using the SampleGrabber filter and interface to get samples from each frame for further image processing. I'm using a callback, so it gets called after each new frame. I've basically just worked from the PlayCap sample application and added a sample filter to the graph.
The problem I'm having is that I'm trying to display the grabbed samples on a different OpenCV window. However, when I try to cast the information in the buffer to an IplImage, I get a garbled mess of pixels. The code for the BufferCB call is below, sans any proper error handling:
STDMETHODIMP BufferCB(double Time, BYTE *pBuffer, long BufferLen)
{
AM_MEDIA_TYPE type;
g_pGrabber->GetConnectedMediaType(&type);
VIDEOINFOHEADER *pVih = (VIDEOINFOHEADER *)type.pbFormat;
BITMAPINFO* bmi = (BITMAPINFO *)&pVih->bmiHeader;
BITMAPINFOHEADER* bmih = &(bmi->bmiHeader);
int channels = bmih->biBitCount / 8;
mih->biPlanes = 1;
bmih->biBitCount = 24;
bmih->biCompression = BI_RGB;
IplImage *Image = cvCreateImage(cvSize(bmih->biWidth, bmih->biHeight), IPL_DEPTH_8U, channels);
Image->imageSize = BufferLen;
CopyMemory(Image->imageData, pBuffer, BufferLen);
cvFlip(Image);
//openCV Mat creation
Mat cvMat = Mat(Image, true);
imshow("Display window", cvMat); // Show our image inside it.
waitKey(2);
return S_OK;
}
My question is, am I doing something wrong here that will make the image displayed look like this:
Am I missing header information or something?
The quoted code is a part of the solution. You create here an image object of certain width/height with 8-bit pixel data and unknown channel/component count. Then you copy data from another buffer of unknown format.
The only chance for it to work well is that all unknowns amazingly match without your effort. So you basically need to start with checking what media type is exactly on Sample Grabber's input pin. Then, if it is not what you wanted, you have to update your code respectively. It might also be important what is the downstream connection of the SG, and whether it is connected to video renderer in particular.