Due to some reasons I need to access directly to image buffer from a camera through OpenCV's VideoCapture but I cannot find a way. To make it more clear, I want to access to the data from cv::VideoCapture::grab() before retrieving it to a cv::Mat.
I check the OpenCV source code here
and it seems OpenCV decodes it automatically before outputing the frame. Intuitionally I am thinking about "encoding" the frame to obtain the original data, however, cv::imencode requires a specific file extension.
Is there a way to access the camera buffer data without tweaking the source code?
Best,
Eric
Related
In order to accomplish some specific editing on some .avi files, I'd like to create an application (in C++) that is able to load, edit, and save those .avi files. But, what is the most efficient way? When first thinking about it, a simple 3D-Array containing a 2D-array of pixels for every frame seems the simplest solution; But then its size would be ENORMOUS. I mean, let's assume that a pixel only needs a color. One color would mean 3bytes (1char r, 1char b, 1char g). If I now have a 1920x1080 video format, this would mean 2MEGABYTES for only one frame! This data may or may not be smaller if using pointers for the colors, so that alreay used colors wont take more size - I don't really know, since I'm pretty new to C++ and the whole low-level stuff. (As a comparison: One of my AVI files recorded with Xvid codec is 40seconds long, 30fps, and only has 2MB.)
So how would you actually store the video data (Not even the audio, just the video) efficiently (while still being easily able to perform per-frame-changes on it)?
As you have realised, uncompressed video is enormous and it is not practical to store an entire video in this way.
Video compression is an extremely complex topic, but more-or-less, it works as follows: certain "key-frames" are compressed using fairly standard compression techniques similar or identical to still-photo compression such as JPEG. Frames following key-frames are compressed by comparing the frame with the previous one and looking for changes (such as moving blocks). Every now and again, a new key-frame is used.
You don't really have to worry much about that as you are not going to write your own video coder/decoder (codec). There are standard ones.
What will happen is that your program will decode the compressed video frame-by-frame and keep a certain number of frames in memory while you are working on them and then re-encode them when it is finished. In the uncompressed form, you will have access to the individual pixels and can work on them how you want.
You are probably not going to do that either by yourself - it is very hard. You probably need to use a framework, such as OpenCV. There are a huge number of standard filters and tools built in to these frameworks, and it may be that what you want to do is already implemented somewhere.
The OpenCV framework can return individual frames in a Mat object and you can then access the pixels. See this post Get Pixels from Mat
OpenCV
Tutorial page: Open CV Tutorial
Is it possible to:
read an image given by just a filename (not knowing the image format) to a 2d matrix rgb uncompressed form (e.g. read an JPG to a 2d array)
access the bytes of that image, copy them, change them... (e.g. inverse the colors, I need a pointer to the image bytes, setters/getters won't do )
save those bytes to any given image format (e.g. save the inversed image to PNG)
Is it possible with boost::gil ? Maybe there is a different library more appropriate for such a task?
Sample code would be highly appreciated.
Yes, you can do all that in boost::gil.
What you should know though, is that boost::gil is only a universal interface and doesn't handle reading/writing images all by itself. You still need to use a second library, e.g. libpng..
Yes, yes and yes.
There are functions that enable you to read and write JPEG, TIFF and PNG images: see here.
For the second bullet, it is what just GIL is meant to do. You can manipulate images using its facilities (click here).
Can I get compressed data from cvCapture, which is stored into buffer after cvGrabFrame, before using cvRetrieveFrame? I need to keep them in the an archive of compressed frames.
I use cvCaptureFromFile for getting rtsp stream from Axis IP camera.
As far as I see it it's not possible. CvCapture is a black box structure. There is not really a possibility to access the data inside.
There is the function cv::imencode to compress images. It is not optimal to compress the image again but it should work.
By the way, you're using the old C API and there is a newer API for C++ that makes capturing more elegant: cv::VideoCapture.
I want to read in an .avi video file for a program that I am making. I have the file location saved as a string. Is there any good tutorials on using .avi files in c++ or does anyone know who to read one in? Is it the same as normal files?
I have a previously asked SO question that goes into better detail but here is what I want to do:
I am making a program that will detect faces (though OpenCV) As of now I have been given a video processor program that will detect each face on a frame, and return the frame as a image and the CvRec of the faces. I want to take these faces and test them to validate that they are all actually faces.
After I have all the faces (tested) I want to then take the images and test them together. I test the faces on each frame for size and distance changes. If the faces pass this for a frame length of two seconds, then I want to crop the face and make it the subject of each frame.
After each frame is cropped I then want to save the new video file for the user.
Hopefully that helps. If anyone needs a better explanation please let me know.
First of all, a little background.
What is AVI?
AVI stands for Audio Video Interleave. It is a special case of the RIFF (Resource Interchange File Format). AVI is defined by Microsoft and it is the most common format for audio/video data.
I assume you would want to read a avi file and decode the compressed video frames. AVI file is just like any other normal file and you can use fread()(in C) or iostream(in C++) to open an avi file and read it contents. But the contents of an avi file are video frames in a compressed format. The compression allows video content of bigger sizes to be efficiently packed in less memory space.To make any sense of this compressed data you would have to decode the encoded data format.You will have to study the standard which describes how AVI encoding is done and then extract and decode the frames. this raw video data now when fed to a video device will be displayed in video format.
It seems you are staying within OpenCV so things are easy. If OpenCV is compiled properly it is capable of delegating io/coding/decoding to other libraries. Quicktime and others for example, but best is to use ffmpeg. You open, read and decode everything using the OpenCV API which gives you the video frame by frame.
Make sure your OpenCV is compiled with ffmpeg support and then read the OpenCV tutorial on how to read/write AVI files. It's really easy.
Getting OpenCV to be built with ffmpeg support might be hard though. You might want to switch to an older version of OpenCV if you can't get ffmpeg running with the current one.
Personally i would not spent time trying to read the video by yourself and delegate the task to OpenCV. That's how it is supposed to be used.
I want to process an image in C++. How can I access the 3D array representing the JPEG image as is done in MATLAB?
I'd suggest using OpenCV for the task; C++ documentation is available here. The relevant (I believe) data structure which you'd have to use is the Point3_ class, which represents a 3D point in the image.
Well, I've never used MATLAB for such a task, but in C++ you will need some JPEG loader library like OpenIL or FreeImage. These will allow you to access the picture as byte arrays.
FreeImage's FreeImage_GetBits function has a detailed example in the documentation on how to access per pixel per channel data.
BTW, if you plan to do image processing in C/C++, I'd suggest you to check out the Insight Segmentation and Registration Toolkit and OpenCV.