I have a program that extracts every n-th frame of a video. The current way of doing it is to iterate trough the entire video and when we get to a desired frame we extract it a process it. But this is slow as we are going trough every frame when if we could just jump to the ones we desire it would be much faster. I have searched for solutions but I couldn't find one.
current slow code
while (true) {
cv::Mat frame;
vid >> frame;
if (frame.empty()) {
break;
}
//check if we need the frame
//process frame
}
Also tried other versions using stuff like vid.set(CAP_PROP_POS_FRAMES,frame_num)
but that also just seems to do the same thing like the code above but sets back the current frame to the original one so its even slower.
Related
I'm using OpenCV4 to read from a camera. Similar to a webcam. Works great, code is somewhat like this:
cv::VideoCapture cap(0);
cap.set(cv::CAP_PROP_FRAME_WIDTH , 1600);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, 1200);
while (true)
{
cv::Mat mat;
// wait for some external event here so I know it is time to take a picture...
cap >> mat;
process_image(mat);
}
Problem is, this gives many video frames, not a single image. This is important because in my case I don't want nor need to be processing 30 FPS. I actually have specific physical events that trigger reading the image from the camera at certain times. Because OpenCV is expecting the caller to want video -- not surprising considering the class is called cv::VideoCapture -- it has buffered many seconds of frames.
What I see in the image is always from several seconds ago.
So my questions:
Is there a way to flush the OpenCV buffer?
Or to tell OpenCV to discard the input until I tell it to take another image?
Or to get the most recent image instead of the oldest one?
The other option I'm thinking of investigating is using V4L2 directly instead of OpenCV. Will that let me take individual pictures or only stream video like OpenCV?
I am using Opencv for video decoding now. I can play the video with no problem. However, I could not find a way to skip frame, get total number of frames and current frame number. Is there a way to use VideoReader like VideoCapture? I have gone through their api and could not get anything.
OpenCV is Open Source library. If you use VideoCapture then the object of this class has any methods:
To skip a frame you should read 2 times a frame.
To get total number of frames as follows:
cv::VideoCapture m_capture;
....
int totalFrames = m_capture.get(cv::CAP_PROP_FRAME_COUNT)
To get a current frame number:
int currentFrame = m_capture.get(cv::CAP_PROP_POS_FRAMES)
For detailed information visit the OpenCV documentation https://docs.opencv.org/3.1.0/d8/dfe/classcv_1_1VideoCapture.html.
I've been having some issues regarding capturing video from a live stream.
I open up the video with the open function of the cv::VideoCapture. However I need to manually ask when a frame is ready in something like this:
while (true){
cv::Mat frame;
if (videoCapture.read(frame)){
// Do stuff ...
}
else{
// Video is done.
}
}
The problem with this code is that it will definitely process a single frame multiple times, the number of times depending on the camera's FPS. This is because the read() function will only return false if the camera is disconnected, according to the documentation.
So my question is, how can I know if there is a NEW frame available? That I'm not just getting the old one again?
I'm attempting to record video from a 1080p webcam into a file. I held a timer up in the video and in every trial the timestamp reported by a video player (VLC is what I used) doesn't sync up with the time in the video. It's always off a few seconds (usually in-video timer is faster than player-reported time).
As seen below, I set up the C++ program to capture video in one thread, and record in another thread. This is working fine as my CPU usage is ~200% (possible max out?). I'm on a Macbook Air w/ OS X 10.8 # 1.8 GHz Intel Core i7.
I've tried changing the framerate to 15fps and that results in very choppy/slow video. I've also tried setting CV_CAP_PROP_FRAME_WIDTH & CV_CAP_PROP_FRAME_HEIGHT to a lower resolution and it results in slow video. It appears that 1080p # 30fps results in good steady video, but it is still always plays faster than it's supposed to. I've also tried putting in waitKey(10); after record << frame; but it did not affect anything.
Any recommendations on how to make the video match up in time?
Thanks!
Aakash
#include "opencv/cv.h"
#include "opencv/highgui.h"
#include <boost/thread.hpp>
using namespace cv;
void captureFunc(Mat *frame, VideoCapture *capture){
for(;;){
// get a new frame from camera
(*capture) >> (*frame);
}
}
int main(int, char**)
{
VideoCapture capture(0); // open the default camera
if( !capture.isOpened() ) {
printf("Camera failed to open!\n");
return -1;
}
capture.set(CV_CAP_PROP_FPS,30); //set capture rate to 30fps
Mat frame;
capture >> frame; // get first frame for size
// initialize recording of video
VideoWriter record("test.avi", CV_FOURCC('D','I','V','X'), 30, frame.size(), true);
if( !record.isOpened() ) {
printf("VideoWriter failed to open!\n");
return -1;
}
boost::thread captureThread(captureFunc, &frame, &capture); //start capture thread
sleep(1); //just to make sure capture thread is ready
for(;;)
{
// add frame to recorded video
record << frame;
}
return 0;
}
I resolved my issue after a bit of debugging; it was an issue with VideoWriter being picky on the rate at which frames were fed to it.
You need to sync your read and write functions. Your code reads as fast as possible, and writes also as fast as possible. Your output video probably looks slow because writing the output happens faster than reading the input (since capture >> is waiting for your camera), and several identical frames are recorded.
Writing without waiting or syncing means you may write the same content several times (what I think is happening here), or lose frames.
If you want to keep having threads, you may, but you will need to make your write process wait until there is something new to write.
Likewise to avoid losing frames or writing corrupted frames, you need the read process to wait until writing is done, so that frame can be safely overwritten.
Since the threads need to wait for each other anyway, there's little point in threads at all.
I'd rather recommend this much simpler way:
for (;;) {
capture >> frame;
process(frame); // whatever smart you need
record << frame;
}
If you need parallelism, you'll need much more complex sync mechanism, and maybe some kind of fifo for your frames.
I read frames from video stream in FFMPEG using this loop:
while(av_read_frame(pFormatCtx, &packet)>=0) {
// Is this a packet from the video stream?
if(packet.stream_index==videoStream) {
// Decode video frame
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);
// Did we get a video frame?
if(frameFinished) {
sws_scale(img_convert_context ,pFrame->data,pFrame->linesize,0,
pCodecCtx->height, pFrameRGBA->data, pFrameRGBA->linesize);
printf("%s\n","Frame read finished ");
ExportFrame(pFrameRGBA->data[0]);
break;
}
}
// Save the frame to disk
}
printf("%s\n","Read next frame ");
// Free the packet that was allocated by av_read_frame
av_free_packet(&packet);
}
So in this way the stream is read sequentially.What I want is to have a random access to the frame to be able reading a specific frame (by frame number).How is it done?
You may want to look
int av_seek_frame(AVFormatContext *s, int stream_index, int64_t timestamp,
int flags);
The above api will seek to the keyframe at give timestamp. After seeking you can read the frame. Also the tutorial below explain conversion between position and timestamp.
http://dranger.com/ffmpeg/tutorial07.html
Since most frames in a video depend on previous and next frames, in general, accessing random frames in a video is not straightforward. However, some frames, are encoded independently of any other frames, and occur regularly throughout the video. These frames are known as I-frames. Accessing these frames is straightforward through seeking.
If you want to "randomly" access any frame in the video, then you must:
Seek to the previous I-frame
Read the frames one by one until you get to the frame number that you want
You've already got the code for the second point, so all you need to do is take care of the first point and you're done. Here's an updated version of the Dranger tutorials that people often refer to -- it may be of help.