I've been having some issues regarding capturing video from a live stream.
I open up the video with the open function of the cv::VideoCapture. However I need to manually ask when a frame is ready in something like this:
while (true){
cv::Mat frame;
if (videoCapture.read(frame)){
// Do stuff ...
}
else{
// Video is done.
}
}
The problem with this code is that it will definitely process a single frame multiple times, the number of times depending on the camera's FPS. This is because the read() function will only return false if the camera is disconnected, according to the documentation.
So my question is, how can I know if there is a NEW frame available? That I'm not just getting the old one again?
Related
I have a program that extracts every n-th frame of a video. The current way of doing it is to iterate trough the entire video and when we get to a desired frame we extract it a process it. But this is slow as we are going trough every frame when if we could just jump to the ones we desire it would be much faster. I have searched for solutions but I couldn't find one.
current slow code
while (true) {
cv::Mat frame;
vid >> frame;
if (frame.empty()) {
break;
}
//check if we need the frame
//process frame
}
Also tried other versions using stuff like vid.set(CAP_PROP_POS_FRAMES,frame_num)
but that also just seems to do the same thing like the code above but sets back the current frame to the original one so its even slower.
I'm using OpenCV4 to read from a camera. Similar to a webcam. Works great, code is somewhat like this:
cv::VideoCapture cap(0);
cap.set(cv::CAP_PROP_FRAME_WIDTH , 1600);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, 1200);
while (true)
{
cv::Mat mat;
// wait for some external event here so I know it is time to take a picture...
cap >> mat;
process_image(mat);
}
Problem is, this gives many video frames, not a single image. This is important because in my case I don't want nor need to be processing 30 FPS. I actually have specific physical events that trigger reading the image from the camera at certain times. Because OpenCV is expecting the caller to want video -- not surprising considering the class is called cv::VideoCapture -- it has buffered many seconds of frames.
What I see in the image is always from several seconds ago.
So my questions:
Is there a way to flush the OpenCV buffer?
Or to tell OpenCV to discard the input until I tell it to take another image?
Or to get the most recent image instead of the oldest one?
The other option I'm thinking of investigating is using V4L2 directly instead of OpenCV. Will that let me take individual pictures or only stream video like OpenCV?
I've been trying to find a way to asynchronously check to see if the next frame of the camera that I get using videocapture is ready.
I came across waitAny() which is described to "Wait for ready frames from VideoCapture.".
in the OpenCV documentation, I didn't find any useful info on how to use this or what are the use cases.
I've been searching the net for two days now and the only thing I found is how to define the parameters this function needs(I'm new to c++) and I don't know how to fill them and what are their use cases.
here is the documentation: https://docs.opencv.org/master/d8/dfe/classcv_1_1VideoCapture.html#ade1c7b8d276fea4d000bc0af0f1017b3
An approach that IMO is fairly common is to have a pair of threads -- one that "produces" frames from VideoCapture (calls VideoCapture::read() in a loop), and another that actually uses these frames (a "consumer"). The producer pushes images onto a queue (that's shared across two threads), while the consumer pops them.
In this situation, checking whether a camera has produced an image amounts to checking whether the queue is empty.
By itself, VideoCapture does not provide such an async API.
That said, if you want to use waitAny, and if you have a single camera, you could do something like this:
VideoCapture cap = /* get the VideoCapture object from somewhere */
constexpr int64 kTimeoutNs = 1000;
std::vector<int> ready_index;
cv::Mat image;
if (VideoCapture::waitAny({cap}, ready_index, kTimeoutNs)) {
// Camera was ready; get image.
cap.retrieve(image);
} else {
// Camera was not ready; do something else.
}
Above, VideoCapture::waitAny will wait for the specified timeout (1 microsecond) for the camera to produce a frame, and will return after this period.
If the camera is ready, it will return true (and will also populate ready_index with the index of the ready camera. Since you only have a single camera, the vector will be either empty or non-empty).
That said, waitAny seems to only be supported by VideoCapture sources that use V4L in the backend.
I'm trying to access multiple usb-cameras in openCV with MacOS 10.11.
My goal is to connect up to 20 cameras to the pc via USB Quad-Channel extensions and take single images. I do not need to have live streaming.
I tried with the following code and I can take a single image from all cameras (currently only 3, via one usb controller).
The question is: does opencv stream a live video from the usb cameras all the time, or does grab() stores an image on the camera which can be retrieve with retrieve() ?
I couldn't find the information, wether opencv uses the grab() command on it's internal video buffer, or on the camera.
int main(int argument_number, char* argument[])
{
std::vector<int> cameraIDs{0,1,2};
std::vector<cv::VideoCapture> cameraCaptures;
std::vector<std::string> nameCaptures{"a","b","c"};
//Load all cameras
for (int i = 0;i<cameraIDs.size();i++)
{
cv::VideoCapture camera(cameraIDs[i]);
if(!camera.isOpened()) return 1;
camera.set(CV_CAP_PROP_FRAME_WIDTH, 640);
camera.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
cameraCaptures.push_back(camera);
}
cv::namedWindow("a");
while(true) {
int c = cvWaitKey(2);
if(27 == char(c)){ //if esc pressed. grab new image and display it.
for (std::vector<cv::VideoCapture>::iterator it=cameraCaptures.begin();it!=cameraCaptures.end();it++)
{
(*it).grab();
}
int i=0;
for (std::vector<cv::VideoCapture>::iterator it=cameraCaptures.begin();it!=cameraCaptures.end();it++)
{
cv::Mat3b frame;
(*it).retrieve(frame);
cv::imshow(nameCaptures[i++], frame);
}
}
}
return 0;
}
Could you please make the statement more clear. You only want the frame from the feed or you want the streams to be connected all the time.
Opencv camera capture is always in running mode unless you release the capture device. So if you say you want only one frame from a device then its better to release this device once you retrieve the frame.
Another point is instead of using grab or retrieve in a multi cam environment, its better to use read() which combines both the above methods and reduces the overhead in decoding streams. So if you want say frame # 2sec position from each of the cams then in time domain they are pretty closely captured as in say the frame x from cam1 captured # 2sec position then frame x at 2.00001sec and frame x from cam3 captured at 2.00015sec..(time multiplexing - multithreading - internal by ocv)
I hope I am clear in explanation.
Thanks
I'm attempting to record video from a 1080p webcam into a file. I held a timer up in the video and in every trial the timestamp reported by a video player (VLC is what I used) doesn't sync up with the time in the video. It's always off a few seconds (usually in-video timer is faster than player-reported time).
As seen below, I set up the C++ program to capture video in one thread, and record in another thread. This is working fine as my CPU usage is ~200% (possible max out?). I'm on a Macbook Air w/ OS X 10.8 # 1.8 GHz Intel Core i7.
I've tried changing the framerate to 15fps and that results in very choppy/slow video. I've also tried setting CV_CAP_PROP_FRAME_WIDTH & CV_CAP_PROP_FRAME_HEIGHT to a lower resolution and it results in slow video. It appears that 1080p # 30fps results in good steady video, but it is still always plays faster than it's supposed to. I've also tried putting in waitKey(10); after record << frame; but it did not affect anything.
Any recommendations on how to make the video match up in time?
Thanks!
Aakash
#include "opencv/cv.h"
#include "opencv/highgui.h"
#include <boost/thread.hpp>
using namespace cv;
void captureFunc(Mat *frame, VideoCapture *capture){
for(;;){
// get a new frame from camera
(*capture) >> (*frame);
}
}
int main(int, char**)
{
VideoCapture capture(0); // open the default camera
if( !capture.isOpened() ) {
printf("Camera failed to open!\n");
return -1;
}
capture.set(CV_CAP_PROP_FPS,30); //set capture rate to 30fps
Mat frame;
capture >> frame; // get first frame for size
// initialize recording of video
VideoWriter record("test.avi", CV_FOURCC('D','I','V','X'), 30, frame.size(), true);
if( !record.isOpened() ) {
printf("VideoWriter failed to open!\n");
return -1;
}
boost::thread captureThread(captureFunc, &frame, &capture); //start capture thread
sleep(1); //just to make sure capture thread is ready
for(;;)
{
// add frame to recorded video
record << frame;
}
return 0;
}
I resolved my issue after a bit of debugging; it was an issue with VideoWriter being picky on the rate at which frames were fed to it.
You need to sync your read and write functions. Your code reads as fast as possible, and writes also as fast as possible. Your output video probably looks slow because writing the output happens faster than reading the input (since capture >> is waiting for your camera), and several identical frames are recorded.
Writing without waiting or syncing means you may write the same content several times (what I think is happening here), or lose frames.
If you want to keep having threads, you may, but you will need to make your write process wait until there is something new to write.
Likewise to avoid losing frames or writing corrupted frames, you need the read process to wait until writing is done, so that frame can be safely overwritten.
Since the threads need to wait for each other anyway, there's little point in threads at all.
I'd rather recommend this much simpler way:
for (;;) {
capture >> frame;
process(frame); // whatever smart you need
record << frame;
}
If you need parallelism, you'll need much more complex sync mechanism, and maybe some kind of fifo for your frames.