Using OpenCV 2.4.3.2 on Ubuntu 12.10 with a PS3-Eye camera I'm not able to capture more than ~60 frames per second (FPS). The camera itself delivers up to 125 FPS. I would like to know what limits the framerate in OpenCV. So here is what I did so far:
#include <sys/time.h>
#include <time.h>
#include <iostream> // for standard I/O
using cv;
using std;
long time_diff( const timespec &t1, const timespec &t2 ) {
return (long)(t2.tv_sec-t1.tv_sec)*1000000000 + (t2.tv_nsec-t1.tv_nsec);
}
int main(int argc, char *argv[]) {
VideoCapture cap(0); // open the default camera
cap.set(CV_CAP_PROP_EXPOSURE, 0);
cap.set(CV_CAP_PROP_FPS, 125);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);
if(!cap.isOpened()) // check if we succeeded
return -1;
Mat frame;
timespec t_start, t_end;
for(int i=1;;++i) {
cap >> frame;
clock_gettime(CLOCK_REALTIME, &t_end);
if( i%20==0 )
std::cout << "FPS ~= " << time_diff(t_start, t_end) << std::endl;
clock_gettime(CLOCK_REALTIME, &t_start);
}
}
This outputs the framerate every 20 frames to stdout. Note that I had to patch the source to be able to set the framerate correctly for the PS3-Eye camera.
First I set the framerate to 30 (cap.set(CV_CAP_PROP_FPS, 30);) to verify that my measurement is correct. Then using higher framerates the reported framerate is capped at ~60 FPS.
The USB is not the problem because I can get the full 120 FPS with guvcview.
I modified the code above to use grab() and retrieve() like this:
clock_gettime(ClOCK_REALTIME, &t_start);
cap->grab();
clock_gettime(ClOCK_REALTIME, &t_end);
cap->retrieve(frame);
but the framerate is capped again at ~60 FPS.
So how can I tell what is limiting the framerate?
After switching to a desktop machine (from laptop) I was able to capture the full framerate. It seems OpenCVs capture implementation is not as efficient as the one in guvcview.
I have same issue with 65 FPS limit in win7 x64 with OpenCV and Delphi X6. Founded problem is in cvWaitKey or/and Windows message queue, which limits redraws in 65 Hz.
The solution - to call cvWaitKey less frequently, less 65 times per second.
Related
I am working in a real time image processing project, I am using a Basler camera model acA1300-200uc with communication by USB3, but I am having troubles with the fps of my c++ program because the camera supports over 200 fps but my program only runs arround 30 fps and I dont know how to increase it, my project need 100 fps aprox.
This is my code, I hope you can help me, thanks in advance.
#include <Windows.h>
#include <opencv2\core\core.hpp>
#include <opencv2\highgui\highgui.hpp>
#include <opencv2\video\video.hpp>
#include <pylon\PylonIncludes.h>
#include <time.h>
using namespace Pylon;
// Settings for using Basler USB cameras.
#include <pylon/usb/BaslerUsbInstantCamera.h>
typedef Pylon::CBaslerUsbInstantCamera Camera_t;
using namespace Basler_UsbCameraParams;
using namespace cv;
using namespace std;
static const uint32_t c_countOfImagesToGrab = 1000;
int main(int argc, char* argv[]) {
int frames = 0;
double seconds = 0,fps;
time_t start, end;
Pylon::PylonAutoInitTerm autoInitTerm;
try
{
CDeviceInfo info;
info.SetDeviceClass(Camera_t::DeviceClass());
Camera_t camera(CTlFactory::GetInstance().CreateFirstDevice(info));
cout << "Dispositivo utilizado: " << camera.GetDeviceInfo().GetModelName() << endl;
camera.Open();
camera.MaxNumBuffer = 10;
CImageFormatConverter formatConverter;
formatConverter.OutputPixelFormat = PixelType_BGR8packed;
CPylonImage pylonImage;
Mat openCvImage, gray_img;
vector<Vec3f> circles;
int64_t W = 800, H = 600;
camera.Width.SetValue(W);
camera.Height.SetValue(H);
camera.StartGrabbing(c_countOfImagesToGrab, GrabStrategy_LatestImageOnly);
CGrabResultPtr ptrGrabResult;
camera.RetrieveResult(5000, ptrGrabResult, TimeoutHandling_ThrowException);
cout << "SizeX: " << ptrGrabResult->GetWidth() << endl;
cout << "SizeY: " << ptrGrabResult->GetHeight() << endl;
cvNamedWindow("OpenCV Display Window", CV_WINDOW_AUTOSIZE);
time(&start);
while (camera.IsGrabbing())
{
camera.RetrieveResult(5000, ptrGrabResult, TimeoutHandling_ThrowException);
if (ptrGrabResult->GrabSucceeded())
{
formatConverter.Convert(pylonImage, ptrGrabResult);
openCvImage = Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(), CV_8UC3, (uint8_t *)pylonImage.GetBuffer());
imshow("OpenCV Display Window", openCvImage);
frames++;
if (waitKey(30)>=0) break;
}
}
time(&end);
}
catch (...) { cout << "error" << endl; }
seconds = difftime(end, start);
fps = frames / seconds;
cout << "fps: " << fps;
Sleep(1000);
}
Frame rate is affected by many parameters. If the manufacturer specifies 200fps as the maximum at full resolution, this is the absolute maximum with:
minimum exposure time (too small for most applications)
nothing else going on on the USB bus
optimal transfer and acquisition parameters (maximum aquisition frame rate, no bandwidth limitations, fast readout mode
...
In case you haven't noticed, that's the marketing guy with big and juicy bait. 200fps cannot be achieved in most applications due to many factors.
You can read out the resulting framerate for your current configuration like that:
// Get the resulting frame rate
double d = camera.ResultingFrameRate.GetValue();
Refer to the camera's user manual... There's an entire chapter on frame rate, framerate limitations, framerate optimization
I also see a waitkey(30) call in your fps measurement. This function will delay your grab loop for at least 30ms unless you press any key. If you display each frame for 30 milliseconds (at least that's how I understand the waitkey documentation), how are you supposed to reach 100 fps? 1 frame / 0.03 s = 33.33 fps.
In OpenCV is there a way to dramatically increase the frame rate of a video (.mp4). I have tried to methods to increase the playback of a video including :
Increasing the frame rate:
cv.SetCaptureProperty(cap, cv.CV_CAP_PROP_FPS, int XFRAMES)
Skipping frames:
for( int i = 0; i < playbackSpeed; i++ ){originalImage = frame.grab();}
&
video.set (CV_CAP_PROP_POS_FRAMES, (double)nextFrameNumber);
is there another way to achieve the desired results? Any suggestions would be greatly appreciated.
Update
Just to clarify, the play back speed is NOT slow, I am just searching for a way to make it much faster.
You're using the old API (cv.CaptureFromFile) to capture from a video file.
If you use the new C++ API, you can grab frames at the rate you want. Here is a very simple sample code :
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int, char**)
{
VideoCapture cap("filename"); // put your filename here
namedWindow("Playback",1);
int delay = 15; // 15 ms between frame acquisitions = 2x fast-forward
while(true)
{
Mat frame;
cap >> frame;
imshow("Playback", frame);
if(waitKey(delay) >= 0) break;
}
return 0;
}
Basically, you just grab a frame at each loop and wait between frames using cvWaitKey(). The number of milliseconds that you wait between each frame will set your speedup. Here, I put 15 ms, so this example will play a 30 fps video at approx. twice the speed (minus the time necessary to grab a frame).
Another option, if you really want control about how and what you grab from video files, is to use the GStreamer API to grab the images, and then convert to OpenCV for image tratment. You can find some info about this on this post : MJPEG streaming and decoding
I am trying to get the fps from my camera so that I can pass it to the VideoWriter for outputting the video. However, I am getting 0 fps by calling VideoCapture::get(CV_CAP_PROP_FPS) from my camera. If I hardcode it, my video may be too slow or too fast.
#include "opencv2/opencv.hpp"
#include <stdio.h>
#include <stdlib.h>
using namespace std;
using namespace cv;
int main(int argc, char *argv[])
{
cv::VideoCapture cap;
int key = 0;
if(argc > 1){
cap.open(string(argv[1]));
}
else
{
cap.open(CV_CAP_ANY);
}
if(!cap.isOpened())
{
printf("Error: could not load a camera or video.\n");
}
Mat frame;
cap >> frame;
waitKey(5);
namedWindow("video", 1);
double fps = cap.get(CV_CAP_PROP_FPS);
CvSize size = cvSize((int)cap.get(CV_CAP_PROP_FRAME_WIDTH),(int)cap.get(CV_CAP_PROP_FRAME_HEIGHT));
int codec = CV_FOURCC('M', 'J', 'P', 'G');
if(!codec){ waitKey(0); return 0; }
std::cout << "CODEC: " << codec << std::endl;
std::cout << "FPS: " << fps << std::endl;
VideoWriter v("Hello.avi",-1,fps,size);
while(key != 'q'){
cap >> frame;
if(!frame.data)
{
printf("Error: no frame data.\n");
break;
}
if(frame.empty()){ break; }
v << frame;
imshow("video", frame);
key = waitKey(5);
}
return(0);
}
How can I get VideoCapture::get(CV_CAP_PROP_FPS) to return the right fps or give a fps to the VideoWriter that works universally for all webcams?
CV_CAP_PROP_FPS only works on videos as far as I know. If you want to capture video data from a webcam you have to time it correctly yourself. For example use a timer to capture a frame from the webcam every 40ms and then save as 25fps video.
You can use VideoCapture::set(CV_CAP_PROP_FPS) to set the desired FPS for a webcam. However, you can't use get for some reason.
Note that sometimes the driver will choose a different FPS than what you have requested depending on the limitations of the webcam.
My workaround: capture frames during a few seconds (4 is fine in my tests, with 0.5 seconds of initial delay), and estimate the fps the camera outputs.
I've never observed CV_CAP_PROP_FPS to work. I have tried with various flavors of OpenCV 2.4.x (currently 2.4.11) using file inputs.
As a workaround in one scenario, I directly used libavformat (from ffmpeg) to get the frame rate, which I can then use in my other OpenCV code:
static double get_frame_rate(const char *filePath) {
AVFormatContext *gFormatCtx = avformat_alloc_context();
av_register_all();
if (avformat_open_input(&gFormatCtx, filePath, NULL, NULL) != 0) {
return -1;
} else if (avformat_find_stream_info(gFormatCtx, NULL) < 0) {
return -1;
}
for (int i = 0; i < gFormatCtx->nb_streams; i++) {
if (gFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
AVRational rate = gFormatCtx->streams[i]->avg_frame_rate;
return (double)av_q2d(rate);
}
}
return -1;
}
Aside from that, undoubtedly one of the slowest possible (although sure to work) methods to get the average fps, would be to step through each frame and divide the current frame number by the current time:
for (;;) {
currentFrame = cap.get(CV_CAP_PROP_POS_FRAMES);
currentTime = cap.get(CV_CAP_PROP_POS_MSEC);
fps = currentFrame / (currentTime / 1000);
# ... code ...
# stop this loop when you're satisfied ...
}
You'd probably only want to do the latter if the other methods of directly finding the fps failed, and further, there were no better way to summarily get overall duration and frame count information.
The example above works on a file -- to adapt to a camera, you could use elapsed wallclock time since beginning of capture, instead of getting CV_CAP_PROP_POS_MSEC. Then the average fps for the session would be the elapsed wall clock time divided by the current frame number.
For live video from webcam use cap.get(cv2.CAP_PROP_FPS)
i have a Raspberry Pi and installed on it the OpenCV and Guvcview. When i open Guvcview, i get ~ 17-21 fps but when i run a simple program (only capture from webcam and display frame) in C++ with Opencv, i get only 6 fps.
What is wrong? i need to configure Opencv to use Guvcview's configuration? why guvcview get 20 fps? What can i do?
thanks.
P.D. I've done the same in my computer and i get 29 fps in both cases.
//*********************************this is the code C++ :
#include <iostream>
#include "opencv2/opencv.hpp"
using namespace std;
using namespace cv;
time_t start, end; //variabile di tipo time_t , contiene tempo in sec.
// inizializzo contatore nella dichiarazione
int counter=0;
int main()
{ time(&start);
VideoCapture cap(1);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 640);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
if (!cap.isOpened())
{ cout << "could not capture";
return 0; }
Mat frame;
namedWindow("camera", 1);
char key = 'a';
while(key != 27)
{ cap.read( frame);
imshow("camera", frame);
//##################
//time at the end of 1 show, Stop the clock and show FPS
time(&end);
++counter;
cout <<"fps: "<< counter/ difftime(end,start) <<endl <<endl;
//##################
key = waitKey(3); }
destroyAllWindows();
return 0;
}
OpenCV is a heavy weight API and following tips may introduce minor improvements:
you can disable RGB conversion:
cap.set(CV_CAP_PROP_CONVERT_RGB , false);
you can increase frame rate if its default frame rate is low:
cap.set(CV_CAP_PROP_FPS , 60);
I'd suggest to do direct video capture via V4L, since OpenCV may do YUYV to RGB transformations and other stuff that involves floating point calculations, that are expensive on this kind of hardware. We have done many robotics projects on embedded systems and the rule of the thumb is that you will be always better of either directly using V4L or small 3rd party libraries like CMVision (http://www.cs.cmu.edu/~jbruce/cmvision/) to do image processing on embedded systems.
I'm capturing frames from a Webcam using OpenCV in a C++ app both on my Windows machine as well as on a RaspberryPi (ARM, Debian Wheezy). The problem is the CPU usage. I only need to process frames like every 2 seconds - so no real time live view. But how to achieve that? Which one would you suggest?
Grab each frame, but process only some: This helps a bit. I get the most recent frames but this option has no significant impact on the CPU usage (less than 25%)
Grab/Process each frame but sleep: Good impact on CPU usage, but the frames that I get are old (5-10sec)
Create/Destroy VideoCapture in each cycle: After some cycles the application crashes - even though VideoCapture is cleaned up correctly.
Any other idea?
Thanks in advance
#include <opencv2/opencv.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <vector>
#include <unistd.h>
#include <stdio.h>
using namespace std;
int main(int argc, char *argv[])
{
cv::VideoCapture cap(0); //0=default, -1=any camera, 1..99=your camera
if(!cap.isOpened())
{
cout << "No camera detected" << endl;
return 0;
}
// set resolution & frame rate (FPS)
cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT,240);
cap.set(CV_CAP_PROP_FPS, 5);
int i = 0;
cv::Mat frame;
for(;;)
{
if (!cap.grab())
continue;
// Version 1: dismiss frames
i++;
if (i % 50 != 0)
continue;
if( !cap.retrieve(frame) || frame.empty() )
continue;
// ToDo: manipulate your frame (image processing)
if(cv::waitKey(255) ==27)
break; // stop on ESC key
// Version 2: sleep
//sleep(1);
}
return 0;
}
Create/Destroy VideoCapture in each cycle: not yet tested
It may be a bit troublesome on Windows (and maybe on other operating systems too) - First frame grabbed after creating VideoCapture is usually black or gray. Second frame should be fine :)
Other ideas:
- modified idea nr 2 - after sleep grab 2 frames. First frame may be old, but second should be new. It's not tested and generally i'm not sure about that, but it's easy to check it.
- Eventually after sleep you may grab frames in while loop (without sleep) waiting till you grab the same frame twice (but it may be hard to achieve especially on RasberryPi).