How calculated fps can be greater than camera's declared fps? - c++

Question

I am trying to measure Frames Per Second when processing frame from camera. Calculations are nothing special and can be found in this question: How to write function with parameter which type is deduced with 'auto' word?
My camera is pretty old and manufacturer declared FPS is no more than 30 with resolution 640x480. However, when I am running those calculations it shows me 40-50 on live streams. How can it be?
Update: Code:
#include <chrono>
#include <iostream>
using std::cerr;
using std::cout;
using std::endl;
#include <string>
#include <numeric>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/imgproc.hpp>
using cv::waitKey;
using cv::Mat;
using time_type = decltype(std::chrono::high_resolution_clock::now());
void showFPS(Mat* frame, const time_type &startTime);
int main(int argc, char** argv) {
cv::VideoCapture capture;
std::string videoDevicePath = "/dev/video0";
if (!capture.open(videoDevicePath)) {
std::cerr << "Unable to open video capture.";
return 1;
}
//TODO normally through cmd or from cameraParameters.xml
bool result;
result = capture.set(CV_CAP_PROP_FOURCC, CV_FOURCC('M', 'J', 'P', 'G'));
if (result) {
std::cout << "Camera: PROP_FOURCC: MJPG option set.";
} else {
std::cerr << "Camera: PROP_FOURCC: MJPG option was not set.";
}
result = capture.set(CV_CAP_PROP_FRAME_WIDTH, 640);
if (result) {
std::cout << "Camera: PROP_FRAME_WIDTH option set.";
} else {
std::cerr << "Camera: PROP_FRAME_WIDTH option was not set.";
}
result = capture.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
if (result) {
std::cout << "Camera: PROP_FRAME_HEIGHT option set.";
} else {
std::cerr << "Camera: PROP_FRAME_HEIGHT option was not set.";
}
result = capture.set(CV_CAP_PROP_FPS, 30);
if (result) {
std::cout << "Camera: PROP_FPS option set.";
} else {
std::cerr << "Camera: PROP_FPS option was not set.";
}
Mat frame, raw;
while (cv::waitKey(5) != 'q') {
auto start = std::chrono::high_resolution_clock::now();
capture >> raw;
if (raw.empty()) {
return 1;
}
if (raw.channels() > 1) {
cv::cvtColor(raw, frame, CV_BGR2GRAY);
} else {
frame = raw;
}
showFPS(&raw1, start);
}
return 0;
}
void showFPS(Mat* frame, const time_type &startTime) {
typedef std::chrono::duration<float> fsec_t;
auto stopTime = std::chrono::high_resolution_clock::now();
fsec_t duration = stopTime - startTime;
double sec = duration.count();
double fps = (1.0 / sec);
std::stringstream s;
s << "FPS: " << fps;
cv::putText(*frame, s.str(), Point2f(20, 20), constants::font,
constants::fontScale, constants::color::green);
}

Answer 1

Camera's FPS is the number of frames that camera could provide per second.
It means that camera provides new frame every 33ms.
On the other side, what you are measuring is not FPS.
You are measuring inverse time of the function of the new frame retrieval plus color converting.
And this time is 20-25ms, based on your results.
This is not correct way of measuring FPS, at least because you can't guarantee the synchronization of these two processes.
If you want to measure FPS correctly, you can measure the time for showing last N frames.
Pseudocode:
counter = 0;
start = getTime();
N = 100;
while (true) {
captureFrame();
convertColor();
counter++;
if (counter == N) {
fps = N / (getTime() - start);
printFPS(fps);
counter = 0;
start = getTime();
}
}

Answer 2

Aleksey Petrov's answer is not bad, but while averaging over the last N frames gives smoother values, one can measure the frame rate relatively accurately without averaging. Here the code from the question modified to do that:
// see question for earlier code
Mat frame, raw;
time_type prevTimePoint; // default-initialized to epoch value
while (waitKey(1) != 'q') {
capture >> raw;
auto timePoint = std::chrono::high_resolution_clock::now();
if (raw.empty()) {
return 1;
}
if (raw.channels() > 1) {
cv::cvtColor(raw, frame, CV_BGR2GRAY);
} else {
frame = raw;
}
showFPS(&frame, prevTimePoint, timePoint);
cv::imshow("frame", frame);
}
return 0;
}
void showFPS(Mat* frame, time_type &prevTimePoint, const time_type &timePoint) {
if (prevTimePoint.time_since_epoch().count()) {
std::chrono::duration<float> duration = timePoint - prevTimePoint;
cv::putText(*frame, "FPS: " + std::to_string(1/duration.count()),
cv::Point2f(20, 40), 2, 2, cv::Scalar(0,255,0));
}
prevTimePoint = timePoint;
}
Note that this measures the time point right after capture >> raw returns, which (without messing with OpenCV) is the closest one can get to when the camera sent the frame, and that the time is measured only once per loop and compared against the previous measurement, which gives a quite precise current frame rate. Of course, if the processing takes more time than 1/(frame rate), the measurement will be off.
The reason the question's code gave too high a frame rate was actually the code between the two time measurements: the now() in showFPS() and the now() in the while loop. My hunch is this code included cv::imshow(), which is not in the question and which together with cv::waitKey(5) and cv::putText() is likely responsible for most of the "missing time" in the frame rate calculation (causing too high a frame rate).

Answer 3

You have a cvtColor in between, so it affects your time computing because the process time of cvtColor may vary in each loop (probably because of the other processes of windows).
Consider this example:
You get the first frame with capture at moment 0, then do a
cvtColor and that takes e.g. 10 ms, then you make a stopTime at
moment 10 ms. 23 ms later (33-10) you capture the second frame. But
this time cvtColor takes 5 ms (It could happen) and you make the
second stopTime at moment 38 (33+5), so the first tick was at moment
10 and the second tick is at moment 38. Now your fps becomes
1000/(38-10) = 35.7