OpenCV Slow camera framerate - c++

I just copied an program to detect faces through webcam but the video capture is really slow, and i dont know how to fix it!
Here is the code:
#include<stdio.h>
#include<math.h>
#include<opencv\cv.h>
#include<opencv\highgui.h>
#include<opencv2\objdetect\objdetect.hpp>
#include<opencv2\highgui\highgui.hpp>
#include<opencv2\imgproc\imgproc.hpp>
#include<vector>
using namespace cv;
using namespace std;
int main()
{
CascadeClassifier face_cascade;
if(!face_cascade.load("c:\\haar\\haarcascade_frontalface_alt2.xml")) {
printf("Error loading cascade file for the face");
return 1;
}
VideoCapture capture(0);
if(!capture.isOpened())
{
printf("Error trying to start the Camera");
return 1;
}
Mat cap_img,gray_img;
vector<Rect> faces;
while(1)
{
capture >> cap_img;
waitKey(10);
cvtColor(cap_img, gray_img, CV_BGR2GRAY);
cv::equalizeHist(gray_img,gray_img);
face_cascade.detectMultiScale(gray_img, faces, 1.1, 10, CV_HAAR_SCALE_IMAGE | CV_HAAR_DO_CANNY_PRUNING, cvSize(0,0), cvSize(300,300));
for(int i=0; i < faces.size();i++)
{
Point pt1(faces[i].x+faces[i].width, faces[i].y+faces[i].height);
Point pt2(faces[i].x,faces[i].y);
rectangle(cap_img, pt1, pt2, cvScalar(191,191,191), 2, 8, 0);
}
imshow("Result", cap_img);
waitKey(3);
char c = waitKey(3);
if(c == 27)
break;
}
return 0;
}
I am using Visual studio 2012 and this is the main.cpp file. Im using OpenCV 2.4.9!

OpenCV comes with prebuild libraries. When you use them in an application you actually want to deploy make sure you use the release libraries. In debug mode you have lots of additional checks in the form of asserts as well as debug symbols allowing you to step into the libraries with a debugger that are removed in release mode.
Another more specific advise regarding the code you posted: Avoid calls to cv::waitKey() as every single call, who would have guessed, makes your main thread wait the specified amount of time in milliseconds. Do not drop calls to it entirely as e.g. cv::imshow() will only work appropriately with these in place.
Edit:
Reduce your while loop to:
while ( true ) {
capture >> cap_img;
imshow("Result", cap_img);
if(waitKey(1) == 27)
break;
}
When you know how much time you need for capturing the image and just displaying it, then you can compare how much of an impact whatever algorithm you are running has on your performance.

Here is solution...everytime you read!!!..everytime you use vcap>>cap or cap.read(vcap)... you must set FPS ..25 is max..if i try to set 26 or higher it runs slow.
...this runs 30 fps for me
If you not set FPS...it runs slow too
if (cap.read(vcap)){
imshow("Cam Feed", vcap);
cap.set(CV_CAP_PROP_FPS, 25);
}

On top of the other answers, Multiprocessing is what fixed my issue. I split my program into 3 processes. One for grabbing the image, one for processing the image, and one for displaying the image. Referencing this page helped me a lot. I also followed the tips of the other answers here.

Related

lag in opencv videocapture when i use rtsp camera stream

So i'm currently working on a project that needs to do a facial recognition on rtsp ip cam , i managed to get the rtsp feed with no problems, but when it comes to applying the face recognition the video feed gets too slow and shows a great delay, i even used multithreading to make it better but with no success,here is my code i'm still a beginner in multi threading matters so any help would be appreciated.
#include <iostream>
#include <thread>
#include "opencv2/opencv.hpp"
#include <vector>
using namespace std;
using namespace cv;
void detect(Mat img, String strCamera) {
string cascadeName1 = "C:\\ocv3.2\\Build\\install\\etc\\haarcascades\\haarcascade_frontalface_alt.xml";
CascadeClassifier facedetect;
bool loaded1 = facedetect.load(cascadeName1);
Mat original;
img.copyTo(original);
vector<Rect> human;
cvtColor(img, img, CV_BGR2GRAY);
equalizeHist(img, img);
facedetect.detectMultiScale(img, human, 1.1, 2, 0 | 1, Size(40, 80), Size(400, 480));
if (human.size() > 0)
{
for (int gg = 0; gg < human.size(); gg++)
{
rectangle(original, human[gg].tl(), human[gg].br(), Scalar(0, 0, 255), 2, 8, 0);
}
}
imshow("Detect " + strCamera, original);
int key6 = waitKey(40);
//End of the detect
}
void stream(String strCamera) {
VideoCapture cap(strCamera);
if (cap.isOpened()) {
while (true) {
Mat frame;
cap >> frame;
resize(frame, frame, Size(640, 480));
detect(frame, strCamera);
}
}
}
int main() {
thread cam1(stream, "rtsp://admin:password#ipaddress:554/live2.sdp?tcp");
thread cam2(stream, "rtsp://admin:password#ipaddress/live2.sdp?tcp");
cam1.join();
cam2.join();
return 0;
}
I had similar issues and was able to resolve them by completely isolating the frame capturing from processing of the images. I also updated OpenCV to the latest (3.2.0) available, but I think this will also resolve problems with earlier versions.
void StreamLoop(String strCamera, LFQueue1P1C<Mat> *imageQueue, bool *shutdown) {
VideoCapture cap(strCamera, CV_CAP_FFMPEG);
Mat image;
while(!(*shutdown) && cap.isOpened()){
*cap >> image;
imageQueue->Produce(image);
}
}
int main(){
Mat aImage1;
bool shutdown(false);
LFQueue1P1C<Mat> imageQueue;
string rstp("rtsp://admin:password#ipaddress:554/live2.sdp?tcp");
thread streamThread(StreamLoop, rtsp, &imageQueue, &shutdown);
...
while(!shutdownCondition){
if(imageQueue.Consume(aImage1)) {
// Process Image
resize(aImage1, aImage1, Size(640, 480));
detect(aImage1, rtsp);
}
}
shutdown = true;
if(streamThread.joinable()) streamThread.join();
...
return 0;
}
It seems that there is some issue with rtsp in OpenCV where it easily hangs up if there are even slight pauses while picking up the frames. As long as I pick up frames without much pause I have not seen a problem.
Also, I didn't have this issue when the video cameras where directly connected to my local network. It was not until we deployed them at a remote site that I started getting the hang ups. Separating frame retrieval and processing into separate threads resolved my issues, hopefully someone else might find this solution useful.
Note: The queue I used is a custom queue for passing items from one thread to another. The code I posted is modified from my original code to make it more readable and applicable to this problem.
i'm still a beginner in multi threading matters so any help would be appreciated
Having threads that have no way of exiting will cause you issues in the future. Even if it is test code, get in the habit of making sure the code has an exit path. As an example: You might copy and paste a section of code later on and forget there is an infinite loop in there and it will cause great grief later trying to track down why you have mysterious crashing or your resources are locked up.
I am not a C++ developer but I had the same problem in Java. I solved my issue by calling VideoCapture.grab() function before reading camera frame. According to OpenCV Doc, the use of the grab function is :
The primary use of the function is in multi-camera environments,
especially when the cameras do not have hardware synchronization.
Besides that, in java application, you should release your frame's Mat objects every time you read new frames.

Can i capture at 20 fps with C++ and Opencv?

i have a Raspberry Pi and installed on it the OpenCV and Guvcview. When i open Guvcview, i get ~ 17-21 fps but when i run a simple program (only capture from webcam and display frame) in C++ with Opencv, i get only 6 fps.
What is wrong? i need to configure Opencv to use Guvcview's configuration? why guvcview get 20 fps? What can i do?
thanks.
P.D. I've done the same in my computer and i get 29 fps in both cases.
//*********************************this is the code C++ :
#include <iostream>
#include "opencv2/opencv.hpp"
using namespace std;
using namespace cv;
time_t start, end; //variabile di tipo time_t , contiene tempo in sec.
// inizializzo contatore nella dichiarazione
int counter=0;
int main()
{ time(&start);
VideoCapture cap(1);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 640);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
if (!cap.isOpened())
{ cout << "could not capture";
return 0; }
Mat frame;
namedWindow("camera", 1);
char key = 'a';
while(key != 27)
{ cap.read( frame);
imshow("camera", frame);
//##################
//time at the end of 1 show, Stop the clock and show FPS
time(&end);
++counter;
cout <<"fps: "<< counter/ difftime(end,start) <<endl <<endl;
//##################
key = waitKey(3); }
destroyAllWindows();
return 0;
}
OpenCV is a heavy weight API and following tips may introduce minor improvements:
you can disable RGB conversion:
cap.set(CV_CAP_PROP_CONVERT_RGB , false);
you can increase frame rate if its default frame rate is low:
cap.set(CV_CAP_PROP_FPS , 60);
I'd suggest to do direct video capture via V4L, since OpenCV may do YUYV to RGB transformations and other stuff that involves floating point calculations, that are expensive on this kind of hardware. We have done many robotics projects on embedded systems and the rule of the thumb is that you will be always better of either directly using V4L or small 3rd party libraries like CMVision (http://www.cs.cmu.edu/~jbruce/cmvision/) to do image processing on embedded systems.

C/C++ OpenCV video processing

Good day everyone! So currently I'm working on a project with video processing, so I decided to give a try to OpenCV. As I'm new to it, I decided to find few sample codes and test them out. First one, is C OpenCV and looks like this:
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <stdio.h>
int main( void ) {
CvCapture* capture = 0;
IplImage *frame = 0;
if (!(capture = cvCaptureFromCAM(0)))
printf("Cannot initialize camera\n");
cvNamedWindow("Capture", CV_WINDOW_AUTOSIZE);
while (1) {
frame = cvQueryFrame(capture);
if (!frame)
break;
IplImage *temp = cvCreateImage(cvSize(frame->width/2, frame->height/2), frame->depth, frame->nChannels); // A new Image half size
cvResize(frame, temp, CV_INTER_CUBIC); // Resize
cvSaveImage("test.jpg", temp, 0); // Save this image
cvShowImage("Capture", frame); // Display the frame
cvReleaseImage(&temp);
if (cvWaitKey(5000) == 27) // Escape key and wait, 5 sec per capture
break;
}
cvReleaseImage(&frame);
cvReleaseCapture(&capture);
return 0;
}
So, this one works perfectly well and stores image to hard drive nicely. But problems begin with next sample, which uses C++ OpenCV:
#include "opencv2/opencv.hpp"
#include <string>
using namespace cv;
int main(int, char**)
{
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) // check if we succeeded
return -1;
Mat edges;
//namedWindow("edges",1);
for(;;)
{
Mat frame;
cap >> frame; // get a new frame from camera
cvtColor(frame, edges, CV_RGB2XYZ);
imshow("edges", edges);
//imshow("edges2", frame);
//imwrite("test1.jpg", frame);
if(waitKey(1000) >= 0) break;
}
// the camera will be deinitialized automatically in VideoCapture destructor
return 0;
}
So, yeah, generally, in terms of showing video (image frames) there is practically no changes, but when it comes to using im** functions, some problems arise.
Using cvSaveImage() works out nicely, but the moment I try to use imwrite(), unhandled exception arises in regards of 'access violation reading location'. Same goes for imread(), when I'm trying to load image.
So, the thing I wanted to ask, is it possible to use most of the functionality with C OpenCV? Or is it necessary to use C++ OpenCV. If yes, is there any solution for the problem I described earlier.
Also as stated here, images initially are in BGR-format, so conversion needed. But doing BGR2XYZ conversion seems to invert colors, while RGB2XYZ preserve them. Examples:
images
Or is it necessary to use C++ OpenCV?
No, there is no necessity whatsoever. You can use any interface you like and you think you are good with it (OpenCV offers C, C++, Python interfaces).
For your problem about imwrite() and imread() :
For color images the order channel is normally Blue, Green, Red , this
is what imshow() , imread() and imwrite() expect
Quoted from there

OpenCV Frame Rate Issue

I am working on a school project with OpenCV. A major part of the program will be a comparison of histograms. There will be a database of histograms and new histograms will be created from a live video feed then compared to the histograms in the database. Right now I am just trying to get the histograms created correctly from the video feed. My problem is that the program crashes or slows down dramatically at random intervals. So my question is how do I prevent the program from crashing or slowing down? OpenCV has always been kind of flaky for me, so I'm not sure if it is an issue with my code or if it is just the nature of OpenCV. If it is to do with my code I think the issue might have something to do with the frame rates (a guess/gut feeling). I am useing "cvWaitKey" to "pace" the loading of frames, but the "Learning OpenCV" book has this to say about "cvWaitKey"
c = cvWaitKey(33);
if( c == 27 ) break;
Once we have displayed the frame, we then wait for 33 ms. If the user hits a key, then c
will be set to the ASCII value of that key; if not, then it will be set to –1. If the user hits
the Esc key (ASCII 27), then we will exit the read loop. Otherwise, 33 ms will pass and
we will just execute the loop again.
It is worth noting that, in this simple example, we are not explicitly controlling
the speed of the video in any intelligent way. We are relying solely on the timer in
cvWaitKey() to pace the loading of frames. In a more sophisticated application it would
be wise to read the actual frame rate from the CvCapture structure (from the AVI) and
behave accordingly!
You will see in my code below (modified from here) that I my loop waits 10ms before starting the next execution. Often times the program will run with no issues at all, but sometimes it will crash less than a minute in, or five minutes in, there really is not pattern that I can detect. Any suggestions on how this crash( or slow down) can be prevented would be welcomed. Also I should add that I am using OpenCV 1.1 (can't ever get OpenCV 2.0 to work right), I am using Visual Studio 2008, and I create an .MSI installer package everytime I modify my code, that is, I do not debug in Visual Studio. Dependencies are cv110.dll, cxcore110.dll, and highgui110.dll. My code is below:
// SLC (Histogram).cpp : Defines the entry point for the console application.
#include "stdafx.h"
#include <cxcore.h>
#include <cv.h>
#include <cvaux.h>
#include <highgui.h>
#include <stdio.h>
#include <sstream>
#include <iostream>
using namespace std;
int main(){
CvCapture* capture = cvCaptureFromCAM(0);
if(!cvQueryFrame(capture)){
cout<<"Video capture failed, please check the camera."<<endl;
}
else{
cout<<"Video camera capture successful!"<<endl;
};
CvSize sz = cvGetSize(cvQueryFrame(capture));
IplImage* image = cvCreateImage(sz, 8, 3);
IplImage* imgHistogram = 0;
IplImage* gray = 0;
CvHistogram* hist;
cvNamedWindow("Image Source",1);
cvNamedWindow("Histogram",1);
for(;;){
image = cvQueryFrame(capture);
//Size of the histogram -1D histogram
int bins = 256;
int hsize[] = {bins};
//Max and min value of the histogram
float max_value = 0, min_value = 0;
//Value and normalized value
float value;
int normalized;
//Ranges - grayscale 0 to 256
float xranges[] = {0, 256};
float* ranges[] = {xranges};
//Create an 8 bit single channel image to hold a grayscale version of the original picture
gray = cvCreateImage(cvGetSize(image), 8, 1);
cvCvtColor(image, gray, CV_BGR2GRAY);
//Planes to obtain the histogram, in this case just one
IplImage* planes[] = {gray};
//Get the histogram and some info about it
hist = cvCreateHist(1, hsize, CV_HIST_ARRAY, ranges,1);
cvCalcHist(planes, hist, 0, NULL);
cvGetMinMaxHistValue(hist, &min_value, &max_value);
printf("Minimum Histogram Value: %f, Maximum Histogram Value: %f\n", min_value, max_value);
//Create an 8 bits single channel image to hold the histogram and paint it white
imgHistogram = cvCreateImage(cvSize(bins, 50),8,3);
cvRectangle(imgHistogram, cvPoint(0,0), cvPoint(256,50), CV_RGB(255,255,255),-1);
//Draw the histogram
for(int i=0; i < bins; i++){
value = cvQueryHistValue_1D(hist, i);
normalized = cvRound(value*50/max_value);
cvLine(imgHistogram,cvPoint(i,50), cvPoint(i,50-normalized), CV_RGB(0,0,0));
}
cvFlip(image, NULL, 1);
cvShowImage("Image Source", image);
cvShowImage("Histogram", imgHistogram);
//Page 19 paragraph 3 of "Learning OpenCV" tells us why we DO NOT use "cvReleaseImage(&image)" in this section
cvReleaseImage(&imgHistogram);
cvReleaseImage(&gray);
cvReleaseHist(&hist);
char c = cvWaitKey(10);
//if ASCII key 27 (esc) is pressed then loop breaks
if(c==27) break;
}
cvReleaseImage(&image);
cvReleaseCapture(&capture);
cvDestroyAllWindows();
}
Only a few things I can see or recommend:
Considering the build, make sure you're building in Release. Also, make sure the build of OpenCV you're using was built with OpenMP enabled, it makes an enormous difference.
Try moving your allocations outside the loop. Every loop you're re-creating gray and other images, when they should be re-used.
The other thing is your style, which makes it difficult to give good recommendations easily. It's poor style to pre-declare a bunch of variables, this is C-style. Declare your variables just prior to their use, and the code will be easier to read.
Update: I found the issue, it was actually my hardware (well the driver I think). I was using a PS3 Eye because of the amazing frame rates, but for some reason OpenCV does not like the PS3 Eye all the time. Sometimes it works great and other times not so great. I have verfied this on three computers, all of which run my code good with a standard web cam but randomly lock up when the PS3 Eye is used. Still, thank you for your suggestions GMan!

Capture a picture from multiple webcams, using C++

I need a program to capture pictures from multiple webcams and save them automatically in Windows Vista. I got the basic code from this link. The code runs in Window XP, but when I tried using it on Vista it says "failed." Different errors pop up every time it is executed. Would it help if I used the SDK platform? Does anyone have any suggestions?
I can't test this on multiple webcams since I only have one, but I'm sure OpenCV2.0 should be able to handle it. Here's some sample code (I use Vista) with one webcam to get you started.
#include <cv.h>
#include <highgui.h>
using namespace cv;
int main()
{
// Start capturing on camera 0
VideoCapture cap(0);
if(!cap.isOpened()) return -1;
// This matrix will store the edges of the captured frame
Mat edges;
namedWindow("edges",1);
for(;;)
{
// Acquire the frame from cap into frame
Mat frame;
cap >> frame;
// Now, find the edges by converting to grayscale, blurring and then Canny edge detection
cvtColor(frame, edges, CV_BGR2GRAY);
GaussianBlur(edges, edges, Size(7,7), 1.5, 1.5);
Canny(edges, edges, 0, 30, 3);
// Display the edges and the frame
imshow("edges", edges);
imshow("frame", frame);
// Terminate by pressing a key
if(waitKey(30) >= 0) break;
}
return 0;
}
Note:
The matrix edges is allocated during
the first frame processing and unless
the resolution will suddenly change,
the same buffer will be reused for
every next frame’s edge map.
As you can see, the code is quite clean and readable! I lifted this from the OpenCV 2.0 documentation (opencv.pdf).
The code not only displays the image from the webcam (under frame) but also does real-time edge detection! Here's a screenshot when I pointed the webcam at my monitor :)
screenshot http://img245.imageshack.us/img245/5014/scrq.png
If you want code to just display the frames from one camera:
#include <cv.h>
#include <highgui.h>
using namespace cv;
int main()
{
VideoCapture cap(0);
if(!cap.isOpened()) return -1;
for(;;)
{
Mat frame;
cap >> frame;
imshow("frame", frame);
if(waitKey(30) >= 0) break;
}
return 0;
}
If the program works with UAC off or when running administrator, make sure the place you choose to save the results are in writable places like the user's my documents folder. Generally speaking root folders and the program files folder is read only for normal users.