I have 2 webcams and I want to get input from both of them at the same time. Therefore I believe I have to work with threads in c++ which is pthread. When I run my code given below, the webcam turns on for a second and the routine exits. I can't figure out what is wrong in my code.
void *WebCam(void *arg){
VideoCapture cap(0);
for (; ; ) {
Mat frame;
*cap >> frame;
resize(frame, frame, Size(640, 480));
flip(frame, frame, 1);
imshow("frame", frame);
if(waitKey(30) >= 0)
break;
}
pthread_exit(NULL);
}
int main(){
pthread_t thread1, thread2;
pthread_create(&thread1, NULL, &WebCam, NULL);
return 0;
}
this is doe for one webcam just to turn and do streaming. Once this one works than other will be just copy of it.
When you create the thread, it starts running, but your main program, which is still running, just terminates, making the child thread finish too. Try adding this after pthread_create:
pthread_join(thread1, NULL);
By the way, even if you have two cameras, you can avoid the use of threads. I am not sure, but they could be problematic when dealing with the highgui functions (imshow, waitKey), because you must make sure they are thread-safe. Otherwise, what will be the result of having two threads calling waitKey at the same time?
You could get rid of threads with a design similar to this one:
VideoCapture cap0(0);
VideoCapture cap1(1);
for(;;)
{
cv::Mat im[2];
cap0 >> im[0];
cap1 >> im[1];
// check which of im[i] is non empty and do something with it
}
Related
I am having an OpenCV program which works like this:
VideoCapture cap(0);
Mat frame;
while(true) {
cap >> frame;
myprocess(frame);
}
The problem is if myprocess takes a long time which longer than camera's IO interval, the captured frame be delayed, cannot get the frame synchronized with the real time.
So, I think to solve this problem, should make the camera streaming and myprocess run parallelly. One thread does IO operation, another does CPU computing. When the camera finished capture, send to work thread to processing.
Is this idea right? Any better strategy to solve this problem?
Demo:
int main(int argc, char *argv[])
{
cv::Mat buffer;
cv::VideoCapture cap;
std::mutex mutex;
cap.open(0);
std::thread product([](cv::Mat& buffer, cv::VideoCapture cap, std::mutex& mutex){
while (true) { // keep product the new image
cv::Mat tmp;
cap >> tmp;
mutex.lock();
buffer = tmp.clone(); // copy the value
mutex.unlock();
}
}, std::ref(buffer), cap, std::ref(mutex));
product.detach();
while (cv::waitKey(20)) { // process in the main thread
mutex.lock();
cv::Mat tmp = buffer.clone(); // copy the value
mutex.unlock();
if(!tmp.data)
std::cout<<"null"<<std::endl;
else {
std::cout<<"not null"<<std::endl;
cv::imshow("test", tmp);
}
}
return 0;
}
Or use a thread keep clearing the buffer.
int main(int argc, char *argv[])
{
cv::Mat buffer;
cv::VideoCapture cap;
std::mutex mutex;
cap.open(0);
std::thread product([](cv::Mat& buffer, cv::VideoCapture cap, std::mutex& mutex){
while (true) { // keep product the new image
cap.grab();
}
}, std::ref(buffer), cap, std::ref(mutex));
product.detach();
int i;
while (true) { // process in the main thread
cv::Mat tmp;
cap.retrieve(tmp);
if(!tmp.data)
std::cout<<"null"<<i++<<std::endl;
else {
cv::imshow("test", tmp);
}
if(cv::waitKey(30) >= 0) break;
}
return 0;
}
The second demo I thought shall be work base on https://docs.opencv.org/3.0-beta/modules/videoio/doc/reading_and_writing_video.html#videocapture-grab, but it's not...
In project with Multitarget tracking I used 2 buffers for frame (cv::Mat frames[2]) and 2 threads:
One thread for capturing the next frame and detect objects.
Second thread for tracking the detected objects and draw result on frame.
I used index = [0,1] for the buffers swap and this index was protected with mutex. For signalling about the end of work was used 2 condition variables.
First works CatureAndDetect with frames[capture_ind] buffer and Tracking works with previous frames[1-capture_ind] buffer. Next step - switch the buffers: capture_ind = 1 - capture_ind.
Do you can this project here: Multitarget-tracker.
I have an opencv program where the image processing is sensitive to having a stable and relatively high framerate in the video capture. Unfortunately, all the image processing I do is slowing down the framerate significantly, resulting in erroneous behavior in my program. I believe that putting the camera on a separate thread and having the image processing happen on its own thread would improve framerate, but I am not sure how to do this. Can anyone guide me through the process?
UPDATE: After doing some research on threads, I managed to implement threading to where the final video feed post-processed is displayed. However, somehow my implementation of it is causing the image processing methods to fail (ex. before I could successfully track a moving object, now it is erroneous whether or not it is tracked). I suspect this has something to do with the image processing algorithms not being able to process each frame fast enough as new frames are read in. How can I improve this implementation so that my processing methods worked as they did without multithreading?
void CaptureFrames() {
VideoCapture capture(0);
if (!capture.isOpened()) {
cout << "cannot open camera" << endl;
}
while (true) {
//CAMERAFRAME is a global Mat defined at the top of my program
capture.read(CAMERAFRAME);
if (waitKey(30) >= 0) { break; }
}
}
void ProcessFrames() {
while (true) {
Mat hsvFrame;
Mat binMat;
Mat grayFrame;
Mat grayBinMat;
if (!CAMERAFRAME.empty()) {
//do image processing functions (these worked before implementing threads and are not causing errors)
imshow("gray binary", grayBinMat);
imshow("multithread test", CAMERAFRAME);
}
if (waitKey(30) >= 0) { break; }
}
}
int main(int argc, char** argv[]) {
thread t1(CaptureFrames);
thread t2(ProcessFrames);
while(true) {
if(waitKey(30) >= 0) {break;}
}
return 0;
}
Try the older version again but remove this last line from the ProcessFrames function.
if (waitKey(30) >= 0) { break; }
On showing images don't make it wait again for 30 m-seconds, the while loop will be enough
I have a system that is typically running a scan time of 100 HZ or 10 ms and performing time critical tasks. I'm trying to add a camera with opencv to once a while (depends on when a user interacts with the system so it can be anywhere from 10 seconds pauses to minutes) capture an image for quality control.
Here is what my code is doing:
int main(int, char**)
{
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) // check if we succeeded
return -1;
UMat frame;
for(;;){
if (timing_variable_100Hz){
cap >> frame; // get a new frame from camera
*Do something time critical*
if(some_criteria_is_met){
if(!frame.empty()) imwrite( "Image.jpg", frame);
}
}
}
return 0;
}
Now the issue I'm having is that cap >> frame takes a lot of time.
My scan time regularly runs around 3ms and now it's at 40ms. Now my question is, are there anyway to open the camera, capture, then not have to capture every frame after until I have to? I tried to move the cap >> frame inside the if(some_criteria_is_met) which allowed me to capture the first image correctly but the second image which was taken a few minutes later was a single frame past the first captured image (I hope that makes sense).
Thanks
The problem is that your camera has a framerate of less than 100fps, probably 30fps (according to the 32ms you measured), so grab wait for a new frame to be available.
Since there is no way to do a non blocking read in opencv, i think that your best option is to do the video grabbing in another thread.
Something like this, if you use c++11 (this is an example, not sure it is entirely correct):
void camera_loop(std::atomic<bool> &capture, std::atomic<bool> &stop)
{
VideoCapture cap(0);
Mat frame;
while(!stop)
{
cap.grab();
if(capture)
{
cap.retrieve(frame);
// do whatever you do with the frame
capture=false;
}
}
}
int main()
{
std::atomic<bool> capture=false, stop=false;
std::thread camera_thread(camera_loop, std::ref(capture), std::ref(stop));
for(;;)
{
// do something time critical
if(some_criteria_is_met)
{
capture=true;
}
}
stop=true;
camera_thread.join();
}
It doesn't answer your question of are there anyway to open the camera, capture, then not have to capture every frame after until I have to?, but a suggestion
You could try and have the cap >> frame in a background thread which is responsible only for capturing the frames.
Once the frame is in memory, push it to some sort of shared cyclic queue to be accessed from the main thread.
I have one problem about thread in MFC using opencv. Let I describe my problem first. I have one GUI that used to display video frame from camera. Hence, I must used one thread to get video from camera and display it in to GUI. It is done. However, I want to extend my problem such as: When video is displaying, I want to show that video in other window of opencv by command
IplImage* image2=cvCloneImage(&(IplImage)original);
cvShowImage("Raw Image", image2);
cvReleaseImage(&image2);
As my knowledge, I need to create a new thread inside thread get video from camera. Is it possible to do it? Let see my code and could you give me some solution or suggestion to do that task? Thank you so much
This is my code
THREADSTRUCT *_param = new THREADSTRUCT;
_param->_this = this;
CWinThread* m_hThread;
m_hThread = AfxBeginThread (StartThread, _param);
In StartThread function, I will call the load video from camera such as
UINT Main_MFCDlg::StartThread (LPVOID param)
{
THREADSTRUCT* ts = (THREADSTRUCT*)param;
cv::VideoCapture cap;
cap.open(0);
while (true)
{
Mat frame;
Mat original;
cap >> frame;
if (!frame.empty()){
original = frame.clone();
//Display video in GUI
CDC* vDC_VIDEO;
vDC_VIDEO=ts->_this->GetDlgItem(IDC_VIDEO)->GetDC();
CRect rect_VIDEO;
ts->_this->GetDlgItem(IDC_VIDEO)->GetClientRect(&rect_VIDEO);
//Is it possible to create a thread in here to show video with other
//delay time such as 1000ms
//To call the function cv::imshow("Second window", original);
}
if (waitKey(30) >= 0) break;// Delay 30ms for first window
}
}
Notethat thread struct look like
//structure for passing to the controlling function
typedef struct THREADSTRUCT
{
Main_MFCDlg* _this;
} THREADSTRUCT;
Of course you can create a thread in a thread. You problem is: the worker thread should not access UI object directly in the UI thread, the details are explained here
In your worker thread, after the background job is done, you can use SendMessage to send an message to the GUI and let it update.
Hey guys,
I'm using OpenCV with the C++ API, and in order for my project to be more reliable I need a certain camera connection\disconnection handling.
I have searched for how-to's, but I could only find answers that require an ugly hack in order to do so.
Can you suggest a cleaner way to do it?
Thnx
Detecting camera connection/disconnection might require some tricks.
I suggest that you start another thread to check the success of cvCreateCameraCapture() in a loop, while your application is running.
Something like the following:
while (run_detection_thread) // global variable controlled by the main thread
{
CvCapture* capture = cvCreateCameraCapture(-1); //-1 or whatever number works for you
if (camera) //camera is connected
{
sleep(1);
}
else
{
// camera was disconnected
}
}
I think that I have a good workaround for this problem. I create an auxiliary Mat array with zeros with the same resolution like the output from camera. I assign it to Mat array to which just after is assign the frame captured from camera and at the end I check the norm of this array. If it is equal zero it means that there was no new frame captured from camera.
VideoCapture cap(0);
if(!cap.isOpened()) return -1;
Mat frame;
cap >> frame;
Mat emptyFrame = Mat::zeros(CV_CAP_PROP_FRAME_WIDTH, CV_CAP_PROP_FRAME_HEIGHT, CV_32F);
for(;;)
{
frame = emptyFrame;
cap >> frame;
if (norm(frame) == 0) break;
}