OpenCV: processing multiple images in a C++ vector using pthreads - c++

Question

I have a large number images in a file that I need to perform various processing operations on. Here is what I am trying to do
1) Read the images into a file, and put them in a C++ vector named imageQueue (a mutable array)
2) Create a number of threads
3) Each thread grabs an image from imageQueue, and then erases that image from the vector
4) Each thread then goes ahead and processes that image
5) When finished processing, each thread grabs the next image from the vector
6) This entire process runs until there are no more images in the imageQueue, at which point the program ends. (currently I have 4 photos in the file that I am using for tests, which is why in my loops I run from i = 0 to i < 4. When I complete this, I will have many more photos.
I have named each of the images in the file 00.jpg, 01.jpg, 02.jpg....
For testing purposes, right now I am simply trying to have each thread display the image it grabbed. However, when I run this only purely white windows pop up, instead of the actual image. Any help on why this is happening and how to correct it?
Thanks!
Here is my code:
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <pthread.h>
#include <vector.h>
#define NUM_THREADS 2
using namespace std;
using namespace cv;
/* Function Declarations */
void* startProcessing(void* args);
/* Global Variables */
vector <Mat> imageQueue;
static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
int main(int argc, char** argv)
{
VideoCapture cap;
cap.open("/Users/nlauer/Documents/ImageSequence/%02d.jpg");
for(int i = 0; i < 4; i++)
{
Mat frame;
vector<Mat>::iterator it;
cap >> frame;
it = imageQueue.end();
it = imageQueue.insert(it, frame);
}
/* Create the threads */
pthread_t tids[NUM_THREADS];
for(int i = 0; i < NUM_THREADS; i++)
{
pthread_create(&tids[i], NULL, startProcessing, NULL);
}
/* Reap the threads */
for(int i = 0; i < NUM_THREADS; i++)
{
pthread_join(tids[i], NULL);
}
imageQueue.clear();
return 0;
}
void* startProcessing(void* args)
{
/* Each thread grabs an image from imageQueue, removes it from the
queue, and then processes it. The grabbing and removing are done
under a lock */
Mat image;
Mat emptyImage;
/* Get the first image for each thread */
pthread_mutex_lock(&mutex);
if(!imageQueue.empty()) {
image = imageQueue[0];
vector<Mat>::iterator it;
it = imageQueue.begin();
it = imageQueue.erase(it);
}
pthread_mutex_unlock(&mutex);
while(!image.empty())
{
/* Process the image - right now I just want to display it */
namedWindow("window", CV_WINDOW_AUTOSIZE);
imshow("window", image);
sleep(10);
/* Obtain the next image in the queue */
pthread_mutex_lock(&mutex);
if(!imageQueue.empty()) {
image = imageQueue[0];
vector<Mat>::iterator it;
it = imageQueue.begin();
it = imageQueue.erase(it);
} else {
image = emptyImage;
}
pthread_mutex_unlock(&mutex);
}
return NULL;
}

Answer 1

What you try to achieve in the items 3)-5) is exactly what Intel's library TBB is designed for. Take a look at tbb::parallel_for.
All you need is a class which has operator(Mat) which processes a single image, the library TBB will take care of the thread handling.
You can go even further if you use tbb::concurrent_queue instead of your vector to keep the images. Then processing can start even while reading has not finished. You might use tbb::parallel_do instead of tbb::parallel_for in this approach.

Answer 2

Issue has been resolved - I simply went ahead and continued with the processing as is. Something strange was happening when I tried to show the image in each thread, but as long as you let each thread finish its routine, then show the image after, everything worked fine.