C++ Multithreading OpenCV locking problems - c++

Question

I'm phasing a frustrating issue here with multithreading and openCV.
I want to read two different video files (video.mp4 and video-copy.mp4). I want to place frames of one video on one corner(a background jpg image) of a canvas and the frames of the second video on another corner on the same canvas.
Below is the code I used to hope to accomplish this goal. It compiles well but runs unpredictably:
one time it will show both frames on the canvas but freezes, or frame from only one video are show and the other video can't be seen
another time it runs correctly but still shows the runtime errors in red at my terminal (i included the runtime errors at the bottom of this cpp file, also I'm running this on ubuntu.)
I'd like to thank you all a head of time for your help. Thanks again everyone.
#include <iostream>
#include "opencv2/highgui/highgui.hpp"
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <string>
#include <thread>
#include <mutex>
using namespace std;
using namespace cv;
//The main canvas (a background image)
Mat canvas=imread("canvas.jpg", CV_LOAD_IMAGE_COLOR);
// the mutex for accessing the canvas variable
mutex mu;
// The function that plays a video on a predefined ROI on the canvas
void play(string n, Point p) //this function could be called n number of times as its own thread
{
VideoCapture cap(n); //open a local video file
int fps=cap.get(CV_CAP_PROP_FPS); // get it's frames per-seconds frame rates
char k; // for exiting the frame fetching loop
Mat temp; // a temporary holding variable for each frame from the stream
while(1)
{
unique_lock<mutex> locker(mu); // lock the mutex (shared memory is the canvas matrix)
cap>>temp; // get a frame from the stream to the temporary buffer
if(temp.empty())// if stream is empty get out of this loop
{
locker.unlock(); // unlock the mutex first before exiting this loop
break; // exit the loop now.
}
// syntax= src_image.copyto(dst_image(Rect(Point(x,y), size(src_image))))
//dest_image is the shared object between the threads.
temp.copyTo(canvas(Rect(p.x, p.y, temp.cols, temp.rows))); // past a frame onto the canvas at location (x,y) and size(of the frame)
locker.unlock();// done interacting with the shared variable, so unlock the mutex
this_thread::sleep_for(chrono::milliseconds(10));// wait briefly before looping again.
}
}
int main()
{
char k; // for breaking out of the main loop ( the display image loop)
namedWindow("MainWindow", WINDOW_AUTOSIZE); // create the main window
// create and start the first thread, pass a filename and a coordinates to place the frame on the canvas
thread P1(play, "video-copy.mp4", Point(0,0));
// create and start the second thread, they should both be able to access the canvas variable (one at a time ofcourse)
thread P2(play, "video.mp4", Point(200,300));
while(1) // keep displaying the canvas as it is changing.
{
unique_lock<mutex> locker(mu); // lock the mutex again, because we are using the canvas (shared object) again,
imshow("MainWindow", canvas); // show image
locker.unlock(); // unlock the mutex again.
k=waitKey(10); // flush the buffer and wait for 10 milliseconds
if(k==27) //exit the display image loop.
break;
}
// Join the child threads back with the parrent thread before exiting.
P1.join();
P2.join();
destroyWindow("MainWindow"); // destroy the main window
return 0; // end the main.
}
/* The errors I always get but cant find what is causing it
(MainWindow:3157): GLib-GObject-CRITICAL **: g_object_remove_weak_pointer: assertion `G_IS_OBJECT (object)' failed
[NULL # 0xb1b06ca0] insufficient thread locking around avcodec_open/close()
[NULL # 0xb1b07aa0] insufficient thread locking around avcodec_open/close()
[NULL # 0xb1b06ca0] insufficient thread locking around avcodec_open/close()
[NULL # 0xb1b07aa0] insufficient thread locking around avcodec_open/close()
[IMGUTILS # 0xb28fde64] Picture size 0x0 is invalid
[IMGUTILS # 0xb28fde94] Picture size 0x0 is invalid
[swscaler # 0xad180300] bad dst image pointers
*/