openCV to dlib - Mat to array2d - c++

Question

I'm successfully opening and displaying a .avi video using OpenCV and I need this to go through OpenCV because I want to learn how to make OpenCV and dlib communicate.
For my understanding, a Mat has to be converted into an array2d in order to be processed by dlib so here's my first attempt:
cv::VideoCapture cap("/home/francesco/Downloads/05-1.avi");
cv::namedWindow("UNLTD", CV_WINDOW_AUTOSIZE);
while(1)
{
cv::Mat temp;
cv_image<bgr_pixel> cimg(temp);
std::vector<rectangle> faces = detector(cimg);
cout << faces.size() << endl;
cv::imshow("UNLTD", temp);
}
This returns the error
Error detected in file /usr/local/include/dlib/opencv/cv_image.h.
Error detected in function dlib::cv_image<pixel_type>::cv_image(cv::Mat) [with pixel_type = dlib::bgr_pixel].
Failing expression was img.depth() == cv::DataType<typename pixel_traits<pixel_type>::basic_pixel_type>::depth && img.channels() == pixel_traits<pixel_type>::num.
The pixel type you gave doesn't match pixel used by the open cv Mat object.
img.depth(): 0
img.cv::DataType<typename pixel_traits<pixel_type>::basic_pixel_type>::depth: 0
img.channels(): 1
img.pixel_traits<pixel_type>::num: 3
I tried swapping bgr_pixel to rgb_pixel but without any luck.
Looking around the internet somebody mentioned that the img.depth() is zero, therefore I should use unsigned char instead of rgb_pixel.
First thing: my video is playing in colors, so it does have 3 channels, I don't understand why it should be interpreted as a 1 channel image.
The strange thing is that, making that change from rgb_pixel to unsigned char, makes the software work but ZERO faces are detected on that video stream (that is the video of a guy talking and the face on the same video is detected with no problems by dlib on python.
I don't understand what I'm doing wrong

Answer 1

In your code, the temp is empty because you have not fed any frame from the video capture to it. Conversion of cv::Mat to dlib::array2d is also not correct. Please see this post for more information.
You may try:
cv::VideoCapture cap("/home/francesco/Downloads/05-1.avi");
cv::namedWindow("UNLTD", CV_WINDOW_AUTOSIZE);
dlib::frontal_face_detector detector = dlib::get_frontal_face_detector();
while(1)
{
cv::Mat temp;
cap >> temp;
dlib::array2d<bgr_pixel> dlibFrame;
dlib::assign_image(dlibFrame, dlib::cv_image<bgr_pixel>(temp));
std::vector<rectangle> faces = detector(dlibFrame);
cout << faces.size() << endl;
cv::imshow("UNLTD", temp);
}