Serializing Face Features for recognition in DLIB - c++

Question

I am writing a dlib code to do face recognition on 1 to 1 basis.
I followed the code sample in dlib samples and did the following:
std::vector<matrix<rgb_pixel>> faces;
for (auto face : detector(img1))
{
auto shape = sp(img1, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img1, get_face_chip_details(shape, 150, 0.25), face_chip);
faces.push_back(move(face_chip));
}
this is for the first image and then did the same for the second image:
for (auto face : detector(img2))
{
auto shape = sp(img2, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img2, get_face_chip_details(shape, 150, 0.25), face_chip);
faces.push_back(move(face_chip));
}
then i continue as per the mentioned link:
std::vector<matrix<float, 0, 1>> face_descriptors = net(faces);
std::vector<sample_pair> edges;
for (size_t i = 0; i < face_descriptors.size(); ++i)
{
for (size_t j = i; j < face_descriptors.size(); ++j)
{
if (length(face_descriptors[i] - face_descriptors[j]) < threshold)
edges.push_back(sample_pair(i, j));
}
}
std::vector<unsigned long> labels;
const int num_clusters = chinese_whispers(edges, labels);
//etc
and now comes my question. img1 is an image already available to the code that is read when i need to match a specific person. (ie if I want to mach personX, img1 is read using
load_image(img1, "personX.jpg");
Instead of having the image saved, i was trying to save the features and load them to reduce the time spent the extraction of features. so what I did is I moved the first for loop at a different function (enrollment like) and made it something like this:
std::vector<matrix<rgb_pixel>> faces;
for (auto face : detector(img1))
{
auto shape = sp(img1, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img1, get_face_chip_details(shape, 150, 0.25), face_chip);
serialize("personX.dat") <<face_chip;
}
then at the recognition instead of the loop i used
matrix<rgb_pixel> face_chip;
deserialize("personX.dat")>>face_chip;
faces.push_back(move(face_chip));
and the rest of the code from the extraction of img2 onward remained the same. the code compiled. But during execution when i reach the recognition i end up with the following error:
**************************** FATAL ERROR DETECTED ****************************
Error detected at line 216.
Error detected in file /usr/local/include/dlib/dnn/input.h.
Error detected in function void dlib::input_rgb_image_sized::to_tensor(forward_iterator, forward_iterator, dlib::resizable_tensor&) const [with forward_iterator = __gnu_cxx::__normal_iterator*, std::vector > >; long unsigned int NR = 150ul; long unsigned int NC = 150ul].
Failing expression was i->nr()==NR && i->nc()==NC input_rgb_image_sized::to_tensor()
All input images must have 150 rows and 150 columns, but we got one with 0 rows and 0 columns.
Is there something wrong with the serialization / de-serialization? or should i do writing the features to a file with another method?
code for the full funtion:
try
{
load_image(img1, check_image);
}
catch (...)
{
cout<<"Name: "<<uname<<" doesn't exist"<<endl;
return;
}
else
{
QElapsedTimer timer;
timer.start();
dlib::assign_image(img2, dlib::cv_image<bgr_pixel>(colorImage));
std::vector<matrix<rgb_pixel>> faces;
for (auto face : detector(img1))
{
auto shape = sp(img1, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img1, get_face_chip_details(shape, 150, 0.25), face_chip);
faces.push_back(move(face_chip));
// serialize("out.dat")<<face_chip; //used whin i dont need to read image
}
// matrix<rgb_pixel> face_chip; //used whin i dont need to read image
// deserialize("out.dat")>>face_chip; //used whin i dont need to read image
// faces.push_back(move(face_chip)); //used whin i dont need to read image
cout<<"Time to extract features for enroled image: "<<timer.elapsed()<<endl;
timer.restart();
for (auto face : detector(img2))
{
auto shape = sp(img2, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img2, get_face_chip_details(shape, 150, 0.25), face_chip);
faces.push_back(move(face_chip));
}
cout<<"Time to extract features for new image: "<<timer.elapsed()<<endl;
timer.restart();
if (faces.size() < 2)
{
cout<<"No Face"<<endl;
}
else
{
std::vector<matrix<float, 0, 1>> face_descriptors = net(faces);
std::vector<sample_pair> edges;
for (size_t i = 0; i < face_descriptors.size(); ++i)
{
for (size_t j = i; j < face_descriptors.size(); ++j)
{
if (length(face_descriptors[i] - face_descriptors[j]) < threshold)
edges.push_back(sample_pair(i, j));
}
}
std::vector<unsigned long> labels;
const int num_clusters = chinese_whispers(edges, labels);
if (num_clusters == 1)
{
cout<<"Recognized"<<endl;
}
else
{
cout<<"Faces don't match";
}
}
cout<<"Needed time is: "<<timer.elapsed()<<" ms"<<endl;
}

Answer 1

instead of serializing the Matrix i serialized the output vector (faces).
serialize("personX.dat")<<faces;
then when doing the recognition i deserialized the dat file and used the resulting vector:
std::vector<matrix<rgb_pixel>> faces;
deserialize("out.dat")>>faces;
for (auto face : detector(img2))
{
auto shape = sp(img2, face);
matrix<rgb_pixel> face_chip;
extract_image_chip(img2, get_face_chip_details(shape, 150, 0.25), face_chip);
faces.push_back(move(face_chip));
}
and I continued as mentioned in the question.
I don't know if this is the best way to do it... but it worked.