OpenCV C++ detectMultiScale crashes UE4 - Access violation writing location - c++

I am trying to create an OpenCV plugin for Unreal Engine 4 to detect faces, but when the code reaches the detectMultiScale function I get this error:
Exception thrown at 0x00007FFD4F34FF1B (UE4Editor-Core.dll) in
UE4Editor-Win64-DebugGame.exe: 0xC0000005: Access violation writing
location 0x0000000000000010.
I am using OpenCV version 4.1.0 with VS2017. I also tried with version 3.4.7 of OpenCV, but having the same issue.
This is my function that gets the webcam input and renders it in UE4:
In the constructor:
if (!face_cascade.load("C:\\models\\haarcascade_frontalface_alt.xml")) { printf("--(!)Error loading\n"); }
And the function:
void AWebcamReader::UpdateTexture()
{
std::vector<cv::Rect> faces;
cv::Mat frame_gray;
//cvtColor(frame, frame_gray, cv::COLOR_BGR2GRAY);
if (isStreamOpen && frame.data)
{
cv::cvtColor(frame, frame_gray, cv::COLOR_BGR2GRAY);
cv::equalizeHist(frame_gray, frame_gray);
face_cascade.detectMultiScale(frame_gray, faces, 1.1, 2, 0 | cv::CASCADE_SCALE_IMAGE, cv::Size(30, 30));
for (size_t i = 0; i < faces.size(); i++)
{
cv::Point center(faces[i].x + faces[i].width*0.5, faces[i].y + faces[i].height*0.5);
ellipse(frame, center, cv::Size(faces[i].width*0.5, faces[i].height*0.5), 0, 0, 360, cv::Scalar(255, 0, 255), 4, 8, 0);
cv::Mat faceROI = frame_gray(faces[i]);
}
// Copy Mat data to Data array
for (int y = 0; y < VideoSize.Y; y++)
{
for (int x = 0; x < VideoSize.X; x++)
{
int i = x + (y * VideoSize.X);
Data[i].B = frame.data[i * 3 + 0];
Data[i].G = frame.data[i * 3 + 1];
Data[i].R = frame.data[i * 3 + 2];
}
}
// Update texture 2D
UpdateTextureRegions(VideoTexture, (int32)0, (uint32)1, VideoUpdateTextureRegion, (uint32)(4 * VideoSize.X), (uint32)4, (uint8*)Data.GetData(), false);
}
}
If I comment out the detectMultiScale function, the plugin runs without a problem and I see the webcam input rendered correctly.
The Exception gets called on this line in VT2.cpp (VT2 is the project name)
IMPLEMENT_PRIMARY_GAME_MODULE( FDefaultGameModuleImpl, VT2, "VT2" );
Any suggestions would be greatly appreciated!

Related

Dlib not detecting face in kurento opencv filter

I have created an opencv filter that can detect if a person blinks for Kurento the WebRTC framework. My code works in a standalone opencv app. However, once I converted to the opencv filter for Kurento it started playing up. When the module/filter was compiled without optimisation flags it would briefly detect the face and draw contours around the eyes. However, after compiling the module/filter with optimisation flags, performance improved, but no face was being detected. Here's the code I have in the filter:
void BlinkDetectorOpenCVImpl::process(cv::Mat &mat) {
std::vector <dlib::rectangle> faces;
// Just resize input image if you want
resize(mat, mat, Size(800, 450));
cv_image <rgb_alpha_pixel> cimg(mat);
dlib::array2d<unsigned char> img_gray;
dlib::assign_image(img_gray, cimg);
faces = detector(img_gray);
std::cout << "XXXXXXXXXXXXXXXXXXXXX FACES: " << faces.size() << std::endl;
std::vector <full_object_detection> shapes;
for (unsigned long i = 0; i < faces.size(); ++i) {
full_object_detection shape = pose_model(cimg, faces[i]);
std::vector <Point> left_eye_points = get_points_for_eye(shape, LEFT_EYE_START, LEFT_EYE_END);
std::vector <Point> right_eye_points = get_points_for_eye(shape, RIGHT_EYE_START, RIGHT_EYE_END);
double left_eye_ear = get_eye_aspect_ratio(left_eye_points);
double right_eye_ear = get_eye_aspect_ratio(right_eye_points);
double ear = (left_eye_ear + right_eye_ear) / 2.0;
// Draw left eye
std::vector <std::vector<Point>> contours;
contours.push_back(left_eye_points);
std::vector <std::vector<Point>> hull(1);
convexHull(contours[0], hull[0]);
drawContours(mat, hull, -1, Scalar(0, 255, 0));
// Draw right eye
contours[0] = right_eye_points;
convexHull(contours[0], hull[0]);
drawContours(mat, hull, -1, Scalar(0, 255, 0));
if (ear < EYE_AR_THRESH) {
counter++;
} else {
if (counter >= EYE_AR_CONSEC_FRAMES) {
total++;
/* std::string sJson = "{\"blink\": \"blink\"}";
try
{
onResult event(getSharedFromThis(), onResult::getName(), sJson);
signalonResult(event);
}
catch (std::bad_weak_ptr &e)
{
}*/
}
counter = 0;
}
cv::putText(mat, (boost::format{"Blinks: %d"} % total).str(), cv::Point(10, 30),
cv::FONT_HERSHEY_SIMPLEX,
0.7, Scalar(0, 0, 255), 2);
cv::putText(mat, (boost::format{"EAR: %.2f"} % ear).str(), cv::Point(300, 30),
cv::FONT_HERSHEY_SIMPLEX,
0.7, Scalar(0, 0, 255), 2);
}
}
} /* blinkdetector */
I was able to fix my own problem. I found that instead of resizing the image to an arbitrary resolution you should resize it by half the width and half the height of the actual image resolution. Resizing an image to a lower size makes Dlib face detection fast. So here's what I did to solve the issue:
Mat tmpMat = mat.clone();
resize(tmpMat, tmpMat, Size(tmpMat.size().width / 2, tmpMat.size().height / 2));
I had to clone the image sent by Kurento to my method because for some odd reason the original Mat doesn't show the contours when turned into a Dlib image with cv_image.

How to load .xml file in the Microsoft Visual Studio

For my project, I'm using Microsoft Visual Studio Ultimate 2013, opencv-3.3.1-vc14 and Microsoft Window 10. In my project, I'm going to load an .xml file but it didn't.
The directory of my project is: C:\Users\Muhammad_Anas_Hashmi\Desktop\Face_and_Eye_Detection. But the .xml file that I want to load has the directory: C:\opencv\sources\data\haarcascades\haarcascade_eye.xml.
The way that I use to load the file is:
CascadeClassifier eye_cascade;
eye_cascade.load("C:\\opencv\sources\data\haarcascades_cuda\haarcascade_eye.xml")
But I check through a message whether the file has been loaded or not through the following code.
if (!eye_cascade.load("C:\\opencv\sources\data\haarcascades_cuda\haarcascade_eye.xml"))
{
printf("Error loading cascade file for eye\n");
}
Every time it prints out the message. It means file has not been loaded.
But the whole code of my project is:
#include<stdio.h>
#include<stdlib.h>
#include<math.h>
#include<opencv\cv.h>
#include<opencv\highgui.h>
#include<opencv2\objdetect\objdetect.hpp>
#include<opencv2\highgui\highgui.hpp>
#include<opencv2\imgproc\imgproc.hpp>
#include<vector>
using namespace cv;
using namespace std;
int main()
{
CascadeClassifier face_cascade, eye_cascade;
if (!face_cascade.load("C:\\Users\Muhammad Anas Hashmi\Desktop\Face_and_Eye_Detection\haarcascade_frontalface_alt2.xml"))
{
printf("Error loading cascade file for face\n");
/*return 1;*/
}
if (!eye_cascade.load("C:\\opencv\sources\data\haarcascades_cuda\haarcascade_eye.xml"))
{
printf("Error loading cascade file for eye\n");
/*return 1;*/
}
VideoCapture capture(0);
if (!capture.isOpened())
{
printf("Error to initialize camera");
return 1;
}
Mat cap_img, gray_img;
vector<Rect> faces, eyes;
while (1)
{
capture >> cap_img;
waitKey(10);
cvtColor(cap_img, gray_img, CV_BGR2GRAY);
cv::equalizeHist(gray_img, gray_img);
face_cascade.detectMultiScale(gray_img, faces, 1.1, 10, CV_HAAR_SCALE_IMAGE | CV_HAAR_DO_CANNY_PRUNING, cvSize(0, 0), cvSize(300, 300));
for (int i = 0; i < faces.size(); i++)
{
Point pt1(faces[i].x + faces[i].width, faces[i].y + faces[i].height);
Point pt2(faces[i].x, faces[i].y);
Mat faceROI = gray_img(faces[i]);
eye_cascade.detectMultiScale(faceROI, eyes, 1.1, 2, 0 | CV_HAAR_SCALE_IMAGE, Size(30, 30));
for (size_t j = 0; j < eyes.size(); j++)
{
Point center(faces[i].x + eyes[j].x + eyes[j].width * 0.5, faces[i].y + eyes[j].y + eyes[j].width * 0.5);
int radius = cvRound((eyes[j].width + eyes[j].height)*0.25);
circle(cap_img, center, radius, Scalar(255, 0, 0), 2, 8, 0);
}
rectangle(cap_img, pt1, pt2, cvScalar(0, 255, 0), 2, 8, 0);
}
imshow("Result", cap_img);
waitKey(3);
char c = waitKey(3);
if (c == 27)
{
break;
}
}
system("PAUSE");
return 0;
}
I found the problem at the start while loading the .xml file.
C:\\opencv\sources\data\haarcascades_cuda\haarcascade_eye.xm
You only escaped the first backslash. You need to escape all:
C:\\opencv\\sources\\data\\haarcascades_cuda\\haarcascade_eye.xml

opencv face detect crashes computer

Hello everyone I have been having this issue with my opencv 3.1 program that I am programming in visual studio 2015 C++. It can use the camera just fine and display images just fine, but I try out a face detect example from the documentation my computer freezes up. Here is the code:
Here are the Two Functions where it gets hung up on: but mainly stops around the faceDetect function, I will do a break point to know specifically where.
void CoreVision::init()
{
face_cascade_name = "C:\\PAD\\opencv\\haarcascades\\haarcascade_frontalface_alt.xml";
eyes_cascade_name = "C:\\PAD\\opencv\\haarcascades\\haarcascade_eye_tree_eyeglasses.xml";
if (!face_cascade.load(face_cascade_name)) { printf("--(!)Error loading face cascade\n"); };
if (!eyes_cascade.load(eyes_cascade_name)) { printf("--(!)Error loading eyes cascade\n"); };
std::cout << "face_dir: " << face_cascade_name << std::endl;
std::cout << "eye_dir: " << eyes_cascade_name << std::endl;
}
cv::Mat CoreVision::detectFace(cv::Mat main)
{
cv::Mat nuller;
try
{
faces.clear();
cv::Mat frame_gray;
cv::cvtColor(main, frame_gray, cv::COLOR_BGR2GRAY);
cv::equalizeHist(frame_gray, frame_gray);
face_cascade.detectMultiScale(frame_gray, faces, 1.1, 2, 0 | cv::CASCADE_SCALE_IMAGE, cv::Size(30, 30));
for (size_t i = 0; i < faces.size(); i++)
{
cv::Point center(faces[i].x + faces[i].width / 2, faces[i].y + faces[i].height / 2);
cv::ellipse(main, center, cv::Size(faces[i].width / 2, faces[i].height / 2), 0, 0, 360, cv::Scalar(255, 0, 255), 4, 8, 0);
cv::Mat faceROI = frame_gray(faces[i]);
std::vector<cv::Rect> eyes;
eyes_cascade.detectMultiScale(faceROI, eyes, 1.1, 2, 0 | cv::CASCADE_SCALE_IMAGE, cv::Size(30, 30));
for (size_t j = 0; j < eyes.size(); j++)
{
cv::Point eye_center(faces[i].x + eyes[j].x + eyes[j].width / 2, faces[i].y + eyes[j].y + eyes[j].height / 2);
int radius = cvRound((eyes[j].width = eyes[j].height) * 0.25);
cv::circle(main, eye_center, radius, cv::Scalar(255, 0, 0), 4, 8, 0);
}
}
return main;
}
catch (...)
{
std::cout << "Exception: have issue with processing face detection.";
return nuller;
}
Here is the main function:
int main()
{
CoreVision _core;
_core.init();
while (true)
{
cv::Mat img = _core.captureImage(-1);
img = _core.detectFace(img);
_core.displayImage(img, "parker");
}
}
And finally the .h file for the two functions:
#include <iostream>
#include <stdio.h>
#include <tchar.h>
#include <iostream>
#include <opencv2\opencv.hpp>
#include <opencv2\world.hpp>
#include "opencv2/objdetect/objdetect.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
class CoreVision
{
public:
CoreVision();
~CoreVision();
cv::Mat loadImage(const char* image_name, int type);
void saveImage(cv::Mat img, const char* filename);
cv::Mat captureImage(int cam);
cv::Mat detectFace(cv::Mat main);
int convertImage(cv::Mat img, int type);
void displayImage(cv::Mat, const char* window_name);
void init(void);
char cCurrentPath[FILENAME_MAX];
cv::String face_cascade_name;
cv::String eyes_cascade_name;
cv::CascadeClassifier face_cascade;
cv::CascadeClassifier eyes_cascade;
std::vector<cv::Rect> faces;
};
Thank you in advance for your help, I am not for sure if it is a driver issues, but I have tried it on multiple computers and the same freezing happens.
you don't seem to be storing the faces vector. You have it in public
std::vector faces; but where are you using it? eyes and faces are 2 different sets of data. Your also mixing two styles of code c style and c++ style opencv. You should stick with c++ since your using the latest opencv version.
faces.clear(); // under here
std::vector<Rect>faces; // see if this helps
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
void saveImage(cv::Mat img, const char* filename); //should be
void imwrite(img, const char* filename);// etc.
cv::Mat img = _core.captureImage(-1);
img = _core.detectFace(img);
_core.displayImage(img, "parker"); // should be
_core.displayImage(Mat frame);
this proably isn't going to work asuming core.captureImage(-1);
is your camera. also what does int convertImage(cv::Mat img, int type); do? are you sure it takes an int value?
c++ convention is
Videocapture cap(0);
Mat frame:
cap >> frame;
namedWindow("video");
imshow(video, frame); //etc
I had exactly the same issue. I figured out that the line containing eyes_cascade.detectMultiScale was the one causing my driver to crash. I have the Intel HD graphics 4400.
The code started working perfectly after I completely uninstalled the Intel HD graphics drivers.

imshow() function freezes

I am developing some video analytic algorithms using openCV. However, after I process a frame and want to display it on a window, it hangs at imshow() function. I have searched for this issue online but still cannot find the problem! Here is the code where I am using multithread and openCV:
void CFeatureExtraction::extract(){
boost::thread OpFlowThread, BGSThread;
while (m_nRun){
ImgPtr frame_ptr;
m_pQueue->wait_and_pop(frame_ptr);
Mat frame1, frame2;
(*frame_ptr).copyTo(frame1);
(*frame_ptr).copyTo(frame2);
OpFlowThread = boost::thread(&COpFlow::op_flow, m_pAlgo,frame1);
BGSThread = boost::thread(&CBGSub::bgsub, m_pBGS, frame2);
OpFlowThread.join();
BGSThread.join();
}
}
And inside op_flow and bgsub function, I am using imshow() and cvWaitKey() together! But it keeps hanging the whole program.
If my question is still not clear for you, pls feel free to ask me for more detail.
Here is the detail code of calling imshow():
CFarnebackAlgo::CFarnebackAlgo() : COpFlow()
{
namedWindow("Optical Flow");
}
CFarnebackAlgo::~CFarnebackAlgo()
{
destroyWindow("Optical Flow");
}
int CFarnebackAlgo::op_flow(Mat frame)
{
Mat flow;
cvtColor(frame, gray, COLOR_BGR2GRAY);
if (prevgray.data)
{
calcOpticalFlowFarneback(prevgray, gray, flow, 0.5, 3, 15, 3, 5, 1.2, 0);
for (int y = 0; y < frame.rows; y += 8)
for (int x = 0; x < frame.cols; x += 8){
const Point2f& fxy = flow.at<Point2f>(y, x);
line(frame, Point(x, y), Point(cvRound(x + fxy.x), cvRound(y + fxy.y)), Scalar(0, 255, 0));
circle(frame, Point(x, y), 2, Scalar(0, 255, 0), -1);
}
if (frame.data == NULL){
cout << "No Img!" << endl;
exit(0);
}
imshow("Optical Flow", frame);
waitKey(50);
}
std::swap(prevgray, gray);
return 0;
}
If I put the namedWindow() in the op_flow(), it is working. But if I put it in the constructor, it's not working and freezes. Anyone knows why?

cv::gpu::HoughLines is not working properly

I am trying to detect some lines using Hough Transform on a cv::gpu::GpuMat structure. I have tried using both gpu::HoughLines and gpu::HoughLinesP but even with extremely low thresholds, I am not getting any results at all. During debugging, I see that the container which should contain the results (houghLines) has only zeros stored inside it. The code I have written is given below,
static cv::Mat drawHoughLinesOnMat (cv::gpu::GpuMat hough_Mat, cv::gpu::GpuMat houghLines)
{
cv::Mat output_Mat;
cv::cvtColor(cv::Mat(hough_Mat), output_Mat, CV_GRAY2BGR);
std::vector<cv::Vec4i> lines_vector;
if (!houghLines.empty())
{
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat (1, houghLines.cols, CV_8UC3, &lines_vector[0]);
houghLines.download (temp_Mat);
}
for (size_t i=0; i<lines_vector.size(); ++i)
{
cv::Vec4i l = lines_vector[i];
cv::line(output_Mat, cv::Point(l[0], l[1]), cv::Point(l[2], l[3]), cv::Scalar(0, 0, 255), 1, 8);
}
return output_Mat;
}
int main()
{
cv::Mat input = cv::imread(INPUT_DATA_1->c_str(), CV_LOAD_IMAGE_GRAYSCALE);
std::string imageType = getImgType(input.type());
cv::gpu::GpuMat mat_input(input), bil_out, mat_thresh, hough_lines;
cv::gpu::HoughLinesBuf hough_buffer;
int bilateral_thresh = 15; // 5 == 0.085s; 15 == 0.467s at run-time
cv::gpu::bilateralFilter(mat_input, bil_out, bilateral_thresh, bilateral_thresh*2, bilateral_thresh/2);
//cv::gpu::threshold(bil_out, mat_thresh, 10, 255, CV_THRESH_BINARY);
cv::gpu::Canny(bil_out, mat_thresh, 10, 60, 5);
cv::gpu::HoughLinesP(mat_thresh, hough_lines, hough_buffer, 1.0f, (float)(CV_PI/180.0f), 5, 1);
//cv::Mat test_hough(hough_lines);
cv::Mat hough_Mat = drawHoughLinesOnMat(mat_input, hough_lines);
cv::gpu::HoughLines(mat_thresh, hough_lines, 1.0f, (float)(CV_PI/180.0f), 1, true);
/*cv::Mat */hough_Mat = drawHoughLinesOnMat(mat_input, hough_lines);
return EXIT_SUCCESS
}
The image I am using is,
Could someone tell me what it is that I am doing wrong..? Thanks in advance.!
The output of the Canny filter is,
EDIT:
I have tested on the CPU version of HoughLines and it seems to work just fine.
EDIT_2:
The solution posted by #jet47 works perfectly.
You use incorrect code for downloading results from GPU back to CPU:
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat (1, houghLines.cols, CV_8UC3, &lines_vector[0]);
houghLines.download (temp_Mat);
You use incorrect type for temp_Mat - CV_8UC3, it must be CV_32SC4.
The correct code is:
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat(1, houghLines.cols, CV_32SC4, &lines_vector[0]);
houghLines.download(temp_Mat);
My guess is that the Method you are using is outdated (but im not entirely sure).
This is how i would do it(as demonstrated in this Example Code):
//d_src filled with your image somewhere
GpuMat d_lines;
{
Ptr<cuda::HoughSegmentDetector> hough = cuda::createHoughSegmentDetector(1.0f, (float) (CV_PI / 180.0f), 50, 5);
hough->detect(d_src, d_lines);
}
vector<Vec4i> lines_gpu;
if (!d_lines.empty())
{
lines_gpu.resize(d_lines.cols);
Mat h_lines(1, d_lines.cols, CV_32SC4, &lines_gpu[0]);
d_lines.download(h_lines);
}
for (size_t i = 0; i < lines_gpu.size(); ++i)
{
Vec4i l = lines_gpu[i];
line(dst_gpu, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0, 0, 255), 3, LINE_AA);
}
EDIT The above uses the OpenCv 3.0 Interface