OpenCV feature detectors - ROI mask for better performance? - c++

Question

In OpenCV it is possible to specify a region of interest via a mask as input for a feature detector algorithm. From my perspective, I would expect a huge performance gain, but a simple test with a small ROI cannot confirm that.
Is it reasonable to expect a better performance when using masks in OpenCV? Or is it necessary to trim the images?

Answer 1

Most likely the mask simply removes all keypoints outside the mask, so OpenCV has still to parse the entire image.
You can reduce the size of your image to improve the speed

Answer 2

I'm not sure if this is something you're looking for (esp since this is in Java), but check out this file, specifically the function at line 121.
Here it is for your convenience:
MatOfRect diceDetections = new MatOfRect(); // Essentially an array of locations where our dice features were detected. (Stupid wrappers)
// Note that detectMultiScale has thrown an unknown exception before (literally, unknown). This is to prevent crashing.
try {
diceCascade.detectMultiScale(image, diceDetections, 1.1, 4, 0, new Size(20, 20), new Size(38, 38));
} catch (Exception e) {
e.printStackTrace();
}
// Debug, used for console output
String curDetect = "";
// Iterates for every Dice ROI
for (int i = 0; i < diceDetections.toArray().length; i++) {
Rect diceRect = diceDetections.toArray()[i];
// Draws rectangles around our detected ROI
Point startingPoint = new Point(diceRect.x, diceRect.y);
Point endingPoint = new Point(diceRect.x + diceRect.width, diceRect.y + diceRect.height);
Imgproc.rectangle(image, startingPoint, endingPoint, new Scalar(255, 255, 0));
MatOfRect pipDetections = new MatOfRect();
try {
/*
* Now this is interesting. We essentially create a sub-array of the image, with our dice ROI as the image. Then we perform the detection on the image. This gives us the relative
* positions of our pip ROIs to the dice ROI. Later on, we can draw the circles around the pip ROI, with the centers' positions adjusted by adding the dice ROI positions, so that it
* renders properly. This is an amazing trick, as it not only eliminates false positives in non-dice ROIs, but it reduces how many pixels the classifier has to analyze to only at most
* 38 x 38 pixels (because of the size restraints provided while detecting dice ROIs). This means we can set the precision to an insane level, without performance loss.
*/
pipCascade.detectMultiScale(image.submat(diceRect), pipDetections, 1.01, 4, 0, new Size(2, 2), new Size(10, 10));
} catch (Exception e) {
e.printStackTrace();
}
// Gets the number of detected pips and draws a cricle around the ROI
int numPips = 0;
for (int y = 0; y < pipDetections.toArray().length; y++) {
Rect pipRect = pipDetections.toArray()[y]; // Provides the relative position of the pips to the dice ROI
/*
* Finds the absolute center of a pip. diceRect.x and diceRect.y provides the top-left position of the dice ROI. pipRect.x and pipRect.y provides the top-left position of the pip ROI.
* Normally, to find a center of an object with size (w, h) with the top-left point (x, y), we divide the width and height by two, and then add on the x pos to the width and y pos to
* the height. Now, since pipDetections only provide relative positioning to the dice ROI, we also need to add the dice position to find our absolute center position (aka relative to
* the entire image).
*/
Point center = new Point(diceRect.x + pipRect.x + pipRect.width / 2, diceRect.y + pipRect.y + pipRect.height / 2);
Imgproc.ellipse(image, center, new Size(pipRect.width / 2, pipRect.height / 2), 0, 0, 360, new Scalar(255, 0, 255), 1, 0, 0);
numPips++;
}
In a nutshell, I have two classifiers, one to recognize dice (line 129) and one to recognize the pips (black dots) on the dice. It gets an array of dice ROI, and then for each item in the array, take a submatrix of the image (located at the ROI), and have the pip classifier scan over that matrix instead of the whole image (line 156). However, if you're trying to display the detections (pips in my example), you'll need to offset it by the positions of the ROI that you're in, hence the work at line 171 and 172.
I'm certain that this achieves the same performance gain you're look for, just not necessarily in the same fashion (subimaging vs masking).