As part of a bigger project, I need to extract the extreme bottom corners of a quadrangle. I have an image and a corresponding binary Mat with 1s where the image is white (the image) and 0 where black (the background).
I've found ways to find the extreme left, right, bottom and top points but they may not give the points I want as the quadrangles are not perfectly rectangular.
https://www.pyimagesearch.com/2016/04/11/finding-extreme-points-in-contours-with-opencv/
Finding Top Left and Bottom Right Points (C++)
Finding extreme points in contours with OpenCV C++
The only way I can think of doing it is not very good. I'm hoping you guys can think of a better way than to just cycle though the matrix for the most bottom row then most left point and then keep points within a certain radius from that most bottom and left point.
And the same for the right, but this is not very computationally efficient.
This is an example quadruple and the corners of interest.
The ideal output is two Mats, similar to the original one, that have 1s only in the region of interest and 0s everywhere.
Any and all help will be greatly appreciated!!
There are at least two possible approaches. Both assume that you've extracted the corners:
Use approxPolyDPfunction to approximate the contour and get 4 vertices of a quadrangle.
2.Fit rectangle to the contour and then find nearest points in your contour to the bottom vertices of this rectangle.
// bin - your binarized image
std::vector<std::vector<cv::Point2i>> contours;
cv::findContours(bin, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);
int biggestContourIdx = -1;
double biggestContourArea = 0;
for (int i = 0; i < contours.size(); ++i)
{
auto area = cv::contourArea(contours[i]);
if (area > biggestContourArea)
{
biggestContourArea = area;
biggestContourIdx = i;
}
}
//first solution:
std::vector<cv::Point2i> approx;
cv::approxPolyDP(contours[biggestContourIdx], approx, 30, true);
auto mean = cv::mean(approx);
std::vector<cv::Point2i> bottomCorners;
for (auto p : approx)
{
if (p.y > mean[1]) bottomCorners.push_back(p);
}
//second solution:
auto rect = cv::minAreaRect(cv::Mat(contours[biggestContourIdx]));
auto center = rect.center;
Point2f rect_points[4];
rect.points(rect_points);
std::vector<cv::Point2i> bottomRectCorners;
std::vector<double> distances(2, std::numeric_limits<double>::max());
for (int i = 0; i < 4; ++i)
{
if (rect_points[i].y > center.y)
bottomRectCorners.push_back(rect_points[i]);
}
bottomCorners.clear();
bottomCorners.resize(2);
for (auto p : contours[biggestContourIdx])
{
for (int i = 0; i < distances.size(); ++i)
{
auto dist = cv::norm(p - bottomRectCorners[i]);
if (dist < distances[i])
{
distances[i] = dist;
bottomCorners[i] = p;
}
}
}
Results of both approaches: red - first method, green second
I am re asking this question with a better explanation.
I have obtained the four points of a rectangle that I wish to perform a warp perspective transformation on. I have successfully achieved a transform on one of my images as I manually selected the points and assigned their location for the transform.
cv::Point2f src_vertices[4];
src_vertices[0] = corners[3];
src_vertices[1] = corners[1];
src_vertices[2] = corners[0];
src_vertices[3] = corners[2];
Point2f dst_vertices[4];
dst_vertices[0] = Point(0, 0);
dst_vertices[1] = Point(box.boundingRect().width-1, 0);
dst_vertices[2] = Point(0, box.boundingRect().height-1);
dst_vertices[3] = Point(box.boundingRect().width-1, box.boundingRect().height-1);
As you can see I assigned the corners to the transformation vertices manually.
As I wish to use this code on multiple slightly different images I would like a more accurate way of assigning the corners.
All the images will look similar to this:
http://imgur.com/t0cgTzr
The corners for this image are as follows:
Top left - (1106, 331),
Top right- (810, 747),
Bottom left- (825, 187),
Bottom right- (510, 537)
I have found the max and min x and y values as follows:
float X, Y;
float maxX= 0;
float minX = 10000;
float maxY= 0;
float minY = 10000;
for(int i=0; i< 4; i++){
if( corners[i].x > maxX){
maxX = corners[i].x;
}
if( corners[i].x < minX){
minX = corners[i].x;
}
if( corners[i].y > maxY){
maxY = corners[i].y;
}
if( corners[i].y < minY){
minY = corners[i].y;
}
}
This gives me:
maxX - 1106, minX - 510, maxY - 747, minY - 187.
I would like to know how to recombine the max and min values to their respective values so I can use the points to perform the transform. I am quite new to opencv so sorry if this is very obvious.
What you can do to get the corners without losing information on the other coordinate is to find the whole point. Instead of storing just float maxX= 0; you can store Point2f maxX(0,0); and modify you for loop to keep track of the point itself, not only the x coordinate.
Better yet, get rid of those loops and make use of STL
Point2f maxX =
*std::max_element(corners, corners+4, [](Point2f a, Point2f b){return a.x < b.x;});
given a shapes orignal centroid + vertices .. i.e. if its a triangle, i know all three vertices coords. How could i then create a scaling function with a scaling factor as a parameter as below.. however my current code is with error and the result are huge shapes, much more than what im scaling by (only want scale factor of 2).
void Shape::scale(double factor)
{
int x, y, xx, xy;
int disx, disy;
for (itr = vertices.begin(); itr != vertices.end(); ++itr) {
//translate obj to origin (0,0)
x = itr->getX() - centroid.getX();
y = itr->getY() - centroid.getY();
//finds distance between centroid and vertex
disx = x + itr->getX();
disy = y + itr->getY();
xx = disx * factor;
xy = disy * factor;
//translate obj back
xx = xx + centroid.getX();
xy = xy + centroid.getY();
//set new coord
itr->setX(xx);
itr->setY(xy);
}
}
I know of using iterations to run through the vertices, my main point of confusion is how can i do the maths between the factor to scale my shapes size?
this is how i declare and itialise a vertex
// could i possible do (scale*x,scale*y)? or would that be problematic..
vertices.push_back(Vertex(x, y));
Also.. the grid is i.e. 100x100. if a scaled shape was to be too big to fit into that grid, i want an exit from the scale function so that the shape wont be enlarged, how can this be done effectively? so far i have a for look but that just loops on vertices, so it will only stop those that would be outside the grid, instead of cancelling the entire shape which would be ideal
if my question is too broad, please ask and i shall edit further to standard
First thing you need to do is find the center of mass of your set of points. That is the arithmetic mean of the coordinates of your points. Then, for each point calculate the line between the center of mass and that point. Now the only thing left is to put the point on that line, but in factor * current_distance away, where current_distance is the distance from the mass center to the given point before rescaling.
void Shape::scale(double factor)
{
Vertex mass_center = Vertex(0., 0.);
for(int i = 0; i < vertices.size(); i++)
{
mass_center.x += vertices[i].x;
mass_center.y += vertices[i].y;
}
mass_center.x /= vertices.size();
mass_center.y /= vertices.size();
for(int i = 0; i < vertices.size(); i++)
{
//this is a vector that leads from mass center to current vertex
Vertex vec = Vertex(vertices[i].x - mass_center.x, vertices[i].y - mass_center.y);
vertices[i].x = mass_center.x + factor * vec.x;
vertices[i].y = mass_center.y + factor * vec.y;
}
}
If you already know the centroid of a shape and the vertexes are the distance from that point then scaling in rectangular coordinates is just multiplying the x and y components of each vertex by the appropriate scaling factor (with a negative value flipping the shape around the axis.
void Shape::scale(double x_factor, double y_factor){
for(auto i=0; i < verticies.size();++i){
verticies[i].x *= x_scale;
verticies[i].y *= y_scale;
}
}
You could then just overload this function with one that takes a single parameter and calls this function with the same value for x and y.
void Shape::scale(double factor){
Shape::scale(factor, factor);
}
If you're vertex values are not centered at the origin then you will also have to multiply those values by your scaling factor.
I was wandering what the best approach would be for detecting 'figures' in an array of 2D points.
In this example I have two 'templates'. Figure 1 is a template and figure 2 is a template.
Each of these templates exists only as a vector of points with an x,y coordinate.
Let's say we have a third vector with points with x,y coordinate
What would be the best way to find out and isolate points matching one of the first two arrays in the third one. (including scaling, rotation)?
I have been trying nearest neigbours(FlannBasedMatcehr) or even SVM implementation but it doesn't seem to get me any result, template matching doesn't seem to be the way to go either, I think. I am not working on images but only on 2D points in memory...
Especially because the input vector always has more points than the original data set to be compared with.
All it needs to do is find points in array that match a template.
I am not a 'specialist' in machine learning or opencv. I guess I am overlooking something from the beginning...
Thank you very much for your help/suggestions.
just for fun I tried this:
Choose two points of the point dataset and compute the transformation mapping the first two pattern points to those points.
Test whether all transformed pattern points can be found in the data set.
This approach is very naive and has a complexity of O(m*n²) with n data points and a single pattern of size m (points). This complexity might be increased for some nearest neighbor search methods. So you have to consider whether it's not efficient enough for your appplication.
Some improvements could include some heuristic to not choose all n² combinations of points but, but you need background information of maximal pattern scaling or something like that.
For evaluation I first created a pattern:
Then I create random points and add the pattern somewhere within (scaled, rotated and translated):
After some computation this method recognizes the pattern. The red line shows the chosen points for transformation computation.
Here's the code:
// draw a set of points on a given destination image
void drawPoints(cv::Mat & image, std::vector<cv::Point2f> points, cv::Scalar color = cv::Scalar(255,255,255), float size=10)
{
for(unsigned int i=0; i<points.size(); ++i)
{
cv::circle(image, points[i], 0, color, size);
}
}
// assumes a 2x3 (affine) transformation (CV_32FC1). does not change the input points
std::vector<cv::Point2f> applyTransformation(std::vector<cv::Point2f> points, cv::Mat transformation)
{
for(unsigned int i=0; i<points.size(); ++i)
{
const cv::Point2f tmp = points[i];
points[i].x = tmp.x * transformation.at<float>(0,0) + tmp.y * transformation.at<float>(0,1) + transformation.at<float>(0,2) ;
points[i].y = tmp.x * transformation.at<float>(1,0) + tmp.y * transformation.at<float>(1,1) + transformation.at<float>(1,2) ;
}
return points;
}
const float PI = 3.14159265359;
// similarity transformation uses same scaling along both axes, rotation and a translation part
cv::Mat composeSimilarityTransformation(float s, float r, float tx, float ty)
{
cv::Mat transformation = cv::Mat::zeros(2,3,CV_32FC1);
// compute rotation matrix and scale entries
float rRad = PI*r/180.0f;
transformation.at<float>(0,0) = s*cosf(rRad);
transformation.at<float>(0,1) = s*sinf(rRad);
transformation.at<float>(1,0) = -s*sinf(rRad);
transformation.at<float>(1,1) = s*cosf(rRad);
// translation
transformation.at<float>(0,2) = tx;
transformation.at<float>(1,2) = ty;
return transformation;
}
// create random points
std::vector<cv::Point2f> createPointSet(cv::Size2i imageSize, std::vector<cv::Point2f> pointPattern, unsigned int nRandomDots = 50)
{
// subtract center of gravity to allow more intuitive rotation
cv::Point2f centerOfGravity(0,0);
for(unsigned int i=0; i<pointPattern.size(); ++i)
{
centerOfGravity.x += pointPattern[i].x;
centerOfGravity.y += pointPattern[i].y;
}
centerOfGravity.x /= (float)pointPattern.size();
centerOfGravity.y /= (float)pointPattern.size();
pointPattern = applyTransformation(pointPattern, composeSimilarityTransformation(1,0,-centerOfGravity.x, -centerOfGravity.y));
// create random points
//unsigned int nRandomDots = 0;
std::vector<cv::Point2f> pointset;
srand (time(NULL));
for(unsigned int i =0; i<nRandomDots; ++i)
{
pointset.push_back( cv::Point2f(rand()%imageSize.width, rand()%imageSize.height) );
}
cv::Mat image = cv::Mat::ones(imageSize,CV_8UC3);
image = cv::Scalar(255,255,255);
drawPoints(image, pointset, cv::Scalar(0,0,0));
cv::namedWindow("pointset"); cv::imshow("pointset", image);
// add point pattern to a random location
float scaleFactor = rand()%30 + 10.0f;
float translationX = rand()%(imageSize.width/2)+ imageSize.width/4;
float translationY = rand()%(imageSize.height/2)+ imageSize.height/4;
float rotationAngle = rand()%360;
std::cout << "s: " << scaleFactor << " r: " << rotationAngle << " t: " << translationX << "/" << translationY << std::endl;
std::vector<cv::Point2f> transformedPattern = applyTransformation(pointPattern,composeSimilarityTransformation(scaleFactor,rotationAngle,translationX,translationY));
//std::vector<cv::Point2f> transformedPattern = applyTransformation(pointPattern,trans);
drawPoints(image, transformedPattern, cv::Scalar(0,0,0));
drawPoints(image, transformedPattern, cv::Scalar(0,255,0),3);
cv::imwrite("dataPoints.png", image);
cv::namedWindow("pointset + pattern"); cv::imshow("pointset + pattern", image);
for(unsigned int i=0; i<transformedPattern.size(); ++i)
pointset.push_back(transformedPattern[i]);
return pointset;
}
void programDetectPointPattern()
{
cv::Size2i imageSize(640,480);
// create a point pattern, this can be in any scale and any relative location
std::vector<cv::Point2f> pointPattern;
pointPattern.push_back(cv::Point2f(0,0));
pointPattern.push_back(cv::Point2f(2,0));
pointPattern.push_back(cv::Point2f(4,0));
pointPattern.push_back(cv::Point2f(1,2));
pointPattern.push_back(cv::Point2f(3,2));
pointPattern.push_back(cv::Point2f(2,4));
// transform the pattern so it can be drawn
cv::Mat trans = cv::Mat::ones(2,3,CV_32FC1);
trans.at<float>(0,0) = 20.0f; // scale x
trans.at<float>(1,1) = 20.0f; // scale y
trans.at<float>(0,2) = 20.0f; // translation x
trans.at<float>(1,2) = 20.0f; // translation y
// draw the pattern
cv::Mat drawnPattern = cv::Mat::ones(cv::Size2i(128,128),CV_8U);
drawnPattern *= 255;
drawPoints(drawnPattern,applyTransformation(pointPattern, trans), cv::Scalar(0),5);
// display and save pattern
cv::imwrite("patternToDetect.png", drawnPattern);
cv::namedWindow("pattern"); cv::imshow("pattern", drawnPattern);
// draw the points and the included pattern
std::vector<cv::Point2f> pointset = createPointSet(imageSize, pointPattern);
cv::Mat image = cv::Mat(imageSize, CV_8UC3);
image = cv::Scalar(255,255,255);
drawPoints(image,pointset, cv::Scalar(0,0,0));
// normally we would have to use some nearest neighbor distance computation, but to make it easier here,
// we create a small area around every point, which allows to test for point existence in a small neighborhood very efficiently (for small images)
// in the real application this "inlier" check should be performed by k-nearest neighbor search and threshold the distance,
// efficiently evaluated by a kd-tree
cv::Mat pointImage = cv::Mat::zeros(imageSize,CV_8U);
float maxDist = 3.0f; // how exact must the pattern be recognized, can there be some "noise" in the position of the data points?
drawPoints(pointImage, pointset, cv::Scalar(255),maxDist);
cv::namedWindow("pointImage"); cv::imshow("pointImage", pointImage);
// choose two points from the pattern (can be arbitrary so just take the first two)
cv::Point2f referencePoint1 = pointPattern[0];
cv::Point2f referencePoint2 = pointPattern[1];
cv::Point2f diff1; // difference vector
diff1.x = referencePoint2.x - referencePoint1.x;
diff1.y = referencePoint2.y - referencePoint1.y;
float referenceLength = sqrt(diff1.x*diff1.x + diff1.y*diff1.y);
diff1.x = diff1.x/referenceLength; diff1.y = diff1.y/referenceLength;
std::cout << "reference: " << std::endl;
std::cout << referencePoint1 << std::endl;
// now try to find the pattern
for(unsigned int j=0; j<pointset.size(); ++j)
{
cv::Point2f targetPoint1 = pointset[j];
for(unsigned int i=0; i<pointset.size(); ++i)
{
cv::Point2f targetPoint2 = pointset[i];
cv::Point2f diff2;
diff2.x = targetPoint2.x - targetPoint1.x;
diff2.y = targetPoint2.y - targetPoint1.y;
float targetLength = sqrt(diff2.x*diff2.x + diff2.y*diff2.y);
diff2.x = diff2.x/targetLength; diff2.y = diff2.y/targetLength;
// with nearest-neighborhood search this line will be similar or the maximal neighbor distance must be relative to targetLength!
if(targetLength < maxDist) continue;
// scale:
float s = targetLength/referenceLength;
// rotation:
float r = -180.0f/PI*(atan2(diff2.y,diff2.x) + atan2(diff1.y,diff1.x));
// scale and rotate the reference point to compute the translation needed
std::vector<cv::Point2f> origin;
origin.push_back(referencePoint1);
origin = applyTransformation(origin, composeSimilarityTransformation(s,r,0,0));
// compute the translation which maps the two reference points on the two target points
float tx = targetPoint1.x - origin[0].x;
float ty = targetPoint1.y - origin[0].y;
std::vector<cv::Point2f> transformedPattern = applyTransformation(pointPattern,composeSimilarityTransformation(s,r,tx,ty));
// now test if all transformed pattern points can be found in the dataset
bool found = true;
for(unsigned int i=0; i<transformedPattern.size(); ++i)
{
cv::Point2f curr = transformedPattern[i];
// here we check whether there is a point drawn in the image. If you have no image you will have to perform a nearest neighbor search.
// this can be done with a balanced kd-tree in O(log n) time
// building such a balanced kd-tree has to be done once for the whole dataset and needs O(n*(log n)) afair
if((curr.x >= 0)&&(curr.x <= pointImage.cols-1)&&(curr.y>=0)&&(curr.y <= pointImage.rows-1))
{
if(pointImage.at<unsigned char>(curr.y, curr.x) == 0) found = false;
// if working with kd-tree: if nearest neighbor distance > maxDist => found = false;
}
else found = false;
}
if(found)
{
std::cout << composeSimilarityTransformation(s,r,tx,ty) << std::endl;
cv::Mat currentIteration;
image.copyTo(currentIteration);
cv::circle(currentIteration,targetPoint1,5, cv::Scalar(255,0,0),1);
cv::circle(currentIteration,targetPoint2,5, cv::Scalar(255,0,255),1);
cv::line(currentIteration,targetPoint1,targetPoint2,cv::Scalar(0,0,255));
drawPoints(currentIteration, transformedPattern, cv::Scalar(0,0,255),4);
cv::imwrite("detectedPattern.png", currentIteration);
cv::namedWindow("iteration"); cv::imshow("iteration", currentIteration); cv::waitKey(-1);
}
}
}
}
I am trying to find the centroid of a contour but am having trouble implementing the example code in C++ (OpenCV 2.3.1). Can anyone help me out?
To find the centroid of a contour, you can use the method of moments. And functions are implemented OpenCV.
Check out these moments function (central and spatial moments).
Below code is taken from OpenCV 2.3 docs tutorial. Full code here.
/// Find contours
findContours( canny_output, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );
/// Get the moments
vector<Moments> mu(contours.size() );
for( int i = 0; i < contours.size(); i++ )
{ mu[i] = moments( contours[i], false ); }
/// Get the mass centers:
vector<Point2f> mc( contours.size() );
for( int i = 0; i < contours.size(); i++ )
{ mc[i] = Point2f( mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00 ); }
Also check out this SOF, although it is in Python, it would be useful. It finds all parameters of a contour.
If you have the mask of the contour area, you can find the centroid location as follows:
cv::Point computeCentroid(const cv::Mat &mask) {
cv::Moments m = moments(mask, true);
cv::Point center(m.m10/m.m00, m.m01/m.m00);
return center;
}
This approach is useful when one has the mask but not the contour. In that case the above method is computationally more efficient vs. using cv::findContours(...) and then finding mass center.
Here's the source
Given the contour points, and the formula from Wikipedia, the centroid can be efficiently computed like this:
template <typename T>
cv::Point_<T> computeCentroid(const std::vector<cv::Point_<T> >& in) {
if (in.size() > 2) {
T doubleArea = 0;
cv::Point_<T> p(0,0);
cv::Point_<T> p0 = in->back();
for (const cv::Point_<T>& p1 : in) {//C++11
T a = p0.x * p1.y - p0.y * p1.x; //cross product, (signed) double area of triangle of vertices (origin,p0,p1)
p += (p0 + p1) * a;
doubleArea += a;
p0 = p1;
}
if (doubleArea != 0)
return p * (1 / (3 * doubleArea) ); //Operator / does not exist for cv::Point
}
///If we get here,
///All points lies on one line, you can compute a fallback value,
///e.g. the average of the input vertices
[...]
}
Note:
This formula works with vertices given both in clockwise and
counterclockwise order.
If the points have integer coordinates, it
might be convenient to adapt the type of p and of the return value to Point2f or Point2d,
and to add a cast to float or double to the denominator in the return statement.
If all you need is an approximation of the centroid here are a couple of simple ways to do it:
sumX = 0; sumY = 0;
size = array_points.size;
if(size > 0){
foreach(point in array_points){
sumX += point.x;
sumY += point.y;
}
centroid.x = sumX/size;
centroid.y = sumY/size;
}
Or with the help of Opencv's boundingRect:
//pseudo-code:
Rect bRect = Imgproc.boundingRect(array_points);
centroid.x = bRect.x + (bRect.width / 2);
centroid.y = bRect.y + (bRect.height / 2);