OpenCV find contours of close objects - c++

I am writing a C++ Application using the OpenCV library to detect objects in images. These images look like this:
http://fs1.directupload.net/images/150311/my6uczfn.png
The upper part of the image, which is black, can be ignored.
I know, that every pixel, which is not part of a desired object, will be colored in white. What I am trying to do is to find out how many objects of interest are on an image and where they are.
Up until now I wrote the following code:
Mat image = imread("2.png", CV_LOAD_IMAGE_COLOR);
if(!image.data)
{
std::cout << "Could not open or find the image." << std::endl;
}
Range range_rows(0, image.size().height);
Range range_columns_left(0, image.size().width);
Range range_columns_middle(image.size().width, image.size().width * 2);
Range range_columns_right(image.size().width * 2, image.size().width * 3);
Mat display_mat(image.size().height, image.size().width * 3, CV_8UC3);
Mat left(display_mat, range_rows, range_columns_left);
image.copyTo(left);
Mat classified_image;
threshold(image, classified_image, 254, 255, THRESH_BINARY);
Mat middle(display_mat, range_rows, range_columns_middle);
classified_image.copyTo(middle);
Mat cimage = Mat::zeros(image.size(), CV_8UC3);
Mat classified_grayscale_image;
cvtColor(classified_image, classified_grayscale_image, CV_RGB2GRAY);
std::vector< std::vector<cv::Point> > contours;
findContours(classified_grayscale_image, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
for(size_t counter = 0; counter < contours.size(); counter++)
{
std::cout << "Contours size: " << contours[counter].size() << std::endl;
if(contours[counter].size() < 6)
continue;
Mat pointsf;
Mat(contours[counter]).convertTo(pointsf, CV_32F);
RotatedRect box = fitEllipse(pointsf);
drawContours(cimage, contours, (int)counter, Scalar::all(255), 1, 8);
ellipse(cimage, box, Scalar(0,0,255), 1, CV_AA);
std::cout << "Ellipse Parameter:\t";
ellipse(cimage, box.center, box.size*0.5f, box.angle, 0, 360, Scalar(0,255,255), 1, CV_AA);
Point2f vtx[4];
box.points(vtx);
for( int j = 0; j < 4; j++ )
line(cimage, vtx[j], vtx[(j+1)%4], Scalar(0,255,0), 1, CV_AA);
}
Mat right(display_mat, range_rows, range_columns_right);
cimage.copyTo(right);
namedWindow("Results", CV_WINDOW_AUTOSIZE);
imshow("Results", display_mat);
waitKey(0);
return 0;
The result looks like this:
http://fs1.directupload.net/images/150311/toiy3aes.png
As you see, the classification, what is an object and what is not, is not perfect, so 2 objects are recognized as one. The classification will be improved, but something like this can happen, if those objects are very close. Even more of a problem is, when they are touching each other.
How can I do a proper object recognition in the case shown above? Any ideas?

You have got few options:
use some filtering/thresholding method on your result image to split objects from each other. Otsu binarization should be enough in this case, alternatively you can try to use dilate operation.
inverse your result image and than use distance transform and Otsu binarization (or some other kind o thresholding - most of them should work fine). It will make you objects smaller but will make counting them much easier.
if you need to mark the objects as precise as possible you need to use more complicated method. Here there is an tutorial which use techniques i've described above and connected components and watershed.

Related

Detect the coin and from there measure the leaf in the image

Friends, I am trying to detect the coin first, since I already know its size is 1.011cm2. And then measure the leaves in the image.
I am using findContours, but I am not always able to distinguish the currency first, I have also tried to use hougCircles but it is not working in my case. Would anyone have any ideas?
OpenCv 4.5.0 C++
My code
//variables for segmentation image
cv::Mat imagem_original, imagem_gray, imagem_binaria, imagem_inRange, imagem_threshold, dst, src;
vector<Vec3f> circles;
cv::Scalar min_color = Scalar(50, 50, 50);
cv::Scalar max_color = Scalar(90, 120, 180);
imagem_original = load_image("IMG_1845.jpg");
//imshow("Imagem Original", imagem_original);
cv::cvtColor(imagem_original, imagem_gray, COLOR_BGR2GRAY);
//imshow("imagem_gray", imagem_gray);
//cv::inRange(imagem_gray, min_color, max_color, imagem_inRange);
cv::threshold(imagem_gray, imagem_threshold, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
imshow(" Threshold", imagem_threshold);
// find outer-contours in the image these should be the circles!
cv::Mat conts = imagem_threshold.clone();
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(conts, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
int total_IAF = 0;
cout << "\n\n";
cout << contours.size() << "\n\n";
for (int i = 0; i < contours.size(); i++) {
int area = contourArea(contours[i]);
if (area <= 10) {
cv::drawContours(imagem_original, contours, i, Scalar(0, 0, 255));
}
else {
cout << area << "\n";
cv::drawContours(imagem_original, contours, i, Scalar(255, 0, 0));
}
if (area > 5000) {
total_IAF += contourArea(contours[i]);
}
}
imshow(" ORIGINAL ", imagem_original);
double iAF_cm2 = total_IAF / 4658;
cout << "\n\n TOTAL AREA IAF: " << total_IAF;
cout << "\n IAF em cm2: " << iAF_cm2 << " cm2\n\n";
If your setup has constant white-ish/gray-ish background and green leaves, I'd use the HSV color space to detect all objects using the S channel (the green leaves and the golden part of the coin will have significantly more saturation than the background) and then distinguish between the coin and the leaves using the H channel (the green leaves will have hue values around 45). The remainder is to determine the image areas of all contours, and set the coin's image area as some kind of reference area to calculate the object areas w.r.t. the coin's object area of 1.011.
That's the saturation channel of the given image:
The saturation channel thresholded at 64:
That's the hue channel of the image:
Here's some code executing the above idea:
int main()
{
// Read image
cv::Mat img = cv::imread("Wcj1R.jpg", cv::IMREAD_COLOR);
// Convert image to HSV color space, and split H, S, V channels
cv::Mat img_hsv;
cv::cvtColor(img, img_hsv, cv::COLOR_BGR2HSV);
std::vector<cv::Mat> hsv;
cv::split(img_hsv, hsv);
// Binary threshold S channel at fixed threshold
cv::Mat img_thr;
cv::threshold(hsv[1], img_thr, 64, 255, cv::THRESH_BINARY);
// Find most outer contours only
std::vector<std::vector<cv::Point>> cnts;
std::vector<cv::Vec4i> hier;
cv::findContours(img_thr.clone(), cnts, hier, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_NONE);
// Iterate found contours
std::vector<cv::Point> cnt_centers;
std::vector<double> cnt_areas;
double ref_area = -1;
for (int i = 0; i < cnts.size(); i++)
{
// Current contour
std::vector<cv::Point> cnt = cnts[i];
// If contour is too small, discard
if (cnt.size() < 100)
continue;
// Calculate and store center (just for visualization) and area of contour
cv::Moments m = cv::moments(cnt);
cnt_centers.push_back(cv::Point(m.m10 / m.m00 - 30, m.m01 / m.m00));
cnt_areas.push_back(cv::contourArea(cnt));
// Check H channel, whether the contour's image parts are mostly green
cv::Mat mask = hsv[0].clone().setTo(cv::Scalar(0));
cv::drawContours(mask, cnts, i, cv::Scalar(255), cv::FILLED);
double h_mean = cv::mean(hsv[0], mask)[0];
// If it's not mostly green, that's the coin, thus the reference area
if (h_mean < 40 || h_mean > 50)
ref_area = cv::contourArea(cnt);
}
// Iterate all contours again
for (int i = 0; i < cnt_centers.size(); i++)
{
// Calculate actual object area
double area = cnt_areas[i] / ref_area * 1.011;
// Put area on image w.r.t. the contour's center
cv::putText(img, std::to_string(area), cnt_centers[i], cv::FONT_HERSHEY_COMPLEX_SMALL, 1, cv::Scalar(255, 255, 255));
}
return 0;
}
And, that'd be the output:
Your code finds all contours in a image and shows them. So I'm confused about the meaning of "detect the coin first".
If you want to draw the contour of the coin first, sort contours vector by size. The coin is the smallest object so it would be the first element of the vector after sorting.(Of course, some unwanted contours should removed before sorting.)

Get rectangular contours from image

How to extract all rectangular contours from an image with a black background?
I want to get the black rectangle in the bottom (with some white text inside)
In the code I try to fetch all contours via cv::RETR_EXTERNAL with 4 corners, but it only fetches the whole image as one big contour?? What am I doing wrong?
std::vector<std::vector<cv::Point>> contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(img, contours, hierarchy, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);
cv::cvtColor(img, img, cv::COLOR_GRAY2BGR);
for(int i = 0; i < contours.size(); i++){
std::cout << "contour found" << std::endl;
std::vector<cv::Point> approx;
cv::approxPolyDP(contours[i], approx, cv::arcLength(contours[i], true) * 0.02, true);
if(!cv::isContourConvex(approx)){
continue;
}
if(approx.size() == 4){
cv::rectangle(img, cv::boundingRect(contours[i]), cv::Scalar(0, 255, 0), 2);
}
}
cv::imwrite("img.png", img);
Found out I need to invert the image before I can use cv::RETR_EXTERNAL
img_inv = 255 - img;
You are using the wrong flag:
cv::RETR_EXTERNAL : If you use this flag, it returns only extreme outer flags. All child contours are left behind
https://docs.opencv.org/3.4.3/d9/d8b/tutorial_py_contours_hierarchy.html
You should use cv::RETR_LIST,because you need all the contours and you do not need the hierarchy.

Get coordinates of contours in OpenCV

Let's say that I have the following output image:
Basically, I have video stream and I want to get coordinates of rectangle only in the output image. Here's my C++ code:
while(1)
{
capture >> frame;
if(frame.empty())
break;
cv::cvtColor( frame, gray, CV_BGR2GRAY ); // Grayscale image
Canny( gray, canny_output, thresh, thresh * 2, 3 );
// Find contours
findContours( canny_output, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );
// Draw contours
Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );
for( int i = 0; i< contours.size(); i++ )
{
Scalar color = Scalar( rng.uniform(0, 255), rng.uniform(0,255), rng.uniform(0,255) );
drawContours( drawing, contours, i, color, 2, 8, hierarchy, 0, Point() );
}
cv::imshow( "w", drawing );
waitKey(20); // waits to display frame
}
Thanks.
Look at the definition of the find contours function in the opencv documentation and see the parameters (link):
void findContours(InputOutputArray image, OutputArrayOfArrays contours, OutputArray hierarchy, int mode, int method, Point offset=Point())
Parameters: here
Look at contours, like Rafa said each contour is stored in a vector of points and each vector of points is stored in a vector, so, by walking in the outer vector and then walking in the inner vector you'll be finding the points you wish.
However, if you want to detect only the bigger contour you might want to use CV_RETR_EXTERNAL as the mode parameter, because it'll detect only most external contour (the big rectangle).
If you still wish to maintain the smaller contours then you might use the CV_RETR_TREE and work out with the hierarchy structure: Using hierarchy contours
Looking at the documentation, the OutputArrayOfArrays contours is the key.
contours – Detected contours. Each contour is stored as a vector of points.
so, you've got a vector< vector<Point> > contours. The vector<Point>(inside) is the coordinates of a contour, and every contour is stored in a vector.
So for instance, to know the 5-th vector, it's vector<Point> fifthcontour = contours.at(4);
and you have the coordinates in that vector.
You can access to those coordinates as:
for (int i = 0; i < fifthcontour.size(); i++) {
Point coordinate_i_ofcontour = fifthcontour[i];
cout << endl << "contour with coordinates: x = " << coordinate_i_ofcontour.x << " y = " << coordinate_i_ofcontour.y;
}

Derivatives in OpenCV

I'm writing a program using opencv that does text detection and extraction.
Im using the Sobel derivative in order to do edge detection and have gotten the following result:
But I wish to get the following result:
(I appologize for the blurry image.)
The problem I'm having is the "blank areas" inside the edges "confuse" the algorithem I'm using so when the algorithem detects the "blank part" seperating between two lines from the lines themselves it gets confused and start running into the letter themselves instead of keepeing between two lines. This error, I believe would be solves by achieving the second result.
Anyone knows what changes i need to make? in the soble derivative? maybe use a different derivative?
Code:
Mat ProfileSeamTextLineExtractor::computeDerivative(){
Mat img = _image;
Mat gradiant_mat;
int scale = 2;
int delta = 0;
int ddepth = CV_16S;
GaussianBlur(img, img, Size(3, 3), 0, 0, BORDER_DEFAULT);
Mat grad_x, grad_y;
Mat abs_grad_x, abs_grad_y;
Sobel(img, grad_x, ddepth, 1, 0, 3, scale, delta, BORDER_DEFAULT);
convertScaleAbs(grad_x, abs_grad_x);
Sobel(img, grad_y, ddepth, 0, 1, 3, scale, delta, BORDER_DEFAULT);
convertScaleAbs(grad_y, abs_grad_y);
/// Total Gradient (approximate)
addWeighted(abs_grad_x, 0.5, abs_grad_y, 0.5, 0, gradiant_mat);
return gradiant_mat;
}
Regards,
Try using the second sobel derivative, add, normalize (this may do the same as addWeighted), and then thresholding optimally. I had results similar to yours with different threshold values.
Here's an example:
cv::Mat result;
cvtColor(image, gray, CV_BGR2GRAY);
cv::medianBlur(gray, gray, 3);
cv::Mat sobel_x, sobel_y, result;
cv::Sobel(gray, sobel_x, CV_32FC1, 2, 0, 5);
cv::Sobel(gray, sobel_y, CV_32FC1, 0, 2, 5);
cv::Mat sum = sobel_x + sobel_y;
cv::normalize(sum, result, 0, 255, CV_MINMAX, CV_8UC1);
//Determine optimal threshold value using THRESH_OTSU.
// This didn't give me optimal results, but was a good starting point.
cv::Mat temp, final;
double threshold = cv::threshold(result, temp, 0, 255, CV_THRESH_BINARY+CV_THRESH_OTSU);
cv::threshold(result, final, threshold*.9, 255, CV_THRESH_BINARY);
I was able to clearly extract both light text on a dark background, and dark text on a light background.
If you need the final image to consistently be white background with black text, you can do this:
cv::Scalar avgPixelIntensity = cv::mean( final );
if(avgPixelIntensity[0] < 127.0)
cv::bitwise_not(final, final);
I tried a lot of different text extraction methods and couldn't find any that worked across the board, but this seems to. This took a lot of trial and error to figure out, so I hope this helps.
I don't really understand what your final aim is. Do you eventually want a nice filled in version of the text so you can recognise the characters? I can give that a shot if that's what you are looking for.
This is what I did while trying to remove inner holes:
For this one I didn't bother:
It fails at the edges where the text is cut off.
Obviously, I had to work with the image that had already gone through some processing. I might be able to give you more help if I had the original and produce a better output. You might not even need to use derivatives at all if the background is clean enough.
Here is the code:
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
using namespace cv;
using namespace std;
void printInnerContours (int contourPos, Mat &filled, vector<vector<Point2i > > &contours, vector<Vec4i> &hierarchy, int area);
int main() {
int areaThresh;
vector<vector<Point2i > > contours;
vector<Vec4i> hierarchy;
Mat text = imread ("../wHWHA.jpg", 0); //write greyscale
threshold (text, text, 50, 255, THRESH_BINARY);
imwrite ("../text1.jpg", text);
areaThresh = (0.01 * text.rows * text.cols) / 100;
findContours (text, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);
Mat filled = Mat::zeros(text.rows, text.cols, CV_8U);
cout << contours.size() << endl;
for (int i = 0; i < contours.size(); i++) {
int area = contourArea(contours[i]);
if (area > areaThresh) {
if ((hierarchy[i][2] != -1) && (hierarchy[i][3] == -1)) {
drawContours (filled, contours, i, 255, -1);
if (hierarchy[i][2] != -1) {
printInnerContours (hierarchy[i][2], filled, contours, hierarchy, area);
}
}
}
}
imwrite("../output.jpg", filled);
return 0;
}
void printInnerContours (int contourPos, Mat &filled, vector<vector<Point2i > > &contours, vector<Vec4i> &hierarchy, int area) {
int areaFrac = 5;
if (((contourArea (contours[contourPos]) * 100) / area) < areaFrac) {
//drawContours (filled, contours, contourPos, 0, -1);
}
if (hierarchy[contourPos][2] != -1) {
printInnerContours (hierarchy[contourPos][2], filled, contours, hierarchy, area);
}
if (hierarchy[contourPos][0] != -1) {
printInnerContours (hierarchy[contourPos][0], filled, contours, hierarchy, area);
}
}

OpenCV contourArea() not working

I'm pretty new to OpenCV and having a slight problem which is probably something very easy to fix.
Basically im doing some basic image processing, I'm trying to find contours which have an contourArea() of < 3000.
The problem is, I'm getting the following error when trying to draw contours and/or call contourArea() function:
The error is occuring on the cv:contourArea() line, the error message is:
OpenCV Error: Assertion failed (contour.checkVector(2) >= 0 && (contour.depth() == CV_32F || contour.depth() == CV_32S)) in cv::contourArea,
file ..\..\..\..\opencv\modules\imgproc\src\contours.cpp, line 1904
Any help is greatly appreciated. The code is below:
using namespace cv;
cv::Mat greyMat, binaryMat, newMat;
cv::Mat image = cv::imread("image.png", 1);
// First convert image to gray scale
cv::cvtColor(image, greyMat, CV_BGR2GRAY);
cv::adaptiveThreshold(greyMat, binaryMat, 255, cv::ADAPTIVE_THRESH_GAUSSIAN_C, cv::THRESH_BINARY_INV, 45, 0);
erode(binaryMat, binaryMat, getStructuringElement(MORPH_ELLIPSE, Size(2, 2)));
dilate(binaryMat, binaryMat, getStructuringElement(MORPH_ELLIPSE, Size(1, 1)));
// Remove unclosed curves (the circled hashtag)
cv::copyMakeBorder(binaryMat, newMat, 1, 1, 1, 1, cv::BORDER_CONSTANT, 0);
cv::floodFill(newMat, cv::Point(0, 0), 255);
newMat = 255 - newMat;
cv::Mat cMat;
newMat.copyTo(cMat);
std::vector<std::vector<cv::Point>> contours;
cv::findContours(cMat, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_NONE);
std::cout << "Found: " << contours.size() << " " << contours[0][0] << std::endl;
for (size_t i = 0; i < contours.size(); i++)
{
if (cv::contourArea(contours[i]) < 3000)
{
cv::drawContours(newMat, contours, i, 255, -1);
}
}
cv::imshow("Debug", newMat);
cv::waitKey(0);
return 0;
Not sure, but from what I read in the error message, the function expects a floating point value, and you give him vector of vector of Point.
According to the current manual, this type is an integer point, so maybe this is the problem.