I'm currently working on a project which reads an image, applies a number of filters, with the purpose of being able to place a bounding rect around regions of interest.
I have an image of handwritten text on lined paper as my input:
string imageLocation = "loctation of image file";
src = imread(imageLocation, 1);
I then convert the image to gray scale and apply adaptive thresholding:
cvtColor(src, newsrc, CV_BGR2GRAY);
adaptiveThreshold(~newsrc, dst, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);
I then use morphological operations to attempt to eliminate the horizontal lines from the image:
Mat horizontal = dst.clone();
int horizontalSize = dst.cols / 30;
Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalSize,1));
erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));
cv::resize(horizontal, horizontal, cv::Size(), 0.5, 0.5, CV_INTER_CUBIC);
imshow("horizontal", horizontal);
Which produces the following (so far so good):
I then try to use the same erode & dilate methods to figure out the vertical:
int verticalsize = dst.rows / 30;
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));
erode(vertical, vertical, verticalStructure, Point(-1, -1));
dilate(vertical, vertical, verticalStructure, Point(-1, -1));
cv::resize(vertical, vertical, cv::Size(), 0.5, 0.5, CV_INTER_CUBIC);
imshow("vertical", vertical);
I'm following OpenCV's example, which can be found here
But, the output i'm getting for the vertical is:
My question is, how would I go about removing these horizontal lines from the image.
Sorry for the lengthy question (I wanted to explain as much as I could) and thanks in advance for any advice.
You can try to make this work in frequency domain like here:
http://lifeandprejudice.blogspot.ru/2012/07/activity-6-enhancement-in-frequency_25.html
http://www.fmwconcepts.com/imagemagick/fourier_transforms/fourier.html
Working with FFT is very effective in adding/removing regilar grids from image.
Related
I've computed the following mask of an image containing corn:
the problem is, now i have black parts inside the seeds. Is there a way to get rid of them while making sure that seeds remain disconnected from each other?
My ultimate goal is to count the seeds on the picture, using watershed algorithm. I observed when the seeds are touching each other, it introduces a imprecision to the algorithm, so I tried to introduce gaps between seeds by using canny and subtracting the borders from the mask, which is the result above.
My code so far:
auto img = GetAreaOfInterest(orig, bt); // <- just a Gauss blur with OTSU thresh
{
cv::Mat gray, edges;
cv::cvtColor(orig, gray, cv::COLOR_BGR2GRAY);
// tried bilateralFIlter here, but things only got worse
cv::Canny(gray, edges, 40, 160);
cv::dilate(edges, edges,
cv::getStructuringElement(cv::MORPH_ELLIPSE, {5, 5}),
{-1, -1}, 1);
img.setTo(0, edges != 0);
}
cv::Mat background, foreground, unknown, markers;
cv::dilate(img, background,
cv::getStructuringElement(cv::MORPH_ELLIPSE, {3,3}));
/* Get area for which we are sure it's the foreground */
cv::distanceTransform(img, img, cv::DIST_L1, 3, CV_8U);
{
double max;
cv::minMaxLoc(img, nullptr, &max);
cv::threshold(img, img, 0.46 * max, 255, cv::THRESH_BINARY);
}
img.convertTo(foreground, CV_8UC3);
show_resized("distance", foreground, 0.5);
/* Mark unknown areas */
cv::subtract(background, foreground, unknown);
/* Set up markers */
cv::connectedComponents(foreground, markers);
markers = markers + 1;
markers.setTo(0, unknown == 255);
/* Expand markers */
cv::watershed(orig, markers);
// Count objects
original image:
the most problematic image:
I am trying to make an average of two blobs in OpenCV. To achieve that I was planning to use watershed algorithm on the image preprocessed in the following way:
cv::Mat common, diff, processed, result;
cv::bitwise_and(blob1, blob2, common); //calc common area of the two blobs
cv::absdiff(blob1, blob2, diff); //calc area where they differ
cv::distanceTransform(diff, processed, CV_DIST_L2, 3); //idea here is that the highest intensity
//will be in the middle of the differing area
cv::normalize(processed, processed, 0, 255, cv::NORM_MINMAX, CV_8U); //convert floats to bytes
cv::Mat watershedMarkers, watershedOutline;
common.convertTo(watershedMarkers, CV_32S, 1. / 255, 1); //change background to label 1, common area to label 2
watershedMarkers.setTo(0, processed); //set 0 (unknown) for area where blobs differ
cv::cvtColor(processed, processed, CV_GRAY2RGB); //watershed wants 3 channels
cv::watershed(processed, watershedMarkers);
cv::rectangle(watershedMarkers, cv::Rect(0, 0, watershedMarkers.cols, watershedMarkers.rows), 1); //remove the outline
//draw the boundary in red (for debugging)
watershedMarkers.convertTo(watershedOutline, CV_16S);
cv::threshold(watershedOutline, watershedOutline, 0, 255, CV_THRESH_BINARY_INV);
watershedOutline.convertTo(watershedOutline, CV_8U);
processed.setTo(cv::Scalar(CV_RGB(255, 0, 0)), watershedOutline);
//convert computed labels back to mask (blob), less relevant but shows my ultimate goal
watershedMarkers.convertTo(watershedMarkers, CV_8U);
cv::threshold(watershedMarkers, watershedMarkers, 1, 0, CV_THRESH_TOZERO_INV);
cv::bitwise_not(watershedMarkers * 255, result);
My problem with the results is that the calculated boundary is (almost) always adjacent to the area common to both blobs. Here are the pictures:
Input markers (black = 0, gray = 1, white = 2)
Watershed input image (distance transform result) with resulting outline drawn in red:
I would expect the boundary to go along the maximum intensity region of the input (that is, along the middle of the differing area). Instead (as you can see) it mostly goes around the area marked as 2, with a bit shifted to touch the background (marked as 1). Do I do something wrong here, or did I misunderstand how watershed works?
Starting from this image:
You can get the correct result simply passing an all-zero image to watershed algorithm. The "basin" is then equally filled of "water" starting from each "side" (then just remember to remove the outer border which is set by default to -1 by watershed algorithm):
Code:
#include <opencv2\opencv.hpp>
using namespace cv;
using namespace std;
int main()
{
Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);
Mat1i markers(img.rows, img.cols, int(0));
markers.setTo(1, img == 128);
markers.setTo(2, img == 255);
Mat3b image(markers.rows, markers.cols, Vec3b(0,0,0));
markers.convertTo(markers, CV_32S);
watershed(image, markers);
Mat3b result;
cvtColor(img, result, COLOR_GRAY2BGR);
result.setTo(Scalar(0, 0, 255), markers == -1);
imshow("Result", result);
waitKey();
return(0);
}
I am a beginner in OpenCV, I need to remove the horizontal and vertical lines in the image so that only the text remains ( The lines were causing trouble when extracting text in ocr ). I am trying to extract text from the Nutrient Fact Table. Can anyone help me?
This was an interesting question, so I gave it a shot. Below I will show you how to extract and remove horizontal and vertical lines. You could extrapolate from it. Also, for sake of saving time, I did not preprocess your image to crop out the background as one should, which is an avenue for improvement.
The result:
The code (edit: added vertical lines):
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main(int, char** argv)
{
// Load the image
Mat src = imread(argv[1]);
// Check if image is loaded fine
if(!src.data)
cerr << "Problem loading image!!!" << endl;
Mat gray;
if (src.channels() == 3)
{
cvtColor(src, gray, CV_BGR2GRAY);
}
else
{
gray = src;
}
//inverse binary img
Mat bw;
//this will hold the result, image to be passed to OCR
Mat fin;
//I find OTSU binarization best for text.
//Would perform better if background had been cropped out
threshold(gray, bw, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
threshold(gray, fin, 0, 255, THRESH_BINARY | THRESH_OTSU);
imshow("binary", bw);
Mat dst;
Canny( fin, dst, 50, 200, 3 );
Mat str = getStructuringElement(MORPH_RECT, Size(3,3));
dilate(dst, dst, str, Point(-1, -1), 3);
imshow("dilated_canny", dst);
//bitwise_and w/ canny image helps w/ background noise
bitwise_and(bw, dst, dst);
imshow("and", dst);
Mat horizontal = dst.clone();
Mat vertical = dst.clone();
fin = ~dst;
//Image that will be horizontal lines
Mat horizontal = bw.clone();
//Selected this value arbitrarily
int horizontalsize = horizontal.cols / 30;
Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));
erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1), 1);
imshow("horizontal_lines", horizontal);
//Need to find horizontal contours, so as to not damage letters
vector<Vec4i> hierarchy;
vector<vector<Point> >contours;
findContours(horizontal, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);
for (const auto& c : contours)
{
Rect r = boundingRect(c);
float percentage_height = (float)r.height / (float)src.rows;
float percentage_width = (float)r.width / (float)src.cols;
//These exclude contours that probably are not dividing lines
if (percentage_height > 0.05)
continue;
if (percentage_width < 0.50)
continue;
//fills in line with white rectange
rectangle(fin, r, Scalar(255,255,255), CV_FILLED);
}
int verticalsize = vertical.rows / 30;
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size(1,verticalsize));
erode(vertical, vertical, verticalStructure, Point(-1, -1));
dilate(vertical, vertical, verticalStructure, Point(-1, -1), 1);
imshow("verticalal", vertical);
findContours(vertical, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);
for (const auto& c : contours)
{
Rect r = boundingRect(c);
float percentage_height = (float)r.height / (float)src.rows;
float percentage_width = (float)r.width / (float)src.cols;
//These exclude contours that probably are not dividing lines
if (percentage_width > 0.05)
continue;
if (percentage_height < 0.50)
continue;
//fills in line with white rectange
rectangle(fin, r, Scalar(255,255,255), CV_FILLED);
}
imshow("Result", fin);
waitKey(0);
return 0;
}
The limitations of this approach are that the lines need to be straight. Due to the curve in the bottom line, it cuts slightly into "E" in "Energy". Perhaps with a hough line detection like suggested (I've never used it), a similar but more robust approach could be devised. Also, filling in the lines with rectangles probably is not the best approach.
I've been following this tutorial to get the skew angle of an image. It seems like HoughLinesP is struggling to find lines when characters are a bit scattered on the target image.
This is my input image:
This is the lines the HoughLinesP has found:
It's not really getting most of the lines and it seems pretty obvious to me why. This is because I've set my minLineWidth to be (size.width / 2.f). The point is that because of the few lines it has found it turns out that the skew angle is also wrong. (-3.15825 in this case, when it should be something close to 0.5)
I've tried to erode my input file to make characters get closer and in this case it seems to work out, but I don't feel this is best approach for situations akin to it.
This is my eroded input image:
This is the lines the HoughLinesP has found:
This time it has found a skew angle of -0.2185 degrees, which is what I was expecting but in other hand it is losing the vertical space between lines which in my humble opinion isn't a good thing.
Is there another to pre-process this kind of image to make houghLinesP get better results for scattered characters ?
Here is the source code I'm using:
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
static cv::Scalar randomColor( cv::RNG& rng )
{
int icolor = (unsigned) rng;
return cv::Scalar( icolor&255, (icolor>>8)&255, (icolor>>16)&255 );
}
void rotate(cv::Mat& src, double angle, cv::Mat& dst)
{
int len = std::max(src.cols, src.rows);
cv::Point2f pt(len/2., len/2.);
cv::Mat r = cv::getRotationMatrix2D(pt, angle, 1.0);
cv::warpAffine(src, dst, r, cv::Size(len, len));
}
double compute_skew(cv::Mat& src)
{
// Random number generator
cv::RNG rng( 0xFFFFFFFF );
cv::Size size = src.size();
cv::bitwise_not(src, src);
std::vector<cv::Vec4i> lines;
cv::HoughLinesP(src, lines, 1, CV_PI/180, 100, size.width / 2.f, 20);
cv::Mat disp_lines(size, CV_8UC3, cv::Scalar(0, 0, 0));
double angle = 0.;
unsigned nb_lines = lines.size();
for (unsigned i = 0; i < nb_lines; ++i)
{
cv::line(disp_lines, cv::Point(lines[i][0], lines[i][1]),
cv::Point(lines[i][2], lines[i][3]), randomColor(rng));
angle += atan2((double)lines[i][3] - lines[i][1],
(double)lines[i][2] - lines[i][0]);
}
angle /= nb_lines; // mean angle, in radians.
std::cout << angle * 180 / CV_PI << std::endl;
cv::imshow("HoughLinesP", disp_lines);
cv::waitKey(0);
return angle * 180 / CV_PI;
}
int main()
{
// Load in grayscale.
cv::Mat img = cv::imread("IMG_TESTE.jpg", 0);
cv::Mat rotated;
double angle = compute_skew(img);
rotate(img, angle, rotated);
//Show image
cv::imshow("Rotated", rotated);
cv::waitKey(0);
}
Cheers
I'd suggest finding individual components first (i.e., the lines and the letters), for example using cv::threshold and cv::findContours.
Then, you could drop the individual components that are narrow (i.e., the letters). You can do this using cv::floodFill for example. This should leave you with the lines only.
Effectively, getting rid of the letters might provide easier input for the Hough transform.
Try to detect groups of characters as blocks, then find contours of these blocks. Below I've done it using blurring, a morphological opening and a threshold operation.
Mat im = imread("yCK4t.jpg", 0);
Mat blurred;
GaussianBlur(im, blurred, Size(5, 5), 2, 2);
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
Mat morph;
morphologyEx(blurred, morph, CV_MOP_OPEN, kernel);
Mat bw;
threshold(morph, bw, 0, 255, CV_THRESH_BINARY_INV | CV_THRESH_OTSU);
Mat cont = Mat::zeros(im.rows, im.cols, CV_8U);
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(bw, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
for(int idx = 0; idx >= 0; idx = hierarchy[idx][0])
{
drawContours(cont, contours, idx, Scalar(255, 255, 255), 1);
}
Then use Hough line transform on contour image.
With accumulator threshold 80, I get following lines that results in an angle of -3.81. This is high because of the outlier line that is almost vertical. With this approach, majority of the lines will have similar angle values except few outliers. Detecting and discarding the outliers will give you a better approximation of the angle.
HoughLinesP(cont, lines, 1, CV_PI/180, 80, size.width / 4.0f, size.width / 8.0f);
I am new to opencv. I am working with opencv and C++ on visual studio 2013. I have a task to extract all horizontal and vertical lines of a document containing a table separately and use that to extract the cells present in the table.
I can only use morphological operation to achieve that.
Can anyone suggest the procedure to achieve that?
Here is a sample document.
Finally got the output.
Look at the code.
string src = "d://sabari//23.jpg";
Mat im = imread(src);
Mat gray;
if (im.channels() == 3)
{
cvtColor(im, gray, CV_BGR2GRAY);
}
else
{
gray = im;
}
adaptiveThreshold(~gray, gray, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);
Mat vertical = gray.clone();
int horizontalsize = gray.cols / 30;
Mat structure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));
erode(gray, gray,structure, Point(-1, -1));
dilate(gray, gray,structure, Point(-1, -1));
imshow("ans", gray);
imwrite("d://out2.jpg", gray);
int verticalsize = vertical.rows / 30;
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));
erode(vertical, vertical, verticalStructure, Point(-1, -1));
dilate(vertical, vertical, verticalStructure, Point(-1, -1));
imshow("ans1", vertical);
imwrite("d://out3.jpg", vertical);