Unsharp mask implementation with OpenCV - c++

I want to apply unsharp mask like Adobe Photoshop,
I know this answer, but it's not as sharp as Photoshop.
Photoshop has 3 parameters in Smart Sharpen dialog: Amount, Radius, Reduce Noise; I want to implement all of them.
This is the code I wrote, according to various sources in SO.
But the result is good in some stages ("blurred", "unsharpMask", "highContrast"), but in the last stage ("retval") the result is not good.
Where am I wrong, what should I improve?
Is it possible to improve the following algorithm in terms of performance?
#include "opencv2/opencv.hpp"
#include "fstream"
#include "iostream"
#include <chrono>
using namespace std;
using namespace cv;
// from https://docs.opencv.org/3.4/d3/dc1/tutorial_basic_linear_transform.html
void increaseContrast(Mat img, Mat* dst, int amountPercent)
{
*dst = img.clone();
double alpha = amountPercent / 100.0;
*dst *= alpha;
}
// from https://stackoverflow.com/a/596243/7206675
float luminanceAsPercent(Vec3b color)
{
return (0.2126 * color[2]) + (0.7152 * color[1]) + (0.0722 * color[0]);
}
// from https://stackoverflow.com/a/2938365/7206675
Mat usm(Mat original, int radius, int amountPercent, int threshold)
{
// copy original for our return value
Mat retval = original.clone();
// create the blurred copy
Mat blurred;
cv::GaussianBlur(original, blurred, cv::Size(0, 0), radius);
cv::imshow("blurred", blurred);
waitKey();
// subtract blurred from original, pixel-by-pixel to make unsharp mask
Mat unsharpMask;
cv::subtract(original, blurred, unsharpMask);
cv::imshow("unsharpMask", unsharpMask);
waitKey();
Mat highContrast;
increaseContrast(original, &highContrast, amountPercent);
cv::imshow("highContrast", highContrast);
waitKey();
// assuming row-major ordering
for (int row = 0; row < original.rows; row++)
{
for (int col = 0; col < original.cols; col++)
{
Vec3b origColor = original.at<Vec3b>(row, col);
Vec3b contrastColor = highContrast.at<Vec3b>(row, col);
Vec3b difference = contrastColor - origColor;
float percent = luminanceAsPercent(unsharpMask.at<Vec3b>(row, col));
Vec3b delta = difference * percent;
if (*(uchar*)&delta > threshold) {
retval.at<Vec3b>(row, col) += delta;
//retval.at<Vec3b>(row, col) = contrastColor;
}
}
}
return retval;
}
int main(int argc, char* argv[])
{
if (argc < 2) exit(1);
Mat mat = imread(argv[1]);
mat = usm(mat, 4, 110, 66);
imshow("usm", mat);
waitKey();
//imwrite("USM.png", mat);
}
Original Image:
Blurred stage - Seemingly good:
UnsharpMask stage - Seemingly good:
HighContrast stage - Seemingly good:
Result stage of my code - Looks bad!
Result From Photoshop - Excellent!

First of all, judging by the artefacts that Photoshop left on the borders of the petals, I'd say that it applies the mask by using a weighted sum between the original image and the mask, as in the answer you tried first.
I modified your code to implement this scheme and I tried to tweak the parameters to get as close as the Photoshop result, but I couldn't without creating a lot of noise. I wouldn't try to guess what Photoshop is exactly doing (I would definitely like to know), however I discovered that it is fairly reproducible by applying some filter on the mask to reduce the noise. The algorithm scheme would be:
blurred = blur(image, Radius)
mask = image - blurred
mask = some_filter(mask)
sharpened = (mask < Threshold) ? image : image - Amount * mask
I implemented this and tried using basic filters (median blur, mean filter, etc) on the mask and this is the kind of result I can get:
which is a bit noisier than the Photoshop image but, in my opinion, close enough to what you wanted.
On another note, it will of course depend on the usage you have for your filter, but I think that the settings you used in Photoshop are too strong (you have big overshoots near petals borders). This is sufficient to have a nice image at the naked eye, with limited overshoot:
Finally, here is the code I used to generate the two images above:
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
Mat usm(Mat original, float radius, float amount, float threshold)
{
// work using floating point images to avoid overflows
cv::Mat input;
original.convertTo(input, CV_32FC3);
// copy original for our return value
Mat retbuf = input.clone();
// create the blurred copy
Mat blurred;
cv::GaussianBlur(input, blurred, cv::Size(0, 0), radius);
// subtract blurred from original, pixel-by-pixel to make unsharp mask
Mat unsharpMask;
cv::subtract(input, blurred, unsharpMask);
// --- filter on the mask ---
//cv::medianBlur(unsharpMask, unsharpMask, 3);
cv::blur(unsharpMask, unsharpMask, {3,3});
// --- end filter ---
// apply mask to image
for (int row = 0; row < original.rows; row++)
{
for (int col = 0; col < original.cols; col++)
{
Vec3f origColor = input.at<Vec3f>(row, col);
Vec3f difference = unsharpMask.at<Vec3f>(row, col);
if(cv::norm(difference) >= threshold) {
retbuf.at<Vec3f>(row, col) = origColor + amount * difference;
}
}
}
// convert back to unsigned char
cv::Mat ret;
retbuf.convertTo(ret, CV_8UC3);
return ret;
}
int main(int argc, char* argv[])
{
if (argc < 3) exit(1);
Mat original = imread(argv[1]);
Mat expected = imread(argv[2]);
// closer to Photoshop
Mat current = usm(original, 0.8, 12., 1.);
// better settings (in my opinion)
//Mat current = usm(original, 2., 1., 3.);
cv::imwrite("current.png", current);
// comparison plot
cv::Rect crop(127, 505, 163, 120);
cv::Mat crops[3];
cv::resize(original(crop), crops[0], {0,0}, 4, 4, cv::INTER_NEAREST);
cv::resize(expected(crop), crops[1], {0,0}, 4, 4, cv::INTER_NEAREST);
cv::resize( current(crop), crops[2], {0,0}, 4, 4, cv::INTER_NEAREST);
char const* texts[] = {"original", "photoshop", "current"};
cv::Mat plot = cv::Mat::zeros(120 * 4, 163 * 4 * 3, CV_8UC3);
for(int i = 0; i < 3; ++i) {
cv::Rect region(163 * 4 * i, 0, 163 * 4, 120 * 4);
crops[i].copyTo(plot(region));
cv::putText(plot, texts[i], region.tl() + cv::Point{5,40},
cv::FONT_HERSHEY_SIMPLEX, 1.5, CV_RGB(255, 0, 0), 2.0);
}
cv::imwrite("plot.png", plot);
}

Here's my attempt at 'smart' unsharp masking. Result isn't very good, but I'm posting anyway. Wikipedia article on unsharp masking has details about smart sharpening.
Several things I did differently:
Convert BGR to Lab color space and apply the enhancements to the brightness channel
Use an edge map to apply enhancement to the edge regions
Original:
Enhanced: sigma=2 amount=3 low=0.3 high=.8 w=2
Edge map: low=0.3 high=.8 w=2
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include <cstring>
cv::Mat not_so_smart_sharpen(
const cv::Mat& bgr,
double sigma,
double amount,
double canny_low_threshold_weight,
double canny_high_threshold_weight,
int edge_weight)
{
cv::Mat enhanced_bgr, lab, enhanced_lab, channel[3], blurred, difference, bw, kernel, edges;
// convert to Lab
cv::cvtColor(bgr, lab, cv::ColorConversionCodes::COLOR_BGR2Lab);
// perform the enhancement on the brightness component
cv::split(lab, channel);
cv::Mat& brightness = channel[0];
// smoothing for unsharp masking
cv::GaussianBlur(brightness, blurred, cv::Size(0, 0), sigma);
difference = brightness - blurred;
// calculate an edge map. I'll use Otsu threshold as the basis
double thresh = cv::threshold(brightness, bw, 0, 255, cv::ThresholdTypes::THRESH_BINARY | cv::ThresholdTypes::THRESH_OTSU);
cv::Canny(brightness, edges, thresh * canny_low_threshold_weight, thresh * canny_high_threshold_weight);
// control edge thickness. use edge_weight=0 to use Canny edges unaltered
cv::dilate(edges, edges, kernel, cv::Point(-1, -1), edge_weight);
// unsharp masking on the edges
cv::add(brightness, difference * amount, brightness, edges);
// use the enhanced brightness channel
cv::merge(channel, 3, enhanced_lab);
// convert to BGR
cv::cvtColor(enhanced_lab, enhanced_bgr, cv::ColorConversionCodes::COLOR_Lab2BGR);
// cv::imshow("edges", edges);
// cv::imshow("difference", difference * amount);
// cv::imshow("original", bgr);
// cv::imshow("enhanced", enhanced_bgr);
// cv::waitKey(0);
return enhanced_bgr;
}
int main(int argc, char *argv[])
{
double sigma = std::stod(argv[1]);
double amount = std::stod(argv[2]);
double low = std::stod(argv[3]);
double high = std::stod(argv[4]);
int w = std::stoi(argv[5]);
cv::Mat bgr = cv::imread("flower.jpg");
cv::Mat enhanced = not_so_smart_sharpen(bgr, sigma, amount, low, high, w);
cv::imshow("original", bgr);
cv::imshow("enhanced", enhanced);
cv::waitKey(0);
return 0;
}

Related

Plotting Velocity Vectors in a Binary Image Using OpenCV

I have a binary (Black and White) image over which I want to plot velocity vectors of certain velocity. In MATLAB, we can use quiver to plot these vectors. I am looking for a solution to this problem in OpenCV using C++. I would be grateful if someone can share a solution to this. Although one of the solutions is provided in using the static image on SO (OpenCV How to Plot velocity vectors as arrows in using single static image), its not clear of how to implement it on a binary image. I would be grateful if someone can guide me.
Look forward to some suggestion towards implementation.
This is my solution to your problem: in this example I start with an rgb image coming from the webcam, then i convert it to grayscale and then to binary after applying a threshold.
The next step, when you have a binary image, is to convert it to RGB again (or BGR as the OpenCV convention) and draw whatever you want on it. The code for the arrow is a copypasta of what you have linked.
Hope it helps
cv::VideoCapture cam(n_source);
cam >> frame;
cv::Mat grey_image;
cv::Mat binary_image; // Your binary image
cv::cvtColor(frame, grey_image, CV_RGB2GRAY);
cv::threshold(grey_image, binary_image, 100, 255, 0);
// Convert the binary to RGB
cv::Mat dst_rgb;
cv::cvtColor(binary_image, dst_rgb, CV_GRAY2BGR);
// Draw the arrow on the RGB image
int x = 200;
int y = 200;
int u = 100;
int v = 100;
cv::Point pt1,pt2;
double Theta;
double PI = 3.1416;
cv::Scalar Color(255,0,0);
int size = 5;
int Thickness = 5;
if(u==0)
Theta=PI/2;
else
Theta=atan2(double(v),(double)(u));
pt1.x=x;
pt1.y=y;
pt2.x=x+u;
pt2.y=y+v;
cv::line(dst_rgb,pt1,pt2,Color,Thickness,8); //Draw Line
size=(int)(size*0.707);
if(Theta==PI/2 && pt1.y > pt2.y)
{
pt1.x=(int)(size*cos(Theta)-size*sin(Theta)+pt2.x);
pt1.y=(int)(size*sin(Theta)+size*cos(Theta)+pt2.y);
cv::line(dst_rgb,pt1,pt2,Color,Thickness,8); //Draw Line
pt1.x=(int)(size*cos(Theta)+size*sin(Theta)+pt2.x);
pt1.y=(int)(size*sin(Theta)-size*cos(Theta)+pt2.y);
cv::line(dst_rgb,pt1,pt2,Color,Thickness,8); //Draw Line
}
else{
pt1.x=(int)(-size*cos(Theta)-size*sin(Theta)+pt2.x);
pt1.y=(int)(-size*sin(Theta)+size*cos(Theta)+pt2.y);
cv::line(dst_rgb,pt1,pt2,Color,Thickness,8); //Draw Line
pt1.x=(int)(-size*cos(Theta)+size*sin(Theta)+pt2.x);
pt1.y=(int)(-size*sin(Theta)-size*cos(Theta)+pt2.y);
cv::line(dst_rgb,pt1,pt2,Color,Thickness,8); //Draw Line
}
// Plot
cv::namedWindow("test rgb");
cv::imshow("test rgb", dst_rgb);
cv::waitKey(0);
An example:
After some discussion I have come across cv::arrowedLine in OpenCV whose one of the usage is as follows:
#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
#include <string>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui.hpp>
using namespace std;
using namespace cv;
int main(int argc, char* argv[])
{
auto width = 320;
auto height = 320;
auto img = cv::Mat(cv::Size(width, height), CV_8UC3); // create background image
auto center = cv::Point(width / 2, height / 2); // center point
int lineType = 8;
int thickness = 1;
double tipLength = 0.1;
img.setTo(255); // clear image - set to black (0) or white (255)
for (int angle = 0; angle < 360; angle += 15)
{
auto angleRad = angle*CV_PI / 180.0; // convert angle to radians
auto length = 150;
auto direction = cv::Point(length * cos(angleRad), length * sin(angleRad)); // calculate direction
tipLength = .01 + 0.4 * (angle % 180) / 360;
cv::arrowedLine(img, center + direction*0.5, center + direction, CV_RGB(255, angle, 0), thickness, lineType, 0, tipLength); // draw arrow!
++thickness;
if (0 == angle % 45)
thickness = 0;
if (180 <= angle)
lineType = CV_AA;
}
imshow("Arrowed Image", img); // show image
waitKey();
return EXIT_SUCCESS;
}

How to prepare image data for kmeans opencv function input? [duplicate]

I am making a function using C++ and OpenCV that will detect the color of a pixel in an image, determine what color range it is in, and replace it with a generic color. For example, green could range from dark green to light green, the program would determine that its still green and replace it with a simple green, making the output image very simple looking. everything is set up but I'm having trouble defining the characteristics of each range and was curious if anyone knows or a formula that, given BGR values, could determine the overall color of a pixel. If not I'll have to do much experimentation and make it myself, but if something already exists that'd save time. I've done plenty of research and haven't found anything so far.
If you want to make your image simpler (i.e. with less colors), but good looking, you have a few options:
A simple approach would be to divide (integer division) by a factor N the image, and then multiply by a factor N.
Or you can divide your image into K colors, using some clustering algorithm such as kmeans showed here, or median-cut algorithm.
Original image:
Reduced colors (quantized, N = 64):
Reduced colors (clustered, K = 8):
Code Quantization:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
uchar N = 64;
img /= N;
img *= N;
imshow("Reduced", img);
waitKey();
return 0;
}
Code kmeans:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
// Cluster
int K = 8;
int n = img.rows * img.cols;
Mat data = img.reshape(1, n);
data.convertTo(data, CV_32F);
vector<int> labels;
Mat1f colors;
kmeans(data, K, labels, cv::TermCriteria(), 1, cv::KMEANS_PP_CENTERS, colors);
for (int i = 0; i < n; ++i)
{
data.at<float>(i, 0) = colors(labels[i], 0);
data.at<float>(i, 1) = colors(labels[i], 1);
data.at<float>(i, 2) = colors(labels[i], 2);
}
Mat reduced = data.reshape(3, img.rows);
reduced.convertTo(reduced, CV_8U);
imshow("Reduced", reduced);
waitKey();
return 0;
}
Yes, what you probably mean by "Overall color of a pixel" is either the "Hue" or "Saturation" of the color.
So you want a formula that transform RGB to HSV (Hue, Saturation, Value), and then you would only be interested by the Hue or Saturation values.
See: Algorithm to convert RGB to HSV and HSV to RGB in range 0-255 for both
EDIT: You might need to max out the saturation, and then convert it back to RGB, and inspect which value is the highest (for instance (255,0,0), or (255,0,255), etc.
If you want to access RGB value of all pixels , then below is code,
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat image = imread("image_path");
for(int row = 1; row < image.rows; row++)
{
for(int col = 1; col < image.cols; col++)
{
Vec3b rgb = image.at<Vec3b>(row, col);
}
}
}

Is there a formula to determine overall color given BGR values? (OpenCV and C++)

I am making a function using C++ and OpenCV that will detect the color of a pixel in an image, determine what color range it is in, and replace it with a generic color. For example, green could range from dark green to light green, the program would determine that its still green and replace it with a simple green, making the output image very simple looking. everything is set up but I'm having trouble defining the characteristics of each range and was curious if anyone knows or a formula that, given BGR values, could determine the overall color of a pixel. If not I'll have to do much experimentation and make it myself, but if something already exists that'd save time. I've done plenty of research and haven't found anything so far.
If you want to make your image simpler (i.e. with less colors), but good looking, you have a few options:
A simple approach would be to divide (integer division) by a factor N the image, and then multiply by a factor N.
Or you can divide your image into K colors, using some clustering algorithm such as kmeans showed here, or median-cut algorithm.
Original image:
Reduced colors (quantized, N = 64):
Reduced colors (clustered, K = 8):
Code Quantization:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
uchar N = 64;
img /= N;
img *= N;
imshow("Reduced", img);
waitKey();
return 0;
}
Code kmeans:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
// Cluster
int K = 8;
int n = img.rows * img.cols;
Mat data = img.reshape(1, n);
data.convertTo(data, CV_32F);
vector<int> labels;
Mat1f colors;
kmeans(data, K, labels, cv::TermCriteria(), 1, cv::KMEANS_PP_CENTERS, colors);
for (int i = 0; i < n; ++i)
{
data.at<float>(i, 0) = colors(labels[i], 0);
data.at<float>(i, 1) = colors(labels[i], 1);
data.at<float>(i, 2) = colors(labels[i], 2);
}
Mat reduced = data.reshape(3, img.rows);
reduced.convertTo(reduced, CV_8U);
imshow("Reduced", reduced);
waitKey();
return 0;
}
Yes, what you probably mean by "Overall color of a pixel" is either the "Hue" or "Saturation" of the color.
So you want a formula that transform RGB to HSV (Hue, Saturation, Value), and then you would only be interested by the Hue or Saturation values.
See: Algorithm to convert RGB to HSV and HSV to RGB in range 0-255 for both
EDIT: You might need to max out the saturation, and then convert it back to RGB, and inspect which value is the highest (for instance (255,0,0), or (255,0,255), etc.
If you want to access RGB value of all pixels , then below is code,
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat image = imread("image_path");
for(int row = 1; row < image.rows; row++)
{
for(int col = 1; col < image.cols; col++)
{
Vec3b rgb = image.at<Vec3b>(row, col);
}
}
}

OpenCv 2.3 C - How to isolate object inside image

i have an image like:
i want to remove the black rows and cols round the number.
So i want that the result is:
i try this:
void findX(IplImage* imgSrc,int* min, int* max){
int i;
int minFound=0;
CvMat data;
CvScalar maxVal=cvRealScalar(imgSrc->width * 255);
CvScalar val=cvRealScalar(0);
//For each col sum, if sum < width*255 then we find the min
//then continue to end to search the max, if sum< width*255 then is new max
for (i=0; i< imgSrc->width; i++){
cvGetCol(imgSrc, &data, i);
val= cvSum(&data);
if(val.val[0] < maxVal.val[0]){
*max= i;
if(!minFound){
*min= i;
minFound= 1;
}
}
}
}
void findY(IplImage* imgSrc,int* min, int* max){
int i;
int minFound=0;
CvMat data;
CvScalar maxVal=cvRealScalar(imgSrc->width * 255);
CvScalar val=cvRealScalar(0);
//For each col sum, if sum < width*255 then we find the min
//then continue to end to search the max, if sum< width*255 then is new max
for (i=0; i< imgSrc->height; i++){
cvGetRow(imgSrc, &data, i);
val= cvSum(&data);
if(val.val[0] < maxVal.val[0]){
*max=i;
if(!minFound){
*min= i;
minFound= 1;
}
}
}
}
CvRect findBB(IplImage* imgSrc){
CvRect aux;
int xmin, xmax, ymin, ymax;
xmin=xmax=ymin=ymax=0;
findX(imgSrc, &xmin, &xmax);
findY(imgSrc, &ymin, &ymax);
aux=cvRect(xmin, ymin, xmax-xmin, ymax-ymin);
//printf("BB: %d,%d - %d,%d\n", aux.x, aux.y, aux.width, aux.height);
return aux;
}
So i use:
IplImage *my_image = cvLoad....
CvRect bb = findBB(my_image);
IplImage *new_image = cvCreateImage(cvSize(bb.width,bb.height), my_image->depth, 1);
cvShowImage("test",new_image);
it doesn't work good, cause i try to check if in new image there are black rows or cols and they are present. what can i do? can someone help me? (sorry for my english!)
One way to do it is to simply execute the bounding box technique to detect the digit, as illustrated by the image below:
Since your image is already processed the bounding box technique I use is a lot simpler.
After that procedure, all you really need to do is set the ROI (Region of Interest) of the original image to the area defined by the box to achieve the crop effect and isolate the object:
Notice that in the resulting image there is one extra row/column of pixels in the border that are not white. Well, they are not black either. That's because I didn't performed any threshold method to binarize the image to black and white. The code below demonstrates the bounding box technique being executed on a grayscale version of the image.
This is pretty much the roadmap to achieve what you want. For educational purposes I'm sharing the code I wrote using the C++ interface of OpenCV. I'm sure you are capable of converting it to the C interface.
#include <cv.h>
#include <highgui.h>
#include <vector>
int main(int argc, char* argv[])
{
cv::Mat img = cv::imread(argv[1]);
// Convert RGB Mat to GRAY
cv::Mat gray;
cv::cvtColor(img, gray, CV_BGR2GRAY);
// Store the set of points in the image before assembling the bounding box
std::vector<cv::Point> points;
cv::Mat_<uchar>::iterator it = gray.begin<uchar>();
cv::Mat_<uchar>::iterator end = gray.end<uchar>();
for (; it != end; ++it)
{
if (*it) points.push_back(it.pos());
}
// Compute minimal bounding box
cv::RotatedRect box = cv::minAreaRect(cv::Mat(points));
// Draw bounding box in the original image (debug purposes)
//cv::Point2f vertices[4];
//box.points(vertices);
//for (int i = 0; i < 4; ++i)
//{
//cv::line(img, vertices[i], vertices[(i + 1) % 4], cv::Scalar(0, 255, 0), 1, CV_AA);
//}
//cv::imshow("box", img);
//cv::imwrite("box.png", img);
// Set Region of Interest to the area defined by the box
cv::Rect roi;
roi.x = box.center.x - (box.size.width / 2);
roi.y = box.center.y - (box.size.height / 2);
roi.width = box.size.width;
roi.height = box.size.height;
// Crop the original image to the defined ROI
cv::Mat crop = img(roi);
cv::imshow("crop", crop);
cv::imwrite("cropped.png", crop);
cvWaitKey(0);
return 0;
}

Need help implementing a special edge detector

I'm implementing an approach from a research paper. Part of the approach calls for a major edge detector, which the authors describe as follows:
Obtain DC image (effectively downsample by 8 for both width and height)
Calculate Sobel gradient of DC image
Threshold Sobel gradient image (using T=120)
Morphological operations to clean up edge image
Note that this NOT Canny edge detection -- they don't bother with things like non-maximum suppression, etc. I could of course do this with Canny edge detection, but I want to implement things exactly as they are expressed in the paper.
That last step is the one I'm a bit stuck on.
Here is exactly what the authors say about it:
After obtaining the binary
edge map from the edge detection process, a binary morphological
operation is employed to remove isolated edge pixels,
which might cause false alarms during the edge detection
Here's how things are supposed to look like at the end of it all (edge blocks have been filled in black):
Here's what I have if I skip the last step:
It seems to be on the right track. So here's what happens if I do erosion for step 4:
I've tried combinations of erosion and dilation to obtain the same result as they do, but don't get anywhere close. Can anyone suggest a combination of morphological operators that will get me the desired result?
Here's the binarization output, in case anyone wants to play around with it:
And if you're really keen, here is the source code (C++):
#include <cv.h>
#include <highgui.h>
#include <stdlib.h>
#include <assert.h>
using cv::Mat;
using cv::Size;
#include <stdio.h>
#define DCTSIZE 8
#define EDGE_PX 255
/*
* Display a matrix as an image on the screen.
*/
void
show_mat(char *heading, Mat const &m)
{
Mat clone = m.clone();
Mat scaled(clone.size(), CV_8UC1);
convertScaleAbs(clone, scaled);
IplImage ipl = scaled;
cvNamedWindow(heading, CV_WINDOW_AUTOSIZE);
cvShowImage(heading, &ipl);
cvWaitKey(0);
}
/*
* Get the DC components of the specified matrix as an image.
*/
Mat
get_dc(Mat const &m)
{
Size s = m.size();
assert(s.width % DCTSIZE == 0);
assert(s.height % DCTSIZE == 0);
Size dc_size = Size(s.height/DCTSIZE, s.width/DCTSIZE);
Mat dc(dc_size, CV_32FC1);
cv::resize(m, dc, dc_size, 0, 0, cv::INTER_AREA);
return dc;
}
/*
* Detect the edges:
*
* Sobel operator
* Thresholding
* Morphological operations
*/
Mat
detect_edges(Mat const &src, int T)
{
Mat sobelx = Mat(src.size(), CV_32FC1);
Mat sobely = Mat(src.size(), CV_32FC1);
Mat sobel_sum = Mat(src.size(), CV_32FC1);
cv::Sobel(src, sobelx, CV_32F, 1, 0, 3, 0.5);
cv::Sobel(src, sobely, CV_32F, 0, 1, 3, 0.5);
cv::add(cv::abs(sobelx), cv::abs(sobely), sobel_sum);
Mat binarized = src.clone();
cv::threshold(sobel_sum, binarized, T, EDGE_PX, cv::THRESH_BINARY);
cv::imwrite("binarized.png", binarized);
//
// TODO: this is the part I'm having problems with.
//
#if 0
//
// Try a 3x3 cross structuring element.
//
Mat elt(3,3, CV_8UC1);
elt.at<uchar>(0, 1) = 0;
elt.at<uchar>(1, 0) = 0;
elt.at<uchar>(1, 1) = 0;
elt.at<uchar>(1, 2) = 0;
elt.at<uchar>(2, 1) = 0;
#endif
Mat dilated = binarized.clone();
//cv::dilate(binarized, dilated, Mat());
cv::imwrite("dilated.png", dilated);
Mat eroded = dilated.clone();
cv::erode(dilated, eroded, Mat());
cv::imwrite("eroded.png", eroded);
return eroded;
}
/*
* Black out the blocks in the image that contain DC edges.
*/
void
censure_edge_blocks(Mat &orig, Mat const &edges)
{
Size s = edges.size();
for (int i = 0; i < s.height; ++i)
for (int j = 0; j < s.width; ++j)
{
if (edges.at<float>(i, j) != EDGE_PX)
continue;
int row = i*DCTSIZE;
int col = j*DCTSIZE;
for (int m = 0; m < DCTSIZE; ++m)
for (int n = 0; n < DCTSIZE; ++n)
orig.at<uchar>(row + m, col + n) = 0;
}
}
/*
* Load the image and return the first channel.
*/
Mat
load_grayscale(char *filename)
{
Mat orig = cv::imread(filename);
std::vector<Mat> channels(orig.channels());
cv::split(orig, channels);
Mat grey = channels[0];
return grey;
}
int
main(int argc, char **argv)
{
assert(argc == 3);
int bin_thres = atoi(argv[2]);
Mat orig = load_grayscale(argv[1]);
//show_mat("orig", orig);
Mat dc = get_dc(orig);
cv::imwrite("dc.png", dc);
Mat dc_edges = detect_edges(dc, bin_thres);
cv::imwrite("dc_edges.png", dc_edges);
censure_edge_blocks(orig, dc_edges);
show_mat("censured", orig);
cv::imwrite("censured.png", orig);
return 0;
}
I can't imagine any combination of morphological operations that would produce the same edges as detected by the supposedly correct result, given your partial result as input.
I note that the underlying image is different; this probably contributes to why your results are so different. The Lena image is fine for indicating the type of result but not for comparisons. Do you have the exact same image as the original authors ?
What the authors described could be implemented with connected component analysis, using 8way connectivity. I would not call that morphological though.
I do think you are missing something else: Their image does not have edges that are thicker than one pixel. Yours has. The paragraph you quoted only talks about removing isolated pixels, so there must be a step you missed or implemented differently.
Good luck!
I think that what you need is a kind of erode or open that is, in a sense, 4-way and not 8-way. The default morphological kernel for OpenCV is a 3x3 rectangle (IplConvKernel with shape=CV_SHAPE_RECT). This is pretty harsh on thin edges.
You might want to try eroding with a 3x3 custom IplConvKernel with shape=CV_SHAPE_CROSS.
If you need an even finer filter, you may want to try eroding with 4 different CV_SHAPE_RECT kernels of size 1x2, 2x1 with the anchor in (0,1) and (1,0) for each.
First of all, your input image has a much higher resolution that the test input image, which can explain the fact less edges are detected - the changes are more smooth.
Second of all, since the edges are thresholded to 0, try dilation on smaller neighborhoods (e.g. compare each pixels with 4 original neighbors (in a non-serial manner)) to get rid of isolated edges.