How can I crop a non rectangular region from image?
Imagine I have four points and I want to crop it, this shape wouldn't be a triangle somehow!
For example I have the following image :
and I want to crop this from image :
How can I do this?
regards..
The procedure for cropping an arbitrary quadrilateral (or any polygon for that matter) part of an image is summed us as:
Generate a "mask". The mask is black where you want to keep the image, and white where you don't want to keep it
Compute the "bitwise_and" between your input image and the mask
So, lets assume you have an image. Throughout this I'll use an image size of 30x30 for simplicity, you can change this to suit your use case.
cv::Mat source_image = cv::imread("filename.txt");
And you have four points you want to use as the corners:
cv::Point corners[1][4];
corners[0][0] = Point( 10, 10 );
corners[0][1] = Point( 20, 20 );
corners[0][2] = Point( 30, 10 );
corners[0][3] = Point( 20, 10 );
const Point* corner_list[1] = { corners[0] };
You can use the function cv::fillPoly to draw this shape on a mask:
int num_points = 4;
int num_polygons = 1;
int line_type = 8;
cv::Mat mask(30,30,CV_8UC3, cv::Scalar(0,0,0));
cv::fillPoly( mask, corner_list, &num_points, num_polygons, cv::Scalar( 255, 255, 255 ), line_type);
Then simply compute the bitwise_and of the image and mask:
cv::Mat result;
cv::bitwise_and(source_image, mask, result);
result now has the cropped image in it. If you want the edges to end up white instead of black you could instead do:
cv::Mat result_white(30,30,CV_8UC3, cv::Scalar(255,255,255));
cv::bitwise_and(source_image, mask, result_white, mask);
In this case we use bitwise_and's mask parameter to only do the bitwise_and inside the mask. See this tutorial for more information and links to all the functions I mentioned.
You may use cv::Mat::copyTo() like this:
cv::Mat img = cv::imread("image.jpeg");
// note mask may be single channel, even if img is multichannel
cv::Mat mask = cv::Mat::zeros(img.rows, img.cols, CV_8UC1);
// fill mask with nonzero values, e.g. as Tim suggests
// cv::fillPoly(...)
cv::Mat result(img.size(), img.type(), cv::Scalar(255, 255, 255));
img.copyTo(result, mask);
Related
i have this image
i want to create a tranigle mask to get only this zone
but with the following code i get this result
Moments mu = moments(red,true);
Point center;
center.x = mu.m10 / mu.m00;
center.y = mu.m01 / mu.m00;
circle(red, center, 2, Scalar(0, 0, 255));
cv::Size sz = red.size();
int imageWidth = sz.width;
int imageHeight = sz.height;
Mat mask3(red.size(), CV_8UC1, Scalar::all(0));
// Create Polygon from vertices
vector<Point> ptmask3(3);
ptmask3.push_back(Point(imageHeight-1, imageWidth-1));
ptmask3.push_back(Point(center.x, center.y));
ptmask3.push_back(Point(0, red.rows - 1));
vector<Point> pt;
approxPolyDP(ptmask3, pt, 1.0, true);
// Fill polygon white
fillConvexPoly(mask3, &pt[0], pt.size(), 255, 8, 0);
// Create new image for result storage
Mat hide3(red.size(), CV_8UC3);
// Cut out ROI and store it in imageDest
red.copyTo(hide3, mask3);
imshow("mask3", hide3);
Updated Version (with the Help of Dan MaĊĦek)
Your Triangle is wrong
This is because you're initializing the vector with size 3, then putting another three points into it, for a total of 6 points of which three have default values. Try this instead:
vector<Point> ptmask3;
Also, make sure that the coordinates of the points are correct. You'll want to have a point in the bottom left corner, but it doesn't seem like your current triangle has one like that.
Your image is gray
You need to initialize hide3 properly, like this:
cv::Mat hide3(img.size(), CV_8UC3, cv::Scalar(0));
I want to change the black pixels in the image to red pixels, such that the ball should look white and red. I want to use OpenCV libraries and code it in C++. I have tried converting the image to RGB.
Common approach is to threshold the image, so in your case you would say that each pixel with an intensity less than some threshold will be considered as being black and then recolored to red. One way to find a good threshold (that divides the image's pixel into two classes ("more black" and "more white") is OTSU thresholding:
int main()
{
cv::Mat input = cv::imread("../inputData/ball_thresholding.jpg");
cv::Mat gray;
cv::cvtColor(input,gray,CV_BGR2GRAY);
cv::Mat mask;
// compute inverse thresholding (dark areas become "active" pixel in the mask) with OTSU thresholding:
double grayThres = cv::threshold(gray, mask, 0, 255, CV_THRESH_BINARY_INV | CV_THRESH_OTSU);
// color all masked pixel red:
input.setTo(cv::Scalar(0,0,255), mask);
// compute median filter to remove the whitish black parts and darker white parts
cv::imshow("input", input);
cv::waitKey(0);
return 0;
}
Giving this mask:
and this result:
For this image, the threshold that was computed by OTSU is 127, which means that each grayscale pixel intensity of 127 or less (or less than 127, I'm not sure) will be recolored to red.
If you want to keep the shading effect withing the black/red region, you can remove input.setTo(cv::Scalar(0,0,255), mask); lind and replace it by:
// keep the shading:
for(int j=0; j<input.rows; ++j)
for(int i=0; i<input.cols; ++i)
{
if(mask.at<unsigned char>(j,i))
{
input.at<cv::Vec3b>(j,i)[2] = 255;
}
}
which will result int:
cv::Mat imBW = imread('bwImg.jpg',CV_LOAD_IMAGE_GRAYSCALE);
cv::Mat RGB_img = cv::Mat(imBW.rows, imBW.cols, CV_8UC3);
cv::Mat R_channel = 255-imBW;
cv::Mat B_channel = cv::Mat::zeros(imBW.rows, imBW.cols, CV_8UC1);
cv::Mat G_channel = cv::Mat::zeros(imBW.rows, imBW.cols, CV_8UC1);
vector<cv::Mat> channels;
channels.push_back(B_channel);
channels.push_back(G_channel);
channels.push_back(R_channel);
cv::merge(channels, RGB_img);
I create a Bird-View-Image with the warpPerspective()-function like this:
warpPerspective(frame, result, H, result.size(), CV_WARP_INVERSE_MAP, BORDER_TRANSPARENT);
The result looks very good and also the border is transparent:
Bird-View-Image
Now I want to put this image on top of another image "out". I try doing this with the function warpAffine like this:
warpAffine(result, out, M, out.size(), CV_INTER_LINEAR, BORDER_TRANSPARENT);
I also converted "out" to a four channel image with alpha channel according to a question which was already asked on stackoverflow:
Convert Image
This is the code: cvtColor(out, out, CV_BGR2BGRA);
I expected to see the chessboard but not the gray background. But in fact, my result looks like this:
Result Image
What am I doing wrong? Do I forget something to do? Is there another way to solve my problem? Any help is appreciated :)
Thanks!
Best regards
DamBedEi
I hope there is a better way, but here it is something you could do:
Do warpaffine normally (without the transparency thing)
Find the contour that encloses the image warped
Use this contour for creating a mask (white values inside the image warped, blacks in the borders)
Use this mask for copy the image warped into the other image
Sample code:
// load images
cv::Mat image2 = cv::imread("lena.png");
cv::Mat image = cv::imread("IKnowOpencv.jpg");
cv::resize(image, image, image2.size());
// perform warp perspective
std::vector<cv::Point2f> prev;
prev.push_back(cv::Point2f(-30,-60));
prev.push_back(cv::Point2f(image.cols+50,-50));
prev.push_back(cv::Point2f(image.cols+100,image.rows+50));
prev.push_back(cv::Point2f(-50,image.rows+50 ));
std::vector<cv::Point2f> post;
post.push_back(cv::Point2f(0,0));
post.push_back(cv::Point2f(image.cols-1,0));
post.push_back(cv::Point2f(image.cols-1,image.rows-1));
post.push_back(cv::Point2f(0,image.rows-1));
cv::Mat homography = cv::findHomography(prev, post);
cv::Mat imageWarped;
cv::warpPerspective(image, imageWarped, homography, image.size());
// find external contour and create mask
std::vector<std::vector<cv::Point> > contours;
cv::Mat imageWarpedCloned = imageWarped.clone(); // clone the image because findContours will modify it
cv::cvtColor(imageWarpedCloned, imageWarpedCloned, CV_BGR2GRAY); //only if the image is BGR
cv::findContours (imageWarpedCloned, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
// create mask
cv::Mat mask = cv::Mat::zeros(image.size(), CV_8U);
cv::drawContours(mask, contours, 0, cv::Scalar(255), -1);
// copy warped image into image2 using the mask
cv::erode(mask, mask, cv::Mat()); // for avoid artefacts
imageWarped.copyTo(image2, mask); // copy the image using the mask
//show images
cv::imshow("imageWarpedCloned", imageWarpedCloned);
cv::imshow("warped", imageWarped);
cv::imshow("image2", image2);
cv::waitKey();
One of the easiest ways to approach this (not necessarily the most efficient) is to warp the image twice, but set the OpenCV constant boundary value to different values each time (i.e. zero the first time and 255 the second time). These constant values should be chosen towards the minimum and maximum values in the image.
Then it is easy to find a binary mask where the two warp values are close to equal.
More importantly, you can also create a transparency effect through simple algebra like the following:
new_image = np.float32((warp_const_255 - warp_const_0) *
preferred_bkg_img) / 255.0 + np.float32(warp_const_0)
The main reason I prefer this method is that openCV seems to interpolate smoothly down (or up) to the constant value at the image edges. A fully binary mask will pick up these dark or light fringe areas as artifacts. The above method acts more like true transparency and blends properly with the preferred background.
Here's a small test program that warps with transparent "border", then copies the warped image to a solid background.
int main()
{
cv::Mat input = cv::imread("../inputData/Lenna.png");
cv::Mat transparentInput, transparentWarped;
cv::cvtColor(input, transparentInput, CV_BGR2BGRA);
//transparentInput = input.clone();
// create sample transformation mat
cv::Mat M = cv::Mat::eye(2,3, CV_64FC1);
// as a sample, just scale down and translate a little:
M.at<double>(0,0) = 0.3;
M.at<double>(0,2) = 100;
M.at<double>(1,1) = 0.3;
M.at<double>(1,2) = 100;
// warp to same size with transparent border:
cv::warpAffine(transparentInput, transparentWarped, M, transparentInput.size(), CV_INTER_LINEAR, cv::BORDER_TRANSPARENT);
// NOW: merge image with background, here I use the original image as background:
cv::Mat background = input;
// create output buffer with same size as input
cv::Mat outputImage = input.clone();
for(int j=0; j<transparentWarped.rows; ++j)
for(int i=0; i<transparentWarped.cols; ++i)
{
cv::Scalar pixWarped = transparentWarped.at<cv::Vec4b>(j,i);
cv::Scalar pixBackground = background.at<cv::Vec3b>(j,i);
float transparency = pixWarped[3] / 255.0f; // pixel value: 0 (0.0f) = fully transparent, 255 (1.0f) = fully solid
outputImage.at<cv::Vec3b>(j,i)[0] = transparency * pixWarped[0] + (1.0f-transparency)*pixBackground[0];
outputImage.at<cv::Vec3b>(j,i)[1] = transparency * pixWarped[1] + (1.0f-transparency)*pixBackground[1];
outputImage.at<cv::Vec3b>(j,i)[2] = transparency * pixWarped[2] + (1.0f-transparency)*pixBackground[2];
}
cv::imshow("warped", outputImage);
cv::imshow("input", input);
cv::imwrite("../outputData/TransparentWarped.png", outputImage);
cv::waitKey(0);
return 0;
}
I use this as input:
and get this output:
which looks like ALPHA channel isn't set to ZERO by warpAffine but to something like 205...
But in general this is the way I would do it (unoptimized)
The implementation of this functionality seems pretty straightforward in Python, as shown here: http://docs.opencv.org/trunk/doc/py_tutorials/py_imgproc/py_grabcut/py_grabcut.html
Yet, when I tried to do exactly the same in C++, I get bad arguments error (for the grabcut function). How to put the mask image in the right format?
I am a newbie at this, so I'd be very thankful if someone could help me understand better. Thank you!
Here's what I have so far:
Mat image;
image= imread(file);
Mat mask;
mask.setTo( GC_BGD );
mask = imread("messi5.png");
Mat image2 = image.clone();
// define bounding rectangle
cv::Rect rectangle(startX, startY, width, height);
cv::Mat result; // segmentation result (4 possible values)
cv::Mat bgModel,fgModel; // the models (internally used)
//// GrabCut segmentation that works, but with a rectangle, not with the mask I need
//cv::grabCut(image, // input image
// result, // segmentation result
// rectangle,// rectangle containing foreground
// bgModel,fgModel, // models
// 1, // number of iterations
// cv::GC_INIT_WITH_RECT); // use rectangle
grabCut( image, mask, rectangle, bgModel, fgModel, 1, GC_INIT_WITH_MASK);
cv::compare(mask,cv::GC_PR_FGD,mask,cv::CMP_EQ);
cv::Mat foreground(image.size(),CV_8UC3,cv::Scalar(255,255,255));
image.copyTo(foreground,mask); // bg pixels not copied
namedWindow( "Display window", WINDOW_AUTOSIZE );
imshow( "Display window", foreground );
waitKey(0);
return 0;
}
It looks like you have misunderstood the guide, repeated here from the linked guide in the question:
# newmask is the mask image I manually labelled
newmask = cv2.imread('newmask.png',0)
# whereever it is marked white (sure foreground), change mask=1
# whereever it is marked black (sure background), change mask=0
mask[newmask == 0] = 0
mask[newmask == 255] = 1
mask, bgdModel, fgdModel = cv2.grabCut(img,mask,None,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_MASK)
mask = np.where((mask==2)|(mask==0),0,1).astype('uint8')
img = img*mask[:,:,np.newaxis]
plt.imshow(img),plt.colorbar(),plt.show()
this is not what you have done i'm afraid. For a start you seem to have set the mask to the rgb image:
mask = imread("messi5.png");
whereas is should be set to the mask image:
mask = imread("newmask.png",CV_LOAD_IMAGE_GRAYSCALE);
EDIT from comments:
from a pure red mask painted over the image (an actual mask would be better).
maskTmp = imread("messi5.png");
std::vector<cv::Mat> channels(3)
split( messi5, channels);
cv::Mat maskRed = channels[2];
now threshold on the red channel to get your binary mask.
On an image frame, I use
void ellipse(Mat& img, Point center, Size axes, double angle, double startAngle, double endAngle, const Scalar& color, int thickness=1, int lineType=8, int shift=0)
to draw an ellipse and I want to set the ellipse color to green [ RGB value : (165, 206, 94) ].
So I set the parameter const Scalar& color to
cv::Scalar(94.0, 206.0, 165.0, 0.0); // as BGR order, suppose the value is 0.0 - 255.0
cv::Scalar(94.0/255.0, 206.0/255.0, 165.0/255.0, 0.0); // suppose the value is 0.0 - 1.0
I also tried RGB alternative.
CV_RGB(165.0, 206.0, 94.0); // as RGB order, suppose the value is 0.0 - 255.0
CV_RGB(165.0/255.0, 206.0/255.0, 94.0/255.0); // suppose the value is 0.0 - 1.0
But the color being displayed is white [ RGB value (255, 255, 255) ] , not the desired green one.
What I missed at this point? Any suggestion please. Thank you.
EDIT:
Let me put whole related code here. According to OpenCV iOS - Video Processing, this is the CvVideoCamera config in - (void)viewDidLoad;:
self.videoCamera = [[CvVideoCamera alloc] initWithParentView:imgView];
[self.videoCamera setDelegate:self];
self.videoCamera.defaultAVCaptureDevicePosition = AVCaptureDevicePositionFront;
self.videoCamera.defaultAVCaptureSessionPreset = AVCaptureSessionPreset352x288;
self.videoCamera.defaultAVCaptureVideoOrientation = AVCaptureVideoOrientationPortrait;
self.videoCamera.defaultFPS = 30;
self.videoCamera.grayscaleMode = NO;
[self.videoCamera adjustLayoutToInterfaceOrientation:UIInterfaceOrientationPortrait];
Then after [self.videoCamera start]; called, the (Mat&)image would be captured and can be processed in the CvVideoCameraDelegate method - (void)processImage:(Mat&)image; and here are the code to draw an ellipse:
- (void)processImage:(Mat&)image {
NSLog(#"image.type(): %d", image.type()); // got 24
// image.convertTo(image, CV_8UC3); // try to convert image type, but with or without this line result the same
NSLog(#"image.type(): %d", image.type()); // also 24
cv::Scalar colorScalar = cv::Scalar( 94, 206, 165 );
cv::Point center( image.size().width*0.5, image.size().height*0.5 );
cv::Size size( 100, 100 );
cv::ellipse( image, center, size, 0, 0, 360, colorScalar, 4, 8, 0 );
}
Eventually, the ellipse is still in white, not the desired green one.
Set alpha to 255 can fix this problem.
Scalar(94,206,165,255)
As mrgloom points correctly in the comment, it might be because of type of your image [ the Mat object where you want to draw, i.e Mat &img in ellipse() function].
cv::Scalar(94, 206, 165) is the desired green color for 8UC3 type images. Setting these values in 32FC3 image will result in white color.
you can use
src.convertTo(src, CV_8UC3);
Where CV_8UC3 means that you use 8 bits unsigned char and 3 color image representation.
More information you can find here OpenCV docs
after that your ellipse should be green, if it doesn't help post the whole code.
I was having similar problem and I have managed to fix it by first converting image to BGR. So in your case processImage function would look like as:
-(void)processImage:(Mat&)image
{
cvtColor(image, image, CV_RGBA2BGR);
cv::Scalar colorScalar = cv::Scalar( 94, 206, 165 );
cv::Point center( image.size().width*0.5, image.size().height*0.5 );
cv::Size size( 100, 100 );
cv::ellipse( image, center, size, 0, 0, 360, colorScalar, 4, 8, 0 );
}
The only line which I have included in your code is:
cvtColor(image, image, CV_RGBA2BGR);
If you also log channel, depth and type information in the above function as follows:
NSLog(#"Before conversion");
NSLog(#"channels %d", image.channels());
NSLog(#"depth %d", image.depth());
NSLog(#"type %d", image.type());
NSLog(#"element size %lu", image.elemSize());
cvtColor(image, image, CV_RGBA2BGR);
NSLog(#"After conversion");
NSLog(#"channels %d", image.channels());
NSLog(#"depth %d", image.depth());
NSLog(#"type %d", image.type());
NSLog(#"element size %lu", image.elemSize());
you will see before conversion:
channels 4
depth 0
type 24
element size 4
which I think is CV_8UC4 and after conversion it becomes:
channels 3
depth 0
type 16
element size 3
which is CV_8UC3.
I guess one of the reason why it does not work without cvtColor is that the opencv drawing functions don't support alpha transparency when the target image is 4-channel as mentioned in opencv documentation. So by converting CV_RGBA2BGR we take out alpha channel. However having said that I do not managed to get it work if I do:
cvtColor(image, image, CV_RGBA2RGB);
In this Red and Blue colors are inverted in the image. So although it seems to work but I am not sure if it is the actual reason.