Related
I want to calculate numbers of all white pixels within every polygon area efficiently.
Given some processes:
// some codes for reading gray image
// cv::Mat gray = cv::imread("gray.jpg");
// given polygons
// vector< vector<cv::Point> > polygons;
cv::Mat cropped;
cv::Mat mask = cv::Mat::zeros(gray.size(), CV_8UC1);
cv::fillPoly(mask, polygons, cv::Scalar(255));
cv::bitwise_and(gray, gray, cropped, mask);
cv::Mat binary;
cv::threshold(cropped, binary, 20, 255, CV_THRESH_BINARY);
So until now, we can get a image with multiple polygon areas(say we have 3 areas) which have white( with value 255) pixels. Then after some operations we expect to get a vector like:
// some efficient operations
// ...
vector<int> pixelNums;
The size of pixelNums should be same with polygons which is 3 here. And if we print them we may get some outputs like(the values are basically depended on the pre-processes):
index: 0; value: 120
index: 1; value: 1389
index: 2; value: 0
Here is my thought. Counting every pixels within every polygon area with help of cv::countNonZero, but I need to call it within a loop which I don't think it's a efficient way, isn't it?
vector<int> pixelNums;
for(auto polygon : polygons)
{
vector< vector<cv::Point> > temp_polygons;
temp_polygons.push_back(polygon);
cv::Mat cropped;
cv::Mat mask = cv::Mat::zeros(gray.size(), CV_8UC1);
cv::fillPoly(mask, temp_polygons, cv::Scalar(255));
cv::bitwise_and(gray, gray, cropped, mask);
cv::Mat binary;
cv::threshold(cropped, binary, 20, 255, CV_THRESH_BINARY);
pixelNums.push_back(cv::countNonZero(binary));
}
If you have some better ways, please kindly answer this post. Here I say better way is consuming as little time as you can just in cpu environment.
There are some minor improvements that can be done, but all of them combined should provide a decent speedup.
Compute the threshold only once
Make most operations on smaller images, using the bounding box of your polygon to get the region of interest
Avoid unneeded copies in the for loop, use const auto&
Example code:
#include <vector>
#include <opencv2/opencv.hpp>
int main()
{
// Your image
cv::Mat1b gray = cv::imread("path/to/image", cv::IMREAD_GRAYSCALE);
// Your polygons
std::vector<std::vector<cv::Point>> polygons
{
{ {15,120}, {45,200}, {160,160}, {140, 60} },
{ {10,10}, {15,30}, {50,25}, {40, 15} },
// etc...
};
// Compute the threshold just once
cv::Mat1b thresholded = gray > 20;
std::vector<int> pixelNums;
for (const auto& polygon : polygons)
{
// Get bbox of polygon
cv::Rect bbox = cv::boundingRect(polygon);
// Make a new (small) mask
cv::Mat1b mask(bbox.height, bbox.width, uchar(0));
cv::fillPoly(mask, std::vector<std::vector<cv::Point>>{polygon}, cv::Scalar(255), 8, 0, -bbox.tl());
// Get crop
cv::Mat1b cropped = thresholded(bbox) & mask;
// Compute the number of white pixels only on the crop
pixelNums.push_back(cv::countNonZero(cropped));
}
return 0;
}
I create a Bird-View-Image with the warpPerspective()-function like this:
warpPerspective(frame, result, H, result.size(), CV_WARP_INVERSE_MAP, BORDER_TRANSPARENT);
The result looks very good and also the border is transparent:
Bird-View-Image
Now I want to put this image on top of another image "out". I try doing this with the function warpAffine like this:
warpAffine(result, out, M, out.size(), CV_INTER_LINEAR, BORDER_TRANSPARENT);
I also converted "out" to a four channel image with alpha channel according to a question which was already asked on stackoverflow:
Convert Image
This is the code: cvtColor(out, out, CV_BGR2BGRA);
I expected to see the chessboard but not the gray background. But in fact, my result looks like this:
Result Image
What am I doing wrong? Do I forget something to do? Is there another way to solve my problem? Any help is appreciated :)
Thanks!
Best regards
DamBedEi
I hope there is a better way, but here it is something you could do:
Do warpaffine normally (without the transparency thing)
Find the contour that encloses the image warped
Use this contour for creating a mask (white values inside the image warped, blacks in the borders)
Use this mask for copy the image warped into the other image
Sample code:
// load images
cv::Mat image2 = cv::imread("lena.png");
cv::Mat image = cv::imread("IKnowOpencv.jpg");
cv::resize(image, image, image2.size());
// perform warp perspective
std::vector<cv::Point2f> prev;
prev.push_back(cv::Point2f(-30,-60));
prev.push_back(cv::Point2f(image.cols+50,-50));
prev.push_back(cv::Point2f(image.cols+100,image.rows+50));
prev.push_back(cv::Point2f(-50,image.rows+50 ));
std::vector<cv::Point2f> post;
post.push_back(cv::Point2f(0,0));
post.push_back(cv::Point2f(image.cols-1,0));
post.push_back(cv::Point2f(image.cols-1,image.rows-1));
post.push_back(cv::Point2f(0,image.rows-1));
cv::Mat homography = cv::findHomography(prev, post);
cv::Mat imageWarped;
cv::warpPerspective(image, imageWarped, homography, image.size());
// find external contour and create mask
std::vector<std::vector<cv::Point> > contours;
cv::Mat imageWarpedCloned = imageWarped.clone(); // clone the image because findContours will modify it
cv::cvtColor(imageWarpedCloned, imageWarpedCloned, CV_BGR2GRAY); //only if the image is BGR
cv::findContours (imageWarpedCloned, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
// create mask
cv::Mat mask = cv::Mat::zeros(image.size(), CV_8U);
cv::drawContours(mask, contours, 0, cv::Scalar(255), -1);
// copy warped image into image2 using the mask
cv::erode(mask, mask, cv::Mat()); // for avoid artefacts
imageWarped.copyTo(image2, mask); // copy the image using the mask
//show images
cv::imshow("imageWarpedCloned", imageWarpedCloned);
cv::imshow("warped", imageWarped);
cv::imshow("image2", image2);
cv::waitKey();
One of the easiest ways to approach this (not necessarily the most efficient) is to warp the image twice, but set the OpenCV constant boundary value to different values each time (i.e. zero the first time and 255 the second time). These constant values should be chosen towards the minimum and maximum values in the image.
Then it is easy to find a binary mask where the two warp values are close to equal.
More importantly, you can also create a transparency effect through simple algebra like the following:
new_image = np.float32((warp_const_255 - warp_const_0) *
preferred_bkg_img) / 255.0 + np.float32(warp_const_0)
The main reason I prefer this method is that openCV seems to interpolate smoothly down (or up) to the constant value at the image edges. A fully binary mask will pick up these dark or light fringe areas as artifacts. The above method acts more like true transparency and blends properly with the preferred background.
Here's a small test program that warps with transparent "border", then copies the warped image to a solid background.
int main()
{
cv::Mat input = cv::imread("../inputData/Lenna.png");
cv::Mat transparentInput, transparentWarped;
cv::cvtColor(input, transparentInput, CV_BGR2BGRA);
//transparentInput = input.clone();
// create sample transformation mat
cv::Mat M = cv::Mat::eye(2,3, CV_64FC1);
// as a sample, just scale down and translate a little:
M.at<double>(0,0) = 0.3;
M.at<double>(0,2) = 100;
M.at<double>(1,1) = 0.3;
M.at<double>(1,2) = 100;
// warp to same size with transparent border:
cv::warpAffine(transparentInput, transparentWarped, M, transparentInput.size(), CV_INTER_LINEAR, cv::BORDER_TRANSPARENT);
// NOW: merge image with background, here I use the original image as background:
cv::Mat background = input;
// create output buffer with same size as input
cv::Mat outputImage = input.clone();
for(int j=0; j<transparentWarped.rows; ++j)
for(int i=0; i<transparentWarped.cols; ++i)
{
cv::Scalar pixWarped = transparentWarped.at<cv::Vec4b>(j,i);
cv::Scalar pixBackground = background.at<cv::Vec3b>(j,i);
float transparency = pixWarped[3] / 255.0f; // pixel value: 0 (0.0f) = fully transparent, 255 (1.0f) = fully solid
outputImage.at<cv::Vec3b>(j,i)[0] = transparency * pixWarped[0] + (1.0f-transparency)*pixBackground[0];
outputImage.at<cv::Vec3b>(j,i)[1] = transparency * pixWarped[1] + (1.0f-transparency)*pixBackground[1];
outputImage.at<cv::Vec3b>(j,i)[2] = transparency * pixWarped[2] + (1.0f-transparency)*pixBackground[2];
}
cv::imshow("warped", outputImage);
cv::imshow("input", input);
cv::imwrite("../outputData/TransparentWarped.png", outputImage);
cv::waitKey(0);
return 0;
}
I use this as input:
and get this output:
which looks like ALPHA channel isn't set to ZERO by warpAffine but to something like 205...
But in general this is the way I would do it (unoptimized)
I have this code:
mapx.create(image.size(), CV_32FC1);
mapy.create(image.size(), CV_32FC1);
what is the values in the mapx and mapy after this? Are all data is zero?
what about this type of initialization:
cv::Mat mapx(image.size(), CV_32FC1);
Do I need explicitly set the value of each element to zero?
How can I set the value of each element to say -1?
Data after create should be undefined. In fact, your are just allocating memory.
cv::Mat mapx(image.size(), CV_32FC1);
is exactly as
cv::Mat1f mapx(image.size());
and
cv::Mat mapy;
mapy.create(image.size(), CV_32FC1);
You can assign an initial value (e.g. -1) like this:
cv::Mat1f(images.size(), -1.f);
Regarding you main question Should I initialize a cv::Mat, the answer is that in general you don't need to. From OpenCV doc:
Instead of writing:
Mat color;
...
Mat gray(color.rows, color.cols, color.depth());
cvtColor(color, gray, CV_BGR2GRAY);
you can simply write:
Mat color;
...
Mat gray;
cvtColor(color, gray, CV_BGR2GRAY);
You can see the opencv documentation :
Mat::zeros
Mat::ones
Mat A = Mat::ones(100, 100, CV_8U)*3; // make 100x100 matrix filled with 3.
http://docs.opencv.org/modules/core/doc/basic_structures.html#mat-zeros
How can I set the value of each element to say -1?
I think something like that :
Mat A = Mat::ones(100, 100, CV_8U)*-1; // make 100x100 matrix filled with -1.
I want to blend two images using multiband blending but I am not clear to the input parameter of this function:
void detail::Blender::prepare(const std::vector<Point>& corners, const std::vector<Size>& sizes)
In my case ,I just input two warped images with black gap, and with masks all white.(forgive me can not add pictures...)
And I set the two corners (0.0,0.0),because the warped images has been registered.
but my result is not good enough.with obvious seam in the result
can someone tell me why?How can I solve this problem?
I'm not sure what do you mean when you say "my result is not good enough". It's better to watch that result, but I'll try to guess. My main part of code, which makes panorama, looks like this:
void makePanorama(Rect bounding_box, vector<Mat> images, vector<Mat> homographies, vector<vector<Point>> corners) {
detail::MultiBandBlender blender;
blender.prepare(bounding_box);
Mat mask, bigImage, curImage;
for (int i = 0; i < (int)images.size(); ++i) {
warpPerspective(images[i], curImage, homographies[i],
bounding_box.size(), INTER_LINEAR, ORDER_TRANSPARENT);
mask = makeMask(curImage.size(), corners[i], homographies[i]);
blender.feed(curImage.clone(), mask, Point(0, 0));
}
blender.blend(bigImage, mask);
bigImage.convertTo(bigImage, (bigImage.type() / 8) * 8);
imshow("Result", bigImage);
waitKey();
}
So, prepare blender and then loop: warp image, make the mask after warped image and feed blender. At the end, turn this blender on and that's all. I met two problems, which influence on my result badly. May be you have one of them or both.
The first is type. My images had CV_16SC3, and after blending you need to convert blended image type into unsigned one. Like this
bigImage.convertTo(bigImage, (bigImage.type() / 8) * 8);
If you not, the result image would be gray.
The second is borders. In the beginning, my function makeMask was calculating non-black area of warped images. As a result, the one could see borders of the warped images on the blended image. The solution is to make mask smaller than non-black warped image area. So, my function makeMask is looks like this:
Mat makeMask(Size sz, vector<Point2f> imageCorners, Mat homorgaphy) {
Scalar white(255, 255, 255);
Mat mask = Mat::zeros(sz, CV_8U);
Point2f innerPoint;
vector<Point2f> transformedCorners(4);
perspectiveTransform(imageCorners, transformedCorners, homorgaphy);
// Calculate inner point
for (auto& point : transformedCorners)
innerPoint += point;
innerPoint.x /= 4;
innerPoint.y /= 4;
// Make indent for each corner
vector<Point> corners;
for (int ind = 0; ind < 4; ++ind) {
Point2f direction = innerPoint - transformedCorners[ind];
double normOfDirection = norm(direction);
corners[ind].x += settings.indent * direction.x / normOfDirection;
corners[ind].y += settings.indent * direction.y / normOfDirection;
}
// Draw borders
Point prevPoint = corners[3];
for (auto& point : corners) {
line(mask, prevPoint, point, white);
prevPoint = point;
}
// Fill with white
floodFill(mask, innerPoint, white);
return mask;
}
I took this pieces of code from my real code, so I could possibly forget to specify something. But I hope, the idea of how to work with MultiBandBlender is clear.
I have 2 vectors (p1 and p2) of point3f variables which represent 2 3D pointclouds. In order to match these two point clouds I want to use SVD to find a transformation for this. The problem is that SVD requires a matrix (p1*p2 transpose). My question is how do I convert a vector of size Y to a Yx3 matrix?
I tried cv::Mat p1Matrix(p1) but this gives me a row vector with two dimensions.I also found fitLine but I think this only works for 2D.
Thank you in advance.
How about something like:
cv::Mat p1copy(3, p1.size(), CV_32FC1);
for (size_t i = 0, end = p1.size(); i < end; ++i) {
p1copy.at<float>(0, i) = p1[i].x;
p1copy.at<float>(1, i) = p1[i].y;
p1copy.at<float>(2, i) = p1[i].z;
}
If this gives you the desired result, you can make the code faster by using a pointer instead of the rather slow at<>() function.
I use reshape function for convert vector of points to Mat.
vector<Point3f> P1,P2;
Point3f c1,c2;//center of two set
... //data association for two set
Mat A=Mat(P1).reshape(1).t();
Mat B=Mat(P2).reshape(1).t();
Mat AA,BB,CA,CB;
repeat(Mat(c1),1,P1.size(),CA);
repeat(Mat(c2),1,P2.size(),CB);
AA=A-CA;
BB=B-CB;
Mat H=AA*BB.t();
SVD svd(H);
Mat R_;
transpose(svd.u*svd.vt,R_);
if(determinant(R_)<0)
R_.at<float>(0,2)*=-1,R_.at<float>(1,2)*=-1,R_.at<float>(2,2)*=-1;
Mat t=Mat(c2)-R_*Mat(c1);