I'm doing a mosaic from a video in Opencv. I'm using this example for stitching the frames of the videos: http://docs.opencv.org/doc/tutorials/features2d/feature_detection/feature_detection.html. At the end I'm doing this for merging the new frame with the stitch created at the passed iteration:
Mat H = findHomography(obj, scene, CV_RANSAC);
static Mat rImg;
warpPerspective(vImg[0], rImg, H, Size(vImg[0].cols, vImg[0].rows), INTER_NEAREST);//(vImg[0], rImg, H, Size(vImg[0].cols * 2, vImg[0].rows * 2), CV_INTER_LINEAR);
static Mat final_img(Size(rImg.cols*2, rImg.rows*2), CV_8UC3);
static Mat roi1(final_img, Rect(0, 0, vImg[1].cols, vImg[1].rows));
Mat roi2(final_img, Rect(0, 0, rImg.cols, rImg.rows));
rImg.copyTo(roi2);
vImg[1].copyTo(roi1);
imwrite("stitch.jpg", final_img);
vImg[0] = final_img;
So here's my problem: obviously the stitch becomes larger at each iteration, so how can I resize it to make it fit in the final_img image?
EDIT
Sorry but I had to remove images
For the second question, what you observe is an error in the homography that was estimated. This may come either from:
drift (if you chain homographies along the sequence), ie, small errors that accumulate and become large after dozens of frames
or (more likely) because your reference image is too old with respect to your new image, and they exhibit too few matching points to give an accurate homography, but yet enough to find one that passes the quality test inside cv::findHomography().
For your first question, you need to add some code that keeps track of the current bounds of the stitched image in a fixed coordinate frame.
I would suggest to choose the coordinates linked with the first image.
Then, when you stitch a new image, what you do really is to project this image onto this coordinate frame.
You can compute first for example the projected coordinates of the 4 corners of the incoming frame, test if they fit into the current stitching result, copy it to a new (bigger) image if necessary, then proceed with stitching the new image.
Related
As we all know,we can use the function cv::getOptimalNewCameraMatrix() with alpha = 1 to get a new CameraMatrix. Then we use the function cv::undistort() with the new CameraMatrix can get the image after dedistortion. However, I find the image after distortion is as large as the original image and some part of the image after distortion covered by black.
So my question is :Does this mean that the original image pixel is lost? and is there any way to avoid pixel loss or get the image whose size larger than origin image with opencv?
cv::Mat NewKMatrixLeft = cv::getOptimalNewCameraMatrix(KMatrixLeft,DistMatrixLeft ,cv::Size(image.cols,image.rows),1);
cv::undistort(image, show_image, KMatrixLeft, DistMatrixLeft,NewKMatrixLeft);
The size of image and show_image are both 640*480,however from my point of view,the size of image after distortion should be larger than 640*480 because some part of it is meaningless.
Thanks!
In order to correct distortion, you basically have to reverse the process that caused the initial distortion. This implies that pixels are stretched and squashed along various directions to correct the distortion. In some cases, this would move the pixels away from the image edge. In OpenCV, this is handled by inserting black pixels. There is nothing wrong with this approach. You can then choose how to crop it to remove the black pixels at the edges.
I implemented stitching algorithm in OpenCV. What I want to do now is to find certain point from one of the original images in the final stitched image. I can find this point in image after it is warped, but then all images blend together and I want to find coordinates of this point.
For example, I have this image with size 499x369:
Now i warp this image and i want to find point(for example 255x185)
This warped image has size 475x372 and selected point from original image is at 232x184
After this, I use OpenCV's MultiBand blending function and i blend 2 warped images together
Is there some way that i could find that point after blending them together?
CODE:
Here i warp the images:
Ptr<detail::RotationWarper> warp = warper_->create(float(scale));
warp->warp(img, K, rotationMatrix, INTER_LINEAR, BORDER_REFLECT, img_warped);
//now I can find where a certain point in warped image is
warp->warpPoint(Point, K, rotationMatrix);
//now I blend the images
blender_->prepare(corners, sizes);
blender_->feed(img_warped_s, mask_warped, corners[img_idx]);
blender_->blend(result, result_mask);
So result is the last image that i posted and i want to find that point in that image.
First of all: I'm new to opencv :-)
I want to perform a vignetting correction on a 24bit RGB Image. I used an area scan camera as a line camera and put together an image from 1780x2 px parts to get an complete image with 1780x3000 px. Because of the vignetting, i made a white reference picture with 1780x2 px to calculate a LUT (with correction factor in it) for the vignetting removal. Here is my code idea:
Mat white = imread("WHITE_REF_2L.bmp", 0);
Mat lut(2, 1780, CV_8UC3, Scalar(0));
lut = 255 / white;
imwrite("lut_test.bmp", lut*white);
As i understood, what the second last line will (hopefully) do, is to divide 255 with every intensity value of every channel and store this in the lut matrice.
I want to use that lut then to to calculate the “real” (not distorted) intensity
level of each pixel by multiplying every element of the src img with every element of the lut matrice.
obviously its not working how i want to do it, i get a memory exception.
Can anybody help me with that problem?
edit: i'm using opencv 3.1.0 and i solved the problem like this:
// read white reference image
Mat white = imread("WHITE_REF_2L_D.bmp", IMREAD_COLOR);
white.convertTo(white, CV_32FC3);
// calculate LUT with vignetting correction factors
Mat vLUT(2, 1780, CV_32FC3, Scalar(0.0f));
divide(240.0f, white, vLUT);
of course that's not optimal, i will read in more white references and calculate the mean value to optimize.
Here's the 2 lines white reference, you can see the shadows at the image borders i want to correct
when i multiply vLUT with the white reference i obviously get a homogenous image as the result.
thanks, maybe this can help anyone else ;)
I am detecting and matching features of a pair of images, using a typical detector-descriptor-matcher combination and then findHomography to produce a transformation matrix.
After this, I want the two images to be overlapped (the second one (imgTrain) over the first one (imgQuery), so I warp the second image using the transformation matrix using:
cv::Mat imgQuery, imgTrain;
...
TRANSFORMATION_MATRIX = cv::findHomography(...)
...
cv::Mat imgTrainWarped;
cv::warpPerspective(imgTrain, imgTrainWarped, TRANSFORMATION_MATRIX, imgTrain.size());
From here on, I don't know how to produce an image that contains the original imgQuery with the warped imgTrainWarped on it.
I consider two scenarios:
1) One where the size of the final image is the size of imgQuery
2) One where the size of the final image is big enough to accommodate both imgQuery and imgTrainWarped, since they overlap only partially, not completely. I understand this second case might have black/blank space somewhere around the images.
You should warp to a destination matrix that has the same dimensions as imgQuery after that, loop over the whole warped image and copy pixel to the first image, but only if the warped image actually holds a warped pixel. That is most easily done by warping an additional mask. Please try this:
cv::Mat imgMask = cv::Mat(imgTrain.size(), CV_8UC1, cv::Scalar(255));
cv::Mat imgMaskWarped;
cv::warpPerspective(imgMask , imgMaskWarped, TRANSFORMATION_MATRIX, imgQuery.size());
cv::Mat imgTrainWarped;
cv::warpPerspective(imgTrain, imgTrainWarped, TRANSFORMATION_MATRIX, imgQuery.size());
// now copy only masked pixel:
imgTrainWarped.copyTo(imgQuery, imgMaskWarped);
please try and tell whether this is ok and solves scenario 1. For scenario 2 you would test how big the image must be before warping (by using the transformation) and copy both images to a destination image big enough.
Are you trying to create a panoramic image out of two overlapping pictures taken from the same viewpoint in different directions? If so, I am concerned about the "the second one over the first one" part. The correct way to stitch the panorama together is to cut both images off down the central line (symmetry axis) of the overlapping part, and not to add a part of one image to the (whole) other one.
Accepted answer works but could be done easier with using BORDER_TRANSPARENT:
cv::warpPerspective(imgTrain, imgQuery, TRANSFORMATION_MATRIX, imgQuery.size(), INTER_LINEAR, BORDER_TRANSPARENT);
When using BORDER_TRANSPARENT the source pixel of imgQuery remains untouched.
For OpenCV 4 INTER_LINEAR and BORDER_TRANSPARENT
can be resolved by using
cv::InterpolationFlags::INTER_LINEAR, cv::BorderTypes::BORDER_TRANSPARENT, e.g.
cv::warpPerspective(imgTrain, imgQuery, TRANSFORMATION_MATRIX, imgQuery.size(), cv::InterpolationFlags::INTER_LINEAR, cv::BorderTypes::BORDER_TRANSPARENT);
I am working on a project which merges the torn pieces of a paper into one image.
Till now I have done the pre-processing to find the contours and find the matched pieces.
I am getting 2 images which are matched but are in separate 2 cv::Mat objects.
Now, I need to merge these 2 images into 1 image. One way of doing so is to copy pixel by pixel of both the images into new image, but this will be very time expensive and processor expensive.
I need a OpenCV library function or workaround with similar function to do the job.
You can use copyTo function of openCV. For example assume that piece1 and piece2 are images of two pieces of paper:
Mat twoPieces (piece1.rows, 2*piece1.cols, piece1.type());
piece1.copyTo (twoPieces(Rect(0, 0, piece1.cols, piece1.rows)));
piece2.copyTo (twoPieces(Rect(piece1.cols, 0, piece2.cols, piece2.rows)));
I assumed here that the images are of the same size but if they not you can update the code. Also the example above showed how to copy whole image. You can define mask and copy only the places with pieces of paper. To draw mask you can use polylines.
Mat mask1(piece1.size(), CV_8U, Scalar(0));
polylines(mask1, piece1Contours, true, 255, CV_FILLED);
piece1.copyTo (twoPieces(Rect(0, 0, piece1.cols, piece1.rows)), mask1);
And of course if you know the shift between the pieces of paper you can change the point of origin in Rect so that both pieces will be adjacent to each other.
Mat mask2(piece2.size(), CV_8U, Scalar(0));
polylines(mask2, piece2Contours, true, 255, CV_FILLED);
piece2.copyTo (twoPieces(Rect(dx, 0, piece2.cols, piece2.rows)), mask2);
Once again I simplified the example assuming that the shift is only in X direction. It can be done with shift in both X and Y direction but it will require more calculations, like estimation of bounding box of your polygons.