What I want to do is overlay stereo images together.
Given a sample set of stereo images I was able to display rectified images of them.
However, given a set of stereo images taken for the Microsoft Kinect, RGB and Infrared, I get really distorted images.
The original and rectified images can be found in the link:
http://img153.imageshack.us/img153/8021/calibration.png
I used the same code for the same set of images. I have tried multiple sets of Kinect "stereo" images and they all came out very distorted.
I am wondering what could be wrong?
The way I am displaying the images is:
I use cvStereoCalibrate() with these two as the last parameters: ...cvTermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, 1e-5), CV_CALIB_FIX_ASPECT_RATIO }
I then use cvStereoRectify and get mapx and mapy of the RGB camera using cvInitUndistortRectifyMap() then cvRemap and display the images.
I was wondering if the parameters of cvStereoCalibrate greatly affect the Kinect "stereo" images?
Thanks,
Tyro
I notice that one of the images has much less brightness and contrast in your samples. While it does find the corners, less brightness and contrast will result in a lot of error in the subpixel accuracy. I struggle a lot with rectification too and find that setting everything up perfectly (to the point of needing less rectification) is the only way to get really good results.
You use too small pattern for calibration.
Related
I have an issue with perspective transformation in OpenCV (I'm working with Qt).
I've attached two images, you can see the contour and how part of the image is lost.
What kind of transformation can I use? Note that the images may or may not have this curvature.
Image edited
Issue with curvature
thanks
That's the expected output: an image of the same size but transformed, as it can be seen in OpenCV's page.
If you want to preserve the whole image you have to embed the original one into a larger image (probably centered, with a background color that matches your problem, maybe black) and the apply the transformation to that image. You can later crop-to-fit and scale it to the original size.
I'm trying to remove foreground from two images, here's a sample pair of images:
As you can see, the Budweiser bottle is removed from the scene before the second shot is taken.
These photos were captured from a pinhole camera (iPhone), and, the tricky part is I'm hand-holding the camera, so it cannot be guaranteed that the images are perfectly aligned pixel by pixel, so a simple minus-threshold method will not work.
Then, I've decided to perform image registration using findHomography and warpPerspective from OpenCV, here's the result image:
This image is warped with the matrix I've got from findHomography, it kind of improved the alignment quality, but still not that aligned so I can use a simple way to remove the foreground.
So, finally, I decided to implement a "fuzzy-minus" algorithm: for every pixel in image1, I'll look through a 7x7 neighbour in image2 (a 7 by 7 kernel?), using the minimal difference in grayscale as the result of minus, and threshold the result into binary image, here's what I've got:
And the result is still not good. Notice the white wholes in the bottle, this is produced due to similar grayscale value of foreground and background. So I'm not sure what to do now.
I can think of two ways to solve the problem, the first is to get a better aligned pair of images, and simply minus the pairs; the second is to use a more robust way to extract the foreground.
Can anyone give me some advice on how to deal with this kind of problem? I believe there should be some state-of-art algorithms or processing pipelines, but after googling around, I get nothing.
I'm using OpenCV with C++, it would be fantastic if you can tell me how to do it with these tools in hand.
Big big thanks in advance!
The problem is not in your algorithm. You are having problem because the two scenes were not taken from exactly the same angle, as shown in the animation below. This slight difference highlight the edges in the subtraction.
You need a static camera in order to apply this approach.
I suggest using mathematical morphology on the mask that you got to get rid of the artifacts.
Try applying both opening and closing to get rid of the black and the white small regions.
Mathematical Morphology
Mathematical Morphology in opencv
The difference between the two picture is pretty huge, so you will need to use a large structure element, but I don't think you will be able to get rid of the shadow.
For the two large strips in the background, you may try to use a horizontally shaped structure element as well.
Edit
Is it possible to produce a grayscale image instead of a binary image? if yes, you may try to experiment with the hat method for the shadow, but I am not sure about this point.
This is what I got using two different structure elements for closing THEN opening
Mat mask = imread("mask.jpg",CV_LOAD_IMAGE_GRAYSCALE);
morphologyEx(mask,mask,MORPH_CLOSE,getStructuringElement(CV_SHAPE_ELLIPSE,Size(50,10)));
morphologyEx(mask,mask,MORPH_OPEN,getStructuringElement(CV_SHAPE_ELLIPSE,Size(10,50)));
imshow("open",mask);
imwrite("maskopenclose.jpg",mask);
I would suggest optical flow for alignment and OpenCV's background subtraction algorithm:
http://docs.opencv.org/trunk/doc/tutorials/video/background_subtraction/background_subtraction.html
I suggest that instead of using findHomography try using some of openCV's stereo correspondence functions: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
there is a sample code here: https://github.com/Itseez/opencv/blob/master/samples/cpp/stereo_calib.cpp
I am new in image processing and opencv. I have two images. I want to find correspond values in the image 2 with the image 1. and then show it. is there any function in opencv to find correspond values between images?
thanks in advance.
Mat corrVals;
bitwise_and(image2, image1>0, corrVals);
image1>0 will create temporary binary image with values 0 and 255. Than the only thing you need is to perform AND operation between pixels of your images, and store result somewhere. This is done by bitwise_and.
This is similar to approach suggested by #Mailerdaimon but uses much cheaper operations.
You can threshold you image1 such that all Values you want are 1 and all other are 0.
Than you multiply image1 with image2.
cv::multiply(image1, image2, result, scale, dtype)
This will return an image with all values greater than zero from image2 that are marked in image1.
It is hard to say without looking at your images. This is a well studied problem in computer vision and OpenCV contains several algorithms for this. The problem you're looking at can be very easy or very hard, depending on:
your images, are the normal images? just shapes? binary?
where on the images lie the corresponding pixels
how fast you need this to run
how much variation there is between images, is it exactly the same pixel value?
is there camera movement?
is there variation in illumination?
You can start by looking at stereo matching and optical flow inside OpenCV.
I am new to OpenCV. I would like to know if we can compare two images (one of the images made by photoshop i.e source image and the otherone will be taken from the camera) and find if they are same or not.
I tried to compare the images using template matching. It does not work. Can you tell me what are the other procedures which we can use for this kind of comparison?
Comparison of images can be done in different ways depending on which purpose you have in mind:
if you just want to compare whether two images are approximately equal (with a few
luminance differences), but with the same perspective and camera view, you can simply
compute a pixel-to-pixel squared difference, per color band. If the sum of squares over
the two images is smaller than a threshold the images match, otherwise not.
If one image is a black-white variant of the other, conversion of the color images is
needed (see e.g. http://www.johndcook.com/blog/2009/08/24/algorithms-convert-color-grayscale). Afterwarts simply perform the step above.
If one image is a subimage of the other, you need to perform registration of the two
images. This means determining the scale, possible rotation and XY-translation that is
necessary to lay the subimage on the larger image (for methods to register images, see:
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A. , Mutual-information-based registration of
medical images: a survey, IEEE Transactions on Medical Imaging, 2003, Volume 22, Issue 8,
pp. 986 – 1004)
If you have perspective differences, you need an algorithm for deskewing one image to
match the other as well as possible. For ways of doing deskewing look for example in
http://javaanpr.sourceforge.net/anpr.pdf from page 15 and onwards.
Good luck!
You should try SIFT. You apply SIFT to your marker (image saved in memory) and you get some descriptors (points robust to be recognized). Then you can use FAST algorithm with the camera frames in order to find the coprrespondent keypoints of the marker in the camera image.
You have many threads about this topic:
How to get a rectangle around the target object using the features extracted by SIFT in OpenCV
How to search the image for an object with SIFT and OpenCV?
OpenCV - Object matching using SURF descriptors and BruteForceMatcher
Good luck
How can I reconstruct an image from the stereo image pairs using OpenCV?
This is not necessarily an easy-to-solve problem. The thing is that both images store almost the same information, but from a slightly different perspective (angle and distance). So you have a perspective for each 2 of the stereo-optics. The only way to restore this is if(a) you knew what this perspective would be, e.g. a relative position-vector between both perspectives and the angle for both, you could create a mapping for a pixel in one of the images to the other.
The color of this (mapped) pixel ought to be the same, but as older stereo-optic-systems mapped to blue and red, you might have different values and thus have gained information doing this. Still, without these perspectives, you will need to correlate both pictures to each other and do quite complex image processing. I would suggest using scholar.google.com, unfortunately I failed to find anything useful, if you also can't find it there, start a phd ;)
Anyone who does know an algorithm of method to somehow restore such images, please let me know :) I am very curious about this as well.