OpenCV/C++ : warpPerspective producing overly-warped results - c++

Question

Or, perhaps, not warped enough...
So I'm trying to take an image and specify four corners - then move those four corners into a (near-)perfect square in the middle of the image.
UPDATE 1 (At bottom): 9/10/13 # 8:20PM GMT The math & matrix tags were added with this update. (If your read this update before 8:20, I must apologize, I gave you really bad info!)
I don't need it super-accurate, but my current results are very clearly not working, yet after looking at multiple other examples I cannot see what I've been doing wrong.
Here is my mock-up:
And through some magical process I obtain the coordinates. For this 640x480 mock-up the points are as follows:
Corners:
Upper-Left: 186, 87
Upper-Right: 471, 81
Lower-Left: 153, 350
Lower-Right: 500, 352
And the points I want to move the corners to are as follows:
Upper-Left: 176, 96
Upper-Right: 464, 96
Lower-Left: 176, 384
Lower-Right: 464, 384
Now, the end-goal here is to get the coordinates of the black dot-thing relative to the corners. I'm not making the box fill up the entire image because that point in the center could be outside the box given a different picture, so I want to keep enough "outside the box" room to generalize the process. I know I can get the point after it's moved, I'm just having trouble moving it correctly. My current warpPerspective attempts provide the following result:
Ok, so it looks like it's trying to fit things properly, but the corners didn't actually end up where we thought they would. The top-left is too far to the right. The Bottom-Left is too high, and the two on the right are both too far right and too close together. Well, ok... so lets try expanding the destination coordinates so it fills up the screen.
Just seems zoomed in? Are my coordinates somehow off? Do I need to feed it new coordinates? Source, destination, or both?
Here's my code: (This is obviously edited down to the key pieces of information, but if I missed something, please ask me to re-include it.)
Mat frame = inputPicture;
Point2f sourceCoords[4], destinationCoords[4];
// These values were pre-determined, and given above for this image.
sourceCoords[0].x = UpperLeft.X;
sourceCoords[0].y = UpperLeft.Y;
sourceCoords[1].x = UpperRight.X;
sourceCoords[1].y = UpperRight.Y;
sourceCoords[2].x = LowerLeft.X;
sourceCoords[2].y = LowerLeft.Y;
sourceCoords[3].x = LowerRight.X;
sourceCoords[3].y = LowerRight.Y;
// We need to make a square in the image. The 'if' is just in case the
// picture used is not longer left-to-right than it is top-to-bottom.
int top = 0;
int bottom = 0;
int left = 0;
int right = 0;
if (frame.cols >= frame.rows)
{
int longSideMidpoint = frame.cols/2.0;
int shortSideFifthpoint = frame.rows/5.0;
int shortSideTenthpoint = frame.rows/10.0;
top = shortSideFifthpoint;
bottom = shortSideFifthpoint*4;
left = longSideMidpoint - (3*shortSideTenthpoint);
right = longSideMidpoint + (3*shortSideTenthpoint);
}
else
{
int longSideMidpoint = frame.rows/2.0;
int shortSideFifthpoint = fFrame.cols/5.0;
int shortSideTenthpoint = frame.cols/10.0;
top = longSideMidpoint - (3*shortSideTenthpoint);
bottom = longSideMidpoint + (3*shortSideTenthpoint);
left = shortSideFifthpoint;
right = shortSideFifthpoint*4;
}
// This code was used instead when putting the destination coords on the edges.
//top = 0;
//bottom = frame.rows-1;
//left = 0;
//right = frame.cols-1;
destinationCoords[0].y = left; // UpperLeft
destinationCoords[0].x = top; // UL
destinationCoords[1].y = right; // UpperRight
destinationCoords[1].x = top; // UR
destinationCoords[2].y = left; // LowerLeft
destinationCoords[2].x = bottom; // LL
destinationCoords[3].y = right; // LowerRight
destinationCoords[3].x = bottom; // LR
Mat warp_matrix = cvCreateMat(3, 3, CV_32FC1);
warp_matrix = getPerspectiveTransform(sourceCoords, destinationCoords); // This seems to set the warp_matrix to 3x3 even if it isn't.
warpPerspective(frame, frame, warp_matrix, frame.size(), CV_INTER_LINEAR, 0);
IplImage *savePic = new IplImage(frame);
sprintf(fileName, "test/%s/photo%i-7_warp.bmp", startupTime, Count);
cvSaveImage(fileName, savePic);
delete savePic;
I've also tried using perspectiveTransform, but that lead to the error:
OpenCV Error: Assertion failed (scn + 1 == m.cols && (depth == CV_32F
|| depth == CV_64F)) in unknown function, file
C:\slace\builds\WinInstallerMegaPack\src\opencv\modules\core\src\matmul.cpp,
line 1926
which lead me to trying getHomography leading to THIS error instead:
OpenCV Error: Assertion failed (npoints >= 0 && points2.checkVector(2)
== npoints && points1.type()) in unknown function, file C:\slave\builds\WinInstallerMegaPack\src\opencv\modules\calib3d\src\fundam.cpp,
line 1074
(I checked - npoints IS greater than zero, because it equals points1.checkVector(2) - it's failing because points1.checkVector(2) and points2.checkVector(2) don't match - but I don't understand what checkVector(2) does. points1 and points2 are taken from the coordinates I tried feeding getHomography - which were the same coordinates as above.
Any idea how to get the output I'm looking for? I've been left confused for a while now. :/
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
UPDATE 1:
Ok, so I've figured out how it's supposed to calculate things.
getPerspectiveTransform is supposed to find a 3x3 matrix such that:
|M00 M10 M20| |X| |c*X'|
|M01 M11 M21| * |Y| = |c*Y'|
|M02 M12 M22| |1| | c |
Where MXX is the constant matrix, X & Y are the input coordinates, and X' & Y' are the output coordinates. This has to work for all four input/output coordinate combos. (The idea is simple, even if the math to get there may not be - I'm still not sure how they're supposed to get that matrix... input here would be appreciated - since I only need one actual coordinate I would not mind just bypassing getPerspectiveTransform and WarpPerspective entirely and just using a mathematical solution.)
After you've gotten the perspective transform matrix, WarpPerspective basically just moves each pixel's coordinates by multiplying:
|M00 M10 M20| |X|
|M01 M11 M21| * |Y|
|M02 M12 M22| |1|
for each coordinate. Then dividing cX' & cY' both by c (obviously). Then some averaging needs to be done since the results are unlikely to be perfect integers.
Ok, the basic idea is easy enough but here's the problem; getPerspectiveTransform does not seem to be working!
In my above example I modified the program to print out the matrix it was getting from getPerspectiveTransform. It gave me this:
|1.559647 0.043635 -37.808761|
|0.305521 1.174385 -50.688854|
|0.000915 0.000132 1.000000|
In my above example, I gave for the upper-left coordinate 186, 87 to be moved to 176, 96. Unfortunately when I multiply the above warp_matrix with the input coordinate (186, 87) I get not 176, 96 - but 217, 92! Which is the same result WarpPerspective gets. So at least we know WarpPerspective is working...

Answer 1

Um, Alex...
Your X & Y coordinates are reversed on both your Source and Destination coordinates.
In other words:
sourceCoords[0].x = UpperLeft.X;
sourceCoords[0].y = UpperLeft.Y;
should be
sourceCoords[0].x = UpperLeft.Y;
sourceCoords[0].y = UpperLeft.X;
and
destinationCoords[0].y = top;
destinationCoords[0].x = left;
should be
destinationCoords[0].y = left;
destinationCoords[0].x = top;
Jus' thought you'd like to know.