what I'm trying to do is transforming an image using an (Matlab) transformation matrix. It is the following 2D transformation matrix with a 3x3 dimension:
aaa bbb 0
ccc ddd 0
eee fff 1
I found a pretty good explanation here: how to transform an image with a transformation Matrix in OpenCv? but I'm not able to fill the 3x3 matrix (and apply it to the image). This is my code:
cv::Mat t(3,3,CV_64F,cvScalar(0.0));
t.at<double>(0, 0) = aaa;
t.at<double>(1, 0) = bbb;
t.at<double>(2, 0) = 0;
t.at<double>(0, 1) = ccc;
t.at<double>(1, 1) = ddd;
t.at<double>(2, 1) = 0;
t.at<double>(0, 2) = eee;
t.at<double>(1, 2) = fff;
t.at<double>(2, 2) = 1;
cv::Mat dest;
cv::Size size(imageToTransform.cols,imageToTransform.rows);
warpAffine(imageToTransform, outputImage, t, size, INTER_LINEAR, BORDER_CONSTANT);
imshow("outputImage", outputImage);
Causes the following error:
OpenCV Error: Assertion failed ((M0.type() == CV_32F || M0.type() == CV_64F) && M0.rows == 2 && M0.cols == 3)
Any idea whats wrong here?
The transformation matrix used in warp affine is an affine transformation matrix, which representation is as follows:
a11 a12 b1
a21 a22 b2
0 0 1
this means it is a 3x3 matrix indeed. However OpenCV doesn't care about the last row, since it will always be the same (look at the link before). Then, you can eliminate this row and have the input needed (2x3):
a11 a12 b1
a21 a22 b2
In getAffineTransform you can see why they want it like that (to have an homgeneous point mapped to the new 2d point and have it already in 2d and avoid extra multiplications).
You have a matrix of the type:
aaa bbb 0
ccc ddd 0
eee fff 1
This seems to be transpose somehow? since I do not know how did you generate the matrix, I can't tell for sure. Maybe you have transpose it and then assign it to opencv (or do it directly). The funny part is that (intentionally or otherwise) you had it in your c++ code the matrix transpose. Remember that OpenCV notation is usually (rows, columns) in most of the cases.
Your code should look like:
cv::Mat t(2,3,CV_64F,cvScalar(0.0));
t.at<double>(0, 0) = aaa;
t.at<double>(1, 0) = bbb;
t.at<double>(0, 1) = ccc;
t.at<double>(1, 1) = ddd;
t.at<double>(0, 2) = eee;
t.at<double>(1, 2) = fff;
cv::Mat dest;
cv::Size size(imageToTransform.cols,imageToTransform.rows);
warpAffine(imageToTransform, outputImage, t, size, INTER_LINEAR, BORDER_CONSTANT);
imshow("outputImage", outputImage);
I hope this helps you with the problem.
Related
As known, in OpenCV I can get affine or perspective transformation between 2 images:
M - affine transformation - by using estimateRigidTransform()
H - perspective (homography) transformation - by using FeatureDetector (SIFT, SURF, BRISK, FREAK, ...), then FlannBasedMatcher and findHomography()
Then I can do:
affine transformation - by using warpAffine(img_src, img_dst, M)
perspective transformation - by using warpPerspective(img_src, img_dst, H)
But if I have 3 or more images, and I already found:
affine: M1 (img1 -> img2), M2 (img2 -> img3)
perspective: H1 (img1 -> img2), H2 (img2 -> img3)
then can I get matix of transformation (img1 -> img3) by simply add two matrix?
of an affine transform: M3 = M1 + M2;
of an perspective transform: H3 = H1 + H2;
Or which of functions should I use for this?
No, you need to multiply the matrices to get the cascaded effect. I won't go into the math, but applying a transformation to coordinates is a matter of performing a matrix multiplication. If you are however curious as to know why that is, I refer you to this good Wikipedia article on cascading matrix transformations. Given a coordinate X and a transformation matrix M, you get the output coordinate Y by:
Y = M*X
Here I use * to refer to matrix multiplication as opposed to element-wise multiplication. What you have is a pair of transformation matrices which go from img1 to img2 then img2 to img3. You'll need to do the operation twice. So to go from img1 to img2 where X belongs to the coordinate space of img1, we have (assuming we're using the affine matrices):
Y1 = M1*X
Next, to go from img2 to img3, we have:
Y2 = M2*Y1 --> Y2 = M2*M1*X --> Y2 = M3*X --> M3 = M2*M1
Therefore, to get the desired chain effect, you need to create a new matrix such that M2 is multiplied by M1. Same as H2 and H1.
So define a new matrix such that:
cv::Mat M3 = M2*M1;
Similarly for your projective matrices, you can do:
cv::Mat H3 = H2*H1;
However, estimateRigidTransform (the output is M in your case) gives you a 2 x 3 matrix. One trick is to augment this matrix so that it becomes 3 x 3 where we add an additional row where it is all 0 except for the last element, which is set to 1. Therefore, you would have the last row such that it becomes [0 0 1]. You would do this for both matrices, multiply them, then extract just the first two rows into a new matrix to pipe into warpAffine. Therefore, do something like this:
// Create padded matrix for M1
cv::Mat M1new = cv::Mat(3,3,M1.type());
M1new.at<double>(0,0) = M1.at<double>(0,0);
M1new.at<double>(0,1) = M1.at<double>(0,1);
M1new.at<double>(0,2) = M1.at<double>(0,2);
M1new.at<double>(1,0) = M1.at<double>(1,0);
M1new.at<double>(1,1) = M1.at<double>(1,1);
M1new.at<double>(1,2) = M1.at<double>(1,2);
M1new.at<double>(2,0) = 0.0;
M1new.at<double>(2,1) = 0.0;
M1new.at<double>(2,2) = 1.0;
// Create padded matrix for M2
cv::Mat M2new = cv::Mat(3,3,M2.type());
M2new.at<double>(0,0) = M2.at<double>(0,0);
M2new.at<double>(0,1) = M2.at<double>(0,1);
M2new.at<double>(0,2) = M2.at<double>(0,2);
M2new.at<double>(1,0) = M2.at<double>(1,0);
M2new.at<double>(1,1) = M2.at<double>(1,1);
M2new.at<double>(1,2) = M2.at<double>(1,2);
M2new.at<double>(2,0) = 0.0;
M2new.at<double>(2,1) = 0.0;
M2new.at<double>(2,2) = 1.0;
// Multiply the two matrices together
cv::Mat M3temp = M2new*M1new;
// Extract out relevant rows and place into M3
cv::Mat M3 = cv::Mat(2, 3, M3temp.type());
M3.at<double>(0,0) = M3temp.at<double>(0,0);
M3.at<double>(0,1) = M3temp.at<double>(0,1);
M3.at<double>(0,2) = M3temp.at<double>(0,2);
M3.at<double>(1,0) = M3temp.at<double>(1,0);
M3.at<double>(1,1) = M3temp.at<double>(1,1);
M3.at<double>(1,2) = M3temp.at<double>(1,2);
When dealing with cv::Mat and the * operator, it is overloaded to specifically perform matrix multiplication.
You can then use M3 and H3 into warpAffine and warpPerspective respectively.
Hope this helps!
The example in the link below is using findHomography to get the transformation between two sets of points. I want to limit the degrees of freedom used in the transformation so want to replace findHomography with estimateRigidTransform.
http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html#feature-homography
Below I use estimateRigidTransform to get the transformation between the object and scene points. objPoints and scePoints are represented by vector <Point2f>.
Mat H = estimateRigidTransform(objPoints, scePoints, false);
Following the method used in the tutorial above, I want to transform the corner values using the transformation H. The tutorial uses perspectiveTransform with the 3x3 matrix returned by findHomography. With the rigid transform it only returns a 2x3 Matrix so this method cannot be used.
How would I transform the values of the corners, represented as vector <Point2f> with this 2x3 Matrix. I am just looking to perform the same functions as the tutorial but with less degrees of freedom for the transformation. I have looked at other methods such as warpAffine and getPerspectiveTransform as well, but so far not found a solution.
EDIT:
I have tried the suggestion from David Nilosek. Below I am adding the extra row to the matrix.
Mat row = (Mat_<double>(1,3) << 0, 0, 1);
H.push_back(row);
However this gives this error when using perspectiveTransform.
OpenCV Error: Assertion failed (mtype == type0 || (CV_MAT_CN(mtype) == CV_MAT_CN(type0) && ((1 << type0) & fixedDepthMask) != 0)) in create, file /Users/cgray/Downloads/opencv-2.4.6/modules/core/src/matrix.cpp, line 1486
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /Users/cgray/Downloads/opencv-2.4.6/modules/core/src/matrix.cpp:1486: error: (-215) mtype == type0 || (CV_MAT_CN(mtype) == CV_MAT_CN(type0) && ((1 << type0) & fixedDepthMask) != 0) in function create
ChronoTrigger suggested using warpAffine. I am calling the warpAffine method below, the size of 1 x 5 is the size of objCorners and sceCorners.
warpAffine(objCorners, sceCorners, H, Size(1,4));
This gives the error below, which suggests the wrong type. objCorners and sceCorners are vector <Point2f> representing the 4 corners. I thought warpAffine would accept Mat images which may explain the error.
OpenCV Error: Assertion failed ((M0.type() == CV_32F || M0.type() == CV_64F) && M0.rows == 2 && M0.cols == 3) in warpAffine, file /Users/cgray/Downloads/opencv-2.4.6/modules/imgproc/src/imgwarp.cpp, line 3280
I've done it this way in the past:
cv::Mat R = cv::estimateRigidTransform(p1,p2,false);
if(R.cols == 0)
{
continue;
}
cv::Mat H = cv::Mat(3,3,R.type());
H.at<double>(0,0) = R.at<double>(0,0);
H.at<double>(0,1) = R.at<double>(0,1);
H.at<double>(0,2) = R.at<double>(0,2);
H.at<double>(1,0) = R.at<double>(1,0);
H.at<double>(1,1) = R.at<double>(1,1);
H.at<double>(1,2) = R.at<double>(1,2);
H.at<double>(2,0) = 0.0;
H.at<double>(2,1) = 0.0;
H.at<double>(2,2) = 1.0;
cv::Mat warped;
cv::warpPerspective(img1,warped,H,img1.size());
which is the same as David Nilosek suggested: add a 0 0 1 row at the end of the matrix
this code warps the IMAGES with a rigid transformation.
I you want to warp/transform the points, you must use perspectiveTransform function with a 3x3 matrix ( http://docs.opencv.org/modules/core/doc/operations_on_arrays.html?highlight=perspectivetransform#perspectivetransform )
tutorial here:
http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html
or you can do it manually by looping over your vector and
cv::Point2f result;
result.x = point.x * R.at<double>(0,0) + point.y * R.at<double>(0,1) + R.at<double>(0,2);
result.y = point.x * R.at<double>(1,0) + point.y * R.at<double>(1,1) + R.at<double>(1,2);
hope that helps.
remark: didn't test the manual code, but should work. No PerspectiveTransform conversion needed there!
edit: this is the full (tested) code:
// points
std::vector<cv::Point2f> p1;
p1.push_back(cv::Point2f(0,0));
p1.push_back(cv::Point2f(1,0));
p1.push_back(cv::Point2f(0,1));
// simple translation from p1 for testing:
std::vector<cv::Point2f> p2;
p2.push_back(cv::Point2f(1,1));
p2.push_back(cv::Point2f(2,1));
p2.push_back(cv::Point2f(1,2));
cv::Mat R = cv::estimateRigidTransform(p1,p2,false);
// extend rigid transformation to use perspectiveTransform:
cv::Mat H = cv::Mat(3,3,R.type());
H.at<double>(0,0) = R.at<double>(0,0);
H.at<double>(0,1) = R.at<double>(0,1);
H.at<double>(0,2) = R.at<double>(0,2);
H.at<double>(1,0) = R.at<double>(1,0);
H.at<double>(1,1) = R.at<double>(1,1);
H.at<double>(1,2) = R.at<double>(1,2);
H.at<double>(2,0) = 0.0;
H.at<double>(2,1) = 0.0;
H.at<double>(2,2) = 1.0;
// compute perspectiveTransform on p1
std::vector<cv::Point2f> result;
cv::perspectiveTransform(p1,result,H);
for(unsigned int i=0; i<result.size(); ++i)
std::cout << result[i] << std::endl;
which gives output as expected:
[1, 1]
[2, 1]
[1, 2]
The affine transformations (the result of cv::estimateRigidTransform) are applied to the image with the function cv::warpAffine.
The 3x3 homography form of a rigid transform is:
a1 a2 b1
-a2 a3 b2
0 0 1
So when using estimateRigidTransform you could add [0 0 1] as the third row, if you want the 3x3 matrix.
I would like to implement some white balance algorithms from wiki
http://en.wikipedia.org/wiki/Color_balance
They are just some simple matrix manipulation
Do openCV offer any functions to do some multiplication
on a group of pixels like following?
example, 2 x 2, 3 channels Mat A =
0 0 0 1 1 1
2 2 2 3 3 3
3 x 3, 1 channels Mat B =
1 0 0
0 2 0
0 0 3
A x B = C and C =
0 0 0 1 2 3
2 4 6 3 6 9
I have wrote some generic functions to deal with pixel transformation
but I would prefer the build in function of openCV if it exist since the
functions of openCV may do some optimization
template<typename T, typename UnaryFunctor>
void transform_channel(cv::Mat &src, int channel, UnaryFunctor functor)
{
int const channels = src.channels();
if(channels == 1 && src.isContinuous()){
return transform_continuous_channel<T>(src, functor);
}
for(int row = 0; row != src.rows; ++row)
{
auto dst_ptr = get_pointer<T>(src, row, channel);
for(int col = 0; col != src.cols; ++col){
*dst_ptr = functor(*dst_ptr);
dst_ptr += channels;
}
}
}
You will notice that color-balance operations consist only of diagonal matrices, which corresponds to an element-wise multiplication by a scalar. Thus, the transform in your example would be:
image = image.mul(cv::Scalar(1,2,3));
for a 3-channel image. I do not know of a function to apply arbitrary pixel-wise matrix transformations.
Matrix multiplication is implementend in OpenCV with the * operator.
If you want to make it work though, you should reshape your RGB images into 1-channel matrices using cv::Mat::reshape() (documentation here).
Hint: cv::Mat::reshape() returns a reference to the new matrix without copying the data (unnecessary and slow). Hence, it is usually a good idea to use it like this:
cv::Mat someMatrix;
someMatrix = someMatrix.reshape(1);
Or this way (using another matrix variable):
cv::Mat rgbMatrix, myMatrix;
myMatrix = rgbMatrix.reshape(1);
Again, I emphasize that this does not copy the data, so you are not losing any memory.
I'm trying to use LogPolar transform to obtain the scale and the rotation angle from two images. Below are two 300x300 sample images. The first rectangle is 100x100, and the second rectangle is 150x150, rotated by 45 degree.
The algorithm:
Convert both images to LogPolar.
Find the translational shift using Phase Correlation.
Convert the translational shift to scale and rotation angle (how to do this?).
My code:
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/imgproc/imgproc_c.h>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
int main()
{
cv::Mat a = cv::imread("rect1.png", 0);
cv::Mat b = cv::imread("rect2.png", 0);
if (a.empty() || b.empty())
return -1;
cv::imshow("a", a);
cv::imshow("b", b);
cv::Mat pa = cv::Mat::zeros(a.size(), CV_8UC1);
cv::Mat pb = cv::Mat::zeros(b.size(), CV_8UC1);
IplImage ipl_a = a, ipl_pa = pa;
IplImage ipl_b = b, ipl_pb = pb;
cvLogPolar(&ipl_a, &ipl_pa, cvPoint2D32f(a.cols >> 1, a.rows >> 1), 40);
cvLogPolar(&ipl_b, &ipl_pb, cvPoint2D32f(b.cols >> 1, b.rows >> 1), 40);
cv::imshow("logpolar a", pa);
cv::imshow("logpolar b", pb);
cv::Mat pa_64f, pb_64f;
pa.convertTo(pa_64f, CV_64F);
pb.convertTo(pb_64f, CV_64F);
cv::Point2d pt = cv::phaseCorrelate(pa_64f, pb_64f);
std::cout << "Shift = " << pt
<< "Rotation = " << cv::format("%.2f", pt.y*180/(a.cols >> 1))
<< std::endl;
cv::waitKey(0);
return 0;
}
The log polar images:
For the sample image images above, the translational shift is (16.2986, 36.9105). I have successfully obtain the rotation angle, which is 44.29. But I have difficulty in calculating the scale. How to convert the given translational shift to obtain the scale?
You have two Images f1, f2 with f1(m, n) = f2(m/a , n/a) That is f1 is scaled by factor a
In logarithmic notation that is equivalent to f1(log m, log n) = f2(logm − log a, log n − log a) where log a is the shift in your phasecorrelated image.
Compare B. S. Reddy, B. N. Chatterji: An FFT-Based Technique for Translation, Rotation and
Scale-Invariant Image Registration, IEEE Transactions On Image Processing Vol. 5
No. 8, IEEE, 1996
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.4387&rep=rep1&type=pdf
here is python version
which tells
ir = abs(ifft2((f0 * f1.conjugate()) / r0))
i0, i1 = numpy.unravel_index(numpy.argmax(ir), ir.shape)
angle = 180.0 * i0 / ir.shape[0]
scale = log_base ** i1
The value for the scale factor is indeed exp(pt.y). However, since you used a "magnitude scale parameter" of 40 for the cvLogPolar function, you now need to divide pt.x by 40 to get the correct value for the displacement:
Scale = exp( pt.x / 40) = exp(16.2986 / 40) = 1.503
The value of the "magnitude scale parameter" for the cvLogPolar function does not affect the displacement produced by the rotation angle pt.x, because according to the math, it cancels out. For that reason, your formula for the rotation gives the correct value.
On another note, I believe the formula for the rotation should actually be:
Rotation = pt.y*360/(a.cols)
But, for some strange reason, the ">> 1" that you added is causing the result to be multiplied by 2 (which I believe you compensated for by multiplying by 180 instead of 360?) Remove it, and you'll see what I mean.
Also, ">>1" is causing a division by 2 in:
cvPoint2D32f(a.cols >> 1, a.rows >> 1)
If to set the center parameter of the cvLogPolar function to the center of the image (which is what you want):
cvPoint2D32f(a.cols/2, a.rows/2)
and
cvPoint2D32f(b.cols/2, b.rows/2)
then, you'll also get the correct value for the rotation (i.e. the same value that you got), and for the scale.
This thread was helpful in getting me started on rotation-invariant phase correlation, so I hope my input will help resolve any lingering issues.
We aim to calculate the scale and rotation (which is incorrectly calculated in the code). Let's start by gathering the equations from the logPolar docs. There they state the following:
(1) I = (dx,dy) = (x-center.x, y-center.y)
(2) rho = M * ln(magnitude(I))
(3) phi = Ky * angle(I)_0..360
Note: rho is pt.x and phi is pt.y in the code above
We also know that
(4) M = src.cols/ln(maxRadius)
(5) Ky = src.rows/360
First, let's solve for scale. Solving for magnitude(I) (i.e. scale) in equation 2, we get
(6) magnitude(I) = scale = exp(rho/M)
Then we substitute for M and simplify to get
(7) magnitude(I) = scale = exp(rho*ln(maxRadius)/src.cols) = pow(maxRadius, rho/src.cols)
Now let's solve for rotation. Solving for angle(I) (i.e. rotation) in equation 3, we get
(8) angle(I) = rotation = phi/Ky
Then we substitute for Ky and simplify to get
(9) angle(I) = rotation = phi*360/src.rows
So, scale and rotation can be calculated using equations 7 and 9, respectively. It might be worth noting that you should use equation 4 for calculation M and Point2f center( (float)a.cols/2, (float)a.rows/2 ) for calculating center as opposed to what is in the code above. There are good bits of info in this logpolar example opencv code.
From the values by phase correlation, the coordinates are rectangular coordinates hence (16.2986, 36.9105) are (x,y). The scale is calculated as
scale = log((x^2 + y^ 2)^0.5) which is approximately 1.6(near to 1.5).
When we calculate angle by using formulae theta = arctan(y/x) = 66(approx).
The theta value is way of the real value(45 in this case).
This question is on the OpenCV functions findHomography, getPerspectiveTransform & getAffineTransform
What is the difference between findHomography and getPerspectiveTransform?. My understanding from the documentation is that getPerspectiveTransform computes the transform using 4 correspondences (which is the minimum required to compute a homography/perspective transform) where as findHomography computes the transform even if you provide more than 4 correspondencies (presumably using something like a least squares method?).
Is this correct?
(In which case the only reason OpenCV still continues to support getPerspectiveTransform should be legacy? )
My next concern is that I want to know if there is an equivalent to findHomography for computing an Affine transformation? i.e. a function which uses a least squares or an equivalent robust method to compute and affine transformation.
According to the documentation getAffineTransform takes in only 3 correspondences (which is the min required to compute an affine transform).
Best,
Q #1: Right, the findHomography tries to find the best transform between two sets of points. It uses something smarter than least squares, called RANSAC, which has the ability to reject outliers - if at least 50% + 1 of your data points are OK, RANSAC will do its best to find them, and build a reliable transform.
The getPerspectiveTransform has a lot of useful reasons to stay - it is the base for findHomography, and it is useful in many situations where you only have 4 points, and you know they are the correct ones. The findHomography is usually used with sets of points detected automatically - you can find many of them, but with low confidence. getPerspectiveTransform is good when you kn ow for sure 4 corners - like manual marking, or automatic detection of a rectangle.
Q #2 There is no equivalent for affine transforms. You can use findHomography, because affine transforms are a subset of homographies.
I concur with everything #vasile has written. I just want to add some observations:
getPerspectiveTransform() and getAffineTransform() are meant to work on 4 or 3 points (respectively), that are known to be correct correspondences. On real-life images taken with a real camera, you can never get correspondences that accurate, not with automatic nor manual marking of the corresponding points.
There are always outliers. Just look at the simple case of wanting to fit a curve through points (e.g. take a generative equation with noise y1 = f(x) = 3.12x + gauss_noise or y2 = g(x) = 0.1x^2 + 3.1x + gauss_noise): it will be much more easier to find a good quadratic function to estimate the points in both cases, than a good linear one. Quadratic might be an overkill, but in most cases will not be (after removing outliers), and if you want to fit a straight line there you better be mightily sure that is the right model, otherwise you are going to get unusable results.
That said, if you are mightily sure that affine transform is the right one, here's a suggestion:
use findHomography, that has RANSAC incorporated in to the functionality, to get rid of the outliers and get an initial estimate of the image transformation
select 3 correct matches-correspondances (that fit with the homography found), or reproject 3 points from the 1st image to the 2nd (using the homography)
use those 3 matches (that are as close to correct as you can get) in getAffineTransform()
wrap all of that in your own findAffine() if you want - and voila!
Re Q#2, estimateRigidTransform is the oversampled equivalent of getAffineTransform. I don't know if it was in OCV when this was first posted, but it's available in 2.4.
There is an easy solution for the finding the Affine transform for the system of over-determined equations.
Note that in general an Affine transform finds a solution to the over-determined system of linear equations Ax=B by using a pseudo-inverse or a similar technique, so
x = (A At )-1 At B
Moreover, this is handled in the core openCV functionality by a simple call to solve(A, B, X).
Familiarize yourself with the code of Affine transform in opencv/modules/imgproc/src/imgwarp.cpp: it really does just two things:
a. rearranges inputs to create a system Ax=B;
b. then calls solve(A, B, X);
NOTE: ignore the function comments in the openCV code - they are confusing and don’t reflect the actual ordering of the elements in the matrices. If you are solving [u, v]’= Affine * [x, y, 1] the rearrangement is:
x1 y1 1 0 0 1
0 0 0 x1 y1 1
x2 y2 1 0 0 1
A = 0 0 0 x2 y2 1
x3 y3 1 0 0 1
0 0 0 x3 y3 1
X = [Affine11, Affine12, Affine13, Affine21, Affine22, Affine23]’
u1 v1
B = u2 v2
u3 v3
All you need to do is to add more points. To make Solve(A, B, X) work on over-determined system add DECOMP_SVD parameter. To see the powerpoint slides on the topic, use this link. If you’d like to learn more about the pseudo-inverse in the context of computer vision, the best source is: ComputerVision, see chapter 15 and appendix C.
If you are still unsure how to add more points see my code below:
// extension for n points;
cv::Mat getAffineTransformOverdetermined( const Point2f src[], const Point2f dst[], int n )
{
Mat M(2, 3, CV_64F), X(6, 1, CV_64F, M.data); // output
double* a = (double*)malloc(12*n*sizeof(double));
double* b = (double*)malloc(2*n*sizeof(double));
Mat A(2*n, 6, CV_64F, a), B(2*n, 1, CV_64F, b); // input
for( int i = 0; i < n; i++ )
{
int j = i*12; // 2 equations (in x, y) with 6 members: skip 12 elements
int k = i*12+6; // second equation: skip extra 6 elements
a[j] = a[k+3] = src[i].x;
a[j+1] = a[k+4] = src[i].y;
a[j+2] = a[k+5] = 1;
a[j+3] = a[j+4] = a[j+5] = 0;
a[k] = a[k+1] = a[k+2] = 0;
b[i*2] = dst[i].x;
b[i*2+1] = dst[i].y;
}
solve( A, B, X, DECOMP_SVD );
delete a;
delete b;
return M;
}
// call original transform
vector<Point2f> src(3);
vector<Point2f> dst(3);
src[0] = Point2f(0.0, 0.0);src[1] = Point2f(1.0, 0.0);src[2] = Point2f(0.0, 1.0);
dst[0] = Point2f(0.0, 0.0);dst[1] = Point2f(1.0, 0.0);dst[2] = Point2f(0.0, 1.0);
Mat M = getAffineTransform(Mat(src), Mat(dst));
cout<<M<<endl;
// call new transform
src.resize(4); src[3] = Point2f(22, 2);
dst.resize(4); dst[3] = Point2f(22, 2);
Mat M2 = getAffineTransformOverdetermined(src.data(), dst.data(), src.size());
cout<<M2<<endl;
getAffineTransform:affine transform is combination of translation, scale, shear, and rotation
https://www.mathworks.com/discovery/affine-transformation.html
https://www.tutorialspoint.com/computer_graphics/2d_transformation.htm
getPerspectiveTransform：perspective transform is project mapping
enter image description here