Why equate an identity matrix to a transformation matrix in OpenCV - c++

In opencv and C++, if I have previously found a transformation matrix between 2 images, why do I need to do this
Mat affineTransform=getAffineTransform(coordinates_1.data(),coordinates_2.data()) ;
Mat perspectiveTransform=Mat::eye(3,3,CV_64FC1);
for(unsigned int y=0; y<2; ++y){
for(unsigned int x=0; x<3; ++x){
perspectiveTransform.at<double>(y,x) = affineTransform.at<double>(y,x);
}
instead of applying the transformation matrix to an image directly. I understand the meaning of Mat::eye() but why go through all this?
Note: originalTranformationMatrix is a Mat object and the found transformation matrix is a 3 x 3 matrix

Affine transform has following form:
(a, b, c)
(d, e, f)
It transform point (x,y) in following way:
x_new = a*x + b*y + c;
y_new = d*x + e*y + f;
Perspective transform has following form:
(a, b, c)
(d, e, f)
(g, h, 1)
It transform point (x,y) in following way:
z = g*x + h*y + 1;
x_new = (a*x + b*y + c)/z;
y_new = (d*x + e*y + f)/z;
That means that if you want to define perspective transform that does only affine transformation, it should be:
(a, b, c)
(d, e, f)
(0, 0, 1)
And this is exactly what your code does. First it creates matrix:
(1, 0, 0)
(0, 1, 0)
(0, 0, 1)
And than replaces first two rows with rows of affine transformation. By the way, it can be done in cleaner way without loops:
perspectiveTransform(Rect(0,0,3,2)) = affineTransform.clone();

Related

Intersection of ray and ellipsoid

Yes, I know that it is a popular problem. But I found nowhere the full clear implementing code without using OpenGL classes or a lot of headers files.
Okay, the math solution is to transfer ellipsoid to sphere. Then find intersections dots (if they exist of course) and make inverse transformation. Because affine transformation respect intersection.
But I have difficulties when trying to implement this.
I tried something for sphere but it is completely incorrect.
double CountDelta(Point X, Point Y, Sphere S)
{
double a = 0.0;
for(int i = 0; i < 3; i++){
a += (Y._coordinates[i] - X._coordinates[i]) * (Y._coordinates[i] - X._coordinates[i]);
}
double b = 0.0;
for(int i = 0; i < 3; i++)
b += (Y._coordinates[i] - X._coordinates[i]) * (X._coordinates[i] - S._coordinates[i]);
b *= 2;
double c = - S.r * S.r;
for(int i = 0; i < 3; i++)
c += (X._coordinates[i] - S._coordinates[i]) * (X._coordinates[i] - S._coordinates[i]);
return b * b - 4 * a * c;
}
Let I have start point P = (Px, Py, Pz), direction V = (Vx, Vy, Vz), ellipsoid = (Ex, Ey, Ec) and (a, b, c). How to construct clear code?
Let a line from P to P + D intersecting a sphere of center C and radius R.
WLOG, C can be the origin and R unit (otherwise translate by -C and scale by 1/R). Now using the parametric equation of the line and the implicit equation of the sphere,
(Px + t Dx)² + (Py + t Dy)² + (Pz + t Dz)² = 1
or
(Dx² + Dy² + Dz²) t² + 2 (Dx Px + Dy Py + Dz Pz) t + Px² + Py² + Pz² - 1 = 0
(Vectorially, D² t² + 2 D P t + P² - 1 = 0 and t = (- D P ±√((D P)² - D²(P² - 1))) / D².)
Solve this quadratic equation for t and get the two intersections as P + t D. (Don't forget to invert the initial transformations.)
For the ellipsoid, you can either plug the parametric equation of the line directly into the implicit equation of the conic, or reduce the conic (and the points simultaneously) and plug in the reduced equation.

Using Eigen matrix multiplication for efficient pointcloud calculation

I am attempting to calculate a pointcloud from an opencv Mat depth image and an intrinsic matrix. Currently I do it as follows (k matrix values were extracted into fx,fy,cx,cy earlier):
for(int i=0; i<depth.rows; i++)
{
const float* row_ptr = depth.ptr<float>(i);
for(int j=0; j<depth.cols; j++)
{
// Only add valid depth points
if(row_ptr[j] != 0)
{
const float x = ((j - cx) * row_ptr[j]/focal_length_x);
const float y = ((i - cy) * row_ptr[j]/focal_length_y);
pointcloud[cnt] = pcl::PointXYZ(x/1000, y/1000, row_ptr[j]/1000);
cnt++;
}
}
}
However I am wondering is it possible to turn this into a matmul operation and use eigen for better performance. I am aware that:
[x, y, z] = depth value * inv(k) * [u, v, 1] with
[fx, 0, cx]
k = [0, fy, cy]
[0, 0, 1]
How would I go about turning this into a full matrix multiplication, my depth image is 1280x800, obviously directly multiplying a 1280x800 matrix with a 3x3 with a 3x1 wont work so what ways can this be done if any?

Normalizing 2D lines in Eigen C++

A line in the 2D plane can be represented with the implicit equation
f(x,y) = a*x + b*y + c = 0
= dot((a,b,c),(x,y,1))
If a^2 + b^2 = 1, then f is considered normalized and f(x,y) gives you the Euclidean (signed) distance to the line.
Say you are given a 3xK matrix (in Eigen) where each column represents a line:
Eigen::Matrix<float,3,Eigen::Dynamic> lines;
and you wish to normalize all K lines. Currently I do this a follows:
for (size_t i = 0; i < K; i++) { // for each column
const float s = lines.block(0,i,2,1).norm(); // s = sqrt(a^2 + b^2)
lines.col(i) /= s; // (a, b, c) /= s
}
I know there must be a more clever and efficient method for this that does not require looping. Any ideas?
EDIT: The following turns out being slower for optimized code... hmmm..
Eigen::VectorXf scales = lines.block(0,0,2,K).colwise().norm().cwiseInverse()
lines *= scales.asDiagonal()
I assume that this as something to do with creating KxK matrix scales.asDiagonal().
P.S. I could use Eigen::Hyperplane somehow, but the docs seem little opaque.

How do I estimate a cube corner from a projected square?

I've got the coordinates of a square's corners projected onto an image. I want to know where I'd find the corners of a cube if they were projected onto the image.
More specifically, assuming a square with sides of length 1. It's at the origin with coordinates (x, y, z) of (0, 0, 0), (1, 0, 0), (0, 1, 0) and (1, 1, 0). I know the corresponding projected locations (u, v) for those square corners. how do I find the (u, v) for the (x, y, z) point (0, 0, 1) ?
I'm really just looking for an equations that will give u001 and v001 in terms of the scalar values:
u000 v000
u100 v100
u010 v010
u110 v110
I think the transformation matrix in homogenous coordinates from (x, y, z, 1) to (u, v, w) will look like:
a b c d
e f g h
i j k 1
But I can't figure out how to find c, g, or k without a z point.
Doing some algebra:
d = u_00
h = v_00
a = u_10 * (i+1) - u_00
b = u_01 * (j+1) - u_00
e = v_10 * (i+1) - v_00
f = v_01 * (j+1) - v_00

Merge two Mat-Objects in OpenCV

I wrote a little function in order to be able to "stick" the pixels of an image on top of another, but it somehow doesnt work: While the "shapes" of my "sticked" image is right, the colors are not.
The example flower is the first image, and as a second image I had a black trapezoid png. As you can see, there are multiple problems:
1. The colors are shown weirdly. Actually there are no colors, just greyscale, and some weird stripes as overlay.
2. Alpha values are not respected. The white part of the overlay image is transparent in the png.
Here is my code:
void mergeMats(Mat mat1, Mat mat2, int x, int y){
//unsigned char * pixelPtr = (unsigned char *)mat2.data;
//unsigned char * pixelPtr2 = (unsigned char *)mat1.data;
//int cn = mat2.channels();
//int cn2 = mat2.channels();
//Scalar_<unsigned char> bgrPixel;
for (int i = 0; i < mat2.cols; i++){
for (int j = 0; j < mat2.rows; j++){
if (x + i < mat1.cols && y + j < mat1.rows){
Vec3b &intensity = mat1.at<Vec3b>(j+y, i+x);
Vec3b intensity2 = mat2.at<Vec3b>(j, i);
for (int k = 0; k < mat1.channels(); k++) {
intensity.val[k] = saturate_cast<uchar>(intensity2.val[k]);
}
//pixelPtr2[(i + x)*mat1.cols*cn2 + (j + y)*cn2 + 0] = pixelPtr[(i + x)*mat2.cols*cn + (j + y)*cn + 0];
//pixelPtr2[(i + x)*mat1.cols*cn2 + (j + y)*cn2 + 1] = pixelPtr[(i + x)*mat2.cols*cn + (j + y)*cn + 1];
//pixelPtr2[(i + x)*mat1.cols*cn2 + (j + y)*cn2 + 2] = pixelPtr[(i + x)*mat2.cols*cn + (j + y)*cn + 2];
}
}
}
}
The commented code was another approach, but had the same results.
So, here are my questions:
1. How do I solve the 2 problems (1. the colors..., 2. alpha ...)
2. How is the pixel-array of any Mat-Object actually, well, organized? I guess it would be easier for me to manipulate those arrays if I knew whats in them.
Because you are iterating mat2 with wrong type. Change Vec3b intensity2 = mat2.at<Vec3b>(j, i); to:
Vec4b intensity2 = mat2.at<Vec4b>(j, i);
and the weird stripes are eliminated. And use intensity2[3] to deal with the alpha channel.
Assume that you are reading the black trapezoid png file using -1 flag:
auto trapezoidImg = cv::imread("trapezoid.png", -1);
where the -1 flag specifies that alpha channel is read. Then trapezoidImg is organized in the following format:
[B, G, R, A, B, G, R, A, ......;
B, G, R, A, B, G, R, A, ......;
......
B, G, R, A, B, G, R, A, ......]
You can print out trapezoidImg, for example using std::cout, to figure out this format.
If you read trapezoidImg using at<Vec3b>, what you get in fact is (B, G, R), (A, B, G), (R, A, B), ......, and this is where the weird stripes came from. Therefore, use at<Vec4b> to read the (R, G, B, A) intensity correctly.
Next, you should define what to do with the alpha channel. You can mixing two Mat or override another, whatever. One simple method is to override mat1 only when the alpha channel in mat2 is large enough:
cv::Vec3b &intensity = mat1.at<cv::Vec3b>(j + y, i + x);
cv::Vec4b intensity2 = mat2.at<cv::Vec4b>(j, i);
for (int k = 0; k < mat1.channels(); k++) {
if (intensity2.val[3] > 250){ //3 for alpha channel
intensity.val[k] = cv::saturate_cast<uchar>(intensity2.val[k]);
}
}
This is enough to deal with your black trapezoid png with transparent background. Or further extend the rule by mixing two Mat:
cv::Vec3b &intensity = mat1.at<cv::Vec3b>(j + y, i + x);
cv::Vec4b intensity2 = mat2.at<cv::Vec4b>(j, i);
auto alphaValue = cv::saturate_cast<uchar>(intensity2.val[3]);
auto alpha = alphaValue / 255.0;
for (int k = 0; k < 2; k++) { //BGR channels only
intensity.val[k] = cv::saturate_cast<uchar>(
intensity2.val[k] * alpha + intensity.val[k] * (1.0 - alpha));
}