I am attempting to calculate a pointcloud from an opencv Mat depth image and an intrinsic matrix. Currently I do it as follows (k matrix values were extracted into fx,fy,cx,cy earlier):
for(int i=0; i<depth.rows; i++)
{
const float* row_ptr = depth.ptr<float>(i);
for(int j=0; j<depth.cols; j++)
{
// Only add valid depth points
if(row_ptr[j] != 0)
{
const float x = ((j - cx) * row_ptr[j]/focal_length_x);
const float y = ((i - cy) * row_ptr[j]/focal_length_y);
pointcloud[cnt] = pcl::PointXYZ(x/1000, y/1000, row_ptr[j]/1000);
cnt++;
}
}
}
However I am wondering is it possible to turn this into a matmul operation and use eigen for better performance. I am aware that:
[x, y, z] = depth value * inv(k) * [u, v, 1] with
[fx, 0, cx]
k = [0, fy, cy]
[0, 0, 1]
How would I go about turning this into a full matrix multiplication, my depth image is 1280x800, obviously directly multiplying a 1280x800 matrix with a 3x3 with a 3x1 wont work so what ways can this be done if any?
Related
I'm using this article: nonlingr as a font to understand non linear transformations, in the section GLYPHS ALONG A PATH he explains how to use a parametric curve to transform an image, i'm trying to apply a cubic bezier to an image, however i have been unsuccessfull, this is my code:
OUT.aloc(IN.width(), IN.height());
//get the control points...
wVector p0(values[vindex], values[vindex+1], 1);
wVector p1(values[vindex+2], values[vindex+3], 1);
wVector p2(values[vindex+4], values[vindex+5], 1);
wVector p3(values[vindex+6], values[vindex+7], 1);
//this is to calculate t based on x
double trange = 1 / (OUT.width()-1);
//curve coefficients
double A = (-p0[0] + 3*p1[0] - 3*p2[0] + p3[0]);
double B = (3*p0[0] - 6*p1[0] + 3*p2[0]);
double C = (-3*p0[0] + 3*p1[0]);
double D = p0[0];
double E = (-p0[1] + 3*p1[1] - 3*p2[1] + p3[1]);
double F = (3*p0[1] - 6*p1[1] + 3*p2[1]);
double G = (-3*p0[1] + 3*p1[1]);
double H = p0[1];
//apply the transformation
for(long i = 0; i < OUT.height(); i++){
for(long j = 0; j < OUT.width(); j++){
//t = x / width
double t = trange * j;
//apply the article given formulas
double x_path_d = 3*t*t*A + 2*t*B + C;
double y_path_d = 3*t*t*E + 2*t*F + G;
double angle = 3.14159265/2.0 + std::atan(y_path_d / x_path_d);
mapped_point.Set((t*t*t)*A + (t*t)*B + t*C + D + i*std::cos(angle),
(t*t*t)*E + (t*t)*F + t*G + H + i*std::sin(angle),
1);
//test if the point is inside the image
if(mapped_point[0] < 0 ||
mapped_point[0] >= OUT.width() ||
mapped_point[1] < 0 ||
mapped_point[1] >= IN.height())
continue;
OUT.setPixel(
long(mapped_point[0]),
long(mapped_point[1]),
IN.getPixel(j, i));
}
}
Applying this code in a 300x196 rgb image all i get is a black screen no matter what control points i use, is hard to find information about this kind of transformation, searching for parametric curves all i find is how to draw them, not apply to images. Can someone help me on how to transform an image with a bezier curve?
IMHO applying a curve to an image sound like using a LUT. So you will need to check for the value of the curve for different image values and then switch the image value with the one on the curve, so, create a Look-Up-Table for each possible value in the image (e.g : 0, 1, ..., 255, for a gray value 8 bit image), that is a 2x256 matrix, first column has the values from 0 to 255 and the second one having the value of the curve.
I have a Mat2d matrix, where each element is a 2D vector. For example:
[[x0, y0], [x1, y1]
[x2, y2], [x3, y3]]
I want to left multiply each of these vectors by a Mat1d camera matrix:
[fx, 0, cx,
0, fy, cy,
0, 0, 1]
(Each vector represents the location of a vertex in a grid which I want to convert from the camera space to the pixel space.)
The resulting matrix, for this example, would be:
[[x0 * fx + cx, y0 * fy + cy], [x1 * fx + cx, y1 * fy + cy]
[x2 * fx + cx, y2 * fy + cy], [x3 * fx + cx, y3 * fy + cy]]
What is the most straightforward and efficient way to accomplish this?
Here is my current approach:
Mat2d points = getMesh();
Mat1d cameraMtrx = getCameraMtrx();
for(int col = 0; col < points.cols; col++){
for(int row = 0; row < points.rows; row++){
points.at<Vec2d>(row, col).val[0] = points.at<Vec2d>(row, col)[0] * cameraMtrx.at<double>(0, 0) + cameraMtrx.at<double>(0, 2);
points.at<Vec2d>(row, col).val[1] = points.at<Vec2d>(row, col)[1] * cameraMtrx.at<double>(1, 1) + cameraMtrx.at<double>(1, 2);
}
}
OpenCV Documentation has already detailed various methods of iterating the cv::Mat efficiently, Out of the presented methods, the most efficient way is to use cv::LUT(), but from the context of this question, I guess the range of input matrix values is not fixed, so a look-up table can't be created, It is very helpful in case of RGB images, because we know beforehand that min value would be 0 and max value would be 255, so we can easily create a lookUp table, but in this problem, we need to multiply two matrices which I assume are not images so we would go with The efficient way.
int cameraMatrix[] = {2, 0, 10, 0, 4, 20, 0, 0, 1};
cv::Mat mat(2, 2, CV_32FC2, cv::Scalar(100, 20));
cv::Size contSize = mat.size();
// Calculate the length of array if the input matrix was flatten, in case of continuous matrix only.
if (mat.isContinuous()) {
contSize.width *= contSize.height;
contSize.height = 1;
}
cv::Vec2f* ptr;
for (int i = 0; i < contSize.height; ++i)
{
ptr = mat.ptr<cv::Vec2f>(i);
for (int j = 0; j < contSize.width; ++j)
{
ptr[j] = cv::Vec2f(ptr[j].val[0]*cameraMatrix[0] + cameraMatrix[2], ptr[j].val[1] * cameraMatrix[4] + cameraMatrix[5]);
}
}
I have a problem accessing elements of a 5D Matrix in OpenCV. I create my Matrix using
int sizes[5] = { height_, width_, range_, range_, range_ };
Mat w_i_ = Mat(2 + channels, sizes, CV_16UC(channels), Scalar(0));
where channels = 3. Then I'm trying to access and modify the matrix elements using for loops:
for (UINT Y = 0; Y < height; ++Y) {
for (UINT X = 0; X < width; ++X) {
// a) Compute the homogeneous vector (wi,w)
Vec3b wi = image.at<Vec3b>(Y, X);
// b) Compute the downsampled coordinates
UINT y = round(Y / sigmaSpatial);
UINT x = round(X / sigmaSpatial);
Vec3b zeta = round( (image.at<Vec3b>(Y, X) - min) / sigmaRange);
// round() here is overloaded for vectors
// c) Update the downsampled S×R space
int idx[5] = { y, x, zeta[0], zeta[1], zeta[2] };
w_i_.at<Vec3b>(idx) = wi;
}
}
I am getting an assertion failed error produced by Mat::at() when I run the code. Specifically the message I get is:
OpenCV Error: Assertion failed (elemSize() == (((((DataType<_Tp>::type) & ((512 - 1) << 3)) >> 3) + 1) << ((((sizeof(size_t)/4+1)*16384|0x3a50) >> ((DataType<_Tp>::type) & ((1 << 3) - 1))*2) & 3))) in cv::Mat::at, file c:\opencv\build\include\opencv2\core\mat.inl.hpp, line 1003
I have searched the web but I can't seem to find any topics on 5D Matrices (similar topics proved of no help).
Thanks in advance
You initialize the zeta variable and do not check its values.
Most likely you get an out-of-range value for zeta[0], zeta[1] and zeta[2] indices and thus the internal range checking in at() function fails.
To prevent such crashes at least add some manual range checking before calling at():
for(int i = 0 ; i < 3 ; i++)
if(zeta[i] < 0 || zeta[i] >= _range)
continue;
Let's say I have image, which I need to rotate 90 degrees in any direction and I just can't understand how to do that clear. I need to work with matrix, where Width - it's X, and Height - it's Y. I've already done rotating an image 180 degrees, but can't figure out 90 degrees.
Here are the examples. Let's say I have an image 3x4 pixels. Width = 3, Height = 4, the amount of data in each cell - N = Width * Height = 3 * 4 = 12. Let's make the matrix:
The formula to easily go through the matrix is y*Width + x. And the formula for our rotating 180 degrees is:
N - Width + x - y * Width
So we have:
DataOut [y * Width + x] = DataIn [N - Width + x - y * Width]
Using this formula we get:
But I can't come up with the formula of rotating 90 degrees. Can you help me, please?
you can simply rotate the matrix by this:
for(int i=0; i<cols; i++)
{
for(int j=1; j<=rows; j++)
{
datOut[i][j]= datIn[rows-j][i];
}
}
and in 1-D array:
for(int i=0; i<cols; i++)
{
for(int j=1; j<=rows; j++)
{
datOut[i * rows + j]= datIn[(rows-j) * cols + i];
}
}
You can easily convert the (x + y * width) to a simpler (x, y) representation.
using P = point_data_type;
P point(int x, int y){
return DataIn[x + y * width]; // N - ...?
}
Now a right angle rotation is just a coordinate switch and maybe a sign correction.
P rotate90(int x, int y){
return point(y, x);
}
This is a positive rotation in a left-handed system.
Subtract x from Width if the rotation is in the wrong direction.
I am using OpenCV for some time and now I hit the point where I need a multiplication of this type:
Define a matrix T, which contains elements of the type Vec3d1 . Matrix T has the size: M X N. Matrix T has to be multiplied with a Vector Phi, which has the size: N X 1, containing doubles as values. Each element of the result has to be the result of a matrix multiplication of both matrices.
I don't want to do a component-wise multiplication, but a "real" matrix multiplication, e.g. multiplying the first element of T2 with the first element of matrix J, then multiplying the second element of matrix T3 with the second element of matrix J. Do this until you completed the first row of T and then sum up the results. The result is a M X 1.
For example, if T would be a 3 X 2 matrix and Phi a 2 X 1 matrix, then the calculation should be T_11 * phi_11 + T_12 * phi_21 for the first value of the result. Currently I'm using two for loops which are slow:
for (int i = 0; i<M; ++i){
cv::Mat summedResult = cv::Mat(3, 1, CV_64F, double(0));
for (uint32 j = 0; j<N; ++j){
summedResult = summedResult +
(cv::Mat(mMatrixT.at<cv::Vec3d>(i, j)) * mMatrixPhi.at<double>(j));
}
// The result matrix contains values of type Vec3d again
mResultMatrix.at<cv::Vec3d>(i) = cv::Vec3d(summedResult);
}
More generally: Is it possible to efficiently multiply matrices containing Vec3ds and doubles in OpenCV?
1. three dimensional vector containing doubles.
2. coordinate: 1,1
3. coordinate: 1,2
I still don't know what kind of result you expect, but maybe try this:
Assuming that you have a MxN matrix of Vec3d and a Nx1 Matrix of type double, your result will be a Mx1 matrix of type Vec3d:
for (int i = 0; i<M; ++i)
{
cv::Vec3d summedResult; // here this must be a Vec3d instead of a matrix, if I assume the right result expection
for (uint32 j = 0; j<N; ++j)
{
summedResult = summedResult + (mMatrixT.at<cv::Vec3d>(i, j) * mMatrixPhi.at<double>(j));
}
// The result matrix contains values of type Vec3d again
mResultMatrix.at<cv::Vec3d>(i) = summedResult;
}
EDIT:
ah sorry, didnt read to the end that your provided code works but is too slow... well, I expect that there is no optimization for that because mathematically this isnt defined. What you can do is to convert your Vec3d mat to a Mx(3*N) matrix and convert your Nx1 mat to a (3*N)x1 mat (3 times the same value before the next value) and use the OpenCV matrix product directly. But probably that's not faster because of the 3* bigger size of both matrices ;)
EDIT: will be a different result, since each element will be the sum of the Vec3d elements...