Pairwise differences between two matrices in Eigen - c++

In matlab/octave pairwise distances between matrices as required for e.g. k-means are calculated by one function call (see cvKmeans.m), to distFunc(Codebook, X) with as arguments two matrices of dimensions KxD.
In Eigen this can be done for a matrix and one vector by using broadcasting, as explained on eigen.tuxfamily.org:
(m.colwise() - v).colwise().squaredNorm().minCoeff(&index);
However, in this case v is not just a vector, but a matrix. What's the equivalent oneliner in Eigen to calculate such pairwise (Euclidean) distances across all entries between two matrices?

I think the appropriate solution is to abstract this functionality into a function. That function may well be templated; and it may well use a loop - the loop will be really short, after all. Many matrix operations are implemented using loops - that's not a problem.
For example, given your example of...
MatrixXd p0(2, 4);
p0 <<
1, 23, 6, 9,
3, 11, 7, 2;
MatrixXd p1(2, 2);
p1 <<
2, 20,
3, 10;
then we can construct a matrix D such that D(i,j) = |p0(i) - p1(j)|2
MatrixXd D(p0.cols(), p0.rows());
for (int i = 0; i < p1.cols(); i++)
D.col(i) = (p0.colwise() - p1.col(i)).colwise().squaredNorm().transpose();
I think this is fine - we can use some broadcasting to avoid 2 levels of nesting: we iterate over p1's points, but not over p0's points, nor over their dimensions.
However, you can make a oneliner if you observe that |p0(i) - p1(j)|2 = |p0(i)|2 + |p1(j)|2 - 2 p0(i)T p1(j). In particular, the last component is just matrix multiplication, so D = -2 p0T p1 + ...
The blank left to be filled is composed of a component that only depends on the row; and a component that only depends on the column: these can be expressed using rowwise and columnwise operations.
The final "oneliner" is then:
D = ( (p0.transpose() * p1 * -2
).colwise() + p0.colwise().squaredNorm().transpose()
).rowwise() + p1.colwise().squaredNorm();
You could also replace the rowwise/colwise trickery with an (outer) product with a 1 vector.
Both methods result in the following (squared) distances:
1 410
505 10
32 205
50 185
You'd have to benchmark which is fastest, but I wouldn't be surprised to see the loop win, and I expect that's more readable too.

Eigen is more of a headache than I thought on first sight.
There is no reshape() functionality for example (and conservativeResize is something else).
It also seems (I'd like to be corrected) to be the case that Map does not just offer a view on the data, but assignments to temporary variables seem to be required.
The minCoeff function after the colwise operator cannot return a minimum element and an index to that element.
It is unclear to me if replicate is actually allocating duplicates of the data. The reason behind broadcasting is that this is not required.
matrix_t data(2,4);
matrix_t means(2,2);
// data points
data << 1, 23, 6, 9,
3, 11, 7, 2;
// means
means << 2, 20,
3, 10;
std::cout << "Data: " << std::endl;
std::cout << data.replicate(2,1) << std::endl;
column_vector_t temp1(4);
temp1 = Eigen::Map<column_vector_t>(means.data(),4);
std::cout << "Means: " << std::endl;
std::cout << temp1.replicate(1,4) << std::endl;
matrix_t temp2(4,4);
temp2 = (data.replicate(2,1) - temp1.replicate(1,4));
std::cout << "Differences: " << std::endl;
std::cout << temp2 << std::endl;
matrix_t temp3(2,8);
temp3 = Eigen::Map<matrix_t>(temp2.data(),2,8);
std::cout << "Remap to 2xF: " << std::endl;
std::cout << temp3 << std::endl;
matrix_t temp4(1,8);
temp4 = temp3.colwise().squaredNorm();
std::cout << "Squared norm: " << std::endl;
std::cout << temp4 << std::endl;//.minCoeff(&index);
matrix_t temp5(2,4);
temp5 = Eigen::Map<matrix_t>(temp4.data(),2,4);
std::cout << "Squared norm result, the distances: " << std::endl;
std::cout << temp5.transpose() << std::endl;
//matrix_t::Index x, y;
std::cout << "Cannot get the indices: " << std::endl;
std::cout << temp5.transpose().colwise().minCoeff() << std::endl; // .minCoeff(&x,&y);
This is not a nice oneliner and seems overkill just to compare every column in data with every column in means and return a matrix with their differences. However, the versatility of Eigen does not seem to be such that this can be written down much shorter.

Related

NTL library GF2X

I am experimenting with the Galois Field using the NTL library. GF2 are the integers mod 2, GF2X are polynomials over GF2 and GF2E is the ring/field extension over GF2.
The problem that I am facing is that I initialize the irreducible polynomial as follows
GF2X irreduc;
SetCoeff(irreduc, 128, 1);
SetCoeff(irreduc, 7, 1);
SetCoeff(irreduc, 2, 1);
SetCoeff(irreduc, 1, 1);
SetCoeff(irreduc, 0, 1);
GF2E::init(irreduc);
and then I also initialize two polynomials:
GF2X a;
SetCoeff(a, 120);
SetCoeff(a, 22);
GF2X b;
SetCoeff(b, 128);
SetCoeff(b, 51);
std::cout << "a: " << a << '\n';
std::cout << "b: " << b << '\n';
and multiply them:
std::cout << "\ndeg(a * b): " << deg(a * b) << '\n';
The output is deg(a * b): 248, which is out of the field/ring of the 2^128, defined by the irreducible polynomial.
I know that I am probably missing something obvious but I am very new to this area so bear with me.
Thank you!
As you already said, GF2X represents polynomials over GF2, so they do not get reduced by the polynomial you initialized GF2E with. You need to convert the polynomials to GF2E and then everything works as expected.
So changing you last line to
std::cout << "\ndeg(a * b): " << deg(conv<GF2X>(conv<GF2E>(a) * conv<GF2E>(b))) << '\n';
results in the output
deg(a * b): 124
This conversions are pretty ugly. I'm not sure if there is a better way to do it and the way NTL is documented it is hard to find the right functions for what you want to do. I only found GF2E::degree(), but this only gives you the degree if the irreducible polynomial. Let me know when you find the right way to do it.

Data assignment without copying to cv::Mat and cv::UMat from other OpenCV Matrix

I'm stuck with the data assignment without copying in C++ OpenCV (version 3.4.1).
Please have a look at the following example code:
std::cout << "test cv::Mat" << std::endl;
cv::Mat a(2, 2, CV_32FC1, cv::Scalar(1.));
cv::Mat b(2, 2, CV_32FC1, cv::Scalar(5.));
std::cout << "a:" << std::endl << a << std::endl;
std::cout << "b:" << std::endl << b << std::endl;
a.col(0) = b.col(0);
std::cout << "a:" << std::endl << a << std::endl;
std::cout << "b:" << std::endl << b << std::endl;
a.col(0) = b.row(0).t();
std::cout << "a:" << std::endl << a << std::endl;
std::cout << "b:" << std::endl << b << std::endl;
std::cout << "test cv::UMat" << std::endl;
cv::UMat c(2, 2, CV_32FC1, cv::Scalar(1.));
cv::UMat d(2, 2, CV_32FC1, cv::Scalar(5.));
std::cout << "c:" << std::endl << c << std::endl;
std::cout << "d:" << std::endl << d << std::endl;
c.col(0) = d.col(0);
std::cout << "c:" << std::endl << c << std::endl;
std::cout << "d:" << std::endl << d << std::endl;
c.col(0) = d.row(0).t();
std::cout << "c:" << std::endl << c << std::endl;
std::cout << "d:" << std::endl << d << std::endl;
This gives the following output:
test cv::Mat
a:
[1, 1;
1, 1]
b:
[5, 5;
5, 5]
a:
[1, 1;
1, 1]
b:
[5, 5;
5, 5]
a:
[5, 1;
5, 1]
b:
[5, 5;
5, 5]
test cv::UMat
[ INFO:0] Initialize OpenCL runtime...
c:
[1, 1;
1, 1]
d:
[5, 5;
5, 5]
c:
[1, 1;
1, 1]
d:
[5, 5;
5, 5]
c:
[1, 1;
1, 1]
d:
[5, 5;
5, 5]
In case of cv::Mat, why is the assigment with a.col(0) = b.col(0); not working?
Why is the assigment with a.col(0) = b.row(0).t(); working?
Why is it different in case of cv::UMat?
The goal behind all this is to concatenate a cv::Mat (or better cv::UMat) from multiple other OpenCV matrices, without copying the data (which might be expensive due to the amount of data).
Is this possible at all?
No, it's not possible to concatenate multiple OpenCV matrices into a single one without copying of data.
The class cv::Mat holds only single pointer to data, that does not allow handling multiple memory regions.
There are also pointers datastart, dataend and datalimit in cv::Mat, but these are just helpers for support of a region of interest. They help handling smaller matrices without data copying, not larger.
When you use a.row() or a.col(), the data in a is not copied
to the new array. Like d = b.col(0);: This expression
means to create a new array header d , and to arrange its data pointer, step array, and so on, such that it will access the data in col 0 in d.
So that's why it does look like it doesn't work and when you add transpose .t() to it then you're not in the same case.
For CPU version, in .t() you are not applying the same rule as .col() but simply assigning a vector to another matrix, hence its value has been copied/modified.
For UMat, I don't have enough information what exactly using GPU or CPU to perform the task. For me it is using my GPU but for you it doesn't seem to be:
test cv::UMat
[ INFO:0] Initialize OpenCL runtime...
[ INFO:0] Successfully initialized OpenCL cache directory: /home/jdros/.cache/opencv/3.4.1-dev/opencl_cache/
[ INFO:0] Preparing OpenCL cache configuration for context: NVIDIA_Corporation--GeForce_930MX--384_81
Recommendations:
With newer versions of OpenCV, you get more optimized code and more faster. So I would recommend you to read about each particular function about copying data in memory and/or try some code yourself where you can use those functions like .clone(), copyTo()..etc. For example, a simple assignment is not the best way of matrix copying, which is what you really need, however .clone() is literally copying data in memory.. But investigate that with every function you're using, it worth it.
I found some more informations in a book called: Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library:
It is critical to understand that the data in an array is not attached rigidly to the array object. The cv::Mat object is really a header for a data area, which—in principle—is an entirely separate thing. For example, it is possible to assign one matrix n to another matrix m (i.e., m=n ). In this case, the data pointer inside of m will be changed to point to the same data as n . The data pointed to previously by the data element of m (if any) will be deallocated (Technically, it will only be deallocated if m was the last cv::Mat that pointed to that particular data). At the same time, the reference counter for the data area that they both now share will be incremented. Last but not least, the members of m that characterize its data (such as rows , cols , and flags ) will be updated to
accurately describe the data now pointed to by data in m . This all results in a very convenient behavior, in which arrays can be assigned to one another, and the work necessary to do this takes place automatically behind the scenes to give the correct result.

Mapping Eigen Matrix Ref objects not consistent with blocks

Quick context, I am working with another C++ library that has functions that expect either a regular or mapped Eigen matrix. I would like to use the mapped version to avoid the memory overhead of copying.
That said, I am trying to work with blocks of matrices. I know these can be easily accessed with the block method returning either an Eigen::Block or Eigen::Ref object. Below I am trying to work with the Ref object. I would like to Map the Eigen::MatrixXd block. However, it appears that I cannot map across columns but only contiguous elements in columns (which I assume is a consequence of the column oriented data). You can see the difference in outputs below.
Is there any way for me to Map a block of an Eigen::MatrixXd?
#include <iostream>
#include <Eigen/Core>
int main()
{
Eigen::MatrixXd A(3,3);
A(0,0) = 1.0;
A(0,1) = 2.0;
A(0,2) = 3.0;
A(1,0) = 4.0;
A(1,1) = 5.0;
A(1,2) = 6.0;
A(2,0) = 7.0;
A(2,1) = 8.0;
A(2,2) = 9.0;
std::cout << "source" << std::endl;
std::cout << A << std::endl;
Eigen::Ref<Eigen::MatrixXd> block = A.block(1,1,1,2);
std::cout << "block" << std::endl;
std::cout << block << std::endl;
Eigen::Map<Eigen::MatrixXd> map(block.data(), block.rows(), block.cols());
std::cout << "map" << std::endl;
std::cout << map << std::endl;
}
Output:
source
1 2 3
4 5 6
7 8 9
block
5 6
map
5 8
The Eigen::Map assumes a unit stride unless specified otherwise. The problem with the Ref object is that the stride is not 1. You can specify the stride (outer in this case) as follows:
Eigen::Map<Eigen::MatrixXd, 0, Eigen::OuterStride<> >
map2(block.data(), block.rows(), block.cols(), Eigen::OuterStride<>(3));
std::cout << "map2" << std::endl;
std::cout << map2 << std::endl;
Better yet, you can use the outer stride of the Ref object to specify it for the map:
Eigen::Map<Eigen::MatrixXd, 0, Eigen::OuterStride<> >
map2(block.data(), block.rows(), block.cols(), Eigen::OuterStride<>(block.outerStride()));

Verifying essential matrix

I'm trying to code a simple structure from motion scenario, using only 2 images taken from the same camera.
I use SIFT to find matching points between the images (total of 72 matches), out of which 62 are correct.
I use OpenCV to calculate the fundamental matrix, then the essential. When I try to verify the essential matrix by doing p2^T * E * p1 I get very high values instead of close to zero.
Am I doing something wrong?
Here's the code: (pts1, pts2 are std::vector<Point2f>. dmat is Mat_<double>)
int n = pts1.size();
std::cout << "Total point matches: " << n << std::endl;
std::vector<unsigned char> status(n);
std::cout << "K=" << K << std::endl;
F = findFundamentalMat(pts1, pts2,FM_RANSAC,3,0.99,status);
std::cout << "F=" << F << std::endl;
std::cout << "Total inliers: " << std::accumulate(status.begin(),status.end(),0) << std::endl;
E = K.t() * F * K;
std::cout << "E=" << E << std::endl;
for (int i = 0; i < n;++i)
{
dmat p1(3,1), p2(3,1);
p1 << pts1[i].x, pts1[i].y, 1;
p2 << pts2[i].x, pts2[i].y, 1;
dmat mv = p2.t() * E * p1;
double v = mv(0, 0);
std::cout << v << std::endl;
}
and here is the output from this code:
Total point matches: 72
K=[390.0703661671206, 0, 319.5;
0, 390.0703661671206, 239.5;
0, 0, 1]
F=[-2.723736291531157e-007, 7.660367616625481e-005, -0.01766345189507435;
-4.219955880897177e-005, 9.025976628215733e-006, -0.04376995849516735;
0.009562535474535394, 0.03723116011143099, 1]
Total inliers: 62
E=[-0.04144297973569942, 11.65562396370436, 0.2325229628055823;
-6.420869252333299, 1.373346486079092, -21.48936503378938;
-0.2462444924550576, 24.91291898830852, -0.03174504032716108]
188648
-38467.5
-34880.7
289671
257263
87504.7
462472
-30138.1
-30569.3
174520
-32342.8
-32342.8
-37543.4
241378
-36875.4
-36899
-38796.4
-38225.2
-38120.9
394285
-440986
396805
455397
543629
14281.6
630398
-29714.6
191699
-37854.1
-39295.8
-3395.93
-3088.56
629769
-28132.9
178537
878596
-58957.9
-31034.5
-30677.3
-29854.5
165689
-13575.9
-13294.3
-6607.96
-3446.41
622355
-31803
-35149
-38455.4
2068.12
82164.6
-35731.2
-36252.7
-36746.9
-35325.3
414185
-35216.3
-126107
-5551.84
100196
2.29755e+006
177785
-31991.8
-31991.8
100340
108897
108897
84660.4
-7828.65
225817
225817
295423
The equation v2^T * E * v1 is true for the essential matrix only when v2 and v1 are in normalized coordinates, i.e. v1 = K^(-1)*p1, with p1 the observed point in pixels. Same goes for v2 and p2.
If you have it, you can refer to definition 9.16 page 257 of Hartley and Zisserman's book. But note that this makes sense, given the relation E = K.t() * F * K.

Best way to sort all columns of an armadillo matrix by an index vector

I'm wondering whether there's a better way to achieve what I'm doing here. I have an arma matrix and I want to reorder all of it's columns by the indices stored in a uvec vector. I think I'm basically copying the whole matrix.
#include <armadillo>
using namespace arma;
int main(){
// get a discrete random matrix
// defined umat because eventually want to
// order by a given column OF A. irrelevant now.
umat A = randi<umat>(4,6,distr_param(0,3));
std::cout << "A " << std::endl;
std::cout << A << std::endl;
// get an index vector with the now row order
uvec b;
b << 3 << 2 << 1 << 0;
std::cout << "sort by b:" << std::endl;
std::cout << b << std::endl;
// get all col indices
uvec cols = linspace<uvec>(0,A.n_cols-1,A.n_cols);
// order ALL cols of A by b
// I'm afraid this just makes a copy
A = A.submat(b, cols );
std::cout << "reordered A by b" << std::endl;
std::cout << A << std::endl;
return 0;
}
You are right in that the code creates a new matrix A and does not exchange the rows in place.
Alternatively you could express the permutation as a product of transpositions and then swap the rows of A one-by-one with swap_rows. This is of course not trivial to implement and I would only go this route if memory usage is of concern or if you only need to permute a few of the rows and will leave the rest as they are. Otherwise rebuilding the matrix will probably be faster due to cache efficiency.
For your example case, which just reverses the row order, you might of course want to swap the last and first row, then the last-1'th and the 2nd and so on.