It is easy to copy data between e.g. Eigen::VectorXd and std::vector<double> or std::vector<Eigen::Vector3d>, for example
std::vector<Eigen::Vector3> vec1(10, {0,0,0});
Eigen::VectorXd vec2(30);
VectorXd::Map(&vec2[0], vec1.size()) = vec1;
(see e.g. https://stackoverflow.com/a/26094708/4069571 or https://stackoverflow.com/a/21560121/4069571)
Also, it is possible to create an Eigen::Ref<VectorXd> from a Matrix block/column/... for example like
MatrixXd mat(10,10);
Eigen::Ref<VectorXd> vec = mat.col(0);
The Question
Is it possible to create an Eigen::Ref<VectorXd> from a std::vector<double> or even std::vector<Eigen::Vector3d> without first copying the data?
I tried and it actually works as I describe in my comment by first mapping and then wrapping it as a Eigen::Ref object. Shown here through a google test.
void processVector(Eigen::Ref<Eigen::VectorXd> refVec) {
size_t size = refVec.size();
ASSERT_TRUE(10 == size);
std::cout << "Sum before change: " << refVec.sum(); // output is 50 = 10 * 5.0
refVec(0) = 10.0; // for a sum of 55
std::cout << "Sum after change: " << refVec.sum() << std::endl;
}
TEST(testEigenRef, onStdVector) {
std::vector<double> v10(10, 5.0);
Eigen::Map<Eigen::VectorXd> mPtr(&v10[0], 10);
processVector(mPtr);
// confirm that no copy is made and std::vector is changed as well
std::cout << "Std vec[0]: " << v10[0] << std::endl; // output is 10.0
}
Made it a bit more elaborate after the 2nd edit. Now I have my google unit test for Eigen::Ref (thank you). Hope this helps.
Related
I currently have the very simple program. I deactivated copy elision in my compiler and as far as I understand it shouldn't even apply here because the compiler doesn't know what I will input. Now, the very weird thing is that when I look at the addresses they match what I would expect if no copy was happening with the value that is returned by the function. However, THERE is a copy happening when I say Eigen::MatrixXf m2 = m. I have no idea why the adresses match as inutitively I expect there to be a copy.
#include <iostream>
#include <Eigen/Dense>
Eigen::MatrixXf no_copy(){
Eigen::MatrixXf m = Eigen::MatrixXf::Random(3,3);
Eigen::MatrixXf m2 = Eigen::MatrixXf::Random(3,3);
int c = 0;
std::cin >> c;
std::cout << "m" << &m(0,0) << std::endl;
std::cout << "m2" << &m2(0,0) << std::endl;
if (c > 0) {
std::cout << "m " << std::endl;
return m;
}
else{
std::cout << "m2 " << std::endl;
return m2;
}
return m;
}
int main() {
//make a 1000 x 1000 matrix of random floats
Eigen::MatrixXf m;
m = no_copy();
Eigen::MatrixXf m2 = m;
std::cout << m << std::endl;
std::cout << &m(0,0) << std::endl;
std::cout << &m2(0,0) << std::endl;
return 0;
}
`
I tried doing Eigen::MatrixXf m2 = std::move(m); to see if a move was the reason for the odd behavior, but I got an assertion error when I tried it. Why isn't Eigen copying the matrix that is returned in the no_copy function?
Eigen::MatrixXf holds a matrix of a dynamically chosen size. Therefore it can't be storing the matrix elements directly inside its own storage. It must be using a dynamic allocation, the same way that std::vector does.
return m;, if NRVO is not applied, will use the move constructor to construct the temporary result object of the function call and m = no_copy(); will also use the move assignment operator to assign from this temporary. If the move operations are implemented properly, they will not do any additional allocation, but instead just take over ownership of the dynamic allocation that the original MatrixXf made.
You are printing addresses into the dynamic allocation, which shouldn't have changed.
Eigen::MatrixXf m2 = m; is copy-construction, not move-construction, so it can't do the same. It needs to make a new dynamic allocation and copy over the values. The addresses must be different. Copy elision is also impossible.
Eigen::MatrixXf m2 = std::move(m); results in an error because you are still trying to index into m afterwards although you have moved from it and its dynamic allocation isn't owned by m anymore.
I would like to calculate some summary functions (e.g. mean) on a section of a column of an armadillo matrix. Key is not to create a temporary copy of the data.
This works with entire columns, using unsafe_col:
arma::mat x;
double d = mean(x.unsafe_col(0));
To obtain a section of a vector, I could use:
x.col(0).subvec(0,100);
However, this creates a copy of the vector data, which I want to avoid.
What would be the most efficient way to apply the armadillo-provided functions to sections of column vectors? Performance is absolutely critical in this applications.
One way I found was to construct a vector with pointers to the underlying memory, using what is called an "advanced constructor" in the armadillo documentation, but first getting pointers to double for the column start and the adding offsets for start and end just to create a shorter vector seems quite a hack...
UPDATE: Here is an example of the behaviour:
#include <armadillo>
#include <iostream>
arma::mat m( {{1,2,3,4},{5,6,7,8},{9,10,11,12},{13,14,15,16}});
const arma::vec view1() {
return m.unsafe_col(0).subvec(0,2);
}
const arma::vec view2() {
double *start = &m[0,0];
const int n = 3;
return arma::vec(start, 3, false, false);
}
int main() {
std::cout << &m(0,0) << " " << &m(1,0) << " " << &m(0,1) << std::endl;
auto v1(view1());
auto v2(view2());
std::cout << &v1(0) << std::endl;
std::cout << &v2(0) << std::endl;
return 0;
}
which returns, for example:
0x56419b070170 0x56419b070178 0x56419b070190
0x7ffcb36f7510
0x56419b070170
So view1() created a copy, but view2() did not.
Quick context, I am working with another C++ library that has functions that expect either a regular or mapped Eigen matrix. I would like to use the mapped version to avoid the memory overhead of copying.
That said, I am trying to work with blocks of matrices. I know these can be easily accessed with the block method returning either an Eigen::Block or Eigen::Ref object. Below I am trying to work with the Ref object. I would like to Map the Eigen::MatrixXd block. However, it appears that I cannot map across columns but only contiguous elements in columns (which I assume is a consequence of the column oriented data). You can see the difference in outputs below.
Is there any way for me to Map a block of an Eigen::MatrixXd?
#include <iostream>
#include <Eigen/Core>
int main()
{
Eigen::MatrixXd A(3,3);
A(0,0) = 1.0;
A(0,1) = 2.0;
A(0,2) = 3.0;
A(1,0) = 4.0;
A(1,1) = 5.0;
A(1,2) = 6.0;
A(2,0) = 7.0;
A(2,1) = 8.0;
A(2,2) = 9.0;
std::cout << "source" << std::endl;
std::cout << A << std::endl;
Eigen::Ref<Eigen::MatrixXd> block = A.block(1,1,1,2);
std::cout << "block" << std::endl;
std::cout << block << std::endl;
Eigen::Map<Eigen::MatrixXd> map(block.data(), block.rows(), block.cols());
std::cout << "map" << std::endl;
std::cout << map << std::endl;
}
Output:
source
1 2 3
4 5 6
7 8 9
block
5 6
map
5 8
The Eigen::Map assumes a unit stride unless specified otherwise. The problem with the Ref object is that the stride is not 1. You can specify the stride (outer in this case) as follows:
Eigen::Map<Eigen::MatrixXd, 0, Eigen::OuterStride<> >
map2(block.data(), block.rows(), block.cols(), Eigen::OuterStride<>(3));
std::cout << "map2" << std::endl;
std::cout << map2 << std::endl;
Better yet, you can use the outer stride of the Ref object to specify it for the map:
Eigen::Map<Eigen::MatrixXd, 0, Eigen::OuterStride<> >
map2(block.data(), block.rows(), block.cols(), Eigen::OuterStride<>(block.outerStride()));
In matlab/octave pairwise distances between matrices as required for e.g. k-means are calculated by one function call (see cvKmeans.m), to distFunc(Codebook, X) with as arguments two matrices of dimensions KxD.
In Eigen this can be done for a matrix and one vector by using broadcasting, as explained on eigen.tuxfamily.org:
(m.colwise() - v).colwise().squaredNorm().minCoeff(&index);
However, in this case v is not just a vector, but a matrix. What's the equivalent oneliner in Eigen to calculate such pairwise (Euclidean) distances across all entries between two matrices?
I think the appropriate solution is to abstract this functionality into a function. That function may well be templated; and it may well use a loop - the loop will be really short, after all. Many matrix operations are implemented using loops - that's not a problem.
For example, given your example of...
MatrixXd p0(2, 4);
p0 <<
1, 23, 6, 9,
3, 11, 7, 2;
MatrixXd p1(2, 2);
p1 <<
2, 20,
3, 10;
then we can construct a matrix D such that D(i,j) = |p0(i) - p1(j)|2
MatrixXd D(p0.cols(), p0.rows());
for (int i = 0; i < p1.cols(); i++)
D.col(i) = (p0.colwise() - p1.col(i)).colwise().squaredNorm().transpose();
I think this is fine - we can use some broadcasting to avoid 2 levels of nesting: we iterate over p1's points, but not over p0's points, nor over their dimensions.
However, you can make a oneliner if you observe that |p0(i) - p1(j)|2 = |p0(i)|2 + |p1(j)|2 - 2 p0(i)T p1(j). In particular, the last component is just matrix multiplication, so D = -2 p0T p1 + ...
The blank left to be filled is composed of a component that only depends on the row; and a component that only depends on the column: these can be expressed using rowwise and columnwise operations.
The final "oneliner" is then:
D = ( (p0.transpose() * p1 * -2
).colwise() + p0.colwise().squaredNorm().transpose()
).rowwise() + p1.colwise().squaredNorm();
You could also replace the rowwise/colwise trickery with an (outer) product with a 1 vector.
Both methods result in the following (squared) distances:
1 410
505 10
32 205
50 185
You'd have to benchmark which is fastest, but I wouldn't be surprised to see the loop win, and I expect that's more readable too.
Eigen is more of a headache than I thought on first sight.
There is no reshape() functionality for example (and conservativeResize is something else).
It also seems (I'd like to be corrected) to be the case that Map does not just offer a view on the data, but assignments to temporary variables seem to be required.
The minCoeff function after the colwise operator cannot return a minimum element and an index to that element.
It is unclear to me if replicate is actually allocating duplicates of the data. The reason behind broadcasting is that this is not required.
matrix_t data(2,4);
matrix_t means(2,2);
// data points
data << 1, 23, 6, 9,
3, 11, 7, 2;
// means
means << 2, 20,
3, 10;
std::cout << "Data: " << std::endl;
std::cout << data.replicate(2,1) << std::endl;
column_vector_t temp1(4);
temp1 = Eigen::Map<column_vector_t>(means.data(),4);
std::cout << "Means: " << std::endl;
std::cout << temp1.replicate(1,4) << std::endl;
matrix_t temp2(4,4);
temp2 = (data.replicate(2,1) - temp1.replicate(1,4));
std::cout << "Differences: " << std::endl;
std::cout << temp2 << std::endl;
matrix_t temp3(2,8);
temp3 = Eigen::Map<matrix_t>(temp2.data(),2,8);
std::cout << "Remap to 2xF: " << std::endl;
std::cout << temp3 << std::endl;
matrix_t temp4(1,8);
temp4 = temp3.colwise().squaredNorm();
std::cout << "Squared norm: " << std::endl;
std::cout << temp4 << std::endl;//.minCoeff(&index);
matrix_t temp5(2,4);
temp5 = Eigen::Map<matrix_t>(temp4.data(),2,4);
std::cout << "Squared norm result, the distances: " << std::endl;
std::cout << temp5.transpose() << std::endl;
//matrix_t::Index x, y;
std::cout << "Cannot get the indices: " << std::endl;
std::cout << temp5.transpose().colwise().minCoeff() << std::endl; // .minCoeff(&x,&y);
This is not a nice oneliner and seems overkill just to compare every column in data with every column in means and return a matrix with their differences. However, the versatility of Eigen does not seem to be such that this can be written down much shorter.
can someone please explain to me in detail why the following code for vectorY will do the assignment but the size of VecY is zero? Also, the begin and end iterators are stuck at the first node. It seems that reserve only works with push back and that you need to construct the vector with the size if you want the iterators for the vectors and the size to work as expected. I am assuming that push_back is doing some type of allocation that the straight assignment is not in this case? I am looking for details explaining this so I can make sure I understand what is happening with a reserve and push_back versus constructing with a size element and then doing assignment as in VecX example.
#include <iostream>
#include <vector>
int main ( int argc, char *argv[])
{
std::vector<int> vecX(2);
vecX[0] = 1;
vecX[1] = 2;
std::cout << " VecX0 Item: " << vecX[0] << std::endl;
std::cout << " VecX1 Item: " << vecX[1] << std::endl;
std::cout << " VectorX Size: " << vecX.size() << std::endl;
std::vector<int> vecY;
vecY.reserve(2);
vecY[0] = 1;
vecY[1] = 2;
std::cout << " VecY0 Item: " << vecY[0] << std::endl;
std::cout << " VecY1 Item: " << vecY[1] << std::endl;
std::cout << " VectorY Size: " << vecY.size() << std::endl;
}
Output
VecX0 Item: 1
VecX1 Item: 2
VectorX Size: 2
VecY0 Item: 1
VecY1 Item: 2
VectorY Size: 0
std::vector<int> vecY;
vecY.reserve(2);
vecY[0] = 1;
vecY[1] = 2;
This code is wrong and evokes Undefined Behavior1. When you reserve a vector, you set the capacity, not the size.
You need to either push_back, or construct the vector as you did in example 1.
"Undefined Behavior" : This invokes Undefined Behavior because of the out-of-range call to operator[] If you call vector::operator[n] where n > vec.size(), the behavior is Undefined.
If you don't want use push_back nor construct, consider using the resize method