Eigen, how to access the underlying array of a MatrixBase<Derived> - c++

I need to access the array that contains the data of a MatrixBase Eigen matrix.
The Eigen library has the data() method which returns a pointer to an array, however it is only accessible from a Matrix type. The MatrixBase doesn't have a similar method, even though the MatrixBase class is supposed to act as a template and the actual type should be just a Matrix. If I try to access MatrixBase.data() I get a compile time error:
template <typename ScalarA, typename Index, typename DerivedB, typename DerivedC>
void uscgemv(float alpha,
const USCMatrix<ScalarA,Index> &a,
const MatrixBase<DerivedB> &b,
const MatrixBase<DerivedC> &c_const)
{
//...some code
float * bMat = b.data();
///more code
}
This code produces the following compile time error.
error: ‘const class Eigen::MatrixBase<Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<float>, Eigen::Matrix<float, -1, 1> > >’ has no member named ‘data’
float * bMat = b.data();
So I have to resort to gimmicks such as...
float * bMat;
int bRows = b.rows();
int bCols = b.cols();
mallocPinnedMemory(&bMat, bRows*bCols*sizeof(float));
Eigen::Map<Matrix<float, Dynamic, Dynamic> > bmat_temp(bMat, bRows, bCols);
bmat_temp = b; //THis is SLOW, we should avoid it.
Then I can access the bMat array...
Those copies back-and-forth are the biggest cost in the gpu matrix multiplication, as I essentially I have to make an extra copy, before even coping to the device...
I can't use Eigen-magma, as this is sparse matrix-in-a-weird-format to a dense matrix (or sometimes vector) multiplication so I can't use any of the automatic gpu functions there. Also I would much rather not declare the matrices as something else, because that would require changing A LOT of lines of code across the whole program (which I didn't write).
EDIT: A static cast solution was proposed:
float * bMat = (static_cast<Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic> >(b)).data();
However I get segfault the first time I try to access an element of the array bMat.
EDIT 2: I'm looking for a zero copy way to access the underlying arrays. I need to only be able to read b, but I also need to able to write to c. Currently c is unconst-d with the following macro:
#define UNCONST(t,c,uc) Eigen::MatrixBase<t> &uc = const_cast<Eigen::MatrixBase<t>&>(c);
EDIT 3: After cross posting to Eigen Forums it would seem I can't do better than the suggested answer.

MatrixBase is the base class of any dense expression. It does not necessarily correspond to an object with storage. For instance, can be the abstract representation of A+B, or in your case the abstract representation of a vector with constant values. You can make uscgemv accepts only expression having appropriate storage using the Ref<> class, e.g.:
template <typename ScalarA, typename Index>
void uscgemv(float alpha,
const USCMatrix<ScalarA,Index> &a,
Ref<const VectorXf> b,
Ref<VectorXf> c);
If the third argument does not match the storage of a VectorXf then it will be evaluated for you. Then you can safely call b.data(). To keep the scalar type of b generic, you can still declare it as MatrixBase<DerivedB>& and then copy it into a Ref<const Matrix<typename DerivedB::Scalar, DerivedB::RowsAtCompileTime, DerivedB::ColsAtCompileTime> >:
typedef Ref<const Matrix<typename DerivedB::Scalar, DerivedB::RowsAtCompileTime, DerivedB::ColsAtCompileTime> > RefB;
RefB actual_b(b);
actual_b.data();

I guess the issue is this: you are not allowed to get a pointer to data of a MatrixBase<Derived>, since the latter can be any kind of expression in Eigen, like a product of matrices for example. To get a pointer you probably have to first implicitly convert the MatrixBase<Derived> into a Matrix<Scalar, Dynamic, Dynamic>, then use the data() member of the latter.
So you can create a deep copy of the expression, i.e. use something like
Eigen::Matrix<typename Derived::Scalar, Eigen::Dynamic, Eigen::Dynamic tmp = b;
then use
tmp.data()
This code works now
#include <Eigen/Dense>
#include <iostream>
template<typename Derived>
void use_data\
(const Eigen::MatrixBase<Derived>& mat)
{
Eigen::Matrix<typename Derived::Scalar, Eigen::Dynamic, Eigen::Dynamic>tmp = mat();
typename Derived::Scalar* p = tmp.data();
std::cout << std::endl;
for(std::size_t i = 0; i < tmp.size(); i++)
std::cout << *(p+i) << " ";
}
int main()
{
Eigen::MatrixXd A = Eigen::MatrixXd::Random(2, 2);
Eigen::MatrixXd B = Eigen::MatrixXd::Random(2, 2);
// now A*B is an expression, of type MatrixBase<EigenSum....>
use_data(A + B);
}

There are an easy solution to solve your question, combine EigenMap, &a(0, 0) and const_cast you could resue the buffer of the MatrixBase.
Example :
template<typename Derived1,
typename Derived2>
void example(Eigen::MatrixBase<Derived1> const &input,
Eigen::MatrixBase<Derived2> const &output)
{
static_assert(std::is_same<Derived1::Scalar, Derived2::Scalar>::value,
"Data type of matrix input, weight, bias and output should be the same");
using Scalar = typename Derived3::Scalar;
using MatType = Eigen::Matrix<Scalar, Eigen::Dynamic, 1>;
using Mapper = Eigen::Map<const MatType, Eigen::Aligned>;
//in the worst case, you can do const_cast<Scalar *> on
//&bias(0, 0).That is, if you cannot explicitly define the Map
//type as const
Mapper Map(&input(0, 0), input.size());
output.colwise() += Map;
}
}
I try it on windows 8, vc2013 32bits, Eigen version is 3.2.5, no segmentation fault occur(yet), every things looks perfectly fine. I also check the address of the Map, it is same as the original input. You can verify it with another example
#include <Eigen/Dense>
#include <iostream>
template<typename Derived>
void example_2(Eigen::MatrixBase<Derived> &input)
{
using Scalar = decltype(input[0]);
Eigen::Map<Derived> map(&input(0, 0),
input.rows(),
input.cols());
map(0, 0) = 300;
}
int main()
{
Eigen::MatrixXd mat(2, 2);
mat<<0, 1, 2, 3;
example_2(mat);
std::cout<<mat<<"\n\n";
return 0;
}
The first element of mat will be "300"

Related

Eigen: function signature which accepts general matrix expression of fixed size and type

The Eigen documentation is filled with examples illustrating how one should write a general function accepting a matrix:
template <typename Derived>
void print_cond(const MatrixBase<Derived>& a)
The reason to use MatrixBase as opposed to Matrix is that all dense Eigen matrix expressions derive from MatrixBase. So, for instance, if I pass a block of a matrix
print_cond ( A.block(...));
Then using the signature const MatrixBase<Derived>& a avoids creating a temporary. Conversely, if we had declared the function with the signature
template <typename T, int rows, int cols>
void print_cond(const Matrix<T,rows,cols>& a)
then Eigen would have to convert the block type to a matrix before passing it to the function, meaning that an unnecessary temporary would have to be created.
Please correct me if this understanding is incorrect...
With that in mind, one of the benefits of the second approach is that we can get compile time checks on the dimensions of the matrix (assuming they are fixed, not dynamic).
What I can't find in the documentation is an example with the generality of the first approach (which helps avoid temporary creation), but which has complie time checks on the type and dimensions of the matrix. Could somebody please tell me how to do that?
Just for completeness, Marc and ggael are suggesting something like this
#include <iostream>
#include "Eigen/Dense"
using namespace Eigen;
using T = double;
const int rows = 5;
const int cols = 3;
template<typename Derived>
void print_cond(const MatrixBase <Derived> &a) {
/* We want to enforce the shape of the input at compile-time */
static_assert(rows == Derived::RowsAtCompileTime);
static_assert(cols == Derived::ColsAtCompileTime);
/* Now that we are guaranteed that we have the
* correct dimensions, we can do something... */
std::cout << a;
}
int main() {
print_cond(Matrix<T, rows, cols>::Ones());
/* These will not compile */
// print_cond(Matrix<T, rows + 1, cols>::Ones());
// print_cond(Matrix<T, rows, cols + 1>::Ones());
// print_cond(Matrix<T, rows + 1, cols + 1>::Ones());
return 0;
}

reduction of eigen based templates

I am trying to do a reduction based on eigen matrix.
#include <iostream>
#include <Eigen/Dense>
#include <type_traits>
template<typename T1, typename T2, int n1, int n2>
auto reduction(Eigen::Matrix<T1, n1, n2> &a1,
Eigen::Matrix<T2, n1, n2> &a2)
-> decltype(T1{}*T2{})
{
using BaseT3 =
typename std::remove_cv<typename std::remove_reference<decltype(T1{}*T2{})>::type>::type;
BaseT3 res = a1(0, 0)*a2(0, 0);
for (int i=0; i<n1; ++i)
for (int j=0; j<n2; ++j)
if (i+j)
res = res + a1(i, j)*a2(i, j);
return res;
}
int main()
{
Eigen::Matrix<double, 3, 3> m;
Eigen::Matrix<Eigen::Vector3d, 3, 3> n;
std::cout << reduction(m, n) << std::endl;
}
Basically, Im a trying to get sum_{i, j} a1[i, j] * a2[i, j] where a1 and a2 are some eigen mathix but I get compilation errors. The error I get is
error: no match for ‘operator=’ (operand types are ‘BaseT3 {aka
Eigen::CwiseUnaryOp<Eigen::internal::scalar_multiple_op<double>,
const Eigen::Matrix<double, 3, 1> >}’
and
‘const Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double>,
const Eigen::CwiseUnaryOp<Eigen::internal::scalar_multiple_op<double>,
const Eigen::Matrix<double, 3, 1> >,
const Eigen::CwiseUnaryOp<Eigen::internal::scalar_multiple_op<double>,
const Eigen::Matrix<double, 3, 1> > >’)
res = res + a1(i, j)*a2(i, j);
^
If I am not mistaken, for the given main, type BaseT3 should have been Eigen::Vector3d. I also tried to static cast so the operator= should not fail but I then get other errors.
This is c++11, I use Eigen3 and the compiler is g++ 5.4.1.
The decltype of T1 * T2 isn't what you expect here - Eigen heavily uses expression templates. The CWiseUnaryOp and CWiseBinaryOp types in your error are indicative of that. In other words, the result of "double * Vector3d" isn't what you'd expect (it's not a Vector3d, it's a cwisebinaryop).
See also: Writing functions taking Eigen Types.
In this specific case you may find a solution by creating partial specializations for Eigen base types for both the first and second parameters of your template function.
Eigen uses expression templates to optimize chains of operations.
So MatrixA*MatrixB is not a Matrix type, but rather an expression that says "when evaluated this will be the product of a MatrixA times a MatrixB".
The result is that A*B+C*D doesn't create (at least as many) temporary matrixes, but instead when stored in an output matrix the results are "lazily" calculated directly into the output matrix.
Now, you are multipying elements. But one of your element types is in turn a matrix. And Eigen does expression template optimization of scalar times vector it turns out.
The type you want is std::decay_t<decltype((T1{}+T2{}).eval())> (well the C++11 verbose version of that).
You could write a fancy SFINAE thing that checks if it can be evaluated, and if so does that. Or you can test for Eigen expression template types.

Eigen binaryExpr with eigen type output

I'm having a problem while trying to use binaryExpr. It is the first use I'm making of it so I have been following the Eigen documentation
For my use I need a functor with Eigen type inputs and outputs but this does not want to compile and I do not understand why. I've looked up the explanation in the code but I didn't think this would apply here because I use floats and an array of floats
// We require Lhs and Rhs to have "compatible" scalar types.
// It is tempting to always allow mixing different types but remember that this is often impossible in the vectorized paths.
// So allowing mixing different types gives very unexpected errors when enabling vectorization, when the user tries to
// add together a float matrix and a double matrix.
Here is a short example of the use I would need that gets me the same compilation error:
#include <eigen3/Eigen/Dense>
using namespace std;
using namespace Eigen;
struct myBinaryFunctor {
EIGEN_EMPTY_STRUCT_CTOR(myBinaryFunctor)
typedef Vector2f result_type;
Vector2f operator()(const Matrix<float,9,1>& a,const float& f) const
{
float x = a.head(4).sum()*f;
float y = a.tail(5).sum()/f;
return Vector2f(x,y);
}
};
int main()
{
constexpr int n = 3;
Matrix<Matrix<float,9,1>,n,n> Ma;
Matrix<float,n,n> F;
Matrix<Vector2f,n,n> R;
for(size_t i = 0, sizeMa = Ma.size(); i<sizeMa; i++)
{
Ma(i).setOnes();
}
F.setConstant(n,n,2);
R = Ma.binaryExpr(F,myBinaryFunctor());
return 0;
}
The compilation output is :
/usr/local/include/eigen3/Eigen/src/Core/CwiseBinaryOp.h:107: erreur : static assertion failed: YOU_MIXED_DIFFERENT_NUMERIC_TYPES__YOU_NEED_TO_USE_THE_CAST_METHOD_OF_MATRIXBASE_TO_CAST_NUMERIC_TYPES_EXPLICITLY
EIGEN_CHECK_BINARY_COMPATIBILIY(BinaryOp,typename Lhs::Scalar,typename Rhs::Scalar);
^
If you have a solution that could make this work this would be a huge help for me :) If not I would still enjoy an explanation to understand what is happening. Thanks a lot.
Adding:
namespace Eigen {
template<>
struct ScalarBinaryOpTraits<Matrix<float,9,1>,float,myBinaryFunctor> {
typedef Vector2f ReturnType;
};
}
will do the job. This is because implicit scalar conversion are explicitly disallowed within Eigen, so you must explicit say that two different scalar types are compatible. For instance adding a VectorXd to a VectorXf is disallowed.
Nonetheless, it seems to me that your abusing Eigen's features here.

Passing fixed-size Eigen matrices as arguments to function calling for dynamic-size matrices

I am writing a small linear algebra utility library on top of Eigen for my personal code-base. To try to make it as flexible as possible, I typedefed different Eigen matrix types for use as parameters. However, an issue I keep running into is that when I use it, I can't pass a fixed-size (i.e. set at compile-time) matrix as the argument to a function that has a dynamically-sized (set at runtime) matrix typedef as a parameter. I could understand the inverse--not being able to pass a dynamic-sized matrix as fixed due to compile-time checks, but it seems this ought to work.
A test-able example is the pdist2 function below (which really ought to have a native implementation in the Eigen API).
#include <Eigen/Core>
namespace Eigen
{
template <typename T>
using MatrixXT = Eigen::Matrix<T, Eigen::Dynamic, Eigen::Dynamic>;
}
// X is M x N
// Y is M x K
// Output is N x K
template <typename T>
inline Eigen::MatrixXT<T> pdist2(const Eigen::MatrixXT<T> &X, const Eigen::MatrixXT<T> &Y)
{
// ASSERT(X.rows() == Y.rows(), "Input observations must have same number of rows (" +
// std::to_string(X.rows()) + "!=" + std::to_string(Y.rows()) + ")");
Eigen::MatrixXT<T> dists = X.colwise().squaredNorm().transpose() * Eigen::MatrixXT<T>::Ones(1, Y.cols()) +
Eigen::MatrixXT<T>::Ones(X.cols(), 1) * Y.colwise().squaredNorm() -
2 * X.transpose() * Y;
return dists;
}
This code does not compile:
Eigen::Matrix<double, 3, 5> X;
X << 8.147236863931790, 9.133758561390193, 2.784982188670484, 9.648885351992766, 9.571669482429456,
9.057919370756192, 6.323592462254095, 5.468815192049838, 1.576130816775483, 4.853756487228412,
1.269868162935061, 0.975404049994095, 9.575068354342976, 9.705927817606156, 8.002804688888002;
Eigen::Matrix<double, 3, 4> Y;
Y << 1.418863386272153, 7.922073295595544, 0.357116785741896, 6.787351548577734,
4.217612826262750, 9.594924263929030, 8.491293058687772, 7.577401305783335,
9.157355251890671, 6.557406991565868, 9.339932477575505, 7.431324681249162;
Eigen::Matrix<double, 5, 4> D = pdist2(X, Y);
The above function has been unit tested and evaluates correctly, but it will only work if X and Y are Eigen::MatrixXd types. It seems it must be my template typedef causing the issue, but it's just a dynamically (i.e. at runtime) sized matrix with a templated type.
The error reads:
error: no matching function for call to ‘pdist2(Eigen::Matrix<double, 3, 5>&, Eigen::Matrix<double, 3, 4>&)’
Eigen::Matrix<double, 5, 4> D = Util::Math::pdist2(X, Y);
^
note: candidate: template<class T> Eigen::MatrixXT<T> Util::Math::pdist2(Eigen::MatrixXT<T>&, Eigen::MatrixXT<T>&)
inline Eigen::MatrixXT<T> pdist2(const Eigen::MatrixXT<T> &X, const Eigen::MatrixXT<T> &Y)
^
note: template argument deduction/substitution failed:
note: template argument ‘3’ does not match ‘#‘integer_cst’ not supported by dump_decl#<declaration error>’
Eigen::Matrix<double, 5, 4> D_est = Util::Math::pdist2(X, Y);
^
note: ‘Eigen::Matrix<double, 3, 5>’ is not derived from ‘Eigen::MatrixXT<T>’
Why is this not working? Or perhaps more specifically, how do I use a templated typedef to ensure that my fixed-size matrices do derive from Eigen::MatrixXT<T>?
Note: This is all using Eigen 3.3.3.
The problem is that Matrix<double, 3, 5> and MatrixXT<double> are not the same type, therefore, the only way to pass the former to pdist2 is to convert it to a MatrixXT<double>. This would be done automatically by the compiler if pdist2 were not a template function:
MatrixXT<double> pdist2(const MatrixXT<double>&,const MatrixXT<double>&);
but since pdist2 is templated and that MatrixXT<double> is not a base class of Matrix<double, 3, 5>, in C++ the compiler is not allowed to automatically deduced the template parameter to instantiate pdist2. The solution for you is to generalize even more your function to take any MatrixBase<> as inputs:
template<typename D1,typename D2>
Matrix<typename D1::Scalar,D1::ColsAtCompileTime,D2::ColsAtCompileTime>
pdist2(const MatrixBase<D1>& _X,const MatrixBase<D2>& _Y);
Since X and Y are going to be used multiple times, and that they can now be arbitrary expressions (including a costly matrix product), you might want to let Eigen evaluate the arguments if needed, to this end, you can use Ref:
Ref<const typename D1::PlainObject> X(_X);
Ref<const typename D2::PlainObject> Y(_Y);
This way X will be evaluated if it cannot be represented as a pointer+stride to actual values.

Removing potentially redundant template parameters when loading a binary file into Eigen matrix

I want to read a plain binary file containing a number of unsigned 16-bit integers into an Eigen matrix, and I wrote a templated utility to do this. This is what the caller looks like:
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data<Matrix<uint16_t, Dynamic, Dynamic>, uint16_t>(
argv[1], data);
And here's what read_data looks like:
template <typename Derived, typename Scalar> // Per #Jarod42, get rid of Scalar here (*)
int read_data(const char* const fname, MatrixBase<Derived>& data) {
// (*) If we don't have Scalar as a template, just uncomment this:
// typedef typename Derived::Scalar Scalar;
ifstream fin(fname, ios::binary);
if (!fin) {
return 2;
}
fin.seekg(0, fin.end);
long long bytes = fin.tellg();
if (bytes % sizeof(Scalar) != 0) {
// The available number of bytes won't fill an even number of Scalar
// values
return 3;
}
long long nscalars = bytes / sizeof(Scalar);
// See http://forum.kde.org/viewtopic.php?f=74&t=107551
MatrixBase<Derived>& data_edit = const_cast<MatrixBase<Derived>&>(data);
data_edit.derived().resize(nscalars, 1);
Scalar* buffer = new Scalar[nscalars]; // Switched to vector per #Casey
fin.seekg(0, fin.beg);
fin.read(reinterpret_cast<char*>(buffer), bytes);
if (!fin) {
// All data not read. fin.gcount() will indicate bytes read.
return 4;
}
for (long long idx = 0; idx < nscalars; ++idx) {
data_edit(idx) = buffer[idx];
}
return 0;
}
In brief,
the file is opened,
its size is obtained,
an array is dynamically allocated to store all the data,
the file is read into the array,
the array's contents are copied into the matrix.
This is reasonable and it works (though I'm open to suggestions for improvements), but I think the function has one too many template parameters, and the function call in the caller is just too verbose. I think there should be a way to eliminate the second template parameter, which only serves to tell read_data the number of bytes per scalar (2 in the case of uint16_t), and which I believe should be inferrable using the first template parameter.
Questions Is there a way to eliminate the seemingly redundant second template parameter to read_data?
Also, is my approach of passing in a matrix reference only to resize it in the read_data function (using the verbose and confusing idiom of creating a modifiable reference to the matrix in order to resize it via derived()) the right way to proceed? I realize this dynamically allocates memory, which is fine, but I think it is not doing anything wasteful---correct?.
(Discussion Is there other improvements to this code one would like to see? I'm a C or Python numerical coder; in C, I'd just deal with void* arrays and pass an extra function argument telling the function the size of each scalar; with Python I'd just do numpy.fromfile('path/to/file.bin', dtype=numpy.uint16) and be done with it. But I'd like to do it right by Eigen and C++.)
NB. I use matrixes instead of vectors because I'll be resizing them into rectangular matrixes later.
NB2. In Fixed Sized Eigen types as parameters the notion of templating the function using the scalar type is promoted. I am not averse to this approach, I chose to pass read_data a matrix reference instead of making it return a Matrix object because I wanted integer return values indicating errors---though now I realize I ought to make those exceptions.
NB3. In c++ check for nested typedef of a template parameter to get its scalar base type an elaborate set of helper classes is used to, I think, achieve a similar effect for templated classes. I'm curious if a simpler approach can be used here, for a templated function.
NB4. A simple improvement that I'm aware of is typedeffing Matrix<uint16_t, Dynamic, Dynamic> to reduce verbosity.
I think that Matrix<uint16_t, Dynamic, Dynamic> should have a typedef to retrieve uint16_t. If so, you may write as first line (assuming it is type):
typedef typename Derived::type Scalar;
If there is no typedef, you may write a traits for that:
template <typename> struct matrix_type;
template <typename T, typename U1, typename U2>
struct matrix_type<Matrix<T, U1, U2> >
{
typedef T type;
typedef U1 type1;
typedef U2 type2;
};
And then write
typedef typename matrix_type<Derived>::type Scalar;
Note:
If you have written the parameters in the other order
template <typename Derived, typename Scalar>
int read_data(const char* const fname, MatrixBase<Derived>& data);
you may write (as the last parameter may be deduced):
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data<uint16_t>(argv[1], data);
And now that the parameter is obsolete, you may write directly:
Matrix<uint16_t, Dynamic, Dynamic> data;
int retval = read_data(argv[1], data);
You may want to look at map in eigen. It takes a chuck a memory directly.
TutorialMapClass.html">http://eigen.tuxfamily.org/dox/group_TutorialMapClass.html
To construct a Map variable, you need two other pieces of information: a pointer to the region of memory defining the array of coefficients, and the desired shape of the matrix or vector. For example, to define a matrix of float with sizes determined at compile time, you might do the following:
Map mf(pf,rows,columns);