Fast way to slice an Eigen SparseMatrix - c++

In finite element analyses it is quite common to apply some prescribed condition(s) to a big sparse matrix and get a reduced one. This can be achieved easily in MATLAB, SciPy and Julia, for instance, in MATLAB
a=sprand(10000,10000,0.2); % create a random sparse matrix; fill %20
tic; c=a(1:2:4000,2:3:5000); toc % slice the matrix to get a reduced one
Assuming that one has a similar sparse matrix in Eigen, what is the most efficient way to slice an Eigen matrix. I don't care about a copy or a view, but the methodology needs to be able to cope up with non-contiguous slicing. The latter requirement makes the Eigen block operations useless in this regard.
I can think of two methodologies that I have tested:
Iterate over the columns and rows using for loops and assign the values to a second sparse matrix (I know this is a truly bad idea).
Create a dummy sparse matrix with zeros and ones and pre and post multiply it with the actual matrix D*A*A.transpose()
I always use setFromTriplets to create a sparse matrices in Eigen and I have been happy with the solvers and assembling of sparse matrices. However it seems that this slicing is the bottleneck in my code at the moment
The timing of MATLAB vs Eigen (using -O3 -DNDEBUG -march=native) is
MATLAB: 0.016 secs
EIGEN LOOP INDEXING: 193 secs
EIGEN PRE-POST MUL: 13.7 secs
The other methodology that I do not know how to go about is to manipulate directly the [I,J,V] triplets outerIndexPtr, innerIndexPtr, valuePtr.
Here is a proof of concept code snippet
#include <Eigen/Core>
#include <Eigen/Sparse>
template<typename T>
using spmatrix = Eigen::SparseMatrix<T,Eigen::RowMajor>;
spmatrix<double> sprand(int rows, int cols, double sparsity) {
std::default_random_engine gen;
std::uniform_real_distribution<double> dist(0.0,1.0);
int sparsity_ = sparsity*100;
typedef Eigen::Triplet<double> T;
std::vector<T> tripletList; tripletList.reserve(rows*cols);
int counter = 0;
for(int i=0;i<rows;++i) {
for(int j=0;j<cols;++j) {
if (counter % sparsity_ == 0) {
auto v_ij=dist(gen);
tripletList.push_back(T(i,j,v_ij));
}
counter++;
}
}
spmatrix<double> mat(rows,cols);
mat.setFromTriplets(tripletList.begin(), tripletList.end());
return mat;
}
int main() {
int m=1000,n=10000;
auto a = sprand(n,n,0.05);
auto b = sprand(m,n,0.1);
spmatrix<double> c;
// this is efficient but definitely not the right way to do this
// c = b*a*b.transpose(); // uncomment to check, much slower than block operation
c = a.block(0,0,1000,1000); // very fast, Faster than MATLAB (I believe this is just a view)
return 0;
}
So Any pointers, in this direction would be useful.

Related

How to efficiently initialize a SparseVector in Eigen

In the Eigen docs for filling a sparse matrix it is recommended to use the triplet filling method as it can be much more efficient than making calls to coeffRef, which involves a binary search.
For filling SparseVectors however, there is no clear recommendation on how to do it efficiently.
The suggested method in this SO answer uses coeffRef which means that a binary search is performed for every insertion.
Is there a recommended, efficient way to build sparse vectors? Should I try to create a single row SparseMatrix and then store that as a SparseVector?
My use case is reading in LibSVM files, in which there can be millions of very sparse features and billions of data points. I'm currently representing these as an std::vector<Eigen::SparseVector>. Perhaps I should just use SparseMatrix instead?
Edit: One thing I've tried is this:
// for every data point in a batch do the following:
Eigen::SparseMatrix<float> features(1, num_features);
// copy the data over
typedef Eigen::Triplet<float> T;
std::vector<T> tripletList;
for (int j = 0; j < num_batch_instances; ++j) {
for (size_t i = batch.offset[j]; i < batch.offset[j + 1]; ++i) {
uint32_t index = batch.index[i];
float fvalue = batch.value;
if (index < num_features) {
tripletList.emplace_back(T(0, index, fvalue));
}
}
features.setFromTriplets(tripletList.begin(), tripletList.end());
samples->emplace_back(Eigen::SparseVector<float>(features));
}
This creates a SparseMatrix using the triplet list approach, then creates a SparseVector from that object. In my experiments with ~1.4M features and very high sparsity this is 2 orders of magnitude slower than using SparseVector and coeffRef, which I definitely did not expect.

C++ Matrix product: increased speed with little changes

I'm writing a code to multiply a sparse matrix with a full matrix.
I've created 2 class: SparseMatrix and Matrix, which store datas as a vector of shared pointers to vectors. In the SparseMatrix case i save item as an object, called SparseMatrixItem with 2 attributes: position and values. In the Matrix case I simply save the value.
They can both be row or column based, by the value of a bool attribute.
Now I'm trying to write an efficient version of the standard product between the 2 matrixes. By semplicity in the first implementation I consider only the case in which the first matrix is a row based SparseMatrix and the second is a column based Matrix. I write the code into the class SparseMatrix by overloading the operator *.
I post my implementation:
template <typename scalar>
Matrix<scalar> SparseVectorMatrix<scalar>::operator*(Matrix<scalar> &input2) {
Matrix<scalar> newMatrix(getNumberOfRows(),input2.getNumberOfColumns(),true);
int numberOfRow=newMatrix.getNumberOfRows();
int numberOfColumn=newMatrix.getNumberOfColumns();
for (int i=0; i<numberOfRow; i++) {
vector<SparseMatrixItem<scalar>>& readRow(*horizontalVectorMatrix[i]);
vector<scalar>& writeRow(*newMatrix.internalMatrix[i]);
for (int j=0; j<numeroColonne; j++) {
vector<scalar>& readColumn1(*input2.internalMatrix[j]);
writeRow[j]=fastScalarProduct(readRow, readColumn1);
}
}
}
The strange fact I cannot figure out is that if I change the 2 loop order performance are dramatically faster.
I test it with 2 matrix: 6040x4000 and 4000*6040, the first implementation tooks nearly 30 seconds,while the second implementation tooks only 12 seconds.
I post it:
template <typename scalar>
Matrix<scalar> SparseVectorMatrix<scalar>::operator*(Matrix<scalar> &input2) {
Matrix<scalar> newMatrix(getNumberOfRows(),input2.getNumberOfColumns(),true);
int numberOfRow=newMatrix.getNumberOfRows();
int numeroColonne=newMatrix.getNumberOfColumns();
for (int j=0; j<numeroColonne; j++) {
vector<scalar>& readColumn(*input2.internalMatrix[j]);
vector<scalar>& writeColumn(*newMatrix.internalMatrix[j]);
for (int i=0; i<numberOfRow; i++) {
vector<SparseMatrixItem<scalar>>& readRow(*matriceOrizzontaleVettori[i]);
writeColumn[i]=fastScalarProduct(readRow, readColumn);
}
}
}
I post also the code of the function fastScalarProduct() that I use:
template <typename scalar>
scalar SparseVectorMatrix<scalar>::fastScalarProduct
( vector<SparseMatrixItem<scalar>> &vector1
, const vector<scalar> &vector2
) {
int totalSum=0;
int position;
auto sizeVector1=vector1.size();
for (int i=0; i<sizeVector1; i++) {
position=vector1[i].position-1;
if (vector2[position]) {
totalSum+=(vector1[i].value)*vector2[position];
}
}
return totalSum;
}
I try the same product with MATLAB and it takes only 1.5 seconds more or less. I think that there are issues with cache memory, but since I'm new to this kind of problems I cannot figure out the real problem.
I'm also trying to write an efficient full matrix product, and I'm facing the same problems.
You are right in saying the "issue" is with cache memory. I suggest you read about locality of reference (http://en.wikipedia.org/wiki/Locality_of_reference) which explains why your program runs faster when the loop with the most iterations is inside the one with less iterations. Basically, arrays are linear data sctructures, and they make great use of spatial locality.
As for the time it took to run the algorithm in matlab vs C++, I suggest you read this post: Why is MATLAB so fast in matrix multiplication?

How can I calculate inverse of sparse matrix in Eigen library

I have a question about Eigen library in C++. Actually, I want to calculate inverse matrix of sparse matrix.
When I used Dense matrix in Eigen, I can use .inverse() operation to calculate inverse of dense matrix.
But in Sparse matrix, I cannot find inverse operation anywhere. Does anyone who know to calculate inverse of sparse matrix? help me.
You cannot do it directly, but you can always calculate it, using one of the sparse solvers. The idea is to solve A*X=I, where I is the identity matrix. If there is a solution, X will be your inverse matrix.
The eigen documentation has a page about sparse solvers and how to use them, but the basic steps are as follows:
SolverClassName<SparseMatrix<double> > solver;
solver.compute(A);
SparseMatrix<double> I(n,n);
I.setIdentity();
auto A_inv = solver.solve(I);
It's not mathematically meaningful.
A sparse matrix does not necessarily have a sparse inverse.
That's why the method is not available.
A small extension on #Soheib and #MatthiasB's answers, if you're using Eigen::SparseMatrix<float> it's better to use SparseLU rather than SimplicialLLT or SimplicialLDLT, they produced wrong answers with me on float matrices
Be warned that the inverse of a sparse matrix is not necessarily sparse, so if you're working with large matrices (which is likely, if you're using sparse representations) then this is going to be expensive. Think carefully about whether you really need the actual matrix inverse. If you're going to use the matrix inverse to solve a system of equations, then you don't need to actually compute the matrix inverse and multiply it out (use the method typically named solve and supply the right-hand-side of the equation). If you need the inverse of the Fisher matrix for covariances, try to approximate.
You can find a example about inverse of Sparse Complex Matrix
I used of SimplicialLLT class,
you can find other class from bellow
http://eigen.tuxfamily.org/dox-devel/group__TopicSparseSystems.html
This page can help you with proper class name for your work (spead, accuracy and dimmenssion of your Matrix)
////////////////////// In His Name \\\\\\\\\\\\\\\\\\\\\\\\\\\
#include <iostream>
#include <vector>
#include <Eigen/Dense>
#include <Eigen/Sparse>
using namespace std;
using namespace Eigen;
int main()
{
SparseMatrix< complex<float> > A(4,4);
for (int i=0; i<4; i++) {
for (int j=0; j<4; j++) {
A.coeffRef(i, i) = i+j;
}
}
A.insert(2,1) = {2,1};
A.insert(3,0) = {0,0};
A.insert(3,1) = {2.5,1};
A.insert(1,3) = {2.5,1};
SimplicialLLT<SparseMatrix<complex<float> > > solverA;
A.makeCompressed();
solverA.compute(A);
if(solverA.info()!=Success) {
cout << "Oh: Very bad" << endl;
}
SparseMatrix<float> eye(4,4);
eye.setIdentity();
SparseMatrix<complex<float> > inv_A = solverA.solve(eye);
cout << "A:\n" << A << endl;
cout << "inv_A\n" << inv_A << endl;
}

Eigen MatrixXd push back in c++

Eigen is a well known matrix Library in c++. I am having trouble finding an in built function to simply push an item on to the end of a matrix. Currently I know that it can be done like this:
Eigen::MatrixXd matrix(10, 3);
long int count = 0;
long int topCount = 10;
for (int i = 0; i < listLength; ++i) {
matrix(count, 0) = list.x;
matrix(count, 1) = list.y;
matrix(count, 2) = list.z;
count++;
if (count == topCount) {
topCount *= 2;
matrix.conservativeResize(topCount, 3);
}
}
matrix.conservativeResize(count, 3);
And this will work (some of the syntax may be out). But its pretty convoluted for a simple thing to do. Is there already an in built function?
There is no such function for Eigen matrices. The reason for this is such a function would either be very slow or use excessive memory.
For a push_back function to not be prohibitively expensive it must increase the matrix's capacity by some factor when it runs out of space as you have done. However when dealing with matrices, memory usage is often a concern so having a matrix's capacity be larger than necessary could be problematic.
If it instead increased the size by rows() or cols() each time the operation would be O(n*m). Doing this to fill an entire matrix would be O(n*n*m*m) which for even moderately sized matrices would be quite slow.
Additionally, in linear algebra matrix and vector sizes are nearly always constant and known beforehand. Often when resizeing a matrix you don't care about the previous values in the matrix. This is why Eigen's resize function does not retain old values, unlike std::vector's resize.
The only case I can think of where you wouldn't know the matrix's size beforehand is when reading from a file. In this case I would either load the data first into a standard container such as std::vector using push_back and then copy it into an already sized matrix, or if memory is tight run through the file once to get the size and then a second time to copy the values.
There is no such function, however, you can build something like this yourself:
using Eigen::MatrixXd;
using Eigen::Vector3d;
template <typename DynamicEigenMatrix>
void push_back(DynamicEigenMatrix& m, Vector3d&& values, std::size_t row)
{
if(row >= m.rows()) {
m.conservativeResize(row + 1, Eigen::NoChange);
}
m.row(row) = values;
}
int main()
{
MatrixXd matrix(10, 3);
for (std::size_t i = 0; i < 10; ++i) {
push_back(matrix, Vector3d(1,2,3), i);
}
std::cout << matrix << "\n";
return 0;
}
If this needs to perform too many resizes though, it's going to be horrendously slow.

Complex matrix exponential in C++

Is it actually possible to calculate the Matrix Exponential of a Complex Matrix in c / c++?
I've managed to take the product of two complex matrices using blas functions from the GNU Science Library. for matC = matA * matB:
gsl_blas_zgemm (CblasNoTrans, CblasNoTrans, GSL_COMPLEX_ONE, matA, matB, GSL_COMPLEX_ZERO, matC);
And I've managed to get the matrix exponential of a matrix by using the undocumented
gsl_linalg_exponential_ss(&m.matrix, &em.matrix, .01);
But this doesn't seems to accept complex arguments.
Is there anyway to do this? I used to think c++ was capable of anything. Now I think its outdated and cryptic...
Several options:
modify the gsl_linalg_exponential_ss code to accept complex matrices
write your complex NxN matrix as real 2N x 2N matrix
Diagonalize the matrix, take the exponential of the eigenvalues, and rotate the matrix back to the original basis
Using the complex matrix product that is available, implement the matrix exponential according to it's definition: exp(A) = sum_(n=0)^(n=infinity) A^n/(n!)
You have to check which methods are appropriate for your problem.
C++ is a general purpose language. As mentioned above, if you need specific functionality you have to find a library that can do it or implement it yourself. Alternatively you could use software like MatLab and Mathematica. If that's too expensive there are open source alternatives, e.g. Sage and Octave.
"I used to think c++ was capable of anything" - if a general-purpose language has built-in complex math in its core, then something is wrong with that language.
Fur such very specific tasks there is a well-accepted solution: libraries. Either write your own, or much better, use an already existing one.
I myself rarely need complex matrices in C++, I always used Matlab and similar tools for that. However, this http://www.mathtools.net/C_C__/Mathematics/index.html might be of interest to you if you know Matlab.
There are a couple other libraries which might be of help:
http://eigen.tuxfamily.org/index.php?title=Main_Page
http://math.nist.gov/lapack++/
I was also thinking to do the same, writing your complex NxN matrix as real 2N x 2N matrix is the best way to solve the problem, then use gsl_linalg_exponential_ss().
Suppose A=Ar+i*Ai, where A is the complex matrix and Ar and Ai are the real matrices. Then write the new matrix B=[Ar Ai ;-Ai Ar] (Here the matrix is written in matlab notation). Now calculate the exponential of B, that is eB=[eB1 eB2 ;eB3 eB4], then exponential of A is given by, eA=eB1+1i.*eB2
(summing the matrices eB1 and 1i.*eB2).
I have written a code to calculate the matrix exponential of the complex matrices with the gsl function, gsl_linalg_exponential_ss(&m.matrix, &em.matrix, .01);
Here you have the complete code, and the compilation results. I have checked the result with the Matlab and result agrees.
#include <stdio.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_linalg.h>
#include <gsl/gsl_complex.h>
#include <gsl/gsl_complex_math.h>
void my_gsl_complex_matrix_exponential(gsl_matrix_complex *eA, gsl_matrix_complex *A, int dimx)
{
int j,k=0;
gsl_complex temp;
gsl_matrix *matreal =gsl_matrix_alloc(2*dimx,2*dimx);
gsl_matrix *expmatreal =gsl_matrix_alloc(2*dimx,2*dimx);
//Converting the complex matrix into real one using A=[Areal, Aimag;-Aimag,Areal]
for (j = 0; j < dimx;j++)
for (k = 0; k < dimx;k++)
{
temp=gsl_matrix_complex_get(A,j,k);
gsl_matrix_set(matreal,j,k,GSL_REAL(temp));
gsl_matrix_set(matreal,dimx+j,dimx+k,GSL_REAL(temp));
gsl_matrix_set(matreal,j,dimx+k,GSL_IMAG(temp));
gsl_matrix_set(matreal,dimx+j,k,-GSL_IMAG(temp));
}
gsl_linalg_exponential_ss(matreal,expmatreal,.01);
double realp;
double imagp;
for (j = 0; j < dimx;j++)
for (k = 0; k < dimx;k++)
{
realp=gsl_matrix_get(expmatreal,j,k);
imagp=gsl_matrix_get(expmatreal,j,dimx+k);
gsl_matrix_complex_set(eA,j,k,gsl_complex_rect(realp,imagp));
}
gsl_matrix_free(matreal);
gsl_matrix_free(expmatreal);
}
int main()
{
int dimx=4;
int i, j ;
gsl_matrix_complex *A = gsl_matrix_complex_alloc (dimx, dimx);
gsl_matrix_complex *eA = gsl_matrix_complex_alloc (dimx, dimx);
for (i = 0; i < dimx;i++)
{
for (j = 0; j < dimx;j++)
{
gsl_matrix_complex_set(A,i,j,gsl_complex_rect(i+j,i-j));
if ((i-j)>=0)
printf("%d+%di ",i+j,i-j);
else
printf("%d%di ",i+j,i-j);
}
printf(";\n");
}
my_gsl_complex_matrix_exponential(eA,A,dimx);
printf("\n Printing the complex matrix exponential\n");
gsl_complex compnum;
for (i = 0; i < dimx;i++)
{
for (j = 0; j < dimx;j++)
{
compnum=gsl_matrix_complex_get(eA,i,j);
if (GSL_IMAG(compnum)>=0)
printf("%f+%fi\t ",GSL_REAL(compnum),GSL_IMAG(compnum));
else
printf("%f%fi\t ",GSL_REAL(compnum),GSL_IMAG(compnum));
}
printf("\n");
}
return(0);
}