I'm thinking of using Boost's Sparse Matrix for a computation where minimal memory usage is the goal. Unfortunately, the documentation page didn't include a discussion of the sparse matrix implementation's memory usage when I looked through it. Nor am I sure how to determine how much memory the sparse matrix is using at any given time.
How much memory will the sparse matrix use? Can you quote a source?
How can I find out how much memory the matrix is using at a given time t?
I cannot give you an exact answer. But generally speaking a sparse matrix uses an amount of memory that is a multiple of the number of nonzero entries of the matrix. A common format stores all nonzero entries in an array 'A' (row by row). Stores than a second array 'B' which gives the column-index of the corresponding nonzero entry from 'A' and a third array telling me where in array 'A' row x begins.
Assuming datatypes type_nnz, type_index a N*N sparse matrix with nnz nonzero elements the memory requirement is
sizeof(type_nnz)*nnz + sizeof(type_index)*(nnz+N)
Related
I am using the Eigen library in C++ for solving sparse linear equations: Ax=b where, A is a square sparse matrix and b is a dense vector with ILU-preconditioned BiCGSTAB. I am initializing the matrix A using the setFromTriplets function. The linear system is generated from discretization of partial differential equations in space and time.
My application changes the matrix slightly at every time-step. I want to modify a small number of rows (around 1% rows) in the matrix in the beginning of each time-step. I am storing the matrix in the row-major format so that I can access the row directly. I don't want to re-assemble the entire matrix from triplets since the number of rows to be modified are around 1%. Moreover, the modification is such that the number of non-zeros in the row are exactly identical. I just want to change the column indices and values. Hence, I do not need to allocate extra memory for the row. After going through the Eigen documentation as well as the forum, I found the functions coeffRef and insert. Both of them will allocate extra memory if the element does not exist. I would like to avoid this since the number of non-zeros are not changing.
Any help is appreciated.
I wrote a derived data type to store banded matrices in Compressed Diagonal Storage format; in particular I store each diagonal of the banded matrix in a column of the 2D array cds(1:N,-L:U), where N is the number of rows of the full matrix and L and U are the number of lower and upper diagonals (this question includes the definition of the type).
I also wrote a function to perform the product between a matrix in this CDS format and a full vector. To obtain each element of the product vector, the elements of the corresponding row of cds are used, which are not contiguous in memory, since the language is Fortran. Because of this I was wandering if a better solution would be to store the diagonals in the rows of a 2D array cds2(-L:U,1:N), which seems pretty reasonable to me.
On the contrary here I read
we can allocate for the matrix A an array val(1:n,-p:q). The declaration with reversed dimensions (-p:q,n) corresponds to the LINPACK band format [132], which, unlike compressed diagonal storage (CDS), does not allow for an efficiently vectorizable matrix-vector multiplication if p + q is small.
Which is just what seems appropriate to C in my opinion. What am I missing?
EDIT
The core of the routine performing matrix vector products is the following
DO i = A%lb(1), A%ub(1)
CDS_mat_x_full_vec(i) = DOT_PRODUCT(A%matrix(i,max(-lband,lv-i):min(uband,uv-i)), &
& v(max(i-lband,lv):min(i+uband,uv)))
END DO
(Where lv and uv are used to take into account the case of the vector indexed from an index other than 1.)
The matrix A is then accessed by rows.
I implemented the derived type which stores the diagonals in an array val(-p:q,1:n) and it is faster, as I supposed. So I think that the link I referenced refers to a row major storage language as C and not a column major one as Fortran. (Or it implements the matrix product in a way I can't even imagine.)
I'm working with large sparse matrices that are not exactly very sparse and I'm always wondering how much sparsity is required for storage of a matrix as sparse to be beneficial? We know that sparse representation of a reasonably dense matrix could have a larger size than the original one. So is there a threshold for the density of a matrix so that it would be better to store it as sparse? I know that the answer to this question usually depends on the structure of the sparsity, etc but I was wondering if there is just some guidelines? for example I have a very large matrix with density around 42%. should I store this matrix as dense or sparse?
scipy.coo_matrix format stores the matrix as 3 np.arrays. row and col are integer indices, data has the same data type as the equivalent dense matrix. So it should be straight forward to calculate the memory it will take as a function of overall shape and sparsity (as well as the data type).
csr_matrix may be more compact. data and indices are the same as with coo, but indptr has a value for each row plus 1. I was thinking that indptr would be shorter than the others, but I just constructed a small matrix where it was longer. An empty row, for example, requires a value in indptr, but none in data or indices. The emphasis with this format is computational efficiency.
csc is similar, but working with columns. Again you should be able to the math to calculate this size.
Brief discussion of memory advantages from MATLAB (using similar storage options)
http://www.mathworks.com/help/matlab/math/computational-advantages.html#brbrfxy
background paper from MATLAB designers
http://www.mathworks.com/help/pdf_doc/otherdocs/simax.pdf
SPARSE MATRICES IN MATLAB: DESIGN AND IMPLEMENTATION
I'm trying to infer a Markov chain of a process I can only simulate. The amount of states/vertices that the final graph will contain is very large, but I do not know the amount of vertices in advance.
Right now I have the following:
My simulation outputs a boost::dynamic_bitset containing 112 bits every timestep.
I use the bitset as a key in a Google Sparse Hash to map to an integer value that can be used as an index to the adjacency matrix I want to construct.
Now I need a good/fast matrix or two-dimensional array to store integers. It should:
Use the integer values I stored in the Google Sparse Hash as row/column numbers. (Eg. I want to access/change a stored integer by doing something like matrix(3,4) = 3.
I do not know the amount of rows or columns I will need in advance. So it should be able to just add rows and columns on the fly.
Most values will be 0, so it should probably be a sparse implementation of something.
The amount of rows and columns will be very large, so it should be very fast.
Simple to use. I don't need a lot of mathematical operations, it should just be a fast and simple way to store and access integers.
I hope I put my question clear enough.
I'd recommend http://www.boost.org/doc/libs/1_54_0/libs/numeric/ublas/doc/matrix_sparse.htm -- boost UBLAS sparse matrices. There are several different implementations of sparse matrix storages, so reading the documentation can help you choose a type that's right for your purpose. (TLDR: sparse matrices have either fast retrieval or fast insertion.)
I am implementing a sparse matrix class in compressed row format. This means i have a fixed number of rows and each row consists of a number of elements (this number can be different for different rows but will not change after the matrix has been initialised.
Is it suitable to implement this via vectors of vectors or will this somehow fragment the memory?
How whould i implement the allocation of this so i will have one big chunk of memory?
Thanks for sharing your wisdom!
Dirk
You can use existing libraries, like boost sparse matrix (http://www.boost.org/doc/libs/1_43_0/libs/numeric/ublas/doc/matrix_sparse.htm)
The general rule with sparse matrices is that you pick a structure that best fits the location of non-zeros in matrix; so maybe you could write a bit more about the matrix structure, and also what (sort of) algorithms will be invoked on it.
About the memory -- if this matrix is not too big, it is better to alloc it as a one big chunk and make some index magic; this may result in a better cache use.
If it is huge, then the chunked version should be used since it will better fit in memory.
You won't get 'one big chunk' with vector-of-vectors. Each row will be a contiguous chunk, but each row's data could be located at a different place on the heap.
Keep in mind that this might be a good thing if you're going to be dealing with huge matrices. The larger the matrix, the less likely you'll be able to acquire a contiguous chunk to fit the whole thing.
I've implemented the compressed column format with 3 std::vector<int> (2 for row & column indices and one for the value). Why do you need std::vector<std::vector<int>>?
Are you sure you're implementing the right format? A description of the compressed column format (and code for matrix-vector multiplication) can be found here.