How to read some elements of a matrix from disk C++ - c++

Question

I have a code where I'm reading 1024x1024 float matrix from disk then I'm getting some elements of it and doing some process on the new matrix as follows.
// mask is the 1Kx1K matrix that 1/64 element of it are 1 other elements are 0;
// it is a mask for **Mat data**
string filename = "filepath";
Mat data(1024,1024,CV_32F);
readMatrix(filename, data);
Mat smallMat(128,128,CV_32F);
getSmallerMat(data, mask, smallMat);
I read from float Mat from disk and fill smallMat using getSmallerMat(...) which is simply two for loops checking if mask(i,j) == 1, write to next position in smallMat
readMatrix(string fpath,Mat& data){
FILE* fp = fopen(fpath.c_str(),"rb");
if (!fp)perror("fopen");
int size = 1024;
data.create(size,size,CV_32F);
float* buffer= new float[size];
for(int i=0;i<size;++i) {
fread(buffer,sizeof(float),size,fp);
for(int j=0;j<size;++j){
data.at<float>(i,j)=buffer[j];
}
}
fclose(fp);
free(buffer);
}
What I want to do is just reading matrix elements whose corresponding value in mask is equal to 1. My problem is how will I pick (i,j)-th element from the disk.
Reading whole matrix and squeezing it takes 15 ms, I want to make it faster but I couldn't achieve to do it.
Consider this pic is my mask matrix. I want to read only white pixels only.
Thanks,

Answer 1

I am not sure that i understand the question correctly, but are you looking for a method to access data on the hard disk more quickly than via a stream? For finding some specific matrix element (i,j) in your stream you need to read the whole file (in the worst case), i.e. the complexity is linear, this can't be helped.
However, if you actually know the position in the fiel exactly (i.e. if you use a fixed length format for representing your doubles, etc.) seekg
http://www.cplusplus.com/reference/istream/istream/seekg/
should be faster than actually reading all characters until the desired position.
EDIT:
Given the discussion in comments to other answers I want to stress that using some seek in a file stream is O(N), hence multiple seeks for specific element will be way slower than just reading the whole file. I am not aware of a method to access data stored on hard disk in O(1). However, if all you ever need is matrices which are zero outside your mask, you should familiarize yourself with the concept of sparse matrices.
See e.g. https://en.wikipedia.org/wiki/Sparse_matrix and the documentation for your library, e.g. http://www.boost.org/doc/libs/1_39_0/libs/numeric/ublas/doc/matrix_sparse.htm

Answer 2

I am not sure if I have understood your problem or not; but if you want to read i,j th element from a file which contains the only float elements you should be able to get it like below -
float get(int i, int j, int rowsize, FILE * fp) {
float retVal = -1.0f; //-infinity may be?
// if you need restoring the stream pos
long lastPos = ftell(fp);
// ff to i*row + j
fseek(fp , ((i * rowsize) + j) * sizeof(float), SEEK_SET);
fread((unsigned char *)&retVal, sizeof(float), 1, fp);
// restore prevpos
// bla bla bla
return retVal;
}
You should be able to read any file which contains fixed size element very fast using the fseek and some arithmatic from start end or current file pointer. check the fseek documentation for more details.

Answer 3

From your code it appears your matrix is stored in binary as a memory image of the floats. What you want is go directly to the index on the disk where the (i,j) float is. You can compute this position using the following formula: index = i*colWidth+j where colWidth is 1024 in your case. You can use fseek and ftell to move your position and get your position in the file opened by fopen.