question
Suppose I have a sequence of float matrix, which has strong temporal correlation. How to compress such a float matrix sequence? I think, it could be nice if there is a method which considers both spatial and temporal correlation as in mp4.
actual problem
I have two sequences, uint8 matrix sequence (RGB Image) and float matrix sequence (Depth Image). The uint8 matrix sequence can be compressed very well by mp4. And, I want to do the similar thing for float sequence because it's size is huge.
For a sequence of floats that are slowly changing, as you imply, the easiest thing to do is subtract the previous value from all but the first value. Then take those differences and send only the significant portion. "Significant" is defined by what you know about your data and the accuracy of the samples. Then use a lossless compressor on that (e.g. gzip).
Related
I am in a discussion on whether you can save disk space by doing PCA on your data. Suppose you have a covariance matrix and your data vectors are of length 1000. The compression method to cut space by 50% would be:
derive a matrix that rotates the covariance matrix into diagonal form such that the eigenvalues are along the diagonal.
drop the smallest 500 diagonal elements - replace by zero
rotate the result using the transpose of the original rotation.
Me: This doesnt save any space for the vectors because there will still be nonzero elements in all the 1000 components after rotation. There is no compression. The data are probably simplified but that is a different thing.
Him: just take the first 500 elements in the result - that is your "compression".
I know I am right but plenty of people say in the literature that they are doing compression with PCA - here is an example:
http://ufldl.stanford.edu/tutorial/unsupervised/PCAWhitening/
I think that this tutorial is mostly right and is a nice description but the conclusion on compression is wrong. But how could something so obvious be overlooked by people who clearly work with data. Makes me things that I am wrong.
Can anyone help me understand their viewpoint?
In my opinion:
1- Yes, you can compress data by PCA because the dimension of the vectors (each one) you have to store is less than the original. Of course, you have to store the matrix to decompress the data too, but if your original dataset is enough large, this is insignificant to the data itself.
2- Of course there is a drawback. The compression is not lossless. You lose the original data forever, and your new version after decompression won't be exactly the same as the original. It will be an approximation.
At this point here's my advice:
If you have a lot of data with the same form (vectors of the same dimension...), your interest in this data is qualitative (you don't care the exact number itself, only the approximate number) and some of the data shows collinearity (dependency between vectors), PCA is a way to save storage space.
It is imperative to check if you lose the variance of the original data or not, because this is the signal you are choosing too much compression.
Anyway, the main purpose of PCA is not saving storage space... it is to do heavy operations with the data quicker to obtain a very similar result.
I hope this is helpful for you.
I want to implement a system where given an input image, it returns a reasonable similar one (approximation is acceptable) in a dataset of (about) 50K images. Time performances are crucial.
I'll use a parallel version of SIFT for obtaining a matrix of descriptors D. I've read about Fisher Vector (FV) (VLfeat and Yael implementations) as a learning and much more precise alternative to Bag of Features (BoF) for representing D as a single vector v.
My question are:
What distance is used for FVs? Is it the Euclidean one? In that case I would use LSH in eucledian distance for quickly find approximate near neighbor of FVs.
There is any other FV efficient (in terms of time) C++ implementation?
Another method you could take into consideration is VLAD encoding. (Basically a non-probabilistic version of FV, replacing GMMs by k-Means clustering)
Implementation differs only slightly from standard vector quantisation, but I my experiments it showed much better performance with significantly lower codebook size.
It uses euclidean distance to find the nearest codebook vector, but instead of just counting elements, it accumulates every elements residual.
An example for image search: Link
FV / VLAD paper: Paper
This is my first image processing application, so please be kind with this filthy peasant.
THE APPLICATION:
I want to implement a fast application (performance are crucial even over accuracy) where given a photo (taken by mobile phone) containing a movie poster finds the most similar photo in a given dataset and return a similarity score. The dataset is composed by similar pictures (taken by mobile phone, containing a movie poster). The images can be of different size, resolutions and can be taken from different viewpoints (but there is no rotation, since the posters are supposed to always be right-oriented).
Any suggestion on how to implement such an application is well accepted.
FEATURE DESCRIPTIONS IN OPENCV:
I've never used OpenCV and I've read this tutorial about Feature Detection and Description by OpenCV.
From what I've understood, these algorithms are supposed to find keypoints (usually corners) and eventually define descriptors (which describe each keypoint and are used for matching two different images). I used "eventually" since some of them (eg FAST) provides only keypoints.
MOST SIMILAR IMAGE PROBLEM AND LSH:
The problems above doesn't solve the problem "given an image, how to find the most similar one in a dataset in a fast way". In order to do that, we can both use the keypoints and descriptors obtained by any of the previous algorithms. The problem stated above seems like a nearest neighbor problem and Locality Sensitive Hashing is a fast and popular solution for find an approximate solution for this problem in high-dimensionality spaces.
THE QUESTION:
What I don't understand is how to use the result of any of the previous algorithms (i.e. keypoints and descriptors) in LSH.
Is there any implementation for this problem?
I will provide a general answer, going beyond the scope of OpenCV library.
Quoting this answer:
descriptors: they are the way to compare the keypoints. They
summarize, in vector format (of constant length) some characteristics
about the keypoints.
With that said, we can imagine/treat (geometrically) a descriptor as point in a D dimensional space. So in total, all the descriptors are points in a D dimensional space. For example, for GIST, D = 960.
So actually descriptors describe the image, using less information that the whole image (because when you have 1 billion images, the size matters). They serve as the image's representatives, so we are processing them on behalf of the image (since they are easier/smaller to treat).
The problem you are mentioning is the Nearest Neighbor problem. Notice that an approximate version of this problem can lead to significant speed ups, when D is big (since the curse of dimensionality will make the traditional approaches, such as a kd-tree very slow, almost linear in N (number of points)).
Algorithms that solve the NN problem, which is a problem of optimization, are usually generic. They may not care if the data are images, molecules, etc., I for example have used my kd-GeRaF for both. As a result, the algorithms expect N points in a D dimensional space, so N descriptors you might want to say.
Check my answer for LSH here (which points to a nice implementation).
Edit:
LSH expects as input N vectors of D dimension and given a query vector (in D) and a range R, will find the vectors that lie within this range from the query vector.
As a result, we can say that every image is represented by just one vector, in SIFT format for example.
You see, LSH doesn't actually solve the k-NN problem directly, but it searches within a range (and can give you the k-NNs, if they are withing the range). Read more about R, in the Experiments section, High-dimensional approximate nearest neighbo. kd-GeRaF and FLANN solve directly the k-NN problem.
I'm working with large sparse matrices that are not exactly very sparse and I'm always wondering how much sparsity is required for storage of a matrix as sparse to be beneficial? We know that sparse representation of a reasonably dense matrix could have a larger size than the original one. So is there a threshold for the density of a matrix so that it would be better to store it as sparse? I know that the answer to this question usually depends on the structure of the sparsity, etc but I was wondering if there is just some guidelines? for example I have a very large matrix with density around 42%. should I store this matrix as dense or sparse?
scipy.coo_matrix format stores the matrix as 3 np.arrays. row and col are integer indices, data has the same data type as the equivalent dense matrix. So it should be straight forward to calculate the memory it will take as a function of overall shape and sparsity (as well as the data type).
csr_matrix may be more compact. data and indices are the same as with coo, but indptr has a value for each row plus 1. I was thinking that indptr would be shorter than the others, but I just constructed a small matrix where it was longer. An empty row, for example, requires a value in indptr, but none in data or indices. The emphasis with this format is computational efficiency.
csc is similar, but working with columns. Again you should be able to the math to calculate this size.
Brief discussion of memory advantages from MATLAB (using similar storage options)
http://www.mathworks.com/help/matlab/math/computational-advantages.html#brbrfxy
background paper from MATLAB designers
http://www.mathworks.com/help/pdf_doc/otherdocs/simax.pdf
SPARSE MATRICES IN MATLAB: DESIGN AND IMPLEMENTATION
I'm currently studying lighting in OpenGL, which utilizes a function in GLSL called normalize. According to OpenGL docs, it says that it "calculates the normalized product of two vectors". However, it still doesn't explain what "normalized" mean. I have tried look for what a normalized product is on Google, however I can't seem to find anything about it. Can anyone explain what normalizing means and provide a few example of a normalized value?
I think the confusion comes from the idea of normalizing "a value" as opposed to "a vector"; if you just think of a single number as a value, normalization doesn't make any sense. Normalization is only useful when applied to a vector.
A vector is a sequence of numbers; in 3D graphics it is usually a coordinate expressed as v = <x,y,z>.
Every vector has a magnitude or length, which can be found using Pythagora's theorem: |v| = sqrt(x^2 + y^2 + z^2) This is basically the length of a line from the origin <0,0,0> to the point expressed by the vector.
A vector is normal if its length is 1. That's it!
To normalize a vector means to change it so that it points in the same direction (think of that line from the origin) but its length is one.
The main reason we use normal vectors is to represent a direction; for example, if you are modeling a light source that is an infinite distance away, you can't give precise coordinates for it. But you can indicate where to find it from a particular point by using a normal vector.
It's a mathematical term and this link explains its meaning in quite simple terms:
Operations in 2D and 3D computer graphics are often performed using copies of vectors that have been normalized ie. converted to unit vectors... Normalizing a vector involves two steps:
calculate its length, then,
divide each of its (xy or xyz) components by its length...
It's something complicated to explain if you don't know too much about vectors or even vectorial algebra. (You can check this article about general concepts as vector, normal vector or even normalization procedure ) Check it
But the procedure or concept of "normalize" refers to the process of making something standard or “normal.”
In the case of vectors, let’s assume for the moment that a standard vector has a length of 1. To normalize a vector, therefore, is to take a vector of any length and, keeping it pointing in the same direction, change its length to 1, turning it into what is called a unit vector.