I have 20 4D matrix and I want to perform PCA on them to get may be 2 or 3 4D matrix that explains most of the variance. I think this means I have 20 observations, but how do I organize my 20 observations into a matrix to perform PCA? Is it correct to just arrange each matrix into a column? I am unsure about it beacuse I feel though the 20 matrix are correlated, elements in the individual matrice are not.
Is there anyone here to help me understand PCA? Thanks a lot!
Maybe you can try Multilinear PCA (a generalization of PCA to higher order tensors). A Matlab implementation can be found here.
Related
I have vectors in 4096 VLAD codes, each one of them representing an image.
I have to run PCA on them without reducing their dimension on learning datasets with less than 4096 images (e.g., the Holiday dataset with <2k images.) to obtain the rotation matrix A.
In this paper (which also explains why I need to run this PCA without dimensionality reduction) they solved the problem with this approach:
For efficient computation of A, we compute at most the first 1,024 eigenvectors by eigendecomposition of the covariance matrix, and
the remaining orthogonal complements up to D-dimensional are filled using Gram-Schmidt orthogonalization.
Now, how do I implement this using C++ libraries? I'm using OpenCV, but cv::PCA seems not to offer such a strategy. Is there any way to do this?
I'm trying to design a line detector in opencv, and to do that, I need to get the Gaussian matrix with variance σs.
The final formula should be
H=Gσs∗(Gσd')T, and H is the detector that I'm going to create, but I have no idea how am I supposed to create the matrix with the variance and furthermore calculate H finally.
Update
This is the full formula.where “T” is the transpose operation.Gσd' is the first-order derivative of a 1-D Gaussian function Gσd with varianceσd in this direction
****Update****
These are the two formulas that I want, I need H for further use so please tell me how to generate the matrix. thx!
As a Gaussian filter is quite common, OpenCV has a built-in operation for it: GaussianBlur.
When you use that function you can set the ksize argument to 0/0 to automatically compute the pixel size of the kernel from the given sigmas.
A Gaussian 2D filter kernel is separable. That means you can first apply a 1D filter along the x axis and then a 1D filter along the y axis. That is the reason for having two 1D filters in the equation above. It is much faster to do two 1D filter operations instead of one 2D.
I have 2D data (I have a zero mean normalized data). I know the covariance matrix, eigenvalues and eigenvectors of it. I want to decide whether to reduce the dimension to 1 or not (I use principal component analysis, PCA). How can I decide? Is there any methodology for it?
I am looking sth. like if you look at this ratio and if this ratio is high than it is logical to go on with dimensionality reduction.
PS 1: Does PoV (Proportion of variation) stands for it?
PS 2: Here is an answer: https://stats.stackexchange.com/questions/22569/pca-and-proportion-of-variance-explained does it a criteria to test it?
PoV (Proportion of variation) represents how much information of data will remain relatively to using all of them. It may be used for that purpose. If POV is high than less information will be lose.
You want to sort your eigenvalues by magnitude then pick the highest 1 or 2 values. Eigenvalues with a very small relative value can be considered for exclusion. You can then translate data values and using only the top 1 or 2 eigenvectors you'll get dimensions for plotting results. This will give a visual representation of the PCA split. Also check out scikit-learn for more on PCA. Precisions, recalls, F1-scores will tell you how well it works
from http://sebastianraschka.com/Articles/2014_pca_step_by_step.html...
Step 1: 3D Example
"For our simple example, where we are reducing a 3-dimensional feature space to a 2-dimensional feature subspace, we are combining the two eigenvectors with the highest eigenvalues to construct our d×kd×k-dimensional eigenvector matrix WW.
matrix_w = np.hstack((eig_pairs[0][1].reshape(3,1),
eig_pairs[1][1].reshape(3,1)))
print('Matrix W:\n', matrix_w)
>>>Matrix W:
[[-0.49210223 -0.64670286]
[-0.47927902 -0.35756937]
[-0.72672348 0.67373552]]"
Step 2: 3D Example
"
In the last step, we use the 2×32×3-dimensional matrix WW that we just computed to transform our samples onto the new subspace via the equation
y=W^T×x
transformed = matrix_w.T.dot(all_samples)
assert transformed.shape == (2,40), "The matrix is not 2x40 dimensional."
I have two images( of same size ) and I want to check whether the second is the same as the first one but with some shift.So more formally I have two matrices A,B of same size and I want to check whether a submatrix of B occurs in A. But as this pictures are big(400x400) I need an efficient way to do so. Acceptable complexity would be O(n^3).Is there a way that can be done or should I make the images smaller ?:)
Thanks in advance.
You could simply use run-of-the-mill 2D cross correlation and detect where the maximum value lies to determine the (x,y) offset. Following the Cross-correlation theorem you can efficiently implement this in the Fourier domain.
See this simple example in Matlab on github, cross correlation and peak finding
EDIT
Here follows the short and mostly incomplete guide to the rigid registration of images. The gist of the cross correlation idea is as follows:
Say I have a 1D vector:
t = [1 2 3 1 2 3 4 ]
I shift this vector -4 places resulting in a new vector t2:
t2 = [2 3 4 1 2 3 1]
Now I have a look at the so called cross correlation c between t and t2:
c = [1 5 11 15 17 25 38 37 28 24 29 18 8]
Now, this cross correlation vector has a maximum of 38 located on position or index 7. This we can use to ascertain the shift as follows:
offset = round((7-(length(c)+1))/2)
offset = -4
where length() gives the dimensionality or number of elements in this dimension, of the cross correlation result.
Now, as should be evident, the cross correlation in the Spacial domain requires a lot of operations. This is where the above mentioned cross-correlation theorem comes in to play which links correlation in the Spacial domain to multiplication in the Fourier domain. The fourier transform is blessed with a number of very fast implementations (FFT) requiring vastly less operations, hence the reason they are used for determining the cross correlation.
There are many methods that deal with so-called rigid registration from stitching of satellite and holiday images a like to overlaying images from different sources as often found in medical imaging applications.
In your particular case, you might want to have a look at Phase correlation. Also the Numercial recipes in c book contains a chapter on fourier and correlation.
This problem is known in the literature as "Two dimentional pattern matching" (hint: google it).
Here is a paper that describes both optimal and naive algorithms:
Fast two dimensional pattern matching
Another popular term is "sub-matrix matching", but this is usually used when you want certain level of fuzziness instead of exact matches. Here is an example for such algorithm:
Partial shape recognition by sub-matrix matching for partial matching guided image labeling
I'm studying the code behind the convolutionFFT2D example of the Nvidia CUDA sdk, but I don't get the point of this line:
cufftPlan2d(&fftPlan, fftH, fftW/2, CUFFT_C2C);
Apparently this initializes a complex plane for the FFT to be running in, but I don't see the point of dividing the plan width by 2.
Just to be precise: the fftH and fftW are rounded values for imageX+kernelX+1 and imageY+kernelY+1 dimensions (just for speed reasons). I know that in the frequency domain you usually have a positive component and a symmetric negative component of the same frequency.. but this sounds like cutting half of my image data away..
Can someone explain this to me a little better? I've never used a FFT (I just know the theory behind a fourier transformation)
When you perform a real to complex FFT half the frequency domain data is redundant due to symmetry. This is only the case in one axis of a 2D FFT though. You can think of a 2D FFT as two 1D FFT operations, the first operates on all the rows, and for a real valued image this will give you complex row values. In the second stage you apply a 1D FFT to every column, but since the row values are now complex this will be a complex to complex FFT with no redundancy in the output. Hence you only need width / 2 points in the horizontal axis, but you still need height pointe in the vertical axis.