I have some problems with opencv flann::Index -
I'm creating index
Mat samples = Mat::zeros(vfv_net_quie.size(),24,CV_32F);
for (int i =0; i < vfv_net_quie.size();i++)
{
for (int j = 0;j<24;j++)
{
samples.at<float>(i,j)=(float)vfv_net_quie[i].vfv[j];
}
}
cv::flann::Index flann_index(
samples,
cv::flann::KDTreeIndexParams(4),
cvflann::FLANN_DIST_EUCLIDEAN
);
flann_index.save("c:\\index.fln");
A fter that I'm tryin to load it and find nearest neiborhoods
cv::flann::Index flann_index(Mat(),
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
cv::Mat resps(vfv_reg_quie.size(), K, CV_32F);
cv::Mat nresps(vfv_reg_quie.size(), K, CV_32S);
cv::Mat dists(vfv_reg_quie.size(), K, CV_32F);
flann_index.knnSearch(sample,nresps,dists,K,cv::flann::SearchParams(64));
And have access violation in miniflann.cpp in line
((IndexType*)index)->knnSearch(_query, _indices, _dists, knn,
(const ::cvflann::SearchParams&)get_params(params));
Please help
You should not load the flann-file into a Mat(), as it is the place where the index is stored. It is a temporary object destroyed after the constructor was called. That's why the index isn't pointing anywhere useful when you call knnSearch().
I tried following:
cv::Mat indexMat;
cv::flann::Index flann_index(
indexMat,
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
resulting in:
Reading FLANN index error: the saved data size (100, 64) or type (5) is different from the passed one (0, 0), 0
which means, that the matrix has to be initialized with the correct dimensions (seems very stupid to me, as I don't necessarily know, how many elements are stored in my index).
cv::Mat indexMat(samples.size(), CV_32FC1);
cv::flann::Index flann_index(
indexMat,
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
does the trick.
In the accepted answer is somehow not clear and misleading why the input matrix in the cv::flann::Index constructor must have the same dimension as the matrix used for generating the saved Index. I'll elaborate on #Sau's comment with an example.
KDTreeIndex was generated using as input a cv::Mat sample, and then saved. When you load it, you must provide the same sample matrix to generate it, something like (using the templated GenericIndex interface):
cv::Mat sample(sample_num, sample_size, ... /* other params */);
cv::flann::SavedIndexParams index_params("c:\\index.fln");
cv::flann::GenericIndex<cvflann::L2<float>> flann_index(sample, index_params);
L2 is the usual Euclidean distance (other types can be found in opencv2/flann/dist.h).
Now the index can be used as shown the find the K nearest neighbours of a query point:
std::vector<float> query(sample_size);
std::vector<int> indices(K);
std::vector<float> distances(K);
flann_index.knnSearch(query, indices, distances, K, cv::flann::SearchParams(64));
The matrix indices will contain the locations of the nearest neighbours in the matrix sample, which was used at first to generate the index. That's why you need to load the saved index with the very matrix used to generate the index, otherwise the returned vector will contain indices pointing to meaningless "nearest neighbours".
In addition you get a distances matrix containing how far are the found neighbours from your query point, which you can later use to perform some inverse distance weighting, for example.
Please also note that sample_size has to match across sample matrix and query point.
Related
I'm working on code that computes dense SIFT features from a set of images, based on SIFT flow: http://people.csail.mit.edu/celiu/SIFTflow/
I'd like to try building a FLANN index on these images by comparing the "energy" between each image in SIFT flow representation.
I have the code to compute the energy from here: http://richardt.name/publications/video-deanaglyph/
Is there a way to create my own distance function for the indexing?
RELATED NOTE:
I was finally able to get an alternate (but not custom) distance function working with flann::Index. The trick is you need to use flann::GenericIndex like so:
flann::GenericIndex<cvflann::ChiSquareDistance<int>> flannIndex(descriptors, cvflann::KDTreeIndexParams());
But you need to give it CV_32S descriptors.
And if you use knnSearch with custom distance function, you have to provide a CV_32S results Mat and CV_32F distances Mat.
Here's my full code in case it's helpful (not a lot of documentation out there):
Mat samples;
loadDescriptors(samples); // loading descriptors from .yml file
samples *= 100000; // scaling up my descriptors to be int
samples.convertTo(samples, CV_32S); // convert float to int
// create flann index
flann::GenericIndex<cvflann::ChiSquareDistance<int>> flannIndex(samples, cvflann::KDTreeIndexParams());
// NOTE lack of distance type in constructor parameters
// (unlike flann::index)
// now try knnSearch
int k=10; // find 10 nearest neighbors
Mat results(1,10,CV_32S), dists(1,10,CV_32F);
// (1,10) Mats for the output, types CV_32S and CV_32F
Mat responseHistogram;
responseHistogram = samples.row(60);
// choose a random row from the descriptors Mat
// to find nearest neighbors
flannIndex.knnSearch(responseHist, results, dists, k, cvflann::SearchParams(200) );
cout << results << endl;
cout << dists << endl;
flannIndex.save(ofToDataPath("indexChi2.txt"));
Using Chi Squared actually seems to work better for me than L2 distance. My feature vectors are BoW histograms in this case.
I have created a PCA solution in Matlab, which works. I am now in the middle of converting it to c++, where I use OpenCV's function cv::PCA. Where I found in a link that you could extract the mean, eigenvalues and eigenvectors using:
// Perform a PCA:
cv::PCA pca(data, cv::Mat(), CV_PCA_DATA_AS_ROW, num_components);
// And copy the PCA results:
cv::Mat mean = pca.mean.clone();
cv::Mat eigenvalues = pca.eigenvalues.clone();
cv::Mat eigenvectors = pca.eigenvectors.clone();
Which compiles and runs. But when I want to use the values and look into the size and allocation, I get a size of the input data, which seems odd, i.e.
cv::Size temp= eigenvectors.size();
//temp has the same size as data.size();
In my mind the eigenvectors should be defined by the num_components and in my 3D space, should only be 3x3 size, i.e. 9 elements. Can anybody explain the reasoning behind the data sizes of pca.x.clone()?
Also what is the correct way of doing matrix operation in opencv and c++, based on the documentation it seems like you can use operators with cv::Mat. Using the above extraction of the pca information can you do:
cv::Mat Z = data - mean; //according to documentation http://stackoverflow.com/questions/10936099/matrix-multiplication-in-opencv
cv::Mat res = Z*eigenvectors;
It has crashed in my tests on runtime, probably because of the issue with "misinterpretation"/"interpretation" of the size.
Question
The basic question is how to use the pca opencv function correctly?
Edit
Another way that I have tested is to use the following code:
cv::PCA pca_analysis(mat, cv::Mat(), CV_PCA_DATA_AS_ROW, num_components);
cv::Point3d cntr = cv::Point3d(static_cast<double>(pca_analysis.mean.at<double>(0, 0)),
static_cast<int>(pca_analysis.mean.at<double>(0, 1)), static_cast<double>(pca_analysis.mean.at<double>(0, 2)));
//Store the eigenvalues and eigenvectors
vector<cv::Point3d> eigen_vecs(num_components);
vector<double> eigen_val(num_components);
for (int i = 0; i < num_components; ++i)
{
eigen_vecs[i] = cv::Point3d(pca_analysis.eigenvectors.at<double>(0, i*num_components),
pca_analysis.eigenvectors.at<double>(0, i * num_components + 1), pca_analysis.eigenvectors.at<double>(0, i*num_components + 2));
eigen_val[i] = pca_analysis.eigenvalues.at<double>(0,i);
}
When I compare the results from the above code with my Matlab implementation then the cntr seems to be plausible (under percentage difference). But the eigenvectors and eigenvalues are just zero, when I look at them in the debugger. This seems to go back to my original question of how to extract and understand pca's output.
Can anybody clarify what I am missing?
I came across your question when searching for how to access the eigenvalues and eigenvectors.
I personally needed to see the full information and would expect that myself. So, it's possible that the developers expected my use-case to more likely than yours.
In any case, as they are sorted you can take the first 'num_components' eigenvalues/eigenvectors as being the information you are after.
Also, from your perspective - i.e. if it worked as you expect it to - think about the converse: If you wanted to see the full info, how would you? The developers would have to implement a new parameter...
If one erodes an image by zeros(3,3) structuring element, it should be all 1s, but in case of OpenCV, it returns the image. Similarly, dilation of image by zeros(3,3) structuring element return image itself instead of all 0s.
The documentation says:
element – structuring element used for dilation; if element=Mat() , a 3x3 rectangular structuring element is used
If you take a look at the implementation, you will see that in case of empty kernel the ksize will be 3x3 inside of morphOp() #1683:
Size ksize = !kernel.empty() ? kernel.size() : Size(3,3);
This one also works for me:
cv::Mat input = cv::Mat::eye(10, 10, CV_8UC1);
cv::Mat output;
cv::erode(input, output, cv::Mat());
This answer states how to copy a row of a matrix to another row: How to copy a row of a Mat to another Mat's column in OpenCv?
However if I try to copy a row of a matrix to a column vector the program is ends abruptly.
Example:
Mat A(640,480,CV_64F);
Mat B(480,1,CV_64F);
A.row(0).copyTo(B.col(0));
A possible workaround is:
Mat A(640,480,CV_64F);
Mat B;
A.row(0).copyTo(B);
B = B.t();
But how can I get type CV_64F in B if A is a different type? Is the data type copied, too? And - most important – why do I need this workaround?
copyTo() function can only copy data between the matrices of same size (width and height) and same type. For this reason you can't use it directly to copy row of matrix to column of matrix. Normally OpenCV will reallocate target matrix so that its size and type will match original image, but call to col() function returns constant temporary object that can't be reallocated. As a result your program crushed.
Actually this problem can be solved. You can copy row to column (if they have same amount of elements) and you can copy data between matrices of different types. Mat provides an iterator to its data, and that means you can use all C++ functions that works with iterators. copy() function for example.
Mat A(640, 480, CV_64F);
Mat B(480, 1, CV_32S); // note that the type don't have to be the same
Mat tmp = A.row(0); // no data is copied here. it is needed only because taking iterator to temporary object is really bad idea
copy(tmp.begin<double>(), tmp.end<double>(), B.begin<int>());
Since code like the following (transposing the row only) is not supported by the API:
A.row(0).t().copyTo( B.col(0) );
The workaround is either create a temporary matrix from the row, transpose it, then copy to your target matrix
Mat temp = A.row(0);
temp = temp.t();
temp.copyTo( B.col(0) );
I would rather do something like this though:
Mat A( 640, 480,CV_64F );
Mat B( 1, 480, CV_64F ); /* rather than 480 x 1, make a 1 x 480 first */
A.row(0).copyTo( B.row(0) );
B = B.t(); /* transpose it afterwards */
I presume this is all just because the API is not supporting it yet.
I am trying to store a IPL_DEPTH_8U, 3 channel image into an array so that I can store 100 images in memory.
To initialise my 4D array I used the following code (rows,cols,channel,stored):
int size[] = { 324, 576, 3, 100 };
CvMatND* cvImageBucket; = cvCreateMatND(3, size, CV_8U);
I then created a matrix and converted the image into the matrix
CvMat *matImage = cvCreateMat(Image->height,Image->width,CV_8UC3 );
cvConvert(Image, matImage );
How would I / access the CvMatND to copy the CvMat into it at the position of stored?
e.g. cvImageBucket(:,:,:,0) = matImage; // copied first image into array
You've tagged this as both C and C++. If you want to work in C++, you could use the (in my opinion) simpler cv::Mat structure to store each of the images, and then use these to populate a vector with all the images.
For example:
std::vector<cv::Mat> imageVector;
cv::Mat newImage;
newImage = getImage(); // where getImage() returns the next image,
// or an empty cv::Mat() if there are no more images
while (!newImage.empty())
{
// Add image to vector
imageVector.push_back(image);
// get next image
newImage = getImage();
}
I'm guessing something similar to:
for ith matImage
memcpy((char*)cvImageBucket->data+i*size[0]*size[1]*size[2],(char*)matImage->data,size[0]*size[1]*size[2]);
Although I agree with #Chris that it is best to use vector<Mat> rather than a 4D matrix, this answer is just to be a reference for those who really need to use 4D matrices in OpenCV (even though it is a very unsupported, undocumented and unexplored thing with so little available online and claimed to be working just fine!).
So, suppose you filled a vector<Mat> vec with 2D or 3D data which can be CV_8U, CV_32F etc.
One way to create a 4D matrix is
vector<int> dims = {(int)vec.size(), vec[0].rows, vec[0].cols};
Mat m(dims, vec[0].type(), &vec[0]);
However, this method fails when the vector is not continuous which is typically the case for big matrices. If you do this for a discontinuous matrix, you will get a segmentation fault or bad access error when you would like to use the matrix (i.e. copying, cloning, etc). To overcome this issue, you can copy matrices of the vector one by one into the 4D matrix as follows:
Mat m2(dims, vec[0].type());
for (auto i = 0; i < vec.size(); i++){
vec[i].copyTo(temp.at<Mat>(i));
}
Notice that both methods require the matrices to be the same resolution. Otherwise, you may get undesired results or errors.
Also, notice that you can always use for loops but it is generally not a good idea to use them when you can vectorize.