VLFeat kmeans C API explanation - c++

I'm trying to use VLFeat's kmeans implementation in C but I'm having a really hard time understanding how it works.
Note: I am using the C API in a C++ program, so any code posted by me here is C++. Additionally, I am using the Eigean header library, so that's where those Matrix data types are coming from.
Things unclear to from the example and API are:
What format does the data have to be in? The kmeans library functions appear to require a one-dimensional array, which could be taken from the backing of a matrix. However, does this matrix need to be column major or row major? That is, how does the function know to differentiate between dimensions of data and different data vectors?
How do I actually access the cluster center info? I ran a test where I declared I wanted 5 clusters, but using their example code from the link above, I only return 1.
Code:
int numData = 1000;
int dims = 10;
// Use float data and the L1 distance for clustering
VlKMeans * kmeans = vl_kmeans_new (VL_TYPE_FLOAT, VlDistanceL1) ;
// Use Lloyd algorithm
vl_kmeans_set_algorithm (kmeans, VlKMeansLloyd) ;
// Initialize the cluster centers by randomly sampling the data
Matrix<float, 1000,10, RowMajor> data = buildData(numData, dims);
vl_kmeans_init_centers_with_rand_data (kmeans, data.data(), dims, numData, 5);
// Run at most 100 iterations of cluster refinement using Lloyd algorithm
vl_kmeans_set_max_num_iterations (kmeans, 100) ;
vl_kmeans_refine_centers (kmeans, &data, numData) ;
// Obtain the energy of the solution
energy = vl_kmeans_get_energy(kmeans) ;
// Obtain the cluster centers
centers = (double*)vl_kmeans_get_centers(kmeans);
cout << *centers << endl;
Example Output: centers = 0.0376879 (a scalar)
How do I get all centers? I tried using an array to store centers, but it won't accept the type.
I also tried the following, assuming that perhaps I was just accessing the center info wrong:
cout << centers[0]<< endl;
cout << centers[1]<< endl;
cout << centers[2]<< endl;
cout << centers[3]<< endl;
cout << centers[4]<< endl;
cout << centers[5]<< endl;
cout << centers[6]<< endl;
cout << centers[7]<< endl;
cout << centers[8]<< endl;
But I should only have none-zero values for indices 0-4 (given 5 cluster centers). I actually expected exceptions to be thrown for higher indices. If this is the right approach, could someone please explain to me what these other values (indices 5-8) are from?
I'm sure there are other confusing pieces as well, but I haven't even addressed them yet as I've been stuck on these two pretty important pieces (I mean what is kmeans if you can't cluster properly to start).
Thank you in advance for your help!

What format does the data have to be in?
The manual says:
All algorithms support float or double data and can use the l1 or the l2 distance for clustering.
You specify that when you create your kmeans handle, e.g:
VlKMeans *kmeans = vl_kmeans_new(VL_TYPE_FLOAT, VlDistanceL2);
does this matrix need to be column major or row major?
It must be in row major, i.e: data + dimension * i is the i-th center.
How do I actually access the cluster center info?
With vl_kmeans_get_centers. For example if you work with float-s:
/* no need to cast here since get centers returns a `void *` */
const float *centers = vl_kmeans_get_centers(kmeans);
(see this answer regarding the cast)
The total size (in bytes) of this array is sizeof(float) * dimension * numCenters. If you want to print out the centers you can do:
int i, j;
for (i = 0; i < numCenters; i++) {
printf("center # %d:\n", i);
for (j = 0; j < dimension; j++) {
printf(" coord[%d] = %f\n", j, centers[dimension * i + j]);
}
}

Related

How can one write a multi-dimensional vector of image data to an output file?

Question:
Is there a good way to write a 3D float vector of size (9000,9000,4) to an output file in C++?
My C++ program generates a 9000x9000 image matrix with 4 color values (R, G, B, A) for each pixel. I need to save this data as an output file to be read into a numpy.array() (or similar) using python at a later time. Each color value is saved as a float (can be larger than 1.0) which will be normalized in the python portion of the code.
Currently, I am writing the (9000,9000,4) sized vector into a CSV file with 81 million lines and 4 columns. This is slow for reading and writing and it creates large files (~650MB).
NOTE: I run the program multiple times (up to 20) for each trial, so read/write times and file sizes add up.
Current C++ Code:
This is the snippet that initializes and writes the 3D vector.
// initializes the vector with data from 'makematrix' class instance
vector<vector<vector<float>>> colorMat = makematrix->getMatrix();
outfile.open("../output/11_14MidRed9k8.csv",std::ios::out);
if (outfile.is_open()) {
outfile << "r,g,b,a\n"; // writes column labels
for (unsigned int l=0; l<colorMat.size(); l++) { // 0 to 8999
for (unsigned int m=0; m<colorMat[0].size(); m++) { // 0 to 8999
outfile << colorMat[l][m][0] << ',' << colorMat[l][m][1] << ','
<< colorMat[l][m][2] << ',' << colorMat[l][m][3] << '\n';
}
}
}
outfile.close();
Summary:
I am willing to change the output file type, the data structures I used, or anything else that would make this more efficient. Any and all suggestions are welcome!
Use the old C file functions and binary format
auto startT = chrono::high_resolution_clock::now();
ofstream outfile;
FILE* f = fopen("example.bin", "wb");
if (f) {
const int imgWidth = 9000;
const int imgHeight = 9000;
fwrite(&imgWidth, sizeof(imgWidth), 1, f);
fwrite(&imgHeight, sizeof(imgHeight), 1, f);
for (unsigned int i=0; i<colorMat.size(); ++i)
{
fwrite(&colorMat[i], sizeof(struct Pixel), 1, f);
}
}
auto endT = chrono::high_resolution_clock::now();
cout << "Time taken : " << chrono::duration_cast<chrono::seconds>(endT-startT).count() << endl;
fclose(f);
The format is the following :
[ImageWidth][ImageHeight][RGBA][RGBA[RGBA]... for all ImageWidth * ImageHeight pixels.
Your sample ran in 119s in my machine. This code ran in 2s.
But please note that the file will be huge anyway : you are writing the equivalent of two 8K files without any kind of compression.
Besides that, some tips on your code :
Don't use a vector of floats to represent your pixels. They won't have more components than RGBA. Instead create a simple struct with four floats.
You don't need to look through width and height separately. Internally all lines are put sequentially one after the other. It is easier to create a one dimension array of width * height size.

Directly accessing entries of armadillo-matrix by memory-pointer (memptr) does not work after matrix is modified

I have a problem where I want to access certain entries of an armadillo-matrix "M" by a pointer in a struct (or class). After initializing M I set the pointer in the struct. By dereferencing the pointer I can see it has the right value (the first entry of M - or M(0,0)).
Then I change M to M * M. But now dereferencing the pointer does not give me the right value anymore. What's weird: If I have a small matrix i.e. 3x3 or 4x4 (see "matrixSize" in my code) the error does not happen. With small matrices dereferencing the pointer gives the RIGHT value.
Bigger matrices though result in the wrong values (with 5x5 something like "2.76282e-320", which is probably some random place in memory).
What am I doing wrong here? How can I solve this problem?
If it helps, I'd like to explain what I want to achieve:
I have a network of delay-coupled nodes that each have some sort of dynamic behaviour. (think delay-coupled differential equations DDEs - delay-coupled Oscillators). As they are delay-coupled I need to store their past states (an array of their histories). Each oscillator also has some LOCAL dynamic with dynamical variables that are not influencing other nodes which means that I don't have to store their histories.
The matrix shall be used to keep the past states of some variable of the nodes. I want to have them in a matrix, because I want to use vector-operations on them (one index of the matrix represents time, while the other is the node-index). But I also want to access them individually to calculate some local dynamic at each of the nodes (oscillators). So I want to update the individual nodes, but also the global state.
That's why having both representations helps: For the local dynamics I access the delayed states through a pointer into the matrix. For the global dynamics I access the coupled variables through a row in the matrix that functions as a history-array.
#include <iostream>
#include <armadillo>
struct someStruct {
const double *currentStateInMatrix;
void printvalue() {std::cout << "*currentState (pointer to first enty of matrix) = " << *currentStateInMatrix << std::endl;}
};
int main(int argc, char* argv[]){
uint M_size = 4;
arma::Mat<double> M(M_size, M_size, arma::fill::randu);
double* pointerIntoMatrix = M.memptr();
std::cout << "pointer [ " << pointerIntoMatrix << " ] has value: " << *pointerIntoMatrix << std::endl;
someStruct myStruct;
myStruct.currentStateInMatrix = pointerIntoMatrix;
std::cout << "original matrix M: \n" << M;
myStruct.printvalue();
std::cout << "\n+++changing contents of matrix M+++\n\n";
M = M * M;
std::cout << "M * M: \n" << M;
myStruct.printvalue();
}
I compiled it using:
g++ file.cpp -std=c++17 -larmadillo -o main
The operation M = M * M in Armadillo is a matrix multiply (not an element by element multiply). So storing the intermediate calculations of M * M directly into M would be problematic. It would overwrite existing data in M that is still needed to complete the M * M operation.
It's probably safe to assume that Armadillo detects this problem and stores the result of M * M into a separate chunk of memory and then assigns that chunk to M.
There are ways around that. Use fixed size matrices like darcamo mentioned in the comments. Use Mat<double>::fixed<4, 4> M; to declare the matrix.
Another way is to manually manage the memory for the matrix elements and tell the M matrix to always use that memory. There are the advanced constructors to do that:
mat(ptr_aux_mem, n_rows, n_cols, copy_aux_mem = true, strict = false)
So we can do:
double* my_mem = (double*)std::malloc(M_size * M_size * sizeof(double));
// or use new[]
Mat<double> M(my_mem, M_size, M_size, false, true);
// don't forget to call std::free(my_mem) or delete[] when done!
A word of caution. The above two workarounds (fixed size matrices and manual memory management) probably have minor performance penalties. There is no way to avoid using a separate chunk of memory for the M = M * M operation. When using these workarounds, Armadillo will store the result of M * M into a temporary buffer and then copy the contents of the buffer into the memory used by M.

Why do I get different values when using different datatypes when accessing pixels in a matrix?

I have a single channel grayscale image (slice).
cout << "num" << slice.channels() << ends; //outputs 1
for(int x = 0;x<=slice.cols;x++){
for(int y = 0;y<=slice.rows;y++){
Vec3b currentPoint = slice.at<Vec3b>(x,y);
cout << currentPoint;
}
}
however, when I try to access a pixel and expect currentPoint to be a single int as it is a single channel image. However, i get [32, 36, 255] which is odd, as it implies three channels. I appreciate I am using a type that says vec3b, but even so, where is it getting the other two elements from?
So I replace Vec3b with uchar, then i get lots of \377. That is even more confusing.
Even when I do have a 3 channel image, I get odd outputs when trying to access a single element of Vec3b (i get more \377).
How can this make sense? I must be mis understanding how the at() method is used.
Firstly, how do I get a single output for each pixel (0-255)?
Also, where am I going wrong when i see \377?
A lot of stuff for a few lines of code...
Since your image is a grayscale image, you should access it with at<uchar>.
Pay attention that the at<> function accepts (rows, cols), which is the opposite of (x,y).
It's faster to scan by line, since the matrix is stored row-wise in memory.
To print out the value of a uchar, you need to cast to int, or you get the ASCII coded character.
The loops should not be <=, but instead <, or you go out of bounds.
So:
for(int y = 0; y < slice.rows; y++) {
for(int x = 0; x < slice.cols; x++) {
uchar currentPoint = slice.at<uchar>(y,x);
cout << int(currentPoint) << " ";
}
cout << "\n";
}

opencv C++ neural network predict() function throws "Bad argument" error

I have managed to train a neural network to recognize numbers in an image and have saved the network parameters to an .xml file.
However, when testing the network against a new image the code fails at the predict() stage with the error:
OpenCV Error: Bad argument (Both input and output must be floating-point matrices of the same type and have the same number of rows) in CvANN_MLP::predict, file ........\opencv\modules\ml\src\ann_mlp.cpp, line 279.
ann_mlp.cpp line 279 is:
if( !CV_IS_MAT(_inputs) || !CV_IS_MAT(_outputs) ||
!CV_ARE_TYPES_EQ(_inputs,_outputs) ||
(CV_MAT_TYPE(_inputs->type) != CV_32FC1 &&
CV_MAT_TYPE(_inputs->type) != CV_64FC1) ||
_inputs->rows != _outputs->rows )
CV_Error( CV_StsBadArg, "Both input and output must be floating-point matrices "
"of the same type and have the same number of rows" );
I have checked input rows by running this code:
cv::Size s = newVec.size();
int rows = s.height;
int cols = s.width;
cout << "newVec dimensions: " << rows << " x " << cols << endl;
...and it comes out with the expected 1 x 900 vector / matrix.
I have set the input and output matrices to be CV_32FC1 as per the error dialog like this:
Input matrix
cv::Mat newVec(1, 900, CV_32FC1);
newVec = crop_img.reshape(0, 1); //reshape / unroll image to vector
CvMat n = newVec;
newVec = cv::Mat(&n);
Output matrix
cv::Mat classOut = cvCreateMatHeader(1, CLASSES, CV_32FC1);
And I try to run the prediction function like this:
CvANN_MLP* nnetwork = new CvANN_MLP;
nnetwork->load("nnetwork.xml", "nnetwork");
int maxIndex = 0;
cv::Mat classOut = cvCreateMatHeader(1, CLASSES, CV_32FC1);
//prediction
nnetwork->predict(newVec, classOut);
float value;
float maxValue = classOut.at<float>(0, 0);
for (int index = 1; index<CLASSES; index++)
{
value = classOut.at<float>(0, index);
if (value>maxValue)
{
maxValue = value;
maxIndex = index;
}
}
Any ideas? Much appreciated...
I suspect the problem is your input, not your output.
First it's important to understand that OpenCV deserves a lot of the blame for this, not you. Their C++ API is quite mediocre, and it caused major confusion to you.
See, normally in C++ when you define a 1x900 matrix of floats, it stays a matrix of floats. C++ has strong type safety.
OpenCV does not. If you assign a matrix of bytes to a matrix of floats, the latter will change its type (!).
Your code initializes newVec to such a matrix of floats, then assigns a second matrix, and then yet another matrix. I suspect that crop_img is still an image, i.e. 8 bits. Reshaping it will make it 1x900, but not floating point. That's the job of .convertTo.

C++ eigenvalue/vector decomposition, only need first n vectors fast

I have a ~3000x3000 covariance-alike matrix on which I compute the eigenvalue-eigenvector decomposition (it's a OpenCV matrix, and I use cv::eigen() to get the job done).
However, I actually only need the, say, first 30 eigenvalues/vectors, I don't care about the rest. Theoretically, this should allow to speed up the computation significantly, right? I mean, that means it has 2970 eigenvectors less that need to be computed.
Which C++ library will allow me to do that? Please note that OpenCV's eigen() method does have the parameters for that, but the documentation says they are ignored, and I tested it myself, they are indeed ignored :D
UPDATE:
I managed to do it with ARPACK. I managed to compile it for windows, and even to use it. The results look promising, an illustration can be seen in this toy example:
#include "ardsmat.h"
#include "ardssym.h"
int n = 3; // Dimension of the problem.
double* EigVal = NULL; // Eigenvalues.
double* EigVec = NULL; // Eigenvectors stored sequentially.
int lowerHalfElementCount = (n*n+n) / 2;
//whole matrix:
/*
2 3 8
3 9 -7
8 -7 19
*/
double* lower = new double[lowerHalfElementCount]; //lower half of the matrix
//to be filled with COLUMN major (i.e. one column after the other, always starting from the diagonal element)
lower[0] = 2; lower[1] = 3; lower[2] = 8; lower[3] = 9; lower[4] = -7; lower[5] = 19;
//params: dimensions (i.e. width/height), array with values of the lower or upper half (sequentially, row major), 'L' or 'U' for upper or lower
ARdsSymMatrix<double> mat(n, lower, 'L');
// Defining the eigenvalue problem.
int noOfEigVecValues = 2;
//int maxIterations = 50000000;
//ARluSymStdEig<double> dprob(noOfEigVecValues, mat, "LM", 0, 0.5, maxIterations);
ARluSymStdEig<double> dprob(noOfEigVecValues, mat);
// Finding eigenvalues and eigenvectors.
int converged = dprob.EigenValVectors(EigVec, EigVal);
for (int eigValIdx = 0; eigValIdx < noOfEigVecValues; eigValIdx++) {
std::cout << "Eigenvalue: " << EigVal[eigValIdx] << "\nEigenvector: ";
for (int i = 0; i < n; i++) {
int idx = n*eigValIdx+i;
std::cout << EigVec[idx] << " ";
}
std::cout << std::endl;
}
The results are:
9.4298, 24.24059
for the eigenvalues, and
-0.523207, -0.83446237, -0.17299346
0.273269, -0.356554, 0.893416
for the 2 eigenvectors respectively (one eigenvector per row)
The code fails to find 3 eigenvectors (it can only find 1-2 in this case, an assert() makes sure of that, but well, that's not a problem).
In this article, Simon Funk shows a simple, effective way to estimate a singular value decomposition (SVD) of a very large matrix. In his case, the matrix is sparse, with dimensions: 17,000 x 500,000.
Now, looking here, describes how eigenvalue decomposition closely related to SVD. Thus, you might benefit from considering a modified version of Simon Funk's approach, especially if your matrix is sparse. Furthermore, your matrix is not only square but also symmetric (if that is what you mean by covariance-like), which likely leads to additional simplification.
... Just an idea :)
It seems that Spectra will do the job with good performances.
Here is an example from their documentation to compute the 3 first eigen values of a dense symmetric matrix M (likewise your covariance matrix):
#include <Eigen/Core>
#include <Spectra/SymEigsSolver.h>
// <Spectra/MatOp/DenseSymMatProd.h> is implicitly included
#include <iostream>
using namespace Spectra;
int main()
{
// We are going to calculate the eigenvalues of M
Eigen::MatrixXd A = Eigen::MatrixXd::Random(10, 10);
Eigen::MatrixXd M = A + A.transpose();
// Construct matrix operation object using the wrapper class DenseSymMatProd
DenseSymMatProd<double> op(M);
// Construct eigen solver object, requesting the largest three eigenvalues
SymEigsSolver< double, LARGEST_ALGE, DenseSymMatProd<double> > eigs(&op, 3, 6);
// Initialize and compute
eigs.init();
int nconv = eigs.compute();
// Retrieve results
Eigen::VectorXd evalues;
if(eigs.info() == SUCCESSFUL)
evalues = eigs.eigenvalues();
std::cout << "Eigenvalues found:\n" << evalues << std::endl;
return 0;
}