I have some training data that consists of a number of features extracted from images and different class labels. I have managed to train Normal Bayes classifiers using OpenCV3 in C++. And I am able to pass new test data into the classifiers to obtain the predicted class labels using predict() function.
However, I don't want to simply get the predicted class label, I also wish to know the probabilities for each class label per test data using predictProb() function of class NormalBayesClassifier.
There is a predictProb() function that seems to be able to return the probabilities per class label:
virtual float cv::ml::NormalBayesClassifier::predictProb
( InputArray inputs,
OutputArray outputs,
OutputArray outputProbs,
int flags = 0
) const
However, when i tested the code, I always get a vector of 0s or a mixture of 0s and Inf for different test images as the predict probabilities even though I do get a correct prediction. I tried adding RAW_OUTPUT to the flags and the result is the same.
int N=4;
vector<string> loc;
loc.push_back("1.jpg");
loc.push_back("2.jpg");
loc.push_back("3.jpg");
loc.push_back("4.jpg");
loc.push_back("5.jpg");
loc.push_back("6.jpg");
Ptr<ml::NormalBayesClassifier> rt = cv::ml::NormalBayesClassifier::create();
Mat img,features,dictionary;
vector<cv::KeyPoint> keyPoints;
Mat X;
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");
Ptr<SURF> detector = SURF::create(400,4,2,1,1);
Ptr<DescriptorExtractor> extractor = detector;
FileStorage fs("Bag-Of-Features.yml", FileStorage::READ);
fs["dictionary"] >> dictionary;
fs.release();
Ptr<BOWImgDescriptorExtractor> bowDE=makePtr<BOWImgDescriptorExtractor>(extractor, matcher);
bowDE->setVocabulary(dictionary);
for(int i=0;i<4;i++)
{
img=imread(loc[i]);
detector->detect(img,keyPoints);
bowDE->compute(img,keyPoints,features);
int rows = features.rows;
int cols = features.cols;
//cout << "r"<< rows << "c "<< cols ;
X.push_back(features);
}
Mat_<int> Y(N,1);
Y << 0,0, 1,1 ;
rt->train(X, ml::ROW_SAMPLE, Y);
rt->save("classifier.yml");
/////////prediction/////////////
Mat features1;
vector<cv::KeyPoint> keyPoints1;
Mat r,p;
Mat inp;
Mat R1,P1;
for (int i=0;i<2;i++)
{
inp=imread(loc[4+i]);
//inp.convertTo(inp,CV_8U);
detector->detect(inp, keyPoints1);
bowDE->compute(inp, keyPoints1, features1);
//features1.convertTo(features1,CV_32F);
rt->predictProb(features1,r,p);
R1.push_back(r);
P1.push_back(p);
}
cout << "Probability"<<P1 <<endl ;
return 0;
}
Ouput:
Probability[0, 0;inf, 0]
There are two conditions that need to be met for the Bayes Classifier to be able to output probabilities:
1) You must train it with multiple samples that pair to the same response value.
2) The training samples should be normalized.
I had this same question, and after scouring the internet for answers finally this link helped me conclude the above statement:
Opencv3 Bayes Classifier predictProb giving strange results
Related
I'm new to SVM. I want to use Multiclass svm for classification in my action recognition project. My data set have 15 class like running jogging walking biking etc. I understand binary SVM and seen lot of examples. I'm getting confusion in one vs one multiclass svm and one vs rest multiclass svm.
My question is: Should I have to create and train a svm for each class like svm1 for running, svm2 for jogging.....etc what should be class lebels for each created svm either (0,1,2,.......14) or (0,1). how to determine the model for multiclass classification.how the result will be predicted ?
I was trying some thing like this:
Mat trainData, trainLabels;
for (int i = 0; i < 15; i++ )
{
sprintf_s(filename, "VocabularyHOG/Dictionary%d.yml", i);
Mat feature;
FileStorage fs(filename, FileStorage::READ);
fs["vocabulary"] >> feature;
feature.convertTo(feature, CV_32F); // make sure we got float data
trainData.push_back(feature.reshape(1, 1)); // as a flat column
trainLabels.push_back(i); // the classlabel
}
// Train the SVM
Ptr<SVM> svm = SVM::create();
svm->setType(SVM::C_SVC);
svm->setKernel(SVM::LINEAR);
svm->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, 100, 1e-6));
svm->train(trainData, ROW_SAMPLE, trainLabels);
Mat testData;
FileStorage fs("DictionaryrunningHOG.yml", FileStorage::READ);
fs["vocabulary"] >> testData;
int response = svm->predict(testData.reshape(1, 1));
if (response == 1)
cout << "boxing";
else
cout << "negative result";
waitKey(27);
return 0;
but getting error. what is wrong here?? how to code one vs rest multiclass svm properly.What is the label for each svm? Thanks
I am trying to use SVM to match a query image with its appropriate class. Right now the classes are just 1 or 0. I extract the class from a .txt file and store it into a Mat. I use BoW to compute a histogram for each image in the training set, and also store it into a Mat.
Mat response_hist;
Mat histograms;
Mat classes;
ifstream ifs("train.txt");
int total_samples_in_file = 0;
vector<string> classes_names;
vector<string> lines;
for (int i = 1; i <= trainingSetSize; i++){
cout << "in for loop iteration"<< i << endl;
_snprintf_s(filepath, 100, "C:/Users/Randal/Desktop/TestCase1Training/train/%d.bmp", i);
Mat temp = imread(filepath, CV_LOAD_IMAGE_GRAYSCALE);
Mat tempBW;
adaptiveThreshold(temp, tempBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
detector->detect(tempBW, keypoints1);
BOW.compute(tempBW, keypoints1, response_hist);
response_hist.convertTo(response_hist, CV_32F);
histograms.push_back(response_hist);
}
//read from the file - ifs and put into a vector
std::string line;
float class_num;
string imgfilepath;
for (int j = 1; getline(ifs, line); j++)
{
istringstream ss(line);
ss >> imgfilepath >> class_num;
classes.push_back(class_num);
}
The Mats class_num and histograms are used in training the SVM. Each row in "histograms" represents a sample (a histogram of an image in the training set). "class_num" is one row with each column being the class (1 or 0) of a corresponding image in the training set.
Ptr<ml::SVM> svm = ml::SVM::create();
svm->setType(ml::SVM::C_SVC);
svm->setKernel(ml::SVM::POLY);
svm->setGamma(3);
Mat trainingDataMat(histograms);
Mat trainingDataClass(classes);
trainingDataMat.convertTo(trainingDataMat, CV_32F);
trainingDataMat = trainingDataMat.reshape(trainingDataMat.cols, 1);
trainingDataClass.convertTo(classes, CV_32F);
svm->train(trainingDataMat, ml::ROW_SAMPLE, trainingDataClass); //incorrect types? I think it is a problem with ROW_SAMPLE
Mat res; // output
svm->predict(output, res);
When I run this I get the error "Assertion failed (samples.type() == CV_32F || samples.type() == CV_32S) in cv::ml::TrainDataImpl::setData". However, I have placed lines of code in to convert both my class Mat and my histogram Mat to type CV_32F. Is the issue with my inputs or does it have something to do with ROW_SAMPLE in svm->train? Any help is greatly appreciated.
Thanks
The error was caused by my input being incorrect. I changed the type of Mat classesMat to CV_32S in its declaration. I also changed
trainingDataMat_32.reshape(trainingDataMat_32.cols, 1);
to have the correct number of channels and rows.
trainingDataMat_32.reshape(1, trainingDataMat_32.rows);
TrainingDataMat.cols was not the correct value at all. It needed 1 channel (first parameter) and the same number of rows as my histogram input.
This caused a new error regarding the kernel I am using (SVM parameter "POLY"). I had to add another line right below the kernel parameter:
svm->setDegree(3);
This fixed the error. My output is not correct, but this solved the assertion failure.
My goal is to take an image as a query and find its best match in a library of images. I am using SURF features, in openCV 3.0.0, and the Bag of Words approach to find a match. I need a way to find out if the query image has a match in the library. If it does, I want to know the index of the image that is the nearest match.
Here is my code for reading in all the images (300 total in the library of images) and extracting and clustering the features:
Mat training_descriptors(1, extractor->descriptorSize(), extractor->descriptorType());
//read in all images and set to binary
char filepath[1000];
for (int i = 1; i < trainingSetSize; i++){
cout << "in for loop, iteration: " << i << endl;
_snprintf_s(filepath, 100, "C:/Users/Randal/Desktop/TestCase1Training/%d.bmp", i);
Mat temp = imread(filepath, CV_LOAD_IMAGE_GRAYSCALE);
Mat tempBW;
adaptiveThreshold(temp, tempBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
detector->detect(tempBW, keypoints1);
extractor->compute(tempBW, keypoints1, descriptors1);
training_descriptors.push_back(descriptors1);
cout << "descriptors added" << endl;
}
cout << "Total descriptors: " << training_descriptors.rows << endl;
trainer.add(training_descriptors);
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");
BOWImgDescriptorExtractor BOW(extractor, matcher);
Mat library = trainer.cluster();
BOW.setVocabulary(library);
I wrote the following code in an attempt to find a match. The problem is that BOW.compute only returns the indices of clusters (words) that exist in both the image and the library of images. imgQ is the query image.
Mat output;
Mat imgQBW;
adaptiveThreshold(imgQ, imgQBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
imshow("query image", imgQBW);
detector->detect(imgQBW, keypoints2);
extractor->compute(imgQBW, keypoints2, descriptors2);
BOW.compute(imgQBW, keypoints1, output);
cout << output.row(0) << endl;
I need to know which clusters in the BoW correspond to which images. My output right now--output.row(0)--is just an array with all the the indices of clusters found in the library. Am I misunderstanding this output? Is there a way to determine which image has the most matching clusters?
I also did something similar based on this code:
https://github.com/royshil/FoodcamClassifier/blob/master/training_common.cpp
But the above part is after the clustering has finished.
What you have to do is to train using your ML(I used SVM) and your cluster centers, the visual bag of words that you have.
More, you need to find all the "closest" points to your clustered points and train them using histograms. Next, you will have a histogram of frequencies(bag of key points) that you need to train.
Ptr<ifstream> ifs(new ifstream("training.txt"));
int total_samples_in_file = 0;
vector<string> classes_names;
vector<string> lines;
//read from the file - ifs and put into a vector
for(int i=0;i<lines.size();i++) {
vector<KeyPoint> keypoints;
Mat response_hist;
Mat img;
string filepath;
string line(lines[i]);
istringstream iss(line);
iss >> filepath;
string class_to_train;
iss >> class_to_train;
class_ml = "class_" + class_to_train;
if(class_ml.size() == 0) continue;
img = imread(filepath);
detector->detect(img,keypoints);
bowide.compute(img, keypoints, response_hist);
cout << "."; cout.flush();
//here create the logic for the class to train(class_0, e.g) and the data you need to train.
}
More you can find at this git project:
https://github.com/royshil/FoodcamClassifier
Documentation here:
http://www.morethantechnical.com/2011/08/25/a-simple-object-classifier-with-bag-of-words-using-opencv-2-3-w-code/
i'm trying to do a training using OpenCV and SVM.
But i have a problem, precisely this error:
OpenCV Error: Bad argument (train data must be floating-point matrix) in cvCheckTrainData
I have to do a train in a dataSet of images where every images has 68 point (X,Y) that i use to do SVM.
In the beginning this was my code :
//for each image
fin_land.open(str_app); assert(fin_land);
for (int i=(0+(68*count_FOR)); i<(num_landCKplus+(68*count_FOR)); i++) {
fin_land >> f1;
fin_land >> f2;
int data[2] = {(int)f1, (int)f2};
Mat actual(1, 2, CV_32FC1, &data);
trainData.push_back(actual);
}
// SVM
CvSVMParams params;
params.svm_type = CvSVM::NU_SVC;
params.kernel_type = CvSVM::POLY;
trainData = trainData.reshape(1, #numImage);
SVM.train(trainData, trainLabels, Mat(), Mat(), params);
The problem with this code was that i thought to use a Mat for the test with 68 row and 2 columns, because every training-class in my SVM has 2 columns, but i received this error :
OpenCV Error: Incorrect size of input array (Input sample must be 1-dimensional vector) in cvPreparePredictData
If I have correctly understood, the problem was that the dimension of the test Mat need to be only one dimensional. So, i thought to modify my code like this:
//for each image
fin_land.open(str_app); assert(fin_land);
for (int i=(0+(68*count_FOR)); i<(num_landCKplus+(68*count_FOR)); i++) {
fin_land >> f1;
fin_land >> f1;
int data = (int)f1;
trainData.push_back(&data);
data = (int)f2;
trainData.push_back(&data);
}
Now every training-class has only one column, so even the Mat of test, but I have a new error and it says:
OpenCV Error: Bad argument (train data must be floating-point matrix) in cvCheckTrainData
The problem is that the type of the new mat of trainingSet is wrong?
I don't know how to fix it...
you need float data (and integer labels)
1 row per feature, 1 label per row.
float f1,f2;
for (int i=(0+(68*count_FOR)); i<(num_landCKplus+(68*count_FOR)); i++) {
fin_land >> f1;
fin_land >> f1;
trainData.push_back(f1); // pushing the 1st thing will determine the type of trainData
trainData.push_back(f2);
}
trainData = trainData.reshape(1, numItems);
SVM.train(trainData, trainLabels, Mat(), Mat(), params);
I'm having problems getting PCA and Eigenfaces working using the latest C++ syntax with the Mat and PCA classes. The older C syntax took an array of IplImage* as a parameter to perform its processing and the current API only takes a Mat that is formatted by Column or Row. I took the Row approach using the reshape function to fit my image's matrix to fit in a single row. I eventually want to take this data and then use the SVM algorithm to perform detection, but when I do that all my data is just a stream of 0s. Can someone please help me out? What am I doing wrong? Thanks!
I saw this question and it's somewhat related, but I'm not sure what the solution is.
This is basically what I have:
vector<Mat> images; //This variable will be loaded with a set of images to perform PCA on.
Mat values(images.size(), 1, CV_32SC1); //Values are the corresponding values to each of my images.
int nEigens = images.size() - 1; //Number of Eigen Vectors.
//Load the images into a Matrix
Mat desc_mat(images.size(), images[0].rows * images[0].cols, CV_32FC1);
for (int i=0; i<images.size(); i++) {
desc_mat.row(i) = images[i].reshape(1, 1);
}
Mat average;
PCA pca(desc_mat, average, CV_PCA_DATA_AS_ROW, nEigens);
Mat data(desc_mat.rows, nEigens, CV_32FC1); //This Mat will contain all the Eigenfaces that will be used later with SVM for detection
//Project the images onto the PCA subspace
for(int i=0; i<images.size(); i++) {
Mat projectedMat(1, nEigens, CV_32FC1);
pca.project(desc_mat.row(i), projectedMat);
data.row(i) = projectedMat.row(0);
}
CvMat d1 = (CvMat)data;
CvMat d2 = (CvMat)values;
CvSVM svm;
svm.train(&d1, &d2);
svm.save("svmdata.xml");
What etarion said is correct.
To copy a column or row you always have to write:
Mat B = mat.col(i);
A.copyTo(B);
The following program shows how to perform a PCA in OpenCV. It'll show the mean image and the first three Eigenfaces. The images I used in there are available from http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html:
#include "cv.h"
#include "highgui.h"
using namespace std;
using namespace cv;
Mat normalize(const Mat& src) {
Mat srcnorm;
normalize(src, srcnorm, 0, 255, NORM_MINMAX, CV_8UC1);
return srcnorm;
}
int main(int argc, char *argv[]) {
vector<Mat> db;
// load greyscale images (these are from http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html)
db.push_back(imread("s1/1.pgm",0));
db.push_back(imread("s1/2.pgm",0));
db.push_back(imread("s1/3.pgm",0));
db.push_back(imread("s2/1.pgm",0));
db.push_back(imread("s2/2.pgm",0));
db.push_back(imread("s2/3.pgm",0));
db.push_back(imread("s3/1.pgm",0));
db.push_back(imread("s3/2.pgm",0));
db.push_back(imread("s3/3.pgm",0));
db.push_back(imread("s4/1.pgm",0));
db.push_back(imread("s4/2.pgm",0));
db.push_back(imread("s4/3.pgm",0));
int total = db[0].rows * db[0].cols;
// build matrix (column)
Mat mat(total, db.size(), CV_32FC1);
for(int i = 0; i < db.size(); i++) {
Mat X = mat.col(i);
db[i].reshape(1, total).col(0).convertTo(X, CV_32FC1, 1/255.);
}
// Change to the number of principal components you want:
int numPrincipalComponents = 12;
// Do the PCA:
PCA pca(mat, Mat(), CV_PCA_DATA_AS_COL, numPrincipalComponents);
// Create the Windows:
namedWindow("avg", 1);
namedWindow("pc1", 1);
namedWindow("pc2", 1);
namedWindow("pc3", 1);
// Mean face:
imshow("avg", pca.mean.reshape(1, db[0].rows));
// First three eigenfaces:
imshow("pc1", normalize(pca.eigenvectors.row(0)).reshape(1, db[0].rows));
imshow("pc2", normalize(pca.eigenvectors.row(1)).reshape(1, db[0].rows));
imshow("pc3", normalize(pca.eigenvectors.row(2)).reshape(1, db[0].rows));
// Show the windows:
waitKey(0);
}
and if you want to build the matrix by row (like in your original question above) use this instead:
// build matrix
Mat mat(db.size(), total, CV_32FC1);
for(int i = 0; i < db.size(); i++) {
Mat X = mat.row(i);
db[i].reshape(1, 1).row(0).convertTo(X, CV_32FC1, 1/255.);
}
and set the flag in the PCA to:
CV_PCA_DATA_AS_ROW
Regarding machine learning. I wrote a document on machine learning with the OpenCV C++ API that has examples for most of the classifiers, including Support Vector Machines. Maybe you can get some inspiration there: http://www.bytefish.de/pdf/machinelearning.pdf.
data.row(i) = projectedMat.row(0);
This will not work. operator= is a shallow copy, meaning no data is actually copied. Use
cv::Mat sample = data.row(i); // also a shallow copy, points to old data!
projectedMat.row(0).copyTo(sample);
The same also for:
desc_mat.row(i) = images[i].reshape(1, 1);
I would suggest looking at the newly checked in tests in svn head
modules/core/test/test_mat.cpp
online here : https://code.ros.org/svn/opencv/trunk/opencv/modules/core/test/test_mat.cpp
has examples for PCA in the old c and new c++
Hope that helps!