OpenCV : Training SVM Error - Assertion failed - c++

Im writing program to classify objects using SVM and BoW. I am getting the following error when I try use the TrainData::create() member function to create the data necessary to train SVM classifier.
OpenCV Error: Assertion failed (responses.type() == CV_32F || responses.type() == CV_32S) in setData
This is my function to read the train data from a director, compute BoW histogram for each train image, create a matrix of all descriptors of all train images in a matrix and the create the train data, labels and then train the SVM
void trainClassifier(string dictionaryPath, string trainDataPath, string saveClassifierPath, int samples){
//Write file
FileStorage readFile(dictionaryPath, FileStorage::READ);
//Load into Dictionary matrix
readFile["Data"] >> dictionary;
if(dictionary.empty() == false)
{
cout << "Error loading visual vocalbulary" << endl;
}
//Set the Bow descripter with the dictionary
testBOW.setVocabulary(dictionary);
//Inititate variables
vector<KeyPoint> keypointTrain;
vector<DMatch> matchTrain;
Mat descriptorTrain;
//inputTrain -> input images, inputFeatures -> BoW descriptor output
Mat inputTrain;
Mat inputFeatures;
//Label array
vector<string> label;
//Create a string to read files from directory
string updatedDataPath;
for(int i = 1; i <= samples; i++)
{
//Update the string updateDataPath to correspond the image FILENAME with each iteration
updatedDataPath.append(trainDataPath);
updatedDataPath += to_string(i);
updatedDataPath.append(".JPEG");
//Read FILE from the updated datapath
inputTrain = imread(updatedDataPath);
//Convert to single channel, since classifier takes only single channel data
cvtColor(inputTrain, inputTrain, CV_BGR2GRAY);
//Generate BoW features/histogram for the train image
testBOW.compute(inputTrain, keypointTrain, inputFeatures);
//Load the data in the descriptor Matrix
descriptorTrain.push_back(inputFeatures);
//Generate label according to the sample
if(samples > 1 && samples <= 10)
{
label.push_back("OBJ1 POSSITIVE");
}
else if (samples > 11 && samples <= 20)
{
label.push_back("OBJ1 NEGATIVE");
}
//Reset data path
updatedDataPath.clear();
}
//Convert the descriptor matrix into 32-pt float to make it compatible with classifier
if(descriptorTrain.type() != CV_32F)
{
descriptorTrain.convertTo(descriptorTrain, CV_32F);
}
//Create train data using TrainData::create()
Ptr<TrainData> trainData = TrainData::create(descriptorTrain, ROW_SAMPLE, label);
//Iniitialize Support vector based classifier (SVM) to classify and detect object
Ptr<SVM>SVM = SVM::create();
SVM->setType(SVM::C_SVC);
SVM->setKernel(SVM::LINEAR);
SVM->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, 100, 1e-6));
//Now train the SVM
SVM->trainAuto(trainData);
SVM->save(saveClassifierPath);
cout << "Classifier training status: SUCCESSFUL" << endl;}
Any help is appreciated. Thanks and cheers :)

You are using a vector<string> as the TrainData responses.
//Label array
vector<string> label;
// [long code]
//Create train data using TrainData::create()
Ptr<TrainData> trainData = TrainData::create(descriptorTrain, ROW_SAMPLE, label);
And it shoud be a Mat CV_32F or CV_32S, as the error says.
You can confirm that at:
documentation: TrainData::create(...)
source-code: TrainData::create(...) that calls setData(...)

Related

Clustering error while using BOW to get the vocabulary of the images

void SceneRecognition::BowRepresentation()
{
Mat dstGray2;
//Folder path is written and file names are taken according to that.
vector <String> fileNames;
String folder("airCond/Train/*.jpg");
glob(folder, fileNames,false);
//File names are checked.
for (auto t : fileNames)
{
cout << t << endl;
}
//Object is opened.
Ptr<SiftFeatureDetector> detector;
//Gray image holder is opened.
Mat dst, dstGray;
//Detector is created.
detector = SiftFeatureDetector::create();
//Keypoint vector is created.
vector<KeyPoint> keypoints;
//Object is opened.
Mat Desp;
Ptr<SiftDescriptorExtractor> extractor;
//Extractor is created.
extractor = SiftDescriptorExtractor::create();
Mat training_descriptors(1, extractor->descriptorSize(), extractor->descriptorType());
// Image matrices are read in a loop.
for (size_t i = 0; i < fileNames.size(); i++)
{
Mat im = imread(fileNames[i]);
//Image is converted to gray.
cvtColor(im, dstGray, COLOR_BGR2GRAY);
detector->detect(dstGray, keypoints);
//Descriptors are extracted.
extractor->compute(dstGray, keypoints, Desp);
training_descriptors.push_back(Desp);
}
cout << training_descriptors.size << endl;
/*Number of clusters are chosen as 1000.*/
//TermCriteria tc(TermCriteria::MAX_ITER + TermCriteria::EPS, 10, 0.001);
//int retries = 1;
//int flags = KMEANS_PP_CENTERS;
BOWKMeansTrainer bowTrainer(100);
bowTrainer.add(training_descriptors);
//Created descriptors are added.
cout << "a" << endl;
//Vocabulary is created by k-means clustering.
Mat vocabulary = bowTrainer.cluster();
}
When I run my code I get an error as followed: OpenCV(4.5.5) Error: Iterations do not converge (kmeans: can't update cluster center (check input for huge or NaN values)) in cv::generateCentersPP, file C:\Users\LENOVO\Desktop\openncvv\opencv-4.5.5\modules\core\src\kmeans.cpp, line 147
I tried different approaches but couldn't come up with an answer. Any suggestions?
One row of an input image
One of the image size is 75*144. Should I change something when I taking the image as input ?

how to code one vs rest multi class SVM for action recognition project

I'm new to SVM. I want to use Multiclass svm for classification in my action recognition project. My data set have 15 class like running jogging walking biking etc. I understand binary SVM and seen lot of examples. I'm getting confusion in one vs one multiclass svm and one vs rest multiclass svm.
My question is: Should I have to create and train a svm for each class like svm1 for running, svm2 for jogging.....etc what should be class lebels for each created svm either (0,1,2,.......14) or (0,1). how to determine the model for multiclass classification.how the result will be predicted ?
I was trying some thing like this:
Mat trainData, trainLabels;
for (int i = 0; i < 15; i++ )
{
sprintf_s(filename, "VocabularyHOG/Dictionary%d.yml", i);
Mat feature;
FileStorage fs(filename, FileStorage::READ);
fs["vocabulary"] >> feature;
feature.convertTo(feature, CV_32F); // make sure we got float data
trainData.push_back(feature.reshape(1, 1)); // as a flat column
trainLabels.push_back(i); // the classlabel
}
// Train the SVM
Ptr<SVM> svm = SVM::create();
svm->setType(SVM::C_SVC);
svm->setKernel(SVM::LINEAR);
svm->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, 100, 1e-6));
svm->train(trainData, ROW_SAMPLE, trainLabels);
Mat testData;
FileStorage fs("DictionaryrunningHOG.yml", FileStorage::READ);
fs["vocabulary"] >> testData;
int response = svm->predict(testData.reshape(1, 1));
if (response == 1)
cout << "boxing";
else
cout << "negative result";
waitKey(27);
return 0;
but getting error. what is wrong here?? how to code one vs rest multiclass svm properly.What is the label for each svm? Thanks

Train and test bag of visual word using VGG descriptors in OpenCV

I had a code which is getting ORB keypoints and descriptors of images and then it trains a bag of word using "BOWKMeansTrainer". the purpose of this code is to cluster photos using k-means. I don't have any problem using ORB.
Here is my problem:
I saw that openCV provides a VGG, You can find it HERE. I want to use VGG descriptor instead of ORB. Based on the documents, VGG doesn't have any function to make keypoints, so I used ORB keypoint and then used ORB keypoints to create VGG descriptors.
Ptr<ORB> detector = ORB::create();
Ptr<cv::xfeatures2d::VGG> detectorVGG = cv::xfeatures2d::VGG::create(VGG::VGG_120, 1.4f, true, true, 0.75f, true);
detector->detect(grayFrame, kp);
detectorVGG->compute(grayFrame, kp, descriptors);
This one also works and I can easily train bag of word and cluster them similar to the way I did using ORB. However when I want to test the bag of words, I'm getting this error:
error: (-215) _queryDescriptors.type() == trainDescType in function knnMatchImpl Aborttrap: 6
I tried to change the type of query descriptor Mat and descriptor Mat but I cannot solve it. In the best case , the error is changed to this:
(-215) (type == CV_8U && dtype == CV_32S) || dtype == CV_32F in function batchDistance
I am not sure if the problem is a small mistake in types or it is because of using ORB keypoints and VGG descriptors at the same time.
Part of my code is here:
vector<Mat> makeDescriptor(vector<Mat> frames)
{
int frameSize = frames.size();
vector<Mat> characteristics;
for (int i = 0 ; i < frameSize ; i++)
{
vector<KeyPoint> kp;
Mat descriptors;
Ptr<cv::xfeatures2d::VGG> detectorVGG = cv::xfeatures2d::VGG::create();
Ptr<ORB> detectorORB = ORB::create();
Mat frame = frames[i];
if (frame.empty()) {
cout << "Frame empty"<< "\n";
continue;
}
Mat grayFrame;
try
{
cvtColor(frame,grayFrame,CV_RGB2GRAY);
}
catch(Exception& e)
{
continue;
}
detectorORB->detect(grayFrame,kp);
detectorVGG->compute(grayFrame,kp,descriptors);
characteristics.push_back(descriptors);
}
return characteristics;
}
Mat makeDict(vector<Mat> characteristics, int _nCluster)
{
BOWKMeansTrainer bow(_nCluster);
for (int j = 0 ; j < characteristics.size() ; j++)
{
Mat descr = characteristics[j];
if (!descr.empty())
{
bow.add(descr);
}
}
Mat voc = bow.cluster();
return voc;
}
BOWImgDescriptorExtractor makeBow(Mat dict)
{
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("BruteForce-Hamming");
Ptr<DescriptorExtractor> extractor = VGG::create();
BOWImgDescriptorExtractor bowDE(extractor,matcher);
Mat voc;
dict.convertTo(voc,CV_32F);
bowDE.setVocabulary(voc);
return bowDE;
}
void testBow(BOWImgDescriptorExtractor bowDE, vector<Mat> frames)
{
Mat features;
vector< vector<KeyPoint> > keypoints = makeORBKeyPoints(frames);
for (int i = 0 ; i < frames.size() ; i++)
{
Mat bowDesc;
if (!keypoints[i].empty())
{
cout << "inside if" << '\n';
Mat frame;
frames[i].convertTo(frame, CV_8U);
bowDE.compute(frame, keypoints[i], bowDesc); //This Line
bowDesc.convertTo(bowDesc,CV_32F);
}
features.push_back(bowDesc);
}
}
The runtime error is happening at this line:
bowDE.compute(frame, keypoints[i], bowDesc); //This Line
I appreciate if someone helps me out to find a solution for it.
Thanks,

Error with SVM training (assertion failed for sample type)

I am trying to use SVM to match a query image with its appropriate class. Right now the classes are just 1 or 0. I extract the class from a .txt file and store it into a Mat. I use BoW to compute a histogram for each image in the training set, and also store it into a Mat.
Mat response_hist;
Mat histograms;
Mat classes;
ifstream ifs("train.txt");
int total_samples_in_file = 0;
vector<string> classes_names;
vector<string> lines;
for (int i = 1; i <= trainingSetSize; i++){
cout << "in for loop iteration"<< i << endl;
_snprintf_s(filepath, 100, "C:/Users/Randal/Desktop/TestCase1Training/train/%d.bmp", i);
Mat temp = imread(filepath, CV_LOAD_IMAGE_GRAYSCALE);
Mat tempBW;
adaptiveThreshold(temp, tempBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
detector->detect(tempBW, keypoints1);
BOW.compute(tempBW, keypoints1, response_hist);
response_hist.convertTo(response_hist, CV_32F);
histograms.push_back(response_hist);
}
//read from the file - ifs and put into a vector
std::string line;
float class_num;
string imgfilepath;
for (int j = 1; getline(ifs, line); j++)
{
istringstream ss(line);
ss >> imgfilepath >> class_num;
classes.push_back(class_num);
}
The Mats class_num and histograms are used in training the SVM. Each row in "histograms" represents a sample (a histogram of an image in the training set). "class_num" is one row with each column being the class (1 or 0) of a corresponding image in the training set.
Ptr<ml::SVM> svm = ml::SVM::create();
svm->setType(ml::SVM::C_SVC);
svm->setKernel(ml::SVM::POLY);
svm->setGamma(3);
Mat trainingDataMat(histograms);
Mat trainingDataClass(classes);
trainingDataMat.convertTo(trainingDataMat, CV_32F);
trainingDataMat = trainingDataMat.reshape(trainingDataMat.cols, 1);
trainingDataClass.convertTo(classes, CV_32F);
svm->train(trainingDataMat, ml::ROW_SAMPLE, trainingDataClass); //incorrect types? I think it is a problem with ROW_SAMPLE
Mat res; // output
svm->predict(output, res);
When I run this I get the error "Assertion failed (samples.type() == CV_32F || samples.type() == CV_32S) in cv::ml::TrainDataImpl::setData". However, I have placed lines of code in to convert both my class Mat and my histogram Mat to type CV_32F. Is the issue with my inputs or does it have something to do with ROW_SAMPLE in svm->train? Any help is greatly appreciated.
Thanks
The error was caused by my input being incorrect. I changed the type of Mat classesMat to CV_32S in its declaration. I also changed
trainingDataMat_32.reshape(trainingDataMat_32.cols, 1);
to have the correct number of channels and rows.
trainingDataMat_32.reshape(1, trainingDataMat_32.rows);
TrainingDataMat.cols was not the correct value at all. It needed 1 channel (first parameter) and the same number of rows as my histogram input.
This caused a new error regarding the kernel I am using (SVM parameter "POLY"). I had to add another line right below the kernel parameter:
svm->setDegree(3);
This fixed the error. My output is not correct, but this solved the assertion failure.

opencv 2.3.1 use bow , svm train images and load/save functions

I'm a newbee in opencv and trying to classify two image categories with opencv2.3.1.
Here is my code.
void trainSVM(map<string,Mat>& classes_training_data, string& file_postfix, int response_cols, int response_type) {
//train 1-vs-all SVMs
vector<string> classes_names;
for (map<string,Mat>::iterator it = classes_training_data.begin(); it != classes_training_data.end(); ++it) {
classes_names.push_back((*it).first);
}
string use_postfix = file_postfix;
for (int i=0;i<classes_names.size();i++) {
string class_ = classes_names[i];
Mat samples(0,response_cols,response_type);
Mat labels(0,1,CV_32FC1);
//copy class samples and label
cout << "adding " << classes_training_data[class_].rows << " positive" << endl;
samples.push_back(classes_training_data[class_]);
Mat class_label = Mat::ones(classes_training_data[class_].rows, 1, CV_32FC1);
labels.push_back(class_label);
//copy rest samples and label
for (map<string,Mat>::iterator it1 = classes_training_data.begin(); it1 != classes_training_data.end(); ++it1) {
string not_class_ = (*it1).first;
if(not_class_.compare(class_)==0) continue;
samples.push_back(classes_training_data[not_class_]);
class_label = Mat::zeros(classes_training_data[not_class_].rows, 1, CV_32FC1);
labels.push_back(class_label);
}
cout << "Train.." << endl;
Mat samples_32f; samples.convertTo(samples_32f, CV_32F);
if(samples.rows == 0) continue; //phantom class?!
CvSVM classifier;
classifier.train(samples_32f,labels);
{
stringstream ss;
ss << "SVM_classifier_";
if(file_postfix.size() > 0) ss << file_postfix << "_";
ss << class_ << ".yml";
cout << "Save.." << endl;
classifier.save(ss.str().c_str());
}
}
}
I had saved the train file successfully.And when I'm trying to load the trained file using the below snippets:
classes_classifiers[catefory[i]].load(fclass.c_str());
it runs normally.
svm.get_support_vector_count()
it's true,too.
But when adds
svm.predict(descriptors,false);
it would reports error
"OpenCV Error: Bad argument (The sample is not a valid vector) in cvPreparePredictData, file ~/OpenCV-2.3.1/modules/ml/src/inner_functions.cpp, line 1099
terminate called after throwing an instance of 'cv::Exception'
what(): ~/modules/ml/src/inner_functions.cpp:1099: error: (-5) The sample is not a valid vector in function cvPreparePredictData"
Has any could help me to solve this problem?
Regards.
Hi maybe your predict variable descriptors is not a vector, commonly the descriptors has the same size with your one train sample data.
Here is my example code for opencv's SVM:
int sample_num = 100;
int feature_size = 256;
Mat_<float> train_data(sample_num,feature_size);
//put your train sample datas
...
Mat_<int> label_data(sample_num,1);
//put your train sample label
TermCriteria criteria( CV_TERMCRIT_EPS, 1000, FLT_EPSILON );
SVMParams param( SVM::C_SVC, SVM::RBF, 10.0, 8.0, 1.0, 10.0, 0.5, 0.1, NULL, criteria );
SVM svm;
//svm training
svm.train(train_data, label_data, Mat(), Mat(), param);
svm.save("svmtest.xml");
SVM _svm;
_svm.load("svmtest.xml");
Mat_<float> test_data(1,feature_size);
//put your test data
...
int predict_label = _svm.predict(test_data);
I think you are missing the load of your vocabulary.Before loading your trained data saved in your .yml file , you must save your vocabulary : when you are using BOW descriptors (BOWImgDescriptorExtractor & BOWKMeansTrainer) , so that you can load it in another project and compute with it the new descriptor for your test image and use your saved yml file for training. Wish this can help you