I am trying to write utility for training svm classifier for image classification in OpenCV3. But I have Floating point exception (core dumped) error during training process.
My main problem is that I don't know, I'm not sure exactly how to form training data to feed svm.train method.
This is code which is forming training data.
TrainingDataType SVMTrainer::prepareDataForTraining() {
cv::Mat trainingData(m_numOfAllImages, 28*28, CV_32FC1);
cv::Mat trainingLabels(m_numOfAllImages, 1, CV_32FC1);
int rowNum = 0;
// Item is pair of classId (int) and vector of images.
for(auto item : m_data){
int classId = item.first;
for(auto item1 : item.second){
Mat temp = item1.reshape(1,1);
temp.copyTo(trainingData.row(rowNum));
trainingLabels.at<float>(rowNum) = item.first;
++rowNum;
}
}
return cv::ml::TrainData::create(trainingData,
cv::ml::SampleTypes::ROW_SAMPLE,
trainingLabels) ;
}
void SVMTrainer::train(std::string& configPath){
// Read and store images in memory.
formClassifierData(configPath);
m_classifier = cv::ml::SVM::create();
// Training parameters:
m_classifier->setType(cv::ml::SVM::C_SVC);
m_classifier->setKernel(cv::ml::SVM::POLY);
m_classifier->setGamma(3);
m_classifier->setDegree(3);
TrainingDataType trainData = prepareDataForTraining();
m_classifier->trainAuto(trainData);
}
All images are already prepared with dimensions 28*28, black&white.
And actual train call is in this method
Can somebody tell me what I am doing wrong.
Thanks,
Its simple. Change the label format to CV_32SC1. It will definitely resolve your issue in opencv 3.0 ml.
Related
The piece of code which retrieves faces from grayscale image (already converted to cv::Mat) works oddly,
what I'm doing wrong?
// in initializer list
model(cv::face::FisherFaceRecognizer::create())
// ....
const cv::Mat grayscale = cv::imread("photo_15.jpeg",cv::IMREAD_GRAYSCALE);
std::vector<cv::Rect> faceCandidates;
m_cascade.detectMultiScale(grayscale, faceCandidates);
uint32 label = -1;
double confidence = 0.0;
// this line for the testing purposes only
model->predict(grayscale, label, confidence);
this works fine : label refers to correct person and confidence within 10.
but lets continue with this function code:
for (auto &faceCandidateRegion : faceCandidates) {
cv::Mat faceResized;
// size_ is a member and contains 1280x720 for my case, equal to model trained photos.
cv::resize( cv::Mat(grayscale, faceCandidateRegion), faceResized, cv::Size(size_.width(), size_.height()));
// Recognize current face.
m_model->predict(faceResized, label, confidence);
// ... other processing
this piece of code works absolutely wrong: it always produces incorrect label and confidence is about ~45-46K even if I use a recognition photo from training photo set
any idea what I'm doing wrong here?
for the testing : I've tried to perform this with fisher, eigen and lbph with the same wrong result
update: each model in the app is a few user's group, where each user presented by 2-6 photos , so this is a reason why I train a few users in the model
here is a code which trains the models:
std::size_t
Recognizer::extractFacesAndConvertGrayscale(const QByteArray &rgb888, std::vector<cv::Mat> &faces)
{
cv::Mat frame = cv::imdecode(std::vector<char>{rgb888.cbegin(), rgb888.cend()}, cv::IMREAD_GRAYSCALE);
std::vector<cv::Rect> faceCandidates;
m_cascade.detectMultiScale(frame, faceCandidates);
int label = 0;
for(const auto &face : faceCandidates) {
cv::Mat faceResized;
cv::resize(cv::Mat{frame, face}, faceResized,
cv::Size(this->m_size.width(), this->m_size.height()));
faces.push_back(faceResized);
}
return faceCandidates.size();
}
bool Recognizer::train(const std::vector<qint32> &labels, const std::vector<QByteArray> &rgb888s)
{
if (labels.empty() || rgb888s.empty() || labels.size() != rgb888s.size())
return false;
std::vector<cv::Mat> mats = {};
std::vector<int32_t> processedLabels = {};
std::size_t i = 0;
for(const QByteArray &data : rgb888s)
{
std::size_t count = this->extractFacesAndConvertGrayscale(data, mats);
if (count)
std::fill_n(std::back_inserter(processedLabels), count, labels[i++]);
}
m_model->train(mats, processedLabels);
return true;
}
We resolved this in the comments, but for future reference:
The fact that this line
// this line for the testing purposes only
model->predict(grayscale, label, confidence);
had better confidence than
// Recognize current face.
m_model->predict(faceResized, label, confidence);
occurred because the model was trained with non-cropped images, while the detector crops the faces.
Rather than using the whole image with prediction, to match the input, the model should be trained with cropped faces:
The classifier performs independently of the size of the faces in the original image, due to the multiscale detection; i.e. size and position of the faces in the image become invariants.
Background does not interfere with classification. The original input had 16:9 aspect ratio, so at least the sides of the image would produce noise in the descriptors.
I try to calculate Haar feature using opencv (Given an image).
Input: an image
output: haar feature
For that, I am using the FeatureEvaluator from OpenCV.
But I got an exception when I try to calculate one feature.
Here is how I am doing:
Ptr<FeatureEvaluator> ptrHaar = FeatureEvaluator::create(FeatureEvaluator::HAAR);
Mat img = imread(image_path); // image of size 2048*1536 correctly loaded
ptrHaar->setImage(img, Size(100, 100));
ptrHaar->setWindow(Point(0, 0));
double res = ptrHaar->calcOrd(0); // get the exception here
I think that you need to load/create some type of Haar feature, not just create an object. Try to load some Haar cascade classifier using load method and than try to use calcOrd method.
Your code is almost right. Only is missing is to read the CascadeClassifier previously trained. You can do this as follow:
FileStorage fs( "cascade.xml", FileStorage::READ );
//2) Then, create a FileNode to access the features:
FileNode featuresNode = fs["cascade"]["features"];
//3) Create the FeatureEvaluator, as you did in your first line
//4) Read the FileNode you have created:
ptrHaar->read(featuresNode);
And continue accordingly your code.
Note that ptrHaar->calcOrd(0) will read just the first feature rectangle, if you have more to read, you will need a loop, like this:
FileNodeIterator it = featuresNode.begin(), it_end = featuresNode.end();
int idx = 0;
for( ; it != it_end; ==it, idx++ )
{
res = ptrHaar.calcOrd(idx);
}
I'm learning Neural Networks from this bytefish machine learning guide and code. I understand it well but I would like to update the code at the previous link to use image pixel data instead of random values as the input data. In this section of the aforementioned code:
cv::randu(trainingData,0,1);
cv::randu(testData,0,1);
the training and test matrices are filled with random data. Then label data is added to the classes matrices here:
cv::Mat trainingClasses = labelData(trainingData, eq);
cv::Mat testClasses = labelData(testData, eq);
using this function:
// label data with equation
cv::Mat labelData(cv::Mat points, int equation) {
cv::Mat labels(points.rows, 1, CV_32FC1);
for(int i = 0; i < points.rows; i++) {
float x = points.at<float>(i,0);
float y = points.at<float>(i,1);
labels.at<float>(i, 0) = f(x, y, equation);
// the f() function used above
//is only a case statement with 5
//switches in it eg on of the switches is:
//case 0:
//return y > sin(x*10) ? -1 : 1;
//break;
}
return labels;
}
Then points are plotted in a window here:
plot_binary(trainingData, trainingClasses, "Training Data");
plot_binary(testData, testClasses, "Test Data");
with this function:
;; Plot Data and Class function
void plot_binary(cv::Mat& data, cv::Mat& classes, string name) {
cv::Mat plot(size, size, CV_8UC3);
plot.setTo(cv::Scalar(255.0,255.0,255.0));
for(int i = 0; i < data.rows; i++) {
float x = data.at<float>(i,0) * size;
float y = data.at<float>(i,1) * size;
if(classes.at<float>(i, 0) > 0) {
cv::circle(plot, Point(x,y), 2, CV_RGB(255,0,0),1);
} else {
cv::circle(plot, Point(x,y), 2, CV_RGB(0,255,0),1);
}
}
imshow(name, plot);
}
The plotted points, as I understand it, represent the input data multiplied by the equations in the f() function and is used by the predict functions to predict which point to plot in the mlp, knn, svm etc. functions. How do I update what is going on here to do something with Image pixel data. Any advice to get me farther would be appreciated.
"How do I update what is going on here to do something with Image pixel data" is a broad and generic question. May I ask in exchange: what do you want to do with "Image pixel data"?
Do you want an answer to: what can be done with "Image pixel data" on machine learning algorithms like ANN, SVM etc. ?
The answer is a loooong list of things encompassing thousands of research papers and hundreds of PhD theses. Some examples include: supervised and/or un-supervised classification of images into labels/tags/categories based on features like image content, objects in image, patterns in image etc. The possibilities are endless. You may perhaps want to take a look at this: http://stuff.mit.edu/afs/athena/course/urop/profit/PDFS/EdwardTolson.pdf
Now, coming back to you original objective: "I would like to update the code at the previous link to use image pixel data instead of random values as the input data"...
The implementation technique would depend largely on what you want to do. I can cite one/two easy techniques for extracting feature vectors from image, which can be fed into any machine learning algorithm of your choice...
Example 1:
You may start with using pixel intensity data as a feature vector. Here's how you may go ahead with it:
Load image using
Mat image = imread(argv[1], CV_LOAD_IMAGE_GRAYSCALE);
Resize image into a smaller area using resize. You may want to begin with small image sizes, like 8x8 or 10x10 pixels.
Loop through the image matrix, somewhat like this:
for(int row = 0; row < img.rows; ++row)
{
uchar* p = img.ptr(row);
for(int col = 0; col < img.cols; ++col)
{
*p++ //points to each pixel value in turn assuming a CV_8UC1 greyscale image
}
}
A collection of all the pixel values will give you a feature vector for that image.
Now suppose you have two classes of image. For each set of feature vector you generate, you'll have to prepare (for supervised classification) a corresponding label Mat (somewhat like the example you've mentioned). It needs to contain the class label (say, 0 and 1) for all the feature vectors present in your feature Mat.
Now feed the feature vectors and label Mat to your machine learning code and see what happens.
However, the ability of image classification based on image pixel data alone is quite limited. There are thousands of techniques for extracting image features, most of which are dependent on the application area.
Example 2:
I'll finish off with one more example for extracting feature vectors, which, in some cases, will prove to be more effective than simple image pixel values.
You may use the Histogram of Oriented Gradients descriptor for slightly better results, use this:
cv::HOGDescriptor hog;
vector<float> descriptors;
hog.compute(mat, descriptors);
The vector descriptors is your feature vector.
HOGDescriptors, when used with SVM, provides a decent classification mechanism.
You can put the pixel data of an image into a Mat called trainingData using something similar to this:
cv::Mat labelData(cv::Mat points, int equation)
{
cv::Mat labels(points.rows, 1, CV_32FC1);
for(int i = 0; i < points.rows; i++)
{
float x = points.at<float>(i,0);
float y = points.at<float>(i,1);
labels.at<float>(i, 0) = f(x, y, equation);
}
return labels;
}
Now, instead of labelData, we're going to return a Mat of pixel data. One obvious way is to use the image itself as a feature vector. However, some machine learning algorithms in openCV, including ANN, SVM etc., required special formatting of input data.
You may try something like this:
cv::Mat trainingData(cv::Mat image)
{
cv::Mat trainingVector(image.rows*image.cols, 1, CV_32FC1);
for(int i = 0; i < image.rows; i++)
{
for(int j = 0; j < image.cols; j++)
{
float valueOfPixel = image.at<float>(i,j);
trainingVector.at<float>((i*image.cols)+j, 0) = valueOfPixel;
}
}
return trainingVector;
}
(Please recheck the syntax of the code before using, I just typed it out here)
So, what the above block effectively does is change the 2D matrix of the image into a 1D array. Now, how and where you use it depends on your requirements.
Please make necessary modifications before invoking the machine learning modules.
Hope this answers your question.
Thanks.
I have create a SVM detector for faces in opencv. I want to separate the process of train and predict. Thus, I want to find a way to store in my hard disk CvSVM SVM object in order to access it whenever I want without train again a new model. In this way, I could use predict method by just loading SVM object from hard disk. This is pretty straight forward in Matlab, but how can I succeed it in opencv c++?
Using load and save functions in order to load and save SVM model it leads to munmap_chunk()::invalid pointer message(glibc detected) message.
CvSVM SVM = detect.SVMtrain("dbpath/");
SVM.save("svmTrain.xml", 0);
cout <<"Detection system in progress progress...\n";
string pathFile = databasePath+ imageFilename;
vector<float> confidence;
detections = detect.detection(pathFile);
CvSVM SVM;
SVM.load("svmTrain.xml",0);
confidence = detect.SVMpredict(detections.at(0), SVM);
//cout << "confidence is: " << confidence.at(0) << endl;
When I ran the above code I received the above message. The problem stands in svmpredict function. And SVMpredict function:
vector<float> Detection::SVMpredict(Mat image, CvSVM SVM){
vector<float> res;
//Mat image = imread(fileName,0); //Read as a gray image
//cvtColor(sampleMat, sampleMat, CV_BGR2GRAY);
image = eigenfacesExtraction(image);
image.convertTo(image, CV_32FC1); // <-- Convert to CV_32F for matrix mult
float response = SVM.predict(image); res.push_back(response);
float val = SVM.predict(image,true); res.push_back(val);
cout << "svm result: "<< response <<" val: "<< val << endl;
return res;
}
Straightforward too. It's a CvStatModel, so it comes with save and load methods.
I'm trying to use CvNormalBayesClassifier to train my program to learn skin pixel colors. I have a set of training images and response images. The response images are in black and white, skin regions are marked white. The following is my code,
CvNormalBayesClassifier classifier;
for (int i = 0; i < numFiles; i++) {
string trainFile = "images/" + int2str(i) + ".jpg";
string responseFile = "images/" + int2str(i) + "_mask.jpg";
Mat trainData = imread(trainFile, 1);
Mat responseData = imread(responseFile, CV_LOAD_IMAGE_GRAYSCALE);
trainData = trainData.reshape(1, trainData.rows * trainData.cols);
responseData = responseData.reshape(0, responseData.rows * responseData.cols);
trainData.convertTo(trainData, CV_32FC1);
responseData.convertTo(responseData, CV_32FC1);
classifier.train(trainData, responseData, Mat(), Mat(), i != 0);
}
However, it is giving the following error,
The function/feature is not implemented (In the current implementation the new training data must have absolutely the same set of class labels as used in the original training data) in CvNormalBayesClassifier::train
Many thanks.
As the error message states, you cannot 'update' the classifier in light of new class labels. The Normal Bayes Classifier learns a Mixture of Gaussians to represent the training data. If you suddenly start adding new labels this mixture model will cease to be correct and a new model must be learned from scratch.
Ok, I found that the problem was that the black and white images have been compressed and thus contain values ranging from 0-255. Therefore, there can be a new class label in the other images.
To solve this problem, use thresholding to make the value all become 0 or 255.