Performing MNIST example with mlpack - c++

Question

I'd like to do an simple dnn example for getting known with mlpack.
The simple example is to train a dnn with the MNist Digits - the classical
ml-hello world case :-)
I managed to preparate all images using opencv-filters - the result is a
monochannel grayscale opencv::mat matrix.
I also managed to convert the pixle values to the armadillo matrix arma::mat
and to label this "pictures".
But somewhere in the past two operations, I did a mistake.
I got N examples and M Input Neurons
IN means "Input Neuron"
OL means "OutputLabel"
Ex means "Example"
The structure of my train-data is like that:
arma::mat TrainSet = {{IN_1/Ex_1,IN_/Ex_2,IN_1/Ex_3,...,IN_1/Ex_N},
{IN_2/Ex_1, IN_2/Ex_2, IN_2/Ex_3,...,IN_2/Ex_N},
{...},
{IN_M/Ex_1, IN_M/Ex_2, IN_M/Ex_3,...,IN_M/Ex_N}}`
arma::mat LabelSet = {OL_Ex_1, OL_Ex_2, ..., OL_Ex_N}
Training my network leeds to an error.
Error training artificial neural network!
Error details: Mat::operator(): index out of bounds
I'm quite sure, that the structure of my train- and labelstack is not correct.
Does anybody know, what I did wrong?
I tried to follow this example and to translate it to my case:
http://www.mlpack.org/docs/mlpack-git/doxygen/cnetutorial.html
Here is a picture of visual studio showing the structure of my trainset:
Here is a picture of visual studio showing the structure of my lables:
Thank you for any help.
Yours,
Jan
This is my code:
#pragma region Includings
#include <iostream>
#include <stdlib.h>
#include <exception>
#include <string>
#include "opencv2/opencv.hpp"
#include <mlpack\\core.hpp>
#include <mlpack/methods/ann/layer/layer.hpp>
#include <mlpack/methods/ann/ffn.hpp>
#include <mlpack/core/optimizers/cne/cne.hpp>
#pragma endregion
#pragma region Globals
std::string TrainFolder = "C:\\HomeC\\MNist\\MNist\\train-labels\\";
#pragma endregion
#pragma region Structs
typedef struct TInputPair {
double Value;
int Index;
};
typedef struct TDigitPairExample {
TInputPair* InputPairArray;
int nNonZero;
char OutputValue;
};
#pragma endregion
#pragma region Identifier
void DisplayImage(cv::Mat* Img, std::string Title = "CV::DefaultForm");
std::vector<TDigitPairExample> GenerateTrainingSet(std::string TrainFolder, int nExamplesPerClass, bool DisplayAtWindow = false);
void DisplayImage(cv::Mat* Img, std::string Title, int Delay = 0);
TInputPair* MatToArray(cv::Mat* img, int* nEntries);
int CharToOutputInt(char c);
void TransferDataToMLPack(std::vector<TDigitPairExample>* ExStack, arma::mat* DataStack, arma::mat* LabelStack, int nInput);
typedef uchar Pixel;
#pragma endregion
int main() {
#pragma region Get training examples from images
std::vector<TDigitPairExample> TrainExamples = GenerateTrainingSet(TrainFolder, 101);
#pragma endregion
#pragma region Convert training vector to armadillo matrix
arma::mat trainset, labels;
TransferDataToMLPack(&TrainExamples, &trainset, &labels, 784);
#pragma endregion
#pragma region Define network
mlpack::ann::FFN<mlpack::ann::NegativeLogLikelihood<> > network;
network.Add<mlpack::ann::Linear<> >(784, 784);
network.Add<mlpack::ann::SigmoidLayer<> >();
network.Add<mlpack::ann::Linear<> >(784, 10);
network.Add<mlpack::ann::LogSoftMax<> >();
#pragma endregion
#pragma region Train network
try {
network.Train(trainset, labels);
}catch (const std::exception& e) {
std::cout << "Error training artificial neural network!" << std::endl << "Error details: " << e.what() << std::endl;
}
#pragma endregion
std::cout << "Application finished. Press ENTER to exit..." << std::endl;
std::cin.get();
}
#pragma region Private_regions
void DisplayImage(cv::Mat* Img, std::string Title, int Delay) {
/***************/
/*Define window*/
/***************/
cv:cvNamedWindow(Title.c_str(), cv::WINDOW_AUTOSIZE);
cv::imshow(Title.c_str(), *Img);
cv::waitKey(Delay);
//cv::destroyWindow(Title.c_str());
return;
}
TInputPair* MatToArray(cv::Mat* img, int* nEntries) {
uchar* ptr = nullptr, *dptr = nullptr;
TInputPair* InPairArr = nullptr;
int j = 0;
if (img->isContinuous()) {
ptr = img->ptr<uchar>();
}else { return nullptr; }
InPairArr = (TInputPair*)malloc((img->cols) * (img->rows) * sizeof(TInputPair));
if (InPairArr == nullptr) { return nullptr; }
for (int i = 0; i < (img->rows)*(img->cols); i++) {
//std::cout << "Index_" + std::to_string(i) + "; " + std::to_string(ptr[i]) << std::endl;
if (ptr[i] != 255) { InPairArr[j].Index = i; InPairArr[j].Value = (double)(255 - ptr[i]) / 255.0; j++; }
}
InPairArr = (TInputPair*)realloc(InPairArr, j * sizeof(TInputPair));
*nEntries = j;
return InPairArr;
}
std::vector<TDigitPairExample> GenerateTrainingSet(std::string TrainFolder, int nExamplesPerClass, bool DisplayAtWindow) {
/********/
/*Localc*/
/********/
int nEntries = 0;
cv::Mat imgMod, imgGrad, imgInv, ptHull, imgHull, imgResize;
std::vector<std::vector<cv::Point>> contours;
std::vector<TDigitPairExample> TrainExamples;
TDigitPairExample TDPE;
for (int i = 1, j = 0;; i++) {
/**************/
/*Reading file*/
/**************/
cv::Mat imgOrig = cv::imread(TrainFolder + std::to_string(j) + "_" + std::to_string(i) + ".bmp", cv::IMREAD_GRAYSCALE);
if (imgOrig.empty() || i > 100) { j++; i = 1; if (j > 9) { break; } continue; }
/****************/
/*Build negative*/
/****************/
cv::subtract(cv::Scalar::all(255.0), imgOrig, imgMod);
/*****************/
/*Cut by treshold*/
/*****************/
cv::threshold(imgMod, imgMod, 230.0, 255.0, cv::THRESH_BINARY);
/**************/
/*Get contours*/
/**************/
//cv::findContours(imgMod, contours, cv::CHAIN_APPROX_NONE, 1);
//cv::Scalar color = cv::Scalar(255, 0, 0);
//cv::drawContours(imgMod, contours, -1, color, 1, 8);
//cv::Laplacian(imgOrig, imgGrad, 16, 1, 1.0, 0.0, cv::BORDER_REFLECT);
/********************/
/*Resize and display*/
/********************/
cv::resize(imgMod, imgResize, cv::Size(300, 300), .0, .0, cv::INTER_LINEAR);
TDPE.InputPairArray = MatToArray(&imgMod, &nEntries);
TDPE.nNonZero = nEntries;
TDPE.OutputValue = std::to_string(j).c_str()[0];
TrainExamples.push_back(TDPE);
if (DisplayAtWindow) { DisplayImage(&imgResize, std::string("After inversion"), 5); }
}
return TrainExamples;
}
int CharToOutputInt(char c) {
switch (c) {
case '0': return 0;
case '1': return 1;
case '2': return 2;
case '3': return 3;
case '4': return 4;
case '5': return 5;
case '6': return 6;
case '7': return 7;
case '8': return 8;
case '9': return 9;
default: throw new std::exception();
}
}
void TransferDataToMLPack(std::vector<TDigitPairExample>* ExStack, arma::mat* DataStack, arma::mat* LabelStack, int nInput) {
*DataStack = arma::zeros(nInput, ExStack->size());
*LabelStack = arma::zeros(1, ExStack->size()); /*...edit...*/
TDigitPairExample DPE;
TInputPair TIP;
/*Looping all digit examples*/
for (int i = 0; i < ExStack->size(); i++) {
DPE = (*ExStack)[i];
/*Looping all nonZero pixle*/
for (int j = 0; j < DPE.nNonZero; j++) {
TIP = DPE.InputPairArray[j];
try {
(*DataStack)(TIP.Index, i) = TIP.Value;
}catch (std::exception& ex) {
std::cout << "Error adding example[" << std::to_string(j) << "] to training stack!" << std::endl <<
"Error details: " << ex.what() << std::endl;
}
}
/*Adding label*/
try {
(*LabelStack)(0, i) = CharToOutputInt(DPE.OutputValue); /*...edit...*/
}catch (std::exception& ex) {
std::cout << "Error adding example[" << std::to_string(i) << "] to label stack!" << std::endl <<
"Error details: " << ex.what() << std::endl;
}
}
return;
}
#pragma endregion

Answer 1

The NegativeLogLikelihood loss function expects the target to be in the range [1, N], so you would have to increase the return value of CharToOutputInt. In case you haven't seen it already there is an interesting example: mlpack - DigitRecognizerCNN which could be helpful as well.

Answer 2

Ok, thanks to Marcus, I found the error. A label "0" is not allowed. I simply changed the lables to:
1 ==> Zero
2 ==> One
3 ==> Two
...
10 => Nine
That worked.