I am running the EM algorithm in OpenCV several times in a loop. Initially the EM runs with default initial parameters. In subsequent iterations we pass parameters to the EM algorithm based on output of previous iteration. Here is the code
Mat meansCombine;
Mat weightsCombine;
vector<Mat> covsCombine;
for(int k=maxComponents; k>=minComponents; k--){
EM model(k,EM::COV_MAT_DIAGONAL,TermCriteria(TermCriteria::COUNT+TermCriteria::EPS,2,0.0001));
Mat labels;
Mat probs;
Mat log_likelihoods;
if( k==maxComponents )
{
model.train(samples,log_likelihoods, labels, probs);
}
else
{
model.trainE(samples, meansCombine, covsCombine, weightsCombine, log_likelihoods, labels, probs); //provide parameters as per previous iteration results
}
double total_likelihood = 0.0;
for(int i=0;i<log_likelihoods.rows;i++){
double t = log_likelihoods.at<double>(i,0);
total_likelihood += t;
}
int dimension =3;
double l = k*(1 + dimension + ((dimension+1)*dimension)/2)-1;
double penalty = 0.5*l*log(samples.rows*dimension);
double mdl = -total_likelihood + penalty;
mdl_output << "********** No. of components=" << k << "***********" << endl;
mdl_output << "Total log likelihood=" << total_likelihood << endl;
mdl_output << "Penalty=" << penalty << endl;
mdl_output << "MDL value=" << mdl << endl;
if(mdl < minMdl)
{
minMdl = mdl;
minK = k;
}
int c1,c2;
Mat means = model.get<Mat>("means");
Mat weights = model.get<Mat>("weights");
vector<Mat> covs = model.get<vector<Mat> >("covs");
leastBhattacharyaDist(means,covs,c1,c2);
mdl_output << "Merging components" << c1 <<" and " << c2 <<endl;
meansCombine = Mat(means.rows-1,means.cols,means.type());
weightsCombine = Mat(weights.rows,(weights.cols)-1,weights.type());
covsCombine.clear();
mergeComponents(means,covs,weights,c1,c2,meansCombine,covsCombine,weightsCombine);
}
Running this code gives me the following assertion failed message.
Assertion failed (0 <= _rowRange.start && _rowRange.start <= _rowRange.end && _rowRange.end <= m.rows) in Mat, file /home/one_more_step/Documents/OpenCV/opencv-2.4.7/modules/core/src/matrix.cpp, line 284
Not able to trace the bug. Thanks in advance.
Asserts usually indicate that the following code was written with certain assumptions in mind - and your parameters do not fit the assumptions. (The dumbest thing you can do is remove the assert - yes, the code may work, but after all the assumptions aren't met so you'll shoot yourself in the foot somewhere in the future).
Asserts sometimes are complicated because they may be triggered by variables or code flows you do not control.
Usually an assertion is pretty easy to debug. Just run your code within the debugger, when the assertion happens: look at the backtrace.
The backtrace will tell you where the call happens from the code you show above.
By stepping along the frames of the backtrace, you can inspect the value of all variables along - which will tell you why the assert went off.
Related
I have a program that performs an FFT on a 2d array. In order to work with the fft library fftw3 I have to use a temporary array (called FTtemp) that reads out the result of the FFT: it is 3d since it contains the x & y axis plus the real and imaginary value for each (x,y) tupel.
The transfer of the data from the FFT array (which has a special variable type) to the ordinary array is working in debug mode but not in release. In release I get the following runtime error: Access violation writing location 0x02913000.
From my google search I found that release version bugs are usually related to uninitialized objects. This led me to explicitly initialise every item in FTtemp with 0.0, however to no avail. Furthermore, I printed the FFt array items to console and numbers appeared which means that they are also initialised. Hence, I am a bit out of ideas and wondered if someone might be wiser than me?
Here is the code snippet I am talking about. Since the program relies on a lot of other things, I was not quite able to recreate a minimal example yet, but I will add one as soon as I got the same error.
Fun fact: I print the I & j values of the loop to the console for trouble shooting and it is another (I,j) tupel where it crashes every time when I run it: eg: 49,212 or 116,169. I am really confused by this.
FTtemp = new double** [width];
for (i = 0; i < width; i++) {
FTtemp[i] = new double*[height];
}
for ( i = 0; i < width; i++)
{
for (j = 0; j < height; j++) {
FTtemp[i][j] = new double[2];
FTtemp[i][j][0] = 0.0;
FTtemp[i][j][1] = 0.0;
}
}
cout << "width,height: " << width << "," << height << endl;
for (i = 0; i < width; i++)
{
for (j = 0; j < height; j++) {
/*
cout << "access to out: " << out[indexFFT(i, j)][0] << endl;
cout << "access to FTtemp: " << FTtemp[i][j][1] << endl;
*/
cout << "i,j is: " << i << "," << j << endl;
FTtemp[i][j][1] = out[indexFFT(i, j)][0]; <--------- error occours here
FTtemp[i][j][2] = out[indexFFT(i, j)][1];
}
}
Thank you for your consideration.
All the best,
Blue
There is an error in this line:
FTtemp[i][j][2] = out[indexFFT(i, j)][1];
Notice that FTtemp[i][j] is initialized to new double[2] earlier in your code, which means that FTtemp[i][j][2] is an out-of-bounds write.
There may be other issues here - perhaps indexFFT(i, j) gives a bad index? - but without seeing how out was initialized or how indexFFT works we can't be sure.
Hope this helps!
I'm currently encountering the most strange behavior in my C++ project. What I'm trying to do is calculate the euclidean distance between two double vectors (well, actually, vectors of double vectors, hence the m_Data[0].size()).
This is the source:
double NEAT::Behavior::Distance_To(NEAT::PhenotypeBehavior* other)
{
double sum = 0.0;
for (int i = 0; i < m_Data[0].size() && i < other->m_Data[0].size(); i++) {
double x1 = m_Data[0][i];
double x2 = b->m_Data[0][i];
double difference = x1 - x2;
difference *= difference;
sum += difference;
}
return sqrt(sum);
}
I initially had all this written in one line, but I've split it up to locate the error. What happens is that after a few thousand calls to this function, it throws a SIGSEGV at the last line of the for loop:
sum += difference;
I have NO idea how this could happen. I've checked the stack trace, it's from the Distance_To(...) function and it gets thrown at this line precisely. As soon as I comment it out, everything's fine (but of course the function won't work lol). The signal gets thrown at the same time each time I run the program with the same objects interacting.
Help would be much appreciated. Thanks!
Edit: I've verified the integrity of the pointers in this method by printing out the needed values before entering the loop. All values get printed correctly. Here is the complete version of the function I used for debugging purposes:
double NEAT::Behavior::Distance_To(NEAT::PhenotypeBehavior* other)
{
double sum = 0.0;
Behavior* b = (Behavior*) other;
// Gets executed without any problems
if (genomeid == 300 && b->genomeid == 399) {
std::cout << "PROBLEM CASE" << std::endl;
std::cout << "Printing values for 300..." << std::endl;
for (int i = 0; i < m_Data[0].size(); i++) std::cout << m_Data[0][i] << std::endl;
std::cout << "Printing values for 399..." << std::endl;
for (int i = 0; i < m_Data[0].size(); i++) std::cout << b->m_Data[0][i] << std::endl;
}
// Doesn't get executed
if (m_Data[0].size() != other->m_Data[0].size()) {
std::cout << "Different sizes, " << m_Data[0].size() << " and " << b->m_Data[0].size() << std::endl;
}
// SIGSEGV at size() call
for (int i = 0; i < m_Data[0].size() && i < b->m_Data[0].size(); i++) {
double x1 = m_Data[0][i];
double x2 = b->m_Data[0][i];
double difference = x1 - x2;
difference *= difference;
// If this line gets commented out, no SIGSEGV but the program starts behaving weirdly afterwards (suddenly different sizes after the faulty run)
sum += difference;
}
return sqrt(sum);
}
ASAN and valgrind are the tools you should use to identify the root cause of this type of errors. Eventhough the error thrown at line sum += difference, your actual error could be somewhere else before hitting this point which corrupts your memory. These tools will help you to track that.
Sorry guys, I missed out on some MultiNEAT framework functions I should've used but didn't for initializing the objects etc. Anyways, thanks a lot to all of you, I learned a lot about using valgrind and ASAN (both are really handy and I didn't know about either of them before! lol) and even got a few good articles to read. Duh!
I am using OpenCV's implementation of Random Forest algorithm (i.e. RTrees) and am facing a little problem when setting parameters.
I have 5 classes and 3 variables and I want to add weight to classes because the samples sizes for each classes vary a lot.
I took a look at the documentation here and here and it seems that the priors array is the solution, but when I try to give it 5 weights (for my 5 classes) it gives me the following error :
OpenCV Error: One of arguments' values is out of range (Every class weight should be positive) in CvDTreeTrainData::set_data, file /home/sguinard/dev/opencv-2.4.13/modules/ml/src/tree.cpp, line 644
terminate called after throwing an instance of 'cv::Exception'
what(): /home/sguinard/dev/opencv-2.4.13/modules/ml/src/tree.cpp:644: error: (-211) Every class weight should be positive in function CvDTreeTrainData::set_data
If I understand well, it's due to the fact that the priors array have 5 elements. And when I try to give it only 3 elements (as my number of variables) everything works.
According to the documentation, this array should be used to add weight to classes but it actually seems that it is used to add weight to variables...
So, does anyone knows how to add weight to classes on OpenCV's RTrees algorithm ? (I'm working with OpenCV 2.4.13 in c++)
Thanks in advance !
Here is my code :
cv::Mat RandomForest(cv::Mat train_data, cv::Mat response_data, cv::Mat sample_data, int size, int size_predict, float weights[5])
{
#undef CV_TERMCRIT_ITER
#define CV_TERMCRIT_ITER 10
#define ATTRIBUTES_PER_SAMPLE 3
cv::RandomTrees RFTree;
float priors[] = {1,1,1};
CvRTParams RFParams = CvRTParams(25, // max depth
500, // min sample count
0, // regression accuracy: N/A here
false, // compute surrogate split, no missing data
5, // max number of categories (use sub-optimal algorithm for larger numbers)
//priors
weights, // the array of priors (use weights or priors)
true,//false, // calculate variable importance
2, // number of variables randomly selected at node and used to find the best split(s).
100, // max number of trees in the forest
0.01f, // forrest accuracy
CV_TERMCRIT_ITER | CV_TERMCRIT_EPS // termination cirteria
);
cv::Mat varIdx = cv::Mat();
cv::Mat vartype( train_data.cols + 1, 1, CV_8U );
vartype.setTo(cv::Scalar::all(CV_VAR_NUMERICAL));
vartype.at<uchar>(ATTRIBUTES_PER_SAMPLE, 0) = CV_VAR_CATEGORICAL;
cv::Mat sampleIdx = cv::Mat();
cv::Mat missingdatamask = cv::Mat();
for (int i=0; i!=train_data.rows; ++i)
{
for (int j=0; j!=train_data.cols; ++j)
{
if(train_data.at<float>(i,j)<0
|| train_data.at<float>(i,j)>10000
|| !float(train_data.at<float>(i,j)))
{train_data.at<float>(i,j)=0;}
}
}
// Training
std::cout << "Training ....." << std::flush;
bool train = RFTree.train(train_data,
CV_ROW_SAMPLE,//tflag,
response_data,//responses,
varIdx,
sampleIdx,
vartype,
missingdatamask,
RFParams);
if (train){std::cout << " Done" << std::endl;}
else{std::cout << " Failed" << std::endl;return cv::Mat();}
std::cout << "Variable Importance : " << std::endl;
cv::Mat VI = RFTree.getVarImportance();
for (int i=0; i!=VI.cols; ++i){std::cout << VI.at<float>(i) << " - " << std::flush;}
std::cout << std::endl;
std::cout << "Predicting ....." << std::flush;
cv::Mat predict(1,sample_data.rows,CV_32F);
float max = 0;
for (int i=0; i!=sample_data.rows; ++i)
{
predict.at<float>(i) = RFTree.predict(sample_data.row(i));
if (predict.at<float>(i)>max){max=predict.at<float>(i);/*std::cout << predict.at<float>(i) << "-"<< std::flush;*/}
}
// Personnal test due to an error I got (everyone sent to 0)
if (max==0){std::cout << " Failed ... Max value = 0" << std::endl;return cv::Mat();}
std::cout << " Done ... Max value = " << max << std::endl;
return predict;
}
The following code is designed to implement this formula:
void integrate ( double R , double E ) //radius and acceptable error, entered by user
{
int i = 1; //iterator
int n = 1; //number of rectangles
double x = 0;
double width = 0; //stores width of each rectangle
double area = 0; //stores calculated area
double error = E + 1; //stores difference between calculated area and pi*(radius)^2
while ( error > E ) //loop continues as long as error is greater than the acceptable limit entered by user
{
width = R / n; //sets the width of rectangles
area = 0; //with this line, calculated area converges to zero. without it, calculated area increases without bound
while ( i <= n )
{
x = i; //assign iterator value to a separate variable to avoid data type issues in calculation
area += width * sqrt ( pow ( R , 2.0 ) - pow ( width * ( x - 0.5 ) , 2.0 ) ); //riemann integral formula
i++;
}
n++; //increase number of rectangles by 1 to be used in next loop
area *= 4; //forumla uses one quarter of a circle, multiply by 4 to get total area
error = abs ( area - ( pi * pow ( R , 2.0 ) ) ); //error calculation
cout << " \n Calculated area = " << area << "\n" //debugging output
<< " \n Error = " << error << "\n";
cin.get();
}
cout << "\n Area: " << area << "\n" //output to user
<< "\n Error: " << error << "\n"
<< "\n Iterations: " << i << "\n";
return;
}
As the comment says, when the line "area = 0;" is included in the first loop, the area converges to zero, and when it is removed, the area increases without bound. I've been working on this one function for the better part of a week without progress. The rest of the program is completed, tested, and works fine. I have to assume that there's either an error in the logic or the math that I just can't see. Any help would be appreciated.
I hope that the comments are sufficient. I'll provided additional clarification as necessary.
(This is a self-contained function. No other part of the program affects it or is affected by it aside from the user input.)
I wrote a function within my code that should create some sort of matrices. It is fine when the size is small, but when it gets bigger, it crashes at the middle of this function without giving any information. I did that with both debug and release mode and same thing happened. Any idea on what can be wrong? Someone suggested me it could be buffer overrun.
In this function when kl.mechelms get bigger than a certain number, it crashes. The following code uses a function and gets a 3 by 3 matrix and stores it in kl.scoff which size is [3][3][kl.mechelms][kl.mechelms]. The problem happens when kl.mechelms are like bigger than 7000, but we need far more than that for our code.
Could the function Calc_3D which I use within this part cause the problem? I think it shouldn't since it just reads some values.
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q] = new double *[kl.mechelms];
}
}
for (int i = 0;i<kl.mechelms;i++) {
cout << "element " << i << "of " << kl.mechelms << endl;
kl.comments << "element " << i << "of " << kl.mechelms << endl;
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q][i] = new double[kl.mechelms];
}
}
for (int j = 0;j<kl.mechelms;j++) {
Calc_3D(i,j, kl.elmx[j], kl.elmy[j], kl.elmz[j], kl.anglemat[j], kl.dip[j], kl.elmx[i],kl.elmy[i],kl.elmz[i],
kl.elma[i],kl.rectza,kl.anglemat[i],kl.dip[i], kl.G, kl.v, kl.scofs, kl.rdepth);
for (int z = 0;z<3;z++) {
for (int q = 0;q<3;q++) {
kl.scofsarr[z][q][i][j] = kl.scofs[z][q];
}
}
}
}