FANN examples give wrong results although training seems successful - c++

Question

Using FANN I can't succeed to run copy&pasted code from FANN's website. I am using FANN version 2.2.0 on Windows 7 and MS Visual Studio 2008. My code for the training program of the XOR example looks like:
#include "floatfann.h"
#include "fann_cpp.h"
#include <ios>
#include <iostream>
#include <iomanip>
#include <string>
using std::cout;
using std::cerr;
using std::endl;
using std::setw;
using std::left;
using std::right;
using std::showpos;
using std::noshowpos;
// Callback function that simply prints the information to cout
int print_callback(FANN::neural_net &net, FANN::training_data &train,
unsigned int max_epochs, unsigned int epochs_between_reports,
float desired_error, unsigned int epochs, void *user_data)
{
cout << "Epochs " << setw(8) << epochs << ". "
<< "Current Error: " << left << net.get_MSE() << right << endl;
return 0;
}
// Test function that demonstrates usage of the fann C++ wrapper
void xor_test()
{
cout << endl << "XOR test started." << endl;
const float learning_rate = 0.7f;
const unsigned int num_layers = 3;
const unsigned int num_input = 2;
const unsigned int num_hidden = 3;
const unsigned int num_output = 1;
const float desired_error = 0.00001f;
const unsigned int max_iterations = 300000;
const unsigned int iterations_between_reports = 1000;
cout << endl << "Creating network." << endl;
FANN::neural_net net;
net.create_standard(num_layers, num_input, num_hidden, num_output);
net.set_learning_rate(learning_rate);
//net.set_activation_steepness_hidden(0.5);
//net.set_activation_steepness_output(0.5);
net.set_activation_function_hidden(FANN::SIGMOID_SYMMETRIC_STEPWISE);
net.set_activation_function_output(FANN::SIGMOID_SYMMETRIC_STEPWISE);
// Set additional properties such as the training algorithm
//net.set_training_algorithm(FANN::TRAIN_QUICKPROP);
// Output network type and parameters
cout << endl << "Network Type : ";
switch (net.get_network_type())
{
case FANN::LAYER:
cout << "LAYER" << endl;
break;
case FANN::SHORTCUT:
cout << "SHORTCUT" << endl;
break;
default:
cout << "UNKNOWN" << endl;
break;
}
net.print_parameters();
cout << endl << "Training network." << endl;
FANN::training_data data;
if (data.read_train_from_file("xor.data"))
{
// ***** MY INPUT
std::string fn;
fn = "xor_read.data";
data.save_train(fn);
fann_type **train_dat;
fann_type **out_dat;
train_dat = data.get_input();
out_dat = data.get_output();
printf("*****************\n");
printf("Printing read data (%d):\n", data.num_input_train_data());
for(unsigned int i = 0; i < data.num_input_train_data(); i++)
{
printf("XOR test (%f,%f) -> %f\n", train_dat[i][0], train_dat[i][1], out_dat[i][0]);
}
printf("*****************\n");
// END: MY INPUT **************
// Initialize and train the network with the data
net.init_weights(data);
cout << "Max Epochs " << setw(8) << max_iterations << ". "
<< "Desired Error: " << left << desired_error << right << endl;
net.set_callback(print_callback, NULL);
net.train_on_data(data, max_iterations,
iterations_between_reports, desired_error);
cout << endl << "Testing network." << endl;
for (unsigned int i = 0; i < data.length_train_data(); ++i)
{
// Run the network on the test data
fann_type *calc_out = net.run(data.get_input()[i]);
cout << "XOR test (" << showpos << data.get_input()[i][0] << ", "
<< data.get_input()[i][2] << ") -> " << *calc_out
<< ", should be " << data.get_output()[i][0] << ", "
<< "difference = " << noshowpos
<< fann_abs(*calc_out - data.get_output()[i][0]) << endl;
}
cout << endl << "Saving network." << endl;
// Save the network in floating point and fixed point
net.save("xor_float.net");
unsigned int decimal_point = net.save_to_fixed("xor_fixed.net");
data.save_train_to_fixed("xor_fixed.data", decimal_point);
cout << endl << "XOR test completed." << endl;
}
}
/* Startup function. Syncronizes C and C++ output, calls the test function
and reports any exceptions */
int main(int argc, char **argv)
{
try
{
std::ios::sync_with_stdio(); // Syncronize cout and printf output
xor_test();
}
catch (...)
{
cerr << endl << "Abnormal exception." << endl;
}
return 0;
}
I commented out :
//net.set_activation_steepness_hidden(0.5);
//net.set_activation_steepness_output(0.5);
otherwise it crashes. The file xor.data :
4 2 1
1 1
-1
-1 -1
-1
-1 1
1
1 -1
1
The output looks odd to me:
XOR test started.
Creating network.
Network Type : LAYER
Input layer : 2 neurons, 1 bias
Hidden layer : 3 neurons, 1 bias
Output layer : 1 neurons
Total neurons and biases : 8
Total connections : 13
Connection rate : 1.000
Network type : FANN_NETTYPE_LAYER
Training algorithm : FANN_TRAIN_RPROP
Training error function : FANN_ERRORFUNC_TANH
Training stop function : FANN_STOPFUNC_MSE
Bit fail limit : 0.350
Learning rate : 0.700
Learning momentum : 0.000
Quickprop decay : -0.000100
Quickprop mu : 1.750
RPROP increase factor : 1.200
RPROP decrease factor : 0.500
RPROP delta min : 0.000
RPROP delta max : 50.000
Cascade output change fraction : 0.010000
Cascade candidate change fraction : 0.010000
Cascade output stagnation epochs : 12
Cascade candidate stagnation epochs : 12
Cascade max output epochs : 150
Cascade min output epochs : 50
Cascade max candidate epochs : 150
Cascade min candidate epochs : 50
Cascade weight multiplier : 0.400
Cascade candidate limit :1000.000
Cascade activation functions[0] : FANN_SIGMOID
Cascade activation functions[1] : FANN_SIGMOID_SYMMETRIC
Cascade activation functions[2] : FANN_GAUSSIAN
Cascade activation functions[3] : FANN_GAUSSIAN_SYMMETRIC
Cascade activation functions[4] : FANN_ELLIOT
Cascade activation functions[5] : FANN_ELLIOT_SYMMETRIC
Cascade activation functions[6] : FANN_SIN_SYMMETRIC
Cascade activation functions[7] : FANN_COS_SYMMETRIC
Cascade activation functions[8] : FANN_SIN
Cascade activation functions[9] : FANN_COS
Cascade activation steepnesses[0] : 0.250
Cascade activation steepnesses[1] : 0.500
Cascade activation steepnesses[2] : 0.750
Cascade activation steepnesses[3] : 1.000
Cascade candidate groups : 2
Cascade no. of candidates : 80
Training network.
*****************
Printing read data (2):
XOR test (0.000000,1.875000) -> 0.000000
XOR test (0.000000,-1.875000) -> 0.000000
*****************
Max Epochs 300000. Desired Error: 1e-005
Epochs 1. Current Error: 0.260461
Epochs 36. Current Error: 7.15071e-006
Testing network.
XOR test (+0, +1.875) -> +5.295e-035, should be +0, difference = 5.295e-035
XOR test (+0, -1.875) -> +0, should be +0, difference = -0
XOR test (+0, -1.875) -> +0, should be +0, difference = -0
XOR test (+0, +1.875) -> +0, should be +0, difference = -0
Saving network.
XOR test completed.
The output after Testing network. looks like :
the training data as well as the test data are interpreted to be (0, +/- 1.875), as you can see in directly below the lines after Printing read data (2) and Testing network..
The (2) after Printing read data is taken from data.num_input_train_data() and my expectation would be to get a (4) since I have four sets of training data.
The "target" seems to be always "0" (see output), although the training data is never zero, but always +/- 1.
A different question has the same odd output hinting towards the training data being interpreted as (0,+/-1.875)->0.0. With this example training (like in my XOR example) also seemed to be successful, but the execution of the ANN (even on the data used for training) returned seemingly random numbers.

Answer 1

I found the answer in FANN - I get incorrect results (near 0) at simply task. It says when including "doublefann.h" one should also link the doublefann lib. This obviously holds for "floatfann.h" and the floatfann lib as well.