Backpropagation giving strange values in C++ neural network - c++

I am trying to solve the iris data set with a neural network I wrote in C++ from scratch, which has 150 rows divided up into 3 different flowers with 4 columns and then a fifth for the flower type which I converted to a 0, 1 or 2.
Problem:
Whenever I run the network it will go through a test set of 90 rows, split into 3 different flowers (30, 30, 30). Every time I run an epoch it will show the output values being all very high like (0.99, 0.99, 0.98). It will do that for a few epochs and then eventually get lower to more sensible values. But when it will get to the later epochs, when I'm doing say 50 epochs, the values for the correct flower will get closer and closer to 1.00, for each flower, then do the same for the next flower and the flower after that, then it will start that process over. Instead of starting close to 1.0 indicating that it had learned and the weights were properly adjusted.
Console output for running epoch (which runs forward_prop(), back_prop() and then update_weights()), after each epoch it prints out the output values for the network. Printing at the end of the epoch means that the actual values are always {0, 0, 1}. When I ran the network I ran it 1000 times, the output values never changed for every epoch after 15. Why is it doing this?
File parsed, weights and bias randomized
Epoch 1
0.97 0.97 0.99 Epoch 2
0.93 0.94 0.99 Epoch 3
0.64 0.70 0.99 Epoch 4
0.27 0.36 0.99 Epoch 5
0.22 0.31 0.99 Epoch 6
0.21 0.30 0.99 Epoch 7
0.21 0.30 0.98 Epoch 8
0.21 0.30 0.98 Epoch 9
0.21 0.30 0.96 Epoch 10
0.21 0.30 0.88 Epoch 11
0.21 0.30 0.66 Epoch 12
0.21 0.30 0.56 Epoch 13
0.21 0.30 0.54 Epoch 14
0.21 0.30 0.53 Epoch 15
0.21 0.30 0.53 completed successfully
End console output.
Example of epoch 9
0.21 0.30 0.98
0.21 0.30 0.98
0.22 0.29 0.98
0.23 0.29 0.98
0.24 0.28 0.98
0.25 0.28 0.98
0.25 0.27 0.98
0.26 0.27 0.98
0.27 0.27 0.98
0.28 0.26 0.98
0.29 0.26 0.98
0.30 0.26 0.98
0.31 0.26 0.98
0.32 0.25 0.98
0.34 0.25 0.98
0.35 0.24 0.98
0.36 0.24 0.98
0.37 0.24 0.98
0.38 0.24 0.98
0.40 0.23 0.98
0.41 0.23 0.98
0.42 0.23 0.98
0.43 0.23 0.98
0.44 0.22 0.98
0.45 0.22 0.98
0.46 0.22 0.98
0.48 0.22 0.98
0.49 0.22 0.98
0.50 0.21 0.98
0.51 0.21 0.98
0.53 0.20 0.98
0.52 0.21 0.98
0.50 0.22 0.98
0.49 0.23 0.98
0.48 0.24 0.98
0.47 0.24 0.98
0.46 0.25 0.98
0.45 0.26 0.98
0.44 0.27 0.98
0.43 0.28 0.98
0.42 0.29 0.98
0.42 0.30 0.98
0.41 0.32 0.98
0.40 0.33 0.98
0.39 0.34 0.98
0.38 0.35 0.98
0.38 0.36 0.98
0.37 0.37 0.98
0.36 0.38 0.98
0.35 0.40 0.98
0.35 0.41 0.98
0.34 0.42 0.98
0.34 0.43 0.98
0.33 0.44 0.98
0.32 0.46 0.98
0.32 0.47 0.98
0.31 0.48 0.98
0.31 0.49 0.98
0.30 0.50 0.98
0.30 0.51 0.97
0.30 0.52 0.98
0.29 0.51 0.98
0.29 0.50 0.98
0.28 0.49 0.98
0.28 0.48 0.98
0.27 0.47 0.98
0.27 0.46 0.97
0.27 0.45 0.98
0.26 0.44 0.98
0.26 0.43 0.98
0.26 0.42 0.98
0.25 0.41 0.98
0.25 0.40 0.98
0.25 0.40 0.98
0.24 0.39 0.98
0.24 0.38 0.98
0.24 0.37 0.98
0.24 0.37 0.98
0.23 0.36 0.98
0.23 0.35 0.98
0.23 0.35 0.98
0.23 0.34 0.98
0.22 0.33 0.98
0.22 0.33 0.98
0.22 0.32 0.98
0.22 0.32 0.98
0.21 0.31 0.98
0.21 0.31 0.98
0.21 0.30 0.98
0.21 0.30 0.98 Epoch 9
So with epoch 9 the first 30 rows have an actual value of {1, 0, 0}, then next 30 have an actual value of {0, 1, 0} and finally the last 30 have an actual value of {0, 0, 1}. See how it inches closer and closer for each row of data, yet the last row stays the same, while not staying the same for all the epochs. This is strange and I am not sure exactly why it is doing this.
So the basic structure of the program is:
main() executes, declare and initialize a class Neural_Network with a input, hidden and output layer.
calling train() then executes epoch() which runs in a loop the amount of times specified when calling train.
epoch() itself runs forward_prop(), back_prop() and finally update_network(), there are also a few variables like arrays for the expected and actual values for the output.
The vectors bias, values, weights and errors all hold the values for the network separately, which I found was better for readability. the first layer or position [0] of the weights vector is empty and the input values use the weights in the hidden layer and the hidden layer uses the weights in the output layer.
Each weight is a vector of weights equal to the amount of nodes in the previous layer, Position [0] of the vector of weights is used for the node at position [0] in the previous layer.
#include <iostream>
#include <cstdlib>
#include <iomanip>
#include <cmath>
#include <fstream>
#include <sstream>
#include <vector>
#include <array>
#include <string>
#include <numeric>
class Neural_Network
{
private:
std::vector<std::array<double, 4>> training_set; // 30 setosa -> 30 versicolor -> 30 virginica
std::vector<std::vector<double>> values, bias, errors;
std::vector<std::vector<std::vector<double>>> weights;
size_t net_size = 0;
double dot_val(std::vector<double> val, std::vector<double> weights);
double sigmoid(const double num);
double random_number();
double transfer_derivitive(double num);
void initialize(std::vector<size_t> layers);
void forward_prop(std::vector<double>& expected);
void back_prop(std::vector<double> expected);
void update_network(double l_rate);
public:
Neural_Network(const std::vector<std::array<double, 4>>& data);
~Neural_Network() = default;
void train(size_t epochs = 1);
void display();
};
Neural_Network::Neural_Network(const std::vector<std::array<double, 4>>& data) : training_set{ data }
{
initialize({ 4, 6, 3 });
}
double Neural_Network::dot_val(std::vector<double> val, std::vector<double> weights)
{
return std::inner_product(val.begin(), val.end(), weights.begin(), 0.0);
}
double Neural_Network::sigmoid(const double num)
{
return (1 / (1 + exp(-num)));
}
double Neural_Network::random_number()
{
return (double)rand() / (double)RAND_MAX;
}
double Neural_Network::transfer_derivitive(double num)
{
return num * (1 - num);
}
void Neural_Network::display()
{
std::cout << std::fixed << std::setprecision(2) << "values:\n";
for (size_t i = 0; i < values.size(); ++i)
{
std::cout << "layer " << i << "\n[ ";
for (size_t j = 0; j < values[i].size(); ++j)
std::cout << values.at(i).at(j) << " ";
std::cout << " ]\n";
}
}
void Neural_Network::initialize(std::vector<size_t> layers)
{
for (size_t i = 0; i < layers.size(); ++i)
{
std::vector<double> v{}, b{}, e{};
std::vector<std::vector<double>> w{};
//initializing the nodes in the layers
for (size_t j = 0; j < layers.at(i); ++j)
{
v.push_back(0);
b.push_back(random_number());
e.push_back(1);
std::vector<double> inner_w{};
if (i != 0) // checking if the current layer is the input
for (size_t k = 0; k < layers.at(i - 1); ++k) // adding weights to the current layer to the amount of nodes in the next layer
inner_w.push_back(random_number()); // adding a weight to the current layer for a node in the next layer
w.push_back(inner_w);
}
values.push_back(v);
bias.push_back(b);
errors.push_back(e);
weights.push_back(w);
++net_size;
}
std::cout << "initialize network success" << std::endl;
}
void Neural_Network::train(size_t epoch_count)
{
const size_t count = epoch_count;
while (epoch_count > 0)
{
std::cout << "\nEpoch " << 1 + (count - epoch_count) << std::endl;
for (size_t i = 0; i < 90; ++i)
{
std::vector<double> expected{ 0, 0, 0 };
if (i < 30)
expected[0] = 1;
else if (i < 60)
expected[1] = 1;
else if (i < 90)
expected[2] = 1;
for (size_t j = 0; j < values[0].size(); ++j) // Initialize input layer values
values.at(0).at(j) = training_set.at(i).at(j); // value[0] is the input layer, j is the node
forward_prop(expected);
back_prop(expected);
update_network(0.05);
}
display();
--epoch_count;
}
}
void Neural_Network::forward_prop(std::vector<double>& expected)
{
for (size_t i = 1; i < net_size - 1; ++i) // looping through every layer except the first and last
for (size_t j = 0; j < values.at(i).size(); ++j) // looping through every node in the current non input/output layer
values.at(i).at(j) = sigmoid(dot_val(values.at(i - 1), weights.at(i).at(j)) + bias.at(i).at(j)); // assigning node j of layer i a sigmoided val that is the dotval + the associated bias
for (size_t i = 0; i < values.at(net_size - 1).size(); ++i) // looping through the ouptut layer
values.at(net_size - 1).at(i) = sigmoid(dot_val(values.at(net_size - 2), weights.at(net_size - 1).at(i)) + bias.at(net_size - 1).at(i));
}
void Neural_Network::back_prop(std::vector<double> expected) // work backwards from the output layer
{
std::vector<double> output_errors{};
for (size_t i = 0; i < errors.at(net_size - 1).size(); ++i) // looping through the output layer
{
output_errors.push_back(expected.at(i) - values.at(net_size - 1).at(i));
errors.at(net_size - 1).at(i) = output_errors.at(i) * transfer_derivitive(values.at(net_size - 1).at(i));
} // output layer finished
for (size_t i = net_size - 2; i > 0; i--) // looping through the non output layers backwards
{
std::vector<double> layer_errors{};
for (size_t j = 0; j < errors.at(i).size(); ++j) // looping through the current layer's nodes
{
double error = 0;
for (size_t k = 0; k < weights.at(i + 1).size(); ++k) // looping through the current set of weights
error += errors.at(i).at(j) * weights.at(i + 1).at(k).at(j);
layer_errors.push_back(error);
}
for (size_t j = 0; j < layer_errors.size(); ++j)
errors.at(i).at(j) = layer_errors.at(j) * transfer_derivitive(values.at(i).at(j));
}
}
void Neural_Network::update_network(double l_rate)
{
for (size_t i = 1; i < net_size; ++i)
{
for (size_t j = 0; j < weights.at(i).size(); ++j)
{
for (size_t k = 0; k < weights.at(i).at(j).size(); ++k)
weights.at(i).at(j).at(k) += l_rate * errors.at(i).at(j) * values.at(i - 1).at(j);
bias.at(i).at(j) += l_rate * errors.at(i).at(j);
}
}
}
int main()
{
std::vector<std::array<double, 4>> data = {
{5.1, 3.5, 1.4, 0.2},
{4.9, 3, 1.4, 0.2},
{4.7, 3.2, 1.3, 0.2},
{4.6, 3.1, 1.5, 0.2},
{5, 3.6, 1.4, 0.2},
{5.4, 3.9, 1.7, 0.4},
{4.6, 3.4, 1.4, 0.3},
{5, 3.4, 1.5, 0.2},
{4.4, 2.9, 1.4, 0.2},
{4.9, 3.1, 1.5, 0.1},
{5.4, 3.7, 1.5, 0.2},
{4.8, 3.4, 1.6, 0.2},
{4.8, 3, 1.4, 0.1},
{4.3, 3, 1.1, 0.1},
{5.8, 4, 1.2, 0.2},
{5.7, 4.4, 1.5, 0.4},
{5.4, 3.9, 1.3, 0.4},
{5.1, 3.5, 1.4, 0.3},
{5.7, 3.8, 1.7, 0.3},
{5.1, 3.8, 1.5, 0.3},
{5.4, 3.4, 1.7, 0.2},
{5.1, 3.7, 1.5, 0.4},
{4.6, 3.6, 1, 0.2},
{5.1, 3.3, 1.7, 0.5},
{4.8, 3.4, 1.9, 0.2},
{5, 3, 1.6, 0.2},
{5, 3.4, 1.6, 0.4},
{5.2, 3.5, 1.5, 0.2},
{5.2, 3.4, 1.4, 0.2},
{4.7, 3.2, 1.6, 0.2},
{7, 3.2, 4.7, 1.4},
{6.4, 3.2, 4.5, 1.5},
{6.9, 3.1, 4.9, 1.5},
{5.5, 2.3, 4, 1.3},
{6.5, 2.8, 4.6, 1.5},
{5.7, 2.8, 4.5, 1.3},
{6.3, 3.3, 4.7, 1.6},
{4.9, 2.4, 3.3, 1},
{6.6, 2.9, 4.6, 1.3},
{5.2, 2.7, 3.9, 1.4},
{5, 2, 3.5, 1},
{5.9, 3, 4.2, 1.5},
{6, 2.2, 4, 1},
{6.1, 2.9, 4.7, 1.4},
{5.6, 2.9, 3.6, 1.3},
{6.7, 3.1, 4.4, 1.4},
{5.6, 3, 4.5, 1.5},
{5.8, 2.7, 4.1, 1},
{6.2, 2.2, 4.5, 1.5},
{5.6, 2.5, 3.9, 1.1},
{5.9, 3.2, 4.8, 1.8},
{6.1, 2.8, 4, 1.3},
{6.3, 2.5, 4.9, 1.5},
{6.1, 2.8, 4.7, 1.2},
{6.4, 2.9, 4.3, 1.3},
{6.6, 3, 4.4, 1.4},
{6.8, 2.8, 4.8, 1.4},
{6.7, 3, 5, 1.7},
{6, 2.9, 4.5, 1.5},
{5.7, 2.6, 3.5, 1},
{6.3, 3.3, 6, 2.5},
{5.8, 2.7, 5.1, 1.9},
{7.1, 3, 5.9, 2.1},
{6.3, 2.9, 5.6, 1.8},
{6.5, 3, 5.8, 2.2},
{7.6, 3, 6.6, 2.1},
{4.9, 2.5, 4.5, 1.7},
{7.3, 2.9, 6.3, 1.8},
{6.7, 2.5, 5.8, 1.8},
{7.2, 3.6, 6.1, 2.5},
{6.5, 3.2, 5.1, 2},
{6.4, 2.7, 5.3, 1.9},
{6.8, 3, 5.5, 2.1},
{5.7, 2.5, 5, 2},
{5.8, 2.8, 5.1, 2.4},
{6.4, 3.2, 5.3, 2.3},
{6.5, 3, 5.5, 1.8},
{7.7, 3.8, 6.7, 2.2},
{7.7, 2.6, 6.9, 2.3},
{6, 2.2, 5, 1.5},
{6.9, 3.2, 5.7, 2.3},
{5.6, 2.8, 4.9, 2},
{7.7, 2.8, 6.7, 2},
{6.3, 2.7, 4.9, 1.8},
{6.7, 3.3, 5.7, 2.1},
{7.2, 3.2, 6, 1.8},
{6.2, 2.8, 4.8, 1.8},
{6.1, 3, 4.9, 1.8},
{6.4, 2.8, 5.6, 2.1},
{7.2, 3, 5.8, 1.6}
};
Neural_Network network{ data };
network.train(1);
return 0;
}
Edit to use .at() instead of [] for accessing std::vector in program
I hope I made everything clear, if not let me know.
note: I had this question of stackoverflow, I was told that I should
move it to codereview.stackexchange, then they told me I should move
it back to stackoverflow again, while reframing my question with more
detail. Please don't tell me to move this question a 3rd time. If there is something wrong with the way I am asking please give me a chance to change it or add information so I can get some help, please and thank you

One obvious mistake is in dot_val:
double Neural_Network::dot_val(std::vector<double> val,std::vector<double> weights)
{
double output; // <-- This is uninitialized
for (size_t i = 0; i < weights.size(); ++i)
output += val[i] * weights[i];
return output; // <-- Who knows what this will be
}
You are using an uninitialized variable. Either initialize output to 0, or you can use
std::inner_product :
#include <numeric>
//...
double Neural_Network::dot_val(std::vector<double> val,std::vector<double> weights)
{
return std::inner_product(val.begin(), val.end(), weights.begin(), 0.0);
}

Related

why does my circle anti aliasing algorithm in C++ give asymmetrical results?

Some background:
So my plan is to create a stippling algorithm in C++ and I basically just plan on storing a whole bunch of data for each radius of a circle to write onto a texture map in OpenGL I'm not sure if this is the right thing to do but I feel like it
would be quicker than the computer dynamically calculating the radius for each circle especially if lots of circles are the same size, my plan is to create a function that just writes a whole text document full of radiuses up to a certain size and this data will be stored bitwise inside an array of long's std::array <long> bit = {0x21, 0x0A ect... } so that I can encode 4X4 arrays of values with 2 bits assigned to the antialiasing value of each pixel however to create this database of ant-aliased circles I need to write a function that I keep getting wrong;
The actual question:
So this may seem lazy but I can promise I have tried everything to wrap my head around what I am getting wrong here basically i have written this code to anti=alias by dividing up the pixels into sub pixels however it seems to be returning values greater than 1 which shouldn't be possible as i have divided each pixel into 100 pixels of size 0.01
float CircleConst::PixelAA(int I, int J)
{
float aaValue = 0;
for (float i = (float) I; i < I + 1; i += 0.1f)
{
for (float j = (float) J; j < J + 1; j += 0.1f)
{
if ((pow((i - center), 2) + pow((j - center), 2) < pow(rad, 2)))
aaValue += 0.01f;
}
}
return aaValue;
}
also here is the code that writes the actual circle
CircleConst::CircleConst(float Rad)
{
rad = Rad;
dataSize = (unsigned int) ceil(2 * rad);
center = (float) dataSize/2;
arrData.reserve((int) pow(dataSize, 2));
for (int i = 0; i < dataSize; i++)
{
for (int j = 0; j < dataSize; j++)
{
if ( CircleBounds(i, j, rad-1) )
arrData.push_back(1);
else if (!CircleBounds(i, j, rad - 1) && CircleBounds(i, j, rad + 1))
{
arrData.push_back(PixelAA(i,j));
}
else
arrData.push_back(0);
}
}
}
so I noticed without the antialiasing that the way the circle is written is shifted over by one line, but this could be fixed by changing the value of the centre of the circle todataSize/2 - 0.5f but this causes problems later on when the circle is asymmetrical with the antialiasing, here is an example of radius 3.5
0.4 1.0 1.1 1.1 1.1 0.4 0.0
1.0 1.0 1.0 1.0 1.0 1.1 0.2
1.1 1.0 1.0 1.0 1.0 1.0 0.5
1.1 1.0 1.0 1.0 1.0 1.0 0.5
1.1 1.0 1.0 1.0 1.0 1.0 0.2
0.4 1.1 1.0 1.0 1.0 0.5 0.0
0.0 0.2 0.5 0.5 0.2 0.0 0.0
as you can see some of the values are over 1.0 which should not be possible, I'm sure there is an obvious answer to why this is but I'm completely missing it.
The problem lies with lines such as this one:
for (float i = (float) I; i < I + 1; i += 0.1f)
Floating point numbers cannot be stored or manipulated with infinite precision. By repeatedly adding one floating point number to another, the inaccuracies accumulate. This is why you're seeing values higher than 1.0.
The solution is to iterate using an integer type and compute the desired floating point numbers. For example:
for (unsigned i = 0U; i < 10U; ++i)
{
float x = 0.1F * static_cast<float>(i);
printf("%f\n", x);
}
In addition to what #Yun (the round-off error of floating point numbers) indicates, you must also pay attention to the sampling point (which must be at the pixel center).
Here your code, with some modification and addition:
#include <iostream>
#include <vector>
#include <iomanip>
#include <math.h>
float rad, radSquared, center;
const int filterSize = 8;
const float invFilterSize = 1.0f / filterSize;
// Sample the circle returning 1 when inside, 0 otherwise.
int SampleCircle(int i, int j) {
float di = (i + 0.5f) * invFilterSize - center;
float dj = (j + 0.5f) * invFilterSize - center;
return ((di * di + dj * dj) < radSquared) ? 1 : 0;
}
// NOTE: This sampling method works with any filter size.
float PixelAA(int I, int J)
{
int aaValue = 0;
for (int i = 0; i < filterSize; ++i)
for (int j = 0; j < filterSize; ++j)
aaValue += SampleCircle(I + i, J + j);
return (float)aaValue / (float)(filterSize * filterSize);
}
// NOTE: This sampling method works only with filter sizes that are power of two.
float PixelAAQuadTree(int i, int j, int filterSize)
{
if (filterSize == 1)
return (float)SampleCircle(i, j);
// We sample the four corners of the filter. Note that on left and bottom corners
// 1 is subtracted to avoid sampling overlap.
int topLeft = SampleCircle(i, j);
int topRight = SampleCircle(i + filterSize - 1, j);
int bottomLeft = SampleCircle(i, j + filterSize - 1);
int bottomRight = SampleCircle(i + filterSize - 1, j + filterSize - 1);
// If all samples have same value we can stop here. All samples lies outside or inside the circle.
if (topLeft == topRight && topLeft == bottomLeft && topLeft == bottomRight)
return (float)topLeft;
// Half the filter dimension.
filterSize /= 2;
// Recurse.
return (PixelAAQuadTree(i, j, filterSize) +
PixelAAQuadTree(i + filterSize, j, filterSize) +
PixelAAQuadTree(i, j + filterSize, filterSize) +
PixelAAQuadTree(i + filterSize, j + filterSize, filterSize)) / 4.0f;
}
void CircleConst(float Rad, bool useQuadTree)
{
rad = Rad;
radSquared = rad * rad;
center = Rad;
int dataSize = (int)ceil(rad * 2);
std::vector<float> arrData;
arrData.reserve(dataSize * dataSize);
if (useQuadTree)
{
for (int i = 0; i < dataSize; i++)
for (int j = 0; j < dataSize; j++)
arrData.push_back(PixelAAQuadTree(i * filterSize, j * filterSize, filterSize));
}
else
{
for (int i = 0; i < dataSize; i++)
for (int j = 0; j < dataSize; j++)
arrData.push_back(PixelAA(i * filterSize, j * filterSize));
}
for (int i = 0; i < dataSize; i++)
{
for (int j = 0; j < dataSize; j++)
std::cout << std::fixed << std::setw(2) << std::setprecision(2)
<< std::setfill('0') << arrData[i + j * dataSize] << " ";
std::cout << std::endl;
}
}
int main() {
CircleConst(3.5f, false);
std::cout << std::endl;
CircleConst(4.0f, false);
std::cout << std::endl;
std::cout << std::endl;
CircleConst(3.5f, true);
std::cout << std::endl;
CircleConst(4.0f, true);
return 0;
}
Which gives these results (the second ones with use of quad-tree to optimize number of samples required to compute the AA value):
0.00 0.36 0.84 1.00 0.84 0.36 0.00
0.36 1.00 1.00 1.00 1.00 1.00 0.36
0.84 1.00 1.00 1.00 1.00 1.00 0.84
1.00 1.00 1.00 1.00 1.00 1.00 1.00
0.84 1.00 1.00 1.00 1.00 1.00 0.84
0.36 1.00 1.00 1.00 1.00 1.00 0.36
0.00 0.36 0.84 1.00 0.84 0.36 0.00
0.00 0.16 0.70 0.97 0.97 0.70 0.16 0.00
0.16 0.95 1.00 1.00 1.00 1.00 0.95 0.16
0.70 1.00 1.00 1.00 1.00 1.00 1.00 0.70
0.97 1.00 1.00 1.00 1.00 1.00 1.00 0.97
0.97 1.00 1.00 1.00 1.00 1.00 1.00 0.97
0.70 1.00 1.00 1.00 1.00 1.00 1.00 0.70
0.16 0.95 1.00 1.00 1.00 1.00 0.95 0.16
0.00 0.16 0.70 0.97 0.97 0.70 0.16 0.00
0.00 0.36 0.84 1.00 0.84 0.36 0.00
0.36 1.00 1.00 1.00 1.00 1.00 0.36
0.84 1.00 1.00 1.00 1.00 1.00 0.84
1.00 1.00 1.00 1.00 1.00 1.00 1.00
0.84 1.00 1.00 1.00 1.00 1.00 0.84
0.36 1.00 1.00 1.00 1.00 1.00 0.36
0.00 0.36 0.84 1.00 0.84 0.36 0.00
0.00 0.16 0.70 0.97 0.97 0.70 0.16 0.00
0.16 0.95 1.00 1.00 1.00 1.00 0.95 0.16
0.70 1.00 1.00 1.00 1.00 1.00 1.00 0.70
0.97 1.00 1.00 1.00 1.00 1.00 1.00 0.97
0.97 1.00 1.00 1.00 1.00 1.00 1.00 0.97
0.70 1.00 1.00 1.00 1.00 1.00 1.00 0.70
0.16 0.95 1.00 1.00 1.00 1.00 0.95 0.16
0.00 0.16 0.70 0.97 0.97 0.70 0.16 0.00
As further notes:
you can see how quad-trees work here https://en.wikipedia.org/wiki/Quadtree
you can further modify the code and implement fixed-point math (https://en.wikipedia.org/wiki/Fixed-point_arithmetic) which has no round-off problems like floats because numbers are always represented as integers
given that these data are part of a pre-calculation phase, privilege the simplicity of the code over performance

C++ double wierd behaviour during incrementing in for loop

I was making a progress bar when I noticed this, Something wierd is happening when incrementing doubles in for loop.
EDIT : I am not asking why floating points are broken !! My issue is that the for loop is iterating one extra time in case 1 and one less time in case 4 !
I have created 4 test cases. Please See the code below :-
#include <iostream>
using namespace std;
int main()
{
cout << endl << "Should End on 0.9 :-" << endl; // Abnormal
for (double i = 0 ; i < 1 ; i += 0.1) cout << i << endl;
cout << endl << "Should End on 1 :-" << endl;
for (double i = 0 ; i <= 1 ; i += 0.1) cout << i << endl;
cout << endl << "Should End on 0.99 :-" << endl;
for (double i = 0 ; i < 1 ; i += 0.01) cout << i << endl;
cout << endl << "Should End on 1 :-" << endl; // Abnormal
for (double i = 0 ; i <= 1 ; i += 0.01) cout << i << endl;
}
Output :-
Should End on 0.9 :-
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Should End on 1 :-
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Should End on 0.99 :-
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
0.51
0.52
0.53
0.54
0.55
0.56
0.57
0.58
0.59
0.6
0.61
0.62
0.63
0.64
0.65
0.66
0.67
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.8
0.81
0.82
0.83
0.84
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
Should End on 1 :-
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.21
0.22
0.23
0.24
0.25
0.26
0.27
0.28
0.29
0.3
0.31
0.32
0.33
0.34
0.35
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
0.51
0.52
0.53
0.54
0.55
0.56
0.57
0.58
0.59
0.6
0.61
0.62
0.63
0.64
0.65
0.66
0.67
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.8
0.81
0.82
0.83
0.84
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
[Program finished]
First case should end on 0.9 but it's ending on 1
And 4th case should end on 1 but it's ending on 0.99
2nd and 3rd case are working normally.
Any ideas what's going on here ?
What am I doing wrong ?
I am using Clang compiler on aarch64 cpu.
The answer is that floating points are "broken" - or at least not trusted for exact values.
In loop 1, when it unexpectedly prints 1.0, the actual value is slightly less (0.999...), so the check that it is less than 1 is still true.
In loop 4, when you expect it to print 1.0, the actual value is slightly higher (1.0001 or something), so the check that it is <= 1 fails.

How do I plot data in a text file depending on the the value present in one of the columns

I have a text file with with a header and a few columns, which represents results of experiments where some parameters were fixed to obtain some metrics. the file is he following format :
A B C D E
0 0.5 0.2 0.25 0.75 1.25
1 0.5 0.3 0.12 0.41 1.40
2 0.5 0.4 0.85 0.15 1.55
3 1.0 0.2 0.11 0.15 1.25
4 1.0 0.3 0.10 0.11 1.40
5 1.0 0.4 0.87 0.14 1.25
6 2.0 0.2 0.23 0.45 1.55
7 2.0 0.3 0.74 0.85 1.25
8 2.0 0.4 0.55 0.55 1.40
So I want to plot x = B, y = C for each fixed value of And E so basically for an E=1.25 I want a series of line plots of x = B, y = C at each value of A then a plot for each unique value of E.
Anyone could help with this?
You could do a combination of groupby() and seaborn.lineplot():
for e,d in df.groupby('E'):
fig, ax = plt.subplots()
sns.lineplot(data=d, x='B', y='C', hue='A', ax=ax)
ax.set_title(e)

How do I refer to the current line/observation number in a loop in the data step?

For example, I have data on various latencies that I iterate through and apply a linear function to, as shown: (the function is just an example here)
data latency;
input lat1 - lat20;
array cost[20];
array lat[20];
do x = 1 to 20;
cost[x] = lat[x] * 1.875;
end;
drop x;
datalines;
0.42 0.85 0.59 0.06 0.21 0.35 0.1 0.08 0.85 0.53 0.81 0.44 0.47 0.2 0.99 0.32 0.18 0.87 0.33 0.84
0.11 0.83 0.02 0.59 0.74 0.65 0.76 0.45 0.57 0.22 0.2 0.13 0.42 0.15 0.05 0.51 0.48 0.95 0.39 0.92
0.8 0.9 0.65 0.29 0.77 0.0 0.24 0.05 0.16 0.72 0.58 0.9 0.35 0.63 0.79 0.41 0.73 0.36 0.82 0.16
0.74 0.21 0.57 0.73 0.83 0.78 0.77 0.92 0.13 0.39 0.52 0.14 0.1 0.77 0.68 0.99 0.26 0.37 0.97 0.83
;
run;
How can I save a variable with the current observation number in each iteration of the loop, so that I can use it in calculations later?
I know that proc print will automatically print the observation number, but how do I access this and store it to a variable in the data step? Is there a way to do this as sas reads the datalines line by line?
I tried this, but then the obs variable is 2 for every observation.
data latency;
input lat1 - lat20;
obs = 1; * ADDED LINE;
array cost[20];
array lat[20];
do x = 1 to 20;
cost[x] = lat[x] * 1.875;
end;
obs = obs + 1; * ADDED LINE;
drop x;
datalines;
0.42 0.85 0.59 0.06 0.21 0.35 0.1 0.08 0.85 0.53 0.81 0.44 0.47 0.2 0.99 0.32 0.18 0.87 0.33 0.84
0.11 0.83 0.02 0.59 0.74 0.65 0.76 0.45 0.57 0.22 0.2 0.13 0.42 0.15 0.05 0.51 0.48 0.95 0.39 0.92
0.8 0.9 0.65 0.29 0.77 0.0 0.24 0.05 0.16 0.72 0.58 0.9 0.35 0.63 0.79 0.41 0.73 0.36 0.82 0.16
0.74 0.21 0.57 0.73 0.83 0.78 0.77 0.92 0.13 0.39 0.52 0.14 0.1 0.77 0.68 0.99 0.26 0.37 0.97 0.83
;
run;
proc print data=latency;
run;
This is a small example, but in reality I can't simply add a new variable that stores the line number to the start of each data line and read it in. That isn't practical for the actual data set.
You just need to add a retain statement so SAS doesn't reset obs to 0 at every new observation.
data latency;
retain obs 0;
obs = obs + 1;
...
run;
Your first attempt was very close. Try again, but this time replace this line:
obs = 1; * ADDED LINE;
With this:
retain obs 0; * ADDED LINE;
That way, your obs variable will be retained across your entire dataset instead of being reset to 1 each time.

Strange profiler behavior: same functions, different performances

I was learning to use gprof and then i got weird results for this code:
int one(int a, int b)
{
int i, r = 0;
for (i = 0; i < 1000; i++)
{
r += b / (a + 1);
}
return r;
}
int two(int a, int b)
{
int i, r = 0;
for (i = 0; i < 1000; i++)
{
r += b / (a + 1);
}
return r;
}
int main()
{
for (int i = 1; i < 50000; i++)
{
one(i, i * 2);
two(i, i * 2);
}
return 0;
}
and this is the profiler output
% cumulative self self total
time seconds seconds calls us/call us/call name
50.67 1.14 1.14 49999 22.80 22.80 two(int, int)
49.33 2.25 1.11 49999 22.20 22.20 one(int, int)
If i call one then two the result is the inverse, two takes more time than one
both are the same functions, but the first calls always take less time then the second
Why is that?
Note: The assembly code is exactly the same and code is being compiled with no optimizations
I'd guess it is some fluke in run-time optimisation - one uses a register and the other doesn't or something minor like that.
The system clock probably runs to a precision of 100nsec. The average call time 30nsec or 25nsec is less than one clock tick. A rounding error of 5% of a clock tick is pretty small. Both times are near enough zero.
My guess: it is an artifact of the way mcount data gets interpreted. The granularity for mcount (monitor.h) is on the order of a 32 bit longword - 4 bytes on my system. So you would not expect this: I get different reports from prof vs gprof on the EXACT same mon.out file.
solaris 9 -
prof
%Time Seconds Cumsecs #Calls msec/call Name
46.4 2.35 2.3559999998 0.0000 .div
34.8 1.76 4.11120000025 0.0000 _mcount
10.1 0.51 4.62 1 510. main
5.3 0.27 4.8929999999 0.0000 one
3.4 0.17 5.0629999999 0.0000 two
0.0 0.00 5.06 1 0. _fpsetsticky
0.0 0.00 5.06 1 0. _exithandle
0.0 0.00 5.06 1 0. _profil
0.0 0.00 5.06 20 0.0 _private_exit, _exit
0.0 0.00 5.06 1 0. exit
0.0 0.00 5.06 4 0. atexit
gprof
% cumulative self self total
time seconds seconds calls ms/call ms/call name
71.4 0.90 0.90 1 900.00 900.00 key_2_text <cycle 3> [2]
5.6 0.97 0.07 106889 0.00 0.00 _findbuf [9]
4.8 1.03 0.06 209587 0.00 0.00 _findiop [11]
4.0 1.08 0.05 __do_global_dtors_aux [12]
2.4 1.11 0.03 mem_init [13]
1.6 1.13 0.02 102678 0.00 0.00 _doprnt [3]
1.6 1.15 0.02 one [14]
1.6 1.17 0.02 two [15]
0.8 1.18 0.01 414943 0.00 0.00 realloc <cycle 3> [16]
0.8 1.19 0.01 102680 0.00 0.00 _textdomain_u <cycle 3> [21]
0.8 1.20 0.01 102677 0.00 0.00 get_mem [17]
0.8 1.21 0.01 $1 [18]
0.8 1.22 0.01 $2 [19]
0.8 1.23 0.01 _alloc_profil_buf [22]
0.8 1.24 0.01 _mcount (675)
Is it always the first one called that is slightly slower? If that's the case, I would guess it is a CPU cache doing it's thing. or it could be lazy paging by the operating system.
BTW: what optimization flags are compiling with?