How do you increase the floating point precision representation of tensorflow variables? I need this cause my network is large and data is complex, so I want to see any amount of improvement no matter how small. When I iterate through the training I occasionally print the mean error to the screen, and all I see is the same 6 digits - it works fine with less complex inputs. Note that tensorboard seems to have similar precision, I would be happy enough with a more precise tensorboard graph.
msquaredError=m_sqerror.eval(sessions=sess,feed_dict={input:ip, output=op,keep_prob:1.0})
print ("MSE: %9f"%msquaredError)
output:
MSE: 0.317513
desired output:
MSE: 0.317513223 ... and many more digits
Stoopid error.
print ("MSE: %9f"%msquaredError)
should be
print ("MSE: %.9f"%msquaredError)
Related
I can't believe I have never had this issue before (nor can I find anyone else with the same issue) but today I have just discovered that SAS sometimes gets simple calculations wrong!?! I noticed that one of my records wasn't getting picked up in the right group based on a value being <3.6 and thought there must be something strange in my data with decimal places. But on investigation I found it was just because SAS was calculating the value wrong! For some reason that I can't fathom, it seems that SAS calculates 90 - 86.4 as 3.59999999999999!!! Simple program below to show this:
code
output
If I alter the calculation to 10 - 6.4 I get the correct value of 3.6000 but for some reason this one is coming out wrong. Could there be some mad setting that is wrong in my installation? I tried both SAS EG and Base SAS and both have the same issue. I feel like I'm going mad! Any help appreciated.
Thanks.
Floating point arithmetic, in any language, will have this same issue. The same issue is possible to understand in human terms, assuming the human doesn't have a concept of infinite. If you only write down 4 digits for your decimals, for example, then:
1 - (1/3) - (1/3) - (1/3)
That's zero, right?
1 - 0.3333 = 0.6667
0.6667 - 0.3333 = 0.3334
0.3334 - 0.3333 = 0.0001
Nope! Computers do the same thing, but in binary, so they have a different (and larger) set of "problem" numbers. 1/10, for example, is not representable in binary - so adding or subtracting 0.1 is not always a "neat" operation like it is in decimal.
SAS uses 8 byte floating points, and so it gets ~15 digits of accuracy. Assuming you're not working in a field where 15 digits of accuracy is needed, you should simply round.
if round(value,.01) ge 3.6 then ... ;
Most of the time this isn't needed, but strictly speaking you should always compare rounded numbers whenever using floating point numbers (as SAS does). Integers are safe, but if you're working with 0.1 etc., use ROUND or FUZZ for integers.
Sorry Cannot replicate your findings.
data x;
a=90-86.4;
run;
Gives the correct result. Are you using any formats or put function. Share the complete code.
In Amazon Athena, I want to round a big number to output me as decimal precision 2.
For example, I have 1.4309491454947177E11 which is equivalent to 143094914549.47177, so I expect result to be 143094914549.47
I am doing
SELECT ROUND(1.4309491454947177E11, 2)
But it is giving me wrong output 1.43.
Any help would be much appreciated!!!
The output is correct it is converting it to 2 decimals and you can use it in any downstream. It is just displayed as exponential. Similar to the way you see in excel and when you click on excel cell it displays actual value.
If you want to see data in 2 digit format.Cast will work make sure no of digits in cast is more than actual value
select cast(Round(1.4309491454947177E11 ,2) as decimal(20,2))
I have a neural network written in standard C++11 which I believe follows the back-propagation algorithm correctly (based on this). If I output the error in each step of the algorithm, however, it seems to oscillate without dampening over time. I've tried removing momentum entirely and choosing a very small learning rate (0.02), but it still oscillates at roughly the same amplitude per network (with each network having a different amplitude within a certain range).
Further, all inputs result in the same output (a problem I found posted here before, although for a different language. The author also mentions that he never got it working.)
The code can be found here.
To summarize how I have implemented the network:
Neurons hold the current weights to the neurons ahead of them, previous changes to those weights, and the sum of all inputs.
Neurons can have their value (sum of all inputs) accessed, or can output the result of passing said value through a given activation function.
NeuronLayers act as Neuron containers and set up the actual connections to the next layer.
NeuronLayers can send the actual outputs to the next layer (instead of pulling from the previous).
FFNeuralNetworks act as containers for NeuronLayers and manage forward-propagation, error calculation, and back-propagation. They can also simply process inputs.
The input layer of an FFNeuralNetwork sends its weighted values (value * weight) to the next layer. Each neuron in each layer afterwards outputs the weighted result of the activation function unless it is a bias, or the layer is the output layer (biases output the weighted value, the output layer simply passes the sum through the activation function).
Have I made a fundamental mistake in the implementation (a misunderstanding of the theory), or is there some simple bug I haven't found yet? If it would be a bug, where might it be?
Why might the error oscillate by the amount it does (around +-(0.2 +- learning rate)) even with a very low learning rate? Why might all the outputs be the same, no matter the input?
I've gone over most of it so much that I might be skipping over something, but I think I may have a plain misunderstanding of the theory.
It turns out I was just staring at the FFNeuralNetwork parts too much and accidentally used the wrong input set to confirm the correctness of the network. It actually does work correctly with the right learning rate, momentum, and number of iterations.
Specifically, in main, I was using inputs instead of a smaller array in to test the outputs of the network.
I am setting floats and visual studios while debugging always just shows me this big notation like.
-4.3176166e+008
-4.3160208e+008
rather than
-0.605
-0.789
Is there any way I can just see my values in decimal rather than that other notation.
Because -4.3176166e+008 != -0.605. If your value was -0.605, it would be displayed like that, but for values with a large number of significant figures, it is easier to read in scientific notation - you don't have to count the digits, and you can compare maginitude quickly and easily by comparing the exponent.
You examples in standard notation are:
-431761660 and -431602080
Not so bad maybe, but for very small or very large numbers, it is beneficial, and the line has to be drawn somewhere. For example 1.0e-20 is much easier to comprehend than 0.00000000000000000001 (and if I have got that wrong, it rather proves my point).
It is not peculiar to Visual Studio; your pocket scientific calculator will do the same thing - it has to as it has a limited width display. So Visual Studio's behaviour is also useful for when you are comparing results in your code with values calculated by hand on a calculator.
Another reason for representing numbers in this way is that binary floating point representations have precision limited to about 6 decimal digits for float and 15 for double, to display a greater number of significant digits would be misleading.
If the issue is that the Visual Studio debugger shows numbers in a certain way, then there is the autoexp.dat file that Visual Studio uses to determine how to display values in the debugger. Changing the definitions in this file will change how variables are displayed.
Here is a tutorial on using autoexp.dat:
http://www.idigitalhouse.com/Blog/?p=83
These links describes the new Visualizer, but mentions autoexp.dat:
http://blogs.msdn.com/b/vcblog/archive/2012/07/12/10329460.aspx
Tutorial/Manual how to create Visualizer in autoexp.dat
If you prefer not to deal with scientific notation mentally, and don't want to change the Visual Studio settings, use a spreadsheet.
Paste the number into a cell, and set the cell format to taste. For example, pasting -4.3176166e+008 into a LibreOffice Calc cell and setting the cell format to "Number" shows -431761660. This has the advantage that you can easily change the format as you are viewing numbers, and display numbers in the same format even if they came from different contexts in different original formats.
A spreadsheet is often useful anyway when debugging numerical programs, for doing quick calculations and keeping the results visible.
I have recently implemented a typical 3 layer neural network (input -> hidden -> output) and I'm using the sigmoid function for activation. So far, the host program has 3 modes:
Creation, which seems to work fine. It creates a network with a specified number of input, hidden and output neurons, initializes the weights to either random values or zero.
Training, which loads a dataset, computes the output of the network then backpropagates the error and updates the weights. As far as I can tell, this works ok. The weights change, but not extremely, after training on the dataset.
Processing, which seems to work ok. However, the data output for the dataset which was used for training, or any other dataset for that matter is very bad. It's usually either just a continuuous stream of 1's, with an occasional 0.999999 or every output value for every input is 0.9999 with the last digits being different between inputs. As far as I could tell there was no correlation between those last 2 digits and what was supposed to be outputed.
How should I go about figuring out what's not working right?
You need to find a set of parameters (number of neurons, learning rate, number of iterations for training) that works well for classifying previously unseen data. People often achieve this by separating their data into three groups: training, validation and testing.
Whatever you decide to do, just remember that it really doesn't make sense to be testing on the same data with which you trained, because any classifcation method close to reasonable should be getting everything 100% right under such a setup.