According to the description of the nanstd and nanmean, and other nan functions in numpy, the nan shall be ignored. However, I got nan as a result of computing standard deviation and mean with nan functions.
Below, I compute the mean and std of a whole array and it's subset. The nan values are at the very end hence are not included if I do the calculations on the subset of it.
>>> nanstd(arr[1,:1000])
0.83957712570078991
>>> nanstd(arr[1,:])
nan
>>> std(arr[1,:])
nan
numpy.__version__
'1.12.0'
According to this answer, there are few reasons why nanstd might return nan.
Here are
If the input is empty (Not the case)
If all of the elements in the input are NaN (Not the case)
one of the elements is either positive or negative infinity. (There is an inf value in my array)
Solved
The solution is here
As mentioned as an answer to a different question , there are few reasons why nanstd might return nan.
Here are
If the input is empty (Not the case)
If all of the elements in the input are NaN (Not the case)
one of the elements is either positive or negative infinity. (There is an inf value in my array)
Going through the list I found out that I do have inf value in my array
Related
Is there a way to combine NaN and ordinary numbers in a different way then usually done in Fortran?
I have several summations which contains 'safe' terms, which cannot be NaN, and some other terms which can be NaN.
I would like the evaluation of the expression to neglect the addends in case they are NaN.
I cannot just get rid of them multiplying them times a null factor when they are NaN as NaN x 0 gives NaN anyway.
Ideas?
Thanks
There is no arithmetic operation that does not propagate NaN. So ideas like multiplying by 0 will not work.
Your only solution is to miss out the NaN terms in the sum. Do that with something based on
IF (IEEE_IS_NAN(x))
If you are not using IEEE754 or are using an older standard of FORTRAN, then you can use
IF(x .NE. x)
which will be TRUE if and only if x is NaN.
A NAN value means Not A Number and IND value means Indeterminate number. But what is the difference between these two. How can we represent both in c++.
But what is the difference between these two.
They are both the same thing. Some platforms choose to display a non-number as some variant of NaN, while others choose to display it as some variant of IND.
How can we represent both in c++.
std::numeric_limits<double>::quiet_NaN() (or float or long double, if you prefer).
If your operation would generate a larger positive number than could be stored in a double, the operation will return 1.#INF on Windows or inf on Linux
Some operations don't make mathematical sense, such as taking the square root of a negative number.Both sqrt(-1.0) and log(-1.0) would return a NaN, the generic term for a "number" that is "not a number".
Windows displays a NaN as -1.#IND ("IND" for "indeterminate") while Linux displays nan. Other operations that would return a NaN include 0/0, 0*∞, and ∞/∞.
Refernce Link1
Refernce Link2
I want to know what makes a float number nan in c++. I am using a large dataset and it is really hard to trace. I want to know the ways of changing a float number to nan to reduce bug possibilities.
I found the code that causes the nan problem. I found that s/m is nan in some cases. But I don't know how to solve it.
float gp(float x){
float e = 2.71828183;
x *= -1;
float s = pow(e,x);
float m = (1 + pow(e,x)) * (1 + pow(e,x));
return s / m;}
Taken from wikipedia -> special values -> nan
0/0
∞×0
sqrt(−1)
in general "invalid operations" (I am not sure wether there are not more than the three above)
Looking at you code: infinity times 0 is possible, is it?
edit:
0 <= s <= +inf
1 <= m <= +inf
s / m:
+inf / +inf does indeed make minus NaN (I tested it)
I think that's the only thing that makes a NaN.
If you can keep x between 0 and FLT_MAX (3.40E+38 in my case), your gp function will not return NaN.
You say in a comment that you only use *, +, -.
[Edit: you've since said that you also use pow and division, which introduce some extra ways to get NaN. For example if the parameter x is a large negative value then pow(e,-x) is infinity, so you can easily end up computing infinity/infinity, which is another NaN]
So, if you have IEEE floating-point then assuming this summary is correct, the only ways you can generate NaN are:
Generate a positive or negative infinity by going out of range,
Multiply it by zero.
or:
Generate a positive and a negative infinity,
Add them (or equivalently, subtract two infinities of the same sign).
So if you check for and catch infinities, you don't have to worry about NaNs as well. That said, the usual way is to let such values propagate as quiet NaNs, and check at the end.
For C++ implementations using non-IEEE arithmetic, I'm not sure what the rules are when a NaN is permitted. I could look them up in the standard, but then again so could you ;-)
sqrt(-1)
give you NaN, for example.
http://www.gnu.org/s/libc/manual/html_node/Infinity-and-NaN.html
EDIT Try use double instead of float.
Probably it depends to compiler you using but general option is:
variable is too small
variable is too big
divide by 0 (zero)
This is exactly the use case for enabling and trapping floating-point exceptions. That way you can detect exactly where the NaN (or other exception value) first appears.
However, that's a platform-dependant feature, so you may have to look into the documentation of your compiler and/or hardware.
I'm messing around with some C code using floats, and I'm getting 1.#INF00, -1.#IND00 and -1.#IND when I try to print floats in the screen. What does those values mean?
I believe that 1.#INF00 means positive infinity, but what about -1.#IND00 and -1.#IND? I also saw sometimes this value: 1.$NaN which is Not a Number, but what causes those strange values and how can those help me with debugging?
I'm using MinGW which I believe uses IEEE 754 representation for float point numbers.
Can someone list all those invalid values and what they mean?
From IEEE floating-point exceptions in C++ :
This page will answer the following questions.
My program just printed out 1.#IND or 1.#INF (on Windows) or nan or inf (on Linux). What happened?
How can I tell if a number is really a number and not a NaN or an infinity?
How can I find out more details at runtime about kinds of NaNs and infinities?
Do you have any sample code to show how this works?
Where can I learn more?
These questions have to do with floating point exceptions. If you get some strange non-numeric output where you're expecting a number, you've either exceeded the finite limits of floating point arithmetic or you've asked for some result that is undefined. To keep things simple, I'll stick to working with the double floating point type. Similar remarks hold for float types.
Debugging 1.#IND, 1.#INF, nan, and inf
If your operation would generate a larger positive number than could be stored in a double, the operation will return 1.#INF on Windows or inf on Linux. Similarly your code will return -1.#INF or -inf if the result would be a negative number too large to store in a double. Dividing a positive number by zero produces a positive infinity and dividing a negative number by zero produces a negative infinity. Example code at the end of this page will demonstrate some operations that produce infinities.
Some operations don't make mathematical sense, such as taking the square root of a negative number. (Yes, this operation makes sense in the context of complex numbers, but a double represents a real number and so there is no double to represent the result.) The same is true for logarithms of negative numbers. Both sqrt(-1.0) and log(-1.0) would return a NaN, the generic term for a "number" that is "not a number". Windows displays a NaN as -1.#IND ("IND" for "indeterminate") while Linux displays nan. Other operations that would return a NaN include 0/0, 0*∞, and ∞/∞. See the sample code below for examples.
In short, if you get 1.#INF or inf, look for overflow or division by zero. If you get 1.#IND or nan, look for illegal operations. Maybe you simply have a bug. If it's more subtle and you have something that is difficult to compute, see Avoiding Overflow, Underflow, and Loss of Precision. That article gives tricks for computing results that have intermediate steps overflow if computed directly.
For anyone wondering about the difference between -1.#IND00 and -1.#IND (which the question specifically asked, and none of the answers address):
-1.#IND00
This specifically means a non-zero number divided by zero, e.g. 3.14 / 0 (source)
-1.#IND (a synonym for NaN)
This means one of four things (see wiki from source):
1) sqrt or log of a negative number
2) operations where both variables are 0 or infinity, e.g. 0 / 0
3) operations where at least one variable is already NaN, e.g. NaN * 5
4) out of range trig, e.g. arcsin(2)
Your question "what are they" is already answered above.
As far as debugging (your second question) though, and in developing libraries where you want to check for special input values, you may find the following functions useful in Windows C++:
_isnan(), _isfinite(), and _fpclass()
On Linux/Unix you should find isnan(), isfinite(), isnormal(), isinf(), fpclassify() useful (and you may need to link with libm by using the compiler flag -lm).
For those of you in a .NET environment the following can be a handy way to filter non-numbers out (this example is in VB.NET, but it's probably similar in C#):
If Double.IsNaN(MyVariableName) Then
MyVariableName = 0 ' Or whatever you want to do here to "correct" the situation
End If
If you try to use a variable that has a NaN value you will get the following error:
Value was either too large or too small for a Decimal.
I'm messing around with some C code using floats, and I'm getting 1.#INF00, -1.#IND00 and -1.#IND when I try to print floats in the screen. What does those values mean?
I believe that 1.#INF00 means positive infinity, but what about -1.#IND00 and -1.#IND? I also saw sometimes this value: 1.$NaN which is Not a Number, but what causes those strange values and how can those help me with debugging?
I'm using MinGW which I believe uses IEEE 754 representation for float point numbers.
Can someone list all those invalid values and what they mean?
From IEEE floating-point exceptions in C++ :
This page will answer the following questions.
My program just printed out 1.#IND or 1.#INF (on Windows) or nan or inf (on Linux). What happened?
How can I tell if a number is really a number and not a NaN or an infinity?
How can I find out more details at runtime about kinds of NaNs and infinities?
Do you have any sample code to show how this works?
Where can I learn more?
These questions have to do with floating point exceptions. If you get some strange non-numeric output where you're expecting a number, you've either exceeded the finite limits of floating point arithmetic or you've asked for some result that is undefined. To keep things simple, I'll stick to working with the double floating point type. Similar remarks hold for float types.
Debugging 1.#IND, 1.#INF, nan, and inf
If your operation would generate a larger positive number than could be stored in a double, the operation will return 1.#INF on Windows or inf on Linux. Similarly your code will return -1.#INF or -inf if the result would be a negative number too large to store in a double. Dividing a positive number by zero produces a positive infinity and dividing a negative number by zero produces a negative infinity. Example code at the end of this page will demonstrate some operations that produce infinities.
Some operations don't make mathematical sense, such as taking the square root of a negative number. (Yes, this operation makes sense in the context of complex numbers, but a double represents a real number and so there is no double to represent the result.) The same is true for logarithms of negative numbers. Both sqrt(-1.0) and log(-1.0) would return a NaN, the generic term for a "number" that is "not a number". Windows displays a NaN as -1.#IND ("IND" for "indeterminate") while Linux displays nan. Other operations that would return a NaN include 0/0, 0*∞, and ∞/∞. See the sample code below for examples.
In short, if you get 1.#INF or inf, look for overflow or division by zero. If you get 1.#IND or nan, look for illegal operations. Maybe you simply have a bug. If it's more subtle and you have something that is difficult to compute, see Avoiding Overflow, Underflow, and Loss of Precision. That article gives tricks for computing results that have intermediate steps overflow if computed directly.
For anyone wondering about the difference between -1.#IND00 and -1.#IND (which the question specifically asked, and none of the answers address):
-1.#IND00
This specifically means a non-zero number divided by zero, e.g. 3.14 / 0 (source)
-1.#IND (a synonym for NaN)
This means one of four things (see wiki from source):
1) sqrt or log of a negative number
2) operations where both variables are 0 or infinity, e.g. 0 / 0
3) operations where at least one variable is already NaN, e.g. NaN * 5
4) out of range trig, e.g. arcsin(2)
Your question "what are they" is already answered above.
As far as debugging (your second question) though, and in developing libraries where you want to check for special input values, you may find the following functions useful in Windows C++:
_isnan(), _isfinite(), and _fpclass()
On Linux/Unix you should find isnan(), isfinite(), isnormal(), isinf(), fpclassify() useful (and you may need to link with libm by using the compiler flag -lm).
For those of you in a .NET environment the following can be a handy way to filter non-numbers out (this example is in VB.NET, but it's probably similar in C#):
If Double.IsNaN(MyVariableName) Then
MyVariableName = 0 ' Or whatever you want to do here to "correct" the situation
End If
If you try to use a variable that has a NaN value you will get the following error:
Value was either too large or too small for a Decimal.