I'm wondering where I can find the standard output (if there is one) when comparing (LT & GT) 2 special single precision IEEE-754 floating point values, being combinations of -inf/+inf/NaN/-0/+0.
I wrote a test program and it gave me following output, but how to check if it is compliant?:
The relevant portion of the IEEE 754-2008 standard, and a late 754-2019 draft, is:
Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself. Comparisons shall ignore the sign of zero (so +0 = −0). Infinite operands of the same sign shall compare equal.
From this, we see that none of the comparisons with a NaN would yield less than or greater than, and therefore any test of less than or greater than should yield false, so the results in the table conform in this regard.
Similarly, the comparisons of an infinity to the same infinity and of either zero to either zero should yield false, so the table conforms in this regard too.
The standard does not explicitly detail comparisons of other values; they are inherited from ordinary arithmetic, and we can see the table conforms in this regard too.
See the links in the Standards section of the Wikipedia IEEE 754 page for sources for official versions of the standard.
The standard says:
nan comparisons are always evaluated as false (even nan == nan)
+inf > x is true if x is neither +inf nor nan, and false otherwise
-inf < x is true if x is neither -inf nor nan, and false otherwise
Related
I did a quick test using the following:
float x = std::numeric_limits<float>::max();
x += 0.1;
that resulted in x == std::numeric_limits::max() so it didn't get any bigger than the limit.
Is this guaranteed behavior across compilers and platforms though? What about HLSL?
Is this guaranteed behavior across compilers and platforms though?
No, the behavior is undefined. The standard says (emphasis mine):
5 Expressions....
If during the evaluation of an expression, the result is not mathematically defined or not in the range of
representable values for its type, the behavior is undefined. [ Note: most existing implementations of C++
ignore integer overflows. Treatment of division by zero, forming a remainder using a zero divisor, and all
floating point exceptions vary among machines, and is usually adjustable by a library function. —end note ]
As #user2079303 mentioned, in practice we can be less restricted:
it is not undefined if std::numeric_limits<float>::has_infinity. Which is often true. In that case, the result is merely unspecified.
The value of std::numeric_limits<T>::max() is defined to be the maximum finite value representable by type T (see 18.3.2.4 [numeric.limits.members] paragraph 4). Thus, the question actually becomes multiple subquestions:
Is it possible to create a value bigger than std::numeric_limits<T>::max(), i.e., is there an infinity?
If so, which value needs to be added to std::numeric_limits<T>::max() to get the infinity?
If not, is the behavior defined?
C++ does not specify the floating point format and different formats may disagree on what the result is. In particular, I don't think floating point formats need to define a value for infinity. For example, IBM Floating Points do not have an infinity. On the other hand the IEEE 754 does have an infinity representation.
Since overflow of arithmetic types may be undefined behavior (see 5 [expr] paragraph 4) and I don't see any exclusion for floating point types. Thus, the behavior would be undefined behavior if there is no infinity. At least, it can be tested whether a type does have an infinity (see 18.3.2.3 [numeric.limits] paragraph 35) in which case the operation can't overflow.
If there is an infinity I think adding any value to std::numeric_limits<T>::max() would get you infinity. However, determining whether that is, indeed, the case would require to dig through the respective floating point specification. I could imagine that IEEE 754 might ignore additions if the value is too small to be relevant as is the case for adding 0.1 to std::numeric_limits<T>::max(). I could also imagine that it decides that it always overflows to infinity.
For floating point values, is it guaranteed that a + b is the same as1 b + a?
I believe this is guaranteed in IEEE754, however the C++ standard does not specify that IEEE754 must be used. The only relevant text seems to be from [expr.add]#3:
The result of the binary + operator is the sum of the operands.
The mathematical operation "sum" is commutative. However, the mathematical operation "sum" is also associative, whereas floating point addition is definitely not associative. So, it seems to me that we cannot conclude that the commutativity of "sum" in mathematics means that this quote specifies commutativity in C++.
Footnote 1:
"Same" as in bitwise identical, like memcmp rather than ==, to distinguish +0 from -0. IEEE754 treats +0.0 == -0.0 as true, but also has specific rules for signed zero. +0 + -0 and -0 + +0 both produce +0 in IEEE754, same for addition of opposite-sign values with equal magnitude. An == that followed IEEE semantics would hide non-commutativity of signed-zero if that was the criterion.
Also, a+b == b+a is false with IEEE754 math if either input is NaN.
memcmp will say whether two NaNs have the same bit-pattern (including payload), although we can consider NaN propagation rules separately from commutativity of valid math operations.
It is not even required that a + b == a + b. One of the subexpressions may hold the result of the addition with more precision than the other one, for example when the use of multiple additions requires one of the subexpressions to be temporarily stored in memory, when the other subexpression can be kept in a register (with higher precision).
If a + b == a + b is not guaranteed, a + b == b + a cannot be guaranteed. If a + b does not have to return the same value each time, and the values are different, one of them necessarily will not be equal to one particular evaluation of b + a.
No, the C++ language generally wouldn't make such a requirement of the hardware. Only the associativity of operators is defined.
All kinds of crazy things do happen in floating-point arithmetic. Perhaps, on some machine, adding zero to an denormal number produces zero. Conceivable that a machine could avoid updating memory in the case of adding a zero-valued register to a denormal in memory. Possible that a really dumb compiler would always put the LHS in memory and the RHS in a register.
Note, though, that a machine with non-commutative addition would need to specifically define how expressions map to instructions, if you're going to have any control over which operation you get. Does the left-hand side go into the first machine operand or the second?
Such an ABI specification, mentioning the construction of expressions and instructions in the same breath, would be quite pathological.
The C++ standard very specifically does not guarantee IEEE 754. The library does have some support for IEC 559 (which is basically just the IEC's version of the IEEE 754 standard), so you can check whether the underlying implementation uses IEEE 754/IEC 559 though (and when it does, you can depend on what it guarantees, of course).
For the most part, the C and C++ standards assume that such basic operations will be implemented however the underlying hardware works. For something as common as IEEE 754, they'll let you detect whether it's present, but still don't require it.
Suppose I have two variables a and b, either both in type float, or both in type double, which hold some values. Do the following assertions always hold? I mean, do the existence of numerical errors alter the conclusions?
a > b is true if and only if a <= b is false
a < b is true if and only if a >= b is false
a >= b is necessarily true if a == b is true
a <= b is necessarily true if a == b is true
For the third and fourth, I mean for example, does "a == b is true" always give you "a >= b is true"?
EDIT:
Assume neither a or b is NaN or Inf.
EDIT 2:
After reading the IEEE 754 standard in the year 1985, I found the following.
First of all, it said the following
Comparisons are exact and never overflow nor underflow.
I understand it as when doing comparison, there is no consideration of numerical error, numbers are compared as-is. Since addition and subtraction such as a - b requires additional effort to define what the numerical error is, I assume the quote above is saying that comparisons like a > b is not done by judging whether a - b > 0 is true or not. Please let me know if I'm wrong.
Second, it listed the four canonical relations which are mutually exclusive.
Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself.
Then in Table 4, it defined the various kinds of operators such as ">" or ">=" in terms of the truth values under these four canonical relations. From the table we immediately have the following:
a >= b is true if and only if a < b is false
a <= b is true if and only if a > b is false
Both a >= b and a <= b are necessarily true if a == b is true
So the assertions in my questions can be concluded as true. However, I wasn't able to find anything in the standard defining whether symmetricity is true or not. In another way, from a > b I don't know if b < a is true or not. Thus I also have no way to derive a <= b is true from b < a is false. So I would be interested to know, in addition to the assertions in the OP, whether the following is always true or not
a < b is true if and only if b > a is true
a <= b is true if and only if b >= a is true
etc.
EDIT 3:
Regarding denormal numbers as mentioned by #Mark Ransom, I read the wikipedia page about it and my current understanding is that the existence of denormal numbers does not alter the conclusions above. In another way, if some hardware claims that it fully support denormal numbers, it also needs to ensure that the definitions of the comparison operators satisfy the above standards.
EDIT 4:
I just read the 2008 revision of IEEE 754, it doesn't say anything about symmetricity either. So I guess this is undefined.
(All the above discussion assumes no NaN or Inf in any of the operands).
If neither number is a NaN or infinity then your assertions hold, if you have an IEEE compatible system. If the standards don't mention it then I'm sure that is just because it was sufficiently obvious to not be worth mentioning. In particular, if "a < b" and "b > a" don't have the same value (even for NaNs and infinities) then we are in crazy town.
Denormals shouldn't affect the conclusions because if you are assuming IEEE compatibility then denormal support is a given.
The only risks I can think of are in cases involving the x87 FPU and it's odd 80-bit long double format. Rounding from 80-bit long double to double or float is expensive, so it is sometimes omitted, and that can lead to an assert like this firing:
assert(sqrt(val) == sqrt(val));
It could fire because the result of the first sqrt() call may be written to memory, and therefore rounded to float or double. It would then be compared against the result of the second sqrt() call, which might not be rounded. However, failing to round the result of the second sqrt() call before doing the comparison is, strictly speaking, not IEEE compliant if sqrt() returns float or double.
Infinity should only be a problem if both numbers are infinity.
I was always assuming that the following test will always succeed for finite values (no INF, no NAN) of somefloat:
assert(somefloat*0.0==0.0);
In Multiply by 0 optimization it was stated that double a=0.0 and double a=-0.0 are not strictly speaking the same thing.
So I was wondering whether this can lead to problems on some platforms e.g. can the result of the above test depend on a beeing positive or negative.
If your implementation uses IEEE 754 arithmetic (which most do), then positive and negative zero will compare equal. Since the left-hand side of your expression can only be either positive or negative zero for finite a, the assertion will always be true.
If it uses some other kind of arithmetic, then only the implementor, and hopefully the implementation-specific documentation, can tell you. Arguably (see comments) the wording of the standard can be taken to imply that they must compare equal in any case, and certainly no sane implementation would do otherwise.
-0.0 == 0.0 according to double comparison rules.
For non-finite values (+-Inf, Nan) somefloat*0.0 != 0.0.
Your assert can never fail, as long as somefloat is not
infinity or NaN. On systems which don't support infinity or
NaN, the compiler can simply optimize it out.
Regarding division by zero, the standards say:
C99 6.5.5p5 - The result of the / operator is the quotient from the division of the first operand by the second; the result of the % operator is the remainder. In both operations, if the value of the second operand is zero, the behavior is undefined.
C++03 5.6.4 - The binary / operator yields the quotient, and the binary % operator yields the remainder from the division of the first expression by the second. If the second operand of / or % is zero the behavior is undefined.
If we were to take the above paragraphs at face value, the answer is clearly Undefined Behavior for both languages. However, if we look further down in the C99 standard we see the following paragraph which appears to be contradictory(1):
C99 7.12p4 - The macro INFINITY expands to a constant expression of type float representing positive or unsigned infinity, if available;
Do the standards have some sort of golden rule where Undefined Behavior cannot be superseded by a (potentially) contradictory statement? Barring that, I don't think it's unreasonable to conclude that if your implementation defines the INFINITY macro, division by zero is defined to be such. However, if your implementation does not define such a macro, the behavior is Undefined.
I'm curious what the consensus is (if any) on this matter for each of the two languages. Would the answer change if we are talking about integer division int i = 1 / 0 versus floating point division float i = 1.0 / 0.0 ?
Note (1) The C++03 standard talks about the <cmath> library which includes the INFINITY macro.
I don't see any contradiction. Division by zero is undefined, period. There is no mention of "... unless INFINITY is defined" anywhere in the quoted text.
Note that nowhere in mathematics it is defined that 1 / 0 = infinity. One might interpret it that way, but it is a personal, "shortcut" style interpretation, rather than a sound fact.
1 / 0 is not infinity, only lim 1/x = ∞ (x -> +0)
This was not a math purest question, but a C/C++ question.
According to the IEEE 754 Standard, which all modern C compilers / FPU's use, we have
3.0 / 0.0 = INF
0.0 / 0.0 = NaN
-3.0 / 0.0 = -INF
The FPU will have a status flag that you can set to generate an exception if so desired, but this is not the norm.
INF can be quite useful to avoiding branching when INF is a useful result. See discussion here
http://people.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF
Why would it?
That doesn't make sense mathematically, it's not as if 1/x is defined as ∞ in mathematics in general. Also, you would at least need two more cases: -1/x and 0/x can't also equal ∞.
See division by zero in general, and the section about computer arithmetic in particular.
Implementations which define __STDC_IEC_559__ are required to abide by the requirements given in Annex F, which in turn requires floating-point semantics consistent with IEC 60559. The Standard imposes no requirements on the behavior of floating-point division by zero on implementations which do not define __STDC_IEC_559__, but does for those which do define it. In cases where IEC 60559 specifies a behavior but the C Standard does not, a compiler which defines __STDC_IEC_559__ is required by the C Standard to behave as described in the IEC standard.
As defined by IEC 60559 (or the US standard IEEE-754) Division of zero by zero yields NaN, division of a floating-point number by positive zero or literal constant zero yields an INF value with the same sign as the dividend, and division of a floating-point number by negative zero yields an INF with the opposite sign.
I've only got the C99 draft. In §7.12/4 it says:
The macro
INFINITY
expands to a constant expression of
type float representing positive or
unsigned infinity, if available; else
to a positive constant of type float
that overflows at translation time.
Note that INFINITY can be defined in terms of floating-point overflow, not necessarily divide-by-zero.
For the INFINITY macro: there is a explicit coding to represent +/- infinity in the IEEE754 standard, which is if all exponent bits are set and all fraction bits are cleared (if a fraction bit is set, it represents NaN)
With my compiler, (int) INFINITY == -2147483648, so an expression that evaluates to int i = 1/0 would definitely produce wrong results if INFINITIY was returned
Bottom line, C99 (as per your quotes) does not say anything about INFINITY in the context of "implementation-defined". Secondly, what you quoted does not show inconsistent meaning of "undefined behavior".
[Quoting Wikipedia's Undefined Behavior page] "In C and C++, implementation-defined behavior is also used, where the language standard does not specify the behavior, but the implementation must choose a behavior and needs to document and observe the rules it chose."
More precisely, the standard means "implementation-defined" (I think only) when it uses those words with respect to the statement made since "implementation-defined" is a specific attribute of the standard. The quote of C99 7.12p4 didn't mention "implementation-defined".
[From C99 std (late draft)] "undefined behavior: behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"
Note there is "no requirement" imposed for undefined behavior!
[C99 ..] "implementation-defined behavior: unspecified behavior where each implementation documents how the choice is made"
[C99 ..] "unspecified behavior: use of an unspecified value, or other behavior where this International Standard provides two or more possibilities and imposes no further requirements on which is chosen in any instance"
Documentation is a requirement for implementation-defined behavior.