I've searched the docs but I can't find anything to explain this.
I have a D program:
import std.stdio;
void main () {
writeln(int.max);
int a = 2;
int b = 180;
writeln(a^^b);
}
It writes:
2147483647
0
I overflowed int, but instead of getting junk or wrapping, I get 0.
If I use real or double, obviously the output will be correct.
I wrote a program to experiment with this in C (not that C is D, but D compiles to native code and C is portable assembler, so they should be comparable):
#include <stdio.h>
#include <math.h>
#include <limits.h>
int main(void) {
double i = pow(2, 128);
int j = (int) i;
printf("int max: %d\n", INT_MAX);
printf("print dbl as int: %d\n", i);
printf("cast dbl -> int: %d\n", j);
printf("double: %f\n", i);
return 0;
}
It gives:
int max: 2147483647
print dbl as int: -1254064128
cast dbl -> int: -2147483648
double: 340282366920938463463374607431768211456.000000
The second and third lines line will rarely be the same thing twice, as I believe it's undefined behaviour, which is the point.
I know that D wants to be a better C, and a way to do this is to eliminate undefined behaviour.
But, if D is a systems programming language (it even has inline asm), why does D refuse to wrap on overflow?
It DID wrap on overflow. You just happened to try the family of cases that happen to wrap to zero. Try 3^^180. I got -949019631. Just because a number happens to look pretty on screen doesn't mean it isn't garbage!
Consider that 2^^n == 1 << n. What happens when you shift a single bit left over and over and over again? Eventually, all the bits on the right become zero! Then when you truncate that to fit in a 64 bit value, you are left with all being zero.
But let me go into some detail anyway. First, a critique of your C:
// snip. note that i is type double
printf("print dbl as int: %d\n", i);
This line is wrong on two levels: it passes a 64 bit double where printf is expecting a 32 bit int, and it is reinterpret casting those bits to int, which is entirely different than doing a conversion to int.
If you wanted to do this in D, you'd want to explicitly reinterpret the bits using a union or cast through an intermediate pointer. You could even slice off the other 32 bits if you wanted to!
The next line, which uses a proper explicit cast, is written correctly, but still undefined behavior because casting a double to int when it is too large to fit is something neither C nor D (nor the underlying hardware) makes any promises about.
And back to D. The ^^ operator in D simply rewrites the expression into std.math.pow(a, b). std.math.pow has different implementations for different types. Since both arguments are integral here, it does no floating point calculations at all - it has a pure int/long implementation that works just like multiplication.
So your C comparison isn't quite right because in C, you used double and tried to convert, whereas in D, it never touched floating point at all. Integer multiplication is defined to work via two's complement and truncation, and that's exactly what happened here. It overflowed, leaving all zeros in the bits left behind.
Related
Consider the following program:
#include <iostream>
int main()
{
unsigned int a = 3;
unsigned int b = 7;
std::cout << (a - b) << std::endl; // underflow here!
return 0;
}
In the line starting with std::cout an underflow is happening because a is lesser than b so a-b is less than 0, but since a and b are unsigend so is a-b.
Is there a compiler flag (for G++) that gives me a warning when I try to calculate the difference of two unsigend integers?
Now, one could argue that an overflow/underflow can happen in any calculation using any operator. But I think it is more dangerous to apply operator - to unsigend ints because with unsigned integers this error may happen with quite low (to me: "more common") numbers.
A (static analysis) tool that finds such things would also be great but I much prefer a compiler flag and warning.
GCC does not (afaict) support it, but Clang's UBSanitizer has the following option [emphasis mine]:
-fsanitize=unsigned-integer-overflow: Unsigned integer overflow, where the result of an unsigned integer computation cannot be represented in its type. Unlike signed integer overflow, this is not undefined behavior, but it is often unintentional. This sanitizer does not check for lossy implicit conversions performed before such a computation
I was investigating the structure of floating-point numbers, and I've found that most of compilers use IEEE 754 standard to store floating point numbers.
And when I tried to do:
float a=0x3f520000; //have to be equal to 0.8203125 in IEEE 754
printf("value of 'a' is: %X [%x] %f\n",(int)a,(int)a, a);
it produces the result:
value of 'a' is: 3F520000 [3f520000] 1062338560.000000
but if I try:
int b=0x3f520000;
float* c = (float*)&b;
printf("value of 'c' is: %X [%x] %f\r\n", *(int*)c, *(int*)c, c[0]);
it gives:
value of 'c' is: 3F520000 [3f520000] 0.820313
The second try gave me the right answer. What is it wrong with the first try? And why does the result differ from that when I cast int to float via pointer?
The difference is that the first converts the value (0x3f520000 is the integer 1062338560), and is equivalent to this:
float a = 1062338560;
printf("value of 'a' is: %X [%x] %f\n",(int)a,(int)a, a);
The second reinterprets the representation of the int - 111111010100100000000000000000 - as being the representation of a float instead.
(It's also undefined, so you shouldn't expect it to do anything in particular.)
[Note: This answer assumes C, not C++, which have different rules]
With
float a=0x3f520000;
you take the integer value 1062338560 and the compiler will convert it to 1062338560.0f.
If you want hexadecimal floating point constant you must use exponent-format using the letter p. As in 0x1.a40010c6f7a0bp-1 (which is the hexadecimal notation for 0.820313).
What happens with
int b=0x3f520000;
float* c = (float*)&b;
is that you break strict aliasing and tell the compiler that c is pointing to a floating-point value (the strict aliasing break is because b isn't a floating point value). The compiler will then reinterpret the bits in *c as a float value.
0x3f520000 is an integer constant. When assigned to a float, the integer is converted.
Some more proper example of how to convert in the second case:
#include <stdio.h>
#include <string.h>
#include <stdint.h>
int main() {
uint32_t fi = 0x3f520000;
float f;
memcpy(&f, &fi, sizeof(f));
printf("%.7g\n", f);
}
it prints:
0.8203125
so that is what you expected.
The approach I used is memcpy that is the safest for all compilers and best choice for modern compilers (GCC since approx. 4.6, Clang since 3.x) that interpret memcpy as "bit cast" in such case and optimize it in a efficient and safe way (at least in "hosted" mode). That's still safe for older compilers, but not nessessarily efficient in the same way; some can prefer cast through union or ever through different pointer type. On dangers of that ways, see here or generally search "type punning and strict aliasing".
(Also, there could be some weird platforms that suffer from endianness issue that integer endianness differs from float one; ones that have byte different than 8 bits, and so on. I don't consider them here.)
UPDATE: I was starting answering to initial version of the question. Yep, bit casting and value conversion will give principally different results. That's how float numbers work.
Consider the following code sample:
#include <iostream>
#include <string>
int main()
{
std::string str("someString"); // length 10
int num = -11;
std::cout << num % str.length() << std::endl;
}
Running this code on http://cpp.sh, I get 5 as a result, while I was expecting it to be -1.
I know that this happens because the type of str.length() is size_t which is an implementation dependent unsigned, and because of the implicit type conversions that happen with binary operators that cause num to be converted from a signed int to an unsigned size_t (more here);
this causes the negative value to become a positive one and messes up the result of the operation.
One could think of addressing the problem with an explicit cast to int:
num % (int)str.length()
This might work but it's not guaranteed, for instance in the case of a string with length larger than the maximum value of int. One could reduce the risk using a larger type, like long long, but what if size_t is unsigned long long? Same problem.
How would you address this problem in a portable and robust way?
Since C++11, you can just cast the result of length to std::string::difference_type.
To address "But what if the size is too big?":
That won't happen on 64 bit platforms and even if you are on a smaller one: When was the last time you actually had a string that took up more than half of total RAM? Unless you are doing really specific stuff (which you would know), using the difference_type is just fine; quit fighting ghosts.
Alternatively, just use int64_t, that's certainly big enough. (Though maybe looping over one on some 32 bit processors is slower than int32_t, I don't know. Won't matter for that single modulus operation though.)
(Fun fact: Even some prominent committee members consider littering the standard library with unsigned types a mistake, for reference see
this panel at 9:50, 42:40, 1:02:50 )
Pre C++11, the sign of % with negative values was implementation defined, for well defined behavior, use std::div plus one of the casts described above.
We know that
-a % b == -(a % b)
So you could write something like this:
template<typename T, typename T2>
constexpr T safeModulo(T a, T2 b)
{
return (a >= 0 ? 1 : -1) * static_cast<T>(std::llabs(a) % b);
}
This won't overflow in 99.98% of the cases, because consider this
safeModulo(num, str.length());
If std::size_t is implemented as an unsigned long long, then T2 -> unsigned long long and T -> int.
As pointed out in the comments, using std::llabs instead of std::abs is important, because if a is the smallest possible value of int, removing the sign will overflow. Promoting a to a long long just before won't result in this problem, as long long has a larger range of values.
Now static_cast<int>(std::llabs(a) % b) will always result in a value that is smaller than a, so casting it to int will never overflow/underflow. Even if a gets promoted to an unsigned long long, it doesn't matter because a is already "unsigned" from std::llabs(a), and so the value is unchanged (i.e. didn't overflow/underflow).
Because of the property stated above, if a is negative, multiply the result with -1 and you get the correct result.
The only case where it results in undefined behavior is when a is std::numeric_limits<long long>::min(), as removing the sign overflows a, resulting in undefined behavior. There is probably another way to implement the function, I'll think about it.
I encountered a strange thing when I was programming under c++. It's about a simple multiplication.
Code:
unsigned __int64 a1 = 255*256*256*256;
unsigned __int64 a2= 255 << 24; // same as the above
cerr()<<"a1 is:"<<a1;
cerr()<<"a2 is:"<<a2;
interestingly the result is:
a1 is: 18446744073692774400
a2 is: 18446744073692774400
whereas it should be:(using calculator confirms)
4278190080
Can anybody tell me how could it be possible?
255*256*256*256
all operands are int you are overflowing int. The overflow of a signed integer is undefined behavior in C and C++.
EDIT:
note that the expression 255 << 24 in your second declaration also invokes undefined behavior if your int type is 32-bit. 255 x (2^24) is 4278190080 which cannot be represented in a 32-bit int (the maximum value is usually 2147483647 on a 32-bit int in two's complement representation).
C and C++ both say for E1 << E2 that if E1 is of a signed type and positive and that E1 x (2^E2) cannot be represented in the type of E1, the program invokes undefined behavior. Here ^ is the mathematical power operator.
Your literals are int. This means that all the operations are actually performed on int, and promptly overflow. This overflowed value, when converted to an unsigned 64bit int, is the value you observe.
It is perhaps worth explaining what happened to produce the number 18446744073692774400. Technically speaking, the expressions you wrote trigger "undefined behavior" and so the compiler could have produced anything as the result; however, assuming int is a 32-bit type, which it almost always is nowadays, you'll get the same "wrong" answer if you write
uint64_t x = (int) (255u*256u*256u*256u);
and that expression does not trigger undefined behavior. (The conversion from unsigned int to int involves implementation-defined behavior, but as nobody has produced a ones-complement or sign-and-magnitude CPU in many years, all implementations you are likely to encounter define it exactly the same way.) I have written the cast in C style because everything I'm saying here applies equally to C and C++.
First off, let's look at the multiplication. I'm writing the right hand side in hex because it's easier to see what's going on that way.
255u * 256u = 0x0000FF00u
255u * 256u * 256u = 0x00FF0000u
255u * 256u * 256u * 256u = 0xFF000000u (= 4278190080)
That last result, 0xFF000000u, has the highest bit of a 32-bit number set. Casting that value to a signed 32-bit type therefore causes it to become negative as-if 232 had been subtracted from it (that's the implementation-defined operation I mentioned above).
(int) (255u*256u*256u*256u) = 0xFF000000 = -16777216
I write the hexadecimal number there, sans u suffix, to emphasize that the bit pattern of the value does not change when you convert it to a signed type; it is only reinterpreted.
Now, when you assign -16777216 to a uint64_t variable, it is back-converted to unsigned as-if by adding 264. (Unlike the unsigned-to-signed conversion, this semantic is prescribed by the standard.) This does change the bit pattern, setting all of the high 32 bits of the number to 1 instead of 0 as you had expected:
(uint64_t) (int) (255u*256u*256u*256u) = 0xFFFFFFFFFF000000u
And if you write 0xFFFFFFFFFF000000 in decimal, you get 18446744073692774400.
As a closing piece of advice, whenever you get an "impossible" integer from C or C++, try printing it out in hexadecimal; it's much easier to see oddities of twos-complement fixed-width arithmetic that way.
The answer is simple -- overflowed.
Here Overflow occurred on int and when you are assigning it to unsigned int64 its converted in to 18446744073692774400 instead of 4278190080
int a = -534;
unsigned int b = (unsigned int)a;
printf("%d, %d", a, b);
prints -534, -534
Why is the typecast not taking place?
I expected it to be -534, 534
If I modify the code to
int a = -534;
unsigned int b = (unsigned int)a;
if(a < b)
printf("%d, %d", a, b);
its not printing anything... after all a is less than b??
Because you use %d for printing. Use %u for unsigned. Since printf is a vararg function, it cannot know the types of the parameters and must instead rely on the format specifiers. Because of this the type cast you do has no effect.
First, you don't need the cast: the value of a is implicitly converted to unsigned int with the assignment to b. So your statement is equivalent to:
unsigned int b = a;
Now, an important property of unsigned integral types in C and C++ is that their values are always in the range [0, max], where max for unsigned int is UINT_MAX (it's defined in limits.h). If you assign a value that's not in that range, it is converted to that range. So, if the value is negative, you add UINT_MAX+1 repeatedly to make it in the range [0, UINT_MAX]. For your code above, it is as if we wrote: unsigned int b = (UINT_MAX + a) + 1. This is not equal to -a (534).
Note that the above is true whether the underlying representation is in two's complement, ones' complement, or sign-magnitude (or any other exotic encoding). One can see that with something like:
signed char c = -1;
unsigned int u = c;
printf("%u\n", u);
assert(u == UINT_MAX);
On a typical two's complement machine with a 4-byte int, c is 0xff, and u is 0xffffffff. The compiler has to make sure that when value -1 is assigned to u, it is converted to a value equal to UINT_MAX.
Now going back to your code, the printf format string is wrong for b. You should use %u. When you do, you will find that it prints the value of UINT_MAX - 534 + 1 instead of 534.
When used in the comparison operator <, since b is unsigned int, a is also converted to unsigned int. This, given with b = a; earlier, means that a < b is false: a as an unsigned int is equal to b.
Let's say you have a ones' complement machine, and you do:
signed char c = -1;
unsigned char uc = c;
Let's say a char (signed or unsigned) is 8-bits on that machine. Then c and uc will store the following values and bit-patterns:
+----+------+-----------+
| c | -1 | 11111110 |
+----+------+-----------+
| uc | 255 | 11111111 |
+----+------+-----------+
Note that the bit patterns of c and uc are not the same. The compiler must make sure that c has the value -1, and uc has the value UCHAR_MAX, which is 255 on this machine.
There are more details on my answer to a question here on SO.
your specifier in the printf is asking printf to print a signed integer, so the underlying bytes are interpreted as a signed integer.
You should specify that you want an unsigned integer by using %u.
edit: a==b is true for the comparison, which is odd behaviour, but it's perfectly valid. You haven't changed the underlying bits you have only asked the compiler to treat the underlying bits in a certain way. Therefore a bitwise comparison yields true.
[speculation] I would suspect that behaviour might vary among compiler implementations -i.e., a fictitious CPU might not use the same logic for both signed and unsigned numerals in which case a bitwise comparison would fail. [/speculation]
C can be an ugly beast sometimes. The problem is that -534 always represents the value 0xfffffdea whether it is stored in a variable with the type unsigned int or signed int. To compare these variables they must be the same type so one will get automatically converted to either an unsigned or signed int to match the other. Once they are the same type they are equal as they represent the same value.
It seems likely that the behaviour you want is provided by the function abs:
int a = -534;
int b = abs(a);
printf("%d, %d", a, b);
I guess the first case of why b is being printed as -534 has been sufficiently answered by Tronic and Hassan. You should not be using %d and should be using %u.
As far as your second case is concered, again an implicit typecasting will be happening and both a and b will be same due to which your comparison does yield the expected result.
As far as I can see, the if fails because the compiler assumes the second variable should be considered the same type as the first. Try
if(b > a)
to see the difference.
Re 2nd question:
comparison never works between two different types - they are always implicitly cast to the "lowest common denominator", which in this case will be unsigned int. Nasty and counter-intuitive, I know.
Casting an integer type from signed to unsigned does not modify the bit pattern, it merely changes the interpretation of the bit pattern.
You also have a format specifier mismatch, %u should be used for unsigned integers, but even then the result will not be 534 as you expect, but 4294966762.
If you want to make a negative value positive, simply negate it:
unsigned b = (unsigned)-a ;
printf("%d, %u", a, b);
As for the second example, operations between types with differing signed-ness involve arcane implicit conversion rules - avoid. You should set your compiler's warning level high to trap many of these errors. I suggest /W4 /WX in VC++ and -Wall -Werror -Wformat for GCC for example.