In all versions of C and C++ prior to 2014, writing
1 << (CHAR_BIT * sizeof(int) - 1)
caused undefined behaviour, because left-shifting is defined as being equivalent to successive multiplication by 2, and this shift causes signed integer overflow:
The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are filled with zeros. [...] If E1 has a signed type and nonnegative value, and E1 × 2E2 is representable in the result type, then that is the resulting value; otherwise, the behavior is undefined.
However in C++14 the text has changed for << but not for multiplication:
The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. [...] Otherwise, if E1 has a signed type and non-negative value, and E1 × 2E2 is representable in the corresponding unsigned type of the result type, then that value, converted to the result type, is the resulting value; otherwise, the behavior is undefined.
The behaviour is now the same as for out-of-range assignment to signed type, i.e. as covered by [conv.integral]/3:
If the destination type is signed, the value is unchanged if it can be represented in the destination type (and bit-field width); otherwise, the value is implementation-defined.
This means it's still non-portable to write 1 << 31 (on a system with 32-bit int). So why was this change made in C++14?
The relevant issue is CWG 1457, where the justification is that the change allows 1 << 31 to be used in constant expressions:
The current wording of 5.8 [expr.shift] paragraph 2 makes it undefined
behavior to create the most-negative integer of a given type by
left-shifting a (signed) 1 into the sign bit, even though this is not
uncommonly done and works correctly on the majority of
(twos-complement) architectures:
...if E1 has a signed type and non-negative value, and E1 * 2E2 is
representable in the result type, then that is the resulting value;
otherwise, the behavior is undefined.
As a result, this technique
cannot be used in a constant expression, which will break a
significant amount of code.
Constant expressions can't contain undefined behavior, which means that using an expression containing UB in a context requiring a constant expression makes the program ill-formed. libstdc++'s numeric_limits::min, for example, once failed to compile in clang due to this.
Related
In c++20, signed integers are now defined to use two's complement,
see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r3.html
This is a welcome change, however one of the bullet-points caught my eye:
Change Left-shift on signed integer types produces the same
results as left-shift on the corresponding unsigned integer type.
This seem like a strange change. Will this not shift away the sign bit?
The C++17 wording for signed left shifts (E1 << E2) was:
Otherwise, if E1 has a signed type and non-negative value, and E1×2E2 is representable in the corresponding unsigned type of the result type, then that value, converted to the result type, is the resulting value; otherwise, the behavior is undefined.
Note that it speaks of being representable in "the corresponding unsigned type". So if you have a 32-bit signed integer whose value is the 0x7FFFFFFF, and you left-shift it by 1, the resulting shift is representable in a 32-bit unsigned integer (0xFFFFFFFE). But then this unsigned value gets converted into the result type. And converting an unsigned integer whose value is too big for the corresponding signed type is implementation-defined.
Overall, in C++17, left-shifting into the sign bit could happen through implementation-defined behavior, and even then only if you don't shift beyond the unsigned result type's size. Going past that is explicitly UB.
The C++20 wording, for both signed and unsigned integers, is:
The value of E1 << E2 is the unique value congruent to E1×2E2 modulo 2N, where N is the width of the type of the result.
Integer congruence modulo a number basically means cutting off the bits beyond the modulo number. The "width" of an integer is explicitly defined as:
The range of representable values for a signed integer type is −2N−1 to 2N−1−1 (inclusive), where N is called the width of the type.
This means that for a 32-bit signed integer, the width is 31. So the modulous of the result of a shift is 31 bits, which cuts off the sign bit, explicitly preventing shifting into it.
So in C++20, we have a harder guarantee; implementations can never do a signed left-shift into the sign bit. This is different from C++17 only in the sense that implementation variance/UB has been explicitly defined to not happen.
So left shift wasn't defined to shift into the sign bit in C++17, and is defined not to do so in C++20.
What exactly that quote means probably refers to the fact that left shift on a negative number is now valid, shifting is always well-defined no matter how much shifting you do, and the wording for the signed/unsigned shifting is overall the same.
Yes, the left shifting signed integer behavior changed with C++20.
With C++17, left-shifting a positive signed integer into the sign bit invokes implementation defined behavior.1 Example:
int i = INT_MAX;
int j = i << 1; // implementation defined behavior with std < C++20
C++20 changed this to defined behavior because it mandates two's complement representation for signed integers.2,3
With C++17, shifting a negative signed integer invokes undefined behavior.1 Example:
int i = -1;
int j = i << 1; // undefined behavior with std < C++20
In C++20, this changed as well and this operation now also invokes defined behavior.3
This seem like a strange change. Will this not shift away the sign bit?
Yes, a signed left shift shifts away the sign bit. Example:
int i = 1 << (sizeof(int)*8-1); // C++20: defined behavior, set most significant bit
int j = i << 1; // C++20: defined behavior, set to 0
The main reason for specifying something as undefined or implementation defined behavior is to allow for efficient implementations on different hardware.
Nowadays, since all CPUs implement two's complement it's natural that the C++ standard mandates it. And if you mandate two's complement it's only consequential that you make the above operations defined behavior because this is also how left shift behaves in all two's complement instruction set architectures (ISAs).
IOW, leaving it implementation defined and undefined wouldn't buy you anything.
Or, if you liked the previous undefined behavior why would you care if it gets changed to defined behavior? You can still avoid this operation as before. You wouldn't have to change your code.
1
The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned
type, the value of the result is E1 × 2**E2, reduced modulo one more than the maximum value representable in
the result type. Otherwise, if E1 has a signed type and non-negative value, and E1 × 2**E2 is representable
in the corresponding unsigned type of the result type, then that value, converted to the result type, is the
resulting value; otherwise, the behavior is undefined.
(C++17 final working draft, Section 8.8 Shift operators [expr.shift], Paragraph 2, page 132 - emphasis mine)
2
[..] For each value x of a signed integer type, the value of the
corresponding unsigned integer type congruent to x modulo 2 N has the same value of corresponding bits in
its value representation. 41) This is also known as two’s complement representation. [..]
(C++20 latest working draft, Section 6.8.1 Fundamental types [basic.fundamental], Paragraph 3, page 66)
3
The value of E1 << E2 is the unique value congruent to E1 × 2**E2 modulo 2**N, where N is the width of the
type of the result. [Note: E1 is left-shifted E2 bit positions; vacated bits are zero-filled. — end note]
(C++20 latest working draft, Section 7.6.7 Shift operators [expr.shift], Paragraph 2, page 129, link mine)
According to cppreference,
For signed a, the value of a << b is a * 2^b if it is
representable [in the unsigned version of the (since C++14)] return
type [(which is then converted to signed: this makes it legal to
create INT_MIN as 1 << 31) (since C++14)], otherwise the behavior
is undefined.
I can't quite understand this specification. What does it mean exactly by if it is representable in the unsigned version of the return type? Does it apply only to the case when the signed value is non-negative? Are INT_MIN << 1 and -1 << 31 well-defined?
If we go to source, the C++14 Standard, this is what we find (with the part about unsigned types highlighted):
The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned type, the value of the result is E1 × 2E2 , reduced modulo one more than the maximum value representable in the result type. Otherwise, if E1 has a signed type and non-negative value, and E1 × 2E2 is representable in the corresponding unsigned type of the result type, then that value, converted to the result type, is the resulting value; otherwise, the behavior is undefined.
For a platform in which std::numeric_limits<int>::digits is 31, it is legal to perform 1 << 31. The resulting value in unsigned int will be 0x80000000 or 2147483648. However, that number is beyond the valid range of value of int for such a platform. However, that bit pattern, when treated as a two's complement representation int is equal to -2147483648, which is the same as INT_MIN for such a platform.
If you use
int a = 2 << 31;
you invoke undefined behavior since 2 * 231 is not representable as an unsigned int in that platform.
Are INT_MIN << 1 and -1 << 31 well-defined?
No, they are not. Bitwise left shifting negative numbers is undefined behavior. Notice the use of non-negative value in the above highlighted text.
Is it legal to do the following in C11, C++11 and C++14?
static_assert(((-4) >> 1) == -2, "my code assumes sign-extending right shift");
or the C equivalent:
_Static_assert(((-4) >> 1) == -2, "my code assumes sign-extending right shift");
I don't know the rules for constant-expressions regarding whether you can use implementation-defined operations like the above.
I'm aware that the opposite, signed shift left of negative numbers, is undefined regardless of machine type.
Yes. The C++11 standard says in [expr.shift]/3:
The value of E1 >> E2 is E1 right-shifted E2 bit positions. If
E1 has an unsigned type or if E1 has a signed type and a non-negative
value, the value of the result is the integral part of the quotient of
E1/2^E2. If E1 has a signed type and a negative value, the
resulting value is implementation-defined.
And nowhere in [expr.const]/2 it is said that such a shift, or expressions with implementation-defined values in general, are not constant expressions.
You will thus get a constant expression that has an implementation-defined value.
This is legal, insofaras it doesn't cause undefined behaviour.
The behaviour of right-shift of negative values is implementation-defined. The C and C++ standards do not guarantee it to be either arithmetic or logical; although so far as I know there has never been a CPU that didn't pick one or the other.
If I have value declared as of type int on a 32 bit platform and perform the following:
int32_t mask;
mask = 1 << 31: // produces 2147483648 (or 0x80000000)
Can someone help me understand why the above line would produce the warning:
The result of the '<<' expression is undefined
231 isn't representable in a int32_t, which goes from -231 to (231-1). This is undefined behavior in C11 and C++11, and implementation-defined in C++14.
C11 §6.5.7/p4 (quoting N1570):
The result of E1 << E2 is E1 left-shifted E2 bit positions;
vacated bits are filled with zeros. [...] If E1 has a signed type
and nonnegative value, and E1 × 2E2 is representable in
the result type, then that is the resulting value; otherwise, the
behavior is undefined.
The C++11 rule in N3337 §5.8 [expr.shift]/p2 is pretty much identical. Since 231 isn't representable, the behavior is undefined.
C++14 §5.8 [expr.shift]/p2 (quoting N3936; see also CWG issue 1457):
The value of E1 << E2 is E1 left-shifted E2 bit positions;
vacated bits are zero-filled. [...] Otherwise, if E1 has a signed
type and non-negative value, and E1×2E2is representable
in the corresponding unsigned type of the result type, then that
value, converted to the result type, is the resulting value;
otherwise, the behavior is undefined.
As 231 is representable in an unsigned 32-bit int, the behavior is defined and the result is 231 converted to int32_t; this conversion is implementation-defined per §4.7 [conv.integral]/p3. In a typical system using two's complement you'd get -231.
I understand the reason why C++ define INT_MIN as (-2147483647 - 1), but why don't they just use 1<<31 ? That prevent the overflow and also easy to understand.
That prevent the overflow and also easy to understand
How could it prevent the overflow, if by left-shifting a positive number you are trying to obtain a negative one? ;)
Keep in mind, that signed integer overflow is Undefined Behavior. Per paragraph 5.8/2 of the C++11 Standard:
The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. [...] Otherwise, if E1 has a signed type and non-negative value, and E1×2^E2 is representable
in the corresponding unsigned type of the result type, then that value, converted to the result type, is the
resulting value; otherwise, the behavior is undefined.
Also, per paragraph 5/4:
If during the evaluation of an expression, the result is not mathematically defined or not in the range of
representable values for its type, the behavior is undefined. [...]
Because 1 << 31 invokes undefined behaviour (assuming 32-bit int).