Related
Consider the following code:
unsigned int n = 0;
unsigned int m = n - 1; // no warning here?
if (n > -1) {
std::cout << "n > -1.\n";
} else {
std::cout << "yes, 0 is not > -1.\n";
}
The code above produces a warning on the if condition if (m > -1) for comparing signed and unsigned integer expressions. I have no contest with that. What bothers me is the first two assignment statements.
unsigned int n = 0;
unsigned int m = n - 1;
My thinking is that the compiler should have given me a warning on the second assignment because it knows that the variable n is unsigned with a value of 0 from the first line and that there was an attempt to subtract from a zero value and assigning it to an unsigned type.
If the next line after the second assignment happened to be different than an if statement or something similar, then the concerned code might have slipped through.
Yes, there is a narrowing conversion before the assignment to m there and yes the compiler do not complain about it which was also mentioned by Marshall Clow in his C++Now 2017 Lightning Talk (Fighting Compiler Warnings).
short s = 3 * 6;
short s = integer * integer;
short s = integer;
So, why can't the compiler tell me about the possible underflow in that code?
Compilers:
Clang 3.7/4.0 (-Wall -Wextra)
GCC 5.3/7.1.1 (-Wall -Wextra -pedantic)
Microsoft C/C++ 19.00.23506
The reason is because if (n > -1) can never be false, but unsigned int m = n - 1; is an actual legal expression you may have wanted to write. From 5/9 there are a bunch of rules about how to get your signed an unsigned types to have a consistent type, and all of them fail except the final default condition
Otherwise, both operands shall be converted to the unsigned integer
type corresponding to the type of the operand with signed integer
type.
Since unsigned arithmetic is well-defined to use modulo operations the entire expression is legal and well defined. They could yet decide to emit a warning but there may be enough legacy code using tricks like this that it would cause too many false positives.
The following code crashes C++ with a runtime error:
#include <string>
using namespace std;
int main() {
string s = "aa";
for (int i = 0; i < s.length() - 3; i++) {
}
}
While this code does not crash:
#include <string>
using namespace std;
int main() {
string s = "aa";
int len = s.length() - 3;
for (int i = 0; i < len; i++) {
}
}
I just don't have any idea how to explain it. What could be the reason for this behavior?
s.length() is unsigned integer type. When you subtract 3, you make it negative. For an unsigned, it means very big.
A workaround (valid as long the string is long up to INT_MAX) would be to do like this:
#include <string>
using namespace std;
int main() {
string s = "aa";
for (int i = 0; i < static_cast<int> (s.length() ) - 3; i++) {
}
}
Which would never enter the loop.
A very important detail is that you have probably received a warning "comparing signed and unsigned value". The problem is that if you ignore those warnings, you enter the very dangerous field of implicit "integer conversion"(*), which has a defined behaviour, but it is difficult to follow: the best is to never ignore those compiler warnings.
(*) You might also be interested to know about "integer promotion".
First of all: why does it crash? Let's step through your program like a debugger would.
Note: I'll assume that your loop body isn't empty, but accesses the string. If this isn't the case, the cause of the crash is undefined behaviour through integer overflow. See Richard Hansens answer for that.
std::string s = "aa";//assign the two-character string "aa" to variable s of type std::string
for ( int i = 0; // create a variable i of type int with initial value 0
i < s.length() - 3 // call s.length(), subtract 3, compare the result with i. OK!
{...} // execute loop body
i++ // do the incrementing part of the loop, i now holds value 1!
i < s.length() - 3 // call s.length(), subtract 3, compare the result with i. OK!
{...} // execute loop body
i++ // do the incrementing part of the loop, i now holds value 2!
i < s.length() - 3 // call s.length(), subtract 3, compare the result with i. OK!
{...} // execute loop body
i++ // do the incrementing part of the loop, i now holds value 3!
.
.
We would expect the check i < s.length() - 3 to fail right away, since the length of s is two (we only every given it a length at the beginning and never changed it) and 2 - 3 is -1, 0 < -1 is false. However we do get an "OK" here.
This is because s.length() isn't 2. It's 2u. std::string::length() has return type size_t which is an unsigned integer. So going back to the loop condition, we first get the value of s.length(), so 2u, now subtract 3. 3 is an integer literal and interpreted by the compiler as type int. So the compiler has to calculate 2u - 3, two values of different types. Operations on primitive types only work for same types, so one has to be converted into the other. There are some strict rules, in this case, unsigned "wins", so 3 get's converted to 3u. In unsigned integers, 2u - 3u can't be -1u as such a number does not exists (well, because it has a sign of course!). Instead it calculates every operation modulo 2^(n_bits), where n_bits is the number of bits in this type (usually 8, 16, 32 or 64). So instead of -1 we get 4294967295u (assuming 32bit).
So now the compiler is done with s.length() - 3 (of course it's much much faster than me ;-) ), now let's go for the comparison: i < s.length() - 3. Putting in the values: 0 < 4294967295u. Again, different types, 0 becomes 0u, the comparison 0u < 4294967295u is obviously true, the loop condition is positively checked, we can now execute the loop body.
After incrementing, the only thing that changes in the above is the value of i. The value of i will again be converted into an unsigned int, as the comparison needs it.
So we have
(0u < 4294967295u) == true, let's do the loop body!
(1u < 4294967295u) == true, let's do the loop body!
(2u < 4294967295u) == true, let's do the loop body!
Here's the problem: What do you do in the loop body? Presumably you access the i^th character of your string, don't you? Even though it wasn't your intention, you didn't only accessed the zeroth and first, but also the second! The second doesn't exists (as your string only has two characters, the zeroth and first), you access memory you shouldn't, the program does whatever it wants (undefined behaviour). Note that the program isn't required to crash immediately. It can seem to work fine for another half an hour, so these mistakes are hard to catch. But it's always dangerous to access memory beyond the bounds, this is where most crashes come from.
So in summary, you get a different value from s.length() - 3 from that what you'd expect, this results in a positive loop condition check, that leads to repetitive execution of the loop body, which in itself accesses memory it shouldn't.
Now let's see how to avoid that, i.e. how to tell the compiler what you actually meant in your loop condition.
Lengths of strings and sizes of containers are inherently unsigned so you should use an unsigned integer in for loops.
Since unsigned int is fairly long and therefore undesirable to write over and over again in loops, just use size_t. This is the type every container in the STL uses for storing length or size. You may need to include cstddef to assert platform independence.
#include <cstddef>
#include <string>
using namespace std;
int main() {
string s = "aa";
for ( size_t i = 0; i + 3 < s.length(); i++) {
// ^^^^^^ ^^^^
}
}
Since a < b - 3 is mathematically equivalent to a + 3 < b, we can interchange them. However, a + 3 < b prevents b - 3 to be a huge value. Recall that s.length() returns an unsigned integer and unsigned integers perform operations module 2^(bits) where bits is the number of bits in the type (usually 8, 16, 32 or 64). Therefore with s.length() == 2, s.length() - 3 == -1 == 2^(bits) - 1.
Alternatively, if you want to use i < s.length() - 3 for personal preference, you have to add a condition:
for ( size_t i = 0; (s.length() > 3) && (i < s.length() - 3); ++i )
// ^ ^ ^- your actual condition
// ^ ^- check if the string is long enough
// ^- still prefer unsigned types!
Actually, in the first version you loop for a very long time, as you compare i to an unsigned integer containing a very large number. The size of a string is (in effect) the same as size_t which is an unsigned integer. When you subtract the 3 from that value it underflows and goes on to be a big value.
In the second version of the code, you assign this unsigned value to a signed variable, and so you get the correct value.
And it's not actually the condition or the value that causes the crash, it's most likely that you index the string out of bounds, a case of undefined behavior.
Assuming you left out important code in the for loop
Most people here seem unable to reproduce the crash—myself included—and it looks like the other answers here are based on the assumption that you left out some important code in the body of the for loop, and that the missing code is what is causing your crash.
If you are using i to access memory (presumably characters in the string) in the body of the for loop, and you left that code out of your question in an attempt to provide a minimal example, then the crash is easily explained by the fact that s.length() - 3 has the value SIZE_MAX due to modular arithmetic on unsigned integer types. SIZE_MAX is a very big number, so i will keep getting bigger until it is used to access an address that triggers a segfault.
However, your code could theoretically crash as-is, even if the body of the for loop is empty. I am unaware of any implementations that would crash, but maybe your compiler and CPU are exotic.
The following explanation does not assume that you left out code in your question. It takes on faith that the code you posted in your question crashes as-is; that it isn't an abbreviated stand-in for some other code that crashes.
Why your first program crashes
Your first program crashes because that is its reaction to undefined behavior in your code. (When I try running your code, it terminates without crashing because that is my implementation's reaction to the undefined behavior.)
The undefined behavior comes from overflowing an int. The C++11 standard says (in [expr] clause 5 paragraph 4):
If during the evaluation of an expression, the result is not mathematically defined or not in the range of representable values for its type, the behavior is undefined.
In your example program, s.length() returns a size_t with value 2. Subtracting 3 from that would yield negative 1, except size_t is an unsigned integer type. The C++11 standard says (in [basic.fundamental] clause 3.9.1 paragraph 4):
Unsigned integers, declared unsigned, shall obey the laws of arithmetic modulo 2n where n is the number of bits in the value representation of that particular size of integer.46
46) This implies that unsigned arithmetic does not overflow because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting unsigned integer type.
This means that the result of s.length() - 3 is a size_t with value SIZE_MAX. This is a very big number, bigger than INT_MAX (the largest value representable by int).
Because s.length() - 3 is so big, execution spins in the loop until i gets to INT_MAX. On the very next iteration, when it tries to increment i, the result would be INT_MAX + 1 but that is not in the range of representable values for int. Thus, the behavior is undefined. In your case, the behavior is to crash.
On my system, my implementation's behavior when i is incremented past INT_MAX is to wrap (set i to INT_MIN) and keep going. Once i reaches -1, the usual arithmetic conversions (C++ [expr] clause 5 paragraph 9) cause i to equal SIZE_MAX so the loop terminates.
Either reaction is appropriate. That is the problem with undefined behavior—it might work as you intend, it might crash, it might format your hard drive, or it might cancel Firefly. You never know.
How your second program avoids the crash
As with the first program, s.length() - 3 is a size_t type with value SIZE_MAX. However, this time the value is being assigned to an int. The C++11 standard says (in [conv.integral] clause 4.7 paragraph 3):
If the destination type is signed, the value is unchanged if it can be represented in the destination type (and bit-field width); otherwise, the value is implementation-defined.
The value SIZE_MAX is too big to be representable by an int, so len gets an implementation-defined value (probably -1, but maybe not). The condition i < len will eventually be true regardless of the value assigned to len, so your program will terminate without encountering any undefined behavior.
The type of s.length() is size_t with a value of 2, therefore s.length() - 3 is also an unsigned type size_t and it has a value of SIZE_MAX which is implementation defined (which is 18446744073709551615 if its size is 64 bit). It is at least 32 bit type (can be 64 bit in 64 bit platforms) and this high number means an indefinite loop. In order to prevent this problem you can simply cast s.length() to int:
for (int i = 0; i < (int)s.length() - 3; i++)
{
//..some code causing crash
}
In the second case len is -1 because it is a signed integer and it does not enter the loop.
When it comes to crashing, this "infinite" loop is not the direct cause of the crash. If you share the code within the loop you can get further explanation.
Since s.length() is unsigned type quantity, when you do s.length()-3, it becomes negative and negative values are stored as large positive values (due to unsigned conversion specifications) and the loop goes infinite and hence it crashes.
To make it work, you must typecast the s.length() as :
static_cast < int > (s.length())
The problem you are having arises from the following statement:
i < s.length() - 3
The result of s.length() is of the unsigned size_t type.
If you imagine the binary representation of two:
0...010
And you then substitute three from this, you are effectively taking off 1 three times, that is:
0...001
0...000
But then you have a problem, removing the third digit it underflows, as it attempts to get another digit from the left:
1...111
This is what happens no matter if you have an unsigned or signed type, however the difference is the signed type uses the Most Significant Bit (or MSB) to represent if the number is negative or not. When the undeflow occurs it simply represents a negative for the signed type.
On the other hand, size_t is unsigned. When it underflows it will now represent the highest number size_t can possibly represent. Thus the loop is practically infinite (Depending on your computer, as this effects the maximum of size_t).
In order to fix this problem, you can manipulate the code you have in a few different ways:
int main() {
string s = "aa";
for (size_t i = 3; i < s.length(); i++) {
}
}
or
int main() {
string s = "aa";
for (size_t i = 0; i + 3 < s.length(); i++) {
}
}
or even:
int main() {
string s = "aa";
for(size_t i = s.length(); i > 3; --i) {
}
}
The important things to note is that the substitution has been omitted and instead addition has been used elsewhere with the same logical evaluations.
Both the first and last ones change the value of i that is available inside the for loop whereas the second will keep it the same.
I was tempted to provide this as an example of code:
int main() {
string s = "aa";
for(size_t i = s.length(); --i > 2;) {
}
}
After some thought I realised this was a bad idea. Readers' exercise is to work out why!
The reason is the same as
int a = 1000000000;
long long b = a * 100000000; would give error. When compilers multiplies these numbers it evaluates it as ints, since a and literal 1000000000 are ints, and since 10^18 is much more large than the upper bound of int, it will give error.
In your case we have s.length() - 3, as s.length() is unsigned int, it cant be negative, and since s.length() - 3 is evaluated as unsigned int, and its value is -1, it gives error here too.
I want to define a function that takes an unsigned int as argument and returns an int congruent modulo UINT_MAX+1 to the argument.
A first attempt might look like this:
int unsigned_to_signed(unsigned n)
{
return static_cast<int>(n);
}
But as any language lawyer knows, casting from unsigned to signed for values larger than INT_MAX is implementation-defined.
I want to implement this such that (a) it only relies on behavior mandated by the spec; and (b) it compiles into a no-op on any modern machine and optimizing compiler.
As for bizarre machines... If there is no signed int congruent modulo UINT_MAX+1 to the unsigned int, let's say I want to throw an exception. If there is more than one (I am not sure this is possible), let's say I want the largest one.
OK, second attempt:
int unsigned_to_signed(unsigned n)
{
int int_n = static_cast<int>(n);
if (n == static_cast<unsigned>(int_n))
return int_n;
// else do something long and complicated
}
I do not much care about the efficiency when I am not on a typical twos-complement system, since in my humble opinion that is unlikely. And if my code becomes a bottleneck on the omnipresent sign-magnitude systems of 2050, well, I bet someone can figure that out and optimize it then.
Now, this second attempt is pretty close to what I want. Although the cast to int is implementation-defined for some inputs, the cast back to unsigned is guaranteed by the standard to preserve the value modulo UINT_MAX+1. So the conditional does check exactly what I want, and it will compile into nothing on any system I am likely to encounter.
However... I am still casting to int without first checking whether it will invoke implementation-defined behavior. On some hypothetical system in 2050 it could do who-knows-what. So let's say I want to avoid that.
Question: What should my "third attempt" look like?
To recap, I want to:
Cast from unsigned int to signed int
Preserve the value mod UINT_MAX+1
Invoke only standard-mandated behavior
Compile into a no-op on a typical twos-complement machine with optimizing compiler
[Update]
Let me give an example to show why this is not a trivial question.
Consider a hypothetical C++ implementation with the following properties:
sizeof(int) equals 4
sizeof(unsigned) equals 4
INT_MAX equals 32767
INT_MIN equals -232 + 32768
UINT_MAX equals 232 - 1
Arithmetic on int is modulo 232 (into the range INT_MIN through INT_MAX)
std::numeric_limits<int>::is_modulo is true
Casting unsigned n to int preserves the value for 0 <= n <= 32767 and yields zero otherwise
On this hypothetical implementation, there is exactly one int value congruent (mod UINT_MAX+1) to each unsigned value. So my question would be well-defined.
I claim that this hypothetical C++ implementation fully conforms to the C++98, C++03, and C++11 specifications. I admit I have not memorized every word of all of them... But I believe I have read the relevant sections carefully. So if you want me to accept your answer, you either must (a) cite a spec that rules out this hypothetical implementation or (b) handle it correctly.
Indeed, a correct answer must handle every hypothetical implementation permitted by the standard. That is what "invoke only standard-mandated behavior" means, by definition.
Incidentally, note that std::numeric_limits<int>::is_modulo is utterly useless here for multiple reasons. For one thing, it can be true even if unsigned-to-signed casts do not work for large unsigned values. For another, it can be true even on one's-complement or sign-magnitude systems, if arithmetic is simply modulo the entire integer range. And so on. If your answer depends on is_modulo, it's wrong.
[Update 2]
hvd's answer taught me something: My hypothetical C++ implementation for integers is not permitted by modern C. The C99 and C11 standards are very specific about the representation of signed integers; indeed, they only permit twos-complement, ones-complement, and sign-magnitude (section 6.2.6.2 paragraph (2); ).
But C++ is not C. As it turns out, this fact lies at the very heart of my question.
The original C++98 standard was based on the much older C89, which says (section 3.1.2.5):
For each of the signed integer types, there is a corresponding (but
different) unsigned integer type (designated with the keyword
unsigned) that uses the same amount of storage (including sign
information) and has the same alignment requirements. The range of
nonnegative values of a signed integer type is a subrange of the
corresponding unsigned integer type, and the representation of the
same value in each type is the same.
C89 says nothing about only having one sign bit or only allowing twos-complement/ones-complement/sign-magnitude.
The C++98 standard adopted this language nearly verbatim (section 3.9.1 paragraph (3)):
For each of the signed integer types, there exists a corresponding
(but different) unsigned integer type: "unsigned char", "unsigned
short int", "unsigned int", and "unsigned long int", each of
which occupies the same amount of storage and has the same alignment
requirements (3.9) as the corresponding signed integer type ; that
is, each signed integer type has the same object representation as
its corresponding unsigned integer type. The range of nonnegative
values of a signed integer type is a subrange of the corresponding
unsigned integer type, and the value representation of each
corresponding signed/unsigned type shall be the same.
The C++03 standard uses essentially identical language, as does C++11.
No standard C++ spec constrains its signed integer representations to any C spec, as far as I can tell. And there is nothing mandating a single sign bit or anything of the kind. All it says is that non-negative signed integers must be a subrange of the corresponding unsigned.
So, again I claim that INT_MAX=32767 with INT_MIN=-232+32768 is permitted. If your answer assumes otherwise, it is incorrect unless you cite a C++ standard proving me wrong.
Expanding on user71404's answer:
int f(unsigned x)
{
if (x <= INT_MAX)
return static_cast<int>(x);
if (x >= INT_MIN)
return static_cast<int>(x - INT_MIN) + INT_MIN;
throw x; // Or whatever else you like
}
If x >= INT_MIN (keep the promotion rules in mind, INT_MIN gets converted to unsigned), then x - INT_MIN <= INT_MAX, so this won't have any overflow.
If that is not obvious, take a look at the claim "If x >= -4u, then x + 4 <= 3.", and keep in mind that INT_MAX will be equal to at least the mathematical value of -INT_MIN - 1.
On the most common systems, where !(x <= INT_MAX) implies x >= INT_MIN, the optimizer should be able (and on my system, is able) to remove the second check, determine that the two return statements can be compiled to the same code, and remove the first check too. Generated assembly listing:
__Z1fj:
LFB6:
.cfi_startproc
movl 4(%esp), %eax
ret
.cfi_endproc
The hypothetical implementation in your question:
INT_MAX equals 32767
INT_MIN equals -232 + 32768
is not possible, so does not need special consideration. INT_MIN will be equal to either -INT_MAX, or to -INT_MAX - 1. This follows from C's representation of integer types (6.2.6.2), which requires n bits to be value bits, one bit to be a sign bit, and only allows one single trap representation (not including representations that are invalid because of padding bits), namely the one that would otherwise represent negative zero / -INT_MAX - 1. C++ doesn't allow any integer representations beyond what C allows.
Update: Microsoft's compiler apparently does not notice that x > 10 and x >= 11 test the same thing. It only generates the desired code if x >= INT_MIN is replaced with x > INT_MIN - 1u, which it can detect as the negation of x <= INT_MAX (on this platform).
[Update from questioner (Nemo), elaborating on our discussion below]
I now believe this answer works in all cases, but for complicated reasons. I am likely to award the bounty to this solution, but I want to capture all the gory details in case anybody cares.
Let's start with C++11, section 18.3.3:
Table 31 describes the header <climits>.
...
The contents are the same as the Standard C library header <limits.h>.
Here, "Standard C" means C99, whose specification severely constrains the representation of signed integers. They are just like unsigned integers, but with one bit dedicated to "sign" and zero or more bits dedicated to "padding". The padding bits do not contribute to the value of the integer, and the sign bit contributes only as twos-complement, ones-complement, or sign-magnitude.
Since C++11 inherits the <climits> macros from C99, INT_MIN is either -INT_MAX or -INT_MAX-1, and hvd's code is guaranteed to work. (Note that, due to the padding, INT_MAX could be much less than UINT_MAX/2... But thanks to the way signed->unsigned casts work, this answer handles that fine.)
C++03/C++98 is trickier. It uses the same wording to inherit <climits> from "Standard C", but now "Standard C" means C89/C90.
All of these -- C++98, C++03, C89/C90 -- have the wording I give in my question, but also include this (C++03 section 3.9.1 paragraph 7):
The representations of integral types shall define values by use of a
pure binary numeration system.(44) [Example: this International
Standard permits 2’s complement, 1’s complement and signed magnitude
representations for integral types.]
Footnote (44) defines "pure binary numeration system":
A positional representation for integers that uses the binary digits 0
and 1, in which the values represented by successive bits are
additive, begin with 1, and are multiplied by successive integral
power of 2, except perhaps for the bit with the highest position.
What is interesting about this wording is that it contradicts itself, because the definition of "pure binary numeration system" does not permit a sign/magnitude representation! It does allow the high bit to have, say, the value -2n-1 (twos complement) or -(2n-1-1) (ones complement). But there is no value for the high bit that results in sign/magnitude.
Anyway, my "hypothetical implementation" does not qualify as "pure binary" under this definition, so it is ruled out.
However, the fact that the high bit is special means we can imagine it contributing any value at all: A small positive value, huge positive value, small negative value, or huge negative value. (If the sign bit can contribute -(2n-1-1), why not -(2n-1-2)? etc.)
So, let's imagine a signed integer representation that assigns a wacky value to the "sign" bit.
A small positive value for the sign bit would result in a positive range for int (possibly as large as unsigned), and hvd's code handles that just fine.
A huge positive value for the sign bit would result in int having a maximum larger than unsigned, which is is forbidden.
A huge negative value for the sign bit would result in int representing a non-contiguous range of values, and other wording in the spec rules that out.
Finally, how about a sign bit that contributes a small negative quantity? Could we have a 1 in the "sign bit" contribute, say, -37 to the value of the int? So then INT_MAX would be (say) 231-1 and INT_MIN would be -37?
This would result in some numbers having two representations... But ones-complement gives two representations to zero, and that is allowed according to the "Example". Nowhere does the spec say that zero is the only integer that might have two representations. So I think this new hypothetical is allowed by the spec.
Indeed, any negative value from -1 down to -INT_MAX-1 appears to be permissible as a value for the "sign bit", but nothing smaller (lest the range be non-contiguous). In other words, INT_MIN might be anything from -INT_MAX-1 to -1.
Now, guess what? For the second cast in hvd's code to avoid implementation-defined behavior, we just need x - (unsigned)INT_MIN less than or equal to INT_MAX. We just showed INT_MIN is at least -INT_MAX-1. Obviously, x is at most UINT_MAX. Casting a negative number to unsigned is the same as adding UINT_MAX+1. Put it all together:
x - (unsigned)INT_MIN <= INT_MAX
if and only if
UINT_MAX - (INT_MIN + UINT_MAX + 1) <= INT_MAX
-INT_MIN-1 <= INT_MAX
-INT_MIN <= INT_MAX+1
INT_MIN >= -INT_MAX-1
That last is what we just showed, so even in this perverse case, the code actually works.
That exhausts all of the possibilities, thus ending this extremely academic exercise.
Bottom line: There is some seriously under-specified behavior for signed integers in C89/C90 that got inherited by C++98/C++03. It is fixed in C99, and C++11 indirectly inherits the fix by incorporating <limits.h> from C99. But even C++11 retains the self-contradictory "pure binary representation" wording...
This code relies only on behavior, mandated by the spec, so requirement (a) is easily satisfied:
int unsigned_to_signed(unsigned n)
{
int result = INT_MAX;
if (n > INT_MAX && n < INT_MIN)
throw runtime_error("no signed int for this number");
for (unsigned i = INT_MAX; i != n; --i)
--result;
return result;
}
It's not so easy with requirement (b). This compiles into a no-op with gcc 4.6.3 (-Os, -O2, -O3) and with clang 3.0 (-Os, -O, -O2, -O3). Intel 12.1.0 refuses to optimize this. And I have no info about Visual C.
The original answer solved the problem only for unsigned => int. What if we want to solve the general problem of "some unsigned type" to its corresponding signed type? Furthermore, the original answer was excellent at citing sections of the standard and analyzing some corner cases, but it did not really help me get a feel for why it worked, so this answer will try to give a strong conceptual basis. This answer will try to help explain "why", and use modern C++ features to try to simplify the code.
C++20 answer
The problem has simplified dramatically with P0907: Signed Integers are Two’s Complement and the final wording P1236 that was voted into the C++20 standard. Now, the answer is as simple as possible:
template<std::unsigned_integral T>
constexpr auto cast_to_signed_integer(T const value) {
return static_cast<std::make_signed_t<T>>(value);
}
That's it. A static_cast (or C-style cast) is finally guaranteed to do the thing you need for this question, and the thing many programmers thought it always did.
C++17 answer
In C++17, things are much more complicated. We have to deal with three possible integer representations (two's complement, ones' complement, and sign-magnitude). Even in the case where we know it must be two's complement because we checked the range of possible values, the conversion of a value outside the range of the signed integer to that signed integer still gives us an implementation-defined result. We have to use tricks like we have seen in other answers.
First, here is the code for how to solve the problem generically:
template<typename T, typename = std::enable_if_t<std::is_unsigned_v<T>>>
constexpr auto cast_to_signed_integer(T const value) {
using result = std::make_signed_t<T>;
using result_limits = std::numeric_limits<result>;
if constexpr (result_limits::min() + 1 != -result_limits::max()) {
if (value == static_cast<T>(result_limits::max()) + 1) {
throw std::runtime_error("Cannot convert the maximum possible unsigned to a signed value on this system");
}
}
if (value <= result_limits::max()) {
return static_cast<result>(value);
} else {
using promoted_unsigned = std::conditional_t<sizeof(T) <= sizeof(unsigned), unsigned, T>;
using promoted_signed = std::make_signed_t<promoted_unsigned>;
constexpr auto shift_by_window = [](auto x) {
// static_cast to avoid conversion warning
return x - static_cast<decltype(x)>(result_limits::max()) - 1;
};
return static_cast<result>(
shift_by_window( // shift values from common range to negative range
static_cast<promoted_signed>(
shift_by_window( // shift large values into common range
static_cast<promoted_unsigned>(value) // cast to avoid promotion to int
)
)
)
);
}
}
This has a few more casts than the accepted answer, and that is to ensure there are no signed / unsigned mismatch warnings from your compiler and to properly handle integer promotion rules.
We first have a special case for systems that are not two's complement (and thus we must handle the maximum possible value specially because it doesn't have anything to map to). After that, we get to the real algorithm.
The second top-level condition is straightforward: we know the value is less than or equal to the maximum value, so it fits in the result type. The third condition is a little more complicated even with the comments, so some examples would probably help understand why each statement is necessary.
Conceptual basis: the number line
First, what is this window concept? Consider the following number line:
| signed |
<.........................>
| unsigned |
It turns out that for two's complement integers, you can divide the subset of the number line that can be reached by either type into three equally sized categories:
- => signed only
= => both
+ => unsigned only
<..-------=======+++++++..>
This can be easily proven by considering the representation. An unsigned integer starts at 0 and uses all of the bits to increase the value in powers of 2. A signed integer is exactly the same for all of the bits except the sign bit, which is worth -(2^position) instead of 2^position. This means that for all n - 1 bits, they represent the same values. Then, unsigned integers have one more normal bit, which doubles the total number of values (in other words, there are just as many values with that bit set as without it set). The same logic holds for signed integers, except that all the values with that bit set are negative.
The other two legal integer representations, ones' complement and sign-magnitude, have all of the same values as two's complement integers except for one: the most negative value. C++ defines everything about integer types, except for reinterpret_cast (and the C++20 std::bit_cast), in terms of the range of representable values, not in terms of the bit representation. This means that our analysis will hold for each of these three representations as long as we do not ever try to create the trap representation. The unsigned value that would map to this missing value is a rather unfortunate one: the one right in the middle of the unsigned values. Fortunately, our first condition checks (at compile time) whether such a representation exists, and then handles it specially with a runtime check.
The first condition handles the case where we are in the = section, which means that we are in the overlapping region where the values in one can be represented in the other without change. The shift_by_window function in the code moves all values down by the size of each of these segments (we have to subtract the max value then subtract 1 to avoid arithmetic overflow issues). If we are outside of that region (we are in the + region), we need to jump down by one window size. This puts us in the overlapping range, which means we can safely convert from unsigned to signed because there is no change in value. However, we are not done yet because we have mapped two unsigned values to each signed value. Therefore, we need to shift down to the next window (the - region) so that we have a unique mapping again.
Now, does this give us a result congruent mod UINT_MAX + 1, as requested in the question? UINT_MAX + 1 is equivalent to 2^n, where n is the number of bits in the value representation. The value we use for our window size is equal to 2^(n - 1) (the final index in a sequence of values is one less than the size). We subtract that value twice, which means we subtract 2 * 2^(n - 1) which is equal to 2^n. Adding and subtracting x is a no-op in arithmetic mod x, so we have not affected the original value mod 2^n.
Properly handling integer promotions
Because this is a generic function and not just int and unsigned, we also have to concern ourselves with integral promotion rules. There are two possibly interesting cases: one in which short is smaller than int and one in which short is the same size as int.
Example: short smaller than int
If short is smaller than int (common on modern platforms) then we also know that unsigned short can fit in an int, which means that any operations on it will actually happen in int, so we explicitly cast to the promoted type to avoid this. Our final statement is pretty abstract and becomes easier to understand if we substitute in real values. For our first interesting case, with no loss of generality let us consider a 16-bit short and a 17-bit int (which is still allowed under the new rules, and would just mean that at least one of those two integer types have some padding bits):
constexpr auto shift_by_window = [](auto x) {
return x - static_cast<decltype(x)>(32767) - 1;
};
return static_cast<int16_t>(
shift_by_window(
static_cast<int17_t>(
shift_by_window(
static_cast<uint17_t>(value)
)
)
)
);
Solving for the greatest possible 16-bit unsigned value
constexpr auto shift_by_window = [](auto x) {
return x - static_cast<decltype(x)>(32767) - 1;
};
return int16_t(
shift_by_window(
int17_t(
shift_by_window(
uint17_t(65535)
)
)
)
);
Simplifies to
return int16_t(
int17_t(
uint17_t(65535) - uint17_t(32767) - 1
) -
int17_t(32767) -
1
);
Simplifies to
return int16_t(
int17_t(uint17_t(32767)) -
int17_t(32767) -
1
);
Simplifies to
return int16_t(
int17_t(32767) -
int17_t(32767) -
1
);
Simplifies to
return int16_t(-1);
We put in the largest possible unsigned and get back -1, success!
Example: short same size as int
If short is the same size as int (uncommon on modern platforms), the integral promotion rule are slightly different. In this case, short promotes to int and unsigned short promotes to unsigned. Fortunately, we explicitly cast each result to the type we want to do the calculation in, so we end up with no problematic promotions. With no loss of generality let us consider a 16-bit short and a 16-bit int:
constexpr auto shift_by_window = [](auto x) {
return x - static_cast<decltype(x)>(32767) - 1;
};
return static_cast<int16_t>(
shift_by_window(
static_cast<int16_t>(
shift_by_window(
static_cast<uint16_t>(value)
)
)
)
);
Solving for the greatest possible 16-bit unsigned value
auto x = int16_t(
uint16_t(65535) - uint16_t(32767) - 1
);
return int16_t(
x - int16_t(32767) - 1
);
Simplifies to
return int16_t(
int16_t(32767) - int16_t(32767) - 1
);
Simplifies to
return int16_t(-1);
We put in the largest possible unsigned and get back -1, success!
What if I just care about int and unsigned and don't care about warnings, like the original question?
constexpr int cast_to_signed_integer(unsigned const value) {
using result_limits = std::numeric_limits<int>;
if constexpr (result_limits::min() + 1 != -result_limits::max()) {
if (value == static_cast<unsigned>(result_limits::max()) + 1) {
throw std::runtime_error("Cannot convert the maximum possible unsigned to a signed value on this system");
}
}
if (value <= result_limits::max()) {
return static_cast<int>(value);
} else {
constexpr int window = result_limits::min();
return static_cast<int>(value + window) + window;
}
}
See it live
https://godbolt.org/z/74hY81
Here we see that clang, gcc, and icc generate no code for cast and cast_to_signed_integer_basic at -O2 and -O3, and MSVC generates no code at /O2, so the solution is optimal.
You can explicitly tell the compiler what you want to do:
int unsigned_to_signed(unsigned n) {
if (n > INT_MAX) {
if (n <= UINT_MAX + INT_MIN) {
throw "no result";
}
return static_cast<int>(n + INT_MIN) - (UINT_MAX + INT_MIN + 1);
} else {
return static_cast<int>(n);
}
}
Compiles with gcc 4.7.2 for x86_64-linux (g++ -O -S test.cpp) to
_Z18unsigned_to_signedj:
movl %edi, %eax
ret
If x is our input...
If x > INT_MAX, we want to find a constant k such that 0 < x - k*INT_MAX < INT_MAX.
This is easy -- unsigned int k = x / INT_MAX;. Then, let unsigned int x2 = x - k*INT_MAX;
We can now cast x2 to int safely. Let int x3 = static_cast<int>(x2);
We now want to subtract something like UINT_MAX - k * INT_MAX + 1 from x3, if k > 0.
Now, on a 2s complement system, so long as x > INT_MAX, this works out to:
unsigned int k = x / INT_MAX;
x -= k*INT_MAX;
int r = int(x);
r += k*INT_MAX;
r -= UINT_MAX+1;
Note that UINT_MAX+1 is zero in C++ guaranteed, the conversion to int was a noop, and we subtracted k*INT_MAX then added it back on "the same value". So an acceptable optimizer should be able to erase all that tomfoolery!
That leaves the problem of x > INT_MAX or not. Well, we create 2 branches, one with x > INT_MAX, and one without. The one without does a strait cast, which the compiler optimizes to a noop. The one with ... does a noop after the optimizer is done. The smart optimizer realizes both branches to the same thing, and drops the branch.
Issues: if UINT_MAX is really large relative to INT_MAX, the above might not work. I am assuming that k*INT_MAX <= UINT_MAX+1 implicitly.
We could probably attack this with some enums like:
enum { divisor = UINT_MAX/INT_MAX, remainder = UINT_MAX-divisor*INT_MAX };
which work out to 2 and 1 on a 2s complement system I believe (are we guaranteed for that math to work? That's tricky...), and do logic based on these that easily optimize away on non-2s complement systems...
This also opens up the exception case. It is only possible if UINT_MAX is much larger than (INT_MIN-INT_MAX), so you can put your exception code in an if block asking exactly that question somehow, and it won't slow you down on a traditional system.
I'm not exactly sure how to construct those compile-time constants to deal correctly with that.
std::numeric_limits<int>::is_modulo is a compile time constant. so you can use it for template specialization. problem solved, at least if compiler plays along with inlining.
#include <limits>
#include <stdexcept>
#include <string>
#ifdef TESTING_SF
bool const testing_sf = true;
#else
bool const testing_sf = false;
#endif
// C++ "extensions"
namespace cppx {
using std::runtime_error;
using std::string;
inline bool hopefully( bool const c ) { return c; }
inline bool throw_x( string const& s ) { throw runtime_error( s ); }
} // namespace cppx
// C++ "portability perversions"
namespace cppp {
using cppx::hopefully;
using cppx::throw_x;
using std::numeric_limits;
namespace detail {
template< bool isTwosComplement >
int signed_from( unsigned const n )
{
if( n <= unsigned( numeric_limits<int>::max() ) )
{
return static_cast<int>( n );
}
unsigned const u_max = unsigned( -1 );
unsigned const u_half = u_max/2 + 1;
if( n == u_half )
{
throw_x( "signed_from: unsupported value (negative max)" );
}
int const i_quarter = static_cast<int>( u_half/2 );
int const int_n1 = static_cast<int>( n - u_half );
int const int_n2 = int_n1 - i_quarter;
int const int_n3 = int_n2 - i_quarter;
hopefully( n == static_cast<unsigned>( int_n3 ) )
|| throw_x( "signed_from: range error" );
return int_n3;
}
template<>
inline int signed_from<true>( unsigned const n )
{
return static_cast<int>( n );
}
} // namespace detail
inline int signed_from( unsigned const n )
{
bool const is_modulo = numeric_limits< int >::is_modulo;
return detail::signed_from< is_modulo && !testing_sf >( n );
}
} // namespace cppp
#include <iostream>
using namespace std;
int main()
{
int const x = cppp::signed_from( -42u );
wcout << x << endl;
}
EDIT: Fixed up code to avoid possible trap on non-modular-int machines (only one is known to exist, namely the archaically configured versions of the Unisys Clearpath). For simplicity this is done by not supporting the value -2n-1 where n is the number of int value bits, on such machine (i.e., on the Clearpath). in practice this value will not be supported by the machine either (i.e., with sign-and-magnitude or 1’s complement representation).
I think the int type is at least two bytes, so the INT_MIN and INT_MAX may change in different platforms.
Fundamental types
≤climits≥ header
My money is on using memcpy. Any decent compiler knows to optimise it away:
#include <stdio.h>
#include <memory.h>
#include <limits.h>
static inline int unsigned_to_signed(unsigned n)
{
int result;
memcpy( &result, &n, sizeof(result));
return result;
}
int main(int argc, const char * argv[])
{
unsigned int x = UINT_MAX - 1;
int xx = unsigned_to_signed(x);
return xx;
}
For me (Xcode 8.3.2, Apple LLVM 8.1, -O3), that produces:
_main: ## #main
Lfunc_begin0:
.loc 1 21 0 ## /Users/Someone/main.c:21:0
.cfi_startproc
## BB#0:
pushq %rbp
Ltmp0:
.cfi_def_cfa_offset 16
Ltmp1:
.cfi_offset %rbp, -16
movq %rsp, %rbp
Ltmp2:
.cfi_def_cfa_register %rbp
##DEBUG_VALUE: main:argc <- %EDI
##DEBUG_VALUE: main:argv <- %RSI
Ltmp3:
##DEBUG_VALUE: main:x <- 2147483646
##DEBUG_VALUE: main:xx <- 2147483646
.loc 1 24 5 prologue_end ## /Users/Someone/main.c:24:5
movl $-2, %eax
popq %rbp
retq
Ltmp4:
Lfunc_end0:
.cfi_endproc
Hi I am new in here so please let me know if anything is wrong and I will try to better the next time .
I am trying to understand how underflow and overflow works in C++ .My understanding is if a variable's range is exceeded it will start from the other end of the range . Thus if minimum of short is -32768 and if we do a -1 to it the new value should be SHRT_MAX .(32767)
Here is my code:
#include<iostream.h>
#include<limits.h>
#include<conio.h>
int main ( void )
{
int testpositive =INT_MIN ;
short testnegative = SHRT_MIN ;
cout<< SHRT_MIN<<"\n";
cout << testnegative-1<<"\n";
cout << INT_MIN << "\n";
cout << testpositive-1 << "\n";
cout<<testpositive-2;
getch();
return 0;
}
The exact behavior on overflow/underflow is only specified for unsigned types.
Unsigned integers shall obey the laws of arithmetic modulo 2^n where n is the number of bits in the value representation of that particular size of integer.
Source: Draft N3690 §3.9.1 sentence 4
This implies that unsigned arithmetic does not overflow because a result that cannot be represented by the resulting
unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the
resulting unsigned integer type.
Source: Draft N3690 Note 47 for §3.9.1
For normal signed integer types instead the C++ standard simply says than anything can happen.
If during the evaluation of an expression, the result is not mathematically defined or not in the range of representable values for its type, the behavior is undefined
Source: Draft N3690 §5 sentence 4
If we're talking about x86 processor (or most other modern processors) indeed the behavior is exactly what you describe and for the CPU there is no difference between a signed value or an unsigned value (there are signed and unsigned operations, but the value themselves are just bits).
Note that compilers can assume (and most modern optimizing compilers actually DO assume) that no signed integer overflow can occur in a correct program and for example in code like:
int do_something();
int do_something_else();
void foo() {
int x = do_something();
int y = x + 1;
if (x < y) {
do_something();
} else {
do_something_else();
}
}
a compiler is free to skip the test and the else branch in the generated code completely because in a valid program a signed int x is always less than x+1 (as signed overflow cannot be considered valid behavior).
If you replace int with unsigned int however the compiler must generate code for the test and for the else branch because for unsigned types it's possible that x > x+1.
For example clang compiles the code for foo to
foo(): # #foo()
push rax
call do_something()
pop rax
jmp do_something() # TAILCALL
where you can see that the ode just calls do_something twice (except for the strange handling of rax) and no mention of do_something_else is actually present. More or less the same code is generated by gcc.
Signed overflows are undefined behavior in C++.
For example:
INT_MIN - 1
-INT_MIN
are expressions that invoke undefined behavior.
SHRT_MIN - 1 and -SHRT_MIN are not undefined behavior in an environment with 16-bit short and 32-bit int because with integer promotions the operand is promoted to int first. In an environment with 16-bit short and int, these expressions are also undefined behavior.
Typically yes. But since this is C++, and C++ is regulated by the C++ standard, you must know that overflows are undefined behavior.
Although what you stated probably applies on most platforms, it's in no way guaranteed, so don't rely on it.
The new value need not be SHRT_MAX it is undefined.
I have an 8-character string representing a hexadecimal number and I need to convert it to an int. This conversion has to preserve the bit pattern for strings "80000000" and higher, i.e., those numbers should come out negative. Unfortunately, the naive solution:
int hex_str_to_int(const string hexStr)
{
stringstream strm;
strm << hex << hexStr;
unsigned int val = 0;
strm >> val;
return static_cast<int>(val);
}
doesn't work for my compiler if val > MAX_INT (the returned value is 0). Changing the type of val to int also results in a 0 for the larger numbers. I've tried several different solutions from various answers here on SO and haven't been successful yet.
Here's what I do know:
I'm using HP's C++ compiler on OpenVMS (using, I believe, an Itanium processor).
sizeof(int) will be at least 4 on every architecture my code will run on.
Casting from a number > INT_MAX to int is implementation-defined. On my machine, it usually results in a 0 but interestingly casting from long to int results in INT_MAX when the value is too big.
This is surprisingly difficult to do correctly, or at least it has been for me. Does anyone know of a portable solution to this?
Update:
Changing static_cast to reinterpret_cast results in a compiler error. A comment prompted me to try a C-style cast: return (int)val in the code above, and it worked. On this machine. Will that still be safe on other architectures?
Quoting the C++03 standard, §4.7/3 (Integral Conversions):
If the destination type is signed, the value is unchanged if it can be represented in the destination type (and bit-field width); otherwise, the value is implementation-defined.
Because the result is implementation-defined, by definition it is impossible for there to be a truly portable solution.
While there are ways to do this using casts and conversions, most rely on undefined behavior that happen to have well-defined behaviors on some machines / with some compilers. Instead of relying on undefined behavior, copy the data:
int signed_val;
std::memcpy (&signed_val, &val, sizeof(int));
return signed_val;
You can negate an unsigned twos-complement number by taking the complement and adding one. So let's do that for negatives:
if (val < 0x80000000) // positive values need no conversion
return val;
if (val == 0x80000000) // Complement-and-addition will overflow, so special case this
return -0x80000000; // aka INT_MIN
else
return -(int)(~val + 1);
This assumes that your ints are represented with 32-bit twos-complement representation (or have similar range). It does not rely on any undefined behavior related to signed integer overflow (note that the behavior of unsigned integer overflow is well-defined - although that should not happen here either!).
Note that if your ints are not 32-bit, things get more complex. You may need to use something like ~(~0U >> 1) instead of 0x80000000. Further, if your ints are no twos-complement, you may have overflow issues on certain values (for example, on a ones-complement machine, -0x80000000 cannot be represented in a 32-bit signed integer). However, non-twos-complement machines are very rare today, so this is unlikely to be a problem.
Here's another solution that worked for me:
if (val <= INT_MAX) {
return static_cast<int>(val);
}
else {
int ret = static_cast<int>(val & ~INT_MIN);
return ret | INT_MIN;
}
If I mask off the high bit, I avoid overflow when casting. I can then OR it back safely.
C++20 will have std::bit_cast that copies bits verbatim:
#include <bit>
#include <cassert>
#include <iostream>
int main()
{
int i = -42;
auto u = std::bit_cast<unsigned>(i);
// Prints 4294967254 on two's compliment platforms where int is 32 bits
std::cout << u << "\n";
auto roundtripped = std::bit_cast<int>(u);
assert(roundtripped == i);
std::cout << roundtripped << "\n"; // Prints -42
return 0;
}
cppreference shows an example of how one can implement their own bit_cast in terms of memcpy (under Notes).
While OpenVMS is not likely to gain C++20 support anytime soon, I hope this answer helps someone arriving at the same question via internet search.
unsigned int u = ~0U;
int s = *reinterpret_cast<int*>(&u); // -1
Сontrariwise:
int s = -1;
unsigned int u = *reinterpret_cast<unsigned int*>(&s); // all ones