I ran into a bug in my program:
for (int i = 0; i < objArray.size() - 1; ++i)
In my case objArray.size() is an unsigned long long and an empty vector minus 1 equals about 18 quintillion. I was wondering, does the loop on every iteration have to cast an int to an unsigned long long? I checked the assembly and while using an int creates different code than size_t without optimisations, with -O2 specified it generates exactly the same assembly. Does this mean it's not implicitly casting?
I don't understand assembly, but the code it generated was:
test rcx, rcx
je .L32
add rdx, rax
and then :
cmp rdx, rax
jne .L28
This may be caused by a compiler optimization. The c++ standard says that overflow of signed integral types is undefined. In this case, i starts at 0. Supposing that i is not written to in the loop, the compiler can thus deduce, that i >= 0, since overflowing is undefined behaviour and can be pruned.
Normally, for an signed-unsigned comparison, the signed value would have to be converted to the unsigned type following the rules you can see here. These rules are the reason for the compiler warnings when comparing a signed and unsigned type (leading to confusion, e.g. -1 > 2U
is true). In this case, that doesn't matter though.
With the assumption i >= 0 and 2-complement signed types though, the compiler can safely reinterpret i as an unsigned long long since he knows the sign-bit is 0. That's what your assemly output shows.
Now, we can see that there is indeed a bug. Suppose objArray.size() - 1 does not fit into a positive signed int. This would cause i to overflow, thus cause undefined behaviour which is always bad news.
Lets dissect the code
for (int i = 0; i < objArray.size() - 1; ++i)
you are doing a comparison between and int and a size_t. The size( )-1 is an unsigned underflow when the array is empty and results in a value of std::numeric_limits::max( ). The comparison, will be signed/unsigned and use the type promotion rules as outlined here Signed/unsigned comparisons
Related
This question already has answers here:
Do C99 signed integer types defined in stdint.h exhibit well-defined behaviour in case of an overflow?
(2 answers)
Closed 3 years ago.
The below code works as expected on Windows but when built with Clang 6.0 and running on an Ubuntu server it does not work. The CurAnimIndex is an int32 and has the value 2147483647 (max int). I would expect it to enter the branch since the value of CurAnimIndex after the increment should be a negative number, however it does not.
CurAnimIndex++;
if (CurAnimIndex >= AnimSelectorDatas.Num() || CurAnimIndex < 0)
{
CurAnimIndex = 0;
}
0x000000000411a12f mov 0x0(%r13),%eax
0x000000000411a133 lea 0x1(%rax),%ecx
0x000000000411a136 movslq 0x10(%r13),%r15
0x000000000411a13a xor %ebp,%ebp
0x000000000411a13c cmp %r15d,%ecx
0x000000000411a13f cmovge %ebp,%ecx
0x000000000411a142 cmp $0xffffffff,%eax
0x000000000411a145 cmovl %ebp,%ecx
0x000000000411a148 mov %ecx,0x0(%r13)
0x000000000411a14c mov 0x8(%r13),%r12 enter code here
CurAnimIndex++
The CurAnimIndex is an int32 and has the value 2147483647 (max int). I would expect it to enter the branch since the value of CurAnimIndex after the increment should be a negative number
2147483647 is a positive number. Why would you expect that incrementing a positive number would yield a negative one? That doesn't happen in normal arithmetic. The compiler knows this and optimises according to that knowledge. If the initial value of CurAnimIndex has been proven to be at least -1, then the check CurAnimIndex < 0 is known always to be false and can be optimised away.
Maybe your expectation is related to the fact that the operation overflows the maximum representable value. That expectation is misplaced because signed overflow isn't guaranteed to have such behaviour. In fact, signed overflow isn't guaranteed to have any particular behaviour. The behaviour of the program is undefined.
A correct way to do this is to first check whether the number is equal to maximum representable value and only increment if it isn't.
For some legacy reasons I have code that casts a double to unsigned byte and we are seeing much difference between the two platforms. Short of doing - "don't try to stuff a signed value into unsigned value; don't try to stuff a double value in integer", is there something else that can be done?
unsigned char newR1 = -40.16;
Value of newR1 is 216 on windows (as we expected from a long time); but on ARM64 it is 0.
Disassembly on Win:
00007FF75E388818 cvttsd2si eax,mmword ptr [R]
00007FF75E38881D mov byte ptr [newR1],al
On ARM64
00007FF6F9E800DC ldr d16,[sp,#0x38 |#0x38 ]
00007FF6F9E800E0 fcvtzu w8,d16
00007FF6F9E800E4 uxtb w8,w8
00007FF6F9E800E8 strb w8,[sp,#0x43 |#0x43 ]
Will try these as well, but just wanted some other opinions
unsigned char newR1 = -40.16;
unsigned char newR2 = (int)-40.16;
unsigned char newR3 = (unsigned char)-40.16;
unsigned char newR4 = static_cast<int>(-40.16);
or may be
int i = -40.16;
unsigned char c = i;
What the C standard says (and there's similar text in the C++ one):
When a finite value of real floating type is converted to an integer
type other than _Bool, the fractional part is discarded (i.e., the
value is truncated toward zero). If the value of the integral part
cannot be represented by the integer type, the behavior is undefined.
So, getting 216 out of -40.16 with a single cast from double to unsigned char is already UB. In fact, getting any result in this case is UB. Which is why the compiler is free to produce anything and not 216 that you desire.
You may want to do two casts:
(unsigned char)(int)-40.16
Again, the first cast (to int) is still subject to the above restriction I quoted.
I'm trying t understand a problem we cleared recently when using Clang 5.0 and Undefined Behavior Sanitizer (UBsan). We have code that processes a buffer in the forward or backwards direction. The reduced case is similar to the code shown below.
The 0-len may look a little unusual, but it is needed for early Microsoft .Net compilers. Clang 5.0 and UBsan produced integer overflow findings:
adv-simd.h:1138:26: runtime error: addition of unsigned offset to 0x000003f78cf0 overflowed to 0x000003f78ce0
adv-simd.h:1140:26: runtime error: addition of unsigned offset to 0x000003f78ce0 overflowed to 0x000003f78cd0
adv-simd.h:1142:26: runtime error: addition of unsigned offset to 0x000003f78cd0 overflowed to 0x000003f78cc0
...
Lines 1138, 1140, 1142 (and friends) are the increment, which may
stride backwards due to the 0-len.
ptr += inc;
According to Pointer comparisons in C. Are they signed or unsigned? (which also discusses C++), pointers are neither signed nor unsigned. Our offsets were unsigned and we relied on unsigned integer wrap to achieve the reverse stride.
The code was fine under GCC UBsan and Clang 4 and earlier UBsan. We eventually cleared it for Clang 5.0 with help with the LLVM devs. Instead of size_t we needed to use ptrdiff_t.
My question is, where was the integer overflow/undefined behavior in the construction? How did ptr + <unsigned> result in signed integer overflow and lead to undefined behavior?
Here is an MSVC that mirrors the real code.
#include <cstddef>
#include <cstdint>
using namespace std;
uint8_t buffer[64];
int main(int argc, char* argv[])
{
uint8_t * ptr = buffer;
size_t len = sizeof(buffer);
size_t inc = 16;
// This sets up processing the buffer in reverse.
// A flag controls it in the real code.
if (argc%2 == 1)
{
ptr += len - inc;
inc = 0-inc;
}
while (len > 16)
{
// process blocks
ptr += inc;
len -= 16;
}
return 0;
}
The definition of adding an integer to a pointer is (N4659 expr.add/4):
I used an image here in order to preserve the formatting (this will be discussed below).
Note that this is a new wording which replaces a less-clear description from previous standards.
In your code (when argc is odd) we end up with code equivalent to:
uint8_t buffer[64];
uint8_t *ptr = buffer + 48;
ptr = ptr + (SIZE_MAX - 15);
For the variables in the standard quote applied to your code, i is 48 and j is (SIZE_MAX - 15) and n is 64.
The question now is whether it is true that 0 ≤ i + j ≤ n. If we interpret "i + j" to mean the result of the expression i + j, then that equals 32 which is less than n. But if it means the mathematical result then it is much greater than n.
The standard uses a font for mathematical equations here and does not use the font for source code. ≤ is not a valid operator either. So I think they intend this equation to describe the mathematical value i.e. this is undefined behaviour.
The C standard defines type ptrdiff_t as being the type yielded by the pointer-difference operator. It would be possible for a system to have a 32-bit size_t and a 64-bit ptrdiff_t; such definitions would be a natural fit for a system which used 64-bit linear or quasi-linear pointers but did required individual objects to be less than 4GiB each.
If objects are known to be less than 2GiB each, storing values of type ptrdiff_t rather than size_t might make the program needlessly inefficient. In such a scenario, however, code should not use size_t to hold pointer differences that might be negative, but instead use int32_t [which will be large enough if objects are less than 2GiB each]. Even if ptrdiff_t is 64 bits, a value of type int32_t will be properly sign-extended before it is added or subtracted from any pointers.
This question is a result of another SO question.
Example Code
#include <iostream>
int main()
{
unsigned long b = 35000000;
int i = 100;
int j = 30000000;
unsigned long n = ( i * j ) / b; // #1
unsigned long m = ( 100 * 30000000 ) / b; // #2
std::cout << n << std::endl;
std::cout << m << std::endl;
}
Output
85 85
Compiling this code with g++ -std=c++11 -Wall -pedantic -O0 -Wextra gives the following warning:
9:28: warning: integer overflow in expression [-Woverflow]
Questions
Am I correct in thinking that #1 and #2 invoke undefined behavior because the intermediate result 100 * 30000000 does not fit into an int? Or is the output I am seeing well-defined?
Why do I only get a warning with #2?
Yes, it is undefined behaviour, and the result you get is usually¹ different if unsigned long is a 64-bit type.
¹ It's UB, so there are no guarantees.
Intermediate result
Yes, this is undefined behaviour. What if you just stopped right there and return m? The compiler needs to get from point A to point B, and you've told it to do it by making that calculation (which isn't possible). A compiler may choose to optimize this statement in such a way that you don't get an overflow, but as far as I know, the standard doesn't require the optimizer to do anything.
Why no error when they're variables?
You're explicitly telling gcc not to optimize at all (-O0), so my assumption is that it doesn't know the values of i and j at that point. Normally you'd learn the values because of constant folding, but like I said, you told it not to optimize.
If you re-run this and it still doesn't mention it, there's also the possibility that this warning is generated before the optimizer runs, so it's just not smart enough to do constant folding at all for this step.
1) Yes, it's undefined behavior.
2) Because #1 involves variables (not constants), so the compiler in general doesn't know whether it will overflow (although in this case it does, and I don't know why it doesn't warn).
You get a warning with two, because the compiler knows the values in the operand. The outputs are right because both use /b which is unsigned long. The temporary value to be divisible by b must be hold greater or equal datatype range, ( i * j ) or ( 100 * 30000000 ) are stored in a CPU register that has the same datatype range of the value to be divided, if b was an int the temporary result would be a int, since b is an ulong, int can't be divided by ulong, the temporary value is stored to an ulong.
It is undefined behavior if it overflows, but it's not overflowing in those cases
A program with the same structure, only changing b to int will have only two lines on .s code.
cltd
idivl (%ecx)
to b = int
movl $0,
%edx divl (%ecx)
to b = unsigned long,
idivl performs signed division, storing the value as signed
divl performs unsigned division, storing the value as unsigned
So you're right, the operation does overflows, the output is correct because of the division operation.
What is the difference of idivl and divl?
https://stackoverflow.com/a/12488534/1513286
As for 5/4 the result is undefined behavior.
However note that if you changed the types to unsigned (for the constants just add the u suffix) not only the values do fit, but according to 3.9.1/4 the arithmetic becomes a modulo arithmetic and the result is perfectly defined even for larger intermediate values that do not fit the type.
Hi I am new in here so please let me know if anything is wrong and I will try to better the next time .
I am trying to understand how underflow and overflow works in C++ .My understanding is if a variable's range is exceeded it will start from the other end of the range . Thus if minimum of short is -32768 and if we do a -1 to it the new value should be SHRT_MAX .(32767)
Here is my code:
#include<iostream.h>
#include<limits.h>
#include<conio.h>
int main ( void )
{
int testpositive =INT_MIN ;
short testnegative = SHRT_MIN ;
cout<< SHRT_MIN<<"\n";
cout << testnegative-1<<"\n";
cout << INT_MIN << "\n";
cout << testpositive-1 << "\n";
cout<<testpositive-2;
getch();
return 0;
}
The exact behavior on overflow/underflow is only specified for unsigned types.
Unsigned integers shall obey the laws of arithmetic modulo 2^n where n is the number of bits in the value representation of that particular size of integer.
Source: Draft N3690 §3.9.1 sentence 4
This implies that unsigned arithmetic does not overflow because a result that cannot be represented by the resulting
unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the
resulting unsigned integer type.
Source: Draft N3690 Note 47 for §3.9.1
For normal signed integer types instead the C++ standard simply says than anything can happen.
If during the evaluation of an expression, the result is not mathematically defined or not in the range of representable values for its type, the behavior is undefined
Source: Draft N3690 §5 sentence 4
If we're talking about x86 processor (or most other modern processors) indeed the behavior is exactly what you describe and for the CPU there is no difference between a signed value or an unsigned value (there are signed and unsigned operations, but the value themselves are just bits).
Note that compilers can assume (and most modern optimizing compilers actually DO assume) that no signed integer overflow can occur in a correct program and for example in code like:
int do_something();
int do_something_else();
void foo() {
int x = do_something();
int y = x + 1;
if (x < y) {
do_something();
} else {
do_something_else();
}
}
a compiler is free to skip the test and the else branch in the generated code completely because in a valid program a signed int x is always less than x+1 (as signed overflow cannot be considered valid behavior).
If you replace int with unsigned int however the compiler must generate code for the test and for the else branch because for unsigned types it's possible that x > x+1.
For example clang compiles the code for foo to
foo(): # #foo()
push rax
call do_something()
pop rax
jmp do_something() # TAILCALL
where you can see that the ode just calls do_something twice (except for the strange handling of rax) and no mention of do_something_else is actually present. More or less the same code is generated by gcc.
Signed overflows are undefined behavior in C++.
For example:
INT_MIN - 1
-INT_MIN
are expressions that invoke undefined behavior.
SHRT_MIN - 1 and -SHRT_MIN are not undefined behavior in an environment with 16-bit short and 32-bit int because with integer promotions the operand is promoted to int first. In an environment with 16-bit short and int, these expressions are also undefined behavior.
Typically yes. But since this is C++, and C++ is regulated by the C++ standard, you must know that overflows are undefined behavior.
Although what you stated probably applies on most platforms, it's in no way guaranteed, so don't rely on it.
The new value need not be SHRT_MAX it is undefined.