bit shift - shifting a shifted value - c++

I have tried to make a longer function but it has acted very weird. I had tried to isolate the problem and I have been able to find the buggy part.
This program made for an Arduino but this phenomenon probably appear in other environment. I have tried to make a lot of searches but I cannot find the solution.
So, my buggy part:
Why do not these two codes give the same result?
How can I construct a one line function without extra variable
but same operation like "Code 1"?
Results:
Code 1: 0b00101100
Code 2: 0b01101100
Source codes:
Code 1: (correct operation but not one line)
#include <binary.h>
const byte value=B00110110;
byte buffer,result;
void setup(){
Serial.begin(115200);
buffer = (value << 3);
result = (buffer >> 2);
Serial.println(result,BIN);
}
void loop(){
}
It gives: 0b00101100
Code 2: (incorrect operation but one line)
#include <binary.h>
const byte value=B00110110;
byte result;
void setup(){
Serial.begin(115200);
result = ((value << 3) >> 2);
Serial.println(result,BIN);
}
void loop(){
}
It gives: 0b01101100

Remove the extra bits by masking with 0xFF:
result = (((value << 3) & 0xFF) >> 2);
You can also trim the upper three bits off after the sequence of shifts:
result = 0x1F & ((value << 3) >> 2);
According to the language specification, default integer promotions are applied to shift operands before shifting, and the result of the operation is the promoted type of the first operand.
5.8.1 The operands shall be of integral or enumerated type and integral promotions are performed. The type of the result is that of the promoted left operand.
If an integral type can fit in an int, then int is the result of the promotion. When you shift left, the most significant bits "spill" into the upper part of the int. Your first snippet cuts them off by assigning back to byte; you can achieve the same result by masking the result with 0xFF.
Link to ideone.

If byte is a uint8_t (thanks Mr. Fernandes!), the result of (value << 3) is wider than a byte. More precisely, value is promoted to a wider type before the shift occurs, and the result is that wider type. To get the behavior of Code 1, you can change Code 2 to do this:
result = (static_cast<byte>(value << 3) >> 2);
This mimics the intermediate assignment to buffer in Code 1 with a cast.

I assume that your byte is some unsigned integral type smaller than int.
Most arithmetic operations in C++ are performed in the domain of int type, after all "smaller" operands are implicitly converted (promoted) to type int.
For this reason, your first group of shifts is equivalent to
buffer = (byte) ((int) value << 3);
result = (byte) ((int) buffer >> 2);
while your second group if shifts is equivalent to
result = (byte) (((int) value << 3) >> 2);
In the first variant, any bits that get shifted outside the range of type byte by the << 3 shift are lost during intermediate conversion and assignment to buffer. In the second variant all calculations are performed within the range of int, so nothing is lost.
If you want your second variant to produce the same results as the first, you need to explicitly "truncate" the intermediate shift result
result = ((byte) (value << 3) >> 2);

Related

Left-shift bit operation for multiplying int-variable: Limited Range for multiplying. Arithmetic pattern after exceeding?

My actual concern is about this:
The left-shift bit operation is used to multiply values of integer variables quickly.
But an integer variable has a defined range of available integers it can store, which is obviously very logical due to the place in bytes which is reserved for it.
Depending on 16-bit or 32-bit system, it preserves either 2 or 4 bytes, which range the available integers from
-32,768 to 32,767 [for signed int] (2 bytes), or
0 to 65,535 [for unsigned int] (2 bytes) on 16-bit
OR
-2,147,483,648 to 2,147,483,647 [for signed int] (4 bytes), or
0 to 4,294,967,295 [for unsigned int] (4 bytes) on 32-bit
My thought is, it should´t be able to multiply the values over the exact half of the maximum integer of the according range.
But what happens then to the values if you proceed the bitwise operation after the value has reached the integer value of the half of the max int value?
Is there an arithmetic pattern which will be applied to it?
One example (in case of 32-bit system):
unsigned int redfox_1 = 2147483647;
unsigned int redfox_2;
redfox_2 = redfox_1 << 1;
/* Which value has redfox_2 now? */
redfox_2 = redfox_1 << 2;
/* Which value has redfox_2 now? */
redfox_2 = redfox_1 << 3;
/* Which value has redfox_2 now? */
/* And so on and on */
/* Is there a arithmetic pattern what will be applied to the value of redfox_2 now? */
the value stored inside redfox_2 shouldn´t be able to go over 2.147.483.647 because its datatype is unsigned int, which can handle only integers up to 4,294,967,295.
What will happen now with the value of redfox_2?
And Is there a arithmetic pattern in what will happen to the value of redfox_2?
Hope you can understand what i mean.
Thank you very much for any answers.
Per the C 2018 standard, 6.5.7 4:
The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are filled with zeros. If E1 has an unsigned type, the value of the result is E1 × 2E2, reduced modulo one more than the maximum value representable in the result type. If E1 has a signed type and nonnegative value, and E1 × 2E2 is representable in the result type, then that is the resulting value; otherwise, the behavior is undefined.
So, for unsigned integer types, the bits are merely shifted left, and vacated bit positions are filled with zeroes. For signed integer types, the consequences of overflow are not defined by the C standard.
Many C implementations will, in signed shifts, slavishly shift the bits, including shifting value bits into the sign bit, resulting in various positive or negative values that a naïve programmer might not expect. However, since the behavior is not defined by the C standard, a C implementation could also:
Clamp the result at INT_MAX or INT_MIN (for int, or the corresponding maxima for the particular type).
Shift the value bits without affecting the sign bit.
Generate a trap.
Transform the program, when the undefined shift is recognized during compilation and optimization, in arbitrary ways, such as removing the entire code path that performs the shift.
If you really want to see the pattern, then just write a program that prints it:
#include <iostream>
#include <ios>
#include <bitset>
int main()
{
unsigned int redfox = 2147483647;
std::bitset<32> b;
for (int i = 0; i < 32; ++i)
{
redfox = redfox << 1;
b = redfox;
std::cout << std::dec << redfox << ", " << std::hex << redfox << ", " << b << std::endl;
}
}
This produces:
4294967294, fffffffe, 11111111111111111111111111111110
4294967292, fffffffc, 11111111111111111111111111111100
4294967288, fffffff8, 11111111111111111111111111111000
4294967280, fffffff0, 11111111111111111111111111110000
4294967264, ffffffe0, 11111111111111111111111111100000
4294967232, ffffffc0, 11111111111111111111111111000000
4294967168, ffffff80, 11111111111111111111111110000000
4294967040, ffffff00, 11111111111111111111111100000000
4294966784, fffffe00, 11111111111111111111111000000000
4294966272, fffffc00, 11111111111111111111110000000000
4294965248, fffff800, 11111111111111111111100000000000
4294963200, fffff000, 11111111111111111111000000000000
4294959104, ffffe000, 11111111111111111110000000000000
4294950912, ffffc000, 11111111111111111100000000000000
4294934528, ffff8000, 11111111111111111000000000000000
4294901760, ffff0000, 11111111111111110000000000000000
4294836224, fffe0000, 11111111111111100000000000000000
4294705152, fffc0000, 11111111111111000000000000000000
4294443008, fff80000, 11111111111110000000000000000000
4293918720, fff00000, 11111111111100000000000000000000
4292870144, ffe00000, 11111111111000000000000000000000
4290772992, ffc00000, 11111111110000000000000000000000
4286578688, ff800000, 11111111100000000000000000000000
4278190080, ff000000, 11111111000000000000000000000000
4261412864, fe000000, 11111110000000000000000000000000
4227858432, fc000000, 11111100000000000000000000000000
4160749568, f8000000, 11111000000000000000000000000000
4026531840, f0000000, 11110000000000000000000000000000
3758096384, e0000000, 11100000000000000000000000000000
3221225472, c0000000, 11000000000000000000000000000000
2147483648, 80000000, 10000000000000000000000000000000
0, 0, 00000000000000000000000000000000

Unexpected output with left shift operator C++

Here is the code which is giving me the unexpected answer
#include<bits/stdc++.h>
using namespace std;
int main()
{
cout<<(1<<50);
}
The answer I get is 0.
But if I change the line to
cout<<pow(2, 50);
I get the right answer.
Could someone explain me the reason.
Assuming your compiler treats the constant 1 as a 32bit integer, you shifted it so far to the left, that only zeroes remain in the 32bit you have. 50 is larger than 32.
Try this (run it):
#include <iostream>
int main()
{
std::int64_t i { 1 }; // wide enough to allow 50 bits shift
std::cout << std::hex << ( i << 50 ); // should display 4000000000000
return 0;
}
From the C++ Standard (5.8 Shift operators)
1 The shift operators << and >> group left-to-right.
shift-expression:
additive-expression
shift-expression << additive-expression
shift-expression >> additive-expression
The operands shall be of integral or unscoped enumeration type and
integral promotions are performed. The type of the result is that of
the promoted left operand. The behavior is undefined if the right
operand is negative, or greater than or equal to the length in
bits of the promoted left operand.
Take into account that the behavior also will be undefined if the right operand is not greater or equal to the length in bits of the left operand but can touch the sign bit because the integer literal 1 has the type signed int.
As for this function call pow(2, 50) then there is used some algorithm that calculates the power.
You shift the "1" out of the 32 bit field, so zero is the result. Pow uses float representation where 2^50 can be handled.
EDIT
Without any modifications like "1LL" or "1ULL" (which generate long 64bit numbers), an integer number is usually handled as 32 bit on a x64 or x86 architectures. You can use
cout << (1ULL << 50);
or
cout << ((long long)1 << 50);
which should to it.
It's exactly what you're doing. You're shifting a single bit by 50 position in a portion of memory that's 32 bit... What's happening according to you? The bit goes somewhere else but it's not inside the memory portion of the integer anymore. pow(2, 50) performs a double casting, so you're not shifting bits anymore.
Also, never use #include<bits/stdc++.h>. It's not standard, and it's slow. You should use it only in precompiled headers but I'd avoid this also in that cases.
cout<<(1<<50);
Your code treats 1 as an int, so overflows. Instead, try:
cout << (1ULL << 50);

Why is the binary equivalent calculation getting incorrect?

I wrote the following program to output the binary equivalent of a integer taking(I checked that int on my system is of 4 bytes) it is of 4 bytes. But the output doesn't come the right. The code is:
#include<iostream>
#include<iomanip>
using namespace std;
void printBinary(int k){
for(int i = 0; i <= 31; i++){
if(k & ((1 << 31) >> i))
cout << "1";
else
cout << "0";
}
}
int main(){
printBinary(12);
}
Where am I getting it wrong?
The problem is in 1<<31. Because 231 cannot be represented with a 32-bit signed integer (range −231 to 231 − 1), the result is undefined [1].
The fix is easy: 1U<<31.
[1]: The behavior is implementation-defined since C++14.
This expression is incorrect:
if(k & ((1<<31)>>i))
int is a signed type, so when you shift 1 31 times, it becomes the sign bit on your system. After that, shifting the result right i times sign-extends the number, meaning that the top bits remain 1s. You end up with a sequence that looks like this:
80000000 // 10000...00
C0000000 // 11000...00
E0000000 // 11100...00
F0000000 // 11110...00
F8000000
FC000000
...
FFFFFFF8
FFFFFFFC
FFFFFFFE // 11111..10
FFFFFFFF // 11111..11
To fix this, replace the expression with 1 & (k>>(31-i)). This way you would avoid undefined behavior* resulting from shifting 1 to the sign bit position.
* C++14 changed the definition so that shifting 1 31 times to the left in a 32-bit int is no longer undefined (Thanks, Matt McNabb, for pointing this out).
A typical internal memory representation of a signed integer value looks like:
The most significant bit (first from the right) is the sign bit and in signed numbers(like int) it represents whether the number is negative or not.
When you shift additional bits sign extension is performed to preserve the number's sign. This is done by appending digits to the most significant side of the number.(following a procedure dependent on the particular signed number representation used).
In unsigned numbers the first bit from the right is just the MSB of the represented number, thus when you shift additional bits no sign extension is performed.
Note: the enumeration of the bits starts from 0, so 1 << 31 replaces your sign bit and after that every bit shift operation to the left >> results in sign extension. (as pointed out by #dasblinkenlight)
So, the simple solution to your problem is to make the number unsigned (this is what U does in 1U << 31) before you start the bit manipulation. (as pointed out by #Yu Hao)
For further reading see signed number representations and two's complement.(as it's the most common)

2 chars to short using bit-shifting results in a weird result with large values

I have an std::string that contains the response of a server. After parsing the string a bit, I come across a short. The short is big-endian and is stored in the string accordingly:
raw[0] == 0xa5;
raw[1] == 0x69;
I know this as
file << raw[0] << std::endl << raw[1];
when viewed as hex results to "0xa5 0x0a 0x69".
I write them to a short like this and then write to a file as inspired by https://stackoverflow.com/a/300837/1318909:
short x = (raw[1] << 8) | raw[0];
file << std::to_string(x);
expected result? 27045 (0x69a5).
actual result? -91 (0xffa5) <-- overflow!.
Why is this? I tested and it worked fine with a value of 2402. I also did some additional testing and it works with
short x = (raw[1] << 8) | 0xa5;
but not with
short x = (0x69 << 8) | raw[0];
char may (or may not) be signed, and it looks like it is on your platform. That means that 0xa5 is sign extended to 0xffa5 before you or in the upper byte.
Try casting each byte to unsigned char before doing your bit manipulation.
raw[0] is an object of type char, which happens to be 8-bit signed integer type on your platform. When you stuff 0xA5 value into such signed char object, it actually acquires value of -91. When raw[0] is used as an operand of | operator, it is subjected to usual arithmetic conversions. The latter convert it to value of type int with value -91 (0xFFA5). That FF in the higher byte if what causes the observed result.

Right shift with zeros at the beginning

I'm trying to do a kind of left shift that would add zeros at the beginning instead of ones. For example, if I left shift 0xff, I get this:
0xff << 3 = 11111000
However, if I right shift it, I get this:
0xff >> 3 = 11111111
Is there any operation I could use to get the equivalent of a left shift? i.e. I would like to get this:
00011111
Any suggestion?
Edit
To answer the comments, here is the code I'm using:
int number = ~0;
number = number << 4;
std::cout << std::hex << number << std::endl;
number = ~0;
number = number >> 4;
std::cout << std::hex << number << std::endl;
output:
fffffff0
ffffffff
Since it seems that in general it should work, I'm interested as to why this specific code doesn't. Any idea?
This is how C and binary arithmetic both work:
If you left shift 0xff << 3, you get binary: 00000000 11111111 << 3 = 00000111 11111000
If you right shift 0xff >> 3, you get binary: 00000000 11111111 >> 3 = 00000000 00011111
0xff is a (signed) int with the positive value 255. Since it is positive, the outcome of shifting it is well-defined behavior in both C and C++. It will not do any arithmetic shifts nor any kind or poorly-defined behavior.
#include <stdio.h>
int main()
{
printf("%.4X %d\n", 0xff << 3, 0xff << 3);
printf("%.4X %d\n", 0xff >> 3, 0xff >> 3);
}
Output:
07F8 2040
001F 31
So you are doing something strange in your program because it doesn't work as expected. Perhaps you are using char variables or C++ character literals.
Source: ISO 9899:2011 6.5.7.
EDIT after question update
int number = ~0; gives you a negative number equivalent to -1, assuming two's complement.
number = number << 4; invokes undefined behavior, since you left shift a negative number. The program implements undefined behavior correctly, since it either does something or nothing at all. It may print fffffff0 or it may print a pink elephant, or it may format the hard drive.
number = number >> 4; invokes implementation-defined behavior. In your case, your compiler preserves the sign bit. This is known as arithmetic shift, and arithmetic right shift works in such a way that the MSB is filled with whatever bit value it had before the shift. So if you have a negative number, you will experience that the program is "shifting in ones".
In 99% of all real world cases, it doesn't make sense to use bitwise operators on signed numbers. Therefore, always ensure that you are using unsigned numbers, and that none of the dangerous implicit conversion rules in C/C++ transforms them into signed numbers (for more info about dangerous conversions, see "the integer promotion rules" and "the usual arithmetic conversions", plenty of good info about those on SO).
EDIT 2, some info from the C99 standard's rationale document V5.10:
6.5.7 Bitwise shift operators
The description of shift operators in K&R suggests that shifting by a
long count should force the left operand to be widened to long before
being shifted. A more intuitive practice, endorsed by the C89
Committee, is that the type of the shift count has no bearing on the
type of the result.
QUIET CHANGE IN C89
Shifting by a long count no longer coerces the shifted operand to
long. The C89 Committee affirmed the freedom in implementation granted
by K&R in not requiring the signed right shift operation to sign
extend, since such a requirement might slow down fast code and since
the usefulness of sign extended shifts is marginal. (Shifting a
negative two’s complement integer arithmetically right one place is
not the same as dividing by two!)
If you explicitly shift 0xff it works as you expected
cout << (0xff >> 3) << endl; // 31
It should be possible only if 0xff is in type of signed width 8 (char and signed char on popular platforms).
So, in common case:
You need to use unsigned ints
(unsigned type)0xff
right shift works as division by 2(with rounding down, if I understand correctly).
So when you have 1 as first bit, you have negative value and after division it's negative again.
The two kinds of right shift you're talking about are called Logical Shift and Arithmetic Shift. C and C++ use logical shift for unsigned integers and most compilers will use arithmetic shift for a signed integer but this is not guaranteed by the standard meaning that the value of right shifting a negative signed int is implementation defined.
Since you want a logical shift you need to switch to using an unsigned integer. You can do this by replacing your constant with 0xffU.
To explain your real code you just need the C++ versions of the quotes from the C standard that Lundin gave in comments:
int number = ~0;
number = number << 4;
Undefined behavior. [expr.shift] says
The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated
bits are zero-filled. If E1 has an unsigned type, the value of the
result is E1 × 2E2, reduced modulo one more than the maximum value
representable in the result type. Otherwise, if E1 has a signed type
and non-negative value, and E1×2E2 is representable in the result
type, then that is the resulting value; otherwise, the behavior is
undefined.
number = ~0;
number = number >> 4;
Implementation-defined result, in this case your implementation gave you an arithmetic shift:
The value of E1 >> E2 is E1 right-shifted E2 bit positions. If E1 has
an unsigned type or if E1 has a signed type and a non-negative value,
the value of the result is the integral part of the quotient of
E1/2E2. If E1 has a signed type and a negative value, the resulting
value is implementation-defined
You should use an unsigned type:
unsigned int number = -1;
number = number >> 4;
std::cout << std::hex << number << std::endl;
Output:
0x0fffffff
To add my 5 cents worth here...
I'm facing exactly the same problem as this.lau! I've done some perfunctory research on this and these are my results:
typedef unsigned int Uint;
#define U31 0x7FFFFFFF
#define U32 0xFFFFFFFF
printf ("U31 right shifted: 0x%08x\n", (U31 >> 30));
printf ("U32 right shifted: 0x%08x\n", (U32 >> 30));
Output:
U31 right shifted: 0x00000001 (expected)
U32 right shifted: 0xffffffff (not expected)
It would appear (in the absence of anyone with detailed knowledge) that the C compiler in XCode for Mac OS X v5.0.1 reserves the MSB as a carry bit that gets pulled along with each shift.
Somewhat annoyingly, the converse is NOT true:-
#define ST00 0x00000001
#define ST01 0x00000002
printf ("ST00 left shifted: 0x%08x\n", (ST00 << 30));
printf ("ST01 left shifted: 0x%08x\n", (ST01 << 30));
Output:
ST00 left shifted: 0x40000000
ST01 left shifted: 0x80000000
I concur completely with the people above that assert that the sign of the operand has no bearing on the behaviour of the shift operator.
Can anyone shed any light on the specification for the Posix4 implementation of C? I feel a definitive answer may rest there.
In the meantime, it appears that the only workaround is a construct along the following lines;-
#define CARD2UNIVERSE(c) (((c) == 32) ? 0xFFFFFFFF : (U31 >> (31 - (c))))
This works - exasperating but necessary.
Just in case if you want the first bit of negative number to be 0 after right shift what we can do is to take the XOR of that negative number with INT_MIN that will make its msb zero, I understand that its not appropriate arithmetic shift but will get work done