Expanding packed nibbles to 5-bit groups - bit-manipulation

I currently have an un unsigned int of 64 bits that contains:
0100
0100
0100
0100
0000...
And i would change it to :
01000
01000
01000
01000
00000...
Is there a way to to do that ?
Thanks

📎 Hi! It looks like you are trying to expand 4-bit nibbles into 5-bit groups.
In general, you can do it like this
uint64_t value = YOUR_DATA; //this has your bits.
for (int i; i< sizeof(value)*2; i++) {
uint8_t nibble = (value & 0xF);
nibble <<= 1; //shift left 1 bit, add 0 to end.
STORE(nibble, i);
value>>=4; //advance to next nibble
}
This will call STORE once for each group of 4 bits. The arguments to STORE are the "expanded" 5 bit value, and the nibble counter, where 0 represents the least significant 4 bits.
The design question to answer is how to store the result? 64 bits / 4 * 5 = 80 bits, so you either need 2 words, or to throw away the data at one end.
Assuming 2 words with the anchor at the LSB, STORE could look like
static uint64_t result[2] = {0,0};
void STORE(uint64_t result[], uint8_t value, int n) {
int idx = (n>12); //which result word?
result[idx] |= value << ( n*5 - idx*64 );
if (n==12) result[1] |= value>>4; //65th bit goes into 2nd result word
}

Omit the leading 0, it serves no purpose => shift left one bit

Related

I need to understand the logic behind this cpp code

int x = 25;
unsigned int g = x & 0x80000000;
how did this code read the most significant bit of in the address of x? does the reference to 0x80000000, or binary 1000 0000 0000 0000 accomplished that task, or was it something else?
For char the most significant bit is typically the sign bit as per Two's Complement, so this should be:
char x = 25;
unsigned int msb = x & (1 << 6);
Where (1 << 6) means the 6 bit, counting from 0, or the 7th counting from 1st. It's the second-to-top bit and equivalent to 0x40.
Since 25 is 0b00011001 you won't get a bit set. You'll need a value >= 64.

Isolating bits and flattening them [duplicate]

This question already has an answer here:
I want to pack the bits based on arbitrary mask
(1 answer)
Closed 5 years ago.
Problem
Suppose I have a bit mask mask and an input n, such as
mask = 0x10f3 (0001 0000 1111 0011)
n = 0xda4d (1101 1010 0100 1101)
I want to 1) isolate the masked bits (remove bits from n not in mask)
masked_n = 0x10f3 & 0xda4d = 0x1041 (0001 0000 0100 0001)
and 2) "flatten" them (get rid of the zero bits in mask and apply those same shifts to masked_n)?
flattened_mask = 0x007f (0000 0000 0111 1111)
bits to discard (___1 ____ 0100 __01)
first shift ( __ _1__ __01 0001)
second shift ( __ _101 0001)
result = 0x0051 (0000 0000 0101 0001)
Tried solutions
a) For this case, one could craft an ad hoc series of bit shifts:
result = (n & 0b10) | (n & 0b11110000) >> 2 | (n & 0b1000000000000) >> 6
b) More generically, one could also iterate over each bit of mask and calculate result one bit at a time.
for (auto i = 0, pos = 0; i < 16; i++) {
if (mask & (1<<i)) {
if (n & (1<<i)) {
result |= (1<<pos);
}
pos++;
}
}
Question
Is there a more efficient way of doing this generically, or at the very least, ad hoc but with a fixed number of operations regardless of bit placement?
A more efficient generic approach would be to loop over the bits but only process the number of bits that are in the mask, removing the if (mask & (1<<i)) test from your loop and looping only 7 times instead of 16 for your example mask. In each iteration of the loop find the rightmost bit of the mask, test it with n, set the corresponding bit in the result and then remove it from the mask.
int mask = 0x10f3;
int n = 0xda4d;
int result = 0;
int m = mask, pos = 1;
while(m != 0)
{
// find rightmost bit in m:
int bit = m & -m;
if (n & bit)
result |= pos;
pos <<= 1;
m &= ~bit; // remove the rightmost bit from m
}
printf("%04x %04x %04x\n", mask, n, result);
Output:
10f3 da4d 0051
Or, perhaps less readably but without the bit temp variable:
if (n & -m & m)
result |= pos;
pos <<= 1;
m &= m-1;
How does it work? First, consider why m &= m-1 clears the rightmost (least significant) set bit. Your (non-zero) mask m is going to be made up of a certain number of bits, then a 1 in the least significant set place, then zero or more 0s:
e.g:
xxxxxxxxxxxx1000
Subtracting 1 gives:
xxxxxxxxxxxx0111
So all the bits higher than the least significant set bit will be unchanged (so ANDing them together leaves them unchanged), the least significant set bit changes from a 1 to a 0, and the less significant bits were all 0s beforehand so ANDing them with all 1s leaves them unchanged. Net result: least significant set bit is cleared and the rest of the word stays the same.
To understand why m & -m gives the least significant set bit, combine the above with the knowledge that in 2s complement, -x = ~(x-1)

Fast way to get the decimal value of certain bits from a bitset

I have a variable mask of type std::bitset<8> as
std::string bit_string = "00101100";
std::bitset<8> mask(bit_string);
Is there an efficient way to quickly mask out the corresponding (three) bits of another given std::bitset<8> input and move all those masked out bits to the rightmost? E.g., if input is 10100101, then I would like to quickly get 00000101 which equals 5 in decimal. Then I can vect[5] to quickly index the 6th element of vect which is std::vector<int> of size 8.
Or rather, can I quickly get the decimal value of the masked out bits (with their relative positions retained)? Or I can't?
I guess in my case the advantage that can be taken is the bitset<8> mask I have. And I'm supposed to manipulate it somehow to do the work fast.
I see it like this (added by Spektre):
mask 00101100b
input 10100101b
---------------
& ??1?01??b
>> 101b
5
First things first: you can't avoid O(n) complexity with n being the number of mask bits if your mask is available as binary. However, if your mask is constant for multiple inputs, you can preprocess the mask into a series of m mask&shift transformations where m is less or equal to your number of value 1 mask bits. If you know the mask at compile time, you can even preconstruct the transformations and then you get your O(m).
To apply this idea, you need to create a sub-mask for each group of 1 bits in your mask and combine it with a shift information. The shift information is constructed by counting the number of zeroes to the right of the current group.
Example:
mask = 00101100b
// first group of ones
submask1 = 00001100b
// number of zeroes to the right of the group
subshift1 = 2
submask2 = 00100000b
subshift2 = 3
// Apply:
input = 10100101b
transformed = (input & submask1) >> subshift1 // = 00000001b
transformed = (input & submask2) >> subshift2 // = 00000100b
+ transformed // = 00000101b
If you make the sub-transforms into an array, you can easily apply them in a loop.
Your domain is small enough that you can brute-force this. Trivially, an unsigned char LUT[256][256] can store all possible outcomes in just 64 KB.
I understand that the mask has at most 3 bits, so you can restrict the lookup table size in that dimension to [224]. And since f(input, mask) == f(input&mask, mask) you can in fact reduce the LUT to unsigned char[224][224].
A further size reduction is possible by realizing that the highest mask is 11100000 but you can just test the lowest bit of the mask. When mask is even, f(input, mask) == f((input&mask)/2, mask/2). The highest odd mask is only 11000001 or 191. This reduces your LUT further, to [192][192].
A more space-efficient algorithm splits input and mask into 2 nibbles (4 bits). You now have a very simple LUT[16][16] in which you look up the high and low parts:
int himask = mask >> 4, lomask = mask & 0xF;
int hiinp = input >> 4, loinp = input & 0xF;
unsigned char hiout = LUT[himask][hiinp];
unsigned char loout = LUT[lomask][loinp];
return hiout << bitsIn[lomask] | loout;
This shows that you need another table, char bitsIn[15].
Taking the example :
mask 0010 1100b
input 1010 0101b
himask = 0010
hiinp = 1010
hiout = 0001
lomask = 1100
loinp = 0101
loout = 0001
bitsIn[lowmask 1100] = 2
return (0001 << 2) | (0001)
Note that this generalizes fairly easily to more than 8 bits:
int bitsSoFar = 0;
int retval = 0;
while(mask) { // Until we've looked up all bits.
int mask4 = mask & 0xF;
int input4 = input & 0xF;
retval |= LUT[mask4][input4] << bitsSoFar;
bitsSoFar += bitsIn[mask4];
mask >>= 4;
input >>= 4;
}
Since this LUT only hold nibbles, you could reduce it to 16*16/2 bytes, but I suspect that's not worth the effort.
I see it like this:
mask 00101100b
input 10100101b
---------------
& ??1?01??b
>> 101b
5
I would create a bit weight table for each set bit in mask by scan bits from LSB and add weights 1,2,4,8,16... for set bits and leave zero for the rest so:
MSB LSB
--------------------------
mask 0 0 1 0 1 1 0 0 bin
--------------------------
weight 0 0 4 0 2 1 0 0 dec (A)
input 1 0 1 0 0 1 0 1 bin (B)
--------------------------
(A.B) 0*1+0*0+4*1+0*0+2*0+1*1+0*0+0*1 // this is dot product ...
4 + 1
--------------------------
5 dec
--------------------------
Sorry I do not code in Python at all so no code ... I still think using integral types for this directly would be better but that is probably just my low level C++ thinking ...

How to set specific bits?

Let's say I've got a uint16_t variable where I must set specific bits.
Example:
uint16_t field = 0;
That would mean the bits are all zero: 0000 0000 0000 0000
Now I get some values that I need to set at specific positions.
val1=1; val2=2, val3=0, val4=4, val5=0;
The structure how to set the bits is the following
0|000| 0000| 0000 000|0
val1 should be set at the first bit on the left. so its only one or zero.
val2 should be set at the next three bits. val3 on the next four bits. val4 on the next seven bits and val5 one the last bit.
The result would be this:
1010 0000 0000 1000
I only found out how to the one specific bit but not 'groups'. (shift or bitset)
Does anyone have an idea how to solve this issue?
There are (at least) two basic approaches. One would be to create a struct with some bitfields:
struct bits {
unsigned a : 1;
unsigned b : 7;
unsigned c : 4;
unsigned d : 3;
unsigned e : 1;
};
bits b;
b.a = val1;
b.b = val2;
b.c = val3;
b.d = val4;
b.e = val5;
To get the 16-bit value, you could (for one example) create a union of that struct with a uint16_t. Just one minor problem: the standard doesn't guarantee what order the bit fields will end up in when you look at the 16-bit value. Just for example, you might need to reverse the order I've given above to get the order from most to least significant bits that you really want (but changing compilers might muck things up again).
The other obvious possibility would be to use shifting and masking to put the pieces together into a number:
int16_t result = val1 | (val2 << 1) | (val3 << 8) | (val4 << 12) | (val5 << 15);
For the moment, I've assumed each of the inputs starts out in the correct range (i.e., has a value that can be represented in the chosen number of bits). If there's a possibility that could be wrong, you'd want to mask it to the correct number of bits first. The usual way to do that is something like:
uint16_t result = input & ((1 << num_bits) - 1);
In case you're curious about the math there, it works like this. Lets's assume we want to ensure an input fits in 4 bits. Shifting 1 left 4 bits produces 00010000 (in binary). Subtracting one from that then clears the one bit that's set, and sets all the less significant bits than that, giving 00001111 for our example. That gives us the first least significant bits set. When we do a bit-wise AND between that and the input, any higher bits that were set in the input are cleared in the result.
One of the solutions would be to set a K-bit value starting at the N-th bit of field as:
uint16_t value_mask = ((1<<K)-1) << N; // for K=4 and N=3 will be 00..01111000
field = field & ~value_mask; // zeroing according bits inside the field
field = field | ((value << N) & value_mask); // AND with value_mask is for extra safety
Or, if you can use struct instead of uint16_t, you can use Bit fields and let the compiler to perform all these actions for you.
finalvle = 0;
finalvle = (val1&0x01)<<15;
finalvle += (val2&0x07)<<12;
finalvle += (val3&0x0f)<<8
finalvle += (val4&0xfe)<<1;
finalvle += (val5&0x01);
You can use the bitwise or and shift operators to achieve this.
Use shift << to 'move bytes to the left':
int i = 1; // ...0001
int j = i << 3 // ...1000
You can then use bitwise or | to put it at the right place, (assuming you have all zeros at the bits you are trying to overwrite).
int k = 0; // ...0000
k |= i // ...0001
k |= j // ...1001
Edit: Note that #Inspired's answer also explains with zeroing out a certain area of bits. It overall explains how you would go about implementing it properly.
try this code:
uint16_t shift(uint16_t num, int shift)
{
return num | (int)pow (2, shift);
}
where shift is position of bit that you wanna set

setBit java method using bit shifting and hexadecimal code - question

I am having trouble understanding what is happening in the two lines with the 0xFF7F and the one below it. There is a link here that explains it to some degree.
http://www.herongyang.com/java/Bit-String-Set-Bit-to-Byte-Array.html
I don't know if 0xFF7F>>posBit) & oldByte) & 0x00FF
are supposed to be 3 values 'AND'ed together or how this is supposed to be read. If anyone can clarify what is happening here a little better, I would greatly appreciate it.
private static void setBit(byte[] data,
final int pos,
final int val) {
int posByte = pos/8;
int posBit = pos%8;
byte oldByte = data[posByte];
oldByte = (byte) (((0xFF7F>>posBit) & oldByte) & 0x00FF);
byte newByte = (byte) ((val<<(8-(posBit+1))) | oldByte);
data[posByte] = newByte;
}
passed into this method as parameters from a selectBits method was setBit(out,i,val);
out = is byte[] out = new byte[numOfBytes]; (numOfBytes can be 7 in this situation)
i = which is number [57], the original number from the PC1 int array holding the 56-integers.
val = which is the bit taken from the byte array from the getBit() method.
First of all 0xFF7F is 1111 1111 0111 1111. This is shifted right by an amount of bits calculated from the bit you pass as a parameter (so the one you want to set).
If you specify third bit posBit = 3 % 8 = 3 so
0xFF7F 1111 1111 0111 1111
>> 3 0001 1111 1110 1111
this value is then ANDed with the original byte you are modifying, the result is that every bit is kept equal to oldBit original bit except the one that is anded with the 0 bit, suppose you have for example oldByte == 0111 1010, you'll obtain:
0111 1010
& 1110 1111
-------------
0110 1010
Then the value is anded with 0xFF just to discard any bit that doesn't fit a byte (because it's at least the ninth bit) before doing the cast.
A better way to write this would be:
private static void setBit(byte[] data, int index, boolean value)
{
final int byteIndex = index / 8;
final int bitIndex = 7 - (index % 8);
final byte mask = (byte) (1 << bitIndex);
final byte valueBit = value ? mask : 0;
data[byteIndex] = (byte) ((data[byteIndex] & ~mask) | valueBit);
}