Optimizing Bitwise Logic - c++

Question

In my code the following lines are currently the hotspot:
int table1[256] = /*...*/;
int table2[512] = /*...*/;
int table3[512] = /*...*/;
int* result = /*...*/;
for(int r = 0; r < r_end; ++r)
{
std::uint64_t bits = bit_reader.value(); // 64 bits, no assumption regarding bits.
// The get_ functions are table lookups from the highest word of the bits variable.
struct entry
{
int sign_offset : 5;
int r_offset : 4;
int x : 7;
};
// NOTE: We are only interested in the highest word in the bits variable.
entry e;
if(is_in_table1(bits)) // branch prediction should work well here since table1 will be hit more often than 2 or 3, and 2 more often than 3.
e = reinterpret_cast<const entry&>(table1[get_table1_index(bits)]);
else if(is_in_table2(bits))
e = reinterpret_cast<const entry&>(table2[get_table2_index(bits)]);
else
e = reinterpret_cast<const entry&>(table3[get_table3_index(bits)]);
r += e.r_offset; // r is 18 bits, top 14 bits are always 0.
int x = e.x; // x is 14 bits, top 18 bits are always 0.
int sign_offset = e.sign_offset;
assert(sign_offset <= 16 && sign_offset > 0);
// The following is the hotspot.
int sign = 1 - (bits >> (63 - sign_offset) & 0x2);
(*result++) = ((x << 18) * sign) | r; // 32 bits
// End of hotspot
bit_reader.skip(sign_offset); // sign_offset is the last bit used.
}
Though I haven't figured out how to further optimize this, maybe something from intrinsics for Operations at Bit-Granularity, __shiftleft128 or _rot could be useful?
Note that I am also doing processing of the resulting data on the GPU, so the important thing is to get something into result which the GPU then can use to calculate the correct.
Suggestions?
EDIT:
Added table look-up.
EDIT:
int sign = 1 - (bits >> (63 - e.sign_offset) & 0x2);
000000013FD6B893 and ecx,1Fh
000000013FD6B896 mov eax,3Fh
000000013FD6B89B sub eax,ecx
000000013FD6B89D movzx ecx,al
000000013FD6B8A0 shr r8,cl
000000013FD6B8A3 and r8d,2
000000013FD6B8A7 mov r14d,1
000000013FD6B8AD sub r14d,r8d

Answer 1

I overlooked the fact that the sign is +/-1, so I'm correcting my answer.
Assuming that mask is an array with properly defined bitmasks for all possible values of sign_offset, this approach might be faster
bool sign = (bits & mask[sign_offset]) != 0;
__int64 result = r;
if (sign)
result |= -(x << 18);
else
result |= x << 18;
The code generated by VC2010 optimized build
OP code (11 instructions)
; 23 : __int64 sign = 1 - (bits >> (63 - sign_offset) & 0x2);
mov rax, QWORD PTR bits$[rsp]
mov ecx, 63 ; 0000003fH
sub cl, BYTE PTR sign_offset$[rsp]
mov edx, 1
sar rax, cl
; 24 : __int64 result = ((x << 18) * sign) | r; // 32 bits
; 25 : std::cout << result;
and eax, 2
sub rdx, rax
mov rax, QWORD PTR x$[rsp]
shl rax, 18
imul rdx, rax
or rdx, QWORD PTR r$[rsp]
My code (8 instructions)
; 34 : bool sign = (bits & mask[sign_offset]) != 0;
mov r11, QWORD PTR sign_offset$[rsp]
; 35 : __int64 result = r;
; 36 : if (sign)
; 37 : result |= -(x << 18);
mov rdx, QWORD PTR x$[rsp]
mov rax, QWORD PTR mask$[rsp+r11*8]
shl rdx, 18
test rax, QWORD PTR bits$[rsp]
je SHORT $LN2#Test1
neg rdx
$LN2#Test1:
; 38 : else
; 39 : result |= x << 18;
or rdx, QWORD PTR r$[rsp]
EDIT by Skizz
To get rid of branch:
shl rdx, 18
lea rbx,[rdx*2]
test rax, QWORD PTR bits$[rsp]
cmove rbx,0
sub rdx,rbx
or rdx, QWORD PTR r$[rsp]

Answer 2

Let's do some equivalent transformations:
int sign = 1 - (bits >> (63 - sign_offset) & 0x2);
int result = ((x << 18) * sign) | r; // 32 bits
Perhaps the processor will find shifting 32-bit values cheaper -- replace the definition of HIDWORD with whatever leads to direct access to the high-order DWORD without shifting. Also, for preparation of the next step, let's rearrange the shifting in the second assignment:
#define HIDWORD(q) ((uint32_t)((q) >> 32))
int sign = 1 - (HIDWORD(bits) >> (31 - sign_offset) & 0x2);
int result = ((x * sign) << 18) | r; // 32 bits
Observe that, in two-s complement, q * (-1) equals ~q + 1, or (q ^ -1) - (-1), while q * 1 equals (q ^ 0) - 0. This justifies the second transformation which gets rid of the nasty multiplication:
int mask = -(HIDWORD(bits) >> (32 - sign_offset) & 0x1);
int result = (((x ^ mask) - mask) << 18) | r; // 32 bits
Now let's rearrange shifting again:
int mask = (-(HIDWORD(bits) >> (32 - sign_offset) & 0x1)) << 18;
int result = (((x << 18) ^ mask) - mask) | r; // 32 bits
Recall the identity concerning - and ~:
int mask = (~(HIDWORD(bits) >> (32 - sign_offset) & 0x1) + 1) << 18;
Shift rearrangement again:
int mask = (~(HIDWORD(bits) >> (32 - sign_offset) & 0x1)) << 18 + (1 << 18);
Who can finally unfiddle this? (Are the transformations corect anyway?)
(Note that only profiling on a real CPU can
assess the performance. Measures like instruction count won't do. I am not even sure that the transformations helped at all.)

Answer 3

Memory access is usually the root of all optimisation problems on modern CPUs. You are being misled by the performance tools as to where the slow down is happening. The compiler is probably re-ordering the code to something like this:-
int sign = 1 - (bits >> (63 - get_sign_offset(bits)) & 0x2);
(*result++) = ((get_x(bits) << 18) * sign) | (r += get_r_offset(bits));
or even:-
(*result++) = ((get_x(bits) << 18) * (1 - (bits >> (63 - get_sign_offset(bits)) & 0x2))) | (r += get_r_offset(bits));
This would highlight the lines you identified as being the hotspot.
I would look at the way you organise your memory and the what the various get_ functions do. Can you post the get_ functions at all?

Answer 4

To calculate the sign, I would suggest this:
int sign = (int)(((int64_t)(bits << sign_offset)) >> 63);
Which is only 2 instructions (shl and sar).
If sign_offset is one bigger than I expected:
int sign = (int)(((int64_t)(bits << (sign_offset - 1))) >> 63);
Which is still not bad. Should be only 3 instructions.
That gives an answer as 0 or -1, with which you can do this:
(*result++) = (((x << 18) ^ sign) - sign) | r;

Answer 5

I think this is the fastest solution:
*result++ = (_rotl64(bits, sign_offset) << 31) | (x << 18) | (r << 0); // 32 bits
And then correct x depending on whether the sign bit is set or not on the GPU.