I'm supposed to pack some shorts into a 32 bit integer. It's a homework assignment that will lead into a larger idea of compression/decompression.
I don't have any problems understanding how to pack the shorts into an integer, but I am struggling to understand how to get each short value stored within the integer.
So, for example, I store the values 2, 4, 6, 8 into the integer. That means I want to print them in the same order I input them.
How do you go about getting these values out from the integer?
EDIT: Shorts in this context refers to an unsigned two-byte integer.
As Craig corrected me, short is a 16 bit variable, therfore only 2 shorts can fit in one int, so here's my answer retrieving shorts:
2|4
000000000000001|00000000000000100
00000000000000100000000000000100
131076
denoting first as the left-most variable and last as the right-most variable, getting the short variable would like like this:
int x = 131076; //00000000000000100000000000000100 in binary
short last = x & 65535; // 65535= 1111111111111111
short first= (x >> 16) & 65535;
and here's my previous answer fixed for compressing chars (8 bit variables):
Let's assume the first char is the one the start on the MSB and the last one is the one that ends on the LSB:
2|4|6|8
00000001|00000010|000000110|00001000
000000010000001000000011000001000
33818120
So, in this example the first char is 2 (0010), followed by 4 (0100), 6 (0110) and last: 8 (1000).
so to get the compressed numbers back one could use this code:
int x = 33818120; //00000010000001000000011000001000 in binary
char last = x & 255; // 255= 11111111
char third = (x >> 8) & 255;
char second = (x >> 16) & 255;
char last = (x >> 24) & 255;
This would be more interesting with char [as you'd get 4]. But, you can only pack two shorts into a single int. So, I'm a bit mystified by this as to what the instructor was thinking.
Consider a union:
union combo {
int u_int;
short u_short[2];
char u_char[4];
};
int
getint1(short s1,short s2)
{
union combo combo;
combo.u_short[0] = s1;
combo.u_short[1] = s2;
return combo.u_int;
}
short
getshort1(int val,int which)
{
union combo combo;
combo.u_int = val;
return combo.u_short[which];
}
Now consider encoding/decoding with shifts:
unsigned int
getint2(unsigned short s1,unsigned short s2)
{
unsigned int val;
val = s1;
val <<= 16;
val |= s2;
return val;
}
unsigned short
getshort2(unsigned int val,int which)
{
val >>= (which * 16);
return val & 0xFFFF;
}
The unsigned code above will probably do what you want.
But, the next uses signed values and probably won't work as well because you may have mixed signs between s1/s2 and encoding/decoding that may present problems
int
getint3(short s1,short s2)
{
int val;
val = s1;
val <<= 16;
val |= s2;
return val;
}
short
getshort3(int val,int which)
{
val >>= (which * 16);
return val & 0xFFFF;
}
Currently I'm working on an NES emulator and I'm working on the CPU core. I've searched all over the internet, but I can't find the right keywords find the answer to my question, so I thought I'd ask it here. What I have is seven booleans that act as my processor status flags. The current opcode I'm working on wants me to push the processor status flags to the stack. My stack is of the unsigned char datatype. I thought I would be able to smash the booleans together like this:
bool C, Z, I, D, B, V, N;
unsigned char example;
example = {C, Z, I, D, B, V, N, 0};
That didn't work, so how do I go about putting all of my boolean processor status flags into an unsigned char datatype so I can push the flags onto my stack?
enum { C=0x01, Z=0x02, I=0x04, D=0x08,
B=0x10, V=0x20, N=0x40 };
unsigned char flags = 0;
This works because only one bit is set in each constant. To determine whether the flag is set, use (flags & FLAGNAME) like (flags & C) to determine if it is set. Be careful not to get the bitwise AND (&) I used confused with the Boolean/logical AND (&&). The && operator won't work in place of the & operator. To set a flag, use (flags |= FLAGNAME) and to unset it, use flags &= ~FLAGNAME;. Adjust the constants for the correct order, so the NES instruction can properly check the flags. After all, if they're not in the correct order, the wrong flag values may be used, which obviously would be a problem.
Also, don't forget the parentheses. The bitwise operators have very low precedence, so parentheses around the expression are essential.
If you don't know about the bitwise operators, they are worth reading up on.
Edit
A handy set of macros for you:
#define FLAG_ISSET(x) (flags & (x))
#define FLAG_SET(x) (flags |= (x))
#define FLAG_CLEAR(x) (flags &= ~(x))
...
if (FLAG_ISSET(C))
...
FLAG_SET(V);
FLAG_CLEAR(Z);
Try std::bitset. http://en.cppreference.com/w/cpp/utility/bitset I couldn't find a way to construct it using initializer lists, but you can always use the operator[] or set() to set each bit, and use to_ulong() to convert it to a number (then you can convert the unsigned long to an unsigned char trivially).
#include <stdio.h>
#include <bitset>
using namespace std;
int main()
{
bitset<7> reg;
bool C = true, Z = false, I = false, D = false, B = true, V = true, N = false;
reg[6] = C;
reg[5] = Z;
reg[4] = I;
reg[3] = D;
reg[2] = B;
reg[1] = V;
reg[0] = N;
unsigned char example = static_cast<unsigned char>(reg.to_ulong());
printf("0x%x\n", example);
return 0;
}
try this:
bool C, Z, I, D, B, V, N;
unsigned char example = 0;
if(C)
example |= (1 << 7)
if(Z)
example |= (1 << 6)
if(I)
example |= (1 << 5)
if(D)
example |= (1 << 4)
if(B)
example |= (1 << 3)
if(V)
example |= (1 << 2)
if(N)
example |= (1 << 1)
Let's say int status is your CPU status register.
You could create some defines like these:
#define CPU_REG_STATUS_C 0x00000040
#define CPU_REG_STATUS_Z 0x00000020
#define CPU_REG_STATUS_I 0x00000010
#define CPU_REG_STATUS_D 0x00000008
#define CPU_REG_STATUS_B 0x00000004
#define CPU_REG_STATUS_V 0x00000002
#define CPU_REG_STATUS_N 0x00000001
and then set/unset the flags indipendently from each other by using the bitwise binary operators & and |
example1: setting the ZERO flag
status |= CPU_REG_STATUS_Z;
example2: checking the value of the ZERO flag:
if(status & CPU_REG_STATUS_Z)
{
//zero flag is set
}
else
{
//zero flag is unset
}
however, there are plenty of CPU core source codes out there, especially for the 6502 CPU (Commodore C64, Nintendo NES and many others) so you have to bother only with the emulation of the console.
http://www.6502.org/tools/emu/
This may be a bit heavy to be putting in an emulator, but why not convert each boolean value into a bit in the character, then decode the character once you need to pull it off your stack.
unsigned char encodeBits(bool bools[8]) {
unsigned char retval = 0;
for(int i = 0; i < 8; i++) {
retval += (bools[i] ? 1 : 0);
retval = (i < 7 ? retval << 1 : retval);
}
return retval;
}
This code basically just shifts the boolean value into a character.
Then to decode, pass an empty array by reference,
void decodeBits(bool (&bools)[8], unsigned char input) {
for(int i = 0; i < 8; i++) {
bools[i] = (input & 0x00000001 ? true : false);
input = (i < 7 ? input >> 1 : input);
}
}
This code should compare the first bit of the character, then shift it etc. etc.
This can be done without arrays but for simplicity's sake I used arrays.
Edit: A working example: http://ideone.com/hQ47nn
This should be what you are looking for:
bool C, Z, I, D, B, V, N;
unsigned char example = 0;
const unsigned char mask = 0x01;
if(N)
example |= mask << 1;
if(V)
example |= mask << 2;
//...
if(C)
example |= mask << 7;
Lets say i have an array dynamically allocated.
int* array=new int[10]
That is 10*4=40 bytes or 10*32=320 bits. I want to read the 2nd bit of the 30th byte or 242nd bit. What is the easiest way to do so? I know I can access the 30th byte using array[30] but accessing individual bits is more tricky.
bool bitset(void const * data, int bitindex) {
int byte = bitindex / 8;
int bit = bitindex % 8;
unsigned char const * u = (unsigned char const *) data;
return (u[byte] & (1<<bit)) != 0;
}
this is working !
#define GET_BIT(p, n) ((((unsigned char *)p)[n/8] >> (n%8)) & 0x01)
int main()
{
int myArray[2] = { 0xaaaaaaaa, 0x00ff00ff };
for( int i =0 ; i < 2*32 ; i++ )
printf("%d", GET_BIT(myArray, i));
return 0;
}
ouput :
0101010101010101010101010101010111111111000000001111111100000000
Be carefull of the endiannes !
First, if you're doing bitwise operations, it's usually
preferable to make the elements an unsigned integral type
(although in this case, it really doesn't make that much
difference). As for accessing the bits: to access bit i in an
array of n int's:
static int const bitsPerWord = sizeof(int) * CHAR_BIT;
assert( i >= 0 && i < n * bitsPerWord );
int wordIndex = i / bitsPerWord;
int bitIndex = i % bitsPerWord;
then to read:
return (array[wordIndex] & (1 << bitIndex)) != 0;
to set:
array[wordIndex] |= 1 << bitIndex;
and to reset:
array[wordIndex] &= ~(1 << bitIndex);
Or you can use bitset, if n is constant, or vector<bool> or
boost::dynamic_bitset if it's not, and let someone else do the
work.
You can use something like this:
!((array[30] & 2) == 0)
array[30] is the integer.
& 2 is an and operation which masks the second bit (2 = 00000010)
== 0 will check if the mask result is 0
! will negate that result, because we're checking if it's 1 not zero....
You need bit operations here...
if(array[5] & 0x1)
{
//the first bit in array[5] is 1
}
else
{
//the first bit is 0
}
if(array[5] & 0x8)
{
//the 4th bit in array[5] is 1
}
else
{
//the 4th bit is 0
}
0x8 is 00001000 in binary. Doing the anding masks all other bits and allows you to see if the bit is 1 or 0.
int is typically 32 bits, so you would need to do some arithmetic to get a certain bit number in the entire array.
EDITED based on comment below - array contains int of 32 bits, not 8 bits uchar.
int pos = 241; // I start at index 0
bool bit242 = (array[pos/32] >> (pos%32)) & 1;
I'm looking for the most efficient way to calculate the minimum number of bytes needed to store an integer without losing precision.
e.g.
int: 10 = 1 byte
int: 257 = 2 bytes;
int: 18446744073709551615 (UINT64_MAX) = 8 bytes;
Thanks
P.S. This is for a hash functions which will be called many millions of times
Also the byte sizes don't have to be a power of two
The fastest solution seems to one based on tronics answer:
int bytes;
if (hash <= UINT32_MAX)
{
if (hash < 16777216U)
{
if (hash <= UINT16_MAX)
{
if (hash <= UINT8_MAX) bytes = 1;
else bytes = 2;
}
else bytes = 3;
}
else bytes = 4;
}
else if (hash <= UINT64_MAX)
{
if (hash < 72057594000000000ULL)
{
if (hash < 281474976710656ULL)
{
if (hash < 1099511627776ULL) bytes = 5;
else bytes = 6;
}
else bytes = 7;
}
else bytes = 8;
}
The speed difference using mostly 56 bit vals was minimal (but measurable) compared to Thomas Pornin answer. Also i didn't test the solution using __builtin_clzl which could be comparable.
Use this:
int n = 0;
while (x != 0) {
x >>= 8;
n ++;
}
This assumes that x contains your (positive) value.
Note that zero will be declared encodable as no byte at all. Also, most variable-size encodings need some length field or terminator to know where encoding stops in a file or stream (usually, when you encode an integer and mind about size, then there is more than one integer in your encoded object).
You need just two simple ifs if you are interested on the common sizes only. Consider this (assuming that you actually have unsigned values):
if (val < 0x10000) {
if (val < 0x100) // 8 bit
else // 16 bit
} else {
if (val < 0x100000000L) // 32 bit
else // 64 bit
}
Should you need to test for other sizes, choosing a middle point and then doing nested tests will keep the number of tests very low in any case. However, in that case making the testing a recursive function might be a better option, to keep the code simple. A decent compiler will optimize away the recursive calls so that the resulting code is still just as fast.
Assuming a byte is 8 bits, to represent an integer x you need [log2(x) / 8] + 1 bytes where [x] = floor(x).
Ok, I see now that the byte sizes aren't necessarily a power of two. Consider the byte sizes b. The formula is still [log2(x) / b] + 1.
Now, to calculate the log, either use lookup tables (best way speed-wise) or use binary search, which is also very fast for integers.
The function to find the position of the first '1' bit from the most significant side (clz or bsr) is usually a simple CPU instruction (no need to mess with log2), so you could divide that by 8 to get the number of bytes needed. In gcc, there's __builtin_clz for this task:
#include <limits.h>
int bytes_needed(unsigned long long x) {
int bits_needed = sizeof(x)*CHAR_BIT - __builtin_clzll(x);
if (bits_needed == 0)
return 1;
else
return (bits_needed + 7) / 8;
}
(On MSVC you would use the _BitScanReverse intrinsic.)
You may first get the highest bit set, which is the same as log2(N), and then get the bytes needed by ceil(log2(N) / 8).
Here are some bit hacks for getting the position of the highest bit set, which are copied from http://graphics.stanford.edu/~seander/bithacks.html#IntegerLogObvious, and you can click the URL for details of how these algorithms work.
Find the integer log base 2 of an integer with an 64-bit IEEE float
int v; // 32-bit integer to find the log base 2 of
int r; // result of log_2(v) goes here
union { unsigned int u[2]; double d; } t; // temp
t.u[__FLOAT_WORD_ORDER==LITTLE_ENDIAN] = 0x43300000;
t.u[__FLOAT_WORD_ORDER!=LITTLE_ENDIAN] = v;
t.d -= 4503599627370496.0;
r = (t.u[__FLOAT_WORD_ORDER==LITTLE_ENDIAN] >> 20) - 0x3FF;
Find the log base 2 of an integer with a lookup table
static const char LogTable256[256] =
{
#define LT(n) n, n, n, n, n, n, n, n, n, n, n, n, n, n, n, n
-1, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3,
LT(4), LT(5), LT(5), LT(6), LT(6), LT(6), LT(6),
LT(7), LT(7), LT(7), LT(7), LT(7), LT(7), LT(7), LT(7)
};
unsigned int v; // 32-bit word to find the log of
unsigned r; // r will be lg(v)
register unsigned int t, tt; // temporaries
if (tt = v >> 16)
{
r = (t = tt >> 8) ? 24 + LogTable256[t] : 16 + LogTable256[tt];
}
else
{
r = (t = v >> 8) ? 8 + LogTable256[t] : LogTable256[v];
}
Find the log base 2 of an N-bit integer in O(lg(N)) operations
unsigned int v; // 32-bit value to find the log2 of
const unsigned int b[] = {0x2, 0xC, 0xF0, 0xFF00, 0xFFFF0000};
const unsigned int S[] = {1, 2, 4, 8, 16};
int i;
register unsigned int r = 0; // result of log2(v) will go here
for (i = 4; i >= 0; i--) // unroll for speed...
{
if (v & b[i])
{
v >>= S[i];
r |= S[i];
}
}
// OR (IF YOUR CPU BRANCHES SLOWLY):
unsigned int v; // 32-bit value to find the log2 of
register unsigned int r; // result of log2(v) will go here
register unsigned int shift;
r = (v > 0xFFFF) << 4; v >>= r;
shift = (v > 0xFF ) << 3; v >>= shift; r |= shift;
shift = (v > 0xF ) << 2; v >>= shift; r |= shift;
shift = (v > 0x3 ) << 1; v >>= shift; r |= shift;
r |= (v >> 1);
// OR (IF YOU KNOW v IS A POWER OF 2):
unsigned int v; // 32-bit value to find the log2 of
static const unsigned int b[] = {0xAAAAAAAA, 0xCCCCCCCC, 0xF0F0F0F0,
0xFF00FF00, 0xFFFF0000};
register unsigned int r = (v & b[0]) != 0;
for (i = 4; i > 0; i--) // unroll for speed...
{
r |= ((v & b[i]) != 0) << i;
}
Find the number of bits by taking the log2 of the number, then divide that by 8 to get the number of bytes.
You can find logn of x by the formula:
logn(x) = log(x) / log(n)
Update:
Since you need to do this really quickly, Bit Twiddling Hacks has several methods for quickly calculating log2(x). The look-up table approach seems like it would suit your needs.
This will get you the number of bytes. It's not strictly the most efficient, but unless you're programming a nanobot powered by the energy contained in a red blood cell, it won't matter.
int count = 0;
while (numbertotest > 0)
{
numbertotest >>= 8;
count++;
}
You could write a little template meta-programming code to figure it out at compile time if you need it for array sizes:
template<unsigned long long N> struct NBytes
{ static const size_t value = NBytes<N/256>::value+1; };
template<> struct NBytes<0>
{ static const size_t value = 0; };
int main()
{
std::cout << "short = " << NBytes<SHRT_MAX>::value << " bytes\n";
std::cout << "int = " << NBytes<INT_MAX>::value << " bytes\n";
std::cout << "long long = " << NBytes<ULLONG_MAX>::value << " bytes\n";
std::cout << "10 = " << NBytes<10>::value << " bytes\n";
std::cout << "257 = " << NBytes<257>::value << " bytes\n";
return 0;
}
output:
short = 2 bytes
int = 4 bytes
long long = 8 bytes
10 = 1 bytes
257 = 2 bytes
Note: I know this isn't answering the original question, but it answers a related question that people will be searching for when they land on this page.
Floor((log2(N) / 8) + 1) bytes
You need exactly the log function
nb_bytes = floor(log(x)/log(256))+1
if you use log2, log2(256) == 8 so
floor(log2(x)/8)+1
You need to raise 256 to successive powers until the result is larger than your value.
For example: (Tested in C#)
long long limit = 1;
int byteCount;
for (byteCount = 1; byteCount < 8; byteCount++) {
limit *= 256;
if (limit > value)
break;
}
If you only want byte sizes to be powers of two (If you don't want 65,537 to return 3), replace byteCount++ with byteCount *= 2.
I think this is a portable implementation of the straightforward formula:
#include <limits.h>
#include <math.h>
#include <stdio.h>
int main(void) {
int i;
unsigned int values[] = {10, 257, 67898, 140000, INT_MAX, INT_MIN};
for ( i = 0; i < sizeof(values)/sizeof(values[0]); ++i) {
printf("%d needs %.0f bytes\n",
values[i],
1.0 + floor(log(values[i]) / (M_LN2 * CHAR_BIT))
);
}
return 0;
}
Output:
10 needs 1 bytes
257 needs 2 bytes
67898 needs 3 bytes
140000 needs 3 bytes
2147483647 needs 4 bytes
-2147483648 needs 4 bytes
Whether and how much the lack of speed and the need to link floating point libraries depends on your needs.
I know this question didn't ask for this type of answer but for those looking for a solution using the smallest number of characters, this does the assignment to a length variable in 17 characters, or 25 including the declaration of the length variable.
//Assuming v is the value that is being counted...
int l=0;
for(;v>>l*8;l++);
This is based on SoapBox's idea of creating a solution that contains no jumps, branches etc... Unfortunately his solution was not quite correct. I have adopted the spirit and here's a 32bit version, the 64bit checks can be applied easily if desired.
The function returns number of bytes required to store the given integer.
unsigned short getBytesNeeded(unsigned int value)
{
unsigned short c = 0; // 0 => size 1
c |= !!(value & 0xFF00); // 1 => size 2
c |= (!!(value & 0xFF0000)) << 1; // 2 => size 3
c |= (!!(value & 0xFF000000)) << 2; // 4 => size 4
static const int size_table[] = { 1, 2, 3, 3, 4, 4, 4, 4 };
return size_table[c];
}
For each of eight times, shift the int eight bits to the right and see if there are still 1-bits left. The number of times you shift before you stop is the number of bytes you need.
More succinctly, the minimum number of bytes you need is ceil(min_bits/8), where min_bits is the index (i+1) of the highest set bit.
There are a multitude of ways to do this.
Option #1.
int numBytes = 0;
do {
numBytes++;
} while (i >>= 8);
return (numBytes);
In the above example, is the number you are testing, and generally works for any processor, any size of integer.
However, it might not be the fastest. Alternatively, you can try a series of if statements ...
For a 32 bit integers
if ((upper = (value >> 16)) == 0) {
/* Bit in lower 16 bits may be set. */
if ((high = (value >> 8)) == 0) {
return (1);
}
return (2);
}
/* Bit in upper 16 bits is set */
if ((high = (upper >> 8)) == 0) {
return (3);
}
return (4);
For 64 bit integers, Another level of if statements would be required.
If the speed of this routine is as critical as you say, it might be worthwhile to do this in assembler if you want it as a function call. That could allow you to avoid creating and destroying the stack frame, saving a few extra clock cycles if it is that critical.
A bit basic, but since there will be a limited number of outputs, can you not pre-compute the breakpoints and use a case statement? No need for calculations at run-time, only a limited number of comparisons.
Why not just use a 32-bit hash?
That will work at near-top-speed everywhere.
I'm rather confused as to why a large hash would even be wanted. If a 4-byte hash works, why not just use it always? Excepting cryptographic uses, who has hash tables with more then 232 buckets anyway?
there are lots of great recipes for stuff like this over at Sean Anderson's "Bit Twiddling Hacks" page.
This code has 0 branches, which could be faster on some systems. Also on some systems (GPGPU) its important for threads in the same warp to execute the same instructions. This code is always the same number of instructions no matter what the input value.
inline int get_num_bytes(unsigned long long value) // where unsigned long long is the largest integer value on this platform
{
int size = 1; // starts at 1 sot that 0 will return 1 byte
size += !!(value & 0xFF00);
size += !!(value & 0xFFFF0000);
if (sizeof(unsigned long long) > 4) // every sane compiler will optimize this out
{
size += !!(value & 0xFFFFFFFF00000000ull);
if (sizeof(unsigned long long) > 8)
{
size += !!(value & 0xFFFFFFFFFFFFFFFF0000000000000000ull);
}
}
static const int size_table[] = { 1, 2, 4, 8, 16 };
return size_table[size];
}
g++ -O3 produces the following (verifying that the ifs are optimized out):
xor %edx,%edx
test $0xff00,%edi
setne %dl
xor %eax,%eax
test $0xffff0000,%edi
setne %al
lea 0x1(%rdx,%rax,1),%eax
movabs $0xffffffff00000000,%rdx
test %rdx,%rdi
setne %dl
lea (%rdx,%rax,1),%rax
and $0xf,%eax
mov _ZZ13get_num_bytesyE10size_table(,%rax,4),%eax
retq
Why so complicated? Here's what I came up with:
bytesNeeded = (numBits/8)+((numBits%8) != 0);
Basically numBits divided by eight + 1 if there is a remainder.
There are already a lot of answers here, but if you know the number ahead of time, in c++ you can use a template to make use of the preprocessor.
template <unsigned long long N>
struct RequiredBytes {
enum : int { value = 1 + (N > 255 ? RequiredBits<(N >> 8)>::value : 0) };
};
template <>
struct RequiredBytes<0> {
enum : int { value = 1 };
};
const int REQUIRED_BYTES_18446744073709551615 = RequiredBytes<18446744073709551615>::value; // 8
or for a bits version:
template <unsigned long long N>
struct RequiredBits {
enum : int { value = 1 + RequiredBits<(N >> 1)>::value };
};
template <>
struct RequiredBits<1> {
enum : int { value = 1 };
};
template <>
struct RequiredBits<0> {
enum : int { value = 1 };
};
const int REQUIRED_BITS_42 = RequiredBits<42>::value; // 6