I have the question of the title, but If not, how could I get away with using only 4 bits to represent an integer?
EDIT really my question is how. I am aware that there are 1 byte data structures in a language like c, but how could I use something like a char to store two integers?
In C or C++ you can use a struct to allocate the required number of bits to a variable as given below:
#include <stdio.h>
struct packed {
unsigned char a:4, b:4;
};
int main() {
struct packed p;
p.a = 10;
p.b = 20;
printf("p.a %d p.b %d size %ld\n", p.a, p.b, sizeof(struct packed));
return 0;
}
The output is p.a 10 p.b 4 size 1, showing that p takes only 1 byte to store, and that numbers with more than 4 bits (larger than 15) get truncated, so 20 (0x14) becomes 4. This is simpler to use than the manual bitshifting and masking used in the other answer, but it is probably not any faster.
You can store two 4-bit numbers in one byte (call it b which is an unsigned char).
Using hex is easy to see that: in b=0xAE the two numbers are A and E.
Use a mask to isolate them:
a = (b & 0xF0) >> 4
and
e = b & 0x0F
You can easily define functions to set/get both numbers in the proper portion of the byte.
Note: if the 4-bit numbers need to have a sign, things can become a tad more complicated since the sign must be extended correctly when packing/unpacking.
Related
I`m working on an LZW compression app in C++. Since there are no data types that can store 12 bit numbers for representing table elements up to 4095 I thought that I can store 2 of those nrs as 3 bytes in a file and then read them as a struct with 2 unsigned short members. Is there a way to do that or I should just use unsigned short? This is what I have tried but it stores 4 bytes as there are 2 unsigned short members.
#define BITS 12
struct LZWStruct {
unsigned short code1 : BITS;
unsigned short code2 : BITS;
};
int main() {
LZWStruct test;
test.code1 = 144;
test.code2 = 233;
FILE* f = fopen("binary.bin", "wb");
fwrite(&test, sizeof(test), 1, f);
fclose(f);
}
Your question title and question body are two different questions with different answers.
No, you absolutely cannot store 3 * 12-bit unsigned numbers (36 bits) in four bytes (32 bits).
Yes, you can store two 12-bit numbers (24 bits) in three bytes (24 bits).
The bit fields in C++, inherited from C, that you are trying to use do not guarantee exactly how the bits are packed in the structure, so you cannot know which three bytes in the structure have your data. You should simply use the shift and or operators to put them in an integer. Then you will know exactly which three bytes to write to the file.
Then to be portable, in particular not dependent on the endianess of the machine, you should write bytes from the integer also using the shift operator. If you write using a pointer to the integer, it won't be portable.
In your example, you could have tried fwrite(&test, 3, 1, f), and it might work, if the compiler put the codes in the low bits of test, and if your machine is little-endian. Otherwise, no.
So to do it reliably:
Put in an integer:
unsigned short code1;
unsigned short code2;
uint32_t test = (code1 & 0x3ff) | ((uint32_t)(code2 & 0x3ff) << 12);
Write to a file:
putc(test, f);
putc(test >> 8, f);
putc(test >> 16, f);
You can skip the intermediate step if you like:
putc(code1, f);
putc(((code1 >> 8) & 0xf) | (code2 << 4), f);
putc(code2 >> 4, f);
(In the above I am assuring that I only store the low 12 bits of each code with the & operators, in case the bits above the low 12 are not zero. If you know for certain that the code values are less than 4096, then you can remove the & operations above.)
From here, multiple adjacent bit fields are usually packed together. The special unnamed bit field of size zero can be forced to break up padding. It specifies that the next bit field begins at the beginning of its allocation unit. Use sizeof to verify the size of your structure.
The exact packing, however, may depend on platform and compiler. This may be less a problem if the data are later loaded by the same program, or some closely related, but may be an issue for some generic format.
I am dealing with very large list of booleans in C++, around 2^N items of N booleans each. Because memory is critical in such situation, i.e. an exponential growth, I would like to build a N-bits long variable to store each element.
For small N, for example 24, I am just using unsigned long int. It takes 64MB ((2^24)*32/8/1024/1024). But I need to go up to 36. The only option with build-in variable is unsigned long long int, but it takes 512GB ((2^36)*64/8/1024/1024/1024), which is a bit too much.
With a 36-bits variable, it would work for me because the size drops to 288GB ((2^36)*36/8/1024/1024/1024), which fits on a node of my supercomputer.
I tried std::bitset, but std::bitset< N > creates a element of at least 8B.
So a list of std::bitset< 1 > is much greater than a list of unsigned long int.
It is because the std::bitset just change the representation, not the container.
I also tried boost::dynamic_bitset<> from Boost, but the result is even worst (at least 32B!), for the same reason.
I know an option is to write all elements as one chain of booleans, 2473901162496 (2^36*36), then to store then in 38654705664 (2473901162496/64) unsigned long long int, which gives 288GB (38654705664*64/8/1024/1024/1024). Then to access an element is just a game of finding in which elements the 36 bits are stored (can be either one or two). But it is a lot of rewriting of the existing code (3000 lines) because mapping becomes impossible and because adding and deleting items during the execution in some functions will be surely complicated, confusing, challenging, and the result will be most likely not efficient.
How to build a N-bits variable in C++?
How about a struct with 5 chars (and perhaps some fancy operator overloading as needed to keep it compatible to the existing code)? A struct with a long and a char probably won't work because of padding / alignment...
Basically your own mini BitSet optimized for size:
struct Bitset40 {
unsigned char data[5];
bool getBit(int index) {
return (data[index / 8] & (1 << (index % 8))) != 0;
}
bool setBit(int index, bool newVal) {
if (newVal) {
data[index / 8] |= (1 << (index % 8));
} else {
data[index / 8] &= ~(1 << (index % 8));
}
}
};
Edit: As geza has also pointed out int he comments, the "trick" here is to get as close as possible to the minimum number of bytes needed (without wasting memory by triggering alignment losses, padding or pointer indirection, see http://www.catb.org/esr/structure-packing/).
Edit 2: If you feel adventurous, you could also try a bit field (and please let us know how much space it actually consumes):
struct Bitset36 {
unsigned long long data:36;
}
I'm not an expert, but this is what I would "try". Find the bytes for the smallest type your compiler supports (should be char). You can check with sizeof and you should get 1. That means 1 byte, so 8 bits.
So if you wanted a 24 bit type...you would need 3 chars. For 36 you would need 5 char array and you would have 4 bits of wasted padding on the end. This could easily be accounted for.
i.e.
char typeSize[3] = {0}; // should hold 24 bits
Now make a bit mask to access each position of typeSize.
const unsigned char one = 0b0000'0001;
const unsigned char two = 0b0000'0010;
const unsigned char three = 0b0000'0100;
const unsigned char four = 0b0000'1000;
const unsigned char five = 0b0001'0000;
const unsigned char six = 0b0010'0000;
const unsigned char seven = 0b0100'0000;
const unsigned char eight = 0b1000'0000;
Now you can use the bit-wise or to set the values to 1 where needed..
typeSize[1] |= four;
*typeSize[0] |= (four | five);
To turn off bits use the & operator..
typeSize[0] &= ~four;
typeSize[2] &= ~(four| five);
You can read the position of each bit with the & operator.
typeSize[0] & four
Bear in mind, I don't have a compiler handy to try this out so hopefully this is a useful approach to your problem.
Good luck ;-)
You can use array of unsigned long int and store and retrieve needed bit chains with bitwise operations. This approach excludes space overhead.
Simplified example for unsigned byte array B[] and 12-bit variables V (represented as ushort):
Set V[0]:
B[0] = V & 0xFF; //low byte
B[1] = B[1] & 0xF0; // clear low nibble
B[1] = B[1] | (V >> 8); //fill low nibble of the second byte with the highest nibble of V
In a single nibble (0-F) I can store one number from 0 to 15. In one byte, I can store a single number from 0 to 255 (00 - FF).
Can I use a byte (00-FF) to store two different numbers each in the range 0-127 (00 - 7F)?
The answer to your question is NO. You can split a single byte into two numbers, but the sum of the bits in the two numbers must be <= 8. Since, the range 0-127 requires 7 bits, the other number in the byte can only be 1 bit, i.e. 0-1.
For obvious cardinality reasons, you cannot store two small integers in the 0 ... 127 range in one byte of 0 ... 255 range. In other words the cartesian product [0;127]×[0;127] has 214 elements which is bigger than 28 (the cardinal of the [0;255] interval, for bytes)
(If you can afford losing precision - which you didn't tell - you could, e.g. by storing only the highest bits ...)
Perhaps your question is: could I store two small integers from [0;15] in a byte? Then of course you could:
typedef unsigned unibble_t; // unsigned nibble in [0;15]
uint8_t make_from_two_nibbles(unibble_t l, unibble_t r) {
assert(l<=15);
assert(r<=15);
return (l<<4) | r;
}
unibble_t left_nible (uint8_t x) { return x >> 4; }
unibble_t right_nibble (uint8_t) { return x & 0xf; }
But I don't think you always should do that. First, you might use bit fields in struct. Then (and most importantly) dealing with nibbles that way might be more inefficient and make less readable code than using bytes.
And updating a single nibble, e.g. with
void update_left_nibble (uint8_t*p, unibble_t l) {
assert (p);
assert (l<=15);
*p = ((l<<4) | ((*p) & 0xf));
}
is sometimes expensive (it involves a memory load and a memory store, so uses the CPU cache and cache coherence machinery), and most importantly is generally a non-atomic operation (what would happen if two different threads are calling simultaneously update_left_nibble on the same address p -i.e. with pointer aliasing- is undefined behavior).
As a rule of thumb, avoid packing more than one data item in a byte unless you are sure it is worthwhile (e.g. you have a billion of such data items).
One byte is not enough for two values in 0…127, because each of those values needs log2(128) = 7 bits, for a total of 14, but a byte is only 8 bits.
You can declare variables with bit-packed storage using the C and C++ bitfield syntax:
struct packed_values {
uint8_t first : 7;
uint8_t second : 7;
uint8_t third : 2;
};
In this example, sizeof(packed_values) should equal 2 because only 16 bits were used, despite having three fields.
This is simpler than using bitwise arithmetic with << and & operators, but it's still not quite the same as ordinary variables: bit-fields have no addresses, so you can't have a pointer (or C++ reference) to one.
Can I use a byte to store two numbers in the range 0-127?
Of course you can:
uint8_t storeTwoNumbers(unsigned a, unsigned b) {
return ((a >> 4) & 0x0f) | (b & 0xf0);
}
uint8_t retrieveTwoNumbers(uint8_t byte, unsigned *a, unsigned *b) {
*b = byte & 0xf0;
*a = (byte & 0x0f) << 4;
}
Numbers are still in range 0...127 (0...255, actually). You just loose some precision, similar to floating point types. Their values increment in steps of 16.
You can store two data in range 0-15 in a single byte, but you should not (one var = one data is a better design).
If you must, you can use bit-masks and bit-shifts to access to the two data in your variable.
uint8_t var; /* range 0-255 */
data1 = (var & 0x0F); /* range 0-15 */
data2 = (var & 0xF0) >> 4; /* range 0-15 */
Excuse me for my english. I have a number of int values stored in it from 0 to 255. To find out what lies in 7 bit number, I translate the numbers into a binary system, then in the line and check the line:
if (informationOctet_.substr(6, 1) == "0")
{
...
}
Two questions arose,
If I use int (which we have 4 bytes), and my number is unsigned int the range [0, 255] How do I determine which byte I need to consider? High byte?
I have found the desired byte, how do you know the contents of, for example, the 6th bit?
P.S.
I do not use spells, because do with unsigned int.
THANK ALL, I test int number:
int k = 3;
for (int i = 0; i < 8; i++)
{
if (k & (1 << i))
{
std::cout << 1;
}
else
{
std::cout << 0;
}
}
print: 11000000
This is implementation-defined and depends on the endianess of the CPU. If you are smart though, you do the check like this: the_int & 0xFF, which will always give you the least significant byte no matter endianess.
byte & (1 << 6). Maybe. Note that bit counting is zero-indexed (just like arrays), so you have to be careful with the terms. "Bit 6" and "the 6th bit" may have different meanings. Always enumerate bits as 7 6 5 4 3 2 1 0, then it will be consistent with the C code.
You can choose the "Char" Data Type to check. It answer your both question. Because, the character Data Type is of 1 byte (8 bits). And it contains integer values as well because char & int are the compatible data types.
I am writing a program and using memcpy to copy some bytes of data, using the following code;
#define ETH_ALEN 6
unsigned char sourceMAC[6];
unsigned char destMAC[6];
char* txBuffer;
....
memcpy((void*)txBuffer, (void*)destMAC, ETH_ALEN);
memcpy((void*)(txBuffer+ETH_ALEN), (void*)sourceMAC, ETH_ALEN);
Now I want to copy some data on to the end of this buffer (txBuffer) that is less than a single byte or greater than one byte, so it is not a multiple of 8 (doesn't finish on a whole byte boundary), so memcpy() can't be used (I don't believe?).
I want to add 16 more bits worth of data which is a round 4 bytes. First I need to add a value into the next 3 bits of txtBuffer which I have stored in an int, and a fourth bit which is always 0. Next I need to copy another 12 bit value, again I have this in an int.
So the first decimal value stored in an int is between 0 and 7 inclusively, the same is true for the second number I mention to go into the final 12 bits. The stored value is within the rang of 2^12. Should I for example 'bit-copy' the last three bits of the int into memory, or merge all these values together some how?
Is there a way I can compile these three values into 4 bytes to copy with memcpy, or should I use something like bitset to copy them in, bit at a time?
How should I solve this issue?
Thank you.
Assuming int is 4 bytes on your platform
int composed = 0;
int three_bits = something;
int twelve_bits = something_else;
composed = (three_bits & 0x07) | (1 << 3) | ((twelve_bits << 4) & 0xFFFFFF0);