Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Bit fields in a structure can be used to save some bytes of memory, I have heard. How can we use this particular bytes for any purposes?
typedef struct
{
char A : 1;
int B : 1;
} Struct1;
The char value itself with a width of one bit is not particularly useful. In fact char values as bit fields are not standard. It's an extension that Microsoft added. On the other hand the B field can be used as on / off value since it can hold the values 1 or 0
Struct1 s;
s.B = 0;
if (s.B) {
...
}
This particular example doesn't really demonstrate the savings offered by bit fields particularly well. Need a more complex strut for that. Consider the following
typedef struct {
int Value1;
int Value2;
} S1;
On most platforms S1 will have a size of 8 (each int field being 4 bytes long). Imagine though that Value1 and Value2 will always have values between 0 and 10. This could be stored in 4 bits but we're using 32 bits meaning. Using bit fields we could reduce the waste significantly here
typedef struct {
int Value1 : 4;
int Value2 : 4;
} S1;
Now the size of S1 is likely 1 byte and can still hold all of the necessary values
In embedded systems, the bit fields in a structure can be used to represent bit fields or a hardware device.
Other uses for bit fields are in protocols (messages). One byte (or 4 bytes) to represent the presences or absences of many things would occupy a lot a space and wasted transmission time. So in 1 byte you could represent 8 Boolean conditions rather than using 8 bytes or 8 words to do so.
The bit fields in a structure are usually used as convenience. The same operations to extract, set or test bit fields can be performed using the arithmetic bit operators (such as AND).
Saving memory means, that you need less memory. This can increase the program performance due to less swapping or cache misses.
In your example the two parts A and B would be stored in a byte (or whatever the compiler decides to use), instead of two. A better example is, if you want to store the occupied seats in an opera house with 1000 seats. If you would store them as a boolean which is often stored in a byte per boolean, they could be stored in 128 bytes, because per seat only one bit is needed.
The downside is the performance loss. Accessing the bits need some additional shifting or xor-ing. Thus, it's a trade-off memory for computation time.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I am not a C++ programmer, but this algorithm appeared in an operating manual for a machine I'm using and I'm struggling to make sense of it. I'd like someone to explain some terms within it, or possibly the flow of the entire code, given that I don't have time to learn C in the course of the project.
Waveform files for the machine in question are made up of a number of tags in curly brackets. The checksum is calculated using the WAVEFORM tag, {WAVEFORM-length: #data}.
The "data" consists of a number of bytes represented as hexadecimal numbers. "length" is the number of bytes in the "data", while "start" apparently points to the first byte in the "data".
I've managed to work out some of the terms, but I'm particularly unsure about my interpretation of ((UINT32 *)start)[i]
UINT32 checksum(void *start, UINT32 length)
{
UINT32 i, result = 0xA50F74FF;
for(i=0; i < length/4; i++)
result = result ^ ((UINT32 *)start)[i];
return(result);
}
So from what I can tell, the code does the following:
Take the address of the first byte in the "data" and the length of the "data"
Create a variable called result, which is an unsigned integer A50F74FF
For each byte in the first 25% of the data string, raise "result" to that power (presumably modulo 2^32)
Return result as the value checksum
Am I correct here or have I misread the algorithm on one of the steps? I feel like I can't be correct, because basing a checksum on only part of the data wouldn't spot errors in the later parts of the data.
For each byte in the first 25% of the data string, raise "result" to that power (presumably modulo 2^32)
This is wrong. ^ is the bitwise XOR operation. It does not raise to a power.
Also, about "of the data string". The algorithm iterates the pointed data as if it is an array of UINT32. In fact, if start doesn't point to (an element of) an array of UINT32, then the behaviour of the program is undefined1. It would be much better to declare the argument to be UINT32* in the first place, and not use the explicit cast.
Also, about "For each byte in the first 25% of the data string", the algorithm appears to go through (nearly2) all bytes from start to start + length. length is presumably measured in bytes, and UINT32 is presumably a type that consists of 4 bytes. Thus an array of UINT32 objects of N bytes contains N/4 elements UINT32 of objects. Note that this assumes that the byte is 8 bits wide which is probably an assumption that the manual can make, but keep in mind that it is not an assumption portable to all systems.
1 UB as far as the C++ language is concerned. But, if it's shown in the operating manual of a machine, then perhaps the special compiler for the particular hardware specifies defined behaviour for this. That said, it is also quite possible for the author of the manual to have made a mistake.
2 If length is not divisible by 4, then the remaining 1-3 bytes are not used.
So the pseudocode for this function is roughly like this:
function checksum(DATA)
RESULT = 0xA50F74FF;
for each DWORD in DATA do
RESULT = RESULT xor DWORD
return RESULT
where DWORD is a four-byte integer value.
The function is actually going though (almost) all of the data (not 25%) but it's doing it in 4-byte increments that's why the length which is in bytes is divided by 4.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I recently posted a question about unaligned memory access, but given the answer, I am a little lost. I often hear that "aligned memory access is far more efficient than unaligned access", but I am actually not sure what is unaligned memory. Consequently:
What is unaligned memory?
How you declare something unaligned in C++? (small example program)
How do you access and manipulate something unaligned in C++? (small example program)
Is there is a way to manipulate unaligned memory in with a defined behavior approach or all of that is platform dependent/undefined behavior in C++?
Whether something is unaligned or not depends on the data type and its size As the answer from Gregg explains.
A well-written program usually does not have unaligned memory access, except when the compiler introduces it. (Yes, that happens during vectorization but let's skip that).
But you can write program in C++ to force unaligned memory access. The code below does just that.
#include <iostream>
using namespace std;
int main() {
int a[3] {1, 2, 3};
cout << *((long long *)(&a[0])) << endl;
cout << *((long long *)(&a[1])) << endl;
cout << (long long) (&a[0]) << endl;
cout << (long long) (&a[1]) << endl;
return 0;
}
The output of the code is this
8589934593
12884901890
70367819479584
70367819479588
What this program does?
I declare an integer array of size 3. This array will be 4 byte aligned because int is a 4 byte data type (at least on my platform). So the address of a[0] is divisible by 4. Now address of both of a[0] and a[1] is divisible by 4 but only address of one of them is divisible by 8.
So if I cast the address of a[0] and a[1] to a pointer to long long (which is an 8 byte data type on my platform) and then deference these two pointers, one of them will be an unaligned memory access. This is not undefined behavior AFAIK, but it is going to be slower than aligned memory access.
As you see this code contains C style casts which is not a good practice. But I think for enforcing some strange behavior that is fine.
let me know if you have question about the output of the code. You should know about endianness and representation of integers to understand the first two lines. The third and fourth line are address of the first two elements of the integer array. That should be easier to understand.
taking an example of a 32 bit computer reading a 4 byte word of data:
In the hardware, a 32 bit computer reads 4 bytes at a time but only on every 4th byte. This is because the memory bus is 4 bytes wide.
If your 4 byte data does not start on one of those 4 byte boundaries, the computer must read the memory twice, and then assemble the 4 bytes to a single register internally.
Based on the architecture chosen, the compiler knows this and places/pads data structures so that two byte data occur on two byte boundaries, 4 byte data starts on 4 byte boundaries, etc. This is specifically to avoid mis-aligned reads.
You can get misaligned reads if you read data in as bytes (like from a serial protocol) and then access them as 32 bit words. Avoid this in speed critical code. Usually it is taken care of for you and is not a problem.
My official question will be: "Is there a clean way to use data types to "encode and compress" data rather than using messy bit masking." The hopes would be to save space in the case of compressing, and I would like to use native data types, structures, and arrays in order to improve readability over bit masking. I am proficient in bit masking from my assembly background but I am learning C++ and OOP. We can store so much information in a 32 bit register by using individual bits and I feel that I am trying to get back to that low level environment while having the readability of C++ code.
I am attempting to save some space because I am working with huge resource requirements. I am still learning more about how c++ treats the bool data type. I realize that memory is stored in byte chunks and not individual bits. I believe that a bool usually uses one byte and is masked somehow. In my head I could use 8 bool values in one byte.
If I malloc in C++ an array of 2 bool elements. Does it allocate two bytes or just one?
Example: We will use DNA as an example since it can be encoded into two bit to represent A,C,G and T. If I make a struct with an array of two bool called DNA_Base, then I make an array of 20 of those.
struct DNA_Base{ bool Bit_1; bool Bit_2; };
DNA_Base DNA_Sequence[7] = {false};
cout << sizeof(DNA_Base)<<sizeof(DNA_Sequence)<<endl;
//Yields a 2 and a 14.
//I would like this to say 1 and 2.
In my example I would also show the case where the DNA sequence can be 20 bases long which would require 40 bits to encode. GATTACA could only take up a maximum of 2 bytes? I suppose an alternative question would have been "How to make C++ do the bit masking for me in a more readable way" or should I just make my own data type and classes and implement the bit masking using classes and operator overloading.
Not fully what you want but you can use bitfield:
struct DNA_Base
{
unsigned char Bit_1 : 1;
unsigned char Bit_2 : 1;
};
DNA_Base DNA_Sequence[7];
So sizeof(DNA_Base) == 1 and sizeof(DNA_Sequence) == 7
So you have to pack the DNA_Base to avoid to lose place with padding, something like:
struct DNA_Base_4
{
unsigned char base1 : 2; // may have value 0 1 2 or 3
unsigned char base2 : 2;
unsigned char base3 : 2;
unsigned char base4 : 2;
};
So sizeof(DNA_Base_4) == 1
std::bitset is an other alternative, but you have to do the interpretation job yourself.
An array of bools will be N-elements x sizeof(bool).
If your goal is to save space in registers, don't bother, because it is actually more efficient to use a word size for the processor in question than to use a single byte, and the compiler will prefer to use a word anyway, so in a struct/class the bool will usually be expanded to a 32-bit or 64-bit native word.
Now, if you like to save room on disk, or in RAM, due to needing to store LOTS of bools, go ahead, but it isn't going to save room in all cases unless you actually pack the structure, and on some architectures packing can also have performance impact because the CPU will have to perform unaligned or byte-by-byte access.
A bitmask (or bitfield), on the other hand, is performant and efficient and as dense as possible, and uses a single bitwise operation. I would look at one of the abstract data types that provide bit fields.
The standard library has bitset http://www.cplusplus.com/reference/bitset/bitset/ which can be as long as you want.
Boost also has something I'm sure.
Unless you are on a 4 bit machine, the final result will be using bit arithmetic. Whether you do it explicitly, have the compiler do it via bit fields, or use a bit container, there will be bit manipulation.
I suggest the following:
Use existing compression libraries.
Use the method that is most readable or understood by people other
than yourself.
Use the method that is most productive (talking about development
time).
Use the method that you will inject the least amount of defects.
Edit 1:
Write each method up as a separate function.
Tell the compiler to generate the assembly language for each function.
Compare the assembly language of each function to each other.
My belief is that they will be very similar, enough that wasting time discussing them is not worthwhile.
You can't operate on bits directly, but you can treat the smallest unit available to you as a multiple data store, and define
enum class DNAx4 : uint8_t {
AAAA = 0x00, AAAC = 0x01, AAAG = 0x02, AAAT = 0x03,
// .... And the rest of them
AAAA = 0xFC, AAAC = 0xFD, AAAG = 0xFE, AAAT = 0xFF
}
I'd actually go further, and create a structure DNAx16 or DNAx32 to efficiently use the native word size on your machine.
You can then define functions on the data type, which will have to use the underlying bit representation, but at least it allows you to encapsulate this and build higher level operations from these primitives.
Before any question is asked: I am dealing with actual hardware.
I am searching for a meta-language that would allow me to specify data structure contents where fields have different bit length (this includes fields like 1, 3 or 24 or 48 bits long), with respect to endianess, and would generate C++ code accessing the data.
The question was put on hold due to being too vague, so I'll try to make it as clear as possible:
I am searching for a language that:
accepts simple structure description and generate useful C++ code,
would allow to precisely specify integers ranging from 1 bit to multiple (up to 8) bytes long, along with data (typically string),
would isolate me from need to convert endianess,
produces exact, predictable output that does not come with overhead (like in protocol buffers)
ASN.1 sounds almost good for the purpose, but it adds its own overhead (meaning, I cannot produce a simple structure that has 2 bytes split into 4 nibbles) - what i'm looking for is a language that would offer exact representation of the structure.
For example, I would want to abstract this:
struct Command {
struct Record {
int8_t track;
int8_t point;
int8_t index;
int16_t start_position; // big endian, misaligned
int32_t length; // big endian, misaligned;
} __attribute__((packed)); // structure length = 11 bytes.
int8_t current : 1;
int8_t command : 7;
int8_t reserved;
int16_t side : 3; // entire int16_t needs to be
int16_t layer : 3; // converted from big endian, because
int16_t laser_mark : 3; // this field spans across bytes.
int16_t laser_power : 3;
int16_t reserved_pad : 2;
int16_t laser_tag : 2;
int32_t mode_number : 8; // again, entire 32 bit field needs to be converted
int32_t record_count : 24; // from big endian to read this count properly.
Record records[];
} __attribute__((packed));
the above needs to be packed exactly to the structure carrying 8 + record_count * 11 bytes, all formed accurately, no additional data, no additional bits or bytes set.
The above is just an example. it's made simple so that I don't clog the site with actual structures that have oftentimes hundreds of fields. It has been simplified, but shows many of the features that I am looking forward to see (two remaining features are 48 or 64-bit integers and plain data (bytes[]))
If this question is still too vague, please explain what it is that I should add in the comments. thanks!
A simple table that tracks individual field sizes and is used to spin out offsets of each element into your structure sounds like the easiest solution. This won't scale to deeply nested structures, but could be tuned to support handling of the unassigned bit cases you identify.
Then, you can use this to generate constants or even named property accessors to extract and update the individual fields. Given the size of the individual elements, macros are likely to make life even harder, but any mainstream compiler should inline the code. You mileage could vary with a template-based implementation.
If would help if you could use a common representation for both sides of the application (host and device) to further reduce the likelihood of transcription errors.
The PLC world has a number of different mechanisms for layout, but these are all very hardwired into their eco-systems and so would not really help.
Alternately, if you have the tooling available, you could consider something like ASN.1 structures for the representation. In the extreme, you could even use an open source generator to come up with an unencoded generator directly from the MIB.
Days ago I heard (maybe I've even seen it!) about library, that helps with packing structures. Unfortunately - I can't recall it's name.
In my program I have to keep large amount of data, therefore I need to pack them and avoid loosing bits on gaps. For example: I have to keep many numbers from range 1...5. If I would keep them in char - it would take 8bits, but this number can be kept on 3 bits. Moreover - if I would keep this numbers in packs of 8bits with maximum number 256 I could pack there 51 numbers (instead of 1 or 2!).
Is there any librarary, which helps this actions, or do I have do this on my own?
As Tomalak Garet'kal already mentioned, this is a feature of ANSI C, called bit-fields. The wikipedia article is quite useful. Typically you declare them as structs.
For your example: as you mentioned you have one number in the range of 0..5 you can use 3 bits on this number, which leaves you 5 bits of use:
struct s
{
unsigned int mynumber : 3;
unsigned int myother : 5;
}
These can now be accesses simply like this:
struct s myinstance;
myinstance.mynumber = 3;
myinstance.myother = 1;
Be awared that bit fields are slower than usual struct members/variables, since the runtime has to perform bit-shifting/masking to allow access to simple bits.