I have declared a union allocating 4100 bytes to variable "sample_union" and made the same union declaration as part of a structure which is allocating 4104 bytes.
union test_size_union {
struct {
uint8_t type;
union {
uint8_t count;
uint8_t list;
};
uint16_t rc;
uint16_t arr_value[2048];
};
uint64_t first_dword;
}__attribute__((packed)) sample_union ;
Placing the above union inside structure is allocating 4104 bytes.
struct test_size_struct {
union {
struct {
uint8_t type;
union {
uint8_t count;
uint8_t list;
};
uint16_t rc;
uint16_t arr_value[2048];
};
uint64_t first_dword;
};
}__attribute__((packed)) sample_struct;
Well, this is not a project requirement, but I would like to know why compiler is behaving differently for this two declaration.
gcc version: (GCC) 4.9.2, x86_64
Platform: Linux, x86_64
When you placed the union inside the struct, you didn't mark the union as packed. The unpacked union has a little padding (four bytes) so that its size is a multiple of the size of uint64_t.
The packed union doesn't have this padding, so it is smaller.
As a side observation, the anonymous struct inside the union is not marked packed. That happens not to matter in this case, because everything is nicely aligned anyway - but it's something to be aware of.
Related
Suppose I have a struct which contains other structs, which are not packed:
struct ContainerOfNonPacked
{
NonPacked1 first;
NonPacked2 second;
};
And then I have a struct which contains other structs which are packed:
struct ContainerOfPacked
{
Packed1 first; // declaration of struct had __attribute__((packed))
Packed2 second; // ditto
};
The first is not going to be packed by the compiler (i.e. there's no guarantee that there will be no "holes" inside the struct). It might coincidentally have no holes, but that's not what the question is about.
What about the second container, that contains packed structs? Is there any guarantee that a struct consisting solely of packed structs as its fields is itself packed?
Edit: it is up to the compiler and is not a behaviour one can depend on.
For the case of GCC and clang, structs that contain only packed structs are themselves "packed". They will not have any padding "holes" in them.
Here is an example to illustrate (godbolt link: https://godbolt.org/g/2noSpP):
We have four structs, Packed1, Packed2, NonPacked1, and NonPacked2.
#include <cstdint>
struct Packed1
{
uint32_t a;
uint64_t b;
} __attribute__((packed));
struct NonPacked1
{
uint16_t a;
// Padding of 2 bytes
uint32_t b;
uint8_t c;
};
// We have two structs of the same size, one packed and one not
static_assert(sizeof(NonPacked1) == 12);
static_assert(sizeof(Packed1) == 12);
struct Packed2
{
uint64_t a;
uint64_t b;
} __attribute__((packed)); // packing has no effect, but better for illustration
struct NonPacked2
{
uint32_t a;
// Padding of 4 bytes
uint64_t b;
};
// And again, two structs of the same size
static_assert(sizeof(Packed2) == 16);
static_assert(sizeof(NonPacked2) == 16);
Packed1 and Packed2 go into a ContainerOfPacked struct, and the other two into a ContainerOfNonPacked.
struct ContainerOfNonPacked
{
NonPacked1 first; // 12 bytes
// Padding of 4 bytes between the non-packed struct-fields
NonPacked2 second; // 16 bytes
};
struct ContainerOfPacked
{
Packed1 first; // 12
// No padding between the packed struct-fields
Packed2 second; // 16
};
static_assert(sizeof(ContainerOfNonPacked) == 32);
static_assert(sizeof(ContainerOfPacked) == 28);
As the code comments and static asserts show, the container of packed structs does not have padding holes inside it. It behaves as if it is itself "packed".
Although this is an answer, I'm also looking for answers which can cite the relevant parts of the Standard or some other authoritative/theoretical explanations.
I have a struct with data members with types of different lengths (some ints, some words, some just single bytes). The struct is packed (using #pragma pack(1)).
My question is: Is it possible to have one of the struct's data members memory address be aligned in memory (for example, have the address be divisable by 16)?
I know it is possible to align the whole struct (or in other words, its first member) using attribute. I want to know if it's possible to align a specific member and not just the first one.
Basically I need the struct to be packed (every member follows the previous one without padding) and I need that the address of one of the members, not the first one, be divisable by 16.
There doesn't seem to be any possibility to align structures with an offset. You must create the offset (padding) by yourself.
typedef struct __attribute__((packed)) special_align {
char misc[3];
int align_me;
};
typedef struct __attribute__((packed, aligned(16))) pad_container {
char padding[13]; //16-3
struct special_align data;
};
Of course accessing the struct will become more cumbersome.
If you don't want to calculate by hand how much padding you need, you can use the offsetof() macro. In the example offsetof(struct special_align,align_me) will return 3, that is to be substracted from the align size. (of course if the offset is greater than align, you must use a corresponding pad size multiplier. `
pad_size = (sizeof(special_align) * align_size - offsetof(special_align,align_me)) % align_size;
#pragma pack(1) is nasty -- once it is set for a given structure, it will apply to every member of that structure. None of alignas, __attribute__((packed, aligned(8))), std::aligned_storage, std::aligned_union, #pragma pack() constructs inside that structure will affect the alignment, the original packing will be maintained for the whole structure (an intermediate #pragma pack() will apply to any inner structure that follows, but the outer structure will be packed with its original packing).
So the only solution is to pack your structure manually, by adding padding bytes where necessary.
I'm using something like this, it triggers a compiler error and also shows what the (incorrect) alignment is:
template<int s> struct OffsetIs;
template<int N>
struct CheckAlignment {
static constexpr size_t ok = sizeof(OffsetIs<N>);
};
template<>
struct CheckAlignment<0> {
static constexpr bool ok = true;
};
#pragma pack(1)
struct MyStruct
{
char a;
//char pad[3]; // uncomment to make it compile
long x;
};
#pragma pack()
static_assert(CheckAlignment<offsetof(MyStruct, x) % 4>::ok, "Alignment of x is wrong");
I have no idea how to decipher this man someone needs to tell me what is going on please help
typedef struct _ARGBCOLOR {
union {
struct {
BYTE B;
BYTE G;
BYTE R;
BYTE A;
};
DWORD ARGB;
};
} ARGBCOLOR, *PARGBCOLOR;
If you have a ARGBCOLOR x;You can access 4 separate bytes as x.B,x.G,x.R, and x.A, or a 32-bit word as x.ARGB.
The C standard guarantees these will overlay properly (assuming the sizes fit and padding requirements don't screw the pooch (not the case here)). But this struct clearly assumes a little-endian system.
One extra complication is that the union is anonymous. It's common to name a union like this u, but some compilers allow internal unions and structs to be anonymous, so its members are accessed as if they were up one level (at the same level as the union itself).
My favorite way to do this sort of overlay type is to put the union at the outermost level. You can duplicate the kind or type member so it's accessible everywhere. But this way removes the temptation to use anonymous unions (which are not available in Ansi-C or C99) because you don't need a bogus u member in the middle.
typedef union _ARGBCOLOR {
//BYTE type;
struct {
//BYTE type;
BYTE B;
BYTE G;
BYTE R;
BYTE A;
} bytes;
struct {
//BYTE type;
DWORD ARGB;
} word;
} ARGBCOLOR, *PARGBCOLOR;
Due to the common initial prefix property, all three of the BYTE type; members would overlay the same memory.
Another variation is to make an array for the individual bytes.
typedef union _ARGBCOLOR {
DWORD dword;
BYTE byte[ sizeof(DWORD)/sizeof(BYTE) ];
} ARGBCOLOR, *PARGBCOLOR;
enum { B,G,R,A };
Now we don't need two levels, and the type-punning is more apparent.
ARGBCOLOR x = { 0x10203040 };
x.byte[B] = 50;
x.byte[G] = 60;
printf("0x%8x\n", x.dword); // prints: 0x10206050
It does the same thing that declaring the structure outside of the union would do, but allows you to do it anonymously, exposing the struct's members as the same scope as the other union members. Your construct in particular allows you to do something like
ARGBCOLOR a;
printf("a = %u.%u.%u.%u or %lu\n", a.B, a.G, a.R, a.A, a.ARGB);
(The union itself lets you see it as one of the types within the union, in this case you can see it either as a sequence of four BYTE values or as a single DWORD value)
If you hoisted the struct definition out of the union:
struct ARGB_VALUES {
BYTE B, G, R, A;
};
struct ARGBCOLOR {
union {
ARGB_VALUES VALUES; /* My god, how can you tell a type from a member? */
DWORD ARGB;
};
};
The previous printf would have to be:
ARGBCOLOR a;
printf("a = %u.%u.%u.%u or %lu\n", a.VALUES.B, a.VALUES.G, a.VALUES.R, a.VALUES.A, a.ARGB);
Note: The struct definition construct you're using is very archaic. You can reduce it as follows and save giving every structure/class a second alias that bloats your symbol table:
typedef struct ARGBCOLOR {
...
} *PARGBCOLOR;
The way you're doing it is equivalent to this:
struct _ARGBCOLOR {}; // First symbol.
typedef _ARGBCOLOR ARGBCOLOR;
typedef _ARGBCOLOR *PARGBCOLOR;
typedef struct _ARGBCOLOR
{
union
{
struct
{
BYTE B;
BYTE G;
BYTE R;
BYTE A;
};
DWORD ARGB;
};
} ARGBCOLOR, *PARGBCOLOR;
If you observe carefully, you can see a struct and a DWORD within the union. You can use either of these from your typedefed struct ARGBCOLOR, whichever is available
In a union, each data member starts at the same location in memory. The DWORD ARGB shares the memory location of the struct with 4 bytes. So, the size of union will be 4 bytes and you may access it as either as a whole using ARGB or byte-wise, using A, R, G, or B (a single byte can be modified without affecting other bytes).
The outermost struct is not really useful in your example, but it is when it has an additional member called kind or some such:
struct ARGBCOLOR {
uint8_t kind;
union {
struct {
BYTE B;
BYTE G;
BYTE R;
BYTE A;
};
DWORD ARGB;
};
} ARGBCOLOR;
Then depending on the value of kind you interpret the union as either a struct with members B, G, R and A, or the DWORD ARGB.
edit: Keep in mind that anonymous members such as the union and the inner struct only became part of the C standard in C11.
I'd like to define an enum to be smaller than one byte while maintaining type safety.
Defining an enum as:
enum MyEnum : unsigned char
{
i ,j, k, w
};
I can shrink it to one byte, however I'd like to make it use only 2 bits since I will at most have 4 values in it. Can this be done?
In my struct where I use the enum, the following does not work
struct MyStruct
{
MyEnum mEnum : 2; // This will be 4 bytes in size
};
Thanks!
Update:
The questions comes from this scenario:
enum MyEnum : unsigned char
{
i ,j, k, w
};
struct MyStruct
{
union
{
signed int mXa:3;
unsigned int mXb:3;
};
union
{
signed int mYa:3;
unsigned int mYb:3;
};
MyEnum mEnum:2;
};
sizeof(MyStruct) is showing 9 bytes. Ideally I'd like the struct to be 1 bytes in size.
Update for implemented solution:
This struct is one byte and offers the same functionality and type safety:
enum MyEnum :unsigned char
{
i,j,k,w
};
struct MyStruct
{
union
{
struct { MyEnum mEnum:2; char mXa:3; char mXb:3;};
struct { MyEnum mEnum:2; unsigned char mYa:3; unsigned char mYb:3;};
};
};
As per standard definition, a types sizeof must be at least 1 byte. This is the smallest addressable unit of memory.
The feature of bitfields you are mentioning allows to define members of structures to have smaller sizes, but the struct itself may not be because
It must be of at least 1 byte too
Alignment considerations might need it to be even bigger
additionally you may not take the address of bitfield members, since as said above, a byte is the smallest addressable unit of memory (You can already see that by sizeofactually returning the number of bytes, not bits, so if you expected less than CHAR_BIT bits, sizeof would not even be able to express it).
bitfields can only share space if they use the same underlying type. And any unused bits are actually left unused; if the sum of bits in an unsigned int bitfield is 3 bits, it still takes 4 bytes total. Since both enums have unsigned int members, they're both 4 bytes, but since they are bitfields, they have an alignment of one. So the first enum is 4 bytes, and the second is four bytes, then the MyEnum is 1 byte. Since all of those have an alignment of one, no padding is needed.
Unfortunately, union doesn't really work with bitfields really at all. Bitfields are for integer types only. The most I could get your data to without serious redesign is 3 bytes: http://coliru.stacked-crooked.com/view?id=c6ad03c93d7893ca2095fabc7f72ca48-e54ee7a04e4b807da0930236d4cc94dc
enum MyEnum : unsigned char
{
i ,j, k, w
};
union MyUnion
{
signed char ma:3; //char to save memory
unsigned char mb:3;
};
struct MyStruct
{
MyUnion X;
MyUnion Y;
MyEnum mEnum;
}; //this structure is three bytes
In the complete redesign category, you have this: http://coliru.stacked-crooked.com/view?id=58269eef03981e5c219bf86167972906-e54ee7a04e4b807da0930236d4cc94dc
No. C++ defines "char" to be the smallest addressable unit of memory for the platform. You can't address 2 bits.
Bit packing 'Works for me'
#include <iostream>
enum MyEnum : unsigned char
{
i ,j, k, w
};
struct MyStruct
{
MyEnum mEnum : 2;
unsigned char val : 6;
};
int main()
{
std::cout << sizeof(MyStruct);
}
prints out 1. How / what are you measuring?
Edit: Live link
Are you doing something like having a pointer as the next thing in the struct? In which case, you'll have 30bits of dead space as pointers must be 4 byte aligned on most 32bit systems.
Edit: With your updated example, its the unions which are breaking you
enum MyEnum : unsigned char
{
i ,j, k, w
};
struct MyStruct
{
unsigned char mXb:3;
unsigned char mYb:3;
MyEnum mEnum:2;
};
Has size 1. I'm not sure how unions and bit packing work together though, so I'm no more help.
Can anybody please explain what's going on?
My MSVC 2008 project's structure member alignment setting is set to 16 bytes (/Zp16) alignment, however one of the following structures is being aligned by 16 bytes and another is aligned only by 8 bytes... WHY?!!!
struct HashData
{
void *pData;
const char* pName;
int crc;
bool bModified;
}; // sizeof (HashData) == 4 + 4 + 4 + 1 + padding = 16 bytes, ok
class StringHash
{
HashData data[1024];
int mask;
int size;
}; // sizeof(StringHash) == 1024 * 16 + 4 + 4 + 0 = 16392 bytes, why not 16400 bytes?
This may not look like a big deal, but it's a big problem for me, since I am forced to emulate the MSVC structures alignment in GCC and specifying the aligned(16) attribute makes the sizeof (StringHash) == 16400!
Please tell me, when and why MSVC overrides the /Zp16 setting, I absolutely can't fathom it...
I think you misunderstood the /Zp16 option.
MSDN says,
When you specify this option, each structure member after the first is
stored on either the size of the member type or n-byte boundaries
(where n is 1, 2, 4, 8, or 16), whichever is smaller.
Please read the "whichever is smaller". It doesn't say that the struct will be padded by 16. It rather defines the boundary of each member relative to each other, starting from the first member.
What you basically want is align (C++) attribute, which says
Use __declspec(align(#)) to precisely control the alignment of user-defined data
So try this:
_declspec(align(16)) struct StringHash
{
HashData data[1024];
int mask;
int size;
};
std::cout << sizeof(StringHash) << std::endl;
It should print what you expect.
Or you can use #pragma pack(16).
Consider using the pack pragma directive:
// Set packing to 16 byte alignment
#pragma pack(16)
struct HashData
{
void *pData;
const char* pName;
int crc;
bool bModified;
};
class StringHash
{
HashData data[1024];
int mask;
int size;
};
// Restore default packing
#pragma pack()
See: pack and Working with Packing Structures