I have a struct iof_header in my code, and I determined it would be 24 bytes wide. I perform a sizeof(iof_header) and it returns 32 bytes wide.
Question 1
Why is it 32 bytes wide instead of 24?
Question 2
Including its members, how is a struct stored in memory?
Question 3
I find any time I create one of my structs that bytes[4-8 & 20-24] are all NULL, I see this apparent in my char array. The array reads as follows {4 bytes of BASEID_Code, 4 NULL bytes, 8 bytes of zeroed padding, 4 bytes of ASID_Code, 4 NULL bytes, 8 bytes of size}
There are NULL bytes at the ends of my unsigned __int32 members, why is this happening?
Is this possibly compile related? Possibly an efficiency thing to make the CPU able to process these data types faster?
struct iof_header
{
union
{
struct
{
unsigned __int32 BASEID_Code;
unsigned __int64 padding;
union
{
char ASID_Type[4];
unsigned __int32 ASID_Code;
};
unsigned __int64 Size;
}header;
char header_c[24];
};
iof_header()
{
header.ASID_Code = 0;
header.BASEID_Code = 0;
header.Size = 0;
header.padding = 0;
}
};
Why is it 32 bytes wide instead of 24?
Probably because padding is added before each __int64 member to meet their alignment requirements.
Including its members, how is a struct stored in memory?
The members are stored in order, with padding inserted where necessary to correctly align each member relative to the start of the structure.
Some compilers have a non-standard extension to "pack" the members, so that padding is not inserted. For example, on GCC you can put __attribute__((packed)) after the structure definition.
Possibly an efficiency thing to make the CPU able to process these data types faster?
Yes. On some processors, unaligned accesses are slow; on others, they aren't allowed at all, and must be emulated by two or more accesses.
A compiler is free to add padding bytes after members to preserve alignment requirements. Your __int64 members are probably aligned to 8 bytes, ergo the 4 padding bytes between BASEID_Code and padding.
Related
Someone explain me how does the order of the member declaration inside a class determines the size of that class.
For Example :
class temp
{
public:
int i;
short s;
char c;
};
The size of above class is 8 bytes.
But when the order of the member declaration is changed as below
class temp
{
public:
char c;
int i;
short s;
};
then the size of class is 12 bytes.
How?
The reason behind above behavior is data structure alignment and padding. Basically if you are creating a 4 byte variable e.g. int, it will be aligned to a four byte boundary i.e. it will start from an address in memory, which is multiple of 4. Same applies to other data types. 2 byte short should start from even memory address and so on.
Hence if you have a 1 byte character declared before the int (assume 4 byte here), there will be 3 free bytes left in between. The common term used for them is 'padded'.
Data structure alignment
Another good pictorial explanation
Reason for alignment
Padding allows faster memory access i.e. for cpu, accessing memory areas that are aligned is faster e.g. reading a 4 byte aligned integer might take a single read call where as if an integer is located at a non aligned address range (say address 0x0002 - 0x0006), then it would take two memory reads to get this integer.
One way to force compiler to avoid alignment is (specific to gcc/g++) to use keyword 'packed' with the structure attribute. packed keyword Also the link specifies how to enforce alignment by a specific boundary of your choice (2, 4, 8 etc.) using the aligned keyword.
Best practice
It is always a good idea to structure your class/struct in a way that variables are already aligned with minimum padding. This reduces the size of the class overall plus it reduces the amount of work done by the compiler i.e. no rearrangement of structure. Also one should always access member variables by their names in the code, rather than trying to read a specific byte from structure assuming a value would be located at that byte.
Another useful SO question on performance advantage of alignment
For the sake of completion, following would still have a size of 8 bytes in your scenario (32 bit machine), but it won't get any better since full 8 bytes are now occupied, and there is no padding.
class temp
{
public:
int i;
short s;
char c;
char c2;
};
class temp
{
public:
int i; //size 4 alignment 4
short s; //size 2 alignment 2
char c; //size 1 alignment 1
}; //Size 8 alignment max(4,2,1)=4
temp[i[0-4];s[4-2];c[6-7]]] -> 8
Padding in (7-8)
class temp
{
public:
char c; //size 1 alignment 1
int i; //size 4 alignment 4
short s; //size 2 alignment 2
};//Size 12 alignment max(4,2,1)=4
temp[c[0-1];i[4-8];s[8-10]]] -> 12
Padding in (1-4) and (10-12)
Someone explain me how does the order of the member declaration inside a class determines the size of that class.
For Example :
class temp
{
public:
int i;
short s;
char c;
};
The size of above class is 8 bytes.
But when the order of the member declaration is changed as below
class temp
{
public:
char c;
int i;
short s;
};
then the size of class is 12 bytes.
How?
The reason behind above behavior is data structure alignment and padding. Basically if you are creating a 4 byte variable e.g. int, it will be aligned to a four byte boundary i.e. it will start from an address in memory, which is multiple of 4. Same applies to other data types. 2 byte short should start from even memory address and so on.
Hence if you have a 1 byte character declared before the int (assume 4 byte here), there will be 3 free bytes left in between. The common term used for them is 'padded'.
Data structure alignment
Another good pictorial explanation
Reason for alignment
Padding allows faster memory access i.e. for cpu, accessing memory areas that are aligned is faster e.g. reading a 4 byte aligned integer might take a single read call where as if an integer is located at a non aligned address range (say address 0x0002 - 0x0006), then it would take two memory reads to get this integer.
One way to force compiler to avoid alignment is (specific to gcc/g++) to use keyword 'packed' with the structure attribute. packed keyword Also the link specifies how to enforce alignment by a specific boundary of your choice (2, 4, 8 etc.) using the aligned keyword.
Best practice
It is always a good idea to structure your class/struct in a way that variables are already aligned with minimum padding. This reduces the size of the class overall plus it reduces the amount of work done by the compiler i.e. no rearrangement of structure. Also one should always access member variables by their names in the code, rather than trying to read a specific byte from structure assuming a value would be located at that byte.
Another useful SO question on performance advantage of alignment
For the sake of completion, following would still have a size of 8 bytes in your scenario (32 bit machine), but it won't get any better since full 8 bytes are now occupied, and there is no padding.
class temp
{
public:
int i;
short s;
char c;
char c2;
};
class temp
{
public:
int i; //size 4 alignment 4
short s; //size 2 alignment 2
char c; //size 1 alignment 1
}; //Size 8 alignment max(4,2,1)=4
temp[i[0-4];s[4-2];c[6-7]]] -> 8
Padding in (7-8)
class temp
{
public:
char c; //size 1 alignment 1
int i; //size 4 alignment 4
short s; //size 2 alignment 2
};//Size 12 alignment max(4,2,1)=4
temp[c[0-1];i[4-8];s[8-10]]] -> 12
Padding in (1-4) and (10-12)
I've written CMyObject class as follows:
class CMyObject
{
public:
CMyOjbect(void) {};
virtual ~CMyOjbect(void) {};
public:
ULONGLONG m_uField1;
UINT m_uField2;
BOOL m_bField3;
int m_iField4;
BOOL m_bField5;
}
To reduce the size of CMyObject, I changed it to:
class CMyObject
{
public:
CMyOjbect(void) {};
virtual ~CMyOjbect(void) {};
public:
ULONGLONG m_uField1;
UINT m_uField2;
short m_sField4; // Change from int to short, since short is enough for the data range
unsigned m_bField3: 1; // Change field 3 and 5 from BOOL to bit field to save spaces
unsigned m_bField5: 1;
}
However, the sizeof(CMyObject) is still not changed, why?
Can I use pargma pack(1) in a class to pack all the member variables, like this:
pargma pack(1)
class CMyObject
{
public:
CMyOjbect(void) {};
virtual ~CMyOjbect(void) {};
public:
ULONGLONG m_uField1;
UINT m_uField2;
short m_sField4; // Change from int to short, since short is enough for the data range
unsigned m_bField3: 1; // Change field 3 and 5 from BOOL to bit field to save spaces
unsigned m_bField5: 1;
}
pargma pack(0)
Because of your ULONGLONG first member, your structure will have 8-byte (64-bit) alignment. Assuming 32-bit ints, your first version uses 18 bytes, which would take 24 bytes to store. (The large and small members are interspersed, which makes the matter worse, but by my count, that doesn't change the answer here.) Your second version also takes 18 bytes:
8 bytes for m_uField1
4 bytes for m_uField2
2 bytes for m_sField4
4 bytes for the two unsigned bitfields (which will have 4-byte alignment, so will also inject 2 bytes of padding after m_sField4)
If you switch to short unsigned m_bField3:1 and short unsigned m_bField4:1, I think there's a good chance your structure will become smaller and fit in only 16 bytes.
Use of #pragma pack is not portable, so I can't comment on the specifics there, but it's possible that could shrink the size of your structure. I'm guessing it may not, though, since it's usually better at compensating for nonoptimal ordering of members with alignment, and by my counts, your variables themselves are just too big. (However, removing the alignment requirement on the ULONGLONG may shrink your structure size in either version, and #pragma pack may do that.)
As #slater mentions, in order to decrease the size and padding in your structure, you should declare your member variables of similar sizes next to each other. It's a pretty good rule of thumb to declare your variables in decreasing size order, which will tend to minimize padding and leverage coinciding alignment requirements.
However, the size of the structure isn't always the most important concern. Members are initialized in declaration order in the constructor, and for some classes, this matters, so you should take that into account. Additionally, if your structure spans multiple cache lines and will be used concurrently by multiple threads, you should ideally put variables that are used together nearby and in the same cache line and variables that are not used together in separate cache lines to reduce/eliminate false sharing.
In regards to your first question "However, the sizeof(CMyObject) is still not changed, why?"
Your BOOLs are not defined contiguously, so they are padded by the compiler for the purposes of memory-alignment.
On most 32-bit systems, this struct uses 16 bytes:
struct {
char b1; // 1 byte for char, 3 for padding
int i1; // 4 bytes
char b2; // 1 byte for char, 3 for padding
int i2; // 4 bytes
}
This struct uses 12 bytes:
struct {
char b1; // 1 byte
char b2; // 1 byte, 2 bytes for padding
int i1; // 4 bytes
int i2; // 4 bytes
}
Alignment and packing are implementation dependent, but typically you can request smaller alignment and better packing by using smaller types. That applies to specifying smaller types in bit-field declarations as well, since many compilers interpret the bit-field type as a request for allocation unit of that size specifically and alignment requirement of that type specifically.
In your case one obvious mistake is using unsigned for bit fields. Use unsigned char for bit fields and it should pack much better
class CMyObject
{
public:
CMyOjbect(void) {};
virtual ~CMyOjbect(void) {};
public:
ULONGLONG m_uField1;
UINT m_uField2;
short m_sField4;
unsigned char m_bField3: 1;
unsigned char m_bField5: 1;
};
This will not necessarily make it as compact as #pragma pack(1) can make it, but it will take it much closer to it.
Someone explain me how does the order of the member declaration inside a class determines the size of that class.
For Example :
class temp
{
public:
int i;
short s;
char c;
};
The size of above class is 8 bytes.
But when the order of the member declaration is changed as below
class temp
{
public:
char c;
int i;
short s;
};
then the size of class is 12 bytes.
How?
The reason behind above behavior is data structure alignment and padding. Basically if you are creating a 4 byte variable e.g. int, it will be aligned to a four byte boundary i.e. it will start from an address in memory, which is multiple of 4. Same applies to other data types. 2 byte short should start from even memory address and so on.
Hence if you have a 1 byte character declared before the int (assume 4 byte here), there will be 3 free bytes left in between. The common term used for them is 'padded'.
Data structure alignment
Another good pictorial explanation
Reason for alignment
Padding allows faster memory access i.e. for cpu, accessing memory areas that are aligned is faster e.g. reading a 4 byte aligned integer might take a single read call where as if an integer is located at a non aligned address range (say address 0x0002 - 0x0006), then it would take two memory reads to get this integer.
One way to force compiler to avoid alignment is (specific to gcc/g++) to use keyword 'packed' with the structure attribute. packed keyword Also the link specifies how to enforce alignment by a specific boundary of your choice (2, 4, 8 etc.) using the aligned keyword.
Best practice
It is always a good idea to structure your class/struct in a way that variables are already aligned with minimum padding. This reduces the size of the class overall plus it reduces the amount of work done by the compiler i.e. no rearrangement of structure. Also one should always access member variables by their names in the code, rather than trying to read a specific byte from structure assuming a value would be located at that byte.
Another useful SO question on performance advantage of alignment
For the sake of completion, following would still have a size of 8 bytes in your scenario (32 bit machine), but it won't get any better since full 8 bytes are now occupied, and there is no padding.
class temp
{
public:
int i;
short s;
char c;
char c2;
};
class temp
{
public:
int i; //size 4 alignment 4
short s; //size 2 alignment 2
char c; //size 1 alignment 1
}; //Size 8 alignment max(4,2,1)=4
temp[i[0-4];s[4-2];c[6-7]]] -> 8
Padding in (7-8)
class temp
{
public:
char c; //size 1 alignment 1
int i; //size 4 alignment 4
short s; //size 2 alignment 2
};//Size 12 alignment max(4,2,1)=4
temp[c[0-1];i[4-8];s[8-10]]] -> 12
Padding in (1-4) and (10-12)
I'm working on a 32-bit machine, so I suppose that the memory alignment should be 4 bytes. Say I have this struct:
typedef struct {
unsigned short v1;
unsigned short v2;
unsigned short v3;
} myStruct;
The plain added size is 6 bytes, and I suppose that the aligned size should be 8, but sizeof(myStruct) returns me 6.
However if I write:
typedef struct {
unsigned short v1;
unsigned short v2;
unsigned short v3;
int i;
} myStruct;
the plain added size is 10 bytes, aligned size shall be 12, and this time sizeof(myStruct) == 12.
Can somebody explain what is the difference?
At least on most machines, a type is only ever aligned to a boundary as large as the type itself [Edit: you can't really demand any "more" alignment than that, because you have to be able to create arrays, and you can't insert padding into an array]. On your implementation, short is apparently 2 bytes, and int 4 bytes.
That means your first struct is aligned to a 2-byte boundary. Since all the members are 2 bytes apiece, no padding is inserted between them.
The second contains a 4-byte item, which gets aligned to a 4-byte boundary. Since it's preceded by 6 bytes, 2 bytes of padding is inserted between v3 and i, giving 6 bytes of data in the shorts, two bytes of padding, and 4 more bytes of data in the int for a total of 12.
Forget about having different members, even if you write two structs whose members are exactly same, with a difference is that the order in which they're declared is different, then size of each struct can be (and often is) different.
For example, see this,
#include <iostream>
using namespace std;
struct A
{
char c;
char d;
int i;
};
struct B
{
char c;
int i; //note the order is different!
char d;
};
int main() {
cout << sizeof(A) << endl;
cout << sizeof(B) << endl;
}
Compile it with gcc-4.3.4, and you get this output:
8
12
That is, sizes are different even though both structs has same members!
Code at Ideone : http://ideone.com/HGGVl
The bottomline is that the Standard doesn't talk about how padding should be done, and so the compilers are free to make any decision and you cannot assume all compilers make the same decision.
By default, values are aligned according to their size. So a 2-byte value like a short is aligned on a 2-byte boundary, and a 4-byte value like an int is aligned on a 4-byte boundary
In your example, 2 bytes of padding are added before i to ensure that i falls on a 4-byte boundary.
(The entire structure is aligned on a boundary at least as big as the biggest value in the structure, so your structure will be aligned to a 4-byte boundary.)
The actual rules vary according to the platform - the Wikipedia page on Data structure alignment has more details.
Compilers typically let you control the packing via (for example) #pragma pack directives.
Assuming:
sizeof(unsigned short) == 2
sizeof(int) == 4
Then I personally would use the following (your compiler may differ):
unsigned shorts are aligned to 2 byte boundaries
int will be aligned to 4 byte boundaries.
typedef struct
{
unsigned short v1; // 0 bytes offset
unsigned short v2; // 2 bytes offset
unsigned short v3; // 4 bytes offset
} myStruct; // End 6 bytes.
// No part is required to align tighter than 2 bytes.
// So whole structure can be 2 byte aligned.
typedef struct
{
unsigned short v1; // 0 bytes offset
unsigned short v2; // 2 bytes offset
unsigned short v3; // 4 bytes offset
/// Padding // 6-7 padding (so i is 4 byte aligned)
int i; // 8 bytes offset
} myStruct; // End 12 bytes
// Whole structure needs to be 4 byte aligned.
// So that i is correctly aligned.
Firstly, while the specifics of padding are left up to the compiler, the OS also imposes some rules as to alignment requirements. This answer assumes that you are using gcc, though the OS may vary
To determine the space occupied by a given struct and its elements, you can follow these rules:
First, assume that the struct always starts at an address that is properly aligned for all data types.
Then for every entry in the struct:
The minimum space needed is the raw size of the element given by sizeof(element).
The alignment requirement of the element is the alignment requirement of the element's base type.
Notably, this means that the alignment requirement for a char[20] array is the same as
the requirement for a plain char.
Finally, the alignment requirement of the struct as a whole is the maximum of the alignment requirements of each of its elements.
gcc will insert padding after a given element to ensure that the next one (or the struct if we are talking about the last element) is correctly aligned. It will never rearrange the order of the elements in the struct, even if that will save memory.
Now the alignment requirements themselves are also a bit odd.
32-bit Linux requires that 2-byte data types have 2-byte alignment (their addresses must be even). All larger data types must have 4-byte alignment (addresses ending in 0x0, 0x4, 0x8 or 0xC). Note that this applies to types larger than 4 bytes as well (such as double and long double).
32-bit Windows is more strict in that if a type is K bytes in size, it must be K byte aligned. This means that a double can only placed at an address ending in 0x0 or 0x8. The only exception to this is the long double which is still 4-byte aligned even though it is actually 12-bytes long.
For both Linux and Windows, on 64-bit machines, a K byte type must be K byte aligned. Again, the long double is an exception and must be 16-byte aligned.
Each data type needs to be aligned on a memory boundary of its own size. So a short needs to be on aligned on a 2-byte boundary, and an int needs to be on a 4-byte boundary. Similarly, a long long would need to be on an 8-byte boundary.
The reason for the second sizeof(myStruct) being 12 is the padding that gets inserted between v3 and i to align i at a 32-bit boundary. There is two bytes of it.
Wikipedia explains the padding and alignment reasonably clearly.
In your first struct, since every item is of size short, the whole struct can be aligned on short boundaries, so it doesn't need to add any padding at the end.
In the second struct, the int (presumably 32 bits) needs to be word aligned so it inserts padding between v3 and i to align i.
Sounds like its being aligned to bounderies based on the size of each var, so that the address is a multiple of the size being accessed(so shorts are aligned to 2, ints aligned to 4 etc), if you moved one of the shorts after the int, sizeof(mystruct) should be 10. Of course this all depends on the compiler being used and what settings its using in turn.
The standard doesn't say much about the layout of structs with complete types - it's up to to the compiler. It decided that it needs the int to start on a boundary to access it, but since it has to do sub-boundary memory addressing for the shorts there is no need to pad them