What is data structure padding in c++ and how do i check the number of bytes padded bytes?
class a { public: int x; int y; int z; };
Processors require that certain types of data have particular alignments. For example, a processor might require that an int be on a 4-byte boundary. So, for example, an int could start at memory location 0x4000 but it could not start at 0x4001. So if you defined a class:
class a
{
public:
char c;
int i;
};
the compiler would have to insert padding between c and i so that i could start on a 4-byte boundary.
struct A
{
char c;
int i;
};
int main(int argc, char *argv[])
{
A a;
cout << "sizeof struct = " << sizeof(A) << endl;
cout << "sizeof items = " << sizeof(a.c) + sizeof(a.i) << endl;
return 0;
}
padding is done for performance reasons - see this article Data Structure Alignment for more info.
To see whether the compiler pads your data structure you could write a simple program:
#include <iostream>
class a {
public:
int x;
int y;
int z;
};
int main()
{
std::cout << sizeof(a) << std::endl; // print the total size in bytes required per class instance
a anInstance;
std::cout << &anInstance.x << std::endl; // print the address of the x member
std::cout << &anInstance.y << std::endl; // as above but for y
std::cout << &anInstance.z << std::endl; // etc
}
I added the public declaration to avoid compiler errors - It will not affect the size or padding.
Edit: Running this on my macbook air gives the following output:
12
0x7fff5fbff650
0x7fff5fbff654
0x7fff5fbff658
This shows that on my machine the total size is 12 bytes, and each member is 4 bytes apart. The ints are 4 bytes each (which can be confirmed with sizeof(int)). There is no padding.
Try this with different members in your class, for example:
class b {
public:
char w;
char x[6];
int y;
long long z;
};
Lol just create 2 identical structs, make one of them packed
e.g.
struct foo {
int a;
char b;
int c;
}
struct bar {
int a;
char b;
int c;
} __attribute__((__packed__));
sizeof(foo) - sizeof(bar)
will give you the amount of padding. Or you could also calculate manually like Duck suggested.
Related
I want to be able to find the size of the individual members in a struct. For example
struct A {
int a0;
char a1;
}
Now sizeof(A) is 8, but let's assume I am writing a function that will print the alignment of A as shown below where "aa" represents the padding.
data A:
0x00: 00 00 00 00
0x04: 00 aa aa aa
*-------------------------
size: 8 padding: 3
In order for me to calculate padding, I need to know the size of each individual members of a struct. So my question is how can I access to individual members of a given struct.
Also, let me know if there is another way to find the number of padding.
A simple approach would be to use sizeof operator (exploiting the fact that it does not evaluate its operand, only determines the size of the type that would result if it was evaluated) and the offsetof() macro (from <cstddef>).
For example;
#include <iostream>
#include <cstddef>
struct A
{
int a0;
char a1;
};
int main()
{
// first calculate sizes
size_t size_A = sizeof(A);
size_t size_a0 = sizeof(((A *)nullptr)->a0); // sizeof will not dereference null
size_t size_a1 = sizeof(((A *)nullptr)->a1);
// calculate positions
size_t pos_a0 = offsetof(A, a0); // will be zero, but calculate it anyway
size_t pos_a1 = offsetof(A, a1);
// now calculate padding amounts
size_t padding_a0 = pos_a1 - pos_a0 - size_a0; // padding between a0 and a1 members
size_t padding_a1 = size_A - pos_a1 - size_a1;
std::cout << "Data A:\n";
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << pos_a0;
size_t i = pos_a0;
while (i < pos_a0 + size_a0) // print out zeros for bytes of a0 member
{
std::cout << " 00";
++i;
}
while (i < pos_a1) // print out aa for each padding byte after a_0
{
std::cout << " aa";
++i;
}
std::cout << std::endl;
std::cout << "0x" << std::hex << std::setw(2) << std::setfill('0') << pos_a1;
while (i < pos_a1 + size_a1) // print out zeros for bytes of a1 member
{
std::cout << " 00";
++i;
}
while (i < size_A) // print out aa for each padding byte after a_1
{
std::cout << " aa";
++i;
}
std::cout << std::endl;
std::cout << "size: " << size_A << " padding: " << padding_a0 + padding_a1 << std::endl;
}
You can work this out if you know the content of the struct. Passing usually works like this,
Assume that your struct is this.
struct A {
int a0; // 32 bits (4 bytes)
char a1; // 8 bits (1 byte)
};
But this is not memory efficient as if you pack these structs in memory, you might get some fragmentation issues thus making the application slower. So the compiler optimizes the struct like this and the final struct to the compiler would look something like this.
struct A {
int a0;
char a1;
// padding
char __padd[3]; // 3 * 1 byte = 3 bytes.
/// Adding this padding makes it 32 bit aligned.
};
Now using this knowledge, you can see why its not easy to get the padding of an object without knowing the content of it. And paddings aren't always placed at the end of the object. For example,
struct Obj {
int a = 0;
char c = 'b';
// Padding is set here by the compiler. It may look something like this: char __padd[3];
int b = 10;
};
So how to get the padding of the struct?
You can use something called Reflection to get the content of the struct at runtime. Then workout the sizes of the data types in the struct and then you can calculate the padding by deducting the size of the previous type and the next type which gives you how the padding would look like.
As other answers have said, the offsetof macro is clearly the best solution here, but just to demonstrate that you could find the positions of your members at run time by looking at the pointers:
#include <iostream>
struct A
{
char x;
int y;
char z;
};
template <typename T>
void PrintSize ()
{
std::cout << " size = " << sizeof(T) << std::endl;
}
void PrintPosition (char * ptr_mem, char * ptr_base)
{
std::cout << " position = " << ptr_mem - ptr_base << std::endl;
}
template <typename T>
void PrintDetails (char member, T * ptr_mem, A * ptr_base)
{
std::cout << member << ":" << std::endl;
PrintSize<T>();
PrintPosition((char*) ptr_mem, (char*) ptr_base);
std::cout << std::endl;
}
int main()
{
A a;
PrintDetails('x', &a.x, &a);
PrintDetails('y', &a.y, &a);
PrintDetails('z', &a.z, &a);
}
Output on my machine:
x:
size = 1
position = 0
y:
size = 4
position = 4
z:
size = 1
position = 8
(Surprisingly, on my intel, with gcc/clang, A is of size 12! I thought that the compiler did a better job of rearranging elements)
To calculate the padding of a structure, you need to know the offset of the last member, and the size:
Concisely, if type T has a member last which is of type U, the padding size is:
sizeof(T) - (offsetof(T, last) + sizeof(U))
To calculate the total amount of padding in a structure, if that is what this question is about, I would use a GCC extension: declare the same structure twice (perhaps with the help of a macro), once without the packed attribute and once with. Then subtract their sizes.
Here is a complete, working sample:
#include <stdio.h>
#define X struct { char a; int b; char c; }
int main(void)
{
printf("%zd\n", sizeof(X) - sizeof(X __attribute__((packed))));
return 0;
}
For the above structure, it outputs 6. This corresponds to the 3 bytes of padding after a necessary for the four-byte alignment of b and at the end of the structure, necessary for the alignment of b if the structure is used as an array member.
The packed attribute defeats all padding, and so the difference between the packed and unpacked structure gives us the total amount of padding.
I've been reading up on placement new, and I'm not sure if I'm "getting" it fully or not when it comes to proper alignment.
I've written the following test program to attempt to allocate some memory to an aligned spot:
#include <iostream>
#include <cstdint>
using namespace std;
unsigned char* mem = nullptr;
struct A
{
double d;
char c[5];
};
struct B
{
float f;
int a;
char c[2];
double d;
};
void InitMemory()
{
mem = new unsigned char[1024];
}
int main() {
// your code goes here
InitMemory();
//512 byte blocks to write structs A and B to, purposefully misaligned
unsigned char* memoryBlockForStructA = mem + 1;
unsigned char* memoryBlockForStructB = mem + 512;
unsigned char* firstAInMemory = (unsigned char*)(uintptr_t(memoryBlockForStructA) + uintptr_t(alignof(A) - 1) & ~uintptr_t(alignof(A) - 1));
A* firstA = new(firstAInMemory) A();
A* secondA = new(firstA + 1) A();
A* thirdA = new(firstA + 2) A();
cout << "Alignment of A Block: " << endl;
cout << "Memory Start: " << (void*)&(*memoryBlockForStructA) << endl;
cout << "Starting Address of firstA: " << (void*)&(*firstA) << endl;
cout << "Starting Address of secondA: " << (void*)&(*secondA) << endl;
cout << "Starting Address of thirdA: " << (void*)&(*thirdA) << endl;
cout << "Sizeof(A): " << sizeof(A) << endl << "Alignof(A): " << alignof(A) << endl;
return 0;
}
Output:
Alignment of A Block:
Memory Start: 0x563fe1239c21
Starting Address of firstA: 0x563fe1239c28
Starting Address of secondA: 0x563fe1239c38
Starting Address of thirdA: 0x563fe1239c48
Sizeof(A): 16
Alignof(A): 8
The output appears to be valid, but I still have some questions about it.
Some questions I have are:
Will fourthA, fifthA, etc... all be aligned as well?
Is there a simpler way of finding a properly aligned memory location?
In the case of struct B, it is set up to not be memory friendly. Do I need to reconstruct it so that the largest members are at the top of the struct, and the smallest members are at the bottom? Or will the compiler automatically pad everything so that it's member d will not be malaligned?
Will fourthA, fifthA, etc... all be aligned as well?
yes if the alignement of a type is a multiple of the size
witch is (i think) always the case
Is there a simpler way of finding a properly aligned memory location?
yes
http://en.cppreference.com/w/cpp/language/alignas
or
http://en.cppreference.com/w/cpp/memory/align
as Dan M said.
In the case of struct B, it is set up to not be memory friendly. Do I need to reconstruct it so that the largest members are at the top of the struct, and the smallest members are at the bottom? Or will the compiler automatically pad everything so that it's member d will not be malaligned?
you should reorganize if you think about it.
i don't think compiler will reorganize element in a struct for you.
because often when interpreting raw data (coming from file, network ...) this data is often just interpreted as a struct and 2 compiler reorganizing differently could break code.
I hope my explanation are clear and that I did not make any mistakes
I was playing with some reference to template base member code. Out of curiosity, I wondered if the compiler optimised out my references behind the scenes somehow (as the references were all to the same variables in the parent class every time). I decided a simple sizeof() test would tell me. To my great surprise, simply referencing the 3 float members of the class I was inheriting from, blew my class memory footprint up by 333%. This surprised and confused me greatly.
After some fiddling, I don't even know what is going on (but I suspect I'm being dumb).
Code: (compiled in Code::Blocks 13.12 using GCC 4.8.2 on Ubuntu 14.04)
#include <iostream>
template <typename T, unsigned int N>
class FooArray{
public:
FooArray(){
for (size_t i = 0; i < N; ++i)
{
m_data[i] = static_cast<T>(0);
}
}
FooArray(const T& _krv){
for (size_t i = 0; i < N; ++i)
{
m_data[i] = _krv;
}
}
T& operator[](unsigned int _index);
private:
T m_data[N];
};
template <typename T, unsigned int N>
T&
FooArray<T, N>::operator[](unsigned int _index)
{
return m_data[_index];
}
class Inherit0Ref : public FooArray<float, 3>{};
class Inherit1Ref : public FooArray<float, 3>
{
public:
Inherit1Ref():a((*this)[0]){}
float& a;
};
class Inherit2Ref : public FooArray<float, 3>
{
public:
Inherit2Ref():a((*this)[0]), b((*this)[1]){}
float& a;
float& b;
};
class Inherit3Ref : public FooArray<float, 3>
{
public:
Inherit3Ref():a((*this)[0]), b((*this)[1]), c((*this)[2]){}
float& a;
float& b;
float& c;
};
class Inherit2RefMul : public FooArray<float, 3>
{
public:
Inherit2RefMul():a((*this)[0]), b((*this)[1]){}
float& a, b;
};
class Inherit3RefMul : public FooArray<float, 3>
{
public:
Inherit3RefMul():a((*this)[0]), b((*this)[1]), c((*this)[2]){}
float& a, b, c;
};
class FloatRef
{
public:
FloatRef(float& _r):a(_r){}
float& a;
};
class WrapFloat
{
float pad;
};
class PadFloatRef
{
public:
PadFloatRef():a(pad), pad(0.0f){}
float& a;
float pad;
};
int main()
{
Inherit3Ref test;
test.a = 1.0f;
test.b = 2.0f;
test.c = 3.14f;
std::cout << test[0] << ", " << test[1] << ", " << test[2] << std::endl;
std::cout << "float size: " << sizeof(float) << std::endl;
std::cout << "float& size: " << sizeof(float&) << std::endl;
std::cout << "FloatRef size: " << sizeof(FloatRef) << std::endl;
std::cout << "WrapFloat size: " << sizeof(WrapFloat) << std::endl;
std::cout << "PadFloatRef size: " << sizeof(PadFloatRef) << std::endl;
std::cout << "FooArray<float, 3> size: " << sizeof(FooArray<float, 3>) << std::endl;
std::cout << "Inherit0Ref size: " << sizeof(Inherit0Ref) << std::endl;
std::cout << "Inherit1Ref size: " << sizeof(Inherit1Ref) << std::endl;
std::cout << "Inherit2Ref size: " << sizeof(Inherit2Ref) << std::endl;
std::cout << "Inherit3Ref size: " << sizeof(Inherit3Ref) << std::endl;
std::cout << "Inherit2RefMul size: " << sizeof(Inherit2RefMul) << std::endl;
std::cout << "Inherit3RefMul size: " << sizeof(Inherit3RefMul) << std::endl;
// Supposedly size 32, lets test assignment and access.
Inherit3RefMul testvar;
testvar.a = 5.0f;
testvar.b = 4.0f;
testvar.c = 3.142f;
// Interesting...
// testvar: 5, 0, 0
std::cout << "testvar: " << testvar[0] << ", " << testvar[1] << ", " << testvar[2] << std::endl;
return 0;
}
Output:
1, 2, 3.14
float size: 4
float& size: 4
FloatRef size: 8
WrapFloat size: 4
PadFloatRef size: 16
FooArray<float, 3> size: 12
Inherit0Ref size: 12
Inherit1Ref size: 24
Inherit2Ref size: 32
Inherit3Ref size: 40
Inherit2RefMul size: 32
Inherit3RefMul size: 32
testvar: 5, 0, 0
Very curious. :)
Why is a wrapped float reference(FloatRef) twice the size of a wrapped float(WrapFloat)?
Why does declaring 3 float references using one float reference keyword (Inherit3RefMul) yield the same footprint as the same class with 2 references rather than 3(Inherit2Ref)?
Why has only TestVar's first variable been populated while all 3 of test's are populated as expected?
Is there some way to achieve the functionality of Inherit3Ref without the ridiculous memory footprint of 40?
EDIT:
Please note:
sizeof(float&) == 4
Update:
Running alignof(float&) returns 4. I don't see the reason for a float[3] float&[3] class to have size 40 when all variables have size of 4 and align of 4.
The compiler must add padding to ensure that any member variables are appropriately aligned. float has size 4 and alignment 4, while references and pointers (to whatever) have typically size 8 (on 64bit machines) and alignment 8. Hence:
struct foo {
float m_data[3]; // size 3*4=12 begin=0 end=12
float&a; // size 8 begin=16 end=24 NOTE: 4 unused bytes
};
(Also note that
float&a,b;
is equivalent to
float&a;
float b;
but not to
typedef float&float_ref;
float_ref a,b;
this notation is hence best avoided for clarity)
A float is 4-bytes, a reference is 8-bytes. So an array of 3 floats will be 12-bytes. The only non-trivial bit is your Inherit3Ref, which will look like:
float[3];
*** padding to 8 byte boundary ***
float &[3];
Hence 12 bytes for the floats + 4 bytes additional padding + 24 bytes for the references to make 40. If you want to avoid the padding, you can make the class packed - which should get you down to 36 bytes.
Not sure why you call it "ridiculous" - it certainly has to be at least 36...
The code that I've written:
#include <iostream>
using std::cout;
using std::endl;
struct S
{
long l;
};
struct A
{
int a;
};
struct B : A, S{ };
int main()
{
cout << "sizeof(A) = " << sizeof(A) << endl; //4
cout << "sizeof(S) = " << sizeof(S) << endl; //8
cout << "sizeof(B) = " << sizeof(B) << endl; //16 != 4 + 8
}
demo
Why do we have to allocate an additional 4 bytes for B? How this additional memory is used?
Memory image of a B instance:
int a; // bytes 0-3
long l; // bytes 4-11
Problem:
l is an 8-byte variable, but its address is not aligned to 8 bytes. As a result, unless the underlying HW architecture supports unaligned load/store operations, the compiler cannot generate correct assembly code.
Solution:
The compiler adds a 4-byte padding after variable a, in order to align variable l to an 8-byte address.
Memory image of a B instance:
int a; // bytes 0-3
int p; // bytes 4-7
long l; // bytes 8-15
For POD data type, there can be padding and alignments.
In B's memory layout, int comes first, then long. The compiler likely inserts 4 bytes of padding between them to 8-align the long assuming any instance of the structure will also have its starting address 8-aligned.
Similar things would happen if you put an A and a S instance on your stack: the compiler would leave 4 bytes empty in between.
See Data Structure Alignment.
I know there should be a delete operator with which I'm not concerned in somewhere. I just wondering, wow, it worked. Where is the argument "size" coming from?
#include<iostream>
#include<string>
class Base {
public:
Base() { }
void *operator new( unsigned int size, std::string str ) {
std::cout << "Logging an allocation of ";
std::cout << size;
std::cout << " bytes for new object '";
std::cout << str;
std::cout << "'";
std::cout << std::endl;
return malloc( size );
}
private:
int var1;
double var2;
};
int main(int argc, char** argv){
Base* b = new ("Base instance 1") Base;
}
Here is the result:
Logging an allocation of 16 bytes for new object 'Base instance 1'
It is provided by the compiler at compile time. When the compiler sees:
new ("Base instance 1") Base;
it will add a call to:
Base::operator new(sizeof(Base), "Base instance 1");
EDIT: The compiler will of course also add a call to Base::Base()
on 32 bit arch int is 4 bytes, double is 8, but the double is going to be aligned to 8 byte boundary so size = 4 + 4(empty space) + 8 = 16