I was learning about the structures in C++ and got to know that if a structure in C++ has 3 variables (let each one be of some data type), then all of them are not allocated in a contiguous fashion. Is this correct?
If yes, then how much memory would be allocated to an object to that structure type?
For e.g. Let's say we have a structure like this:
struct a
{
int x;
int y;
char c;
};
Now how intuitively an object of type a, must occupy some space = sizeOf(int) + sizeOf(int) + sizeOf(char). But, if they are not allocated continuously, there could be some memory locations allocated for that object, that are just present for providing some padding - i.e. the memory allocated for that object could look something like this:
xxxx[4-bytes]xxxxx[4-bytes]xxxx[1-byte]xxx
(NOTE: In the above blockquote x corresponds to a memory location of size 1 byte. I also assumed that sizeOf(int) = 4-bytes and sizeOf(char) = 1- byte.)
So in the above one, we can see that the object a occupies more than 9-bytes (because there are some memory locations (x's) used for padding.)
So, does something like this happen?
Thanks for your replies!
P.S. Please let me know if I hadn't written something clearly.
When it comes to structs and classes, The layout is implementation-defined between compilers, os, and architecture... Some will use automatic alignment, others will use padding, some may even auto arrange it's members. If you need to know the size of a struct, use sizeof(Your Struct).
Here's a code snippet...
#include <iostream>
struct A {
char a;
float b;
int c;
};
struct B {
float a;
int b;
char c;
};
int main() {
std::cout << "Sizeof(A) = " << sizeof(A) << '\n';
std::cout << "Sizeof(B) = " << sizeof(B) << '\n';
return 0;
}
Output:
Sizeof(A) = 12
Sizeof(B) = 12
For my particular machine, I'm running Windows 7 - 64bit, It is an Intel Core2 Quad Extreme, and I'm using Visual Studio 2017 running it with C++17.
With my particular setup, both structures are being generated with a different layout, but have the same size in bytes.
In A's case...
char a; // 1 byte
// 3 bytes of padding
float b; // 4 bytes
int c; // 4 bytes (int is 32bit even on x64).
In B's case...
float a; // 4 bytes
int b; // 4 bytes
char c; // 1 byte
// 3 bytes of padding.
Also, your compiler flags and optimizations may have an effect. This isn't always guaranteed, as it is implementation-defined as stated in the standard.
--Edit--
Also, if you don't want this exact behavior there are some pragmas directives and macros such as pragma pack and alignas() that can be used to modify your implementation details. Here are a few references.
How to use alignas to replace pragma pack?
https://en.cppreference.com/w/cpp/preprocessor/impl
https://en.cppreference.com/w/cpp/language/alignas
https://learn.microsoft.com/en-us/cpp/cpp/alignment-cpp-declarations?view=msvc-160
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.cbclx01/pragma_pack.htm
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.cbclx01/packqua.htm
https://www.iditect.com/how-to/57426535.html
https://downloads.ctfassets.net/oxjq45e8ilak/1LriV4eAdhNlu9Zv06H9NJ/53576095f772b5f6cddbbedccb7ebd8a/Alexander_Titov_Know_your_hardware_CPU_memory_hierarchy.pdf
https://cpc110.blogspot.com/2020/10/vs2019-alignas-in-struct-definition.html
So, structure Object variables take contiguous memory locations and the variables are stored in memory in the same order in which they are defined.
Please refer to the code below you will get an idea.
struct c{
int a;
int b;
int c;
};
int main()
{
c obj;
cout << &(obj.a) << endl; //0x7ffe128fe1c4
cout << &(obj.b) << endl; //0x7ffe128fe1c8
cout << &(obj.c) << endl; //0x7ffe128fe1cc
return 0;
}
For Structures Padding Concept Refer here
Related
Consider the following structure:
struct S
{
int a;
int b;
double arr[0];
} __attribute__((packed));
As you can see, this structure is packed and has Zero sized array at the end.
I'd like to send this as binary data over the network (assume I took care of endianity).
In C/C++ I could just use malloc to allocate as much space as I want and use free later.
I'd like this memory to be handled by std::shared_ptr.
Is there a straight forward way of doing so without special hacks?
I'd like this memory to be handled by std::shared_ptr.
Is there a straight forward way of doing so without special hacks?
Sure, there is:
shared_ptr<S> make_buffer(size_t s)
{
auto buffer = malloc(s); // allocate as usual
auto release = [](void* p) { free(p); }; // a deleter
shared_ptr<void> sptr(buffer, release); // make it shared
return { sptr, new(buffer) S }; // an aliased pointer
}
This works with any objects that are placed in a malloced buffer, not just when there are zero-sized arrays, provided that the destructor is trivial (performs no action) because it is never called.
The usual caveats about zero-sized arrays and packed structures still apply as well, of course.
double arr[0];
} __attribute__((packed));
Zero sized arrays are not allowed as data members (nor as any other variable) in C++. Furthermore, there is no such attribute as packed in C++; it is a language extension (as such, it may be considered to be a special hack). __attribute__ itself is a language extension. The standard syntax for function attributes uses nested square brackets like this: [[example_attribute]].
I'd like to send this as binary data over the network
You probably should properly serialise the data. There are many serialisation specifications although none of them is universally ubiquitous and none of them is implemented in the C++ standard library. Indeed, there isn't a standard API for network commnication either.
A straightforward solution is to pick an existing serialisation format and use an existing library that implements it.
First, let me explain again why I have this packed structure:
it is used for serialization of data over the network so there's a header file with all network packet structures.
I know it generates bad assembly due to alignment issues, but I guess that this problem persists with regular serialization (copy to char * buffer with memcpy).
Zero size arrays are supported both by gcc and clang which I use.
Here's an example of a full program with a solution to my question and it's output (same output for gcc and g++).
compiled with -O3 -std=c++17 flags
#include <iostream>
#include <memory>
#include <type_traits>
#include <cstddef>
struct S1
{
~S1() {std::cout << "deleting s1" << std::endl;}
char a;
int b;
int c[0];
} __attribute__((packed));
struct S2
{
char a;
int b;
int c[0];
};
int main(int argc, char **argv)
{
auto s1 = std::shared_ptr<S1>(static_cast<S1 *>(::operator
new(sizeof(S1) + sizeof(int) * 1e6)));
std::cout << "is standart: " << std::is_standard_layout<S1>::value << std::endl;
for (int i = 0; i < 1e6; ++i)
{
s1->c[i] = i;
}
std::cout << sizeof(S1) << ", " << sizeof(S2) << std::endl;
std::cout << offsetof(S1, c) << std::endl;
std::cout << offsetof(S2, c) << std::endl;
return 0;
}
This is the output:
is standart: 1
5, 8
5
8
deleting s1
Is there anything wrong with doing this?
I made sure using valgrind all allocations/frees work properly.
#include <iostream>
using namespace std;
struct node1{
char b[3];
int c[0];
};
struct node2{
int c[0];
};
struct node3{
char b[3];
};
int main() {
cout << sizeof(node1) << endl; // prints 4
cout << sizeof(node2) << endl; // prints 0
cout << sizeof(node3) << endl; // prints 3
}
My Question is why does the compiler allocate 0 bytes for int c[0] in node2
but allocate 1 byte for its when part of node1.
I'm assuming that this 1 byte is the reason why sizeof(node1) returns 4 since without it (like in node3) its size is 3 or is that due to padding??
Also trying to understand that shouldn't node2 have enough space to hold a pointer to an array (which will be allocated in the further down in the code as part of the flexible array/struct hack?
Yes, it's about padding/alignment. If you add __attribute__((__packed__)) to the end [useful when writing device drivers], you'll get 3 0 3 for your output.
If node1 had defined c[1], the size is 8 not 7, because the compiler will align c to an int boundary. With packed, sizeof would be 7
Yes, padding makes the difference. The reason why node1 has a padding byte, while node3 doesn't, lies in the typical usage of zero-length arrays.
Zero-length arrays are typically used with casting: You cast a larger, (possibly variable-sized) object to the struct containing the zero-length array. Then you access the "rest" of the large object using the zero-length array, which, for this purpose, has to be aligned properly. The padding byte is inserted before the zero-sized array, such that the ints are aligned. Since you can't do that with node3, no padding is needed.
Example:
struct Message {
char Type[3];
int Data[]; // it compiles without putting 0 explicitly
};
void ReceiveMessage(unsigned char* buffer, size_t length) {
if(length < sizeof(Message))
return;
Message* msg = (Message*)buffer;
if(!memcmp(msg->Type, "GET", 3)) {
HandleGet(msg->Data, (length - sizeof(Message))/sizeof(int));
} else if....
Note: this is rather hackish, but efficient.
c doesn't allocate one byte in node1. Its because of the padding added to b.
For b, to be easily obtainable by a 32-bit CPU, it is four bytes big. 32-bit CPUs can read 4 consecutive bytes from memory at a time. To read three, they have to read four and then remove the one not necessary. Therefore, to optimize this behavior, the compiler padds the struct with some bytes.
You can observe similar compiler optimizations when values are pushed on the stack (that is, arguments or local variables are allocated). The stack is always kept aligned to the CPU's data bus size (commonly 32 or 64 bits).
int main() {
cout << sizeof(node1) << endl; // prints 4
cout << sizeof(node2) << endl; // prints 0
cout << sizeof(node3) << endl; // prints 3
}
the main function queries the the size of the user defined structs, not of the array members. sizeof() will return the number of bytes allocated to the struct, with each character allocated in the character array being allocated 1 byte. A character array is really a C style string which is terminated by the sentinel character '\0'. It is likely to include the byte allocated to hold the sentinel character when evaluating the sizeof(node1) as there is another variable after it so it reads over it, but not include the sentinel in sizeof(node3) where the string and the struct terminates
Giving simple structure (POD) containing only one array of shorts (bytes, ints from <cstdint>, etc) and no more fields will be added later:
#define FIXED_SIZE 128 // 'fixed' in long term, shouldn’t change in future versions
struct Foo {
uint16_t bar[FIXED_SIZE];
};
is it any possibility to end up with padding at the end of the structure added by compiler for any reason ?
It seems reasonable not to make any padding as it is no any obvious need of it, but is it any guarantees by standard (could you provide any links where it is explained)?
Later I would like to use arrays of Foo structs in simple serialization (IPC) within different platforms and don't want to use any libraries for this simple task (code simplified for demonstration):
#define FOO_ELEMS 1024
...
// sender
Foo *from = new Foo[FOO_ELEMS];
uint8_t *buff_to = new uint8_t[FOO_ELEMS * FIXED_SIZE * sizeof(uint16_t) ];
memcpy(buff_to, from, ...);
...
// receiver
uint8_t *buff_from = new uint8_t[ ... ];
Foo *to = new Foo[FOO_ELEMS];
memcpy(to, buff_from, ...);
I would like to use struct here instead of plain arrays as it will be some auxiliary methods within struct and it seems more convenient then to use plain functions + arrays pointers instead.
Intersects with this (plain C) question, but seems a little bit different for me:
Alignment of char array struct members in C standard
The various standards provide for padding to occur (but not at the start).
There is no strict requirement at all that it will only appear to align the members and the object in arrays.
So the truly conformant answer is:
Yes, there may be padding because the compiler can add it but not at the start or between array elements.
There is no standard way of forcing packing either.
However every time this comes up and every time I ask no one has ever identified a real compiler on a platform that pads structures for any other reason than for internal alignment and array alignment.
So for all know practical purposes that structure will not be packed on any known platform.
Please consider this yet another request for someone to find a real platform that breaks that principle.
Since we are already guaranteed that there will no padding at the beginning of the structure don't have to worry about that. At the end I could see padding being added if the sizeof of the array was not divisible by the word size of the machine.
The only way I could get any padding to be added to the struct though was to add an int member to the struct as well. In doing so the struct was padded to make them the same size.
#include <iostream>
#include <cstdint>
struct a
{
uint16_t bar[128];
};
struct b
{
uint16_t bar[127];
};
struct c
{
int test;
uint16_t bar[128];
};
struct d
{
int test;
uint16_t bar[127];
};
struct e
{
uint16_t bar[128];
int test;
};
struct f
{
uint16_t bar[127];
int test;
};
int main()
{
std::cout << sizeof(a) << "\t" << sizeof(b) << "\t" << sizeof(c) << "\t" << sizeof(d) << "\t" << sizeof(e) << "\t" << sizeof(f);
}
Live Example
I am trying to understand the object layout for C++ in multiple inheritance.
For this purpose, I have two superclasses A, B and one subclass C.
What I expected to see when trying dumping it was:
vfptr | fields of A | vfptr | fields of B | fields of C.
I get this model but with some zeros that I don't understand.
Here is the code I am trying
#include <iostream>
using namespace std;
class A{
public:
int a;
A(){ a = 5; }
virtual void foo(){ }
};
class B{
public:
int b;
B(){ b = 10; }
virtual void foo2(){ }
};
class C : public A, public B{
public:
int c;
C(){ c = 15; a = 20;}
virtual void foo2(){ cout << "Heeello!\n";}
};
int main()
{
C c;
int *ptr;
ptr = (int *)&c;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
ptr++;
cout << *ptr << endl;
return 0;
}
And here is the output I get:
4198384 //vfptr
0
20 // value of a
0
4198416 //vfptr
0
10 // value of b
15 // value of c
What is the meaning of the zeros in between?
Thanks in advance!
That depends upon your compiler. With clang-500, I get:
191787296
1
20
0
191787328
1
10
15
1785512560
I am sure there's a GDB way too, but this is what I get if I dump pointer-sized words with LLDB at the address of the class object:
0x7fff5fbff9d0: 0x0000000100002120 vtable for C + 16
0x7fff5fbff9d8: 0x0000000000000014
0x7fff5fbff9e0: 0x0000000100002140 vtable for C + 48
0x7fff5fbff9e8: 0x0000000f0000000a
This layout seems sensible, right? Just what you expect.
The reason why that doesn't show as clean in your program as it does in the debugger is that you are dumping int-sized words. On a 64-bit system sizeof(int)==4 but sizeof(void*)==8
So, you see your pointers split into (int,int) pairs. On Linux, your pointers don't have any bit set beyond the low 32, on OSX my pointers do - hence the reason for the 0 vs. 1 disparity
this is hugely architecture and compiler dependant... Possibly for you the size of a pointer might not be the size of an int... What architecture/compiler are you using?
If you're working on a 64-bit system, then:
The first zero is the 4 most-significant-bytes of the first vfptr.
The second zero is padding, so that the second vfptr will be aligned to an 8-byte address.
The third zero is the 4 most-significant-bytes of the second vfptr.
You can check if sizeof(void*) == 8 in order to assert that.
Hard to tell without knowing your platform and compiler, but this might be an alignment issue. In effect, the compiler might attempt to align class data along 8-byte boundaries, with zeroes used for padding.
Without the above details, this is merely speculation.
This is completely dependant on your compiler, system, bitness.
The virtual table pointer will have the size of a pointer. This depends on whether you are compiling your file as 32-bit or 64-bit. Pointers will also be aligned at a multiple address of their size (like any type will typically be). This is probably why you are seeing the 0 padding after the 20.
The integers will have the size of an integer on your specific system. This is usually always 32-bit. Note that if this isn't the case on your machine you will get unexpected results because you are increasing your ptr by sizeof(int) with pointer arithmetic.
If you use MVSC, you can dump all memory layout of all class in your solution with -d1reportAllClassLayout like that:
cl -d1reportAllClassLayout main.cpp
Hope it helpful to you
On VS (release), I run the following:
int main(void)
{
char b[] = "123";
char a[] = "1234567";
printf("%x %x\n", b,a);
return 0;
}
I can see that, the mem address of a is b+3(the length of the string). Which shows that the memory are allocated with no gaps. And this guarantee that least memories are used.
So, I now kind of believe that all compilers will do so.
I want to make sure of this guess here. Can somebody give me an more formal proof or tell me that my guess is rooted on a coincidence.
No, it's not guaranteed that there will always be perfect packing of data.
For example, I compiled and runned this code on g++, and the difference is 8.
You can read more about this here.
tl;dr: Compilers can align objects in memory to only addresses divisible by some constant (always machine-word length) to help processor(for them it's easier to work with such addresses)
UPD: one interesting example about alignment:
#include <iostream>
using namespace std;
struct A
{
int a;
char b;
int c;
char d;
};
struct B
{
int a;
int c;
char b;
char d;
};
int main()
{
cout << sizeof(A) << " " << sizeof(B) << "\n";
}
For me, it prints
16 12
There is no guarantee what addresses will be chosen for each variable. Different processors may have different requirements or preferences for alignment of variables for instance.
Also, I hope there were at least 4 bytes between the addresses in your example. "123" requires 4 bytes - the extra byte being for the null terminator.
Try reversing the order of declaring a[] and b[], and/or increase the length of b.
You are making a very big assumption about how storage is allocated. Depending on your compiler the string literals might get stored in a literal pool that is NOT on the stack. Yet a[] and b[] do occupy elements on the stack. So, another test would be to add int c and compare those addresses.