struct hack - zero sized array - c++

#include <iostream>
using namespace std;
struct node1{
char b[3];
int c[0];
};
struct node2{
int c[0];
};
struct node3{
char b[3];
};
int main() {
cout << sizeof(node1) << endl; // prints 4
cout << sizeof(node2) << endl; // prints 0
cout << sizeof(node3) << endl; // prints 3
}
My Question is why does the compiler allocate 0 bytes for int c[0] in node2
but allocate 1 byte for its when part of node1.
I'm assuming that this 1 byte is the reason why sizeof(node1) returns 4 since without it (like in node3) its size is 3 or is that due to padding??
Also trying to understand that shouldn't node2 have enough space to hold a pointer to an array (which will be allocated in the further down in the code as part of the flexible array/struct hack?

Yes, it's about padding/alignment. If you add __attribute__((__packed__)) to the end [useful when writing device drivers], you'll get 3 0 3 for your output.
If node1 had defined c[1], the size is 8 not 7, because the compiler will align c to an int boundary. With packed, sizeof would be 7

Yes, padding makes the difference. The reason why node1 has a padding byte, while node3 doesn't, lies in the typical usage of zero-length arrays.
Zero-length arrays are typically used with casting: You cast a larger, (possibly variable-sized) object to the struct containing the zero-length array. Then you access the "rest" of the large object using the zero-length array, which, for this purpose, has to be aligned properly. The padding byte is inserted before the zero-sized array, such that the ints are aligned. Since you can't do that with node3, no padding is needed.
Example:
struct Message {
char Type[3];
int Data[]; // it compiles without putting 0 explicitly
};
void ReceiveMessage(unsigned char* buffer, size_t length) {
if(length < sizeof(Message))
return;
Message* msg = (Message*)buffer;
if(!memcmp(msg->Type, "GET", 3)) {
HandleGet(msg->Data, (length - sizeof(Message))/sizeof(int));
} else if....
Note: this is rather hackish, but efficient.

c doesn't allocate one byte in node1. Its because of the padding added to b.
For b, to be easily obtainable by a 32-bit CPU, it is four bytes big. 32-bit CPUs can read 4 consecutive bytes from memory at a time. To read three, they have to read four and then remove the one not necessary. Therefore, to optimize this behavior, the compiler padds the struct with some bytes.
You can observe similar compiler optimizations when values are pushed on the stack (that is, arguments or local variables are allocated). The stack is always kept aligned to the CPU's data bus size (commonly 32 or 64 bits).

int main() {
cout << sizeof(node1) << endl; // prints 4
cout << sizeof(node2) << endl; // prints 0
cout << sizeof(node3) << endl; // prints 3
}
the main function queries the the size of the user defined structs, not of the array members. sizeof() will return the number of bytes allocated to the struct, with each character allocated in the character array being allocated 1 byte. A character array is really a C style string which is terminated by the sentinel character '\0'. It is likely to include the byte allocated to hold the sentinel character when evaluating the sizeof(node1) as there is another variable after it so it reads over it, but not include the sentinel in sizeof(node3) where the string and the struct terminates

Related

What is the memory lay out of a C++ structure?

I was learning about the structures in C++ and got to know that if a structure in C++ has 3 variables (let each one be of some data type), then all of them are not allocated in a contiguous fashion. Is this correct?
If yes, then how much memory would be allocated to an object to that structure type?
For e.g. Let's say we have a structure like this:
struct a
{
int x;
int y;
char c;
};
Now how intuitively an object of type a, must occupy some space = sizeOf(int) + sizeOf(int) + sizeOf(char). But, if they are not allocated continuously, there could be some memory locations allocated for that object, that are just present for providing some padding - i.e. the memory allocated for that object could look something like this:
xxxx[4-bytes]xxxxx[4-bytes]xxxx[1-byte]xxx
(NOTE: In the above blockquote x corresponds to a memory location of size 1 byte. I also assumed that sizeOf(int) = 4-bytes and sizeOf(char) = 1- byte.)
So in the above one, we can see that the object a occupies more than 9-bytes (because there are some memory locations (x's) used for padding.)
So, does something like this happen?
Thanks for your replies!
P.S. Please let me know if I hadn't written something clearly.
When it comes to structs and classes, The layout is implementation-defined between compilers, os, and architecture... Some will use automatic alignment, others will use padding, some may even auto arrange it's members. If you need to know the size of a struct, use sizeof(Your Struct).
Here's a code snippet...
#include <iostream>
struct A {
char a;
float b;
int c;
};
struct B {
float a;
int b;
char c;
};
int main() {
std::cout << "Sizeof(A) = " << sizeof(A) << '\n';
std::cout << "Sizeof(B) = " << sizeof(B) << '\n';
return 0;
}
Output:
Sizeof(A) = 12
Sizeof(B) = 12
For my particular machine, I'm running Windows 7 - 64bit, It is an Intel Core2 Quad Extreme, and I'm using Visual Studio 2017 running it with C++17.
With my particular setup, both structures are being generated with a different layout, but have the same size in bytes.
In A's case...
char a; // 1 byte
// 3 bytes of padding
float b; // 4 bytes
int c; // 4 bytes (int is 32bit even on x64).
In B's case...
float a; // 4 bytes
int b; // 4 bytes
char c; // 1 byte
// 3 bytes of padding.
Also, your compiler flags and optimizations may have an effect. This isn't always guaranteed, as it is implementation-defined as stated in the standard.
--Edit--
Also, if you don't want this exact behavior there are some pragmas directives and macros such as pragma pack and alignas() that can be used to modify your implementation details. Here are a few references.
How to use alignas to replace pragma pack?
https://en.cppreference.com/w/cpp/preprocessor/impl
https://en.cppreference.com/w/cpp/language/alignas
https://learn.microsoft.com/en-us/cpp/cpp/alignment-cpp-declarations?view=msvc-160
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.cbclx01/pragma_pack.htm
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.cbclx01/packqua.htm
https://www.iditect.com/how-to/57426535.html
https://downloads.ctfassets.net/oxjq45e8ilak/1LriV4eAdhNlu9Zv06H9NJ/53576095f772b5f6cddbbedccb7ebd8a/Alexander_Titov_Know_your_hardware_CPU_memory_hierarchy.pdf
https://cpc110.blogspot.com/2020/10/vs2019-alignas-in-struct-definition.html
So, structure Object variables take contiguous memory locations and the variables are stored in memory in the same order in which they are defined.
Please refer to the code below you will get an idea.
struct c{
int a;
int b;
int c;
};
int main()
{
c obj;
cout << &(obj.a) << endl; //0x7ffe128fe1c4
cout << &(obj.b) << endl; //0x7ffe128fe1c8
cout << &(obj.c) << endl; //0x7ffe128fe1cc
return 0;
}
For Structures Padding Concept Refer here

Struct sizes and address of struct members [duplicate]

This question already has answers here:
Weird Pointer Address for Individual Struct Data Member
(2 answers)
Closed 8 years ago.
2 questions about the below code:
1 Why is the size of struct b 12 bytes (line 2 of output)? I can understand why a is 12 bytes (to align k on the 4 byte boundary), however shouldnt b be 9 bytes and not 12?
2 Why is using the & operator to get the address of the char members not displaying a valid address? (middle output in the last 2 lines)
#include<iostream>
using namespace std;
struct a
{
int i;
char j;
int k;
};
struct b
{
int i;
int k;
char j;
};
int main()
{
a s1;
b s2;
cout<<sizeof(a)<<endl;
cout<<sizeof(b)<<endl;
cout<<sizeof(int)<<endl;
cout<<sizeof(char)<<endl;
cout<<&s1.i<<'\t'<<&s1.j<<'\t'<<&s1.k<<endl;
cout<<&s2.i<<'\t'<<&s2.j<<'\t'<<&s2.k<<endl;
}
Output:
12
12
4
1
0x28ff14 - 0x28ff1c
0x28ff08 4A 0x28ff0c
If b were 9 bytes large and you had an array
b foo[2];
then foo[1].i would not be aligned to four-byte boundaries.
As for the char members, their address is of type char *, and std::ostream has an operator<< for those that interprets them as a C-style string. If you want to see their address, static_cast<void*>(&s1.j) is your friend.
EDIT: Full disclosure of egg on my face: This originally stated that some compilers defined minimum struct alignment sizes; upon investigation, this turns out to be untrue. Well, it could be true, since the size of struct types is not specified in the standard to that extent, but common compilers don't appear to do it. Sorry about that.
Why is the size of struct b 12 bytes (line 2 of output)?
It has int fields, and so has the same alignment requirement as int - four bytes, in your case. Three padding bytes are added at the end, so the size is a multiple of four, to give the required alignment.
This applies even at the end of the structure, so that the next element of an array is properly aligned.
Why is using the & operator to get the address of the char members not displaying a valid address?
Because << interprets a pointer to a character type as a pointer to a C-style string, and prints memory contents until it finds a zero value. To print the address, cast to a non-character pointer type:
cout << (void*)&j;

Processing Arrays

An Example demonstrating Passing an array as argument
#include <iostream>
#include <malloc.h>
using namespace std;
typedef unsigned char U8;
#define MAX_LEN 20
void func1(U8* arr)
{
printf(" Passing Base address Element1 = %s\n",arr);
}
void func2(U8* arr)
{
printf(" Passing Pointer address Element1 = %s \n",arr);
}
int main()
{
U8 arr[MAX_LEN] = "Stack Overflow\n";
U8* arr1 = (U8*)malloc(sizeof(MAX_LEN));
func1(arr); /* Passing Base address */
memcpy(arr1,arr,sizeof(arr));
/*
memcpy(arr1,arr,sizeof(MAX_LEN)); Leads to Heap Corruption why ?
*/
func2(arr1);/* Passing pointer */
free(arr1);
cout << "Array Freed" << endl;
cin.get();
return 0;
}
Queries :
1. which is the best Practise in consideration with data Processing.[data on stack or Heap]
2. Please suggest reliable methodology to be used for such cases
memcpy(arr1,arr,sizeof(MAX_LEN)); // Leads to Heap Corruption why ?
Because sizeof(MAX_LEN) is equivalent to sizeof(20) which is equivalent to sizeof(int).
This means you'll copy 4 or 8 bytes (depending on your platform). In fun1 you then print the array as if it were a null terminated string. There is no null terminator though, since you didn't copy it and printf happily runs out of bounds.
sizeof(arr), on the other hand, gives you correct size of 20.
sizeof operator queries the size of the type of the expression you give it as operand, not the value. It's purely compile time operator. The type of integer literal 20 is int and it'll return the size of that type.
Honestly? If you write in C++, simply use std::vector, pass it by reference and forget about the whole problem.
If you really have to use C-style arrays, I'd say, that in most cases (I mean, 99.9%), you'll have to allocate the array on the heap, because programs usually have limited stack memory available and placing an array there is generally not a good idea.
Remember though, that constant text expressions defined in the following way:
const char * myText = "Alice has a cat";
may not be stored on the stack, but somewhere (depending on C++ compiler). These won't occupy place on the stack and mostly probably this is the case in your example. In this example, a pointer to that text is stored on the stack, but the text itself is (mostly probably) stored elsewhere.

Why struct size do not match with plain sum of item's sizes? [duplicate]

This question already has answers here:
Why isn't sizeof for a struct equal to the sum of sizeof of each member?
(13 answers)
Closed 9 years ago.
With C/C++ structs, normally, the final struct's size is the plain sum of the sizes of all its elements, but there are cases which this is not true.
I am looking at the technical answer (it has to be one) of why the following struct's size is not equal to the size of all its members:
#include <iostream>
struct {
int a;
long b;
char c;
} typedef MyStruct;
int main() {
MyStruct sss;
std::cout << "Size of int: " << sizeof(int) << std::endl << "Size of long: " << sizeof(long) << std::endl << "Size of char: " << sizeof(char) << std::endl;
std::cout << "Size of MyStruct: " << sizeof(sss) << std::endl;
return 0;
}
Thas has the following output:
Size of int: 4
Size of long: 8
Size of char: 1
Size of MyStruct: 24
So it can be seen than the size of MyStruct is not 4+8+1 (13) rather, it is actually 24, but why?
Because C allows compilers to add padding between the elements of a struct to make the generated code more performant.
Some hardware allows accessing multi-byte data faster when such data is placed at memory addresses divisible by the size of the data. For example, accessing a 32-bit int may go significantly faster when the int is located at address 0x8004 or 0x8008 than when the same int is located at an address 0x8003 or 0x8006. The standard allows compilers to adjust offsets of struct members to make use of such optimization.
Moreover, there are platforms where accessing a multi-byte value not aligned with the memory space results in an error: for example, reading a 16-bit int from an odd address in 68000 architecture triggers bus error. Compilers know this, and they add unused space to your struct so that accessing multi-byte fields does not trigger an error.

How is memory allocated for stack variables?

On VS (release), I run the following:
int main(void)
{
char b[] = "123";
char a[] = "1234567";
printf("%x %x\n", b,a);
return 0;
}
I can see that, the mem address of a is b+3(the length of the string). Which shows that the memory are allocated with no gaps. And this guarantee that least memories are used.
So, I now kind of believe that all compilers will do so.
I want to make sure of this guess here. Can somebody give me an more formal proof or tell me that my guess is rooted on a coincidence.
No, it's not guaranteed that there will always be perfect packing of data.
For example, I compiled and runned this code on g++, and the difference is 8.
You can read more about this here.
tl;dr: Compilers can align objects in memory to only addresses divisible by some constant (always machine-word length) to help processor(for them it's easier to work with such addresses)
UPD: one interesting example about alignment:
#include <iostream>
using namespace std;
struct A
{
int a;
char b;
int c;
char d;
};
struct B
{
int a;
int c;
char b;
char d;
};
int main()
{
cout << sizeof(A) << " " << sizeof(B) << "\n";
}
For me, it prints
16 12
There is no guarantee what addresses will be chosen for each variable. Different processors may have different requirements or preferences for alignment of variables for instance.
Also, I hope there were at least 4 bytes between the addresses in your example. "123" requires 4 bytes - the extra byte being for the null terminator.
Try reversing the order of declaring a[] and b[], and/or increase the length of b.
You are making a very big assumption about how storage is allocated. Depending on your compiler the string literals might get stored in a literal pool that is NOT on the stack. Yet a[] and b[] do occupy elements on the stack. So, another test would be to add int c and compare those addresses.