I am using eclipse with cygwin. The application is 64bit. In cygwin the structure is defined as :
struct addrinfo {
int ai_flags; /* input flags */
int ai_family; /* address family of socket */
int ai_socktype; /* socket type */
int ai_protocol; /* ai_protocol */
socklen_t ai_addrlen; /* length of socket address */
char *ai_canonname; /* canonical name of service location */
struct sockaddr *ai_addr; /* socket address of socket */
struct addrinfo *ai_next; /* pointer to next in list */
};
The sizeof(addrinfo) result is 48. The size of socketlen_t is 4 bytes. The int type size is 4 bytes. The pointer is 8 bytes in the 64 bits application. The total bytes is 44(4 ints = 16 bytes, socket_len = 4 bytes, 3 pointers = 24; 20+4+24 = 44). I am wondering what the missing 4 bytes for? Are they for padding? I thought 44 bytes do not need to be aligned. Any thought?
Thanks for the answer in advance.
It's padding, after the socklen_t. The next variable in the struct is a pointer, which is 64-bits in length, and will (in this case) be aligned to 64-bits as well. Note however that padding is dependent on architecture and compiler settings; it happens here, but is not guaranteed to always happen.
Note that since you are sharing this struct with the operating system, you should NOT try to change the padding yourself (most compilers allow this using compiler switches and/or pragmas). The OS is expecting this struct with a certain amount of padding included. If you fail to provide it, all the pointers at the end of the struct will have their values misinterpreted.
It's "struct member alignment", the /Zp flag
Data structure alignment,
Microsoft documentation
Related
I am taking binary input from a file to a buffer vector then casting the pointer of that buffer to be my struct type.
The goal is for the data to populate the struct perfectly.
I know the size of all the various fields and the order they're going to come in.
As a result my struct needs to be tightly packed and be 42 bytes long.
My issue is that it is coming out at 44 bytes long when I test it.
Also, the first value lines up. After that, the data is incorrect.
Here's the struct:
#pragma pack(push, 1)
struct myStruct
{
uint8_t ID;
uint32_t size: 24;
uint16_t value;
char name[12];
char description[4];
char shoppingList[14];
char otherValue[6];
};
#pragma pack(pop)
Also, the first value lines up. After that, the data is incorrect.
uint32_t size: 24;
If you want to guarantee portably that this is three bytes with no padding before the next member, you're going to need to use a byte buffer and do the conversions yourself.
#pragma pack is an extension, and the packing of bitfield members is anyway implementation-defined.
FWIW both GCC and CLANG do seem to do what you want in this case, but unless it's defined by a platform ABI depending on this is still brittle.
I have a struct iof_header in my code, and I determined it would be 24 bytes wide. I perform a sizeof(iof_header) and it returns 32 bytes wide.
Question 1
Why is it 32 bytes wide instead of 24?
Question 2
Including its members, how is a struct stored in memory?
Question 3
I find any time I create one of my structs that bytes[4-8 & 20-24] are all NULL, I see this apparent in my char array. The array reads as follows {4 bytes of BASEID_Code, 4 NULL bytes, 8 bytes of zeroed padding, 4 bytes of ASID_Code, 4 NULL bytes, 8 bytes of size}
There are NULL bytes at the ends of my unsigned __int32 members, why is this happening?
Is this possibly compile related? Possibly an efficiency thing to make the CPU able to process these data types faster?
struct iof_header
{
union
{
struct
{
unsigned __int32 BASEID_Code;
unsigned __int64 padding;
union
{
char ASID_Type[4];
unsigned __int32 ASID_Code;
};
unsigned __int64 Size;
}header;
char header_c[24];
};
iof_header()
{
header.ASID_Code = 0;
header.BASEID_Code = 0;
header.Size = 0;
header.padding = 0;
}
};
Why is it 32 bytes wide instead of 24?
Probably because padding is added before each __int64 member to meet their alignment requirements.
Including its members, how is a struct stored in memory?
The members are stored in order, with padding inserted where necessary to correctly align each member relative to the start of the structure.
Some compilers have a non-standard extension to "pack" the members, so that padding is not inserted. For example, on GCC you can put __attribute__((packed)) after the structure definition.
Possibly an efficiency thing to make the CPU able to process these data types faster?
Yes. On some processors, unaligned accesses are slow; on others, they aren't allowed at all, and must be emulated by two or more accesses.
A compiler is free to add padding bytes after members to preserve alignment requirements. Your __int64 members are probably aligned to 8 bytes, ergo the 4 padding bytes between BASEID_Code and padding.
How can I know the actual structure size?
using sizeof returns the number of bytes after alignment.
For example :
struct s {
char c;
int i
}
sizeof(s) = 8;
I am interested to get the size of bytes without alignment ,i.e 5
The actual size of structure is size of structure after alignment. I mean sizeof return size of structure in the memory. if you want to have an structure that aligned on byte you should tell the compiler to do that. for example something like this:
#ifdef _MSVC
# pragma pack(push)
# pragma pack(1)
# define PACKED
#else
# define PACKED __attribute((packed))
#endif
struct byte_aligned {
char c;
int i;
} PACKED;
#ifdef _MSVC
# pragma pack(pop)
#endif
now sizeof(byte_aligned) return 5 and its actually 5 byte in memory
sizeof returns the actual structure size.
The size without padding is a bit meaningless, because the compiler will insert the padding it feels appropriate. Some compilers support pragmas that turn off structure padding, in which case you should use that, but you'll probably get all sorts of fun things happening.
You have misunderstood the documentation.
using sizeof returns the number of bytes after alignment
This is incorrect. sizeof returns the number of bytes after padding, which is not the same thing.
Thus, if what you want to know is "how much space will my struct take", sizeof() is telling you the correct answer.
Let's say I want to send the following data to a socket using C or C++, all in one packet:
Headers
-------
Field 1: 2 byte hex
Field 2: 2 byte hex
Field 3: 4 byte hex
Data
----
Field1 : 2 byte hex
Field1 : 8 byte hex
What would the code typically look like to create and send the packet containing all this data?
Let's suppose that your program is already organized to have the header in one struct and the data in another struct. For example, you might have these data structures:
#include <stdint.h>
struct header {
uint16_t f1;
uint16_t f2;
uint32_t f3;
};
struct data {
uint16_t pf1;
uint64_t pf2;
};
Let's call this organization "host format". It really doesn't matter to me what the host format is, as long as it is useful to the rest of your program. Let's call the format that you will pass to the send() call "network format". (I chose these names to match the htons (host-to-network-short) and htonl (host-to-network-long) names.)
Here are some conversion functions that we might find handy. Each of these converts your host format structures to a network format buffer.
#include <arpa/inet.h>
#include <string.h>
void htonHeader(struct header h, char buffer[8]) {
uint16_t u16;
uint32_t u32;
u16 = htons(h.f1);
memcpy(buffer+0, &u16, 2);
u16 = htons(h.f2);
memcpy(buffer+2, &u16, 2);
u32 = htonl(h.f3);
memcpy(buffer+4, &u32, 4);
}
void htonData(struct data d, char buffer[10]) {
uint16_t u16;
uint32_t u32;
u16 = htons(d.pf1);
memcpy(buffer+0, &u16, 2);
u32 = htonl(d.pf2>>32);
memcpy(buffer+2, &u32, 4);
u32 = htonl(d.pf2);
memcpy(buffer+6, u32, 4);
}
void htonHeaderData(struct header h, struct data d, char buffer[18]) {
htonHeader(h, buffer+0);
htonData(d, buffer+8);
}
To send your data, do this:
...
char buffer[18];
htonHeaderData(myPacketHeader, myPacketData, buffer);
send(sockfd, buffer, 18, 0);
...
Again, you don't have to use the header and data structs that I defined. Just use whatever your program needs. The key is that you have a conversion function that writes all of the data, at well-defined offsets, in a well-defined byte order, to a buffer, and that you pass that buffer to the send() function.
On the other side of the network connection, you will need a program to interpret the data it receives. On that side, you need to write the corresponding functions (ntohHeader, etc). Those function will memcpy the bits out of a buffer and into a local variable, which it can pass to ntohs or ntohl. I'll leave those functions for you to write.
Well, typically it would look like it's preparing that packet structure into a memory buffer (making judicious calls the the htonl family of functions).
If would then use the send, sendto, sendmsg or write functions, hopefully with a lot of care taken with the length of the buffer and good error handling/reporting.
(Or one of the Win32 apis for the send, if that is the target plateforms.)
You'll find a good presentation about all this at Beej's Guide to Network Programming.
Specifially for the byte packing part (with endian consideration), look at the serialization topic. (There's way more detail in that section than what you need for plain fixed-size integer data types.
The code would look different depending on the OS's networking library (*nix uses Berkeley sockets, Windows uses Winsock, etc.). However, you could create a struct containing all the data you wanted to send in a packet, e.g.,
typedef struct
{
short field1;
short field2;
int field3;
} HeaderStruct;
typedef struct
{
short field1;
long long field2;
} PacketDataStruct;
assuming a 32-bit int size.
Edit:
As someone kindly reminded me in the comments, don't forget about converting to and from Network Order. Networking libraries will have functions to assist with this, such as ntohs, nothl, htons, and htonl.
One simple answer is that it would be sent in the format that the receiver expects. That begs the question a bit, though. Assuming the data is a fixed size as shown and the receiving end expects, then you could use a packed (1 byte alignment) structure and store the data in each field. The reason for using 1 byte alignment is that it is typically easier to make sure both ends are expecting the same data. Without 1 byte alignment, then the structure would possibly look different based on compiler options, 32-bit versus 64-bit architecture, etc.) And, typically, it is expected that you would send the values in network byte order if the hex values are integers. You can use functions such as htons and htonl (and possibly htobe64 if available) to convert them.
Assuming that the data is in the structure with the desired byte order, then the send call may be something like this:
ret = send( socket, &mystruct, sizeof( mystruct ), 0 );
That assumes that mystruct is declared as an instance of the structure as opposed to a pointer to the structure.
I have the following struct declaration and typedef in my code:
struct blockHeaderStruct {
bool allocated;
unsigned int length;
};
typedef struct blockHeaderStruct blockHeader;
When I do sizeof(blockheader), I get the value of 4 bytes back, but when I do sizeof(struct blockHeaderStruct), I get 8 bytes.
Why is this happening? Why am I not getting 5 back instead?
Firstly, you cannot do sizeof(blockHeaderStruct). That simply will not compile. What you can do is sizeof(struct blockHeaderStruct), which could indeed give you 8 bytes as result.
Secondly, getting a different result from sizeof(blockheader) is highly unlikely. Judging by your reference to sizeof(blockHeaderStruct) (which, again, will not even compile) your description of the problem is inaccurate. Take a closer look at what is it you are really doing. Most likely, you are taking a sizeof of a pointer type (which gives you 4), not a struct type.
In any case, try posting real code.
Looking at the definition of your struct, you have 1 byte value followed by 4 byte Integer. This integer needs to be allocated on 4 byte boundary, which will force compiler to insert a 3 byte padding after your 1 byte bool. Which makes the size of struct to 8 byte. To avoid this you can change order of elements in the struct.
Also for two sizeof calls returning different values, are you sure you do not have a typo here and you are not taking size of pointer or different type or some integer variable.
The most likely scenario is that you are actually looking at the size of a pointer, not the struct, on a 32-bit system.
However, int may be 2 bytes (16 bits). In that case, the expected size of the structure is 4:
2 bytes for the int
1 byte for the bool
round up to the next multiple of 2, because the size of the struct is usually rounded to a multiple of the size of its largest primitive member.
Nothing could explain sizeof(blockHeaderStruct) != sizeof(struct blockHeader), though, given that typedef. That is completely impossible.
Struct allocation normally occurs on a 4 byte boundary. That is the compiler will pad data types within a struct up to the 4 byte boundary before starting with the next data type. Given that this is c++ (bool is a sizeof 1) and not c (bool needs to be #define as something)
struct blockHeaderStruct {
bool allocated; // 1 byte followed by 3 pad bytes
unsigned int length; // 4 bytes
};
typedef struct blockHeaderStruct blockHeader;
typedef struct blockHeaderStruct *blockHeaderPtr;
A sizeof operation would result:
sizeof(blockHeader) == 8
sizeof(struct blockHeader) == 8
sizeof(blockHeaderPtr) == 4
(Note: The last entry will be 8 for a
64 bit compiler. )
There should be no difference in sizes between the first two lines of code. A typedef merely assigns an alias to an existing type. The third is taking the sizeof a pointer which is 4 bytes in a 32 bit machine and 8 bytes on a 64 bit machine.
To fix this, simply apply the #pragma pack directive before a structure is defined. This forces the compiler to pack on the specified boundary. Usually set as 1,2,or 4 (although 4 is normally the default and doesn't need to be set).
#include <stddef.h>
#include <stdio.h>
#pragma pack(1)
struct blockHeaderStruct {
bool allocated;
unsigned int length;
};
typedef struct blockHeaderStruct blockHeader;
int main()
{
printf("sizeof(blockHeader) == %li\n", sizeof(blockHeader));
printf("sizeof(struct blockHeader) == %li\n", sizeof(struct blockHeaderStruct));
return 0;
}
Compiled with g++ (Ubuntu
4.4.1-4ubuntu9) 4.4.1
Results in:
sizeof(blockHeader) == 5
sizeof(structblockHeader) == 5
You don't normally need this directive. Just remember to pack your structs efficiently. Group smaller data types together. Do not alternate < 4 byte datatypes and 4 byte data types as your structs will be mostly unused space. This can cause unnecessary bandwidth for network related applications.
I actually copied that snippet direct
from my source file.
OK.
When I do sizeof(blockheader), I get
the value of 4 bytes back
It looks like blockheader is typedef'ed somewhere and its type occupies 4 bytes or its type requires 4 byte alignment.
If you try sizeof(blockHeader) you'll get your type.
when I do sizeof(blockHeaderStruct), I
get 8 bytes.
The reason why alignment matters is that if you need an array of blockHeaders then you can compute how much memory you need. Or if you have an array and need to compute how much memory to copy, you can compute it.
If you want to align all struct members to addresses that are multiples of 1 instead of 4 or instead of your compiler's defaults, your compiler might offer a #pragma to do it. Then you'll save memory but accesses might be slower in order to access unaligned data.
Some compilers will optimize to always allocate data by powers of 2 (4 stays 4, but 5 is rounded up to 8).