Most compiliers support changing the packing of a class by using a #pragma pack(N) directive, where N is the new minimum acceptable alignment for each member.
Is it possible to check at compile-time whether or not a #pragma pack(N) has been specified. Additionally, is there a way to determine N?
You cannot test the struct packing directly, instead you have to create a test structure and check its size:
struct Test_Pack_Struct {
unsigned char bVal;
__int64 lVal;
};
#define GetStructPacking() (sizeof(Test_Pack_Struct)-8)
At compile time you may check the apprriate size with a static assert (requires C++ 11 or higher), for example:
static_assert( GetStructPacking() == 4, "Error: 4 byte packing assumed" );
At runtime you can assign the value of GetStructPacking macro to a variable or use it in expressions:
int iPacking = GetStructPacking()
Keep in mind, that the size of the Test_Pack_Struct structure depends on the posistion where it is defined (headers, code files etc).
A drawback is that, if you want to make several packing checks in the same context, you have to defined different structures (and macros if you want to use it).
Related
I have embedded device connected to PC
and some big struct S with many fields and arrays of custom defined type FixedPoint_t.
FixedPoint_t is a templated POD class with exactly one data member that vary in size from char to long depending on template params. Anyway it passes static_assert((std::is_pod<FixedPoint_t<0,8,8> >::value == true),"");
It will be good if this big struct has compatible underlaying memory representation on both embedded system and controlling PC. This allows significant simplification of communication protocol to commands like "set word/byte with offset N to value V". Assume endianess is the same on both platforms.
I see 3 solutions here:
Use something like #pragma packed on both sides.
BUT i got warning when i put attribute((packed)) to struct S declaration
warning: ignoring packed attribute because of unpacked non-POD field.
This is because FixedPoint_t is not declared as packed.
I don't want declare it as packed because this type is widely used in whole program and packing can lead to performance drop.
Make correct struct serialization. This is not acceptable because of code bloat, additional RAM usege...Protocol will be more complicated because i need random access to the struct. Now I think this is not an option.
Control padding manually. I can add some field, reorder others...Just to acheive no padding on both platforms. This will satisfy me at the moment. But i need good way to write a test that shows me is padding is there or not.
I can compare sum of sizeof() each field to sizeof(struct).
I can compare offsetof() each struct field on both platforms.
Both variants are ugly enough...
What do you recommend? Especially i am interested in manual padding controling and automaic padding detection in tests.
EDIT: Is it sufficient to compare sizeof(big struct) on two platforms to detect layout compatibility(assume endianess is equal)?? I think size should not match if padding will be different.
EDIT2:
//this struct should have padding on 32bit machine
//and has no padding on 8bit
typedef struct
{
uint8_t f8;
uint32_t f32;
uint8_t arr[5];
} serialize_me_t;
//count of members in struct
#define SERTABLE_LEN 3
//one table entry for each serialize_me_t data member
static const struct {
size_t width;
size_t offset;
// size_t cnt; //why we need cnt?
} ser_des_table[SERTABLE_LEN] =
{
{ sizeof(serialize_me_t::f8), offsetof(serialize_me_t, f8)},
{ sizeof(serialize_me_t::f32), offsetof(serialize_me_t, f32)},
{ sizeof(serialize_me_t::arr), offsetof(serialize_me_t, arr)},
};
void serialize(void* serialize_me_ptr, char* buf)
{
const char* struct_ptr = (const char*)serialize_me_ptr;
for(int i=0; i<SERTABLE_LEN; I++)
{
struct_ptr += ser_des_table[i].offset;
memcpy(buf, struct_ptr, ser_des_table[i].width );
buf += ser_des_table[i].width;
}
}
I strongly recommend to use option 2:
You are save for future changes (new PCD/ABI, compiler, platform, etc.)
Code-bloat can be kept to a minimum if well thought. There is just one function needed per direction.
You can create the required tables/code (semi-)automatically (I use Python for such). This way both sides will stay in sync.
You definitively should add a CRC to the data anyway. As you likely do not want to calculate this in the rx/tx-interrupt, you'll have to provide an array anyway.
Using a struct directly will soon become a maintenance nightmare. Even worse if someone else has to track this code.
Protocols, etc. tend to be reused. If it is a platform with different endianess, the other approach goes bang.
To create the data-structs and ser/des tables, you can use offsetof to get the offset of each type in the struct. If that table is made an include-file, it can be used on both sides. You can even create the struct and table e.g. by a Python script. Adding that to the build-process ensures it is always up-to-date and you avoid additional typeing.
For instance (in C, just to get idea):
// protocol.inc
typedef struct {
uint32_t i;
uint 16_t s[5];
uint32_t j;
} ProtocolType;
static const struct {
size_t width;
size_t offset;
size_t cnt;
} ser_des_table[] = {
{ sizeof(ProtocolType.i), offsetof(ProtocolType.i), 1 },
{ sizeof(ProtocolType.s[0]), offsetof(ProtocolType.s), 5 },
...
};
If not created automatically, I'd use macros to generate the data. Possibly by including the file twice: one to generate the struct definition and one for the table. This is possible by redefining the macros in-between.
You should care about the representation of signed integers and floats (implementation defined, floats are likely IEEE754 as proposed by the standard).
As an alternative to the width field, you can use an "type" code (e.g. a char which maps to an implementation-defined type. This way you can add custom types with same width, but different encoding (e.g. uint32_t and IEEE754-float). This will completely abstract the network protocol encoding from the physical machine (the best solution). Note noting hinders you from using common encodings which do not complicate code a single bit (literally).
What is the best practice for exporting a packed structure containing booleans?
I ask this because I'm trying to find the best way to do that. Current I do:
#ifndef __cplusplus
#if __STDC_VERSION__ >= 199901L
#include <stdbool.h> //size is 1.
#else
typedef enum {false, true} bool; //sizeof(int)
#endif
#endif
now in the above, the size of a boolean can be 1 or sizeof(int)..
So in a structure like:
#pragma pack(push, 1)
typedef struct
{
long unsigned int sock;
const char* address;
bool connected;
bool blockmode;
} Sock;
#pragma pack(pop)
the alignment is different if using C compared to C99 & C++. If I export it as an integer then languages where boolean is size 1 have alignment problems and need to pad the structure.
I was wondering if it would be best to typedef a bool as a char in the case of pre-C99 but it just doesn't feel right.
Any better ideas?
It depends on what you're looking for: preserve space but run a few extra instructions, or waste a few bytes but run faster.
If you're looking to be fast, but can "waste" a few bytes of space (i.e. a single value for each boolean flag, see sizeof bool discussion), your current approach is the superior. That is because it can load and compare the boolean values directly without having to mask them out of a packed field (see next).
If you're looking to preserve space then you should look into C bitfields:
struct Sock {
...
int connected:1; // For 2 flags, you could also use char here.
int blockmode:1;
}
or roll your own "flags" and set bits in integer values:
#define SOCKFLAGS_NONE 0
#define SOCKFLAGS_CONNECTED (1<<0)
#define SOCKFLAGS_BLOCKMODE (1<<1)
struct Sock {
...
int flags; // For 2 flags, you could also use char here.
}
Both examples lead to more or less the same code which masks bits and shifts values around (extra instructions) but is denser packed than simple bool values.
IMHO, using #pragma pack is more pain (in long term) than the gain (in short term).
It is compiler specific; non-standard and non-portable
I understand the embedded systems or protocols scenarios. With little extra effort, the code can be written pragma free.
I too want to pack my structure as much as possible and lay out the members in wider-first way as you did. However, I do not mind losing 2 bytes, if that allows my code to be standard-compliant and portable.
I would do the following three things:
Declare the flags as bool (you already did) and assign true/false
Put them as last members of the struct (you already did)
Use bitfield (as suggested by fellow stackers)
Combining these:
typedef struct Sock
{
long unsigned int sock;
const char* address;
bool connected : 1;
bool blockmode : 1;
} Sock;
In the pre-C99 case, it is risky to typedef char bool;. That will silently break code like:
bool x = (foo & 0x100);
which is supposed to set x to be true if that bit is set in foo. The enum has the same problem.
In my code I actually do typedef unsigned char bool; but then I am careful to write !! everywhere that an expression is converted to this bool. It's not ideal.
In my experience, using flags in an integral type leads to fewer issues than using bool in your structure, or bitfields, for C90.
I have this sample code which I decorated with pack to make sure that its size if 5 byte (4 for int and 1 for char).
but it print out that the size of struct is 8 byte.
#pragma pack push
#pragma pack 1
struct mystruct
{
int x;
char y;
};
#pragma pack pop
//static_assert(sizeof(mystruct) == 5, "Size of mystruct should be 5 byte.");
int _tmain(int argc, _TCHAR* argv[])
{
int x=sizeof(mystruct);
printf("size of struct is %d\n",x);
return 0;
}
Why te pck is not working?
How can make sure that size of struct is always 5.
Your pragma syntax is wrong.
Try this:
#pragma pack(push,1)
...
#pragma pack(pop)
Demo.
I believe the compiler is permitted to ignore the pack pragma; perhaps it can be overridden with compiler switches.
The compiler can always ignore your attempts to stop packing. This, in some ways, is a good thing, as the packing allows a higher level of efficiency... Think of an array, when you access element 1, it is efficiently traversed to as the distance, in terms of memory, between the elements is uniform. This is also performed with structures in order to ensure efficiency is maintained.
I direct you to this stack overflow
Disable structure padding in C without using pragma
Strange- I have structs in a 3rd party program that are packed on 1-byte boundaries and they do work as expected (though not liked!).
If the compiler is ignoring your pragmas, try setting the alignment compiler options as well - C/C++ | Code Generation | Struct Member Alignment to 1.
If the compiler is doing an "I know best" optimisation, perhaps putting the char first would stop it.
I've seen several libraries and some C++ header files that provide compiler independent types but I don't understand quite why they are compiler independent.
For example:
int Number; // Not compiler Independent
typedef unsigned int U32;
U32 Number2; // Now this is compiler independent
Is this above true? If so, why? I don't quite understand why the usage of a typedef would mean that the size of Number2 is the same across compilers.
Elaborating on the comment,
Proposition : Use a typedef for compiler independence.
Rationale : Platform independence is a Good Thing
Implementation:
#ifdef _MSC_VER
#if _MSC_VER < 1400
typedef int bar;
#elif _MSC_VER < 1600
typedef char bar;
#else
typedef bool bar;
#else
#error "Unknown compiler"
#endif
The preprocessor macro chain is the important part not the typedef.
Disclaimer: I haven't compiled it!
I'm assuming that you meant for the types to be the same with unsigned int Number.
But no, these are exactly the same. Both declarations, Number and Number2, have the same type. Neither is more compiler independent than the other.
However, the point of using a typedef like this is so that the developers of the library can easily change the integer type used by all functions that use U32. If, for example, they are on a system that where an unsigned int is not 32 bits, but an unsigned long is, they could change the typedef to:
typedef unsigned long U32;
In fact, it's possible to use the build system to conditionally change the typedef depending on the target platform.
However, if you want a nice standardised way to ensure that the type is a 32 bit unsigned integer type, I recommend using std::uint32_t from the <cstdint> header. However, this type is not guaranteed to exist if you're on a machine with no 32 bit integer type. Instead, you can use std::uint_least32_t, which will give you the smallest integer type with at least 32 bits.
As stated in the comments, the shown typedef is not compiler independent.
If you want a compiler independent way to get fixed sizes, you might want to use cstdint.
This header file actually comes with your compiler and assures you a minimum size, but no maximum for bigger types (64 bit, 128 bit).
If you want to be completely sure about all the sizes of your types, you need to check it.
I have defined this struc to read a binary file
struct cabecera{
unsigned long time;
short lrec;
short eddimdat;
short edmaxdat;
short edncn;
short estindefmax;
long maxiedisc;
long edbuit;
long edusat;
short estindefusat;
long libdoff;
long vidoff;
long dgoff;
long estindefoff;
long estinoff;
long sedoff;
long esdoff;
int libvers;
long offie;
long tiueoff;
};
I have a class that extends from fstream to read the data
open(fNombre.c_str(),ios::in|ios::binary);
if(fail()||bad()) return;
int pos = 160 ;
cabecera cb={};
seekg(pos,ios::beg);
read((char*)&cb, sizeof(cb));
But the variable maxiedisc gets a wrong value (1052835858) and the rest on the variables from here too
If I read this variable without the structure, the value I obtain is correct(1200000)
int tmLong = sizeof(long);
int tmULong = sizeof(unsigned long);
int tmShort = sizeof(short);
int pos = 160 + tmULong + (tmShort*5);
seekg(pos,ios::beg);
long maxiedisc;
read((char*)&maxiedisc, tmLong);
What is the problem with the structure? Why I obtain different results?
Almost certainly you have padding in your struct. The compiler has placed two extra bytes between the estindefmax and maxiedisc members. This is why reading directly into a struct is a bad idea, unless you wrote the file with the exact same struct.
Do it your second way, without the struct. Fill the struct in later if that is what you need.
Reading and writing files directly from memory to disk is not portable.
Some of the problems you may have are
Memory padding. (Compiler dependent) You can use #pragma pack (vs) to avoid this but those structures will be used by CPu in a more inefficient way.
Endian-ess. Integer types can be stored in Little-Ending or Big-Endian format (Platform dependent). Can be converted by using boost::endian family of functions
Save complex data structures (STL list, vector, etc)
struct versioning. Load an older version of a struct in a newer program.
The right approach is to use a serialization library that already encapsulate all this problems (like Boost::serialization or google's ProtoBuff) or if the overhead of a library is too big for you to write a little serializer yourself. It's easier than it sounds. Just write two member functions (save/load) that writes/reads members to/from a stream. You should handle yourself endian-ess and versioning.
You need to disable the padding of your compiler. It will add padding bytes to your structure making it bigger than you would expect.
Since you did not mention which compiler you use: Here is how it is done on msvc. If I remember corrctly gcc has the same syntax. But Inever tried it.
looks like padding issues
to read directly into a struct with gcc try
struct my_struct {
/* ... */
} __attribute__((packed));
this ensures, that no padding is used
As mentioned by #john the problem seems to be the struct padding.
You have two solutions to get rid of the padding, first one is to write each struct component one by one (but in structs of that size is not the best way) the second one is disable the padding as suggested by other users.
#ifndef LINUX
#pragma pack(1)
#endif
struct cabecera
{
// your stuff...
}__attribute__((packed));
#else
};
#endif
PS: don't mix languages on your code, it looks silly ;) "si programas en inglés, usa el inglés para nombrar variables y datos"
You could use something like Google protobuf to do the serialization/unserialization of a struct for you. This approach is much safer and works across compiler and system boundaries. The alternative would be to serialize each field individually. Packing is sometimes the quickest option, but has problems with compiler compatibility and binary compatibility between systems.
#pragma pack(push,1)
// struct definition
#pragma push