I know structure packing is a common thing in C++ programming (at least on low memory systems). But what about classes.
I know it works because I tried it
#include <iostream>
#pragma pack(push, 1)
class Test_Packed {
uint8_t t;
uint32_t test;
};
#pragma pack(pop)
class Test_Unpacked {
uint8_t t;
uint32_t test;
};
int main() {
std::cout<<sizeof(Test_Packed) << " / " << sizeof(Test_Unpacked)<<std::endl;
return 0;
}
Which correctly outputs "5 / 8".
Can I assume this to be the case on all conforming Compilers, or is this implementation defined?
I know that adding virtual members (and thus needing a vtable) will add additional data in the front. What can be other reasons for this to fail?
Can this cause any problems except for poor performance on some platforms?
The documentation for #pragma states:
Implementation defined behavior is controlled by #pragma directive.
So the effect, if any, of #pragma pack(push, 1) and #pragma pack(pop)
is completely "implementation defined."
Related
Is there a way to write a compile-time assertion that checks if some type has any padding in it?
For example:
struct This_Should_Succeed
{
int a;
int b;
int c;
};
struct This_Should_Fail
{
int a;
char b;
// because there are 3 bytes of padding here
int c;
};
Since C++17 you might be able to use std::has_unique_object_representations.
#include <type_traits>
static_assert(std::has_unique_object_representations_v<This_Should_Succeed>); // succeeds
static_assert(std::has_unique_object_representations_v<This_Should_Fail>); // fails
Although, this might not do exactly what you want it to do. Check the linked cppreference page for details.
Edit: Check Indiana's answer.
Is there a way to write a compile-time assertion that checks if some type has any padding in it?
Yes.
You can sum the sizeof of all members and compare it to the size of the class itself:
static_assert(sizeof(This_Should_Succeed) == sizeof(This_Should_Succeed::a)
+ sizeof(This_Should_Succeed::b)
+ sizeof(This_Should_Succeed::c));
static_assert(sizeof(This_Should_Fail) != sizeof(This_Should_Fail::a)
+ sizeof(This_Should_Fail::b)
+ sizeof(This_Should_Fail::c));
This unfortunately requires explicitly naming the members for the sum. An automatic solution requires (compile time) reflection. Unfortunately, C++ language has no such feature yet. Maybe in C++23 if we are lucky. For now, there are solutions based on wrapping the class definition in a macro.
A non-portable solution might be to use -Wpadded option provided by GCC, which promises to warn if structure contains any padding. This can be combined with #pragma GCC diagnostic push to only do it for chosen structures.
type I'm checking, the type is a template input.
A portable, but not fully satisfactory approach might be to use a custom trait that the user of the template can use to voluntarily promise that the type does not contain padding allowing you to take advantage of the knowledge.
The user would have to rely on explicit or pre-processor based assertion that their promise holds true.
To get the total field size without retyping each struct member you can use an X Macro
First define all the fields
#define LIST_OF_FIELDS_OF_This_Should_Fail \
X(int, a) \
X(char, b) \
X(int, c)
#define LIST_OF_FIELDS_OF_This_Should_Succeed \
X(long long, a) \
X(long long, b) \
X(int, c) \
X(int, d) \
X(int, e) \
X(int, f)
then declare the structs
struct This_Should_Fail {
#define X(type, name) type name;
LIST_OF_FIELDS_OF_This_Should_Fail
#undef X
};
struct This_Should_Succeed {
#define X(type, name) type name;
LIST_OF_FIELDS_OF_This_Should_Succeed
#undef X
};
and check
#define X(type, name) sizeof(This_Should_Fail::name) +
static_assert(sizeof(This_Should_Fail) == LIST_OF_FIELDS_OF_This_Should_Fail 0);
#undef X
#define X(type, name) sizeof(This_Should_Succeed::name) +
static_assert(sizeof(This_Should_Succeed) == LIST_OF_FIELDS_OF_This_Should_Succeed 0);
#undef X
or you can just reuse the same X macro to check
#define X(type, name) sizeof(a.name) +
{
This_Should_Fail a;
static_assert(sizeof(This_Should_Fail) == LIST_OF_FIELDS_OF_This_Should_Fail 0);
}
{
This_Should_Succeed a;
static_assert(sizeof(This_Should_Succeed) == LIST_OF_FIELDS_OF_This_Should_Succeed 0);
}
#undef X
See demo on compiler explorer
For more information about this you can read Real-world use of X-Macros
An alternate non-portable solution is to compare the size of the struct with a packed version with #pragma pack or __attribute__((packed)). #pragma pack is also supported by many other compilers like GCC or IBM XL
#ifdef _MSC_VER
#define PACKED_STRUCT(declaration) __pragma(pack(push, 1)) declaration __pragma(pack(pop))
#else
#define PACKED_STRUCT(declaration) declaration __attribute((packed))
#endif
#define THIS_SHOULD_FAIL(name) struct name \
{ \
int a; \
char b; \
int c; \
}
PACKED_STRUCT(THIS_SHOULD_FAIL(This_Should_Fail_Packed));
THIS_SHOULD_FAIL(This_Should_Fail);
static_assert(sizeof(This_Should_Fail_Packed) == sizeof(This_Should_Fail));
Demo on Compiler Explorer
See Force C++ structure to pack tightly. If you want to have an even more portable pack macro then try this
Related:
How to check the size of struct w/o padding?
Detect if struct has padding
In GCC and Clang there's a -Wpadded option for this purpose
-Wpadded
Warn if padding is included in a structure, either to align an element of the structure or to align the whole structure. Sometimes when this happens it is possible to rearrange the fields of the structure to reduce the padding and so make the structure smaller.
In case the struct is in a header that you can't modify then in some cases it can be worked around like this to get a packed copy of the struct
#include "header.h"
// remove include guard to include the header again
#undef HEADER_H
// Get the packed versions
#define This_Should_Fail This_Should_Fail_Packed
#define This_Should_Succeed This_Should_Succeed_Packed
// We're including the header again, so it's quite dangerous and
// we need to do everything to prevent duplicated identifiers:
// rename them, or define some macros to remove possible parts
#define someFunc someFunc_deleted
// many parts are wrapped in SOME_CONDITION so this way
// we're preventing them from being redeclared
#define SOME_CONDITION 0
#pragma pack(push, 1)
#include "header.h"
#pragma pack(pop)
#undef This_Should_Fail
#undef This_Should_Succeed
static_assert(sizeof(This_Should_Fail_Packed) == sizeof(This_Should_Fail));
static_assert(sizeof(This_Should_Succeed_Packed) == sizeof(This_Should_Succeed ));
This won't work for headers that use #pragma once or some structs that include structs in other headers though
I have a problem that a struct shall be checked - at compile time - if it is well aligned or if it contains gaps.
The checking may be done in additional test code, but I don't want "packed" data in the real implementation code.
This is an example header file (MyData.h) with the typical include guards:
#ifndef MYDATA_H_
#define MYDATA_H_
struct uneven
{
int bla_u32;
short bla_u16;
char bla_u8;
/* <-- this gap will be filled in the unpacked version */
};
#endif // MYDATA_H
I found one possible solution - see below.
Questions:
Is there an elegant way to check if the struct uneven contains a different number of bytes compared to its unpacked counterpart at compile time?
Is there maybe even a solution that will work in C (without using a namespace)?
A compiler specific solution that works for both C and C++: GCC has a warning option -Wpadded, that produces a warning for every definition that change size due to alignment.
You could use a function instead of a namespace (on ideone):
This solution also works in C
Header File:
typedef struct
{
int bla_u32;
short bla_u16;
char bla_u8;
/* <-- this gap will be filled in the unpacked version */
} uneven;
Source File:
#include "MyData.h"
#define StaticAssert(cond, msg) switch(0){case 0:case cond:;}
void checkSizes()
{
uneven unpacked_uneven;
#pragma pack(push, 1)
#undef MYDATA_H_ // force re-including "MyData.h"
#include "MyData.h"
#pragma pack(pop)
uneven packed_uneven;
StaticAssert(sizeof(unpacked_uneven) == sizeof(packed_uneven), "uneven contains gaps");
}
You can place your StaticAssert into the function for a compile time error.
I found one (somehow nasty and very tricky) solution for the problem that only works with C++, not C.
#define StaticAssert(cond, msg) switch(0){case 0:case cond:;}
#pragma pack(push, 1)
namespace packed
{
#include "MyData.h"
}
#pragma pack(pop)
#undef MYDATA_H_ // force re-including "MyData.h"
#include "MyData.h"
void checkSizes()
{
StaticAssert(sizeof(packed::uneven) == sizeof(uneven), "uneven contains gaps");
}
This StaticAssertmacro fails for the given uneven struct data - as the packed version's size is of 7 bytes and the unpacked (normal) version is 8 bytes. If an additional charis added at the end of the struct the test succeeds - both versions have 8 bytes then.
I have one question about using pragma in c++ classes.
I've read there (Use of #pragma pack on a class) that use pragma around c++ class is not recommended, but can I use pragma like this:
class TestClass {
public:
ConfigProtocol();
#pragma pack(1)
struct t_config_header {
quint8 version;
quint8 da;
quint16 sa;
quint16 counter;
};
#pragma pack()
};
will not it be mistake?
Short answer: Yes you can (and in your case, as it seems you are implementing a communcation protocol, in fact, should) do it.
The way you are using the pragma is only affecting the structure which invalidates the arguments made in the StackOverflow answer you are linking to: The struct is not significantly changing as long as it stays a struct.
Let's say we build 2 C++ DLLs as part of a huge in-house codebase: a Core.DLL and Another.DLL calling the Core.DLL. Each DLL gets built with the same version of Visual Studio but with very different project settings, including e.g. C/C++->Code Generation->Struct Member Alignment.
Now let's say we pass a reference to CoreData when calling Core.DLL from Another.DLL, but the CoreData is not dllexported, it is defined by each DLL independently.
What makes it dangerous to use a non-dllexported class CoreData in such context? Which project settings are particularly important to keep identical between the DLLs, if we had to (at gunpoint) employ the generally unsafe practice ?
I know that we should be dll-interfacing only with POD types and abstract interfaces (COM-like), the purpose of the question is to learn what really goes on behind the curtains, if one doesn't follow those rules. I will follow these rules, just have to learn how to figure out what to fix first in a large codebase.
Below is my pseudo-code to illustrate the question.
In-depth explanations as well as links to articles or books are very welcome. I've seen a few similar questions on SO but none seemed to be deep enough.
EDIT: wrapped API header in #pragma pack to counter the different Struct Member Alignment project settings. Is it safe now, or are there any other project settings to keep an eye on?
Cheers!
// CoreDllApi.h
#pragma once
#ifdef COREDLL_EXPORTS
#define COREDLL_API __declspec(dllexport)
#else
#define COREDLL_API __declspec(dllimport)
#endif
#pragma pack(push,8)
class CoreData
{
public:
CoreData():a(0),b(0),c(.0){}
CoreData(int a, int b, double d):a(a),b(b),c(c){}
virtual bool IsValid() {return true;}
int a;
int b;
double c;
};
#pragma pack(pop)
class COREDLL_API CoreRetriever
{
static void get(int id, CoreData &out);
};
// AnotherDll.cpp:
#include "CoreDllApi.h"
#include <iostream>
static void ProcessData(int id)
{
CoreData data;
CoreRetriever::get(id, data); // danger?
std::cout << data.a << ";" << data.b << ";" << data.c << std::endl;
}
I am reading simple binary data, without pointers, using C++ classes without padding with the following code:
#include <fstream>
#include <iostream>
using namespace std;
class Data {
public:
int a;
int b;
short int c;
double d;
}__attribute__((packed));
int main() {
Data myData;
ifstream ifs("test.bin", ios::binary);
ifs.read((char *)&myData, sizeof(myData));
ifs.close();
}
I am using this method because the data might have 20+ different formats and I want to write 20+ different classes to cover all the formats that might show up. I also read that other options include using bit-fields, pragma directives, and even the boost serialization routines (I can't because I have to use std). My question is: is this the best way to read simple binary data using classes without padding? Do you suggest any other alternative? I would like to learn what is the safest and most widely used method out there.
Typically, one would use a struct instead of a class, but yes, the same concept applies to both.
I've used these macros to allow packed structs to compile on both gcc and VC:
#ifdef _MSC_VER
#define BEGIN_PACK __pragma( pack(push, 1) )
#define END_PACK __pragma( pack(pop) )
#else
#define BEGIN_PACK
#define END_PACK __attribute__((packed))
#endif
So then you'd use them like this:
BEGIN_PACK
struct Data {
int a;
int b;
short int c;
double d;
} END_PACK;
But yes, that's usually how it's done. Note that these are non-standard extensions.
C++11 has defined packing directives, but I don't know if they're supported by compilers yet.