recently I've read about compiler ability to reorded members in a class. From C++ 11 standard:
§ 9.2.13
[...] The order of allocation of non-static data members with different access control is unspecified.
I would like to know how does it look like in practice. Do the major compilers (I'm interested in g++, clang and msvc) reorder class members in some situtations?
If no, is there anything else that could happen which would result in different object layout on different compilers (or when using different compiler flags)?
Assume that no virtual methods are used and so no vtable is created.
The question is, in fact, moot. The only standard-compliant way to ensure the layout of the class follows your expectation is to make sure the class is a Standard Layout Type - and that would guarantee the same layout on every conformant compiler.
One of the requirements for such a type is that all members have the same access control.
Related
There are a few questions and answers on the site already concerning pointer-interconvertibility of structs and their first member variable, as well as structs and their first public base. This question is one of them, for example.
However, what I'm interested in is not the fact that it's undefined behavior to reinterpret_cast (or static_cast through a void *) between a non-standard-layout struct and its public base, but rather the reasoning why the C++ standard currently forbids such casts. The existing questions and answers don't cover this aspect.
Consider the following example in particular (Godbolt):
#include <type_traits>
struct Base {
int m_base_var = 1;
};
struct Derived: public Base {
int m_derived_var = 2;
};
Derived g_derived;
constexpr Derived *g_pDerived = &g_derived;
constexpr Base *g_pBase = &g_derived;
constexpr void *g_pvDerived = &g_derived;
//These assertions all hold
static_assert(!std::is_pointer_interconvertible_base_of_v<Base, Derived>);
static_assert((void *)g_pDerived == (void *)g_pBase);
static_assert((void *)g_pDerived == g_pvDerived);
static_assert((void *)g_pBase == g_pvDerived);
//This is well-defined and returns &g_derived
Derived * getDerived() {
return static_cast<Derived *>(g_pvDerived);
}
//This is also well-defined; outer static_cast added to illustrate the sequence of conversions
Base * getBase() {
return static_cast<Base *>(static_cast<Derived *>(g_pvDerived));
}
//This is UB due to the first static_assert!
Base * getBaseUB() {
return static_cast<Base *>(g_pvDerived);
}
As you can see from the Godbolt link, all three functions compile to the exact same assembly on x86-64 GCC. However, the standard forbids the third variant since Base is not a pointer-interconvertible base of Derived.
My question is: Is there an obvious reason why the standard forbids this kind of cast? In particular, on all implementations that I know of, the value of a pointer to the Base subobject is the same as that of the pointer to the whole Derived, and I don't see a particular reason why Derived should not be considered standard-layout anymore. (In other words, Base lives at offset zero within Derived.) Would it be legal for a C++ implementation to place Base at a non-zero offset within Derived? (Is there maybe an implementation already that does this?)
Note that this question is only about cases without virtual member functions / virtual inheritance / multiple inheritance.
This is really two question:
What is standard layout, like really?
Why is pointer-interconvertibility linked to standard layout?
Standard layout was constructed out of one half of the pre-C++11 concept of "plain old data" types. The other half is trivial copyability (ie: memcpying an instance of the object is just as good as a copy constructor). These two halves didn't really interact, but POD required both.
From a purely standard C++ perspective, standard layout is a requirement for the ability to construct a type whose layout matches an existing type, such that if you shove both of those types into a union, you can access subobjects of the non-active members. This is the core functionality that standard layout enabled within the language since it was invented in C++11.
This is why standard layout only allows one member in the class hierarchy to have non-static data members. If only one class has NSDMs, then there's no question about the ordering between NSDMs of different base classes and so forth. And being able to know a priori what that ordering is is vital for being able to know that two types match.
This is also useful for communicating across languages.
Once standard layout was defined however, it started getting used for things that were... less clearly part of its domain.
For example, standard layout became the determinator for whether offsetof was valid behavior. This is due to offsetof oroginally being based on being a POD type, so when the layout part was spun off, offsetof was updated to use that. However, this was suboptimal, as the only layout-based thing that would break offsetof is having virtual base classes (the offset of members of a virtual base class depends on the most-derived class, which depends on the runtime type of the object). Now, the new restriction was still was better than POD, but it could have been expanded to include more stuff. But that would mean coming up with a new definition.
Something similar likely goes with pointer-interconvertibility. This concept was invented in C++17 to resolve various issues with the object model. There is no evidence in the papers explaining why they picked standard layout to hang pointer-interconvertibility on. But it was an existing tool with well-defined rules that already had well-defined rules on what the "first subobject" was for any given type.
Expanding the rules like you wants requires creating a new definition for "first suboject".
Is a base class pointer-interconvertible to a particular derived class? Well, that depends on what other classes that derived class inherits from. Is the first NSDM pointer-interconvertible to the class it is a member of? That depends on what other classes are involved in the inheritance diagram.
These dependencies already exist, but they all key off of a specific, pre-existing rule. What you want requires creating a new rule that's more complex. It will have to copy 90% of the existing standard-layout rules (forbidding virtual, public/private members, etc) and then add its own rules. The first NSDM is pointer-interconvertible unless any base classes are non-empty. Any particular base class is pointer-interconvertible only so long as all previous base classes in declaration order are empty of NSDMs.
It's so much easier to just piggyback off of the standard layout rules and say "a standard-layout type is pointer-interconvertible with its first NSDM and all of its base classes."
Having an extra rule also imposes some burden on the specification. It's a partial redundancy, and that breeds errors. For example, C++23 is on track to expand standard layout types by removing the forbidding of mixing public and private members, forcing layout to be ordered strictly by declaration. If pointer-interconvertibility had its own rules, it would have been possible to update standard-layout but not pointer-interconvertibility.
Both C and C++ were defined by tradition before standards were written. When the first standards were written, it was more important to avoid requiring that any implementations break programs that existed for them, than to forbid implementations that would processing various programs from changing in such a fashion as to break them.
Although it would be typical for a derived class object to store all of the parent data in a sequence of consecutive bytes, followed by the remainder of the data for the object, it could sometimes be useful for implementations to deviate from that. If a base class had a char field, a derived class had an int, and sub-derived class had another five char fields, an implementation that nestles one of the sub-derived class fields between the base-class field and first-derived class might be able to store data more compactly than one which places everything from each layer in a single sequence of consecutive bytes.
Note that the C++ Standard expressly waives jurisdiction over what C++ programs should be viewed as "conforming". As such, there was no perceived need to ensure that it didn't classify as UB any programs which most implementations should process usefully. If the authors had foreseen the way compilers would treat the Committee's decisions to waive jurisdiction over various constructs as an invitation to process them nonsensically, they probably would have defined behavior in many more corner cases than they actually did.
Currently, offsetof works only with standard layout types. However, I've never understood this limit. Sure, for types with virtual inheritance, offsetof cannot work. But for types without virtual inheritance, it could. I know, the standard allows non-standard layout types to have differing offsets of members for each instance. But, I've never heard of any compiler implementations which uses this. Why would it do something like this?
All the compilers I know, for types without virtual inheritance, offsets of members are compile-time constants.
So the question is:
is there any compiler, for which offsets of members are not compile-time constants (for types without virtual inheritance)?
why doesn't the committee relax the requirements of offsetof?
(I've read a lot of related answers here in stackoverflow, and discussions of this at various places, but I haven't found the reason of this).
The arbitrary nature of this restriction was discussed in July by the committee. It's not in the working paper yet, but I believe it will be relaxed (to this degree, at least) in C++20.
Relax the requirements to what? offsetofis okay to use except when it gives incorrect results?
The way to access selected data members of a C++ object is with a pointer to data member.
There are various pragmas for controlling the struct/class layout, such as pragma pack. But as far as I know, there's no pragma for saying "I don't care about the layout. It's internal, the code doesn't rely on it. Reorder it for best performance/size.". AFAIK, that's the typical case, and it could improve performance/size in many cases. Also, even if the programmer was careful enough to reorder it for performance/size, a different target architecture might have a different optimal layout.
Edit: to clarify, I'm talking about the order of members. Padding is already controllable.
Also, PVS-Studio has a relevant message. That's what I'm talking about - why can't this be done by a compiler with a pragma?
The language specifically calls out that class members will be ordered in memory the same way they are in each access level (like private). There is no way a pragma could override this behavior.
See 9.2/14:
Nonstatic data members of a (non-union) class with the same access
control (Clause 11) are allocated so that later members have higher
addresses within a class object. The order of allocation of non-static
data members with different access control is unspecified
Bear in mind that reordering members changes the order in which sub-object constructors and destructors would be called, and possibly other things. It seems extremely risky even given a pragma for the compiler to make these sorts of changes behind the scenes (what if you have a member that depends on the initialization of another member).
Such a pragma would be permitted by the language standard, but I'm not aware of any compiler that implements such a thing.
In C, the behavior of #pragma is specified in section 6.10.6 of the standard (the link is to the latest draft):
A preprocessing directive of the form
# pragma pp-tokensopt new-line
where the preprocessing token STDC does not immediately
follow pragma in the directive (prior to any macro replacement)
causes the implementation to behave in an implementation-defined
manner. The behavior might cause translation to fail or cause the
translator or the resulting program to behave in a non-conforming
manner. Any such pragma that is not recognized by the implementation
is ignored.
So a #pragma can, in effect, violate the rules of the language.
The relevant rule in this case is that struct members are laid out in the order in which they're declared. 6.7.2.1 paragraph 15:
Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa.
There may be unnamed padding within a structure object, but not at its
beginning.
The bad news: The C standard requires struct members to be laid out in the order in which they're declared. The first member must be at offset 0. There may be arbitrary padding between members, or after the last one, but they cannot be reordered.
The good news: The language permits an implementation to define a #pragma that specifies a layout that violates the above rule.
The bad news: As far as I know, no implementation actually does so. Even if one did, there are other implementations that do not, so any code that uses such a #pragma would be non-portable. (Though at least if the name of the #pragma is unique, any compilers that don't recognize it are required to ignore it, so your code would still compile.)
That's for C. The C++ rules for #pragma are very similar to the C rules. I'm reasonably sure the C++ rules for struct layout are also similar to C's; inheritance makes things a little more complex.
I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Given the following code:
// that one is a standard layout, and trivial which makes it a c++11 POD
struct Bar
{
public:
int x;
public:
int y;
};
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
In C++03, it can. In C++11 it cannot. C++11's standard layout rules only require that all of the members have the same access control. They don't have to be declared in the same access control region.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
I think you're misunderstanding things. The C++11 rules allow more types to be standard-layout (and thus potentially layout-compatible with C types), not less. Thus, there's no real downside to relaxing the rules.
I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Not exactly the whole point of it, but yes, that is one of the properties of PODs.
// that one is a standard layout, and trivial which makes it a c++11 POD
Correct.
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
We already established it is a POD, which means the compiler will maintain compatibility with C. Maintaining compatibility with C does not break compatibility with C.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
Because it doesn't break anything.
The point of POD is not just to make sure the type is compatible with C - note that a type with an access specifier (public, private, etc.) is by definition not compatible with C since C doesn't have access specifiers. The main property of a POD type is that it can be memcpy'ed around.
Having more than one access specifier in a C++ type does permit the compiler to lay out the type in a non-specified way, and that's been true for a while (it's not new with C++11):
From C++03 9.2/12
Nonstatic data members of a (non-union) class declared without an
intervening access-specifier are allocated so that later members have
higher addresses within a class object. The order of allocation of
nonstatic data members separated by an access-specifier is unspecified
(11.1).
However, that doesn't make a type a non-POD - it can still be a POD, just not one that can be portably expressed in C.
This is a question that was sparked by Rob Walker's answer here.
Suppose I declare a class/struct like so:
struct
{
char A;
int B;
char C;
int D;
};
Is it safe to assume that these members will be declared in exactly that order in memory, or is this a compiler dependent thing? I'm asking because I had always assumed that the compiler can do whatever it wants with them.
This leads into my next question. If the above example causes memory alignment issues, why can the compiler not just turn that into something like this implicitly:
struct
{
char A;
char C;
int B;
int D;
};
(I'm primarily asking about C++, but I'd be interested to hear the C answer as well)
Related topics
Why doesn't GCC optimize structs?
C99 §6.7.2.1 clause 13 states:
Within a structure object, the
non-bit-field members and the units in
which bit-fields reside have addresses
that increase in the order in which
they are declared.
and goes on to say a bit more about padding and addresses. The C89 equivalent section is §6.5.2.1.
C++ is a bit more complicated. In the 1998 and 2003 standards, there is §9.2 clause 12 (clause 15 in C++11):
Nonstatic data members of a
(non-union) class declared without an
intervening access-specifier are
allocated so that later members have
higher addresses within a class
object. The order of allocation of
nonstatic data members separated by an
access-specifier is unspecified
(11.1). Implementation alignment
requirements might cause two adjacent
members not to be allocated
immediately after each other; so might
requirements for space for managing
virtual functions (10.3) and virtual
base classes (10.1).
The data members are arranged in the order declared. The compiler is free to intersperse padding to arrange the memory alignment it likes (and you'll find that many compilers have a boatload a alignment specification options---useful if mixing bits compiled by different programs.).
See also Why doesn't GCC optimize structs?.
It appears that this answer is somewhat obsolete for C++. You learn something everyday. Thanks aib, Nemanja.
I cannot speak for C++, but in C the order is guaranteed to be the same order in memory as declared in the struct.
Basically, you can count on that only for the classes with a standard layout. Strictly speaking, standard layout is a C++0x thing, but it is really just standardizing existing practice/
Aside from padding for alignment, no structure optimization is allowed by any compiler (that I am aware of) for C or C++. I can't speak for C++ classes, as they may be another beast entirely.
Consider your program is interfacing with system/library code on Windows but you want to use GCC. You would have to verify that GCC used an identical layout-optimization algorithm so all your structures would be packed correctly before sending them to the MS-compiled code.
While browsing the related topics at the right, I looked at this question. I figure this may be an interesting corner case when thinking about these issues (unless it's more common than I realize).
To paraphrase, if you have a struct in C that looks something like this:
struct foo{};
and subclass it like so in C++ (using a separate compilation unit):
extern "C" foo;
struct bar: public foo{};
Then the memory alignment won't necessarily be the same for the reasons aib mentions (even amongst compilers from the same vendor).