I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Given the following code:
// that one is a standard layout, and trivial which makes it a c++11 POD
struct Bar
{
public:
int x;
public:
int y;
};
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
In C++03, it can. In C++11 it cannot. C++11's standard layout rules only require that all of the members have the same access control. They don't have to be declared in the same access control region.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
I think you're misunderstanding things. The C++11 rules allow more types to be standard-layout (and thus potentially layout-compatible with C types), not less. Thus, there's no real downside to relaxing the rules.
I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Not exactly the whole point of it, but yes, that is one of the properties of PODs.
// that one is a standard layout, and trivial which makes it a c++11 POD
Correct.
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
We already established it is a POD, which means the compiler will maintain compatibility with C. Maintaining compatibility with C does not break compatibility with C.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
Because it doesn't break anything.
The point of POD is not just to make sure the type is compatible with C - note that a type with an access specifier (public, private, etc.) is by definition not compatible with C since C doesn't have access specifiers. The main property of a POD type is that it can be memcpy'ed around.
Having more than one access specifier in a C++ type does permit the compiler to lay out the type in a non-specified way, and that's been true for a while (it's not new with C++11):
From C++03 9.2/12
Nonstatic data members of a (non-union) class declared without an
intervening access-specifier are allocated so that later members have
higher addresses within a class object. The order of allocation of
nonstatic data members separated by an access-specifier is unspecified
(11.1).
However, that doesn't make a type a non-POD - it can still be a POD, just not one that can be portably expressed in C.
Related
recently I've read about compiler ability to reorded members in a class. From C++ 11 standard:
§ 9.2.13
[...] The order of allocation of non-static data members with different access control is unspecified.
I would like to know how does it look like in practice. Do the major compilers (I'm interested in g++, clang and msvc) reorder class members in some situtations?
If no, is there anything else that could happen which would result in different object layout on different compilers (or when using different compiler flags)?
Assume that no virtual methods are used and so no vtable is created.
The question is, in fact, moot. The only standard-compliant way to ensure the layout of the class follows your expectation is to make sure the class is a Standard Layout Type - and that would guarantee the same layout on every conformant compiler.
One of the requirements for such a type is that all members have the same access control.
There are various pragmas for controlling the struct/class layout, such as pragma pack. But as far as I know, there's no pragma for saying "I don't care about the layout. It's internal, the code doesn't rely on it. Reorder it for best performance/size.". AFAIK, that's the typical case, and it could improve performance/size in many cases. Also, even if the programmer was careful enough to reorder it for performance/size, a different target architecture might have a different optimal layout.
Edit: to clarify, I'm talking about the order of members. Padding is already controllable.
Also, PVS-Studio has a relevant message. That's what I'm talking about - why can't this be done by a compiler with a pragma?
The language specifically calls out that class members will be ordered in memory the same way they are in each access level (like private). There is no way a pragma could override this behavior.
See 9.2/14:
Nonstatic data members of a (non-union) class with the same access
control (Clause 11) are allocated so that later members have higher
addresses within a class object. The order of allocation of non-static
data members with different access control is unspecified
Bear in mind that reordering members changes the order in which sub-object constructors and destructors would be called, and possibly other things. It seems extremely risky even given a pragma for the compiler to make these sorts of changes behind the scenes (what if you have a member that depends on the initialization of another member).
Such a pragma would be permitted by the language standard, but I'm not aware of any compiler that implements such a thing.
In C, the behavior of #pragma is specified in section 6.10.6 of the standard (the link is to the latest draft):
A preprocessing directive of the form
# pragma pp-tokensopt new-line
where the preprocessing token STDC does not immediately
follow pragma in the directive (prior to any macro replacement)
causes the implementation to behave in an implementation-defined
manner. The behavior might cause translation to fail or cause the
translator or the resulting program to behave in a non-conforming
manner. Any such pragma that is not recognized by the implementation
is ignored.
So a #pragma can, in effect, violate the rules of the language.
The relevant rule in this case is that struct members are laid out in the order in which they're declared. 6.7.2.1 paragraph 15:
Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa.
There may be unnamed padding within a structure object, but not at its
beginning.
The bad news: The C standard requires struct members to be laid out in the order in which they're declared. The first member must be at offset 0. There may be arbitrary padding between members, or after the last one, but they cannot be reordered.
The good news: The language permits an implementation to define a #pragma that specifies a layout that violates the above rule.
The bad news: As far as I know, no implementation actually does so. Even if one did, there are other implementations that do not, so any code that uses such a #pragma would be non-portable. (Though at least if the name of the #pragma is unique, any compilers that don't recognize it are required to ignore it, so your code would still compile.)
That's for C. The C++ rules for #pragma are very similar to the C rules. I'm reasonably sure the C++ rules for struct layout are also similar to C's; inheritance makes things a little more complex.
Consider the following struct:
class Foo {
int a;
};
Testing in g++, I get that sizeof(Foo) == 4 but is that guaranteed by the standard? Would a compiler be allowed to notice that a is an unused private field and remove it from the in-memory representation of the class (leading to a smaller sizeof)?
I don't expect any compilers to actually do that kind of optimization but this question popped up in a language lawyering discussion so now I'm curious.
The C++ standard doesn't define a lot about memory layouts. The fundamental rule for this case is item 4 under section 9 Classes:
4 Complete objects and member subobjects of class type shall have nonzero size. [ Note: Class objects can be assigned, passed as arguments to functions, and returned by functions (except objects of classes for which copying or moving has been restricted; see 12.8). Other plausible operators, such as equality comparison, can be defined by the user; see 13.5. — end note ]
Now there is one more restriction, though: Standard-layout classes. (no static elements, no virtuals, same visibility for all members) Section 9.2 Class members requires layout compatibility between different classes for standard-layout classes. This prevents elimination of members from such classes.
For non-trivial non-standard-layout classes I see no further restriction in the standard. The exact behavior of sizeof(), reinterpret_cast(), ... are implementation defined (i.e. 5.2.10 "The mapping function is implementation-defined.").
The answer is yes and no. A compiler could not exhibit exactly that behaviour within the standard, but it could do so partly.
There is no reason at all why a compiler could not optimise away the storage for the struct if that storage is never referenced. If the compiler gets its analysis right, then no program that you could write would ever be able to tell whether the storage exists or not.
However, the compiler cannot report a smaller sizeof() thereby. The standard is pretty clear that objects have to be big enough to hold the bits and bytes they contain (see for example 3.9/4 in N3797), and to report a sizeof smaller than that required to hold an int would be wrong.
At N3797 5.3.2:
The sizeof operator yields the number of bytes in the object
representation of its operand
I do not se that 'representation' can change according to whether the struct or member is referenced.
As another way of looking at it:
struct A {
int i;
};
struct B {
int i;
};
A a;
a.i = 0;
assert(sizeof(A)==sizeof(B));
I do not see that this assert can be allowed to fail in a standards-conforming implementation.
If you look at templates, you'll notice that "optimization" of such often ends up with nearly nothing in the output even though the template files may be thousands of lines...
I think that the optimization you are talking about will nearly always occur in a function when the object is used on the stack and the object doesn't get copied or passed down to another function and the private field is never accessed (not even initialized... which could be viewed as a bug!)
The standard defines that Unions cannot be used as Base class, but is there any specific reasoning for this? As far as I understand Unions can have constructors, destructors, also member variables, and methods to operate on those varibales. In short a Union can encapsulate a datatype and state which might be accessed through member functions. Thus it in most common terms qualifies for being a class and if it can act as a class then why is it restricted from acting as a base class?
Edit: Though the answers try to explain the reasoning I still do not understand how Union as a Derived class is worst than when Union as just a class. So in hope of getting more concrete answer and reasoning I will push this one for a bounty. No offence to the already posted answers, Thanks for those!
Tony Park gave an answer which is pretty close to the truth. The C++ committee basically didn't think it was worth the effort to make unions a strong part of C++, similarly to the treatment of arrays as legacy stuff we had to inherit from C but didn't really want.
Unions have problems: if we allow non-POD types in unions, how do they get constructed? It can certainly be done, but not necessarily safely, and any consideration would require committee resources. And the final result would be less than satisfactory, because what is really required in a sane language is discriminated unions, and bare C unions could never be elevated to discriminated unions in way compatible with C (that I can imagine, anyhow).
To elaborate on the technical issues: since you can wrap a POD-component only union in a struct without losing anything, there's no advantage allowing unions as bases. With POD-only union components, there's no problem with explicit constructors simply assigning one of the components, nor with using a bitblit (memcpy) for compiler generated copy constructor (or assignment).
Such unions, however, aren't useful enough to bother with except to retain them so existing C code can be considered valid C++. These POD-only unions are broken in C++ because they fail to retain a vital invariant they possess in C: any data type can be used as a component type.
To make unions useful, we must allow constructable types as members. This is significant because it is not acceptable to merely assign a component in a constructor body, either of the union itself, or any enclosing struct: you cannot, for example, assign a string to an uninitialised string component.
It follows one must invent some rules for initialising union component with mem-initialisers, for example:
union X { string a; string b; X(string q) : a(q) {} };
But now the question is: what is the rule? Normally the rule is you must initialise every member and base of a class, if you do not do so explicitly, the default constructor is used for the remainder, and if one type which is not explicitly initialised does not have a default constructor, it's an error [Exception: copy constructors, the default is the member copy constructor].
Clearly this rule can't work for unions: the rule has to be instead: if the union has at least one non-POD member, you must explicitly initialise exactly one member in a constructor. In this case, no default constructor, copy constructor, assignment operator, or destructor will be generated and if any of these members are actually used, they must be explicitly supplied.
So now the question becomes: how would you write, say, a copy constructor? It is, of course quite possible to do and get right if you design your union the way, say, X-Windows event unions are designed: with the discriminant tag in each component, but you will have to use placement operator new to do it, and you will have to break the rule I wrote above which appeared at first glance to be correct!
What about default constructor? If you don't have one of those, you can't declare an uninitialised variable.
There are other cases where you can determine the component externally and use placement new to manage a union externally, but that isn't a copy constructor. The fact is, if you have N components you'd need N constructors, and C++ has a broken idea that constructors use the class name, which leaves you rather short of names and forces you to use phantom types to allow overloading to choose the right constructor .. and you can't do that for the copy constructor since its signature is fixed.
Ok, so are there alternatives? Probably, yes, but they're not so easy to dream up, and harder to convince over 100 people that it's worthwhile to think about in a three day meeting crammed with other issues.
It is a pity the committee did not implement the rule above: unions are mandatory for aligning arbitrary data and external management of the components is not really that hard to do manually, and trivial and completely safe when the code is generated by a suitable algorithm, in other words, the rule is mandatory if you want to use C++ as a compiler target language and still generate readable, portable code. Such unions with constructable members have many uses but the most important one is to represent the stack frame of a function containing nested blocks: each block has local data in a struct, and each struct is a union component, there is no need for any constructors or such, the compiler will just use placement new. The union provides alignment and size, and cast free component access.
[And there is no other conforming way to get the right alignment!]
Therefore the answer to your question is: you're asking the wrong question. There's no advantage to POD-only unions being bases, and they certainly can't be derived classes because then they wouldn't be PODs. To make them useful, some time is required to understand why one should follow the principle used everywhere else in C++: missing bits aren't an error unless you try to use them.
Union is a type that can be used as any one of its members depending on which member has been set - only that member can be later read.
When you derive from a type the derived type inherits the base type - the derived type can be used wherever the base type could be. If you could derive from a union the derived class could be used (not implicitly, but explicitly through naming the member) wherever any of the union members could be used, but among those members only one member could be legally accessed. The problem is the data on which member has been set is not stored in the union.
To avoid this subtle yet dangerous contradiction that in fact subverts a type system deriving from a union is not allowed.
Bjarne Stroustrup said 'there seems little reason for it' in The Annotated C++ Reference Manual.
The title asks why unions can't be a base class, but the question appears to be about unions as a derived class. So, which is it?
There's no technical reason why unions can't be a base class; it's just not allowed. A reasonable interpretation would be to think of the union as a struct whose members happen to potentially overlap in memory, and consider the derived class as a class that inherits from this (rather odd) struct. If you need that functionality, you can usually persuade most compilers to accept an anonymous union as a member of a struct. Here's an example, that's suitable for use as a base class. (And there's an anonymous struct in the union for good measure.)
struct V3 {
union {
struct {
float x,y,z;
};
float f[3];
};
};
The rationale for unions as a derived class is probably simpler: the result wouldn't be a union. Unions would have to be the union of all their members, and all of their bases. That's fair enough, and might open up some interesting template possibilities, but you'd have a number of limitations (all bases and members would have to be POD -- and would you be able to inherit twice, because a derived type is inherently non-POD?), this type of inheritance would be different from the other type the language sports (OK, not that this has stopped C++ before) and it's sort of redundant anyway -- the existing union functionality would do just as well.
Stroustrup says this in the D&E book:
As with void *, programmers should know that unions ... are inherently dangerous, should be avoided wherever possible, and should be handled with special care when actually needed.
(The elision doesn't change the meaning.)
So I imagine the decision is arbitrary, and he just saw no reason to change the union functionality (it works fine as-is with the C subset of C++), and so didn't design any integration with the new C++ features. And when the wind changed, it got stuck that way.
I think you got the answer yourself in your comments on EJP's answer.
I think unions are only included in C++ at all in order to be backwards compatible with C. I guess unions seemed like a good idea in 1970, on systems with tiny memory spaces. By the time C++ came along I imagine unions were already looking less useful.
Given that unions are pretty dangerous anyway, and not terribly useful, the vast new opportunities for creating bugs that inheriting from unions would create probably just didn't seem like a good idea :-)
Here's my guess for C++ 03.
As per $9.5/1, In C++ 03, Unions can not have virtual functions. The whole point of a meaningful derivation is to be able to override behaviors in the derived class. If a union cannot have virtual functions, that means that there is no point in deriving from a union.
Hence the rule.
You can inherit the data layout of a union using the anonymous union feature from C++11.
#include <cstddef>
template <size_t N,typename T>
struct VecData {
union {
struct {
float x;
float y;
float z;
};
float a[N];
};
};
template <size_t N, typename T>
class Vec : public VecData<N,T> {
//methods..
};
In general its almost always better not work with unions directly but enclose them within a struct or class. Then you can base your inheritance off the struct outer layer and use unions within if you need to.
Is the way C++ structs are laid out set by the standard, or at least common across compilers?
I have a struct where one of its members needs to be aligned on 16 byte boundaries, and this would be easier if I can guarantee the ordering of the fields.
Also, for non-virtual classes, is the address of the first element also likely to be the address of the struct?
I'm most interested in GCC and MSVS.
C and C++ both guarantee that fields will be laid out in memory in the same order as you define them. For C++ that's only guaranteed for a POD type1 (anything that would be legitimate as a C struct [Edit: C89/90 -- not, for example, a C99 VLA] will also qualify as a POD).
The compiler is free to insert padding between members and/or at the end of the struct. Most compilers give you some way to control that (e.g., #pragma pack(N)), but it does vary between compilers.
1Well, there is one corner case they didn't think of, where it isn't guaranteed for a POD type -- an access specifier breaks the ordering guarantee:
struct x {
int x;
int y;
public:
int z;
};
This is a POD type, but the public: between y and z means they could theoretically be re-ordered. I'm pretty sure this is purely theoretical though -- I don't know of any compiler that does reorder the members in this situation (and unless memory fails me even worse than usual today, this is fixed in C++0x).
Edit: the relevant parts of the standard (at least most of them) are §9/4:
A POD-struct is an aggregate class that has no non-volatile
data members of type pointer to member, non-POD-struct, non-
POD-union (or array of such types) or reference, and has no
user-defined copy assignment operator and no user-defined
destructor.
and §8.5.1/1:
An aggregate is an array or a class (clause 9) with no user-
declared constructors (12.1), no private or protected non-
static data members (clause 11), no base classes (clause 10)
and no virtual functions (10.3).
and §9.2/12:
...the order of allocation of non-static data members
separated by an access-specifier is unspecified (11.1).
Though that's restricted somewhat by §9.2/17:
A pointer to a POD-struct object, suitably converted using
a reinterpret_cast, points to its initial member...
Therefore, (even if preceded by a public:, the first member you define must come first in memory. Other members separated by public: specifiers could theoretically be rearranged.
I should also point out that there's some room for argument about this. In particular, there's also a rule in §9.2/14:
Two POD-struct (clause 9) types are layout-compatible if they have the same number of nonstatic data members, and corresponding nonstatic data members (in order) have layout-compatible types (3.9).
Therefore, if you have something like:
struct A {
int x;
public:
int y;
public:
int z;
};
It is required to be layout compatible with:
struct B {
int x;
int y;
int z;
};
I'm pretty sure this is/was intended to mean that the members of the two structs must be laid out the same way in memory. Since the second one clearly can't have its members rearranged, the first one shouldn't be either. Unfortunately, the standard never really defines what "layout compatible" means, rendering the argument rather weak at best.
C++ inherits from c the need/desire to work efficiently on many platforms, and therefore leaves certain things up to the compiler. Aside from requiring elements to appear in the specified order, this is one of them.
But many compilers support #pragmas and options to let you establish come control of the packing. See your compiler's documentation.
Is the way C++ structs are laid out
set by the standard, or at least
common across compilers?
There is no guarantee in the standard. I would not depend on compilers having the same alignment
I have a struct where one of its
members needs to be aligned on 16 byte
boundaries, and this would be easier
if I can guarantee the ordering of the
fields.
Compilers will not change the order of the fields.
Here are some links on how to set them for both GCC and MSVC:
For GCC: http://developer.apple.com/mac/library/documentation/DeveloperTools/gcc-4.0.1/gcc/Structure_002dPacking-Pragmas.html
MSVC: http://msdn.microsoft.com/en-us/library/ms253935(VS.80).aspx and http://msdn.microsoft.com/en-us/library/2e70t5y1(VS.80).aspx
I would keep them as structures and use an extern "C" to ensure that it works properly. Maybe not needed but that would definitely work.
C structs (and C++ PODs, for compatibility) are required by the standard to have sequential layout.
The only difference between compilers is alignment, but fortunately, both MSVC and GCC support #pragma pack.
If you want to see the memory layout of all class types in memory under MSVC, add the /d1reportAllClassLayout switch to the compiler command line.
If you want to see what one class is layed out like, add /d1reportSingleClassLayoutNNNNN where NNNNN is the name of the class.
Structures are laid out sequentially in memory. However, the way they're aligned in memory varies based on the operating system.
For things bigger than 4 bytes, there's a difference between Windows and Linux. Linux aligns them as if they're 4 bytes, so for example a double (8 bytes) could start at p+4, p+8, p+12, etc. where p is the start of the structure. In Windows, a double (8 bytes) needs to start at an address that's a multiple of 8 so p+8, p+16, p+24, etc.