Is the way C++ structs are laid out set by the standard, or at least common across compilers?
I have a struct where one of its members needs to be aligned on 16 byte boundaries, and this would be easier if I can guarantee the ordering of the fields.
Also, for non-virtual classes, is the address of the first element also likely to be the address of the struct?
I'm most interested in GCC and MSVS.
C and C++ both guarantee that fields will be laid out in memory in the same order as you define them. For C++ that's only guaranteed for a POD type1 (anything that would be legitimate as a C struct [Edit: C89/90 -- not, for example, a C99 VLA] will also qualify as a POD).
The compiler is free to insert padding between members and/or at the end of the struct. Most compilers give you some way to control that (e.g., #pragma pack(N)), but it does vary between compilers.
1Well, there is one corner case they didn't think of, where it isn't guaranteed for a POD type -- an access specifier breaks the ordering guarantee:
struct x {
int x;
int y;
public:
int z;
};
This is a POD type, but the public: between y and z means they could theoretically be re-ordered. I'm pretty sure this is purely theoretical though -- I don't know of any compiler that does reorder the members in this situation (and unless memory fails me even worse than usual today, this is fixed in C++0x).
Edit: the relevant parts of the standard (at least most of them) are §9/4:
A POD-struct is an aggregate class that has no non-volatile
data members of type pointer to member, non-POD-struct, non-
POD-union (or array of such types) or reference, and has no
user-defined copy assignment operator and no user-defined
destructor.
and §8.5.1/1:
An aggregate is an array or a class (clause 9) with no user-
declared constructors (12.1), no private or protected non-
static data members (clause 11), no base classes (clause 10)
and no virtual functions (10.3).
and §9.2/12:
...the order of allocation of non-static data members
separated by an access-specifier is unspecified (11.1).
Though that's restricted somewhat by §9.2/17:
A pointer to a POD-struct object, suitably converted using
a reinterpret_cast, points to its initial member...
Therefore, (even if preceded by a public:, the first member you define must come first in memory. Other members separated by public: specifiers could theoretically be rearranged.
I should also point out that there's some room for argument about this. In particular, there's also a rule in §9.2/14:
Two POD-struct (clause 9) types are layout-compatible if they have the same number of nonstatic data members, and corresponding nonstatic data members (in order) have layout-compatible types (3.9).
Therefore, if you have something like:
struct A {
int x;
public:
int y;
public:
int z;
};
It is required to be layout compatible with:
struct B {
int x;
int y;
int z;
};
I'm pretty sure this is/was intended to mean that the members of the two structs must be laid out the same way in memory. Since the second one clearly can't have its members rearranged, the first one shouldn't be either. Unfortunately, the standard never really defines what "layout compatible" means, rendering the argument rather weak at best.
C++ inherits from c the need/desire to work efficiently on many platforms, and therefore leaves certain things up to the compiler. Aside from requiring elements to appear in the specified order, this is one of them.
But many compilers support #pragmas and options to let you establish come control of the packing. See your compiler's documentation.
Is the way C++ structs are laid out
set by the standard, or at least
common across compilers?
There is no guarantee in the standard. I would not depend on compilers having the same alignment
I have a struct where one of its
members needs to be aligned on 16 byte
boundaries, and this would be easier
if I can guarantee the ordering of the
fields.
Compilers will not change the order of the fields.
Here are some links on how to set them for both GCC and MSVC:
For GCC: http://developer.apple.com/mac/library/documentation/DeveloperTools/gcc-4.0.1/gcc/Structure_002dPacking-Pragmas.html
MSVC: http://msdn.microsoft.com/en-us/library/ms253935(VS.80).aspx and http://msdn.microsoft.com/en-us/library/2e70t5y1(VS.80).aspx
I would keep them as structures and use an extern "C" to ensure that it works properly. Maybe not needed but that would definitely work.
C structs (and C++ PODs, for compatibility) are required by the standard to have sequential layout.
The only difference between compilers is alignment, but fortunately, both MSVC and GCC support #pragma pack.
If you want to see the memory layout of all class types in memory under MSVC, add the /d1reportAllClassLayout switch to the compiler command line.
If you want to see what one class is layed out like, add /d1reportSingleClassLayoutNNNNN where NNNNN is the name of the class.
Structures are laid out sequentially in memory. However, the way they're aligned in memory varies based on the operating system.
For things bigger than 4 bytes, there's a difference between Windows and Linux. Linux aligns them as if they're 4 bytes, so for example a double (8 bytes) could start at p+4, p+8, p+12, etc. where p is the start of the structure. In Windows, a double (8 bytes) needs to start at an address that's a multiple of 8 so p+8, p+16, p+24, etc.
Related
As it was made know to me, adjacent members of the same type can have padding between them.
Even a structure like this could have it.
struct S{
int x;
int y;
};
First, why would this structure need padding anyway?
Second, why would padding be between the members of the same type?
The C++ standard allows padding between sub objects (members and bases) of classes. As such, there "can" be padding because the standard allows it. The standard allows the padding because there are cases where that is necessary in order to satisfy the alignment requirement of the members.
The standard doesn't enumerate cases where padding isn't needed. It is entirely up to the language implementation to choose when to pad and when not to as long as the alignment requirements are satisfied.
why would this structure need padding anyway?
There is no practical reason (that I know of) why that class would need padding. However, that is not the only class definition that exists and there are other classes which may need padding.
Consider the following struct:
class Foo {
int a;
};
Testing in g++, I get that sizeof(Foo) == 4 but is that guaranteed by the standard? Would a compiler be allowed to notice that a is an unused private field and remove it from the in-memory representation of the class (leading to a smaller sizeof)?
I don't expect any compilers to actually do that kind of optimization but this question popped up in a language lawyering discussion so now I'm curious.
The C++ standard doesn't define a lot about memory layouts. The fundamental rule for this case is item 4 under section 9 Classes:
4 Complete objects and member subobjects of class type shall have nonzero size. [ Note: Class objects can be assigned, passed as arguments to functions, and returned by functions (except objects of classes for which copying or moving has been restricted; see 12.8). Other plausible operators, such as equality comparison, can be defined by the user; see 13.5. — end note ]
Now there is one more restriction, though: Standard-layout classes. (no static elements, no virtuals, same visibility for all members) Section 9.2 Class members requires layout compatibility between different classes for standard-layout classes. This prevents elimination of members from such classes.
For non-trivial non-standard-layout classes I see no further restriction in the standard. The exact behavior of sizeof(), reinterpret_cast(), ... are implementation defined (i.e. 5.2.10 "The mapping function is implementation-defined.").
The answer is yes and no. A compiler could not exhibit exactly that behaviour within the standard, but it could do so partly.
There is no reason at all why a compiler could not optimise away the storage for the struct if that storage is never referenced. If the compiler gets its analysis right, then no program that you could write would ever be able to tell whether the storage exists or not.
However, the compiler cannot report a smaller sizeof() thereby. The standard is pretty clear that objects have to be big enough to hold the bits and bytes they contain (see for example 3.9/4 in N3797), and to report a sizeof smaller than that required to hold an int would be wrong.
At N3797 5.3.2:
The sizeof operator yields the number of bytes in the object
representation of its operand
I do not se that 'representation' can change according to whether the struct or member is referenced.
As another way of looking at it:
struct A {
int i;
};
struct B {
int i;
};
A a;
a.i = 0;
assert(sizeof(A)==sizeof(B));
I do not see that this assert can be allowed to fail in a standards-conforming implementation.
If you look at templates, you'll notice that "optimization" of such often ends up with nearly nothing in the output even though the template files may be thousands of lines...
I think that the optimization you are talking about will nearly always occur in a function when the object is used on the stack and the object doesn't get copied or passed down to another function and the private field is never accessed (not even initialized... which could be viewed as a bug!)
I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Given the following code:
// that one is a standard layout, and trivial which makes it a c++11 POD
struct Bar
{
public:
int x;
public:
int y;
};
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
In C++03, it can. In C++11 it cannot. C++11's standard layout rules only require that all of the members have the same access control. They don't have to be declared in the same access control region.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
I think you're misunderstanding things. The C++11 rules allow more types to be standard-layout (and thus potentially layout-compatible with C types), not less. Thus, there's no real downside to relaxing the rules.
I thought that the whole point of PODs (c++11, trivial + standard-layout) is to make sure the type is compatible with C.
Not exactly the whole point of it, but yes, that is one of the properties of PODs.
// that one is a standard layout, and trivial which makes it a c++11 POD
Correct.
AFAIU, compiler might reorder x and y. Wouldn't that break compatibility with C?
We already established it is a POD, which means the compiler will maintain compatibility with C. Maintaining compatibility with C does not break compatibility with C.
Why that 98/03 POD definition relaxation in c++11 considered to be a good idea?
Because it doesn't break anything.
The point of POD is not just to make sure the type is compatible with C - note that a type with an access specifier (public, private, etc.) is by definition not compatible with C since C doesn't have access specifiers. The main property of a POD type is that it can be memcpy'ed around.
Having more than one access specifier in a C++ type does permit the compiler to lay out the type in a non-specified way, and that's been true for a while (it's not new with C++11):
From C++03 9.2/12
Nonstatic data members of a (non-union) class declared without an
intervening access-specifier are allocated so that later members have
higher addresses within a class object. The order of allocation of
nonstatic data members separated by an access-specifier is unspecified
(11.1).
However, that doesn't make a type a non-POD - it can still be a POD, just not one that can be portably expressed in C.
I was wondering what the size limit for a class is. I did a simple test:
#define CLS(name,other) \
class name\
{\
public: \
name() {};\
other a;\
other b;\
other c;\
other d;\
other e;\
other f;\
other g;\
other h;\
other i;\
other j;\
other k;\
};
class A{
int k;
public:
A(){};
};
CLS(B,A);
CLS(C,B);
CLS(D,C);
CLS(E,D);
CLS(F,E);
CLS(G,F);
CLS(H,G);
CLS(I,H);
CLS(J,I);
It fails to compile with
"'J' : class is too large"
If I remove the final declaration - CLS(J,I);, it all compiles fine.
Is this a compiler-imposed restriction, or is it somewhere in the standard?
In C++11 this is Annex B. Implementations can impose limits, but they should be at least:
Size of an object [262 144].
Data members in a single class [16 384].
Members declared in a single class [4 096].
The third one isn't directly relevant to the kind of construction you're using, I mention it just because it indicates that the second one is indeed the total members, presumably including those in bases and I'm not sure about members-of-members. But it's not just about the members listed in a single class definition.
Your implementation appears to have given up either 2^31 data members, or at size 2^32, since it accepts I but not J. It's fairly obviously reasonable for a compiler to refuse to consider classes with size greater than SIZE_MAX, even if the program happens not to instantiate it or use sizeof on the type. So even with the best possible effort on the part of the compiler I wouldn't ever expect this to work on a 32 bit implementation.
Note that "these quantities are only guidelines and do not determine compliance", so a conforming implication can impose an arbitrary smaller limit even where it has sufficient resources to compile a program that uses larger numbers. There's no minimum limit for conformance.
There are various opportunities in the C++ standard for a conforming implementation to be useless due to ridiculously small resource limits, so there's no additional harm done if this is another one.
C++03 is more-or-less the same:
Size of an object [262 144].
Data members in a single class, structure, or union [16 384].
Members declared in a single class [4 096].
I wanted to mention another place in which class size limit is mentioned, which is in section 1.2 of the Itanium C++ ABI draft
Various representations specified by this ABI impose limitations on
conforming user programs. These include, for the 64-bit Itanium ABI:
The offset of a non-virtual base subobject in the full object
containing it must be representable by a 56-bit signed integer (due to
RTTI implementation). This implies a practical limit of 2**55 bytes on
the size of a class.
I'm sure its compiler dependent. You can run your compiler in a preprocess only mode to see what the generated output is if you're curious. You might also want to look at template expansion rather than macros.
This is a question that was sparked by Rob Walker's answer here.
Suppose I declare a class/struct like so:
struct
{
char A;
int B;
char C;
int D;
};
Is it safe to assume that these members will be declared in exactly that order in memory, or is this a compiler dependent thing? I'm asking because I had always assumed that the compiler can do whatever it wants with them.
This leads into my next question. If the above example causes memory alignment issues, why can the compiler not just turn that into something like this implicitly:
struct
{
char A;
char C;
int B;
int D;
};
(I'm primarily asking about C++, but I'd be interested to hear the C answer as well)
Related topics
Why doesn't GCC optimize structs?
C99 §6.7.2.1 clause 13 states:
Within a structure object, the
non-bit-field members and the units in
which bit-fields reside have addresses
that increase in the order in which
they are declared.
and goes on to say a bit more about padding and addresses. The C89 equivalent section is §6.5.2.1.
C++ is a bit more complicated. In the 1998 and 2003 standards, there is §9.2 clause 12 (clause 15 in C++11):
Nonstatic data members of a
(non-union) class declared without an
intervening access-specifier are
allocated so that later members have
higher addresses within a class
object. The order of allocation of
nonstatic data members separated by an
access-specifier is unspecified
(11.1). Implementation alignment
requirements might cause two adjacent
members not to be allocated
immediately after each other; so might
requirements for space for managing
virtual functions (10.3) and virtual
base classes (10.1).
The data members are arranged in the order declared. The compiler is free to intersperse padding to arrange the memory alignment it likes (and you'll find that many compilers have a boatload a alignment specification options---useful if mixing bits compiled by different programs.).
See also Why doesn't GCC optimize structs?.
It appears that this answer is somewhat obsolete for C++. You learn something everyday. Thanks aib, Nemanja.
I cannot speak for C++, but in C the order is guaranteed to be the same order in memory as declared in the struct.
Basically, you can count on that only for the classes with a standard layout. Strictly speaking, standard layout is a C++0x thing, but it is really just standardizing existing practice/
Aside from padding for alignment, no structure optimization is allowed by any compiler (that I am aware of) for C or C++. I can't speak for C++ classes, as they may be another beast entirely.
Consider your program is interfacing with system/library code on Windows but you want to use GCC. You would have to verify that GCC used an identical layout-optimization algorithm so all your structures would be packed correctly before sending them to the MS-compiled code.
While browsing the related topics at the right, I looked at this question. I figure this may be an interesting corner case when thinking about these issues (unless it's more common than I realize).
To paraphrase, if you have a struct in C that looks something like this:
struct foo{};
and subclass it like so in C++ (using a separate compilation unit):
extern "C" foo;
struct bar: public foo{};
Then the memory alignment won't necessarily be the same for the reasons aib mentions (even amongst compilers from the same vendor).