There is a quote from cppreference:
Every object type has the property called alignment requirement, which
is an integer value (of type std::size_t, always a power of 2)
representing the number of bytes between successive addresses at which
objects of this type can be allocated.
I understand, this reference is non-normative. But there is no something about value of alignof(T) in the standard, rather than it is no more than alignof(std::max_align_t).
It is not obviously, that alignment is power of 2. Why does alignment not be a 3?
The standard has the final word for the language, so here a quote of that section. I bolded the power-of-2 requirement:
3.11 Alignment [basic.align]
1 Object types have alignment requirements (3.9.1, 3.9.2) which place restrictions on the addresses at which an object of that type may be allocated. An alignment is an implementation-defined integer value representing the number of bytes between successive addresses at which a given object can be allocated. An object type imposes an alignment requirement on every object of that type; stricter alignment can be requested using the alignment specifier (7.6.2).
2 A fundamental alignment is represented by an alignment less than or equal to the greatest alignment supported by the implementation in all contexts, which is equal to alignof(std::max_align_t) (18.2). The alignment required for a type might be different when it is used as the type of a complete object and when it is used as the type of a subobject. [ Example:
struct B { long double d; };
struct D : virtual B { char c; }
When D is the type of a complete object, it will have a subobject of type B, so it must be aligned appropriately for a long double. If D appears as a subobject of another object that also has B as a virtual base class, the B subobject might be part of a different subobject, reducing the alignment requirements on the D subobject. —end example ] The result of the alignof operator reflects the alignment requirement of the type in the complete- object case.
3 An extended alignment is represented by an alignment greater than alignof(std::max_align_t). It is implementation-defined whether any extended alignments are supported and the contexts in which they are supported (7.6.2). A type having an extended alignment requirement is an over-aligned type. [ Note: every over-aligned type is or contains a class type to which extended alignment applies (possibly through a non-static data member). —end note ]
4 Alignments are represented as values of the type std::size_t. Valid alignments include only those values returned by an alignof expression for the fundamental types plus an additional implementation-defined set of values, which may be empty. Every alignment value shall be a non-negative integral power of two.
5 Alignments have an order from weaker to stronger or stricter alignments. Stricter alignments have larger alignment values. An address that satisfies an alignment requirement also satisfies any weaker valid alignment requirement.
Why did all implementations conform to that requirement (That's part of the reason it could be included at all)?
Well, because it is natural to multiply / divide / mask powers of 2 in binary, and all systems were (excluding some really ancient ones), are, and for the foreseeable future will stay fundamentally binary.
Being natural means it is much more efficient than any other multiplications / divisions / modulo arithmetic, sometimes by orders of magnitude.
As #MooingDuck points out, this fundamental binary nature of computing platforms has already pervaded the language and its standard to such an extent, trying to build a non-binary conforming implementation is about on-par with untying the gordian knot without just cutting it. There are really few computer languages where that's not true.
Also, see a table of word sizes on wikipedia for corroboration.
That's how computers are built.
A computer has a natural 'word' size that is handled more easily than other sizes. On 64-bit CPUs, the size is 8-bytes. Operating on 8-bytes is most efficient. The hardware is built in a way that fetching memory that is aligned to this word size is also more efficient. So alignment is usually based on the CPU's word size.
Word sizes are powers of two because, again, that's how computers are built. Everything comes down to bits - so does the number of bits in a word. It's easier to design the hardware where the number of bits in a word is itself a power of two.
Related
Section 3.6 of C11 standard defines "byte" as "addressable unit of data storage ... to hold ... character".
Section 1.7 of C++11 standard defines "byte" as "the fundamental storage unit in the C++ memory model ... to contain ... character".
Both definitions does not say that "byte" is the minimum addressable unit. Is this because standards intentionally want to abstract from a specific machine ? Can you provide a real example of machine where C/C++ compiler were decided to have "byte" longer/shorter than the minimum addressable unit ?
A byte is the smallest addressable unit in strictly conforming C code. Whether the machine on which the C implementation executes a program supports addressing smaller units is irrelevant to this; the C implementation must present a view in which bytes are the smallest addressable unit in strictly conforming C code.
A C implementation may support addressing smaller units as an extension, such as simply by defining the results of certain pointer operations that are otherwise undefined by the C standard.
One example of a real machine and its compiler where the minimal addressable unit is smaller than a byte is the 8051 family. One compiler I was used to is Keil C51.
The minimal addressable unit is a bit. You can define a variable of this type, you can read and write it. However, the syntax to define the variable is non-standard. Of course, C51 needs several extensions to support all of this. BTW, pointers to bits are not allowed.
For example:
unsigned char bdata bitsAdressable;
sbit bitAddressed = bitsAdressable^5;
void f(void) {
bitAddressed = 1;
}
bit singleBit;
void g(bit value) {
singleBit = value;
}
Both definitions does not say that "byte" is the minimum addressable unit.
That's because they don't need to. Byte-wise types (char, unsigned char, std::byte, etc) have sufficient restrictions that enforce this requirement.
The size of byte-wise types is explicitly defined to be precisely 1:
sizeof(char), sizeof(signed char) and sizeof(unsigned char) are 1.
The alignment of byte-wise types is the smallest alignment possible:
Furthermore, the narrow character types (6.9.1) shall have the weakest alignment requirement
This doesn't have to be an alignment of 1, of course. Except... it does.
See, if the alignment were higher than 1, that would mean that a simple byte array wouldn't work. Array indexing is based on pointer arithmetic, and pointer arithmetic determines the next address based on sizeof(T). But if alignof(T) is greater than sizeof(T), then the second element in any array of T would be misaligned. That's not allowed.
So even though the standard doesn't explicitly say that the alignment of bytewise types is 1, other requirements ensure that it must be.
Overall, this means that every pointer to an object has an alignment at least as restrictive as a byte-wise type. So no object pointer can be misaligned, relative to the alignment of byte-wise types. All valid, non-NULL pointers (pointers to a live object or to a past-the-end pointer) must therefore be at least aligned enough to point to a char.
Similarly, the difference between two pointers is defined in C++ as the difference between the array indices of the elements pointed to by those pointers (pointer arithmetic in C++ requires that the two pointers point into the same array). Additive pointer arithmetic is as previously stated based on the sizeof the type being pointed to.
Given all of these facts, even if an implementation has pointers whose addresses can address values smaller than char, it is functionally impossible for the C++ abstract model to generate a pointer and still have that pointer count as valid (pointing to an object/function, a past-the-end of an array, or be NULL). You could create such a pointer value with a cast from an integer. But you would be creating an invalid pointer value.
So while technically there could be smaller addresses on the machine, you could never actually use them in a valid, well-formed C++ program.
Obviously compiler extensions could do anything. But as far as conforming programs are concerned, it simply isn't possible to generate valid pointers that are misaligned for byte-wise types.
I programmed both the TMS34010 and its successor TMS34020 graphics chips back in the early 1990's and they had a flat address space and were bit addressable i.e. addresses indexed each bit. This was very useful for computer graphics of the time and back when memory was a lot more precious.
The embedded C-compiler didn't really have away to access individual bits directly, since from a (standard) C language point of view the byte was still the smallest unit as pointed out in a previous post.
Thus if you want to read/write a stream of bits in C, you need to read/write (at least) a byte at a time and buffer (for example when writing a Arithmetic or Huffman Coder).
(Thank you everyone who commented and answered, every word helps)
Memory model of a programming language and memory model of the target machine are different things.
Yes, byte is the minimum addressable unit in context of memory model of programming language.
No, byte is not the minimum addressable unit in context of memory model of machine. For example, there are machines where minimum addressable unit is longer or shorter than the "byte" of programming language:
longer: HP Saturn - 4-bit unit vs 8-bit byte gcc (thanks Nate).
shorter: IBM 360 - 36-bit unit vs 6-bit byte (thanks Antti)
longer: Intel 8051 - 1-bit unit vs 8-bit byte (thanks Busybee)
longer: Ti TMS34010 - 1-bit unit vs 8-bit byte (thanks Wcochran)
N3797::3.9/4 [basic.types] :
The object representation of an object of type T is the sequence of N
unsigned char objects taken up by the object of type T, where N equals
sizeof(T). The value representation of an object is the set of bits
that hold the value of type T. For trivially copyable types, the value
representation is a set of bits in the object representation that
determines a value, which is one discrete element of an
implementation-defined set of values
N3797::3.9.1 [basic.fundamental] says:
For narrow character types, all bits of the object representation
participate in the value representation.
Consider the following struct:
struct A
{
char a;
int b;
}
I think for A not all bits of the object representation participate in the value representation because of padding added by implementation. But what about others fundamentals type?
The Standard says:
N3797::3.9.1 [basic.fundamental]
For narrow character types, all bits of the object representation
participate in the value representation.
These requirements do not hold for other types.
I can't imagine why it doesn't hold for say int or long. What's the reason? Could you clarify?
An example might be the Unisys mainframes, where an int has 48
bits, but only 40 participate in the value representation (and INT_MAX is 2^39-1); the
others must be 0. I imagine that any machine with a tagged
architecture would have similar issues.
EDIT:
Just some further information: the Unisys mainframes are
probably the only remaining architectures which are really
exotic: the Unisys Libra (ex-Burroughs) have a 48 bit word, use signed
magnitude for integers, and have a tagged architecture, where
the data itself contains information concerning its type. The
Unisys Dorado are the ex-Univac: 36 bit one's complement (but no
reserved bits for tagging) and 9 bit char's.
From what I understand, however, Unisys is phasing them out (or
has phased them out in the last year) in favor of Intel based
systems. Once they disappear, pretty much all systems will be
2's complement, 32 or 64 bits, and all but the IBM mainframes
will use IEEE floating poing (and IBM is moving or has moved in
that direction as well). So there won't be any motivation for
the standard to continue with special wording to support them;
in the end, in a couple of years at lesat, C/C++ could probably
follow the Java path, and impose a representation on all of its
basic data types.
This is probably meant to give the compiler headroom for optimizations on some platforms.
Consider for example a 64 bit platform where handling non-64 bit values incurs a large penalty, then it would make sense to have e.g. short only use 16 bits (value repr), but still use 64 bit storage (obj repr).
Similar rationale applies to the Fastest minimum-width integer types mandated by <stdint>. Sometimes larger types are not slower, but faster to use.
As far as I understand at least one case for this is dealing with trap representations, usually on exotic architectures. This issue is covered in N2631: Resolving the difference between C and C++ with regards to object representation of integers. It is is very long but I will quote some sections(The author is James Kanze, so if we are lucky maybe he will drop by and comment further) which says (emphasis mine).
In recent discussions in comp.lang.c++, it became clear that C and C++ have different requirements concerning the object representation of integers, and that at least one real implementation of C does not meet the C++ requirements. The purpose of this paper is to suggest wording to align the C++ standard with C.
It should be noted that the issue only concerns some fairly “exotic” hardware. In this regard, it raises a somewhat larger issue
and:
If C compatibility is desired, it seems to me that the simplest and surest way of attaining this is by incorporating the exact words from the C standard, in place of the current wording. I thus propose that we adopt the wording from the C standard, as follows
and:
Certain object representations need not represent a value of the object type. If the stored value of an object has such a representation and is read by an lvalue expression that does not have character type, the behavior is undefined. If such a representation is produced by a side effect that modifies all or any part of the object by an lvalue expression that does not have character type, the behavior is undefined. Such a representation is called a trap representation.
and:
For signed integer types [...] Which of these applies is implementation-defined, as is whether the value with sign bit 1 and all value bits zero (for the first two), or with sign bit and all value bits 1 (for one's complement), is a trap representation or a normal value. In the case of sign and magnitude and one's complement, if this representation is a normal value it is called a negative zero.
This question was asked to me in an interview, that size of char is 2 bytes in some OS, but in some operating system it is 4 bytes or different.
Why is that so?
Why is it different from other fundamental types, such as int?
That was probably a trick question. The sizeof(char) is always 1.
If the size differs, it's probably because of a non-conforming compiler, in which case the question should be about the compiler itself, not about the C or C++ language.
5.3.3 Sizeof [expr.sizeof]
1 The sizeof operator yields the number of bytes in the object
representation of its operand. The operand is either an expression,
which is not evaluated, or a parenthesized type-id. The sizeof
operator shall not be applied to an expression that has function or
incomplete type, or to an enumeration type before all its enumerators
have been declared, or to the parenthesized name of such types, or to
an lvalue that designates a bit-field. sizeof(char), sizeof(signed
char) and sizeof(unsigned char) are 1. The result of sizeof applied to any other fundamental type (3.9.1) is
implementation-defined. (emphasis mine)
The sizeof of other types than the ones pointed out are implementation-defined, and they vary for various reasons. An int has better range if it's represented in 64 bits instead of 32, but it's also more efficient as 32 bits on a 32-bit architecture.
The physical sizes (in terms of the number of bits) of types are usually dictated by the target hardware.
For example, some CPUs can access memory only in units not smaller than 16-bit. For the best performance, char can then be defined a 16-bit integer. If you want 8-bit chars on this CPU, the compiler has to generate extra code for packing and unpacking of 8-bit values into and from 16-bit memory cells. That extra packing/unpacking code will make your code bigger and slower.
And that's not the end of it. If you subdivide 16-bit memory cells into 8-bit chars, you effectively introduce an extra bit in addresses/pointers. If normal addresses are 16-bit in the CPU, where do you stick this extra, 17th bit? There are two options:
make pointers bigger (32-bit, of which 15 are unused) and waste memory and reduce the speed further
reduce the range of addressable address space by half, wasting memory, and loosing speed
The latter option can sometimes be practical. For example, if the entire address space is divided in halves, one of which is used by the kernel and the other by user applications, then application pointers will never use one bit in their addresses. You can use that bit to select an 8-bit byte in a 16-bit memory cell.
C was designed to run on as many different CPUs as possible. This is why the physical sizes of char, short, int, long, long long, void*, void(*)(), float, double, long double, wchar_t, etc can vary.
Now, when we're talking about different physical sizes in different compilers producing code for the same CPU, this becomes more of an arbitrary choice. However, it may be not that arbitrary as it may seem. For example, many compilers for Windows define int = long = 32 bits. They do that to avoid programmer's confusion when using Windows APIs, which expect INT = LONG = 32 bits. Defining int and long as something else would contribute to bugs due to loss of programmer's attention. So, compilers have to follow suit in this case.
And lastly, the C (and C++) standard operates with chars and bytes. They are the same concept size-wise. But C's bytes aren't your typical 8-bit bytes, they can legally be bigger than that as explained earlier. To avoid confusion you may use the term octet, whose name implies the number 8. A number of protocols uses this word for this very purpose.
Is the byte alignment requirement of a given data type guaranteed to be a power of 2?
Is there something that provides this guarantee other than it "not making sense otherwise" because it wouldn't line up with system page sizes?
(background: C/C++, so feel free to assume data type is a C or C++ type and give C/C++ specific answers.)
Alignment requirement are based on the hardware. Most, if not all, "modern" chips have addresses that are divisible by 8, not just a power of 2. In the past there were non-divisible by 8 chips (I know of a 36 bit architecture).
Things you can assume about alignment, per the C standard:
The alignment requirement of any type divides the size of that type (as determined by sizeof).
The character types char, signed char, and unsigned char have no alignment requirement. (This is actually just a special case of the first point.)
In the modern real world, integer and pointer types have sizes that are powers of two, and their alignment requirements are usually equal to their sizes (the only exception being long long on 32-bit machines). Floating point is a bit less uniform. On 32-bit machines, all floating point types typically have an alignment of 4, whereas on 64-bit machines, the alignment requirement of floating point types is typically equal to the size of the type (4, 8, or 16).
The alignment requirement of a struct should be the least common multiple of the alignment requirements of its members, but a compiler is allowed to impose stricter alignment. However, normally each cpu architecture has an ABI standard that includes alignment rules, and compilers which do not adhere to the standard will generate code that cannot be linked with code built by compilers which follow the ABI standard, so it would be very unusual for a compiler to break from the standard except for very special-purpose use.
By the way, a useful macro that will work on any sane compiler is:
#define alignof(T) ((char *)&((struct { char x; T t; } *)0)->t - (char *)0)
The alignment of a field inside a "struct", optimized for size could very well be on a odd boundary. other then that your "It wouldn't make sense" would probably apply, but I think there is NO guarantee, especially if the program was small model, optimized for size. - Joe
The standard doesn't require alignment, but allows struct/unions/bit fields to silently add padding bytes to get a correct alignment. The compiler is also free to align all your data types on even addresses should it desire.
That being said, this is CPU dependent, and I don't believe there exists a CPU that has an alignment requirement on odd addresses. There are plenty of CPUs with no alignment requirements however, and the compiler may then place variables at any address.
In short, no. It depends on the hardware.
However, most modern CPUs either do byte alignment (e.g., Intel x86 CPUs), or word alignment (e.g., Motorola, IBM/390, RISC, etc.).
Even with word alignment, it can be complicated. For example, a 16-bit word would be aligned on a 2-byte (even) address, a 32-bit word on a 4-byte boundary, but a 64-bit value may only require 4-byte alignment instead of an 8-byte aligned address.
For byte-aligned CPUs, it's also a function of the compiler options. The default alignmen for struct members can usually be specified (usually also with a compiler-specific #pragma).
For basic data types (ints, floats, doubles) usually the alignment matches the size of the type. For classes/structs, the alignment is at least the lowest common multiple of the alignment of all its members (that's the standard)
In Visual Studio you can set your own alignment for a type, but it has to be a power of 2, between 1 and 8192.
In GCC there is a similar mechanism, but it has no such requirement (at least in theory)
What is maximal bit width for bit struct field?
struct i { long long i:127;}
Can I define a bit field inside struct, with size of bitfield up to 128 bit, or 256 bit, or larger? There are some extra-wide vector types, like sse2 (128-bit), avx1/avx2 (256-bit), avx-512 (512-bit for next Xeon Phis) registers; and also extensions like __int128 in gcc.
C99 §6.7.2.1, paragraph 3:
The expression that specifies the
width of a bit-field shall be an
integer constant expression that has
nonnegative value that shall not
exceed the number of bits in an object
of the type that is specified if the
colon and expression are omitted. If
the value is zero, the declaration
shall have no declarator.
C++0xa §9.6, paragraph 1:
... The constant-expression shall be an
integral constant expression with a
value greater than or equal to zero.
The value of the integral constant
expression may be larger than the
number of bits in the object
representation (3.9) of the
bit-field’s type; in such cases the
extra bits are used as padding bits
and do not participate in the value
representation (3.9) of the bit-field.
So in C you can't do that at all, and in C++ it won't do what you want it to.
The C++ Standard sets no limits on the size of a bit-field, other than that it must be greater or equal to zero - section 9.6/1. It also says:
Bit-fields are packed into some
addressable allocation unit. [Note:
bit-fields straddle allocation units
on some machines and not on others.
Bit-fields are assigned right-to-left
on some machines, left-to-right on
others. ]
Which I suppose could be taken to indicate some sort of maximum size.
This does not mean that your specific compiler implementation supports arbitrarily sized bit-fields, of course.
Typically, you cannot allocate more bits than the underlying type has. If long long is 64 bits, then your bitfield is probably limited to :64.
Since the values of bit-fields are assigned to integers, I'd assume that the largest bit-field value you can use is that of the size of intmax_t.
Edit:
From the C99 Spec:
6.7.2.1 Bullet 9:
A bit-field is interpreted as a signed
or unsigned integer type consisting of
the specified number of bits. If
the value 0 or 1 is stored into a
nonzero-width bit-field of type
_Bool, the value of the bit-field shall compare equal to the value
stored.
6.7.2.1 Bullet 10:
An implementation may allocate any
addressable storage unit large enough
to hold a bit- field. If enough space
remains, a bit-field that immediately
follows another bit-field in a
structure shall be packed into
adjacent bits of the same unit. If
insufficient space remains, whether a
bit-field that does not fit is put into
the next unit or overlaps adjacent
units is implementation-defined. The
order of allocation of bit-fields
within a unit (high-order to low-order
or low-order to high-order) is
implementation-defined. The alignment
of the addressable storage unit is
unspecified.