In C++, it's legal for a char, unsigned char, or std::byte pointer to alias any T pointer. Such aliased pointer is capable of accessing the object's representation in raw memory.
My question is whether it's legal and free of undefined-behavior to use a T* pointer derived from arithmetic on a char/unsigned char/std::byte pointer aliasing an original T array object sequence -- provided that the resultant pointer is still aligned correctly and within the original reachability of the aliased sequence.
As a concrete example:
// Array sequence is 2 in length
auto array = std::array<unsigned,2>{0xdead, 0xbeef};
// Derive the address of &array[1] using std::byte pointer
auto p = reinterpret_cast<std::byte*>(array.data());
auto p1 = reinterpret_cast<unsigned*>(p + sizeof(unsigned));
assert(p1 == &array[1]); // This expression should be legal since the pointer is safely derived
assert(*p1 == array[1]); // But is this legal, as per the C++ standard?
The pointer p1 should be safely derived in the above example since it doesn't exceed the reachability of array's bound. I know that in many cases in C++ it's legal to cast a pointer to a type/representation that can't actually be used/dereferenced. Is the resultant pointer here legal to dereference as far as the standard is concerned?
For the purpose of this question, assume that T may not be standard-layout type.
Note: I'm not asking about whether this will work in practice, since I have yet to see this type of code fail on compilers; but I'm curious to know whether this is, strictly-speaking, legal per the C++ standard.
Edit:
I'd rather not derail this question, but there are a few comments that are asserting that p + sizeof(unsigned) is undefined behavior as a result of [expr.add], in particular quoting both [expr.add]/4 and [expr.add]/6 due to p not actually being a byte-array.
There is, in fact, a byte sequence[1], and both [expr.add]/4 and [expr.add]/6 simply do not apply[2]
[1]: char, unsigned char, and std::byte view the object representation of an object, which is defined under [basic.types.general]/4 as being:
The object representation of an object of type T is the sequence of N unsigned char objects taken up by the object of type T, where N equals sizeof(T).
The above means that there is a formal sequence of unsigned char (which std::byte is capable of representing), which means p + sizeof(unsigned) is valid. We aren't viewing through a similar pointer, we are viewing exactly as the underlying object-representation which is legal.
[2]: [expr.add]/4 is simply referring to reachability of array objects. However, as pointed out above, there is a valid and reachable object due to the unsigned char sequence, and we do not exceed reachability -- thus this doesn't apply as a case of undefined behavior.
As far as I can tell, [expr.add]/6 primarily exists to avoid cases where an array of derived types is cast to a base pointer and attempted to be indexed. However it wouldn't apply in this case because we aren't viewing through a "similar" pointer, we are viewing an object representation that is formally defined as a sequence of unsigned chars. This isn't a similar representation, it's exact.
Related
Given..:
struct S {
float f;
int i;
};
.. can you convert from an int * pointing to an i field to a struct S *?
I am interested in whether there is a spec-conforming way to do this conversion for either/both C and C++. And, if not, I am also in interested in practical implementation specific conversions for clang / gcc / MSVC.
For example, would ..:
void f(int *i) {
struct S *s = (struct S *) (((char *) i) - offsetof(S, i));
printf("%f\n", s->f);
}
.. be correct?
.. can you convert from an int * pointing to an i field to a struct S *?
Answer: Yes, but ...
Based on
Pointer
Notes
Although any pointer to object can be cast to pointer to object of a different type, dereferencing a pointer to the type different from the declared type of the object is almost always undefined behavior. See strict aliasing for details.
and
Strict aliasing
Given an object with effective type T1, using an lvalue expression (typically, dereferencing a pointer) of a different type T2 is undefined behavior, unless:
T2 is a character type (char, signed char, or unsigned char)
and
offsetof
The macro offsetof expands to an integer constant expression of type size_t, the value of which is the offset, in bytes, from the beginning of an object of specified type to its specified subobject, including padding if any.
it should be alright, but here is the issue:
Pointer arithmetic
For the purpose of pointer arithmetic, a pointer to an object that is not an element of any array is treated as a pointer to the first element of an array of size 1.
The behavior is defined only if both the original pointer and the result pointer are pointing at elements of the same array or one past the end of that array. Note that executing p-1 when p points at the first element of an array is undefined behavior and may fail on some platforms.
If the pointer P1 points at an element of an array with index I (or one past the end) and P2 points at an element of the same array with index J (or one past the end), then
P1-P2 has the value equal to I-J and the type ptrdiff_t (which is a signed integer type, typically half as large as the size of the largest object that can be declared)
The behavior is defined only if the result fits in ptrdiff_t.
Although, we know (assuming the parameter i of the function really points to a member of struct S), that the pointer is pointing to a valid region in memory (begin of struct S), the compiler might not see it that way. But i am not a language lawyer and since the comments above mentioned the usage in the linux kernel, i will and cannot come to a final assessment.
This question is a follow-up to: Is adding to a "char *" pointer UB, when it doesn't actually point to a char array?
In CWG 1314, CWG affirmed that it is legal to perform pointer arithmetic within a standard-layout object using an unsigned char pointer. This would appear to imply that some code similar to that in the linked question should work as intended:
struct Foo {
float x, y, z;
};
Foo f;
unsigned char *p = reinterpret_cast<unsigned char*>(&f) + offsetof(Foo, z); // (*)
*reinterpret_cast<float*>(p) = 42.0f;
(I have replaced char with unsigned char for greater clarity.)
However, it seems that the new changes in C++17 imply that this code is now UB unless std::launder is used after both reinterpret_casts. The result of a reinterpret_cast between two pointer types is equivalent to two static_casts: the first to cv void*, the second to the destination pointer type. But [expr.static.cast]/13 implies that this produces a pointer to the original object, not to an object of the destination type, since an object of type Foo is not pointer-interconvertible with an unsigned char object at its first byte, nor is an unsigned char object at the first byte of f.z pointer-interconvertible with f.z itself.
I find it hard to believe that the committee intended a change that would break this very common idiom, making all pre-C++17 usages of offsetof undefined.
You question was:
Do we need to use std::launder when doing pointer arithmetic within a
standard-layout object (e.g., with offsetof)?
No.
std::launder won't change anything in this case and therefore has nothing to do with the presented example (imo edit launder out of the question or ask another question).
std::launder is usually just needed in a subset of cases (eg. due to a const member) where you change (or create) an underlying object in some runtime manner (eg. via placement new). Mnemonic: the object is 'dirty' and needs to be std::launder'ed.
Using only a standard layout type cannot result in a situation where you would ever need to use std::launder.
The current draft standard (and presumably C++17) say in [basic.compound/4]:
[ Note: An array object and its first element are not pointer-interconvertible, even though they have the same address. — end note ]
So a pointer to an object cannot be reinterpret_cast'd to get its enclosing array pointer.
Now, there is std::launder, [ptr.launder/1]:
template<class T> [[nodiscard]] constexpr T* launder(T* p) noexcept;
Requires: p represents the address A of a byte in memory. An object X that is within its lifetime and whose type is similar to T is located at the address A. All bytes of storage that would be reachable through the result are reachable through p (see below).
And the definion of reachable is in [ptr.launder/3]:
Remarks: An invocation of this function may be used in a core constant expression whenever the value of its argument may be used in a core constant expression. A byte of storage is reachable through a pointer value that points to an object Y if it is within the storage occupied by Y, an object that is pointer-interconvertible with Y, or the immediately-enclosing array object if Y is an array element. The program is ill-formed if T is a function type or cv void.
Now, at first sight, it seems that std::launder is can be used to do the aforementioned conversion, because of the part I've put emphasis.
But. If p points to an object of an array, the bytes of the array is reachable according to this definition (even though p is not pointer-interconvertible to array-pointer), just like the result of the launder. So, it seems that the definition doesn't say anything about this issue.
So, can std::launder be used to convert an object pointer to its enclosing array pointer?
This depends on whether the enclosing array object is a complete object, and if not, whether you can validly access more bytes through a pointer to that enclosing array object (e.g., because it's an array element itself, or pointer-interconvertible with a larger object, or pointer-interconvertible with an object that's an array element). The "reachable" requirement means that you cannot use launder to obtain a pointer that would allow you to access more bytes than the source pointer value allows, on pain of undefined behavior. This ensures that the possibility that some unknown code may call launder does not affect the compiler's escape analysis.
I suppose some examples could help. Each example below reinterpret_casts a int* pointing to the first element of an array of 10 ints into a int(*)[10]. Since they are not pointer-interconvertible, the reinterpret_cast does not change the pointer value, and you get a int(*)[10] with the value of "pointer to the first element of (whatever the array is)". Each example then attempts to obtain a pointer to the entire array by calling std::launder on the cast pointer.
int x[10];
auto p = std::launder(reinterpret_cast<int(*)[10]>(&x[0]));
This is OK; you can access all elements of x through the source pointer, and the result of the launder doesn't allow you to access anything else.
int x2[2][10];
auto p2 = std::launder(reinterpret_cast<int(*)[10]>(&x2[0][0]));
This is undefined. You can only access elements of x2[0] through the source pointer, but the result (which would be a pointer to x2[0]) would have allowed you to access x2[1], which you can't through the source.
struct X { int a[10]; } x3, x4[2]; // assume no padding
auto p3 = std::launder(reinterpret_cast<int(*)[10]>(&x3.a[0])); // OK
This is OK. Again, you can't access through a pointer to x3.a any byte you can't access already.
auto p4 = std::launder(reinterpret_cast<int(*)[10]>(&x4[0].a[0]));
This is (intended to be) undefined. You would have been able to reach x4[1] from the result because x4[0].a is pointer-interconvertible with x4[0], so a pointer to the former can be reinterpret_cast to yield a pointer to the latter, which then can be used for pointer arithmetic. See https://wg21.link/LWG2859.
struct Y { int a[10]; double y; } x5;
auto p3 = std::launder(reinterpret_cast<int(*)[10]>(&x5.a[0]));
And this is again undefined, because you would have been able to reach x5.y from the resulting pointer (by reinterpret_cast to a Y*) but the source pointer can't be used to access it.
Remark: any non schizophrenic compiler will probably gladly accept that, as it would accept a C-style cast or a re-interpret cast, so just try and see is not an option.
But IMHO, the answer to your question is no. The emphasized immediately-enclosing array object if Y is an array element lies in a Remark paragraph, not in the Requires one. That means that provided the requires section is respected, the remarks one also applies. As an array and its element type are not similar types, the requirement is not satisfied and std::launder cannot be used.
What follows is more of a general (philosophycal?) interpretation. At the time of K&R C (in the 70's), C was intended to be able to replace assembly language. For that reason the rule was: the compiler must obey the programmer provided the source code can be translated. So no strict aliasing rule and a pointer was no more that an address with additional arithmetics rules. This strongly changed in C99 and C++03 (not speaking of C++11 +). Programmers are now supposed to use C++ as a high level language. That means that a pointer is just an object that allows to access another object of a given type, and an array and its element type are totally different types. Memory addresses are now little more than implementation details. So trying to convert a pointer to an array to a pointer to its first element is then against the philosophy of the language and could bite the programmer in a later version of the compiler. Of course real life compiler still accept it for compatibility reasons, but we should not even try to use it in modern programs.
Now we know that doing out-of-bounds-pointer-arithmetic has undefined behavior as described in this SO question.
My question is: can we workaround such restriction by casting to std::uintptr_t for arithmetic operations and then cast back to pointer? is that guaranteed to work?
For example:
char a[5];
auto u = reinterpret_cast<std::uintptr_t>(a) - 1;
auto p = reinterpret_cast<char*>(u + 1); // OK?
The real world usage is for optimizing offsetted memory access -- instead of p[n + offset], I want to do offset_p[n].
EDIT To make the question more explicit:
Given a base pointer p of a char array, if p + n is a valid pointer, will reinterpret_cast<char*>(reinterpret_cast<std::uintptr_t>(p) + n) be guaranteed to yield the same valid pointer?
No, uintptr_t cannot be meaningfully used to avoid undefined behavior when performing pointer arithmetic.
For one thing, at least in C there is no guarantee that uintptr_t even exists. The requirement is that any value of type void* may be converted to uintptr_t and back again, yielding the original value without loss of information. In principle, there might not be any unsigned integer type wide enough to hold all pointer values. (I presume the same applies to C++, since C++ inherits most of the C standard library and defines it by reference to the C standard.)
Even if uintptr_t does exist, there is no guarantee that a given arithmetic operation on a uintptr_t value does the same thing as the corresponding operation on a pointer value.
For example, I've worked on systems (Cray vector systems, T90 and SV1) on which byte pointers are implemented in software. A native address is a 64-bit address that refers to a 64-bit word; there is no hardware support for byte addressing. A char* or void* pointer consists of a word pointer with a 3-bit offset stored in the otherwise unused high-order bits. Conversion between integers and pointers simply copies the bits. So incrementing a char* would advance it to point to the next 8-bit byte in memory; incrementing a uintptr_t obtained by converting a char* would advance it to point to the next 64-bit word.
That's just one example. More generally, conversions between pointers and integers are implementation-defined, and the language standard makes no guarantee about the semantics of those conversions (other than, in some cases, converting back to a pointer).
So yes, you can convert a pointer value to uintptr_t (if that type exists) and perform arithmetic on it without risking undefined behavior -- but the result may or may not be meaningful.
It happens that, on most systems, the mapping between pointers and integers is simpler, and you probably can get away with that kind of game. But you're better off using pointer arithmetic directly, and just being very careful to avoid any invalid operations.
Yes, that is legal, but you must reinterpret_cast exactly the same uintptr_t value back to char*.
(Therefore, what it you're intending to do is illegal; that is, converting a different value back to a pointer.)
5.2.10 Reinterpret cast
4 . A pointer can be explicitly converted to any integral type large enough to hold it. The mapping function is
implementation-defined.
5 . A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted
to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type
will have its original value;
(Note that there'd be no way, in general, for the compiler to know that you subtracted one and then added it back.)
I had a discussion with someone on IRC and this question turned up. We are allowed by the Standard to change an object of type int by a char lvalue.
int a;
char *b = (char*) &a;
*b = 0;
Would we be allowed to do this in the opposite direction, if we know that the alignment is fine?
The issue I'm seeing is that the aliasing rule does not cover the simple case of the following, if one considers the aliasing rule as a non-symmetric relation
int a;
a = 0;
The reason is, that each object contains a sequence of sizeof(obj) unsigned char objects (called the "object representation"). If we change the int, we will change some or all of those objects. However, the aliasing rule only states we are allowed to change a int by an char or unsigned char, but not the other way around. Another example
int a[1];
int *ra = a;
*ra = 0;
Only one direction is described by 3.10/15 ("An aggregate or union type that includes..."), but this time we need the other way around ("A type that is the element or non-static data member type of an aggregate...").
Is the other direction implied? This question also applies to C.
The aliasing rule simply states that there's one "effective type" (C99 6.5.7, plus footnote 73) for any given object in memory, and any accesses to such an object go through one of:
A type compatible with the effective type (qualifiers such as const and restrict, as well signed/unsigned-ness may vary)
A struct or union containing one such type
A character type
The effective type isn't specified in advanced, of course - it's just a construct that is used to specify aliasing. But the intent is simply that you don't access the same object with two different non-character types.
So the answer is, yes, you can indeed go the other direction.
The standard (C99 6.3.2.3 §7) defines pointer casts like this as "just fine", the casted pointer will point at the same address. (Unless the CPU has alignment which makes the cast impossible, then it is undefined behavior.)
That is, the actual cast in itself is fine. What will happen if you start to manipulate the data... now that's another implementation-defined story.
Here's from the standard:
"A pointer to an object or incomplete type may be converted to a pointer to a different object or incomplete type. If the resulting pointer is not correctly aligned (57) for the pointed-to type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer.
When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers
to the remaining bytes of the object."
"57) In general, the concept ‘‘correctly aligned’’ is transitive: if a pointer to type A is correctly aligned for a pointer to type B, which in turn is correctly aligned for a pointer to type C, then a pointer to type A is
correctly aligned for a pointer to type C."
I think I'm a bit confused by the question, but the 2nd and 3rd examples are accessing the int through an lvalue that has the object's type (int in the examples).
C++ 3.10/15 states as it's first item that it's OK to access the object through an lvalue that has the the type of "the dynamic type of the object".
What am I misunderstanding in the question?