Given..:
struct S {
float f;
int i;
};
.. can you convert from an int * pointing to an i field to a struct S *?
I am interested in whether there is a spec-conforming way to do this conversion for either/both C and C++. And, if not, I am also in interested in practical implementation specific conversions for clang / gcc / MSVC.
For example, would ..:
void f(int *i) {
struct S *s = (struct S *) (((char *) i) - offsetof(S, i));
printf("%f\n", s->f);
}
.. be correct?
.. can you convert from an int * pointing to an i field to a struct S *?
Answer: Yes, but ...
Based on
Pointer
Notes
Although any pointer to object can be cast to pointer to object of a different type, dereferencing a pointer to the type different from the declared type of the object is almost always undefined behavior. See strict aliasing for details.
and
Strict aliasing
Given an object with effective type T1, using an lvalue expression (typically, dereferencing a pointer) of a different type T2 is undefined behavior, unless:
T2 is a character type (char, signed char, or unsigned char)
and
offsetof
The macro offsetof expands to an integer constant expression of type size_t, the value of which is the offset, in bytes, from the beginning of an object of specified type to its specified subobject, including padding if any.
it should be alright, but here is the issue:
Pointer arithmetic
For the purpose of pointer arithmetic, a pointer to an object that is not an element of any array is treated as a pointer to the first element of an array of size 1.
The behavior is defined only if both the original pointer and the result pointer are pointing at elements of the same array or one past the end of that array. Note that executing p-1 when p points at the first element of an array is undefined behavior and may fail on some platforms.
If the pointer P1 points at an element of an array with index I (or one past the end) and P2 points at an element of the same array with index J (or one past the end), then
P1-P2 has the value equal to I-J and the type ptrdiff_t (which is a signed integer type, typically half as large as the size of the largest object that can be declared)
The behavior is defined only if the result fits in ptrdiff_t.
Although, we know (assuming the parameter i of the function really points to a member of struct S), that the pointer is pointing to a valid region in memory (begin of struct S), the compiler might not see it that way. But i am not a language lawyer and since the comments above mentioned the usage in the linux kernel, i will and cannot come to a final assessment.
Related
An external API expects a pointer to an array of values (int as simple example here) plus a size.
It is logically clearer to deal with the elements in groups of 4.
So process elements via a "group of 4" struct and then pass the array of those structs to the external API using a pointer cast. See code below.
Spider sense says: "strict aliasing violation" in the reinterpret_cast => possible UB?
Are the static_asserts below enough to ensure:
a) this works in practice
b) this is actually standards compliant and not UB?
Otherwise, what do I need to do, to make it "not UB". A union? How exactly please?
or, is there overall a different, better way?
#include <cstddef>
void f(int*, std::size_t) {
// external implementation
// process array
}
int main() {
static constexpr std::size_t group_size = 4;
static constexpr std::size_t number_groups = 10;
static constexpr std::size_t total_number = group_size * number_groups;
static_assert(total_number % group_size == 0);
int vals[total_number]{};
struct quad {
int val[group_size]{};
};
quad vals2[number_groups]{};
// deal with values in groups of four using member functions of `quad`
static_assert(alignof(int) == alignof(quad));
static_assert(group_size * sizeof(int) == sizeof(quad));
static_assert(sizeof(vals) == sizeof(vals2));
f(vals, total_number);
f(reinterpret_cast<int*>(vals2), total_number); /// is this UB? or OK under above asserts?
}
No, this is not permitted. The relevant C++ standard section is §7.6.1.10. From the first paragraph, we have (emphasis mine)
The result of the expression reinterpret_cast<T>(v) is the result of converting the expression v to type T.
If T is an lvalue reference type or an rvalue reference to function type, the result is an lvalue; if T is an rvalue reference to object type, the result is an xvalue; otherwise, the result is a prvalue and the lvalue-to-rvalue, array-to-pointer, and function-to-pointer standard conversions are performed on the expression v.
Conversions that can be performed explicitly using reinterpret_cast are listed below.
No other conversion can be performed explicitly using reinterpret_cast.
So unless your use case is listed on that particular page, it's not valid. Most of the sections are not relevant to your use case, but this is the one that comes closest.
An object pointer can be explicitly converted to an object pointer of a different type.[58]
When a prvalue v of object pointer type is converted to the object pointer type “pointer to cv T”, the result is static_cast<cv T*>(static_cast<cv void*>(v)).
So a reinterpret_cast from one pointer type to another is equivalent to a static_cast through an appropriately cv-qualified void*. Now, a static_cast that goes from T* to S* can be acceptably used as a S* if the types T and S are pointer-interconvertible. From §6.8.4
Two objects a and b are pointer-interconvertible if:
they are the same object, or
one is a union object and the other is a non-static data member of that object ([class.union]), or
one is a standard-layout class object and the other is the first non-static data member of that object or any base class subobject of that object ([class.mem]), or
there exists an object c such that a and c are pointer-interconvertible, and c and b are pointer-interconvertible.
If two objects are pointer-interconvertible, then they have the same address, and it is possible to obtain a pointer to one from a pointer to the other via a reinterpret_cast ([expr.reinterpret.cast]).
[Note 4: An array object and its first element are not pointer-interconvertible, even though they have the same address.
— end note]
To summarize, you can cast a pointer to a class C to a pointer to its first member (and back) if there's no vtable to stop you. You can cast a pointer to C into another pointer to C (that can come up if you're adding cv-qualifiers; for instance, reinterpret_cast<const C*>(my_c_ptr) is valid if my_c_ptr is C*). There are also some special rules for unions, which don't apply here. However, you can't factor through arrays, as per Note 4. The conversion you want here is quad[] -> quad -> int -> int[], and you can't convert between the quad[] and the quad. If quad was a simple struct that contained only an int, then you could reinterpret a quad* as an int*, but you can't do it through arrays, and certainly not through a nested layer of them.
None of the sections I've cited say anything about alignment. Or size. Or packing. Or padding. None of that matters. All your static_asserts are doing is slightly increasing the probability that the undefined behavior (which is still undefined) will happen to work on more compilers. But you're using a bandaid to repair a dam; it's not going to work.
No amount of static_asserts is going to make something which is categorically UB into well-defined behavior in accord with the standard. You did not create an array of ints; you created a struct containing an array of ints. So that's what you have.
It's legal to convert a pointer to a quad into a pointer to an int[group_size] (though you'll need to alter your code appropriately. Or you could just access the array directly and cast that to an int*.
Regardless of how you get a pointer to the first element, it's legal to do pointer arithmetic within that array. But the moment you try to do pointer arithmetic past the boundaries of the array within that quad object, you achieve undefined behavior. Pointer arithmetic is defined based on the existence of an array: [expr.add]/4
When an expression J that has integral type is added to or subtracted from an expression P of pointer type, the result has the type of P.
If P evaluates to a null pointer value and J evaluates to 0, the result is a null pointer value.
Otherwise, if P points to an array element i of an array object x with n elements ([dcl.array]), the expressions P + J and J + P (where J has the value j) point to the (possibly-hypothetical) array element i+j of x if 0≤i+j≤n and the expression P - J points to the (possibly-hypothetical) array element i−j of x if 0≤i−j≤n.
Otherwise, the behavior is undefined.
The pointer isn't null, so case 1 doesn't apply. The n above is group_size (because the array is the one within quad), so if the index is > group_size, then case 2 doesn't apply.
Therefore, undefined behavior will happen whenever someone tries to access the array past index 4. There is no cast that can wallpaper over that.
Otherwise, what do I need to do, to make it "not UB". A union? How exactly please?
You don't. What you're trying to do is simply not valid with respect to the C++ object model. You need an array of ints, so you must create an array of ints. You cannot treat an array of something other than ints as an array of ints (well, with minor exceptions of byte-wise arrays, but that's unhelpful to you).
The simplest valid way to process the array in groups is to just... do some nested loops:
int arr[total_number];
for(int* curr = arr; curr != std::end(arr); curr += 4)
{
//Use `curr[0]` to `curr[3]`;
//Or create a `std::span<int, 4> group(curr)`;
}
In C++, it's legal for a char, unsigned char, or std::byte pointer to alias any T pointer. Such aliased pointer is capable of accessing the object's representation in raw memory.
My question is whether it's legal and free of undefined-behavior to use a T* pointer derived from arithmetic on a char/unsigned char/std::byte pointer aliasing an original T array object sequence -- provided that the resultant pointer is still aligned correctly and within the original reachability of the aliased sequence.
As a concrete example:
// Array sequence is 2 in length
auto array = std::array<unsigned,2>{0xdead, 0xbeef};
// Derive the address of &array[1] using std::byte pointer
auto p = reinterpret_cast<std::byte*>(array.data());
auto p1 = reinterpret_cast<unsigned*>(p + sizeof(unsigned));
assert(p1 == &array[1]); // This expression should be legal since the pointer is safely derived
assert(*p1 == array[1]); // But is this legal, as per the C++ standard?
The pointer p1 should be safely derived in the above example since it doesn't exceed the reachability of array's bound. I know that in many cases in C++ it's legal to cast a pointer to a type/representation that can't actually be used/dereferenced. Is the resultant pointer here legal to dereference as far as the standard is concerned?
For the purpose of this question, assume that T may not be standard-layout type.
Note: I'm not asking about whether this will work in practice, since I have yet to see this type of code fail on compilers; but I'm curious to know whether this is, strictly-speaking, legal per the C++ standard.
Edit:
I'd rather not derail this question, but there are a few comments that are asserting that p + sizeof(unsigned) is undefined behavior as a result of [expr.add], in particular quoting both [expr.add]/4 and [expr.add]/6 due to p not actually being a byte-array.
There is, in fact, a byte sequence[1], and both [expr.add]/4 and [expr.add]/6 simply do not apply[2]
[1]: char, unsigned char, and std::byte view the object representation of an object, which is defined under [basic.types.general]/4 as being:
The object representation of an object of type T is the sequence of N unsigned char objects taken up by the object of type T, where N equals sizeof(T).
The above means that there is a formal sequence of unsigned char (which std::byte is capable of representing), which means p + sizeof(unsigned) is valid. We aren't viewing through a similar pointer, we are viewing exactly as the underlying object-representation which is legal.
[2]: [expr.add]/4 is simply referring to reachability of array objects. However, as pointed out above, there is a valid and reachable object due to the unsigned char sequence, and we do not exceed reachability -- thus this doesn't apply as a case of undefined behavior.
As far as I can tell, [expr.add]/6 primarily exists to avoid cases where an array of derived types is cast to a base pointer and attempted to be indexed. However it wouldn't apply in this case because we aren't viewing through a "similar" pointer, we are viewing an object representation that is formally defined as a sequence of unsigned chars. This isn't a similar representation, it's exact.
C++17 (expr.add/4) say:
When an expression that has integral type is added to or subtracted
from a pointer, the result has the type of the pointer operand. If the
expression P points to element x[i] of an array object x with n
elements, the expressions P + J and J + P (where J has the value j)
point to the (possibly-hypothetical) element x[i+j] if 0≤i+j≤n;
otherwise, the behavior is undefined. Likewise, the expression P - J
points to the (possibly-hypothetical) element x[i−j] if 0≤i−j≤n;
otherwise, the behavior is undefined.
struct Foo {
float x, y, z;
};
Foo f;
char *p = reinterpret_cast<char*>(&f) + offsetof(Foo, z); // (*)
*reinterpret_cast<float*>(p) = 42.0f;
Has the line marked with (*) UB? reinterpret_cast<char*>(&f) doesn't point to a char array, but to a float, so it should UB according to the cited paragraph. But, if it is UB, then offsetof's usefulness would be limited.
Is it UB? If not, why not?
The addition is intended to be valid, but I do not believe the standard manages to say so clearly enough. Quoting N4140 (roughly C++14):
3.9 Types [basic.types]
2 For any object (other than a base-class subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes (1.7) making up the object can be copied into an array
of char or unsigned char.42 [...]
42) By using, for example, the library functions (17.6.1.2) std::memcpy or std::memmove.
It says "for example" because std::memcpy and std::memmove are not the only ways in which the underlying bytes are intended to be allowed to be copied. A simple for loop which copies byte by byte manually is supposed to be valid as well.
In order for that to work, addition has to be defined for pointers to the raw bytes that make up an object, and the way definedness of expressions works, the addition's definedness cannot depend on whether the addition's result will subsequently be used to copy the bytes into an array.
Whether that means those bytes form an array already or whether this is a special exception to the general rules for the + operator that is somehow omitted in the operator description, is not clear to me (I suspect the former), but either way would make the addition you're performing in your code valid.
Any interpretation that disallows the intended usage of offsetof must be wrong:
#include <assert.h>
#include <stddef.h>
struct S { float a, b, c; };
const size_t idx_S[] = {
offsetof(struct S, a),
offsetof(struct S, b),
offsetof(struct S, c),
};
float read_S(struct S *sp, unsigned int idx)
{
assert(idx < 3);
return *(float *)(((char *)sp) + idx_S[idx]); // intended to be valid
}
However, any interpretation that allows one to step past the end of an explicitly-declared array must also be wrong:
#include <assert.h>
#include <stddef.h>
struct S { float a[2]; float b[2]; };
static_assert(offsetof(struct S, b) == sizeof(float)*2,
"padding between S.a and S.b -- should be impossible");
float read_S(struct S *sp, unsigned int idx)
{
assert(idx < 4);
return sp->a[idx]; // undefined behavior if idx >= 2,
// reading past end of array
}
And we are now on the horns of a dilemma, because the wording in both the C and C++ standards, that was intended to disallow the second case, probably also disallows the first case.
This is commonly known as the "what is an object?" problem. People, including members of the C and C++ committees, have been arguing about this and related issues since the 1990s, and there have been multiple attempts to fix the wording, and to the best of my knowledge none has succeeded (in the sense that all existing "reasonable" code is rendered definitely conforming and all existing "reasonable" optimizations are still allowed).
(Note: All of the above code is written as it would be written in C to emphasize that the same problem exists in both languages, and can be encountered without the use of any C++ constructs.)
See CWG 1314
According to 6.9 [basic.types] paragraph 4,
The object representation of an object of type T is the sequence of N unsigned char objects taken up by the object of type T, where N equals sizeof(T).
and 4.5 [intro.object] paragraph 5,
An object of trivially copyable or standard-layout type (6.9 [basic.types]) shall occupy contiguous bytes of storage.
Do these passages make pointer arithmetic (8.7 [expr.add] paragraph 5) within a standard-layout object well-defined (e.g., for writing one's own version of memcpy?
Rationale (August, 2011):
The current wording is sufficiently clear that this usage is permitted.
I strongly disagree with CWG's statement that "the current wording is sufficiently clear", but nevertheless, that's the ruling we have.
I interpret CWG's response as suggesting that a pointer to unsigned char into an object of trivially copyable or standard-layout type, for the purposes of pointer arithmetic, ought to be interpreted as a pointer to an array of unsigned char whose size equals the size of the object in question. I don't know whether they intended that it would also work using a char pointer or (as of C++17) a std::byte pointer. (Maybe if they had decided to actually clarify it instead of claiming the existing wording was clear enough, then I would know the answer.)
(A separate issue is whether std::launder is required to make the OP's code well-defined. I won't go into this here; I think it deserves a separate question.)
As far as I know, your code is valid. Aliasing an object as a char array is explicitly allowed as per § 3.10 ¶ 10.8:
If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined:
[…]
a char or unsigned char type.
The other question is whether casting the char* pointer back to float* and assigning through it is valid. Since your Foo is a POD type, this is okay. You are allowed to compute the address of a POD's member (given that the computation itself is not UB) and then access the member through that address. You must not abuse this to, for example, gain access to a private member of a non-POD object. Furthermore, it would be UB if you'd, say, cast to int* or write at an address where no object of type float exists. The reasoning behind this can be found in the section quoted above.
Yes, this is undefined. As you have stated in your question,
reinterpret_cast<char*>(&f) doesn't point to a char array, but to a float, ...
... reinterpret_cast<char*>(&f) does even not point to a char, so even if the object representation is a char array, the behavior is still undefined.
For offsetof, you can still use it like
struct Foo {
float x, y, z;
};
Foo f;
auto p = reinterpret_cast<std::uintptr_t>(&f) + offsetof(Foo, z);
// ^^^^^^^^^^^^^^
*reinterpret_cast<float*>(p) = 42.0f;
The size of pointer is same irrespective of datatype it is pointing to.Then why do we need to declare the datatype it points to?
For example,
int *p; //p will point to an integer data type.
char *p; //p will point to a character
Then,why can't we generalize a pointer declaration like this
pointer p; //where p is an object of pointer class.
TL;DR because, different data types occupy different size in memory and has different alignment requirement.
To elaborate, a pointer variable holds an address which points to some type of data. Without the type associated, there would be no way to dereference the pointer and get the value.
In other words, to access the data pointed to by a pointer, the associated data type must be known.
The size of the pointer itself, has little connection to the actual data it points to.
There exists a pointer , void * which is considered a generic pointer, but then, you can't dereference it, as the result of dereference would attempt to produce an incomplete type. You need to cast it to a complete type to be able to dereference or apply pointer arithmetic on a void pointer.
The reason behind considering void * a generic pointer is as below, quoting from the C11 standard, chapter §6.3.2.3
A pointer to void may be converted to or from a pointer to any object type. A pointer to
any object type may be converted to a pointer to void and back again; the result shall
compare equal to the original pointer.
So, a void * can be used as a generic container which can hold any pointer type, but to make some operation on the pointer (which includes the knowledge of the data type), you need to cast it to a complete type first.
You can generalize pointers using void*. void* is a pointer to some address without any type information associated. However, there is little you can do with these pointers without casting them to an explicit pointer type first.
Consider the following example. It will not compile, because it's impossible to deduce the "value" pointed to by ptr is. You can't even know how many bytes constitute the value.
void print(const void * ptr) {
std::cout << *ptr; // What's the "value" of ptr?
}
Pointer arithmetic.
Explanation:
int arr[] = {4, 6, 9, 10};
int* x = arr;
*(x+0) = 4
*(x+1) = 6
Compiler knows that x+1 is actually x+sizeof(int) and not 1.
To read the second element the compiler has to skip sizeof(int) each time. Without knowing the actual type you can't dereference it and extract the data correctly.
Plus types differ in size, and the compiler needs to know how many bytes to read from the memory pointed to. A char will be 1 byte and int will be more than 1 byte.
A void * pointer can be viewed as a "generalized" pointer.
It can't be dereferenced, though, because by being generic it doesn't point to any one type of object. To actually use a void * pointer and access whatever it's pointing to, you need to cast it to the type of object you're accessing.
And since it doesn't point to any one type of object, there's no proper way to perform pointer arithmetic on a void * pointer.
The size of pointer is same irrespective of data type it is pointing to.
This is not necessarily true - pointers to different types may have different sizes and representations:
A pointer to void shall have the same representation and alignment requirements as a
pointer to a character type.48) Similarly, pointers to qualified or unqualified versions of
compatible types shall have the same representation and alignment requirements. All
pointers to structure types shall have the same representation and alignment requirements
as each other. All pointers to union types shall have the same representation and
alignment requirements as each other. Pointers to other types need not have the same
representation or alignment requirements.
C 2011 Online Draft, §6.2.5 ¶28
The value representation of pointer types
is implementation-defined. Pointers to layout-compatible types shall have the same value representation and
alignment requirements (3.11).
C++ 2014 Working Draft, §3.9.2, ¶3
On most modern desktop and servers they're the same, but don't expect it to be universally true.
Then why do we need to declare the data type it is pointing to?
Pointer arithmetic depends on the pointed-to type - if p points to an object of type T, then p + 1 points to the next object of type T. Since different types have different sizes, you need to know the pointed-to type to compute the offset correctly.
Then,why can't we generalize a pointer declaration like this
pointer p; //where p is an object of pointer class.
You actually can using void*. But the consequence is that all type (and size information) is lost at that point.
You'll have to keep track of that somehow (hard-coded type casts or whatever), to make that useful.
I had a discussion with someone on IRC and this question turned up. We are allowed by the Standard to change an object of type int by a char lvalue.
int a;
char *b = (char*) &a;
*b = 0;
Would we be allowed to do this in the opposite direction, if we know that the alignment is fine?
The issue I'm seeing is that the aliasing rule does not cover the simple case of the following, if one considers the aliasing rule as a non-symmetric relation
int a;
a = 0;
The reason is, that each object contains a sequence of sizeof(obj) unsigned char objects (called the "object representation"). If we change the int, we will change some or all of those objects. However, the aliasing rule only states we are allowed to change a int by an char or unsigned char, but not the other way around. Another example
int a[1];
int *ra = a;
*ra = 0;
Only one direction is described by 3.10/15 ("An aggregate or union type that includes..."), but this time we need the other way around ("A type that is the element or non-static data member type of an aggregate...").
Is the other direction implied? This question also applies to C.
The aliasing rule simply states that there's one "effective type" (C99 6.5.7, plus footnote 73) for any given object in memory, and any accesses to such an object go through one of:
A type compatible with the effective type (qualifiers such as const and restrict, as well signed/unsigned-ness may vary)
A struct or union containing one such type
A character type
The effective type isn't specified in advanced, of course - it's just a construct that is used to specify aliasing. But the intent is simply that you don't access the same object with two different non-character types.
So the answer is, yes, you can indeed go the other direction.
The standard (C99 6.3.2.3 §7) defines pointer casts like this as "just fine", the casted pointer will point at the same address. (Unless the CPU has alignment which makes the cast impossible, then it is undefined behavior.)
That is, the actual cast in itself is fine. What will happen if you start to manipulate the data... now that's another implementation-defined story.
Here's from the standard:
"A pointer to an object or incomplete type may be converted to a pointer to a different object or incomplete type. If the resulting pointer is not correctly aligned (57) for the pointed-to type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer.
When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers
to the remaining bytes of the object."
"57) In general, the concept ‘‘correctly aligned’’ is transitive: if a pointer to type A is correctly aligned for a pointer to type B, which in turn is correctly aligned for a pointer to type C, then a pointer to type A is
correctly aligned for a pointer to type C."
I think I'm a bit confused by the question, but the 2nd and 3rd examples are accessing the int through an lvalue that has the object's type (int in the examples).
C++ 3.10/15 states as it's first item that it's OK to access the object through an lvalue that has the the type of "the dynamic type of the object".
What am I misunderstanding in the question?