I recently learned that it is Undefined Behavior to reinterpret a POD as a different POD by reinterpret_casting its address. So I'm just wondering what a potential use-case of reinterpret_cast might be, if it can't be used for what its name suggests?
There are two situations in which I’ve used reinterpret_cast:
To cast to and from char* for the purpose of serialisation or when talking to a legacy API. In this case the cast from char* to an object pointer is still strictly speaking UB (even though done very frequently). And you don’t actually need reinterpret_cast here — you could use memcpy instead, but the cast might under specific circumstances avoid a copy (but in situations where reinterpreting the bytes is valid in the first place, memcpy usually doesn’t generate any redundant copies either, the compiler is smart enough for that).
To cast pointers from/to std::uintptr_t to serialise them across a legacy API or to perform some non-pointer arithmetic on them. This is definitely an odd beast and doesn’t happen frequently (even in low-level code) but consider the situation where one wants to exploit the fact that pointers on a given platform don’t use the most significant bits, and these bits can thus be used to store some bit flags. Garbage collector implementations occasionally do this. The lower bits of a pointer can sometimes also be used, if the programmer knows that the pointer will always be aligned e.g. at an 8 byte boundary (so the lowest three bits must be 0).
But to be honest I can’t remember the last concrete, legitimate situation where I’ve actually used reinterpret_cast. It’s definitely many years ago.
Conforming implementations of C and C++ are allowed to extend the semantics of C or C++ by behaving meaningfully even in cases where the Standards would not require them to do so. Implementations that do so will may be more suitable for a wider range of tasks than implementations that do not. In many cases, it is useful to have consistent syntax to specify constructs which will be processed meaningfully and consistently by implementations that are designed to be suitable for low-level programming tasks, even if implementations which are not designed to be suitable for such purposes would process them nonsensically.
One very frequent use case is when you're working with C library functions that take an opaque void * that gets forwarded to a callback function. Using reinterpret_cast on both sides of the fence, so to speak, keeps everything proper.
Related
Take the following code:
char chars[4] = {0x5B, 0x5B, 0x5B, 0x5B};
int* b = (int*) &chars[0];
The (int*) &chars[0] value is going to be used in a loop (a long loop). Is there any advantage in using (int*) &chars[0] over b in my code? Is there any overhead in creating b? Since I only want to use it as an alias and improve code readability.
Also, is it OK to do this kind of type casting as long as I know what I'm doing? Or should I always memcpy() to another array with the correct type and use that? Would I encounter any kind of undefined behavior? because in my testing so far, it works, but I've seen people discouraging this kind of type casting.
is it OK to do this kind of type casting as long as I know what I'm doing?
No, this is not OK. This is not safe. The C++ standard does not allow that. You can access to an object representation (ie. casting an object pointer to char*) although the result is dependent of the target platform (due to the endianess and padding). However, you cannot do the opposite safely (ie. without an undefined behaviour).
More specifically, the int type can have different alignment requirements (typically aligned to 4 or 8 bytes) than char (not aligned). Thus, your array is likely not aligned and the cast cause an undefined behaviour when b will be dereferenced. Note that it can cause a crash on some processors (AFAIK, POWER for example) although mainstream x86-64 processors supports that. Moreover, compilers can assume that b is aligned in memory (ot alignof(int)).
Or should I always memcpy() to another array with the correct type and use that?
Yes, or alternative C++ operations like the new std::bit_cast available since C++20. Do not worry about performance: most compilers (GCC, Clang, ICC, and certainly MSVC) does optimize such operations (called type punning).
Would I encounter any kind of undefined behavior?
As said before, yes, as long as the type punning is not done correctly. For more information about this you can read the following links:
What is the Strict Aliasing Rule and Why do we care?
reinterpret_cast conversion
Objects and alignment
because in my testing so far, it works, but I've seen people discouraging this kind of type casting.
It often works on simple examples on x86-64 processors. However, when you are dealing with a big code, compilers does perform silly optimizations (but totally correct ones regarding the C++ standard). To quote cppreference: "Compilers are not required to diagnose undefined behaviour (although many simple situations are diagnosed), and the compiled program is not required to do anything meaningful.". Such issue are very hard to debug as they generally only appear when optimizations are enabled and in some specific cases. The result of a program can change regarding the inlining of functions which is dependent of compiler heuristics. In your case, this is dependent of the alignment of the stack which is dependent of compiler optimizations and declared/used variables in the current scope. Some processors does not support unaligned accesses (eg. accesses that cross a cache line boundary) which resulting in an hardware exception of data corruption.
So put it shortly, "it works" so far does not means it will always work everywhere anytime.
The (int*) &chars[0] value is going to be used in a loop (a long loop). Is there any advantage in using (int*) &chars[0] over b in my code? Is there any overhead in creating b? Since I only want to use it as an alias and improve code readability.
Assuming you use a correct way to do type punning (eg. memcpy), then optimizing compilers can fully optimize this initialization as long as optimization flags are enabled. You should not worry about that unless you find that the generated code is poorly optimized. Correctness matters more than performance.
AFAIK, a C compiler does not insert any code when casting a pointer - which means that both chars and b are just memory addresses. Normally a C++ compiler should compile this in the same way as a C compiler - this the reason C++ has different, more advanced, casting semantics.
But you can always compile this and then disassemble it in gdb to see for yourself.
Otherwise, as long as you are aware of the endianness problems or potentially different int sizes on exotic platforms, your casting is safe.
See this question also: In C, does casting a pointer have overhead?
If code performs multiple discrete byte operations using a pointer derived from a pointer to a type which requires word alignment, clang will sometimes replace the discrete writes with a word write that would succeed if the original object was aligned for that word type, but would fail on systems that don't support unaligned accesses if the object isn't aligned the way the compiler expects.
Among other things, this means that if one casts a pointer to T into a pointer to a union containing T, code which attempts to use the union pointer to access the original type may fail if the union contains any types that require an alignment stricter than the original type, even if the union is only accessed via the member of the original type.
The accepted answer to What is the strict aliasing rule? mentions that you can use char * to alias another type but not the other way.
It doesn't make sense to me — if we have two pointers, one of type char * and another of type struct something * pointing to the same location, how is it possible that the first aliases the second but the second doesn't alias the first?
if we have two pointers, one of type char * and another of type struct something * pointing to the same location, how is it possible that the first aliases the second but the second doesn't alias the first?
It does, but that's not the point.
The point is that if you have one or more struct somethings then you may use a char* to read their constituent bytes, but if you have one or more chars then you may not use a struct something* to read them.
The wording in the referenced answer is slightly erroneous, so let’s get that ironed out first:
One object never aliases another object, but two pointers can “alias” the same object (meaning, the pointers point to the same memory location — as M.M. pointed out, this is still not 100% correct wording but you get the idea). Also, the standard itself doesn’t (to the best of my knowledge) actually talk about strict aliasing at all, but merely lays out rules that govern through which kinds of expressions an object may be accessed or not. Compiler flags like -fno-strict-aliasing tell the compiler whether it can assume the programmer followed those rules (so it can perform optimizations based on that assumption) or not.
Now to your question: Any object can be accessed through a pointer to char, but a char object (especially a char array) may not be accessed through most other pointer types.
Based on that, the compiler is required to make the following assumptions:
If the type of the actual object itself is not known, both char* and T* pointers may always point to the same object (alias each other) — symmetric relationship.
If types T1 and T2 are not “related” and neither is char, then T1* and T2* may never point to the same object — symmetric relationship.
A char* pointer may point to a char object or an object of any type T.
A T* pointer may not point to a char object — asymmetric relationship.
I believe, the main rationale behind the asymmetric rules about accessing object through pointers is that a char array might not satisfy the alignment requirements of, e.g., an int.
So, even without compiler optimizations based on the strict aliasing rule, writing an int to the location of a 4-byte char array at addresses 0x1, 0x2, 0x3, 0x4, for instance, will — in the best case — result in poor performance and — in the worst case — access a different memory location, because the CPU instructions might ignore the lowest two address bits when writing a 4-byte value (so here this might result in a write to 0x0, 0x1, 0x2, and 0x3).
Please also be aware that the meaning of “related” differs between C and C++, but that is not relevant to your question.
if we have two pointers, one of type char * and another of type struct something * pointing to the same location, how is it possible that the first aliases the second but the second doesn't alias the first?
Pointers don't alias each other; that's sloppy use of language. Aliasing is when an lvalue is used to access an object of a different type. (Dereferencing a pointer gives an lvalue).
In your example, what's important is the type of the object being aliased. For a concrete example let's say that the object is a double. Accessing the double by dereferencing a char * pointing at the double is fine because the strict aliasing rule permits this. However, accessing a double by dereferencing a struct something * is not permitted (unless, arguably, the struct starts with double!).
If the compiler is looking at a function which takes char * and struct something *, and it does not have available the information about the object being pointed to (this is actually unlikely as aliasing passes are done at a whole-program optimization stage); then it would have to allow for the possibility that the object might actually be a struct something *, so no optimization could be done inside this function.
Many aspects of the C++ Standard are derived from the C Standard, which needs to be understood in the historical context when it was written. If the C Standard were being written to describe a new language which included type-based aliasing, rather than describing an existing language which was designed around the idea that accesses to lvalues were accesses to bit patterns stored in memory, there would be no reason to give any kind of privileged status to the type used for storing characters in a string. Having explicit operations to treat regions of storage as bit patterns would allow optimizations to be simultaneously more effective and safer. Had the C Standard been written in such fashion, the C++ Standard presumably would have been likewise.
As it is, however, the Standard was written to describe a language in which a very common idiom was to copy the values of objects by copying all of the bytes thereof, and the authors of the Standard wanted to allow such constructs to be usable within portable programs.
Further, the authors of the Standard intended that implementations process many non-portable constructs "in a documented manner characteristic of the environment" in cases where doing so would be useful, but waived jurisdiction over when that should happen, since compiler writers were expected to understand their customers' and prospective customers' needs far better than the Committee ever could.
Suppose that in one compilation unit, one has the function:
void copy_thing(char *dest, char *src, int size)
{
while(size--)
*(char volatile *)(dest++) = *(char volatile*)(src++);
}
and in another compilation unit:
float f1,f2;
float test(void)
{
f1 = 1.0f;
f2 = 2.0f;
copy_thing((char*)&f2, (char*)&f1, sizeof f1);
return f2;
}
I think there would have been a consensus among Committee members that no quality implementation should treat the fact that copy_thing never writes to an object of type float as an invitation to assume that the return value will always be 2.0f. There are many things about the above code that should prevent or discourage an implementation from consolidating the read of f2 with the preceding write, with or without a special rule regarding character types, but different implementations would have different reasons for their forfearance.
It would be difficult to describe a set of rules which would require that all implementations process the above code correctly without blocking some existing or plausible implementations from implementing what would otherwise be useful optimizations. An implementation that treated all inter-module calls as opaque would handle such code correctly even if it was oblivious to the fact that a cast from T1 to T2 is a sign that an access to a T2 may affect a T1, or the fact that a volatile access might affect other objects in ways a compiler shouldn't expect to understand. An implementation that performed cross-module in-lining and was oblivious to the implications of typecasts or volatile would process such code correctly if it refrained from making any aliasing assumptions about accesses via character pointers.
The Committee wanted to recognize something in the above construct that compilers would be required to recognize as implying that f2 might be modified, since the alternative would be to view such a construct as Undefined Behavior despite the fact that it should be usable within portable programs. The fact that they chose the fact that the access was made via character pointer was the aspect that forced the issue was never intended to imply that compilers be oblivious to everything else, even though unfortunately some compiler writers interpret the Standard as an invitation to do just that.
According to the C standard:
When two pointers are subtracted, both shall point to elements of the
same array object, or one past the last element of the array object
(sect. 6.5.6 1173)
[Note: do not assume that I know much of the standard or UB, I just happen to have found out this one]
I understand that in almost all cases, taking the difference of pointers in two different arrays would be a bad idea anyway.
I also know that on some architectures ("segmented machine" as I read somewhere), there are good reasons that the behavior is undefined.
Now on the other hand
It may useful in some corner cases. For example, in this post, it would allow to use a library interface with different arrays, instead of copying everything in one array that will be split just after.
It seems that on "ordinary" architectures, the way of thinking "all objects are stored in a big array starting at approx. 0 and ending at approx. memory size" is a reasonable description of the memory. When you actually do look at pointer differences of different arrays, you get sensible results.
Hence my question: from experiment, it seems that on some architectures (e.g. x86-64), pointer difference between two arrays provides sensible, reproducible results. And it seems to correspond reasonably well to the hardware of these architectures. So does some implementation actually insure a specific behavior?
For example, is there an implementation out there in the wild that guarantees for a and b being char*, we have a + (reinterpret_cast<std::ptrdiff_t>(b)-reinterpret_cast<std::ptrdiff_t>(a)) == b?
Why make it UB, and not implementation-defined? (where of course, for some architectures, implementation-defined will specify it as UB)
That is not how it works.
If something is documented as "implementation-defined" by the standard, then any conforming implementation is expected to define a behavior for that case, and document it. Leaving it undefined is not an option.
As labeling pointer difference between unrelated arrays "implementation defined" would leave e.g. segmented or Harvard architectures with no way to have a fully-conforming implementation, this case remains undefined by the standard.
Implementations could offer a defined behavior as a non-standard extension. But any program making use of such an extension would no longer be strictly conforming, and non-portable.
Any implementation is free to document a behaviour for which the standard does not require behaviour to be documented - it is well within the limits of the standard. The problem with implementation-defined behaviour in this case is that the implementations must then carefully document them, and when C was standardized, the committee presumably found out that the different implementations were so wildly variable, that no sensible common ground would exist, so they decided to make it UB altogether.
I do not know any compilers that do make it defined, but I know a compiler which does explicitly keep it undefined, even if you try to cheat with casts:
When casting from pointer to integer and back again, the resulting pointer must reference the same object as the original pointer, otherwise the behavior is undefined. That is, one may not use integer arithmetic to avoid the undefined behavior of pointer arithmetic as proscribed in C99 and C11 6.5.6/8.
I believe another compiler also has the same behaviour, though, unfortunately it doesn't document it in an accessible way.
That those two compilers do not define it would be a good reason to avoid depending on it in any programs, even if compiled with another compiler that would specify a behaviour, because you can never be too sure what compiler you need to use 5 years from now...
The more implementation-defined behavior you have and someone's code depends on, the less portable that code is. In this case, there's already an implementation-defined way out of this: reinterpret_cast the pointers to integers and do your math there. That makes it clear to everyone that you're relying on behavior specific to the implementation (or at least, behavior that may not be portable everywhere).
Plus, while the runtime environment may in fact be "all objects are stored in a big array starting at approx. 0 and ending at approx. memory size," that is not true of the compile-time behavior. At compile-time, you can get pointers to objects and do pointer arithmetic on them. But treating such pointers as just addresses into memory could allow a user to start indexing into compiler data and such. By making such things UB, it makes it expressly forbidden at compile-time (and reinterpret_cast is explicitly disallowed at compile-time).
One big reason for saying that things are UB is to allow the compiler to perform optimizations. If you want to allow such a thing, then you remove some optimizations. And as you say, this is only (if even then) useful in some small corner cases. I would say that in most cases where this might seem like a viable option, you should instead reconsider your design.
From comments below:
I agree but the problem it that while I can reconsider my design, I can't reconsider the design of other libraries..
It is very rare that the standard adopts to such things. It has happened however. That's the reason why int *p = 0 is perfectly valid, even though p is a pointer and 0 is an int. This made it in the standard because it was so commonly used instead of the more correct int *p = NULL. But in general, this does not happen, and for good reasons.
First, I feel like we need to get some terms straight, at least with respect to C.
From the C2011 online draft:
Undefined behavior - behavior, upon use of a nonportable or erroneous program construct or of erroneous data,
for which this International Standard imposes no requirements. Possible undefined behavior ranges from ignoring the situation completely with unpredictable
results, to behaving during translation or program execution in a documented manner characteristic of the
environment (with or without the issuance of a diagnostic message), to terminating a translation or
execution (with the issuance of a diagnostic message).
Unspecified behavior - use of an unspecified value, or other behavior where this International Standard provides
two or more possibilities and imposes no further requirements on which is chosen in any
instance. An example of unspecified behavior is the order in which the arguments to a function are
evaluated.
Implementation-defined behavior - unspecified behavior where each implementation documents how the choice is made. An example of implementation-defined behavior is the propagation of the high-order bit
when a signed integer is shifted right.
The key point above is that unspecified behavior means that the language definition provides multiple values or behaviors from which the implementation may choose, and there are no further requirements on how that choice is made. Unspecified behavior becomes implementation-defined behavior when the implementation documents how it makes that choice.
This means that there are restrictions on what may be considered implementation-defined behavior.
The other key point is that undefined does not mean illegal, it only means unpredictable. It means you've voided the warranty, and anything that happens afterwards is not the responsibility of the compiler implementation. One possible outcome of undefined behavior is to work exactly as expected with no nasty side effects. Which, frankly, is the worst possible outcome, because it means as soon as something in the code or environment changes, everything could blow up and you have no idea why (been in that movie a few times).
Now to the question at hand:
I also know that on some architectures ("segmented machine" as I read somewhere), there are good reasons that the behavior is undefined.
And that's why it's undefined everywhere. There are some architectures still in use where different objects can be stored in different memory segments, and any differences in their addresses would be meaningless. There are just so many different memory models and addressing schemes that you cannot hope to define a behavior that works consistently for all of them (or the definition would be so complicated that it would be difficult to implement).
The philosophy behind C is to be maximally portable to as many architectures as possible, and to do that it imposes as few requirements on the implementation as possible. This is why the standard arithmetic types (int, float, etc.) are defined by the minimum range of values that they can represent with a minimum precision, not by the number of bits they take up. It's why pointers to different types may have different sizes and alignments.
Adding language that would make some behaviors undefined on this list of architectures vs. unspecified on that list of architectures would be a headache, both for the standards committee and various compiler implementors. It would mean adding a lot of special-case logic to compilers like gcc, which could make it less reliable as a compiler.
As far as I know, when two pointers (or references) do not type alias each other, it is legal to for the compiler to make the assumption that they address different locations and to make certain optimizations thereof, e.g., reordering instructions. Therefore, having pointers to different types to have the same value may be problematic. However, I think this issue only applies when the two pointers are passed to functions. Within the function body where the two pointers are created, the compiler should be able to make sure the relationship between them as to whether they address the same location. Am I right?
As far as I know, when two pointers (or references) do not type alias
each other, it is legal to for the compiler to make the assumption
that they address different locations and to make certain
optimizations thereof, e.g., reordering instructions.
Correct. GCC, for example, does perform optimizations of this form which can be disabled by passing the flag -fno-strict-aliasing.
However, I think this issue only applies when the two pointers are
passed to functions. Within the function body where the two pointers
are created, the compiler should be able to make sure the relationship
between them as to whether they address the same location. Am I right?
The standard doesn't distinguish between where those pointers came from. If your operation has undefined behavior, the program has undefined behavior, period. The compiler is in no way obliged to analyze the operands at compile time, but he may give you a warning.
Implementations which are designed and intended to be suitable for low-level programming should have no particular difficulty recognizing common patterns where storage of one type is reused or reinterpreted as another in situations not involving aliasing, provided that:
Within any particular function or loop, all pointers or lvalues used to access a particular piece of storage are derived from lvalues of a common type which identify the same object or elements of the same array, and
Between the creation of a derived-type pointer and the last use of it or any pointer derived from it, all operations involving the storage are performed only using the derived pointer or other pointers derived from it.
Most low-level programming scenarios requiring reuse or reinterpretation of storage fit these criteria, and handling code that fits these criteria will typically be rather straightforward in an implementation designed for low-level programming. If an implementation cache lvalues in registers and performs loop hoisting, for example, it could support the above semantics reasonably efficiently by flushing all cached values of type T whenever T or T* is used to form a pointer or lvalue of another type. Such an approach may be optimal, but would degrade performance much less than having to block all type-based optimizations entirely.
Note that it is probably in many cases not worthwhile for even an implementation intended for low-level programming to try to handle all possible scenarios involving aliasing. Doing that would be much more expensive than handling the far more common scenarios that don't involve aliasing.
Implementations which are specialized for other purposes are, of course, not required to make any attempt whatsoever to support any exceptions to 6.5p7--not even those that are often treated as part of the Standard. Whether such an implementation should be able to support such constructs would depend upon the particular purposes for which it is designed.
My knowledge of C++ at this point is more academic than anything else. In all my reading thus far, the use of explicit conversion with named casts (const_cast, static_cast, reinterpret_cast, dynamic_cast) has come with a big warning label (and it's easy to see why) that implies that explicit conversion is symptomatic of bad design and should only be used as a last resort in desperate circumstances. So, I have to ask:
Is explicit conversion with named casts really just jury rigging code or is there a more graceful and positive application to this feature? Is there a good example of the latter?
There're cases when you just can't go without it. Like this one. The problem there is that you have multiple inheritance and need to convert this pointer to void* while ensuring that the pointer that goes into void* will still point to the right subobject of the current object. Using an explicit cast is the only way to achieve that.
There's an opinion that if you can't go without a cast you have bad design. I can't agree with this completely - different situations are possible, including one mentioned above, but perhaps if you need to use explicit casts too often you really have bad design.
There are situations when you can't really avoid explicit casts. Especially when interacting with C libraries or badly designed C++ libraries (like the COM library sharptooth used as examples).
In general, the use of explicit casts IS a red herring. It does not necessarily means bad code, but it does attract attention to a potential dangerous use.
However you should not throw the 4 casts in the same bag: static_cast and dynamic_cast are frequently used for up-casting (from Base to Derived) or for navigating between related types. Their occurrence in the code is pretty normal (indeed it's difficult to write a Visitor Pattern without either).
On the other hand, the use of const_cast and reinterpret_cast is much more dangerous.
using const_cast to try and modify a read-only object is undefined behavior (thanks to James McNellis for correction)
reinterpret_cast is normally only used to deal with raw memory (allocators)
They have their use, of course, but should not be encountered in normal code. For dealing with external or C APIs they might be necessary though.
At least that's my opinion.
How bad a cast is typically depends on the type of cast. There are legitimate uses for all of these casts, but some smell worse than others.
const_cast is used to cast away constness (since adding it doesn't require a cast). Ideally, that should never be used. It makes it easy to invoke undefined behavior (trying to change an object originally designated const), and in any case breaks the const-correctness of the program. It is sometimes necessary when interfacing with APIs that are not themselves const-correct, which may for example ask for a char * when they're going to treat it as const char *, but since you shouldn't write APIs that way it's a sign that you're using a really old API or somebody screwed up.
reinterpret_cast is always going to be platform-dependent, and is therefore at best questionable in portable code. Moreover, unless you're doing low-level operations on the physical structure of objects, it doesn't preserve meaning. In C and C++, a type is supposed to be meaningful. An int is a number that means something; an int that is basically the concatenation of chars doesn't really mean anything.
dynamic_cast is normally used for downcasting; e.g. from Base * to Derived *, with the proviso that either it works or it returns 0. This subverts OO in much the same way as a switch statement on a type tag does: it moves the code that defines what a class is away from the class definition. This couples the class definitions with other code and increases the potential maintenance load.
static_cast is used for data conversions that are known to be generally correct, such as conversions to and from void *, known safe pointer casts within the class hierarchy, that sort of thing. About the worst you can say for it is that it subverts the type system to some extent. It's likely to be needed when interfacing with C libraries, or the C part of the standard library, as void * is often used in C functions.
In general, well-designed and well-written C++ code will avoid the use cases above, in some cases because the only use of the cast is to do potentially dangerous things, and in other cases because such code tends to avoid the need for such conversions. The C++ type system is generally seen as a good thing to maintain, and casts subvert it.
IMO, like most things, they're tools, with appropriate uses and inappropriate ones. Casting is probably an area where the tools frequently get used inappropriately, for example, to cast between an int and pointer type with a reinterpret_cast (which can break on platforms where the two are different sizes), or to const_cast away constness purely as a hack, and so on.
If you know what they're for and the intended uses, there's absolutely nothing wrong with using them for what they were designed for.
There is an irony to explicit casts. The developer whose poor C++ design skills lead him to write code requiring a lot of casting is the same developer who doesn't use the explicit casting mechanisms appropriately, or at all, and litters his code with C-style casts.
On the other hand, the developer who understands their purpose, when to use them and when not to, and what the alternatives are, is not writing code that requires much casting!
Check out the more fine-grained variations on these casts, such as polymorphic_cast, in the boost conversions library, to give you an idea of just how careful C++ programmers are when it comes to casting.
Casts are a sign that you're trying to put a round peg in a square hole. Sometimes that's part of the job. But if you have some control over both the hole and the peg, it would be better not create this condition, and writing a cast should trigger you to ask yourself if there was something you could have done so this was a little smoother.