It appears from other StackOverflow questions and reading §9.5.1 of the ISO/IEC draft C++ standard standard that the use of unions to do a literal reinterpret_cast of data is undefined behavior.
Consider the code below. The goal is to take the integer value of 0xffff and literally interpret it as a series of bits in IEEE 754 floating point. (Binary convert shows visually how this is done.)
#include <iostream>
using namespace std;
union unionType {
int myInt;
float myFloat;
};
int main() {
int i = 0xffff;
unionType u;
u.myInt = i;
cout << "size of int " << sizeof(int) << endl;
cout << "size of float " << sizeof(float) << endl;
cout << "myInt " << u.myInt << endl;
cout << "myFloat " << u.myFloat << endl;
float theFloat = *reinterpret_cast<float*>(&i);
cout << "theFloat " << theFloat << endl;
return 0;
}
The output of this code, using both GCC and clang compilers is expected.
size of int 4
size of float 4
myInt 65535
myFloat 9.18341e-41
theFloat 9.18341e-41
My question is, does the standard actually preclude the value of myFloat from being deterministic? Is the use of a reinterpret_cast better in any way to perform this type of conversion?
The standard states the following in §9.5.1:
In a union, at most one of the non-static data members can be active at any time, that is, the value of at most one of the non-static data members can be stored in a union at any time. [...] The size of a union is sufficient to contain the largest of its non-static data members. Each non-static data member is allocated as if it were the sole member of a struct. All non-static data members of a union object have the same address.
The last sentence, guaranteeing that all non-static members have the same address, seems to indicate the use of a union is guaranteed to be identical to the use of a reinterpret_cast, but the earlier statement about active data members seems to preclude this guarantee.
So which construct is more correct?
Edit:
Using Intel's icpc compiler, the above code produces even more interesting results:
$ icpc union.cpp
$ ./a.out
size of int 4
size of float 4
myInt 65535
myFloat 0
theFloat 0
The reason it's undefined is because there's no guarantee what exactly the value representations of int and float are. The C++ standard doesn't say that a float is stored as an IEEE 754 single-precision floating point number. What exactly should the standard say about you treating an int object with value 0xffff as a float? It doesn't say anything other than the fact it is undefined.
Practically, however, this is the purpose of reinterpret_cast - to tell the compiler to ignore everything it knows about the types of objects and trust you that this int is actually a float. It's almost always used for machine-specific bit-level jiggery-pokery. The C++ standard just doesn't guarantee you anything once you do it. At that point, it's up to you to understand exactly what your compiler and machine do in this situation.
This is true for both the union and reinterpret_cast approaches. I suggest that reinterpret_cast is "better" for this task, since it makes the intent clearer. However, keeping your code well-defined is always the best approach.
It's not undefined behavior. It's implementation defined behavior. The first does mean that bad things can happen. The other means that what will happen has to be defined by the implementation.
The reinterpret_cast violates the strict aliasing rule. So I do not think it will work reliably. The union trick is what people call type-punning and is usually allowed by compilers. The gcc folks document the behavior of the compiler: http://gcc.gnu.org/onlinedocs/gcc/Structures-unions-enumerations-and-bit_002dfields-implementation.html#Structures-unions-enumerations-and-bit_002dfields-implementation
I think this should work with icpc as well (but they do not appear to document how they implemented that). But when I looked the assembly, it looks like icc tries to cheat with float and use higher precision floating point stuff. Passing -fp-model source to the compiler fixed that. With that option, I get the same results as with gcc.
I do not think you want to use this flag in general, this is just a test to verify my theory.
So for icpc, I think if you switch your code from int/float to long/double, type-punning will work on icpc as well.
Undefined behavior does not mean bad things must happen. It means only that the language definition doesn't tell you what happens. This kind of type pun has been part of C and C++ programming since time immemorial (i.e., since 1969); it would take a particularly perverse implementor to write a compiler where this didn't work.
Related
The template function below is part of a sequence generator. Instead of manual shifts, I came up with the following union-based solution to make the operations more explicit. It works great on all the compilers tested. Godbolt link.
However despite working in practice, I am afraid there are aliasing rules that are being violated, which means it might not work in the future or in another compiler other than GCC and CLANG.
Strictly in view of the C++ standard: is the code below well formed? Does it incur in undefined behavior?
template <int BITS>
uint64_t flog2(uint64_t num) {
constexpr uint64_t MAXNUM = (uint64_t(1) << BITS);
if (num < MAXNUM) return num;
union FP {
double dbl;
struct {
uint64_t man: 52;
uint32_t exp: 11;
uint32_t sign: 1;
};
struct {
uint64_t xman: 52-BITS;
uint32_t xexp: 11+BITS;
uint32_t xsgn: 1;
};
};
FP fp;
fp.dbl = num;
fp.exp -= 1023-1+BITS;
return fp.xexp;
}
Thanks!
First of all, the program is syntactically ill-formed in ISO standard C++. Anonymous struct members are not standard C++ (in contrast to C). They are an extension. In ISO standard C++ the struct must be named and accessed through that name.
I'll ignore that for the rest of the answer and pretend you were accessing through such a name.
It is not an aliasing violation technically, but undefined behavior for reading an inactive member of the union object in
fp.exp -= 1023-1+BITS;
The types don't really matter for this (in contrast to aliasing). There is always only at most one active member of a union, which would the last one which was either explicitly created or written with a member access/assignment expression. In your case fp.dbl = num; means that dbl is the active member and the only one that may be read from.
There is one exception in the standard for accessing the common initial sequence of standard layout class type members of a union, in which case the non-active one may be accessed as if it was the active one. But even your two struct { members have a non-empty common initial sequence only for BITS == 0.
However, in practice compilers typically explicitly support this kind of type punning, probably already for C compatibility where it is allowed.
Of course, even setting all of this aside, the layout of bitfields and the representations of the involved types are completely implementation-defined and you can't expect this to be generally portable.
It is undefined behaviour to read from a union member that was not most recently written.
Furthermore, the layout of bit-fields is implementation-defined.
Hence, from a strict C++ Standard view, this code invokes both undefined behaviour (by reading exp after writing dbl) and relies on implementation-defined behaviour in assuming that the bit-field layout corresponds to the double floating point representation (which by the way is also implementation-defined).
My question is regarding a code fragment, such as below:
#include <iostream>
int main() {
double a = -50;
std::cout << a << "\n";
uint8_t* b = reinterpret_cast<uint8_t*>(&a);
b[7] &= 0x7F;
std::cout << a << "\n";
return 0;
}
As far as I can tell I am not breaking any rules and everything is well defined (as noted below I forgot that uint8_t is not allowed to alias other types). There is some implementation defined behavior going on, but for the purpose of this question I don't think that is relevant.
I would expect this code to print -50, then 50 on systems where the double follows the IEEE standard, is 8 bytes long and is stored in little endian format. Now the question is. Does the compiler guarantee that this happens. More specifically, turning on optimisations can the compiler optimise away the middle b[7], either explicitly or implicitly, by simply keeping a in a register through the whole function. The second one obviously could be solved by specifying volatile double a, but is that needed?
Edit: As an a note I (mistakenly) remembered that uint8_t was required to be an alias for unsigned char, but indeed the standard does not specify such. I have also written the question in a way that, yes the compiler can ahead of time know everything here, but modified to
#include <iostream>
int main() {
double a;
std::cin >> a;
std::cout << a << "\n";
unsigned char* b = reinterpret_cast<unsigned char*>(&a);
b[7] &= 0x7F;
std::cout << a << "\n";
return 0;
}
one can see where the problem might arise. Here the strict aliasing rule is no longer violated, and a is not a compile time constant. Richard Critten's comment however is curious if the aliased data can be examined, but not written, is there a way one can set individual bytes, while still following the standard?
More specifically, turning on optimisations can the compiler optimise away the middle b[7], either explicitly or implicitly, by simply keeping a in a register through the whole function.
The compiler can generate the double value 50 as a constant, and pass that directly to the output function. b can be optimised away completely. Like most optimisation, this is due to the as-if rule:
[intro.abstract]
The semantic descriptions in this document define a parameterized nondeterministic abstract machine.
This document places no requirement on the structure of conforming implementations.
In particular, they need not copy or emulate the structure of the abstract machine.
Rather, conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below.
The second one obviously could be solved by specifying volatile double a
That would prevent the optimisation, which would generally be considered to be the opposite of a solution.
Does the compiler guarantee that [50 is printed].
You didn't mention what compiler you are asking about. I'm going to assume that you mean whether the standard guarantees this. It doesn't guarantee that universally. You are relying on several assumptions about the implementation:
If sizeof(double) < 8, then you access the object outside of its bounds, and behaviour of the program is undefined.
If std::uint8_t is not an a type alias of unsigned char, then it isn't allowed to alias double, and the behaviour of the program is undefined.
Given the assumptions hold and thus behviour is well-defined, then the second output will be of a double value that is like -50, but whose most significant bit(s from 8th forward) of the byte at position 7 will have been set to 0. In case of little endian IEEE-754 representation, that value would be 50. volatile is not needed to guarantee this, and it won't add a guarantee in case the behaviour of the program is undefined.
I want to know if it is possible to "reduce" the alignment of a datatype in C++. For example, the alignment of int is 4; I want to know if it's possible to set the alignment of int to 1 or 2. I tried using the alignas keyword but it didn't seem to work.
I want to know if this is something not being done by my compiler or the C++ standard doesn't allow this; for either case, I would like to know the reason why it is as such.
I want to know if it is possible to "reduce" the alignment of a datatype in C++.
It is not possible. From this Draft C++ Standard:
10.6.2 Alignment specifier [dcl.align]
…
5 The combined effect of all
alignment-specifiers in a declaration shall not specify an alignment
that is less strict than the alignment that would be required for the
entity being declared if all alignment-specifiers appertaining to that
entity were omitted.
The 'reason' for this is that, in most cases, alignment requirements are dictated by the hardware that is being targeted: if a given CPU requires that an int be stored in a 4-byte-aligned address then, if the compiler were allowed to generate code that puts such an int in a less strictly aligned memory location, the program would cause a hardware fault, when run. (Note that, on some platforms, the alignment requirement for an int is only 1 byte, even though access may be optimized when more strictly aligned.)
Some compilers may offer ways that appear to allow alignment reduction; for example, MSVC has the __declspec(align(#)) extension, which can be applied in a typedef statement. However, from the documentation: __declspec(align(#)) can only increase alignment restrictions:
#include <iostream>
typedef __declspec(align(1)) int MyInt; // No compiler error, but...
int main()
{
std::cout << alignof(int) << "\n"; // "4"
std::cout << alignof(MyInt) << "\n"; // "4" ...doesn't reduce the aligment requirement
return 0;
}
It appears from other StackOverflow questions and reading §9.5.1 of the ISO/IEC draft C++ standard standard that the use of unions to do a literal reinterpret_cast of data is undefined behavior.
Consider the code below. The goal is to take the integer value of 0xffff and literally interpret it as a series of bits in IEEE 754 floating point. (Binary convert shows visually how this is done.)
#include <iostream>
using namespace std;
union unionType {
int myInt;
float myFloat;
};
int main() {
int i = 0xffff;
unionType u;
u.myInt = i;
cout << "size of int " << sizeof(int) << endl;
cout << "size of float " << sizeof(float) << endl;
cout << "myInt " << u.myInt << endl;
cout << "myFloat " << u.myFloat << endl;
float theFloat = *reinterpret_cast<float*>(&i);
cout << "theFloat " << theFloat << endl;
return 0;
}
The output of this code, using both GCC and clang compilers is expected.
size of int 4
size of float 4
myInt 65535
myFloat 9.18341e-41
theFloat 9.18341e-41
My question is, does the standard actually preclude the value of myFloat from being deterministic? Is the use of a reinterpret_cast better in any way to perform this type of conversion?
The standard states the following in §9.5.1:
In a union, at most one of the non-static data members can be active at any time, that is, the value of at most one of the non-static data members can be stored in a union at any time. [...] The size of a union is sufficient to contain the largest of its non-static data members. Each non-static data member is allocated as if it were the sole member of a struct. All non-static data members of a union object have the same address.
The last sentence, guaranteeing that all non-static members have the same address, seems to indicate the use of a union is guaranteed to be identical to the use of a reinterpret_cast, but the earlier statement about active data members seems to preclude this guarantee.
So which construct is more correct?
Edit:
Using Intel's icpc compiler, the above code produces even more interesting results:
$ icpc union.cpp
$ ./a.out
size of int 4
size of float 4
myInt 65535
myFloat 0
theFloat 0
The reason it's undefined is because there's no guarantee what exactly the value representations of int and float are. The C++ standard doesn't say that a float is stored as an IEEE 754 single-precision floating point number. What exactly should the standard say about you treating an int object with value 0xffff as a float? It doesn't say anything other than the fact it is undefined.
Practically, however, this is the purpose of reinterpret_cast - to tell the compiler to ignore everything it knows about the types of objects and trust you that this int is actually a float. It's almost always used for machine-specific bit-level jiggery-pokery. The C++ standard just doesn't guarantee you anything once you do it. At that point, it's up to you to understand exactly what your compiler and machine do in this situation.
This is true for both the union and reinterpret_cast approaches. I suggest that reinterpret_cast is "better" for this task, since it makes the intent clearer. However, keeping your code well-defined is always the best approach.
It's not undefined behavior. It's implementation defined behavior. The first does mean that bad things can happen. The other means that what will happen has to be defined by the implementation.
The reinterpret_cast violates the strict aliasing rule. So I do not think it will work reliably. The union trick is what people call type-punning and is usually allowed by compilers. The gcc folks document the behavior of the compiler: http://gcc.gnu.org/onlinedocs/gcc/Structures-unions-enumerations-and-bit_002dfields-implementation.html#Structures-unions-enumerations-and-bit_002dfields-implementation
I think this should work with icpc as well (but they do not appear to document how they implemented that). But when I looked the assembly, it looks like icc tries to cheat with float and use higher precision floating point stuff. Passing -fp-model source to the compiler fixed that. With that option, I get the same results as with gcc.
I do not think you want to use this flag in general, this is just a test to verify my theory.
So for icpc, I think if you switch your code from int/float to long/double, type-punning will work on icpc as well.
Undefined behavior does not mean bad things must happen. It means only that the language definition doesn't tell you what happens. This kind of type pun has been part of C and C++ programming since time immemorial (i.e., since 1969); it would take a particularly perverse implementor to write a compiler where this didn't work.
Pointers store addresses, either on the heap or on the stack. After some search, I tried to understand what is an "address"; I found it is just an integer value mapping a memory area.
I wonder: as long as an address is just an integer why can't I assign it to an integer variable:
#include <iostream>
using std::cout;
using std::endl;
int main()
{
int a = 1024;
int* ptrA = &a;
cout << "ptrA: " << ptrA << endl; // 0018FF44
cout << "*ptrA: " << *ptrA << endl; // 1024
cout << "&a: " << &a << endl; // 0018FF44
cout << "a: " << a << endl; // 1024
// int b = ptrA; // why this is incorrect
int b = (int)ptrA; // why I need this?
cout << "b: " << std::hex << b << endl; // 18FF44
// so b is identic to ptrA!
std::cout << std::endl;
return 0;
}
There are two primary reasons.
An int may not be big enough to store a pointer value. This is an implementation-specific detail. It is still common to find C++ implementations on 64-bit platforms, with 64-bit memory addresses, where an int is only 32 bits.
Type safety. One of the reasons different types exist it for the compiler to catch obvious mistakes, such as using a pointer to one class when a pointer to a different class is expected. If all pointers, to every object, were just plain ints, these errors will be uncaught.
I found it is just an integer value that is the way memory is mapped.
Usually yes, but this is not necessarily true.
The C++ standard allows many problematic things, like eg. bits in the variable that are not used for storing the value, specific values that are invalid and can cause crashes etc., and so on. Pointers and integers could differ in that points.
A part of it is an actual problem on commonly used platforms: int and pointers can have different sizes (eg. 4 and 8 bytes).
I wonder: as long as an address is just an integer why can't I assign
it to an integer variable
...and that's why you can't do this: There no guarantee that assigning the value in memory is possible and/or makes sense.
On x86-like platforms, as long as you use the correct int size, there are no real problems, but C++ is more than x86...
You can, but only for integer types that are large enough to hold the value of the pointer. On 64-bit systems, pointers are 64 bits long, whereas an int is typically only 32 bits. While you could use a long int or long long int (depending on which kind of processor and which operating system), there is also the type intptr_t from inttypes.h, which is guaranteed to be able to hold the value of a pointer.
Note however that you should almost never have to put the value of a pointer into an integer variable.
From the C language standard ISO/IEC9899 §6.3.2.3/p6 Pointers:
Any pointer type may be converted to an integer type. Except as
previously specified, the result is implementation-defined. If the
result cannot be represented in the integer type, the behaviour is
undefined. The result need not be in the range of values of any
integer type.
As the standard states there are certain issues when converting a pointer to an integer.
The result is implementation-defined.
The result may not be representable by the integer type, which leads in undefined behaviour.
The result need not be in the range of values of any integer type.
Having in mind the above reasons, C++ as a strongly typed language forbids an implicit conversion of such kind, in order for accidental behaviour to be avoided. It is allowed though, to do explicit conversions of such kind in the spirit that as a programmer you know what you're doing.
A type is defined by the data representation that it uses and by the operations that can be applied to it. "An address is just an integer" is about the data representation, not about operations. An address is not an integer.
Well c++ is a strong typed language. Of course an adress of a pointer is just, in most cases, an hexadecimal representation of an integer which represents a spot in the memory. The type of data will tell the compiler how much space it needs to save on the stack to store the variable. Pointers being usually 4 bytes on 32 bits machines.
As for the conversion, it is simply how the language is made. A pointer can only point to the location of a same-type variable initially. The only exception is void.
As for the conversion using the cast, it is required for ehat has been mentionned above, c++ is a strongly-typed language and must know what type it is dealing with.