The following function shifts-left a double operand:
double shl(double x,unsigned long long n)
{
unsigned long long* p = (unsigned long long*)&x;
*p += n << 52;
return x;
}
Is this function guaranteed to work correctly on all platforms?
You may assume a valid combination of x and n (i.e., x*2^n does not overflow).
In other words, I am asking whether or not the language standard dictates the following:
Exponent bits of type double are 52,53,54,55,56,57,58,59,60,61,62
Bit-size of double equals bit-size of unsigned long long equals 64
Strict-aliasing rule for double and unsigned long long is not broken
If the answers for C and for C++ are different, then I would like to know each one of them.
No, neither C nor C++ dictates the representation of floating-point types in the language standard, although the IEEE format is recommended and a macro (__STDC_IEC_559__) is supposed to be available to detect whether it is in use.
Your solution has multiple problems in addition to different representations. You already spotted the strict aliasing violation... the optimizer might turn your whole function into a no-op, since no double is modified between the beginning of the function and the return value, it can assume x isn't changed. You could additionally have an overflow problem -- you would need some form of saturating arithmetic that doesn't allow the result to carry into the sign bit.
However, you don't need to mess with any of that, since the standard library already contains the function you're trying to write.
It is named ldexp (and ldexpf for float).
For C++:
No. C++ does not require IEEE floating point. It doesn't even require binary exponents.
Absolutely not. Unsigned long long may be more than 64 bits. Double need not be 64 bits.
That sort of type punning is unsafe.
This code definitely doesn't shift a double operand to the left. It does some kind of bit manipulation, probably in the hope that the exponent of a double number would be changed.
As it is, the code invokes undefined behaviour, because an lvalue is written using the type long long and then read using the type double. As a result, anything could happen. That's the most unportable code you could get.
Related
Aside from %hn and %hhn (where the h or hh specifies the size of the pointed-to object), what is the point of the h and hh modifiers for printf format specifiers?
Due to default promotions which are required by the standard to be applied for variadic functions, it is impossible to pass arguments of type char or short (or any signed/unsigned variants thereof) to printf.
According to 7.19.6.1(7), the h modifier:
Specifies that a following d, i, o, u, x, or X conversion specifier applies to a
short int or unsigned short int argument (the argument will
have been promoted according to the integer promotions, but its value shall
be converted to short int or unsigned short int before printing);
or that a following n conversion specifier applies to a pointer to a short
int argument.
If the argument was actually of type short or unsigned short, then promotion to int followed by a conversion back to short or unsigned short will yield the same value as promotion to int without any conversion back. Thus, for arguments of type short or unsigned short, %d, %u, etc. should give identical results to %hd, %hu, etc. (and likewise for char types and hh).
As far as I can tell, the only situation where the h or hh modifier could possibly be useful is when the argument passed it an int outside the range of short or unsigned short, e.g.
printf("%hu", 0x10000);
but my understanding is that passing the wrong type like this results in undefined behavior anyway, so that you could not expect it to print 0.
One real world case I've seen is code like this:
char c = 0xf0;
printf("%hhx", c);
where the author expects it to print f0 despite the implementation having a plain char type that's signed (in which case, printf("%x", c) would print fffffff0 or similar). But is this expectation warranted?
(Note: What's going on is that the original type was char, which gets promoted to int and converted back to unsigned char instead of char, thus changing the value that gets printed. But does the standard specify this behavior, or is it an implementation detail that broken software might be relying on?)
One possible reason: for symmetry with the use of those modifiers in the formatted input functions? I know it wouldn't be strictly necessary, but maybe there was value seen for that?
Although they don't mention the importance of symmetry for the "h" and "hh" modifiers in the C99 Rationale document, the committee does mention it as a consideration for why the "%p" conversion specifier is supported for fscanf() (even though that wasn't new for C99 - "%p" support is in C90):
Input pointer conversion with %p was added to C89, although it is obviously risky, for symmetry with fprintf.
In the section on fprintf(), the C99 rationale document does discuss that "hh" was added, but merely refers the reader to the fscanf() section:
The %hh and %ll length modifiers were added in C99 (see §7.19.6.2).
I know it's a tenuous thread, but I'm speculating anyway, so I figured I'd give whatever argument there might be.
Also, for completeness, the "h" modifier was in the original C89 standard - presumably it would be there even if it wasn't strictly necessary because of widespread existing use, even if there might not have been a technical requirement to use the modifier.
In %...x mode, all values are interpreted as unsigned. Negative numbers are therefore printed as their unsigned conversions. In 2's complement arithmetic, which most processors use, there is no difference in bit patterns between a signed negative number and its positive unsigned equivalent, which is defined by modulus arithmetic (adding the maximum value for the field plus one to the negative number, according to the C99 standard). Lots of software- especially the debugging code most likely to use %x- makes the silent assumption that the bit representation of a signed negative value and its unsigned cast is the same, which is only true on a 2's complement machine.
The mechanics of this cast are such that hexidecimal representations of value always imply, possibly inaccurately, that a number has been rendered in 2's complement, as long as it didn't hit an edge condition of where the different integer representations have different ranges. This even holds true for arithmetic representations where the value 0 is not represented with the binary pattern of all 0s.
A negative short displayed as an unsigned long in hexidecimal will therefore, on any machine, be padded with f, due to implicit sign extension in the promotion, which printf will print. The value is the same, but it is truly visually misleading as to the size of the field, implying a significant amount of range that simply isn't present.
%hx truncates the displayed representation to avoid this padding, exactly as you concluded from your real-world use case.
The behavior of printf is undefined when passed an int outside the range of short that should be printed as a short, but the easiest implementation by far simply discards the high bit by a raw downcast, so while the spec doesn't require any specific behavior, pretty much any sane implementation is going to just perform the truncation. There're generally better ways to do that, though.
If printf isn't padding values or displaying unsigned representations of signed values, %h isn't very useful.
The only use I can think of is for passing an unsigned short or unsigned char and using the %x conversion specifier. You cannot simply use a bare %x - the value may be promoted to int rather than unsigned int, and then you have undefined behaviour.
Your alternatives are either to explicitly cast the argument to unsigned; or to use %hx / %hhx with a bare argument.
The variadic arguments to printf() et al are automatically promoted using the default conversions, so any short or char values are promoted to int when passed to the function.
In the absence of the h or hh modifiers, you would have to mask the values passed to get the correct behaviour reliably. With the modifiers, you no longer have to mask the values; the printf() implementation does the job properly.
Specifically, for the format %hx, the code inside printf() can do something like:
va_list args;
va_start(args, format);
...
int i = va_arg(args, int);
unsigned short s = (unsigned short)i;
...print s correctly, as 4 hex digits maximum
...even on a machine with 64-bit `int`!
I'm blithely assuming that short is a 16-bit quantity; the standard does not actually guarantee that, of course.
I found it useful to avoid casting when formatting unsigned chars to hex:
sprintf_s(tmpBuf, 3, "%2.2hhx", *(CEKey + i));
It's a minor coding convenience, and looks cleaner than multiple casts (IMO).
another place it's handy is snprintf size check.
gcc7 added size check when using snprintf
so this will fail
char arr[4];
char x='r';
snprintf(arr,sizeof(arr),"%d",r);
so it forces you to use bigger char when using %d when formatting a char
here is a commit that shows those fixes instead of increasing the char array size they changed %d to %h. this also give more accurate description
https://github.com/Mellanox/libvma/commit/b5cb1e34a04b40427d195b14763e462a0a705d23#diff-6258d0a11a435aa372068037fe161d24
I agree with you that it is not strictly necessary, and so by that reason alone is no good in a C library function :)
It might be "nice" for the symmetry of the different flags, but it is mostly counter-productive because it hides the "conversion to int" rule.
I know that in order to get the 4 least significant bytes of a number of type long I can cast it to int/unsigned int or use a bitwise AND (& 0xFFFFFFFF).
This code produces the following output:
#include <stdio.h>
int main()
{
long n = 0x8899AABBCCDDEEFF;
printf("0x%016lX\n", n);
printf("0x%016X\n", (int)n);
printf("0x%016X\n", (unsigned int)n);
printf("0x%016lX\n", n & 0xFFFFFFFF);
}
Output:
0x8899AABBCCDDEEFF
0x00000000CCDDEEFF
0x00000000CCDDEEFF
0x00000000CCDDEEFF
Does that mean that the two methods used are equivalent? If so, do they always produce the same output regardless of the platform/compiler?
Also, is there any catch or pitfall while casting to unsigned int rather than int for the purpose of this question?
Finally, why is the output the same if you change the number n to be an unsigned long instead?
The methods are definitely different.
According to integral conversion rules (cf, for example, this online c++11 standard), a conversion (e.g. through an explicit cast) from one integral type to another depends on whether the destination type is signed or unsigned. If the destination type is unsigned, one can rely on a "modulo 2n" truncation, whereas with signed destination types one could tap into implementation defined behaviour:
4.7 Integral conversions [conv.integral]
2 If the destination type is unsigned, the resulting value is the
least unsigned integer congruent to the source integer (modulo 2n
where n is the number of bits used to represent the unsigned type). [
Note: In a two's complement representation, this conversion is
conceptual and there is no change in the bit pattern (if there is no
truncation). — end note ]
3 If the destination type is signed, the value is unchanged if it can
be represented in the destination type (and bit-field width);
otherwise, the value is implementation-defined.
For your first question, as others have pointed out, the size of int and long is dependent on the platform, so the methods are not equivalent. In C data types, check that the types say "at least XX bits in size"
For the second question, it comes down to this: long and int are signed, meaning that one bit is reserved for sign (take a look also to two's complement). If you were the compiler, what can you do with negative values (especially the long ones)? As Stepahn Lechner mentioned, this is implementation defined (that is, is up to the compiler).
Finally, in the spirit of "your code must do what it says it does", the best thing to do if you need to do masks is to use masks (and, if you use masks, use unsigned types). Don't try to use cleaver answers. Believe me, they always bite you in the rear. I've dealt with a lot of legacy code to know that by heart.
What's the difference between casting a long to int versus using a bitwise AND in order to get the 4 least significant bytes?
Type. Casting makes the value an int. And'ing does not change the type.
Range. Depending on int,long range, a cast may not change the value at all.
IDB and UB. implementation defined behavior and undefined behavior are present with mixing signed-ness.
To "get" the 4 LSBytes, use & 0xFFFFFFFFu or cast to uint32_t.
OP's question is unnecessarily convoluted.
long n = 0x8899AABBCCDDEEFF; --> Converting a value outside the range of a signed integer type is implementation-defined.
Otherwise, the new type is signed and the value cannot be represented in it; either the
result is implementation-defined or an implementation-defined signal is raised.
C11 §6.3.1.3 3
printf("0x%016lX\n", n); --> Printing a long with a "%lX" outside the the common range of long/unsigned long is undefined behavior.
Let's go forward with unsigned long:
unsigned long n = 0x8899AABBCCDDEEFF; // no problem,
printf("0x%016lX\n", n); // no problem,
printf("0x%016X\n", (int)n); // problem, C11 6.3.1.3 3
printf("0x%016X\n", (unsigned int)n); // no problem,
printf("0x%016lX\n", n & 0xFFFFFFFF); // no problem,
The "no problem" are OK even is unsigned long is 32-bit or 64-bit. The output will differ, yet is OK.
Recall that int,long are not always 32,64 bit. (16,32), (32,32), (32,64) are common.
int is at least 16 bit.
long is at least that of int and at least 32 bit.
unsigned int t = 10;
int d = 16;
float c = t - d;
int e = t - d;
Why is the value of c positive but e negative?
Let's start by analysing the result of t - d.
t is an unsigned int while d is an int, so to do arithmetic on them, the value of d is converted to an unsigned int (C++ rules say unsigned gets preference here). So we get 10u - 16u, which (assuming 32-bit int) wraps around to 4294967290u.
This value is then converted to float in the first declaration, and to int in the second one.
Assuming the typical implementation of float (32-bit single-precision IEEE), its highest representable value is roughly 1e38, so 4294967290u is well within that range. There will be rounding errors, but the conversion to float won't overflow.
For int, the situation's different. 4294967290u is too big to fit into an int, so wrap-around happens and we arrive back at the value -6. Note that such wrap-around is not guaranteed by the standard: the resulting value in this case is implementation-defined(1), which means it's up to the compiler what the result value is, but it must be documented.
(1) C++17 (N4659), [conv.integral] 7.8/3:
If the destination type is signed, the value is unchanged if it can be represented in the destination type;
otherwise, the value is implementation-defined.
First, you have to understand "usual arithmetic conversions" (that link is for C, but the rules are the same in C++). In C++, if you do arithmetic with mixed types (you should avoid that when possible, by the way), there's a set of rules that decides which type the calculation is done in.
In your case, you are subtracting a signed int from an unsigned int. The promotion rules say that the actual calculation is done using unsigned int.
So your calculation is 10 - 16 in unsigned int arithmetic. Unsigned arithmetic is modulo arithmetic, meaning that it wraps around. So, assuming your typical 32-bit int, the result of this calculation is 2^32 - 6.
This is the same for both lines. Note that the subtraction is completely independent from the assignment; the type on the left side has absolutely no influence on how the calculation happens. It is a common beginner mistake to think that the type on the left side somehow influences the calculation; but float f = 5 / 6 is zero, because the division still uses integer arithmetic.
The difference, then, is what happens during the assignment. The result of the subtraction is implicitly converted to float in one case, and int in the other.
The conversion to float tries to find the closest value to the actual one that the type can represent. This will be some very large value; not quite the one the original subtraction yielded though.
The conversion to int says that if the value fits into the range of int, the value will be unchanged. But 2^32 - 6 is far larger than the 2^31 - 1 that a 32-bit int can hold, so you get the other part of the conversion rule, which says that the resulting value is implementation-defined. This is a term in the standard that means "different compilers can do different things, but they have to document what they do".
For all practical purposes, all compilers that you'll likely encounter say that the bit pattern stays the same and is just interpreted as signed. Because of the way 2's complement arithmetic works (the way that almost all computers represent negative numbers), the result is the -6 you would expect from the calculation.
But all this is a very long way of repeating the first point, which is "don't do mixed type arithmetic". Cast the types first, explicitly, to types that you know will do the right thing.
I was going through this example which has a function outputting a hex bit pattern to represent an arbitrary float.
void ExamineFloat(float fValue)
{
printf("%08lx\n", *(unsigned long *)&fValue);
}
Why take the address of fValue, cast to unsigned long pointer, then dereference? Isn't all that work just equivalent to a direct cast to unsigned long?
printf("%08lx\n", (unsigned long)fValue);
I tried it and the answer isn't the same, so confused.
(unsigned long)fValue
This converts the float value to an unsigned long value, according to the "usual arithmetic conversions".
*(unsigned long *)&fValue
The intention here is to take the address at which fValue is stored, pretend that there is not a float but an unsigned long at this address, and to then read that unsigned long. The purpose is to examine the bit pattern which is used to store the float in memory.
As shown, this causes undefined behavior though.
Reason: You may not access an object through a pointer to a type that is not "compatible" to the object's type. "Compatible" types are for example (unsigned) char and every other type, or structures that share the same initial members (speaking of C here). See §6.5/7 N1570 for the detailed (C11) list (Note that my use of "compatible" is different - more broad - than in the referenced text.)
Solution: Cast to unsigned char *, access the individual bytes of the object and assemble an unsigned long out of them:
unsigned long pattern = 0;
unsigned char * access = (unsigned char *)&fValue;
for (size_t i = 0; i < sizeof(float); ++i) {
pattern |= *access;
pattern <<= CHAR_BIT;
++access;
}
Note that (as #CodesInChaos pointed out) the above treats the floating point value as being stored with its most significant byte first ("big endian"). If your system uses a different byte order for floating point values you'd need to adjust to that (or rearrange the bytes of above unsigned long, whatever's more practical to you).
Floating-point values have memory representations: for example the bytes can represent a floating-point value using IEEE 754.
The first expression *(unsigned long *)&fValue will interpret these bytes as if it was the representation of an unsigned long value. In fact in C standard it results in an undefined behavior (according to the so-called "strict aliasing rule"). In practice, there are issues such as endianness that have to be taken into account.
The second expression (unsigned long)fValue is C standard compliant. It has a precise meaning:
C11 (n1570), § 6.3.1.4 Real floating and integer
When a finite value of real floating type is converted to an integer type other than _Bool, the fractional part is discarded (i.e., the value is truncated toward zero). If the value of the integral part cannot be represented by the integer type, the behavior is undefined.
*(unsigned long *)&fValue is not equivalent to a direct cast to an unsigned long.
The conversion to (unsigned long)fValue converts the value of fValue into an unsigned long, using the normal rules for conversion of a float value to an unsigned long value. The representation of that value in an unsigned long (for example, in terms of the bits) can be quite different from how that same value is represented in a float.
The conversion *(unsigned long *)&fValue formally has undefined behaviour. It interprets the memory occupied by fValue as if it is an unsigned long. Practically (i.e. this is what often happens, even though the behaviour is undefined) this will often yield a value quite different from fValue.
Typecasting in C does both a type conversion and a value conversion. The floating point → unsigned long conversion truncates the fractional portion of the floating point number and restricts the value to the possible range of an unsigned long. Converting from one type of pointer to another has no required change in value, so using the pointer typecast is a way to keep the same in-memory representation while changing the type associated with that representation.
In this case, it's a way to be able to output the binary representation of the floating point value.
As others have already noted, casting a pointer to a non-char type to a pointer to a different non-char type and then dereferencing is undefined behavior.
That printf("%08lx\n", *(unsigned long *)&fValue) invokes undefined behavior does not necessarily mean that running a program that attempts to perform such a travesty will result in hard drive erasure or make nasal demons erupt from ones nose (the two hallmarks of undefined behavior). On a computer in which sizeof(unsigned long)==sizeof(float) and on which both types have the same alignment requirements, that printf will almost certainly do what one expects it to do, which is to print the hex representation of the floating point value in question.
This shouldn't be surprising. The C standard openly invites implementations to extend the language. Many of these extensions are in areas that are, strictly speaking, undefined behavior. For example, the POSIX function dlsym returns a void*, but this function is typically used to find the address of a function rather than a global variable. This means the void pointer returned by dlsym needs to be cast to a function pointer and then dereferenced to call the function. This is obviously undefined behavior, but it nonetheless works on any POSIX compliant platform. This will not work on a Harvard architecture machine on which pointers to functions have different sizes than do pointers to data.
Similarly, casting a pointer to a float to a pointer to an unsigned integer and then dereferencing happens to work on almost any computer with almost any compiler in which the size and alignment requirements of that unsigned integer are the same as that of a float.
That said, using unsigned long might well get you into trouble. On my computer, an unsigned long is 64 bits long and has 64 bit alignment requirements. This is not compatible with a float. It would be better to use uint32_t -- on my computer, that is.
The union hack is one way around this mess:
typedef struct {
float fval;
uint32_t ival;
} float_uint32_t;
Assigning to a float_uint32_t.fval and accessing from a ``float_uint32_t.ival` used to be undefined behavior. That is no longer the case in C. No compiler that I know of blows nasal demons for the union hack. This was not UB in C++. It was illegal. Until C++11, a compliant C++ compiler had to complain to be compliant.
Any even better way around this mess is to use the %a format, which has been part of the C standard since 1999:
printf ("%a\n", fValue);
This is simple, easy, portable, and there is no chance of undefined behavior. This prints the hexadecimal/binary representation of the double precision floating point value in question. Since printf is an archaic function, all float arguments are converted to double prior to the call to printf. This conversion must be exact per the 1999 version of the C standard. One can pick up that exact value via a call to scanf or its sisters.
I have the below simple program:
#include <iostream>
#include <stdio.h>
void SomeFunction(int a)
{
std::cout<<"Value in function: a = "<<a<<std::endl;
}
int main(){
size_t a(0);
std::cout<<"Value in main: "<<a-1<<std::endl;
SomeFunction(a-1);
return 0;
}
Upon executing this I get:
Value in main: 18446744073709551615
Value in function: a = -1
I think I roughly understand why the function gets the 'correct' value of -1: there is an implicit conversion from the unsigned type to the signed one i.e. 18446744073709551615(unsigned) = -1(signed).
Is there any situation where the function will not get the 'correct' value?
Since size_t type is unsigned, subtracting 1 is well defined:
A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.
However, the resultant value of 264-1 is out of ints range, so you get implementation-defined behavior:
[when] the new type is signed and the value cannot be represented in it, either the result is implementation-defined or an implementation-defined signal is raised.
Therefore, the answer to your question is "yes": there are platforms where the value of a would be different; there are also platforms where instead of calling SomeFunction the program will raise a signal.
Not on your computer... but technically yes, there is a situation where things can go wrong.
All modern PCs use the "two's complement" system for signed integer arithmetic (read Wikipedia for details). Two's complement has many advantages, but one of the biggest is this: unsaturated addition and subtraction of signed integers is identical to that of unsigned integers. As long as overflow/underflow causes the result to "wrap around" (i.e., 0-1 = UINT_MAX), the computer can add and subtract without even knowing whether you're interpreting the numbers as signed or unsigned.
BUT! C/C++ do not technically require two's complement for signed integers. There are two other permissible systems, known as "sign-magnitude" and "one's complement". These are unusual systems, never found outside antique architectures and embedded processors (and rarely even there). But in those systems, signed and unsigned arithmetic do not match up, and (signed)(a+b) will not necessarily equal (signed)a + (signed) b.
There's another, more mundane caveat when you're also narrowing types, as is the case between size_t and int on x64, because C/C++ don't require compilers to follow a particular rule when narrowing out-of-range values to signed types. This is likewise more a matter of language lawyering than actual unsafeness, though: VC++, GCC, Clang, and all other compilers I'm aware of narrow through truncation, leading to the expected behavior.
It's easier to compare and contrast signed and insigned of type same basic type, say signed int and unsigned int.
On a system that uses 32 bits for int, the range of unsigned int is [0 - 4294967295] and the range of signed int is [-2147483647 - 2147483647].
Say you have a variable of type unsigned int and its value is greater than 2147483647. If you pass such a variable to SomeFunction, you will see an incorrect value in the function.
Conversely, say you have a variable of type signed int and its value is less than zero. If you pass such a variable to a function that expects an unsigned int, you will see an incorrect value in the function.