Save float16 max number in float32 - c++

How to save the float16 (https://en.wikipedia.org/wiki/Half-precision_floating-point_format) max number in float32 (https://en.wikipedia.org/wiki/Single-precision_floating-point_format) format?
I want to have a function which could convert 0x7bff to 65504. 0x7bff is the max value can be represented by floating point half precision:
0 11110 1111111111 -> decimal value: 65504
I want to have 0x7bff to represent the actual bits in my program.
float fp16_max = bit_cast(0x7bff);
# want "std::cout << fp16_max" to be 65504
I tried to implement such a function but it didn't seem to work:
float bit_cast (uint32_t fp16_bits) {
float i;
memcpy(&i, &fp16_bits, 4);
return i;
}
float test = bit_cast(0x7bff);
# print out test: 4.44814e-41

#include <cmath>
#include <cstdio>
/* Decode the IEEE-754 binary16 encoding into a floating-point value.
Details of NaNs are not handled.
*/
static float InterpretAsBinary16(unsigned Bits)
{
// Extract the fields from the binary16 encoding.
unsigned SignCode = Bits >> 15;
unsigned ExponentCode = Bits >> 10 & 0x1f;
unsigned SignificandCode = Bits & 0x3ff;
// Interpret the sign bit.
float Sign = SignCode ? -1 : +1;
// Partition into cases based on exponent code.
float Significand, Exponent;
// An exponent code of all ones denotes infinity or a NaN.
if (ExponentCode == 0x1f)
return Sign * (SignificandCode == 0 ? INFINITY : NAN);
// An exponent code of all zeros denotes zero or a subnormal.
else if (ExponentCode == 0)
{
/* Subnormal significands have a leading zero, and the exponent is the
same as if the exponent code were 1.
*/
Significand = 0 + SignificandCode * 0x1p-10;
Exponent = 1 - 0xf;
}
// Other exponent codes denote normal numbers.
else
{
/* Normal significands have a leading one, and the exponent is biased
by 0xf.
*/
Significand = 1 + SignificandCode * 0x1p-10;
Exponent = ExponentCode - 0xf;
}
// Combine the sign, significand, and exponent, and return the result.
return Sign * std::ldexp(Significand, Exponent);
}
int main(void)
{
unsigned Bits = 0x7bff;
std::printf(
"Interpreting the bits 0x%x as an IEEE-754 binary16 yields %.99g.\n",
Bits,
InterpretAsBinary16(Bits));
}

By the very declaration float fp16_max, your value is already a 32-bit float; no need to cast here. I guess you can simply:
float i = fp16_max;
The assumption here is that your "magic" bit_cast function already returned a 32-bit float properly. Since you haven't shown us what bit-cast does or actually returns, I'll assume it does indeed return a proper float value.

How to save the float16 max number in float32 format?
65504
You can simply convert the integer to float:
float half_max = 65504;
If you would like to calculate the value, you can use ldexpf:
float half_max = (2 - ldexpf(1, -10)) * ldexpf(1, 15)
Or generally, for any IEEE float:
// in case of half float
int bits = 16;
int man_bits = 10;
// the calculation
int exp_bits = bits - man_bits - 1;
int exp_max = (1 << (exp_bits - 1)) - 1;
long double max = (2 - ldexp(1, -1 * man_bits)) * ldexp(1, exp_max);
Bit casting 0x7bff does not work, because 0x7bff is the representation in the binary16 format (in some endianness), not in binary32 format. You cannot bit cast conflicting representations.

Related

round double with N significant decimal digits in an overflow-safe manner

I want a overflow-safe function that round a double like std::round in addition it can handle the number of significant decimal digts.
f.e.
round(-17.747, 2) -> -17.75
round(-9.97729, 2) -> -9.98
round(-5.62448, 2) -> -5.62
round(std::numeric_limits<double>::max(), 10) ...
My first attempt was
double round(double value, int precision)
{
double factor=pow(10.0, precision);
return floor(value*factor+0.5)/factor;
}
but this can easily overflow.
Assuming IEEE, it is possible to decrease the possibility of overflows, like this.
double round(double value, int precision)
{
// assuming IEEE 754 with 64 bit representation
// the number of significant digits varies between 15 and 17
precision=std::min(17, precision);
double factor=pow(10.0, precision);
return floor(value*factor+0.5)/factor;
}
But this still can overflow.
Even this performance disaster does not work.
double round(double value, int precision)
{
std::stringstream ss;
ss << std::setprecision(precision) << value;
std::string::size_type sz;
return std::stod(ss.str(), &sz);
}
round(std::numeric_limits<double>::max(), 2.0) // throws std::out_of_range
Note:
I'm aware of setprecision, but i need rounding not only for displaying purpose. So that is not a solution.
Unlike this post here How to round a number to n decimal places in Java , my question is especially on overflow safety and in C++ (the anwser in the topic above are Java-specific or do not handle overflows)
I haven't heavily tested this code:
/* expects x in (-1, 1) */
double round_precision2(double x, int precision2) {
double iptr, factor = std::exp2(precision2);
double y = (x < 0) ? -x : x;
std::modf(y * factor + .5, &iptr);
return iptr/factor * ((x < 0) ? -1 : 1);
}
double round_precision(double x, int precision) {
int bits = precision * M_LN10 / M_LN2;
/* std::log2(std::pow(10., precision)); */
double iptr, frac = std::modf(x, &iptr);
return iptr + round_precision2(frac, bits);
}
The idea is to avoid overflow by only operating on the fractional part of the number.
We compute the number of binary bits to achieve the desired precision. You should be able to put a bound on them with the limits you describe in your question.
Next, we extract the fractional and integer parts of the number.
Then we add the integer part back to the rounded fractional part.
To compute the rounded fractional part, we compute the binary factor. Then we extract the integer part of the rounded number resulting from multiplying fractional part by the factor. Then we return the fraction by dividing the integral part by the factor.

Is it correct to state that the first number that collide in single precision is 131072.02? (positive, considering 2 digits as mantissa)

I was trying to figure it out for my audio application if float can be used to represent correctly the range of parameters I'll use.
The "biggest" mask it needs is for frequency params, which is positive, and allow max two digits as mantissa (i.e. from 20.00 hz to 22000.00 hz). Conceptually, the following digits will be rounded out, so I don't care for them.
So I made this script to check the first number that collide in single precision:
float temp = 0.0;
double valueDouble = 0.0;
double increment = 1e-2;
bool found = false;
while(!found) {
double oldValue = valueDouble;
valueDouble += increment;
float value = valueDouble;
// found
if(temp == value) {
std::cout << "collision found: " << valueDouble << std::endl;
std::cout << " collide with: " << oldValue << std::endl;
std::cout << "float stored as: " << value << std::endl;
found = true;
}
temp = value;
}
and its seems its 131072.02 (with 131072.01, stored as the same 131072.015625 value), which is far away than 22000.00. And it seems I would be ok using float.
But I'd like to understand if that reasoning is correct. It is?
The whole problem would be if I set a param of XXXXX.YY (7 digits) and it collides with some other numbers having a less number of digits (because single precision only guarantee 6 digits)
Note: of course numbers such as 1024.0002998145910169114358723163604736328125 or 1024.000199814591042013489641249179840087890625 collide, and they are within the interval, but they do it at a longer significative digits than my required mantissa, so I don't care.
IEEE 754 Single precision is defined as
1 sign bit
8 exponent bits: range 2^-126 to 2^127 ~ 10^-38 to 10^38)
23 fraction (mantissa) bits: decimal precision depending on the exponent)
At 22k the exponent will represent an offset of 16384=2^14, so the 23-bit mantissa will give you a precision of 2^14/2^23= 1/2^9 = 0.001953125... which is sufficient for your case.
For 131072.01, the exponent will represent an offset 131072 = 2^17, so the mantissa will give a precision of 2^17/2^23 = 1/2^6 = 0.015625 which is larger then your target precision of 0.01
Your program does not verify exactly what you want, but your underlying reasoning should be ok.
The problem with the program is that valueDouble will accumulate slight errors (since 0.01 isn't represented accurately) - and converting the string "20.01" to a floating point number will introduce slight round-off errors.
But those errors should be on the order of DBL_EPSILON and be much smaller than the error you see.
If you really wanted to test it you would have to write "20.00" to "22000.00" and scan them all using the scanf-variant you plan to use and verify that they differ.
Is it correct to state that the first number that collide in single precision is 131072.02? (positive, considering 2 digits as mantissa after the decimal point)
Yes.
I'd like to understand if that reasoning is correct. It is?
For values just less than 131072.0f, each successive representable float value is 1/128th apart.
For values in the range [131072.0f ... 2*131072.0f), each successive representable float value is 1/64th apart.
With values of the decimal textual form "131072.xx", there are 100 combinations, yet only 64 differ float. It is not surprising that 100-64 or 36 collisions occurs - see below. For numbers of this form, this is the first place the density of float is too sparse: the least significant bit in float > 0.01 in this range.
int main(void) {
volatile float previous = 0.0;
for (long i = 1; i <= 99999999; i++) {
volatile float f1 = i / 100.0;
if (previous == f1) {
volatile float f0 = nextafterf(f1, 0);
volatile float f2 = nextafterf(f1, f1 * 2);
printf("%f %f %f delta fraction:%f\n", f0, f1, f2, 1.0 / (f1 - f0));
static int count = 100 - 64;
if (--count == 0) return 0;
}
previous = f1;
}
printf("Done\n");
}
Output
131072.000000 131072.015625 131072.031250 delta fraction:64.000000
131072.031250 131072.046875 131072.062500 delta fraction:64.000000
131072.046875 131072.062500 131072.078125 delta fraction:64.000000
...
131072.921875 131072.937500 131072.953125 delta fraction:64.000000
131072.937500 131072.953125 131072.968750 delta fraction:64.000000
131072.968750 131072.984375 131073.000000 delta fraction:64.000000
Why floating-points number's significant numbers is 7 or 6 may also help.

Storing non-negative floating point values

Is there an efficient way to store non-negative floating point values using the existing float32 and float64 formats?
Imagine the default float32 behaviour which allows negative/positive:
val = bytes.readFloat32();
Is it possible to allow for greater positive values if negative values are not necessary?
val = bytes.readFloat32() + 0xFFFFFFFF;
Edit: Essentially when I know I'm storing only positive values, the float format could be modified a bit to allow for greater range or precision for the same amount of bits.
Eg. The float32 format is defined as 1 bit for sign, 8 bits for exponent, 23 bits for fraction
What if I don't need the sign bit, can we have 8 bits for exponent, 24 bits for fraction to give greater precision for the same 32 bits?
Floating-point numbers (float32 and float64) have an explicit sign bit. The equivalent of unsigned integers doesn't exist for floating-point numbers.
So there is no easy way to double the range of positive floating-point numbers.
No, not for free.
You can extend the range/accuracy in many ways using other numeric representations. The intent won't be clear, and the performance will typically be poor if you want the range and accuracy of float or double using another numeric representation (of equal size).
Just stick with float or double unless performance/storage is very very important, and you can represent your values well (or better!) using another numeric representation.
There's almost no support for unsigned float in hardware so you won't have such off-the-shelf feature but you can still have quite efficient unsigned float by storing the least significant bit in the sign bit. This way you can utilize the available floating-point hardware support instead of writing a software float solution. To do that you can
manipulate it manually after each operation
This way you need some small correction to the lsb (A.K.A sign bit), for example 1 more long division step, or a 1-bit adder for the addition
or by doing the math in higher precision if available
For example for float you can do operations in double then cast back to float when storing if sizof(float) < sizeof(double)
Here's a simple PoC implementation:
#include <cmath>
#include <cfenv>
#include <bit>
#include <type_traits>
// Does the math in double precision when hardware double is available
#define HAS_NATIVE_DOUBLE
class UFloat
{
public:
UFloat(double d) : UFloat(0.0f)
{
if (d < 0)
throw std::range_error("Value must be non-negative!");
uint64_t dbits = std::bit_cast<uint64_t>(d);
bool lsb = dbits & lsbMask;
dbits &= ~lsbMask; // turn off the lsb
d = std::bit_cast<double>(dbits);
value = lsb ? -(float)d : (float)d;
}
UFloat(const UFloat &rhs) : UFloat(rhs.value) {}
// =========== Operators ===========
UFloat &operator+=(const UFloat &rhs)
{
#ifdef HAS_NATIVE_DOUBLE
// Calculate in higher precision then round back
setValue((double)value + rhs.value);
#else
// Calculate the least significant bit manually
bool lhsLsb = std::signbit(value);
bool rhsLsb = std::signbit(rhs.value);
// Clear the sign bit to get the higher significant bits
// then get the sum
value = std::abs(value);
value += std::abs(rhs.value);
if (std::isfinite(value))
{
if (lhsLsb ^ rhsLsb) // Only ONE of the 2 least significant bits is 1
{
// The sum's lsb is 1, so we'll set its sign bit
value = -value;
}
else if (lhsLsb)
{
// BOTH least significant bits are 1s,
// so we'll add the carry to the next bit
value = std::nextafter(value, INFINITY);
// The lsb of the sum is 0, so the sign bit isn't changed
}
}
#endif
return *this;
}
UFloat &operator*=(const UFloat &rhs)
{
#ifdef HAS_NATIVE_DOUBLE
// Calculate in higher precision then round back
setValue((double)value * rhs.value);
#else
// Calculate the least significant bit manually
bool lhsLsb = std::signbit(value);
bool rhsLsb = std::signbit(rhs.value);
// Clear the sign bit to get the higher significant bits
// then get the product
float lhsMsbs = std::abs(value);
float rhsMsbs = std::abs(rhs.value);
// Suppose we have X.xPm with
// X: the high significant bits
// x: the least significant one
// and m: the exponent. Same to Y.yPn
// X.xPm * Y.yPn = (X + 0.x)*2^m * (Y + 0.y)*2^n
// = (X + x/2)*2^m * (Y + y/2)*2^n
// = (X*Y + X*y/2 + Y*x/2 + x*y/4)*2^(m + n)
value = lhsMsbs * rhsMsbs; // X*Y
if (std::isfinite(value))
{
uint32_t rhsMsbsBits = std::bit_cast<uint32_t>(rhsMsb);
value += rhsMsbs*lhsLsb / 2; // X*y/2
uint32_t lhsMsbsBits = std::bit_cast<uint32_t>(lhsMsbs);
value += lhsMsbs*rhsLsb / 2; // Y*x/2
int lsb = (rhsMsbsBits | lhsMsbsBits) & 1; // the product's lsb
lsb += lhsLsb & rhsLsb;
if (lsb & 1)
value = -value; // set the lsb
if (lsb > 1) // carry to the next bit
value = std::nextafter(value, INFINITY);
}
#endif
return *this;
}
UFloat &operator/=(const UFloat &rhs)
{
#ifdef HAS_NATIVE_DOUBLE
// Calculate in higher precision then round back
setValue((double)value / rhs.value);
#else
// Calculate the least significant bit manually
// Do just one more step of long division,
// since we only have 1 bit left to divide
throw std::runtime_error("Not Implemented yet!");
#endif
return *this;
}
double getUnsignedValue() const
{
if (!std::signbit(value))
{
return value;
}
else
{
double result = std::abs(value);
uint64_t doubleValue = std::bit_cast<uint64_t>(result);
doubleValue |= lsbMask; // turn on the least significant bit
result = std::bit_cast<double>(doubleValue);
return result;
}
}
private:
// The unsigned float value, with the least significant bit (lsb)
// being stored in the sign bit
float value;
// the first bit after the normal mantissa bits
static const uint64_t lsbMask = 1ULL << (DBL_MANT_DIG - FLT_MANT_DIG - 1);
// =========== Private Constructor ===========
UFloat(float rhs) : value(rhs)
{
std::fesetround(FE_TOWARDZERO); // We'll round the value ourselves
#ifdef HAS_NATIVE_DOUBLE
static_assert(sizeof(float) < sizeof(double));
#endif
}
void setValue(double d)
{
// get the bit pattern of the double value
auto bits = std::bit_cast<std::uint64_t>(d);
bool lsb = bits & lsbMask;
// turn off the lsb to avoid rounding when converting to float
bits &= ~lsbMask;
d = std::bit_cast<double>(bits);
value = (float)d;
if (lsb)
value = -value;
}
}
Some more tuning may be necessary to get the correct lsb
Either way you'll need more operations than normal so this may only be good for big arrays where cache footprint is a concern. In that case I suggest to use this as a storage format only, like how FP16 is treated on most current architectures: there are only load/store instructions for it which expands to float or double and convert back. All arithmetic operations are done in float or double only
So the unsigned float should exist in memory only, and will be decoded to the full double on load. This way you work on the native double type and won't need the correction after each operator
Alternatively this can be used with SIMD to operate on multiple unsigned floats at the same time

Generating random floating-point values based on random bit stream

Given a random source (a generator of random bit stream), how do I generate a uniformly distributed random floating-point value in a given range?
Assume that my random source looks something like:
unsigned int GetRandomBits(char* pBuf, int nLen);
And I want to implement
double GetRandomVal(double fMin, double fMax);
Notes:
I don't want the result precision to be limited (for example only 5 digits).
Strict uniform distribution is a must
I'm not asking for a reference to an existing library. I want to know how to implement it from scratch.
For pseudo-code / code, C++ would be most appreciated
I don't think I'll ever be convinced that you actually need this, but it was fun to write.
#include <stdint.h>
#include <cmath>
#include <cstdio>
FILE* devurandom;
bool geometric(int x) {
// returns true with probability min(2^-x, 1)
if (x <= 0) return true;
while (1) {
uint8_t r;
fread(&r, sizeof r, 1, devurandom);
if (x < 8) {
return (r & ((1 << x) - 1)) == 0;
} else if (r != 0) {
return false;
}
x -= 8;
}
}
double uniform(double a, double b) {
// requires IEEE doubles and 0.0 < a < b < inf and a normal
// implicitly computes a uniform random real y in [a, b)
// and returns the greatest double x such that x <= y
union {
double f;
uint64_t u;
} convert;
convert.f = a;
uint64_t a_bits = convert.u;
convert.f = b;
uint64_t b_bits = convert.u;
uint64_t mask = b_bits - a_bits;
mask |= mask >> 1;
mask |= mask >> 2;
mask |= mask >> 4;
mask |= mask >> 8;
mask |= mask >> 16;
mask |= mask >> 32;
int b_exp;
frexp(b, &b_exp);
while (1) {
// sample uniform x_bits in [a_bits, b_bits)
uint64_t x_bits;
fread(&x_bits, sizeof x_bits, 1, devurandom);
x_bits &= mask;
x_bits += a_bits;
if (x_bits >= b_bits) continue;
double x;
convert.u = x_bits;
x = convert.f;
// accept x with probability proportional to 2^x_exp
int x_exp;
frexp(x, &x_exp);
if (geometric(b_exp - x_exp)) return x;
}
}
int main() {
devurandom = fopen("/dev/urandom", "r");
for (int i = 0; i < 100000; ++i) {
printf("%.17g\n", uniform(1.0 - 1e-15, 1.0 + 1e-15));
}
}
Here is one way of doing it.
The IEEE Std 754 double format is as follows:
[s][ e ][ f ]
where s is the sign bit (1 bit), e is the biased exponent (11 bits) and f is the fraction (52 bits).
Beware that the layout in memory will be different on little-endian machines.
For 0 < e < 2047, the number represented is
(-1)**(s) * 2**(e – 1023) * (1.f)
By setting s to 0, e to 1023 and f to 52 random bits from your bit stream, you get a random double in the interval [1.0, 2.0). This interval is unique in that it contains 2 ** 52 doubles, and these doubles are equidistant. If you then subtract 1.0 from the constructed double, you get a random double in the interval [0.0, 1.0). Moreover, the property about being equidistant is preserve.
From there you should be able to scale and translate as needed.
I'm surprised that for question this old, nobody had actual code for the best answer. User515430's answer got it right--you can take advantage of IEEE-754 double format to directly put 52 bits into a double with no math at all. But he didn't give code. So here it is, from my public domain ojrandlib:
double ojr_next_double(ojr_generator *g) {
uint64_t r = (OJR_NEXT64(g) & 0xFFFFFFFFFFFFFull) | 0x3FF0000000000000ull;
return *(double *)(&r) - 1.0;
}
NEXT64() gets a 64-bit random number. If you have a more efficient way of getting only 52 bits, use that instead.
This is easy, as long as you have an integer type with as many bits of precision as a double. For instance, an IEEE double-precision number has 53 bits of precision, so a 64-bit integer type is enough:
#include <limits.h>
double GetRandomVal(double fMin, double fMax) {
unsigned long long n ;
GetRandomBits ((char*)&n, sizeof(n)) ;
return fMin + (n * (fMax - fMin))/ULLONG_MAX ;
}
This is probably not the answer you want, but the specification here:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3225.pdf
in sections [rand.util.canonical] and [rand.dist.uni.real], contains sufficient information to implement what you want, though with slightly different syntax. It isn't easy, but it is possible. I speak from personal experience. A year ago I knew nothing about random numbers, and I was able to do it. Though it took me a while... :-)
The question is ill-posed. What does uniform distribution over floats even mean?
Taking our cue from discrepancy, one way to operationalize your question is to define that you want the distribution that minimizes the following value:
Where x is the random variable you are sampling with your GetRandomVal(double fMin, double fMax) function, and means the probability that a random x is smaller or equal to t.
And now you can go on and try to evaluate eg a dabbler's answer. (Hint all the answers that fail to use the whole precision and stick to eg 52 bits will fail this minimization criterion.)
However, if you just want to be able to generate all float bit patterns that fall into your specified range with equal possibility, even if that means that eg asking for GetRandomVal(0,1000) will create more values between 0 and 1.5 than between 1.5 and 1000, that's easy: any interval of IEEE floating point numbers when interpreted as bit patterns map easily to a very small number of intervals of unsigned int64. See eg this question. Generating equally distributed random values of unsigned int64 in any given interval is easy.
I may be misunderstanding the question, but what stops you simply sampling the next n bits from the random bit stream and converting that to a base 10 number number ranged 0 to 2^n - 1.
To get a random value in [0..1[ you could do something like:
double value = 0;
for (int i=0;i<53;i++)
value = 0.5 * (value + random_bit()); // Insert 1 random bit
// or value = ldexp(value+random_bit(),-1);
// or group several bits into one single ldexp
return value;

Converting from unsigned long long to float with round to nearest even

I need to write a function that rounds from unsigned long long to float, and the rounding should be toward nearest even.
I cannot just do a C++ type-cast, since AFAIK the standard does not specify the rounding.
I was thinking of using boost::numeric, but i could not find any useful lead after reading the documentation. Can this be done using that library?
Of course, if there is an alternative, i would be glad to use it.
Any help would be much appreciated.
EDIT: Adding an example to make things a bit clearer.
Suppose i want to convert 0xffffff7fffffffff to its floating point representation. The C++ standard permits either one of:
0x5f7fffff ~ 1.9999999*2^63
0x5f800000 = 2^64
Now if you add the restriction of round to nearest even, only the first result is acceptable.
Since you have so many bits in the source that can't be represented in the float and you can't (apparently) rely on the language's conversion, you'll have to do it yourself.
I devised a scheme that may or may not help you. Basically, there are 31 bits to represent positive numbers in a float so I pick up the 31 most significant bits in the source number. Then I save off and mask away all the lower bits. Then based on the value of the lower bits I round the "new" LSB up or down and finally use static_cast to create a float.
I left in some couts that you can remove as desired.
const unsigned long long mask_bit_count = 31;
float ull_to_float2(unsigned long long val)
{
// How many bits are needed?
int b = sizeof(unsigned long long) * CHAR_BIT - 1;
for(; b >= 0; --b)
{
if(val & (1ull << b))
{
break;
}
}
std::cout << "Need " << (b + 1) << " bits." << std::endl;
// If there are few enough significant bits, use normal cast and done.
if(b < mask_bit_count)
{
return static_cast<float>(val & ~1ull);
}
// Save off the low-order useless bits:
unsigned long long low_bits = val & ((1ull << (b - mask_bit_count)) - 1);
std::cout << "Saved low bits=" << low_bits << std::endl;
std::cout << val << "->mask->";
// Now mask away those useless low bits:
val &= ~((1ull << (b - mask_bit_count)) - 1);
std::cout << val << std::endl;
// Finally, decide how to round the new LSB:
if(low_bits > ((1ull << (b - mask_bit_count)) / 2ull))
{
std::cout << "Rounding up " << val;
// Round up.
val |= (1ull << (b - mask_bit_count));
std::cout << " to " << val << std::endl;
}
else
{
// Round down.
val &= ~(1ull << (b - mask_bit_count));
}
return static_cast<float>(val);
}
I did this in Smalltalk for arbitrary precision integer (LargeInteger), implemented and tested in Squeak/Pharo/Visualworks/Gnu Smalltalk/Dolphin Smalltalk, and even blogged about it if you can read Smalltalk code http://smallissimo.blogspot.fr/2011/09/clarifying-and-optimizing.html .
The trick for accelerating the algorithm is this one: IEEE 754 compliant FPU will round exactly the result of an inexact operation. So we can afford 1 inexact operation and let the hardware rounds correctly for us. That let us handle easily first 48 bits. But we cannot afford two inexact operations, so we sometimes have to care of the lowest bits differently...
Hope the code is documented enough:
#include <math.h>
#include <float.h>
float ull_to_float3(unsigned long long val)
{
int prec=FLT_MANT_DIG ; // 24 bits, the float precision
unsigned long long high=val>>prec; // the high bits above float precision
unsigned long long mask=(1ull<<prec) - 1 ; // 0xFFFFFFull a mask for extracting significant bits
unsigned long long tmsk=(1ull<<(prec - 1)) - 1; // 0x7FFFFFull same but tie bit
// handle trivial cases, 48 bits or less,
// let FPU apply correct rounding after exactly 1 inexact operation
if( high <= mask )
return ldexpf((float) high,prec) + (float) (val & mask);
// more than 48 bits,
// what scaling s is needed to isolate highest 48 bits of val?
int s = 0;
for( ; high > mask ; high >>= 1) ++s;
// high now contains highest 24 bits
float f_high = ldexpf( (float) high , prec + s );
// store next 24 bits in mid
unsigned long long mid = (val >> s) & mask;
// care of rare case when trailing low bits can change the rounding:
// can mid bits be a case of perfect tie or perfect zero?
if( (mid & tmsk) == 0ull )
{
// if low bits are zero, mid is either an exact tie or an exact zero
// else just increment mid to distinguish from such case
unsigned long long low = val & ((1ull << s) - 1);
if(low > 0ull) mid++;
}
return f_high + ldexpf( (float) mid , s );
}
Bonus: this code should round according to your FPU rounding mode whatever it may be, since we implicitely used the FPU to perform rounding with + operation.
However, beware of aggressive optimizations in standards < C99, who knows when the compiler will use extended precision... (unless you force something like -ffloat-store).
If you always want to round to nearest even, whatever the current rounding mode, then you'll have to increment high bits when:
mid bits > tie, where tie=1ull<<(prec-1);
mid bits == tie and (low bits > 0 or high bits is odd).
EDIT:
If you stick to round-to-nearest-even tie breaking, then another solution is to use Shewchuck EXPANSION-SUM of non adjacent parts (fhigh,flow) and (fmid) see http://www-2.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-arithmetic.ps :
#include <math.h>
#include <float.h>
float ull_to_float4(unsigned long long val)
{
int prec=FLT_MANT_DIG ; // 24 bits, the float precision
unsigned long long mask=(1ull<<prec) - 1 ; // 0xFFFFFFull a mask for extracting significant bits
unsigned long long high=val>>(2*prec); // the high bits
unsigned long long mid=(val>>prec) & mask; // the mid bits
unsigned long long low=val & mask; // the low bits
float fhigh = ldexpf((float) high,2*prec);
float fmid = ldexpf((float) mid,prec);
float flow = (float) low;
float sum1 = fmid + flow;
float residue1 = flow - (sum1 - fmid);
float sum2 = fhigh + sum1;
float residue2 = sum1 - (sum2 - fhigh);
return (residue1 + residue2) + sum2;
}
This makes a branch-free algorithm with a bit more ops. It may work with other rounding modes, but I let you analyze the paper to make sure...
What is possible between between 8-byte integers and the float format is straightforward to explain but less so to implement!
The next paragraph concerns what is representable in 8 byte signed integers.
All positive integers between 1 (2^0) and 16777215 (2^24-1) are exactly representable in iEEE754 single precision (float). Or, to be precise, all numbers between 2^0 and 2^24-2^0 in increments of 2^0. The next range of exactly representable positive integers is 2^1 to 2^25-2^1 in increments of 2^1 and so on up to 2^39 to 2^63-2^39 in increments of 2^39.
Unsigned 8-byte integer values can be expressed up to 2^64-2^40 in increments of 2^40.
The single precison format doesn't stop here but goes on all the way up to the range 2^103 to 2^127-2^103 in increments of 2^103.
For 4-byte integers (long) the highest float range is 2^7 to 2^31-2^7 in 2^7 increments.
On the x86 architecture the largest integer type supported by the floating point instruction set is the 8 byte signed integer. 2^64-1 cannot be loaded by conventional means.
This means that for a given range increment expressed as "2^i where i is an integer >0" all integers that end with the bit pattern 0x1 up to 2^i-1 will not be exactly representable within that range in a float
This means that what you call rounding upwards is actually dependent on what range you are working in. It is of no use to try to round up by 1 (2^0) or 16 (2^4) if the granularity of the range you are in is 2^19.
An additional consequence of what you propose to do (rounding 2^63-1 to 2^63) could result in an (long integer format) overflow if you attempt the following conversion: longlong_int=(long long) ((float) 2^63).
Check out this small program I wrote (in C) which should help illustrate what is possible and what isn't.
int main (void)
{
__int64 basel=1,baseh=16777215,src,dst,j;
float cnvl,cnvh,range;
int i=0;
while (i<40)
{
src=basel<<i;
cnvl=(float) src;
dst=(__int64) cnvl; /* compare dst with basel */
src=baseh<<i;
cnvh=(float) src;
dst=(__int64) cnvh; /* compare dst with baseh */
j=basel;
while (j<=baseh)
{
range=(float) j;
dst=(__int64) range;
if (j!=dst) dst/=0;
j+=basel;
}
++i;
}
return i;
}
This program shows the representable integer value ranges. There is overlap beteen them: for example 2^5 is representable in all ranges with a lower boundary 2^b where 1=