I have stumbled upon an interesting case of comparing (==, !=) float types.
I encountered this problem while porting my own software from windows to linux. It's a bit of a bummer. The relevant code is the following:
template<class T> class PCMVector2 {
public:
T x, y;
public:
bool operator == ( const PCMVector2<T>& a ) const {
return x == a.x && y == a.y;
}
bool operator != ( const PCMVector2<T>& a ) const {
return x != a.x || y != a.y;
}
// Mutable normalization
PCMVector2<T>& Normalize() {
const T l = 1.0f / Length();
x *= l;
y *= l;
return *this;
}
// Immutable normalization
const PCMVector2<T> Normalized() {
const T l = 1.0f / Length();
return PCMVector2<T>(x*l,y*l);
}
// Vector length
T Length() const { return sqrt(x*x+y*y); }
};
I cleverly designed a unit test functions which check all available functionality regarding those classes, before porting to linux. And, in contrast to msvc, the g++ doesn't complain, but gives incorrect results at runtime.
I was stumped, so I did some additional logging, type-puns, memcmp's, etc. and they all showed that memory is 1:1 the same! Anyone has any ideas about this?
My flags are: -Wall -O2 -j2
Thanks in advance.
EDIT2: The failed test case is:
vec2f v1 = vec2f(2.0f,3.0f);
v1.Normalize(); // mutable normalization
if( v1 != vec2f(2.0f,3.0f).Normalized() ) //immutable normalization
// report failure
Note: Both normalizations are the same, and yield same results (according to memcmp).
RESOLUTION: Turns out that you should never trust the compiler about floating numbers! No matter how sure you are about the memory you compare. Once data goes to the registers, it can change, and you have no control over it. After some digging regarding registers, I found this neat source of information. Hope it's useful to someone in the future.
Floating point CPU registers can be larger than the floating point type you're working with. This is especially true with float which is typically only 32 bits. A calculation will be computed using all the bits, then the result will be rounded to the nearest representable value before being stored in memory.
Depending on inlining and compiler optimization flags, it is possible that the generated code may compare one value from memory with another one from a register. Those may compare as unequal, even though their representation in memory will be bit-for-bit identical.
This is only one of the many reasons why comparing floating-point values for equality is not recommended. Especially when, as in your case, it appears to work some of the time.
Related
I scanned my code with PVS Studio analyzer and I am confused on why this error and how to fix this.
V550 An odd precise comparison: * dest == value. It's probably better to use a comparison with defined precision: fabs(A - B) < Epsilon.
bool PipelineCache::SetShadowRegister(float* dest, uint32_t register_name) {
float value = register_file_->values[register_name].f32;
if (*dest == value) {
return false;
}
*dest = value;
return true;
}
I am guessing to change code like this:
bool PipelineCache::SetShadowRegister(float* dest, float* epsilon uint32_t register_name) {
float value = register_file_->values[register_name].f32;
return fabs(dest - value) < epsilon;
}
Whoever's wondering, we're talking about this code.
I'll try to explain what PVS studio developers were trying to achieve with this message. Citing their reference about V550:
Consider this sample:
double a = 0.5;
if (a == 0.5) //OK
x++;
double b = sin(M_PI / 6.0);
if (b == 0.5) //ERROR
x++;
The first comparison 'a == 0.5' is true. The second comparison 'b == 0.5' may be both true and false. The result of the 'b == 0.5' expression depends upon the processor, compiler's version and settings being used. For instance, the 'b' variable's value was 0.49999999999999994 when we used the Visual C++ 2010 compiler.
What they are trying to say, is, comparing floating point numbers is tricky. If you just assign your floating point number, store it and move it around the memory to later compare with itself in this function - feel free to dismiss this error message.
If you are looking to perform some bit-representation check (which I honestly think you are doing), see below.
If you are performing some massive calculations on floating point numbers, and you are a game developer, calculating the coordinates of the enemy battlecruisers - this warning is one of your best friends.
Anyway, let's return to your case. As it usually happens with PVS-Studio, they did not see the exact error, but they pointed you to the right direction. You actually want to compare two float values, but you are doing it wrong. The thing is, if both float numbers you are comparing contain NaN (even in the same bit representation), you'll get *dest != value, and your code will not work in the way you want.
In this scenario, you better reinterpret the memory under float * as uint32_t (or whatever integer type has the same size as float on your target) and compare them instead.
For example, in your particular case, register_file_->values[register_name] is of type xe::gpu::RegisterFile::RegisterValue, which already supports uint32_t representation.
As a side effect, this will draw the warning away :)
The following code snippet is scattered all over the web and seems to be used in multiple different projects with very little changes:
union Float_t {
Float_t(float num = 0.0f) : f(num) {}
// Portable extraction of components.
bool Negative() const { return (i >> 31) != 0; }
int RawMantissa() const { return i & ((1 << 23) - 1); }
int RawExponent() const { return (i >> 23) & 0xFF; }
int i;
float f;
};
inline bool AlmostEqualUlpsAndAbs(float A, float B, float maxDiff, int maxUlpsDiff)
{
// Check if the numbers are really close -- needed
// when comparing numbers near zero.
float absDiff = std::fabs(A - B);
if (absDiff <= maxDiff)
return true;
Float_t uA(A);
Float_t uB(B);
// Different signs means they do not match.
if (uA.Negative() != uB.Negative())
return false;
// Find the difference in ULPs.
return (std::abs(uA.i - uB.i) <= maxUlpsDiff);
}
See, for example here or here or here.
However, I don't understand what is going on here. To my (maybe naive) understanding, the floating-point member variable f is initialized in the constructor, but the integer member i is not.
I'm not terribly familiar with the binary operators that are used here, but I fail to understand how accesses of uA.i and uB.i produce anything but random numbers, given that no line in the code actually connects the values of f and i in any meaningful way.
If somebody could enlighten my on why (and how) exactly this code produces the desired result, I would be very delighted!
A lot of Undefined Behaviour are being exploited here. First assumption is that fields of union can be accessed in place of each other, which is, in itself, UB. Furthermore, coder assumes that: sizeof(int) == sizeof(float), that floats have a given length of mantissa and exponent, that all union members are aligned to zero, that the binary representation of float coincides with the binary representation with int in a very specific way. In short, this will work as long as you're on x86, have specific int and float types and you say a prayer at every sunrise and sunset.
What you probably didn't note is that this is a union, therefore int i and float f is usually aligned in a specific manner in a common memory array by most compilers. This is, in general, still UB and you can't even safely assume that the same physical bits of memory will be used without restricting yourself to a specific compiler and a specific architecture. All that's guaranteed is, the address of both members will be the same (but there might be alignment and/or typedness issues). Assuming that your compiler uses the same physical bits (which is by no means guaranteed by standard) and they both start at offset 0 and have the same size, then i will represent the binary storage format of f.. as long as nothing changes in your architecture. Word of advice? Do not use it until you don't have to. Stick to floating point operations for AlmostEquals(), you can implement it like that. It's the very final pass of optimization when we consider these specialities and we usually do it in a separate branch, you shouldn't plan your code around it.
I have an image processing program which uses floating point calculations. However, I need to port it to a processor which does not have floating point support in it. So, I have to change the program to use fixed point calculations. For that I need proper scaling of those floating point numbers, for which I need to know the range of all values, including intermediate values of the floating point calculations.
Is there a method where I just run the program and it automatically give me the range of all the floating point calculations in the program? Trying to figure out the ranges manually would be too cumbersome, so if there is some tool for doing it, that would be awesome!
You could use some "measuring" replacement for your floating type, along these lines (live example):
template<typename T>
class foo
{
T val;
using lim = std::numeric_limits<int>;
static int& min_val() { static int e = lim::max(); return e; }
static int& max_val() { static int e = lim::min(); return e; }
static void sync_min(T e) { if (e < min_val()) min_val() = int(e); }
static void sync_max(T e) { if (e > max_val()) max_val() = int(e); }
static void sync(T v)
{
v = std::abs(v);
T e = v == 0 ? T(1) : std::log10(v);
sync_min(std::floor(e)); sync_max(std::ceil(e));
}
public:
foo(T v = T()) : val(v) { sync(v); }
foo& operator=(T v) { val = v; sync(v); return *this; }
template<typename U> foo(U v) : foo(T(v)) {}
template<typename U> foo& operator=(U v) { return *this = T(v); }
operator T&() { return val; }
operator const T&() const { return val; }
static int min() { return min_val(); }
static int max() { return max_val(); }
};
to be used like
int main ()
{
using F = foo<float>;
F x;
for (F e = -10.2; e <= 30.4; e += .2)
x = std::pow(10, e);
std::cout << F::min() << " " << F::max() << std::endl; // -11 31
}
This means you need to define an alias (say, Float) for your floating type (float or double) and use it consistently throughout your program. This may be inconvenient but it may prove beneficial eventually (because then your program is more generic). If your code is already templated on the floating type, even better.
After this parametrization, you can switch your program to "measuring" or "release" mode by defining Float to be either foo<T> or T, where T is your float or double.
The good thing is that you don't need external tools, your own code carries out the measurements. The bad thing is that, as currently designed, it won't catch all intermediate results. You would have to define all (e.g. arithmetic) operators on foo for this. This can be done but needs some more work.
It is not true that you cannot use floating point code on hardware that does not support floating point - the compiler will provide software routines to perform floating point operations - they just may be rather slow - but if it is fast enough for your application , that is the path of least resistance.
It is probably simplest to implement a fixed point data type class and have its member functions detect over/underflow as a debug option (because the checking will otherwise slow your code).
I suggest you look at Anthony Williams' fixed-Point math C++ library. It is in C++ and defines a fixed class with extensive function and operator overloading, so it can largely be used simply by replacing float or double in your existing code with fixed. It uses int64_t as the underlying integer data type, with 34 integer bits and 28 fractional bits (34Q28), so is good for about 8 decimal places and a wider range than int32_t.
It does not have the under/overflow checking I suggested, but it is a good starting point for you to add your own.
On 32bit ARM this library performs about 5 times faster than software-floating point and is comparable in performance to ARM's VFP unit for C code.
Note that the sqrt() function in this library has poor precision performance for very small values as it looses lower-order bits in intermediate calculations that can be preserved. It can be improved by replacing it with the code the version I presented in this question.
For self-contained C programs, you can use Frama-C's value analysis to obtain ranges for the floating-point variables, for instance h below:
And variable g computed from h:
There is a specification language to describe the ranges of the inputs (information without which it is difficult to say anything informative). In the example above, I used that language to specify what function float_interval was expected to do:
/*# ensures \is_finite(\result) && l <= \result <= u ; */
float float_interval(float l, float u);
Frama-C is easiest to install on Linux, with Debian and Ubuntu binary packages for a recent (but usually not the latest) version available from within the distribution.
If you could post your code, it would help telling if if this approach is realistic. If your code is C++, for instance (your question does not say, it is tagged with several language tags), then the current version of Frama-C will be no help as it only accepts C programs.
I am writing a numerical code that needs to make extensive (and possibly fast) comparisons among double precision numbers. My solution to compare two numbers A and B involves shifting A to the left (or right) by an epsilon and checking whether the result is bigger (or smaller) than B. If so, the two doubles are the same. (Extra coding needs to be done for negative or zero numbers).
This is the comparing function:
#define S_
inline double s_l (double x){
if(x>0){return 0.999999999*x;}
else if(x<0){return 1.00000001*x;}
else {return x-0.000000000001;}
}
inline double s_r (double x){
if(x>0){return 1.00000001*x;}
else if(x<0){return 0.999999999*x;}
else{return x+0.000000000001;}
}
inline bool s_equal (double x,double y){
if(x==y){return true;}
else if(x<y && s_r(x)>y){return true;}
else if(x>y && s_l(x)<y){return true;}
else{return false;}
}
#endif
Since this is part of a MonteCarlo algorithm and s_equal(x,y) is called millions of times, I wonder if there is any better or faster way to code this, understandable at a simple level.
I do something like abs( (x-y)/x ) < 1.0e-10 .
You need to divide by x in case both values are huge or tiny.
I was surprised to find a significant speedup by avoiding all the double-precision math:
#define S_L(x) (x)+((x)<0?1024:-1024)
#define S_R(x) (x)+((x)<0?-1024:1024)
#define S_EQUAL(x,y) (S_L(x)<(y) && S_R(x)>(y))
double foo;
double bar;
long *pfoo;
long *pbar;
pfoo = (long*)&foo;
pbar = (long*)&bar;
double result1 = S_R(*pfoo);
double result2 = S_L(*pbar);
bool result3 = S_EQUAL(*pfoo, *pbar);
(In testing, I operated on randomly-generated doubles between -1M and 1M, executing each operation 100M times with different input for each iteration. Each operation was timed in an independent loop, comparing system times - not wall times. Including loop overhead and generation of random numbers, this solution was about 25% faster.)
A word of warning: there are lots of dependencies here on your hardware, the range of your doubles, the behavior of your optimizer, etc., etc. Such pitfalls are commonplace when you start second-guessing your compiler. I was shocked to see how much faster this was for me, since I'd always been told that integer and floating point units are kept so separate on hardware that the transport of bits from one to the other always requires a hardware memory operation. Who knows how well this will work for you.
You will likely have to play with the magic numbers a bit (the 1024s) to get it to do about what you want it to - if it's even possible.
If you're using the C++11, then you could use the new math library functions, such as:
bool isgreater(float x, float y)
More documentation on std::isgreater can be had here.
Otherwise, there's always is_equal in boost. Also, SO already has a bunch of related (not sure if same) questions such as the ones here, here and here.
I'm currently implementing a hash table in C++ and I'm trying to make a hash function for floats...
I was going to treat floats as integers by padding the decimal numbers, but then I realized that I would probably reach the overflow with big numbers...
Is there a good way to hash floats?
You don't have to give me the function directly, but I'd like to see/understand different concepts...
Notes:
I don't need it to be really fast, just evenly distributed if possible.
I've read that floats should not be hashed because of the speed of computation, can someone confirm/explain this and give me other reasons why floats should not be hashed? I don't really understand why (besides the speed)
It depends on the application but most of time floats should not be hashed because hashing is used for fast lookup for exact matches and most floats are the result of calculations that produce a float which is only an approximation to the correct answer. The usually way to check for floating equality is to check if it is within some delta (in absolute value) of the correct answer. This type of check does not lend itself to hashed lookup tables.
EDIT:
Normally, because of rounding errors and inherent limitations of floating point arithmetic, if you expect that floating point numbers a and b should be equal to each other because the math says so, you need to pick some relatively small delta > 0, and then you declare a and b to be equal if abs(a-b) < delta, where abs is the absolute value function. For more detail, see this article.
Here is a small example that demonstrates the problem:
float x = 1.0f;
x = x / 41;
x = x * 41;
if (x != 1.0f)
{
std::cout << "ooops...\n";
}
Depending on your platform, compiler and optimization levels, this may print ooops... to your screen, meaning that the mathematical equation x / y * y = x does not necessarily hold on your computer.
There are cases where floating point arithmetic produces exact results, e.g. reasonably sized integers and rationals with power-of-2 denominators.
If your hash function did the following you'd get some degree of fuzziness on the hash lookup
unsigned int Hash( float f )
{
unsigned int ui;
memcpy( &ui, &f, sizeof( float ) );
return ui & 0xfffff000;
}
This way you'll mask off the 12 least significant bits allowing for a degree of uncertainty ... It really depends on yout application however.
You can use the std hash, it's not bad:
std::size_t myHash = std::cout << std::hash<float>{}(myFloat);
unsigned hash(float x)
{
union
{
float f;
unsigned u;
};
f = x;
return u;
}
Technically undefined behavior, but most compilers support this. Alternative solution:
unsigned hash(float x)
{
return (unsigned&)x;
}
Both solutions depend on the endianness of your machine, so for example on x86 and SPARC, they will produce different results. If that doesn't bother you, just use one of these solutions.
You can of course represent a float as an int type of the same size to hash it, however this naive approach has some pitfalls you need to be careful of...
Simply converting to a binary representation is error prone since values which are equal wont necessarily have the same binary representation.
An obvious case: -0.0 wont match 0.0 for example. *
Further, simply converting to an int of the same size wont give very even distribution, which is often important (implementing a hash/set that uses buckets for example).
Suggested steps for implementation:
filter out non-finite cases (nan, inf) and (0.0, -0.0 whether you need to do this explicitly or not depends on the method used).
convert to an int of the same size(that is - use a union for example to represent the float as an int, not simply cast to an int).
re-distribute the bits, (intentionally vague here!), this is basically a speed vs quality tradeoff. But if you have many values in a small range you probably don't want them to in a similar range too.
*: You may wan't to check for (nan and -nan) too. How to handle those exactly depends on your use case (you may want to ignore sign for all nan's as CPython does).
Python's _Py_HashDouble is a good reference for how you might hash a float, in production code (ignore the -1 check at the end, since that's a special value for Python).
If you're interested, I just made a hash function that uses floating point and can hash floats. It also passes SMHasher ( which is the main bias-test for non-crypto hash functions ). It's a lot slower than normal non-cryptographic hash functions due to the float calculations.
I'm not sure if tifuhash will become useful for all applications, but it's interesting to see a simple floating point function pass both PractRand and SMHasher.
The main state update function is very simple, and looks like:
function q( state, val, numerator, denominator ) {
// Continued Fraction mixed with Egyptian fraction "Continued Egyptian Fraction"
// with denominator = val + pos / state[1]
state[0] += numerator / denominator;
state[0] = 1.0 / state[0];
// Standard Continued Fraction with a_i = val, b_i = (a_i-1) + i + 1
state[1] += val;
state[1] = numerator / state[1];
}
Anyway, you can get it on npm
Or you can check out the github
Using is simple:
const tifu = require('tifuhash');
const message = 'The medium is the message.';
const number = 333333333;
const float = Math.PI;
console.log( tifu.hash( message ),
tifu.hash( number ),
tifu.hash( float ),
tifu.hash( ) );
There's a demo of some hashes on runkit here https://runkit.com/593a239c56ebfd0012d15fc9/593e4d7014d66100120ecdb9
Side note: I think that in future using floating point,possibly big arrays of floating point calculations, could be a useful way to make more computationally-demanding hash functions in future. A weird side effect I discovered of using floating point is that the hashes are target dependent, and I surmise maybe they could be use to fingerprint the platforms they were calculated on.
Because of the IEEE byte ordering the Java Float.hashCode() and Double.hashCode() do not give good results. This problem is wellknown and can be adressed by this scrambler:
class HashScrambler {
/**
* https://sites.google.com/site/murmurhash/
*/
static int murmur(int x) {
x ^= x >> 13;
x *= 0x5bd1e995;
return x ^ (x >> 15);
}
}
You then get a good hash function, which also allows you to use Float and Double in hash tables. But you need to write your own hash table that allows a custom hash function.
Since in a hash table you need also test for equality, you need an exact equality to make it work. Maybe the later is what President James K. Polk intends to adress?