I use the two following C++ compilers:
cl.exe : Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24210 for x86
g++ : g++ (Ubuntu 5.2.1-22ubuntu2) 5.2.1 20151010
When using the built-in sine function, I get different results. This is not critical, but sometimes results are too significants for my use. Here is an example with a 'hard-coded' value:
printf("%f\n", sin(5451939907183506432.0));
Result with cl.exe:
0.528463
Result with g++:
0.522491
I know that g++'s result is more accurate and that I could use an additional library to get this same result, but that's not my point here. I would really understand what happens here: why is cl.exe that wrong?
Funny thing, if I apply a modulo of (2 * pi) on the param, then I get the same result than g++...
[EDIT] Just because my example looks crazy for some of you: this is a part of a pseudorandom number generator. It is not important to know if the result of the sine is accurate or not: we just need it to give some result.
You have a 19-digit literal, but double usually has 15-17 digit precision. As a result, you can get a small relative error (when converting to double), but big enough (in the context of sine calculation) absolute error.
Actually, different implementations of the standard library have differences in treating such large numbers. For example, in my environment, if we execute
std::cout << std::fixed << 5451939907183506432.0;
g++ result would be 5451939907183506432.000000
cl result would be 5451939907183506400.000000
The difference is because versions of cl earlier than 19 have a formatting algorithm that uses only a limited number of digits and fills the remaining decimal places with zero.
Furthermore, let's look at this code:
double a[1000];
for (int i = 0; i < 1000; ++i) {
a[i] = sin(5451939907183506432.0);
}
double d = sin(5451939907183506432.0);
cout << a[500] << endl;
cout << d << endl;
When executed with my x86 VC++ compiler the output is:
0.522491
0.528463
It appears that when filling the array sin is compiled to the call of __vdecl_sin2, and when there is a single operation, it is compiled to the call of __libm_sse2_sin_precise (with /fp:precise).
In my opinion, your number is too large for sin calculation to expect the same behavior from different compilers and to expect the correct behavior in general.
I think Sam's comment is closest to the mark. Whereas you're using a recentish version of GCC/glibc, which implements sin() in software (calculated at compile time for the literal in question), cl.exe for x86 likely uses the fsin instruction. The latter can be very imprecise, as described in the Random ASCII blog post, "Intel Underestimates Error Bounds by 1.3 quintillion".
Part of the problem with your example in particular is that Intel uses an imprecise approximation of pi when doing range reduction:
When doing range reduction from double-precision (53-bit mantissa) pi the results will have about 13 bits of precision (66 minus 53), for an error of up to 2^40 ULPs (53 minus 13).
According to cppreference:
The result may have little or no significance if the magnitude of arg is large
(until C++11)
It's possible that this is the cause of the problem, in which case you will want to manually do the modulo so that arg is not large.
Related
I was benchmarking some essential routines by executing cycles such as:
float *src, *dst;
for (int i=0; i<cnt; i++) dst[i] = round(src[i]);
All with AVX2 target, newest CLANG. Interestingly floor(x), ceil(x), int(x)... all seem fast. But round(x) seems exremely slow and looking into disassembly there's some weird spaghetti code instead of the newer SSE or AVX versions. Even when blocking the ability to vectorize the loops by introducing some dependency, round is like 10x slower. For floor etc. the generated code uses vroundss, for round there's the spaghetti code... Any ideas?
Edit: I'm using -ffast-math, -mfpmath=sse, -fno-math-errno, -O3, -std=c++17, -march=core-avx2 -mavx2 -mfma
The problem is that none of the SSE rounding modes specify the correct rounding for round:
These functions round x to the nearest integer, but round halfway cases away from zero
(regardless of the current rounding direction, see fenv(3)), instead of to the nearest
even integer like rint(3).
If you want faster code, you could try testing rint instead of round, as that specifies a rounding mode that SSE does support.
One thing to note is that an expression like floor(x + 0.5), while not having the exact same semantics that round(x) does, is a valid substitute in almost all use cases, and I doubt it is anywhere near 10x slower than floor(x).
I've come across a surprising oddity with floating point reciprocals that only seems to occur sometimes.
Why is it that, at unexpectable times, given two floats,
float a = ..., b = ...;
testing their equivalence at one time shows they're equal,
cout << (a == b ? "T" : "F") << endl; // prints T
yet, when adjusting the same line to test the equality of the reciprocals, and running the program no differently, they are suddenly not equal:
cout << (1/a == 1/b ? "T" : "F") << endl; // prints F
Here, a and b are not NaN, they're neither -INF nor +INF, and they're also not 0 (they're typically in the 3000 range with decimal values). I also noticed that, when compiling and running with both cout expressions somehow they then both print T.
Why would this be the case? I am very familiar with floating point numbers and related precision issues, but I would expect an operation on the same value to generate the same result. Or can division at times be a guesstimation/approximation instruction on certain CPUs?
Any clarification would be very appreciated.
EDIT:
As a side note, since I am starting to think this is something my compiler is doing, I am using MinGW32 GCC version 4.8.1. For targeting C++14 I'm using the flag -std=c++1y, as this version doesn't seem to support the -std=c++14 flag.
EDIT
Could this be a compiler error? I've determined that this is an issue in the GCC 4.8.1 compiler that occurs only when compiling with optimizations (-O2 or -O3).
I'm using the awesome Eigen3 library to write a MATLAB MEX file. But I am experiencing some accuracy issues (compared to MATLAB), even when using long double.
The most critical computation seems to be the one where I compute a probability according to the normal distribution.
Here is the code snippet:
p.fill( 1/( M_PIl * sigma * sigma ) );
p.array() *= ( - 0.5/pow( sigma, 2.0 ) * ( mu.array() - x.array() ).array().square() ).array().exp();
where x, p and mu are Eigen::Matrix< long double, Dynamic, 1 >. Usually these vectors have a length of 3000.
What are possible steps I can take to get the maximum possible precision?
What are the correct GCC compiler flags I can use to force 80 bit precision wherever possible?
P.S: I compile the C++ code (in MATLAB with MEX) with gcc 4.9 and my linux reports the following available instruction sets: Intel MMX, Intel SSE, Intel SSE2, Intel SSE3, Intel SSE4
Edit:
I tried what #Avi Ginsburg suggested below and compiled it using the following command:
mex -g -largeArrayDims '-I/usr/include/eigen3' CXXFLAGS='-DEIGEN_DONT_VECTORIZE -std=c++11 -fPIC' test.cpp
with double and long double and each of these options gives me the same error with respect to the solution from MATLAB.
I'm hazarding a guess here. You are using SSE instructions with your array calculations, most notably, ...array().exp(). I'm pretty sure there is no extended precision with SSE, hence the differences between MATLAB and Eigen.
By default Eigen uses a faster but slightly less accurate implementation of several mathematical functions, including the exponential. You can explicitly disable these optimizations by compiling with the -DEIGEN_FAST_MATH=0 option.
If you use gcc as your compiler also make sure that you don't use the -Ofast or -ffast-math options, as these can result in reduced precision.
If you want to compute the probability density of a (1 dimensional) normal distribution, the factor at the beginning should be 1/std::sqrt( 2* M_PIl * sigma * sigma ).
Also, the p.fill() at the beginning of your snippet is inefficient. Just write this in one line:
p = (1/std::sqrt(2*M_PIl*sigma*sigma)) *
( -0.5/(sigma*sigma) * (mu-x).array().square() ).exp();
N.B.: If you are only performing element-wise operations on your arrays, consider declaring them as Eigen::Array<...> instead of Eigen::Matrix<...>. The template parameters are the same, also the binary layout is the same, but you don't need to write .array() every time you want to make element-wise operations.
Immediately warn you that this is a difficult task.
There is a test. The test was the result of parsing a large problem to a bug in which we encountered at work. Construction __ attribute__((noinline)) prohibits the compiler to do the substitution function (for optimizations to something there not imploded). This is the easiest way to optimize guaranteed not to kill an interesting situation.
#include <stdio.h>
double d = 5436277361664796672.000000;
long long ll = 5436277361664796253LL;
int __attribute__((noinline))
func1 (void)
{
double d1 = (double)ll;
if (d > d1)
return 1;
else
return 0;
}
int __attribute__((noinline))
func2 (void)
{
if (d > (double)ll)
return 1;
else
return 0;
}
int
main (void)
{
printf ("%d %d\n", func1(), func2());
return 0;
}
I ran this test on intel and sparc. Gcc used in a mode with optimizations and without optimizations. Obtained the following results:
sparc: "gcc" printed "0 0"
sparc: "gcc -O2" printed "0 0"
intel: "gcc" printed "0 1"
intel: "gcc -O2" printed "1 1"
What is the cause differences? Anyway in the analysis situation would be useful to be able to repeat it all myself, but, of course, almost no one has the possibility to run this code on sparc. Instead sparc can try to run under Windows using microsoft or borland C compiler. I do not know what they will be given the results, but in any case something does not match with anything (because we see three different results)
Edit 1
_attribute_ ((noinline)) - an extension of the compiler gcc (forgot to write about it). Therefore VisualStudio can not compile it.
I note that the declaration of the double constant has 19 significant figures which is more precision than can be represented by a IEEE double (which allows 15 to 17 significant figures). So d cannot hold 5436277361664796672.000000 exactly.
The two constant definition strings become different at the 16th digit, so you are in the region where the inaccuracies in the double are of the same magnitude as the difference between these two numbers. Hence the comparison cannot be relied upon.
I do not know if the C++ standard specifies what happens when an over-precise string is converted to a double, but I would not be surprised if the exact result was either undefined or implementation-dependent.
Seems solved the problem. In general, all written correctly. But actually works correctly sparc version. Because standard to convert int64-> float64 must be a loss of precision. And in the code when you convert (for intel) int64-> float80 loss occurs. Ie intel-based code works with higher accuracy , but it is in contradiction with the standard.
Perhaps it is some sort of agreement for the platform Intel, which is permissible by default to work this way. Surely there are some options on which the code runs in strict accordance with the standard (but becomes slower)
The Intel C++ compiler provides two options for controlling floating point:
-fp-speculation (fast/safe/strict/off)
-fp-model (precise/fast/strict and source/double/extended)
I think I understand what fp-model does. But what is fp-speculation and how does it relate to fp-model? I have yet to find any intel doc which explains this!
-fp-model influences how floating-point computations are carried out, and can change the numeric result (by licensing unsafe optimizations or by changing the precision at which intermediate results are evaluated).
-fp-speculation does not change the numerical results, but can effect what floating-point flags are raised by an operation (or what traps are taken if floating-point traps are enabled). 99.99% of programmers don't need care about these things, so you can probably run with the default and not worry about it.
Here's a concrete example; suppose you have the following function:
double foo(double x) {
// lots of computation
if (x >= 0) return sqrt(x);
else return x;
}
sqrt is, relatively speaking, slow. It would be nice to hoist the computation of sqrt(x) like this:
double foo(double x) {
const double sqrtx = sqrt(x);
// lots of computation
if (x >= 0) return sqrtx;
else return x;
}
By doing this, we allow the computation of sqrt to proceed simultaneously with other computations, reducing the latency of our function. However, there's a problem; if x is negative, then sqrt(x) raises the invalid flag. In the original program, this could never happen, because sqrt(x) was only computed if x was non-negative. In the modified program, sqrt(x) is computed unconditionally. Thus, if x is negative, the modified program raises the invalid flag, whereas the original program did not.
The -fp-speculation flag gives you a way to tell the compiler whether or not you care about these cases, so it knows whether or not it is licensed to make such transformations.
Out of order execution and speculative execution can result in extraneous exceptions or raise exceptions at the wrong time.
If that matters to you, you can use the fp-speculation option to control speculation of floating-point instructions.
For (a little bit) more information: http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/fortran/lin/compiler_f/copts/common_options/option_fp_speculation.htm
On Windows OS:
1.Intel compiler floating calculation 32 bit application vs 64 bit application , same code Can give to you different result!!!! No matter what flag you choose:)!!!!
2.Visual studio compiler floating calculation 32 bit vs 64 bit application , same code output same result.