The question is in the title. It seems that the software I deliver to my customer has varying behaviors depending on wether some parameters passed as integers or as floats. I build a DLL for my customer with MinGW and he integrates it in his Visual Studio project, which uses some other compiler (no idea which, I guess the standard one for VS).
Could it be that floats are represented differently by him than by me ?
Thanks for the heads up,
Charles
Yes, floating point representation is compiler dependent.
In theory you can use std::numeric_limits to determine the main aspects of the representation, such as whether it's IEEE 754, or whether it's binary or decimal.
In practice you can't rely on that except for the memory layout, because with a main compiler, g++, the semantics of floating point operations are influenced strongly by the optimization options (e.g., whether NaN compares equal to itself or not).
I.e. in practice it's not only compiler dependent but also option-dependent.
Happily compilers for a given platform will generally conform to that platform's standard memory layout for floating point, e.g. IEEE 754 in Windows (the standard originated on the PC platform). And so floating point values should in general work OK when exchanged between g++ and Visual C++, say. One exception is that with g++ long double maps to 80-bit IEEE 754, while with Visual C++ it maps to ordinary double, i.e. 64-bit, and that could conceivably be what makes trouble for you.
Related
I intend to use half-precision floating-point in my code but I am not able to figure out how to declare them. For Example, I want to do something like the following-
fp16 a_fp16;
bfloat a_bfloat;
However, the compiler does not seem to know these types (fp16 and bfloat are just dummy types, for demonstration purposes)
I remember reading that bfloat support was added into GCC-10, but I am not able to find it in the manual.I am especially interested in bfloat floating numbers
Additional Questions -
FP16 now has hardware support on Intel / AMD support as today? I think native hardware support was added since Ivy Bridge itself. (https://scicomp.stackexchange.com/questions/35187/is-half-precision-supported-by-modern-architecture)
I wanted to confirm whether using FP16 will indeed increase FLOPs. I remember reading somewhere that all arithmetic operations on fp16 are internally converted to fp32 first, and only affect cache footprint and bandwidth.
SIMD intrinsic support for half precision float, especially bfloat(I am aware of intrinsics like _mm256_mul_ph, but not sure how to pass the 16bit FP datatype, would really appreciate if someone could highlight this too)
Are these types added to Intel Compilers as well ?
PS - Related Post - Half-precision floating-point arithmetic on Intel chips , but it does not cover on declaring half precision floating point numbers.
TIA
Neither C++ nor C language has arithmetic types for half floats.
The GCC compiler supports half floats as a language extension. Quote from the documentation:
On x86 targets with SSE2 enabled, GCC supports half-precision (16-bit) floating point via the _Float16 type. For C++, x86 provides a builtin type named _Float16 which contains same data format as C.
...
On x86 targets with SSE2 enabled, without -mavx512fp16, all operations will be emulated by software emulation and the float instructions. The default behavior for FLT_EVAL_METHOD is to keep the intermediate result of the operation as 32-bit precision. This may lead to inconsistent behavior between software emulation and AVX512-FP16 instructions. Using -fexcess-precision=16 will force round back after each operation.
Using -mavx512fp16 will generate AVX512-FP16 instructions instead of software emulation. The default behavior of FLT_EVAL_METHOD is to round after each operation. The same is true with -fexcess-precision=standard and -mfpmath=sse. If there is no -mfpmath=sse, -fexcess-precision=standard alone does the same thing as before, It is useful for code that does not have _Float16 and runs on the x87 FPU.
I'm asking this question because I'm trying to achieve bitwise (hash) equality between Visual Studio 2017 (cl.exe) and gcc 5.4.0. The problematic function makes use of sin() and cos(). All variables are double, and FMAD is also relevant.
I've been reading extensively on SO and the web about floating point determinism, reproducibility, and lock-stock MP game design. I'm aware that single-compiler, single-build determinism is not hard, but I am attempting 2-compiler, single-build determinism.
Efficiency is not a concern here. I just want the results to match.
I ask because I hope to narrow my concerns for what to test/try.
Are these things relevant for x86_64 processors and builds?
functions that control the x87 fpu
XPFPA_{DECLARE,RESTORE,etc}
"<"fpu_control.h>, _FPU_SINGLE, _FPU_DOUBLE, etc.
_controlfp_s(), _PC24, _PC53, _PC_64
I ask because I have read that platforms with SSE (x86_64) default to using SSE for floating point, so fpu control functions should be irrelevant?
I have found this and this to be most informative. This MSDN article says setting the floating point precision mask is not supported on x64 arch. And this SO post says SSE has fixed precision.
My testing has shown that /fp:{strict,precise,fast} are not changing the hashes. Neither is optimization level. So I'm hoping to narrow my focus to sin, cos.
Most floating point functions have to perform rounding one way or an other. The C/C++ standard is rather vague on the subject, and IEEE conformance is not strict enough on trigonometric functions. Which means that in practice it is useless to try to squeeze correct rounding out of your compilers default math implementation in a portable way.
For instance, the libm implementation (used by gcc) of sin/cos is written in assembly and the algorithm is different for different architectures and most probably depends on the version of the library.
You therefore have two possibilities:
implement your own sin/cos using only floating point operations with exact rounding (fused multiply-accumulate + Taylor series)
use a 3rd party library with strong rounding considerations
I personally use the MPFR library as a gold standard when dealing with rounding errors. There will be a runtime cost, although I never tried to benchmark it against libm performance.
Custom Implementation
Note that if you decide to implement it yourself, you need to choose the rounding mode and inform the compiler that it matter to you.
In C++ it is done this way:
#include <cfenv>
#pragma STDC FENV_ACCESS ON
#pragma STDC FP_CONTRACT OFF
int main(int, char**) {
...
if(!std::fesetround(FE_TONEAREST))
throw std::runtime_error("fesetround failed!");
...
}
If you are programming with the C language for a microprocessor that does not have an FPU, does the compiler signal errors when floating point literals and keywords are encountered (0.75, float, double, etc)?
Also, what happens if the result of an expression is fractional?
I understand that there are software libraries that are used so you can do floating-point math, but I am specifically wondering what the results will be if you did not use one.
Thanks.
A C implementation is required to implement the types float and double, and arithmetic expressions involving them. So if the compiler knows that the target architecture doesn't have floating-point ops then it must bring in a software library to do it. The compiler is allowed to link against an external library, it's also allowed to implement floating point ops in software by itself as intrinsics, but it must somehow generate code to get it done.
If it doesn't do so [*] then it is not a conforming C implementation, so strictly speaking you're not "programming with the C language". You're programming with whatever your compiler docs tell you is available instead.
You'd hope that code involving float or double types will either fail to compile (because the compiler knows you're in a non-conforming mode and tells you) or else fails to link (because the compiler emits calls to emulation routines in the library, but the library is missing). But you're on your own as far as C is concerned, if you use something that isn't C.
I don't know the exact details (how old do I look?), but I imagine that back in the day if you took some code compiled for x87 then you might be able to link and load it on a system using an x86 with no FPU. Then the CPU would complain about an illegal instruction when you tried to execute it -- quite possibly the system would hang depending what OS you were running. So the worst possible case is pretty bad.
what happens if the result of an expression is fractional?
The actual result of an expression won't matter, because the expression itself was either performed with integer operations (in which case the result is not fractional) or else with floating-point operations (in which case the problem arises before you even find out the result).
[*] or if you fail to specify the options to make it do so ;-)
Floating-point is a required part of the C language, according to the C standard. If the target hardware does not have floating-point instructions, then a C implementation must provide floating-point operations in some other way, such as by emulating them in software. (All calculations are just functions of bits. If you have elementary operations for manipulating bits and performing tests and branches, then you can compute any function that a general computer can.)
A compiler could provide a subset of C without floating-point, but then it would not be a standard-compliant C compiler.
Software floating point can take two forms:
a compiler may generate calls to built-in floating point functions directly - for example the operation 1.2 * 2.5 may invoke (for example) fmul( 1.2, 2.5 ),
alternatively for architectures that support an FPU, but for which some device variants may omit it, it is common to use FPU emulation. When an FP instruction is encountered an invalid instruction exception will occur and the exception handler will vector to code that emulates the instruction.
FPU emulation has the advantage that when the same code is executed on a device with a real FPU, it will be used automatically and accelerate execution. However without an FPU there is usually a small overhead compared with direct software implementation, so if the application is never expected to run on an FPU, emulation might best be avoided is the compiler provides the option.
Software floating point is very much slower that hardware supported floating point. Use of fixed-point techniques can improve performance with acceptable precision in many cases.
Typically, such microprocessor comes along either with a driver-package or even with a complete BSP (board-support-package, consisting of drivers and OS linked together), both of which contain FP library routines.
The compiler replaces every floating-point operation with an equivalent function call. This should be taken into consideration, especially when invoking such operations iteratively (inside a for / while loop), since the compiler cannot apply loop-unrolling optimization as a result.
The result of not including the required libraries within the project would be linkage errors.
I have a file spec (here: http://www.septentrio.com/secure/asterx1v_2_1/SBF%20Reference%20Guide.pdf) that has fields marked as both 32-bit and 64-bit floats (see page 8). How can I use both widths in my program? I am developing on Mac OSX right now but I will also deploy on a Linux machine.
More details:
I know I could tell the compiler the width, but how could I distinguish two different float widths? Maybe someone also has a suggestion for changing the way I parse, which is to reinterpret_cast(buffer+offset) and then use the values. These file sizes are huge (4GB) so I need performance.
This might seem obvious, nevertheless:
On Intel platform and many others float is 32-bit floating point value, and double is 64-bit floating point value. Try this approach. Most likely it will work.
To be absolutely sure check sizeof of your types at the start of your program or statically during compilation if your compiler allows this.
Once again, try the simple solution first.
Float and double arithmetic is both implemented on Intel and it is fast. In any case native arithmetic is the fastest of what you can get from the CPU.
IEEE 754 (http://en.wikipedia.org/wiki/IEEE_floating_point) defines not one floating point format, but several, like 4, 8, 16 bytes, etc. They all have different range and precision but they are all still IEEE values.
I'm working on updating a serialization library to add support for serializing floating point in a portable manner. Ideally I'd like to be able to test the code in an environment where IEEE754 isn't supported. Would it be sufficient to test using a soft-float library? Or any other suggestions about how I can properly test the code?
Free toolchains that you can find for ARM (embedded Linux) development, mostly do not support hard-float operations but soft-float only. You could try with one of these (i.e. CodeSourcery) but you would need some kind of a platform to run the compiled code (real HW or QEMU).
Or if you would want to do the same but on x86 machine, take a look at: Using software floating point on x86 linux
Should your library work on a system where both hardware floating point and soft-float are not available ? If so, if you test using a compiler with soft-float, your code may not compile/work on such a system.
Personally, I would test the library on a ARM9 system with a gcc compiler without soft-float.
Not an answer to your actual question, but describing what you must do to solve the problem.
If you want to support "different" floating point formats, your code would have to understand the internal format of floats [unless you only support "same architecture both ends"], pick the floating point number apart into your own format [which of course may be IEEE-754, but beware of denormals, 128-bit long doubles, NaN, INFINITY, and other "exception values", and of course out of range numbers], and then put it back together to the format required by the other end. If you are not doing this, there is no point in hunting down a non-IEEE-754 system, because it won't work.