I am testing out using the clock_t function's in c++ and I ran across a problem. When I compile I do it on 2 different compilers. Visual studio on my Windows 7 computer (2012), and g++ on a Unix system called "ranger". When I just compiled my code in an attempt to output the time in seconds (up to a thousandth of a second) it takes to run different sort functions, it seems that the g++ compiler completely ignores my attempt to divide the time stamp by 1000 in order to convert it from milliseconds to a second format. Any advice? Is there a difference between g++ and Visual Studio's compiler in regard's to this?
A short code snippet(Output and what i do for division):
//Select Sort
begin = clock(); //Begin time
selectionSort(array, n);
end = clock(); //End time
d_select = ((float)(end/1000.0) - (float)(begin/1000.0)); //Calculates the time in MS, and converts from MS, to a float second.
//Output data
cout << setprecision(3) << fixed; //Set precision to 3 decimal places, with a fixed output (0's are displayed regardless of rounding)
cout << n << "\t" << d_bubble << "\t" << d_insert << "\t" << d_merge << "\t" << d_quick << "\t"
<< d_select << endl;
Visual Studio output (Correct):
n Bubble Insert Merge Quick Select
100000 12.530 1.320 0.000 0.030 2.900
Unix output(Incorrect) :
n Bubble Insert Merge Quick Select
100000 51600.000 11700.000 30.000 150.000 18170.000
Any suggestions? Thanks!
Divide by CLOCKS_PER_SEC, not 1000. On Unix, and POSIX in general, clock() gives a value in microseconds, not milliseconds.
Note that that, and clock_t, are integers; so if you want fractional seconds, convert to a floating-point format before dividing:
d_select = float(end - begin) / CLOCKS_PER_SEC;
Related
This question already has answers here:
How to make C++ cout not use scientific notation
(8 answers)
Closed 3 years ago.
I try to write a large double number which is the time that takes my function to produce the result into txt file( using c++).
for exemple in my console, the function take 0.000029 (time unit), when i write the value into my txt file it is converted into : 2.9e-05
My question: how can i write the value as it is in the console i.e 0.000029 ?
here is my code:
*`
clock_t cPrec1=0;
double duration1 =0.0;
clock_t cTime1;
cTime1 = clock();
bool h= check(4, copy);
duration1 = ( cTime1 - cPrec1 ) / (double) CLOCKS_PER_SEC;
cPrec1 = clock();
outfile << space<< 1 << space << duration1<< space <<'\n' ;
printf(" saved\n");
`
Thanks for helping me.
You can use std::setprecision() and std::fixed function.
when you use a stream you can add a "<< std::setprecision(n)" at the beginning ( or even before the double number ) to set how many number after the dot you want to see. The parameter of this function is an integer that specify the number after the dot that are printed.
Another useful function is std::fixed that can be used like the previous on in the stream ( ex. "<< std::fixed" ). I report to you the definition of the function :
When floatfield is set to fixed, floating-point values are written using fixed-point notation: the value is represented with exactly as many digits in the decimal part as specified by the precision field (precision) and with no exponent part.
I leave you also 2 useful links for this function:
std::setprecision
std::fixed
Use:
outfile << std::setprecision(7) << duration1
I am currently recording some frame times in MS instead of ticks. I know this can be an issue as we are adding all the frame times (in MS) together and then dividing by the number of frames. This could cause bad results due to floating point precision.
It would make more sense to add all the tick counts together then convert to MS once at the end.
However, I am wondering what the actual difference would be for a small number of samples? I expect to have between 900-1800 samples. Would this be an issue at all?
I have made this small example and run it on GCC 4.9.2:
// Example program
#include <iostream>
#include <string>
int main()
{
float total = 0.0f;
double total2 = 0.0f;
for(int i = 0; i < 1000000; ++i)
{
float r = static_cast <float> (rand()) / static_cast <float> (RAND_MAX);
total += r;
total2 += r;
}
std::cout << "Total: " << total << std::endl;
std::cout << "Total2: " << total2 << std::endl;
}
Result:
Total: 500004 Total2: 500007
So as far as I can tell with 1 million values we do not lose a lot of precision. Though I am not sure if what I have written is a reasonable test or actually testing what I want to test.
So my question is, how many floats can I add together before precision becomes an issue? I expect my values to be between 1 and 60 MS. I would like the end precision to be within 1 millisecond. I have 900-1800 values.
Example Value: 15.1345f for 15 milliseconds.
Counterexample
Using the assumptions below about the statement of the problem (times are effectively given as values such as .06 for 60 milliseconds), if we convert .06 to float and add it 1800 times, the computed result is 107.99884796142578125. This differs from the mathematical result, 108.000, by more than .001. Therefore, the computed result will sometimes differ from the mathematical result by more than 1 millisecond, so the goal desired in the question is not achievable in these conditions. (Further refinement of the problem statement and alternate means of computation may be able to achieve the goal.)
Original Analysis
Suppose we have 1800 integer values in [1, 60] that are converted to float using float y = x / 1000.f;, where all operations are implemented using IEEE-754 basic 32-bit binary floating-point with correct rounding.
The conversions of 1 to 60 to float are exact. The division by 1000 has an error of at most ½ ULP(.06), which is ½ • 2−5 • 2−23 = 2−29. 1800 such errors amount to at most 1800 • 2−29.
As the resulting float values are added, there may be an error of at most ½ ULP in each addition, where the ULP is that of the current result. For a loose analysis, we can bound this with the ULP of the final result, which is at most around 1800 • .06 = 108, which has an ULP of 26 • 2−23 = 2−17. So each of the 1799 additions has an error of at most 2−17, so the total errors in the additions is at most 1799 • 2−18.
Thus, the total error during divisions and additions is at most 1800 • 2−29 + 1799 • 2−18, which is about .006866.
That is a problem. I expect a better analysis of the errors in the additions would halve the error bound, as it is an arithmetic progression from 0 to the total, but that still leaves a potential error above .003, which means there is a possibility the sum could be off by several milliseconds.
Note that if the times are added as integers, the largest potential sum is 1800•60 = 108,000, which is well below the first integer not representable in float (16,777,217). Addition of these integers in float would be error-free.
This bound of .003 is small enough that some additional constraints on the problem and some additional analysis might, just might, push it below .0005, in which case the computed result will always be close enough to the correct mathematical result that rounding the computed result to the nearest millisecond would produce the correct answer.
For example, if it were known that, while the times range from 1 to 60 milliseconds, the total is always less than 7.8 seconds, that could suffice.
As much as possible, reduce the errors caused by floating point calculations
Since you've already described measuring your individual timings in milliseconds, it's far better if you accumulate those timings using integer values before you finally divide them:
std::milliseconds duration{};
for(Timing const& timing : timings) {
//Lossless integer accumulation, in a scenario where overflow is extremely unlikely
//or possibly even impossible for your problem domain
duration += std::milliseconds(timing.getTicks());
}
//Only one floating-point calculation performed, error is minimal
float averageTiming = duration.count() / float(timings.size());
The Errors that accumulate are highly particular to the scenario
Consider these two ways of accumulating values:
#include<iostream>
int main() {
//Make them volatile to prevent compilers from optimizing away the additions
volatile float sum1 = 0, sum2 = 0;
for(float i = 0.0001; i < 1000; i += 0.0001) {
sum1 += i;
}
for(float i = 1000; i > 0; i -= 0.0001) {
sum2 += i;
}
std::cout << "Sum1: " << sum1 << std::endl;
std::cout << "Sum2: " << sum2 << std::endl;
std::cout << "% Difference: " << (sum2 - sum1) / (sum1 > sum2 ? sum1 : sum2) * 100 << "%" << std::endl;
return 0;
}
Results may vary on some machines (particularly machines that don't have IEEE754 floats), but in my tests, the second value was 3% different than the first value, a difference of 13 million. That can be pretty significant.
Like before, the best option is to minimize the number of calculations performed using floating point values until the last possible step before you need them as floating point values. That will minimize accuracy losses.
Just for what it's worth, here's some code to demonstrate that yes, after 1800 items, a simple accumulation can be incorrect by more than 1 millisecond, but Kahan summation maintains the required level of accuracy.
#include <iostream>
#include <iterator>
#include <iomanip>
#include <vector>
#include <numeric>
template <class InIt>
typename std::iterator_traits<InIt>::value_type accumulate(InIt begin, InIt end)
{
typedef typename std::iterator_traits<InIt>::value_type real;
real sum = real();
real running_error = real();
for (; begin != end; ++begin)
{
real difference = *begin - running_error;
real temp = sum + difference;
running_error = (temp - sum) - difference;
sum = temp;
}
return sum;
}
int main()
{
const float addend = 0.06f;
const float count = 1800.0f;
std::vector<float> d;
std::fill_n(std::back_inserter(d), count, addend);
float result = std::accumulate(d.begin(), d.end(), 0.0f);
float result2 = accumulate(d.begin(), d.end());
float reference = count * addend;
std::cout << " simple: " << std::setprecision(20) << result << "\n";
std::cout << " Kahan: " << std::setprecision(20) << result2 << "\n";
std::cout << "Reference: " << std::setprecision(20) << reference << "\n";
}
For this particular test, it appears that double precision is sufficient, at least for the input values I tried--but to be honest, I'm still a bit leery of it, especially when exhaustive testing isn't reasonable, and better techniques are easily available.
I am creating a testing utility that requires high usage of sqrt() function. After digging in possible optimisations, I have decided to try inline assembler in C++. The code is:
#include <iostream>
#include <cstdlib>
#include <cmath>
#include <ctime>
using namespace std;
volatile double normalSqrt(double a){
double b = 0;
for(int i = 0; i < ITERATIONS; i++){
b = sqrt(a);
}
return b;
}
volatile double asmSqrt(double a){
double b = 0;
for(int i = 0; i < ITERATIONS; i++){
asm volatile(
"movq %1, %%xmm0 \n"
"sqrtsd %%xmm0, %%xmm1 \n"
"movq %%xmm1, %0 \n"
: "=r"(b)
: "g"(a)
: "xmm0", "xmm1", "memory"
);
}
return b;
}
int main(int argc, char *argv[]){
double a = atoi(argv[1]);
double c;
std::clock_t start;
double duration;
start = std::clock();
c = asmSqrt(a);
duration = std::clock() - start;
cout << "asm sqrt: " << c << endl;
cout << duration << " clocks" <<endl;
cout << "Start: " << start << " end: " << start + duration << endl;
start = std::clock();
c = normalSqrt(a);
duration = std::clock() - start;
cout << endl << "builtin sqrt: " << c << endl;
cout << duration << " clocks" << endl;
cout << "Start: " << start << " end: " << start + duration << endl;
return 0;
}
I am compiling this code using this script that sets number of iterations, starts profiling, and opens profiling output in VIM:
#!/bin/bash
DEFAULT_ITERATIONS=1000000
if [ $# -eq 1 ]; then
echo "Setting ITERATIONS to $1"
DEFAULT_ITERATIONS=$1
else
echo "Using default value: $DEFAULT_ITERATIONS"
fi
rm -rf asd
g++ -msse4 -std=c++11 -O0 -ggdb -pg -DITERATIONS=$DEFAULT_ITERATIONS test.cpp -o asd
./asd 16
gprof asd gmon.out > output.txt
vim -O output.txt
true
The output is:
Using default value: 1000000
asm sqrt: 4
3802 clocks
Start: 1532 end: 5334
builtin sqrt: 4
5501 clocks
Start: 5402 end: 10903
The question is why the sqrtsd instruction takes only 3802 clocks, to count square root of 16, and sqrt() takes 5501 clocks?
Does it have something to do with HW implementation of certain instructions? Thank you.
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 48
Model name: AMD A8-7600 Radeon R7, 10 Compute Cores 4C+6G
Stepping: 1
CPU MHz: 3100.000
CPU max MHz: 3100,0000
CPU min MHz: 1400,0000
BogoMIPS: 6188.43
Virtualization: AMD-V
L1d cache: 16K
L1i cache: 96K
L2 cache: 2048K
NUMA node0 CPU(s): 0-3
Floating point arithmetic has to take into consideration rounding. Most C/C++ compilers adopt IEEE 754, so they have an "ideal" algorithm to perform operations such as square root. They are then free to optimize, but they must return the same result down to the last decimal, in all cases. So their freedom to optimize is not complete, in fact it is severely constrained.
Your algorithm probably is off by a digit or two part of the time. Which could be completely negligible for some users, but could also cause nasty bugs for some others, so it's not allowed by default.
If you care more for speed than standard compliance, try poking around with the options of your compiler. For instance in GCC the first I'd try is -funsafe-math-optimizations, which should enable optimizations disregarding strict standard compliance. Once you tweak it enough, you should come closer to and possibly pass your handmade implementation's speed.
Ignoring the other problems, it will still be the case that sqrt() is a bit slower than sqrtsd, unless compiled with specific flags.
sqrt() has to potentially set errno, it has to check whether it's in that case. It will still boil down to the native square root instruction on any reasonable compiler, but it will have a little overhead. Not a lot of overhead like your flaws test showed, but still some.
You can see that in action here.
Some compile flags suppress this test. For example for GCC, fno-math-errno and ffinite-math-only.
In book Game Engine Architecture : "..., let’s say we use a floating-point variable to track absolute game time in seconds. How long can we run our game before the magnitude of our clock variable gets so large that adding 1/30th of a second to it no longer changes its value? The answer is roughly 12.9 days."
Why 12.9 days, how to calculate it ?
When the result of a floating point computation can't be represented exactly, it is rounded to the nearest value. So you want to find the smallest value x such that the increment f = 1/30 is less than half the width h between x and the next largest float, which means that x+f will round back to x.
Since the gap is the same for all elements in the same binade, we know that x must be the smallest element in its binade, which is a power of 2.
So if x = 2k, then h = 2k-23 since a float has a 24-bit significand. So we need to find the smallest integer k such that
2k-23/2 > 1/30
which implies k > 19.09, hence k = 20, and x = 220 = 1048576 (seconds).
Note that x / (60 × 60 × 24) = 12.14 (days), which is a little bit less that what your answer proposes, but checks out empirically: in Julia
julia> x = 2f0^20
1.048576f6
julia> f = 1f0/30f0
0.033333335f0
julia> x+f == x
true
julia> p = prevfloat(x)
1.04857594f6
julia> p+f == p
false
UPDATE: Okay, so where did the 12.9 come from? The 12.14 is in game time, not actual time: these will have diverged due to the rounding error involved in floating point (especially near the end, when the rounding error is actually quite large relative to f). As far as I know, there's no way to calculate this directly, but it's actually fairly quick to iterate through 32-bit floats.
Again, in Julia:
julia> function timestuff(f)
t = 0
x = 0f0
while true
t += 1
xp = x
x += f
if x == xp
return (t,x)
end
end
end
timestuff (generic function with 1 method)
julia> t,x = timestuff(1f0/30f0)
(24986956,1.048576f6)
x matches our result we calculated earlier, and t is the clock time in 30ths of a second. Converting to days:
julia> t/(30*60*60*24)
9.640029320987654
which is even further away. So I don't know where the 12.9 came from...
UPDATE 2: My guess is that the 12.9 comes from the calculation
y = 4 × f / ε = 1118481.125 (seconds)
where ε is the standard machine epsilon (the gap between 1 and the next largest floating point number). Scaling this to days gives 12.945. This provides an upper bound on x, but it is not the correct answer as explained above.
#include <iostream>
#include <iomanip>
/*
https://en.wikipedia.org/wiki/Machine_epsilon#How_to_determine_machine_epsilon
*/
typedef union
{
int32_t i32;
float f32;
} fi32_t;
float float_epsilon(float nbr)
{
fi32_t flt;
flt.f32 = nbr;
flt.i32++;
return (flt.f32 - nbr);
}
int main()
{
// How to calculate 32-bit floating-point epsilon?
const float one {1.}, ten_mills {10e6};
std::cout << "epsilon for number " << one << " is:\n"
<< std::fixed << std::setprecision(25)
<< float_epsilon(one)
<< std::defaultfloat << "\n\n";
std::cout << "epsilon for number " << ten_mills << " is:\n"
<< std::fixed << std::setprecision(25)
<< float_epsilon(ten_mills)
<< std::defaultfloat << "\n\n";
// In book Game Engine Architecture : "..., let’s say we use a
// floating-point variable to track absolute game time in seconds.
// How long can we run our game before the magnitude of our clock
// variable gets so large that adding 1/30th of a second to it no
// longer changes its value? The answer is roughly 12.9 days."
// Why 12.9 days, how to calculate it ?
const float one_30th {1.f/30}, day_sec {60*60*24};
float time_sec {}, time_sec_old {};
while ((time_sec += one_30th) > time_sec_old)
{
time_sec_old = time_sec;
}
std::cout << "We can run our game for "
<< std::fixed << std::setprecision(5)
<< (time_sec / day_sec)
<< std::defaultfloat << " days.\n";
return EXIT_SUCCESS;
}
This outputs
epsilon for number 1 is:
0.0000001192092895507812500
epsilon for number 10000000 is:
1.0000000000000000000000000
We can run our game for 12.13630 days.
This is due to zones of expressibility in floating point representation.
Check out this lecture from my uni.
As the exponent gets larger, the jump on the really number line between the values actually represented increases; when the exponent is low, the density of representation is high. To give an example, imaging decimal numbers with a finite number of place values. given 1.0001e1 and 1.0002e1 the difference is 0.0001 between the two values. But if the exponent increases 1.0001-10 1.0002-10 the difference between the two is 0.000100135. Obviously this gets larger as the exponent increases. In the case you talk about, it is possible that the jump becomes so large, the increase does not promote a rounding increase of the least significant bit
Interestingly, towards the limits of the representations, the accuracy of a larger float type is worse! Simply because the increase in the bit pattern in the mantissa jumps much further on the number line when more bits are available for the exponent; as is the case for double, over float
I'm on Manjaro 64 bit, latest edition. HP pavilion g6, Codeblocks
Release 13.12 rev 9501 (2013-12-25 18:25:45) gcc 5.2.0 Linux/unicode - 64 bit.
There was a discussion between students on why
sn = 1/n diverges
sn = 1/n^2 converges
So decided to write a program about it, just to show them what kind of output they can expect
#include <iostream>
#include <math.h>
#include <fstream>
using namespace std;
int main()
{
long double sn =0, sn2=0; // sn2 is 1/n^2
ofstream myfile;
myfile.open("/home/Projects/c++/test/test.csv");
for (double n =2; n<100000000;n++){
sn += 1/n;
sn2 += 1/pow(n,2);
myfile << "For n = " << n << " Sn = " << sn << " and Sn2 = " << sn2 << endl;
}
myfile.close();
return 0;
}
Starting from n=9944 I got sn2 = 0.644834, and kept getting it forever. I did expect that the compiler would round the number and ignore the 0s at some point, but this is just too early, no?
So at what theoretical point does 0s start to be ignored? And what to do if you care about all 0s in a number? If long double doesn't do it, then what does?
I know it seems like a silly question but I expected to see a longer number, since you can store big part of pi in long doubles. By the way same result for double too.
The code that you wrote suffers from a classic programming mistake: it sums a sequence of floating-point numbers by adding larger numbers to the sum first and smaller numbers later.
This will inevitably lead to precision loss during addition, since at some point in the sequence the sum will become relatively large, while the next member of the sequence will become relatively small. Adding a sufficiently small floating-point value to a sufficiently large floating-point sum does not affect the sum. Once you reach that point, it will look as if the addition operation is "ignored", even though the value you attempt to add is not zero.
You can observe the same effect if you try calculating 100000000.0f + 1 on a typical machine: it still evaluates to 100000000. This does not happen because 1 somehow gets rounded to zero. This happens because the mathematically-correct result 100000001 is rounded back to 100000000. In order to force 100000000.0f to change through addition, you need to add at least 5 (and the result will be "snapped" to 100000008).
So, the issue here is not that the compiler "rounds the number when it gets so small", as you seem to believe. Your 1/pow(n,2) number is probably fine and sufficiently precise (not rounded to 0). The issue here is that at some iteration of your cycle the small non-zero value of 1/pow(n,2) just cannot affect the sum anymore.
While it is true that adjusting output precision will help you to see better what is going on (as stated in the comments), the real issue is what is described above.
When calculating sums of floating-point sequences with large differences in member magnitudes, you should do it by adding smaller members of the sequence first. Using my 100000000.0f example again, you can easily see that 4.0f + 4.0f + 100000000.0f correctly produces 100000008, while 100000000.0f + 4.0f + 4.0f is still 100000000.
You're not running into precision issues here. The sum doesn't stop at 0.644834; it keeps going to roughly the correct value:
#include <iostream>
#include <math.h>
using namespace std;
int main() {
long double d = 0;
for (double n = 2; n < 100000000; n++) {
d += 1/pow(n, 2);
}
std::cout << d << endl;
return 0;
}
Result:
0.644934
Note the 9! That's not 0.644834 any more.
If you were expecting 1.644934, you should have started the sum at n=1. If you were expecting visible changes between successive partial sums, you didn't see those because C++ is truncating the representation of the sums to 6 significant digits. You can configure your output stream to display more digits with std::setprecision from the iomanip header:
myfile << std::setprecision(9);