Trying to solve a computer vision problem, I have to minimize a nonlinear energy function, implementing it in C++. Although I didn't find a library to help me with the specific function, I have the math for it. So what's the best way to go from symbolic math to C++ code?
Example: given the functions g(x):=x^2 and f(x):=x+2, let's imagine I am interested in converting f(g(x)) to C code; the obvious C code would be y=x^2+2; however for complicated math including jacobians, etc, it is not so easy, translating to pages and pages of operations.
I tried already Matlab and it's conversion module to C code, but the code is far from being optimized (ex: same operations repeating many times instead of reusing the result).
there exists NLopt library callable from C++, C, Matlab, Fortran, (...) for nonlinear optimizations. The implementation of minimization procedure using this library might look like this:
#include <nlopt.hpp>
nlopt::opt opt(nlopt::LD_MMA, 2);
std::vector<double> lb(2);
lb[0] = -HUGE_VAL; lb[1] = 0;
opt.set_lower_bounds(lb);
opt.set_min_objective(myfunc, NULL);
my_constraint_data data[2] = { {2,0}, {-1,1} };
opt.add_inequality_constraint(myconstraint, &data[0], 1e-8);
opt.add_inequality_constraint(myconstraint, &data[1], 1e-8);
opt.set_xtol_rel(1e-4);
std::vector<double> x(2);
x[0] = 1.234; x[1] = 5.678;
double minf;
nlopt::result result = opt.optimize(x, minf);
Related
The overloaded function float pow(float base, int iexp ) was removed in C++11 and now pow returns a double. In my program, I am computing lots of these (in single precision) and I am interested in the most efficient way how to do it.
Is there some special function (in standard libraries or any other) with the above signature?
If not, is it better (in terms of performance in single precision) to explicitly cast result of pow into float before any other operations (which would cast everything else into double) or cast iexp into float and use overloaded function float pow(float base, float exp)?
EDIT: Why I need float and do not use double?
The primarily reason is RAM -- I need tens or hundreds of GB so this reduction is huge advantage. So I need from float to get float. And now I need the most efficient way to achieve that (less casts, use already optimize algorithms, etc).
You could easily write your own fpow using exponentiation by squaring.
float my_fpow(float base, unsigned exp)
{
float result = 1.f;
while (exp)
{
if (exp & 1)
result *= base;
exp >>= 1;
base *= base;
}
return result;
}
Boring part:
This algorithm gives the best accuracy, that can be archived with float type when |base| > 1
Proof:
Let we want to calculate pow(a, n) where a is base and n is exponent.
Let's define b1=a1, b2=a2, b3=a4, b4=a8,and so on.
Then an is a product over all such bi where ith bit is set in n.
So we have ordered set B={bk1,bk1,...,bkn} and for any j the bit kj is set in n.
The following obvious algorithm A can be used for rounding error minimization:
If B contains single element, then it is result
Pick two elements p and q from B with minimal modulo
Remove them from B
Calculate product s = p*q and put it to B
Go to the first step
Now, lets prove that elements in B could be just multiplied from left to right without loosing accuracy. It comes form the fact, that:
bj > b1*b2*...*bj-1
because bj=bj-1*bj-1=bj-1*bj-2*bj-2=...=bj-1*bj-2*...*b1*b1
Since, b1 = a1 = a and its modulo more than one then:
bj > b1*b2*...*bj-1
Hence we may conclude, that during multiplication from left to right the accumulator variable is less than any element from B.
Then, expression result *= base; (except the very first iteration, for sure) does multiplication of two minimal numbers from B, so the rounding error is minimal. So, the code employs algorithm A.
Another question that can only be honestly answered with "wrong question". Or at least: "Are you really willing to go there?". float theoretically needs ca. 80% less die space (for the same number of cycles) and so can be much cheaper for bulk processing. GPUs love float for this reason.
However, let's look at x86 (admittedly, you didn't say what architecture you're on, so I picked the most common). The price in die space has already been paid. You literally gain nothing by using float for calculations. Actually, you may even lose throughput because additional extensions from float to double are required, and additional rounding to intermediate float precision. In other words, you pay extra to have a less accurate result. This is typically something to avoid except maybe when you need maximum compatibility with some other program.
See Jens' comment as well. These options give the compiler permission to disregard some language rules to achieve higher performance. Needless to say this can sometimes backfire.
There are two scenarios where float might be more efficient, on x86:
GPU (including GPGPU), in fact many GPUs don't even support double and if they do, it's usually much slower. Yet, you will only notice when doing very many calculations of this sort.
CPU SIMD aka vectorization
You'd know if you did GPGPU. Explicit vectorization by using compiler intrinsics is also a choice – one you could make, for sure, but this requires quite a cost-benefit analysis. Possibly your compiler is able to auto-vectorize some loops, but this is usually limited to "obvious" applications, such as where you multiply each number in a vector<float> by another float, and this case is not so obvious IMO. Even if you pow each number in such a vector by the same int, the compiler may not be smart enough to vectorize this effectively, especially if pow resides in another translation unit, and without effective link time code generation.
If you are not ready to consider changing the whole structure of your program to allow effective use of SIMD (including GPGPU), and you're not on an architecture where float is indeed much cheaper by default, I suggest you stick with double by all means, and consider float at best a storage format that may be useful to conserve RAM, or to improve cache locality (when you have a lot of them). Even then, measuring is an excellent idea.
That said, you could try ivaigult's algorithm (only with double for the intermediate and for the result), which is related to a classical algorithm called Egyptian multiplication (and a variety of other names), only that the operands are multiplied and not added. I don't know how pow(double, double) works exactly, but it is conceivable that this algorithm could be faster in some cases. Again, you should be OCD about benchmarking.
If you're targeting GCC you can try
float __builtin_powif(float, int)
I have no idea about it's performance tough.
Is there some special function (in standard libraries or any other) with the above signature?
Unfortunately, not that I know of.
But, as many have already mentioned benchmarking is necessary to understand if there is even an issue at all.
I've assembled a quick benchmark online. Benchmark code:
#include <iostream>
#include <boost/timer/timer.hpp>
#include <boost/random/mersenne_twister.hpp>
#include <boost/random/uniform_real_distribution.hpp>
#include <cmath>
int main ()
{
boost::random::mt19937 gen;
boost::random::uniform_real_distribution<> dist(0, 10000000);
const size_t size = 10000000;
std::vector<float> bases(size);
std::vector<float> fexp(size);
std::vector<int> iexp(size);
std::vector<float> res(size);
for(size_t i=0; i<size; i++)
{
bases[i] = dist(gen);
iexp[i] = std::floor(dist(gen));
fexp[i] = iexp[i];
}
std::cout << "float pow(float, int):" << std::endl;
{
boost::timer::auto_cpu_timer timer;
for(size_t i=0; i<size; i++)
res[i] = std::pow(bases[i], iexp[i]);
}
std::cout << "float pow(float, float):" << std::endl;
{
boost::timer::auto_cpu_timer timer;
for(size_t i=0; i<size; i++)
res[i] = std::pow(bases[i], fexp[i]);
}
return 0;
}
Benchmark results (quick conclusions):
gcc: c++11 is consistently faster than c++03.
clang: indeed int-version of c++03 seems a little faster. I'm not sure if it is within a margin of error, since I only run the benchmark online.
Both: even with c++11 calling pow with int seems to be a tad more performant.
It would be great if others could verify if this holds for their configurations as well.
Try using powf() instead. This is C99 function that should be also available in C++11.
A little background: I was working on some data conversion from C to C# by using a C++/CLI midlayer, and I noticed a peculiarity with the way the debugger shows floats and doubles, depending on which dll the code is executing in (see code and images below). At first I thought it had something to do with managed/unmanaged differences, but then I realized that if I completely left the C# layer out of it and only used unmanaged data types, the same behaviour was exhibited.
Test Case: To further explore the issue, I created an isolated test case to clearly identify the strange behaviour. I am assuming that anyone who may be testing this code already has a working Solution and dllimport/dllexport/ macros set up. Mine is called DLL_EXPORT. If you need a minimal working header file, let me know. Here the main application is in C and calling a function from a C++/CLI dll. I am using Visual Studio 2015 and both assemblies are 32 bit.
I am a bit concerned, as I am not sure if this is something I need to worry about or it's just something the debugger is doing (I am leaning towards the latter). And to be quite honest, I am just outright curious as to what's happening here.
Question: Can anyone explain the observed behaviour or at least point me in the right direction?
C - Calling Function
void floatTest()
{
float floatValC = 42.42f;
double doubleValC = 42.42;
//even if passing the address, behaviour is same as all others.
float retFloat = 42.42f;
double retDouble = 42.42;
int sizeOfFloatC = sizeof(float);
int sizeOfDoubleC = sizeof(double);
floatTestCPP(floatValC, doubleValC, &retFloat, &retDouble);
//do some dummy math to make compiler happy (i.e. no unsused variable warnings)
sizeOfFloatC = sizeOfFloatC + sizeOfDoubleC;//break point here
}
C++/CLI Header
DLL_EXPORT void floatTestCPP(float floatVal, double doubleVal,
float *floatRet, double *doubleRet);
C++/CLI Source
//as you can see, there are no managed types in this function
void floatTestCPP(float floatVal, double doubleVal, float *floatRet, double *doubleRet)
{
float floatLocal = floatVal;
double doubleLocal = doubleVal;
int sizeOfFloatCPP = sizeof(float);
int sizeOfDoubleCPP = sizeof(double);
*floatRet = 42.42f;
*doubleRet = 42.42;
//do some dummy math to make compiler happy (no warnings)
floatLocal = (float)doubleLocal;//break point here
sizeOfDoubleCPP = sizeOfFloatCPP;
}
Debugger in C - break point on last line of floatTest()
Debugger in C++/CLI - break point on the second to last line of floatTestCPP()
Consider Debugger in C++/CLI itself is not necessarily coded in C, C# or C++.
MS libraries support the "R" format: A string that can round-trip to an identical number. I suspect this or a g format was used.
Without MS source code, the following is only a good supposition:
The debug output is enough to distinguish the double from other nearby double. So code need not print "42.420000000000002", but "42.42" is sufficient - whatever format is used.
42.42 as an IEEE double is about 42.4200000000000017053025658242404460906982... and the debugger certainly need not print the exact value.
Potential; similar C code
int main(void) {
puts("12.34567890123456");
double d = 42.42;
printf("%.16g\n", nextafter(d,0));
printf("%.16g\n", d);
printf("%.17g\n", d);
printf("%.16g\n", nextafter(d,2*d));
d = 1 / 3.0f;
printf("%.9g\n", nextafterf(d,0));
printf("%.9g\n", d);
printf("%.9g\n", nextafterf(d,2*d));
d = 1 / 3.0f;
printf("%.16g\n", nextafter(d,0));
printf("%.16g\n", d);
printf("%.16g\n", nextafter(d,2*d));
}
output
12.34567890123456
42.41999999999999
42.42
42.420000000000002 // this level of precision not needed.
42.42000000000001
0.333333313
0.333333343
0.333333373
0.3333333432674407
0.3333333432674408
0.3333333432674409
For your code to convert a double to text with sufficient textual precision and back to double to "round-trip" the number, see Printf width specifier to maintain precision of floating-point value.
I have completed a port from Fortran to C++ but have discovered some differences in the COMPLEX type. Consider the following codes:
PROGRAM CMPLX
COMPLEX*16 c
REAL*8 a
c = (1.23456789, 3.45678901)
a = AIMAG(1.0 / c)
WRITE (*, *) a
END
And the C++:
#include <complex>
#include <iostream>
#include <iomanip>
int main()
{
std::complex<double> c(1.23456789, 3.45678901);
double a = (1.0 / c).imag();
std::cout << std::setprecision(15) << " " << a << std::endl;
}
Compiling the C++ version with clang++ or g++, I get the output: -0.256561150444368
Compiling the Fortran version however gives me: -0.25656115049876993
I mean, doesn't both languages follow the IEEE 754? If I run the following in Octave (Matlab):
octave:1> c=1.23456789+ 3.45678901i
c = 1.2346 + 3.4568i
octave:2> c
c = 1.2346 + 3.4568i
octave:3> output_precision(15)
octave:4> c
c = 1.23456789000000e+00 + 3.45678901000000e+00i
octave:5> 1 / c
ans = 9.16290109820952e-02 - 2.56561150444368e-01i
I get the same as the C++ version. What is up with the Fortran COMPLEX type? Am I missing some compiler flags? -ffast-math doesn't change anything. I want to produce the exact same 15 decimals in C++ and Fortran, so I easier can spot porting differences.
Any Fortran gurus around? Thanks!
In the Fortran code replace
c = (1.23456789, 3.45678901)
with
c = (1.23456789d0, 3.45678901d0)
Without a kind the real literals you use on the rhs are, most likely, 32-bit reals and you probably want 64-bit reals. The suffix d0 causes the compiler to create 64-bit reals closest to the values you provide. I've glossed over some details in this, and there are other (possibly better) ways of specifying the kind of a real number literal but this approach should work OK on any current Fortran compiler.
I don't know C++ very well, I'm not sure if the C++ code has the same problem.
If I read your question correctly the two codes produce the same answer to 8sf, the limit of single precision.
As for IEEE-754 compliance, that standard does not cover, so far as I am aware, the issues of complex arithmetic. I expect the f-p arithmetic used behind the scenes produces results on complex numbers within expected error bounds in most cases, but I'm not aware that they are guaranteed as error bounds on f-p arithmetic are.
I would propose to change all Fortran contants to DP
1.23456789_8 (or 1.23456789D00) etc
and use DIMAG instead of AIMAG
I want to use boltzmann constant in my functions. I am using the following code to declare the boltzmann constant
const double boltzmann_constant = 1.3806503 * pow (10,-23);
Will this get calculated at the compile time itself? If now, how should i ensure that it does get calculated at compile time? Any other method to declare the constant?
The pow() function is very unlikely to be calculated at compile time. However, the operation requested is directly expressible in scientific notation, a standard aspect of floating point numbers:
const double boltzmann_constant = 1.3806503e-23;
For a more complex situation, like sin(M_PI / 3), it can be useful to write a program to calculate and display such values so they can be edited into a program. If you do this, do everyone a favor and include a comment explaining what the constant is:
const double magic_val = 0.8660254037844385965883; // sin(M_PI / 3);
I am trying to implement my own version of pow() and sqrt() function as my custom library doesn't have pow()/sqrt() floating point support.
Can anyone help?
Yes, Sun can (Oracle now, I guess):
fdlibm, the "freely distributable math library", has sqrt and pow, along with many other math functions.
They're fairly high-tech implementations, though, and of course nothing is ever the "most efficient" implementation of something like this. Are you after source code to get it done, or are you really not so much looking for pow and sqrt, but actually looking for an education in floating-point algorithms programming?
Sure - it's easy if you have exponential and natural log functions.
Since y = x^n, you can take the natural log of both sides:
ln(y) = n*ln(x)
Then taking the exponential of both sides gives you what you want:
y = exp(n*ln(x))
If you want something better, the best place I know to look is Abramowitz and Stegun.
Note that if your instruction set has an instruction for square root or power, you'll be much better off using that. The x87 floating point instructions, for example, have an instruction fsqrt, and the SSE2 additions include another instruction sqrtsd, which are probably going to be much faster than most solutions written in C. In fact, atleast gcc uses the two instructions when compilation takes place on an x86 machine.
For power, however, things get somewhat murky. There's an instruction in the x87 floating point instruction set that can be used to calculate n*log2(n), namely fyl2x. Another instruction, fldl2e, stores log2(e) in the floating point stack. You might want to give these a look.
You might also want to take a look at how individual C libraries do this. dietlibc, for example, simply uses fsqrt:
sqrt:
fldl 4(%esp)
fsqrt
ret
glibc uses Sun's implementation for machines where a hardware square root instruction is not available (under sysdeps/ieee754/flt-32/e-sqrtf.c), and uses fsqrt on the x86 instruction set (though gcc can be instructed to instead use the sqrtsd instruction.)
Square root is properly implemented with an iterative Newtons method.
double ipow(int base, int exp)
{
bool flag=0;
if(exp<0) {flag=1;exp*=-1;}
int result = 1;
while (exp)
{
if (exp & 1)
result *= base;
exp >>= 1;
base *= base;
}
if(flag==0)
return result;
else
return (1.0/result);
}
//most suitable way to implement power function for integer to power integer
For calculating the square root of a float in C I'd recommend using fsqrt if you target x86.
You can use such ASM instruction with:
asm("fsqrt" : "+t"(myfloat));
For GCC or
asm {
fstp myfloat
fsqrt
fldp myfloat
}
Or something like that for Visual Studio.
For implementing pow, using a big switch statement like the one at upitasoft.com/link/powLUT.h should do.
It can cause some cache problems but if you keep it like that it shouldn't be a problem, just limit the range (note, you can still optimize the code I provided).
If you want to support floating point powers, is way harder...
You can try using the natural logarithm and exponential functions, such as:
float result = exp(number * log(power));
But usually it is slow and/or imprecise.
Hope I helped.
The fastest way I can think of doing a pow() would be along these lines (note, this is pretty complicated):
//raise x^y
double pow(double x, int y) {
int power;
map<int, double> powers;
for (power = 1; power < y; power *= 2, x *= x)
powers.insert(power, x);
while (power > y) {
//figure out how to get there
map<int, double>::iterator p = powers.lower_bound(power - y);
//p is an iterator that points to the biggest power we have that doesn't go over power - y
power -= p->first;
x /= p->second;
}
return x;
}
I have no idea about how to implement a decimal power. My best guess would be to use logarithms.
Edit: I'm attempting a logarithmic solution (based on y), as opposed to a linear solution, which you propose. Let me work this out and edit it, because I know it works.
Edit 2: Hehe, my bad. power *= 2 instead of power++