Related
I have to find the set of integers that minimize this objective function:
The costraints are:
every x must be a non-negative integer
T, A and B are double known numbers.
I have been looking at the OR-Tools C++ library in order to solve this problem, specifically at the CP-SAT solver.
Is it the right tool from such problems?
If yes, would it be feasible to convert all the double to int in the objective function?
If not, what else do you suggest? (I'm also open to other open source C++ libraries)
It will fit in the CP-SAT solver. You will need to scale floating point coefficients to integers.
The objective function accepts floating point coefficients.
But (x1 + A1)^2 will propagate better if you keep it that way instead of A1^2 + 2 * A1 * x1 + x1^2. which fits into the linear with double coefficient limitation of CP-SAT, provided you use temporary variables sx1 = x1 * x1.
Then make sure to use at least 8 workers for that. (parameters num_search_workers:8).
Now, I believe there are least square solvers that are more suited for this.
I'm implementing an arbitrary precision arithmetic library in C++ and I'm pretty much stuck when implementing the gamma function.
By using the equivalences gamma(n) = gamma(n - 1) * n and gamma(n) = gamma(n + 1) / n, respectively, I can obtain a rational number r in the range (1; 2] for all real values x.
However, I don't know how to evaluate gamma(r). For the Lanczos approximation (https://en.wikipedia.org/wiki/Lanczos_approximation), I need precomputed values p which happen to calculate a factorial of a non-integer value (?!) and can't be calculated dynamically with my current knowledge... Precomputing values for p wouldn't make much sense when implementing an arbitrary precision library.
Are there any algorithms that compute gamma(r) in a reasonable amount of time with arbitrary precision? Thanks for your help.
Spouge's approximation is similar to Lanczos's approximation, but probably easier to use for arbitrary precision, as you can set the desired error.
Lanczos approximation doesn't seem too bad. What exactly do you suspect?
Parts of code which calculate p, C (Chebyshev polynomials) and (a + 1/2)! can be implemented as stateful objects so that, for example, you can calculate p(i) from p(i-1) and Chebyshev coefficients and be computed once, maintaining their matrix.
I am trying to implement a root finding algorithm. I am using the hybrid Newton-Raphson algorithm found in numerical recipes that works pretty nicely. But I have a problem in bracketing the root.
While implementing the root finding algorithm I realised that in several cases my functions have 1 real root and all the other imaginary (several of them, usually 6 or 9). The only root I am interested is in the real one so the problem is not there. The thing is that the function approaches the root like a cubic function, touching with the point the y=0 axis...
Newton-Rapson method needs some brackets of different sign and all the bracketing methods I found don't work for this specific case.
What can I do? It is pretty important to find that root in my program...
EDIT: more problems: sometimes due to reaaaaaally small numerical errors, say a variation of 1e-6 in some value the "cubic" function does NOT have that real root, it is just imaginary with a neglectable imaginary part... (checked with matlab)
EDIT 2: Much more information about the problem.
Ok, I need root finding algorithm.
Info I have:
The root I need to find is between [0-1] , if there are more roots outside that part I am not interested in them.
The root is real, there may be imaginary roots, but I don't want them.
Probably all the rest of the roots will be imaginary
The root may be double in that point, but I think that actually doesn't mater in numerical analysis problems
I need to use the root finding algorithm several times during the overall calculations, but the function will always be a polynomial
In one of the particular cases of the root finding, my polynomial will be similar to a quadratic function that touches Y=0 with the point. Example of a real case:
The coefficient may not be 100% precise and that really slight imprecision may make the function not to touch the Y=0 axis.
I cannot solve for this specific case because in other cases it may be that the polynomial is pretty normal and doesn't make any "strange" thing.
The method I am actually using is NewtonRaphson hybrid, where if the derivative is really small it makes a bisection instead of NewRaph (found in numerical recipes).
Matlab's answer to the function on the image:
roots:
0.853553390593276 + 0.353553390593278i
0.853553390593276 - 0.353553390593278i
0.146446609406726 + 0.353553390593273i
0.146446609406726 - 0.353553390593273i
0.499999999999996 + 0.000000040142134i
0.499999999999996 - 0.000000040142134i
The function is a real example I prepared where I know that the answer I want is 0.5
Note:
I still haven't check completely some of the answers I you people have give me (Thank you!), I am just trying to give al the information I already have to complete the question.
Assuming you have a one-dimensional polynomial problem (which I assume from the imaginary solutions) you can use Sturm sequences to bracket all real roots. See Sturm's theorem.
Welcome to the wonderful world of numerical methods. Watch your hairline; it might start receding as you pull your hair out in frustration.
First off, with numerical root finding, you are toast if you can't bracket the problem. Newton Raphson is nice for polishing off a solution once you get close, and it only works if the derivative near the root is well away from zero. You always need to have some slower technique at hand as a backup because Newton Raphson can send you off to never-never land (i.e., somewhere well outside the bracket). If your function is not a polynomial, the first thing to try is Brent's method. If your function is a polynomial, try Laguerre's method or Jenkins-Traub.
BTW, it sounds like you have a pathological problem. You shouldn't expect particularly good performance. Pathological problems are, well, pathological.
Addendum
If you are having problems with things that appear to be roots, but aren't, you need to take care how you evaluate your function. If you do have a polynomial, form each term of the polynomial, sort by absolute value, and add smallest to largest. This produces better accuracy most of the time, but fails if you have large terms whose sum is nearly zero. If that's the case, you might want to add those canceling terms separately, add the rest smallest to largest, and then compute a grand total -- and your still kinda screwed. That big addition that nearly cancels loses a lot of precision. There's no escape other than extended precision arithmetic.
Ander, thanks for responding to my question (about the interval); sorry for the delay in following up - I have very busy work. Also - before I found the additional information you've provided - I had in mind to explain quite a few things how to handle this and was contemplating how to present that. However, I now believe your case is not too difficult and we can get at it without too much additional stuff, since you apparently have an explicit polynomial expression (coefficients to the various powers).
Let's start with a simple case, to pinpoint the approach.
Step 1.
If you have a 2nd degree polynomial, its derivative is first order and has a simple zero (which you can find by bracketing or simply by explicitly solving the equation). (Yes, I know there's a closed formula for the roots of a 2nd degree polynomial also, but for the sake of the current argument, let us forget that).
The zero's of the 2nd degree polynomial are then located one at the left side and one at the right side of the zero of the derivative. So, if you also have the interval where the roots of the original function (the 2nd degree polynomial) are to be found, you now have two intervals - left and right of the derivative-zero, each with one zero.
It is important to realize that the original function is MONOTONIC on each subinterval (decreasing on one of them, increasing on the other). Therefore, simply by checking the function values at the ends of the (sub)interval you can determine whether or not they actually bracket a zero. If not, there's a multiple zero (double, in this case) exactly at the zero of the derivative IF the function is zero there (otherwise, it is a double imaginary root of which you've now found the real part).
In case the zero of the derivative lies OUTSIDE the total interval, you will have at most one root inside your interval and you need to check only that particular (sub)interval.
Step 2.
Consider now a 3rd order polynomial.
Its derivative is 2nd order.
The derivative of THAT 2nd order polynomial is again 1st order and you proceed as before to get two subintervals to find the roots of the derivative of the original function. These two roots give you THREE (at most) intervals where you will find the 3 roots of the original (3rd order) function.
And also here, you will have intervals (3) where the original function is monotonic (alternatingly increasing/decreasing), making the analysis per subinterval quite easy.
Again, zeros may coincide (2 or even all 3) and may in addition turn out to be complex-valued (i.e. having non-zero imaginary parts). The analysis of the cases is straightforward: check function values at the borders of the intervals to assess whether not there's a sign-change (function is monotonic on each subinterval) and/or whether the function is zero at one of the subinterval-borders.
Step 4.
Generalize this with the known polynomial. Let's say - your example - it is 6th order:
a) construct the 5th derivative (i.e. reducing the original to a 1st order polynomial). Compute it's zero (it is at precisely 0.5 in your example). In this case you're already done, but suppose you don't realize that. So you have now 2 intervals 0..0.5 and 0.5..1
b) construct the 4th derivative. Inspect its values at the subinterval-boundaries (0, 0.5, 1)
For each subinterval determine if it has a real zero inside. If so, you re-partition your original interval in 3 subintervals, using the two found zeros (you forget about the zero of the 5th derivative). If they coincide (at the previous cut, 0.5) you stick with that 0.5 (don't care whether you've found a true double zero of your 4th derivative there or a "double imaginary") and still have only 2 intervals, but for the sake of the argument let's say you now have 3.
c) construct the 3rd derivative and do likewise as before. You will then have 4 (at most) intervals.
d) And so on. After having processed the 2nd derivative in this fashion you have 5 (at most) intervals, and after processing the 1st derivative you have 6 intervals (or less...) and knowing the function is monotonic on each subinterval, you'll quickly determine in each of them if there's a real root, as always using the know monotonicity of the function in each of the final subintervals.
Adding a note on numerical accuracy at evaluating a function:
A first (probably sufficient, in this case) method to reduce noise is NOT to evaluate your function in the way suggested by the original form (i.e. a6 x*6 + a5 x*5 +..), but to rewrite it as:
a0 + x*(a1 + x*(a2 + x*(a3 + x*(a4 + x*(a5 + x*a6)))))
So, in evaluating you proceed:
tmp = a6
tmp = x*tmp + a5
tmp = x*tmp + a4
etcetera.
In case this little rewriting is not sufficient for numerical stability, you should rewrite your polynomial in (for instance) a chebyshev-polynomial expansion and evaluate that one with its recurrence relations. Both (getting the expansion and applying the recurrence relations for evaluation) are rather simple. I can explain, if you need help, but I guess it won't be necessary here.
In all cases, you HAVE to allow for some inaccuracy, i.e. accept that a computation will, generally speaking, NEVER give you the mathematically exact function value. So the assessment whether the function is presumably zero at some point must include some "tolerance", there's no way around this, unfortunately; the best you can aim for is to minimize the noise.
Well, if your function touches zero but never crosses it, you seem to be looking for a minimum (or a maximum). In which case, you're better off telling computer to do exactly that --- either find the root of a derivative (if you can calculate it analytically), or use a minimization routine. Then check that the function value at the minimum is 'close enough' to zero.
Just to reiterate what was already said by other people:
don't start with Newton-Raphson method; it's almost always better to start with Brent or even a straightforward bisection (provided you can bracket the root).
An instability where 'small numerical errors' of the order of 1e-6 have bad effects is worth investigating. Immediate suspects: mixing floats and doubles, loss of precision somewhere etc.
EDIT: So, depending on some parameters, your function has either a zero crossing, or a minimum with zero value, is this correct? In this case, what I'd do is this: use a simple and robust bracketing strategy (e.g. start from [-1, 1], multiply the endpoints by 1.1, check the signs, keep multiplying, something like this). If that succeeds, there's a zero crossing, use a root finding routine. If bracketing fails, use minimization.
Using Newton-Raphson is an act of desperation. You are much better off finding the continued fraction that represents your function and calculating that. A CF will converge much faster and will produce the real root(s). Also, because the CF produces a ratio of two integers you have tight control over numeric precision and don't have to worry about accumulation of rounding errors and other similar hair-pulling-out problems.
To find the real roots of any polynomial function refer to "A Continued Fraction Algorithm for Approximating All Real Polynomial Roots" by David Rosen (1978).
------------ ADDENDUM 1 --- 11 OCT-----------------
Ok, you are solving a sextic. You have several options. The simplest is to use a Taylor approximation (say to the 3rd degree) in conjunction with Halley's method. This is much superior to Newton because it has cubic convergence and you can detect imaginary solutions. The disadvantage is that you will have rounding problems which may result in an incorrect answer.
The ideal option is to find the continued fraction that represents the monic root, because this CF will be computable as an integer ratio of any desired precision, thus elminating the problem of rounding.
One approach to computing this CF is via the Jacobi-Perron algorithm. See the paper Hendy and Jeans: http://www.ams.org/mcom/1981-36-154/S0025-5718-1981-0606514-X/S0025-5718-1981-0606514-X.pdf. This paper shows the exact algorithm for computing cubic and quartic roots via CF approximation.
Note that if the sextic is reducible then it can converted into a quartic and quadratic: http://elib.mi.sanu.ac.rs/files/journals/tm/21/tm1124.pdf. The quartic is then solvable by the algorithm in the Hendy paper.
The general solution to generate a CF for a sextic can be done via the Rogers-Ramunajan CF. See the following paper for the method: http://arxiv.org/pdf/1111.6023v2. This will generate the CF for any sextic.
As in your case, you are interested in the real factorization of a real polynomial. One may see that all complex roots come in conjugate pairs which correspond to a real quadratic factor. By finding this real quadratic and completing the square to get the form (x-r)^2 + s you will be able to see the "real" even order root r with an "error" given by s. If s > 0 is too large, you may discard it as probably being complex. If s < 0 is also large, then you have two faraway real roots given by x = r ± √(-s). If s is very small then you might suspect r is a real double root and keep it.
Finding such a quadratic factor may be done using Bairstow's method, which actually applies a two-dimensional Newton method. This gives x^2 + ux + v and r = -u/2; s = v - r^2.
In C++,
const double Pi = 3.14159265;
cout << sin(Pi); // displays: 3.58979e-009
it SHOULD display the number zero
I understand this is because Pi is being approximated, but is there any way I can have a value of Pi hardcoded into my program that will return 0 for sin(Pi)? (a different constant maybe?)
In case you're wondering what I'm trying to do: I'm converting polar to rectangular, and while there are some printf() tricks I can do to print it as "0.00", it still doesn't consistently return decent values (in some cases I get "-0.00")
The lines that require sin and cosine are:
x = r*sin(theta);
y = r*cos(theta);
BTW: My Rectangular -> Polar is working fine... it's just the Polar -> Rectangular
Thanks!
edit: I'm looking for a workaround so that I can print sin(some multiple of Pi) as a nice round number to the console (ideally without a thousand if-statements)
What Every Computer Scientist Should Know About Floating-Point Arithmetic (edit: also got linked in a comment) is pretty hardcore reading (I can't claim to have read all of it), but the crux of it is this: you'll never get perfectly accurate floating point calculations. From the article:
Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation.
Don't let your program depend on exact results from floating point calculations - always allow a tolerance range. FYI 3.58979e-009 is about 0.0000000036. That's well within any reasonable tolerance range you choose!
Let's put it this way, 3.58979e-009 is as close to 0 as your 3.14159265 value is to the real Pi. What you got is, technically, what you asked for. :)
Now, if you only put 9 significant figures (8 decimal places) in, then instruct the output to also display no more, i.e. use:
cout.precision(8);
cout << sin(Pi);
it's equal to zero if your equality operator has enough tolerance
Did you try M_PI, available in most <cmath> or <math.h> implementations?
Even so, using floating point in this way will always introduce a certain amount of error.
This should display zero:
cout << fixed << sin(Pi);
(I don't think you should be trying to round anything. If you are worried about display, deal with the display functions, not with the value itself.)
3.58979e-009 this is 0,0000000358979
Is a ~~0 like yours ~~PI.
You could throw in some more digits to get a better result (try for example 3.1415926535897932384626433832795029L), but you'll still get rounding errors.
Still, you can create your own sin and cos versions that check against your known Pi value and return exactly zero in those cases.
namespace TrigExt
{
const double PI = 3.14159265358979323846;
inline double sin(double theta)
{
return theta==PI?(0.0):(std::sin(theta));
}
}
You may also expand this thing for the other trigonometric functions and to handle Pi multiples.
You could write a little wrapper function:
double mysin(const double d) {
double ret = sin(d);
if(fabs(ret) < 0.0000001) {
return 0.0;
} else {
return ret;
}
}
As others have noted, floating-point maths is notoriously inexact. You need some kind of tolerance if you want something to appear as exactly zero.
why not force to however many digits you need
int isin = (int)(sin(val) * 1000);
cout << (isin/1000.0)
sin(PI) should equal 0, for an exact value of PI. You are not entering the exact value of PI. As other people are pointing out, the result you are getting rounded to 7 decimal places is 0, which is pretty good for your approximation.
If you need different behavior you should write your own sine function.
If you use float or double in math operations you will never have exact results.
The reason is that in a computer everything is stored as a power of 2. This does not translate exactly to our decimal number system. (An example is that there is n o representation in base 2 of 0.1)
In addition float and double are 64 bits at least on some compilers and platforms. (I think - somebody correct me on that if needed). This will cause some rounding errors for either very large values or for very small values (0.0000000000xxx)
In order to get exact results you are going to need some big integer library.
As written in the comments to the question above see the site ...
http://docs.sun.com/source/806-3568/ncg_goldberg.html
double cut(double value, double cutoff=1e-7) {
return (abs(value) > cutoff)*value;
}
this will zero values below threshold, use it like this cut(sin(Pi))
More significant figures might help. My C compiler (gcc) uses the constant 3.14159265358979323846 for M_PI in "math.h". Other than that, there aren't many options. Creating your own function to validate the answer (as described in another answer to your question) is probably the best idea.
You know, just for the mathematical correctness out there: sin(3.14159265) ins't zero. It's approximately zero, which is exactly what the program is telling you. For calculations, this number ought to give you a good result. For displaying, it sucks, so whenever you print a float, make sure to format the number.
I don't really think that there are any float mechanics in the work here... it's just simple math.
About the code though, be careful... doesn't make your code give the wrong result by making the approximations before the display, just display the information the right way.
I have the following code using C++:
double value = .3;
double result = cos(value);
When I look at the values in the locals window for "value" it shows 0.2999999999
Then, when I get the value of "result" I get: 0.95533648912560598
However, when I run cos(.3) on the computers calculator I get: .9999862922474
So clearly there is something that I am doing wrong.
Any thoughts on what might be causing the difference in results?
I am running Win XP on an Intel processor.
Thanks
The difference in results is because:
Your computer's calculator is returning the cosine of an angle specified in degrees.
The C++ cos() function is returning cosine of an angle specified in radians.
The .2999999999 is due to the way floating point numbers are handled in computers. .3 cannot be represented exactly in a double. For details, I recommend reading What Every Computer Scientist Should Know about Floating Point Arithmetic.
cos(.3 radians) = 0.95533...
cos(.3 degrees) = 0.99998...
cos(0.3) = 0.99998629224742679269138848004408 using degrees
cos(0.3) = 0,95533648912560601964231022756805 using radians
When I look at the values in the locals window for "value" it shows 0.2999999999
Long story short, your calculator uses decimal arithmetic, while your C++ code uses binary arithmetic (double is a binary floating-point number). Decimal number 0.3 cannot be represented exactly as a binary floating-point number. Read What Every Computer Scientist Should Know About Floating-Point Arithmetic, that will explain all implications in more detail.
Your calculator is using degrees. For example:
>>> import math
>>> math.cos (.3)
0.95533648912560598
>>> math.cos (.3 * math.pi / 180) # convert to degrees
0.99998629224742674
C++ does not exactly represent floating point numbers due to the insane amount of storage that would be required to get the infinite precision necessary. For a demonstration of this, try the following:
double ninth = 1.0/9.0;
double result = 9.0 * ninth;
This should yield a value in result of .99999999999
So, in essence, you need to compare floating point values within a small epsilon (I tend to use 1e-7). You can do a strict bit-by-bit comparison, but this consists of converting the memory used by the floating point to an array of characters of length sizeof(float), then comparing the characters.
Another thing to check would be whether or not you are using degrees. The computer's calculator uses degrees for its cosine calculation (notice how the result from the calculator is .99999..., which is very close to 1. The cosine of zero is 1 exactly), whereas the cosine function offered in <math> is in radians. Try multiplying your value by PI/180.0 and seeing if the result is more inline with your expectations.