I am studying numerical analysis and also solving algorithms which is described in book. My problem is about Newton's method. In general, if some function is given and we have to find root, how can we determine derivative of function in code? or even limit? because as you know Newton's method involves derivative and makes iteration like this.
Suppose some function f(x) and initial guess,p0, then p(n)=p(n-1)+f(p(n-1))/f'(p(n-1)) here f' denotes derivative of f.
How can I approximate it in code? Thanks a lot.
If you want to use Newton's Method, you will need to know the derivative of the function and code it in.
Otherwise, you can go with the Secant Method which doesn't require knowing the derivative. But it converges at a slower rate.
Depending on how the function is given, you can do a couple of things
symbolic differentiation, if you have a symbolic representation of your function
Numerical differentiation, if you only have point-value pairs
Interpolate with a polynomial and differentiate that (symobolically of course)
All Options are viable. Which of these is most suited to your problem depends on the function and also the time you want to invest in coding and/or reading up on how to do it.
Edit: If you already know the function before execution time, then compute the differential by hand and implement it as a function. You should also already have implemented your f(x) as a function like this
float f (float x) {
// ...
}
And thus:
float df_dx (float x) {
// ...
}
Related
Suppose I want to implement a gradient descent based optimizer of some function f(x),
which is a combination of simple functions: +,*,sin,cos (leaving out / for simplicity)
Is there a way, using templates to symbolically calculate the derivative and make a function f'(x)
in C++, then use the function and its gradient at runtime to optimize it.
I'm comfortable with symbolic mathematics, so that isn't the focus of the question.
I could write a parser and input the function as a string, expanding it dynamically at runtime, but especially for more complex functions, this is liable to be slow.
If there's a way to produce the function at compile time, that would be awesome.
I am using the conjugate gradient method
gsl_multimin_fdfminimizer_conjugate_fr
in the GSL package to minimize a function f. This method requires derivatives, so I have passed the derivative in two ways:
static void dfun(const gsl_vector *v, void *params, gsl_vector *df) {
return fun(x+epsilon)-f(x)/epsilon
}
and another way which is less direct but faster. Both functions return the same values for the gradient in all variables, up to a precision of ~10 digits. However, when I run the minimization using the latter, I do not reach the minimum and the minimizer returns 27: unable to get a better result. When using dfun as the derivative, I converge to the correct minimum.
What may be causing this behavior, and what steps can I take to fix it?
I have a cyclic program in C++ which includes composing of a function (every time it is different) and further minimization of it. Composing of a function is implemented with GiNaC package (symbolic expressions).
I tried to minimize functions using Matlab fmincon function but it ate all the memory while converting string to lambda function (functions are rather complicated). And I couldn't manage to export function from C++ to Matlab in any way but as a string.
Is there any way to compose a complicated function (3 variables, sin-cos-square root etc.) and minimize it without determing gradient by myself because I don't know how functions look before running the program?
I also looked at NLopt and as I understood it requires gradients to be writte by programmer.
Most optimization algorithms do require the gradient. However, if it's impossible to 'know' it directly, you may evaluate it considering a small increment of every coordinate. If your F function depends on coordinates of x vector, you may approximate the i's component of you gradient vector G as
x1 = x;
x1[i] += dx;
G[i] = (F(x1) - F(x))/dx;
where dx is some small increment. Although such a calculation is approximate it's usually absolutely good for a minimum finding provided that dx is small enough.
I am trying to solve a minimisation problem and I want to minimise an expression
a/b
Where both a & b are variables. Hence this is not a linear problem...
How can I transform this function into an other one (being a linear one).
There is a detailed section on how to handle ratios in Linear Programming on the lpsolve site. It should be general enough to apply to AMPL and CPLEX as well.
There are several ways to do this, but the simplest to explain requires that you solve a series of linear programs. First, remove the objective and add a constraint
a <= c * b
Where c is a known upper bound on the solution. Then do a binary search on c you can a range where c_l, c_u where the problem is infeasible for
a <= c_l * b
but feasible for
a <= c_u * b
The general form of the obj should be a linear fractional function, something like f_{0}(x)=(c^Tx+d)/(e^Tx+f). For your case, X=(a,b),c=(1,0),(e=0,1),d=f=0.
To solve this kind of opt, something called linear fractional programming can be used. it's like linear constrainted version of linear fractional function and Charnes-Cooper transformation is applied to transform into a LP. You can find the main idea from wiki. Many OR books talk more about this such as pp53, pp165 in the Boyd's "convex optimization" (free to download).
Background
I need to solve polynomials in multiple variables using Horner's scheme in Fortran90/95. The main reason for doing this is the increased efficiency and accuracy that occurs when using Horner's scheme to evaluate polynomials.
I currently have an implementation of Horner's scheme for univariate/single variable polynomials. However, developing a function to evaluate multivariate polynomials using Horner's scheme is proving to be beyond me.
An example bivariate polynomial would be: 12x^2y^2+8x^2y+6xy^2+4xy+2x+2y which would factorised to x(x(y(12y+8))+y(6y+4)+2)+2y and then evaluated for particular values of x & y.
Research
I've done my research and found a number of papers such as:
staff.ustc.edu.cn/~xinmao/ISSAC05/pages/bulletins/articles/147/hornercorrected.pdf
citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.40.8637&rep=rep1&type=pdf
www.is.titech.ac.jp/~kojima/articles/B-433.pdf
Problem
However, I'm not a mathematician or computer scientist, so I'm having trouble with the mathematics used to convey the algorithms and ideas.
As far as I can tell the basic strategy is to turn a multivariate polynomial into separate univariate polynomials and compute it that way.
Can anyone help me? If anyone could help me turn the algorithms into pseudo-code that I can implement into Fortran myself, I would be very grateful.
For two variables one can store the polynomial coefficients in a rank=2 matrix K(n+1,n+1) where n is the order of the polynomial. Then observe the following pattern (in pseudo-code)
p(x,y) = (K(1,1)+y*(K(1,2)+y*(K(1,3)+...y*K(1,n+1))) +
x*(K(2,1)+y*(K(2,2)+y*(K(2,3)+...y*K(2,n+1))) +
x^2*(K(3,1)+y*(K(3,2)+y*(K(3,3)+...y*K(3,n+1))) +
...
x^n*(K(n+1,1)+y*(K(n+1,2)+y*(K(n+1,3)+...y*K(n+1,n+1)))
Each row is a separate homer's scheme in terms of y and all-together is a final homer's scheme in terms of x.
To code in FORTRAN or any language create an intermediate vector z(n+1) such that
z(i) = homers(y,K(i,1:n+1))
and
p = homers(x,z(1:n+1))
where homers(value,vector) is an implementation of the single variable evaluation with polynomial coefficients stored in vector.