I can calculate the Matrix representation of first degree Linear recurrence equations. And I calculate for higher order by using fast matrix exponentiation. I learnt this from this tutorial
http://fusharblog.com/solving-linear-recurrence-for-programming-contest/
But I am facing problem in calculating the matrix representation of Second Degree Linear recurrence equations. For example -
S(n) = a * (S(n - 1))^2 + b * S(n - 1) + c
where S(0) = d
Can you help me to figure out the matrix representation of the above equation or give me some insights? Thanks in advance.
This is polynomial of second degree. The well-known recurrence
x_(n+1) = (x_n)^2 + c
that is often called the quadratic map is not in general solvable in closed form. Quadratic iteration
x_(n+1) = a (x_n)^2 + b x_n + c
is iteration of the Mandelbrot fractals.
This is the real version of the complex map defining the Mandelbrot set.
Related
I am trying to write a short C++ routine to calculate the following function F(i,j,z) for given integers j > i (typically they lie between 0 and 100) and complex number z (bounded by |z| < 100), where L are the associated Laguerre Polynomials:
The issue is that I want this function to be callable from within a CUDA kernel (i.e. with a __device__ attribute). Standard library/Boost/etc functions are therefore out of the questions, unless they are simple enough to re-implement on my own - this especially relates to the Laguerre polynomials which exist in Boost and C++17. Regardless if I manage to wrap any standard function for Laguerre polynomials, I still have a similar pre-factor to calculate of the form (z^j/j!).
Question: How can I do a relatively simple implementation of such a function, without introducing significant numerical instability?
My idea so far is to calculate L and its pre-factor independently. The pre-factor I will calculate by first looping from 0 to j-i and calculate (z^1 * z^2/2 * ... * z^(j-1)/(j-i)!). I will then calculate the remaining factor exp(-|z|^2/2) *(j-i)! * sqrt(i!/j!) (either in a similar way, or through the Gamma-function, which is implemented in CUDA math). The idea is then to find a minimal algorithm to calculate the associated Laguerre polynomial, unless I manage to wrap an implementation from e.g. Boost or GNU C++.
Edit/side note: The expression for F actually blows up numerically for some values of i/j. It was derived wrong in the source where I got it, and the indices of the associated Laguerre polynomials should instead be L_i^(j-i). That does not invalidate the approaches suggested in the answers/comments.
I recommend finding a recurrence relation for the coefficients of the Laguerre Polynomial:
C(k+1) = g(k)C(k)
g(k) = C(k+1) / C(k)
g(k) = -z * (j - k) / ((j - i + k + 1) * (k + 1)) //Verify this yourself :)
This allows you to avoid most of factorials in computing the polynomial.
After that I would follow Severin's idea of doing the calculations in logarithms
so as to not overload the double floating point range:
log(F) = log(sqrt(i!/j!)) - |z|^2 + (j-i) * log(-z) + log(L(|z|^2))
log(L) = log((2*j - i)!) + log(sum) // where the summation is computed using the recurrence relation above
and using the fact that:
log(a!) = sum(k=1..a, log(k))
and also:
log(z) = log(|z|) + I * arg(z) for complex z
log(-z) = log(|z|) + I * arg(-z)
log(-z) = log(|z|) - I * arg(z)
for the log(sqrt(i!/j!)) part I would do (assuming that j >= i):
log(sqrt(i!/j!))
= 0.5 * (log(i!) - log(j!))
= -0.5 * sum(k==i+1..j, log(k))
I haven't tried this out so there could definitely be little mistakes here and there. This answer is more about the technique rather than a copy-paste-ready answer
Well, what you should do is to logarithm it
Assuming natural logarithm,
q = log(z^j/j!) = log(z^j) - log(j!) = j*log(z) - log(Gamma(j+1))
First term is simple, second term is standard C++ function lgamma(x) (or you could use GSL).
compute value of q and return cexp(q)
You could fold exponent in this method as well
Consider fast matrix multiplication of XDX^T for X an n by m matrix, and D an m by m diagonal matrix. Here m>>n (suppose n around 1000, m around 100000). In my application, X is a fixed matrix and values of D can change at every iteration.
What would be a fast way to calculate this? At the moment I am just doing simple multiplication in C++.
EDIT: I should clarify my current procedure, it is not "simple multiplication". In particular, I am columnise multiplying the X by the square root of diagonal entries of D to get A:=XD^{1/2}. Then I am directly calculating A*t(A) (which is the multiplication of an n by m matrix with its transpose).
Thank you.
If you know that D is diagonal, the you can just do simple multiplication. Hopefully, you are not multiplying the zeros.
Motivation. It is well known that generating function for Catalan numbers satisfies quadratic equation. I would like to have first several coefficients of a function, implicitly defined by an algebraic equation (not necessarily a quadratic one!).
Example.
import sympy as sp
sp.init_printing() # math as latex
from IPython.display import display
z = sp.Symbol('z')
F = sp.Function('F')(z)
equation = 1 + z * F**2 - F
display(equation)
solution = sp.solve(equation, F)[0]
display(solution)
display(sp.series(solution))
Question. The approach where we explicitly solve the equation and then expand it as power series, works only for low-degree equations. How to obtain first coefficients of formal power series for more complicated algebraic equations?
Related.
Since algebraic and differential framework may behave differently, I posted another question.
Sympy: how to solve differential equation in formal power series?
I don't know a built-in way, but plugging in a polynomial for F and equating the coefficients works well enough. Although one should not try to find all coefficients at once from a large nonlinear system; those will give SymPy trouble. I take iterative approach, first equating the free term to zero and solving for c0, then equating 2nd and solving for c1, etc.
This assumes a regular algebraic equation, in which the coefficient of z**k in the equation involves the k-th Taylor coefficient of F, and does not involve higher-order coefficients.
from sympy import *
z = Symbol('z')
d = 10 # how many coefficients to find
c = list(symbols('c:{}'.format(d))) # undetermined coefficients
for k in range(d):
F = sum([c[n]*z**n for n in range(k+1)]) # up to z**k inclusive
equation = 1 + z * F**2 - F
coeff_eqn = Poly(series(equation, z, n=k+1).removeO(), z).coeff_monomial(z**k)
c[k] = solve(coeff_eqn, c[k])[0]
sol = sum([c[n]*z**n for n in range(d)]) # solution
print(series(sol + z**d, z, n=d)) # add z**d to get SymPy to print as a series
This prints
1 + z + 2*z**2 + 5*z**3 + 14*z**4 + 42*z**5 + 132*z**6 + 429*z**7 + 1430*z**8 + 4862*z**9 + O(z**10)
Using Chebyshev polynomials, we can compute sin(2*Pi/n) exactly using the CGAL and CORE library, like the following piece of codes:
#include <CGAL/CORE_Expr.h>
#include <CGAL/Polynomial.h>
#include <CGAL/number_utils.h>
//return sin(theta) and cos(theta) for theta = 2pi/n
static std::pair<AA, AA> sin_cos(unsigned short n) {
// We actually use -x instead of x since root_of will give the k-th
// smallest root but we want the second largest one without counting.
Polynomial x(CGAL::shift(Polynomial(-1), 1));
Polynomial twox(2*x);
Polynomial a(1), b(x);
for (unsigned short i = 2; i <= n; ++i) {
Polynomial c = twox*b - a;
a = b;
b = c;
}
a = b - 1;
AA cos = -CGAL::root_of(2, a.begin(), a.end());
AA sin = CGAL::sqrt(AA(1) - cos*cos);
return std::make_pair(sin, cos);
}
But if I want to compute sin(2*m*Pi/n) exactly, where m and n are integers, what is the formula of the polynomial that I should use? Thanks.
(Partial solution.)
This is essentially computing the real and imaginary part of the roots of unity as algebraic numbers. Let's denote w(m) = exp(2*pi*I*m/n). Then, w(m) itself is a complex root of En(x) = x^n-1.
You need to find a defining polynomial of Re(w(m)). Resultants are a tool to find such a polynomial: 2*Re(w(m)) is a root of Res (En(x-y), En(y); y).
For an explanation why this is the case: Note that 2*Re(w(m)) = w(m) + conj(w(m)), and that the complex roots of En come in conjugate pairs; hence, also conj(w(m)) is a root of En. Now loosely speaking, the En(y) part "constrains" y to be any (complex) root of En, and combining this with the first argument allows x to take any complex value such that x-y is a root of En as well. Hence, a possible assignment is y = conj(w(m)) and x-y = w(m), hence x = w(m)+conj(w(m)) = 2*Re(w(m)).
CGAL can compute resultants of multivariate polynomials, so you can compute this resultant, and you simply have to pick the correct real root. (The largest one will obviously be w(0) = 1, the smallest one is 2*Re(w(floor(n/2))).)
Unfortunately, the resultant has a high complexity (degree n^2), and resultant computation will not be the fastest operation you've ever seen. Also, you'll pay for dense polynomials although your instances are very sparse and structured. YMMV; I have no clue about your use case, and if you need higher degrees.
However, I did a few tests in a computer algebra system, and I found that the resultant splits into factors of more reasonable size, and that all its real roots actually belong to a much simpler polynomial of degree floor(n/2)+1 only. (No proof, just an observation.)
I don't know of a direct formula to write down this factor, and I don't want to speculate about it. But maybe some people at mathoverflow or math.stackexchange can help?
EDIT: Here is a guess for at least a recursive formula.
I write s(n,x) for the significant factor of the resultant polynomial containing all real roots but 0. This means that s(n,x) has all values 2*Re(w(m)) for m != n/4, 3*n/4 as roots.
s(0,x) = 0
s(1,x) = x - 2
s(2,x) = x^2 - 4
s(3,x) = x^2 - x - 2
s(4,x) = x^2 - 4
s(5,x) = x^3 - x^2 - 3*x + 2
s(6,x) = x^4 - 5*x^2 + 4
s(7,x) = x^4 - x^3 - 4*x^2 + 3*x + 2
s(8,x) = x^4 - 6*x^2 + 8
s(n,x) = (x^2-2)*s(n-4,x) - s(n-8,x)
Waiting for a proof...
I have implemented a MCMC algorithm in C++ using the Eigen library. The main part of the algorithm is a loop in which first some some matrix calculations are performed after which the determinant of the resulting matrix is obtained and added to the output. E.g.:
MatrixXd delta0;
NumericVector out(3);
out[0] = 0;
out[1] = 0;
for (int i = 0; i < s; i++) {
...
delta0 = V*(A.cast<double>()-(A+B).cast<double>()*theta.asDiagonal());
...
I = delta0.determinant()
out[1] += I;
out[2] += std::sqrt(I);
}
return out;
Now on certain matrices I unfortunately observe a numerical underflow so that the determinant is outputted as zero (which it actually isn't).
How can I avoid this underflow?
One solution would be to obtain, instead of the determinant, the log of the determinant. However,
I do not know how to do this;
how could I then add up these logs?
Any help is greatly appreciated.
There are 2 main options that come to my mind:
The product of eigenvalues of square matrix is the determinant of this matrix, therefore a sum of logarithms of each eigenvalue is a logarithm of the determinant of this matrix. Assume det(A) = a and det(B) = b for compact notation. After applying aforementioned for 2 matrices A and B, we end up with log(a) and log(b), then actually the following is true:
log(a + b) = log(a) + log(1 + e ^ (log(b) - log(a)))
Yes, we get a logarithm of the sum. What would you do with it next? I don't know, depends on what you have to. If you have to remove logarithm by e ^ log(a + b) = a + b, then you might be lucky that the value of a + b does not underflow now, but in some cases it can still underflow as well.
Perform clever preconditioning; there might be tons of options here, and you better read about them from some trusted sources as this is a serious topic. The simplest (and probably the cheapest ever) example of preconditioning for this particular problem could be to recall that det(c * A) = (c ^ n) * det(A), where A is n by n matrix, and to premultiply your matrix with some c, compute the determinant, and then to divide it by c ^ n to get the actual one.
Update
I thought about one more option. If on the last stages of #1 or #2 you still experience underflow too frequently, then it might be a good idea to increase precision specifically for these last operations, for example, by utilizing GNU MPFR.
You can use Householder elimination to get the QR decomposition of delta0. Then the determinant of the Q part is +/-1 (depending on whether you did an even or odd number of reflections) and the determinant of the R part is the product of the diagonal elements. Both of these are easy to compute without running into underflow hell---and you might not even care about the first.