I'm trying to solve a system of first order differential equations that looks like this:
import numpy as np
from sympy import *
tR1201 = Rational(1,2)
tR1212 = Rational(1,2)
tB12 = Rational(1,2)
tB01 = Rational(1,2)
Gamma = Rational(1,4)
delta = Rational(1,10)
hbar = 1
t = Symbol('t')
p1 = Function('p1')(t)
p2 = Function('p2')(t)
p3 = Function('p3')(t)
p4 = Function('p4')(t)
Psi = Matrix([[p1],[p2],[p3],[p4]])
Coupling = Matrix([[0,tB01,tR1201,0],[tB01,0,0,tR1212],[tR1201,0,delta +(-I*Gamma),tB12],[0,tR1212,tB12,delta +(-I*Gamma)]])
diffeq = I*hbar * diff(Psi, t) - Coupling*Psi
solution = dsolve(diffeq,[p1,p2,p3,p4])
When I run it I get this error:
line 1122, in eigenvals raise MatrixError("Could not compute
eigenvalues for {}".format(self))
MatrixError: Could not compute eigenvalues for Matrix([[0, 1/2, 1/2, 0], [1/2, 0, 0, 1/2], [1/2, 0, 1/10 - I/4, 1/2], [0, 1/2, 1/2, 1/10 - I/4]])
Part of dsolve involves finding eigenvalues, and this error suggests it's not possible to compute eigenvalues for the matrix Coupling. However, I've found the eigenvalues with another method and they are complex. Are complex eigenvalues an issue for Sympy?
I've found similar questions asked (like this one: Computation of symbolic eigenvalues with sympy) but the answers to these other questions did not help in this case.
Related
I am not pratice in Sympy manipulation.
I need to find roots on particular poly:
-4x**(11/2)-24x**(9/2)-16x**(7/2)+2x**(5/2)+16x**(5)+23x**(4)+5x**(3)-x**(2)
I verified that I have 2 real solution and I find one of them with Sympy function
nsolve(mypoly,x,1).
Why the previous step doesn't look the other?
How can I proceed to find ALL roots?
Thank you to all for assistance
A.
To my knowledge, nsolve looks in the proximity of the provided initial guess to find one root for each equations.
I would plot the expression to find suitable initial guesses:
from sympy import *
from sympy.plotting import PlotGrid
expr = -4*x**(S(11)/2)-24*x**(S(9)/2)-16*x**(S(7)/2)+2*x**(S(5)/2)+16*x**(5)+23*x**(4)+5*x**(3)-x**(2)
p1 = plot(expr, (x, 0, 0.5), adaptive=False, n=1000, ylim=(-0.01, 0.05), show=False)
p2 = plot(expr, (x, 0, 5), adaptive=False, n=1000, ylim=(-200, 200), show=False)
PlotGrid(1, 2, p1, p2)
Now, we can do:
nsolve(expr, x, 0.2)
# out: 0.169003536680445
nsolve(expr, x, 4)
# out: 4.28968831654177
EDIT: to find all roots (even the complex one), we can:
compute the derivative of the expression.
convert both the expression and the derivative to numerical functions with sympy's lambdify.
visually inspect the expression in the complex plane to determine good initial values for the root finding algorithm. I'm going to use this plotting module, SymPy Plotting Backend which exposes a very handy function, plot_complex, to generate domain coloring plots. In particular, I will plot alternating black and white stripes corresponding to modulus.
use scipy's newton method to compute the actual roots. EDIT: I just discovered that nsolve works too :)
# step 1 and 2
f = lambdify(x, expr)
f_der = lambdify(x, expr.diff(x))
# step 3
from spb import plot_complex
r = (x, -1-0.8j, 4.5+0.8j)
w = r[1].real - r[2].real
h = r[1].imag - r[2].imag
# number of discretization points, watch out memory usage
n1 = 1500
n2 = int(h / w * n1)
plot_complex(expr, r, {"interpolation": "spline36"}, grid=False, coloring="e", n1=n1, n2=n2, size=(10, 5))
In the above picture we see circular stripes getting bigger and deforming. The center of these circular stripes represent a pole or a zero. But this is an easy case: there are no poles. So, from the above pictures we count 7 zeros. We already know 3, the two computed above and the value 0. Let's find the others:
from scipy.optimize import newton
r1 = newton(f, x0=-0.9+0.1j, fprime=f_der)
r2 = newton(f, x0=-0.9-0.1j, fprime=f_der)
r3 = newton(f, x0=0.6+0.6j, fprime=f_der)
r4 = newton(f, x0=0.6-0.6j, fprime=f_der)
for r in (r1, r2, r3, r4):
print(r, ": is it a zero?", expr.subs(x, r).evalf())
# out:
# (-0.9202719950522663+0.09010409402273806j) : is it a zero? -8.21787666002984e-15 + 2.06697764417957e-15*I
# (-0.9202719950522663-0.09010409402273806j) : is it a zero? -8.21787666002984e-15 - 2.06697764417957e-15*I
# (0.6323265751497729+0.6785871500619469j) : is it a zero? -2.2103533615688e-15 - 2.77549897301442e-15*I
# (0.6323265751497729-0.6785871500619469j) : is it a zero? -2.2103533615688e-15 + 2.77549897301442e-15*I
As you can see, inserting those values into the original expression get values very very close to zero. It is perfectly normal to see these kind of errors.
I just discovered that you can use also use nsolve instead of newton to compute complex roots. This makes step 1 and 2 unnecessary.
nsolve(expr, x, -0.9+0.1j)
# out: −0.920271995052266+0.0901040940227375𝑖
I'm solving an eigenvalue problem when the matrix and the eigenvectors are time dependent. The matrix has dimension 8x8 and is hermitian. The time dependent matrix has the form:
import sympy as sp
t, lbd = sp.symbols(r't,\lambda', real=True)
Had = ...
print(repr(Had))
Matrix([[2*t,0, 0, 0, 0, 0, 0,0],
[ 0,-2*t, 2*t*(1 - t), 0, 0, 0,0,0],
[0, 2*t*(1 - t),0,0, 2 - 2*t, 0,0,0],
[0,0,0,0, 0, 2 - 2*t, 0,0],
[0,0,2 - 2*t,0,0,0,0,0],
[0,0,0, 2 - 2*t,0,0, 2*t*(1 - t),0],
[0,0,0,0,0, 2*t*(1 - t),-2*t,0],
[0,0,0,0,0,0,0,2*t]])
Now the characteristic polynomial has the following for:
P = p.simplify(sp.collect(sp.factor(Had.charpoly(lbd).as_expr()),lbd))
and get
Then I choose the second term and find the solution for lambda:
P_list = sp.factor_list(P)
a,b = P_list[1]
eq,exp = sp.simplify(b)
sol = sp.solve(eq)
With that I get the roots in a list:
r_list = []
for i in range(len(sol)):
a = list(sol[i].values())
r_list.append(a[0])
Solving the problem using sp.eigenvecs:
val_mult_vec = Had.eigenvects()
e_vals = []
mults = []
e_vecs = []
for i in range(len(val_mult_vec)):
val, mult, [vec_i, vec_j] = val_mult_vec[i]
e_vals.append(val)
e_vals.append(val)
mults.append(mult)
e_vecs.append(vec_i)
e_vecs.append(vec_j)
Solving the eigenvectors I get complicated expressions like this:
But I know that this complicated expression can be expressed in terms of the solution of the second term in the characteristic polynomial something like this:
Where r1 are one of the roots of that equation. With the solution to the characteristic polynomial how can I rewrite the eigenvectors in a simplified way like the last image using sympy? rewrite e_vec[i] in terms of r_list[j]
Seems like you want to obtain a compact version of the eigenvectors.
Recepy:
We can create as many symbols as the number of eigenvalues. Each symbol represents an eigenvalue.
Loop over the eigenvectors and for each of its elements substitute the long eigenvalue expression with the respective symbol.
r_symbols = symbols("r0:%s" % len(e_vals))
final_evecs = []
for vec, val, s in zip(e_vecs, e_vals, r_symbols):
final_evecs.append(
vec.applyfunc(lambda t: t.subs(val, s))
)
final_evecs is a list containing eigenvectors in a compact notation.
Let's test one output:
final_evecs[7]
Motivation. It is well known that generating function for Catalan numbers satisfies quadratic equation. I would like to have first several coefficients of a function, implicitly defined by an algebraic equation (not necessarily a quadratic one!).
Example.
import sympy as sp
sp.init_printing() # math as latex
from IPython.display import display
z = sp.Symbol('z')
F = sp.Function('F')(z)
equation = 1 + z * F**2 - F
display(equation)
solution = sp.solve(equation, F)[0]
display(solution)
display(sp.series(solution))
Question. The approach where we explicitly solve the equation and then expand it as power series, works only for low-degree equations. How to obtain first coefficients of formal power series for more complicated algebraic equations?
Related.
Since algebraic and differential framework may behave differently, I posted another question.
Sympy: how to solve differential equation in formal power series?
I don't know a built-in way, but plugging in a polynomial for F and equating the coefficients works well enough. Although one should not try to find all coefficients at once from a large nonlinear system; those will give SymPy trouble. I take iterative approach, first equating the free term to zero and solving for c0, then equating 2nd and solving for c1, etc.
This assumes a regular algebraic equation, in which the coefficient of z**k in the equation involves the k-th Taylor coefficient of F, and does not involve higher-order coefficients.
from sympy import *
z = Symbol('z')
d = 10 # how many coefficients to find
c = list(symbols('c:{}'.format(d))) # undetermined coefficients
for k in range(d):
F = sum([c[n]*z**n for n in range(k+1)]) # up to z**k inclusive
equation = 1 + z * F**2 - F
coeff_eqn = Poly(series(equation, z, n=k+1).removeO(), z).coeff_monomial(z**k)
c[k] = solve(coeff_eqn, c[k])[0]
sol = sum([c[n]*z**n for n in range(d)]) # solution
print(series(sol + z**d, z, n=d)) # add z**d to get SymPy to print as a series
This prints
1 + z + 2*z**2 + 5*z**3 + 14*z**4 + 42*z**5 + 132*z**6 + 429*z**7 + 1430*z**8 + 4862*z**9 + O(z**10)
I need to solve a classic problem of the form Ax = b for a vector x that is of size 4. A is on the order of ~500 data points and thus is a dense 500x4 matrix.
Currently I can solve this using the normal equations described here and it works fine however I would like to constrain one of my parameters in x to never be above a certain value.
Is there a good way to do this programmatically with Eigen?
You can try my quadradic programming solver based on Eigen there. You'll still have to form the normal equation.
Here is some python-demo (using numpy which is not that far away from Eigen) showing an accelerated projected-gradient algorithm for your kind of problem. Typically this approach is used for large-scale problems (where other algorithms incorporating second-order information might struggle), but it's also nice to implement.
This version, which is a small modification from some old code of mine is not the most simple approach which can be used, as we are using:
acceleration / momentum (faster iteration)
line-search (saves us some step-size tuning trouble)
You could remove the line-search and tune the step-size. Momentum is also not needed.
As i'm not doing much C++ right now, i don't think i will port this to Eigen. But i'm sure, that if you would wanto to port it, it's not that hard. Eigen should not be too different from numpy.
I did not measure the performance, but the results are calculated instantly (perceptually).
Edit: some non-scientific timings (more momentum; lesser tolerance than in following code):
A=(500,4):
Solve non-bounded with scipys lsq_linear
used: 0.004898870004675975
cost : 244.58267993
Solve bounded (0.001, 0.05) with scipys lsq_linear
used: 0.005605718416479959
cost : 246.990611225
Solve bounded (0.001, 0.05) with accelerated projected gradient
early-stopping # it: 3
used: 0.002282825315435914
cost: 246.990611225
A=(50000, 500):
Solve non-bounded with scipys lsq_linear
used: 4.118898701951786 secs
cost : 24843.7115776
Solve bounded (0.001, 0.05) with scipys lsq_linear
used: 14.727660030288007 secs
cost : 25025.0328661
Solve bounded (0.001, 0.05) with accelerated projected gradient
early-stopping # it: 14
used: 5.319953458329618 secs
cost: 25025.0330754
The basic idea is to use some gradient-descent-like algorithm, and project onto our constraints after each gradient-step. This approach is very powerful, if that projection can be done efficiently. Box-constraint-projections are simple!
Page 4 in this pdf shows you the box-constraint projection.
We just clip our solution-vector to lower_bound, upper_bound. Clipping in numpy is described as: Given an interval, values outside the interval are clipped to the interval edges. For example, if an interval of [0, 1] is specified, values smaller than 0 become 0, and values larger than 1 become 1.
It's an iterative-algorithm approximating the solution and i think every algorithm in use will be an iterated one.
Code
import numpy as np
from scipy.optimize import lsq_linear
np.random.seed(1)
A = np.random.normal(size=(500, 4))
b = np.random.normal(size=500)
""" Solve Ax=b
----------
"""
print('Solve non-bounded with scipys lsq_linear')
sol = lsq_linear(A, b)
print('Ax=b sol: ', sol['x'])
print('cost : ', sol['cost'])
print()
""" Solve Ax=b with box-constraints
-------------------------------
"""
print('Solve bounded (0.001, 0.05) with scipys lsq_linear')
sol = lsq_linear(A, b, bounds=(0.001, 0.05))
print('Ax=b constrained sol: ', sol['x'])
print('cost : ', sol['cost'])
print()
""" Solve Ax=b with box-constraints using a projected gradient algorithm
--------------------------------------------------------------------
"""
def solve_pg(A, b, bounds=(-np.inf, np.inf), momentum=0.9, maxiter=1000):
""" remarks:
algorithm: accelerated projected gradient
projection: proj onto box constraints
line-search: armijo-rule along projection-arc (Bertsekas book)
stopping-criterion: naive
gradient-calculation: precomputes AtA
"""
lb = np.empty(A.shape[1])
ub = np.empty(A.shape[1])
if len(bounds) == 2:
# apply lb & ub to all variables
lb = bounds[0]
ub = bounds[1]
else:
# assume dimensions are ok
lb = np.array(bounds[0])
ub = np.array(bounds[1])
M, N = A.shape
x = np.zeros(N)
AtA = A.T.dot(A)
Atb = A.T.dot(b)
stop_count = 0
def gradient(x):
return AtA.dot(x) - Atb
def obj(x):
return 0.5 * np.linalg.norm(A.dot(x) - b)**2
it = 0
while True:
grad = gradient(x)
# line search
alpha = 1
beta = 0.5
sigma=1e-2
old_obj = obj(x)
while True:
new_x = x - alpha * grad
new_obj = obj(new_x)
if old_obj - new_obj >= sigma * grad.dot(x - new_x):
break
else:
alpha *= beta
x_old = x[:]
x = x - alpha*grad
# projection
np.clip(x, lb, ub, out=x) # Projection onto box constraints
# see SO-text
# in-place clipping
y = x + momentum * (x - x_old)
if np.abs(old_obj - obj(x)) < 1e-2:
stop_count += 1
else:
stop_count = 0
if stop_count == 3:
print('early-stopping # it: ', it)
return x
it += 1
if it == maxiter:
return x
print('Solve bounded (0.001, 0.05) with accelerated projected gradient')
sol = solve_pg(A, b, bounds=(0.001, 0.05))
print(sol)
print('cost: ', 0.5 * (np.square(np.linalg.norm(A.dot(sol) - b))))
Output
Solve non-bounded with scipys lsq_linear
Ax=b sol: [ 0.06627173 -0.06104991 -0.07010355 0.04024075]
cost : 244.58267993
Solve bounded (0.001, 0.05) with scipys lsq_linear
Ax=b constrained sol: [ 0.05 0.001 0.001 0.03902291]
cost : 246.990611225
Solve bounded (0.001, 0.05) with accelerated projected gradient
early-stopping # it: 3
[ 0.05 0.001 0.001 0.03902229]
cost: 246.990611225
it is quite common to calculate the probability density of a value within a probability density function (PDF). Imagine we have a gaussian distribution with mean = 40, a standard deviation of 5 and now would like to get the probability density of value 32. We'd go like:
In [1]: import scipy.stats as stats
In [2]: print stats.norm.pdf(32, loc=40, scale=5)
Out [2]: 0.022
--> The probability density is 2.2%.
But now, let's consider the inverse problem. I have the mean value, I have the value at probabilty density of 0.05 and I would like to get the standard deviation (i.e. the scale parameter).
What I could implement is a numerical approach: create stats.norm.pdf several times with the scale-parameter increased stepwise and take that one with the result getting as closest as possible.
In my case, I specify the value 30 as the 5% mark. So I need to solve this "equation":
stats.norm.pdf(30, loc=40, scale=X) = 0.05
There is a scipy function called "ppf" which is the inverse of the PDF, so it will return the value for a specific probability density, but I haven't found a function to return the scale parameter.
Implementing an iteration would take too much time (both creating and calculating). My script is going to be huge, so I should save computation time. Could the lambda-function help in this case? I roughly know what it's doing, but I haven't used it so far. Any ideas on this?
Thank you!
The normal probability density function, f is given by
Given f and x we wish to solve for 𝞼. Let's ask sympy if it can solve the equation:
import sympy as sy
from sympy.abc import x, y, sigma
expr = (1/(sy.sqrt(2*sy.pi)*sigma) * sy.exp(-x**2/(2*sigma**2))) - y
ans = sy.solve(expr, sigma)[0]
print(ans)
# sqrt(2)*exp(LambertW(-2*pi*x**2*y**2)/2)/(2*sqrt(pi)*y)
So it appears there is a closed-formed solution in terms of the LambertW function, W, which satisfies
z = W(z) * exp(W(z))
for all complex-valued z.
We could use sympy to also find the numerical result for given x and y, but
perhaps it would be faster to do the numerical work with
scipy.special.lambertw:
import numpy as np
import scipy.special as special
def sigma_func(x, y):
results = set([np.real_if_close(
np.sqrt(2)*np.exp(special.lambertw(-2*np.pi*x**2*y**2, k=k)/2)
/(2*np.sqrt(np.pi)*y)).item() for k in (0, -1)])
results = [s for s in results if np.isreal(s)]
return results
In general, the LambertW function returns complex values, but we are only
interested in real-valued solutions for sigma. Per the
docs,
special.lambertw has two partially-real branches, when k=0 and k=1. So the
code above checks if the returned value (for those two branches) is real, and
returns a list of any real solutions if they exist. If no real solution exists,
then an empty list is returned. That happens if the pdf value y is not
attained for any real value of sigma (for the given value of x).
You can use it like this:
x = 30.0
loc = 40.0
y = 0.02
s = sigma_func(loc-x, y)
print(s)
# [16.65817044316178, 6.830458938511113]
import scipy.stats as stats
for si in s:
assert np.allclose(stats.norm.pdf(x, loc=loc, scale=si), y)
In the example you gave, with y = 0.025, there is no solution for sigma:
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
x = 30.0
loc = 40.0
y = 0.025
s = np.linspace(5, 20, 100)
plt.plot(s, stats.norm.pdf(x, loc=loc, scale=s))
plt.hlines(y, 4, 20, color='red') # the horizontal line y = 0.025
plt.ylabel('pdf')
plt.xlabel('sigma')
plt.show()
and so sigma_func(40-30, 0.025) returns an empty list:
In [93]: sigma_func(40-30, 0.025)
Out [93]: []
The plot above is typical in the sense that when y is too large there are zero
solutions, at the maximum of the curve (let's call it y_max) there is one
solution
In [199]: y_max = np.nextafter(np.sqrt(1/(np.exp(1)*2*np.pi*(10)**2)), -np.inf)
In [200]: y_max
Out[200]: 0.024197072451914336
In [201]: sigma_func(40-30, y_max)
Out[201]: [9.9999999776424]
and for y smaller than the y_max there are two solutions.
The will be two solutions, because normal PDF is symmetric around the mean.
As it stands, you have a single-variable equation to solve. It won't have a closed-form solution, so you can use e.g. scipy.optimize.fsolve to solve it.
EDIT: see #unutbu's answer for the closed form solution in terms of Lambert W function.