Symbolic entropy maximization in SymPy - sympy

A simple problem of entropy maximization in statistical mechanics in Physics, is formulated as follows.
The goal is to maximize an entropy function (LaTeX is still missing in stackexchange?):
H = - sum_x P_x ln P_x
subject to the following constraints: the normalization constraint
1 = sum_x P_x
and the constraint of average energy
U = sum_i E_x P_x
where the index i runs over x=1,2,...,n. E_x represents the energy of the system when it is in microscopic state x and P_x is the probability for the system to be in the microscopic state x.
The solution to such a problem can be obtained by the method of Lagrange multipliers. In this context, it works as follows...
Firstly, the Lagrangian is defined as
L = H + a( 1 - sum_i P_x ) + b( U - sum_i P_x E_x )
Here, a and b are the Lagrange multipliers. The Lagrangian L is a function of a, b and the probabilities P_x for x=1,2,...,n. The term a( 1 - sum_x P_x ) correspond to the normalization constraint and the term b( E - sum_x P_x E_x ) to the average energy constraint.
Secondly, the partial derivatives of L with respect to a, b and the P_x for the different x=1,2,...,n are calculated. These result in
dL/da = 1 - sum_x P_x
dL/db = E - sum_x E_x P_x
dL/P_x = dH/P_x - a - b E_x
= - ln P_x - 1 - a - b E_x
Thirdly, we find the solution by equating these derivatives to zero. This makes sense since there are 2+n equations and we have 2+n unknowns: the P_x, a and b. The solution from these equations read
P_x = exp( - b E_x ) / Z
where
Z = sum_x exp( - b E_x )
and b is implicitly determined by the relation
E = sum_x P_x E_x = ( 1 / Z ) sum_x exp( -b E_x ) E_x
Now that the "mathematical" problem is defined, lets state the "computational" problem (which is the one I want to ask here).
The computational problem is the following: I would like to reproduce the previous derivation in Sympy.
The idea is to automatize the process so, eventually, I can attack similar but more complicate problems.
I already made certain progress. Still, I think I haven't used the best approach. This is my solution.
# Lets attempt to derive these analytical result using SymPy.
import sympy as sy
import sympy.tensor as syt
# Here, n is introduced to specify an abstract range for x and y.
n = sy.symbols( 'n' , integer = True )
a , b = sy.symbols( 'a b' ) # The Lagrange-multipliers.
x = syt.Idx( 'x' , n ) # Index x for P_x
y = syt.Idx( 'y' , n ) # Index y for P_y; this is required to take derivatives according to SymPy rules.
>>> P = syt.IndexedBase( 'P' ) # The unknowns P_x.
>>> E = syt.IndexedBase( 'E' ) # The knowns E_x; each being the energy of state x.
>>> U = sy.Symbol( 'U' ) # Mean energy.
>>>
>>> # Entropy
>>> H = sy.Sum( - P[x] * sy.log( P[x] ) , x )
>>>
>>> # Lagrangian
>>> L = H + a * ( 1 - sy.Sum( P[x] , x ) ) + b * ( U - sy.Sum( E[x] * P[x] , x ) )
>>> # Lets compute the derivatives
>>> dLda = sy.diff( L , a )
>>> dLdb = sy.diff( L , b )
>>> dLdPy = sy.diff( L , P[y] )
>>> # These look like
>>>
>>> print dLda
-Sum(P[x], (x, 0, n - 1)) + 1
>>>
>>> print dLdb
U - Sum(E[x]*P[x], (x, 0, n - 1))
>>>
>>> print dLdPy
-a*Sum(KroneckerDelta(x, y), (x, 0, n - 1)) - b*Sum(KroneckerDelta(x, y)*E[x], (x, 0, n - 1)) + Sum(-log(P[x])*KroneckerDelta(x, y) - KroneckerDelta(x, y), (x, 0, n - 1))
>>> # The following approach does not work
>>>
>>> tmp = dLdPy.doit()
>>> print tmp
-a*Piecewise((1, 0 <= y), (0, True)) - b*Piecewise((E[y], 0 <= y), (0, True)) + Piecewise((-log(P[y]) - 1, 0 <= y), (0, True))
>>>
>>> sy.solve( tmp , P[y] )
[]
>>> # Hence, we try an ad-hoc procedure
>>> Px = sy.Symbol( 'Px' )
>>> Ex = sy.Symbol( 'Ex' )
>>> tmp2 = dLdPy.doit().subs( P[y] , Px ).subs( E[y] , Ex ).subs( y , 0 )
>>> print tmp2
-Ex*b - a - log(Px) - 1
>>> Px = sy.solve( tmp2 , Px )
>>> print Px
[exp(-Ex*b - a - 1)]
Is there a "better" way to proceed? In particular, I don't like the idea of substituting P[y] with Px and E[y] with Ex in order to solve the equations. Why the equations cannot be solved in terms of P[y]?

Related

SymPy `subs` Not Doing Anything

I have a differential equation for which I use sympy.solvers.ode.dsolve to solve, I get out
___________ ___________
-x⋅╲╱ E - V_max x⋅╲╱ E - V_max
ψ(x) = C₁⋅ℯ + C₂⋅ℯ
From (I put some of the code to I used to generate this equation at the end):
2
d
-ψ(x)⋅E + ψ(x)⋅V_max + ───(ψ(x)) = 0
2
dx
This is all well and good, the problem comes when I know that C₁ and C₂ happen to be equal and want to substitute one for the other. So I try something like
psi_high.subs( sp.Symbol( "C_2" ), sp.Symbol( "C_1" ) )
However it just comes out the same as before
___________ ___________
-x⋅╲╱ E - V_max x⋅╲╱ E - V_max
ψ(x) = C₁⋅ℯ + C₂⋅ℯ
I am thinking this may be a memory issue, that a reference to a sympy.Symbol object must refer to not only a sympy.Symbol object with the same value/symbol but which also must be the same underlying object.
This is only speculation (but I can say, do psi_high.subs( x, 0 ) and it works), but my question is how do I resolve it?
Curiously, it seems to work here (I did try this using the sympy.symbols function and by enclosing the symbol references in a tuple and list like shown in the question)
Thanks!
well_length = sq.Quantity( 'L' )
highest_potential = sq.Quantity( "V_max" )
x = sp.Symbol( 'x' )
m = sq.Quantity( 'm' )
hbar = sq.Quantity( "hbar" )
total_energy = sq.Quantity( 'E' )
inverse_total_energy = 1.0 / total_energy
psi_symbol = ud.lookup( "GREEK SMALL LETTER PSI" )
psi = sp.Function( "psi" )
second_derivative = sp.Derivative( psi( x ), x, 2 )
make_shrodinger_left = lambda potential, psi_parameter : ( second_derivative + ( psi( psi_parameter ) * potential ) )
make_shrodinger_right = lambda psi_parameter : total_energy * psi( psi_parameter )
make_psi_equal = lambda input_value, value : sp.Eq( psi( sp.Eq( x, input_value ) ), value )
set_equal = lambda to_set, value : sp.Eq( to_set, value )
shrodinger_left_high = sp.simplify( make_shrodinger_left( highest_potential, x ) )
shrodinger_right = make_shrodinger_right( x )
high_diff = sp.simplify( set_equal( shrodinger_left_high - shrodinger_right, 0 ) )
Here's a simpler example:
In [3]: eq = Eq(f(x).diff(x, 2), 0)
In [4]: eq
Out[4]:
2
d
───(f(x)) = 0
2
dx
In [5]: sol = dsolve(eq)
In [7]: sol
Out[7]: f(x) = C₁ + C₂⋅x
We can inspect these symbols:
In [8]: sol.free_symbols
Out[8]: {C₁, C₂, x}
In [9]: [s.name for s in sol.free_symbols]
Out[9]: ['C2', 'C1', 'x']
Note that there are no underscores in the symbol names. What we want to do then is:
In [10]: sol.subs(Symbol("C1"), Symbol("C2"))
Out[10]: f(x) = C₂⋅x + C₂

Is this a bug of SymPy?

Why do we have a strange result with integration of expression by SymPy.
>>> from sympy import *
>>> from sympy import __version__
>>> __version__
'1.8'
>>> x = Symbol('x')
>>> f = (x**2 - Rational(1, 4))**2 * sqrt(1 - x**2); f
sqrt(1 - x**2)*(x**2 - 1/4)**2
>>> integrate(f, (x, -1, 1))
0
The integrand is strictly positive, this result is wrong.
However, dividing the interval (x, -1, 1) into (x, -1, 0) and (x, 0, 1), we have the correct.
>>> integrate(f, (x, -1, 0))
pi/64
>>> integrate(f, (x, 0, 1))
pi/64
Expanding the integrand, the result is also correct.
>>> g = f.expand(); g
x**4*sqrt(1 - x**2) - x**2*sqrt(1 - x**2)/2 + sqrt(1 - x**2)/16
>>> integrate(g, (x, -1, 1))
pi/32
This strange phenomenon has occurred since version 1.5 of SymPy.
>>> from sympy import *
>>> from sympy import __version__
>>> __version__
'1.4'
>>> x = Symbol('x')
>>> f = (x**2 - Rational(1, 4))**2 * sqrt(1 - x**2); f
sqrt(1 - x**2)*(x**2 - 1/4)**2
>>> integrate(f, (x, -1, 1))
pi/32
Is this a bug?
Discussion is continued to the following.
https://github.com/sympy/sympy/issues/22033?fbclid=IwAR3oPgk-sLipSDWe7lsRmqG_hpw0fEEgED5XU5K96IKDi-UnVyOzqQqjSYY
I'm not sure the code of integrate function, but I understand as follows.
The primitive function of f = (x**2 - Rational(1,4))**2 * sqrt(1 - x**2) is this.
>>> from sympy import *
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> x = Symbol('x')
>>> f = (x**2 - Rational(1,4))**2 * sqrt(1 - x**2)
>>> F = integrate(f).simplify(); F
Piecewise((-x**3*(1 - x**2)**(3/2)/6 + x*sqrt(1 - x**2)/32 + asin(x)/32, (x > -1) & (x < 1)))
>>> X = np.linspace(-1, 1, 1000)
>>> Y = np.vectorize(Lambda(x, F))(X)
>>> plt.plot(X, Y)
>>> plt.show()
This is correct.
However, in the definite integral with respect to a real number, another expression fr of xr is used as folows.
>>> xr = Symbol('xr', real=True)
>>> fr = (xr**2 - Rational(1,4))**2 * sqrt(1 - xr**2)
>>> Fr = integrate(fr).simplify()
>>> Fr
Piecewise((xr**4*sqrt(1 - xr**2)*Abs(xr)/6 - xr**2*sqrt(1 - xr**2)*Abs(xr)/6 + sqrt(1 - xr**2)*Abs(xr)/32 - asin(sqrt(1 - xr**2))/32, (xr > -1) & (xr < 1) & Ne(xr, 0)))
>>> Yr = np.vectorize(Lambda(xr, Fr))(X)
>>> plt.plot(X, Yr)
>>> plt.show()
This is incorrect.
I think that integration by substitution with y = x**2 is used inadequetly like this..
>>> y = Symbol('y')
>>> g = (y - Rational(1,4))**2 * sqrt(1 - y)/2/sqrt(y)
>>> G = integrate(g).simplify(); G
Piecewise((y**(5/2)*sqrt(1 - y)/6 - y**(3/2)*sqrt(1 - y)/6 + sqrt(y)*sqrt(1 - y)/32 - asin(sqrt(1 - y))/32, (y <= 1) & (y > 0)))
>>> Gr = G.subs(y, x**2).simplify(); Gr
Piecewise((sqrt(1 - x**2)*(x**2)**(5/2)/6 - sqrt(1 - x**2)*(x**2)**(3/2)/6 + sqrt(1 - x**2)*sqrt(x**2)/32 - asin(sqrt(1 - x**2))/32, (x > -1) & (x < 1) & Ne(x, 0)))
>>> Z = np.vectorize(Lambda(x, Gr))(X)
>>> plt.plot(X, Z)
>>> plt.show()
This function is same as Fr above.
However, I don't know why the well known integration by substitution with x = sin(t) isn't used.
>>> t = Symbol('t')
>>> h = (sin(t)**2 - Rational(1,4))**2 * cos(t)**2
>>> H = integrate(h).simplify()
>>> H = integrate(h)
>>> H
t*sin(t)**6/16 + 3*t*sin(t)**4*cos(t)**2/16 - t*sin(t)**4/16 + 3*t*sin(t)**2*cos(t)**4/16 - t*sin(t)**2*cos(t)**2/8 + t*sin(t)**2/32 + t*cos(t)**6/16 - t*cos(t)**4/16 + t*cos(t)**2/32 + sin(t)**5*cos(t)/16 - sin(t)**3*cos(t)**3/6 - sin(t)**3*cos(t)/16 - sin(t)*cos(t)**5/16 + sin(t)*cos(t)**3/16 + sin(t)*cos(t)/32
>>> H1 = H.subs(t, asin(x)).simplify(); H1
x**5*sqrt(1 - x**2)/6 - x**3*sqrt(1 - x**2)/6 + x*sqrt(1 - x**2)/32 + asin(x)/32
This is same as primitive function F of f above and has another expression.
>>> H2 = H.simplify().subs(t, asin(x)); H2
sin(6*asin(x))/192 + asin(x)/32
>>> plt.show()
Using this we are able to have the following correct definite integral of f over [-1, 1].
>>> H2.subs(x, 1) - H2.subs(x, -1)
pi/32

How do I draw graph in sympy?

I'm high school student, and I'm writing a report about profile velocity.
I don't know much about differential equations and Python, but I have to use both of them.
I'm trying to induce the velocity from (ma = mg - kv), and caculate a and s from v.
I caculated v successfully, but I have few questions.
import sympy
init_printing()
%matplotlib inline
(m, g, k, t) = symbols('m g k t')
v = Function('v')
deq = Eq( m*v(t).diff(t), m*g - k*v(t) )
eq = dsolve( deq, v(t) )
C1 = Symbol('C1')
C1_ic = solve( eq.rhs.subs( {t:0}), C1)[0]
r = expand(eq.subs({C1:C1_ic}))
the simple way to caculate C1 doesn't work
v(0) = 0
so I write
eq = dsolve( deq, ics={v(0):0})
but it has same result with
eq = dsolve( deq, v(t) )
how to caculate acc and draw a graph?
I try this code, but it doesn't work
a = diff(r, t)
r = dsolve( a, v(t))
r.subs({m:1, g:9.8, k:1})
plot( r , (t,0,100))
I don't get the same result from eq = dsolve( deq, ics={v(0):0}). Also you should declare m, g and k with positive=True.
In [50]: m, g, k = symbols('m g k', positive=True)
In [51]: t = Symbol('t')
In [52]: v = Function('v')
In [53]: deq = Eq( m*v(t).diff(t), m*g - k*v(t) )
In [54]: deq
Out[54]:
d
m⋅──(v(t)) = g⋅m - k⋅v(t)
dt
In [55]: dsolve(deq, v(t))
Out[55]:
k⋅(C₁ - t)
──────────
m
g⋅m + ℯ
v(t) = ─────────────────
k
In [56]: dsolve(deq, v(t), ics={v(0):0})
Out[56]:
⎛ m⋅log(g) m⋅log(m) ⅈ⋅π⋅m⎞
k⋅⎜-t + ──────── + ──────── + ─────⎟
⎝ k k k ⎠
────────────────────────────────────
m
g⋅m + ℯ
v(t) = ───────────────────────────────────────────
k
In [57]: sol = dsolve(deq, v(t), ics={v(0):0}).rhs
In [58]: sol.expand()
Out[58]:
-k⋅t
─────
m
g⋅m g⋅m⋅ℯ
─── - ──────────
k k
In [59]: factor_terms(sol.expand())
Out[59]:
⎛ -k⋅t ⎞
⎜ ─────⎟
⎜ m ⎟
g⋅m⋅⎝1 - ℯ ⎠
────────────────
k
You can compute and plot the acceleration like
In [62]: sol = factor_terms(sol.expand())
In [64]: a = sol.diff(t)
In [65]: a = sol.diff(t).subs({m:1, g:9.8, k:1})
In [66]: a
Out[66]:
-t
9.8⋅ℯ
In [67]: plot(a, (t, 0, 100))

Polynomial Coefficients from Sympy to Array

In the below code, L1 simplifies to the transfer function that I want:
import sympy as sy
z = sy.symbols('z')
L1 = sy.simplify(((z**2 - 0.5*z + 0.16) / (z-1)**2 ) - 1)
L1
After this, I manually enter the coefficients for the numerator and denominator as follow:
num = [1.5, -0.84]
den = [1., -2., 1.]
Is there a way to do this from code? I'm not sure how to convert the sympy result to something that I can work with again without manually creating the arrays num and den.
You can use as_numer_denom to get the numerator and denominator and then as_poly and coeffs to get the coefficients:
In [16]: import sympy as sy
...: z = sy.symbols('z')
...: L1 = sy.simplify(((z**2 - 0.5*z + 0.16) / (z-1)**2 ) - 1)
...: L1
Out[16]:
1.0⋅(1.5⋅z - 0.84)
────────────────────
2
1.0⋅z - 2.0⋅z + 1.0
In [17]: num, den = L1.as_numer_denom()
In [18]: num.as_poly(z).coeffs()
Out[18]: [1.5, -0.84]
In [19]: den.as_poly(z).coeffs()
Out[19]: [1.0, -2.0, 1.0]
Or to get the whole expression, you could do :
from sympy import *
z = symbols('z')
L1 = simplify(((z**2 - 0.5*z + 0.16) / (z-1)**2 ) - 1)
srepr(L1)
output:
"Mul(Float('1.0', precision=53), Add(Mul(Float('1.5', precision=53), Symbol('z')),
Float('-0.83999999999999997', precision=53)), Pow(Add(Mul(Float('1.0', precision=53),
Pow(Symbol('z'), Integer(2))), Mul(Integer(-1), Float('2.0', precision=53),
Symbol('z')), Float('1.0', precision=53)), Integer(-1)))"

Gradient Descent in Python 2

Part of my assignment is to implement the Gradient Descent to find the best approximation of values c_1, c_2 and r_1 for the function
.
Given is only a list of 30 y-values corresponding to x from 0 to 30. I am implementing this in Enthought Canopy like this:
First I start with random values:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as pyplt
c1 = -0.1
c2 = 0.1
r1 = 0.1
x = np.linspace(0,29,30) #start,stop,numitems
y = c1*np.exp(r1*x) + (c1*x)**3.0 - (c2*x)**2.0
pyplt.plot(x,y)
values_x = np.linspace(0,29,30)
values_y = np.array([0.2, -0.142682939241718, -0.886680607211679, -2.0095087143494, -3.47583798747496, -5.24396052331554, -7.2690008846359, -9.50451068338581, -11.9032604272567, -14.4176327390446, -16.9998176236069, -19.6019094345634, -22.1759550265352, -24.6739776668383, -27.0479889096801, -29.2499944927101, -31.2319972651608, -32.945998641919, -34.3439993255969, -35.3779996651013, -35.9999998336943, -36.161999917415, -35.8159999589895, -34.9139999796348, -33.4079999898869, -31.249999994978, -28.3919999975061, -24.7859999987616, -20.383999999385, -15.1379999996945])
pyplt.plot(values_x,values_y)
The squared error is quite high:
def Error(y,y0):
return ( (1.0)*sum((y-y0)**2.0) )
print Error(y,values_y)
Now, to implement the gradient descent, I derived the partial derivative functions for c_1, c_2 and r_1 and implemented the Gradient Descent:
step_size = 0.0000005
accepted_Error = 50
dc1 = c1
dc2 = c2
dr1 = r1
y0 = values_y
previous_Error = 100000
left = True
for _ in range(1000):
gc1 = (2.0) * sum( ( y - dc1*np.exp(dr1*x) - (dc1*x)**3 + (dc2*x)**2 ) * ( -1*np.exp(dr1*x) - (3*(dc1**2)*(x**3)) ) )
gc2 = (2.0) * sum( ( y - dc1*np.exp(dr1*x) - (dc1*x)**3 + (dc2*x)**2 ) * ( 2*dc2*(x**2) ) )
gr1 = (2.0) * sum( ( y - dc1*np.exp(dr1*x) - (dc1*x)**3 + (dc2*x)**2 ) * ( -1*dc1*x*np.exp(dr1*x) ) )
dc1 = dc1 - step_size*gc1
dc2 = dc2 - step_size*gc2
dr1 = dr1 - step_size*gr1
y1 = dc1*np.exp(dr1*x) + (dc1*x)**3.0 - (dc2*x)**2.0
current_Error = Error(y0,y1)
if (current_Error > accepted_Error):
print currentError
else:
break
if (current_Error > previous_Error):
print currentError
print "DIVERGING"
break
if (current_Error==previous_Error):
print "CAN'T IMPROVE"
break
previous_Error = current_Error
However, the error is not improving at all, and I tried to vary the step size. Is there a mistake in my code?