How to compute gradient wrt concatenated variables in theano - gradient

I have a loss function y which is a function of multiple shared variables theta1, theta2, ...
Then, gradient descent of y wrt theta s can be written simply as
import theano.tensor as T
theta_list = [theta1, theta2, theta3, theta4]
grad_list = T.grad(y, theta_list)
for theta, gradient in zip(theta_list, grad_list):
theta = theta - learning_rate * gradient
However I want not to use the list representation of theta. That is,
import theano.tensor as T
thetas = <Properly concatenate thetas>
gradient = T.grad(y, thetas)
thetas = thetas - learning_rate * gradient
Is there any way to enable this?
Simple thetas = T.concatenate(theta_list) raises DisconnectedInputError when calculate the gradient.

Related

Plot sine function using radians with xlim and ylim

I would like to plot a sine function with xlim measured in radians:
import sympy as sp
from sympy.functions.elementary.trigonometric import sin
x = sp.symbols('x')
eqn = sin(x)
p = sp.plot(eqn, xlim = (-2*sp.pi, 2*sp.pi), ylim = (-1, 1))
This blows up because of how I'm using pi. If I replace pi with a numerical approximation (e.g. 3.14), the plot displays correctly.
This looks like a bug in the handling of xlim. Perhaps it's not been noticed because xlim is rarely used, with the x-limits usually being passed in a tuple along with the symbol itself:
sp.plot(eqn, (x, -2*sp.pi, 2*sp.pi), ylim=(-1, 1))
works fine.

How to fit a 2D ellipse to given points

I would like to fit a 2D array by an elliptic function: (x / a)² + (y / b)² = 1 ----> (and so get the a and b)
And then, be able to replot it on my graph.
I found many examples on internet, but no one with this simple Cartesian equation. I probably have searched badly ! I think a basic solution for this problem could help many people.
Here is an example of the data:
Sadly, I can not put the values... So let's assume that I have an X,Y arrays defining the coordinates of each of those points.
This can be solved directly using least squares. You can frame this as minimizing the sum of squares of quantity (alpha * x_i^2 + beta * y_i^2 - 1) where alpha is 1/a^2 and beta is 1/b^2. You have all the x_i's in X and the y_i's in Y so you can find the minimizer of ||Ax - b||^2 where A is an Nx2 matrix (i.e. [X^2, Y^2]), x is the column vector [alpha; beta] and b is column vector of all ones.
The following code solves the more general problem for an ellipse of the form Ax^2 + Bxy + Cy^2 + Dx +Ey = 1 though the idea is exactly the same. The print statement gives 0.0776x^2 + 0.0315xy+0.125y^2+0.00457x+0.00314y = 1 and the image of the ellipse generated is also below
import numpy as np
import matplotlib.pyplot as plt
alpha = 5
beta = 3
N = 500
DIM = 2
np.random.seed(2)
# Generate random points on the unit circle by sampling uniform angles
theta = np.random.uniform(0, 2*np.pi, (N,1))
eps_noise = 0.2 * np.random.normal(size=[N,1])
circle = np.hstack([np.cos(theta), np.sin(theta)])
# Stretch and rotate circle to an ellipse with random linear tranformation
B = np.random.randint(-3, 3, (DIM, DIM))
noisy_ellipse = circle.dot(B) + eps_noise
# Extract x coords and y coords of the ellipse as column vectors
X = noisy_ellipse[:,0:1]
Y = noisy_ellipse[:,1:]
# Formulate and solve the least squares problem ||Ax - b ||^2
A = np.hstack([X**2, X * Y, Y**2, X, Y])
b = np.ones_like(X)
x = np.linalg.lstsq(A, b)[0].squeeze()
# Print the equation of the ellipse in standard form
print('The ellipse is given by {0:.3}x^2 + {1:.3}xy+{2:.3}y^2+{3:.3}x+{4:.3}y = 1'.format(x[0], x[1],x[2],x[3],x[4]))
# Plot the noisy data
plt.scatter(X, Y, label='Data Points')
# Plot the original ellipse from which the data was generated
phi = np.linspace(0, 2*np.pi, 1000).reshape((1000,1))
c = np.hstack([np.cos(phi), np.sin(phi)])
ground_truth_ellipse = c.dot(B)
plt.plot(ground_truth_ellipse[:,0], ground_truth_ellipse[:,1], 'k--', label='Generating Ellipse')
# Plot the least squares ellipse
x_coord = np.linspace(-5,5,300)
y_coord = np.linspace(-5,5,300)
X_coord, Y_coord = np.meshgrid(x_coord, y_coord)
Z_coord = x[0] * X_coord ** 2 + x[1] * X_coord * Y_coord + x[2] * Y_coord**2 + x[3] * X_coord + x[4] * Y_coord
plt.contour(X_coord, Y_coord, Z_coord, levels=[1], colors=('r'), linewidths=2)
plt.legend()
plt.xlabel('X')
plt.ylabel('Y')
plt.show()
Following the suggestion by ErroriSalvo, here is the complete process of fitting an ellipse using the SVD. The arrays x, y are coordinates of the given points, let's say there are N points. Then U, S, V are obtained from the SVD of the centered coordinate array of shape (2, N). So, U is a 2 by 2 orthogonal matrix (rotation), S is a vector of length 2 (singular values), and V, which we do not need, is an N by N orthogonal matrix.
The linear map transforming the unit circle to the ellipse of best fit is
sqrt(2/N) * U * diag(S)
where diag(S) is the diagonal matrix with singular values on the diagonal. To see why the factor of sqrt(2/N) is needed, imagine that the points x, y are taken uniformly from the unit circle. Then sum(x**2) + sum(y**2) is N, and so the coordinate matrix consists of two orthogonal rows of length sqrt(N/2), hence its norm (the largest singular value) is sqrt(N/2). We need to bring this down to 1 to have the unit circle.
N = 300
t = np.linspace(0, 2*np.pi, N)
x = 5*np.cos(t) + 0.2*np.random.normal(size=N) + 1
y = 4*np.sin(t+0.5) + 0.2*np.random.normal(size=N)
plt.plot(x, y, '.') # given points
xmean, ymean = x.mean(), y.mean()
x -= xmean
y -= ymean
U, S, V = np.linalg.svd(np.stack((x, y)))
tt = np.linspace(0, 2*np.pi, 1000)
circle = np.stack((np.cos(tt), np.sin(tt))) # unit circle
transform = np.sqrt(2/N) * U.dot(np.diag(S)) # transformation matrix
fit = transform.dot(circle) + np.array([[xmean], [ymean]])
plt.plot(fit[0, :], fit[1, :], 'r')
plt.show()
But if you assume that there is no rotation, then np.sqrt(2/N) * S is all you need; these are a and b in the equation of the ellipse.
You could try a Singular Value Decomposition of the data matrix.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.svd.html
First center the data by subtracting mean values of X,Y from each column respectively.
X=X-np.mean(X)
Y=Y-np.mean(Y)
D=np.vstack(X,Y)
Then, apply SVD and extract
-eigenvalues (members of s) -> axis length
-eigenvectors(U) -> axis orientation
U, s, V = np.linalg.svd(D, full_matrices=True)
This should be a least-squares fit.
Of course, things can get more complicated than this, please see
https://www.emis.de/journals/BBMS/Bulletin/sup962/gander.pdf

Drawing a circle with cos() sin(), no repeat pixel, no gaps?

I'm interested in drawing a circle of a vary radius using sin() and cos() functions.
Is there a golden rule to increment the radians so that there isn't multiple plots to the same location and no gaps in the circle drawn on a pixel based display?
x = cos(r) * radius
y = sin(r) * radius
r = r + s
My guess would be that s is something to do with dividing 2 × PI by the a number derived from the radius?
I'm sure this is either really simple or impossible due to the limitations of floating point calculations.
Thanks for your time
Anthony
The length of arc is simply s = r * delta_fi where r is the radius of the circle, fi is the angle and delta_fi is the change of the angle.
The projection of this arc to x-axis is delta_x = s * sin(fi) and to y-axis it is delta_y = s * cos(fi)
You want such delta_fi that either delta_x or delta_y is 1.
Obviously, the problem is symmetrical and we can solve it for fi from -45° to 45° and for delta y and then apply the same solution in other quadrants. We have:
r * delta_fi * cos(fi) = 1
Hence:
delta_fi = 1/cos(fi)/r
The coordinates of a circle can indeed be completely defined using the trigonometric functions sine and cosine:
x = cos(angle)
y = sin(angle)
If the radius is any other value than 1 (which happens to define the unit circle), the underlying principles of trigonometric functions still apply and, therefore, the following equations can be derived:
x = cos(angle) * radius
y = sin(angle) * radius
To implement this in Python (with the kind help of Numpy) all that is necessary in addition to what we have already defined is a suitable vector (or 1-dimensional array) for the angle, which will be evaluated by the function x and y.
import numpy as np
r = 2 # An arbitrary value for the radius
angle = np.linspace(0, 2*np.pi, 1000) # A vector covering all angles from
# 0 to 2*pi (the full circle in radians)
# with an arbitrary number of
# elements, 1000 in this example
x = np.cos(angle)*r
y = np.sin(angle)*r
On plotting this circle don't forget to adjust the size of the figure to a square, otherwise the circle will be distorted.
import matplotlib.pyplot as plt
plt.figure(figsize=(3, 3))
plt.plot(x, y)

Matrix related calculation in python

I am finding a very interesting problem while calculating a matrix update in python . I have to calculate the error (which is difference between previous n updated matrix ).
import numpy as np
import matplotlib.pyplot as plt
#from matplotlib import animation
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
def update(A):
C=A
D=A
D[1:-1,1:-1]=(C[0:-2,1:-1]+C[2:,1:-1]+C[1:-1,0:-2]+C[1:-1,2:])/4
return(np.abs(D-C),D)
def error(A,B):
C=np.zeros(np.shape(A),np.float64)
#e=np.max(np.max(np.abs(C)))
e=(np.abs(C))
return (e.sum(dtype='float64'))
def initial(C):
C[0,:]=0 ## Top Boundary
C[-1,:]=0 ## Bottom Boundary
C[:,0]=0 ## left Boundary
C[:,-1]=100 ## Right Boundary
return(C)
def SolveLaplace(nx, ny,epsilon,imax):
## Initialize the mesh with some values
U = np.zeros((nx, ny),np.float64)
## Set boundary conditions for the problem
U=initial(U)
## Store previous grid values to check against error tolerance
UN=np.zeros((nx, ny),np.float64)
UN=initial(UN)
## Constants
k = 1 ## Iteration counter
## Iterative procedure
while k<imax:
err,U=update(U)
print(err.sum())
k+=1
return (U)
nx = 50.0
ny = 50.0
dx = 0.001
epsilon = 1e-6 ## Absolute Error tolerance
imax = 5000 ## Maximum number of iterations allowed
Z = SolveLaplace(nx, ny,epsilon,imax)
#x = np.linspace(0, nx * dx, nx)
#y = np.linspace(0, ny * dx, ny)
#X, Y = np.meshgrid(x,y)
##===================================================================
def PlotSolution(nx,ny,dx,T):
## Set up x and y vectors for meshgrid
x = np.linspace(0, nx * dx, nx)
y = np.linspace(0, ny * dx, ny)
fig = plt.figure()
ax = fig.gca(projection='3d')
X, Y = np.meshgrid(x,y)
ax.plot_surface(X, Y, T.transpose(), rstride=1, cstride=1, cmap=cm.cool, linewidth=0, antialiased=False)
plt.xlabel("X")
plt.ylabel("Y")
#plt.zlabel("T(X,Y)")
plt.figure()
plt.contourf(X, Y, T.transpose(), 32, rstride=1, cstride=1, cmap=cm.cool)
plt.colorbar()
plt.xlabel("X")
plt.ylabel("Y")
plt.show()
##===================================================================
PlotSolution(nx, ny, dx, Z)
I am suppose to solve Laplace equation for 2-d sheet(temperature distribution) and when error is less than certain minimum value ,equilibrium will be achieved. But while calculating error, I am always getting 0 but when I print my matrix then I find it should not be a zero . Guys I think I have some conceptual problem here and So please help .
Your problem is that you use shallow copies, i.e., only copy the reference, when assigning C=A; D=A in the update function. Essentially, after the construction of D, all three variables A,C,D point to the same object. Use
def update(A):
C=1.0*A
D=1.0*A
D[1:-1,1:-1]=(C[0:-2,1:-1]+C[2:,1:-1]+C[1:-1,0:-2]+C[1:-1,2:])/4
return(np.abs(D-C),D)
or even shorter
def update(A):
D=A.copy()
D[1:-1,1:-1]=(A[0:-2,1:-1]+A[2:,1:-1]+A[1:-1,0:-2]+A[1:-1,2:])/4
return(np.abs(D-A),D)
Passing arguments and performing arithmetic operations results automatically in a deep copy.
You know that the (geometric, first order) convergence rate is something like max(1-C/(nx^2), 1-C/(ny^2)), i.e., very slow for even moderately large grids? For real applications, better use conjugate gradients, other Krylov-related algorithms or multi-grid approaches (or sparse solver libraries, UMFpack ...).
In the (unused) error procedure, should there be not something like
e = abs(A-B)
At the moment, you return the norm of the freshly generated zero matrix C.

Colouring the surface of a sphere with a set of scalar values in matplotlib

I am rather new to matplotlib (and this is also my first question here). I'm trying to represent the scalp surface potential as recorded by an EEG. So far I have a two-dimensional figure of a sphere projection, which I generated using contourf, and pretty much boils down to an ordinary heat map.
Is there any way this can be done on half a sphere?, i.e. generating a 3D sphere with surface colours given by a list of values? Something like this, http://embal.gforge.inria.fr/img/inverse.jpg, but I have more than enough with just half a sphere.
I have seen a few related questions (for example, Matplotlib 3d colour plot - is it possible?), but they either don't really address my question or remain unanswered to date.
I have also spent the morning looking through countless examples. In most of what I've found, the colour at one particular point of a surface is indicative of its Z value, but I don't want that... I want to draw the surface, then specify the colours with the data I have.
You can use plot_trisurf and assign a custom field to the underlying ScalarMappable through set_array method.
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import matplotlib.tri as mtri
(n, m) = (250, 250)
# Meshing a unit sphere according to n, m
theta = np.linspace(0, 2 * np.pi, num=n, endpoint=False)
phi = np.linspace(np.pi * (-0.5 + 1./(m+1)), np.pi*0.5, num=m, endpoint=False)
theta, phi = np.meshgrid(theta, phi)
theta, phi = theta.ravel(), phi.ravel()
theta = np.append(theta, [0.]) # Adding the north pole...
phi = np.append(phi, [np.pi*0.5])
mesh_x, mesh_y = ((np.pi*0.5 - phi)*np.cos(theta), (np.pi*0.5 - phi)*np.sin(theta))
triangles = mtri.Triangulation(mesh_x, mesh_y).triangles
x, y, z = np.cos(phi)*np.cos(theta), np.cos(phi)*np.sin(theta), np.sin(phi)
# Defining a custom color scalar field
vals = np.sin(6*phi) * np.sin(3*theta)
colors = np.mean(vals[triangles], axis=1)
# Plotting
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
cmap = plt.get_cmap('Blues')
triang = mtri.Triangulation(x, y, triangles)
collec = ax.plot_trisurf(triang, z, cmap=cmap, shade=False, linewidth=0.)
collec.set_array(colors)
collec.autoscale()
plt.show()