I want to generate an n x m Matrix where each entry is a different number. The purpose of this program is to generate matrices so that I can then practice the technique of reducing these matrices to reduced row-echelon form via Gaussian elimination.
from sympy import MatrixSymbol,Eq
import random
k=0
def random_value(k):
return random.choice([k for k in range(-4,4) if k not in [0]])
rowlength = abs(random_value(k))
columnlength = abs(random_value(k))
C = MatrixSymbol('C',rowlength ,columnlength)
display(Eq(C,C.as_explicit()))
for i in range(0,rowlength):
for j in range(0,columnlength):
p = C.as_explicit().row(i).col(j)
display(p[0])
display(Eq(C,C.as_explicit()))
#display(C.rref()[0])
I tried p[0] == random_value(k) and it gave me the error:
TypeError: Cannot set values of <class 'sympy.matrices.immutable.ImmutableDenseMatrix'>
I am assuming the problem is that the Matrices I am generating are immutable.
I looked at the similar post SymPy - substitute sybolic entries in a matrix and did not see a viable solution.
SymPy has a randMatrix function:
In [16]: randMatrix(3, 4)
Out[16]:
⎡78 25 63 64⎤
⎢ ⎥
⎢83 72 43 91⎥
⎢ ⎥
⎣82 81 71 66⎦
https://docs.sympy.org/latest/modules/matrices/matrices.html#sympy.matrices.dense.randMatrix
Related
I'm not sure if the following is possible. I tried looking at the docs, but see a lot of physics stuff and not quite what I want.
Is it possible to be like matrixcalculus.org, and specify say a general matrix Q of shape (m,n), x, a vector of shape n, and do this kind of calculation. Rather than limiting it to a set number of dimensions and specifying each element individually
You can use MatrixSymbol:
In [5]: n = Symbol('n')
In [6]: Q = MatrixSymbol('Q', n, n)
In [7]: x = MatrixSymbol('x', n, 1)
In [8]: f = x.T # Q # x
In [9]: f
Out[9]:
T
x ⋅Q⋅x
In [10]: diff(f, x)
Out[10]:
T
Q⋅x + Q ⋅x
If you substitute a concrete value for n then as_explicit can give you the expanded result:
In [11]: diff(f, x).subs(n, 2).as_explicit()
Out[11]:
⎡2⋅Q₀₀⋅x₀₀ + Q₀₁⋅x₁₀ + Q₁₀⋅x₁₀⎤
⎢ ⎥
⎣Q₀₁⋅x₀₀ + Q₁₀⋅x₀₀ + 2⋅Q₁₁⋅x₁₀⎦
https://docs.sympy.org/latest/modules/matrices/expressions.html
Is there a way of vectorizing the following array calculation (i.e. without using for loops):
for i in range(numCells):
z[i] = ((i_mask == i)*s_image).sum()/pixel_counts[i]
s_image is an image stored as a 2-dimensional ndarray (I removed the colour dimension here for simplicity). i_mask is also a 2-dimensional array of the same size as s_image but it contains integers which are indexes to a list of 'cells' of length numCells. The result, z, is a 1-dimensional array of length numCells. The purpose of the calculation is to sum all the pixel values where the mask contains the same index and put the results in the z vector. (pixel_counts is also a 1-dimensional array of length numCells).
As one vectorized approach, you can take advantage of broadcasting and matrix-multiplication, like so -
# Generate a binary array of matches for all elements in i_mask against
# an array of indices going from 0 to numCells
matches = i_mask.ravel() == np.arange(numCells)[:,None]
# Do elementwise multiplication against s_image and sum those up for
# each such index going from 0 to numCells. This is essentially doing
# matix multiplicatio. Finally elementwise divide by pixel_counts
out = matches.dot(s_image.ravel())/pixel_counts
Alternatively, as another vectorized approach, you can do those multiplication and summation with np.einsum as well, which might give a boost to the performance, like so -
out = np.einsum('ij,j->i',matches,s_image.ravel())/pixel_counts
Runtime tests -
Function definitions:
def vectorized_app1(s_image,i_mask,pixel_counts):
matches = i_mask.ravel() == np.arange(numCells)[:,None]
return matches.dot(s_image.ravel())/pixel_counts
def vectorized_app2(s_image,i_mask,pixel_counts):
matches = i_mask.ravel() == np.arange(numCells)[:,None]
return np.einsum('ij,j->i',matches,s_image.ravel())/pixel_counts
def org_app(s_image,i_mask,pixel_counts):
z = np.zeros(numCells)
for i in range(numCells):
z[i] = ((i_mask == i)*s_image).sum()/pixel_counts[i]
return z
Timings:
In [7]: # Inputs
...: numCells = 100
...: m,n = 100,100
...: pixel_counts = np.random.rand(numCells)
...: s_image = np.random.rand(m,n)
...: i_mask = np.random.randint(0,numCells,(m,n))
...:
In [8]: %timeit org_app(s_image,i_mask,pixel_counts)
100 loops, best of 3: 8.13 ms per loop
In [9]: %timeit vectorized_app1(s_image,i_mask,pixel_counts)
100 loops, best of 3: 7.76 ms per loop
In [10]: %timeit vectorized_app2(s_image,i_mask,pixel_counts)
100 loops, best of 3: 4.08 ms per loop
Here is my solution (with all three colours handled). Not sure how efficient this is. Anyone got a better solution?
import numpy as np
import pandas as pd
# Unravel the mask matrix into a 1-d array
i = np.ravel(i_mask)
# Unravel the image into 1-d arrays for
# each colour (RGB)
r = np.ravel(s_image[:,:,0])
g = np.ravel(s_image[:,:,1])
b = np.ravel(s_image[:,:,2])
# prepare a dictionary to create the dataframe
data = {'i' : i, 'r' : r, 'g' : g, 'b' : b}
# create a dataframe
df = pd.DataFrame(data)
# Use pandas pivot table to average the colour
# intensities for each cell index value
pixAvgs = pd.pivot_table(df, values=['r', 'g', 'b'], index='i')
pixAvgs.head()
Output:
b g r
i
-1 26.719482 68.041868 101.603297
0 75.432432 170.135135 202.486486
1 92.162162 184.189189 208.270270
2 71.179487 171.897436 201.846154
3 76.026316 178.078947 211.605263
In the end I solved this problem a different way and it drastically increased the speed. Instead of using i_mask as above, a 2-dimensional array of indices into the 1-d array of output intensities, z, I created a different array, mask1593, of dimensions (numCells x 45). Each row is a list of about 35 to 45 indices into the flattened 256x256 pixel image (0 to 65536).
In [10]: mask1593[0]
Out[10]:
array([14853, 14854, 15107, 15108, 15109, 15110, 15111, 15112, 15363,
15364, 15365, 15366, 15367, 15368, 15619, 15620, 15621, 15622,
15623, 15624, 15875, 15876, 15877, 15878, 15879, 15880, 16131,
16132, 16133, 16134, 16135, 16136, 16388, 16389, 16390, 16391,
16392, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
Then I was able to achieve the same transformation as follows using numpy's advanced indexing:
def convert_image(self, image_array):
"""Convert 256 x 256 RGB image array to 1593 RGB led intensities."""
global mask1593
shape = image_array.shape
img_data = image_array.reshape(shape[0]*shape[1], shape[2])
return np.mean(img_data[mask1593], axis=1)
And here is the result! A 256x256 pixel colour image transformed into an array of 1593 colours for display on this irregular LED display:
boost::number::ublas contains the M::size_type lu_factorize(M& m) function. Its name suggests that it performs the LU decomposition of a given matrix m, i.e. should produce two matrices that m = L*U. There seems to be no documentation provided for this function.
It is easy to deduce that it returns 0 to indicate successful decomposition, and a non-zero value when the matrix is singular. However, it is completely unclear where is the result. Taking the matrix by reference suggests that it works in-place, however it should produce two matrices (L and U) not one. So what does it do?
There is no documentation in boost, but looking at the documentation of SciPy's lu_factor one can see, that it's not uncommon to return one result for the LU decomposition.
This is enough, because in a typical approach to LU decomposition, L's diagonal consists of ones only, as presented in this answer from Mathematics, for example.
So, it is possible to fit both L and U into one matrix, putting L in result's lower part, omitting the diagonal (which is assumed to contain only ones), and U in the upper part. For example, for a 3x3 problem the result is:
u11 u12 u13
m = l21 u22 u23
l31 l32 u33
which implies:
1 0 0
L = l21 1 0
l31 l32 1
and
u11 u12 u13
U = 0 u22 u23
0 0 u33
Inspecting boost's void lu_substitute(const M& m, vector_expression<E>& e) function, from the same namespace seems to confirm this. It solves the equation LUx = e, where both L and U are contained in its m argument in two steps.
First solve Lz = e for z, where z = Ux, using lower part of m:
inplace_solve(m, e, unit_lower_tag ());
then, having computed z = Ux (with e modified in place), Ux = e can be solved, using upper part of m:
inplace_solve(m, e, upper_tag ());
inplace_solve is mentioned in the documentation, and it:
Solves a system of linear equations with triangular form, i.e. A is triangular.
So everything seems to make sense.
The boost doesn't have document of LU factorization (a lower triangular matrix L and upper triangular matrix U), but the source code shared with the public.
If the code is hard to follow, please check the webpage by Nick Higham. It had an detailed explanation. Here are an example from the link:
Let's say we need to solve Ax = b.
(1) Make LU from input matrix, A
[3 -1 1 1]
[-1 3 1 -1] ->
[-1 -1 3 1]
[1 1 1 3]
Low
[1 0 0 0]
[-1/3 1 0 0]
[-1/3 -1/2 1 0]
[1/3 1/2 0 1]
Upper
[3 -1 1 1]
[0 8/3 4/3 -2/3]
[0 0 4 1]
[0 0 0 3]
This example looks straight forward to human but algorithm wise could be numerous steps. This is why LU Factorization came. Methodically, Relation with Gaussian Elimination, Schur Complements, and Block Implementations are some.
(2) Solve the triangular systems Ly = b and Ux = y, since then b = L(Ux).
I have a python function which generates a sympy.Matrix with symbolic entries. It works effectively like:
import sympy as sp
M = sp.Matrix([[1,0,2],[0,1,2],[1,2,0]])
def make_symbolic_matrix(M):
M_sym = sp.zeros(3)
syms = ['a0:3']
for i in xrange(3):
for j in xrange(3):
if M[i,j] == 1:
M_sym = syms[i]
elif M[i,j] == 2:
M_sym = 1 - syms[i]
return M_sym
This works just fine. I get a matrix out, which I can use for all the symbolical calculations I need.
My issue is that now I want to evaluate my matrix at specified parameter-value. Usually I would just use the .subs attribute. However, since the symbols, that are now used as entries in my matrix, were originally defined as temporary elements in a function, I don't know how to call them.
It seems as if it should be possible, since I'm able to perform symbolic calculations.
What I want to do would look something like (following the code above):
M_sym = make_matrix(M)
M_eval = M_sym.subs([(a0,.8),(a1,.3),(a2,.5)])
But all I get is "name 'a0' is not defined".
I'd be super happy if someone out there got a solution!
PS. I'm not just defining the symbols globally, because in the actual problem I don't know how many parameters I have from time to time.
In the general case, I assume you're looking for an n-by-m matrix of symbolic elements.
import sympy
def make_symbolic(n, m):
rows = []
for i in xrange(n):
col = []
for j in xrange(m):
col.append(sympy.Symbol('a%d%d' % (i,j)))
rows.append(col)
return sympy.Matrix(rows)
which could be used in the following way:
make_symbolic(3, 4)
to give:
Matrix([
[a00, a01, a02, a03],
[a10, a11, a12, a13],
[a20, a21, a22, a23]])
once you've got that matrix you can substitute in any values required.
Given that the answer from Andrew was helpful it seems like you might be interested in the MatrixSymbol.
In [1]: from sympy import *
In [2]: X = MatrixSymbol('X', 3, 4)
In [3]: X # completely symbolic
Out[3]: X
In [4]: Matrix(X) # Expand to explicit matrix
Out[4]:
⎡X₀₀ X₀₁ X₀₂ X₀₃⎤
⎢ ⎥
⎢X₁₀ X₁₁ X₁₂ X₁₃⎥
⎢ ⎥
⎣X₂₀ X₂₁ X₂₂ X₂₃⎦
But answering your original question, perhapcs you could get the symbols out of the matrix that you produce?
x12 = X[1, 2]
Symbols are defined by their name. Two symbols with the same name will considered to be the same thing by Sympy. So if you know the name of the symbols you want, just create them again using symbols.
You may also find good use of the zip function of Python.
I have the following system of 3 nonlinear equations that I need to solve:
-xyt + HF = 0
-2xzt + 4yzt - xyt + 4z^2t - M1F = 0
-2xt + 2yt + 4zt - 1 = 0
where x, HF, and M1F are known parameters. Therefore, y,z, and t are the parameters to be calculated.
Attemp to solve the problem:
def equations(p):
y,z,t = p
f1 = -x*y*t + HF
f2 = -2*x*z*t + 4*y*z*t - x*y*t + 4*t*z**2 - M1F
f3 = -2*x*t + 2*y*t + 4*z*t - 1
return (f1,f2,f3)
y,z,t = fsolve(equations)
print equations((y,z,t))
But the thing is that if I want to use scipy.optimize.fsolve then I should input an initial guess. In my case, I do not have any initial conditions.
Is there another way to solve 3 nonlinear equations with 3 unknowns in python?
Edit:
It turned out that I have a condition! The condition is that HF > M1F, HF > 0, and M1F > 0.
#Christian, I don't think the equation system can be linearize easily, unlike the post you suggested.
Powell's Hybrid method (optimize.fsolve()) is quite sensitive to initial conditions, so it is very useful if you can come up with a good initial parameter guess. In the following example, we firstly minimize the sum-of-squares of all three equations using Nelder-Mead method (optimize.fmin(), for small problem like OP, this is probably already enough). The resulting parameter vector is then used as the initial guess for optimize.fsolve() to get the final result.
>>> from numpy import *
>>> from scipy import stats
>>> from scipy import optimize
>>> HF, M1F, x=1000.,900.,10.
>>> def f(p):
return abs(sum(array(equations(p))**2)-0)
>>> optimize.fmin(f, (1.,1.,1.))
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 131
Function evaluations: 239
array([ -8.95023217, 9.45274653, -11.1728963 ])
>>> optimize.fsolve(equations, (-8.95023217, 9.45274653, -11.1728963))
array([ -8.95022376, 9.45273632, -11.17290503])
>>> pr=optimize.fsolve(equations, (-8.95023217, 9.45274653, -11.1728963))
>>> equations(pr)
(-7.9580786405131221e-13, -1.2732925824820995e-10, -5.6843418860808015e-14)
The result is pretty good.