I'm new to Pymc3 and I'm trying to create the Categorical Mixture Model shown in https://en.wikipedia.org/wiki/Mixture_model#Categorical_mixture_model . I'm having difficulty hooking up the 'x' variable. I think it's because I have to make the z variable Deterministic, but I'm getting an error message at the line where 'x' is assigned : "ValueError: We expected 3 inputs but got 2.". It looks like the p function only accepts 2 inputs so I'm stuck. I've tried a bunch of different things, but haven't been able to get this to work yet.
import numpy as np
from pymc3 import *
import theano.tensor as t
K = 3 #NUMBER OF TOPICS
V = 20 #NUMBER OF WORDS
N = 15 #NUMBER OF DOCUMENTS
#GENERAETE RANDOM CATEGORICAL MIXTURES
data = np.ones([N,V])
#theano.compile.ops.as_op(itypes=[t.lscalar, t.dscalar, t.dscalar],otypes=[t.dvector])
def p(z=z, phi=phi):
return [phi[z[i,j]] for i in range(D) for j in range(W)]
model = Model()
with model:
alpha = np.ones(V)
beta = np.ones(K)
theta = [Dirichlet('theta_%i' % i, alpha, shape=V) for i in range(K)]
phi = Dirichlet('phi', beta, shape=K)
z = [Categorical('z_%i' % i, p = phi, shape=V) for i in range(N)]
x = [Categorical('x_%i_%i' % (i,j), p=p(z[i][j],phi), observed=data[i,j]) for i in range(N) for j in range(V)]
#x = [Categorical('x_%i_%i' % (i,j), p=theta[z[i][j]], observed=data[i,j]) for i in range(N) for j in range(V)]
print "Created model. Now begin sampling"
step = Slice()
trace = sample(n, step)
trace.get_values('phi')
For starters, in your example above, z and phi have no value which would allow them to be used as default values. We also don't have values for D and W.
As for the number of arguments, the function you define has 2 but your theano decorator above it has 3. I'd suggest
#theano.compile.ops.as_op(itypes=[t.lscalar, t.dvector],otypes=[t.dvector])
def p(z, phi):
return [phi[z[i,j]] for i,j in zip(range(D),range(W))]
Related
Wondering if you can help me understand where the critical flaw may be with my attempt at implementing Karger's algorithm in python. My program appears to take far too long to run and my computer starts to overwork running large sets of vertices. The purpose of the program is to output the minimum cut of the graph.
from random import choice
from statistics import mode
import math
fhand = open("mincuts.txt", "r")
vertices = fhand.readlines()
d = {}
for index,line in enumerate(vertices):
d["{0}".format(index+1)] = line.split()
def randy(graph, x):
y = str(choice(list(graph)))
if x == y:
y = randy(graph, x)
return y
count = 0
def contract(graph):
global count
if len(graph) == 2:
a = list(graph.keys())[0]
b = list(graph.keys())[1]
for i in range(1, len(graph[a])):
if graph[a][i] in graph[b]:
count = count + 1
#print(graph)
return
x = str(choice(list(graph)))
y = randy(graph, x)
#print(x)
#print(y)
graph[x] = graph[x] + graph[y]
graph.pop(y)
#remove self loops
for key in graph:
#method to remove duplicate entries in the arrays of the vertices. Source: www.w3schools.com
graph[key] = list(dict.fromkeys(graph[key]))
contract(graph)
N = len(d)
runs = int(N*N*(math.log(N)))
outcomes = []
for i in range(runs):
e = d.copy()
count = 0
contract(e)
outcomes.append(count)
print(outcomes)
#returns most common minimum cut value
print(mode(outcomes))
Below is a link to the graph I am running in mincuts.txt:
https://github.com/BigSoundCode/Misc-Algorithm-Implementations/blob/main/mincuts.txt
Background
I'm trying to use a surrogate model in Pyomo. Given a set of data labeled x, y, and z, I would like to write z as an inexpensive function of x and y.
Issue
Pyomo has tools for multivariate piecewise linear functions. See here. I setup a simple example and my function is evaluating correctly. But there there doesn't seem to be any constraint getting added for Z. I would expect the model value to be equal to the interpolated value below. I'm guessing I did something wroing in setting up TransformedPiecewiseLinearFunctionND, but I couldn't find any examples in the documentation. Any insights would be appreciated.
Code
from pyomo.core import ConcreteModel, Var, Constraint
import pyomo.environ as pe
from pyomo.core.kernel.piecewise_library.transforms_nd import (
PiecewiseLinearFunctionND,
TransformedPiecewiseLinearFunctionND
)
from pyomo.core.kernel.variable import (
variable,
variable_list
)
import pyomo.core.kernel.piecewise_library.util as util
import numpy as np
from scipy.spatial import Delaunay
npts = 100
vlist = variable_list([variable(lb=-1, ub=1),
variable(lb=-1, ub=1)])
tri = util.generate_delaunay(vlist, num=npts)
x, y = tri.points.T
z = np.cos(x) * np.sin(y)
model = ConcreteModel()
model.X = Var(initialize=0)
model.Y = Var(initialize=0)
model.Z = Var(initialize=999)
f = PiecewiseLinearFunctionND(tri, z)
model.g = TransformedPiecewiseLinearFunctionND(
f=f,
input=(model.X, model.Y),
output=model.Z
)
def x_rule(model):
return model.X == 0.5
def y_rule(model):
return model.Y == 0.5
model.x_const = Constraint(rule=x_rule)
model.y_const = Constraint(rule=y_rule)
solver = pe.SolverFactory('ipopt')
solver.solve(model)
z_exact = np.cos(0.5) * np.sin(0.5)
z_interp = f([0.5, 0.5])
x_model = pe.value(model.X)
y_model = pe.value(model.Y)
z_model = pe.value(model.Z)
print(f'Z Exact: {z_exact}')
print(f'Z Interpolated: {z_interp}')
print(f'Model values (X, Y, Z): {x_model}, {y_model}, {z_model}')
Output
Z Exact: 0.42073549240394825
Z Interpolated: 0.42067082611089646
Model values (X, Y, Z): 0.5, 0.5, 999
I've also tried adding a constraint for Z manually. This produces an error:
def z_rule(model):
return model.Z == f([model.X, model.Y])
model.z_const = Constraint(rule=z_rule)
You are mixing modeling components between the pyomo.kernel and pyomo.environ modeling layers. This is not supported (this page has more information).
The multi-dimensional piecewise functionality is currently only available using the pyomo.kernel interface. An example of how to use it can be found here:
https://github.com/Pyomo/pyomo/blob/main/examples/kernel/piecewise_nd_functions.py
I'm trying to practice using pymc3 on the kinds of data I come across in my research, but I'm having trouble thinking through how to fit the model when each person gives me multiple data points, and each person comes from a different group (so trying a hierarchical model).
Here's the practice scenario I'm using: Suppose we have 2 groups of people, N = 30 in each group. All 60 people go through a 10 question survey, where each person can response ("1") or not respond ("0") to each question. So, for each person, I have an array of length 10 with 1's and 0's.
To model these data, I assume each person has some latent trait "theta", and each item has a "discrimination" a and a "difficulty" b (this is just a basic item response model), and the probability of responding ("1") is given by: (1 + exp(-a(theta - b)))^(-1). (Logistic applied to a(theta - b) .)
Here is how I tried to fit it using pymc3:
traces = {}
for grp in range(2):
group = prac_data["Group ID"] == grp
data = prac_data[group]["Response"]
with pm.Model() as irt:
# Priors
a_tmp = pm.Normal('a_tmp',mu=0, sd = 1, shape = 10)
a = pm.Deterministic('a', np.exp(a_tmp))
# We do this transformation since we must have a >= 0
b = pm.Normal('b', mu = 0, sd = 1, shape = 10)
# Now for the hyperpriors on the groups:
theta_mu = pm.Normal('theta_mu', mu = 0, sd = 1)
theta_sigma = pm.Uniform('theta_sigma', upper = 2, lower = 0)
theta = pm.Normal('theta', mu = theta_mu,
sd = theta_sigma, shape = N)
p = getProbs(Disc, Diff, theta, N)
y = pm.Bernoulli('y', p = p, observed = data)
traces[grp] = pm.sample(1000)
The function "getProbs" is supposed to give me an array of probabilities for the Bernoulli random variable, as the probability of responding 1 changes across trials/survey questions for each person. But this method gives me an error because it says to "specify one of p or logit_p", but I thought I did with the function?
Here's the code for "getProbs" in case it's helpful:
def getProbs(Disc, Diff, THETA, Nprt):
# Get a large array of probabilities for the bernoulli random variable
n = len(Disc)
m = Nprt
probs = np.array([])
for th in range(m):
for t in range(n):
p = item(Disc[t], Diff[t], THETA[th])
probs = np.append(probs, p)
return probs
I added the Nprt parameter because if I tried to get the length of THETA, it would give me an error since it is a FreeRV object. I know I can try and vectorize the "item" function, which is just the logistic function I put above, instead of doing it this way, but that also got me an error when I tried to run it.
I think I can do something with pm.Data to fix this, but the documentation isn't exactly clear to me.
Basically, I'm used to building models in JAGS, where you loop through each data point, but pymc3 doesn't seem to work like that. I'm confused about how to build/index my random variables in the model to make sure that the probabilities change how I'd like them to from trial-to-trial, and to make sure that the parameters I'm estimating correspond to the right person in the right group.
Thanks in advance for any help. I'm pretty new to pymc3 and trying to get the hang of it, and wanted to try something different from JAGS.
EDIT: I was able to solve this by first building the array I needed by looping through the trials, then transforming the array using:
p = theano.tensor.stack(p, axis = 0)
I then put this new variable in the "p" argument of the Bernoulli instance and it worked! Here's the updated full model: (below, I imported theano.tensor as T)
group = group.astype('int')
data = prac_data["Response"]
with pm.Model() as irt:
# Priors
# Item parameters:
a = pm.Gamma('a', alpha = 1, beta = 1, shape = 10) # Discrimination
b = pm.Normal('b', mu = 0, sd = 1, shape = 10) # Difficulty
# Now for the hyperpriors on the groups: shape = 2 as there are 2 groups
theta_mu = pm.Normal('theta_mu', mu = 0, sd = 1, shape = 2)
theta_sigma = pm.Uniform('theta_sigma', upper = 2, lower = 0, shape = 2)
# Individual-level person parameters:
# group is a 2*N array that lets the model know which
# theta_mu to use for each theta to estimate
theta = pm.Normal('theta', mu = theta_mu[group],
sd = theta_sigma[group], shape = 2*N)
# Here, we're building an array of the probabilities we need for
# each trial:
p = np.array([])
for n in range(2*N):
for t in range(10):
x = -a[t]*(theta[n] - b[t])
p = np.append(p, x)
# Here, we turn p into a tensor object to put as an argument to the
# Bernoulli random variable
p = T.stack(p, axis = 0)
y = pm.Bernoulli('y', logit_p = p, observed = data)
# On my computer, this took about 5 minutes to run.
traces = pm.sample(1000, cores = 1)
print(az.summary(traces)) # Summary of parameter distributions
I'm trying to recreate results from Bayesian Data Analysis Third Edition.
Chapter 5 Section 3 concerns tumors in rats. a Hierarchical model is fit and the hyperprior used is not one of the densities included in pymc3.
The hyperprior is a*b*(a+b)^-2.5. Here is my attempt using pymc3.
import pymc3 as pm
with pm.Model() as model:
def ab_dist(x):
#Should be log density, from what I have read
a = x[0]
b = x[1]
return a+b-5/2*(a+b)
ab = pm.DensityDist('ab', ab_dist, shape = 2, testval=[2,2])
a = ab[0]
b = ab[1]
theta = pm.Beta('theta',alpha = a,beta = b)
Y= pm.Binomial('y', n = n, p = theta, observed = y)
At this stage, I am returned an error
ValueError: Input dimension mis-match. (input[0].shape[0] = 71, input[1].shape[0] = 20000)
What have I done wrong? Have I correctly implemented the density?
I have a point cloud in 4 dimensions, where each point in the cloud has a location and a value (x,y,z,Value). In addition, I have a 'special' point, S0, within the 3d point cloud; I've used this example to find the closest 10 points in the cloud, relative to S0. Now, I have a numpy array for each of the 10 closest points and their values. How can I interpolate these 10 points, to find the interpolated value at point S0? Example code is shown below:
import numpy as np
import matplotlib.pyplot as plt
numpoints = 20
linexs = 320
lineys = 40
linezs = 60
linexe = 20
lineye = 20
lineze = 0
# Create vectors of points
xpts = np.linspace(linexs, linexe, numpoints)
ypts = np.linspace(lineys, lineye, numpoints)
zpts = np.linspace(linezs, lineze, numpoints)
lin = np.dstack((xpts,ypts,zpts))
# Image line of points
fig = plt.figure()
ax = fig.add_subplot(211, projection='3d')
ax.set_xlim(0,365); ax.set_ylim(-85, 85); ax.set_zlim(0, 100)
ax.plot_wireframe(xpts, ypts, zpts)
ax.view_init(elev=12, azim=78)
def randrange(n, vmin, vmax):
return (vmax - vmin)*np.random.rand(n) + vmin
n = 10
for n in range(21):
xs = randrange(n, 0, 350)
ys = randrange(n, -75, 75)
zs = randrange(n, 0, 100)
ax.scatter(xs, ys, zs)
dat = np.dstack((xs,ys,zs))
ax.set_xlabel('X Label')
ax.set_xlim(0,350)
ax.set_ylabel('Y Label')
ax.set_ylim(-75,75)
ax.set_zlabel('Z Label')
ax.set_zlim(0,100)
ax = fig.add_subplot(212, projection='3d')
ax.set_xlim(0,365); ax.set_ylim(-85, 85); ax.set_zlim(0, 100)
ax.plot_wireframe(xpts,ypts,zpts)
ax.view_init(elev=12, azim=78)
plt.show()
dist = []
# Calculate distance from first point to all other points in cloud
for l in range(len(xpts)):
aaa = lin[0][0]-dat
dist.append(np.sqrt(aaa[0][l][0]**2+aaa[0][l][1]**2+aaa[0][l][2]**2))
full = np.dstack((dat,dist))
aaa = full[0][full[0][:,3].argsort()]
print(aaa[0:10])
A basic example. Note that the meshgrid is not needed for the interpolation, but only to make a fast ufunc to generate an example function A=f(x,y,z), here A=x+y+z.
from scipy.interpolate import interpn
import numpy as np
#make up a regular 3d grid
X=np.linspace(-5,5,11)
Y=np.linspace(-5,5,11)
Z=np.linspace(-5,5,11)
xv,yv,zv = np.meshgrid(X,Y,Z)
# make up a function
# see http://docs.scipy.org/doc/numpy/reference/ufuncs.html
A = np.add(xv,np.add(yv,zv))
#this one is easy enough for us to know what to expect at (.5,.5,.5)
# usage : interpn(points, values, xi, method='linear', bounds_error=True, fill_value=nan)
interpn((X,Y,Z),A,[0.5,0.5,0.5])
Output:
array([ 1.5])
If you pass in an array of points of interest, it will give you multiple answers.