I am just starting with Pyomo and I have a big problem.
I want to run an Abstract Model without using the terminal. I can do it with a concrete model but I have serious problems to do it in with the abstract one.
I just want to use F5 and run the code.
This ismy program:
import pyomo
from pyomo.environ import *
#
# Model
#
model = AbstractModel()
#Set: Indices
model.Unit = Set()
model.Block = Set()
model.DemBlock = Set()
#Parameters
model.EnergyBid = Param(model.Unit, model.Block)
model.PriceBid = Param(model.Unit, model.Block)
model.EnergyDem = Param(model.DemBlock)
model.PriceDem = Param(model.DemBlock)
model.Pmin = Param(model.Unit)
model.Pmax = Param(model.Unit)
#Variables definition
model.PD = Var(model.DemBlock, within=NonNegativeReals)
model.PG = Var(model.Unit,model.Block, within=NonNegativeReals)
#Binary variable
model.U = Var(model.Unit, within = Binary)
#Objective
def SocialWellfare(model):
SocialWellfare = sum([model.PriceDem[i]*model.PD[i] for i in model.DemBlock]) - sum([model.PriceBid[j,k]*model.PG[j,k] for j in model.Unit for k in model.Block ])
return SocialWellfare
model.SocialWellfare = Objective(rule=SocialWellfare, sense=maximize)
#Constraints
#Max and min Power generated
def PDmax_constraint(model,p):
return ((model.PD[p] - model.EnergyDem[p])) <= 0
model.PDmax = Constraint(model.DemBlock, rule=PDmax_constraint)
def PGmax_constraint(model,n,m):
return ((model.PG[n,m] - model.EnergyBid[n,m])) <= 0
model.PGmax = Constraint(model.Unit, model.Block,rule = PGmax_constraint)
def Power_constraintDW(model,i):
return ((sum(model.PG[i,k] for k in model.Block))-(model.Pmin[i] * model.U[i]) ) >= 0
model.LimDemandDw = Constraint(model.Unit, rule=Power_constraintDW)
def Power_constraintUP(model,i):
return ((sum(model.PG[i,k] for k in model.Block) - (model.Pmax[i])*model.U[i])) <= 0
model.LimDemandaUp = Constraint(model.Unit, rule=Power_constraintUP)
def PowerBalance_constraint(model):
return (sum(model.PD[i] for i in model.DemBlock) - sum(model.PG[j,k] for j in model.Unit for k in model.Block)) == 0
model.PowBalance = Constraint(rule = PowerBalance_constraint)
model.pprint()
instance = model.create('datos_transporte.dat')
## Create the ipopt solver plugin using the ASL interface
solver = 'ipopt'
solver_io = 'nl'
opt = SolverFactory(solver,solver_io=solver_io)
results = opt.solve(instance)
results.write()
Any help with the last part??
Thanks anyway,
I think your example is actually working. Starting in Pyomo 4.1, the solution returned from the solver is stored directly into the instance that was solved, and is not returned in the solver results object. This change was made because generating the representation of the solution in the results object was rather expensive and was not easily parsed by people. Working with the model instance directly is more natural.
It is unfortunate that the results object reports the number of solutions: 0, although this is technically correct: the results object holds no solutions ... but the Solver section should indicate that a solution was returned and stored into the model instance.
If you want to see the result returned by the solver, you can print out the current status of the model using:
instance.display()
after the call to solve(). That will report the current Var values that were returned from the solver. You will want to pay attention to the stale column:
False indicates the value is not "stale" ... that is, it was either set by the user (before the call to solve()), or it was returned from the solver (after the call to solve()).
True indicates the solver did not return a value for that variable. This is usually because the variable was not referenced by the objective or any enabled constraints, so Pyomo never sent the variable to the solver.
[Note: display() serves a slightly different role than pprint(): pprint() outputs the model structure, whereas display() outputs the model state. So, for example, where pprint() will output the constraint expressions, display() will output the numerical value of the expression (using the current variable/param values).]
Edited to expand the discussion of display() & pprint()
Related
I'm trying to practice using pymc3 on the kinds of data I come across in my research, but I'm having trouble thinking through how to fit the model when each person gives me multiple data points, and each person comes from a different group (so trying a hierarchical model).
Here's the practice scenario I'm using: Suppose we have 2 groups of people, N = 30 in each group. All 60 people go through a 10 question survey, where each person can response ("1") or not respond ("0") to each question. So, for each person, I have an array of length 10 with 1's and 0's.
To model these data, I assume each person has some latent trait "theta", and each item has a "discrimination" a and a "difficulty" b (this is just a basic item response model), and the probability of responding ("1") is given by: (1 + exp(-a(theta - b)))^(-1). (Logistic applied to a(theta - b) .)
Here is how I tried to fit it using pymc3:
traces = {}
for grp in range(2):
group = prac_data["Group ID"] == grp
data = prac_data[group]["Response"]
with pm.Model() as irt:
# Priors
a_tmp = pm.Normal('a_tmp',mu=0, sd = 1, shape = 10)
a = pm.Deterministic('a', np.exp(a_tmp))
# We do this transformation since we must have a >= 0
b = pm.Normal('b', mu = 0, sd = 1, shape = 10)
# Now for the hyperpriors on the groups:
theta_mu = pm.Normal('theta_mu', mu = 0, sd = 1)
theta_sigma = pm.Uniform('theta_sigma', upper = 2, lower = 0)
theta = pm.Normal('theta', mu = theta_mu,
sd = theta_sigma, shape = N)
p = getProbs(Disc, Diff, theta, N)
y = pm.Bernoulli('y', p = p, observed = data)
traces[grp] = pm.sample(1000)
The function "getProbs" is supposed to give me an array of probabilities for the Bernoulli random variable, as the probability of responding 1 changes across trials/survey questions for each person. But this method gives me an error because it says to "specify one of p or logit_p", but I thought I did with the function?
Here's the code for "getProbs" in case it's helpful:
def getProbs(Disc, Diff, THETA, Nprt):
# Get a large array of probabilities for the bernoulli random variable
n = len(Disc)
m = Nprt
probs = np.array([])
for th in range(m):
for t in range(n):
p = item(Disc[t], Diff[t], THETA[th])
probs = np.append(probs, p)
return probs
I added the Nprt parameter because if I tried to get the length of THETA, it would give me an error since it is a FreeRV object. I know I can try and vectorize the "item" function, which is just the logistic function I put above, instead of doing it this way, but that also got me an error when I tried to run it.
I think I can do something with pm.Data to fix this, but the documentation isn't exactly clear to me.
Basically, I'm used to building models in JAGS, where you loop through each data point, but pymc3 doesn't seem to work like that. I'm confused about how to build/index my random variables in the model to make sure that the probabilities change how I'd like them to from trial-to-trial, and to make sure that the parameters I'm estimating correspond to the right person in the right group.
Thanks in advance for any help. I'm pretty new to pymc3 and trying to get the hang of it, and wanted to try something different from JAGS.
EDIT: I was able to solve this by first building the array I needed by looping through the trials, then transforming the array using:
p = theano.tensor.stack(p, axis = 0)
I then put this new variable in the "p" argument of the Bernoulli instance and it worked! Here's the updated full model: (below, I imported theano.tensor as T)
group = group.astype('int')
data = prac_data["Response"]
with pm.Model() as irt:
# Priors
# Item parameters:
a = pm.Gamma('a', alpha = 1, beta = 1, shape = 10) # Discrimination
b = pm.Normal('b', mu = 0, sd = 1, shape = 10) # Difficulty
# Now for the hyperpriors on the groups: shape = 2 as there are 2 groups
theta_mu = pm.Normal('theta_mu', mu = 0, sd = 1, shape = 2)
theta_sigma = pm.Uniform('theta_sigma', upper = 2, lower = 0, shape = 2)
# Individual-level person parameters:
# group is a 2*N array that lets the model know which
# theta_mu to use for each theta to estimate
theta = pm.Normal('theta', mu = theta_mu[group],
sd = theta_sigma[group], shape = 2*N)
# Here, we're building an array of the probabilities we need for
# each trial:
p = np.array([])
for n in range(2*N):
for t in range(10):
x = -a[t]*(theta[n] - b[t])
p = np.append(p, x)
# Here, we turn p into a tensor object to put as an argument to the
# Bernoulli random variable
p = T.stack(p, axis = 0)
y = pm.Bernoulli('y', logit_p = p, observed = data)
# On my computer, this took about 5 minutes to run.
traces = pm.sample(1000, cores = 1)
print(az.summary(traces)) # Summary of parameter distributions
This paper describes Pyomo's Differential and Algebraic Equations framework. It also mentions multi-stage problems; however, it does not show a complete example of such a problem. Does such an example exist somewhere?
The following demonstrates a complete minimum working example of a multi-stage optimization problem using Pyomo's DAE system:
#!/usr/bin/env python3
#http://www.gpops2.com/Examples/OrbitRaising.html
from pyomo.environ import *
from pyomo.dae import *
from pyomo.opt import SolverStatus, TerminationCondition
import random
import matplotlib.pyplot as plt
T = 10 #Maximum time for each stage of the model
STAGES = 3 #Number of stages
m = ConcreteModel() #Model
m.t = ContinuousSet(bounds=(0,T)) #Time variable
m.stages = RangeSet(0, STAGES) #Stages in the range [0,STAGES]. Can be thought of as an integer-valued set
m.a = Var(m.stages, m.t) #State variable defined for all stages and times
m.da = DerivativeVar(m.a, wrt=m.t) #First derivative of state variable with respect to time
m.u = Var(m.stages, m.t, bounds=(0,1)) #Control variable defined for all stages and times. Bounded to range [0,1]
#Setting the value of the derivative.
def eq_da(m,stage,t): #m argument supplied when function is called. `stage` and `t` are given values from m.stages and m.t (see below)
return m.da[stage,t] == m.u[stage,t] #Derivative is proportional to the control variable
m.eq_da = Constraint(m.stages, m.t, rule=eq_da) #Call constraint function eq_da for each unique value of m.stages and m.t
#We need to connect the different stages together...
def eq_stage_continuity(m,stage):
if stage==m.stages.last(): #The last stage doesn't connect to anything
return Constraint.Skip #So skip this constraint
else:
return m.a[stage,T]==m.a[stage+1,0] #Final time of each stage connects with the initial time of the following stage
m.eq_stage_continuity = Constraint(m.stages, rule=eq_stage_continuity)
#Boundary conditions
def _init(m):
yield m.a[0,0] == 0 #Initial value (at zeroth stage and zeroth time) of `a` is 0
yield ConstraintList.End
m.con_boundary = ConstraintList(rule=_init) #Repeatedly call `_init` until `ConstraintList.End` is returned
#Objective function: maximize `a` at the end of the final stage
m.obj = Objective(expr=m.a[STAGES,T], sense=maximize)
#Get a discretizer
discretizer = TransformationFactory('dae.collocation')
#Disrectize the model
#nfe (number of finite elements)
#ncp (number of collocation points within finite element)
discretizer.apply_to(m,nfe=30,ncp=6,scheme='LAGRANGE-RADAU')
#Get a solver
solver = SolverFactory('ipopt', keepfiles=True, log_file='/z/log', soln_file='/z/sol')
solver.options['max_iter'] = 100000
solver.options['print_level'] = 1
solver.options['linear_solver'] = 'ma27'
solver.options['halt_on_ampl_error'] = 'yes'
#Solve the model
results = solver.solve(m, tee=True)
print(results.solver.status)
print(results.solver.termination_condition)
#Retrieve the results in a pleasant format
r_t = [t for s in sorted(m.stages) for t in sorted(m.t)]
r_a = [value(m.a[s,t]) for s in sorted(m.stages) for t in sorted(m.t)]
r_u = [value(m.u[s,t]) for s in sorted(m.stages) for t in sorted(m.t)]
plt.plot(r_t, r_a, label="r_a")
plt.plot(r_t, r_u, label="r_u")
plt.legend()
plt.show()
Is there a way of changing the values of a constraint as the solver is running?
Basically, I have a constraint that depends on the value of a variable. The problem is that the constraint is evaluated based on the initial value of the variable, but isn't updated as the variable changes.
Here's a simple example:
from pyomo.environ import *
from pyomo.opt import SolverFactory
import numpy as np
# Setup
model = ConcreteModel()
model.A = Set(initialize = [0,1,2])
model.B = Set(initialize = [0,1,2])
model.x = Var(model.A, model.B, initialize=0)
# A constraint that I'd like to keep updating, based on the value of x
def changing_constraint_rule(model, a):
x_values = list((model.x[a, b].value for b in model.B))
if np.max(x_values) == 0:
return Constraint.Skip
else:
# Not really important what goes here, just as long as it updates the constraint list
if a == 1 : return sum(model.x[a,b] for b in model.B) == 0
else: return sum(model.x[a,b] for b in model.B) == 1
model.changing_constraint = Constraint(model.A, rule = changing_constraint_rule)
# Another constraint that changes the value of x
def bounding_constraint_rule(model, a):
return sum(model.x[a, b] for b in model.B) == 1
model.bounding_constraint = Constraint(
model.A,
rule = bounding_constraint_rule)
# Some objective function
def obj_rule(model):
return(sum(model.x[a,b] for a in model.A for b in model.B))
model.objective = Objective(rule=obj_rule)
# Results
opt = SolverFactory("glpk")
results = opt.solve(model)
results.write()
model.x.display()
If I run model.changing_constraint.pprint() I can see that no constraints have been made, since the initial value of the variable model.x was set to 0.
If it's not possible to change the constraint values while solving, how could I formulate this problem differently to achieve what I'm looking for? I've read this other post but couldn't figure it out from the instructions.
I am giving you the same answer in the other question by #Gabe:
Any if-logic you use inside of rules should not involve the values of
variables (unless it is based on the initial value of a variable, in
which case you would wrap the variable in value() wherever you use it
outside of the main expression that is returned).
for example:
model.x[a, b].value should be model.x[a, b].value()
But still this might not give you the solution what you are looking for.
I am trying to save the solutions of this optimization problem but they are Nonetype. Therefore I want to convert them to float but I get this error:
float() argument must be a string or a number, not 'NoneType'
It is rare due to in the printed solution from results.write() x1 is 6.57142857142857.
from coopr . pyomo import *
from pyomo.opt import SolverFactory
def create_model(N=[], M=[], c={}, a={}, b={}):
model = ConcreteModel()
model.x = Var(N, within=NonNegativeReals)
model.obj = Objective(expr=sum(c[i]*model.x[i] for i in N))
def con_rule(model, m):
return sum(a[i,m]*model.x[i] for i in N) >= b[m]
model.con = Constraint(M, rule=con_rule)
return model
model = create_model(N = [1,2], M = [1,2], c = {1:1, 2:2},
a = {(1,1):5, (2,1):4, (1,2):7, (2,2):3},
b = {1:11, 2:46})
#model.pprint()
instance = model.create()
#instance.pprint()
opt = SolverFactory("glpk")
results = opt.solve(instance, load_solutions=False)
results.write()
x_1=float( model.x[1].value)
#x_2=float( model.x[2].value or 0)
First, model.create() is deprecated on the most recent version of Pyomo. I believe it is now renamed to model.create_instance.
Second, you are solving the instance object returned from model.create(), which is a different object from model. Therefore, you should be accessing the .value attribute of variables on the instance object and not the model object.
Third, you are starting from a ConcreteModel, which means there is no need to call model.create() (or model.create_instance()). This is simply creating an unnecessary copy of what is already a "concrete instance". I.e., you could send the model object to the solver and the code accessing .value would work as is.
The create_instance method is only necessary when you start from an AbstractModel, where you then typically pass the name of some .dat file to it.
import tensorflow as tf
array = tf.Variable(tf.random_normal([10]))
i = tf.constant(0)
l = []
def cond(i,l):
return i < 10
def body(i,l):
temp = tf.gather(array,i)
l.append(temp)
return i+1,l
index,list_vals = tf.while_loop(cond, body, [i,l])
I want to process a tensor array in the similar way as described in the above code. In the body of the while loop I want to process the array by element by element basis to apply some function. For demonstration, I have given a small code snippet. However, it is giving an error message as follows.
ValueError: Number of inputs and outputs of body must match loop_vars: 1, 2
Any help in resolving this is appreciated.
Thanks
Citing the documentation:
loop_vars is a (possibly nested) tuple, namedtuple or list
of tensors that is passed to both cond and body
You cannot pass regular python array as a tensor. What you can do, is:
i = tf.constant(0)
l = tf.Variable([])
def body(i, l):
temp = tf.gather(array,i)
l = tf.concat([l, [temp]], 0)
return i+1, l
index, list_vals = tf.while_loop(cond, body, [i, l],
shape_invariants=[i.get_shape(),
tf.TensorShape([None])])
The shape invariants are there, because normally tf.while_loop expects the shapes of tensors inside while loop won't change.
sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run(list_vals)
Out: array([-0.38367489, -1.76104736, 0.26266089, -2.74720812, 1.48196387,
-0.23357525, -1.07429159, -1.79547787, -0.74316853, 0.15982138],
dtype=float32)
TF offers a TensorArray to deal with such cases. From the doc,
Class wrapping dynamic-sized, per-time-step, write-once Tensor arrays.
This class is meant to be used with dynamic iteration primitives such as while_loop and map_fn. It supports gradient back-propagation via special "flow" control flow dependencies.
Here is an example,
import tensorflow as tf
array = tf.Variable(tf.random_normal([10]))
step = tf.constant(0)
output = tf.TensorArray(dtype=tf.float32, size=0, dynamic_size=True)
def cond(step, output):
return step < 10
def body(step, output):
output = output.write(step, tf.gather(array, step))
return step + 1, output
_, final_output = tf.while_loop(cond, body, loop_vars=[step, output])
final_output = final_output.stack()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(sess.run(final_output))