How to conditionally add variables to a Pyomo constraint based on predetermined data settings - pyomo

I'm new to Pyomo.
I would like to know if there is an elegant way to code a constraint that contains variables that we may or may not want to include. The option to include them will be known at the time the model is solved and will be based on settings it reads in from the database.
A good example might be having slacks in the constraint, sometimes we want them and sometimes we don't.
I have tapped out a small demo below for a warehouse location example. In equation buildLimit I have added a slack to allow the number of warehouses to exceed the build limit [the code might contain some syntax errors I haven't run it]
# Import pyomo
from pyomo.environ import *
from pyomo.opt import SolverStatus, TerminationCondition
N = ['Harlingen', 'Memphis', 'Ashland']
M = ['NYC', 'LA', 'Chicago', 'Houston']
d = {('Harlingen', 'NYC'): 1956, \
('Harlingen', 'LA'): 1606, \
('Harlingen', 'Chicago'): 1410, \
('Harlingen', 'Houston'): 330, \
('Memphis', 'NYC'): 1096, \
('Memphis', 'LA'): 1792, \
('Memphis', 'Chicago'): 531, \
('Memphis', 'Houston'): 567, \
('Ashland', 'NYC'): 485, \
('Ashland', 'LA'): 2322, \
('Ashland', 'Chicago'): 324, \
('Ashland', 'Houston'): 1236 }
P = 2
model = ConcreteModel("warehouse location problem")
model.N = Set(dimen=1, initialize=N)
model.M = Set(dimen=1, initialize=M)
model.d = Param(model.N, model.M, within=PositiveIntegers, initialize=d)
model.P = Param(initialize=P)
model.y = Var(model.N, within=Binary)
model.x = Var(model.N, model.M, bounds=(0,1))
##########################
model.buildLimitSlack = Var(within=NonNegativeIntegers)
model.useSlacks = Param() # assume some data read will populate this at some stage before the solve
##########################
# Objective, minimise delivery costs
def obj_rule(model):
return sum(model.d[n,m] * model.x[n,m] for n in model.N for m in model.M) + 99*model.buildLimitSlack
model.obj = Objective(rule=obj_rule)
# All customer demand must be met
def demand_rule(model, m):
return sum(model.x[n,m] for n in model.N) == 1
model.demand = Constraint(model.M, rule=demand_rule)
# Can only ship from a warehouse if that warehouse is built
def supplyOnlyIfBuilt_rule(model, m, n):
return model.x[n,m] <= model.y[n]
model.supplyOnlyIfBuilt = Constraint(model.M, model.N, rule=supplyOnlyIfBuilt_rule)
##############################
#### WE WANT THE SLACK IN THIS EQUATION TO BE OPTIONAL BASED ON DATA SETTINGS
def buildLimit_rule(model):
return sum(model.y[n] for n in model.N) <= model.P + model.buildLimitSlack
model.buildLimit = Constraint(rule=buildLimit_rule)
##############################
I imagine we could have an if statement in the constraint, something like the following. But we don't want that as our model equations will likely have many such optional variables in the same constraint and I don't want to have tons of nested if statements [unless there is a nice way to do this?].
def buildLimit_rule(model):
if model.useSlacks:
return sum(model.y[n] for n in model.N) <= model.P + model.buildLimitSlack
else:
return sum(model.y[n] for n in model.N) <= model.P
model.buildLimit = Constraint(rule=buildLimit_rule)
Any advice?
Thanks in advance

(The question is quite general, so I'll propose some approaches but it's up to you to decide which ones work best in which situation.)
EDIT 1 - sparse components: for sparsity-related issues, see this question: Create a variable with sparse index in pyomo
Essentially, you should initialize your sets so that they only contain the "valid" values, and that will automatically ensure that you are only initializing the required components.
EDIT 2 - optional components: there are several ways to add optional components to the model. A simple if statement is a strategy, or you could look into BuildActions. Another option is to create Expressions (combinations of parameters and variables), and "optionally" change those instead of the constraint definition itself.
If all your constraints have that form (i.e. if 0 is the "neutral" state for your variables), what you can do is re-write them as:
def buildLimit_rule(m):
return sum(m.y[n] for n in m.N) <= m.P + m.buildLimitSlack * m.useSlacks
If you have more than one slack variable with independent on/off parameters:
def buildLimit_rule(m):
return sum(m.y[n] for n in m.N) <= m.P + m.v1 * m.use_v1 + m.v2 * m.use_v2 + ...
The most general (but less readable) approach is to index all the potential slack variables:
model.SLACKS = Set(...) # set for slack variables
model.slack = Var(model.SLACKS)
model.use_slack = Param(model.SLACKS, within=Binary)
def buildLimit_rule(m):
return sum(m.y[n] for n in m.N) <= m.P + sum(m.slack[i] * m.use_slack[i] for i in m.SLACKS)
One other way is to fix the variables you don't need:
if not m.useSlacks:
m.Slack.fix(0)
m.OtherSlack.fix(0)

Related

Is it possible to formulate constraints with pyomo in a matrix way?

I'm a fresh with pyomo.There is a problem that has me very confused recently.
Here is the code:
def unit_commitment():
model = pyo.ConcreteModel()
model.GN = pyo.Set(initialize = GN) # GN=280
model.TN = pyo.Set(initialize = TN) # TN=24
model.LN = pyo.Set(initialize = LN) # LN=6060
model.Pc = pyo.Var(model.GN, model.TN, domain = pyo.NonNegativeReals)
def branch_Cap1(model, l, t):
return sum(Tc[l, n] * model.Pc[n, t] for n in range(GenCount)) <= Fmax_re[l, t] #Tc is a ndarray:(6060,280), GenCount = 280, Fmax_re is a ndarray:(6060,24)
model.branch_Cap1 = pyo.Constraint(model.LN, model.TN, rule=branch_Cap1)
return model
As you can see, for each constraint branch_Cap1[l, t], it has to "for" n=280 times to formulate one single constraint.
And if I want to formulate all of the constraints, it has to take about 280 * 6060 * 24 = 40,723,200 times of calculation.
It takes a very long time.(about 1 hour, which is unacceptable for me)
And I have noticed that pyomo has a formulation called:
pyomo.core.kernel.matrix_constraint.matrix_constraint,matrix_constraint which I think may be helpful to me.
matrix_constraint
So I gave it a try:
model.branhc_Cap = pyomo.core.kernel.matrix_constraint.matrix_constraint(A=Tc, x=model.Pc, ub=Fmax_re)
then it came to an error:
self.x = x
raise ValueError(
ValueError: Argument length must be 280 not 6720
It seems that my model.Pc's length is 6720, which equals to 280 *24, not 280.
So how can I fix the problem? Or if there is some other way for me to take to formulate the constraints more efficiently?
Your help is very important to me.
Thanks a lot.

Is it possible to find all integer solutions?

I wanna get all integer solutions in a limited time, is it possible?
This is a linear, integer constraint satisfaction problem, which can be solved efficiently by OR Tools' CP-SAT. I've modified their example to solve your problem in Python:
from ortools.sat.python import cp_model
class VarArraySolutionPrinter(cp_model.CpSolverSolutionCallback):
"""Print intermediate solutions."""
def __init__(self, variables):
cp_model.CpSolverSolutionCallback.__init__(self)
self.__variables = variables
self.__solution_count = 0
def on_solution_callback(self):
self.__solution_count += 1
for v in self.__variables:
print('%s=%i' % (v, self.Value(v)), end=' ')
print()
def solution_count(self):
return self.__solution_count
def SearchForAllSolutionsSampleSat():
"""Showcases calling the solver to search for all solutions."""
# Creates the model.
model = cp_model.CpModel()
p = [1, 2, 3, 4]
ceq = 30
cgeq = 2
N = len(p)
# Creates the variables
x = [model.NewIntVar(0, 100, f'x{i}') for i in range(N)]
# Create the constraints.
model.Add(sum([xi*pi for xi, pi in zip(x, p)]) == ceq)
model.Add(sum(x) >= cgeq)
# Create a solver and solve.
solver = cp_model.CpSolver()
solution_printer = VarArraySolutionPrinter(x)
status = solver.SearchForAllSolutions(model, solution_printer)
print('Status = %s' % solver.StatusName(status))
print('Number of solutions found: %i' % solution_printer.solution_count())
SearchForAllSolutionsSampleSat()

Change constraints based on variable value in Pyomo

Is there a way of changing the values of a constraint as the solver is running?
Basically, I have a constraint that depends on the value of a variable. The problem is that the constraint is evaluated based on the initial value of the variable, but isn't updated as the variable changes.
Here's a simple example:
from pyomo.environ import *
from pyomo.opt import SolverFactory
import numpy as np
# Setup
model = ConcreteModel()
model.A = Set(initialize = [0,1,2])
model.B = Set(initialize = [0,1,2])
model.x = Var(model.A, model.B, initialize=0)
# A constraint that I'd like to keep updating, based on the value of x
def changing_constraint_rule(model, a):
x_values = list((model.x[a, b].value for b in model.B))
if np.max(x_values) == 0:
return Constraint.Skip
else:
# Not really important what goes here, just as long as it updates the constraint list
if a == 1 : return sum(model.x[a,b] for b in model.B) == 0
else: return sum(model.x[a,b] for b in model.B) == 1
model.changing_constraint = Constraint(model.A, rule = changing_constraint_rule)
# Another constraint that changes the value of x
def bounding_constraint_rule(model, a):
return sum(model.x[a, b] for b in model.B) == 1
model.bounding_constraint = Constraint(
model.A,
rule = bounding_constraint_rule)
# Some objective function
def obj_rule(model):
return(sum(model.x[a,b] for a in model.A for b in model.B))
model.objective = Objective(rule=obj_rule)
# Results
opt = SolverFactory("glpk")
results = opt.solve(model)
results.write()
model.x.display()
If I run model.changing_constraint.pprint() I can see that no constraints have been made, since the initial value of the variable model.x was set to 0.
If it's not possible to change the constraint values while solving, how could I formulate this problem differently to achieve what I'm looking for? I've read this other post but couldn't figure it out from the instructions.
I am giving you the same answer in the other question by #Gabe:
Any if-logic you use inside of rules should not involve the values of
variables (unless it is based on the initial value of a variable, in
which case you would wrap the variable in value() wherever you use it
outside of the main expression that is returned).
for example:
model.x[a, b].value should be model.x[a, b].value()
But still this might not give you the solution what you are looking for.

AttributeError: GaussianMixture instance has no attribute 'loglike'

I'm trying to implement Gaussian Mixture Model with Expectation–Maximization algorithm and I get this error.
This is the Gaussian Mixture model that I used:
class GaussianMixture:
"Model mixture of two univariate Gaussians and their EM estimation"
def __init__(self, data, mu_min=min(data), mu_max=max(data), sigma_min=.1, sigma_max=1, mix=.5):
self.data = data
#init with multiple gaussians
self.one = Gaussian(uniform(mu_min, mu_max),
uniform(sigma_min, sigma_max))
self.two = Gaussian(uniform(mu_min, mu_max),
uniform(sigma_min, sigma_max))
#as well as how much to mix them
self.mix = mix
def Estep(self):
"Perform an E(stimation)-step, freshening up self.loglike in the process"
# compute weights
self.loglike = 0. # = log(p = 1)
for datum in self.data:
# unnormalized weights
wp1 = self.one.pdf(datum) * self.mix
wp2 = self.two.pdf(datum) * (1. - self.mix)
# compute denominator
den = wp1 + wp2
# normalize
wp1 /= den
wp2 /= den
# add into loglike
self.loglike += log(wp1 + wp2)
# yield weight tuple
yield (wp1, wp2)
def Mstep(self, weights):
"Perform an M(aximization)-step"
# compute denominators
(left, rigt) = zip(*weights)
one_den = sum(left)
two_den = sum(rigt)
# compute new means
self.one.mu = sum(w * d / one_den for (w, d) in zip(left, data))
self.two.mu = sum(w * d / two_den for (w, d) in zip(rigt, data))
# compute new sigmas
self.one.sigma = sqrt(sum(w * ((d - self.one.mu) ** 2)
for (w, d) in zip(left, data)) / one_den)
self.two.sigma = sqrt(sum(w * ((d - self.two.mu) ** 2)
for (w, d) in zip(rigt, data)) / two_den)
# compute new mix
self.mix = one_den / len(data)
def iterate(self, N=1, verbose=False):
"Perform N iterations, then compute log-likelihood"
def pdf(self, x):
return (self.mix)*self.one.pdf(x) + (1-self.mix)*self.two.pdf(x)
def __repr__(self):
return 'GaussianMixture({0}, {1}, mix={2.03})'.format(self.one,
self.two,
self.mix)
def __str__(self):
return 'Mixture: {0}, {1}, mix={2:.03})'.format(self.one,
self.two,
self.mix)
And then , while training I get that error in the conditional statement.
# Check out the fitting process
n_iterations = 5
best_mix = None
best_loglike = float('-inf')
mix = GaussianMixture(data)
for _ in range(n_iterations):
try:
#train!
mix.iterate(verbose=True)
if mix.loglike > best_loglike:
best_loglike = mix.loglike
best_mix = mix
except (ZeroDivisionError, ValueError, RuntimeWarning): # Catch division errors from bad starts, and just throw them out...
pass
Any ideas why I have the following error?
AttributeError: GaussianMixture instance has no attribute 'loglike'
The loglike attribute is only created when you call the Estep method.
def Estep(self):
"Perform an E(stimation)-step, freshening up self.loglike in the process"
# compute weights
self.loglike = 0. # = log(p = 1)
You didn't call Estep between creating the GaussianMixture instance and mix.loglike:
mix = GaussianMixture(data)
for _ in range(n_iterations):
try:
#train!
mix.iterate(verbose=True)
if mix.loglike > best_loglike:
And the iterate method is empty (It looks like you forgot some code here).
def iterate(self, N=1, verbose=False):
"Perform N iterations, then compute log-likelihood"
So, there'll be no loglike attribute set on the mix instance by the time you do if mix.loglike. Hence, the AttributeError.
You need to do one of the following:
Call the Estep method (since you set self.loglike there)
Define a loglike attribute in __init__

PYOMO: How to use abstract models with internal data

Hei all,
I am trying to set up an abstract model for a very simple QP of the form
min (x-x0)^2
s.t.
A x = b
C x <= d
I would like to use an abstract model, as I need to resolve with changing parameters (mainly x0, but potentially also A, b, C, d). I am right now struggeling with simply setting the parameters in the model instance. I do not want to use an external data file, but rather internal python variables. All examples I find online use AMPL formatted data files.
This is the code I have right now
import pyomo.environ as pe
model = pe.AbstractModel()
# the sets
model.n = pe.Param(within=pe.NonNegativeIntegers)
model.m = pe.Param(initialize = 1)
model.ss = pe.RangeSet(1, model.n)
model.os = pe.RangeSet(1, model.m)
# the starting point and the constraint parameters
model.x_hat = pe.Param(model.ss)
model.A = pe.Param(model.os, model.ss)
model.b = pe.Param(model.os)
model.C = pe.Param(model.os, model.os)
model.d = pe.Param(model.ss, model.os)
# the decision variables
model.x_projected = pe.Var(model.ss)
# the cosntraints
# A x = b
def sum_of_elements_rule(model):
value = model.A * model.x_projected
return value == model.d
model.sumelem = pe.Constraint(model.os, rule=sum_of_elements_rule)
# C x <= d
def positivity_constraint(model):
return model.C*model.x_projected <= model.d
model.bounds = pe.Constraint(model.ss, rule=positivity_constraint)
# the cost
def cost_rule(model):
return sum((model.x_projected[i] - model.x[i])**2 for i in model.ss)
model.cost = pe.Objective(rule=cost_rule)
instance = model.create_instance()
And somehow here I am stuck. How do I set the parameters now?
Thanks and best, Theo
I know this is an old post but a solution to this could have helped me so here is the solution to this problem:
## TEST
data_init= {None: dict(
n = {None : 3},
d = {0:0, 1:1, 2:2},
x_hat = {0:10, 1:-1, 2:-100},
b = {None: 10}
)}
# create instance
instance = model.create_instance(data_init)
This creates the instance in an equivalent way than what you did but in a more formal way.
Ok, I seemed to have figured out what the problem is. If I want to set a parameter after I create an instance, I need the
mutable=True
flag. Then, I can set the parameter with something like
for i in range(model_dimension):
getattr(instance, 'd')[i] = i
The model dimension I need to choose before i create an instance (which is ok for my case). The instance can be reused with different parameters for the constraints.
The code below should work for the problem
min (x-x_hat)' * (x-x_hat)
s.t.
sum(x) = b
x[i] >= d[i]
with x_hat, b, d as parameters.
import pyomo.environ as pe
model = pe.AbstractModel()
# model dimension
model.n = pe.Param(default=2)
# state space set
model.ss = pe.RangeSet(0, model.n-1)
# equality
model.b = pe.Param(default=5, mutable=True)
# inequality
model.d = pe.Param(model.ss, default=0.0, mutable=True)
# decision var
model.x = pe.Var(model.ss)
model.x_hat = pe.Param(model.ss, default=0.0, mutable=True)
# the cost
def cost_rule(model):
return sum((model.x[i] - model.x_hat[i])**2 for i in model.ss)
model.cost = pe.Objective(rule=cost_rule)
# CONSTRAINTS
# each x_i bigger than d_i
def lb_rule(model, i):
return (model.x[i] >= model.d[i])
model.state_bound = pe.Constraint(model.ss, rule=lb_rule)
# sum of x == P_tot
def sum_rule(model):
return (sum(model.x[i] for i in model.ss) == model.b)
model.state_sum = pe.Constraint(rule=sum_rule)
## TEST
# define model dimension
model_dimension = 3
model.n = model_dimension
# create instance
instance = model.create_instance()
# set d
for i in range(model_dimension):
getattr(instance, 'd')[i] = i
# set x_hat
xh = (10,1,-100)
for i in range(model_dimension):
getattr(instance, 'x_hat')[i] = xh[i]
# set b
instance.b = 10
# solve
solver = pe.SolverFactory('ipopt')
result = solver.solve(instance)
instance.display()