Set up model in CPLEX using Python API - python-2.7

In fact, I'm trying to implement the very simple model formulation:
min sum_i(y_i*f_i) + sum_i(sum_j(x_ij*c_ij))
s.t. sum_i(x_ij) = 1 for all j
x_ij <= y_i for all i,j
x_ij, y_i are binary
But I simply cannot figure out how the Python API works. They suggest creating variables like this:
model.variables.add(obj = fixedcost,
lb = [0] * num_facilities,
ub = [1] * num_facilities,
types = ["B"] * num_facilities)
# Create one binary variable for each facility/client pair. The variables
# model whether a client is served by a facility.
for c in range(num_clients):
model.variables.add(obj = cost[c],
lb = [0] * num_facilities,
ub = [1] * num_facilities,
types = ["B"] * num_facilities)
# Create corresponding indices for later use
supply = []
for c in range(num_clients):
supply.append([])
for f in range(num_facilities):
supply[c].append((c+1)*(num_facilities)+f)
# Constraint no. 1:
for c in range(num_clients):
assignment_constraint = cplex.SparsePair(ind = [supply[c][f] for f in \
range(num_facilities)],
val = [1] * num_facilities)
model.linear_constraints.add(lin_expr = [assignment_constraint],
senses = ["L"],
rhs = [1])
For this constraints I have no idea how the variables from above are refered to since it only mentions an auxiliary list of lists. Can anyone explain to me how that should work? The problem is easy, I also know how to do it in C++ but the Python API is a closed book to me.
The problem is the uncapacitated facility location problem and I want to adapt the example file facility.py
EDIT: One idea for constraint no.2 is to create one-dimensional vectors and use vector addition to create the final constraint. But that tells me that this is an unsupported operand for SparsePairs
for f in range(num_facilities):
index = [f]
value = [-1.0]
for c in range(num_clients):
open_constraint = cplex.SparsePair(ind = index, val = value) + cplex.SparsePair(ind = [supply[c][f]], val = [1.0])
model.linear_constraints.add(lin_expr=[open_constraint],
senses = ["L"],
rhs = [0])

The Python API is closer to the C Callable Library in nature than the C++/Concert API. Variables are indexed from 0 to model.variables.get_num() - 1 and can be referred to by index, for example, when creating constraints. They can also be referred to by name (the add method has an optional names argument). See the documentation for the VariablesInterface here (this is for version 12.5.1, which I believe you are using, given your previous post).
It may help to start looking at the most simple examples like lpex1.py (and read the comments). Finally, I highly recommend playing with the Python API from the interactive Python prompt (aka the REPL). You can read the help there and type things in to see what they do.
You also might want to take a look at the docplex package. This is a modeling layer built on top of the CPLEX Python API (or which can solve on the cloud if you don't have a local installation of CPLEX installed).

Related

How to look-up constraint by number in a Pyomo model?

We are attempted to debug a Pyomo model. Ipopt with halt_on_ampl_error True tells up Error evaluating constraint 30. Is there an easy way to programmatically look up a constraint in a Pyomo model by its number (using the numbering of Ipopt)?
You can just modify your call to solve slightly:
opt = SolverFactory('ipopt')
res = opt.solve(model, symbolic_solver_labels=True)
Then, you should see a more useful error message in the ipopt output.
Let me extend this answer to address the other part of the question regarding looking up a constraint by its number. Because Pyomo variables and constraints do not have indices, this is very solver-interface-specific. For Pynumero, you have a couple of options. Suppose you have the following ConcreteModel called m and PyomoNLP called nlp.
import pyomo.environ as pyo
from pyomo.contrib.pynumero.interfaces.pyomo_nlp import PyomoNLP
m = pyo.ConcreteModel()
m.x = pyo.Var()
m.y = pyo.Var()
m.obj = pyo.Objective(expr=m.y)
m.c1 = pyo.Constraint(expr=m.y >= m.x)
m.c2 = pyo.Constraint(expr=m.y >= -m.x)
nlp = PyomoNLP(m)
If you just want to get the index of of a few variables or constraints, you can use
var_indices = nlp.get_primal_indices([m.x, m.y])
con_indices = nlp.get_constraint_indices([m.c1, m.c2])
If you want complete maps both directions, you can use
con_to_index = dict()
index_to_con = dict()
var_to_index = pyo.ComponentMap()
index_to-var = dict()
for ndx, var in enumerate(nlp.get_pyomo_variables()):
var_to_index[var] = ndx
index_to_var[ndx] = var
for ndx, con in enumerate(nlp.get_pyomo_constraints()):
con_to_index[con] = ndx
index_to_con[ndx] = con
I will assume that you are using the PyomoNLP interface in PyNumero. If you are using the asl interface, this is a little more difficult. With the PyomoNLP interface in PyNumero, there are several methods that should be able to do what you need. Have a look at the comments in pyomo_nlp.py. Note that this stuff is pretty new and the API is subject to change.

Converting segments of large .cif files to smaller .pdb files

I'm trying to carve out some binding sites with ligands from cif-files of ribosome crystal structures, and have encountered an annoying problem involving a type error.
TypeError: %c requires int or char
Using the code below,
from Bio.PDB import *
from Bio import PDB
class save_res(Select):
def accept_residue(self, residue):
if residue in keep_res_list:
print(residue)
return 1
else:
return 0
keep_res_list = []
parser = MMCIFParser()
structure = parser.get_structure("1vvj.cif", "./1vvj.cif")
structure = structure[0]
atom_list = Selection.unfold_entities(structure, "A") # A for atoms
ns = NeighborSearch(atom_list)
for residue in structure.get_residues():
if residue.get_resname() == "PAR":
for atom in residue:
center = atom.get_coord()
neighbors = ns.search(center, 5.0)
neighbor_residue_list = Selection.unfold_entities(neighbors, "R")
for res in neighbor_residue_list:
if res not in keep_res_list:
keep_res_list.append(res)
io = PDBIO()
io.set_structure(structure)
io.save("1vvj_bs.pdb", save_res())
gives me the error:
File "/scratch/software/anaconda3/envs/my-devel-3.6/lib/python3.6/site-packages/Bio/PDB/PDBIO.py", line 112, in _get_atom_line
return _ATOM_FORMAT_STRING % args
TypeError: %c requires int or char
This code works well when changing the pdb-id to 1fyb, which also has the same ligand id.
I'm thinking the problem stems from the vast amounts of chains and their IDs in the original file. Am I completely wrong in this assumption or does anyone know how to fix this? I've been trying to find a way to rename the chain IDs, but haven't found a viable method to do this.
Your help is appreciated.
The chain name format in _ATOM_FORMAT_STRING is %c, while in this case you have chain named QA.
Chain names in PDB files were traditionally single characters.
But there are only so many letters and digits. For ribosome it's necessary to use longer names. The pdb format has space for a second letter -- empty column on the left from the 1-character chain name. Many programs support it, but not all, and this is not part of the official specification.
So you can either use PDB files with 2-character chains (if the rest of your workflow supports it) or rename chains in the output (your output is only a tiny part of the original structure).
Here is how to do it in gemmi:
import gemmi
structure = gemmi.read_structure('1vvj.cif')
model = structure[0]
ns = gemmi.NeighborSearch(model, structure.cell, 5.0).populate()
for chain in model:
for residue in chain:
if residue.name == 'PAR':
for atom in residue:
for nb in ns.find_neighbors(atom):
nb.to_cra(model).residue.flag = 'y'
sel = gemmi.Selection().set_residue_flags('y')
new_structure = sel.copy_structure_selection(structure)
#new_structure.remove_empty_chains()
#new_structure.shorten_chain_names()
new_structure.write_minimal_pdb('1vvj-par.pdb')
The two commented out lines are renaming the chains.
One difference comparing with your code is that the NeighborSearch in gemmi is symmetry-aware. It finds also nearby atoms from symmetry mates. In BioPython you search only in asymmetric unit (asu).
Both are different than the biological assembly --
PDB-101 covers it nicely.
If you'd like to search in asu only -- replace structure.cell with gemmi.UnitCell() above, i.e. don't pass the unit cell information.
(You can ask such questions on bioinformatics.SE -- it should get answer sooner there).

Sparse index use for optimization with Pyomo model

I have a Pyomo model connected to a Django-created website.
My decision variable has 4 indices and I have a huge amount of constraints running on it.
Since Pyomo takes a ton of time to read in the constraints with so many variables, I want to sparse out the index set to only contain variables that actually could be 1 (i have some conditions on that)
I saw this post
Create a variable with sparse index in pyomo
and tried a for loop for all my conditions. I created a set "AllowedVariables" to later put this inside my constraints.
But Django's server takes so long to create this set while performing the system check, it never comes out.
Currently i have this model:
model = AbstractModel()
model.x = Var(model.K, model.L, model.F, model.Z, domain=Boolean)
def ObjRule(model):
# some rule, sense maximize
model.Obj = pyomo.environ.Objective(rule=ObjRule, sense=maximize)
def ARule(model,l):
maxA = sum(model.x[k,l,f,z] * for k in model.K for f in model.F
for z in model.Z and (k,l,f,z) in model.AllowedVariables)
return maxA <= 1
model.maxA = Constraint(model.L, rule=ARule)
The constraint is exemplary, I have 15 more similar ones. I currently create "AllowedVariables" this way:
AllowedVariables = []
for k in model.K:
for l in model.L:
..... check all sorts of conditions, break if not valid
AllowedVaraibles.append((k,l,f,z))
model.AllowedVariables = Set(initialize=AllowedVariables)
Using this, the Django server starts checking....and never stops
performing system checks...
Sadly, I somehow need some restriction on the variables or else the reading for the solver will take way to long since the constraints contain so many unnecessary variables that have to be 0 anyways.
Any ideas on how I can sparse my variable set?

Changing index reference for Set in Pyomo

Is there a way to change the Set indexing from 1 indexed to 0 indexed in Pyomo? It's very difficult to keep everything straight when you are dealing with multiple objects where Pyomo is 1 referenced and everything else from Python is 0 referenced.
The reason for this is to generate a model fitting routine for multiple circuit devices. Instead of recreating the entire model over and over, I want to define it once with an AbstractModel. Then I can just reload the data and resolve for each device.
In my objective function, I'm defining intermediate values using list comprehension. Once these intermediate values are generated, they are now 0 referenced. An example of what I'm doing is below. As you can see, I have to have some parameters declared with [i] and others with [i-1]. It just becomes difficult and confusing when the functions become large. It would make a whole lot more sense if everything was just 0 referenced so that it was consistent with standard Python code. I was hoping there was some easy option or setting to declare whether a Set is 0 or 1 referenced.
y11intre = [1 / m.Ra[1] + 1 / m.Rb[1] for i in m.n]
y11intim = [m.w[i] * (m.Ca[1] + m.Cb[1]) for i in m.n]
y12intre = ...
...
z11intre = [-y22intim[i-1] * ... for i in m.n]
...
z11re = [m.Rae[1] + z11intre[i-1] for i in m.n]
z11im = [m.w[i] * m.Lae[1] + z11intim[i-1] for i in m.n]
You can provide the starting and stopping point to RangeSet to give you the values you want:
m.r = RangeSet(0,5) # [0,1,2,3,4,5]
m.s = RangeSet(0,4) # [0,1,2,3,4]

Pyomo - Disjunctive Programming - Passing a BigM Suffix

I am implementing a disjunctive programming problem in Pyomo. In the disjuncts I have nonconvex nonlinear constraints. Therefore I can't use the BigM transformation directly. I have to pass suffixes with the values of the bigM. Here is a snippet of my code:
m = ConcreteModel()
m.Block1 = Block()
#... Here some variables and other constraints are defined.
# Defining the rule for the disjunct:
def _d(disjunct, flag):
m = disjunct.parent_block()
if flag:
# Here 2 constraints are valid
disjunct.c1 = Constraint(expr=x1==0)
disjunct.c2 = Constraint(expr=x2==0)
else:
# Here 7 constraints are valid
disjunct.c1 = Constraint(expr=y1==0)
disjunct.c2 = Constraint(expr=y2==0)
disjunct.c3 = Constraint(expr=y3==0)
...
# Define the disjunct:
m.Block1.d1 = Disjunct([0,1], rule=_d)
# Define the disjunction
def _c1(model):
return [model.d1[0], model.d1[1]]
m.Block1.c1 = Disjunction(rule=_c1)
# Applying the transformation:
TransformationFactory('gdp.bigm').apply_to(m)
I have to pass suffixes for the bigM and don't know how to do this. Also how can this be done with different bigM-values for different constraints?
I'm looking forward for your help.
For linear constraints, Pyomo can automatically compute appropriate big-M values for you. For nonlinear constraints, you'll want to add:
m.BigM = Suffix(direction=Suffix.LOCAL)
m.BigM[m.Block1.d1[0].c1] = your_value_here
Note that different ways of expressing disjunctions are also available, which might make things easier for you. I personally like option 2 the best if I want to be clear, and option 3 if I want to be more concise: http://pyomo.readthedocs.io/en/latest/modeling_extensions/gdp.html