I constructed an abstract model in Pyomo and it worked well.
However, when I try to use a dictionary to instantiate the abstract model, I got the following errors "ERROR: Rule failed when generating expression for objective value:
RuntimeError: Cannot iterate over abstract Set 'I' before it has been
constructed (initialized)."
To be specific, here's the issue:
from pyomo.environ import *
model = AbstractModel()
model.D = Set()
model.I = Set()
model.w = Param(model.D)
model.S_0 = Param(model.D)
model.x = Var(real_model.I, model.D)
def sum_cubic(m):
return sum(w[j]*(m.x[i][j]-m.S_0[j])**3 for i in model.I for j in model.D)
model.value = Objective(rule = sum_cubic, sense = maximize)
model.pprint()
The above code runs just fine. But errors are given when I add the following codes right after it where names and S_0 are predefined dictionaries:
data = {None:{
'D':{None: names},
'I':{None: list(range(1,4))},
'w':[0.3,0.3,0.4],
'S_0':S_0,
}
}
real_model = model.create_instance(data)
ERROR: Rule failed when generating expression for objective value:
RuntimeError: Cannot iterate over abstract Set 'I' before it has been
constructed (initialized).
ERROR: Constructing component 'value' from data=None failed:
RuntimeError: Cannot iterate over abstract Set 'I' before it has been
constructed (initialized).
Could anyone help me with that? Thanks.
You have a couple things biting you here...
you are mixing m. and model. in your function, where you should be using m. because that is the self-referencing parameter of your function signature
you are indexing m.x incorrectly. It should be tuple-indexed (see mine)
your data dictionary is not set up right for the parameters. (see mine)
Good luck!
from pyomo.environ import *
model = AbstractModel()
model.D = Set()
model.I = Set()
model.w = Param(model.D)
model.S_0 = Param(model.D)
model.x = Var(model.I, model.D)
def sum_cubic(m):
return sum(m.w[j]*(m.x[i, j]-m.S_0[j])**3 for i in m.I for j in m.D)
model.value = Objective(rule = sum_cubic, sense = maximize)
model.pprint()
names=['a', 'b', 'c']
data = { None:{
'D': {None: names},
'I': {None: [1, 2, 3]},
'w': {'a': 0.4, 'b': 0.3, 'c': 0.2},
'S_0': {'a': 1.0, 'b': 2.0, 'c': 3.0} }
}
real_model = model.create_instance(data)
Related
I am trying to add a simple constraint to my Pyomo model:
maximum{x[1]-a,b}+maximum{x[2]-a,b} >= c
where a,b,c are given predefined parameters and x is the decision variable indexed by set model.I which is initialize to [1,2].
In order to achieve this, I tried the following method with no success:
from pyomo.environ import *
from pyomo.opt import SolverFactory
from pyomo.environ import *
model = AbstractModel()
model.I = Set()
model.a = Param()
model.b = Param()
model.c = Param()
model.x = Var(model.I)
def my_rule(model):
total = 0
for i in model.I:
val = model.x[i] - model.a
if val < model.b:
val = model.b
total = total + val
return total >= model.c
model.my_constraint = Constraint(rule=my_rule)
data = {None:{
'I': {None: [1,2]},
'a': {None: 3},
'b': {None: 5},
'c': {None: 4}}
}
real_model = model.create_instance(data)
The error given is:
WARNING: DEPRECATED: Chained inequalities are deprecated. Use the inequality()
function to express ranged inequality expressions.
ERROR: Rule failed when generating expression for constraint my_constraint:
TypeError: Attempting to form a compound inequality with two lower bounds
The inequality expression:
x[1] - 3 < 5.0
contains non-constant terms (variables) that were evaluated in an
unexpected Boolean context at
File '<ipython-input-11-e3a6c44c5b8e>', line 14:
if val < model.b:
Evaluating Pyomo variables in a Boolean context, e.g.
if expression <= 5:
is generally invalid. If you want to obtain the Boolean value of
the expression based on the current variable values, explicitly
evaluate the expression using the value() function:
if value(expression) <= 5:
or
if value(expression <= 5):
Could any one help me with this? Thanks a lot.
Couple problems...
You can't bury an "if" statement inside of the constraint construction where the condition depends on the evaluation of variables.
max and min functions can't be directly represented in a LP/IP if that is what you are trying to do. There may be ways to reformulate...however...
Your constraint is somewhat nonsensical. If b and c are fixed values then the constraint you presented is ALWAYS true because b>c as stated, so:
max(anything, b) + max(anything else, b) > c is always true.
Time to re-examine your problem formulation. :)
I want to import parameters from 1 excel sheet, (in the future also 1 csv file) and some parameters that I want to set in the code.
I am importing this values using pandas. But than I don´t know how to pass them to the instance. I tried various options but I am only guessing...
I saw variable examples but I am not able to understand and adopt them.
import pandas as pd
from pyomo.environ import *
from pyomo.opt import SolverFactory
from pyomo.core import Var
infinity = float('inf')
opt = SolverFactory('glpk') # GNU Linear Programming Kit for solving large-scale linear programming (LP), mixed integer programming (MIP), and other
df1 = pd.read_excel("datosPvaD.xlsx")
df2 = pd.read_excel("otrosDatos.xlsx")
#demand = consumption['Consumo (Wh)']
#demand.index += 1
#demand_list = demand.tolist()
data1 = df1.to_dict()
#data2 = df2.to_dict(orient='index')
#data2 = df2.to_dict()
"""
# is the same as otros datos
data2 = {None: {
'pRdImp': {None: 0.35},
'pRdExp': {None: 0.1},
'rend': {None: 0.9},
'CAB': {None: 0.082},
'CABasic': {None: 0.082},
'CAPV': {None: 0.224},
'CI': {None: 0.06849},
'M': {None: 1000},
'dt': {None: 1},
}}
"""
data2 = {'pRdImp': 0.35,
'pRdExp': 0.1,
'rend': 0.9,
'CAB': 0.08,
'CABasic': 0.082,
'CAPV': 0.224,
'CI': 0.06849,
'M': 1000,
'dt': 1
}
#z = {**x, **y}
data = {**data1, **data2}
#from Fotovoltaica_V2_csvread import model # import model
from Fotovoltaica_V1 import model # import model
#instance = model.create_instance('Fotovoltaica_V2.dat')
#instance = model.create_instance(data)
instance = model.create_instance(data1,'Fotovoltaica_V2.dat')
It's hard to tell without seeing your entire model, but the section you have commented out for data2 should work:
data2 = {
None:{
'param':{None:val},
...
}
}
I'm assuming that all of your parameters are not indexed. If they are indexed, then you would need something as follows:
model = AbstractModel()
model.t = Set()
model.thing = Param(t)
input_data = {
None:{
't':{None:[1, 2, 3]},
'thing':{1:100, 2:200, 3:300}
}
}
You would then create a model instance by calling
model.create_instance(input_data)
You can import the data from a csv into python as you normally would with pandas, but then there will be a little rework you need to do to get it in the correct pyomo format
Take a look at this example: https://github.com/Pyomo/pyomo/blob/master/examples/doc/pyomobook/overview-ch/wl_excel.py
I would suggest using a ConcreteModel instead of an AbstractModel when using Pandas to load the data. Instead of creating Param objects the dataframe can be used directly in the Constraint.
I have 2 lists that I want to convert them into a dict with key and values. I managed to do so but there are too many steps so I would like to know if there's a simpler way of achieving this. Basically I would like to create the dict directly in the loop without having the extra steps bellow. I just started working with python and I don't quite understand all the datatypes that it provides.
The jName form can be modified if needed.
jName=["Nose", "Neck", "RShoulder", "RElbow", "RWrist", "LShoulder", "LElbow", "LWrist", "RHip",
"RKnee","RAnkle","LHip", "LKnee", "LAnkle", "REye", "LEye", "REar", "LEar"]
def get_joints(subset, candidate):
joints_per_skeleton = [[] for i in range(len(subset))]
# for each detected skeleton
for n in range(len(subset)):
# for each joint
for i in range(18):
cidx = subset[n][i]
if cidx != -1:
y = candidate[cidx.astype(int), 0]
x = candidate[cidx.astype(int), 1]
joints_per_skeleton[n].append((y, x))
else:
joints_per_skeleton[n].append(None)
return joints_per_skeleton
joints = get_joints(subset,candidate)
print joints
Here is the output of the joints list of list
[[None, (48.0, 52.0), (72.0, 50.0), None, None, (24.0, 55.0), (5.0, 105.0), None, (63.0, 159.0), (57.0, 221.0), (55.0, 281.0), (28.0, 154.0), (23.0, 219.0), (23.0, 285.0), None, (25.0, 17.0), (55.0, 18.0), (30.0, 21.0)]]
Here I defined a function to create the dictionary from the 2 lists
def create_dict(keys, values):
return dict(zip(keys, values))
my_dict = create_dict(jointsName, joints[0])
Here is the result:
{'LAnkle': (23.0, 285.0),
'LEar': (30.0, 21.0),
'LElbow': (5.0, 105.0),
'LEye': (25.0, 17.0),
'LHip': (28.0, 154.0),
'LKnee': (23.0, 219.0),
'LShoulder': (24.0, 55.0),
'LWrist': None,
'Neck': (48.0, 52.0),
'Nose': None,
'RAnkle': (55.0, 281.0),
'REar': (55.0, 18.0),
'RElbow': None,
'REye': None,
'RHip': (63.0, 159.0),
'RKnee': (57.0, 221.0),
'RShoulder': (72.0, 50.0),
'RWrist': None}
I think defaultdict could help you. I made my own example to show that you could predefine the keys and then go through a double for loop and have the values of the dict be lists of potentially different sizes. Please let me know if this answers your question:
from collections import defaultdict
import random
joint_names = ['hip','knee','wrist']
num_skeletons = 10
d = defaultdict(list)
for skeleton in range(num_skeletons):
for joint_name in joint_names:
r1 = random.randint(0,10)
r2 = random.randint(0,10)
if r1 > 4:
d[joint_name].append(r1*r2)
print d
Output:
defaultdict(<type 'list'>, {'hip': [0, 5, 30, 36, 56], 'knee': [35, 50, 10], 'wrist': [27, 5, 15, 64, 30]})
As a note I found it very difficult to read through your code since there were some variables that were defined before the snippet you posted.
I have currently these to utils functions.
The only difference between unique_account_link_generator and unique_order_id is what they filter within qs_exists. It's either .filter(slug=new_id) or .filter(order_id=new_id)
I now wonder is there a way to combine them and then being able to define the filter method when I call the function: unique_id_generator(instance, _filter = "order_id")
import random
import string
def random_string_generator(size=10, chars=string.ascii_lowercase + string.digits):
return ''.join(random.choice(chars) for _ in range(size))
def unique_account_link_generator(instance):
"""
1. Generates random string
2. Check if string unique in database
3. If already exists, generate new string
"""
new_id = random_string_generator()
myClass = instance.__class__
qs_exists = myClass.objects.filter(slug=new_id).exists()
if qs_exists:
return unique_account_link_generator(instance)
return new_id
# How to send field_name via function?
def unique_id_generator(instance):
"""
1. Generates random string
2. Check if string unique in database
3. If already exists, generate new string
"""
new_id = random_string_generator()
myClass = instance.__class__
qs_exists = myClass.objects.filter(order_id=new_id).exists()
if qs_exists:
return unique_id_generator(instance)
return new_id
Not sure I understood the question, as the answer is very simple:
def unique_id_generator(instance, _filter="order_id"):
new_id = random_string_generator()
myClass = instance.__class__
qs_exists = myClass.objects.filter(**{_filter:new_id}).exists()
if qs_exists:
return unique_id_generator(instance, _filter)
return new_id
I want to give you an answer to your question in the comments. Since the comment section doesn't allow much text I would like to attach this as an addition to the accepted answer.
It's actually correct that **{_filter:new_id} will unpack what's inside the _filter parameter
If you call the function with (instance, _filter="order_id")
this part **{_filter:new_id} will look like this **{"order_id":"randomGeneratedCode123"}
Now you have a dictionary with the key "order_id" and the value "randomGeneratedCode123"
You goal is to transform the key "order_id" to a parameter name and the value of the "order_id" key to the value of the parameter order_id
order_id = "randomGeneratedCode123"
As you already said you can unpack a dictionary with the double stars **
After unpacking it, the keys in the dictionary will be your parameter names and the values of the keys the parameter values
Here is a small example for better understanding:
Let's say you have a dictionary and a function
dict = {'a': 1, 'b': 2}
def example(a, b):
print("Variable a is {}, b is {}".format(a, b))
example(**dict)
**dict is converted to:
a = 1, b = 2 so the function will be called with
example(a = 1, b = 2)
It's important that the keys in your dictionary have the same name as your function parameter names
So this wouldn't work:
dict = {'a': 1, 'c': 2}
example(**dict)
because it's "translated " as
example(a = 1, c = 2)
and the function doesn't have a parameter with the name c
An example interview question I have seen come up several times is
Given a mapping of employee_id to boss_id (for direct chain of command
only), return how many employees each boss indirectly manages.
I have found several solutions in SQL but can't find any examples of pythonic approaches.
More detailed problem definition:
Imagine that we have a dict containing the employee_id and the boss_id. In the example below C is manager of A, C is also manager of B, F is manager of C and so on. F is the manager of themselves and therefore the root of the hierarchy (i.e. the CEO). Each employee directly reports to exactly one manager.
emp_boss_map = dict([
("A","C"),
("B","C"),
("C","F"),
("D","E"),
("E","F"),
("F","F")])
Write a function to build a dictionary of the quantity of employees under each employee, not just their direct reports.
Output should be the following for example above:
{A:0,B:0,C:2,D:0,E:1,F:5}
How would one approach this problem in Python?
UPDATE:
Upon #prune 's suggestion I reversed the dict using
newdict = {}
for key, value in emp_boss_map.items():
for string in value:
newdict.setdefault(string, []).append(key)
Which returns
{'C': ['A', 'B'], 'E': ['D'], 'F': ['C', 'E', 'F']}
This is a closure problem on a tree. First and foremost, invert (reverse) the dictionary. From then, it's a straightforward count of nodes in each sub-tree, a recursive algorithm on each individual node: recur on each child, sum their return values, add 1 for each child, and that's the total direct reports for the current node.
Does that get you moving?
full_report = {}
def report_count(node):
global full_report
report_list = newdict[node]
count = len(report_list) # start with quantity of direct reports
# Find number of reports for each of those reports
for report in report_list:
count += report_count(report)
full_report[node] = count # add entry for this node to global count list
return count + len(report_list) # return this count to supervisor
I've left a lot of details open for you, such as finding all of the root nodes (from the original dictionary) and perhaps finding something better than a global variable for the master list.
Does that get you to the final solution?
You can solve this problem by reversing a dictionary and traversing it recursively till the end.
result_count=[]
def get_me_the_children(root):
if reverse_d.get(root,0) == 0:
return
for _ in reverse_d.get(root,0):
if _== root:
continue
else:
get_me_the_children(_)
print(_,end=" ")
result_count.append(_)
if __name__ == "__main__":
input_d = {"A":"A","B":"A","C":"B","D":"B","E":"D","F":"E"}
reverse_d={}
result_count_dict = {}
for key,values in input_d.items():
reverse_d.setdefault(values,[])
reverse_d[values].append(key)
#reverse_d = {'A': ['A', 'B'], 'B': ['C', 'D'], 'D': ['E'], 'E': ['F']}
for key,value in input_d.items():
result_count=[]
print(key,end=": ")
get_me_the_children(key)
result_count_dict[key] = len(result_count)
print("")
print(result_count_dict) #{'A': 5, 'B': 4, 'C': 0, 'D': 2, 'E': 1, 'F': 0}
"""output:
A: C F E D B
B: C F E D
C:
D: F E
E: F
F:
"""
def m1(dict):
new_dict = {}
keys_list = {keys for keys in dict.keys()}
for key, val in dict.items():
if key==val:
continue
if val not in new_dict.keys():
new_dict[val] = 1
else:
new_dict[val] = new_dict[val] + 1
if key in new_dict.keys():
new_dict[val] = new_dict[val] + new_dict[key]
for keys in keys_list:
if keys not in new_dict.keys():
new_dict[keys]=0
print(new_dict)
emp_boss_map={'A': 'C', 'B': 'C', 'C': 'F', 'D': 'E', 'E': 'F', 'F': 'F'}
m1(emp_boss_map)