I am working on an Pyomo optimization script. It is moderately complex with indexed parameters and variables and 8 constraints. If I leave the decision variables with no initial value, I get "ERROR: evaluating expression: No value for uninitialized NumericValue object" errors for constraints involving a decision variable. If I initialize the decision variables to 1 or 0 the solver(glpk) returns Status: optimal and objective function value = 0. All decision variables are also set to 0. I am looking for ideas about how to troubleshoot this. Below is relevant code (w/o data) and the output from the solver and model.display, which has been edited to make it more compact.
#Sets
#clinics
model.C = Set(initialize=list(clinics.keys()))
#clients
model.B = Set(initialize=list(client_blocks.keys()))
#scalar params
model.clt_stf_max = 10
model.tt_max = 45
model.selected_clinic_max = 4
model.stf_max = sum(staff_cap.values()) #total available staff is sum of clinic new staff capacity
model.z_M = 5000 #used in clients at selected clinics constraint
model.e_M = 500 #used in staff at selected clinics constraint
#indexed params
model.cnc_stf_cap = Param(C, initialize=staff_cap)
model.clt_blk = Param(B, initialize=client_blocks)
model.trav_time = Param(T, initialize=trav_time)
#decision vars
#x new staff at each clinic
model.new_stf = Var(model.C, domain=NonNegativeIntegers, initialize=0)
#y new clients at each clinic/block combo
model.new_clt = Var(model.C, model.B, domain=NonNegativeIntegers, initialize=0)
#z client at selected clinics only
model.z = Var(model.C, model.B, domain=Binary, initialize=0)
#e staff at selected clinics only
model.e = Var(model.C, domain=Binary, initialize=0)
#objective function
def o_min_tt_rule(model):
return sum(model.trav_time[c,b]*model.new_clt[c,b] for c in model.C for b in model.B)
model.o_min_tt = Objective(rule=o_min_tt_rule, sense=minimize)
#constraints
# limit new clients at clinic to staff capacity
def limit_clients_to_clinic_staff_cap_rule(model, c):
return sum(model.new_clt[c,b] for c in model.C for b in model.B) <= (model.cnc_stf_cap[c] * model.clt_stf_max)
model.limit_clients_to_clinic_staff_cap = Constraint(model.C, rule=limit_clients_to_clinic_staff_cap_rule)
#total of new clients served in block should not exceed the number new clients in the block
def limit_newclient_block_rule(model, b):
return sum(model.new_clt[c,b] for c in model.C for b in model.B) <= (model.clt_blk[b])
model.limit_newclient_block = Constraint(model.B, rule=limit_newclient_block_rule)
#limit new clients to selected clinics
def client_to_selected_clinic_rule(model, c, b):
return model.new_clt[c,b] <= model.z[c,b] * model.z_M
model.client_to_selected_clinic = Constraint(model.C, model.B, rule=client_to_selected_clinic_rule)
#limit single client travel time to max travel time minutes
def limit_client_travtime_rule(model, c, b):
return (model.trav_time[c,b] * model.z[c,b]) <= model.tt_max
model.limit_client_travtime = Constraint(model.C, model.B, rule=limit_client_travtime_rule)
#limit selected clinics to max number
def limit_selected_clinic_to_max_rule(model):
return summation(model.e) <= model.selected_clinic_max
model.limit_selected_clinic_to_max = Constraint(rule=limit_selected_clinic_to_max_rule,)
#limit new staff to selected clinics
def staff_to_selected_clinic_rule(model, c):
return model.new_stf[c] <= model.e[c] * model.e_M
model.staff_to_selected_clinic = Constraint(model.C, rule=staff_to_selected_clinic_rule)
#limit new staff at clinic to clinic capacity
def limit_staff_to_clnic_cap_rule(model, c):
return model.new_stf[c] <= model.cnc_stf_cap[c]
model.limit_staff_to_clnic_cap = Constraint(model.C, rule=limit_staff_to_clnic_cap_rule)
#limit total new staff to staff max
def limit_tot_staff_rule(model):
return summation(model.new_stf) <= model.stf_max
model.limit_tot_staff = Constraint(rule=limit_tot_staff_rule)
# solve the model
solver = SolverFactory("glpk", tee=True, warmstart=True, symbolic_solver_labels=True)
OUTPUT from solver and model.display()
SOLVER:
Problem:
- Name: unknown
Lower bound: 0.0
Upper bound: 0.0
Number of objectives: 1
Number of constraints: 753
Number of variables: 665
Number of nonzeros: 29213
Sense: minimize
Solver:
- Status: ok
Termination condition: optimal
Statistics:
Branch and bound:
Number of bounded subproblems: 1
Number of created subproblems: 1
Error rc: 0
Time: 0.10507559776306152
Solution:
- number of solutions: 1
number of solutions displayed: 1
- Gap: 0.0
Status: optimal
Message: None
Objective:
o_min_tt:
Value: 0
Variable: No nonzero values
Constraint: No values
MODEL:
Model (Staff_Allocation)
Variables:
new_stf : Size=4, Index=Clinics
Key : Lower : Value : Upper : Fixed : Stale : Domain
FEDCAP Counseling Center - 01004 : 0 : 0 : None : False : False : NonNegativeIntegers
The Salvation Army Belmont Center - 04002 : 0 : 0 : None : False : False : NonNegativeIntegers
VIP Wellness Center - 12000 : 0 : 0 : None : False : False : NonNegativeIntegers
Westchester Center of Excellence - 11006 : 0 : 0 : None : False : False : NonNegativeIntegers
new_clt : Size=392, Index=new_clt_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
('FEDCAP Counseling Center - 01004', 360050019001022) : 0 : 0 : None : False : False : NonNegativeIntegers
z : Size=392, Index=z_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
('FEDCAP Counseling Center - 01004', 360050019001022) : 0 : 0 : 1 : False : False : Binary
e : Size=4, Index=Clinics
Key : Lower : Value : Upper : Fixed : Stale : Domain
FEDCAP Counseling Center - 01004 : 0 : 0 : 1 : False : False : Binary
The Salvation Army Belmont Center - 04002 : 0 : 0 : 1 : False : False : Binary
VIP Wellness Center - 12000 : 0 : 0 : 1 : False : False : Binary
Westchester Center of Excellence - 11006 : 0 : 0 : 1 : False : False : Binary
Objectives:
o_min_tt : Size=1, Index=None, Active=True
Key : Active : Value
None : True : 0
Constraints:
limit_clients_to_clinic_staff_cap : Size=4
Key : Lower : Body : Upper
FEDCAP Counseling Center - 01004 : None : 0 : 200.0
The Salvation Army Belmont Center - 04002 : None : 0 : 150.0
VIP Wellness Center - 12000 : None : 0 : 150.0
Westchester Center of Excellence - 11006 : None : 0 : 250.0
limit_newclient_block : Size=98
Key : Lower : Body : Upper
360050019001022 : None : 0 : 1.0
.... 97 more like this
client_to_selected_clinic : Size=392
Key : Lower : Body : Upper
('FEDCAP Counseling Center - 01004', 360050019001022) : None : 0 : 0.0
.... 391 more like this
limit_client_travtime : Size=392
Key : Lower : Body : Upper
('FEDCAP Counseling Center - 01004', 360050019001022) : None : 0 : 75.0
.... 391 more like this
limit_selected_clinic_to_max : Size=1
Key : Lower : Body : Upper
None : None : 0 : 4.0
staff_to_selected_clinic : Size=4
Key : Lower : Body : Upper
FEDCAP Counseling Center - 01004 : None : 0 : 0.0
The Salvation Army Belmont Center - 04002 : None : 0 : 0.0
VIP Wellness Center - 12000 : None : 0 : 0.0
Westchester Center of Excellence - 11006 : None : 0 : 0.0
limit_staff_to_clnic_cap : Size=4
Key : Lower : Body : Upper
FEDCAP Counseling Center - 01004 : None : 0 : 4.0
The Salvation Army Belmont Center - 04002 : None : 0 : 3.0
VIP Wellness Center - 12000 : None : 0 : 3.0
Westchester Center of Excellence - 11006 : None : 0 : 5.0
limit_tot_staff : Size=1
Key : Lower : Body : Upper
None : None : 0 : 15.0
Related
I am working on a linear optimization problem where I have a set of cities and a set of powerplants. The cities have an electricity demand that needs to be satisfied. However, in the context of my problem, in certain time periods, the cities have no electricity demand (from the power plants because they can produce some of their own). I do not think the specific details are very important so below is my best description of the issue.
The objective function contains the following term:
Term in objective function
I created the appropriate city and month sets and set up my objective function as:
sum(sum(1/model.monthly_demand[c,t]*model.theta[c] for c in model.cities) for t in model.months)
The issue clearly arises when monthly_demand[c,t] = 0 as i get a division by zero error. And I am not sure how to deal with this. Ideally I would like theta[c] to be set to zero in that case but I am unsure how to do this. I tried adding some if/else statements in the sum() function but this is not possible as far as I understand.
I think I can also define a function that is passed into the pyomo objective, so my idea was to try something like an if statement that sets theta[c] to zero when the monthly demand is zero, but this was not successful.
Another idea was to set the demands to something like 0.000001 but I would like this to be a last resort solution because I think it will cause issues.
You should just make a subset wherever convenient of the non-zero elements of demand. I am assuming demand is a parameter which would make sense, but you didn't specify. Then use that subset as the basis of summation. This will avoid including the empty/zero elements.
Note the elements of the objective in the printout of the model.
Code:
import pyomo.environ as pyo
demand_data = { ('LA', 0) : 20,
('LA', 1) : 0,
('SF', 1) : 15,
('NY', 0) : 20,
('NY', 1) : 30}
m = pyo.ConcreteModel('electricity')
m.C = pyo.Set(initialize=['LA', 'SF', 'NY'])
m.T = pyo.Set(initialize=list(range(2)))
# VARS
m.x = pyo.Var(m.C, m.T, domain=pyo.NonNegativeReals)
# PARAMS
m.demand = pyo.Param(m.C, m.T, initialize=demand_data, default=0)
# CONSTRAINTS
# ...
# OBJ
# make a subset of the non-zero demand periods
nonzero_demand_domain = {(c,t) for c in m.C for t in m.T if m.demand[c, t] > 0}
# use that in your objective...
m.obj = pyo.Objective(expr=sum(m.x[c, t] for c, t in nonzero_demand_domain))
m.pprint()
Output:
4 Set Declarations
C : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 3 : {'LA', 'SF', 'NY'}
T : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 2 : {0, 1}
demand_index : Size=1, Index=None, Ordered=True
Key : Dimen : Domain : Size : Members
None : 2 : C*T : 6 : {('LA', 0), ('LA', 1), ('SF', 0), ('SF', 1), ('NY', 0), ('NY', 1)}
x_index : Size=1, Index=None, Ordered=True
Key : Dimen : Domain : Size : Members
None : 2 : C*T : 6 : {('LA', 0), ('LA', 1), ('SF', 0), ('SF', 1), ('NY', 0), ('NY', 1)}
1 Param Declarations
demand : Size=6, Index=demand_index, Domain=Any, Default=0, Mutable=False
Key : Value
('LA', 0) : 20
('LA', 1) : 0
('NY', 0) : 20
('NY', 1) : 30
('SF', 1) : 15
1 Var Declarations
x : Size=6, Index=x_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
('LA', 0) : 0 : None : None : False : True : NonNegativeReals
('LA', 1) : 0 : None : None : False : True : NonNegativeReals
('NY', 0) : 0 : None : None : False : True : NonNegativeReals
('NY', 1) : 0 : None : None : False : True : NonNegativeReals
('SF', 0) : 0 : None : None : False : True : NonNegativeReals
('SF', 1) : 0 : None : None : False : True : NonNegativeReals
1 Objective Declarations
obj : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : minimize : x[NY,0] + x[NY,1] + x[LA,0] + x[SF,1]
7 Declarations: C T x_index x demand_index demand obj
The question is why Pyomo - MindtPy is unable to get an optimal solution with nonlinear exponential function is built in constraints, while being able to get an optimal solution with nonlinear exponential function is built in objective?
The nonlinear optimization problem is
Max Total_Reach_Freq
Constraints are:
Total_Reach_Freq =[1-exp( -2.4208*x[0])] +[1-exp(-1.6278*x[1])]
8000 <= 266*x[0]+494*x[1] <= 12000, 0 <=x[0] <=1755, 0 <=x[1]<=2873, x[0] and x[1] are integers
where exp is the exponential function.
The Python code is below
import pandas as pd
import numpy as np
import pyomo.environ as pe
from pyomo.opt import SolverStatus, TerminationCondition
from pyomo.util.infeasible import log_infeasible_constraints
model = pe.ConcreteModel(name="Pyomo Test Optimization")
########### Define integer variables x[0], x[1]
### such that 0 <= x[0] <= 1755
### 0 <= x[1] <= 2873
cap_list=[1755, 2873]
def cap_rule( model, i):
return (0, cap_list[i])
model.x = pe.Var( [0, 1], within=pe.NonNegativeIntegers, bounds= cap_rule )
C=[-2.4208, -1.6278]
##### Define objective variable Total_Reach_Freq >=0
### Total_Reach_Freq =[1-pe.exp( -2.4208*x[0])] +[1-pe.exp(-1.6278*x[1])]
model.Total_Reach_Freq = pe.Var( within=pe.NonNegativeReals, bounds=(0, None) )
model.Constraint_Define_Objective= pe.Constraint(expr= (1-pe.exp(C[0]*model.x[0])) +(1-pe.exp(C[1]*model.x[1])) == model.Total_Reach_Freq )
model.objective = pe.Objective(expr=model.Total_Reach_Freq, sense= pe.maximize)
#model.objective = pe.Objective(expr=(1-pe.exp(C[0]*model.x[0])) +(1-pe.exp(C[1]*model.x[1])), sense= pe.maximize)
##### Define constraints
#### 8000 <= 266*x[0]+494*x[1] <= 12000
D=[266, 494]
model.Constraint_Budget_lb= pe.Constraint(expr= D[0]*model.x[0]+D[1]*model.x[1] >= 8000 )
model.Constraint_Budget_ub= pe.Constraint(expr= D[0]*model.x[0]+D[1]*model.x[1] <= 12000 )
model.pprint()
results=pe.SolverFactory('mindtpy').solve(model, tee=True)
log_infeasible_constraints(model)
#model.load(results)
#model.solutions.load_from(results)
print()
print("Results=", results)
status= results.solver.termination_condition
print("############################## Status ############################")
print()
print( "results.solver.status =", results.solver.status)
print("results.solver.termination_condition =", results.solver.termination_condition )
print("SolverStatus.warning =", SolverStatus.warning)
print()
print("Solution:")
print()
print( "pe.value(model.objective)=", pe.value(model.objective) )
print( "Decision variable x =", [model.x[0].value, model.x[1].value ] )
print("################################################################")
R_var= [ np.round_( 1-pe.exp( C[0]*model.x[0].value ), decimals=4), np.round_( 1-pe.exp(C[1]*model.x[1].value), decimals=4) ]
print()
print("R_i", R_var)
print()
print("sum (R_i)=", np.round_(sum(R_var), decimals=2))
This problem has optimal solution. However, the output shows a feasible solution
1 Set Declarations
x_index : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 2 : {0, 1}
2 Var Declarations
Total_Reach_Freq : Size=1, Index=None
Key : Lower : Value : Upper : Fixed : Stale : Domain
None : 0 : None : None : False : True : NonNegativeReals
x : Size=2, Index=x_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
0 : 0 : None : 1755 : False : True : NonNegativeIntegers
1 : 0 : None : 2873 : False : True : NonNegativeIntegers
1 Objective Declarations
objective : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : maximize : Total_Reach_Freq
3 Constraint Declarations
Constraint_Budget_lb : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 8000.0 : 266*x[0] + 494*x[1] : +Inf : True
Constraint_Budget_ub : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : -Inf : 266*x[0] + 494*x[1] : 12000.0 : True
Constraint_Define_Objective : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 0.0 : 1 - exp(-2.4208*x[0]) + 1 - exp(-1.6278*x[1]) - Total_Reach_Freq : 0.0 : True
7 Declarations: x_index x Total_Reach_Freq Constraint_Define_Objective objective Constraint_Budget_lb Constraint_Budget_ub
---------------------------------------------------------------------------------------------
Mixed-Integer Nonlinear Decomposition Toolbox in Pyomo (MindtPy)
---------------------------------------------------------------------------------------------
For more information, please visit https://pyomo.readthedocs.io/en/stable/contributed_packages/mindtpy.html
Original model has 3 constraints (1 nonlinear) and 0 disjunctions, with 3 variables, of which 0 are binary, 2 are integer, and 1 are continuous.
rNLP is the initial strategy being used.
===============================================================================================
Iteration | Subproblem Type | Objective Value | Primal Bound | Dual Bound | Gap | Time(s)
- Relaxed NLP 2 -inf 2 nan% 0.05
WARNING: DEPRECATED: The 'ignore_integrality' argument no longer has any
functionality. (deprecated in 6.4.2) (called from
1 MILP 2 -inf 2 nan% 0.17
* 1 Fixed NLP 1 1 2 100.00% 0.21
MILP main problem is infeasible. Problem may have no more feasible binary configurations.
MindtPy exiting due to MILP main problem infeasibility.
===============================================================================================
Primal integral : 0.0000
Dual integral : 0.0000
Primal-dual gap integral : 0.0000
Results=
Problem:
- Name: Pyomo Test Optimization
Lower bound: 1.0
Upper bound: 2.0000000000256444
Number of objectives: 1
Number of constraints: 3
Number of variables: 3
Number of binary variables: 0
Number of integer variables: 2
Number of continuous variables: 1
Number of nonzeros: None
Sense: maximize
Number of disjunctions: 0
Solver:
- Name: MindtPyOA
Status: ok
Message: None
User time: None
System time: None
Wallclock time: None
Termination condition: feasible
Termination message: None
Timing: Call after main solve: 1.1296011507511139e-05
Call after subproblem solve: 1.0070973075926304e-05
OA cut generation: 0.0011551158968359232
fixed subproblem: 0.028742596972733736
initialization: 0.1482729569543153
main loop: 0.07431022403761744
main: 0.0369135340442881
main_timer_start_time: 804870.380321695
total: 0.22866498795337975
Iterations: 2
Num infeasible nlp subproblem: 0
Best solution found time: 0.20502531598322093
Primal integral: 0.0
Dual integral: 0.0
Primal dual gap integral: 0.0
############################## Status ############################
results.solver.status = ok
results.solver.termination_condition = feasible
SolverStatus.warning = warning
Solution:
pe.value(model.objective)= 1.0
Decision variable x = [45.0, 0.0]
################################################################
R_i [1.0, 0.0]
sum (R_i)= 1.0
I eliminated the nonlinear constraint and instead define it in the objective function. Pyomo - Mindtpy is able to get the optimal solution. The Python code is below.
import pandas as pd
import numpy as np
import pyomo.environ as pe
from pyomo.opt import SolverStatus, TerminationCondition
from pyomo.util.infeasible import log_infeasible_constraints
model = pe.ConcreteModel(name="Pyomo Test Optimization")
########### Define integer variables x[0], x[1]
### such that 0 <= x[0] <= 1755
### 0 <= x[1] <= 2873
cap_list=[1755, 2873]
def cap_rule( model, i):
return (0, cap_list[i])
model.x = pe.Var( [0, 1], within=pe.NonNegativeIntegers, bounds= cap_rule )
C=[-2.4208, -1.6278]
##### Define objective variable Total_Reach_Freq >=0
### Total_Reach_Freq =[1-pe.exp( -2.4208*x[0])] +[1-pe.exp(-1.6278*x[1])]
# model.Total_Reach_Freq = pe.Var( within=pe.NonNegativeReals, bounds=(0, None) )
# model.Constraint_Define_Objective= pe.Constraint(expr= (1-pe.exp(C[0]*model.x[0])) +(1-pe.exp(C[1]*model.x[1])) == model.Total_Reach_Freq )
# model.objective = pe.Objective(expr=model.Total_Reach_Freq, sense= pe.maximize)
model.objective = pe.Objective(expr=(1-pe.exp(C[0]*model.x[0])) +(1-pe.exp(C[1]*model.x[1])), sense= pe.maximize)
##### Define constraints
#### 8000 <= 266*x[0]+494*x[1] <= 12000
D=[266, 494]
model.Constraint_Budget_lb= pe.Constraint(expr= D[0]*model.x[0]+D[1]*model.x[1] >= 8000 )
model.Constraint_Budget_ub= pe.Constraint(expr= D[0]*model.x[0]+D[1]*model.x[1] <= 12000 )
model.pprint()
results=pe.SolverFactory('mindtpy').solve(model, tee=True)
log_infeasible_constraints(model)
#model.load(results)
#model.solutions.load_from(results)
print()
print("Results=", results)
status= results.solver.termination_condition
print("############################## Status ############################")
print()
print( "results.solver.status =", results.solver.status)
print("results.solver.termination_condition =", results.solver.termination_condition )
print("SolverStatus.warning =", SolverStatus.warning)
print()
print("Solution:")
print()
print( "pe.value(model.objective)=", pe.value(model.objective) )
print( "Decision variable x =", [model.x[0].value, model.x[1].value ] )
print("################################################################")
R_var= [ np.round_( 1-pe.exp( C[0]*model.x[0].value ), decimals=4), np.round_( 1-pe.exp(C[1]*model.x[1].value), decimals=4) ]
print()
print("R_i", R_var)
print()
print("sum (R_i)=", np.round_(sum(R_var), decimals=2))
The output is below
1 Set Declarations
x_index : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 2 : {0, 1}
1 Var Declarations
x : Size=2, Index=x_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
0 : 0 : None : 1755 : False : True : NonNegativeIntegers
1 : 0 : None : 2873 : False : True : NonNegativeIntegers
1 Objective Declarations
objective : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : maximize : 1 - exp(-2.4208*x[0]) + 1 - exp(-1.6278*x[1])
2 Constraint Declarations
Constraint_Budget_lb : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 8000.0 : 266*x[0] + 494*x[1] : +Inf : True
Constraint_Budget_ub : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : -Inf : 266*x[0] + 494*x[1] : 12000.0 : True
5 Declarations: x_index x objective Constraint_Budget_lb Constraint_Budget_ub
---------------------------------------------------------------------------------------------
Mixed-Integer Nonlinear Decomposition Toolbox in Pyomo (MindtPy)
---------------------------------------------------------------------------------------------
For more information, please visit https://pyomo.readthedocs.io/en/stable/contributed_packages/mindtpy.html
Original model has 2 constraints (0 nonlinear) and 0 disjunctions, with 2 variables, of which 0 are binary, 2 are integer, and 0 are continuous.
Objective is nonlinear. Moving it to constraint set.
rNLP is the initial strategy being used.
===============================================================================================
Iteration | Subproblem Type | Objective Value | Primal Bound | Dual Bound | Gap | Time(s)
- Relaxed NLP 2 -inf 2 nan% 0.06
WARNING: DEPRECATED: The 'ignore_integrality' argument no longer has any
functionality. (deprecated in 6.4.2) ()
1 MILP 2 -inf 2 nan% 0.18
* 1 Fixed NLP 1.99243 1.99243 2 0.38% 0.22
2 MILP 2 1.99243 2 0.38% 0.23
* 2 Fixed NLP 1.99851 1.99851 2 0.07% 0.27
MindtPy exiting on bound convergence. |Primal Bound: 1.998513324083709 - Dual Bound: 2.0| / (1e-10 + |Primal Bound|:1.998513324083709) <= relative tolerance: 0.001
===============================================================================================
Primal integral : 0.0013
Dual integral : 0.0000
Primal-dual gap integral : 0.0013
Results=
Problem:
- Name: Pyomo Test Optimization
Lower bound: 1.998513324083709
Upper bound: 2.0
Number of objectives: 1
Number of constraints: 2
Number of variables: 2
Number of binary variables: 0
Number of integer variables: 2
Number of continuous variables: 0
Number of nonzeros: None
Sense: maximize
Number of disjunctions: 0
Solver:
- Name: MindtPyOA
Status: ok
Message: None
User time: None
System time: None
Wallclock time: None
Termination condition: optimal
Termination message: None
Timing: Call after main solve: 2.035603392869234e-05
Call after subproblem solve: 1.256109680980444e-05
OA cut generation: 0.0016140680527314544
fixed subproblem: 0.06024683394934982
initialization: 0.1535441749729216
main loop: 0.10393345193006098
main: 0.02929913904517889
main_timer_start_time: 805180.471824162
total: 0.27106013789307326
Iterations: 2
Num infeasible nlp subproblem: 0
Best solution found time: 0.2668719069333747
Primal integral: 0.0013113116202245812
Dual integral: 1.0499250925580773e-09
Primal dual gap integral: 0.0013113126701496738
############################## Status ############################
results.solver.status = ok
results.solver.termination_condition = optimal
SolverStatus.warning = warning
Solution:
pe.value(model.objective)= 1.9985133053813313
Decision variable x = [23.0, 4.0]
################################################################
R_i [1.0, 0.9985]
sum (R_i)= 2.0
I have a indexed variable New_UnitsBuilt[p] and this variabele should be integer for the index "GasPowerplant"
but linear for the index "batterystorage".
new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(new_units_built_set, domain=(pyo.NonNegativeIntegers if p="GasPowerplant" else NonNegativeReals)
Please help me how to do this in pyomo.
I am new in pyomo
Best Greetings
Gerhard
There are a couple ways you can accomplish this. For the following, I am assuming that your params.Installable_units = ["GasPowerplant", "batterystorage"]:
If the number of elements in new_units_built_set is small, then you can use a dictionary:
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set,
domain={"GasPowerplant": pyo.NonNegativeIntegers, "batterystorage": pyo.NonNegativeReals})
Or if there are a lot – or there is a simple formula to get the return value – you can use a function (rule):
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
def _new_unitsbuilt_domain(m, p):
return pyo.NonNegativeIntegers if p=="GasPowerplant" else pyo.NonNegativeReals
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set, domain=_new_unitsbuilt_domain)
Or you can just set everything to one value and override later (assuming you are using a ConcreteModel):
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set, domain=pyo.NonNegativeReals)
model.New_UnitsBuilt["GasPowerplant"].domain = pyo.NonNegativeIntegers
All of these will produce:
>>> model.pprint()
1 Set Declarations
new_units_built_set : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 2 : {'GasPowerplant', 'batterystorage'}
1 Var Declarations
New_UnitsBuilt : Size=2, Index=new_units_built_set
Key : Lower : Value : Upper : Fixed : Stale : Domain
GasPowerplant : 0 : None : None : False : True : NonNegativeIntegers
batterystorage : 0 : None : None : False : True : NonNegativeReals
2 Declarations: new_units_built_set New_UnitsBuilt
I am trying to get acc, accel and rx from below text only if accounting value is true.
dataRx: 21916 drx: 1743625
ota: 191791 orx: 74164489
dataDropped: 14 dropped:1134
id: 65535 waitress BE nginxid: 0 kbps: 0.000
accounting: false
drop : 1
rx : 48392 bytes: 483920
id: 65533 waitress BE nginxid: 1 kbps: 0.000
accounting: false
drop : 4
rx : 122914 bytes: 70081939
id: 4232 nginx BE nginxid: 3 kbps: 0.000
accounting: false
drop : 0
rx : 3084 bytes: 94357
id: 10482 server BE nginxid: 4 kbps: 0.000
accounting: false
drop : 0
rx : 15 bytes: 2477
id: 20344 serve BE nginxid: 10 kbps: 62914.560
accounting: true
drop : 2
rx : 2217 bytes: 309637
accel : 482 bytes: 264318
acc :349 bytes: 225181
Below python code gets accounting and accel values using below regex
accounting:\s*((?P<accounting>\S*)[\S\s]*?accel:[\S\s]*?bytes:\s*(?P<accel>\S*)[\S\s]*?)
for match in re.finditer(re_exp, text):
group = match.groupdict()
print group
Output:
{"accounting": false, "accel": 264318}
But, the expected output should be
{"accounting": true, "accel": 264318}
Need help with regex expression. Any help would be greatly appreciated.
Also, is there a regex way to group all the data fields under id?
Thanks
Try this
content = open("acc.txt",'r')
ar = content.read()
import re
getdata = re.findall(r"accounting: (true).+?accel.+?bytes:\s(\d+)",ar,re.S)
print getdata
Try as the follow for group the all data corresponding to their id
content = open("acc.txt",'r')
ar = content.readlines()
arv = []
flag = 0
m = ""
for j in ar:
if("id:"in j):
arv.append(m)
m = ""
flag = 1
if (flag == 1):
m+=j
for j in (arv):
print j
Iterate through lines one by one:
import re
rx = ""
accel = ""
acc = ""
lines = text.split("\n")
for i in range(len(lines)):
line = lines[i]
if line == "accounting: true":
match = re.search(r"rx\s*:\s+(\d*)", lines[i+2])
if match:
rx = match.group(1)
match = re.search(r"accel\s*:\s+(\d*)", lines[i+3])
if match:
accel = match.group(1)
match = re.search(r"acc\s*:\s*(\d*)", lines[i+4])
if match:
acc = match.group(1)
print("'accounting': true, 'rx': {}, 'accel': {}, 'acc': {}".format(rx, accel, acc))
I would like to add columns to a data.table based on a string in another column. This is my data and the approach that I have tried:
Params
1: { clientID : 459; time : 1386868908703; version : 6}
2: { clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}
3: { clientID : 988; time : 1388939739771}
4: { clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}
5: { clientID : 459; time : 1388090530634}
Code to create this table:
DT = data.table(Params=c("{ clientID : 459; time : 1386868908703; version : 6}","{ clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}","{ clientID : 988; time : 1388939739771}","{ clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}","{ clientID : 459; time : 1388090530634}"))
I would like to parse the text in the "Params"-column and create new columns based on the text in it. For example I would like to have a new column named "user" that only holds the number after "user:" in the Params string. The added column should look like this:
Params user
1: { clientID : 459; time : 1386868908703; version : 6} NA
2: { clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001} 459001
3: { clientID : 988; time : 1388939739771} NA
4: { clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001} 459001
5: { clientID : 459; time : 1388090530634} 459001
I created the following function to parse (in this case for the "user"):
myparse <- function(searchterm, s) {
s <-gsub("{","",s, fixed = TRUE)
s <-gsub(" ","",s, fixed = TRUE)
s <-gsub("}","",s, fixed = TRUE)
s <-strsplit(s, '[;:]')
s <-unlist(s)
if (length(s[which(s==searchterm)])>0) {s[which(s==searchterm)+1]} else {NA}
}
Then I use the following function to add a column:
DT <- transform(DT, user = myparse("user", Params))
This works in the case of "time" which is included in all the rows but does not work in the case of "user" which is only included in two of the rows. The following error is returned:
Error in data.table(list(Params = c("{ clientID : 459; time : 1386868908703; version : 6}", :
argument 2 (nrow 2) cannot be recycled without remainder to match longest nrow (5)
How can I address this? Thanks!
Here's a way to use regular expressions for this task:
myparse <- function(searchterm, s) {
res <- rep(NA_character_, length(s)) # NA vector
idx <- grepl(searchterm, s) # index for strings including the search term
pattern <- paste0(".*", searchterm, " : ([^;}]+)[;}].*") # regex pattern
res[idx] <- sub(pattern, "\\1", s[idx]) # extract target string
return(res)
}
You can use this function to add new columns, e.g., for user:
DT[, user := myparse("user", Params)]
The new column contains NA for the rows with no user field:
DT[, user]
# [1] NA "459001" NA "459001" NA
I would use some external parser, for example:
library(yaml)
DT = data.frame(
Params=c("{ clientID : 459; time : 1386868908703; version : 6}","{ clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}","{ clientID : 988; time : 1388939739771}","{ clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}","{ clientID : 459; time : 1388090530634}"),
stringsAsFactors=F
)
conv.to.yaml <- function(x){
gsub('; ','\n',substr(x, 3, nchar(x)-1))
}
tmp <- lapply( DT$Params, function(x) yaml.load(conv.to.yaml(x)) )
then combine the parsed lists into data frame:
unames <- unique( unlist(sapply( tmp, names) ) )
res <- as.data.frame( do.call(rbind, lapply(tmp, function(x)x[unames]) ) )
colnames( res ) <- unames
res
the result is pretty much close to what you have in mind, but you need to think about better handling for the time values:
> res
clientID time version id user
1 459 -405527905 6 NULL NULL
2 459 -405612269 NULL 52a9ea8b534b2b0b5000575f 459001
3 988 1665303163 NULL NULL NULL
4 459 -405626089 NULL 52a9ec00b73cbf0b210057e9 459001
5 459 816094026 NULL NULL NULL