Related
I have to find the optimum cost of building links between nodes. In my objective function, I am trying to minimise the cost. The problem can be solved to determine the variable, however the optimal value of my cost is incorrect as I want it to take the absolute value of each cost. How can I modify my codes as I cannot use abs() in the objective function?
cost+=(model.a[i,k]-model.a[j,k])model.cmodel.d[i,j].
This value can be negative if model.a[j,k]=1 or positive if model.a[i,k]=1
from pyomo.environ import *
# Creation of a Concrete Model
model = ConcreteModel()
# Sets
model.i = Set(initialize=[1,2,3,4,5,6,7,8,9,10,11,12,13], doc='Nodes')
model.k = Set(initialize=['Orange','SFR', 'Bouygues'], doc='Companies')
# Parameters
model.c = Param(initialize=25, doc='Cost of transforming an existing link into a backbone link in euro/km')
links={
(1, 2) : 1.8,
(1, 7) : 1,
(1, 13) : 5.4,
(2, 8) : 2.3,
(2, 3) : 1.7,
(2, 5) : 7,
(2, 7) : 2,
(2, 12) : 3,
(3, 4) : 2,
(3, 10) : 6.5,
(4, 5) : 1,
(4, 6) : 2,
(5, 8) : 5,
(5, 10) : 1,
(5, 11) : 1.5,
(6, 11) : 2.1,
(7, 12) : 2,
(8, 9) : 2,
(8, 13) : 0.7,
(9, 10) : 1.1,
(10, 11) : 1,
(12, 13) : 2.5,
}
model.d=Param(model.i, model.i,default=0, initialize=links, doc='distance in 10 km between nodes')
# Variables
model.a = Var(model.i, model.k, within=Binary, doc='Binary variable indicating whether node i belongs to company k (0 if it does not belong and 1 if it belongs)')
#Contraints#
def allocation_rule(model, i):
return sum(model.a[i,k] for k in model.k) == 1
model.allocation = Constraint(model.i, rule=allocation_rule, doc='Each node can only belong to one company')
def minimum_rule(model, k):
return sum(model.a[i,k] for i in model.i) >= 2
model.minimum = Constraint(model.k, rule=minimum_rule, doc='Each company must have at least 2 nodes')
#objective
def totalcost(model):
cost=0
for i in model.i:
for j in model.i:
if model.d[i,j]!=0:
for k in model.k:
cost+=(model.a[i,k]-model.a[j,k])*model.c*model.d[i,j]
return cost
model.z = Objective(rule=totalcost, sense=minimize, doc='Minimize the cost of implementing a backbone connecting the three sub-networks')
def total(model):
return model.cost_postive-model.cost_negative
## Display of the output ##
optimizer = SolverFactory("glpk",executable='/usr/bin/glpsol') #creates an optimizer object that uses the glpk package installed to your usr/bin.
optimizer.solve(model) #tells your optimizer to solve the model object
model.display()
I have tried using the cost+=abs((model.a[i,k]-model.a[j,k])model.cmodel.d[i,j]) but this makes the problem non-linear so it cannot be solved.
edited to introduce a new variable p, and added 2 constraints to p>=(model.a[i,k]-model.a[j,k])model.cmodel.d[i,j]) and
p>=-(model.a[i,k]-model.a[j,k])model.cmodel.d[i,j]). However, it returns with error: ERROR:pyomo.core:Rule failed for Param 'd' with index (1, 2):
from pyomo.environ import *
# Creation of a Concrete Model
model = ConcreteModel()
# Sets
model.i = Set(initialize=[1,2,3,4,5,6,7,8,9,10,11,12,13],
doc='Nodes')
model.i = Set(initialize=['Orange','SFR', 'Bouygues'],
doc='Companies')
# Parameters
model.c = Param(initialize=25, doc='Cost of transforming an
existing link into a backbone link in euro/km')
links={
(1, 2) : 1.8,
(1, 7) : 1,
(2, 3) : 1.7,
(2, 5) : 7,
(2, 7) : 2,
(2, 12) : 3,
(3, 4) : 2,
(3, 10) : 6.5,
(4, 5) : 1,
(4, 6) : 2,
(5, 8) : 5,
(5, 10) : 1,
(5, 11) : 1.5,
(6, 11) : 2.1,
(7, 12) : 2,
(8, 9) : 2,
(8, 13) : 0.7,
(9, 10) : 1.1,
(10, 11) : 1,
(12, 13) : 2.5,
(1, 13) : 5.4,
(2, 8) : 2.3,
}
model.d=Param(model.i, model.i,default=0, initialize=links, doc='distance in 10 km between nodes')
# Variables
model.a = Var(model.i, model.k, within=Binary, doc='Binary variable indicating whether node i belongs to company k (0 if it does not belong and 1 if it belongs)')
model.p = Var(model.i,model.k, within=(0.0,None), doc='Cost of building backbone link p_ij')
#Contraints#
def allocation_rule(model, i):
return sum(model.a[i,k] for k in model.k) == 1
model.allocation = Constraint(model.i, rule=allocation_rule, doc='Each node can only belong to one company')
def minimum_rule(model, k):
return sum(model.a[i,k] for i in model.i) >= 2
model.minimum = Constraint(model.k, rule=minimum_rule, doc='Each company must have at least 2 nodes')
def absolute_rule1(model):
return model.p >=(model.a[i,k]-
model.a[j,k])*model.c*model.d[i,j]
model.absolute1 = Constraint(model.i, rule=absolute_rule1, doc='To take the positive cost')
def absolute_rule2(model):
for i in model.i:
for j in model.i:
if model.d[i,j]!=0:
for k in model.k:
return model.p >=-(model.a[i,k]-
model.a[j,k])*model.c*model.d[i,j]
model.absolute2 = Constraint(model.i, rule=absolute_rule2, doc='To take the positive cost')
#objective
def totalcost(model):
cost=0
for i in model.i:
for j in model.i:
if model.d[i,j]!=0:
for k in model.k:
cost+=model.p
return cost
model.z = Objective(rule=totalcost, sense=minimize, doc='Minimize the cost of implementing a backbone connecting the three sub-networks')
Below is a slightly modified approach.
You could put in the helper variables to get to absolute value, but I think that might lead you a bit astray in your objective, as I mentioned in the comment. Specifically, if you have 3 companies, the best you could do for "ownership" would be 1 company owning it, so as you summed over all three companies, you would get one "zero" cost and two actual costs, which is probably not desired.
I reformulated a bit to something which kinda does the same thing with a couple new variables. Realize there is "upward pressure" in the model for link ownership... cost is reduced (good) if more links are owned, so the variable I put in assesses each link by company and only allows ownership if they own both nodes.
The other new variable indicates whether a link is owned or not, independent of company. I think you could probably do without that, but it adds a little clarity. You could get the same thing (remove the variable, I think) by observing:
build_link >= 1 - sum(own_link)
Also, a reminder... I didn't see in your original code that you were inspecting the solver results. Always, always, always do that to ensure the status is "optimal" or you are looking at junk response.
Code:
from pyomo.environ import *
links={
(1, 2) : 1.8,
(1, 7) : 1,
(1, 13) : 5.4,
(2, 8) : 2.3,
(2, 3) : 1.7,
(2, 5) : 7,
(2, 7) : 2,
(2, 12) : 3,
(3, 4) : 2,
(3, 10) : 6.5,
(4, 5) : 1,
(4, 6) : 2,
(5, 8) : 5,
(5, 10) : 1,
(5, 11) : 1.5,
(6, 11) : 2.1,
(7, 12) : 2,
(8, 9) : 2,
(8, 13) : 0.7,
(9, 10) : 1.1,
(10, 11) : 1,
(12, 13) : 2.5,
}
# Creation of a Concrete Model
model = ConcreteModel()
# Sets
model.i = Set(initialize=[1,2,3,4,5,6,7,8,9,10,11,12,13], doc='Nodes')
model.k = Set(initialize=['Orange','SFR', 'Bouygues'], doc='Companies')
model.links = Set(within=model.i*model.i, initialize=links.keys())
# Parameters
model.c = Param(initialize=25, doc='Cost of transforming an existing link into a backbone link in euro/km')
model.d = Param(model.links, default=0, initialize=links, doc='distance in 10 km between nodes')
# Variables
model.a = Var(model.i, model.k, within=Binary, doc='Binary variable indicating whether node i belongs to company k (0 if it does not belong and 1 if it belongs)')
model.own_link = Var(model.links, model.k, within=Binary, doc='Own the link')
model.build_link = Var(model.links, within=Binary, doc='build link')
#Contraints#
def allocation_rule(model, i):
return sum(model.a[i,k] for k in model.k) == 1
model.allocation = Constraint(model.i, rule=allocation_rule, doc='Each node can only belong to one company')
def minimum_rule(model, k):
return sum(model.a[i,k] for i in model.i) >= 2
model.minimum = Constraint(model.k, rule=minimum_rule, doc='Each company must have at least 2 nodes')
def link_owner(model, k, n1, n2):
return model.own_link[n1, n2, k] <= 0.5 * (model.a[n1, k] + model.a[n2, k])
model.link1 = Constraint(model.k, model.links, rule=link_owner)
# link the "build link" variable to lack of link ownership
def link_build(model, *link):
return model.build_link[link] >= 1 - sum(model.own_link[link, k] for k in model.k)
model.build_constraint = Constraint(model.links, rule=link_build)
# objective
cost = sum(model.build_link[link]*model.c*model.d[link] for link in model.links)
model.z = Objective(expr=cost, sense=minimize, doc='Minimize the cost of implementing a backbone connecting the three sub-networks')
## Display of the output ##
optimizer = SolverFactory("glpk") #creates an optimizer object that uses the glpk package installed to your usr/bin.
result = optimizer.solve(model) #tells your optimizer to solve the model object
print(result)
print('Link Ownership Plan:')
for idx in model.own_link.index_set():
if model.own_link[idx].value: # will be true if it is 1, false if 0
print(idx, model.own_link[idx].value)
print('\nLink Build Plan:')
for idx in model.build_link.index_set():
if model.build_link[idx].value: # will be true if it is 1, false if 0
print(idx, model.build_link[idx].value)
Output:
Problem:
- Name: unknown
Lower bound: 232.5
Upper bound: 232.5
Number of objectives: 1
Number of constraints: 105
Number of variables: 128
Number of nonzeros: 365
Sense: minimize
Solver:
- Status: ok
Termination condition: optimal
Statistics:
Branch and bound:
Number of bounded subproblems: 2183
Number of created subproblems: 2183
Error rc: 0
Time: 0.21333098411560059
Solution:
- number of solutions: 0
number of solutions displayed: 0
Link Ownership Plan:
(1, 2, 'Orange') 1.0
(1, 7, 'Orange') 1.0
(1, 13, 'Orange') 1.0
(2, 8, 'Orange') 1.0
(2, 5, 'Orange') 1.0
(2, 7, 'Orange') 1.0
(2, 12, 'Orange') 1.0
(3, 10, 'SFR') 1.0
(4, 6, 'Bouygues') 1.0
(5, 8, 'Orange') 1.0
(6, 11, 'Bouygues') 1.0
(7, 12, 'Orange') 1.0
(8, 9, 'Orange') 1.0
(8, 13, 'Orange') 1.0
(12, 13, 'Orange') 1.0
Link Build Plan:
(2, 3) 1.0
(3, 4) 1.0
(4, 5) 1.0
(5, 10) 1.0
(5, 11) 1.0
(9, 10) 1.0
(10, 11) 1.0
I got a list1 which has N items in each item it has (x,y,z). For z in list2, I need to create a new list which has items from list1 with only (x,y) this need to be done for list3, list4 too.
list1 = [(1250, 1442, 0), (1280, 1655, 1), (1029, 1680, 2), (624, 1573, 3), (732, 1159, 4), (1530, 1634, 5), (1885, 1628, 6), (2152, 1834, 7), (1252, 2459, 8), (1309, 3023, 9), (1376, 3585, 10), (1571, 2388, 11), (1682, 2952, 12), (1686, 3579, 13), (1184, 1391, 14), (1291, 1382, 15), (1117, 1440, 16), (1361, 1400, 17)]
list2 = [0,1,14,15,16,17]
list3 = [2,3,4,5,6,7,8,11]
list4 = [9,10,12,13]
For example, list5 which is between list1 and list2 looks like
list5 = [(1250, 1442),(1280, 1655),(1184, 1391)......]
Can anyone suggest a fast way to do it? Thank you
Easy enough:
def getXYfromIndex(l, indexes):
"""Returns x,y from bigger list 'l' containing (x,y,z).
Uses only those elements (by index) of 'l' that are in 'indexes'"""
# list comprehension: returns x,y for each index in 'l' that is in 'indexes'
return [(x,y) for x,y,_ in (l[i] for i in indexes)]
list1 = [(1250, 1442, 0), (1280, 1655, 1), (1029, 1680, 2), (624, 1573, 3), (732, 1159, 4),
(1530, 1634, 5), (1885, 1628, 6), (2152, 1834, 7), (1252, 2459, 8),
(1309, 3023, 9), (1376, 3585, 10), (1571, 2388, 11), (1682, 2952, 12),
(1686, 3579, 13), (1184, 1391, 14), (1291, 1382, 15), (1117, 1440, 16),
(1361, 1400, 17)]
list2 = [0,1,14,15,16,17]
list3 = [2,3,4,5,6,7,8,11]
list4 = [9,10,12,13]
print(getXYfromIndex(list1,list2)) # use list5 = getXYfromIndex(list1,list2)
print(getXYfromIndex(list1,list3)) # to work with those (x,y) - I just print them
print(getXYfromIndex(list1,list4))
Output:
[(1250, 1442), (1280, 1655), (1184, 1391), (1291, 1382), (1117, 1440), (1361, 1400)]
[(1029, 1680), (624, 1573), (732, 1159), (1530, 1634), (1885, 1628), (2152, 1834),
(1252, 2459), (1571, 2388)]
[(1309, 3023), (1376, 3585), (1682, 2952), (1686, 3579)]
I want to sort a dict, first by the value and then by the key. The key being a string.
Example,
>>> words = {'fall':4, 'down':4, 'was':3, 'a':3, 'zebra':2, 'bitter':1, 'betty':1}
>>> sorted(words.items(), key=itemgetter(1,0), reverse=True)
[('fall', 4), ('down', 4), ('was', 3), ('a', 3), ('zebra', 2), ('bitter', 1), ('betty', 1)]
As you can see above, the dict is getting sorted by value but not by the key.
Thanks.
Edit: I forgot to point out that I wanted the values to get sorted top to down and the key bottom to top.
This should do the trick. Take advantage of the fact that the values are numbers.
from operator import itemgetter
words = {'fall':4, 'down':4, 'was':3, 'a':3, 'zebra':2, 'bitter':1, 'betty':1}
sorted_words = [v for v in sorted(words.iteritems(), key=lambda(k, v): (-v, k))]
print(sorted_words)
Output:
[('down', 4), ('fall', 4), ('a', 3), ('was', 3), ('zebra', 2), ('betty', 1), ('bitter', 1)]
I am trying to construct a structured array in Python that can be accessed by the names of the columns and rows. Is this possible with the structured array method of numpy?
Example:
My array should have roughly this form:
My_array = A B C
E 1 2 3
F 4 5 6
G 7 8 9
And i want to have the possibility to do the following:
My_array["A"]["E"] = 1
My_array["C"]["F"] = 6
Is it possible to do this in pyhton using structured arrays or is there another type of structure which is more suitable for such a task?
A basic structured array gives you something that can be indexed with one name:
In [276]: dt=np.dtype([('A',int),('B',int),('C',int)])
In [277]: x=np.arange(9).reshape(3,3).view(dtype=dt)
In [278]: x
Out[278]:
array([[(0, 1, 2)],
[(3, 4, 5)],
[(6, 7, 8)]],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
In [279]: x['B'] # index by field name
Out[279]:
array([[1],
[4],
[7]])
In [280]: x[1] # index by row (array element)
Out[280]:
array([(3, 4, 5)],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
In [281]: x['B'][1]
Out[281]: array([4])
In [282]: x.shape # could be reshaped to (3,)
Out[282]: (3, 1)
The view approach produced a 2d array, but with just one column. The usual columns are replaced by dtype fields. It's 2d but with a twist. By using view the data buffer is unchanged; the dtype just provides a different way of accessing those 'columns'. dtype fields are, technically, not a dimension. They don't register in either the .shape or .ndim of the array. Also you can't use x[0,'A'].
recarray does the same thing, but adds the option of accessing fields as attributes, e.g. x.B is the same as x['B'].
rows still have to be accessed by index number.
Another way of constructing a structured array is to defined values as a list of tuples.
In [283]: x1 = np.arange(9).reshape(3,3)
In [284]: x2=np.array([tuple(i) for i in x1],dtype=dt)
In [285]: x2
Out[285]:
array([(0, 1, 2), (3, 4, 5), (6, 7, 8)],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
In [286]: x2.shape
Out[286]: (3,)
ones, zeros, empty also construct basic structured arrays
In [287]: np.ones((3,),dtype=dt)
Out[287]:
array([(1, 1, 1), (1, 1, 1), (1, 1, 1)],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
I can construct an array that is indexed with 2 field names, by nesting dtypes:
In [294]: dt1=np.dtype([('D',int),('E',int),('F',int)])
In [295]: dt2=np.dtype([('A',dt1),('B',dt1),('C',dt1)])
In [296]: y=np.ones((),dtype=dt2)
In [297]: y
Out[297]:
array(((1, 1, 1), (1, 1, 1), (1, 1, 1)),
dtype=[('A', [('D', '<i4'), ('E', '<i4'), ('F', '<i4')]), ('B', [('D', '<i4'), ('E', '<i4'), ('F', '<i4')]), ('C', [('D', '<i4'), ('E', '<i4'), ('F', '<i4')])])
In [298]: y['A']['F']
Out[298]: array(1)
But frankly this is rather convoluted. I haven't even figured out how to set the elements to arange(9) (without iterating over field names).
Structured arrays are most commonly produced by reading csv files with np.genfromtxt (or loadtxt). The result is a named field for each labeled column, and a numbered 'row' for each line in the file.
With a recarray, you can access columns with dot notation or with specific reference to the column name. For rows, they are accessed by row number. I haven't seen them accessed via a row name, for example:
>>> import numpy as np
>>> a = np.arange(1,10,1).reshape(3,3)
>>> dt = np.dtype([('A','int'),('B','int'),('C','int')])
>>> a.dtype = dt
>>> r = a.view(type=np.recarray)
>>> r
rec.array([[(1, 2, 3)],
[(4, 5, 6)],
[(7, 8, 9)]],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
>>> r.A
array([[1],
[4],
[7]])
>>> r['A']
array([[1],
[4],
[7]])
>>> r.A[0]
array([1])
>>> a['A'][0]
array([1])
>>> # now for the row
>>> >>> r[0]
rec.array([(1, 2, 3)],
dtype=[('A', '<i4'), ('B', '<i4'), ('C', '<i4')])
>>>
You can specify the dtype and the type at the same time
>>> a = np.ones((3,3))
>>> b = a.view(dtype= [('A','<f8'), ('B','<f8'),('C', '<f8')], type = np.recarray)
>>> b
rec.array([[(1.0, 1.0, 1.0)],
[(1.0, 1.0, 1.0)],
[(1.0, 1.0, 1.0)]],
dtype=[('A', '<f8'), ('B', '<f8'), ('C', '<f8')])
>>> b.A
array([[ 1.],
[ 1.],
[ 1.]])
>>> b.A[0]
array([ 1.])
I'm working on a directed network problem and trying to compute all valid paths between two points. I need a way to look at paths up to 30 "trips" (represented by an [origin, destination] pair) in length. The full route is then composed of a series of these pairs:
route = [[start, city2], [city2, city3], [city3, city4], [city4, city5], [city5, city6], [city6, city7], [city7, city8], [city8, stop]]
So far my best solution is as follows:
def numRoutes(graph, start, stop, minStops, maxStops):
routes = []
route = [[start, stop]]
if distance(graph, route) != "NO SUCH ROUTE" and len(route) >= minStops and len(route) <= maxStops:
routes.append(route)
if maxStops >= 2:
for city2 in routesFromCity(graph, start):
route = [[start, city2],[city2, stop]]
if distance(graph, route) != "NO SUCH ROUTE" and len(route) >= minStops and len(route) <= maxStops:
routes.append(route)
if maxStops >= 3:
for city2 in routesFromCity(graph, start):
for city3 in routesFromCity(graph, city2):
route = [[start, city2], [city2, city3], [city3, stop]]
if distance(graph, route) != "NO SUCH ROUTE" and len(route) >= minStops and len(route) <= maxStops:
routes.append(route)
if maxStops >= 4:
for city2 in routesFromCity(graph, start):
for city3 in routesFromCity(graph, city2):
for city4 in routesFromCity(graph, city3):
route = [[start, city2], [city2, city3], [city3, city4], [city4, stop]]
if distance(graph, route) != "NO SUCH ROUTE" and len(route) >= minStops and len(route) <= maxStops:
routes.append(route)
if maxStops >= 5:
for city2 in routesFromCity(graph, start):
for city3 in routesFromCity(graph, city2):
for city4 in routesFromCity(graph, city3):
for city5 in routesFromCity(graph, city4):
route = [[start, city2], [city2, city3], [city3, city4], [city4, city5], [city5, stop]]
if distance(graph, route) != "NO SUCH ROUTE" and len(route) >= minStops and len(route) <= maxStops:
routes.append(route)
return routes
Where numRoutes is fed my network graph where numbers represent distances:
[[0, 5, 0, 5, 7], [0, 0, 4, 0, 0], [0, 0, 0, 8, 2], [0, 0, 8, 0, 6], [0, 3, 0, 0, 0]]
a start city, an end city and the parameters for the length of the routes.
distance checks if a route is viable and routesFromCity returns the attached nodes to each fed in city.
I have a feeling there's a far more efficient way to generate all of the routes especially as I move toward many more steps, but I can't seem to get anything else to work.
You could use a recursive function. Your maxStops can be a parameter and each time you call you decrease this by 1. When minStops is 0, you yield a result, When the maxStops is 0 you stop recursing.
Here is a code example:
def routesFromCity(x):
for i in range(2, 10):
yield x * i
def findRoutes(start, stop, minStops, maxStops):
if start == stop:
if minStops <= 0:
yield []
else:
if maxStops > 0:
for nextCity in routesFromCity(start):
for route in findRoutes(nextCity, stop, minStops - 1, maxStops - 1):
yield [(start, nextCity)] + route
for route in findRoutes(1, 12, 2, 5):
print route
Output:
[(1, 2), (2, 4), (4, 12)]
[(1, 2), (2, 6), (6, 12)]
[(1, 2), (2, 12)]
[(1, 3), (3, 6), (6, 12)]
[(1, 3), (3, 12)]
[(1, 4), (4, 12)]
[(1, 6), (6, 12)]