The problem consists of a set of tasks requiring some resource from renewable resource pool.
The goal is to complete all tasks in the given planning horizon, so that
the number of resource used is minimised.
The main constraint is no overlap of tasks for the same resource.
The code below does not scale well if I increase the number of tasks.
There should be a simple optimal solution: to allocate the resource 1 to 15 to all tasks. But the solver seems to struggle to find it when n_tasks is higher than say 1000. I have tried a number of things with the search annotations, but no breakthrough so far.
Here is the model and data:
n_tasks = 3000;
duration = [1 | i in 1..n_tasks];
n_resources = 35;
number_resource_needed = [15 | i in 1..n_tasks];
t_max = 18000;
include "cumulative.mzn";
%-----------------------------------------------------------------------------%
% MODEL PARAMETERS
% Tasks
int: n_tasks; % The number of tasks
set of int: Tasks = 1..n_tasks; % The set of all tasks
array[Tasks] of int : duration ; % The task durations
% Resources
int: n_resources;
set of int: Resources = 1..n_resources;
array[Tasks] of int: number_resource_needed; % The resource requirements
% Maximum duration
int: t_max;
%~~~~~~~~~~~~~~~~~
% MODEL VARIABLES.
%~~~~~~~~~~~~~~~~~
array [Tasks] of var 1..t_max: start; % The start times
array[Tasks, Resources] of var 0..1: resource_allocation; %Selection of resources per task.
%~~~~~~~~~~~~~~~~~
% CONSTRAINTS
%~~~~~~~~~~~~~~~~~
% Number of Resources per Task constraint
constraint forall(t in Tasks) (
sum(r in Resources) (resource_allocation[t, r]) = number_resource_needed[t]
);
% Constraint allocated to only one task at a time
constraint forall(r in Resources)(
cumulative(start, duration, [resource_allocation[t, r] | t in Tasks], 1)
);
var int: objective = sum(r in Resources) (r * max([resource_allocation[t, r] | t in Tasks]));
var int: nb_active_workers = sum(r in Resources) (max([resource_allocation[t, r] | t in Tasks]));
% solve minimize objective;
solve :: seq_search([
int_search(resource_allocation, input_order, indomain_max),
int_search(start, input_order, indomain_min),
])
minimize objective;
output ["objective = \(objective) \n"];
output ["nb_active_workers = \(nb_active_workers) \n"];
I have used Chuffed, and different options.
My results:
n_tasks = 100, optimal found in 34s
n_tasks = 500, optimal found in 3m 43s
n_tasks = 3000, first solution 2m 40s, optimal not found at 4m 00s
I would like to see a first solution faster, and also the optimal faster.
Related
I started working on a problem in the past several days...
A company plans its business in a three month period. It can produce
110 units at a cost of 600 each. The minimum amount it must produce
per month is 15 units if active (but of course, it can choose to be closed
during the month, and produce 0 units). Each month it can subcotract the
prodution of 60 units, at a cost of 660 each. Storing a unit for one month
costs 20$ per unit per month. The marketing department has forcasted
sales of 100, 130 and 150 units for the next three months, respectively.
The goal is to meet the demand each month while minimizing the total
cost.
I deduced that we need to have an objective function of form min[Sum(i=0..3) 600*x1+660*x2+20*x3].
We need to add some constrains on x1>=15, and on x2 0<=x2<=60
Also we will also need another constraint for each month...
For the first one i=1 => x1+x2 = 100 - x3last (x3last is an extra variable that should hold the amount existing in deposit from the previous month), and for i=2 and i=3 same constraints.
I don't have any idea how to write this in pulp, and i would appreciate some help. Thx ^_^
I'd tend to agree with #Erwin that you should focus on formulating the problem as a Linear Program. It is then easy to translate this into code in PULP or one of many other PULP libraries/tools/languages.
As an example of this - lets work through this process for the example problem you have written out in your question.
Decision Variables
The first thing to decide is what you can/should decide. This set of information is called the decision variables. Picking the best/easiest decision variables for your problem comes with practice - the important thing is that once you know the values of the variables you have a unique solution to the problem.
Here I would suggest the following. These assume that the forecasts for demand are perfect. For each month i:
Whether the production line should be open - o[i]
How much to produce in that month - p[i]
How much to hold in storage for next month - s[i]
How much to get made externally - e[i]
Objective Function
The objective in your case is obvious - minimise the total cost. So we can just write this down: sum(i=0...2)[p[i]*600 + s[i]*20 + e[i]*660]
Constraints
Let's lift these directly our of your problem description:
"It can produce 110 units at a cost of 600 each. The minimum amount it must produce per month is 15 units if active (but of course, it can choose to be closed during the month, and produce 0 units)."
p[i] >= o[i]*15
p[i] <= o[i]*110
The first constraint forces the minimum production about to be 15 if the production is open that month (o[i] == 1), if the production is not open this constraint has not effect. The second constraint sets a maximum value on p[i] of 110 if the production is open and a maximum production of 0 if the production is closed that month (o[i] == 0).
"Each month it can subcotract the prodution of 60 units, at a cost of 660 each"
e[i] <= 60
"The marketing department has forcasted sales of 100, 130 and 150 units for the next three months, respectively. The goal is to meet the demand each month while minimizing the total cost." If we declare the sales in each mongth to be sales[i], we can define our "flow constraint" as:
p[i] + e[i] + s[i-1] == s[i] + sales[i]
The way to think of this constraint is inputs on the left, and outputs on the right. Inputs of units are production, external production, and stuff taken out of storage from last month. Outputs are units left/put in storage for next month and sales.
Finally in code:
from pulp import *
all_i = [1,2,3]
all_i_with_0 = [0,1,2,3]
sales = {1:100, 2:130, 3:150}
o = LpVariable.dicts('open', all_i, cat='Binary')
p =LpVariable.dicts('production', all_i, cat='Linear')
s =LpVariable.dicts('stored', all_i_with_0, lowBound=0, cat='Linear')
e =LpVariable.dicts('external', all_i, lowBound=0, cat='Linear')
prob = LpProblem("MinCost", LpMinimize)
prob += lpSum([p[i]*600 + s[i]*20 + e[i]*660 for i in all_i]) # Objective
for i in all_i:
prob += p[i] >= o[i]*15
prob += p[i] <= o[i]*110
prob += e[i] <= 60
prob += p[i] + e[i] + s[i-1] == sales[i] + s[i]
prob += s[0] == 0 # No stock inherited from previous monts
prob.solve()
# The status of the solution
print ("Status:", LpStatus [prob.status])
# Dislay the optimums of each var
for v in prob.variables ():
print (v.name, "=", v.varValue)
# Objective fcn
print ("Obj. Fcn: ", value(prob.objective))
Which returns:
Status: Optimal
external_1 = 0.0
external_2 = 10.0
external_3 = 40.0
open_1 = 1.0
open_2 = 1.0
open_3 = 1.0
production_1 = 110.0
production_2 = 110.0
production_3 = 110.0
stored_0 = 0.0
stored_1 = 10.0
stored_2 = 0.0
stored_3 = 0.0
Obj. Fcn: 231200.0
What I am trying to do is a little addon which would let me know how much time I have spent casting during combat in %,
function()
local spell, _, _, _, _, endTime = UnitCastingInfo("player")
-- getting information from the game itself whether im "Casting"
local inCombat = UnitAffectingCombat("player")
-- getting information form the game if combat is true (1) or not (nil)
local casting = {}
local sum = 0
if inCombat == 1 then
if spell then
table.insert(casting, 1)
else
table.insert(casting, 0)
end
else
for k in pairs (casting) do
casting [k] = nil
end
end
for i=1, #casting, 1 do
sum = sum + casting[i]
end
return( sum / #casting ) end
-- creating a list which adds 1 every frame I am casting and I am in combat,
-- or adds 0 every frame I'm not casting and I'm not in combat.
-- Then I sum all the numbers and divide it by the number of inputs to figure
-- out how much % I have spent "casting".
-- In case the combat condition is false, delete the list
For some reason these numbers don't add up at all, I only see "1" when both conditions are satisfied, or 0 if the combat condition is satisfied.
There might be some better approach I'm sure, but I am kind of new to lua and programming in general.
You say you're new to Lua, so I will attempt to explain in detail what is wrong and how it can be improved, so brace yourself for a long read.
I assume that your function will be called every frame/step/tick/whateveryouwanttocallit of your game. Since you set sum = 0 and casting = {} at the beginning of the function, this will be done every time the function is called. That's why you always get 0 or 1 in the end.
Upvalues to the rescue!
Lua has this nice thing called lexical scoping. I won't go into much detail, but the basic idea is: If a variable is accessible (in scope) when a function is defined, that function remembers that variable, no matter where it is called. For example:
local foo
do
local var = 10
foo = function() return var end
end
print(bar) -- nil
print(foo()) -- 10
You can also assign a new value to the variable and the next time you call the function, it will still have that new value. For example, here's a simple counter function:
local counter
do
count = 0
counter = function() count = count + 1; return count; end
end
print(counter()) -- 1
print(counter()) -- 2
-- etc.
Applying that to your situation, what are the values that need to persist from one call to the next?
Number of ticks spent in combat
Number of ticks spent casting
Define those two values outside of your function, and increment / read / reset them as needed; it will persist between repeated calls to your function.
Things to keep in mind:
You need to reset those counters when the player is no longer casting and/or in combat, or they will just continue where they left off.
casting doesn't need to be a table. Tables are slow compared to integers, even if you reuse them. If you need to count stuff, a number is more than enough. Just make casting = 0 when not in combat and increase it by 1 when in combat.
Thank you for feedback everyone, in the end after your suggestions and some research my code looks like this and works great:
function()
local spell, _, _, _, startTime, endTime, _, _, _ = UnitCastingInfo("player")
local inCombat = UnitAffectingCombat("player")
local inLockdown = InCombatLockdown()
local _, duration, _, _ = GetSpellCooldown("Fireball")
casting = casting or {}
local sum = 0
if inCombat == 1 or inLockdown == 1 then
if spell or duration ~= 0 then
casting[#casting+1] = 1
elseif spell == nil or duration == 0 then
casting[#casting+1] = 0
end
else
local next = next
local k = next(casting)
while k ~= nil do
casting[k] = nil
k = next(casting, k)
end
end
for i=1, #casting, 1 do
sum = sum + casting[i]
end
return(("%.1f"):format( (sum / #casting)*100 ).. "%%") end
what i noticed is there was a problem with reseting the table in the original code:
for k in pairs (casting) do
casting [k] = nil
it seemed like either some zeros stayed there, or the table size didnt "shrink" i dont know.
Maybe intereger would be faster then a table, but honestly i dont see any performance issues even when the table gets ridicolously big (5 min, 60 fps, thats 18k inputs) also for the sake of learning a new language its better to do it harder way in my opinion
Regards
How to increase the offet and limit more than 100 ? At present we are unable to request more than 100 limit. Can you please help me ???
Pagination works like below;
- offset = 0, limit = 100 -> 0-100 devices
- offset = 101, limit = 100 -> 100-200 devices
- offset = 201, limit = 100 -> 200-300 devices
- ....continues....
As you can see above limit can be a positive integer(less than 100). Most of the time limit is constant, and we keep increasing the offset for the next API calls.
For example;
let offset = 0 limit = 100 when starting; you can do offset = offset + limit for the next call.
- https://192.168.1.18:8243/api/device-mgt/v1.0/devices?offset=0&limit=100
- https://192.168.1.18:8243/api/device-mgt/v1.0/devices?offset=101&limit=100
- https://192.168.1.18:8243/api/device-mgt/v1.0/devices?offset=201&limit=100
In spite of the fact that there are online plenty of algorithms and functions for generating unique combinations of any size from a list of unique items, there is none available in case of a list of non-unique items (i.e. list containing repetitions of same value.)
The question is how to generate ON-THE-FLY in a generator function all
the unique combinations from a non-unique list without the
computational expensive need of filtering out duplicates?
I consider combination comboA to be unique if there is no other combination comboB for which sorted lists for both combinations are the same. Let's give an example of code checking for such uniqueness:
comboA = [1,2,2]
comboB = [2,1,2]
print("B is a duplicate of A" if sorted(comboA)==sorted(comboB) else "A is unique compared to B")
In the above given example B is a duplicate of A and the print() prints B is a duplicate of A.
The problem of getting a generator function capable of providing unique combinations on-the-fly in case of a non-unique list is solved here: Getting unique combinations from a non-unique list of items, FASTER?, but the provided generator function needs lookups and requires memory what causes problems in case of a huge amount of combinations.
The in the current version of the answer provided function does the job without any lookups and appears to be the right answer here, BUT ...
The goal behind getting rid of lookups is to speed up the generation of unique combinations in case of a list with duplicates.
I have initially (writing the first version of this question) wrongly assumed that code which doesn't require creation of a set used for lookups needed to assure uniqueness is expected to give an advantage over code needing lookups. It is not the case. At least not always. The code in up to now provided answer does not using lookups, but is taking much more time to generate all the combinations in case of no redundant list or if only a few redundant items are in the list.
Here some timings to illustrate the current situation:
-----------------
k: 6 len(ls): 48
Combos Used Code Time
---------------------------------------------------------
12271512 len(list(combinations(ls,k))) : 2.036 seconds
12271512 len(list(subbags(ls,k))) : 50.540 seconds
12271512 len(list(uniqueCombinations(ls,k))) : 8.174 seconds
12271512 len(set(combinations(sorted(ls),k))): 7.233 seconds
---------------------------------------------------------
12271512 len(list(combinations(ls,k))) : 2.030 seconds
1 len(list(subbags(ls,k))) : 0.001 seconds
1 len(list(uniqueCombinations(ls,k))) : 3.619 seconds
1 len(set(combinations(sorted(ls),k))): 2.592 seconds
Above timings illustrate the two extremes: no duplicates and only duplicates. All other timings are between this two.
My interpretation of the results above is that a pure Python function (not using any C-compiled modules) can be extremely faster, but it can be also much slower depending on how many duplicates are in a list. So there is probably no way around writing C/C++ code for a Python .so extension module providing the required functionality.
Instead of post-processing/filtering your output, you can pre-process your input list. This way, you can avoid generating duplicates in the first place. Pre-processing involves either sorting (or using a collections.Counter on) the input. One possible recursive realization is:
def subbags(bag, k):
a = sorted(bag)
n = len(a)
sub = []
def index_of_next_unique_item(i):
j = i + 1
while j < n and a[j] == a[i]:
j += 1
return j
def combinate(i):
if len(sub) == k:
yield tuple(sub)
elif n - i >= k - len(sub):
sub.append(a[i])
yield from combinate(i + 1)
sub.pop()
yield from combinate(index_of_next_unique_item(i))
yield from combinate(0)
bag = [1, 2, 3, 1, 2, 1]
k = 3
i = -1
print(sorted(bag), k)
print('---')
for i, subbag in enumerate(subbags(bag, k)):
print(subbag)
print('---')
print(i + 1)
Output:
[1, 1, 1, 2, 2, 3] 3
---
(1, 1, 1)
(1, 1, 2)
(1, 1, 3)
(1, 2, 2)
(1, 2, 3)
(2, 2, 3)
---
6
Requires some stack space for the recursion, but this + sorting the input should use substantially less time + memory than generating and discarding repeats.
The current state-of-the-art inspired initially by a 50 than by a 100 reps bounties is at the moment (instead of a Python extension module written entirely in C):
An efficient algorithm and implementation that is better than the obvious (set + combinations) approach in the best (and average) case, and is competitive with it in the worst case.
It seems to be possible to fulfill this requirement using a kind of "fake it before you make it" approach. The current state-of-the-art is that there are two generator function algorithms available for solving the problem of getting unique combinations in case of a non-unique list. The below provided algorithm combines both of them what becomes possible because it seems to exist a threshold value for percentage of unique items in the list which can be used for appropriate switching between the two algorithms. The calculation of the percentage of uniqueness is done with so tiny amount of computation time that it even doesn't clearly show up in the final results due to common variation of the taken timing.
def iterFastUniqueCombos(lstList, comboSize, percUniqueThresh=60):
lstListSorted = sorted(lstList)
lenListSorted = len(lstListSorted)
percUnique = 100.0 - 100.0*(lenListSorted-len(set(lstListSorted)))/lenListSorted
lstComboCandidate = []
setUniqueCombos = set()
def idxNextUnique(idxItemOfList):
idxNextUniqueCandidate = idxItemOfList + 1
while (
idxNextUniqueCandidate < lenListSorted
and
lstListSorted[idxNextUniqueCandidate] == lstListSorted[idxItemOfList]
): # while
idxNextUniqueCandidate += 1
idxNextUnique = idxNextUniqueCandidate
return idxNextUnique
def combinate(idxItemOfList):
if len(lstComboCandidate) == sizeOfCombo:
yield tuple(lstComboCandidate)
elif lenListSorted - idxItemOfList >= sizeOfCombo - len(lstComboCandidate):
lstComboCandidate.append(lstListSorted[idxItemOfList])
yield from combinate(idxItemOfList + 1)
lstComboCandidate.pop()
yield from combinate(idxNextUnique(idxItemOfList))
if percUnique > percUniqueThresh:
from itertools import combinations
allCombos = combinations(lstListSorted, comboSize)
for comboCandidate in allCombos:
if comboCandidate in setUniqueCombos:
continue
yield comboCandidate
setUniqueCombos.add(comboCandidate)
else:
yield from combinate(0)
#:if/else
#:def iterFastUniqueCombos()
The below provided timings show that the above iterFastUniqueCombos() generator function provides a clear advantage
over uniqueCombinations() variant in case the list has less than 60 percent of unique elements and is not worse as
the on (set + combinations) based uniqueCombinations() generator function in the opposite case where it gets much faster than the iterUniqueCombos() one (due to switching between
the (set + combinations) and the (no lookups) variant at 60% threshold for amount of unique elements in the list):
=========== sizeOfCombo: 6 sizeOfList: 48 noOfUniqueInList 1 percUnique 2
Combos: 12271512 print(len(list(combinations(lst,k)))) : 2.04968 seconds.
Combos: 1 print(len(list( iterUniqueCombos(lst,k)))) : 0.00011 seconds.
Combos: 1 print(len(list( iterFastUniqueCombos(lst,k)))) : 0.00008 seconds.
Combos: 1 print(len(list( uniqueCombinations(lst,k)))) : 3.61812 seconds.
========== sizeOfCombo: 6 sizeOfList: 48 noOfUniqueInList 48 percUnique 100
Combos: 12271512 print(len(list(combinations(lst,k)))) : 1.99383 seconds.
Combos: 12271512 print(len(list( iterUniqueCombos(lst,k)))) : 49.72461 seconds.
Combos: 12271512 print(len(list( iterFastUniqueCombos(lst,k)))) : 8.07997 seconds.
Combos: 12271512 print(len(list( uniqueCombinations(lst,k)))) : 8.11974 seconds.
========== sizeOfCombo: 6 sizeOfList: 48 noOfUniqueInList 27 percUnique 56
Combos: 12271512 print(len(list(combinations(lst,k)))) : 2.02774 seconds.
Combos: 534704 print(len(list( iterUniqueCombos(lst,k)))) : 1.60052 seconds.
Combos: 534704 print(len(list( iterFastUniqueCombos(lst,k)))) : 1.62002 seconds.
Combos: 534704 print(len(list( uniqueCombinations(lst,k)))) : 3.41156 seconds.
========== sizeOfCombo: 6 sizeOfList: 48 noOfUniqueInList 31 percUnique 64
Combos: 12271512 print(len(list(combinations(lst,k)))) : 2.03539 seconds.
Combos: 1114062 print(len(list( iterUniqueCombos(lst,k)))) : 3.49330 seconds.
Combos: 1114062 print(len(list( iterFastUniqueCombos(lst,k)))) : 3.64474 seconds.
Combos: 1114062 print(len(list( uniqueCombinations(lst,k)))) : 3.61857 seconds.
I have this program that is supposed to search for perfect numbers.
(X is a perfect number if the sum of all numbers that divide X, divided by 2 is equal to X)
sum/2 = x
Now It has found the first four, which were known in Ancient Greece, so it's not really a anything awesome.
The next one should be 33550336.
I know it is a big number, but the program has been going for about 50 minutes, and still hasn't found 33550336.
Is it because I opened the .txt file where I store all the perfect numbers while the program was running, or is it because I don't have a PC fast enough to run it*, or because I'm using Python?
*NOTE: This same PC factorized 500 000 in 10 minutes (while also running the perfect number program and Google Chrome with 3 YouTube tabs), also using Python.
Here is the code to the program:
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 0
for x in range(1, i+1):
if i%x == 0:
sum += x
if sum / 2 == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
The next one should be 33550336.
Your code (I fixed the indentation so that it does in principle what you want):
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 0
for x in range(1, i+1):
if i%x == 0:
sum += x
if sum / 2 == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
does i divisions to find the divisors of i.
So to find the perfect numbers up to n, it does
2 + 3 + 4 + ... + (n-1) + n = n*(n+1)/2 - 1
divisions in the for loop.
Now, for n = 33550336, that would be
Prelude> 33550336 * (33550336 + 1) `quot` 2 - 1
562812539631615
roughly 5.6 * 1014 divisions.
Assuming your CPU could do 109 divisions per second (it most likely can't, 108 is a better estimate in my experience, but even that is for machine ints in C), that would take about 560,000 seconds. One day has 86400 seconds, so that would be roughly six and a half days (more than two months with the 108 estimate).
Your algorithm is just too slow to reach that in reasonable time.
If you don't want to use number-theory (even perfect numbers have a very simple structure, and if there are any odd perfect numbers, those are necessarily huge), you can still do better by dividing only up to the square root to find the divisors,
i = 2
a = open("perfect.txt", 'w')
a.close()
while True:
sum = 1
root = int(i**0.5)
for x in range(2, root+1):
if i%x == 0:
sum += x + i/x
if i == root*root:
sum -= x # if i is a square, we have counted the square root twice
if sum == i:
a = open("perfect.txt", 'a')
a.write(str(i) + "\n")
a.close()
i += 1
that only needs about 1.3 * 1011 divisions and should find the fifth perfect number in a couple of hours.
Without resorting to the explicit formula for even perfect numbers (2^(p-1) * (2^p - 1) for primes p such that 2^p - 1 is prime), you can speed it up somewhat by finding the prime factorisation of i and computing the divisor sum from that. That will make the test faster for all composite numbers, and much faster for most,
def factorisation(n):
facts = []
multiplicity = 0
while n%2 == 0:
multiplicity += 1
n = n // 2
if multiplicity > 0:
facts.append((2,multiplicity))
d = 3
while d*d <= n:
if n % d == 0:
multiplicity = 0
while n % d == 0:
multiplicity += 1
n = n // d
facts.append((d,multiplicity))
d += 2
if n > 1:
facts.append((n,1))
return facts
def divisorSum(n):
f = factorisation(n)
sum = 1
for (p,e) in f:
sum *= (p**(e+1) - 1)/(p-1)
return sum
def isPerfect(n):
return divisorSum(n) == 2*n
i = 2
count = 0
out = 10000
while count < 5:
if isPerfect(i):
print i
count += 1
if i == out:
print "At",i
out *= 5
i += 1
would take an estimated 40 minutes on my machine.
Not a bad estimate:
$ time python fastperf.py
6
28
496
8128
33550336
real 36m4.595s
user 36m2.001s
sys 0m0.453s
It is very hard to try and deduce why this has happened. I would suggest that you run your program either under a debugger and test several iteration manually to check if the code is really correct (I know you have already calculated 4 numbers but still). Alternatively it would be good to run your program under a python profiler just to see if it hasn't accidentally blocked on a lock or something.
It is possible, but not likely that this is an issue related to you opening the file while it is running. If it was an issue, there would have probably been some error message and/or program close/crash.
I would edit the program to write a log-type output to a file every so often. For example, everytime you have processed a target number that is an even multiple of 1-Million, write (open-append-close) the date-time and current-number and last-success-number to a log file.
You could then Type the file once in a while to measure progress.