Modeling XNOR in Pyomo - linear-programming

I am writing a nurse patient matching algorithm, and I want to incorporate something into the objective function that measures how well the new patient-nurse assignments match the patient-nurse assignments from the previous day. I've introduced a new binary variable model.MATCHES_PREVIOUS which should be equal to the XNOR operation of model.Prev_Assignments and model.Assignments (if both are 0 or both are 1, then model.MATCHES_PREVIOUS is 1, otherwise it is 0).
#Sets
model.PatientIDs = {0, 1, 2}
model.NurseIDs = {a, b}
Indexed parameter (indexed by (patient, nurse))
model.Prev_Assignments = {(0, a): 1, (0, b): 0, (1, a) : 0, (1, b) : 1, (2, a) : 1, (2, b) : 0}
Indexed variable (indexed by (patient, nurse))
model.Assignments = {(0, a): 1, (0, b): 0, (1, a) : 0, (1, b) : 1, (2, a) : 0, (2, b) : 1}
model.Matches_Previous = {(0, a): 1, (0, b): 1, (1, a) : 1, (1, b) : 1, (2, a) : 0, (2, b) : 0}
Currently, I'm trying to implement this with the below constraint, which I thought was the proper way to translate XNOR into a linear expression:
def matches_previous(self, patient, nurse):
return model.Matches_Previous[patient, nurse] == (model.Assignments[patient, nurse] * model.Prev_Assignments[patient, nurse]) + (1 - model.Assignments[patient, nurse]) * (1 - model.Prev_Assignments[patient, nurse])
In the objective function, I include (-model.Matches_Previous) along with the other components (since I am minimizing the objective, this would maximize model.Matches_Previous).
However, this isn't giving the behavior I want. Note that there are other aspects of the model which would motivate it to produce assignments that are different from the previous assignments (e.g. changing patient workloads), but I want it to match the previous assignments as best as possible.
Any idea how to implement this better? I looked into Pyomo.GDP and LogicalConstriants, but I haven't been able to get this to work and the documentation is lacking for these modeling extensions.

We always can write
z = x xnor y
as
z >= 1-x-y
z <= 1-x+y
z <= 1+x-y
z >= x+y-1
where x,y,x are binary variables. This is rigorous (derivation is here) and tight (we can even relax z to be continuous between 0 and 1).
Sometimes we can drop the <= or >= inequalities because of how the objective (or constraint) works.

So here is a strategy you might try. Note your attempt above makes the problem non-linear by multiplying variables, which is probably NOT desired.
You can code the XNOR variable in linear fashion, assuming there is some "pressure" on it... Let me explain.
In pseudocode:
Let X = binary variable
Let X_prev = other binary variable
Let P = binary variable {1: if X, X_prev are not equal, else 0}
So, we intend to make P a penalty that can be indexed (optionally) and summed up, then introduce 2 constraints
P >= X - X_prev
P >= X_prev - X
And in your objective function, use P to induce a penalty not a reward and it should work... something like:
Obj = <some other components> + p_weight * sum(P) ; minimized

Related

Tiles Stacking Problem to build a stable stack

I am studying Dynamic Programming on GeeksForGeeks and have a problem with Tiles Stacking Problem and the way it is solved
A stable tower of height n is a tower consisting of exactly n tiles of unit height stacked vertically in such a way, that no bigger tile is placed on a smaller tile. An example is shown below :
We have infinite number of tiles of sizes 1, 2, …, m. The task is calculate the number of different stable tower of height n that can be built from these tiles, with a restriction that you can use at most k tiles of each size in the tower.
Note: Two tower of height n are different if and only if there exists a height h (1 <= h <= n), such that the towers have tiles of different sizes at height h.
For example:
Input : n = 3, m = 3, k = 1.
Output : 1
Possible sequences: { 1, 2, 3}.
Hence answer is 1.
Input : n = 3, m = 3, k = 2.
Output : 7
{1, 1, 2}, {1, 1, 3}, {1, 2, 2},
{1, 2, 3}, {1, 3, 3}, {2, 2, 3},
{2, 3, 3}.
The way to solve is to count number of decreasing sequences of length n using numbers from 1 to m where every number can be used at most k times. We can recursively compute count for n using count for n-1.
Declare a 2D array dp[][], where each state dp[i][j] denotes the number of decreasing sequences of length i using numbers from j to m. We need to take care of the fact that a number can be used a most k times. This can be done by considering 1 to k occurrences of a number. Hence our recurrence relation becomes:
Also, we can use the fact that for a fixed j we are using the consecutive values of previous k values of i. Hence, we can maintain a prefix sum array for each state. Now we have got rid of the k factor for each state.
I have read this algorithm for many times but I don't understand it and how to prove the accuracy of it. I have tried to find the guide on the internet but only its variations. Please help me to explain it.
Observe that the largest size tile (m) can appear only at the bottom.
Its appearances are consecutive
Your recurrence becomes:
T(n,m,k) = SIGMA_{i=0,...,k} T(n-i,m-1,k)
Then you have to define the base cases of the recurrence:
T(n,m,1) = // can you tell what this is?
T(n,1,k) = // can you tell what this is?
T(1,m,k) = m // this is easy
We can prove it by forming a logical recurrence:
(A) If the maximum stack height, given m and k is smaller than n, we cannot create any stack.
(B) If only one tile is allowed, we can choose m different sizes for that tile.
(C) If only one size is allowed, if k is greater than or equal to n, we can construct one stack of n tiles of size 1; otherwise, zero stacks.
(D) For each possible count, x, of tiles of size m stacked, we have one way that is multiplied by the number of ways to stack (n - x) tiles, using sizes of at most (m - 1) since we used m.
To convert the recurrence to bottom-up dynamic programming, we initialise the matrix using the base cases of the recurrence, and fill in subsequent entries using its general-case logical branch.
Here's a demonstration of the recurrence in JavaScript (sorry I'm not versed in C++ but the first function, f, which calculates just the count, should be very easy to convert):
// Returns the count
function f(n, m, k){
if (n > m * k)
return 0;
if (n == 1)
return m;
if (m == 1)
return n <= k ? 1 : 0;
let result = 0;
for (let x=0; x<=k; x++)
result += f(n - x, m - 1, k);
return result;
}
// Returns the sequences
function g(n, m, k){
if (n > m * k)
return [];
if (n == 1)
return new Array(m).fill(0).map((_, i) => [i + 1]);
if (m == 1)
return n <= k ? [new Array(n).fill(1)] : [];
let result = [];
for (let x=0; x<=k; x++){
const pfx = new Array(x).fill(m);
const prev = g(n - x, m - 1, k);
for (let s of prev)
result.push(pfx.concat(s));
}
return result;
}
var inputs = [
[3, 3, 1],
[3, 3, 2],
[1, 2, 2]
];
for (let args of inputs){
console.log('' + args);
console.log(f(...args));
console.log(JSON.stringify(g(...args)));
console.log('');
}

Numpy - row-wise outer product of two matrices

I have two numpy arrays: A of shape (b, i) and B of shape (b, o). I would like to compute an array R of shape (b, i, o) where every line l of R contains the outer product of the row l of A and the row l of B. So far what i have is:
import numpy as np
A = np.ones((10, 2))
B = np.ones((10, 6))
R = np.asarray([np.outer(a, b) for a, b in zip(A, B)])
assert R.shape == (10, 2, 6)
I think this method is too slow, because of the zip and the final transformation into a numpy array.
Is there a more efficient way to do it ?
That is possible with numpy.matmul, which can do multiplication of "matrix stacks". In this case we want to multiply a stack of column vectors with a stack of row vectors. First bring matrix A to shape (b, i, 1) and B to shape (b, 1, o). Then use matmul to perform b times the outer product:
import numpy as np
i, b, o = 3, 4, 5
A = np.ones((b, i))
B = np.ones((b, o))
print(np.matmul(A[:, :, np.newaxis], B[:, np.newaxis, :]).shape) # (4, 3, 5)
An alternative could be to use numpy.einsum, which can directly represent your index notation:
np.einsum('bi,bo->bio', A, B)
Why not simply
A[:, :, None] * B[:, None, :]
Depending on your convention and your dtype, you might need to throw in another np.conj somewhere. Note that np.newaxis is simply None

Advanced comparison of lists

I am currently making a program for a school project which is supposed to help farmers overfertilize less (this is kind of a big deal in Denmark). The way the program is supposed to work is that you enter some information about your fields(content of NPK, field size, type of dirt and other things), and then i'll be able to compare their field's content of nutrition to the recommended amount. Then I can create the theoretical ideal composition of fertilizer for this field.
This much I have been able to do, but here is the hard part.
I have a long list of fertilizers that are available in Denmark, and I want my program to compare 10 of them to my theoretical ideal composition, and then automatically pick the one that fits best.
I literally have no idea how to do this!
The way I format my fertilizer compositions is in lists like this
>>>print(idealfertilizercomp)
[43.15177154944473, 3.9661554732945534, 43.62771020624008, 4.230565838180857, 5.023796932839768]
Each number represent one element in percent. An example could be the first number, 43.15177154944473, which is the amount of potassium I want in my fertilizer in percent.
TL;DR:
How do I make a program or function that can compare a one list of integers to a handfull other lists of integers, and then pick the one that fits best?
So, while i had dinner i actually came up with a way to compare multiple lists in proportion to another:
def numeric(x):
if x >= 0:
return x
else:
return -x
def comparelists(x,y):
z1 = numeric(x[0]-y[0])
z2 = numeric(x[1]-y[1])
z3 = numeric(x[2]-y[2])
z4 = numeric(x[3]-y[3])
z5 = numeric(x[4]-y[4])
zt = z1+z2+z3+z4+z5
return zt
def compare2inproportion(x, y, z):
n1 = (comparelists(x, y))
n2 = (comparelists(x, z))
print(n1)
print(n2)
if n1 < n2:
print(y, "is closer to", x, "than", z)
elif n1 > n2:
print(z, "is closer to", x, "than", y)
else:
print("Both", y, "and", z, "are equally close to", x)
idealfertilizer = [1, 2, 3, 4, 5]
fertilizer1 = [2, 3, 4, 5, 6]
fertilizer2 = [5, 4, 3, 2, 1]
compare2inproportion(idealfertilizer, fertilizer1, fertilizer2)
This is just a basic version that compares two lists, but its really easy to expand upon. The output looks like this:
[2, 3, 4, 5, 6] is closer to [1, 2, 3, 4, 5] than [5, 4, 3, 2, 1]
Sorry for taking your time, and thanks for the help.

Mincemeat: supply additional parameters to map and reduce functions with closures

I would like to try out the Mincemeat map/reduce Python application for matrix multiplication. I am using Python 2.7. I found several web pages that describe how to do matrix multiplication using Hadoop in Java, and I have been referring to this one http://importantfish.com/one-step-matrix-multiplication-with-hadoop/ both because it is simple and because the pseudocode that it displays is very close to Python code already.
I noticed in the Java code that is also included that the matrix dimensions are supplied to the map and reduce functions via an additional argument of type Context. Mincemeat doesn't provide such a thing, but I got a suggestion that I could provide these values to my map and reduce functions using closures. The map and reduce functions I wrote look like this:
def make_map_fn(num_rows_result, num_cols_result):
m = num_rows_result
p = num_cols_result
def map_fn(key, value):
# value is ('A', i, j, a_ij) or ('B', j, k, b_jk)
if value[0] == 'A':
i = value[1]
j = value[2]
a_ij = value[3]
for k in xrange(1, p):
yield ((i, k), ('A', j, a_ij))
else:
j = value[1]
k = value[2]
b_jk = value[3]
for i in xrange(1, m):
yield ((i, k), ('B', j, b_jk))
return map_fn
def make_reduce_fn(inner_dim):
n = inner_dim
def reduce_fn(key, values):
# key is (i, k)
# values is a list of ('A', j, a_ij) and ('B', j, b_jk)
hash_A = {j: a_ij for (x, j, a_ij) in values if x == 'A'}
hash_B = {j: b_jk for (x, j, b_jk) in values if x == 'B'}
result = 0
for j in xrange(1, n):
result += hash_A[j] * hash_B[j]
return (key, result)
return reduce_fn
Then I assign them to Mincemeat like this:
s = mincemeat.Server()
s.mapfn = make_map_fn(num_rows_A, num_cols_B)
s.reducefn = make_reduce_fn(num_cols_A)
When I run this in Mincemeat, I get this error message:
error: uncaptured python exception, closing channel <__main__.Client connected at 0x2ada4d0>
(<type 'exceptions.TypeError'>:arg 5 (closure) must be tuple
[/usr/lib/python2.7/asyncore.py|read|83]
[/usr/lib/python2.7/asyncore.py|handle_read_event|444]
[/usr/lib/python2.7/asynchat.py|handle_read|140]
[/usr/local/lib/python2.7/dist-packages/mincemeat.py|found_terminator|96]
[/usr/local/lib/python2.7/dist-packages/mincemeat.py|process_command|194]
[/usr/local/lib/python2.7/dist-packages/mincemeat.py|set_mapfn|159])
I searched around on the net with search terms like |python closure must be tuple| and the things that I found seemed to be dealing with cases where someone is trying to construct a function using lambda or function() and need to make sure they didn't omit certain things when defining them as closures. In my case, the map_fn and reduce_fn values returned by make_map_fn and make_reduce_fn look like valid function objects, their func_closure values are tuples of cells containing the array dimensions that I want to supply, but something is still missing. What form do I need to pass these functions in to be usable by Mincemeat?
I hate to be the bearer of bad news, but this is just the result of a few off-by-one errors in your code, plus two errors in the input file provided by the site you linked. It is unrelated to your usage of a closure, misleading error messages notwithstanding.
Off-by-one errors
Notice that the innermost loops in the pseudocode look like this:
for k = 1 to p:
for i = 1 to m:
for j = 1 to n:
In pseudocode, this typically indicates that the endpoint is included, i.e. for k = 1 to p means k = 1, 2, ..., p-1, p. On the other hand, the corresponding loops in your code look like this:
for k in xrange(1, p):
for i in xrange(1, m):
for j in xrange(1, n):
And of course, xrange(1, p) yields 1, 2, ..., p-2, p-1. Assuming you indexed the matrices from 0 (as they did on the site you linked), all your xranges should start at 0 (e.g. xrange(0, p)), as their equivalents in the Java code do (for (int k = 0; k < p; k++)). This fixes one of your problems.
Input file errors
In case you didn't catch this, the input file for A and B that the site provides is incorrect - they forgot the (0,0) entries of both matrices. In particular, you should add a line to the beginning of the form A,0,0,0.0, and a line between 9 and 10 of the form B,0,0,0.0. (I guess where exactly you put it doesn't matter, but for consistency, you may as well put them where they naturally fit.)
Once I correct these two errors, mincemeat gives me the result we expect (formatted):
{(0, 1): ((0, 1), 100.0),
(1, 2): ((1, 2), 310.0),
(0, 0): ((0, 0), 90.0),
(0, 2): ((0, 2), 110.0),
(1, 0): ((1, 0), 240.0),
(1, 1): ((1, 1), 275.0)}
I haven't figured out exactly what's going on with the error message, but I think it boils down to the fact that the incorrect loop indices in the map function are resulting in garbage data being passed to the reduce nodes, which is why the error mentions the reduce function.
Basically, what happens is that hash_A and hash_B in the reduce function sometimes don't have the same keys, so when you try to multiply hash_A[j] * hash_B[j], you'll get a KeyError because j is not a key of one or the other, and this gets caught somewhere upstream and rethrown as a TypeError instead.

How do I dereference in python? (Image Processing with openCV)

I've been looking all over the internet for a simple thinning algorithm and I stumbled across this: Thinning algorithm The problem is, I do not have too much experience with the dereference operator. Also, my project is in python which has a different way of handling this situation. So I have a few questions
1: What is this bit of code doing?
void myThinningInit (CvMat ** kpw, CvMat ** kpb)
{
/ / Kernel for cvFilter2D
/ / The algorithm kpw kernel binary image and it has become a matching white, black,
/ / Convolution is divided into two sets of binary image was inverted kpb kernel, then take the AND
for (int i = 0; i <8; i + +) {
* (Kpw + i) = cvCreateMat (3, 3, CV_8UC1);
* (Kpb + i) = cvCreateMat (3, 3, CV_8UC1);
cvSet (* (kpw + i), cvRealScalar (0), NULL);
cvSet (* (kpb + i), cvRealScalar (0), NULL);
}.....
And 2: How can I translate this kernels creation into python?
He ends up making 8 kernels but I have no idea what their matrix form looks like.
I don't understand what "* (kpw + i)" or "* (kpb + i)" does in the grand scheme of the program.
3) Can I just make the kernels and store them in a list? If so, how could I do that?
UPDATE:
k = [1, 2, 3, 5, 6, 7, 8]
kpw = []
kpb = []
for i in k:
kpw.append [i] = cv.CreateMat (3, 3, cv.CV_8UC1)
kpb.append [i] = cv.CreateMat (3, 3, cv.CV_8UC1)
cv.cvSet (kpw [i], cv.RealScalar (0), cv.NULL)
cv.cvSet (kpb [i], cv.RealScalar (0), cv.NULL)
At first I didn't just had kpw [i] and it was throwing me an error. After a quick google search I found that you needed to index the array first and the way they did that was through append. I tried this bit of code in order to get 8 base kernels of 3x3 in size but I received this error:
Traceback (most recent call last):
File "/home/krtzer/Documents/python_scripts/thinning.py", line 14, in
kpw.append [i] = cv.CreateMat (3, 3, cv.CV_8UC1)
TypeError: 'builtin_function_or_method' object does not support item assignment
Does this mean I cannot have matrices in lists?
That dereference is just creating a Matrix, without initialising its data. The data is manually set to zero by those lines like cvSet (* (kpw + i), cvRealScalar (0), NULL).
In python, you can just do the same thing in one hit with numpy.zeros and then use cv.fromarray. Alternatively, use x = cv.CreateMat(3, 3, cv.CV_8UC1) and then cv.set(x, 0.).
Edit - made a (pretty big) mistake in this answer, will explain
Looks like an array of CvMats in both kpw and kpb.
Suppose I made a list of arrays kpw = [] in Python.
The *(kpw + i) = ... is just like saying kpw[i] = ....
Looks like the other code initialising the list of kernels to 3x3 matrices of 0, so you could do:
# make a list of 8 3x3 matrices of 0.
kpw = []
for i in xrange(8):
kpw.append(np.zeros((3,3)))
kpb.append(np.zeros((3,3)))
Note: I previously had:
kpw = [np.zeros((3,3))] * 8
kpb = [np.zeros((3,3))] * 8
which is wrong ! It produces 8 references to the same matrix within kpw, and so modifying kpw[0] will also modify all the other kpw[i]!
Then the cvSet2D(*(kpb+0), 0, 0, cvRealScalar(0)); can be translated to :
kpb[0][0,0] = 0
Because *(kpb+0) grabs the matrix in kpg[0], the 0,0 means element 0,0 of the matrix, and 0 is the value.
So: every time you see *(kpb+i) just substitute kpb[i] and you should be find translating that code.
I made a new one in python. Thinning(Python)