Adding constraints for a known solution causes out of bounds exception - linear-programming

I have a linear optimization goal to Maximize EE+FF, where EE and FF each consist of some C and D.
With code I've written, I can get solver to find:
EE_quantity: 0, FF_quantity: 7
...but I know there to be another solution:
EE_quantity: 1, FF_quantity: 6
In order to validate user input for other valid solutions, I added a constraint for both EE and FF. So I added the EE_quantity == 0, FF_quantity == 7 in the code below, which is a runnable example:
SolverContext c2 = SolverContext.GetContext();
Model m2 = c2.CreateModel();
p.elements = elements_multilevel_productmix();
Decision C_quantity = new Decision(Domain.IntegerNonnegative, "C_quantity");
Decision D_quantity = new Decision(Domain.IntegerNonnegative, "D_quantity");
Decision EE_quantity = new Decision(Domain.IntegerNonnegative, "EE_quantity");
Decision FF_quantity = new Decision(Domain.IntegerNonnegative, "FF_quantity");
m2.AddDecisions(C_quantity, D_quantity, EE_quantity, FF_quantity);
m2.AddConstraints("production",
6 * C_quantity + 4 * D_quantity <= 100,
1 * C_quantity + 2 * D_quantity <= 200,
2 * EE_quantity + 1 * FF_quantity <= C_quantity,
1 * EE_quantity + 2 * FF_quantity <= D_quantity,
EE_quantity == 0,
FF_quantity == 7
);
m2.AddGoal("fixed_EE_FF", GoalKind.Maximize, "EE_quantity + FF_quantity");
Solution sol = c2.Solve(new SimplexDirective());
foreach (var item in sol.Decisions)
{
System.Diagnostics.Debug.WriteLine(
item.Name + ": " + item.GetDouble().ToString()
);
}
It seems that Solver Foundation really doesn't like this specific combination. Using EE_quantity == 1, FF_quantity == 6 is fine, as is using just EE_quantity == 0 or FF_quantity == 7. But using both, AND having one of them being zero, throws an exception:
Index was outside the bounds of the array.
What is going on under the hood, here? And how do I specify that I want to find "all" solutions for a specific problem?

(Note: no new releases of Solver Foundation are forthcoming - it's essentially been dropped by Microsoft.)
The stack trace indicates that this is a bug in the simplex solver's presolve routine. Unfortunately the SimplexDirective does not have a way to disable presolve (unlike InteriorPointDirective). Therefore the way to get around this problem is to specify the fixed variables differently.
Remove the last two constraints that set EE_quantity and FF_quantity, and instead set both the upper and lower bounds to be 0 and 7 respectively when you create the Decision objects. This is equivalent to what you wanted to express, but appears to avoid the MSF bug:
Decision EE_quantity = new Decision(Domain.IntegerRange(0, 0), "EE_quantity");
Decision FF_quantity = new Decision(Domain.IntegerRange(7, 7), "FF_quantity");
The MSF simplex solver, like many mixed integer solvers, only returns the optimal solution. If you want MSF to return all solutions, change to the constraint programming solver (ConstraintProgrammingDirective). If you review the documentation for Solution.GetNext() you should figure out how to do this.
Of course the CP solver is not guaranteed to produce the globally optimal solution immediately. But if you iterate through solutions long enough, you'll get there.

Related

LP / MILP If-Then Statements

I have the following statements for a MILP:
Variables: c (can be 1 or 0), αj (real numbers with 0 <= αj <= 1)
I have a linear inequality system for aj:
∑vj * αj = 0 (with vj = constants)
∑αj = c
and I have the following logic:
If there exists a solution for c = 1, the formulation should be infeasible
If there exists the only one solution c = 0 (each αj must be 0) the formulation should be feasible
I need some more equations or changes so that the logic above holds.
First idea:
When I use an additional constraint c = 1 the MILP finds a solution for c = 1 und no solution for c = 0. This helps to identify if c can be 1 but this flips the feasible solution space since the solver breaks when c = 0 which should be the feasible one. Adding the constraint c = 0 will not help, since it is not enough that c = 0 is one potential solution, it must be the only one valid solution.
Second idea:
When I use the objective function max(c) i can conclude that
IF max(c) = 1 THEN not feasible (or IF max(c) = 0 THEN feasible)
However I don't want to use c in the objective function.
Is there any other possibility to change the formulation so that the logic above holds?

How to interpret decision trees' graph results and find most informative features?

I am using sk-learn python 27 and have output some decision tree feature results. Though I am not sure how to interpret the results. At first, I thought the features are listed from the most informative to least informative (from top to bottom), but examining the \nvalue it suggests otherwise. How do I identify the top 5 most informative features from the outputs or using python lines?
from sklearn import tree
tree.export_graphviz(classifierUsed2, feature_names=dv.get_feature_names(), out_file=treeFileName)
# Output below
digraph Tree {
node [shape=box] ;
0 [label="avg-length <= 3.5\ngini = 0.0063\nsamples = 250000\nvalue = [249210, 790]"] ;
1 [label="name-entity <= 2.5\ngini = 0.5\nsamples = 678\nvalue = [338, 340]"] ;
0 -> 1 [labeldistance=2.5, labelangle=45, headlabel="True"] ;
2 [label="first-name=wm <= 0.5\ngini = 0.4537\nsamples = 483\nvalue = [168, 315]"] ;
1 -> 2 ;
3 [label="name-entity <= 1.5\ngini = 0.4016\nsamples = 435\nvalue = [121, 314]"] ;
2 -> 3 ;
4 [label="substring=ee <= 0.5\ngini = 0.4414\nsamples = 73\nvalue = [49, 24]"] ;
3 -> 4 ;
5 [label="substring=oy <= 0.5\ngini = 0.4027\nsamples = 68\nvalue = [49, 19]"] ;
4 -> 5 ;
6 [label="substring=im <= 0.5\ngini = 0.3589\nsamples = 64\nvalue = [49, 15]"] ;
5 -> 6 ;
7 [label="lastLetter-firstName=w <= 0.5\ngini = 0.316\nsamples = 61\nvalue = [49, 12]"] ;
6 -> 7 ;
8 [label="firstLetter-firstName=w <= 0.5\ngini = 0.2815\nsamples = 59\nvalue = [49, 10]"] ;
7 -> 8 ;
9 [label="substring=sa <= 0.5\ngini = 0.2221\nsamples = 55\nvalue = [48, 7]"] ;
... many many more lines below
In Python you can use DecisionTreeClassifier.feature_importances_, which according to the documentation contains
The feature importances. The higher, the more important the feature.
The importance of a feature is computed as the (normalized) total
reduction of the criterion brought by that feature. It is also known
as the Gini importance [R66].
Simply do a np.argsort on the feature importances and you get a feature ranking (ties are not accounted for).
You can look at the Gini impurity (\ngini in the graphviz output) to get a first idea. Lower is better. However, be aware that you will need a way to combine impurity values if a feature is used in more than one split. Typically, this is done by taking the average information gain (or 'purity gain') over all splits on a given feature. This is done for you if you use feature_importances_.
Edit:
I see the problem goes deeper than I thought. The graphviz thing is merely a graphical representation of the tree. It shows the tree and every split of the tree in detail. This is a representation of the tree, not of the features. Informativeness (or importance) of the features does not really fit into this representation because it accumulates information over multiple nodes of the tree.
The variable classifierUsed2.feature_importances_ contains importance information for every feature. If you get for example [0, 0.2, 0, 0.1, ...] the first feature has an importance of 0, the second feature has an importance of 0.2, the third feature has an importance of 0, the fourth feature an importance of 0.1, and so on.
Let's sort features by their importance (most important first):
rank = np.argsort(classifierUsed2.feature_importances_)[::-1]
Now rank contains the indices of the features, starting with the most important one: [1, 3, 0, 1, ...]
Want to see the five most important features?
print(rank[:5])
This prints the indices. What index corresponds to what feature? That's something you should know yourself because you supposedly constructed the feature matrix. Chances are, that this works:
print(dv.get_feature_names()[rank[:5]])
Or maybe this:
print('\n'.join(dv.get_feature_names()[i] for i in rank[:5]))
As kazemakase already pointed out you can get the most important features using the classifier.feature_importances_:
print(sorted(list(zip(classifierUsed2.feature_importances_, dv.get_feature_names()))))
Just as an addendum, I personally prefer the following printing structure (modified from this question/answer):
# Print Decision rules:
def print_decision_tree(tree, feature_names):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
features = [feature_names[i] for i in tree.tree_.feature]
value = tree.tree_.value
def recurse(left, right, threshold, features, node, indent=""):
if (threshold[node] != -2):
print (indent+"if ( " + features[node] + " <= " + str(threshold[node]) + " ) {")
if left[node] != -1:
recurse (left, right, threshold, features,left[node],indent+" ")
print (indent+"} else {")
if right[node] != -1:
recurse (left, right, threshold, features,right[node],indent+" ")
print (indent+"}")
else:
print (indent+"return " + str(value[node]))
recurse(left, right, threshold, features, 0)
# Use it like this:
print_decision_tree(classifierUsed2, dv.get_feature_names())

Understanding (and forming) the regular expression of this finite automaton

For the above automaton, the regular expression that has been given in my textbook is the following :
a*(a*ba*ba*ba*)*(a+a*ba*ba*ba*)
I am having trouble deriving this...the following is my attempt at it :
aa* + aa*(ba*ba*ba*)* + ba*ba*ba* + ba*ba*ba*(ba*ba*ba*)*
Either I am wrong or I am not being able to simplify it into the form given in the book. Can someone please guide me here, point out the mistake or explain it to me step-by-step?
I'd really really thankful and appreciate that.
Check this out. It presents three good, algorithmic methods for answering questions like these. Learn one of them, or all three of them if you have the time or inclination. State removal is fairly intuitive, although I like Kleene's transitive closure method.
http://krchowdhary.com/toc/dfa-to-reg-exp.pdf
EDIT: Your RE is equivalent to the one provided. here's the reduction of theirs to yours:
0. a*(a*ba*ba*ba*)*(a+a*ba*ba*ba*)
1. = a*(a*ba*ba*ba*)*a + a*(a*ba*ba*ba*)*a*ba*ba*ba*
2. = a*(ba*ba*ba*)*a + a*(ba*ba*ba*)*ba*ba*ba*
3. = a*a + a*(ba*ba*ba*)*a + a*(ba*ba*ba*)*ba*ba*ba*
4. = aa* + a*(ba*ba*ba*)*ba*ba*ba*a + a*(ba*ba*ba*)*ba*ba*ba*
5. = aa* + a*(ba*ba*ba*)*ba*ba*ba*
6. = aa* + aa*(ba*ba*ba*)*ba*ba*ba* + (ba*ba*ba*)*ba*ba*ba*
7. = aa* + aa*(ba*ba*ba*)* + (ba*ba*ba*)*ba*ba*ba*
8. = aa* + aa*(ba*ba*ba*)* + ba*ba*ba* + (ba*ba*ba*)*ba*ba*ba*
9. = aa* + aa*(ba*ba*ba*)* + ba*ba*ba* + ba*ba*ba*(ba*ba*ba*)*
Step 1 is right since r(s+t) = rs + rt.
Step 2 is right since r*(r*sr*)* = r*(sr*)*.
Step 3 is right since r = r + s if L(s) is a subset of L(r).
Step 4 is right since r*r = rr* and rs + rq*s = rs + rqq*s.
Step 5 is right since rs + r = r.
Step 6 is right since r*s = rr*s + s.
Step 7 is right since rs + rqq*s = rs + rq*s.
Step 8 is right since r = r + s if L(s) is a subset of L(r).
Step 9 is right since r*r = rr*.
Please feel free to ask any questions or point out any mistakes I may have made.
EDIT2: If you are interested in these kinds of questions, show some love for the new Computer Science StackExchange by going to this link and committing!!!
http://area51.stackexchange.com/proposals/35636/computer-science-non-programming?referrer=rpnXA1_2BNYzXN85c5ibxQ2
The textbook seems correct. Taking it step by step:
a*(a*
If this part of the regular expression is true (in other words you do actually read in an 'a'), you will move to state 3. Following the rest of the expression:
ba*
will have you in state 2,
ba*
in state 4 and
ba*
will have you back in state 3.
Now, suppose that you did not read in an 'a' during a*(a*, reading the next b will move you to state 2. You then end up in exactly the same situation as previously, and by following the rest a*ba*ba*) you end up back in state 3.
Since you are now back in state 3, the part (a*ba*ba*ba*)* can execute as many times as it wants, as this will simply be the same as our first scenario (where you read in an 'a' during a*(a*).
The second part simply explains the first scenario again, where you have to read at least one 'a', and then the rest is the same.
Hope this helps, let me know if it still does not make sense. Don't know if my explanation is too clear.

Which language understands 'variable a = 0 , 20, ..., 300'?

Which language is smart so that it could understand variable a = 0 , 20, ..., 300 ? so you could easily create arrays with it giving step start var last var (or, better no last variable (a la infinite array)) and not only for numbers (but even complex numbers and custom structures like Sedenion's which you would probably define on your own as a class or whatever...)
Point is, find a language or algorithm usable in a language that can cach the law of how array of variables you've given (or params of that variables) change. And compose using that law a structure from which you would be able to get any variable(s).
To everyone - examples you provide are very helpful for all beginners out there. And at the same time are the basic knowledge required to build such 'Smart Array' class. So thank you wary much for your enthusiastic help.
As JeffSahol noticed
all possible rules might include some
that require evaluation of some/all
existing members to generate the nth
member.
So it is a hard Question. And I think language that would do it 'Naturally' would be great to play\work with, hopefully not only for mathematicians.
Haskell:
Prelude> let a=[0,20..300]
Prelude> a
[0,20,40,60,80,100,120,140,160,180,200,220,240,260,280,300]
btw: infinite lists are possible, too:
Prelude> let a=[0,20..]
Prelude> take 20 a
[0,20,40,60,80,100,120,140,160,180,200,220,240,260,280,300,320,340,360,380]
Excel:
Write 0 in A1
Write 20 in A2
Select A1:2
Drag the corner downwards
MatLab:
a = [0:20:300]
F#:
> let a = [|0..20..300|];;
val a : int [] =
[|0; 20; 40; 60; 80; 100; 120; 140; 160; 180; 200; 220; 240; 260; 280; 300|]
With complex numbers:
let c1 = Complex.Create( 0.0, 0.0)
let c2 = Complex.Create(10.0, 10.0)
let a = [|c1..c2|]
val a : Complex [] =
[|0r+0i; 1r+0i; 2r+0i; 3r+0i; 4r+0i; 5r+0i; 6r+0i; 7r+0i; 8r+0i; 9r+0i; 10r+0i|]
As you can see it increments only the real part.
If the step is a complex number too, it will increment the real part AND the imaginary part, till the last var real part has been reached:
let step = Complex.Create(2.0, 1.0)
let a = [|c1..step..c2|]
val a: Complex [] =
[|0r+0i; 2r+1i; 4r+2i; 6r+3i; 8r+4i; 10r+5i|]
Note that if this behavior doesn't match your needs you still can overload (..) and (.. ..) operators. E.g. you want that it increments the imaginary part instead of the real part:
let (..) (c1:Complex) (c2:Complex) =
seq {
for i in 0..int(c2.i-c1.i) do
yield Complex.Create(c1.r, c1.i + float i)
}
let a = [|c1..c2|]
val a : Complex [] =
[|0r+0i; 0r+1i; 0r+2i; 0r+3i; 0r+4i; 0r+5i; 0r+6i; 0r+7i; 0r+8i; 0r+9i; 0r+10i|]
And PHP:
$a = range(1,300,20);
Wait...
Python:
print range(0, 320, 20)
gives
[0, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280, 300]
Props to the comments (I knew there was a more succinct way :P)
Scala:
scala> val a = 0 to 100 by 20
a: scala.collection.immutable.Range = Range(0, 20, 40, 60, 80, 100)
scala> a foreach println
0
20
40
60
80
100
Infinite Lists:
scala> val b = Stream from 1
b: scala.collection.immutable.Stream[Int] = Stream(1, ?)
scala> b take 5 foreach println
1
2
3
4
5
In python you have
a = xrange(start, stop, step)
(or simply range in python 3)
This gives you an iterator from start to stop. It can be infinite since it is built lazily.
>>> a = xrange(0, 300, 20)
>>> for item in a: print item
...
0
20
40
60
80
100
120
140
160
180
200
220
240
260
280
And C++ too [use FC++ library]:
// List is different from STL list
List<int> integers = enumFrom(1); // Lazy list of all numbers starting from 1
// filter and ptr_to_fun definitions provided by FC++
// The idea is to _filter_ prime numbers in this case
// prime is user provided routine that checks if a number is prime
// So the end result is a list of infinite primes :)
List<int> filtered_nums = filter( ptr_to_fun(&prime), integers );
FC++ lazy list implementation: http://www.cc.gatech.edu/~yannis/fc++/New/new_list_implementation.html
More details: http://www.cc.gatech.edu/~yannis/fc++/
Arpan
Groovy,
assert [ 1, *3..5, 7, *9..<12 ] == [1,3,4,5,7,9,10,11]
The SWYM language, which appears to no longer be online, could infer arithmetic and geometric progressions from a few example items and generate an appropriate list.
I believe the syntax in perl6 is start ... *+increment_value, end
You should instead use math.
- (int) infiniteList: (int)x
{
return (x*20);
}
The "smart" arrays use this format since I seriously doubt Haskel could let you do this:
a[1] = 15
after defining a.
C# for example does implement Enumerable.Range(int start, int count), PHP offers the function range(mixed low, mixed high, number step), ... There are programming languages that are "smart" enough.
Beside that, an infinite array is pretty much useless - it's not infinite at all but all-memory-consuming.
You cannot do this enumerating simply with complex numbers as there is no direct successor or predecessor for a given number. Edit: This does not mean that you cannot compare complex numbers or create an array with a specified step!
I may be misunderstanding the question, but the answers that specify way to code the specific example you gave (counting by 20's) don't really meet the requirement that the array "cache" an arbitrary rule for generating array members...it seems that almost any complete solution would require a custom collection class that allows generation of the members with a delegated function/method, especially since all possible rules might include some that require evaluation of some/all existing members to generate the nth member.
Just about any program language can give you this sequence. The question is what syntax you want to use to express it. For example, in C# you can write:
Enumerable.Range(0, 300).Where(x => (x % 20) == 0)
or
for (int i = 0; i < 300; i += 20) yield return i;
or encapsulated in a class:
new ArithmaticSequence(0, 301, 20);
or in a method in a static class:
Enumerable2.ArithmaticSequence(0, 301, 20);
So, what is your criteria?
Assembly:
Assuming edi contains the address of the desired array:
xor eax, eax
loop_location:
mov [edi], eax
add edi, #4
add eax, #20
cmp eax, #300
jl loop_location
MATLAB
it is not a Programming language itself but its a tool but still u can use it like a programming language.
It is built for such Mathematics operations to easily arrays are a breeze there :)
a = 0:1:20;
creates an array from 0 to 20 with an increment of 1.
instead of the number 1 you can also provide any value/operation for the increment
Php always does things much simpler, and sometimes dangerously simple too :)
Well… Java is the only language I've ever seriously used that couldn't do that (although I believe using a Vector instead of an Array allowed that).

Math question in regards to functions in the form (1) / ( b ^ c )

I've found functions which follow the pattern of 1 / bc produce nice curves which can be coupled with interpolation functions really nicely.
The way I use the function is by treating 'c' as the changing value, i.e. the interpolation value between 0 and 1, while varying b for 'sharpness'. I use it to work out an interpolation value between 0 and 1, so generelly the function I use is as such:
float interpolationvalue = 1 - 1/pow(100,c);
linearinterpolate( val1, val2, interpolationvalue);
Up to this point I've been using a hacked approach to make it 'work' since when interpolation value = 1 the value is very close to but not quite 0.
So I was wondering, is there a function in the form of or one which can reproduce similar curves to the ones produced by 1 / bc where at c = 0 result = 1 and c = 1 result = 0.
Or even C = 0, result = 0 and C = 1 result = 1.
Thanks for any help!
For interpolation the approach offering the most flexibility is using splines, in your case quadratic splines would seem sufficient. The wikipedia page is math heavy, but you can find adapted desciptions on google.
1 - c ^ b with small values for b? Another option would be to use a cubic polynomial and specifying the slope at 0 and 1.
You could use a similar curve of the form A - 1 / b^(c + a), choosing values of A and a to match your constraints. So, for c = 0, result = 1:
1 = A - 1/b^a => A = 1 + 1/b^a
and for c = 1, result = 0:
0 = A - 1/b^(1+a) => A = 1/b^(1+a)
Combining these, we can find a in terms of b:
1 + 1/b^a = 1/b^(1+a)
b^(1+a) + b = 1
b * (b^a - 1) = 1
b^a = 1/b - 1
So:
a = log_b(1/b - 1) = log(1/b - 1) / log(b)
A = 1 + 1/b^a = 1 / (1-b)
In real numbers, the ones that mathematician use, no function of the form you specify is ever going to return 0, division can't do that. (1/x)==0 has no real solutions. In floating point arithmetic, the poor relation of real arithmetic that computers use, you could write 1/(MAX_FP_VALUE^1) which will give you as close to 0 as you are ever going to get (actually, it might give you a NaN or one of the other odd returns that IEEE 754 allows).
And, as I'm sure you've noticed, 1/(b^0) always returns 1 since b^0 is, by definition of 0-th power, always 1.
So, no function with c = 0 will produce a result of 0.
For c = 1, result = 1, set b = 1
But I guess this is only a partial answer, I'm not terribly sure I understand what you are trying to do.
Regards
Mark