Related
I'm trying to use Z3 (with C++ API) to check if lots of variable configurations satisfy my constraints, but I'm having big performance issues.
I'm looking for advice about which logic or parameter setting I might be able to use to improve the runtime, or hints about how I could try and feed the problem to Z3 in a different way.
Short description of what I'm doing and how I'm doing it:
//_______________Pseudocode and example_______________
context ctx()
solver s(ctx)
// All my variables are finite domain, maybe some 20 values at most, but usually less.
// They can only be ints, bools, or enums.
// There are not that many variables, maybe 10 or 20 for now.
//
// Since I need to be able to solve constraints of the type (e == f), where
// e and f are two different enum variables, all my
// enum types are actually contained in only one enumeration_sort(), populated
// with all the different values.
sort enum_sort = {"green", "red", "yellow", "blue", "null"}
expr x = ctx.int_const("x")
expr y = ctx.int_const("y")
expr b = ctx.bool_const("b")
expr e = ctx.constant("e", enum_sort)
expr f = ctx.constant("f", enum_sort)
// now I assert the finite domains, for each variable
// enum_value(s) is a helper function, that returns the matching enum expression
//
// Let's say that these are the domains:
//
// int x is from {1, 3, 4, 7, 8}
// int y is from {1, 2, 3, 4}
// bool b is from {0, 1}
// enum e is from {"green", "red", "yellow"}
// enum f is from {"red", "blue", "null"}
s.add(x == 1 || x == 3 || x == 3 || x == 7 || x == 8)
s.add(y == 1 || y == 2 || y == 3 || y == 4)
s.add(b == 0 || b == 1)
s.add(e == enum_value("green") || e == enum_value("red") || enum_value("yellow"))
s.add(f == enum_value("red") || f == enum_value("blue") || enum_value("null"))
// now I add in my constraints. There are also about 10 or 20 of them,
// and each one is pretty short
s.add(b => (x + y >= 5))
s.add((x > 1) => (e != f))
s.add((y == 4 && x == 1) || b)
// setup of the solver is now done. Here I start to query different combinations
// of values, and ask the solver if they are "sat" or "unsat"
// some values are left empty, because I don't care about them
expr_vector vec1 = {x == 1, y == 3, b == 1, e == "red"}
print(s.check(vec1))
expr_vector vec2 = {x == 4, e == "green", f == "null"}
print(s.check(vec2))
....
// I want to answer many such queries.
Of course, in my case this isn't hardcoded, but I read and parse the constraints, variables and their domains from files, then feed the info to Z3.
But it's slow.
Even for something like ten thousand queries, my program is already running over 10s. All of this is inside s.check(). Is it possible to make it run faster?
Hopefully it is, because what I'm asking of the solver doesn't look like it's overly difficult.
No quantifiers, finite domain, no functions, everything is a whole number or an enum, domains are small, the values of the numbers are small, there's only simple arithmetic, constraints are short, etc.
If I try to use parameters for parallel processing, or set the logic to "QF_FD", the runtime doesn't change at all.
Thanks in advance for any advice.
Is it always slow? Or does it get progressively slower as you query for more and more configurations using the same solver?
If it's the former, then your problem is just too hard and this is the price to pay. I don't see anything obviously wrong in what you've shown; though you should never use booleans as integers. (Just looking at your b variable in there. Stick to booleans as booleans, and integers as integers, and unless you really have to, don't mix the two together. See this answer for some further elaboration on this point: Why is Z3 slow for tiny search space?)
If it's the latter, you might want to create a solver from scratch for each query to clean-up all the extra stuff the solver created. While additional lemmas always help, they could also hurt performance if the solver cannot make good use of them in subsequent queries. And if you follow this path, then you can simply "parallelize" the problem yourself in your C++ program; i.e., create many threads and call the solver separately for each problem, taking advantage of many-cores your computer no doubt has and OS-level multi-tasking.
Admittedly, this is very general advice and may not apply directly to your situation. But, without a particular "running" example that we can see and inspect, it's hard to be any more specific than this.
Some Ideas:
1. Replace x == 1 || x == 3 || x == 3 || x == 7 || x == 8 with (1 <= x && x <= 8) && (x <= 1 || (3 <= x) && (x <= 4 || 7 <= x). Similar change with y.
rationale: the solver for linear arithmetic now knows that x is always confined in the interval [1,8], this can be useful information for other linear equalities/inequalities; it may be useful to also learn the trivial mutual exclusion constraints not(x <= 1) || not(3 <= x) and not(x <= 4) || not(7 <= x); there are now exactly 3 boolean assignments that cover your original 5 cases, this makes the reasoning of the linear arithmetic solver more cost-efficient because each invocation deals with a larger chunk of the search space. (Furthermore, it is more likely that clauses learned from conflicts are going to be useful with subsequent calls to the solver)
(Your queries may also contain set of values rather than specific assignments of values; this may allow one to prune some unsatisfiable ranges of values with fewer queries)
2. Just like #alias mentioned, Boolean variables ought to be Booleans and not 0/1 Integer variables. The example you provided is a bit confusing, b is declared as a bool const but then you state b == 0 || b == 1
3. I am not familiar with the enum_sort of z3, meaning that I don't know how it is internally encoded and what solving techniques are applied to deal with it. Therefore, I am not sure whether the solver may try to generate trivially inconsistent truth-assignments in which e == enum_value("green") e e == enum_value("red") are both assigned to true at the same time. This might be worth a bit of investigation. For instance, another possibility could be to declare e and f as Int and give them an appropriate interval domain (as contiguous as possible) with the same approach shown in 1., that will be interpreted by your software as a list of enum values. This should remove a number of Boolean assignments from the search space, make conflict clauses more effective and possibly speed-up the search.
4. Given the small number of problem variables, values and constraints, I would suggest you to try to encode everything using just the Bit-Vector theory and nothing else (using small-but-big-enough domains). If you then configure the solver to encode Bit-Vectors eagerly, then everything is bit-blasted into SAT, and z3 should only use Boolean Constraint Propagation for satisfiability, which is the cheapest technique.
This might be an X Y problem, why are you performing thousands of queries, what are you trying to achieve? Are you trying to explore all possible combination of values? Are you trying to perform model counting?
I have an optimization problem that I need to solve. It's a binary linear programming problem, so all of the decision variables are equal to 0 or 1. I need certain combinations of these decision variables to add up to either 0 or 2+, they cannot sum to 1. I'm struggling with how to accomplish this in PROC OPTMODEL.
Something like this is what I need:
con sum_con: x+y+z~=1;
Unfortunately, this just throws a syntax error... Is there any way to accomplish this?
See below for a linear reformulation. However, you may not need it. In SAS 9.4m2 (SAS/OR 13.2), your expression works as written. You just need to invoke the (experimental) CLP solver:
proc optmodel;
/* In SAS/OR 13.2 you can use your code directly.
Just invoke the experimental CLP solver */
var x binary, y binary, z binary;
con sum_con: x+y+z~=1;
solve with clp / findall;
print {i in 1 .. _NSOL_} x.sol[i]
{i in 1 .. _NSOL_} y.sol[i]
{i in 1 .. _NSOL_} z.sol[i];
produces immediately:
[1] x.SOL y.SOL z.SOL
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0
5 1 1 1
In older versions of SAS/OR, you can still call PROC CLP directly,
which is not experimental.
The syntax for your example will be very similar to PROC OPTMODEL's.
I am sure, however, that your model has other variables and constraints.
In that case, remember that no matter how you formulate this,
it is still a search space with a hole in the middle.
So it potentially can make the solver perform poorly.
How poorly is hard to predict. It depends on other features of your model.
If MILP is a better fit for the rest of your model,
you can reformulate your constraint as a valid MILP in two steps.
First, add a binary variable that is zero only when the expression is zero:
/* If solve with CLP is not available, you can linearize the disjunction: */
var IsGTZero binary; /* 1 if any variable in the expression is 1 */
con IsGTZeroBoundsExpression: 3 * IsGTZero >= x + y + z;
Then add another constraint that forces the expression to be
at least the constant you want (in this case 2) when it is nonzero.
num atLeast init 2;
con ZeroOrAtLeast: x + y + z >= atLeast * IsGTZero;
min f=0; /* Explicit objectives are unnecessary in 13.2 */
solve;
The following equation should work:
(x+y-z)*z + (y+z-x)*x + (x+z-y)*y > -1
It can be generalized to more than three variables and if you have some large number you should be able to use index expansions to make it easier.
I have a linear optimization goal to Maximize EE+FF, where EE and FF each consist of some C and D.
With code I've written, I can get solver to find:
EE_quantity: 0, FF_quantity: 7
...but I know there to be another solution:
EE_quantity: 1, FF_quantity: 6
In order to validate user input for other valid solutions, I added a constraint for both EE and FF. So I added the EE_quantity == 0, FF_quantity == 7 in the code below, which is a runnable example:
SolverContext c2 = SolverContext.GetContext();
Model m2 = c2.CreateModel();
p.elements = elements_multilevel_productmix();
Decision C_quantity = new Decision(Domain.IntegerNonnegative, "C_quantity");
Decision D_quantity = new Decision(Domain.IntegerNonnegative, "D_quantity");
Decision EE_quantity = new Decision(Domain.IntegerNonnegative, "EE_quantity");
Decision FF_quantity = new Decision(Domain.IntegerNonnegative, "FF_quantity");
m2.AddDecisions(C_quantity, D_quantity, EE_quantity, FF_quantity);
m2.AddConstraints("production",
6 * C_quantity + 4 * D_quantity <= 100,
1 * C_quantity + 2 * D_quantity <= 200,
2 * EE_quantity + 1 * FF_quantity <= C_quantity,
1 * EE_quantity + 2 * FF_quantity <= D_quantity,
EE_quantity == 0,
FF_quantity == 7
);
m2.AddGoal("fixed_EE_FF", GoalKind.Maximize, "EE_quantity + FF_quantity");
Solution sol = c2.Solve(new SimplexDirective());
foreach (var item in sol.Decisions)
{
System.Diagnostics.Debug.WriteLine(
item.Name + ": " + item.GetDouble().ToString()
);
}
It seems that Solver Foundation really doesn't like this specific combination. Using EE_quantity == 1, FF_quantity == 6 is fine, as is using just EE_quantity == 0 or FF_quantity == 7. But using both, AND having one of them being zero, throws an exception:
Index was outside the bounds of the array.
What is going on under the hood, here? And how do I specify that I want to find "all" solutions for a specific problem?
(Note: no new releases of Solver Foundation are forthcoming - it's essentially been dropped by Microsoft.)
The stack trace indicates that this is a bug in the simplex solver's presolve routine. Unfortunately the SimplexDirective does not have a way to disable presolve (unlike InteriorPointDirective). Therefore the way to get around this problem is to specify the fixed variables differently.
Remove the last two constraints that set EE_quantity and FF_quantity, and instead set both the upper and lower bounds to be 0 and 7 respectively when you create the Decision objects. This is equivalent to what you wanted to express, but appears to avoid the MSF bug:
Decision EE_quantity = new Decision(Domain.IntegerRange(0, 0), "EE_quantity");
Decision FF_quantity = new Decision(Domain.IntegerRange(7, 7), "FF_quantity");
The MSF simplex solver, like many mixed integer solvers, only returns the optimal solution. If you want MSF to return all solutions, change to the constraint programming solver (ConstraintProgrammingDirective). If you review the documentation for Solution.GetNext() you should figure out how to do this.
Of course the CP solver is not guaranteed to produce the globally optimal solution immediately. But if you iterate through solutions long enough, you'll get there.
Consider two vectors, A and B, of size n, 7 <= n <= 23. Both A and B consists of -1s, 0s and 1s only.
I need a fast algorithm which computes the inner product of A and B.
So far I've thought of storing the signs and values in separate uint32_ts using the following encoding:
sign 0, value 0 → 0
sign 0, value 1 → 1
sign 1, value 1 → -1.
The C++ implementation I've thought of looks like the following:
struct ternary_vector {
uint32_t sign, value;
};
int inner_product(const ternary_vector & a, const ternary_vector & b) {
uint32_t psign = a.sign ^ b.sign;
uint32_t pvalue = a.value & b.value;
psign &= pvalue;
pvalue ^= psign;
return __builtin_popcount(pvalue) - __builtin_popcount(psign);
}
This works reasonably well, but I'm not sure whether it is possible to do it better. Any comment on the matter is highly appreciated.
I like having the 2 uint32_t, but I think your actual calculation is a bit wasteful
Just a few minor points:
I'm not sure about the reference (getting a and b by const &) - this adds a level of indirection compared to putting them on the stack. When the code is this small (a couple of clocks maybe) this is significant. Try passing by value and see what you get
__builtin_popcount can be, unfortunately, very inefficient. I've used it myself, but found that even a very basic implementation I wrote was far faster than this. However - this is dependent on the platform.
Basically, if the platform has a hardware popcount implementation, __builtin_popcount uses it. If not - it uses a very inefficient replacement.
The one serious problem here is the reuse of the psign and pvalue variables for the positive and negative vectors. You are doing neither your compiler nor yourself any favors by obfuscating your code in this way.
Would it be possible for you to encode your ternary state in a std::bitset<2> and define the product in terms of and? For example, if your ternary types are:
1 = P = (1, 1)
0 = Z = (0, 0)
-1 = M = (1, 0) or (0, 1)
I believe you could define their product as:
1 * 1 = 1 => P * P = P => (1, 1) & (1, 1) = (1, 1) = P
1 * 0 = 0 => P * Z = Z => (1, 1) & (0, 0) = (0, 0) = Z
1 * -1 = -1 => P * M = M => (1, 1) & (1, 0) = (1, 0) = M
Then the inner product could start by taking the and of the bits of the elements and... I am working on how to add them together.
Edit:
My foolish suggestion did not consider that (-1)(-1) = 1, which cannot be handled by the representation I proposed. Thanks to #user92382 for bringing this up.
Depending on your architecture, you may want to optimize away the temporary bit vectors -- e.g. if your code is going to be compiled to FPGA, or laid out to an ASIC, then a sequence of logical operations will be better in terms of speed/energy/area than storing and reading/writing to two big buffers.
In this case, you can do:
int inner_product(const ternary_vector & a, const ternary_vector & b) {
return __builtin_popcount( a.value & b.value & ~(a.sign ^ b.sign))
- __builtin_popcount( a.value & b.value & (a.sign ^ b.sign));
}
This will lay out very well -- the (a.value & b.value & ... ) can enable/disable an XOR gate, whose output splits into two signed accumulators, with the first pathway NOTed before accumulation.
I've found functions which follow the pattern of 1 / bc produce nice curves which can be coupled with interpolation functions really nicely.
The way I use the function is by treating 'c' as the changing value, i.e. the interpolation value between 0 and 1, while varying b for 'sharpness'. I use it to work out an interpolation value between 0 and 1, so generelly the function I use is as such:
float interpolationvalue = 1 - 1/pow(100,c);
linearinterpolate( val1, val2, interpolationvalue);
Up to this point I've been using a hacked approach to make it 'work' since when interpolation value = 1 the value is very close to but not quite 0.
So I was wondering, is there a function in the form of or one which can reproduce similar curves to the ones produced by 1 / bc where at c = 0 result = 1 and c = 1 result = 0.
Or even C = 0, result = 0 and C = 1 result = 1.
Thanks for any help!
For interpolation the approach offering the most flexibility is using splines, in your case quadratic splines would seem sufficient. The wikipedia page is math heavy, but you can find adapted desciptions on google.
1 - c ^ b with small values for b? Another option would be to use a cubic polynomial and specifying the slope at 0 and 1.
You could use a similar curve of the form A - 1 / b^(c + a), choosing values of A and a to match your constraints. So, for c = 0, result = 1:
1 = A - 1/b^a => A = 1 + 1/b^a
and for c = 1, result = 0:
0 = A - 1/b^(1+a) => A = 1/b^(1+a)
Combining these, we can find a in terms of b:
1 + 1/b^a = 1/b^(1+a)
b^(1+a) + b = 1
b * (b^a - 1) = 1
b^a = 1/b - 1
So:
a = log_b(1/b - 1) = log(1/b - 1) / log(b)
A = 1 + 1/b^a = 1 / (1-b)
In real numbers, the ones that mathematician use, no function of the form you specify is ever going to return 0, division can't do that. (1/x)==0 has no real solutions. In floating point arithmetic, the poor relation of real arithmetic that computers use, you could write 1/(MAX_FP_VALUE^1) which will give you as close to 0 as you are ever going to get (actually, it might give you a NaN or one of the other odd returns that IEEE 754 allows).
And, as I'm sure you've noticed, 1/(b^0) always returns 1 since b^0 is, by definition of 0-th power, always 1.
So, no function with c = 0 will produce a result of 0.
For c = 1, result = 1, set b = 1
But I guess this is only a partial answer, I'm not terribly sure I understand what you are trying to do.
Regards
Mark