When Sympy generates C code,
is there a way to enforce CSE optimizations for pow (or powf) occurrences in an expression?
For example, this code snippet
c, s = symbols('c s')
myexpr = c**6/1800 - c**5/100 - 0.00833333333333333*c**4*s**2 + 19*c**4/200 + 0.1*c**3*s**2 - 9*c**3/20 + c**2*s**4/120 - 0.57*c**2*s**2 + 43*c**2/40 - c*s**4/20 + 1.35*c*s**2 + 23*c/50 - 0.000555555555555556*s**6 + 19*s**4/200 - 1.075*s**2 - 2107/1800
import sympy
from sympy.codegen.ast import real, float64
sub_exprs,final_expr = sympy.cse([myexpr])
for var,expr in sub_exprs : print "const real", printing.ccode(expr, standard='C99', assign_to=var, type_aliases={real: float64})
print "return ",printing.ccode(final_expr[0], standard='C99', type_aliases={real: float64}),";"
produces the following disappointing output:
const real x0 = pow(c, 2);
const real x1 = pow(c, 3);
const real x2 = pow(c, 4);
const real x3 = pow(s, 2);
const real x4 = pow(s, 4);
return (1.0/1800.0)*pow(c, 6) - 1.0/100.0*pow(c, 5) + 1.3500000000000001*c*x3 - 1.0/20.0*c*x4 + (23.0/50.0)*c - 0.00055555555555555599*pow(s, 6) - 0.56999999999999995*x0*x3 + (1.0/120.0)*x0*x4 + (43.0/40.0)*x0 + 0.10000000000000001*x1*x3 - 9.0/20.0*x1 - 0.0083333333333333297*x2*x3 + (19.0/200.0)*x2 - 1.075*x3 + (19.0/200.0)*x4 - 2107.0/1800.0 ;
Pow optimizations have been completely ignored.
What is the workaround for this?
Remark: I saw that this issue is partially mentioned here:
"The code printers don’t print optimal code in many cases. An example of this is powers in C. x**2 prints as pow(x, 2) instead of x*x. Other optimizations (like mathematical simplifications) should happen before the code printers."
The CSE routine in sympy is not perfect (improved CSE is listed as an area for improvement), e.g.:
>>> sympy.cse([x**4, x**3*y])
([], [x**4, x**3*y])
Expanding pow in the printer or before the printer has been discussed some time, there is now a create_expand_pow optimization which can help some:
>>> expand_opt = create_expand_pow_optimization(3)
>>> expand_opt(x**5 + x**3)
x**5 + x*x*x
Note however that most compilers will already generate optimal assembly (regardless of CSE in the source code) if you pass them the right optimization flags.
Related
I am trying to make a solver for nonlinear system of equations using the Z3 library in c++.
It will exists with multiple equations and variables. Depending on previous steps in my software, it could be that some variables are unknown.
When too many variables are unknown and the system does not have a single solution, I want to be notified about this.
I made a simple fictional system of equations for testing that looks as follows:
Ax = 1
Ay = 1
Bx = 3
By = 2
Cx = 7
Cy = Ay + (By - Ay) / (Bx - Ax) * (Cx - Ax)
I wrote the following code to solve the system, in this example fairly easy:
context ctx;
expr Ax = ctx.real_const("Ax");
expr Ay = ctx.real_const("Ay");
expr Bx = ctx.real_const("Bx");
expr By = ctx.real_const("By");
expr Cx = ctx.real_const("Cx");
expr Cy = ctx.real_const("Cy");
solver s(ctx);
s.add(Ax == 1);
s.add(Ay == 1);
s.add(Bx == 3);
s.add(By == 2);
s.add(Cx == 7);
s.add(Cy == Ay + (By - Ay) / (Bx - Ax) * (Cx - Ax));
std::cout << s.check() << "\n";
model m = s.get_model();
auto CySolution = m.eval(Cy, true);
std::cout << CySolution << "\n";
As output I get the following:
sat
4.0
When for example "Ax" would be unknown, this can be simulated using following code:
context ctx;
expr Ax = ctx.real_const("Ax");
expr Ay = ctx.real_const("Ay");
expr Bx = ctx.real_const("Bx");
expr By = ctx.real_const("By");
expr Cx = ctx.real_const("Cx");
expr Cy = ctx.real_const("Cy");
solver s(ctx);
//s.add(Ax == 1);
s.add(Ay == 1);
s.add(Bx == 3);
s.add(By == 2);
s.add(Cx == 7);
s.add(Cy == Ay + (By - Ay) / (Bx - Ax) * (Cx - Ax));
std::cout << s.check() << "\n";
model m = s.get_model();
auto CySolution = m.eval(Cy, true);
std::cout << CySolution << "\n";
Now the output is:
sat
(/ 78.0 23.0)
I have tried detect when this system of equations give an implicit solution using functions as "expr.get_sort()" or "expr.is_real", but there is never a difference between the complete solution and the implicit one.
Now I have two questions:
How can I interpret "(/ 78.0 23.0)"?
Is there a proper way to detect if the solution is an implicit function or not?
Thanks in advance for any help!
(/ 78.0 23.0) simply means 78/23, i.e., approx. 3.39. Z3 prints real-values in this format since they are infinitely precise; as the fraction might require an unbounded number of digits in general.
Your question regarding "proper way to detect is an implicit function" is a bit ambiguous. I assume what you mean is if there's a unique solution? That is, if a variable's value is "forced" by all the other equations or not?
If that's the case, i.e., to check that a solution is unique, you'd essentially get the first solution, then assert the negation of that solution and ask if there's some other solution with this additional constraint: If the solver comes back unsat then you know the solution was unique. See many questions on stack-overflow regarding how to get multiple solutions from the solver.
Can lp_solve return a unifrom solution? (Is there a flag or something that will force this kinf of behavior?)
Say that I have this:
max: x + y + z + w;
x + y + z + w <= 100;
Results in:
Actual values of the variables:
x 100
y 0
z 0
w 0
However, I would like to have something like:
Actual values of the variables:
x 25
y 25
z 25
w 25
This is an oversimplyfied example, but the idea is that if the variables have the same factor in the objective function, then the result should idealy be more uniform and not everything for one, and the other what is left.
Is this possible to do? (I've tested other libs and some of them seem to do this by default like the solver on Excel or Gekko for Python).
EDIT:
For instance, Gekko has already this behavior without me especifing anything...
from gekko import GEKKO
m = GEKKO()
x1,x2,x3,x4 = [m.Var() for i in range(4)]
#upper bounds
x1.upper = 100
x2.upper = 100
x3.upper = 100
x4.upper = 100
# Constrain
m.Equation(x1 + x2 + x3 + x4 <= 100)
# Objective
m.Maximize(x1 + x2 + x3 + x4)
m.solve(disp=False)
print(x1.value, x2.value, x3.value, x4.value)
>> [24.999999909] [24.999999909] [24.999999909] [24.999999909]
You would need to explicitly model this (as another objective). A solver does nothing automatically: it just finds a solution that obeys the constraints and optimizes the objective function.
Also, note that many linear solvers will produce so-called basic solutions (corner points). So "all variables in the middle" does not come naturally at all.
The example in Gekko ended on [25,25,25,25] because of how the solver took a step towards the solution from an initial guess of [0,0,0,0] (default in Gekko). The problem is under-specified so there are an infinite number of feasible solutions. Changing the guess values gives a different solution.
from gekko import GEKKO
m = GEKKO()
x1,x2,x3,x4 = m.Array(m.Var,4,lb=0,ub=100)
x1.value=50 # change initial guess
m.Equation(x1 + x2 + x3 + x4 <= 100)
m.Maximize(x1 + x2 + x3 + x4)
m.solve(disp=False)
print(x1.value, x2.value, x3.value, x4.value)
Solution with guess values [50,0,0,0]
[3.1593723566] [32.280209196] [32.280209196] [32.280209196]
Here is one method with equality constraints m.Equations([x1==x2,x1==x3,x1==x4]) to modify the problem to guarantee a unique solution that can be used by any linear programming solver.
from gekko import GEKKO
m = GEKKO()
x1,x2,x3,x4 = m.Array(m.Var,4,lb=0,ub=100)
x1.value=50 # change initial guess
m.Equation(x1 + x2 + x3 + x4 <= 100)
m.Maximize(x1 + x2 + x3 + x4)
m.Equations([x1==x2,x1==x3,x1==x4])
m.solve(disp=False)
print(x1.value, x2.value, x3.value, x4.value)
This gives a solution:
[25.000000002] [25.000000002] [25.000000002] [25.000000002]
QP Solution
Switching to a QP solver allows a slight penalty for deviations but doesn't consume a degree of freedom.
from gekko import GEKKO
m = GEKKO()
x1,x2,x3,x4 = m.Array(m.Var,4,lb=0,ub=100)
x1.value=50 # change initial guess
m.Equation(x1 + x2 + x3 + x4 <= 100)
m.Maximize(x1 + x2 + x3 + x4)
penalty = 1e-5
m.Minimize(penalty*(x1-x2)**2)
m.Minimize(penalty*(x1-x3)**2)
m.Minimize(penalty*(x1-x4)**2)
m.solve(disp=False)
print(x1.value, x2.value, x3.value, x4.value)
Solution with QP penalty
[24.999998377] [25.000000544] [25.000000544] [25.000000544]
Tell me please, How to forbid to open brackets? For example,
8 * (x + 1) It should be that way, not 8 * x + 8
Using evaluate = False doesn't help
The global evaluate flag will allow you to do this in the most natural manner:
>>> with evaluate(False):
... 8*(x+1)
...
8*(x + 1)
Otherwise, Mul(8, x + 1, evaluate=False) is a lower level way to do this. And conversion from a string (already in that form) is possible as
>>> S('8*(x+1)',evaluate=False)
8*(x + 1)
In general, SymPy will convert the expression to its internal format, which includes some minimal simplifications. For example, sqrt is represented internally as Pow(x,1/2). Also, some reordering of terms may happen.
In your specific case, you could try:
from sympy import factor
from sympy.abc import x, y
y = x + 1
g = 8 * y
g = factor(g)
print(g) # "8 * (x + 1)"
But, if for example you have g = y * y, SymPy will either represent it as a second power ((x + 1)**2), or expand it to x**2 + 2*x + 1.
PS: See also this answer by SymPy's maintainer for some possible workarounds. (It might complicate things later when you would like to evaluate or simplify this expression in other calculations.)
How about sympy.collect_const(sympy.S("8 * (x + 1)"), 8)?
In general you might be interested in some of these expression manipulations: https://docs.sympy.org/0.7.1/modules/simplify/simplify.html
I don't understand why the expression a * (... + 1) - a is not being removed while simplification. The example below shows the bug:
import sympy as sy
a,b,c = sy.symbols('a b c')
expr = a * (b - c + 1) - a + (b - c) * (a - b)
print expr # printed: a*(b - c + 1) - a + (a - b)*(b - c)
print expr.simplify() # printed: a*(b - c + 1) - a + (a - b)*(b - c)
On the other side, if I change the expression by
expr = a * (b - c + 1) - a
and call simplify(), I will obtain the expected result a * (b - c).
Sympy version is 1.1rc1.
simplify usually can only do a limited amount of magic. It could be arguably more in this case, but if you want that, you need to make a feature request. In any case it’s better to tell SymPy what specific kind of modifications you want to make.
Here, the following will probably satisfy you:
print(expr.factor()) # (2*a - b)*(b - c)
Consider the following piece of code showing some simple arithmetic operations
int result = 0;
result = c * (a + b) + d * (a + b) + e;
To get the result in the expression above the cpu would need to execute two integer multiplications and three integer additions. However algebraically the above expression could be simplified to the code below.
result = (c + d) * (a + b) + e
The two expressions are algebraically identical however the second expression only contains one multiplication and three additions. Is gcc (or other compilers for that matter) able to make this simple optimization on their own.
Now assuming that the compiler is intelligent enough to make this simple optimization, would it be able to optimize something more complex such as the Trapezoidal rule (used for numerical integration). Example below approximates the area under sin(x) where 0 <= x <= pi with a step size of pi/4 (small for the sake of simplicity). Please assume all literals are runtime variables.
#include <math.h>
// Please assume all literals are runtime variables. Have done it this way to
// simplify the code.
double integral = 0.5 * ((sin(0) + sin(M_PI/4) * (M_PI/4 - 0) + (sin(M_PI/4) +
sin(M_PI/2)) * (M_PI/2 - M_PI/4) + (sin(M_PI/2) + sin(3 * M_PI/4)) *
(3 * M_PI/4 - M_PI/2) + (sin(3 * M_PI/4) + sin(M_PI)) * (M_PI - 3 * M_PI/4));
Now the above function could be written like this once simplified using the trapezoidal rule. This drastically reduces the number of multiplications/divisions needed to get the same answer.
integral = 0.5 * (1 / no_steps /* 4 in th case above*/) *
(M_PI - 0 /* Upper and lower limit*/) * (sin(0) + 2 * (sin(M_PI/4) +
sin(3 * M_PI/4)) + sin(M_PI));
GCC (and most C++ compilers, for that matter) does not refactor algebraic expressions.
This is mainly because as far as GCC and general software arithmetic is concerned, the lines
double x = 0.5 * (4.6 + 6.7);
double y = 0.5 * 4.6 + 0.5 * 6.7;
assert(x == y); //Will probably fail!
Are not guaranteed to be evaluate to the exact same number. GCC can't optimize these structures without that kind of guarantee.
Furthermore, order of operations can matter a lot. For example:
int x = y;
int z = (y / 16) * 16;
assert(x == z); //Will only be true if y is a whole multiple of 16
Algebraically, those two lines should be equivalent, right? But if y is an int, what it will actually do is make x equal to "y rounded to the lower whole multiple of 16". Sometimes, that's intended behavior (Like if you're byte aligning). Other times, it's a bug. The important thing is, both are valid computer code and both can be useful depending on the circumstances, and if GCC optimized around those structures, it would prevent programmers from having agency over their code.
Yes, optimizers, gcc's included, do optimizations of this type. Not necessarily the expression that you quoted exactly, or other arbitrarily complex expressions. But a simpler expresion, (a + a) - a is likely to be optimized to a for example. Another example of possible optimization is a*a*a*a to temp = a*a; (temp)*(temp)
Whether a given compiler optimizes the expressions that you quote, can be observed by reading the output assembly code.
No, this type of optimization is not used with floating points by default (unless maybe if the optimizer can prove that no accuracy is lost). See Are floating point operations in C associative? You can let for example gcc do this with -fassociative-math option. At your own peril.