I have to find all basic solutions of some tiny linear-programming problems.
Here's an example (in lp_solve format):
max: x1 + x2;
x1 + x2 <= 1;
x1 <= 0.8;
x2 <= 0.8;
All 2 basic solutions:
x1 = 0.2, x2 = 0.8
x1 = 0.8, x2 = 0.2
Of course there is a way of finding alternative solutions, but I really prefer using existing libraries instead of crafting my own simplex code.
I'm using Python as my programming language, and hoping there's some method in lp_solve or GLPK's C API can do this.
Thanks.
There is no routine to do that with glpk; and IMHO it is very unlikely that any real-world solver implements something like that, since it is not very useful in practise and it is certainly not a simple problem.
What is indeed easy to find one other basic solution once you reached optimality with the simplex algorithm, which does not mean that it is easy to list them all.
Consider a LP whose domain has dimension n; the set S of the optimal solutions is a convex polyhedron whose dimension m can be anything from 0 to n-1.
You want a method to list all the basic solutions of the problem, that is all the vertices of S: as soon as m is greater than 2, you will need to carefully avoid cycling when you move from one basic solution to another.
However, there is (luckily!) no need to write your own simplex code: you can access the internals of the current basis with the glpk library, and probably with lpsolve too.
Edit: two possible solutions
The better way would be to use another library such as PPL for this.
Assume that you have a problem of the form:
min cx; subject to: Ax <= b
First solve your problem with glpk, this will give you the optimal value V of the problem. From this point, you can use PPL to get the description of the polyedron of optimal values:
cx = V and Ax <= b
as the convex hull of its extreme points, which correspond to the BFSs you are looking for.
You can (probably) use the glpk simplex routines. Once you get an optimal BFS, you can get the reduced cost associated with all non-basic columns using the routine glp_get_row_dual (the basis status of the variable can be obtained with glp_get_row_stat), so you can find a non-basic variables with a null reduced cost. Then, I think that you can use function glp_set_row_stat to change the basis status of this column in order to have it enter the basis.
(And then, you can iterate this process as long as you avoid cycling.)
Note that I did not try any of those solutions myself; I think that the first one is by far the best, although it will require that you learn the PPL API. If you wish to go for the second one, I strongly suggest that you send an e-mail to the glpk maintainer (or look at the source code), because I am really not sure it will work as is.
Related
Introduction
Let’s assume that I need the Jacobian matrix for the following set of ODE:
dxdt[ 0 ] = -90.0 * x[0] - 50.0 * x[1];
dxdt[ 1 ] = x[0] + 3*x[1];
dxdt[ 2 ] = x[1] + 50*x[2];
In Matlab/Octave this would be pretty easy:
syms x0 x1 x2;
f = [-90.0*x0-50.0*x1, x0+3*x1, x1+50*x2]
v=[x0, x1, x2]
fp = jacobian(f,v)
This would results with following output matrix:
[-90 -50 0 ]
[ 1 3 0 ]
[ 0 1 50]
What I need
Now I want to reproduce the same results in C++. I can’t compute the Jacobian before and hard-code it, as it will depend for example on user inputs and time. So my question is: How to do this? Usually for mathematics operations, I use the Boost library, however in this case I can’t find any solution. There’s only short note about this in implicit systems, but the following code doesn’t work:
sys.second( x , jacobi , t )
It also requests the time (t), so it probably doesn’t generate an analytic form of solution. Do I misunderstand the documentations? Or should I use another function? I would prefer to stay within Boost, as I need the Jacobian as ublas::matrix and I want to avoid conversion.
EDIT:
More specific I will use Jacobian inside rosenbrock4 ODE solver. Example here - lines 47-52. I need automatic generation of this structure as the ODE set may be changed later and I want to avoid manually rewriting Jacobian ever time. Also some variables inside ODE definitions are not constant in time.
I know this is long after the fact, but I have recently been wanting to do the same thing and have come across many auto differentiation (AD) libraries that do this pretty well. I have mostly been using Eigen's AD because I am already using Eigen everywhere. Here's an example of how you can use Eigen's AD to get the jacobian like you asked.
There's also a long list of c++ AD libraries on autodiff.org.
Hope this helps someone!
The Jacobian is based on derivatives of the function. If the function f is only known at run-time (and there are no constraints such as linearity), you have to automatise the differentiation. If you want this to happen exactly (as opposed to a numerical estimation), you need to use symbolic computation. Look for example here and here for libraries supporting this.
Note that the Jacobian usually depends on the state and the time, so it’s impossible to represent it as a constant matrix (such as in your example), unless your problem is so boring that you can solve it analytically anyway.
I am looking for a simple way to obtain a lot of "good" solutions in a LP problem (not MIP) with CPLEX, and not only (one of the) optimal basic solution(s). By "good" solutions I mean that the corresponding objective values are not so far from the real optimal value. Such pool of solutions could help the decision-maker...
More precisely, given a certain polyedron Ax<=b with x>=0 and an objective function z=cx I want to maximize, after running the LP, I can obtain the optimal value z*. Then I want to enumerate all the extreme points of the polyhedron given by the set of constraints
Ax <= b
cx >= z* - epsilon
x >= 0
when epsilon is a given tolerance.
I know that CPLEX offers way to generate solution pool (see here), but it will not function because this method is for MIP : it enumerates all the solutions of an IP (or one solution for every given set of fixed integer variables if the problem is a MIP).
An interesting efficient way is to visit the adjacent solutions of the optimal basic solution, i.e. all the adjacent extreme points : if I suppose the polyhedron is not degenerative, for each pair of basic variable x_B and non-basic variable x_N, I compute the basic solution obtained when x_B leaves the basis and x_N enters in the basis. Then I throw the solutions with cx < z*-epsilon, and for the others I repeat the procedure. [I know that I could improve this algorithm, but this is the general idea].
The routine CPPXpivot of the Callable Library could help to do this pivoting operation, but I did not find an equivalent in the C++ API (concert technology). Does someone know if such an equivalent exist, or could propose me an other way to answer my original problem ?
Thanks a lot :) !
Rémi L.
There is one interesting way to make this suitable for use with the Cplex solution pool. Use binary variables to encode the current basis, e.g. basis[k] = 0 meaning nonbasic and basis[k] = 1 indicating variable (or row) k is basic. Of course we have sum(k, basis[k]) = m (number of rows). Finally we have x[k] <= basis[k] * upperbound[k] (i.e. if nonbasic then zero -- assuming positive variables). When we add this to the LP model we end up with a MIP and can enumerate (all or some, optimal or near optimal) bases using the Cplex solution pool. See here and here.
Here's the problem:
I am currently trying to create a control system which is required to find a solution to a series of complex linear equations without a unique solution.
My problem arises because there will ever only be six equations, while there may be upwards of 20 unknowns (usually way more than six unknowns). Of course, this will not yield an exact solution through the standard Gaussian elimination or by changing them in a matrix to reduced row echelon form.
However, I think that I may be able to optimize things further and get a more accurate solution because I know that each of the unknowns cannot have a value smaller than zero or greater than one, but it is free to take on any value in between them.
Of course, I am trying to create code that would find a correct solution, but in the case that there are multiple combinations that yield satisfactory results, I would want to minimize Sum of (value of unknown * efficiency constant) over all unknowns, i.e. Sigma[xI*eI] from I=0 to n, but finding an accurate solution is of a greater priority.
Performance is also important, due to the fact that this algorithm may need to be run several times per second.
So, does anyone have any ideas to help me on implementing this?
Edit: You might just want to stick to linear programming with equality and inequality constraints, but here's an interesting exact solution that does not incorporate the constraint that your unknowns are between 0 and 1.
Here's a powerpoint discussing your problem: http://see.stanford.edu/materials/lsoeldsee263/08-min-norm.pdf
I'll translate your problem into math to make things a bit easier to figure out:
you have a 6x20 matrix A and a vector x with 20 elements. You want to minimize (x^T)e subject to Ax=y. According to the slides, if you were just minimizing the sum of x, then the answer is A^T(AA^T)^(-1)y. I'll take another look at this as soon as I get the chance and see what the solution is to minimizing (x^T)e (ie your specific problem).
Edit: I looked in the powerpoint some more and near the end there's a slide entitled "General norm minimization with equality constraints". I am going to switch the notation to match the slide's:
Your problem is that you want to minimize ||Ax-b||, where b = 0 and A is your e vector and x is the 20 unknowns. This is subject to Cx=d. Apparently the answer is:
x=(A^T A)^-1 (A^T b -C^T(C(A^T A)^-1 C^T)^-1 (C(A^T A)^-1 A^Tb - d))
it's not pretty, but it's not as bad as you might think. There's really aren't that many calculations. For example (A^TA)^-1 only needs to be calculated once and then you can reuse the answer. And your matrices aren't that big.
Note that I didn't incorporate the constraint that the elements of x are within [0,1].
It looks like the solution for what I am doing is with Linear Programming. It is starting to come back to me, but if I have other problems I will post them in their own dedicated questions instead of turning this into an encyclopedia.
I am trying to solve two point boundary problem with odeint. My equation has the form of
y'' + a*y' + b*y + c = 0
It is pretty trivial when I have boundary conditions of y(x_1) = y_1 , y'(x_2) = y_2, but when boundary conditions are y(x_1) = y_1 , y(x_2) = y_2 I am lost. Does anybody know the way to deal with problems like this with odeint or other scientific library?
In this case you need a shooting method. odeint does not have such a method, it solved the initial value problem (IVP) which is your first case. I think in the Numerical Recipies this method is explained and you can use Boost.Odeint to do the time stepping.
An alternative and more efficient method to solve this type of problem is finite differences or finite elements method. For finite differences you can check Numerical Recipes. For finite elements I recommend dealii library.
Another approach is to use b-splines: Assuming you do know the initial x0 and final xfinal points of integration, then you can expand the solution y(x) in a b-spline basis, defined over (x0,xfinal), i.e.
y(x)= \sum_{i=1}^n A_i*B_i(x),
where A_i are constant coefficients to be determined, and B_i(x) are b-spline basis (well defined polynomial functions, that can be differentiated numerically). For scientific applications you can find an implementation of b-splines in GSL.
With this substitution the boundary value problem is reduced to a linear problem, since (am using Einstein summation for repeated indices):
A_i*[ B_i''(x) + a*B_i'(x) + b*B_i(x)] + c =0
You can choose a set of points x and create a linear system from the above equation. You can find information for this type of method in the following review paper "Applications of B-splines in Atomic and Molecular Physics" - H Bachau, E Cormier, P Decleva, J E Hansen and F Martín
http://iopscience.iop.org/0034-4885/64/12/205/
I do not know of any library solving directly this problem, but there are several libraries for B-splines (I recommend GSL for your needs), that will allow you to form the linear system. See this stackoverflow question:
Spline, B-Spline and NURBS C++ library
In a C++ application I'm coding, I need to solve a system of non-linear equations (N equations, N unknowns).
The systems I'm solving will be rather small (up to 10 equations/unknowns), so performance is not going to be a real issue.
I've searched the web a bit for a non-linear solver library, and I couldn't get to something which looks easy to use (got to NOX and C/C++ Minpack, but both seem to be an overkill for my need).
Any thoughts and ideas of easy-to-use libraries for this purpose?
One thing should be clear: non-linear equation solution isn't easy. It's not the same as solving linear equations. You aren't always guaranteed to get a solution. And your choice of initial condition and incrementation strategy can have a profound effect on the solution you do get.
With that said, I can't recommend a particular library, but you should be on the lookout for a linear algebra package that includes Newton-Raphson iteration in its menu of choices.
There are two options for you, you can use the sundials packages which includes a nonlinear solver, written in C I think. The only problem I've found with it is that you need to give it good initial estimates. The second option is to use NLEQ or NLEQ2 which I think are superior (writtein in FORTRAN but easy to link to C like langages. However I have had some problems locating it just now. There is a good web site with a list of possible options at: http://plato.asu.edu/sub/zero.html
Numerical Recipes has a routine that will do the job for you.
It depends on how non-linear the equations are. If they possess some "nice" properties...most obvious being positive-semi-definite matrix or convexity, there may be specialized algorithms available. I use IBM/ILOG CPLEX for most of my linear programming needs. Libraries are provided that can be pulled into C++ applications. Although I have not used their quadratic programming module, it is really the state-of-the-art in high horse-power linear and (well-behaved) non-linear programming.
There is always GSL, but all the comments made in the other answers apply to this as well:
http://www.gnu.org/software/gsl/manual/html_node/Multidimensional-Root_002dFinding.html#index-nonlinear-systems-of-equations_002c-solution-of-2426
Have you looked at COIN-OR? It might help if you submit your question to the OR-Exchange.
It's not free by any means, but Solver would work here.
Microsoft Z3 https://github.com/Z3Prover/z3/blob/master/examples/c%2B%2B/example.cpp
also consider omnn::math:
https://github.com/ohhmm/openmind/blob/master/omnn/math/test/08_System.cpp
Lets say system of equations is like this:
(x-a1)^2 + (y-b1)^2 = c1
(x-a2)^2 + (y-b2)^2 = c2
Then you have couple options:
Valuable a1, a2, b1, b2; // init with values
System sys;
Variable x,y;
sys << (x-a1)^2 + (y-b1)^2 - c1; // addin an equation as an equality to 0
sys << (x-a2)^2 + (y-b2)^2 - c2;
for(auto& solution : sys.Solve(x))
std::cout << solution;
alternative way is to make single equation (see why):
((x-a1)^2 + (y-b1)^2 - c1)^2 + ((x-a2)^2 + (y-b2)^2 - c2)^2 = 0
Variable x,y;
Valuable a1, a2, b1, b2; // init with values
auto eq = ((x-a1)^2 + (y-b1)^2 - c1)^2 + ((x-a2)^2 + (y-b2)^2 - c2)^2;
eq.SetView(Valuable::View::Equation); // optional: equation optimizations
// get y function:
auto fn = eq(y);
// show
std::cout << fn << std::endl;
// evaluate
auto evaluate = fn;
evaluate.eval(x, 10);
evaluate.optimize(); // calculate
// show calculated value at x=10:
std::cout << evaluate << std::endl;