If we consider the following linear regression example for PyMC3:
http://docs.pymc.io/notebooks/getting_started.html#A-Motivating-Example:-Linear-Regression
How would we include a constraint such as a + b1 + b2 = 1 or a^2 + b1^2 = 25?
I understand that we can use Bound to create bounds for variables, but I wasn't sure how to add a more complex constraint.
Thanks for the help!
A general solution will be to use a Potential.
const = pm.Potential('const', pm.math.switch(pm.math.eq(a**2 + b1**2, 25),
0,
-np.inf))
A potential is an arbitrary factor that you can add to the model likelihood. In this example if the parameters satisfies you constraints you add nothing otherwise you add -inf.
For future reference you can also ask questions here
Related
While the result of the my linear programming model in Cplex seems to make sense, the q variable sometimes randomly (at least to me it seems random) shows tiny values such as 4e^-14. This doesn't have an effect on the decision variable but is still very irritating as I am not sure if something in my model isn't correct. You can see the results of the q variable with the mini residuals here: Results q variable. These residuals only started appearing in my model once I introduced binary variables.
q is defined as: dexpr float q [t in Years, i in Options] = (c[i] * (a[t+s[i]][i]-a[t+s[i]-1][i]));
a is a decision variable
This is a constraint q is subject to: q[i][t] == a[i] * p[i]* y[t][i])
Since y is a binary variable, q should either be the value of a[i] * p[i] or 0. This is why I am very irritated with the residual values.
Does anybody have any idea why these values appear and how to get rid of them? I have already spent a lot of time on this problem and no idea how to solve it.
Things I noticed while trying to solve it:
Turning all the input variables into integer variables doesn't change
anything
Turning q into an integer variable solves the problem, but ruins the model since a[i][t] needs to be a float variable
Adding a constraint making q >= 0 does not eliminate the negative residual values such as -4e^-14
Adding a constraint making q = 0 for a specific t helps eliminate the residual values there, but of course also ruins the model
Thank you very much for your help!
This is a tolerance issue. MIP solvers such as Cplex have a bunch of them. The ones in play here are integer feasibility tolerance (epint) and the feasibility tolerance (eprhs). You can tighten them, but I usually leave them as they are. Sometimes it helps to round results before printing them or just use fewer digits in the formatting of the output.
I have been working on a combinatorial optimization problem which can be modeled as an integer linear programming. I implemented it as a c++ project in visual studio 2017 and CPLEX1271. Since there are exponentially many constraints, I implemented lazy constraints by IloCplex::LazyConstraintCallbackI. In my mind the following procedure is how an optimal soution is produced: everytime an integer solution is identified, LazyConstraintCallbackI will check it and add some violated constraints to the model untill an optimal integer solution is obtained.
However, the objective values given by my implementation for different inputs are not always correct. After almost one year's intermittent debug and test, I finally figured out the reason, which is very problem related but can be explained(hopefully) by the following tiny example: an integer linear programming involving four bool variables x1, x2, x3 and x4
minimize x1
subject to:
x1 ≥ x2
x1 ≥ x3
x1 ≥ x4
x2 + x3 ≥ 1
x1, x2, x3 and x4 ∈ {0, 1}
The result given by cplex is:
Solution status = Optimal
Objective value = 1
x1 = 1
x2 = 1
x3 = 1
x4 = 1
Undoubtedly, the objective value is correct .The weird thing is that cplex sets x4 = 1. Although whether x4 equals 1 or 0 has no impact on the objective value in this programming. But, when lazy constraint callback is used, this could lead to a problem by adding some incorrect contraint
and integer programming is solved by iteratively adding violated constraints. I'd like to know:
why cplex set "unrelated" variable x4 as 1, instead of 0?
What should I do to tell CPLEX that I want to leave such "unrelated" variable as 0?
Since there is nothing that forces x4=0 in an optimal solution, there is no guarantee that CPLEX will set x4=0? Why would it? Why is 0 preferred over 1? The model has two optimal solutions, one with x4=0 and one with x4=1. Both have objective value 1. CPLEX is completely free to choose one of them.
Like sascha said in his comments, the only way to force x4=0 in an optimal solution is to add this constraint to the model, for example by setting the objective coefficient to a small positive value.
However, it seems strange that your lazy constraint callback would generate invalid constraints from integer feasible solutions. This looks like a bug: Either the callback makes invalid assumptions about the model (namely that x4=0 in any optimal solution) or there is a bug somewhere in the logic of the callback. Note that in the model at hand it would actually be fine for the callback to cut off the solution with x4=1 since that still leaves the equivalent optimal solution with x4=0 and CPLEX would eventually find that.
Is it possible to constrain parameters in Stata's xttobit to be non-negative? I read a paper where the authors said they did just that, and I am trying to work out how.
I know that you can constrain parameters to be strictly positive by exponentially transforming the variables (e.g. gen x1_e = exp(x1)) and then calling nlcom after estimation (e.g. nlcom exp(_b[x1:_y]) where y is the independent variable. (That may not be exactly right, but I am pretty sure the general idea is correct. Here is a similar question from Statlist re: nlsur).
But what would a non-negative constraint look like? I know that one way to proceed is by transforming the variables, for example squaring them. However, I tried this with the author's data and still found negative estimates from xttobit. Sorry if this is a trivial question, but it has me a little confused.
(Note: this was first posted on CV by mistake. Mea culpa.)
Update: It seems I misunderstand what transformation means. Suppose we want to estimate the following random effects model:
y_{it} = a + b*x_{it} + v_i + e_{it}
where v_i is the individual random effect for i and e_{it} is the idiosyncratic error.
From the first answer, would, say, an exponential transformation to constrain all coefficients to be positive look like:
y_{it} = exp(a) + exp(b)*x_{it} + v_i + e_{it}
?
I think your understanding of constraining parameters by transforming the associated variable is incorrect. You don't transform the variable, but rather you fit your model having reexpressed your model in terms of transformed parameters. For more details, see the FAQ at http://www.stata.com/support/faqs/statistics/regression-with-interval-constraints/, and be prepared to work harder on your problem than you might have expected to, since you will need to replace the use of xttobit with mlexp for the transformed parameterization of the tobit log-likelihood function.
With regard to the difference between non-negative and strictly positive constraints, for continuous parameters all such constraints are effectively non-negative, because (for reasonable parameterization) a strictly positive constraint can be made arbitrarily close to zero.
I have to find all basic solutions of some tiny linear-programming problems.
Here's an example (in lp_solve format):
max: x1 + x2;
x1 + x2 <= 1;
x1 <= 0.8;
x2 <= 0.8;
All 2 basic solutions:
x1 = 0.2, x2 = 0.8
x1 = 0.8, x2 = 0.2
Of course there is a way of finding alternative solutions, but I really prefer using existing libraries instead of crafting my own simplex code.
I'm using Python as my programming language, and hoping there's some method in lp_solve or GLPK's C API can do this.
Thanks.
There is no routine to do that with glpk; and IMHO it is very unlikely that any real-world solver implements something like that, since it is not very useful in practise and it is certainly not a simple problem.
What is indeed easy to find one other basic solution once you reached optimality with the simplex algorithm, which does not mean that it is easy to list them all.
Consider a LP whose domain has dimension n; the set S of the optimal solutions is a convex polyhedron whose dimension m can be anything from 0 to n-1.
You want a method to list all the basic solutions of the problem, that is all the vertices of S: as soon as m is greater than 2, you will need to carefully avoid cycling when you move from one basic solution to another.
However, there is (luckily!) no need to write your own simplex code: you can access the internals of the current basis with the glpk library, and probably with lpsolve too.
Edit: two possible solutions
The better way would be to use another library such as PPL for this.
Assume that you have a problem of the form:
min cx; subject to: Ax <= b
First solve your problem with glpk, this will give you the optimal value V of the problem. From this point, you can use PPL to get the description of the polyedron of optimal values:
cx = V and Ax <= b
as the convex hull of its extreme points, which correspond to the BFSs you are looking for.
You can (probably) use the glpk simplex routines. Once you get an optimal BFS, you can get the reduced cost associated with all non-basic columns using the routine glp_get_row_dual (the basis status of the variable can be obtained with glp_get_row_stat), so you can find a non-basic variables with a null reduced cost. Then, I think that you can use function glp_set_row_stat to change the basis status of this column in order to have it enter the basis.
(And then, you can iterate this process as long as you avoid cycling.)
Note that I did not try any of those solutions myself; I think that the first one is by far the best, although it will require that you learn the PPL API. If you wish to go for the second one, I strongly suggest that you send an e-mail to the glpk maintainer (or look at the source code), because I am really not sure it will work as is.
I am trying to solve two point boundary problem with odeint. My equation has the form of
y'' + a*y' + b*y + c = 0
It is pretty trivial when I have boundary conditions of y(x_1) = y_1 , y'(x_2) = y_2, but when boundary conditions are y(x_1) = y_1 , y(x_2) = y_2 I am lost. Does anybody know the way to deal with problems like this with odeint or other scientific library?
In this case you need a shooting method. odeint does not have such a method, it solved the initial value problem (IVP) which is your first case. I think in the Numerical Recipies this method is explained and you can use Boost.Odeint to do the time stepping.
An alternative and more efficient method to solve this type of problem is finite differences or finite elements method. For finite differences you can check Numerical Recipes. For finite elements I recommend dealii library.
Another approach is to use b-splines: Assuming you do know the initial x0 and final xfinal points of integration, then you can expand the solution y(x) in a b-spline basis, defined over (x0,xfinal), i.e.
y(x)= \sum_{i=1}^n A_i*B_i(x),
where A_i are constant coefficients to be determined, and B_i(x) are b-spline basis (well defined polynomial functions, that can be differentiated numerically). For scientific applications you can find an implementation of b-splines in GSL.
With this substitution the boundary value problem is reduced to a linear problem, since (am using Einstein summation for repeated indices):
A_i*[ B_i''(x) + a*B_i'(x) + b*B_i(x)] + c =0
You can choose a set of points x and create a linear system from the above equation. You can find information for this type of method in the following review paper "Applications of B-splines in Atomic and Molecular Physics" - H Bachau, E Cormier, P Decleva, J E Hansen and F Martín
http://iopscience.iop.org/0034-4885/64/12/205/
I do not know of any library solving directly this problem, but there are several libraries for B-splines (I recommend GSL for your needs), that will allow you to form the linear system. See this stackoverflow question:
Spline, B-Spline and NURBS C++ library