I have a working ILP Gurobi model (exclusively binary variables). Reducing runtime and finding a feasible solution is of far more value to me than the optimal solution. Reducing my SolutionLimit to 1 does help. I realized that my objective function is summing up hundreds of thousands of variables together. If I don't truly care about optimality, can I somehow simplify my objective function to reduce the burden on the solver?
Here is my current objective function:
m.setObjective(quicksum(h[x,y,c,p,t] + v[x,y,c,p,t]
for x in range(0,Nx)
for y in range(0,Ny)
for c in range(0,C)
for p in range(0,P)
for t in range(0,T)), GRB.MINIMIZE)
I don't want to nitpick but there is no such thing as a "more optimal" solution - "optimal" is already the superlative. In case you are really only looking for a feasible solution without regard for the objective function, you should follow Erwin's advice and don't set an objective function at all. It's hard to believe, though, that your current objective function is completely meaningless, so a better approach is probably to reduce the objective function to include only a few variables and also to set a higher MIPGap to terminate the solve earlier.
Related
I have a set of data z(0), z(1), z(2)...,z(n) that I am currently fitting with a 2 variables polynomial of the kind p(x,y) = a(1)*x^2+a(2)*y^2+a(3)*x*y+a(4). I have i=1,...,n (x(i),y(i)) coordinates that I impose to be p(x(i),y(i))=z(i). In this way I have a Overdetermined System that I can solve using Eigen SVD . I am looking for a more sophisticated method that can take care of outliers, like a Least Median of Squares robust regression (as described here) but I haven't found a C++ implementation for 2 variables. I looked in GSL but it seems there is nothing for 2 variable functions. The only other solution I can think of is using a TGraph2D in ROOT. Do you know any other solution? Numerical recipes maybe? Since I am writing C++ code I would prefer C or C++ implementations.
Since non answer has been given yet, but I am still working on this problem, I will share my progresses here.
The class TLinearFitter has a fit method that allows you to select Robust fitting - Least Trimmed Squares regression (LTS):
https://root.cern.ch/root/html532/TLinearFitter.html
Another possible solution, more time consuming maybe, but maybe more efficient on the long run is to write my own function to be minimized, and the use:
https://projects.coin-or.org/Ipopt to minimize it. Although in this approach there is a bigger "step". I don't know how to use the library and I haven't (yet?) found a nice tutorial to understand it.
here: https://wis.kuleuven.be/stat/robust/software there is a Fortran implementation of the LMedS algorithm called PROGRESS. So another possible solution could be to port this software to C/C++ and make a library out of it.
I'm trying to wrap my head around how people that code math functions for games and rendering engines can use an optimized math function in an efficient way; let me explain that further.
There is an high need for fast trigonometric functions in those fields, at times you can optimize a sin, a cos or other functions by rewriting them in a different form that is valid only for a given interval, often times this means that your approximation of f(x) is just for the first quadrant, meaning 0 <= x <= pi/2 .
Now the input for your f(x) is still about all 4 quadrants, but the real formula only covers 1/4 of that interval, the straightforward solution is to detect the quadrant by analyzing the input and see in which range it belongs to, then you adjust the result of the formula accordingly if the input comes from a quadrant that is not the first quadrant .
This is good in theory but this also presents a couple of really bad problems, especially considering the fact that you are doing all this to steal a couple of cycles from your CPU ( you also get a consistent implementation, that is not platform dependent like an hardcoded fsin in Intel x86 asm that only works on x86 and has a certain error range, all of this may differ on other platforms with other asm instructions ), so you should keep things working at a concurrent and high performance level .
The reason I can't wrap my head around the "switch case" with quadrants solution is:
it just prevents possible optimizations, namely memoization, considering that you usually want to put that switch-case inside the same functions that actually computes the f(x), probably the situation can be improved by implementing the formula for f(x) outside said function, but this is will lead to a doubling in the number of functions to maintain for any given math library
increase probability of more branching with a concurrent execution
generally speaking doesn't lead to better, clean, dry code, and conditional statements are often times a potential source of bugs, I don't really like switch-cases and similar things .
Assuming that I can implement my cross-platform f(x) in C or C++, how the programmers in this field usually address the problem of translating and mapping the inputs, the quadrants to the result via the actual implementation ?
Note: In the below answer I am speaking very generally about code.
Assuming that I can implement my cross-platform f(x) in C or C++, how the programmers in this field usually address the problem of translating and mapping the inputs, the quadrants to the result via the actual implementation ?
The general answer to this is: In the most obvious and simplest way possible that achieves your purpose.
I'm not entirely sure I follow most of your arguments/questions but I have a feeling you are looking for problems where really none exist. Do you truly have the need to re-implement the trigonometric functions? Don't fall into the trap of NIH (Not Invented Here).
the straightforward solution is to detect the quadrant
Yes! I love straightforward code. Code that is perfectly obvious at a glance what it does. Now, sometimes, just sometimes, you have to do some crazy things to get it to do what you want: for performance, or avoiding bugs out of your control. But the first version should be most obvious and simple code that solves your problem. From there you do testing, profiling, and benchmarking and if (only if) you find performance or other issues, then you go into the crazy stuff.
This is good in theory but this also presents a couple of really bad problems,
I would say that this is good in theory and in practice for most cases and I definitely don't see any "bad" problems. Minor issues in specific corner cases or design requirements at most.
A few things on a few of your specific comments:
approximation of f(x) is just for the first quadrant: Yes, and there are multiple reasons for this. One simply is that most trigonometric functions have identities so you can easily use these to reduce range of input parameters. This is important as many numerical techniques only work over a specific range of inputs, or are more accurate/performant for small inputs. Next, for very large inputs you'll have to trim the range anyways for most numerical techniques to work or at least work in a decent amount of time and have sufficient accuracy. For example, look at the Taylor expansion for cos() and see how long it takes to converge sufficiently for large vs small inputs.
it just prevents possible optimizations: Chances are your c++ compiler these days is way better at optimizations than you are. Sometimes it isn't but the general procedure is to let the compiler do its optimization and only do manual optimizations where you have measured and proven that you need it. Theses days it is very non-intuitive to tell what code is faster by just looking at it (you can read all the questions on SO about performance issues and how crazy some of the root causes are).
namely memoization: I've never seen memoization in place for a double function. Just think how many doubles are there between 0 and 1. Now in reduced accuracy situations you can take advantage of it but this is easily implemented as a custom function tailored for that exact situation. Thinking about it, I'm not exactly sure how to implement memoization for a double function that actually means anything and doesn't loose accuracy or performance in the process.
increase probability of more branching with a concurrent execution: I'm not sure I'd implement trigonometric functions in a concurrent manner but I suppose its entirely possible to get some performance benefits. But again, the compiler is generally better at optimizations than you so let it do its jobs and then benchmark/profile to see if you really need to do better.
doesn't lead to better, clean, dry code: I'm not sure what exactly you mean here, or what "dry code" is for that matter. Yes, sometimes you can get into trouble by too many or too complex if/switch blocks but I can't see a simple check for 4 quadrants apply here...it's a pretty basic and simple case.
So for any platform I get the same y for the same values of x: My guess is that getting "exact" values for all 53 bits of double across multiple platforms and systems is not going to be possible. What is the result if you only have 52 bits correct? This would be a great area to do some tests in and see what you get.
I've used trigonometric functions in C for over 20 years and 99% of the time I just use whatever built-in function is supplied. In the rare case I need more performance (or accuracy) as proven by testing or benchmarking, only then do I actually roll my own custom implementation for that specific case. I don't rewrite the entire gamut of <math.h> functions in the hope that one day I might need them.
I would suggest try coding a few of these functions in as many ways as you can find and do some accuracy and benchmark tests. This will give you some practical knowledge and give you some hard data on whether you actually need to reimplement these functions or not. At the very least this should give you some practical experience with implementing these types of functions and chances are answer a lot of your questions in the process.
My professor gave me a binary linear programming problem, but this problem is slightly different from optimization problems I used to solve(i.e. this is probably not maximizing or minimizing the object function.)
The problem is as follows,
Given a matrix M, for entries m_ij != 0, there are corresponding x_ijk variables.
Entries m_ij = 0 can be ignored.
x_ijk is either 0 or 1, and I want to try 5 x_ijk variables for each m_ij (that is, x_ij1, x_ij2, x_ij3, x_ij4, and x_ij5. One of them is 1 and the others are 0) are enough to satisfy some conditions(a set of inequalities).
More simply, this is to check if the set of constraints involving 5 x_ijk variables for each m_ij is a valid(or feasible) constraints.
I have solved some optimization problems, but I have never solved a problem without an objective function.
What should I set as my objective function here?
0? nothing?
I might be using lp_solve or CPLEX.
Thank you in advance for your advice!
That is correct, you can set an arbitrary constant value as an objective function.
Most of the solvers I have tried allow an empty objective function. Simply leave it out from your model.
Depending on the solver and the API you are using, it can happen that you have to set the coefficients of all variables in the objective to zero.
Don't worry, it has to work.
In response to your comment: Yes, constraint programming tools can provide better performance on feasibility problems than LP solvers (such as CPLEX). I have played with the IBM ILOG CPLEX CP Optimizer a few months ago, it is free for Academic users. Both the LP solver and the CP solver failed on my problems. Don't expect a miracle from constraint programming.
Keep in mind the that time needed to solve a constraint program grows exponentially with the size of the problem in the worse case. Sooner or later, your problems will most likely become unsolvable with either tool.
Just for your information: in the end, the constraint programming solver will call the LP solver (for example CPLEX).
My advice is: try the tool you already have / use the problem formulation that is more natural to you. Check whether the tool can solve your problem. Switch tool only if the tool fails and you cannot improve your model.
I am doing (trying to do) numerical optimization in Fortran 90, on a Windows 7 machine with the gfortran compiler. I have a function, pre-written by someone else, which returns the loglikelihood of a function, given a large set of parameters (about 60 parameters in total) passed in. I am trying to replicate someone's results, so I know the final parameter values, but I was to try and re-estimate them and, eventually, extend their model and use different data. I've been trying the uobyqa.f90 routine available here, which has not been particularly successful thus far.
My questions are: First, for an optimization problem with a large number of parameters (over 60), can anyone suggest the best freely available routine? Derivatives are not available, and would be costly to estimate numerically, hence trying the uobyqa routine first. Also, would implementing parallelization aid significantly in solving this problem? And, if so, could anyone suggest an optimization routine that already implements parallelization using openmp?
Thanks!
I don't have a good suggestion for a specific optimization strategy, but the NLopt package has a few derivative-free optimizers that can handle larger numbers of variables. Worth checking out. I've found the Fortran interface to be very easy to use.
Do a regular (published academic) literature search on this question first.
Maybe try including "LAPACK" with your other search terms (e.g. "optimization", "uobyqa", etc) to see relavant work by other parties.
I am new to gecode and constraint programming in general.
So far, I haven't had much trouble picking up gecode, it's great. But I was wondering what is the best way to perform a "nested" cost function. Specifically, I am looking to minimize X, but within the space of solutions for which X is equal, prefer solutions which minimize Y? I could probably hack it by defining a cost function that looks like X*large_number+Y, but I'd prefer to do this properly if there's a good solution.
If anyone can point me to explain how to implement this in Gecode, that would be really helpful. Thanks!
You can define any kind of optimization criteria using the constrain member in a space in Gecode. See Section 2.5 in Modeling and Programming with Gecode for an example. In your case, the straight forward way would be to add a constrain member that adds a lexicographic constraint between the previous best solutions answer and the current space.
That being said, in general optimizing based on a lexicographic order can be wasteful (too much searching). It may often be better to first run a search optimizing the first component (X in your case). After that, re-run the search with the first components value fixed (X set to best possible value), and optimize the second value (Y in your case). Iterate as needed for all elements in the cost.