Pyomo can't locate GLPK solver - python-2.7

I'm trying to use the GLPK solver with Pyomo. I have a working model that's been tested, but keep getting an error saying GLPK can't be found.
WARNING: Could not locate the 'glpsol' executable, which is required for solver 'glpk'
I've installed glpk sucessfully. I also added the directory to my path variable so the executed can be called globally. I tested this with glpsol --help from my command line, and see the help info printed.
The below thread says it should be working, but alas, it is not.
How do you install glpk-solver along with pyomo in Winpython
Any ideas?

This answer is late but I want to share the solution that worked for me.
solvername='glpk'
solverpath_folder='C:\\glpk\\w64' #does not need to be directly on c drive
solverpath_exe='C:\\glpk\\w64\\glpsol' #does not need to be directly on c drive
I used to do this:
sys.path.append(solverpath_folder)
solver=SolverFactory(solvername)
This works for the cbc solver in coin-or but it does not work for glpk. Then I tried something different:
solver=SolverFactory(solvername,executable=solverpath_exe)
This worked for both cbc and glpk. No idea why this works (I really didn't do anything else).
Version: Python 2.7 or Python 3.7 (tested both), glpk 4.65

You can install glpk solver using this command -
brew install glpk

Installing the glpk package worked for me. As I use Anaconda:
conda install -c conda-forge glpk
This was after already including the 'glpsol' exectuable's folder path in my PATH variables.

So it looks like the set path variable is not handled by your Python installation.
A normal Python installation is set up for a seperated "PYTHONPATH" environment variable to look up additional modules.
There is also the option to make an entry in the windows registry or (like you already mentioned) move the files to the Python home directory, which is recognized relative to your installation directory if "PYTHONHOME" is not set.
More information in the Python Documentary under 3.3.3.
https://docs.python.org/2/using/windows.html#finding-modules

I was having the same issue. I don't know if this is a solution but it definitely got the solver working.
After downloading the Windows installation. I copied all the files in the w64 folder and pasted them directly into my Python working directory.
After that my python code could locate the solver.
NOTE: this was for Python 3.4.3.4, Windows 8.1 64 bit

Reading the source code here suggests you try:
from pyutilib.services import register_executable, registered_executable
register_executable(name='glpsol')
maybe will it give a clue

I had the same issue on Windows 10 and it was down to glpk being installed in a different conda environment. Full steps for installing pyomo and glpk below.
Test the installation by running 'Repeated Solves' example from https://pyomo.readthedocs.io/en/latest/working_models.html
Instructions (run at an anaconda prompt)
conda create --name myenv
conda activate myenv
conda install -c conda-forge pyomo
conda install -c conda-forge pyomo.extras
conda install -c conda-forge glpk
Run spyder from myenv so that if finds everything
spyder activate myenv

Here is the relevant part where pyomo 6.2 searches for the glpsol executable
https://github.com/Pyomo/pyomo/blob/568c6595a56570c6ea69c3ae3198b73b9f473abd/pyomo/common/fileutils.py#L288
def _path():
return (os.environ.get('PATH','') or os.defpath).split(os.pathsep)
There are two options to solve the PATH problem:
Putting the executable in an available folder in PATH (recommended practice). The glpsol executable must be in one of the folders present in the PATH system environment variable. Use in your code print(os.environ['PATH']) to see the available folders and put it there.
Adding the folder to PATH at runtime. You can add it to the system PATH statically or use code to add it dynamically (only while your script is running):
GLPK_FOLDER_PATH = "path/to/glpk"
os.environ["PATH"] += os.pathsep + str(GLPK_FOLDER_PATH)
In my case, my Python project has a virtual environment .venv, and I have an installation process that pastes the files essential to the glpsol executable when I install the project inside the .venv/Scripts folder. Because that folder is added automatically to the system PATH when Python is called from the virtual environment, libraries like Pyomo can find it. And I don't have to remember to add the folder to PATH at runtime whenever I want to use Pyomo.

For anyone that has the same issue, I found a workaround (not a solution!). I copied all the glpk files into my C:/Python27 directory, and (Surprise!) Python can now find them.
I'll hold out for a real solution before accepting this one.

Related

Global name 'col2im_6d_cython' is not defined, CS231n

I'm following CS231n and met a problem when doing assignment2: ConvolutionalNetworks: global name 'col2im_6d_cython' is not defined.
I think the problem was due to a failure in importing functions from im2col_cython.pyx, which used cython.
I've installed Xcode 7.3.1, as shown below, but the problem was still not solved.
I'm running the ipynb files in Jupyter from Anaconda. There is a related discussion on reddit, but unfortunately the solution here was for Windows, not Mac OS X.
Thank you for your time.
I wanted to add my input as a comment but didn't have enough reputation points to do so.
The issue was resolved for me when I closed the jupyter notebook and opened it again. I compiled the cython extension after I got the import error and probably have to relaunch it when the .so file is available.
I solved this with 2 easy steps:
In the terminal, run python setup.py build_ext --inplace in the cs231n directory.
Then reopen the notebook (if necessary, shutdown the notebook, the open it again);
Ps.: I tried this through the notebook using !python ./cs231n/setup.py build_ext --inplace as well. It does not work! You have to that outside the notebook, using the terminal.
I had this problem recently. I googled it a lot and also tried un-/-reinstall Anaconda. However it does work further. So I used "which python" to figure out which python's been using. And it turns out that the python included in Anakonda directory is used as default. I then tried the python2.7 in my macOS, which is located in /usr/bin/python2.7. Although I got a couple of warnings, but now it works likes charm. Perhaps it's kind of version problem. Solved in macOS Sierra 10.12.4.
I compared the two compiling results, as it shows that the include files are totally different. The one in Anaconda includes all header files in python3.6. Instead we need here corresponding header files in python2.7(I suppose). As the red circles point out.
enter image description here
Supported by python 3 onwards. Go to setup_googlecloud.sh and change the line
virtualenv .env
to
virtualenv -p python3 .env and run the setup again as explained in assignment1 setup..
Works well after that..
It occurred to me as well.
My problem:
I saw that the extention file that is created is named "im2col_cython.cp37-win_amd64.pyd" and the import is looking for im2col_cython alone, so I changed the file name to "im2col_cython.pyd" and ran the setup script again.
Now when I ran the code in the notebook it found the module but it said that the dll was compiled with a different python version. I use Anaconda envs and it turns out that since I ran the setup script from the cmd, it used a different python version than the one of the environment. I deleted the created files from the cs231n directory (im2col_cython.cp37-win_amd64.pyd and im2col_cython.c) and ran the setup script again, this time from the env Anaconda Prompt and it worked.
Solution:
remove already created files (the .c and .pyd files)
run the setup
script from the environment prompt (not plain cmd)
change the .pyd
file name into im2col_cython.pyd
Enjoy!

Using IPOPT with Openmdao (or pyoptsparse) in python

Hello everyone. I have a little problem : I am working with openmdao and pyOptSparseDriver. It is working for some solvers (SLSQP, PSQP for instance) so there is no problem about that installation.
Now i'd like to try the same with IPOPT, but the code is not given with pyoptsparse. I followed the COIN-OR documentation to install IPOPT (http://www.coin-or.org/Ipopt/documentation/node10.html), and everything SEEMS ok (i don't know how to check it). Now i have a COIN-OR folder on my desk and i don't know how to make pyoptsparse take IPOPT from it (it is the line "from . import pyoptcore" that crashes), from pyIPOPT/pyIPOPT.py.
Can anyone help me ?
Thank you by advance
we're using IPOPT with OpenMDAO through the pytoptsparse package, and installed it as a standalone library, like you've done. To get that to work you need to set the IPOPT_DIR environment variable pointing to the location you installed IPOPT before running the install of pyoptsparse.
So assuming you installed IPOPT in e.g. /usr/local/IPOPT:
$ export IPOPT_DIR=/usr/local/IPOPT
$ cd /path/to/pyoptsparse/
$ python setup.py install
this should result in pyoptsparse compiling the python wrapper for IPOPT and produce the file pyoptcore.so that will be placed in Python's site-packages/pyoptsparse/pyIPOPT.
On our cluster we had to modify the pyoptsparse/pyIPOPT/setup.py file slightly since we compiled everything with Intel, but if you compile with gfortran the official version of pyoptsparse should work for you.
Alternatively, you can do like Justin suggests, which is essentially the instructions you find in the pyoptsparse docs.
you have to put the ipopt source code into the src folder of pyopt-sparse and then recompile the package.

Use Python libraries without installing them

I recently completed a school project in Python, and I used some non-included libraries like numpy and nltk.
The problem is I am required to demo the projects on school computers (not my own laptop) and the school computers run Python but don't allow me to install any additional packages (so no pip install numpy)
Is there any way I can include these libraries on my USB and help Python find them so my program can still run?
You can do this by downloading to folders and setting the PYTHONPATH before executing.
https://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH
Also, to get the dependencies on there in a convenient manner (other than copying files etc.), you can give pip install options to specify a prefix while installing. See Install a Python package into a different directory using pip? for more details on this.

Unable to import lib tiff after installing Anaconda on Mac OS X 10.9.2

I have developed some software using Python under Windows 7.
I have given it to a colleague to run on a Mac (OS X 10.9.2). I have never used a Mac and am having trouble helping them to get started. I have downloaded and installed Anaconda 1.9.2 on the Mac. According to the continuum documentation, libtiff is included, but when I run my python file using the Spyder IDE I get the following error when it tries to import libtiff:
ImportError: No module named libtiff.
Following one of the answers on Stack Ooverflow, I tried:
conda install libtiff
This runs and returns:
All requested packages already installed.
However on Windows 7 I can see a libtiff folder under \python27\lib\site-packages. On the Mac there is no libtiff folder under /lib/python2.7/site-packages.
Can anyone tell me what I am missing?
This question is answered here:
Installing Python modules with Anaconda or Canopy
If pip install libtiff does not work, you can download the source for PyLibTiff as directed at https://code.google.com/p/pylibtiff/source/checkout and run setup.py with whichever interpreter you would like for PyLibTiff to be installed to.
Also, you do not have to have the the C libraries that Anaconda installs installed for PyLibTiff to work if you have libtiff libraries installed elsewhere.
Unclear on this. But what you can do to begin with is to type echo $PATH from the terminal and see what paths are set. Unclear on how Anaconda interacts with the system, but a good hunch is that if the library file is not in a path then that would cause this.
Also, looking at this thread on Google Groups it seems that Anaconda installs it’s own libraries that might need to be symbolically linked into the main /usr/local/lib directory. The user Denis Engemann—who made the post—posts this bash script in the last response in the thread:
for lib in ~/anaconda/lib/*;
do
ln -s $lib /usr/local/lib/$(basename $lib);
done
I would recommend checking those two directories first before linking to make sure all is as expected.

Install Spatialite for python (GeoDjango) on OS X

I am tearing my hair out trying to install Spatialite for GeoDjango!
I am already using Homebrew, it's generally easy and convenient so I initially tried to follow the Homebrew instructions for GeoDjango.
But this stops short of installing any database, i.e. Spatialite. The next step is to try and install Spatialite itself, but there are no Homebrew-specific instructions provided by Django docs.
I found this tutorial which looks perfect - a Homebrew and virtualenv-friendly install of Spatialite for GeoDjango.
But it doesn't work... it appears that my pysqlite is linked against the non-spatial-enabled version of SQLite that comes with OS X, rather than the Spatial-ised one I installed from Homebrew, I get this error when Django tried to connect to the db:
"The pysqlite library does not support C extension loading. Both SQLite and pysqlite must be configured to allow the loading of extensions to use SpatiaLite."
The author of pysqlite hasn't responded to my pleas for help on Github and I haven't found anything via Google.
So I went back to the drawing board and decided to follow the "Mac OS X-specific instructions" in the GeoDjango docs... by installing the various geo libs from the KyngChaos binary packages.
The docs say "Install the packages in the order they are listed above" but I found I couldn't install UnixImageIO without installing PROJ first. The link in the docs to download Spatialite binaries (http://www.gaia-gis.it/spatialite-2.3.1/binaries.html) is broken so I used the "Spatialite Tools v4.1" from KyngChaos instead.
Proceeding to the next step I get this error:
$ spatialite geodjango.db "SELECT InitSpatialMetaData();"
SQLite header and source version mismatch
2013-10-17 12:57:35 c78be6d786c19073b3a6730dfe3fb1be54f5657a
2013-09-03 17:11:13 7dd4968f235d6e1ca9547cda9cf3bd570e1609ef
Not really sure what's wrong at this point.
There is someone else here on SO who has gone the KyngChaos route and just ends up with the same "Both SQLite and pysqlite must be configured to allow the loading of extensions" error I got from the Homebrew route anyway.
I found this ticket #17756 for adding pyspatialite support to Django - pyspatialite is supposed to be an easier way to pip install everything but unfortunately it doesn't work either (see comments towards bottom of ticket).
I'm a bit reluctant to start trying to build everything from source by hand as it seems likely I'll just run into the same problems again, but spending hours Googling for info about cryptic compiler errors, magic flags and paths etc along the way.
I'm about ready to give up and just use Postgres/PostGIS.
I was able to get this working now, using the tip here:
https://github.com/ghaering/pysqlite/issues/60#issuecomment-50345210
I'm not sure if it was using the real paths that fixed it, or just the Homebrew kegs or underlying packages have been updated and now install cleanly. Still, it works now.
I reproduce below the steps I took:
brew update
brew install sqlite # 3.8.5
brew install libspatialite # 4.2.0
brew install spatialite-tools # 4.1.1
git clone https://github.com/ghaering/pysqlite.git
cd pysqlite
(where brew reported I had existing versions I unlinked them and installed the latest as commented above)
then edited setup.cfg to comment out #define=SQLITE_OMIT_LOAD_EXTENSION and specify the paths:
include_dirs=/usr/local/opt/sqlite/include
library_dirs=/usr/local/opt/sqlite/lib
activated the virtualenv where I want it installed, then
python setup.py build
python setup.py install
(build_static still fails with clang: error: no such file or directory: 'sqlite3.c')
(maybe I should have done pip install . as suggested in the github issue)
now the spatialite geodjango.db "SELECT InitSpatialMetaData();" succeeds, albeit with an ignorable error:
InitSpatiaMetaData() error:"table spatial_ref_sys already exists"
i.e. it's probably not even necessary to run that command
When I was istalling this i follow this instructions https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/spatialite/#pysqlite2
pysqlite2
If you’ve decided to use a newer version of pysqlite2 instead of the sqlite3 Python stdlib module, then you need to make sure it can load external extensions (i.e. the required enable_load_extension method is available so SpatiaLite can be loaded).
This might involve building it yourself. For this, download pysqlite2 2.6, and untar:
$ wget https://pypi.python.org/packages/source/p/pysqlite/pysqlite-2.6.3.tar.gz
$ tar xzf pysqlite-2.6.3.tar.gz
$ cd pysqlite-2.6.3
Next, use a text editor (e.g., emacs or vi) to edit the setup.cfg file to look like the following:
[build_ext]
#define=
include_dirs=/usr/local/include
library_dirs=/usr/local/lib
libraries=sqlite3
#define=SQLITE_OMIT_LOAD_EXTENSION
I had the same error: SQLite header and source version mismatch.
For me it was enough to update libsqlite3-dev.
After that invoking $ spatialite geo.db "SELECT InitSpatialMetaData();" creates proper database.