The Problem
I'm trying to use scikit-learn on a virtual environment and I cannot get it to run.
So, for example, I go to a project folder and install NumPy, SciPy, and scikit-learn
virtualenv venv
source venv/bin/activate
pip install numpy
pip install scipy
pip install scikit-learn
Then I open python and try to import each of them.
import numpy
import scipy
import sklearn
NumPy and SciPy import fine, but when I try to import I get an error saying Library not loaded: #rpath/./libgfortran.3.dylib.
Is there something I'm missing in the installation?
Full error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/sklearn/__init__.py", line 57, in <module>
from .base import clone
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/sklearn/base.py", line 11, in <module>
from .utils.fixes import signature
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/sklearn/utils/__init__.py", line 11, in <module>
from .validation import (as_float_array,
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/sklearn/utils/validation.py", line 16, in <module>
from ..utils.fixes import signature
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/sklearn/utils/fixes.py", line 324, in <module>
from scipy.sparse.linalg import lsqr as sparse_lsqr
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/scipy/sparse/linalg/__init__.py", line 109, in <module>
from .isolve import *
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
from .iterative import *
File "/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/scipy/sparse/linalg/isolve/iterative.py", line 7, in <module>
from . import _iterative
ImportError: dlopen(/Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/scipy/sparse/linalg/isolve/_iterative.so, 2): Library not loaded: #rpath/./libgfortran.3.dylib
Referenced from: /Users/Peter/git/dataSandbox/flaskApps/sfSalary/venv/lib/python2.7/site-packages/scipy/sparse/linalg/isolve/_iterative.so
Reason: image not found
EDIT:
this was some troubleshooting to host a Heroku app using sklearn.
I would strongly recommend against using virtual environments for NumPy/SciPy/Scikit-learn and other packages which include compiled code. Virtual environments deal with pure Python dependencies reasonably well, but for compiled extensions with dependencies outside Python, they can fail in unexpected ways (as you have seen).
If you want to manage multiple environments with different versions of Python packages including compiled extensions, the best tool out there is probably conda, and in particular conda environments.
With it, you can run the following:
$ conda create -n myenv python=3.4 numpy scipy scikit-learn
$ source activate myenv
(myenv) $ python
Python 3.4.3
>>> import sklearn
and you'll be good to go.
Related
I'm using spyder 2 anaconda 2.3.0 on windows as a python IDE for python 2.7. I've been using this for a long time to code in python but suddenly I don't know what happened and it said I did not have biopython package installed, when I installed biopython using
conda -c install anaconda biopython
command, biopython started running however it broke pandas and numpy.
now when I import pandas I get this:
>>> import pandas as pd
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Anaconda\lib\site-packages\pandas\__init__.py", line 13, in
<module>
"extensions first.".format(module))
ImportError: C extension: hashtable not built. If you want to import pandas
from the source directory, you may need to run 'python setup.py build_ext --
inplace' to build the C extensions first.
and then when I import numpy I get this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Anaconda\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import add_newdocs
File "C:\Anaconda\lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "C:\Anaconda\lib\site-packages\numpy\lib\__init__.py", line 8, in <module>
from .type_check import *
File "C:\Anaconda\lib\site-packages\numpy\lib\type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "C:\Anaconda\lib\site-packages\numpy\core\__init__.py", line 26, in <module>
raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: DLL load failed: The specified module could not be found.
I uninstalled numpy and pandas and then installed them back using pip ... still the same. I entirely unistalled anaconda and installed it back still the same errors. I really do not know what's causing this. Any help will be truly appreciated. I've been trying to fix this for the past three days and its not working out. Thanks for your time.
When I am trying to run python file .py that uses numba it gives me the below error
Traceback (most recent call last):
File "C:\Myfile.py", line 16, in <module>
import myfile2 as m
File "C:\myfile2.py", line 8, in <module>
import numba
File "C:\Python27\lib\site-packages\numba\__init__.py", line 9, in <module>
from . import config, errors, runtests, types
File "C:\Python27\lib\site-packages\numba\config.py", line 11, in <module>
import llvmlite.binding as ll
File "C:\Python27\lib\site-packages\llvmlite\binding\__init__.py", line 6, in <module>
from .dylib import *
File "C:\Python27\lib\site-packages\llvmlite\binding\dylib.py", line 4, in <module>
from . import ffi
File "C:\Python27\lib\site-packages\llvmlite\binding\ffi.py", line 50, in <module>
raise e
WindowsError: [Error 126] The specified module could not be found
I've tried to isntall numba from command propmt as below :
pip install numba
but it gives me a message that it already exists before
Requirements already satisfied...........etc
so could anyone help me to fix this error ? also How to install numba and link it with llvm and llvmlite ?
I've uninstall the python and re-install it again
and install the below packages in order
pip install scikit_learn
pip install numby+mkl
pip install scipy
pip install numba
pip install matplotlib
also I've installed mini conda and it is working fine as well
I want to make a python package with numpy, pandas, scipy and sklearn, so I can take it to any linux without install python, but i came across this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data1/sigmoidguo/TOOLS/python27/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/data1/sigmoidguo/TOOLS/python27/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/data1/sigmoidguo/TOOLS/python27/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/data1/sigmoidguo/TOOLS/python27/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/data1/sigmoidguo/TOOLS/python27/lib/python2.7/site-packages/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: libblas.so.3: cannot open shared object file: No such file or directory
How can I fix it without root permission?
PS: I don't have root permission, so I can't install python site-packages to python...
You can install anaconda. It is a multi-platform python distribution that can be installed in your home folder (with user rights). It comes with pip and conda commands to install any package you need. It already comes with all the packages you mention (numpy, pandas, scipy and sklearn), so sounds like a good fit for your needs.
Would http://python-xy.github.io/ do the trick?
(if it was windows I would suggest the portable http://winpython.sourceforge.net/)
I recently came across a blog(http://stronginference.com/ScipySuperpack/) on how to install sklearn . I successfully installed it and it was stored on the path:
/usr/local/lib/python2.7/site-packages/sklearn
I then went to the properties of my eclipse; under the Interpreter-Python and added the path to the PYTHONPATH. I could import sklearn but when I ran this simple code to test everything:
from sklearn import datasets
iris = datasets.load_iris()
digits = datasets.load_digits()
print digits.data
It threw the following errors:
Traceback (most recent call last):
File "/Users/licheng/Documents/workspaceII/SkLearnTutorial/Src/my_first_scilearn.py", line 6, in <module>
from sklearn import datasets
File "/usr/local/lib/python2.7/site-packages/sklearn/datasets/__init__.py", line 7, in <module>
from .base import load_diabetes
File "/usr/local/lib/python2.7/site-packages/sklearn/datasets/base.py", line 24, in <module>
from ..utils import check_random_state
File "/usr/local/lib/python2.7/site-packages/sklearn/utils/__init__.py", line 10, in <module>
from .murmurhash import murmurhash3_32
File "numpy.pxd", line 155, in init sklearn.utils.murmurhash (/private/var/folders/fv/vhs3w0zn2q961_gyp8f2z_rw0000gn/T/pip-EVsI8B-build/sklearn/utils/murmurhash.c:5029)
ValueError: numpy.dtype has the wrong size, try recompiling
any help on what could be wrong? I have been on these for hours now.
Thanks
Try uninstalling NumPy, SciPy and scikit-learn.
Then reinstall those packages.
If you are using Windows, binaries for those packages have been provided by Christoph Gohlke, which can be quite convenient. Just download them and then use pip to install.
Unofficial Windows Binaries for Python Extension Packages
I just upgraded to Scipy 0.14.0 and Ubuntu 14.04. Now when I try to import stats I get an ImportError.
import scipy.stats as stats
Traceback (most recent call last):
File "/home/user1/Documents/test/importTest/importScipy.py", line 1, in <module>
import scipy.stats as stats
File "/usr/local/lib/python2.7/dist-packages/scipy/stats/__init__.py", line 334, in <module>
from .stats import *
File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py", line 181, in <module>
import scipy.special as special
File "/usr/local/lib/python2.7/dist-packages/scipy/special/__init__.py", line 548, in <module>
from .basic import *
File "/usr/local/lib/python2.7/dist-packages/scipy/special/basic.py", line 17, in <module>
from . import orthogonal
File "/usr/local/lib/python2.7/dist-packages/scipy/special/orthogonal.py", line 90, in <module>
from scipy import linalg
File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/__init__.py", line 159, in <module>
from .misc import *
File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/misc.py", line 5, in <module>
from . import blas
File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/blas.py", line 145, in <module>
from scipy.linalg import _fblas
ImportError: /usr/local/lib/python2.7/dist-packages/scipy/linalg/_fblas.so: undefined symbol: clange_
Installing from source solved the issue.
First download the numpy and scipy tarballs.
Then locate the BLAS and LAPACK *.so files on my machine. I found them in /usr/lib/. If they are not there you can install them. I used the standard Ubuntu finder to locate them:
find -iname 'libblas.so'
find -iname 'liblapack.so'
So I set some environment variables for the installation.
export BLAS=/usr/lib/libblas.so
export LAPACK=/usr/lib/liblapack.so
Then I downloaded the source tarballs of numpy and scipy, untarred the tarball and then installed by cd into the install directories and
python setup.py build
sudo python setup.py install
NOTE: You may want to specify the install directory.
Comments welcome. Perhaps there is a better way.
The problem is related to Scipy missing proper reference to LAPACK. I followed the instructions suggested by #jtorca and found that you don't even need to install via tarballs. For me, the following did the trick:
apt-get install libopenblas-dev liblapack-dev
export BLAS=/usr/lib/libblas.so
export LAPACK=/usr/lib/liblapack.so
pip install numpy
pip install scipy