I got a Django project that was started without virtualenv. Now migrating to virtualenv and my requirements.txt created before is huge and not installable in the virtualenv (many errors as shown below). How can i generate a minimal list of required packages? Or is there some reference anywhere?
Could not find any downloads that satisfy the requirement PIL==1.1.7 (from -r requirements.txt (line 8))
Some externally hosted files were ignored (use --allow-external PIL to allow).
Cleaning up...
No distributions at all found for PIL==1.1.7 (from -r requirements.txt (line 8))
it's not only PIL that throws errors, if I comment it another package shows and i can't know what's really used for my application to work.
Thanks in advance!
You can run pip freeze (related to system python used before virtualenv), this give you list of installed packages;
Then filter that list using following:
1) INSTALLED_APPS in settings
2) also check all from and import statement (search through the project)
I've had the same problem and there are issues getting PIL to install using PIP as there's no PIL 1.7 in the default Python repos.
The easiest fix is this to add these options to the pip:
--allow-external PIL --allow-unverified PIL
For example:
pip install -r requirements.txt --allow-external PIL --allow-unverified PIL
The problem with this it is a potential security issue and you don't want to do this on a production server! :)
Your options are to use Pillow which is a fork of PIL:
https://pypi.python.org/pypi/Pillow
Comments from the Pillow author, and you should verify that it works with you code.
Or try PIL 1.1.6 which is the Python Repos:
https://pypi.python.org/pypi/PIL
Or create your own repo and include the PIL 1.1.7 sources.
Or, if your on a Linux system install PIL using your distro's package management tool and remove PIL from your requirements file, and then rebuild your virutalenv.
You can this on Debian based distros like this:
sudo apt-get install python-imaging
Red Hat distros like this:
sudo yum install python-imaging
Related
I have legacy production servers that are still running Python 2.7.6. We have a local environment built from the docker image for ubuntu 14.04 intended to replicate that environment (things still work there once everything is installed.) The packer build script that creates this environment recently stopped working apparently due to PyPi dropping non-SNI support.
I tried using get-pip.py from the docs to download pip:
wget -c https://bootstrap.pypa.io/pip/2.7/get-pip.py
python2 get-pip.py
This gives me the following warning:
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
/tmp/tmpBb3LJu/pip.zip/pip/_vendor/urllib3/util/ssl_.py:424: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
/tmp/tmpBb3LJu/pip.zip/pip/_vendor/urllib3/util/ssl_.py:164: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
ERROR: Could not find a version that satisfies the requirement pip<21.0 (from versions: none)
ERROR: No matching distribution found for pip<21.0
The proposed solution for that is to use pip to upgrade urllib3
https://serverfault.com/questions/866062/easy-install-and-pip-fail-with-ssl-warnings
I don't have pip so I installed a legacy version using
apt-get install python-pip
This installs pip 1.5.4
When I try to pip install "urllib3[secure]" I get the following:
Requirement already satisfied (use --upgrade to upgrade): urllib3[secure] in /usr/lib/python2.7/dist-packages
Installing extra requirements: 'secure'
Cleaning up...
If I try pip install "urllib3[secure]" --upgrade or pip install --index-url https://pypi.python.org/simple/ --upgrade pip I get:
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement urllib3[secure] in /usr/lib/python2.7/dist-packages
Downloading/unpacking urllib3[secure]
Cleaning up...
No distributions at all found for urllib3[secure] in /usr/lib/python2.7/dist-packages
Storing debug log for failure in /root/.pip/pip.log
(the pip message reflects pip, not urllib3[secure])
When I try to use pip 1.5 to install uWSGI
pip install uWSGI
I get the following:
Downloading/unpacking uWSGI
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement uWSGI
Cleaning up...
No distributions at all found for uWSGI
Storing debug log for failure in /root/.pip/pip.log
Upgrading pip doesn't work here either
Downloading/unpacking uWSGI==2.0.18 (from -r /root/requirements.txt (line 1))
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement uWSGI==2.0.18 (from -r /root/requirements.txt (line 1))
Cleaning up...
No distributions at all found for uWSGI==2.0.18 (from -r /root/requirements.txt (line 1))
Storing debug log for failure in /root/.pip/pip.log
Reinstalling pip doesn't work:
python -m pip install -U --force-reinstall pip
Gives me:
Downloading/unpacking pip
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pip
Cleaning up...
No distributions at all found for pip
Storing debug log for failure in /root/.pip/pip.log
If I open /root/.pip/pip.log I see the following:
Downloading/unpacking pip
Getting page https://pypi.python.org/simple/pip/
Could not fetch URL https://pypi.python.org/simple/pip/: 403 Client Error: [[[!!! BREAKING CHANGE !!!]]] Support for clients that do not support Server Name Indication is temporarily disabled and will be permanently deprecated soon. See https://status.python.org/incidents/hzmjhqsdjqgb and https://github.com/pypa/pypi-support/issues/978 [[[!!! END BREAKING CHANGE !!!]]]
The link says that SNI support was dropped:
For users of Python 2.7.{0...8}
Upgrading to the last Python 2.7 release is an option.
However, note that Python 2.7 series itself is now End of Life and support in pip was dropped with version 21.0.
For users of Python 2.6.x and lower:
Neither the Python core developers, or pip maintainers support Python 2.6 and below.
If someone is aware of a work around for this issue (SNI support specifically) they are welcome to share it here for others.
There is no recommended solution from the PyPI team.
How can I get a local environment set up for new developers to work on our legacy application? I've created a new Python 3 dev server and local environment but it will be some time before I can roll out the staging and live environments, get everything moved over, and test it.
As the message says, PyPi has discontinued support for Python <2.7.9 as of May 6th 2021. If you're running a version < 2.7.9 and you cannot upgrade to a newer version of Python then your only option is to manually download the wheels from PyPi.
These are the modification I needed to make to my build script to make it work:
I needed to install software-properties-common and gcc
apt-get install -y software-properties-common gcc
Then I downloaded (setuptools](https://pypi.org/project/setuptools/44.1.1/#files) and unzipped and installed it:
python ./setuptools-44.1.1/setup.py install
Next, I downloaded pip and added it to a folder called wheels. Then I could use the whl file to run pip to get pip
python ./wheels/pip-20.3.4-py2.py3-none-any.whl/pip install --no-index --find-links ./wheels/ pip --ignore-installed
It was suggested to build a Docker container using Ubuntu 16.04 with Python 2.7.17 and use that to download the packages.
pip download -r requirements.txt
But the versions of the packages were wrong, so I ended up going through the requirements.txt and downloading each package manually from PyPi and adding it to the wheels folder. A running instance is useful so you can run pip freeze or look at a requirements.txt file to grab the version numbers of all the packages you need.
Now that I could use pip, I can install my other packages:
python pip install --no-index --find-links ./wheels/ -r /root/requirements.txt
This uncovered some dependencies that I hadn't downloaded packages for yet so I had to go through and download those and added them to the wheels folder. There were a few other things I found needed different versions than I had originally downloaded and a few packages relied on pbr and many more wanted wheel:
pip install --no-index --find-links ./wheels/ pbr==5.5.1 wheel==0.36.2
I also needed to download cMake and add it to the wheels folder
After that I could install my requirements.txt:
pip install --no-index --find-links ./wheels/ -r /root/requirements.txt --ignore-installed
May be late to the party but something similar happened to me while trying to make an HTTPS request with Python 2.7.6 (lack of SNI support). This was causing a lot of issues on a remote web server I work on.
Looking for answers I tried installing urllib3[secure] and entered a loophole since pip was complaining about a lack of SNI support to install this and other packages as well.
I found out this StackOverflow answer which helped me install the required dependencies to make Python 2.7.6 and pip itself support SNI as well as install urllib[secure].
You need to create a folder containing the required wheels (download them from PyPi using wget for instance):
pip, asn1crypto, enum34, idna, six, ipaddress, pyOpenSSL, cffi,
cryptography wheels; and also pycparser (a non-wheel, it will be a
tar.gz)
Make sure the wheels you download support Python 2.7 and that you install pip before the rest of them.
In the original answer, its stated you can use python -m OpenSSL.debug to verify everything worked correctly (a ModuleNotFoundError would mean the pyOpenSSL package was not installed). You can also use pip -Vto check that the new pip version was installed correctly as well.
After updating pip and installing these dependencies I was able to install urllib3[secure] and get SNI support from python as well as pip.
Good luck!
I am sharing this answer as an update to Jonathan Rys's answer that contains the steps required as of the date of this answer. I tried to keep this concise.
As the message says, PyPi has discontinued support for Python <2.7.9 as of May 6th 2021. If you're running a version < 2.7.9 and you cannot upgrade to a newer version of Python then your only option is to manually download the wheels from PyPi.
For Ubuntu 20.04, I have installed build-essential sudo apt-get install build-essential
I installed Python 2 from source, downloading tar bundle and built and installed this. Note, I removed the python command, to avoid that old confusion, so we have python2 and python2.7 and also python3 (for example).
tar xf Python-2.7.18.tgz
cd Python-2.7.18
./configure && make && sudo make install
(cd /usr/local/bin;sudo rm python python-config)
cd ..
Now to get pip installed. Download setuptools zip archive. Then install it:
unzip setuptools-44.1.1.zip
cd setuptools-44.1.1
python2 bootstrap.py
sudo python2 setup.py install
cd ..
Next, I downloaded pip .tar.gz archive. Then unpack and install it. Note, I took extra steps to preserve and restore the original python3 pip in /usr/local/bin. Pip for Python2 are still available as pip2 and pip2.7 in the same directory.
tar xf pip-20.3.4.tar.gz
cd pip-20.3.4
(cd /usr/local/bin;sudo mv pip pip-save)
sudo python2 setup.py install
(cd /usr/local/bin;sudo mv pip-save pip)
Now that I could use pip, I can install the most important package any Python user should install, ipython. Note that I do a user install for this and also preserve my "ipython" command to run python3 and have ipython2 to run python2:
pip2.7 install ipython
(cd ~/.local/bin;rm ipython;ln -s ipython3 ipython)
and it works!
$ ipython2
Python 2.7.18 (default, Jun 22 2022, 09:38:45)
Type "copyright", "credits" or "license" for more information.
IPython 5.10.0 -- An enhanced Interactive Python.
Have found a similar issue, however haven't found proper solution.
Here's a code:
import matplotlib.pyplot as plt
plt.plot([1,2,3],[4,2,5])
plt.show()
Run, got the message:
ImportError: No module named 'matplotlib.pyplot'; 'matplotlib' is not a package
I run Linux Mint 18 with preinstalled python-2.7 and python-3.5 (I use python3), before that I was installing modules with a simple sudo apt-get install method and that worked great.
Before running this the code above, I've installed matplotlib in a usual way sudo apt-get install python-matplotlib. As it haven't worked out, started to look for solution.
Python location
which python3 /usr/bin/python3
Current Matplotlib installed
sudo find /usr | grep matplotlib /usr/lib/python3/dist-packages/matplotlib
My tries:
1) I've removed matplotlib with autoremove, and tried to make it sudo apt-get install python3-matplotlib instead. Didn't worked out.
2) Used: pip3 install matplotlib or sudo pip3 install matplotlib. Received errors like:
command python setup.py egg_info failed with error code 1 in /tmp/pip-build- ....
3) Then I found another solution:
sudo apt-get install virtualenv
virtualenv -p /usr/bin/python3 py3env
source py3env/bin/activate
pip install matplotlib
Same outcome.
Haven't tried to use import sys sys.path.append('/usr/lib/pymodules/python2.7/')(proposed in link above), but as I am not sure what exactly this command does (quite a newbie to python and programming itself) - haven't risked.
If you are using pycharm and have matplotlib.py in your current working directory than you get this error. Just delete or rename the matplotlib.py file and it will work.
don't name any file as matplotlib.py within your working directory
In your working directory, check if there is any file matplotlib.py
Delete that file and import matplotib again. That should work.
simply install:
python -m pip install -U pip
python -m pip install -U matplotlib
I had the same problem reasons in my case were
python 3.8.2 with matplotlib that I downloaded was for 3.7. Make sure they are the same.
python 64 bits version with matplotlib 32bits. Make sure they are the same
Use python -m pip install package_which_you_need to install packages for Windows
Make sure to add to PATH the environment variables I forgot to do in my case
pip version was old use
Use python -m pip install --upgrade pip to upgrade pip to the latest for Windows
I saved the file name as matplotlib.py. Try to avoid that
Finally I typed matplotlib.pyplot as matplotlib.plyplot remember to check for typos first. The error message looks similar even though I corrected all the steps above.
ModuleNotFoundError: No module named 'matplotlib.pyplot'
ModuleNotFoundError: No module named 'matplotlib.plyplot'
I'm trying to get up and running with sorl thumbnail but I'm getting an error "decoder zip not available".
I have read through a ton of similar pages saying that it is a PIL / Pilow issue.
I have tried re-installing pil via:
easy_install http://dist.plone.org/thirdparty/PIL-1.1.7.tar.gz
As well as:
pip uninstall PIL Pillow
pip install Pillow
pip install PIL
I read that Ubuntu uses a directory called /lib/x86_64-linux-gnu for x64 architectures. If you are using that architecture you need to create a symbolic link for that as well as for other shared libraries.
$ sudo ln -s /lib/x86_64-linux-gnu/libz.so.1 /lib/libz.so
Additional info:
OS: Ubuntu Precice
I have not tried installing the package zlib from source which some others have also mentioned worked for them.
I found two solutions for this that worked for me.
The first way that worked for me was to use a different imaging library all together. I installed pgmagic and that worked fine.
sudo apt-get install libgraphicsmagick++-dev
sudo apt-get install libboost-python1.40-dev
There is also other imaging libraries available and they are all listed in the sorl-thumbnail docs:
http://sorl-thumbnail.readthedocs.org/en/latest/requirements.html
The second way that worked for me (preferred) was literally a fresh install of everything where I removed both pillow and PIL and the re-installed them starting with pillow.
pip uninstall pillow
pip uninstall PIL
pip install pillow
pip install PIL
I'm trying to install django-nonrel the correct way - and to be able to reproduce the process.
I've installed django-nonrel using pip - as following:
pip install git+https://github.com/django-nonrel/django-nonrel.git
pip install git+https://github.com/django-nonrel/django-dbindexer.git
pip install git+https://github.com/django-nonrel/django-permission-backend-nonrel
pip install hg+https://bitbucket.org/wkornewald/djangoappengine
pip install hg+https://bitbucket.org/wkornewald/djangotoolbox
pip install hg+https://bitbucket.org/twanschik/django-autoload
pip install hg+https://bitbucket.org/twanschik/nonrel-search/src
After installation, I got this req.txt file (pip freeze > req.txt):
Django==1.3.1
django-autoload==0.01
django-dbindexer==0.3
djangoappengine==1.0
djangotoolbox==0.9.2
nonrel-search==0.1
permission-backend-nonrel==0.1
wsgiref==0.1.2
But I can't use my req.txt file to get the same stuff.
If I uninstall a package (e.g. django-autoload) and try to get it again using the requirements file
(gae-first)bentzy#lama:~/.virtualenvs/gae-first$ pip uninstall django-autoload
Uninstalling django-autoload:
...
Successfully uninstalled django-autoload
(gae-first)bentzy#lama:~/.virtualenvs/gae-first$ pip install -r req.txt
Requirement already satisfied (use --upgrade to upgrade): Django==1.3.1 in ./lib/python2.7/site-packages (from -r req.txt (line 1))
Downloading/unpacking django-autoload==0.01 (from -r req.txt (line 2))
Could not find any downloads that satisfy the requirement django-autoload==0.01 (from -r req.txt (line 2))
No distributions at all found for django-autoload==0.01 (from -r req.txt (line 2))
Storing complete log in /home/bentzy/.pip/pip.log
Why aren't those packages at pip repository?
It still make sense to use pip to install them?
The problem is that your requirements file does not have enough information.
What pip is going to do when you request it to install django-autoload, for instance, is look at PyPI for that package (and scrap some pages after finding the PyPI entry).
If you want to have a requirements file that downloads those packages the same way you did while installing one by one, do the same: tell pip where to find packages.
Create a requirements file like:
git+https://github.com/django-nonrel/django-nonrel.git
git+https://github.com/django-nonrel/django-dbindexer.git
git+https://github.com/django-nonrel/django-permission-backend-nonrel
hg+https://bitbucket.org/wkornewald/djangoappengine
hg+https://bitbucket.org/wkornewald/djangotoolbox
hg+https://bitbucket.org/twanschik/django-autoload
hg+https://bitbucket.org/twanschik/nonrel-search/src
Or if you want to install from specific tag or commit, do:
git+https://github.com/django-nonrel/django-nonrel.git#1.3.1#egg=Django
Read more about requirements file at http://www.pip-installer.org/en/latest/logic.html#requirements-file-format
it doesn't really make sense to use pip if you're using GAE, since all the packages you use need to be in your actual GAE project folder. Packages installed in your system or virtualenv environments won't get uploaded to GAE production servers.
Currently I'm installing PIL into my virtual env as follows:
pip install -E . -r ./releases/%s/requirements.txt
where requirements.txt contains:
pil
I can upload png images but not jpeg images currently. From reading on the web it seems i may need libjpeg decoder? Am i installing pil incorrectly? What is the proper way to install pil for django in a virtual env with libjpeg?
You should install the libraries that others recommended but most importantly you should tell PIL where to find them. Edit the setup.py so that
JPEG_ROOT = None
becomes
JPEG_ROOT = libinclude("/usr/lib")
I found that the easiest way was to download the source with pip but not install:
pip install --no-install PIL
edit the setup (inside the build directory of the virtual environment) and the install
pip install PIL
you can find some more information in my blog
You can also try pillow which seems to do great job with little hassle (pip install pillow)
On Ubuntu precise, PIL doesn't find the jpeg library files, even once they are installed. The easiest way to fix this is to make a symlink after you have installed the jpeg dev package.
So, I needed an extra step:
pip uninstall PIL
sudo apt-get install libjpeg8-dev
sudo ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib
pip install PIL
For Ubuntu 11.04, what finally worked for me is:
pip uninstall PIL
sudo apt-get install libjpeg8-dev
pip install PIL
The Python Imaging Library (PIL) seems really picky about version and location of the jpeg libraries. And because PIL is written in C and compiled, you need the development versions of the library in addition to the runtime versions.
The situation is so bad the community forked PIL to create a softer version: Pillow:
http://pypi.python.org/pypi/Pillow#why-a-fork
On OSX, I used the following binary to get libpng and libjpeg simultaneously installed systemwide:
libpng & libjpeg for OSX
Because I already had PIL installed (via pip on a virtualenv), I ran:
pip uninstall PIL
pip install PIL --upgrade
This resolved the decoder JPEG not available error for me.
You must install the libraries:
sudo aptitude install libjpeg62 libjpeg62-dev zlib1g-dev
if pip raises an error, try
easy_install PIL