I wrote a lambda function that starts like this:
from paramikoimport SSHClient, AutoAddPolicy, RSAKey
I am trying to import this module as a layer, but it still does not work. It keeps showing me that there is no module named paramiko.
I followed this guide: Guide
I execute this command:
pip3 install paramiko -t .
into the python/ folder and then compress everything and load it onto a layer.
Do you have any ideas to suggest to me? Do you see anything wrong with this procedure?
EDIT:
pip3 install command:
pip3 install paramiko -t .
Collecting paramiko
Using cached paramiko-2.11.0-py2.py3-none-any.whl (212 kB)
Collecting cryptography>=2.5
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
Collecting bcrypt>=3.1.3
Using cached bcrypt-4.0.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (594 kB)
Collecting pynacl>=1.0.1
Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pycparser, cffi, cryptography, bcrypt, pynacl, six, paramiko
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
boto3 1.24.38 requires botocore<1.28.0,>=1.27.38, but you'll have botocore 1.23.54 which is incompatible.
awsebcli 3.20.3 requires six<1.15.0,>=1.11.0, but you'll have six 1.16.0 which is incompatible.
Successfully installed bcrypt-4.0.0 cffi-1.15.1 cryptography-37.0.4 paramiko-2.11.0 pycparser-2.21 pynacl-1.5.0 six-1.16.0
Related
I am still using python27.
When I upgrade my boto installation, I got the following output:
$ pip install --upgrade boto3
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting boto3
Downloading https://files.pythonhosted.org/packages/3a/47/c7c92c453593a7b2a062bde3c5f714a4c5c12763f97b8b9c7a7480932bb1/boto3-1.9.167-py2.py3-none-any.whl (128kB)
|████████████████████████████████| 133kB 7.6MB/s
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3)
Using cached https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from boto3)
Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting botocore<1.13.0,>=1.12.167 (from boto3)
Downloading https://files.pythonhosted.org/packages/7b/00/8437c07663969bd219aab33299f17b9d0ecd82622f4e19f482483efbfc6d/botocore-1.12.167-py2.py3-none-any.whl (5.5MB)
|████████████████████████████████| 5.5MB 5.6MB/s
Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" (from s3transfer<0.3.0,>=0.2.0->boto3)
Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting urllib3<1.26,>=1.20; python_version == "2.7" (from botocore<1.13.0,>=1.12.167->boto3)
Using cached https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1; python_version >= "2.7" (from botocore<1.13.0,>=1.12.167->boto3)
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting docutils>=0.10 (from botocore<1.13.0,>=1.12.167->boto3)
Using cached https://files.pythonhosted.org/packages/50/09/c53398e0005b11f7ffb27b7aa720c617aba53be4fb4f4f3f06b9b5c60f28/docutils-0.14-py2-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore<1.13.0,>=1.12.167->boto3)
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: futures, urllib3, jmespath, six, python-dateutil, docutils, botocore, s3transfer, boto3
Successfully installed boto3-1.9.167 botocore-1.12.167 docutils-0.14 futures-3.2.0 jmespath-0.9.4 python-dateutil-2.8.0 s3transfer-0.2.1 six-1.12.0 urllib3-1.25.3
As you can see it installed boto3-1.9.167 botocore-1.12.167
So boto is of the latest version at the time of writing this question: https://github.com/boto/boto3/releases/tag/1.9.167
However I have trouble locating any release note. I just want to make sure AWS Sign v4 is supported and used in this version of boto.
How can I verify?
The botocore changelog indicates that from version 1.7.51, all s3 requests use sigv4 by default. Since boto3 is based on botocore you can safely assume that sigv4 is supported by the versions you downloaded.
I'm a newbie in Django and I've been trying to setup pylint in my virtual environment on Vscode. During the installation I made sure that I was in my virtual environment as well as had the virtual environment python interpreter selected.
Method used to install :
/home/nived/Documents/BTRE_PROJECTF/venv/bin/python -m pip install -U pylint
This is the log if it does matter :
Collecting pylint
Using cached https://files.pythonhosted.org/packages/a5/06/ecef826f319055e6b231716730d7f9047dd7524ffda224b521d989f085b6/pylint-2.2.2-py3-none-any.whl
Collecting astroid>=2.0.0 (from pylint)
Using cached https://files.pythonhosted.org/packages/fc/53/8809bc008bad0300897281a7b320b286dc0e84e836396c0cff6279841e8a/astroid-2.1.0-py3-none-any.whl
Collecting isort>=4.2.5 (from pylint)
Using cached https://files.pythonhosted.org/packages/1f/2c/22eee714d7199ae0464beda6ad5fedec8fee6a2f7ffd1e8f1840928fe318/isort-4.3.4-py3-none-any.whl
Collecting mccabe (from pylint)
Using cached https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Collecting typed-ast; python_version < "3.7" and implementation_name == "cpython" (from astroid>=2.0.0->pylint)
Using cached https://files.pythonhosted.org/packages/de/50/7571200eff27c5c30dafa595e355495e1de85aad5fa3fe4c206791d827ff/typed_ast-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting lazy-object-proxy (from astroid>=2.0.0->pylint)
Using cached https://files.pythonhosted.org/packages/65/1f/2043ec33066e779905ed7e6580384425fdc7dc2ac64d6931060c75b0c5a3/lazy_object_proxy-1.3.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting wrapt (from astroid>=2.0.0->pylint)
Collecting six (from astroid>=2.0.0->pylint)
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: typed-ast, lazy-object-proxy, wrapt, six, astroid, isort, mccabe, pylint
Successfully installed astroid-2.1.0 isort-4.3.4 lazy-object-proxy-1.3.1 mccabe-0.6.1 pylint-2.2.2 six-1.12.0 typed-ast-1.1.1 wrapt-1.10.11
settings.json in the .vscode file contains the following code :
{
"python.pythonPath": "venv/bin/python"
}
Despite all this. The linter does not work and I'm not notified of any errors.
I've found what went wrong, had to install pylint for Django
pip install pylint-django
I installed python 2.7 in Windows 10, and set HOME, PATH environment variables.
Then typed
pip install jupyter
The result is like this:
Collecting jupyter
Using cached jupyter-1.0.0-py2.py3-none-any.whl
Collecting ipykernel (from jupyter)
Using cached ipykernel-4.5.2-py2.py3-none-any.whl
Collecting jupyter-console (from jupyter)
Using cached jupyter_console-5.0.0-py2.py3-none-any.whl
Collecting ipywidgets (from jupyter)
Using cached ipywidgets-5.2.2-py2.py3-none-any.whl
Collecting nbconvert (from jupyter)
Using cached nbconvert-5.0.0-py2.py3-none-any.whl
Collecting notebook (from jupyter)
Using cached notebook-4.3.1-py2.py3-none-any.whl
Collecting qtconsole (from jupyter)
Using cached qtconsole-4.2.1-py2.py3-none-any.whl
Collecting tornado>=4.0 (from ipykernel->jupyter)
Using cached tornado-4.4.2.tar.gz
Collecting jupyter-client (from ipykernel->jupyter)
Using cached jupyter_client-4.4.0-py2.py3-none-any.whl
Collecting ipython>=4.0.0 (from ipykernel->jupyter)
Using cached ipython-5.1.0-py2-none-any.whl
Collecting traitlets>=4.1.0 (from ipykernel->jupyter)
Using cached traitlets-4.3.1-py2.py3-none-any.whl
Collecting prompt-toolkit<2.0.0,>=1.0.0 (from jupyter-console->jupyter)
Using cached prompt_toolkit-1.0.9-py2-none-any.whl
Collecting pygments (from jupyter-console->jupyter)
Using cached Pygments-2.1.3-py2.py3-none-any.whl
Collecting widgetsnbextension>=1.2.6 (from ipywidgets->jupyter)
Using cached widgetsnbextension-1.2.6-py2.py3-none-any.whl
Collecting testpath (from nbconvert->jupyter)
Using cached testpath-0.3-py2.py3-none-any.whl
Collecting pandocfilters>=1.4.1 (from nbconvert->jupyter)
Using cached pandocfilters-1.4.1.tar.gz
Collecting jinja2 (from nbconvert->jupyter)
Using cached Jinja2-2.9.4-py2.py3-none-any.whl
Collecting jupyter-core (from nbconvert->jupyter)
Using cached jupyter_core-4.2.1-py2.py3-none-any.whl
Collecting bleach (from nbconvert->jupyter)
Using cached bleach-1.5.0-py2.py3-none-any.whl
Collecting mistune!=0.6 (from nbconvert->jupyter)
Using cached mistune-0.7.3-py2.py3-none-any.whl
Collecting nbformat (from nbconvert->jupyter)
Using cached nbformat-4.2.0-py2.py3-none-any.whl
Collecting entrypoints>=0.2.2 (from nbconvert->jupyter)
Using cached entrypoints-0.2.2-py2.py3-none-any.whl
Collecting ipython-genutils (from notebook->jupyter)
Using cached ipython_genutils-0.1.0-py2.py3-none-any.whl
Collecting singledispatch (from tornado>=4.0->ipykernel->jupyter)
Using cached singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting certifi (from tornado>=4.0->ipykernel->jupyter)
Using cached certifi-2016.9.26-py2.py3-none-any.whl
Collecting backports_abc>=0.4 (from tornado>=4.0->ipykernel->jupyter)
Using cached backports_abc-0.5-py2.py3-none-any.whl
Collecting pyzmq>=13 (from jupyter-client->ipykernel->jupyter)
Using cached pyzmq-16.0.2-cp27-cp27m-win32.whl
Collecting win-unicode-console>=0.5; sys_platform == "win32" (from ipython>=4.0.0->ipykernel->jupyter)
Using cached win_unicode_console-0.5.zip
Collecting pickleshare (from ipython>=4.0.0->ipykernel->jupyter)
Using cached pickleshare-0.7.4-py2.py3-none-any.whl
Collecting decorator (from ipython>=4.0.0->ipykernel->jupyter)
Using cached decorator-4.0.10-py2.py3-none-any.whl
Collecting backports.shutil-get-terminal-size; python_version == "2.7" (from ipython>=4.0.0->ipykernel->jupyter)
Using cached backports.shutil_get_terminal_size-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=18.5 in c:\python27\lib\site-packages (from ipython>=4.0.0->ipykernel->jupyter)
Collecting pathlib2; python_version == "2.7" or python_version == "3.3" (from ipython>=4.0.0->ipykernel->jupyter)
Using cached pathlib2-2.2.0-py2.py3-none-any.whl
Collecting simplegeneric>0.8 (from ipython>=4.0.0->ipykernel->jupyter)
Using cached simplegeneric-0.8.1.zip
Collecting colorama; sys_platform == "win32" (from ipython>=4.0.0->ipykernel->jupyter)
Using cached colorama-0.3.7-py2.py3-none-any.whl
Collecting enum34; python_version == "2.7" (from traitlets>=4.1.0->ipykernel->jupyter)
Using cached enum34-1.1.6-py2-none-any.whl
Collecting six (from traitlets>=4.1.0->ipykernel->jupyter)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter)
Using cached wcwidth-0.1.7-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->nbconvert->jupyter)
Using cached MarkupSafe-0.23.tar.gz
Collecting html5lib!=0.9999,!=0.99999,<0.99999999,>=0.999 (from bleach->nbconvert->jupyter)
Using cached html5lib-0.9999999.tar.gz
Collecting jsonschema!=2.5.0,>=2.4 (from nbformat->nbconvert->jupyter)
Using cached jsonschema-2.5.1-py2.py3-none-any.whl
Collecting configparser>=3.5; python_version == "2.7" (from entrypoints>=0.2.2->nbconvert->jupyter)
Using cached configparser-3.5.0.tar.gz
Collecting scandir (from pathlib2; python_version == "2.7" or python_version == "3.3"->ipython>=4.0.0->ipykernel->jupyter)
Using cached scandir-1.4.zip
Exception:
Traceback (most recent call last):
File "c:\python27\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "c:\python27\lib\site-packages\pip\commands\install.py", line 324, in run
requirement_set.prepare_files(finder)
File "c:\python27\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "c:\python27\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "c:\python27\lib\site-packages\pip\download.py", line 821, in unpack_url
hashes=hashes
File "c:\python27\lib\site-packages\pip\download.py", line 663, in unpack_http_url
unpack_file(from_path, location, content_type, link)
File "c:\python27\lib\site-packages\pip\utils\__init__.py", line 599, in unpack_file
flatten=not filename.endswith('.whl')
File "c:\python27\lib\site-packages\pip\utils\__init__.py", line 499, in unzip_file
fp = open(fn, 'wb')
IOError: [Errno 22] invalid mode ('wb') or filename: 'c:\\users\\wonhee\\appdata\\local\\temp\\pip-build-o9imsa\\scandir\\test/testdir/subdir/unicod\xc6\x8f.txt
I don't know about the errno 22.
Please give me some solutions for this problem. Thank you
I think your main problem is of 'scandir' package.
Actually, I had same problem as you, and I solved that the way as below.
you need to install 'scandir' package manually.
go to here: https://pypi.python.org/pypi/scandir
and download 'scandir-1.4.zip (md5)', and unzip it
open cmd, and go to directory where 'scandir-1.4' folder is
type 'python setup.py install'
after finishing install, try again to install jupyter
cheers.
In my case, the scandir install was failing because I hadn't installed python dev tools
sudo apt-get install python-dev
To install scandir without a binary package, use an official binary package of scandir from http://www.lfd.uci.edu/~gohlke/
Then run (e.g.)
pip install scandir‑1.5‑cp27‑cp27m‑win32.whl
I got a Django project that was started without virtualenv. Now migrating to virtualenv and my requirements.txt created before is huge and not installable in the virtualenv (many errors as shown below). How can i generate a minimal list of required packages? Or is there some reference anywhere?
Could not find any downloads that satisfy the requirement PIL==1.1.7 (from -r requirements.txt (line 8))
Some externally hosted files were ignored (use --allow-external PIL to allow).
Cleaning up...
No distributions at all found for PIL==1.1.7 (from -r requirements.txt (line 8))
it's not only PIL that throws errors, if I comment it another package shows and i can't know what's really used for my application to work.
Thanks in advance!
You can run pip freeze (related to system python used before virtualenv), this give you list of installed packages;
Then filter that list using following:
1) INSTALLED_APPS in settings
2) also check all from and import statement (search through the project)
I've had the same problem and there are issues getting PIL to install using PIP as there's no PIL 1.7 in the default Python repos.
The easiest fix is this to add these options to the pip:
--allow-external PIL --allow-unverified PIL
For example:
pip install -r requirements.txt --allow-external PIL --allow-unverified PIL
The problem with this it is a potential security issue and you don't want to do this on a production server! :)
Your options are to use Pillow which is a fork of PIL:
https://pypi.python.org/pypi/Pillow
Comments from the Pillow author, and you should verify that it works with you code.
Or try PIL 1.1.6 which is the Python Repos:
https://pypi.python.org/pypi/PIL
Or create your own repo and include the PIL 1.1.7 sources.
Or, if your on a Linux system install PIL using your distro's package management tool and remove PIL from your requirements file, and then rebuild your virutalenv.
You can this on Debian based distros like this:
sudo apt-get install python-imaging
Red Hat distros like this:
sudo yum install python-imaging
I was running heroku push master, and got this:
----- Python app detected
----- No runtime.txt provided; assuming python-2.7.3.
----- Using Python runtime (python-2.7.3)
----- Installing dependencies using Pip (1.2.1)
Downloading/unpacking Django-1.5c2 from https://www.djangoproject.com/download/1.5c2/tarball (from -r
requirements.txt (line 1))
Cannot determine compression type for file /tmp/pip-rYIGHS-unpack/tarball.ksh
Running setup.py egg_info for package Django-1.5c2
Installing collected packages: Django-1.5c2
Running setup.py install for Django-1.5c2
changing mode of build/scripts-2.7/django-admin.py from 600 to 755
changing mode of /app/.heroku/python/bin/django-admin.py to 755
========
WARNING!
========
You have just installed Django over top of an existing
installation, without removing it first. Because of this,
your install may now include extraneous files from a
previous version that have since been removed from
Django. This is known to cause a variety of problems. You
should manually remove the
/app/.heroku/python/lib/python2.7/site-packages/django
directory and re-install Django.
Successfully installed Django-1.5c2
How can I remove the previous Django package?
UPDATE:
My requirements.txt:
https://www.djangoproject.com/download/1.5c2/tarball/**#egg=django**
South==0.7.6
argparse==1.2.1
distribute==0.6.24
dj-database-url==0.2.1
psycopg2==2.4.6
wsgiref==0.1.2
PIL==1.1.7
The text in bold fixed the above warning.
UPDATE 2:
Since Django 1.5 was officially released, I just used pip freeze:
Django==1.5
South==0.7.6
argparse==1.2.1
distribute==0.6.24
dj-database-url==0.2.1
psycopg2==2.4.6
wsgiref==0.1.2
PIL==1.1.7
I've had problems where Heroku caches broken packages and there's no way to get them out. The Python buildpack should have some kind of support for flushing this cache (CACHE_DIR), but it does not.
There is a workaround: follow these instructions to change your Python runtime to, for instance, 3.3.0 (it doesn't matter if your app actually supports Python 3 or not). Then change it back to the default. The act of changing your Python runtime and then deploying will force the buildpack to totally erase the cache. As far as I know this is the only practical way to erase the cache at the moment.
Push current virtenv package to file
pip freeze > requirements.txt
Commit
git commit -am 'update packages'
And push to heroku
git push heroku
Then heroku will rebuild the environment
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (13/13), 1.26 KiB, done.
Total 13 (delta 3), reused 0 (delta 0)
-----> Python app detected
-----> No runtime.txt provided; assuming python-2.7.3.
-----> Preparing Python runtime (python-2.7.3)
-----> Installing Distribute (0.6.34)
-----> Installing Pip (1.2.1)
-----> Installing dependencies using Pip (1.2.1)
Downloading/unpacking Flask==0.9 (from -r requirements.txt (line 1))
Running setup.py egg_info for package Flask
Thought I removed the faulting package and all other packages that depends on it but nope. Each time I deploy I keep getting an error. Finally I somehow striped out all other packages that depends on the faulting package and everything worked perfectly. Hope somebody will find this useful