Getting peewee to work on elastic beanstalk - amazon-web-services

I'm trying to install the peewee package on an elastic beanstalk instance to control a bottle application. I think the problem is with the MySQL-python dependency, but peewee doesn't load either. My requirements file looks like this:
bottle==0.11.6
peewee==2.1.1
MySQL-python
I also tried MySQL-python as MySQL-python==1.2.4b4. For a number of reasons which I believe are summed up here, this doesn't work; peewee isn't installed and I get this error message in the log:
2013-05-18 06:57:08,407 [INFO] (15947 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Script succeeded.
2013-05-18 06:57:08,408 [INFO] (15947 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Executing script: /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py
2013-05-18 06:57:18,663 [INFO] (15947 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Output from script: New python executable in /opt/python/run/venv/bin/python2.6
Not overwriting existing python script /opt/python/run/venv/bin/python (you must use /opt/python/run/venv/bin/python2.6)
Installing distribute..................................................................................................................................................................................................done.
Installing pip................done.
Requirement already satisfied (use --upgrade to upgrade): bottle==0.11.6 in /opt/python/run/venv/lib/python2.6/site-packages (from -r /opt/python/ondeck/app/requirements.txt (line 1))
Downloading/unpacking peewee==2.1.1 (from -r /opt/python/ondeck/app/requirements.txt (line 2))
Running setup.py egg_info for package peewee
Downloading/unpacking MySQL-python (from -r /opt/python/ondeck/app/requirements.txt (line 3))
Running setup.py egg_info for package MySQL-python
The required version of distribute (>=0.6.28) is not available,
and can't be installed while this script is running. Please
install a more recent version first, using
'easy_install -U distribute'.
(Currently using distribute 0.6.27 (/opt/python/run/venv/lib/python2.6/site-packages/distribute-0.6.27-py2.6.egg))
Complete output from command python setup.py egg_info:
The required version of distribute (>=0.6.28) is not available,
and can't be installed while this script is running. Please
install a more recent version first, using
'easy_install -U distribute'.
So pip doesn't work for these packages. I'm pretty sure I could fix this by connecting to the EC2 instance and installing MySQL-python without pip, but it seems to me that would defeat the purpose of elastic beanstalk, since every time a new instance started it would need me to go in and tinker with it manually. Is there a proper way for me to deal with this problem? I've also had the same issue trying to install things like scipy and matplotlib, so broadly, is there a way I can install python packages on elastic beanstalk instances without pip, and without going into each instance individually? Should I be looking into how to make a custom AMI and use it on beanstalk?
Thanks a lot,
Alex

Sorry I forgot I asked this.
I eventually solved this issue by ssh'ing into the instance and installing MySQL-Python using easy_install instead of pip.
Basically I went to the python virtual environment direction at:
opt/python/run/venv/bin
and installed version 1.2.3 using easy_install:
sudo ./easy_install MySQL-Python==1.2.3
I checked to see it was installed by trying to import MySQLdb in the virtual environment's python:
./python
>>>import MySQLdb
>>>
and it worked for me. Then I reset the server by pushing to it again with the elastic beanstalk command line tool:
git aws.push
and that's about it. Somewhat annoying to have to do it, but it seems to continue to work when a new instance is started up.
Thanks,
Alex

Related

Getting error "post-link script failed for package defaults ::ipykernel-4.10.0-py27_0" while installing ipykernel in anaconda with python 2.7

I am trying to install ipykernel in a separate python 2.7 environment (py27) for use of jupyter notebook. My base environment uses python3.9 that works great. When trying to use conda install ipykernel, I get the error below. Yes, this was done in a separate environment from my base and I am not aware of any virus softwares that are impeding this from successfully completing. Those are the only answers that I've seemed to find online. I also tried reinstalling anaconda. I did not attach the output from conda install -v ipykernel, but saw an error saying invalid syntax in decorator.py on line 162. It was pointing to the file=stderr in a print function call.
Similar to this problem https://github.com/conda/conda/issues/7351, but was unable to solve the problem with the suggestions.
(py27) C:\Users\Heath>conda install ipykernel
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\Heath\anaconda3\envs\py27
added / updated specs:
- ipykernel
The following NEW packages will be INSTALLED:
backports pkgs/main/noarch::backports-1.0-pyhd3eb1b0_2
backports.shutil_~ pkgs/main/noarch::backports.shutil_get_terminal_size-1.0.0-pyhd3eb1b0_3
backports_abc pkgs/main/noarch::backports_abc-0.5-py_1
colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0
decorator pkgs/main/noarch::decorator-5.1.0-pyhd3eb1b0_0
enum34 pkgs/main/win-64::enum34-1.1.6-py27_1
futures pkgs/main/win-64::futures-3.3.0-py27_0
ipykernel pkgs/main/win-64::ipykernel-4.10.0-py27_0
ipython pkgs/main/win-64::ipython-5.8.0-py27_0
ipython_genutils pkgs/main/noarch::ipython_genutils-0.2.0-pyhd3eb1b0_1
jupyter_client pkgs/main/win-64::jupyter_client-5.3.4-py27_0
jupyter_core pkgs/main/win-64::jupyter_core-4.6.1-py27_0
libsodium pkgs/main/win-64::libsodium-1.0.16-h8b3e59e_0
pathlib2 pkgs/main/win-64::pathlib2-2.3.5-py27_0
pickleshare pkgs/main/win-64::pickleshare-0.7.5-py27_0
prompt_toolkit pkgs/main/win-64::prompt_toolkit-1.0.15-py27h3a8ec6a_0
pygments pkgs/main/noarch::pygments-2.5.2-py_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0
pywin32 pkgs/main/win-64::pywin32-227-py27h0c8e037_1
pyzmq pkgs/main/win-64::pyzmq-18.1.0-py27hc56fc5f_0
scandir pkgs/main/noarch::scandir-1.10.0-pyh5d7bf9c_3
simplegeneric pkgs/main/win-64::simplegeneric-0.8.1-py27_2
singledispatch pkgs/main/noarch::singledispatch-3.7.0-pyhd3eb1b0_1001
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_0
tornado pkgs/main/win-64::tornado-5.1.1-py27h0c8e037_0
traitlets pkgs/main/win-64::traitlets-4.3.3-py27_0
wcwidth pkgs/main/noarch::wcwidth-0.2.5-pyhd3eb1b0_0
win_unicode_conso~ pkgs/main/win-64::win_unicode_console-0.5-py27hc037021_0
zeromq pkgs/main/win-64::zeromq-4.3.1-h2880e7c_3
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
ERROR conda.core.link:_execute(699): An error occurred while installing package 'defaults::ipykernel-4.10.0-py27_0'.
Rolling back transaction: done
LinkError: post-link script failed for package defaults::ipykernel-4.10.0-py27_0
location of failed script: C:\Users\Heath\anaconda3\envs\py27\Scripts\.ipykernel-post-link.bat
==> script messages <==
<None>
==> script output <==
stdout:
stderr:
return code: 1
()```

Installing requirements with pip in an environment terminates (2.7)

I'm trying to setup my work environment on my new MBP M1. The executable is python2.7, so I've been trying to create a virtualenv and install the requirements.
Alas, I can not.
I create an environment, and get the output:
virtualenv env
created virtual environment CPython2.7.16.final.0-64 in 183ms
creator CPython2macOsFramework(dest=/Users/v/dev/skolplattformen/env, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/v/Library/Application Support/virtualenv)
added seed packages: pip==20.3.3, setuptools==44.1.1, wheel==0.36.2
activators PythonActivator,CShellActivator,FishActivator,PowerShellActivator,BashActivator
Then I activate that environment, and try to pip install the requirements.
Although, I get this
1716 killed pip2.7 install -r requirements.txt
I can't find anything in system.log, I can't find what process 1716 is...
which pip2.7 points to the right place, although which pip, in the environment, says pip: aliased to noglob pip.
I can't execute a python script either in the environment, or open a repl, it gives the same kill message.
I had the same problem. I solved it by downloading Python 2.7 from https://www.python.org/ to replace the system's default.

Failed building wheel for python-ldap (windows) *first stackoverflow ask*

I've installed flask, flask_login, and ldap in the virtualenv for my flask package (to authenticate ldap users with the usernames & passwords of the other users at my company).
When I run my flask app, I get "AttributeError: 'module' object has no attribute 'initialize'"
I'm tried "pip install python-ldap"
but I get a red error (sandwiched between other white text) that reads:
"Failed building wheel for python-ldap"
I have thoroughly examined these:
How to install python-ldap on a python 2.7 virtualenv on windows without compiling
Installing python-ldap in a virtualenv on Windows
https://www.python-ldap.org/docs.html
https://www.linuxjournal.com/article/6988?page=0,1
to no avail.
I've had this error for days now...
(also:
pip install openldap
gives:
Collecting openldap
Could not find a version that satisfies the requirement openldap (from versions: )
No matching distribution found for openldap
...(in and outside of my venv) if that information helps)
I had this error last week and after switching work computers, I recreated the virtualenv and installed all of the other packages and dependencies successfully.
my requirements.txt file contains these lines after a line of git stuff:
click==6.7
Flask==0.12.2
Flask-Login==0.4.0
itsdangerous==0.24
Jinja2==2.9.6
ldap==1.0.2
ldap3==2.2.4
MarkupSafe==1.0
pyasn1==0.2.3
Werkzeug==0.12.2
Has anyone successfully installed python-ldap into a flask virtualenv with windows? If so... how?!
Go to:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-ldap
download python_ldap-2.4.38-cp27-cp27m-win_amd64.whl in the same directory that your venv folder is listed under
(ex.)
/myrepo
/myapp
/myapp
/venv
/run.py
/python_ldap-2.4.38-cp27-cp27m-win_amd64.whl
install the .whl file while inside of your venv
venv\Scripts\activate
pip install python_ldap-2.4.38-cp27-cp27m-win_amd64.whl
Voila!
The error
ERROR: Failed building wheel for python-ldap
can occur for several reasons. You actually have to look at what happened above this line. In my case, I just had to install missing dependencies to build python-ldap. You can look them up here. For CentOS 8, I had to install:
sudo dnf install openldap-devel python3-devel gcc
to make
python3 -m pip install --user python-ldap -U
work.

Error /usr/bin/env: ‘python’: No such file or directory - installing presto on AWS EC2 instance

I unpacked the presto on AWS EC2 instance and followed the documentation on prestodb
when I run bin/launcher startI get the following error:
/usr/bin/env: ‘python’: No such file or directory
I guess documentation could mention that explicitly, but you need certain piece of software installed to run presto. This includes, but is not limited to:
Supported Java JVM
Python (i guess 2.7 would do)
For Ubuntu/Debian:
Install Python if you did not do before:
apt-get install python3
If you have python already:
whereis python3
sudo ln -s /usr/bin/python3 /usr/bin/pytho

Django on Elastic Beanstalk "requirements.txt" is invalid

I'm trying to deploy a django project with Elastic Beanstalk cli. I followed the Official Amazon Tutorial as well as this Tutorial on realpython.com.
The Deploymentprocess works fine as long as I stick to a newly created Django Project with just a view dependencies for pip.
When try it with my Django Project The Deployment fails because the requirements.txt is invalid. I created it inside my virutalenv like this: pip freeze > requirements.txt
braintree==3.32.0
cffi==1.9.1
cryptography==1.7
Django==1.10.4
django-allauth==0.29.0
django-betterforms==1.1.4
django-contrib-comments==1.7.3
django-formtools==1.0
django-payments==0.9.6
django-tinymce==2.4.0
enum34==1.1.6
idna==2.1
ipaddress==1.0.17
oauthlib==2.0.1
Pillow==3.4.2
pyasn1==0.1.9
pycparser==2.17
PyJWT==1.4.2
python-openid==2.2.5
requests==2.12.3
requests-oauthlib==0.7.0
six==1.10.0
stripe==1.43.0
suds-jurko==0.6
xmltodict==0.10.2
In my despair I tried to activate the dependencies one by one. The Deployment process is so slow that I gave up after a while. But now I know now that Django==1.10.4, django-allauth==0.29.0, django-betterforms==1.1.4, django-contrib-comments==1.7.3, django-formtools==1.0 and django-tinymce==2.4.0 are not the problem. With django-payments==0.9.6 it fails but its not the only dependency that causes the problem.
This is in my activity.log:
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/opt/python/run/venv/bin/python2.7 -c "import setuptools, tokenize;
__file__='/tmp/pip-build-BF9Oen/cffi/setup.py';
exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-BsXWzo-record/install-record.txt --single-version-externally-managed --compile --install-headers
/opt/python/run/venv/include/site/python2.7/cffi" failed with error code 1 in /tmp/pip-build-BF9Oen/cffi
You are using pip version 7.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
2016-12-13 14:49:05,155 ERROR
Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
Traceback (most recent call last):
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
install_dependencies()
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command
'/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt'
returned non-zero exit status 1 (Executor::NonZeroExitStatus)
I't could have something to do with missing Software. I know how to use the .ebextensions .config files but I don't know whats missing. Is there a descent way to debug this locally?
You seem to have problems with both FFI and Python. You have Cryptography in use but don't probably have all the required libraries Install Python and FFI requirements and OpenSSL dependencies for Cryptography.
$ eb ssh # SSH into your EB environment
$ sudo su # gain root privileges
$ yum install python-devel # or python27-devel
$ yum install libffi-devel # CFFI / FFI requirements
$ yum install openssl-devel # Cryptography requirements
If Pip complains, make sure you have the latest pip installed:
# repeat SSH and root privilege steps
$ source /opt/python/run/venv/bin/activate # activate Python environment
$ pip install --upgrade pip # upgrade to latest pip
To make the changes permanent, append them to your EB configuration. This also works if you do not have direct SSH access.
# .ebextensions/10_packages.config excerpt
packages:
yum:
gcc: []
libffi-devel: []
openssl-devel: []
python-devel: [] # or python27-devel
.ebextensions folder files with the .config suffix are state declarations for the Elastic Beanstalk environment. That is, they are instructions that are run on each Elastic Beanstalk deployment when you run eb deploy or your CI environment deploys code to EB.
Amazon Linux is an RPM based Linux that uses YUM for package management. You will find RHEL / CentOS / OpenSUSE instructions for Python package installation helpful for the Amazon Linux distribution.
There typically isn’t a way to debug these issues locally, because people don’t normally run Amazon linux on their local machines, and EB has a very specific environment setup that is not easy to reproduce (at least it wasn’t for me).
If you want to do it fast, just ssh to one of your EB/EC2 instances and try to install the pip requirements one-by-one by hand. If you want to do it right, you do have to go through this list of requirements one-by-one. (i.e., start with one requirement in the requirements file, deploy, then two requirements, then three requirements, etc.). You may want to start with upgrading pip by adding
pip==9.0.1
At the top of your requirements file and also making sure the following Amazon Linux (if that is what you are using) are installed:
python-devel
libffi
libffi-devel