Heroku not installing psycopg2 from requirements.txt - django

Solved
In requirements.txt, changed the gunicorn==19.0.0 to gunicorn==18.0.0, and it did the trick. Version 19.0.0 breaks with Python 2.x, and it prevented psycopg2 from installing.
Original question
I have the following in requirements.txt :
Django==1.6.5
South==0.8.4
argparse==1.2.1
dj-database-url==0.3.0
dj-static==0.0.5
django-toolbelt==0.0.1
gunicorn==19.0.0
psycopg2==2.5.2
pystache==0.5.4
static==1.0.2
wsgiref==0.1.2
When I push to Heroku, however, it does not install psycopg2, and doesn't even mention any failure regarding this dependency.
Initializing repository, done.
Counting objects: 1257, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1213/1213), done.
Writing objects: 100% (1257/1257), 504.62 KiB | 376.00 KiB/s, done.
Total 1257 (delta 642), reused 0 (delta 0)
-----> Python app detected
-----> Installing runtime (python-2.7.8)
-----> Installing dependencies with pip
Downloading/unpacking Django==1.6.5 (from -r requirements.txt (line 1))
Downloading/unpacking South==0.8.4 (from -r requirements.txt (line 2))
Downloading/unpacking argparse==1.2.1 (from -r requirements.txt (line 3))
Running setup.py (path:/tmp/pip_build_u3159/argparse/setup.py) egg_info for package argparse
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
Downloading/unpacking dj-database-url==0.3.0 (from -r requirements.txt (line 4))
Downloading dj_database_url-0.3.0-py2.py3-none-any.whl
Downloading/unpacking dj-static==0.0.5 (from -r requirements.txt (line 5))
Downloading dj-static-0.0.5.tar.gz
Running setup.py (path:/tmp/pip_build_u3159/dj-static/setup.py) egg_info for package dj-static
Downloading/unpacking gunicorn==19.0.0 (from -r requirements.txt (line 6))
Running setup.py (path:/tmp/pip_build_u3159/gunicorn/setup.py) egg_info for package gunicorn
Downloading/unpacking pystache==0.5.4 (from -r requirements.txt (line 7))
Running setup.py (path:/tmp/pip_build_u3159/pystache/setup.py) egg_info for package pystache
pystache: using: version '5.4.1' of <module 'setuptools' from '/app/.heroku/python/lib/python2.7/site-packages/setuptools-5.4.1-py2.7.egg/setuptools/__init__.pyc'>
Downloading/unpacking static==1.0.2 (from -r requirements.txt (line 8))
Downloading static-1.0.2.tar.gz
Running setup.py (path:/tmp/pip_build_u3159/static/setup.py) egg_info for package static
Installing collected packages: Django, South, argparse, dj-database-url, dj-static, gunicorn, pystache, static
Running setup.py install for argparse
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
Running setup.py install for dj-static
Running setup.py install for gunicorn
File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/gaiohttp.py", line 67
yield from self.wsgi.close()
^
SyntaxError: invalid syntax
Installing gunicorn_paster script to /app/.heroku/python/bin
Installing gunicorn script to /app/.heroku/python/bin
Installing gunicorn_django script to /app/.heroku/python/bin
Running setup.py install for pystache
pystache: using: version '5.4.1' of <module 'setuptools' from '/app/.heroku/python/lib/python2.7/site-packages/setuptools-5.4.1-py2.7.egg/setuptools/__init__.pyc'>
Installing pystache script to /app/.heroku/python/bin
Installing pystache-test script to /app/.heroku/python/bin
Running setup.py install for static
Installing static script to /app/.heroku/python/bin
Successfully installed Django South argparse dj-database-url dj-static gunicorn pystache static
Cleaning up...
-----> Preparing static assets
Collectstatic configuration error. To debug, run:
$ heroku run python ./manage.py collectstatic --noinput
-----> Discovering process types
Procfile declares types -> web
-----> Compressing... done, 34.7MB
-----> Launching... done, v5
When I try to sync the database, of course, it complains about psycopg2 not being installed.
How can I get Heroku to install the missing dependency? I'll much appreciate any constructive idea.
My database setup, in case it is interesting for the issue:
DATABASES = {}
if ENV_TYPE == 'development':
DATABASES['default'] = {'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),}
else:
DATABASES['default'] = dj_database_url.config()
Update
There is one SyntaxError mentioned in the report, for gunicorn. Is is possibly braking the installation of psycopg2? I think it should not.

yield from added in Python 3.3.
File "/app/.heroku/python/lib/**python2.7**/site-packages/gunicorn/workers/gaiohttp.py"‌​, line 67
yield from self.wsgi.close()

Related

Confusing error while deploying Django on Heroku: No module named site

I've got two Heroku apps, running on the same codebase. The 1st is used as staging env and it works as expected (it auto deploys the devel branch) and the 2nd is used as the production app (auto deploying the master branch).
Today I did a minor code-level change and submitted a PR from devel to master and merged it. From that moment, heroku cannot deploy my master branch in production.
Full logs for this as given by heroku build log:
----->
Python app detected
-----> Uninstalling stale dependencies
Uninstalling django-elasticsearch-0.5:
Successfully uninstalled django-elasticsearch-0.5
Uninstalling pymongo-3.3.0:
Successfully uninstalled pymongo-3.3.0
$ pip install -r requirements.txt
Collecting git+https://github.com/liberation/django_elasticsearch.git (from -r requirements.txt (line 69))
Cloning https://github.com/liberation/django_elasticsearch.git to /tmp/pip-ZPaLWp-build
Collecting Delorean==0.6.0 (from -r requirements.txt (line 10))
Downloading Delorean-0.6.0.tar.gz
Collecting humanize==0.5.1 (from -r requirements.txt (line 37))
Downloading humanize-0.5.1.tar.gz
Collecting pynamodb==2.0.2 (from -r requirements.txt (line 46))
Downloading pynamodb-2.0.2-py2.py3-none-any.whl (73kB)
Collecting tzlocal==1.3 (from -r requirements.txt (line 63))
Downloading tzlocal-1.3.tar.gz
Installing collected packages: humanize, tzlocal, Delorean, django-traffic, pynamodb, django-elasticsearch
Running setup.py install for humanize: started
Running setup.py install for humanize: finished with status 'done'
Running setup.py install for tzlocal: started
Running setup.py install for tzlocal: finished with status 'done'
Running setup.py install for Delorean: started
Running setup.py install for Delorean: finished with status 'done'
Running setup.py install for django-elasticsearch: started
Running setup.py install for django-elasticsearch: finished with status 'done'
Successfully installed Delorean-0.6.0 django-elasticsearch-0.5 django-traffic-1.2.4 humanize-0.5.1 pynamodb-2.0.2 tzlocal-1.3
-----> Django app detected
-----> Running Django migrations
ImportError: No module named site
! Push rejected, failed to compile Django app.
! Push failed
Well, as fas as I know, there's no "site" module/app or anything in my code, so I cannot understand what's the problem in this case. I remind you, the devel branch auto deploys as expected in the respective Heroku app.
Any ideas?

Manually remove Python package on Heroku

I was running heroku push master, and got this:
----- Python app detected
----- No runtime.txt provided; assuming python-2.7.3.
----- Using Python runtime (python-2.7.3)
----- Installing dependencies using Pip (1.2.1)
Downloading/unpacking Django-1.5c2 from https://www.djangoproject.com/download/1.5c2/tarball (from -r
requirements.txt (line 1))
Cannot determine compression type for file /tmp/pip-rYIGHS-unpack/tarball.ksh
Running setup.py egg_info for package Django-1.5c2
Installing collected packages: Django-1.5c2
Running setup.py install for Django-1.5c2
changing mode of build/scripts-2.7/django-admin.py from 600 to 755
changing mode of /app/.heroku/python/bin/django-admin.py to 755
========
WARNING!
========
You have just installed Django over top of an existing
installation, without removing it first. Because of this,
your install may now include extraneous files from a
previous version that have since been removed from
Django. This is known to cause a variety of problems. You
should manually remove the
/app/.heroku/python/lib/python2.7/site-packages/django
directory and re-install Django.
Successfully installed Django-1.5c2
How can I remove the previous Django package?
UPDATE:
My requirements.txt:
https://www.djangoproject.com/download/1.5c2/tarball/**#egg=django**
South==0.7.6
argparse==1.2.1
distribute==0.6.24
dj-database-url==0.2.1
psycopg2==2.4.6
wsgiref==0.1.2
PIL==1.1.7
The text in bold fixed the above warning.
UPDATE 2:
Since Django 1.5 was officially released, I just used pip freeze:
Django==1.5
South==0.7.6
argparse==1.2.1
distribute==0.6.24
dj-database-url==0.2.1
psycopg2==2.4.6
wsgiref==0.1.2
PIL==1.1.7
I've had problems where Heroku caches broken packages and there's no way to get them out. The Python buildpack should have some kind of support for flushing this cache (CACHE_DIR), but it does not.
There is a workaround: follow these instructions to change your Python runtime to, for instance, 3.3.0 (it doesn't matter if your app actually supports Python 3 or not). Then change it back to the default. The act of changing your Python runtime and then deploying will force the buildpack to totally erase the cache. As far as I know this is the only practical way to erase the cache at the moment.
Push current virtenv package to file
pip freeze > requirements.txt
Commit
git commit -am 'update packages'
And push to heroku
git push heroku
Then heroku will rebuild the environment
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (13/13), 1.26 KiB, done.
Total 13 (delta 3), reused 0 (delta 0)
-----> Python app detected
-----> No runtime.txt provided; assuming python-2.7.3.
-----> Preparing Python runtime (python-2.7.3)
-----> Installing Distribute (0.6.34)
-----> Installing Pip (1.2.1)
-----> Installing dependencies using Pip (1.2.1)
Downloading/unpacking Flask==0.9 (from -r requirements.txt (line 1))
Running setup.py egg_info for package Flask
Thought I removed the faulting package and all other packages that depends on it but nope. Each time I deploy I keep getting an error. Finally I somehow striped out all other packages that depends on the faulting package and everything worked perfectly. Hope somebody will find this useful

fixed virtualenvwrapper now setuptools fails on pip install

I just got thru fixing virtualenvwrapper using this page as a guide:
http://www.jongales.com/blog/2012/07/25/fixing-virtualenv-after-installing-mountain-lion/#comment-37964
now when I got to pip install -r requirements.txt to get my Django project working Im failing on setuptools stuff?
anyone have some advices?
MacBook-Pro:staging macbook$ ls
Procfile manage.py settings.py urls.py wsgi.py
local_settings.py medint settings.pyc urls.pyc
local_settings.pyc requirements.txt stuff wiki
macBook-Pro:macbook$ pip install -r requirements.txt
Downloading/unpacking Django==1.4.1 (from -r requirements.txt (line 1))
Downloading Django-1.4.1.tar.gz (7.7MB): 7.7MB downloaded
Running setup.py egg_info for package Django
Downloading/unpacking distribute==0.6.10 (from -r requirements.txt (line 2))
Downloading distribute-0.6.10.tar.gz (389kB): 389kB downloaded
Running setup.py egg_info for package distribute
Downloading/unpacking dj-database-url==0.2.1 (from -r requirements.txt (line 3))
Downloading dj-database-url-0.2.1.tar.gz
Running setup.py egg_info for package dj-database-url
Downloading/unpacking gunicorn==0.14.6 (from -r requirements.txt (line 4))
Downloading gunicorn-0.14.6.tar.gz (219kB): 219kB downloaded
Running setup.py egg_info for package gunicorn
Downloading/unpacking mailsnake==1.6.1 (from -r requirements.txt (line 5))
Downloading mailsnake-1.6.1.tar.gz
Running setup.py egg_info for package mailsnake
file mailsnake.py (for module mailsnake) not found
Downloading/unpacking newrelic==1.5.0.103 (from -r requirements.txt (line 6))
Downloading newrelic-1.5.0.103.tar.gz (186kB): 186kB downloaded
Running setup.py egg_info for package newrelic
Downloading/unpacking psycopg2==2.4.5 (from -r requirements.txt (line 7))
Downloading psycopg2-2.4.5.tar.gz (719kB): 719kB downloaded
Running setup.py egg_info for package psycopg2
no previously-included directories found matching 'doc/src/_build'
Downloading/unpacking requests==0.14.2 (from -r requirements.txt (line 8))
Downloading requests-0.14.2.tar.gz (361kB): 361kB downloaded
Running setup.py egg_info for package requests
warning: no files found matching 'tests/*.'
Downloading/unpacking simplejson==2.6.2 (from -r requirements.txt (line 9))
Downloading simplejson-2.6.2.tar.gz (53kB): 53kB downloaded
Running setup.py egg_info for package simplejson
Requirement already satisfied (use --upgrade to upgrade): wsgiref==0.1.2 in /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7 (from -r requirements.txt (line 10))
Installing collected packages: Django, distribute, dj-database-url, gunicorn, mailsnake, newrelic, psycopg2, requests, simplejson
Running setup.py install for Django
changing mode of build/scripts-2.7/django-admin.py from 644 to 755
changing mode of /Users/tmacbook/Envs/medint2/bin/django-admin.py to 755
Running setup.py install for distribute
Before install bootstrap.
Scanning installed packages
Setuptools installation detected at /Users/tmacbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
Egg installation
Patching...
Renaming /Users/macbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg into /Users/macbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg.OLD.1356316395.42
Patched done.
Relaunching...
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'install' is not defined
Complete output from command /Users/macbook/Envs/medint2/bin/python -c "import setuptools;__file__='/Users/macbook/Envs/medint2/build/distribute/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/qm/dvrz87yj6px11mw0sxcy88lh0000gn/T/pip-RJZBEP-record/install-record.txt --single-version-externally-managed --install-headers /Users/tmacbook/Envs/medint2/bin/../include/site/python2.7:
Before install bootstrap.
Scanning installed packages
Setuptools installation detected at /Users/macbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
Egg installation
Patching...
Renaming /Users/macbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg into /Users/macbook/Envs/medint2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg.OLD.1356316395.42
Patched done.
Relaunching...
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'install' is not defined
----------------------------------------
Command /Users/macbook/Envs/medint2/bin/python -c "import setuptools;__file__='/Users/tmacbook/Envs/medint2/build/distribute/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/qm/dvrz87yj6px11mw0sxcy88lh0000gn/T/pip-RJZBEP-record/install-record.txt --single-version-externally-managed --install-headers /Users/tmacbook/Envs/medint2/bin/../include/site/python2.7 failed with error code 1 in /Users/tmacbook/Envs/medint2/build/distribute
Storing complete log in /Users/tmacbook/.pip/pip.log
upgraded distribute==0.6.32
that seems to fix it.

Django requirements.txt pip for mdbtools

I'm trying to use mdbtools in a django app I have. I've included the following two lines in my requirements.txt file:
mdbtools-dev
mdbtools
but am running into some errors:
....
Requirement already satisfied (use --upgrade to upgrade): stripe in ./lib/python2.7/site-packages (from -r requirements.txt (line 7))
Downloading/unpacking mdbtools-dev (from -r requirements.txt (line 8))
Could not find any downloads that satisfy the requirement mdbtools-dev (from -r requirements.txt (line 8))
No distributions at all found for mdbtools-dev (from -r requirements.txt (line 8))
Storing complete log in /app/.pip/pip.log
! Heroku push rejected, failed to compile Python app
I sudo apt-get install both mdbtools and mdbtools-dev without a problem. Does anyone have any idea what's going on?
requirements.txt file is not for general system requirements - it is only for Python libraries that are necessary for your application.
Since there is no Python package called mdbtools, you are getting that error.

django-sha2 not installing on heroku server

Im having problems deploying my django app to heroku
in the first part, heroku's push fails when it could not find any downloads for django-sha2==0.4 , however when i do pip install on my computer locally, it works
hh:djangoherokutest Kinnovate$ git push heroku master
Counting objects: 12, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 958 bytes, done.
Total 10 (delta 7), reused 0 (delta 0)
-----> Heroku receiving push
-----> Python/Django app detected
-----> Preparing virtualenv version 1.7
New python executable in ./bin/python
Installing distribute..............................................................................................................................................................................................done.
Installing pip...............done.
Overwriting ./bin/activate with new content
Overwriting ./bin/activate.fish with new content
Overwriting ./bin/activate.csh with new content
-----> Activating virtualenv
-----> Installing dependencies using pip version 1.0.2
Downloading/unpacking git+https://github.com/django/django.git (from -r requirements.txt (line 4))
Cloning https://github.com/django/django.git to /tmp/pip-Jvyfwt-build
Running setup.py egg_info for package from git+https://github.com/django/django.git
Requirement already satisfied (use --upgrade to upgrade): py-bcrypt==0.2 in ./lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): ipython==0.12 in ./lib/python2.7/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): Flask==0.8 in ./lib/python2.7/site-packages (from -r requirements.txt (line 3))
Downloading/unpacking psycopg2==2.4.2 (from -r requirements.txt (line 5))
Running setup.py egg_info for package psycopg2
no previously-included directories found matching 'doc/src/_build'
Requested psycopg2==2.4.2 (from -r requirements.txt (line 5)), but installing version 2.4.4
Downloading/unpacking django-sha2==0.4 (from -r requirements.txt (line 6))
Could not find any downloads that satisfy the requirement django-sha2==0.4 (from -r requirements.txt (line 6))
No distributions at all found for django-sha2==0.4 (from -r requirements.txt (line 6))
Storing complete log in /app/.pip/pip.log
! Heroku push rejected, failed to compile Python/django app
To git#heroku.com:growing-mountain-9962.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git#heroku.com:growing-mountain-9962.git'
hh:djangoherokutest Kinnovate$ pip install django-sha2
Requirement already satisfied (use --upgrade to upgrade): django-sha2 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django_sha2-0.4-py2.7.egg
Requirement already satisfied (use --upgrade to upgrade): Django>=1.2 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from django-sha2)
Cleaning up...
how do i fix this?
Try putting
git+https://github.com/fwenzel/django-sha2.git
instead of
django-sha2==0.4
in the requirements.txt. This will link straight to the package.