tl;dr - django channels app runs local with manage.py runserver but not on heroku.
I'm new to django channels - trying to deploy a very basic django app using channels to heroku. I initially built the project using the standard django polls tutorial and deployed that to heroku. Then i added in a chat app using the django channels tutorial. Managed to get that running fine locally using docker to run a redis server as they suggested and "python manage.py runserver".
I'm getting stuck trying to deploy this to heroku or run it locally using heroku local. I've already added the redis addon in heroku and modified settings.py to point to the REDIS_URL environment variable. I also modified my template to use wss if appropriate (I believe that's necessary for heroku):
var ws_scheme = window.location.protocol == "https:" ? "wss" : "ws";
var target = ws_scheme + '://'
+ window.location.host
+ '/ws/chat/'
+ roomName
+ '/';
const chatSocket = new WebSocket(
target
);
...
Thus I'm concluding that the problem is with the procfile. I'm not sure what the instructions to use there are. The initial polls tutorial used:
web: gunicorn gettingstarted.wsgi --log-file -
If I just use that 'heroku local' works fine and deploying works fine, but when I try to send a chat message it does nothing and shows a 404 in the console. I know I have to change it to use an asgi server instead of gunicorn. found this tutorial on deploying an app with channels to heroku, which used:
web: daphne chat.asgi:channel_layer --port $PORT --bind 0.0.0.0 -v2
worker: python manage.py runworker -v2
I tried that, but that's where I get stuck. Here's what I'm getting when I run heroku local:
krishnas-air:python-getting-started Krishna$ heroku local
[OKAY] Loaded ENV .env File as KEY=VALUE Format
6:46:50 PM worker.1 | Traceback (most recent call last):
6:46:50 PM worker.1 | File "manage.py", line 8, in <module>
6:46:50 PM worker.1 | from django.core.management import execute_from_command_line
6:46:50 PM worker.1 | ImportError: No module named django.core.management
[DONE] Killing all processes with signal SIGINT
6:46:50 PM worker.1 Exited with exit code null
6:46:50 PM web.1 | Traceback (most recent call last):
6:46:50 PM web.1 | File "/usr/local/bin/daphne", line 5, in <module>
6:46:50 PM web.1 | from daphne.cli import CommandLineInterface
6:46:50 PM web.1 | File "/usr/local/lib/python3.7/site-packages/daphne/cli.py", line 1, in <module>
6:46:50 PM web.1 | import argparse
6:46:50 PM web.1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
6:46:50 PM web.1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
6:46:50 PM web.1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
6:46:50 PM web.1 | File "<frozen importlib._bootstrap_external>", line 724, in exec_module
6:46:50 PM web.1 | File "<frozen importlib._bootstrap_external>", line 857, in get_code
6:46:50 PM web.1 | File "<frozen importlib._bootstrap_external>", line 525, in _compile_bytecode
6:46:50 PM web.1 | KeyboardInterrupt
6:46:50 PM web.1 Exited with exit code null
The import error message made me think my requirements.txt could be missing something, so I've included that here as well for reference:
django
gunicorn
django-heroku
requests
channels
channels_redis
asgi_redis
asgiref
daphne
redis
gevent
gevent-websocket
greenlet
Thanks for any help!
I just figured out a very similar issue. First of all, although not Heroku specific, these docs are a must read.
I think the problem is that Heroku is routing ws:// requests through WSGI instead of ASGI. So the first step is to create an asgi.py file in the same folder as wsgi.py with something like this:
import django
from channels.routing import get_default_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "dropdjango.settings")
django.setup()
application = get_default_application()
Then, in the Procfile, define dynos for web worker:
web: daphne <my-web-app>.asgi:application --port $PORT --bind 0.0.0.0 -v2
worker: python manage.py runworker channel_layer -v2
If the dynos don't exist yet in Heroku, create them using the Heroku CLI.
In my case, the web dyno already existed so I only created the worker dyno:
heroku ps:scale worker=1:free -a <your-heroku-app-name>
Finally, double-check your settings.py to make sure you have:
ASGI_APPLICATION="<my-web-app>.routing.application"
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {"hosts": [os.environ.get("REDIS_URL", "redis://localhost:6379")]},
},
}
Warning: This will run all your requests through daphne. I've read of caveats but I haven't experienced them yet.
I just want to say this thread is a mess. I will figure out the correct configuration of everything and post back here my results.
I'll make a github repo to share all the files.
Related
I am following this tutorial to setup a Django-gunicorn-nginx server in AWS EC2. After installing all dependancies and making a change in wsgi.py as follows
import os, sys
# add the hellodjango project path into the sys.path
sys.path.append('/home/ubuntu/project/ToDo-application/')
# add the virtualenv site-packages path to the sys.path
sys.path.append('/home/ubuntu/.local/lib/python3.6/site-packages')
# poiting to the project settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "todo_app.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I run gunicorn todo_app.wsgi and get the following error:
ubuntu#ip-172-31-61-163:~/project/ToDo-application$ gunicorn todo_app.wsgi
[2018-11-07 11:25:35 +0000] [8211] [INFO] Starting gunicorn 19.7.1
[2018-11-07 11:25:35 +0000] [8211] [INFO] Listening at: http://127.0.0.1:8000 (8211)
[2018-11-07 11:25:35 +0000] [8211] [INFO] Using worker: sync
[2018-11-07 11:25:35 +0000] [8215] [INFO] Booting worker with pid: 8215
[2018-11-07 11:25:35 +0000] [8215] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 578, in spawn_worker
worker.init_process()
File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
self.load_wsgi()
File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 377, in import_app
__import__(module)
File "/home/ubuntu/urbanpiper/ToDo-application/todo_app/wsgi.py", line 20, in <module>
from django.core.wsgi import get_wsgi_application
File "/home/ubuntu/.local/lib/python3.6/site-packages/django/__init__.py", line 1, in <module>
from django.utils.version import get_version
File "/home/ubuntu/.local/lib/python3.6/site-packages/django/utils/version.py", line 71, in <module>
#functools.lru_cache()
AttributeError: 'module' object has no attribute 'lru_cache'
Is this because of gunicorn having python2 dependancies and Django being on python3? I tried uninstalling gunicorn and trying it again but it did not work.
# WRONG:
# add the virtualenv site-packages path to the sys.path
sys.path.append('/home/ubuntu/.local/lib/python3.6/site-packages')
You ought to create a virutalenv for each uwsgi application you wish to host on the server, rather than setting the virtualenv to the path above. If you followed the linked tutorial word-by-word, then this is the part which needs more explaining:
Make a virtualenv and install your pip requirements
Essentially:
# install virtualenv3
sudo apt-get install virtualenv3
# create the virtual environment, specifically for the stated python version
virtualenv -p python3.6 TITLE_OF_VENV
# You now have a directory called TITLE_OF_VENV (You may wish to replace this
# with something more subtle).
# Activate the virtualenv for your current shell session
. TITLE_OF_VENV/bin/activate
# The dot above is intentional and is a quick way to write source, which
# imports the environment vars
Your shell prompt should now look like this: (TITLE_OF_VENV) ubuntu#ip-172-31-61-163:~/project/ToDo-application$ indicating that the venv is active. To switch out of the venv run the command deactivate.
Anything which you install with pip here will then live in the directory TITLE_OF_VENV/python3.6/site-packages (while this virutal environment is active). This has the advantage of keeping different project requirements separate.
Test the python version (with the venv still active):
(TITLE_OF_VENV)$ python --version
Python 3.6
Now install gunicorn into this virtual environment, along with any other project requirements:
(TITLE_OF_VENV)$ pip install gunicorn
(TITLE_OF_VENV)$ pip install -r requirements.txt
Update your uwsgi.py:
import os
# poiting to the project settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "todo_app.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
And then launch it from within the virtual environment:
(TITLE_OF_VENV)$ gunicorn todo_app.wsgi:application
You could add the -D flag to the gunicorn command also, which makes it run in the background. Also don't make this server publicly accessible. If it's a production box, you need to run it behind nginx!
I'm trying to run my django application on heroku.
Folder structure:
app/
Procfile
docs/
...
project/
manage.py
wsgi.py
<django apps>
Procfile
web: gunicorn --pythonpath="$PWD/project" wsgi:application --log-file=-
Error I'm getting:
2015-02-16T16:05:00.646316+00:00 heroku[web.1]: Starting process with command `gunicorn --pythonpath="$PWD/project" wsgi:application --log-file=-`
2015-02-16T16:05:02.697633+00:00 app[web.1]: [2015-02-16 16:05:02 +0000] [3] [INFO] Listening at: http://0.0.0.0:44846 (3)
2015-02-16T16:05:02.709567+00:00 app[web.1]: [2015-02-16 16:05:02 +0000] [9] [INFO] Booting worker with pid: 9
2015-02-16T16:05:02.696968+00:00 app[web.1]: [2015-02-16 16:05:02 +0000] [3] [INFO] Starting gunicorn 19.1.1
2015-02-16T16:05:02.697790+00:00 app[web.1]: [2015-02-16 16:05:02 +0000] [3] [INFO] Using worker: sync
2015-02-16T16:05:02.793753+00:00 app[web.1]: [2015-02-16 16:05:02 +0000] [10] [INFO] Booting worker with pid: 10
2015-02-16T16:05:03.157305+00:00 app[web.1]: Traceback (most recent call last):
2015-02-16T16:05:03.157311+00:00 app[web.1]: File "/app/.heroku/python/bin/gunicorn", line 11, in <module>
2015-02-16T16:05:03.157351+00:00 app[web.1]: sys.exit(run())
2015-02-16T16:05:03.157383+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
2015-02-16T16:05:03.157461+00:00 app[web.1]: WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
2015-02-16T16:05:03.157463+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/base.py", line 185, in run
2015-02-16T16:05:03.157506+00:00 app[web.1]: super(Application, self).run()
2015-02-16T16:05:03.157527+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/base.py", line 71, in run
2015-02-16T16:05:03.157604+00:00 app[web.1]: Arbiter(self).run()
2015-02-16T16:05:03.157607+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 196, in run
2015-02-16T16:05:03.157635+00:00 app[web.1]: self.halt(reason=inst.reason, exit_status=inst.exit_status)
2015-02-16T16:05:03.157656+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 292, in halt
2015-02-16T16:05:03.157730+00:00 app[web.1]: self.stop()
2015-02-16T16:05:03.157744+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 343, in stop
2015-02-16T16:05:03.157814+00:00 app[web.1]: time.sleep(0.1)
2015-02-16T16:05:03.157836+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 209, in handle_chld
2015-02-16T16:05:03.157887+00:00 app[web.1]: self.reap_workers()
2015-02-16T16:05:03.157908+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
2015-02-16T16:05:03.158009+00:00 app[web.1]: raise HaltServer(reason, self.WORKER_BOOT_ERROR)
2015-02-16T16:05:03.158075+00:00 app[web.1]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
2015-02-16T16:05:03.904714+00:00 heroku[web.1]: Process exited with status 1
2015-02-16T16:05:03.914410+00:00 heroku[web.1]: State changed from starting to crashed
Update 1
My wsgi.py file
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config")
os.environ.setdefault("DJANGO_CONFIGURATION", "Production")
from configurations.wsgi import get_wsgi_application
application = get_wsgi_application()
Here I am just adding some text because SO has this silly minimum amount of text that must be written in a question. I mean, I do get it that quality needs to be kept, but if the problem is self explanatory why force people to write unneeded text? Thanks and have a great day!
Adding --preload to the gunicorn command in the Procfile will make your Traceback a lot more readable and show you the actual errors.
Exmaple:
gunicorn project.wsgi:application --preload --workers 1
I had a similar problem, after reading he docs for gunicorn, I was able to add
--log-level debug
to the
web: gunicorn project.wsgi:application --log-file -
such that its
web: gunicorn project.wsgi:application --log-file - --log-level debug
In my case the path was also an issue.
Finally found the solution, it was a missing dependency of django-organizations. I find it crazy that there is no way (at least not that I could find) to see any useful error output from heroku. I finally did
heroku run bash --app <app_name>
and then ran the wsgi.py file line by line, finally getting some meaningful error on
from configurations.wsgi import get_wsgi_application # noqa
It was then clear that it's a missing module error, installed it and everything runs perfectly fine.
Change your Procfile to:
web: gunicorn project.wsgi:application --log-file=-
You're missing the project module in the python path to the wsgi.py file.
I solved it. I followed these steps:
Remove all the unused libraries.
Delete requirements.txt file.
Create a new requirements.txt file.
Commit to Git and then deploy on Heroku.
I have created a Django application but now have plans to use some asynchronous (real-time) functionality in some areas of the site. After doing some research I think I should use gevent-socketio and therefore it is required I switch the application server to Gunicorn.
I have fallen at the first hurdle of deploying Gunicorn, I have installed with the command sudo apt-get install gunicorn and try to run my application with gunicorn project.wsgi:application but it fails and produces the following error:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 473, in spawn_worker
worker.init_process()
File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 100, in init_process
self.wsgi = self.app.wsgi()
File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 115, in wsgi
self.callable = self.load()
File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 33, in load
return util.import_app(self.app_uri)
File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 362, in import_app
__import__(module)
File "/home/alex/django_projects/fantasymatchday_1/fantasymatchday_1/wsgi.py", line 13, in <module>
from django.core.wsgi import get_wsgi_application
ImportError: No module named django.core.wsgi
2014-11-20 17:31:45 [6605] [INFO] Worker exiting (pid: 6605)
2014-11-20 17:31:45 [6600] [INFO] Shutting down: Master
2014-11-20 17:31:45 [6600] [INFO] Reason: Worker failed to boot.
Can anybody give me a clue to what I need to do from here?
I am using python 3.4.0 and Django 1.6
You have to install django and gunicorn in the same environment.
If you use virtualenv make sure you have both in the same virtual environment.
My goal:
I intend to follow "The Twelve-Factor App" methodology for building my Django app on Heroku.
Introduction:
I'm following the "Getting Started with Django on Heroku" quick start guide. At the moment I have the following directory structure:
~/Projects/
hellodjango_rep/
.env (empty)
.git
.gitignore
Procfile
requirements.txt
hellodjango/
manage.py
hellodjango/
__init__.py
settings/
urls.py
wsgi.py
I installed django-toolbelt, created my simple Django application, started the process in my Procfile... Everything seemed to be working fine, but the problems started when I configured the application for the Heroku environment and added:
import dj_database_url
DATABASES['default'] = dj_database_url.config()
to the bottom of my settings.py file.
I pushed my application’s repository to Heroku, visited the app in my browser with $ heroku open successfully, but locally: dj_database_url.config() returned an empty dictionary.
Locally:
OS X 10.8.4
pip==1.4.1
virtualenv==1.10.1
virtualenvwrapper==4.1.1
wsgiref==0.1.2
Postgres.app running on Port 5432
Environment variables:
mac-pol:hellodjango_rep oubiga$ python
>>> import os
>>> os.environ
{
'PROJECT_HOME': '/Users/oubiga/Projects'...
'PATH': '/usr/local/heroku/bin:/usr/local/share/python:/usr/local/bin:/Applications/Postgres.app/Contents/MacOS/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin'...
'HOME': '/Users/oubiga'...
'WORKON_HOME': '/Users/oubiga/Envs'...
'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/oubiga/Envs'...
'PWD': '/Users/oubiga/Projects/hellodjango_rep'
}
hellodjango_venv:
Django==1.5.2
dj-database-url==0.2.2
dj-static==0.0.5
django-toolbelt==0.0.1
gunicorn==18.0
psycopg2==2.5.1
static==0.4
This is what I have in my wsgi.py file:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "hellodjango.hellodjango.settings")
from django.core.wsgi import get_wsgi_application
from dj_static import Cling
application = Cling(get_wsgi_application())
This is what I have in my manage.py file:
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "hellodjango.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
This is what I have in my Procfile:
web: gunicorn hellodjango.hellodjango.wsgi
Environment variables:
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ python hellodjango/manage.py shell
>>> import os
>>> os.environ
{
'PROJECT_HOME': '/Users/oubiga/Projects'...
'PATH': '/Users/oubiga/Envs/hellodjango_venv/bin:/usr/local/heroku/bin:/usr/local/share/python:/usr/local/bin:/Applications/Postgres.app/Contents/MacOS/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin',
'HOME': '/Users/oubiga'...
'WORKON_HOME': '/Users/oubiga/Envs'...
'VIRTUAL_ENV': '/Users/oubiga/Envs/hellodjango_venv'...
'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/oubiga/Envs'...
'PWD': '/Users/oubiga/Projects/hellodjango_rep'...
'DJANGO_SETTINGS_MODULE': 'hellodjango.settings'
}
On Heroku:
Environment variables:
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ heroku run python hellodjango/manage.py shell
>>> import os
>>> os.environ
{
'DATABASE_URL': 'postgres://dbuser:dbpassword#ec2-184-73-162-34.compute-1.amazonaws.com:5432/dbname',
'HEROKU_POSTGRESQL_ORANGE_URL': 'postgres://dbuser:dbpassword#ec2-184-73-162-34.compute-1.amazonaws.com:5432/dbname',
'LIBRARY_PATH': '/app/.heroku/vendor/lib', 'PWD': '/app'...
'DJANGO_SETTINGS_MODULE': 'hellodjango.settings',
'PYTHONHOME': '/app/.heroku/python'...
'PYTHONPATH': '/app/'...
'DYNO': 'run.9068',
'LD_LIBRARY_PATH': '/app/.heroku/vendor/lib'...
'HOME': '/app', '_': '/app/.heroku/python/bin/python',
'PATH': '/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin'...
}
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ heroku config
=== damp-dusk-5382 Config Vars
DATABASE_URL: postgres://dbuser:dbpassword#ec2-184-73-162-34.compute-1.amazonaws.com:5432/dbname
HEROKU_POSTGRESQL_ORANGE_URL: postgres://dbuser:dbpassword#ec2-184-73-162-34.compute-1.amazonaws.com:5432/dbname
Research:
Store config in the environment: from The Twelve-Factor App
#adamwiggins wrote:
The twelve-factor app stores config in environment variables... Env
vars are easy to change between deploys without changing any code;
unlike config files.
dj_database_url.config() is returning an empty object: from Heroku Forums
#chrisantonick replied:
... dj_database_url.config() gets the Postgres credentials from the
Heroku environment variables. But, on your local machine, those
variables aren't there. You have to put them in your
/venv/bin/activate shell script... put the variables in there.
something like DATABASE_URL = "xxx" export DATABASE_URL
For each thing it needs. Then... "deactivate"... and ..."activate"
again to restart it.
Getting Started with Django and Heroku instructions raised ImproperlyConfigured error: from Heroku Forums
#jwpe replied:
... dj-database-url is a great utility, as it allows you to use
exactly the same settings.py code in your development and production
environments, as recommended in the "12 factor app principles"... what
dj_database_url.config() is doing is looking for the DATABASE_URL
environment variable, and then parsing it into Django's preferred
format... if you haven't manually created and promoted a postgres DB
on Heroku, DATABASE_URL will not be present and the
ImproperlyConfigured error will be raised. Setting the default for
dj_database_url.config() as your local DB URL is one way to make sure
that your application will work in a development environment.
However, it is not necessarily the only way. Perhaps a better
alternative is to manually set DATABASE_URL in your local .env file.
Then, when running your app locally using Foreman, it will be loaded
as an environment variable and dj_database_url will find it. So your
.env would contain:
DATABASE_URL=postgres://user:pass#localhost/dbname
Meaning that in settings.py you would only need to have:
DATABASES['default']= dj_database_url.config()
...The advantage of using a local environment variable instead of a
single, hard-coded default is that your code will run in any
environment where the DATABASE_URL is set. If you change the name of
your local DB, or want to run your code on a different dev machine,
you only need to update your .env file instead of tinkering with
settings.py.
How to manage production/staging/dev Django settings?: from Heroku Forums
#rdegges replied:
... trying to get your application to behave in a way such that:
When you're running the app on your laptop, it uses your local Postgres server.
When you're running the app on your staging Heroku app, it uses the Postgres server addon.
When you're running the app on your production Heroku app, it uses the Postgres server addon.
The best way to accomplish this is by using environment variables!…
Environment variables are the most elegant (and scalable) way to
handle application configuration between different environments…
Instead of having many settings files, define a single file:
settings.py, and have it make use of environment variables to pull
service information and credentials... On Heroku, you can set
environment variables manually by running:
$ heroku config:set SOME_VARIABLE=some_value
... there's always Kenneth Reitz's great autoenv tool. This lets you
define a simple .env file in your project directory… And each time you
enter your project directory, those environment variables will be
automatically set so that you don't have to do anything special! Just
run your project and everything will work as expected: python
manage.py runserver
As a First Attempt:
I manually set DATABASE_URL in my .env file: DATABASE_URL=postgres://dbuser:dbpassword#ec2-184-73-162-34.compute-1.amazonaws.com:5432/dbname
But when I run $ foreman start command:
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ foreman start
17:25:39 web.1 | started with pid 319
17:25:39 web.1 | 2013-09-11 17:25:39 [319] [INFO] Starting gunicorn 18.0
17:25:39 web.1 | 2013-09-11 17:25:39 [319] [INFO] Listening at: http://0.0.0.0:5000 (319)
17:25:39 web.1 | 2013-09-11 17:25:39 [319] [INFO] Using worker: sync
17:25:39 web.1 | 2013-09-11 17:25:39 [322] [INFO] Booting worker with pid: 322
and tried to open my app in the browser http://0.0.0.0:5000:
17:26:59 web.1 | 2013-09-11 10:26:59 [322] [ERROR] Error handling request
17:26:59 web.1 | Traceback (most recent call last):
17:26:59 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 131, in handle_request
17:26:59 web.1 | respiter = self.wsgi(environ, resp.start_response)
17:26:59 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/dj_static.py", line 59, in __call__
17:26:59 web.1 | return self.application(environ, start_response)
17:26:59 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 236, in __call__
17:26:59 web.1 | self.load_middleware()
17:26:59 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 53, in load_middleware
17:26:59 web.1 | raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
17:26:59 web.1 | ImproperlyConfigured: Error importing middleware django.contrib.auth.middleware: "dlopen(/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Library not loaded: #loader_path/../lib/libssl.1.0.0.dylib
17:26:59 web.1 | Referenced from: /Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/psycopg2/_psycopg.so
17:26:59 web.1 | Reason: image not found"
However, dj_database_url.config() returned:
{
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbname',
'HOST': 'ec2-23-21-196-147.compute-1.amazonaws.com',
'USER': 'dbuser',
'PASSWORD': 'dbpassword',
'PORT': 5432
}
As a Second Attempt:
I manually set DATABASE_URL in my .env file changing the host. I replaced "ec2-184-73-162-34.compute-1.amazonaws.com:5432" by "localhost:5000". $ deactivate and then $ workon hellodjango_venv again.
DATABASE_URL=postgres://dbuser:dbpassword#localhost:5000/dbname
But, when I run $ foreman start command:
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ foreman start
17:38:41 web.1 | started with pid 687
17:38:41 web.1 | 2013-09-11 17:38:41 [687] [INFO] Starting gunicorn 18.0
17:38:41 web.1 | 2013-09-11 17:38:41 [687] [INFO] Listening at: http://0.0.0.0:5000 (687)
17:38:41 web.1 | 2013-09-11 17:38:41 [687] [INFO] Using worker: sync
17:38:41 web.1 | 2013-09-11 17:38:41 [690] [INFO] Booting worker with pid: 690
and tried to open my app in the browser http://0.0.0.0:5000:
17:38:46 web.1 | 2013-09-11 10:38:46 [690] [ERROR] Error handling request
17:38:46 web.1 | Traceback (most recent call last):
17:38:46 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 131, in handle_request
17:38:46 web.1 | respiter = self.wsgi(environ, resp.start_response)
17:38:46 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/dj_static.py", line 59, in __call__
17:38:46 web.1 | return self.application(environ, start_response)
17:38:46 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 236, in __call__
17:38:46 web.1 | self.load_middleware()
17:38:46 web.1 | File "/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 53, in load_middleware
17:38:46 web.1 | raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
17:38:46 web.1 | ImproperlyConfigured: Error importing middleware django.contrib.auth.middleware: "dlopen(/Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Library not loaded: #loader_path/../lib/libssl.1.0.0.dylib
17:38:46 web.1 | Referenced from: /Users/oubiga/Envs/hellodjango_venv/lib/python2.7/site-packages/psycopg2/_psycopg.so
17:38:46 web.1 | Reason: image not found"
This time, dj_database_url.config() returned:
{
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbname',
'HOST': 'localhost',
'USER': 'dbuser',
'PASSWORD': 'dbpassword',
'PORT': 5000
}
As a Third Attempt:
I installed autoenv mac-pol:~ oubiga$ pip install autoenv
From this Cookbook Kenneth Reitz wrote, I put:
use_env() {
typeset venv
venv="$1"
if [[ "${VIRTUAL_ENV:t}" != "$venv" ]]; then
if workon | grep -q "$venv"; then
workon "$venv"
else
echo -n "Create virtualenv $venv now? (Yn) "
read answer
if [[ "$answer" == "Y" ]]; then
mkvirtualenv "$venv"
fi
fi
fi
}
in my .bashrc file.
I run $ foreman start command:
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ foreman start
18:11:57 web.1 | started with pid 1104
18:11:57 web.1 | 2013-09-11 18:11:57 [1104] [INFO] Starting gunicorn 18.0
18:11:57 web.1 | 2013-09-11 18:11:57 [1104] [INFO] Listening at: http://0.0.0.0:5000 (1104)
18:11:57 web.1 | 2013-09-11 18:11:57 [1104] [INFO] Using worker: sync
18:11:57 web.1 | 2013-09-11 18:11:57 [1107] [INFO] Booting worker with pid: 1107
and tried to open my app in the browser http://0.0.0.0:5000: It worked!
^CSIGINT received
18:12:06 system | sending SIGTERM to all processes
18:12:06 web.1 | 2013-09-11 11:12:06 [1107] [INFO] Worker exiting (pid: 1107)
SIGTERM received
18:12:06 web.1 | 2013-09-11 18:12:06 [1104] [INFO] Handling signal: int
18:12:06 web.1 | 2013-09-11 18:12:06 [1104] [INFO] Shutting down: Master
18:12:06 web.1 | exited with code 0
But, dj_database_url.config() again returns an empty dictionary.
As a Final Attempt:
I was curious about the python manage.py runserver command and I checked it out.
(hellodjango_venv)mac-pol:hellodjango_rep oubiga$ foreman run python hellodjango/manage.py runserver
Validating models...
0 errors found
September 11, 2013 - 18:42:37
Django version 1.5.2, using settings 'hellodjango.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
and tried to open my app in the browser http://127.0.0.1:8000/: It didn't work!
An ImportError: No module named hellodjango.urls was raised.
I replaced ROOT_URLCONF = 'hellodjango.hellodjango.urls' in my settings.py file by ROOT_URLCONF = 'hellodjango.urls' and it finally worked.
As expected, dj_database_url.config() returned an empty dictionary.
So:
Now, I feel a little overwhelmed. I'm afraid I'm misunderstanding some core concept here.
What's the point to use gunicorn instead of the Django development server?
Why does dj_database_url.config() sometimes return a fully populated dictionary, and sometimes an empty one?
Can I manually set environment variables in the .env file? Do I need to install tools like autoenv, heroku-config...?
Thank you in advance.
I got stuck with the postgres as well, here's what I did in the settings.py to add local settings:
DATABASES = {
'default': dj_database_url.config(default='postgres://<user>:<password>#<host>/<dbname>')
}
Of course you have to have created the database following postgres steps.
Solution was from https://discussion.heroku.com/t/dj-database-url-config-is-returning-an-empty-object/55/9
I can help with the third question in your post:
"Can I manually set environment variables in the .env file? Do I need to install tools like autoenv, heroku-config...?"
I ran into some similar confusion when deploying my django app to heroku and hope this helps you or someone else.
Locally:
I used django-environ to help set my environment variables locally. I set all the variables in the .env file,
In the settings.py file, I imported environ, added this to the top of the file:
env = environ.Env( # set casting, default value DEBUG=(bool, False) )
and then added 'environ' to the list of INSTALLED_APPS. Finally I retrieved the environment variables with this format:
'NAME': os.environ['NAME'], 'USER': os.environ['USER'], 'PASSWORD': os.environ['PASSWORD']
Remotely:
I used heroku config to see what environment variables, if any, were already set. To set environment variables in heroku you can use the gui or heroku config:set SECRET_KEY='whateveryoursecretekeyis'
If you aren't sure what your credentials are for the heroku database, as they are different to what you're running locally, you can run heroku pg:credentials:url which lists it out for you.
One gotcha for me as a beginner was that you need to be sure that whatever variables you set in your settings.py file are declared as environment variables locally and remotely.
Another thing that might help you or someone else searching for this question is that if you were like me and first built your django app (meaning editing views building out models etc) and then started work on getting it deployed to heroku, you might want to start over, re-create the simple basic django app (barebones with the rocket ship landing page) and deploy that to heroku to make sure that pipeline works.
Then, after you're sure your app works both locally and remotely, and have edited your environment variables and set up the correct database for your project (for example postgres maybe, instead of the default database that django uses), you can go back and fill out your models and views.
Hope this helps someone!
Also just noticed your procfile seems off.
Instead of:
web: gunicorn hellodjango.hellodjango.wsgi
You'd want:
web: gunicorn hellodjango.wsgi
Since the project name is hellodjango.
Would greatly appreciate input from anyone with experience configuring redis as a backend for a celery-brokered django project on heroku. My task scheduling worked fine from localhost but I'm finding it really frustrating getting it deployed on heroku:
At the moment I'm running 3 dynos, 1 web, 1 scheduler and 1 worker
I added the redistogo addon to my project. Redistogo set to the free nano plan, which gives me 10 connections, 1 DB and a 5MB size instance
I followed the redistogo documentation (https://devcenter.heroku.com/articles/redistogo#install-redis-in-python) for configuring settings.py and, alternatively, also tried implementing a variation of the solution here. Neither working for me. Here's what I have in settings.py:
redis_url = os.environ.get('REDISTOGO_URL', 'http://localhost:6959')
CACHES = {
'default': {
'BACKEND': 'redis_cache.RedisCache',,
'LOCATION': '%s:%s' % (redis_url.hostname, redis_url.port),
'OPTIONS': {
'DB': 0, # or 1?
'PASSWORD': redis_url.password,
#'PARSER_CLASS': 'redis.connection.HiredisParser'
},
},
}
CELERY_RESULT_BACKEND = redis_url
BROKER_URL = 'redis://localhost:6959/0'
Here's my heroku logs when I try to run the app:
2013-07-11T12:16:10.998516+00:00 app[web.1]: apps = settings.INSTALLED_APPS
2013-07-11T12:16:10.998516+00:00 app[web.1]: mod = importlib.import_module(self.SETTINGS_MODULE)
2013-07-11T12:16:10.998263+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
2013-07-11T12:16:10.998263+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 263, in fetch_command
2013-07-11T12:16:10.998516+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
2013-07-11T12:16:10.998516+00:00 app[web.1]: self._setup(name)
2013-07-11T12:16:10.998516+00:00 app[web.1]: self._wrapped = Settings(settings_module)
2013-07-11T12:16:10.998516+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 132, in __init__
2013-07-11T12:16:10.998516+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
2013-07-11T12:16:10.998516+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in _setup
2013-07-11T12:16:10.998712+00:00 app[web.1]: 'LOCATION': '%s:%s' % (redis_url.hostname, redis_url.port),
2013-07-11T12:16:10.998712+00:00 app[web.1]: AttributeError: 'str' object has no attribute 'hostname'
2013-07-11T12:16:12.201202+00:00 heroku[web.1]: Process exited with status 1
2013-07-11T12:16:12.250743+00:00 heroku[web.1]: State changed from starting to crashed
how do I get redis_url treated like a URI and not a str?
my procfile:
web: python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 3 --log-level info
scheduler: python manage.py celeryd -B -E
worker: python manage.py celeryd -E -B --loglevel=INFO
In requirements I have django-redis-cache==0.10.0, redis==2.7.6, django-celery==3.0.17, celery celery==3.0.20 and kombu==2.5.12
Use the python urlparse library. It parses URLs into components.
redis_url = os.environ.get('REDISTOGO_URL', 'http://localhost:6959')
redis_url = urlparse.urlparse(redis_url)
Looks like os.environ.get is returning a String (or str? not to familiar with python) and you're expecting it to be more like a URI object or something. Do normal python strings respond to methods like hostname?
The documentation also has this step:
redis = redis.from_url(redis_url)
which according to these docs that parses the string into a redis object.