Why isn't collectstatic being run automatically when I deploy my Django app to Heroku? - django

I've followed the official Heroku docs on Django and Static Assets; I've installed dj-static and added it to my requirements.txt file, properly configured all the variables in my settings.py file:
STATIC_ROOT = os.path.join(CONFIG_ROOT, 'served/static/')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
os.path.join(CONFIG_ROOT, 'static'),
)
And this is what my wsgi.py looks like:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_django_project.settings")
from django.core.wsgi import get_wsgi_application
from dj_static import Cling
application = Cling(get_wsgi_application())
The contents of Procfile:
web: gunicorn --bind 0.0.0.0:$PORT my_django_project.wsgi:application
In the docs, it says that "collectstatic is run automatically when it is configured properly." But when I navigate to my site there's clearly no css.
I've tried debugging using heroku run, but that just copies the static files as expected.
I've noticed that when I include the collectstatic command in my Procfile, i.e.
web: python my_django_project/manage.py collectstatic --noinput ; gunicorn -b 0.0.0.0:$PORT my_django_project.wsgi:application
...that works as expected, and the static files are served.
However what's strange about that is when I run heroku run bash and look inside the directory that STATIC_ROOT is pointing to, there's nothing there! In fact, the entire served/ directory is missing, and yet, the static files are still being served!
I'd still like to know why isn't collectstatic being run automatically though -- like mentioned in the docs -- when I deploy my Django app to Heroku.

It looks like you might be using a specific settings module for Heroku/production. Further, you've set the environment variable DJANGO_SETTINGS_MODULE to point to this settings module (and that way, when the app runs, Django knows to use that one and not, say, your default/development one). Finally, you've probably configured static asset settings in Heroku/production settings module (perhaps, STATIC_ROOT).
Okay, so if this is all correct, then here is the issue: heroku environment variables are only set at serve-time, not at compile-time. This is important because collectstatic is a compile-time operation for Heroku. (Heroku goes through 2 stages when you push: 1) compiling, which involves setting the application up (collectstatic, syncdb, etc) 2) serving, the normal operation of your application).
So, essentially, you've done everything correctly, but Heroku hasn't exposed your environment variables, including your specification of a different settings module, to collectstatic.
To have your environment variables set to compile-time, enable Heroku's user-env-compile lab feature like this:
heroku labs:enable user-env-compile
I think this is a silly thing to do by default, and would be interested in hearing why Heroku thought it was a good idea.

Have you tried adding the user_env_compile setting to your heroku config?
heroku labs:enable user-env-compile
With that enabled collectstatic should be run whenever you deploy to heroku automatically.

I am using the heroku python buildpack with dokku, and collectstatic was not being run because it had no execute permission. They fixed that in a recent commit (Dec 13, 2013), so it should work now.

Related

whitenoise not adding random string

I just put up another copy of my Django app in production. The only intended difference between my "old" app and the "new" one is that the old app runs in "traditional" Heroku, and the new app runs in "Dockerized" Heroku.
The weirdest thing is happening. I use Whitenoise to serve my static assets. On the older app, links to my static assets look like this in the generated pages:
<link href="/static/assets/css/bootstrap.min.9f236e18d5bf.css" rel="stylesheet">
But on my new app, that little string (9f236e18d5bf) is missing. it looks like:
<link href="/static/assets/css/bootstrap.min.css" rel="stylesheet">
As a result, I don't have any stylesheets on my new site.
Is there an additional step that I have to take in the Dockerized version to get this to work? Or am I missing some combination of STATIC* settings values that are needed.
My temporary workaround was to add python manage.py collectstatic --noinput to Dockerfile.web, and then set:
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
Which works, but I'm wondering if I'm going to get a scaling issue on Monday.
It looks the answer comes in two parts, and I had one of each working at different times.
Be sure to have this in your Dockerfile.web:
RUN python manage.py collectstatic --noinput
CMD CMD gunicorn --bind 0.0.0.0:8000 <project-name>.wsgi
Note that you want to use RUN for your collectstatic command. Apparently only the last CMD value in a Dockerfile makes any sense.
In settings.py, have
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
That setting will cause collectstatic to add the hashes to the names of each file that is copied into the directory specified in STATICFILES_ROOT.

serve static file with herokku, django and wsgi

I was working to deploy a website to heroku. The site seems to recognize my static files if I run with django's builtin server namely "runserver". However if I run with gunicorn The static files cannot be recognized. I was wondering if there's any special setting I need to tweak to magically make the recognition happen. Can anyone enlighten me how these two commands differ specifically or does it have anything to do with wsgi staff?
Thanks!!!
This is how I do with runserver in Procfile, which is quite neat.
web:python manage.py runserver
And here's what I do with gunicorn in Procfile, which is a mess
web: gunicorn some.dotted.path.to.mywsgi:application
UPDATE
Luckily I worked around this problem by including the following line to my urls.py. Though I know it's not a perfect solution because in real word, you need to shutdown DEBUG. but as for now in development. it's working well. Can anyone explain what this line magically do?
if settings.DEBUG:
urlpatterns += patterns('django.contrib.staticfiles.views',
url(r'^static/(?P<path>.*)$', 'serve'),
)
Serving static files on Heroku with Django is a bit tricky. Assuming you're using the 'staticfiles' app, you have to run 'collectstatic' to collect your static files after deploying. The problem with Heroku is that running 'collectstatic' in the shell will actually run in a new dyno, which disappears as soon as it's finished.
One potential solution is outlined here:
Basically, the idea is to combine a few commands in your Procfile so that 'collectstatic' is run during the dyno spinup process:
web: python my_django_app/manage.py collectstatic --noinput; bin/gunicorn_django --workers=4 --bind=0.0.0.0:$PORT my_django_app/settings.py
You also have to add the 'static' views to your urls.py (see https://docs.djangoproject.com/en/dev/howto/static-files/#serving-files-uploaded-by-a-user, but duplicate for STATIC_URL and STATIC_ROOT). It's worth noting that the Django docs recommend against using this in production.
This solution isn't ideal though, since you are still using your gunicorn process to serve static files. IMHO the best approach to dealing with static files on Heroku is to host them on something like S3.

Django serves static files with runserver but not with foreman

I have the opposite problem to the one described in this question.
My Django site works correctly when the server is started using manage.py runserver but static files are not served when the server is started with foreman start.
My directory structure:
project_name/
project/
settings.py
...
app/
...
venv/
...
public/
static/
# static files go here #
media/
...
Procfile
requirements.txt
manage.py
Procfile (as described in the Getting Started with Django on Heroku tutorial):
web: gunicorn project.wsgi
settings.py:
import os
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
UP_ROOT = os.path.abspath(os.path.join(SITE_ROOT, '..'))
...
MEDIA_ROOT = UP_ROOT + '/public/media/'
...
STATIC_ROOT = UP_ROOT + '/public/static'
...
STATIC_URL = '/static/'
...
STATICFILES_DIRS = (
UP_ROOT + '/public',
UP_ROOT + '/public/static',
)
Like I said, all of this works correctly with runserveron my local machine, but not with foreman start.
Will post more info if requested.
The difference between runserver and foreman:
Django's runserver command serves static files because django.contrib.staticfiles automatically searches through your project's static folders for you (provided a few prerequisites are in place here) and returns them on static requests.
Foreman, however, is a separate utility from Django and knows nothing about Django's internals. So, you need some added help. Two good options are dj-static or whitenoise. Either can be installed using pip. You will need to make a small modification to your wsgi.py file, as described in the setup instructions for whichever you choose (see links).
Both dj-static and whitenoise look in STATIC_ROOT in your settings.py file for static files. This is different than Django's runserver, which automatically traverses the various static folders in your project. This means you need to run manage.py collectstatic (which gathers your static files into STATIC_ROOT) before dj-static or whitenoise will find them.
Here is a set of example steps (for whitenoise):
Run:
pip install whitenoise
Modify wsgi.py to look like this:
from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(get_wsgi_application())
Then static files will work:
# Run collectstatic before foreman whenever you've changed static files
python manage.py collectstatic
foreman start
Foreman requires a little more effort than runserver. However, foreman can closely mimic a production environment, whereas runserver will not (hopefully, it will not). Using dj-static or whitenoise in production is an okay choice in many situations. So, developing using foreman with dj-static or whitenoise will give you reassurance that major problems won't surprise you when your site moves to production. You don't get that reassurance from some other options (e.g. adding 'django.views.static.serve' to urls.py).
This is a bit of a hack but I got it working by adding this route to the urls:
# serve static
(r'^static/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.STATIC_ROOT}),

Heroku collectstatic not run during deployment

I have a django app that I am successfully deplying to heroku. When I dry-run the collectstatic command locally everything works fine.
python manage.py collectstatic --dry-run --noinput
....
Pretending to copy '/Users/hari/.virtualenvs/bsc2/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/admin/ordering.js'
Pretending to copy '/Users/hari/.virtualenvs/bsc2/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/admin/RelatedObjectLookups.js'
71 static files copied.
Despite this ..my django admin staticfiles do not get used and I get a bare-bones django admin site on heroku with Debug set to False.
If I set Debug to True I get a "rich" admin site on heroku. With Debug set to True or False "git push heroku master " command terminal output does not have anything about collecting staticfiles.
I tried the example "helloworld" application that uses gunicorn from Heroku and that did display the "collecting static" messages.I also tried inserting this code snippet into my urls.py. But that too does not help.
from django.conf import settings
if not settings.DEBUG:
urlpatterns += patterns('',
(r'^static/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.STATIC_ROOT}),
Next, I tried adding the following to my heroku config
heroku config:add DISABLE_COLLECTSTATIC=0
But that too did not show my django admin site with all the styles.
Finally I tried switching to gunicorn with my Procfile and that also did not show the admin styles. Only setting Debug=True works to show my admin styles.
I tried this with Django 1.4.2 and 1.5.1 on Heroku and neither is showing me a "normal" admin site. Is there any way out to have my admin files on heroku without going the S3 route.
command terminal output does not have anything about collecting staticfiles.
Looking at heroku-buildpack-python:bin/steps/collectstatic it seems that it tries to do a collectstatic --dry-run --noinput and pipes it's output to /dev/null before displaying the -----> Collecting static files message. This means that if there is an error there that is not present on your local box, you will never see the error on heroku : it will silently fail. (The best kind of failure ;)
have you tried running a one-off worker to test out the collectstatic command and see if it's a problem in their environment?
heroku run python manage.py collectstatic --dry-run --noinput
If this fails, it will give you an error or traceback to look into and further diagnose the issue.
Check this out Django and Static Assets. It seems to be updated recently and you can serve static files nicely using this dj-static package.
Try these three things:
Create this Heroku config variable: DJANGO_SETTINGS_MODULE with a
value of myapp.settings.prod--or as appropriate for your Heroku settings file
Use Whitenoise as described in the Heroku docs:
https://devcenter.heroku.com/articles/django-assets
Check it in and redeploy your dyno: git push heroku master
I found I was missing the first item, the DJANGO_SETTINGS_MODULE" e.g., Command line collectstatic would work but that didn't matter b/c it was an ephemeral dyno
throwing it out there, since dumb mistakes happen:
I spent much time trying to figure out why my module wasn't found on heroku when it was working fine locally, only to realize it was ignored by an entry into .slugignore

Using collectstatic with multiple environments

I have a Django app on Heroku, with staging and production environments. Static files are hosted on S3. I'm streamlining my deployment process and plan to set up fabfiles once I have things working manually.
How can I configure collectstatic to push to multiple places? If I run it locally, it uses my dev settings (with a local STATIC_ROOT). If I run it on one of my Heroku apps (heroku run ./manage.py collectstatic), then it can't grab the files (since .slugignore ensures they're never pushed to Heroku). The same applies if I include collectstatic in my Procfile.
I'm also using django-pipeline, though it's not yet doing much since I'm stuck on the collectstatic bit.
UPDATE
In response to Marat's question, I tried passing a settings file as an option to collectstatic: ./manage.py collectstatic --settings=project.settings.prod, but got an error: Unknown command: 'collectstatic' I checked on the server though and Installed Apps does include django.contrib.staticfiles and I can also run collectstatic remotely, so I'm not sure what would cause that.
You can set the environment variable DJANGO_SETTINGS_MODULE so you don't need specify --settings everywhere:
heroku config:set DJANGO_SETTINGS_MODULE=project.settings.prod
First, if you are going to serve static via CloudFront, you can use custom origin and always use local STATIC_ROOT. Actually it has some advantages over S3 source, eg gzip support.
Another good thing you can do is to have environment dependent settings in a separate file and then import it in settings.py, eg:
local_settings.py (not in project repository, yet you can have local_settings.py.example):
#environment dependent settings
DATABASES = { .. }
CACHES = { .. }
STATIC_ROOT = 'your_path/static'
settings.py:
import local_settings
I've just replied a similar question on Upload Media from Heroku to Amazon S3. If you customise your settings to take in account environmental vars, you can use filesystem storage backends locally and S3 storage backends when pushing to Heroku. This will collect and upload your static files when your slug is compiled.