Django serves static files with runserver but not with foreman - django

I have the opposite problem to the one described in this question.
My Django site works correctly when the server is started using manage.py runserver but static files are not served when the server is started with foreman start.
My directory structure:
project_name/
project/
settings.py
...
app/
...
venv/
...
public/
static/
# static files go here #
media/
...
Procfile
requirements.txt
manage.py
Procfile (as described in the Getting Started with Django on Heroku tutorial):
web: gunicorn project.wsgi
settings.py:
import os
SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
UP_ROOT = os.path.abspath(os.path.join(SITE_ROOT, '..'))
...
MEDIA_ROOT = UP_ROOT + '/public/media/'
...
STATIC_ROOT = UP_ROOT + '/public/static'
...
STATIC_URL = '/static/'
...
STATICFILES_DIRS = (
UP_ROOT + '/public',
UP_ROOT + '/public/static',
)
Like I said, all of this works correctly with runserveron my local machine, but not with foreman start.
Will post more info if requested.

The difference between runserver and foreman:
Django's runserver command serves static files because django.contrib.staticfiles automatically searches through your project's static folders for you (provided a few prerequisites are in place here) and returns them on static requests.
Foreman, however, is a separate utility from Django and knows nothing about Django's internals. So, you need some added help. Two good options are dj-static or whitenoise. Either can be installed using pip. You will need to make a small modification to your wsgi.py file, as described in the setup instructions for whichever you choose (see links).
Both dj-static and whitenoise look in STATIC_ROOT in your settings.py file for static files. This is different than Django's runserver, which automatically traverses the various static folders in your project. This means you need to run manage.py collectstatic (which gathers your static files into STATIC_ROOT) before dj-static or whitenoise will find them.
Here is a set of example steps (for whitenoise):
Run:
pip install whitenoise
Modify wsgi.py to look like this:
from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(get_wsgi_application())
Then static files will work:
# Run collectstatic before foreman whenever you've changed static files
python manage.py collectstatic
foreman start
Foreman requires a little more effort than runserver. However, foreman can closely mimic a production environment, whereas runserver will not (hopefully, it will not). Using dj-static or whitenoise in production is an okay choice in many situations. So, developing using foreman with dj-static or whitenoise will give you reassurance that major problems won't surprise you when your site moves to production. You don't get that reassurance from some other options (e.g. adding 'django.views.static.serve' to urls.py).

This is a bit of a hack but I got it working by adding this route to the urls:
# serve static
(r'^static/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.STATIC_ROOT}),

Related

Django serve files from STATIC_ROOT and use AppDirectoriesFinder at same time

I'm trying and failing to figure out a config for serving static files that works for me under these requirements:
I want the collectstatic command to work always (when Debug=True, and Debug=False), and to collect app dir statics. This implies I need both 'django.contrib.staticfiles' in INSTALLED_APPS and 'django.contrib.staticfiles.finders.AppDirectoriesFinder' in STATICFILES_FINDERS.
I want to serve some files out of STATIC_ROOT with Debug=True (Debug=False I want to serve all files out of STATIC_ROOT), that don't belong to any particular app. I just do.
I want the dev server (./manage.py runserver) AppDirectoriesFinder lookups to work when Debug=True.
As a compromise I could live with all static files out of STATIC_ROOT, but I still need Debug=True and collectstatic to work.
But I can't figure out how to do this.
I can't add STATIC_ROOT to STATICFILES_DIRS (not allowed)
Adding urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) to urls as suggested here does seem to not work in Debug=True with the dev server because the dev server overrides it and uses static finders.
Any ideas?

Why isn't collectstatic being run automatically when I deploy my Django app to Heroku?

I've followed the official Heroku docs on Django and Static Assets; I've installed dj-static and added it to my requirements.txt file, properly configured all the variables in my settings.py file:
STATIC_ROOT = os.path.join(CONFIG_ROOT, 'served/static/')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
os.path.join(CONFIG_ROOT, 'static'),
)
And this is what my wsgi.py looks like:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_django_project.settings")
from django.core.wsgi import get_wsgi_application
from dj_static import Cling
application = Cling(get_wsgi_application())
The contents of Procfile:
web: gunicorn --bind 0.0.0.0:$PORT my_django_project.wsgi:application
In the docs, it says that "collectstatic is run automatically when it is configured properly." But when I navigate to my site there's clearly no css.
I've tried debugging using heroku run, but that just copies the static files as expected.
I've noticed that when I include the collectstatic command in my Procfile, i.e.
web: python my_django_project/manage.py collectstatic --noinput ; gunicorn -b 0.0.0.0:$PORT my_django_project.wsgi:application
...that works as expected, and the static files are served.
However what's strange about that is when I run heroku run bash and look inside the directory that STATIC_ROOT is pointing to, there's nothing there! In fact, the entire served/ directory is missing, and yet, the static files are still being served!
I'd still like to know why isn't collectstatic being run automatically though -- like mentioned in the docs -- when I deploy my Django app to Heroku.
It looks like you might be using a specific settings module for Heroku/production. Further, you've set the environment variable DJANGO_SETTINGS_MODULE to point to this settings module (and that way, when the app runs, Django knows to use that one and not, say, your default/development one). Finally, you've probably configured static asset settings in Heroku/production settings module (perhaps, STATIC_ROOT).
Okay, so if this is all correct, then here is the issue: heroku environment variables are only set at serve-time, not at compile-time. This is important because collectstatic is a compile-time operation for Heroku. (Heroku goes through 2 stages when you push: 1) compiling, which involves setting the application up (collectstatic, syncdb, etc) 2) serving, the normal operation of your application).
So, essentially, you've done everything correctly, but Heroku hasn't exposed your environment variables, including your specification of a different settings module, to collectstatic.
To have your environment variables set to compile-time, enable Heroku's user-env-compile lab feature like this:
heroku labs:enable user-env-compile
I think this is a silly thing to do by default, and would be interested in hearing why Heroku thought it was a good idea.
Have you tried adding the user_env_compile setting to your heroku config?
heroku labs:enable user-env-compile
With that enabled collectstatic should be run whenever you deploy to heroku automatically.
I am using the heroku python buildpack with dokku, and collectstatic was not being run because it had no execute permission. They fixed that in a recent commit (Dec 13, 2013), so it should work now.

deploying Django - Dreamhost

I am neewbie with django and I am stuck trying to deploy my project on the server ( a shared Dreamhost server). I have followed the tutorials I found but it doesnt work. My project structure is:
/usr/mydomain.com
- public
* media
* static
- project
* __init__.py
* manage.py
* settings.py
* urls.py
* views.py
- tmp
- django-setup.py
- manage.py
- passenger_wsgi.py
- passenger_wsgi.pyc
It only works on my domain.com/admin, but I have a simple 'hello world' view to try but it isnt showing. I get only an HTTP 404 when I access my domain...
EDIT
I have installed Python 2.7 and Django 1.5 . Now I get a 500 Internal Server Error.
This is how I do it, with Django 1.5, on DreamHost:
A virtualenv dedicated to my project is in ~/virtualenv/myproject
My Django project is in ~/projects/myproject, with setup:
The database file is in the project root, named sqlite3.db
STATIC_ROOT is set to os.path.join(os.path.abspath(os.path.dirname(__file__)), '..', 'static'), that is, the static directory in the project root.
When the static files are updated in the project, I have to run python manage.py collectstatic
I have per-environment settings in ~/projects/myproject/myproject, named prod_settings.py, beta_settings.py, and so on.
My site is in ~/sites/www.myproject.com, the layout inside:
myproject -- symlink to the Django project
sqlite3.db -- symlink to the database file in the Django project
public/static -- symlink to the STATIC_ROOT defined in the Django project
tmp/restart.txt -- touch this file to have Passenger reload the site settings
passenger_wsgi.py -- the Passenger configuration
Create passenger_wsgi.py like this:
projectname = 'myproject'
virtualenv_root = '/home/jack/virtualenv/' + projectname
import sys
import os
INTERP = os.path.join(virtualenv_root, 'bin', 'python')
if sys.executable != INTERP:
os.execl(INTERP, INTERP, *sys.argv)
sys.path.append(os.path.join(os.getcwd(), projectname))
os.environ['DJANGO_SETTINGS_MODULE'] = projectname + '.prod_settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Create prod_settings.py like this:
from myproject.settings import *
DEBUG = False
TEMPLATE_DEBUG = DEBUG
ADMINS = (('My Project', 'info#myproject.com'), )
MANAGERS = ADMINS
DATABASES = {} # Appropriately for your production environment
SECRET_KEY = "..." # Your secret key
ALLOWED_HOSTS = ["myproject.com", "www.myproject.com"]
For Django 1.4 or earlier, you need to make some minor modifications, but the same idea works. I've been using variations of this setup since Django 1.2.
I setup virtualenv like this:
# install pip and virtualenv in my home directory
easy_install --user pip
pip install --user virtualenv
# create a virtualenv dedicated to my django project
mkdir ~/virtualenv
virtualenv --distribute ~/virtualenv/myproject
# activate the virtualenv, install django and all project dependencies
. ~/virtualenv/myproject/bin/activate
cd ~/projects/myproject
pip install -r requirements.txt
Notice that after activating the virtualenv, you don't need the --user flag anymore when you install packages with pip. When a virtualenv is active, all packages are installed in that virtualenv.
About requirements.txt:
Create it in your development like this: pip freeze > requirements.txt
Edit it, leave only the packages that your project really really needs. It's better to remove too much and add back later as needed.
Note that virtualenv is not necessary, but recommended. You can do without by setting INTERP in your passenger_wsgi.py to /usr/bin/python. virtualenv is useful to have multiple Django sites with different dependencies on the same account. It is also useful when you want to upgrade Django for your site, so that you can test the upgrade on beta.myproject.com without disrupting traffic on your main site.
Finally, if you are using shared hosting, DreamHost support encourages using different user accounts for each website, but I don't know what difference that makes. Be careful with heavy operations, for example large data imports, because if the process uses much memory, it can get killed by DreamHost. The memory ceiling is unspecified, but it's quite low. So if your site needs to do some heavy operations, you need to make such operations fault tolerant, in order to resume after killed.

Identical and simple way of serving static files in development and production - is it possible?

Is it possible to have static files only in PROJECT_DIR/static directory without duplicates of static files in app dirs and without needing to do collectstatic command? On local computer Django dev server is used, in production - some other web server. From what I have read so far I decided that it's impossible, but maybe it's not true.
Of course it's possible.. the static files app is there to be useful. If you dont like "duplicates" (that's the whole point - you can have files per app, all merged into one area), don't use the staticfiles app.
Just use any folder, anywhere, as your assets folder. In production, serve it at some url, say MY_URL. In development, wire up your URLConf to serve files at your asset folder at MY_URL
https://docs.djangoproject.com/en/1.5/howto/static-files/#serving-files-uploaded-by-a-user
from django.conf import settings
from django.conf.urls.static import static
urlpatterns = patterns('',
# ... the rest of your URLconf goes here ...
) + static('MY_URL', document_root='path-to-my-files')
This is the old school method of doing this, before staticfiles brought its goodness.
Are you sure you can't solve this problem by just using the staticfiles app? It's not much work to add in a python manage.py collectstatic --noinput in your deployment script.

Django admin, static files use wrong directory

I am using django 1.4, and when I have this in settings:
STATIC_URL = '/_s/'
STATICFILES_DIRS = (
('admin', '/usr/local/lib/python2.7/dist-packages/django/contrib/admin/static/admin'),
)
And then access /_s/admin/css/base.css Django tries to load it from a different path and writes:
"/usr/local/lib/python2.7/dist-packages/django/contrib/admin/media/css/base.css" does not exist
Why does this happens?
UPD
I use ./manage.py runcserver 0.0.0.0:8001
The STATICFILES_DIRS is used for collecting static files, ./manage.py collectstatic. It tell Django which folders to look in in order to pull all static files together in to one place.
When serving up static files Django will only look in STATIC_ROOT.
Official docs
The problem was in
./manage.py runcserver 0.0.0.0:8001
I used concurrent server, not default django debug server (I thought they both work in the same way (except threads))
But it is not true =(
UPD
From 1.4 you can use django debug server, it is multithreaded by default.