I'm exploring using whitenoise to serve static files in a Django application that's packaged in a Docker container. In the documentation it says:
As part of deploying your application you’ll need to run ./manage.py collectstatic to put all your static files into STATIC_ROOT. (If you’re running on Heroku then this is done automatically for you.)
Is that really needed? I'm not running ./manage.py collectstatic and static files are still served.
If it's not needed, is it an optimization? I'm trying to avoid having needless steps in my deployment process.
As that quote states, Whitenoise serves files from STATIC_ROOT. collectstatic collects files from STATICFILES_DIRS and any app-specific static directories and puts them into STATIC_ROOT.
But there are two situations where it will serve files without running collectstatic. The first is if your files are already in STATIC_ROOT. They shouldn't be, but it's a common mistake to set the value of STATIC_ROOT to the directory containing the source files. However, in this case it won't for example find the admin files, which would need to be collected.
The other situation is if you are running with DEBUG=True. Whitenoise inherits this value for its USE_FINDERS setting, which makes it look in the same places as collectstatic itself to serve files. Clearly, you wouldn't want to run in production with DEBUG on, but you can set this setting explicitly. But as that docs link points out, doing so will disable the caching and compression features of Whitenoise.
Related
I put static files under a app directory which is related to the static files.
for example,
image.jpg is used for templates under exmapleapp. so I locate image.jpg file in a directory /project/exampleapp/static/image.jpg
In Debug=True settings, Dajngo finds static files automatically by django.contrib.staticfiles. If you put static files projoject directory /project/static/, you can set STATICFILES_DIRS = []in settings.py or add like urlspatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) in urls.py(if you want to serve static files manually)
so far, is nothing problem in development right? but you want to deploy, you need to do collectstatic. I am confused from now.
This is what I thought initially.
step1 : you work and test your code in local with DEBUG = True in setting. you continously save your code in git repo.
step2 : you are ready to deploy your code in server. you clone your repo and set apache server for running django framework properly(Alias static, WSGI Daemon process). you do python manage.py collectstatic to serve static file with apache server.
step3 : you keep work in local to improve your code and apply this improvement into your code in server with test.
I got confused and got questions.
Q1 : if you do collectstatic for deployment, there will be static folder(according to STATIC_ROOT settings) with all static files which is spread around in each app folders.
do you do collectstatic in local and send only the static folder which collects all to server? or do you do collectstatic in server according to Static Alias setting in Apache server?
Q2 : Do you change DEBUG = False and ALLOWED_HOSTS =[server IP] in local and save code in git repo and pull it in server??
well, now I am confused with working during development with github and deploying it to server with github with Django settings.
Tell me if you can't understand my question clearly please.
You are using static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) for serving static files in development mode. But in production, (means DEBUG = False) it will be empty list. (static returns empty list). Then collectstatic, in you can use development or production.Its intention is to copy all static files to STATIC_ROOT.So
Q1: Everyone do collectstatic on production to serve static files through webserver(Apache, Nginx)
Q2: For production and development write different settings file. You can see here for ex How to manage local vs production settings in Django?
What is the necessity to do collectstatic in django? Why can't I just copy files to the static folder and make my server refer to that folder? Why does that not work?
Convenience? You can manually copy over the static files and there's no problem with doing that, but when you have multiple different folders where static files are stored and you're deploying to a production server, it's much more of a hassle to go individually to each folder and copy them instead of having collectstatic run as an automated task and do the work for you.
From the Django Docs:
Deployment django.contrib.staticfiles provides a convenience
management command for gathering static files in a single directory so
you can serve them easily.
Set the STATIC_ROOT setting to the directory from which you’d like to
serve these files, for example:
STATIC_ROOT = "/var/www/example.com/static/"
Run the collectstatic management command:
$ python manage.py collectstatic
This will copy all files from your
static folders into the STATIC_ROOT directory.
Use a web server of your choice to serve the files. Deploying static
files covers some common deployment strategies for static files.
What's the purpose of copying the files, why not just serve them from the directory they live in within the app?
Why not just serve your static directory? You might use more than one app, and some of your apps may not be under your control. Before the staticfiles app existed, you then had to either manually copy the static files for all apps to a common directory, upload them to your CDN, or symlink them to the document root of your web server.
The staticfiles app established a convention: put static files for each app under a static directory and let Django do the work for you.
The STATIC_ROOT can be on a different machine than the application, so copying your static files to the static root means that you can serve your static files from a different server (CDN FTW!) which you wouldn't be able to do if those files where only located within their respective app directories.
I'm using django-compressor to concatenate and compress my CSS and JS files on this site. I'm serving static files from an S3 bucket.
On my local copy of the site, using a different S3 bucket, this all works perfectly. But on the live site, hosted on Heroku, it all works except the relative URLs for images in the CSS files do not get re-written.
eg, this line in the CSS file:
background-image: url("../img/glyphicons-halflings-grey.png");
gets rewritten to:
background-image:url('https://my-dev-bucket-name.s3.amazonaws.com/static/img/glyphicons-halflings-grey.png')
on my development site, but isn't touched on the live site. So the live site ends up looking in pepysdiary.s3.amazonaws.com/static/CACHE/img/ for the images (as it's relative to the new, compressed CSS file).
For now, I've put a directory at that location containing the images, but I can't work out why there's this difference. Both sites have this in their settings:
COMPRESS_CSS_FILTERS = [
# Creates absolute urls from relative ones.
'compressor.filters.css_default.CssAbsoluteFilter',
# CSS minimizer.
'compressor.filters.cssmin.CSSMinFilter'
]
And the CSS files are being minimised just fine... but it's like the other filter isn't being applied on the live site.
I recently ran into this issue on heroku, and running the latest version of django-compressor (1.3) does not solve the problem. I will provide the solution that I am using, as well as an explanation of the problems I ran into along the way.
The solution
I created my own 'CssAbsoluteFilter' that removes the settings.DEBUG check from the 'find' method like this:
# compress_filters.py
from compressor.filters.css_default import CssAbsoluteFilter
from compressor.utils import staticfiles
class CustomCssAbsoluteFilter(CssAbsoluteFilter):
def find(self, basename):
# The line below is the original line. I removed settings.DEBUG.
# if settings.DEBUG and basename and staticfiles.finders:
if basename and staticfiles.finders:
return staticfiles.finders.find(basename)
# settings.py
COMPRESS_CSS_FILTERS = [
# 'compressor.filters.css_default.CssAbsoluteFilter',
'app.compress_filters.CustomCssAbsoluteFilter',
'compressor.filters.cssmin.CSSMinFilter',
]
The absolute urls now always work for me whether DEBUG = True or False.
The Problem
The issue is connected to 'compressor.filters.css_default.CssAbsoluteFilter', your DEBUG setting, and the fact that heroku has a read-only file system and overwrites your app files every time you deploy.
The reason compress works correctly on your development server is because CssAbsoluteFilter will always find your static files when DEBUG = True, even if you never run 'collectstatic'. It looks for them in STATICFILES_DIRS.
When DEBUG = False on your production server, CssAbsoluteFilter assumes that static files have already been collected into your COMPRESS_ROOT and won't apply the absolute filter if it can't find the files.
Jerdez, django-compressor's author, explains it like this:
the CssAbsoluteFilter works with DEBUG = False if you've successfully provided the files to work with. During development compressors uses the staticfiles finder as a convenience so you don't have to run collectstatic all the time.
Now for heroku. Even though you are storing your static files on S3, you need to also store them on heroku (using CachedS3BotoStorage). Since heroku is a read-only file system, the only way to do this is to let heroku collect your static files automatically during deployment (see https://devcenter.heroku.com/articles/django-assets).
In my experience, running 'heroku run python manage.py collectstatic --noinput' manually or even in your Procfile will upload the files to S3, but it will NOT save the files to your STATIC_ROOT directory (which compressor uses by default as the COMPRESS_ROOT). You can confirm that your static files have been collected on heroku using 'heroku run ls path/to/collected'.
If your files have been collected on heroku successfully, you should be able to compress your files successfully as well, without the solution I provided above.
However, it seems heroku will only collect static files if you have made changes to your static files since your last deploy. If no changes have been made to your static files, you will see something like "0 of 250 static files copied". This is a problem because heroku completely replaces your app contents when you deploy, so you lose whatever static files were previously collected in COMPRESS_ROOT/STATIC_ROOT. If you try to compress your files after the collected files no longer exist on heroku and DEBUG = False, the CssAbsoluteFilter will not replace the relative urls with absolute urls.
My solution above avoids the heroku problem altogether, and replaces relative css urls with absolute urls even when DEBUG = False.
Hopefully other people will find this information helpful as well.
I've had this exact same problem for a month now, but this is fixed in the 1.3 release (3/18/13, so you were probably on 1.2), so just upgrade:
pip install -U django-compressor
The exact problem I gave up on working out, but it's related to Heroku and CssAbsoluteFilter being called but failing at _converter method. Looking at the 1.3 changelog, the only related commit is this: https://github.com/jezdez/django_compressor/commit/8254f8d707f517ab154ad0d6d77dfc1ac292bf41
I gave up there, life's too short.
Meanwhile this has been fixed in django-compressor 1.6. From the changelog:
Apply CssAbsoluteFilter to precompiled css even when compression is disabled
i.e. the absolute filter is run even with DEBUG = True.
I have a Django app on Heroku, with staging and production environments. Static files are hosted on S3. I'm streamlining my deployment process and plan to set up fabfiles once I have things working manually.
How can I configure collectstatic to push to multiple places? If I run it locally, it uses my dev settings (with a local STATIC_ROOT). If I run it on one of my Heroku apps (heroku run ./manage.py collectstatic), then it can't grab the files (since .slugignore ensures they're never pushed to Heroku). The same applies if I include collectstatic in my Procfile.
I'm also using django-pipeline, though it's not yet doing much since I'm stuck on the collectstatic bit.
UPDATE
In response to Marat's question, I tried passing a settings file as an option to collectstatic: ./manage.py collectstatic --settings=project.settings.prod, but got an error: Unknown command: 'collectstatic' I checked on the server though and Installed Apps does include django.contrib.staticfiles and I can also run collectstatic remotely, so I'm not sure what would cause that.
You can set the environment variable DJANGO_SETTINGS_MODULE so you don't need specify --settings everywhere:
heroku config:set DJANGO_SETTINGS_MODULE=project.settings.prod
First, if you are going to serve static via CloudFront, you can use custom origin and always use local STATIC_ROOT. Actually it has some advantages over S3 source, eg gzip support.
Another good thing you can do is to have environment dependent settings in a separate file and then import it in settings.py, eg:
local_settings.py (not in project repository, yet you can have local_settings.py.example):
#environment dependent settings
DATABASES = { .. }
CACHES = { .. }
STATIC_ROOT = 'your_path/static'
settings.py:
import local_settings
I've just replied a similar question on Upload Media from Heroku to Amazon S3. If you customise your settings to take in account environmental vars, you can use filesystem storage backends locally and S3 storage backends when pushing to Heroku. This will collect and upload your static files when your slug is compiled.