I'm using django-compressor to concatenate and compress my CSS and JS files on this site. I'm serving static files from an S3 bucket.
On my local copy of the site, using a different S3 bucket, this all works perfectly. But on the live site, hosted on Heroku, it all works except the relative URLs for images in the CSS files do not get re-written.
eg, this line in the CSS file:
background-image: url("../img/glyphicons-halflings-grey.png");
gets rewritten to:
background-image:url('https://my-dev-bucket-name.s3.amazonaws.com/static/img/glyphicons-halflings-grey.png')
on my development site, but isn't touched on the live site. So the live site ends up looking in pepysdiary.s3.amazonaws.com/static/CACHE/img/ for the images (as it's relative to the new, compressed CSS file).
For now, I've put a directory at that location containing the images, but I can't work out why there's this difference. Both sites have this in their settings:
COMPRESS_CSS_FILTERS = [
# Creates absolute urls from relative ones.
'compressor.filters.css_default.CssAbsoluteFilter',
# CSS minimizer.
'compressor.filters.cssmin.CSSMinFilter'
]
And the CSS files are being minimised just fine... but it's like the other filter isn't being applied on the live site.
I recently ran into this issue on heroku, and running the latest version of django-compressor (1.3) does not solve the problem. I will provide the solution that I am using, as well as an explanation of the problems I ran into along the way.
The solution
I created my own 'CssAbsoluteFilter' that removes the settings.DEBUG check from the 'find' method like this:
# compress_filters.py
from compressor.filters.css_default import CssAbsoluteFilter
from compressor.utils import staticfiles
class CustomCssAbsoluteFilter(CssAbsoluteFilter):
def find(self, basename):
# The line below is the original line. I removed settings.DEBUG.
# if settings.DEBUG and basename and staticfiles.finders:
if basename and staticfiles.finders:
return staticfiles.finders.find(basename)
# settings.py
COMPRESS_CSS_FILTERS = [
# 'compressor.filters.css_default.CssAbsoluteFilter',
'app.compress_filters.CustomCssAbsoluteFilter',
'compressor.filters.cssmin.CSSMinFilter',
]
The absolute urls now always work for me whether DEBUG = True or False.
The Problem
The issue is connected to 'compressor.filters.css_default.CssAbsoluteFilter', your DEBUG setting, and the fact that heroku has a read-only file system and overwrites your app files every time you deploy.
The reason compress works correctly on your development server is because CssAbsoluteFilter will always find your static files when DEBUG = True, even if you never run 'collectstatic'. It looks for them in STATICFILES_DIRS.
When DEBUG = False on your production server, CssAbsoluteFilter assumes that static files have already been collected into your COMPRESS_ROOT and won't apply the absolute filter if it can't find the files.
Jerdez, django-compressor's author, explains it like this:
the CssAbsoluteFilter works with DEBUG = False if you've successfully provided the files to work with. During development compressors uses the staticfiles finder as a convenience so you don't have to run collectstatic all the time.
Now for heroku. Even though you are storing your static files on S3, you need to also store them on heroku (using CachedS3BotoStorage). Since heroku is a read-only file system, the only way to do this is to let heroku collect your static files automatically during deployment (see https://devcenter.heroku.com/articles/django-assets).
In my experience, running 'heroku run python manage.py collectstatic --noinput' manually or even in your Procfile will upload the files to S3, but it will NOT save the files to your STATIC_ROOT directory (which compressor uses by default as the COMPRESS_ROOT). You can confirm that your static files have been collected on heroku using 'heroku run ls path/to/collected'.
If your files have been collected on heroku successfully, you should be able to compress your files successfully as well, without the solution I provided above.
However, it seems heroku will only collect static files if you have made changes to your static files since your last deploy. If no changes have been made to your static files, you will see something like "0 of 250 static files copied". This is a problem because heroku completely replaces your app contents when you deploy, so you lose whatever static files were previously collected in COMPRESS_ROOT/STATIC_ROOT. If you try to compress your files after the collected files no longer exist on heroku and DEBUG = False, the CssAbsoluteFilter will not replace the relative urls with absolute urls.
My solution above avoids the heroku problem altogether, and replaces relative css urls with absolute urls even when DEBUG = False.
Hopefully other people will find this information helpful as well.
I've had this exact same problem for a month now, but this is fixed in the 1.3 release (3/18/13, so you were probably on 1.2), so just upgrade:
pip install -U django-compressor
The exact problem I gave up on working out, but it's related to Heroku and CssAbsoluteFilter being called but failing at _converter method. Looking at the 1.3 changelog, the only related commit is this: https://github.com/jezdez/django_compressor/commit/8254f8d707f517ab154ad0d6d77dfc1ac292bf41
I gave up there, life's too short.
Meanwhile this has been fixed in django-compressor 1.6. From the changelog:
Apply CssAbsoluteFilter to precompiled css even when compression is disabled
i.e. the absolute filter is run even with DEBUG = True.
Related
I am running a website in Divio Cloud. My Test server serves the static files fine but my Live server gives 404. When I switch DEBUG to True on the Live server it starts serving the static files fine.
When the Live server starts, it has this in the log:
[uwsgi-static] added mapping for /static/ => /app/static_collected
Shouldn't that take care of serving the static files correctly?
--edit--
More info. I actually didn't have any "normal" static files. I added a static file using {% static %} and it is working! The files that are not working are collected with django-sass-processor and django-compressor. I have in settings.py:
STATICFILES_FINDERS.extend([
'compressor.finders.CompressorFinder',
])
Shouldn't this only affect when the files are collected?
I think the issue is most likely in the fact that you are using Django Compressor. There are various different ways in which to use Django Compressor.
In its documentation, please see Offline Compression. This is what you need for it to work on Divio Cloud.
For future reference:
In Divio environment to get the compressor based processors (in this case it was django-sass-processor) to work is add something along following lines in the end of the Dockerfile:
RUN DJANGO_MODE=build python manage.py compilescss
# <STATIC>
RUN DJANGO_MODE=build python manage.py collectstatic --noinput
# </STATIC>
# Remove the css files in development environments (in Live they
# are already collected)
RUN DJANGO_MODE=build python manage.py compilescss --delete-files
I'm exploring using whitenoise to serve static files in a Django application that's packaged in a Docker container. In the documentation it says:
As part of deploying your application you’ll need to run ./manage.py collectstatic to put all your static files into STATIC_ROOT. (If you’re running on Heroku then this is done automatically for you.)
Is that really needed? I'm not running ./manage.py collectstatic and static files are still served.
If it's not needed, is it an optimization? I'm trying to avoid having needless steps in my deployment process.
As that quote states, Whitenoise serves files from STATIC_ROOT. collectstatic collects files from STATICFILES_DIRS and any app-specific static directories and puts them into STATIC_ROOT.
But there are two situations where it will serve files without running collectstatic. The first is if your files are already in STATIC_ROOT. They shouldn't be, but it's a common mistake to set the value of STATIC_ROOT to the directory containing the source files. However, in this case it won't for example find the admin files, which would need to be collected.
The other situation is if you are running with DEBUG=True. Whitenoise inherits this value for its USE_FINDERS setting, which makes it look in the same places as collectstatic itself to serve files. Clearly, you wouldn't want to run in production with DEBUG on, but you can set this setting explicitly. But as that docs link points out, doing so will disable the caching and compression features of Whitenoise.
How do I resolve? I don't want to push a copy of Django Admin's CSS to my static files unless I have to. Is that my only option or ??
Once you move your application to production, in addition to setting DEBUG = False in your settings.py, you also have to run collectstatic and then upload these files to your webserver.
collectstatic will put all the static files that are required for all installed applications, including the django.contrib apps (like the django admin), into the folder you specified as STATIC_ROOT. You should then copy the entire contents of this folder to wherever your STATIC_URL is pointing to.
If you don't do the above, your stylesheets and other assets will not appear correctly.
As you are using S3, the excellent django-extensions package provides a sync_s3 command that will handle synchronizing your bucket for you.
Starting from django 1.6, there is an official list of things you should do before you are ready to deploy; so if you are on the current version of django make sure you visit that page. It is also pointed to in the comments in settings.py.
in Django 1.3 you could set STATIC_URL to the S3 URL and ADMIN_PREFIX to your local server, creating a webserver alias that serves the files
(in Apache for example: Alias /static/admin/ /absolute/path/to/static/admin/)
but AFAIK this is deprecated in Django 1.4 and not possible with Django >= 1.5, since they always will point to {{STATIC_URL}}admin/.
but i don't understand, why wouldn't you upload your admin static!? i strongly suggest to colletstatic all your media to S3 :)
(i don't know if i understood you question correctly, i'll update the answer if not)
I wanted to start playing around with Django again (I'm not an expert in Python/Django, but I can make nice things work tho). I used Mezzanine once just to see how it worked. The 'mezzanine-project myproject' command worked like a charm as I had a nice small app running quickly. So, today I downloaded the new Mezzanine 1.3 along with Django 1.4.3 and all its dependencies (pillow, pytz, html5lib, etc) and tried to create another project so I could now work on it in a more consistent manner for personal purposes.
For my surprise, when I ran the server, I got lots of 404 errors pointing to missing /static/ files. Also, after creating the database (with manage.py createdb command), the only thing created was the static folder containing only the pictures of the predefined gallery that come along with Mezzanine. Also, there is no Log in or signup buttons as well.
I've tried making a clean install of all Python and its site-packages with the same result. I also tried copying/pasting the folders containing missing files from the /site-packages/mezzanine folder into my project, but the result was just reducing the number of 404 messages.
I've been doing an extensive research on this issue (with no luck but maybe because of the release being recent?) and even trying to contact someone on the Mezzanine IRC channel with no success.
I hope I'm not missing something silly. Do I have to change anything (note that I'm ok with the old mezzanine default settings) in my settings.py or in a specific file before running manage.py createdb command?
For the record: before running createdb, The only thing I edited was settings.py and changed the database parameters to make it work with my MySQL Server and commenting the local_settings configuration as I do not need it.
Some parameters that could be of help:
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(PROJECT_ROOT, STATIC_URL.strip("/"))
By default, DEBUG is set to False in settings.py, and DEBUG is set to True in local_setting.py
Given that, if you just commented out importing local_settings.py, DEBUG would be False.
Django's development server won't serve static files when DEBUG is set to False, see the staticfiles section of the Django docs for more details.
I have a very simple question which I was not able to find an answer. Using Django staticfiles app, I call the statics collect routine by typing this command line : python manage.py collectstatic.
My problem is that I've several applications serving statics + a media directory containing user-uploaded files. Django copies all the files to the static directory, including media files!
I just would like Django to not copy user-uploaded files to the static directory when I call python manage.py collectstatic. Does anyone have an idea? Tried --ignore option but without success...
Thank you in advance.
That was a problem of django.contrib.staticfiles.finders.DefaultStorageFinder, try removing that from STATICFILES_FINDERS.
So what exactly doesn't work if you use --ignore?
Though the real solution is not to mix user generated content and static files.
If you keep both types separated you can easily put your static files under version control and don't have to deal with excludes on this front as well. And in fact the separation from media (user generated) and static files is the rationale behind staticfiles:
In previous versions of Django, it was common to place static assets
in MEDIA_ROOT along with user-uploaded files, and serve them both at
MEDIA_URL. Part of the purpose of introducing the staticfiles app is
to make it easier to keep static files separate from user-uploaded
files.
The docs on this topic are really comprehensive: https://docs.djangoproject.com/en/dev/howto/static-files/