I am trying to optimize my Django web application by leveraging browser caching. I set a Cache-Control header for max-age equal to a year in my home view function returned response. When I load up my site however, and check the response header of some of the images on my home page, the cache-control header isn't there. I tried two different methods of setting the response header. First I tried using Django's built in cache-control decorator. I also tried simply taking the rendered response, and setting the header in my view function before its return statement. Are static images cached differently ?
View Function
def view_home(request, page=None):
# FIND THE HOME PAGE IF WE DO NOT HAVE ONE
# IF NOT FOUND RETURN 404
if not page:
try:
page = WebPage.objects.get(template='home')
except WebPage.DoesNotExist:
raise Http404
try:
billboards = Billboard.get_published_objects()
except Exception as e:
logging.error(e)
billboards = None
project_types = ProjectType.get_published_objects()
budgets = Budget.get_published_objects()
deadlines = Deadline.get_published_objects()
contact_descriptions = ContactDescription.get_published_objects()
contact_form = ContactForm(type_list=project_types, budget_list=budgets,
deadline_list=deadlines, description_list=contact_descriptions)
context = {'page': page, 'billboards': billboards, 'contact_form': contact_form}
set_detail_context(request, context)
template = 'home.html'
# Add Cache control to response header
expiry_date = datetime.datetime.now() + datetime.timedelta(days=7)
response = render(request, template, context)
response['Cache-Control'] = 'max-age=602000'
response['Expires'] = expiry_date
return response
It sounds like you're setting the headers on a per-view basis. But those views are handling specific URLs, which presumably are not the URLs for your static image files. So it won't have any effect on those.
How you set the headers for your static files depends on how you're serving them.
The most straightforward solution is to use the whitenoise app. This serves static files from Django in the same way in both development and production, and has a setting to control the max-age.
If you're using an external server (e.g. ngnix or Apache) you'll need to configure it to set any custom headers. It doesn't have anything to do with Django.
If you're using the Django development server you'll have to opt out of having it handle the static files automatically, and instead use a custom view that sets the headers. (Or you could just not bother when using the development server.)
Related
I am hosting some video files in rackspace cloud files, and each user is allowed to download the files that are assigned to them.
Because of the file sizes it is not feasible to buffer the object in the webserver(webfaction)
I tried a redirect to the file, with Content-Disposition set to attachment, but to no avail.
What kind of options do I have, if any?
Ideally the file download would pop as coming from my domain after clicking a link that points to something like example.com/video/42/download/ so I can handle authentication ect. but im not sure how to structure my view for that to happen.
You are probably best served by using an HttpResponseRedirect unless there is something I am misunderstanding...?
# urls.py
from django.http import HttpResponseRedirect
url(r'^applications/(?P<id>\d+)/image\.png$', 'core.views.serve_image', name='image'),
This will serve a view at http://localhost/application/12345/image.png.
# core/views.py
def serve_application_image(request, id):
# redirect to temp_url
application = Application.objects.get(id=id)
return HttpResponseRedirect(application.image.temp_url)
And this will redirect users that hit that URL to the Rackspace URL. It can work for embedding videos, images, etc, in html <img> tags and such. Browser clients will be able to see the redirected URL (at rackcdn.com).
I have configured my apps to serve a temp_url property that expires after 15 minutes. The temporary URL is created for the CDN at Rackspace.com and their documentation may be out of the scope for this question so I'll leave it off for now... but the code I use to sub-class ImageField to serve image attributes with the .temp_url code follows:
import hmac
from hashlib import sha1
from time import time
class ImageFieldFile_With_Temp_Url(ImageFieldFile):
#property
def temp_url(self):
container_name, file_name = (self.storage.container.name, self.name)
key = settings.CUMULUS['CUSTOM__X_ACCOUNT_META_TEMP_URL_KEY']
public_url = settings.CUMULUS['CUSTOM__X_STORAGE_URL']
method = 'GET'
expires = int(time() + settings.CUMULUS['CUSTOM__X_TEMP_URL_TIMEOUT'])
url = '%s/%s/%s' % (public_url, container_name, file_name)
base_url, object_path = url.split('/v1/')
object_path = '/v1/' + object_path
hmac_body = '%s\n%s\n%s' % (method, expires, object_path)
sig = hmac.new(key, hmac_body, sha1).hexdigest()
return '%s%s?temp_url_sig=%s&temp_url_expires=%s' % (base_url, object_path, sig, expires)
class ImageField_With_Temp_Url(models.ImageField):
attr_class = ImageFieldFile_With_Temp_Url
models.ImageField = ImageField_With_Temp_Url
Note that I am using the django-cumulus project in this approach.
Importing this function anywhere at the top of your models.py will extend ImageField with a new temp_url property (since I assign it to models.ImageField ...).
I am writing a Django based website, but need to serve a a simple text file. Is the correct way to so this by putting it in the static directory and bypassing Django?
If the file is static (not generated by the django app) then you can put it in the static directory.
If the content of this file is generated by Django then you can return it in a HttpResponse with text/plain as mimetype.
content = 'any string generated by django'
return HttpResponse(content, content_type='text/plain')
You can also give a name to the file by setting the Content-Disposition of the response.
filename = "my-file.txt"
content = 'any string generated by django'
response = HttpResponse(content, content_type='text/plain')
response['Content-Disposition'] = 'attachment; filename={0}'.format(filename)
return response
I agree with #luc, however another alternative is to use X-Accel-Redirect header.
Imagine that you have to serve big protected (have to login to view it) static files. If you put the file in static directory, then it's access is open and anybody can view it. If you serve it in Django by opening the file and then serving it, there is too much IO and Django will use more RAM since it has to load the file into RAM. The solution is to have a view, which will authenticate a user against a database, however instead of returning a file, Django will add X-Accel-Redirect header to it's response. Now since Django is behind nginx, nginx will see this header and it will then serve the protected static file. That's much better because nginx is much better and much faste at serving static files compared to Django. Here are nginx docs on how to do that. You can also do a similar thing in Apache, however I don't remember the header.
I was using a more complex method until recently, then I discovered this and this:
path('file.txt', TemplateView.as_view(template_name='file.txt',
content_type='text/plain')),
Then put file.txt in the root of your templates directory in your Django project.
I'm now using this method for robots.txt, a text file like the original asker, and a pre-generated sitemap.xml (eg, change to content_type='text/xml').
Unless I'm missing something, this is pretty simple and powerful.
I had a similar requirement for getting a text template for a form via AJAX. I choose to implement it with a model based view (Django 1.6.1) like this:
from django.http import HttpResponse
from django.views.generic import View
from django.views.generic.detail import SingleObjectMixin
from .models import MyModel
class TextFieldView(SingleObjectMixin, View):
model = MyModel
def get(self, request, *args, **kwargs):
myinstance = self.get_object()
content = myinstance.render_text_content()
return HttpResponse(content, content_type='text/plain; charset=utf8')
The rendered text is quite small and dynamically generated from other fields in the model.
I have enabled site-wide Django caching, but the third-party apps I am using have not specified any cache-control expectations. So, I am guessing that their views will get cached.
The problem is that I do not want Django to cache the views of some apps. How do I apply url-level cache control on include()?
url(r"^account/", include("pinax.apps.account.urls")) #How to apply cache control here?
You can't. The per-site cache is achieved through middlewares which considers only request and response instead of specific view.
However, you could achieve this by providing a patched django.middleware.cache.FetchFromCacheMiddleware.
class ManagedFetchFromCacheMiddle(FetchFromCacheMiddleware):
def process_request(self, request):
if should_exempt(request):
request._cache_update_cache = False
return
return super(ManagedFetchFromCacheMiddle, self).process_request(request)
def should_exempt(request):
"""Any predicator to exempt cache on a request
For your case, it looks like
if request.path.startswith('/account/'):
return True
"""
Replace 'django.middleware.cache.FetchFromCacheMiddleware' with the path of the above in MIDDLEWARE_CLASSES.
Maybe generic version of above is suitable for commiting patch to Django community =p
I have a number of sites under one Django application that I would like to implement site wide caching on. However it is proving to be a real hassle.
what happens is that settings.CACHE_MIDDLEWARE_KEY_PREFIX is set once on startup, and I cannot go ahead and change it depending on what the current site is. As a result if a page of url http://website1.com/abc/ is cached then http://website2.com/abc/ renders the cached version of http://website1.com/abc/. Both these websites are running on the same Django instance as this is what Django Sites appears to allow us to do.
Is this an incorrect approach? Because I cannot dynamically set CACHE_MIDDLEWARE_KEY_PREFIX during runtime I am unable to cache multiple sites using Django's Site wide caching. I also am unable to do this for template and view caching.
I get the impression that the way this really needs to be setup is that each site needs its own Django instance which is pretty much identical except for the settings file, which in my case will differ only by the value of CACHE_MIDDLEWARE_KEY_PREFIX. These Django instances all read and write to the same database. This concerns me as it could create a number of new issues.
Am I going down the right track or am I mistaken about how multi site architecture needs to work? I have checked the Django docs and there is not real mention of how to handle caching (that isn't low level caching) for Django applications that serve multiple sites.
(Disclaimer: the following is purely speculation and has not been tested. Consume with a pinch of salt.)
It might be possible to use the vary_on_headers view decorator to include the 'Host' header in the cache key. That should result in cache keys that include the HTTP Host header, thus effectively isolating the caches for your sites.
#vary_on_headers('Host')
def my_view(request):
# ....
Of course, that will only work on a per-view basis, and having to add a decorator to all views can be a big hassle.
Digging into the source of #vary_on_headers reveals the use of patch_vary_headers() which one might be able to use in a middleware to apply the same behaviour on a site level. Something along the lines of:
from django.utils.cache import patch_vary_headers
class VaryByHostMiddleware(object):
def process_response(self, request, response):
patch_vary_headers(response, ('Host',))
return response
I faced this problem recently. What I did based on the documentation was to create a custom method to add the site id to the key used to cache the view.
In settings.py add the KEY_FUNCTION argument:
CACHES = {
'default': {
'BACKEND': 'path.to.backend',
'LOCATION': 'path.to.location',
'TIMEOUT': 60,
'KEY_FUNCTION': 'path.to.custom.make_key_per_site',
'OPTIONS': {
'MAX_ENTRIES': 1000
}
}
}
And my custom make_key method:
def make_key_per_site(key, key_prefix, version):
site_id = ''
try:
site = get_current_site() # Whatever you use to get your site's data
site_id = site['id']
except:
pass
return ':'.join([key_prefix, site_id, str(version), key])
You need to change get_full_path to build_absolute_uri in django.util.cache
def _generate_cache_header_key(key_prefix, request):
"""Returns a cache key for the header cache."""
#path = md5_constructor(iri_to_uri(request.get_full_path()))
path = md5_constructor(iri_to_uri(request.build_absolute_uri())) # patch using full path
cache_key = 'views.decorators.cache.cache_header.%s.%s' % (
key_prefix, path.hexdigest())
return _i18n_cache_key_suffix(request, cache_key)
def _generate_cache_key(request, method, headerlist, key_prefix):
"""Returns a cache key from the headers given in the header list."""
ctx = md5_constructor()
for header in headerlist:
value = request.META.get(header, None)
if value is not None:
ctx.update(value)
#path = md5_constructor(iri_to_uri(request.get_full_path()))
path = md5_constructor(iri_to_uri(request.build_absolute_uri()))
cache_key = 'views.decorators.cache.cache_page.%s.%s.%s.%s' % (
key_prefix, request.method, path.hexdigest(), ctx.hexdigest())
return _i18n_cache_key_suffix(request, cache_key)
Or create you own slightly changed cache middleware for multisite.
http://macrotoma.blogspot.com/2012/06/custom-multisite-caching-on-django.html
I want all the production data for my web-app to also flow through my testing environment. Essentially, I want to forward every http request for the production site to the test site (and also have the production website serve it!).
What is a good way to do this? My site is built with Django, and served by mod_wsgi. Is this best implemented at the app-level (Django), web server level (Apache), or the mod_wsgi-level?
I managed to forward request like this
def view(request):
# do what you planned to do here
...
# processing headers
def format_header_name(name):
return "-".join([ x[0].upper()+x[1:] for x in name[5:].lower().split("_") ])
headers = dict([ (format_header_name(k),v) for k,v in request.META.items() if k.startswith("HTTP_") ])
headers["Cookie"] = "; ".join([ k+"="+v for k,v in request.COOKIES.items()])
# this conversion is needed to avoid http://bugs.python.org/issue12398
url = str(request.get_full_path())
# forward the request to SERVER_DOMAIN
conn = httplib.HTTPConnection("SERVER_DOMAIN")
conn.request(
request.method,
url,
request.raw_post_data,
headers
)
response = conn.getresponse()
# some error handling if needed
if response.status != 200:
...
# render web page as usual
return render_to_response(...)
For code reuse, consider decorators