I'm using template cache with django-redis and it works properly but, although I don't use decorators (like cache_page, cache_control etc..), django is caching automatically every loaded page.
I verified this using redis-cli. This is keys * output after page load:
1) ":1:views.decorators.cache.cache_header..11786bb66822aef24b9fe0dac22e6e4e..."
2) ":1:views.decorators.cache.cache_page..GET.11786bb66822aef24b9fe0dac22e6e4e..."
3) ":1:django.contrib.sessions.cached_db8ss2k5s9jmp42cer0fs1nd..."
I tried to use never_cache on every view and it works but I don't see that as a good solution.
There's a better way?
My configuration is simple
CACHES = {
'default':
{
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': 'redis://redis_ip:redis_port',
'TIMEOUT': 60
},
}
MIDDLEWARE.PY
MIDDLEWARE = [
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
...
'django.middleware.cache.FetchFromCacheMiddleware',
...]
Thanks to all
Remove UpdateCacheMiddleware and FetchFromCacheMiddleware from your MIDDLEWARE settings. They are meant for per-site caching, so every page is cached when you have those middleware.
Related
I'm trying to use a Memcached instance of AWS ElastiCache with a Django project. It seems to be caching a view for a user, but if you come in on a different PC, it isn't cached until called from that PC (or same PC with different browser).
I'm not sure what I've got wrong.
Within settings.py I have
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
}
}
MIDDLEWARE = [
'core.middleware.DenyIndexMiddleware',
'core.middleware.XForwardedForMiddleware',
'core.middleware.PrimaryHostRedirectMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
'masquerade.middleware.MasqueradeMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sites.middleware.CurrentSiteMiddleware',
'cms.middleware.user.CurrentUserMiddleware',
'cms.middleware.page.CurrentPageMiddleware',
'cms.middleware.toolbar.ToolbarMiddleware',
'cms.middleware.language.LanguageCookieMiddleware',
'cms.middleware.utils.ApphookReloadMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
]
I've then cached the views using cache_page
path('<str:service_type>/<str:location>/', cache_page(60*60)(views.canonical_search), name="canonical-search"),
How do I cache the site so that the page is cached irrespective of the user?
EDIT
I've noticed that it never caches when using the user is logged in.
Watch out for the Vary header, that cache_page() takes into account.
Usually, some middlewares may add a Vary header, for example :
CsrfViewMiddleware adds Cookie,
GZipMiddlewareadds Accept-Encoding
LanguageCookieMiddleware may add Accept-Language
meaning that as soon as you have a different Cookie (session), encoding, or language, you have a different version of cache for your page.
As for you case, the CsrfViewMiddleware may be the problem, you can add the decorator #csrf_exempt to your view so that the Vary: Cookie header is not set in the response.
More info at https://docs.djangoproject.com/en/3.0/topics/cache/#using-vary-headers
Though Django documentation, you can read this:
Django’s cache framework
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
If you need to store a page in the local
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': 'c:/foo/bar',
}
}
But, my recommendation is store the results (database result, assets, etc), as #PVSK show in this thread:
from django.core.cache import cache
def sample(request):
cached_data = cache.get_many(['query1', 'query2'])
if cached_data:
return render(request, 'sample.html', {'query1': cached_data['query1'], 'query2': cached_data['query2']})
else:
queryset1 = Model.objects.all()
queryset2 = Model2.objects.all()
cache.set_many({'query1': queryset1 , 'query2': queryset2 }, None)
return render(request, 'sample.html', {'query1': queryset1 , 'query2': queryset2})
Hm, at first I wondered if you're running into some cache default limitations. You are not using OPTIONS in your CACHE backend definition, so the cache is limited to 300 entries per default.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
'OPTIONS': {
'MAX_ENTRIES': 1000
}
}
}
The next possible problem, that we also had, is that the cache_key generation takes the full QUERY_STRING into account, so (the ?param=bla). But you already stated that the url is the same for all users.
Next up, as SebCorbin correctly identified, are the potential Vary issues.
UpdateCacheMiddleware will never cache a cookie-setting response to a cookie-less request.
def process_response(self, request, response):
#...
# Don't cache responses that set a user-specific (and maybe security
# sensitive) cookie in response to a cookie-less request.
if not request.COOKIES and response.cookies and has_vary_header(response, 'Cookie'):
return response
# ...
The order of execution of middlewares is top-to-bottom for process_request and bottom-to-top for process_response.
I suspect that one of lower middlewares (or the view) is setting a cookie, so you might be able to work around this issue by moving the 'django.middleware.cache.UpdateCacheMiddleware' below the offending middleware, but you risk loosing features if you don't move feature-middlewares like LocaleMiddleware as well.
If your view code is setting cookies you'll need to switch to the low-level cache API to cache costly operations (or move the cookie logic into a middleware that lives above the UpdateCacheMiddleware middleware).
I'd like to keep the list of 'Last Seen' 20 requests into a Django view. Also I want to avoid creating a separate model for it and just keep the requested urls in memcached, by pushing each new request to a fixed size queue and then retrieve the queue in views.
But since cache is just key:value dictionary I'm wondering how best to achieve this?
One way to do this with memcache would be to store a pickled python object as a single key in the cache.
In this case we could use a Python deque which has exactly the properties we'd want for a list of 20 most recent items
Each time we record a new page view we need to update the deque, which means to get it, unpickle it, append, pickle it and set the new value back to memcache.
Fortunately Django's cache framework will handle the pickling and unpickling for us. However one thing we need to take care of is the possibility of a race condition - in other words if another process also updates the deque after we get our copy and before we have a chance to set it back to the cache.
For this reason we should use memcache's CAS ('compare-and-set') operation. There is an extended Django cache backend that enables CAS available here:
https://github.com/anentropic/django-cas-cache
pip install django-cas-cache
We'd have some code in a custom Django middleware to update the cache on each page view, looking roughly like this:
middleware.py
from collections import deque
from django.core.cache import cache
class LastSeenMiddleware(object):
def process_response(request, response):
# you might want some logic like this to only
# record successful requests
if response.status != 200:
return response
# in case we don't already have a deque, try to add
# (add will not overwrite if key already exists)
added = cache.add('last_seen', deque([request.path], maxlen=20))
if not added:
def update(val):
val.append(request.path)
return val
cache.cas('last_seen', update)
return response
views.py
from django.core.cache import cache
from django.shortcuts import render_to_response
def myview(request):
last_seen = cache.get('last_seen')
# whatever
return render_to_response('mytemplate.html', {'last_seen': last_seen})
settings.py
CACHES = {
'default': {
'BACKEND': 'cascache.backends.memcached.MemcachedCASCache',
'LOCATION': '127.0.0.1:11211',
}
}
# as a response processor, our middleware should probably come
# first in this list, in order to run *last*
MIDDLEWARE_CLASSES = (
'myapp.middleware.LastSeenMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
)
We are trying to implement caching in Django. We have tried memcache, file system caching and local memory caching. No mater what it just isn't working -- a timestamp we put on the template to test caching is always updating so we know it's not working.
Here are relevant portions of code.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': 'c:/tmp',
}
}
CACHE_MIDDLEWARE_ALIAS = "default"
CACHE_MIDDLEWARE_SECONDS = 600
MIDDLEWARE_CLASSES = [
"django.middleware.cache.UpdateCacheMiddleware",
... OTHER ONES ...
"django.middleware.cache.FetchFromCacheMiddleware",
]
VIEWS.PY
from django.views.decorators.cache import cache_page
#cache_page(60 * 10)
def profiles(request, template_name="profiles/profiles.html", extra_context=None):
...
I've built a RESTful API on top of the Django Rest Framework. The URL conf for the API is composed of class based views.
I would like to cache these views, however, the following is failing. Any thoughts on why that might be and how I could change it?
from django.views.decorators.cache import cache_page
urlpatterns = patterns('',
url(r'^dounces/?$', cache_page(60*60)(DounceListView.as_view(resource=DounceResource)), name='dounces_api'),
I have the following middleware installed.
'django.middleware.gzip.GZipMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.transaction.TransactionMiddleware',
'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
AND for local testing, i'm using the default caching backend:
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
change your urlconf to
urlpatterns = patterns('',
url(r'^dounces/?$', cache_page(60*60)(DounceListView.as_view(resource=DounceResource)), name='dounces_api'),
)
also see https://docs.djangoproject.com/en/1.7/topics/cache/#specifying-per-view-cache-in-the-urlconf
The other answer is out of date. The correct way is detailed here:
https://docs.djangoproject.com/en/1.6/topics/cache/#specifying-per-view-cache-in-the-urlconf
It has to now be done this way:
urlpatterns = patterns('',
url(r'^dounces/?$', cache_page(60*60)(DounceListView.as_view(resource=DounceResource)), name='dounces_api'),
)
I dont see any "django.middleware.cache.UpdateCacheMiddleware" and "django.middleware.cache.FetchFromCacheMiddleware". I dont think it can cache properly without it.
https://docs.djangoproject.com/en/2.2/topics/cache/
So I want to cache some data in mysql and some in memcached.
at the moment I have this In my config file, but i don't know how to write router for cache back end.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
I use multi databases structure and I know how to write multi database routers.
in settings.py
DATABASE_ROUTERS = ['oceankeys.dbrouter.SphinxRouter','oceankeys.dbrouter.DefaultDbRouter']
Any one know how to make Django caching BACKEND router?
Thanks
I don't believe the Django cache framework can mimic db routing in general.
For the site cache using the cache framework middleware you have to specify the name of the cache in settings.py, e.g.:
CACHE_MIDDLEWARE_ALIAS = "my_cache_alias"
For a page cache you can manually specify the name of the cache in the decorator, e.g.:
#cache_page(60 * 15, cache="my_cache_alias")
def my_view(request):
...
I'm not sure cache routing really makes sense for site and page caching so I don't have a problem with the way this is designed.
Now, for your case where you are using MySQL as a database cache backend you can set it up and make a router as per the Django docs section on database caching. For example, this would be your CACHES setting:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
'my_cache_alias': {
'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'my_cache_table',
}
}
Then create a router that identifies which cache backend to use for which models. It looks and works exactly like DB router (as you should be able to see from the doc section on database caching and multiple databases) with the exception that it returns a cache alias instead of db alias.
e.g
settings.py
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
},
'myalias':{
'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'my_cache_table',
}
}
views.py
from django.core.cache import caches
cache = caches['myalias']
cache.set('my_key', 'hello, world!', 30)
print cache.get('my_key')
You can see the detail in Django’s cache framework (section:Accessing the cache)