Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 days ago.
This post was edited and submitted for review 7 days ago.
Improve this question
I am currently looking for a way to manage the Django cache from the administration (having the possibility to turn it off, turn it on, or leave it on with a timeout of 8 hours).
I have tried to use for this the django cache framework, with 3 buttons in the administration to different functions that turn off, activate or modify the cache timeout with no results (the cache remains activated with no possibility to turn it off or modify its timeout).
I tried to use a script that restarts the server once one of these options is chosen, but it doesn't work as it should (this must happen because I can't dynamically modify the settings.py file).
First I create two buttons in the admin panel
<a href="/caching/turn_on_cache/">
<button class="btn btn-success">Turn on cache for 8 hours</button>
</a>
<a href="/caching/turn_off_cache/">
<button class="btn btn-danger">Turn off cache</button>
</a>
Then, I add the two views for set up cache
from django.shortcuts import redirect
from django.core.cache import cache
from django.contrib import messages
import logging
logger = logging.getLogger(__name__)
def turn_on_cache(request):
cache.set('cache', True, 60*60*8)
logging.info(cache.get('cache')) ->>> This return True
messages.success(request, "Cache turned on for 8 hours")
return redirect('admin:index')
def turn_off_cache(request):
cache.set('cache', False, 60*60*8)
logging.info(cache.get('cache')) ->>> This return false
messages.success(request, "Cache turned off")
return redirect('admin:index')
But when I see the django-debug-toolbar the system keeps making sql queries
my settings.py
DEBUG = True
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
#'waffle.middleware.WaffleMiddleware',
"debug_toolbar.middleware.DebugToolbarMiddleware",
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'my_cache_table',
'TIMEOUT': 604800 # 7 days
}
}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 60
CACHE_MIDDLEWARE_KEY_PREFIX = ''
Any different approach to this problem?
Thanks in advance
Related
I'm trying to use a Memcached instance of AWS ElastiCache with a Django project. It seems to be caching a view for a user, but if you come in on a different PC, it isn't cached until called from that PC (or same PC with different browser).
I'm not sure what I've got wrong.
Within settings.py I have
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
}
}
MIDDLEWARE = [
'core.middleware.DenyIndexMiddleware',
'core.middleware.XForwardedForMiddleware',
'core.middleware.PrimaryHostRedirectMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
'masquerade.middleware.MasqueradeMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sites.middleware.CurrentSiteMiddleware',
'cms.middleware.user.CurrentUserMiddleware',
'cms.middleware.page.CurrentPageMiddleware',
'cms.middleware.toolbar.ToolbarMiddleware',
'cms.middleware.language.LanguageCookieMiddleware',
'cms.middleware.utils.ApphookReloadMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
]
I've then cached the views using cache_page
path('<str:service_type>/<str:location>/', cache_page(60*60)(views.canonical_search), name="canonical-search"),
How do I cache the site so that the page is cached irrespective of the user?
EDIT
I've noticed that it never caches when using the user is logged in.
Watch out for the Vary header, that cache_page() takes into account.
Usually, some middlewares may add a Vary header, for example :
CsrfViewMiddleware adds Cookie,
GZipMiddlewareadds Accept-Encoding
LanguageCookieMiddleware may add Accept-Language
meaning that as soon as you have a different Cookie (session), encoding, or language, you have a different version of cache for your page.
As for you case, the CsrfViewMiddleware may be the problem, you can add the decorator #csrf_exempt to your view so that the Vary: Cookie header is not set in the response.
More info at https://docs.djangoproject.com/en/3.0/topics/cache/#using-vary-headers
Though Django documentation, you can read this:
Django’s cache framework
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
If you need to store a page in the local
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': 'c:/foo/bar',
}
}
But, my recommendation is store the results (database result, assets, etc), as #PVSK show in this thread:
from django.core.cache import cache
def sample(request):
cached_data = cache.get_many(['query1', 'query2'])
if cached_data:
return render(request, 'sample.html', {'query1': cached_data['query1'], 'query2': cached_data['query2']})
else:
queryset1 = Model.objects.all()
queryset2 = Model2.objects.all()
cache.set_many({'query1': queryset1 , 'query2': queryset2 }, None)
return render(request, 'sample.html', {'query1': queryset1 , 'query2': queryset2})
Hm, at first I wondered if you're running into some cache default limitations. You are not using OPTIONS in your CACHE backend definition, so the cache is limited to 300 entries per default.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
'OPTIONS': {
'MAX_ENTRIES': 1000
}
}
}
The next possible problem, that we also had, is that the cache_key generation takes the full QUERY_STRING into account, so (the ?param=bla). But you already stated that the url is the same for all users.
Next up, as SebCorbin correctly identified, are the potential Vary issues.
UpdateCacheMiddleware will never cache a cookie-setting response to a cookie-less request.
def process_response(self, request, response):
#...
# Don't cache responses that set a user-specific (and maybe security
# sensitive) cookie in response to a cookie-less request.
if not request.COOKIES and response.cookies and has_vary_header(response, 'Cookie'):
return response
# ...
The order of execution of middlewares is top-to-bottom for process_request and bottom-to-top for process_response.
I suspect that one of lower middlewares (or the view) is setting a cookie, so you might be able to work around this issue by moving the 'django.middleware.cache.UpdateCacheMiddleware' below the offending middleware, but you risk loosing features if you don't move feature-middlewares like LocaleMiddleware as well.
If your view code is setting cookies you'll need to switch to the low-level cache API to cache costly operations (or move the cookie logic into a middleware that lives above the UpdateCacheMiddleware middleware).
I'm using template cache with django-redis and it works properly but, although I don't use decorators (like cache_page, cache_control etc..), django is caching automatically every loaded page.
I verified this using redis-cli. This is keys * output after page load:
1) ":1:views.decorators.cache.cache_header..11786bb66822aef24b9fe0dac22e6e4e..."
2) ":1:views.decorators.cache.cache_page..GET.11786bb66822aef24b9fe0dac22e6e4e..."
3) ":1:django.contrib.sessions.cached_db8ss2k5s9jmp42cer0fs1nd..."
I tried to use never_cache on every view and it works but I don't see that as a good solution.
There's a better way?
My configuration is simple
CACHES = {
'default':
{
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': 'redis://redis_ip:redis_port',
'TIMEOUT': 60
},
}
MIDDLEWARE.PY
MIDDLEWARE = [
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
...
'django.middleware.cache.FetchFromCacheMiddleware',
...]
Thanks to all
Remove UpdateCacheMiddleware and FetchFromCacheMiddleware from your MIDDLEWARE settings. They are meant for per-site caching, so every page is cached when you have those middleware.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
I use django_comments as comment system of my site, but when I post the comment, it gives 403 error. Why?
This is my settings:
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
#'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
)
The others forms works well. They won't give 403 error.
Your other forms work because you have disabled the CSRF middleware. This is a bad idea, because it makes your website vulnerable to a CSRF attack.
The post_comment view from django_comments explicitly uses the csrf_protect decorator. Therefore, you must include {% csrf_token %} in the form tag in your template to prevent CSRF errors.
If you still have problems, then it is probably the view. As the docs say, you must make sure that the template is rendered with the request object, otherwise the {% csrf_token %} tag will not work.
I'd like to keep the list of 'Last Seen' 20 requests into a Django view. Also I want to avoid creating a separate model for it and just keep the requested urls in memcached, by pushing each new request to a fixed size queue and then retrieve the queue in views.
But since cache is just key:value dictionary I'm wondering how best to achieve this?
One way to do this with memcache would be to store a pickled python object as a single key in the cache.
In this case we could use a Python deque which has exactly the properties we'd want for a list of 20 most recent items
Each time we record a new page view we need to update the deque, which means to get it, unpickle it, append, pickle it and set the new value back to memcache.
Fortunately Django's cache framework will handle the pickling and unpickling for us. However one thing we need to take care of is the possibility of a race condition - in other words if another process also updates the deque after we get our copy and before we have a chance to set it back to the cache.
For this reason we should use memcache's CAS ('compare-and-set') operation. There is an extended Django cache backend that enables CAS available here:
https://github.com/anentropic/django-cas-cache
pip install django-cas-cache
We'd have some code in a custom Django middleware to update the cache on each page view, looking roughly like this:
middleware.py
from collections import deque
from django.core.cache import cache
class LastSeenMiddleware(object):
def process_response(request, response):
# you might want some logic like this to only
# record successful requests
if response.status != 200:
return response
# in case we don't already have a deque, try to add
# (add will not overwrite if key already exists)
added = cache.add('last_seen', deque([request.path], maxlen=20))
if not added:
def update(val):
val.append(request.path)
return val
cache.cas('last_seen', update)
return response
views.py
from django.core.cache import cache
from django.shortcuts import render_to_response
def myview(request):
last_seen = cache.get('last_seen')
# whatever
return render_to_response('mytemplate.html', {'last_seen': last_seen})
settings.py
CACHES = {
'default': {
'BACKEND': 'cascache.backends.memcached.MemcachedCASCache',
'LOCATION': '127.0.0.1:11211',
}
}
# as a response processor, our middleware should probably come
# first in this list, in order to run *last*
MIDDLEWARE_CLASSES = (
'myapp.middleware.LastSeenMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
)
I have been trying to set up CSRF protection on my POST forms on my django 1.4.3 site and have been getting the 403 CSRF cookie not set error.
I believe I have followed the doc in detail, followed advice on many forums but all to no avail. I have stripped back the code as much as possible and the cookie just does not set. The csrfmiddlewaretoken is present, so that's not the source of the problem. I'm at my wit's end. No doubt the answer is stupidly simple, I just need a fresh set of eyes to point it out to me...
So here's some code:
#urls.py
...
url(r'^csrf/$', csrf_test, name='csrf_test'),
...
#views.py
...
from django.views.decorators.csrf import csrf_protect
#csrf_protect
def csrf_test(request):
c = {}
c.update(csrf(request))
return render_to_response('csrf.html', c)
...
<!-- csrf.html -->
<html>
<body>
<form action="" method="post">
{% csrf_token %}
<input name="Submit" value="Submit" type="submit"/>
</form>
</body>
</html>
#settings.py
...
MIDDLEWARE_CLASSES = (
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
#for db transactions. Check middleware order if faulty
'django.middleware.transaction.TransactionMiddleware',
# Uncomment the next line for simple clickjacking protection:
# 'django.middleware.clickjacking.XFrameOptionsMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware',
)
...
TEMPLATE_CONTEXT_PROCESSORS = global_settings.TEMPLATE_CONTEXT_PROCESSORS + (
'django.core.context_processors.csrf',
'django.core.context_processors.request',
)
...
I am running the django development server on 127.0.0.1:8000.
I have cookies enabled on my browsers and have confirmed cookies are being set from other websites.
I have checked the csrfmiddlewaretoken is being passed as a POST variable.
I always get the error, this is not intermittent.
I have no problem with POST forms when I have disabled the middleware/used #csrf_exempt.
As I say, I'm at my wit's end. What on earth have I missed?
Thanks
Nathan
EDIT:
It now works, but I have no idea what has changed. For anyone reading this in the future I created a new project and app, it created the cookie straight away. Thinking it must have been a settings issue I changed the order of my MIDDLEWARE_CLASSES entries by moving 'django.middleware.csrf.CsrfViewMiddleware' down below 'django.contrib.sessions.middleware.SessionMiddleware'. I ran the original app and it worked. However to test whether the settings change was responsible I reversed the change. It still worked. Go figure.
I hope this helps someone in the future FWIW.
Nathan
if you're using render_to_response then you should probably use a RequestContext
This will handle the csrf protection for you -
def csrf_test(request):
render_to_response('csrf.html', context_instance=RequestContext(request))
You don't need to include the csrf_protect decorator, since you've got the csrf middleware.
def csrf_test(request):
.....
return render(request, 'csrf.html', { ..... })