Im trying to use Google Cloud CDN to cache Django responses. Django properly sets the cache-control header with the max age, but it is missing the 'public' part of the header.
Basically, currently all cached views have:
cache-control: max-age=3600
But I want:
cache-control: max-age=3600,public
Edit:
My current settings.py have the following for caching:
MIDDLEWARE = [
...
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
...
]
....
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 3600
CACHE_MIDDLEWARE_KEY_PREFIX = ''
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": os.environ.get('CACHE_URL'),
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"IGNORE_EXCEPTIONS": True,
"MAX_ENTRIES": 10000,
}
}
}
So I basically followed https://docs.djangoproject.com/en/3.1/topics/cache/. So I cached the entire site. And for those views which I don't want cached I use #never_cache.
Edit 2:
I could add "public=True" to each view with an annotation but I want a solution for the entire site.
For now I ended up just going for a simple middleware. Though I am open for suggestions.
class PublicCacheMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
if response.has_header('Cache-Control') and 'private' not in response.get('Cache-Control'):
response['Cache-Control'] += ',public'
return response
Related
With Ionic Angular running in on my localhost I made this call to my Django backend (running on different localhost):
test() {
return this.httpClient.get(endpoint + '/test', {
headers: { mode: 'no-cors' },
});
}
And on the backend side I have following code to respond:
#csrf_exempt
def test(request):
response = json.dumps({'success': True})
return HttpResponse(response, content_type='application/json', headers={'Access-Control-Allow-Origin': '*'})
I have also this in my settings.py file:
INSTALLED_APPS = [
...
'corsheaders',
]
MIDDLEWARE = [
...
'corsheaders.middleware.CorsMiddleware',
]
CORS_ALLOW_ALL_ORIGINS = True
CORS_ALLOW_CREDENTIALS = True
Still, I get this error message in my console:
Access to XMLHttpRequest at 'http://127.0.0.1:8000/test' from origin 'http://localhost:8100' has been blocked by CORS policy: Request header field mode is not allowed by Access-Control-Allow-Headers in preflight response.
What am I doing wrong?
You need to just add one more setting
CORS_ALLOW_ALL_HEADERS=True
Other than the above you do not need to set a header on each response. Just simply respond back with payload as
#csrf_exempt
def test(request):
response = json.dumps({'success': True})
return HttpResponse(response, content_type='application/json', headers={'Access-Control-Allow-Origin': '*'})
I want to make cross-site JavaScript call from third-party domain (in this case my desktop/localhost server) to my remote Django server hosted on my_domain.com/ and calling REST WS exposed on my_domain.com/msg/my_service with using session/cookies for storing session state.
But when I call this service (hosted on remote Django server) from my desktop browser or localhost Django server (JS is in index.html), Django doesn't create session cookie and on remote server are doesn't store session state. But when i call this service from Postman or from same localhost JS to localhost instance of same Django service it works and session is created.
My JS script in index.html which make call to WS send_message:
fetch('http://my_domain.com/ws/my_service', {
method:"POST",
credentials: 'include',
body:JSON.stringify(data)
})
.then(res => res.json())
.then(json => {
showResponse(json.message);
})
When I run this script from my desktop browser or my localhost server it runs correctly with cookies and sessions parameters.
Django my_service implementation view
#csrf_exempt
def my_service(request):
if request.method == "POST":
message_bstream= request.body
request.session.set_expiry(0)
message= json.loads(message_bstream)
out,sta=state_machine_response(message["message"],int(request.session["state"]))
request.session["state"] =sta
respo={"message":out}
response = HttpResponse(json.dumps(respo), content_type="application/json")
response.set_cookie(key='name', value='my_value', samesite='None', secure=True)
#return JsonResponse(respo, safe=False, status=200)
return response
else:
return HttpResponseNotFound ("Sorry this methode is not allowed")
Or I try generate response like
return JsonResponse(respo, safe=False, status=200)
My settings.py
INSTALLED_APPS = [
...
'corsheaders',
]
CORS_ALLOW_CREDENTIALS = True
CORS_ORIGIN_WHITELIST = (
'http://localhost:8000',
)
CORS_ALLOWED_ORIGINS = [
'http://localhost:8000',
]
CSRF_TRUSTED_ORIGINS = [
'http://localhost:8000',
]
MIDDLEWARE = [
'corsheaders.middleware.CorsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
CSRF_COOKIE_SECURE = True
CSRF_COOKIE_SAMESITE = 'None'
SESSION_COOKIE_SAMESITE = 'None'
Please do you have any idea?
You can't save cookies from a third-party API call unless you use SameSite=None with the Secure option in the Set-Cookie header. You can achieve this for the sessionid and CSRF cookie with the following settings:
CSRF_COOKIE_SECURE = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SAMESITE = 'None'
SESSION_COOKIE_SAMESITE = 'None'
In this case, you must use the HTTPS protocol scheme.
Another solution would be to use a proxy, which is helpful in a localhost development environment. This is an example using vue.js on the vue.config.js file:
const { defineConfig } = require('#vue/cli-service')
if (process.env.NODE_ENV == "development"){
module.exports = defineConfig({
transpileDependencies: true,
devServer: {
proxy: {
"^/api/": {
target: process.env.VUE_APP_BASE_URL,
changeOrigin: true,
logLevel: "debug",
pathRewrite: { "^/api": "/api" }
}
}
},
})
}
Some useful doc https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite#samesitenone_requires_secure
I'm trying to use a Memcached instance of AWS ElastiCache with a Django project. It seems to be caching a view for a user, but if you come in on a different PC, it isn't cached until called from that PC (or same PC with different browser).
I'm not sure what I've got wrong.
Within settings.py I have
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
}
}
MIDDLEWARE = [
'core.middleware.DenyIndexMiddleware',
'core.middleware.XForwardedForMiddleware',
'core.middleware.PrimaryHostRedirectMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
'masquerade.middleware.MasqueradeMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sites.middleware.CurrentSiteMiddleware',
'cms.middleware.user.CurrentUserMiddleware',
'cms.middleware.page.CurrentPageMiddleware',
'cms.middleware.toolbar.ToolbarMiddleware',
'cms.middleware.language.LanguageCookieMiddleware',
'cms.middleware.utils.ApphookReloadMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
]
I've then cached the views using cache_page
path('<str:service_type>/<str:location>/', cache_page(60*60)(views.canonical_search), name="canonical-search"),
How do I cache the site so that the page is cached irrespective of the user?
EDIT
I've noticed that it never caches when using the user is logged in.
Watch out for the Vary header, that cache_page() takes into account.
Usually, some middlewares may add a Vary header, for example :
CsrfViewMiddleware adds Cookie,
GZipMiddlewareadds Accept-Encoding
LanguageCookieMiddleware may add Accept-Language
meaning that as soon as you have a different Cookie (session), encoding, or language, you have a different version of cache for your page.
As for you case, the CsrfViewMiddleware may be the problem, you can add the decorator #csrf_exempt to your view so that the Vary: Cookie header is not set in the response.
More info at https://docs.djangoproject.com/en/3.0/topics/cache/#using-vary-headers
Though Django documentation, you can read this:
Django’s cache framework
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
If you need to store a page in the local
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': 'c:/foo/bar',
}
}
But, my recommendation is store the results (database result, assets, etc), as #PVSK show in this thread:
from django.core.cache import cache
def sample(request):
cached_data = cache.get_many(['query1', 'query2'])
if cached_data:
return render(request, 'sample.html', {'query1': cached_data['query1'], 'query2': cached_data['query2']})
else:
queryset1 = Model.objects.all()
queryset2 = Model2.objects.all()
cache.set_many({'query1': queryset1 , 'query2': queryset2 }, None)
return render(request, 'sample.html', {'query1': queryset1 , 'query2': queryset2})
Hm, at first I wondered if you're running into some cache default limitations. You are not using OPTIONS in your CACHE backend definition, so the cache is limited to 300 entries per default.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': os.environ.get('CACHE_LOCATION','127.0.0.1:11211'),
'OPTIONS': {
'MAX_ENTRIES': 1000
}
}
}
The next possible problem, that we also had, is that the cache_key generation takes the full QUERY_STRING into account, so (the ?param=bla). But you already stated that the url is the same for all users.
Next up, as SebCorbin correctly identified, are the potential Vary issues.
UpdateCacheMiddleware will never cache a cookie-setting response to a cookie-less request.
def process_response(self, request, response):
#...
# Don't cache responses that set a user-specific (and maybe security
# sensitive) cookie in response to a cookie-less request.
if not request.COOKIES and response.cookies and has_vary_header(response, 'Cookie'):
return response
# ...
The order of execution of middlewares is top-to-bottom for process_request and bottom-to-top for process_response.
I suspect that one of lower middlewares (or the view) is setting a cookie, so you might be able to work around this issue by moving the 'django.middleware.cache.UpdateCacheMiddleware' below the offending middleware, but you risk loosing features if you don't move feature-middlewares like LocaleMiddleware as well.
If your view code is setting cookies you'll need to switch to the low-level cache API to cache costly operations (or move the cookie logic into a middleware that lives above the UpdateCacheMiddleware middleware).
I have both JWT and session authentication middleware being used.
What i found was that i was not able to login to admin. Later when i removed JWT middleware it worked.
My site will is serving as both api login and normal browser login. How to use it for both.
The only option left is the below condition for jwt.
if request.content_type == 'application/json':
How to resolve this
I am not using DRF to create api endpoints. That why i have to create custom middleware to verify JWT token
Django settings:
MIDDLEWARE = (
'debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'webarticles.middleware.jwtWebtoken_mod.BaseJSONWebTokenAuthentication_mod',
)
webarticles.middleware.jwtWebtoken_mod.BaseJSONWebTokenAuthentication_mod
from rest_framework_jwt.authentication import JSONWebTokenAuthentication
from rest_framework import exceptions
import json
from django.http import HttpResponse
from rest_framework.settings import api_settings as api_settings2
from rest_framework_jwt.settings import api_settings
jwt_decode_handler = api_settings.JWT_DECODE_HANDLER
jwt_get_username_from_payload = api_settings.JWT_PAYLOAD_GET_USERNAME_HANDLER
class BaseJSONWebTokenAuthentication_mod(JSONWebTokenAuthentication):
"""
Token based authentication using the JSON Web Token standard.
"""
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
if request.content_type == 'application/json':
try:
user_auth_tuple = self.authenticate(request)
except exceptions.APIException as e:
self._not_authenticated(request)
hare = e.get_full_details()
#hare = {"e": str(e)}
# return HttpResponse(
# json.dumps(hare),
# content_type="application/json"
# )
return HttpResponse(
json.dumps(hare),
content_type="application/json",
status=e.status_code
)
if user_auth_tuple is not None:
request._authenticator = self
request.user, request.auth = user_auth_tuple
else:
self._not_authenticated(request)
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response
def _not_authenticated(self,request):
"""
Set authenticator, user & authtoken representing an unauthenticated request.
Defaults are None, AnonymousUser & None.
"""
request._authenticator = None
if api_settings2.UNAUTHENTICATED_USER:
request.user = api_settings2.UNAUTHENTICATED_USER()
else:
request.user = None
if api_settings2.UNAUTHENTICATED_TOKEN:
request.auth = api_settings2.UNAUTHENTICATED_TOKEN()
else:
request.auth = None
Presently I have put the below condition to manage
if request.content_type == 'application/json':
Use Postman to make requests and once you are successfully able to make requests. Generate code by pressing code button placed at right hand side.
I have made this node.js script from postman to login to particular website.
var request = require("request");
var options = {
method: 'POST',
url: 'http://abc.xyz.com/',
headers:
{
'postman-token': 'xxxx-xxxx-xx-xx-xxx',
'cache-control': 'no-cache',
authorization: 'Basic xxxxxxxxxxxxxxxxxxxxx',
'content-type': 'application/json'
},
body:
{
username: 'yourusername.com',
password: 'Your#password'
},
json: true
};
request(options, function (error, response, body)
{
if (error) throw new Error(error);
console.log(body);
});
We are trying to implement caching in Django. We have tried memcache, file system caching and local memory caching. No mater what it just isn't working -- a timestamp we put on the template to test caching is always updating so we know it's not working.
Here are relevant portions of code.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': 'c:/tmp',
}
}
CACHE_MIDDLEWARE_ALIAS = "default"
CACHE_MIDDLEWARE_SECONDS = 600
MIDDLEWARE_CLASSES = [
"django.middleware.cache.UpdateCacheMiddleware",
... OTHER ONES ...
"django.middleware.cache.FetchFromCacheMiddleware",
]
VIEWS.PY
from django.views.decorators.cache import cache_page
#cache_page(60 * 10)
def profiles(request, template_name="profiles/profiles.html", extra_context=None):
...