I implemented django's per site cache in a django-oscar project with django-oscar-api. I use LocMemCache as a cache backend.
Before adding django.middleware.cache.UpdateCacheMiddleware to my middlewares, everything worked fine and I could make a request to the "basket" endpoint like this and it returned a 200 response:
import requests
session = session.Session()
r_basket = session.get("http://127.0.0.1:8000/api/basket/")
After adding the per-site caching the response has the status code 500. I debugged it and it fails in the UpdateCacheMiddleware with the following error: {TypeError}cannot pickle '_io.BufferedReader' object.
Other endpoints seem to work fine, but I haven't tested them all yet.
The error can also be reproduced on a freshly installed django-oscar sandbox. I pip-installed the django-oscar-api and added it to the installed apps, and the MIDDLEWARE setting looks like this:
MIDDLEWARE = [
'debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
# Allow languages to be selected
'django.middleware.locale.LocaleMiddleware',
'django.middleware.http.ConditionalGetMiddleware',
'django.middleware.common.CommonMiddleware',
# per site caching
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
# Ensure a valid basket is added to the request instance for every request
'oscar.apps.basket.middleware.BasketMiddleware',
]
My package versions are Django v.3.2.9, django-oscar v.2.2 or 3.0, django-oscar-api v.2.1.1.
What object can not be pickled? Or is the order of the middlewares possibly wrong?
Traceback:
[2021-11-13 08:43:33,256] Internal Server Error: /api/basket/
Traceback (most recent call last):
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/utils/deprecation.py", line 119, in __call__
response = self.process_response(request, response)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/middleware/cache.py", line 111, in process_response
response.add_post_render_callback(
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/template/response.py", line 92, in add_post_render_callback
callback(self)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/middleware/cache.py", line 112, in <lambda>
lambda r: self.cache.set(cache_key, r, timeout)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/debug_toolbar/panels/cache.py", line 39, in wrapped
value = method(self, *args, **kwargs)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/debug_toolbar/panels/cache.py", line 94, in set
return self.cache.set(*args, **kwargs)
File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/core/cache/backends/locmem.py", line 56, in set
pickled = pickle.dumps(value, self.pickle_protocol)
TypeError: cannot pickle '_io.BufferedReader' object
expected response
{'id': 1,
'owner': None,
'status': 'Open',
'lines': 'http://127.0.0.1:8000/api/baskets/1/lines/',
'url': 'http://127.0.0.1:8000/api/baskets/1/',
'total_excl_tax': '0.00',
'total_excl_tax_excl_discounts': '0.00',
'total_incl_tax': '0.00',
'total_incl_tax_excl_discounts': '0.00',
'total_tax': '0.00',
'currency': None,
'voucher_discounts': [],
'offer_discounts': [],
'is_tax_known': True}
View: oscarapi.views.basket.BasketView
You need to ensure that UpdateCacheMiddleware is before SessionMiddleware and LocaleMiddleware in your settings. DOCS
FetchFromCacheMiddleware needs to be after those middleware, can be left where it is
MIDDLEWARE = [
'debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
# Moved UpdateCacheMiddleware to here
'django.middleware.cache.UpdateCacheMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
# Allow languages to be selected
'django.middleware.locale.LocaleMiddleware',
'django.middleware.http.ConditionalGetMiddleware',
'django.middleware.common.CommonMiddleware',
# per site caching
'django.middleware.cache.FetchFromCacheMiddleware',
# Ensure a valid basket is added to the request instance for every request
'oscar.apps.basket.middleware.BasketMiddleware',
]
I am using django 1.8 along with mongoengine and a custom Middleware that is supposed to add a user and a toked to a django request.
These two are passed in the header of the request.
The middleware class is the following :
from models import MongoToken
class TokenAuthenticationMiddleware(object):
def process_request(self, request):
if "HTTP_AUTHORIZATION" not in request.META:
return
tokenkey = get_authorization_header(request).split()[1]
token = MongoToken.objects.get(key=tokenkey)
user = User.objects.get(username=request.META.get("HTTP_USERNAME"))
if token.key == tokenkey and token.user.is_active:
request.user = user
request.token = tokenkey
I also put this in my MIDDLEWARE_CLASSES block of the settings.py files:
MIDDLEWARE_CLASSES = [
'corsheaders.middleware.CorsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'myproject.middleware.MongoAuthenticationMiddleware',
'myproject.middleware.TokenAuthenticationMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
And when the considered view is reached, my token is here because thanks to the header params but the user is Null.
Then I am wondering if I did something wrong with this because it does not work.
Thank you for your help.
Alex.
I am struggling getting MiddleWare to work. I put this in my settings.py:
MIDDLEWARE_CLASSES = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'portal.middleware.SimpleMiddleware'
]
and I implemented this class in portal/middleware/MiddleWare.py:
class SimpleMiddleware:
def __init__(self, get_response):
self.get_response = get_response
# One-time configuration and initialization.
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response
But when running, I get a TypeError:
TypeError: __init__() takes exactly 2 arguments (1 given)
Since you are writing a new-style middleware, you should use MIDDLEWARE instead of MIDDLEWARE_CLASSES in your settings.
You also need to make sure that your entry in the settings matches the location of the middleware. The entry 'portal.middleware.SimpleMiddleware' suggests a SimpleMiddleware class in portal/middleware.py, which does not match your filename portal/middleware/MiddleWare.py.
I am trying my hands around using memcache with django on per-view cache.
The trouble is cache only gets set if i refresh a page,clicking on same link does not sets the cache.(that is if i set cache_view on dispatch and reload page, i see the number of queries fall down to 3-4 queries otherwise on clicking the same link, cache is not set, and i get the same number of queries even after hitting the same url again and again)
Here is my views:
class ProductCategoryView(TemplateView):
"""
Browse products in a given category
"""
context_object_name = "products"
template_name = 'catalogue/category.html'
enforce_paths = True
#method_decorator(cache_page(30))
def dispatch(self, request, *args, **kwargs):
return super(ProductCategoryView, self).dispatch(request, *args, **kwargs)
My cache settings is:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
My middlewares are:
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
'oscar.apps.basket.middleware.BasketMiddleware',
'django.middleware.transaction.TransactionMiddleware',
)
Thanks.
I am trying to get middleware to work with Django, but I get an error:
object.__new__() takes no parameters
I have added the middleware tuple:
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
)
My view looks like so:
from middleware import my_mw
#my_mw
def start(request):
do stuff...
return render_to_response('a.html', {})
middleware.py:
class my_mw(object):
def process_request(self, request):
x = 6
return None
I have also tried various other middleware functions that I have copy pasted from examples. I always get the same error. At this point I'd really just like to get any middleware function to work!
I needed to add the middleware function to the middleware tuple:
'my_app.middleware.my_mw',
And then I don't include #my_mw