Code to run in all views in a views.py file - django

What would be the best way of putting a bit of code to run for all views in a views.py file?
I come from a PHP background and I normally put this in the constructor/index bit so that it always ran whatever page is being requested. It has to be specific for that views.py file though, I want to check that the user has access to 'this app/module' and want to avoid having to use decorators on all views if possible?

TL;DR
You should check about middlewares. It allows to execute some code before the view execution, the template rendering and other stuff.
Some words about middlewares
You can represent middlewares in your head like this:
As you can see, the request (orange arrow) go through every middleware before executing the view and then can hitting every middleware after (if you want to do something before the template processing for example).
Using Django 1.10
Arcitecture of middlewares have changed in Django 1.10, and are now represented by simple function. For example, here's a counter of visits for each page:
def simple_middleware(get_response):
# One-time configuration and initialization.
def middleware(request):
try:
p = Page.objects.get(url=request.path)
p.nb_visits += 1
p.save()
except Page.DoesNotExist:
Page(url=request.path).save()
response = get_response(request)
if p:
response.content += "This page has been seen {0} times.".format(p.nb_visits)
return response
return middleware
And voilĂ .
Using Django
Here's an example of middleware, which would update a counter for each visit of a page (admit that a Page Model exists with two field : url and nb_visits)
class StatsMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
try:
p = Page.objects.get(url=request.path)
p.nb_visits += 1
p.save()
except Page.DoesNotExist:
Page(url=request.path).save()
def process_response(self, request, response):
if response.status_code == 200:
p = Page.objects.get(url=request.path)
# Let's say we add our info after the html response (dirty, yeah I know)
response.content += u"This page has been seen {0} times.".format(p.nb_visits)
return response
Hopes this will help you :)

Middleware is the solution but keep in mind the the order to define the middleware in the settings.py matters.

Related

How does interacting with the django rest api through the url work?

I get that the Django rest framework is for interacting with the Django server programmatically but one thing I still don't understand is how.what i want to do is have my client app (mobile app) send data (somehow) to the Django server in order to create/retrieve data based on variables and obviously this has to be done through the URL since there will be no direct GUI interaction with the API. (unless I'm mistaken, which I probably am) I have gone through the official documentation and followed the tutorial to the end and still don't understand how this is supposed to work.all I ask for is a quick and simple explanation because I have searched everywhere and haven't found a simple enough explanation to grasp the core concept of how this is all supposed to work.
I think what you're looking for is JSONResponse and related objects:
This will allow you to send JSON in response to a request.
from django.http import JsonResponse
def my_view_json(request):
response = JsonResponse({'foo': 'bar'})
return response
If your templates or webpages need to make a request to a view and specify different parameters, they can do so by adding POST variables (examples). These can be parsed in the view like so:
def myView(request):
my_post_var = request.POST.get('variable_name', 'default_value')
my_get_var = request.GET.get('variable_name', 'default_value')
You can then parse what was sent any way you like and decide what you want to do with it.
Basically,
You define the URLS upon which you perform Get/POST/PUT Requests and You can Send Data to that.
Eg:
urls.py
from django.conf.urls import url,include
from app import views
urlpatterns = [
url(r'^(?i)customertype/$',views.CustomerViewSet.as_view()),
url(r'^(?i)profile/$', views.Save_Customer_Profile.as_view()),
url(r'^(?i)customer_image/$', views.Save_Customer_Image.as_view()),
]
Now Whenever User would send a Request to:
example.com/profile ==> This would be received in the Save_Customer_Profile View based on the Method Type, Save_Customer_Profile is as follows:
class Save_Customer_Profile(APIView):
"""Saves and Updates User Profile!"""
def get(self, request, format=None):
return AllImports.Response({"Request":"Method Type is GET Request"})
def post(self, request, format=None):
return AllImports.Response({"Request":"Method Type is Post Request"})
def put(self,request, format=None):
return AllImports.Response({"Request":"Method Type is Put Request"})
I think the OP was referring to how to do GET/POST request programmatically. In that case, it is enough to do (values are dummy):
GET:
import requests
r = requests.get('http://localhost:8000/snippets/')
print(r.json())
print(r.status_code, r.reason)
POST:
data = {'code': 'print(" HELLO !!!")', 'language': 'java','owner': 'testuser'}
r = requests.post('http://localhost:8000/snippets/', data=data, auth=('testuser', 'test'))

Django cache_page with keys [duplicate]

The #cache_page decorator is awesome. But for my blog I would like to keep a page in cache until someone comments on a post. This sounds like a great idea as people rarely comment so keeping the pages in memcached while nobody comments would be great. I'm thinking that someone must have had this problem before? And this is different than caching per url.
So a solution I'm thinking of is:
#cache_page( 60 * 15, "blog" );
def blog( request ) ...
And then I'd keep a list of all cache keys used for the blog view and then have way of expire the "blog" cache space. But I'm not super experienced with Django so I'm wondering if someone knows a better way of doing this?
This solution works for django versions before 1.7
Here's a solution I wrote to do just what you're talking about on some of my own projects:
def expire_view_cache(view_name, args=[], namespace=None, key_prefix=None):
"""
This function allows you to invalidate any view-level cache.
view_name: view function you wish to invalidate or it's named url pattern
args: any arguments passed to the view function
namepace: optioal, if an application namespace is needed
key prefix: for the #cache_page decorator for the function (if any)
"""
from django.core.urlresolvers import reverse
from django.http import HttpRequest
from django.utils.cache import get_cache_key
from django.core.cache import cache
# create a fake request object
request = HttpRequest()
# Loookup the request path:
if namespace:
view_name = namespace + ":" + view_name
request.path = reverse(view_name, args=args)
# get cache key, expire if the cached item exists:
key = get_cache_key(request, key_prefix=key_prefix)
if key:
if cache.get(key):
# Delete the cache entry.
#
# Note that there is a possible race condition here, as another
# process / thread may have refreshed the cache between
# the call to cache.get() above, and the cache.set(key, None)
# below. This may lead to unexpected performance problems under
# severe load.
cache.set(key, None, 0)
return True
return False
Django keys these caches of the view request, so what this does is creates a fake request object for the cached view, uses that to fetch the cache key, then expires it.
To use it in the way you're talking about, try something like:
from django.db.models.signals import post_save
from blog.models import Entry
def invalidate_blog_index(sender, **kwargs):
expire_view_cache("blog")
post_save.connect(invalidate_portfolio_index, sender=Entry)
So basically, when ever a blog Entry object is saved, invalidate_blog_index is called and the cached view is expired. NB: haven't tested this extensively, but it's worked fine for me so far.
The cache_page decorator will use CacheMiddleware in the end which will generate a cache key based on the request (look at django.utils.cache.get_cache_key) and the key_prefix ("blog" in your case). Note that "blog" is only a prefix, not the whole cache key.
You can get notified via django's post_save signal when a comment is saved, then you can try to build the cache key for the appropriate page(s) and finally say cache.delete(key).
However this requires the cache_key, which is constructed with the request for the previously cached view. This request object is not available when a comment is saved. You could construct the cache key without the proper request object, but this construction happens in a function marked as private (_generate_cache_header_key), so you are not supposed to use this function directly. However, you could build an object that has a path attribute that is the same as for the original cached view and Django wouldn't notice, but I don't recommend that.
The cache_page decorator abstracts caching quite a bit for you and makes it hard to delete a certain cache object directly. You could make up your own keys and handle them in the same way, but this requires some more programming and is not as abstract as the cache_page decorator.
You will also have to delete multiple cache objects when your comments are displayed in multiple views (i.e. index page with comment counts and individual blog entry pages).
To sum up: Django does time based expiration of cache keys for you, but custom deletion of cache keys at the right time is more tricky.
I wrote Django-groupcache for this kind of situations (you can download the code here). In your case, you could write:
from groupcache.decorators import cache_tagged_page
#cache_tagged_page("blog", 60 * 15)
def blog(request):
...
From there, you could simply do later on:
from groupcache.utils import uncache_from_tag
# Uncache all view responses tagged as "blog"
uncache_from_tag("blog")
Have a look at cache_page_against_model() as well: it's slightly more involved, but it will allow you to uncache responses automatically based on model entity changes.
With the latest version of Django(>=2.0) what you are looking for is very easy to implement:
from django.utils.cache import learn_cache_key
from django.core.cache import cache
from django.views.decorators.cache import cache_page
keys = set()
#cache_page( 60 * 15, "blog" );
def blog( request ):
response = render(request, 'template')
keys.add(learn_cache_key(request, response)
return response
def invalidate_cache()
cache.delete_many(keys)
You can register the invalidate_cache as a callback when someone updates a post in the blog via a pre_save signal.
This won't work on django 1.7; as you can see here https://docs.djangoproject.com/en/dev/releases/1.7/#cache-keys-are-now-generated-from-the-request-s-absolute-url the new cache keys are generated with the full URL, so a path-only fake request won't work. You must setup properly request host value.
fake_meta = {'HTTP_HOST':'myhost',}
request.META = fake_meta
If you have multiple domains working with the same views, you should cycle them in the HTTP_HOST, get proper key and do the clean for each one.
Django view cache invalidation for v1.7 and above. Tested on Django 1.9.
def invalidate_cache(path=''):
''' this function uses Django's caching function get_cache_key(). Since 1.7,
Django has used more variables from the request object (scheme, host,
path, and query string) in order to create the MD5 hashed part of the
cache_key. Additionally, Django will use your server's timezone and
language as properties as well. If internationalization is important to
your application, you will most likely need to adapt this function to
handle that appropriately.
'''
from django.core.cache import cache
from django.http import HttpRequest
from django.utils.cache import get_cache_key
# Bootstrap request:
# request.path should point to the view endpoint you want to invalidate
# request.META must include the correct SERVER_NAME and SERVER_PORT as django uses these in order
# to build a MD5 hashed value for the cache_key. Similarly, we need to artificially set the
# language code on the request to 'en-us' to match the initial creation of the cache_key.
# YMMV regarding the language code.
request = HttpRequest()
request.META = {'SERVER_NAME':'localhost','SERVER_PORT':8000}
request.LANGUAGE_CODE = 'en-us'
request.path = path
try:
cache_key = get_cache_key(request)
if cache_key :
if cache.has_key(cache_key):
cache.delete(cache_key)
return (True, 'successfully invalidated')
else:
return (False, 'cache_key does not exist in cache')
else:
raise ValueError('failed to create cache_key')
except (ValueError, Exception) as e:
return (False, e)
Usage:
status, message = invalidate_cache(path='/api/v1/blog/')
I had same problem and I didn't want to mess with HTTP_HOST, so I created my own cache_page decorator:
from django.core.cache import cache
def simple_cache_page(cache_timeout):
"""
Decorator for views that tries getting the page from the cache and
populates the cache if the page isn't in the cache yet.
The cache is keyed by view name and arguments.
"""
def _dec(func):
def _new_func(*args, **kwargs):
key = func.__name__
if kwargs:
key += ':' + ':'.join([kwargs[key] for key in kwargs])
response = cache.get(key)
if not response:
response = func(*args, **kwargs)
cache.set(key, response, cache_timeout)
return response
return _new_func
return _dec
To expired page cache just need to call:
cache.set('map_view:' + self.slug, None, 0)
where self.slug - param from urls.py
url(r'^map/(?P<slug>.+)$', simple_cache_page(60 * 60 * 24)(map_view), name='map'),
Django 1.11, Python 3.4.3
FWIW I had to modify mazelife's solution to get it working:
def expire_view_cache(view_name, args=[], namespace=None, key_prefix=None, method="GET"):
"""
This function allows you to invalidate any view-level cache.
view_name: view function you wish to invalidate or it's named url pattern
args: any arguments passed to the view function
namepace: optioal, if an application namespace is needed
key prefix: for the #cache_page decorator for the function (if any)
from: http://stackoverflow.com/questions/2268417/expire-a-view-cache-in-django
added: method to request to get the key generating properly
"""
from django.core.urlresolvers import reverse
from django.http import HttpRequest
from django.utils.cache import get_cache_key
from django.core.cache import cache
# create a fake request object
request = HttpRequest()
request.method = method
# Loookup the request path:
if namespace:
view_name = namespace + ":" + view_name
request.path = reverse(view_name, args=args)
# get cache key, expire if the cached item exists:
key = get_cache_key(request, key_prefix=key_prefix)
if key:
if cache.get(key):
cache.set(key, None, 0)
return True
return False
Instead of using the cache page decorator, you could manually cache the blog post object (or similar) if there are no comments, and then when there's a first comment, re-cache the blog post object so that it's up to date (assuming the object has attributes that reference any comments), but then just let that cached data for the commented blog post expire and then no bother re-cacheing...
Instead of explicit cache expiration you could probably use new "key_prefix" every time somebody comment the post. E.g. it might be datetime of the last post's comment (you could even combine this value with the Last-Modified header).
Unfortunately Django (including cache_page()) does not support dynamic "key_prefix"es (checked on Django 1.9) but there is workaround exists. You can implement your own cache_page() which may use extended CacheMiddleware with dynamic "key_prefix" support included. For example:
from django.middleware.cache import CacheMiddleware
from django.utils.decorators import decorator_from_middleware_with_args
def extended_cache_page(cache_timeout, key_prefix=None, cache=None):
return decorator_from_middleware_with_args(ExtendedCacheMiddleware)(
cache_timeout=cache_timeout,
cache_alias=cache,
key_prefix=key_prefix,
)
class ExtendedCacheMiddleware(CacheMiddleware):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
if callable(self.key_prefix):
self.key_function = self.key_prefix
def key_function(self, request, *args, **kwargs):
return self.key_prefix
def get_key_prefix(self, request):
return self.key_function(
request,
*request.resolver_match.args,
**request.resolver_match.kwargs
)
def process_request(self, request):
self.key_prefix = self.get_key_prefix(request)
return super().process_request(request)
def process_response(self, request, response):
self.key_prefix = self.get_key_prefix(request)
return super().process_response(request, response)
Then in your code:
from django.utils.lru_cache import lru_cache
#lru_cache()
def last_modified(request, blog_id):
"""return fresh key_prefix"""
#extended_cache_page(60 * 15, key_prefix=last_modified)
def view_blog(request, blog_id):
"""view blog page with comments"""
Most of the solutions above didn't work in our case because we use https. The source code for get_cache_key reveals that it uses request.get_absolute_uri() to generate the cache key.
The default HttpRequest class sets the scheme as http. Thus we need to override it to use https for our dummy request object.
This is the code that works fine for us :)
from django.core.cache import cache
from django.http import HttpRequest
from django.utils.cache import get_cache_key
class HttpsRequest(HttpRequest):
#property
def scheme(self):
return "https"
def invalidate_cache_page(
path,
query_params=None,
method="GET",
):
request = HttpsRequest()
# meta information can be checked from error logs
request.META = {
"SERVER_NAME": "www.yourwebsite.com",
"SERVER_PORT": "443",
"QUERY_STRING": query_params,
}
request.path = path
key = get_cache_key(request, method=method)
if cache.has_key(key):
cache.delete(key)
Now I can use this utility function to invalidate the cache from any of our views:
page = reverse('url_name', kwargs={'id': obj.id})
invalidate_cache_page(path)
Duncan's answer works well with Django 1.9. But if we need invalidate url with GET-parameter we have to make a little changes in request.
Eg for .../?mykey=myvalue
request.META = {'SERVER_NAME':'127.0.0.1','SERVER_PORT':8000, 'REQUEST_METHOD':'GET', 'QUERY_STRING': 'mykey=myvalue'}
request.GET.__setitem__(key='mykey', value='myvalue')
I struggled with a similar situation and here is the solution I came up with, I started it on an earlier version of Django but it is currently in use on version 2.0.3.
First issue: when you set things to be cached in Django, it sets headers so that downstream caches -- including the browser cache -- cache your page.
To override that, you need to set middleware. I cribbed this from elsewhere on StackOverflow, but can't find it at the moment. In appname/middleware.py:
from django.utils.cache import add_never_cache_headers
class Disable(object):
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
add_never_cache_headers(response)
return response
Then in settings.py, to MIDDLEWARE, add:
'appname.middleware.downstream_caching.Disable',
Keep in mind that this approach completely disables downstream caching, which may not be what you want.
Finally, I added to my views.py:
def expire_page(request, path=None, query_string=None, method='GET'):
"""
:param request: "real" request, or at least one providing the same scheme, host, and port as what you want to expire
:param path: The path you want to expire, if not the path on the request
:param query_string: The query string you want to expire, as opposed to the path on the request
:param method: the HTTP method for the page, if not GET
:return: None
"""
if query_string is not None:
request.META['QUERY_STRING'] = query_string
if path is not None:
request.path = path
request.method = method
# get_raw_uri and method show, as of this writing, everything used in the cache key
# print('req uri: {} method: {}'.format(request.get_raw_uri(), request.method))
key = get_cache_key(request)
if key in cache:
cache.delete(key)
I didn't like having to pass in a request object, but as of this writing, it provides the scheme/protocol, host, and port for the request, pretty much any request object for your site/app will do, as long as you pass in the path and query string.
One more updated version of Duncan's answer: had to figure out correct meta fields: (tested on Django 1.9.8)
def invalidate_cache(path=''):
import socket
from django.core.cache import cache
from django.http import HttpRequest
from django.utils.cache import get_cache_key
request = HttpRequest()
domain = 'www.yourdomain.com'
request.META = {'SERVER_NAME': socket.gethostname(), 'SERVER_PORT':8000, "HTTP_HOST": domain, 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br'}
request.LANGUAGE_CODE = 'en-us'
request.path = path
try:
cache_key = get_cache_key(request)
if cache_key :
if cache.has_key(cache_key):
cache.delete(cache_key)
return (True, 'successfully invalidated')
else:
return (False, 'cache_key does not exist in cache')
else:
raise ValueError('failed to create cache_key')
except (ValueError, Exception) as e:
return (False, e)
The solution is simple, and do not require any additional work.
Example
#cache_page(60 * 10)
def our_team(request, sorting=None):
...
This will set the response to the cache with the default key.
Expire a view cache
from django.utils.cache import get_cache_key
from django.core.cache import cache
def our_team(request, sorting=None):
# This will remove the cache value and set it to None
cache.set(get_cache_key(request), None)
Simple, Clean, Fast.

Django: How to provide context to all views (not templates)?

I want to provide some context to all my function-based views (FBV) similar to the way TEMPLATE_CONTEXT_PROCESSORS (CP) provides context to all of one's templates. The latter doesn't work for me because I need that context prior to rendering the templates.
In particular, on my site I have a function which takes a request and returns the model for the Category of item in focus. My CP provides this for all templates, but I find myself making the same call from my FBV's and would like to remove this redundancy.
This question is similar but it presupposes the approach of accessing the output of the CP from the views. This seems hacky, and I'm not sure it's the best approach.
What's the Django way to do this?
Use Middleware...
class MyModelMiddleware(object):
def process_request(self, request):
request.extra_model = self.get_model(request.user)
Based on mwjackson 's answer and on docs, for Django 1.11, I think the middleware should be:
# middleware/my_middleware.py
class MyModelMiddleware(object):
def __init__(self, get_response):
self.get_response = get_response
# One-time configuration and initialization.
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
# TODO - your processing here
request.extra_model = result_from_processing
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response
In settings.py, add the path to your Middleware on MIDDLEWARE = () . Following the tips from this site, I had created a folder inside my app called middleware and added a new file, say my_middleware.py, with a class called, say, MyModelMiddleware. So, the path that I had added to MIDDLEWARE was my_app.middleware.my_middleware.MyModelMiddleware.
# settings.py
MIDDLEWARE = (
...
'my_app.middleware.my_middleware.MyModelMiddleware',
)

Alternative of Requests in django

In my project I'm trying to hit a url(which is running in same project) in my view.
so in simplest way I can explain here.
#login_required
def my_api_view(request):
if requests.method == 'GET':
# do stuff
return JsonResponse()
# and its url is `/api/get-info/`
another view which is consuming above api
#login_required
def show_info(request):
url = get_base_url + /api/get-info/ # http://localhost:8000/api/get-info/
r = requests.get(url)
return HttpResponse(r.json())
Now I have to use same session (login required) so when I hit the url using requests it complains that user is not loggedin which is obviously correct.
How can I do this elegantly and efficienty. session use of logged in user. (I dont want to call the view as a function, I want to hit the api-url end point and consume.
PS: we have something similar in django Test self.client.get(...)
Just call that view function and pass the request object as parameter to it.
#login_required
def show_info(request):
r = my_api_view(request)
return HttpResponse(r.json())
Or a better option would be to simply separate the logic into a separate function, as mentioned by #koniiiik in the comments.
EDIT: Or if you really want to hit the URL endpoint, you can simply pass on the cookie values to the request you make.
#login_required
def show_info(request):
url = get_base_url + "/api/get-info/" # http://localhost:8000/api/get-info/
r = requests.get(url, cookies=request.COOKIES)
return HttpResponse(r.json())

How to display a custom error page for HTTP status 405 (method not allowed) in Django when using #require_POST

My question is simple, how do I display a custom error page for HTTP status 405 (method not allowed) in Django when using the #require_POST decorator?
I'm using the django.views.decorators.http.require_POST decorator, and when the page is visited by GET request, the console shows a 405 error, but the page is just blank (not even a Django error page). How do I get Django to display a custom and/or default error page for this kind of error?
EDIT:
It's worth mentioning that I've tried putting a 404.html, 500.html and 405.html page in my templates folder - but that does not help either. I have also varied between DEBUG = True and False, to no avail.
You have to write custom Django middleware. You can start with this one and extend it to check if 405.html file exists and so on:
from django.http import HttpResponseNotAllowed
from django.template import RequestContext
from django.template import loader
class HttpResponseNotAllowedMiddleware(object):
def process_response(self, request, response):
if isinstance(response, HttpResponseNotAllowed):
context = RequestContext(request)
response.content = loader.render_to_string("405.html", context_instance=context)
return response
Check docs if you don't know how to install middleware:
http://docs.djangoproject.com/en/dev/topics/http/middleware/
You can also check this article:
http://mitchfournier.com/2010/07/12/show-a-custom-403-forbidden-error-page-in-django/
If you look into the documentation and the source code of django.views.defaults you see that only 404 and 500 errors are supported in a way that you only have to add the 404.html resp. 500.html to your templates directory.
In the doc. you can also read the following
Returning HTTP error codes in Django
is easy. There are subclasses of
HttpResponse for a number of common
HTTP status codes other than 200
(which means "OK"). You can find the
full list of available subclasses in
the request/response documentation.
Thus if you want to return a 405 error, you have to use the HttpResponseNotAllowed class
An example
I'm not sure that's possible. Perhaps you should consider filing a bug report.
I landed here in 2022. The above accepted answer is not working for me. I use django rest framework. My sollution was to create a middleware with
#app/middleware.py
from django.http import HttpResponse
from django.template import loader
class HttpResponseNotAllowedMiddleware:
def __init__(self, get_response):
self.get_response = get_response
# One-time configuration and initialization.
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
if response.status_code == 405:
context = {}
template = loader.get_template('app/405.html')
return HttpResponse(template.render(context, request))
return response
then install it by adding this to settings
MIDDLEWARE = [
.......
'app.middleware.HttpResponseNotAllowedMiddleware',
]
the 405.html template is just a plain not allowed text