Execute code in Django after response has been sent to the client - django

In my Django application I want to keep track of whether a response has been sent to the client successfully. I am well aware that there is no "watertight" way in a connectionless protocol like HTTP to ensure the client has received (and displayed) a response, so this will not be mission-critical functionality, but still I want to do this at the latest possible time. The response will not be HTML so any callbacks from the client (using Javascript or IMG tags etc.) are not possible.
The "latest" hook I can find would be adding a custom middleware implementing process_response at the first position of the middleware list, but to my understanding this is executed before the actual response is constructed and sent to the client. Are there any hooks/events in Django to execute code after the response has been sent successfully?

The method I am going for at the moment uses a subclass of HttpResponse:
from django.template import loader
from django.http import HttpResponse
# use custom response class to override HttpResponse.close()
class LogSuccessResponse(HttpResponse):
def close(self):
super(LogSuccessResponse, self).close()
# do whatever you want, this is the last codepoint in request handling
if self.status_code == 200:
print('HttpResponse successful: %s' % self.status_code)
# this would be the view definition
def logging_view(request):
response = LogSuccessResponse('Hello World', mimetype='text/plain')
return response
By reading the Django code I am very much convinced that HttpResponse.close() is the latest point to inject code into the request handling. I am not sure if there really are error cases that are handled better by this method compared to the ones mentioned above, so I am leaving the question open for now.
The reasons I prefer this approach to the others mentioned in lazerscience's answer are that it can be set up in the view alone and does not require middleware to be installed. Using the request_finished signal, on the other hand, wouldn't allow me to access the response object.

If you need to do this a lot, a useful trick is to have a special response class like:
class ResponseThen(Response):
def __init__(self, data, then_callback, **kwargs):
super().__init__(data, **kwargs)
self.then_callback = then_callback
def close(self):
super().close()
self.then_callback()
def some_view(request):
# ...code to run before response is returned to client
def do_after():
# ...code to run *after* response is returned to client
return ResponseThen(some_data, do_after, status=status.HTTP_200_OK)
...helps if you want a quick/hacky "fire and forget" solution without bothering to integrate a proper task queue or split off a separate microservice from your app.

I suppose when talking about middleware you are thinking about the middleware's process_request method, but there's also a process_response method that is called when the HttpResponse object is returned. I guess that will be the latest moment where you can find a hook that you can use.
Furthermore there's also a request_finished signal being fired.

I modified Florian Ledermann's idea a little bit... So someone can just use the httpresponse function normally, but allows for them to define a function and bind it to that specific httpresponse.
old_response_close = HttpResponse.close
HttpResponse.func = None
def new_response_close(self):
old_response_close(self)
if self.func is not None:
self.func()
HttpResponse.close = new_response_close
It can be used via:
def myview():
def myfunc():
print("stuff to do")
resp = HttpResponse(status=200)
resp.func = myfunc
return resp
I was looking for a way to send a response, then execute some time consuming code after... but if I can get a background (most likely a celery) task to run, then it will have rendered this useless to me. I will just kick off the background task before the return statement. It should be asynchronous, so the response will be returned before the code is finished executing.
---EDIT---
I finally got celery to work with aws sqs. I basically posted a "how to". Check out my answer on this post:
Cannot start Celery Worker (Kombu.asynchronous.timer)

I found a filthy trick to do this by accessing a protected member in HttpResponse.
def some_view(request):
# ...code to run before response is returned to client
def do_after():
# ...code to run *after* response is returned to client
response = HttpResponse()
response._resource_closers.append(do_after)
return response
It works in Django 3.0.6 , check the "close" function in the prototype of HttpResponse.
def close(self):
for closer in self._resource_closers:
try:
closer()
except Exception:
pass
# Free resources that were still referenced.
self._resource_closers.clear()
self.closed = True
signals.request_finished.send(sender=self._handler_class)

Related

How to execute code in Django after response has been sent to the client (on PythonAnywhere)?

I'm looking for a way to execute code in Django after the response has been sent to the client. I know the usual way is to implement a task queue (e.g., Celery). However, the PaaS service I'm using (PythonAnywhere) doesn't support task queues as of May 2019. It also seems overly complex for a few simple use cases. I found the following solution on SO: Execute code in Django after response has been sent to the client. The accepted answer works great when run locally. However, in production on PythonAnywhere, it still blocks the response from being sent to the client. What is causing that?
Here's my implementation:
from time import sleep
from datetime import datetime
from django.http import HttpResponse
class HttpResponseThen(HttpResponse):
"""
WARNING: THIS IS STILL BLOCKING THE PAGE LOAD ON PA
Implements HttpResponse with a callback i.e.,
The callback function runs after the http response.
"""
def __init__(self, data, then_callback=lambda: 'hello world', **kwargs):
super().__init__(data, **kwargs)
self.then_callback = then_callback
def close(self):
super().close()
return_value = self.then_callback()
print(f"Callback return value: {return_value}")
def my_callback_function():
sleep(20)
print('This should print 20 seconds AFTER the page loads.')
print('On PA, the page actually takes 20 seconds to load')
def test_view(request):
return HttpResponseThen("Timestamp: "+str(datetime.now()),
then_callback=my_callback_function) # This is still blocking on PA
I'm expecting the response to be sent to the client immediately, but it actually takes a full 20 seconds for the page to load. (On my laptop, the code works great. The response is sent immediately and the print statements execute 20 seconds later.)

How does interacting with the django rest api through the url work?

I get that the Django rest framework is for interacting with the Django server programmatically but one thing I still don't understand is how.what i want to do is have my client app (mobile app) send data (somehow) to the Django server in order to create/retrieve data based on variables and obviously this has to be done through the URL since there will be no direct GUI interaction with the API. (unless I'm mistaken, which I probably am) I have gone through the official documentation and followed the tutorial to the end and still don't understand how this is supposed to work.all I ask for is a quick and simple explanation because I have searched everywhere and haven't found a simple enough explanation to grasp the core concept of how this is all supposed to work.
I think what you're looking for is JSONResponse and related objects:
This will allow you to send JSON in response to a request.
from django.http import JsonResponse
def my_view_json(request):
response = JsonResponse({'foo': 'bar'})
return response
If your templates or webpages need to make a request to a view and specify different parameters, they can do so by adding POST variables (examples). These can be parsed in the view like so:
def myView(request):
my_post_var = request.POST.get('variable_name', 'default_value')
my_get_var = request.GET.get('variable_name', 'default_value')
You can then parse what was sent any way you like and decide what you want to do with it.
Basically,
You define the URLS upon which you perform Get/POST/PUT Requests and You can Send Data to that.
Eg:
urls.py
from django.conf.urls import url,include
from app import views
urlpatterns = [
url(r'^(?i)customertype/$',views.CustomerViewSet.as_view()),
url(r'^(?i)profile/$', views.Save_Customer_Profile.as_view()),
url(r'^(?i)customer_image/$', views.Save_Customer_Image.as_view()),
]
Now Whenever User would send a Request to:
example.com/profile ==> This would be received in the Save_Customer_Profile View based on the Method Type, Save_Customer_Profile is as follows:
class Save_Customer_Profile(APIView):
"""Saves and Updates User Profile!"""
def get(self, request, format=None):
return AllImports.Response({"Request":"Method Type is GET Request"})
def post(self, request, format=None):
return AllImports.Response({"Request":"Method Type is Post Request"})
def put(self,request, format=None):
return AllImports.Response({"Request":"Method Type is Put Request"})
I think the OP was referring to how to do GET/POST request programmatically. In that case, it is enough to do (values are dummy):
GET:
import requests
r = requests.get('http://localhost:8000/snippets/')
print(r.json())
print(r.status_code, r.reason)
POST:
data = {'code': 'print(" HELLO !!!")', 'language': 'java','owner': 'testuser'}
r = requests.post('http://localhost:8000/snippets/', data=data, auth=('testuser', 'test'))

Faking a Streaming Response in Django to Avoid Heroku Timeout

I have a Django app that uses django-wkhtmltopdf to generate PDFs on Heroku. Some of the responses exceed the 30 second timeout. Because this is a proof-of-concept running on the free tier, I'd prefer not to tear apart what I have to move to a worker/ poll process. My current view looks like this:
def dispatch(self, request, *args, **kwargs):
do_custom_stuff()
return super(MyViewClass, self).dispatch(request, *args, **kwargs)
Is there a way I can override the dispatch method of the view class to fake a streaming response like this or with the Empy Chunking approach mentioned here to send an empty response until the PDF is rendered? Sending an empty byte will restart the timeout process giving plenty of time to send the PDF.
I solved a similar problem using Celery, something like this.
def start_long_process_view(request, pk):
task = do_long_processing_stuff.delay()
return HttpResponse(f'{"task":"{task.id}"}')
Then you can have a second view that can check the task state.
from celery.result import AsyncResult
def check_long_process(request, task_id):
result = AsyncResult(task_id)
return HttpResponse(f'{"state":"{result.state}"')
Finally using javascript you can just fetch the status just after the task is being started. Updating every half second will more than enough to give your users a good feedback.
If you think Celery is to much, there are light alternatives that will work just great: https://djangopackages.org/grids/g/workers-queues-tasks/

Exception handling for missing parameters of a view passed on as json data

Lately, I found out the elementary truth related to exception handling - our server should rarely (preferably, never) through 5XX error.
Having said that, let's consider a REST view with some parameters, and a respective URL.
urlpatterns = [
url(r'^some_view/', some_view),
]
#api_view(['POST'])
def some_view(request):
# ...
param1 = request.data['param1']
In this code, I have to manually handle the exception where the developer calls some_view without assigning value to param1, otherwise - if I don't handle this case explicitly - they will get 500 error (MultipleKeyValueError) which is bad. And we have a dilemma:
Handling it will cause irritating and repetitive try-except blocks, especially when we have multiple params
Not handling will lead to 500 error
The solution is to rewrite the view this way:
urlpatterns = [
url(r'^some_view/(?P<param1>\w+/', some_view),
]
#api_view(['POST'])
def some_view(request, param1):
# ...
Here Django will throw 400 exception - as opposed to 500 - saying that url is not found. But on the other hand, the first option (where I use request.data['param1']) gives the pleasant benefit that I can call the REST resource not only from an outside app, but also from my web app submitting params by serializing HTML form.
So here I am asking about the best practice. How do you, guys, handle this situation? Do you explicitly write try-except blocks watching for missing params, or do you use the url-parameters options, or maybe there's some third option that I didn't mention here ?
If you're writing REST views, you should be using Django Rest Framework. One of the benefits of that framework is that you define serializers in which you declare which fields you accept and which are mandatory, just as you do with forms in vanilla Django. You can therefore return validation errors rather than 500s.
So here I am asking about the best practice. How do you, guys, handle
this situation? Do you explicitly write try-except blocks watching for
missing params, or do you use the url-parameters options, or maybe >there's some third option that I didn't mention here ?
If I am trying to handle a common pattern for exception handling across views, I typically us a “third option” of writing a decorator that handles the exception and returns a JSON response with success = False:
from functools import wraps
def handle_missing_key(func)
#wraps(func)
def _decorator(*args, **kwargs):
try:
func(*args, **kwargs)
except KeyError as ex:
return JsonResponse({
'success': False,
'error': '%s' % ex
})
return _decorator
Then for your methods you can just do:
#handle_missing_key
#api_view(['POST'])
def some_view(request, param1):
pass
This way you're just sending back a 200 response but the JSON defines the error message and lets any calling application know what is missing. You can find the missing key using KeyError.args or the error message in the JsonResponse. If you want to send back a response other than a JsonResponse (i.e., status response 400 or the like), you can, of course, use something like this:
response = HttpResponse(status=400)
response.reason_phrase = 'Key %s is missing' % ex
return response

Django how to test if message is sent

I want to test if message is sent to user after submit. I'm using django.contrib.messages. Everything seems to be working during manual testing (runserver), but in unit test I don't get messages.
Code that stores message:
messages.success(request, _('Internationalized evil message.'))
Code that should test message:
from django.contrib.messages.api import get_messages
...
def test_message_should_sent_to_user(self):
"""After successful phone number submit, message should be displayed."""
response = self.client.post(
reverse('evil.views.evil_data_submit'), self.valid_data)
messages = get_messages(response.request)
self.assertNotEqual(len(messages), 0)
It looks like that no middleware is called during test client post method call.
Update after #Tisho answer
Messages should be found in response.context, even my guts say that it should work, but it doesn't. I've placed import pdb; pdb.set_trace() in django/contrib/messages/context_processors.py to see if its called during test client.post, its not.
I've double checked TEMPLATE_CONTEXT_PROCESSORS, MIDDLEWARE_CLASSES and INSTALLED_APPS - probably tomorrow I'll discover that I missed something.
Important detail
Forgot to mention that in case of successful submit view returns HttpResponseRedirect therefore response.context is empty.
Solution
View returns redirect (which has no context data), to solve that we can pass follow=True to client.post method (method suggested by #Tisho).
During unit tests, the message could be found in
response = self.client.post(
reverse('evil.views.evil_data_submit'), self.valid_data)
messages = response.context['messages']
If your view returns a redirect, response.context will be empty unless you pass follow=True, like so:
response = self.client.post(
reverse('evil.views.evil_data_submit'),
self.valid_data,
follow=True)
The best way I found is by using mock
http://www.voidspace.org.uk/python/mock/patch.html#patch-methods-start-and-stop
class SimpleCommentTestDirect(TestCase):
def setUp(self):
self._patcher1 = patch('django.contrib.messages.error')
self.mock_error = self._patcher1.start()
def tearDown(self):
self._patcher1.stop()
def test_error_message(self):
self.client.get('/vote/direct/unknownapp/comment/1/up/')
self.assertEqual(self.mock_error.call_args[0][1], 'Wrong request. Model.')
BTW: There are also should be a way to get request by using mock. And by using request object get message from django.contrib.messages.get_messages