How to properly handle an aborted HTTP request in Django - django

Whenever a lengthy HTTP requests is aborted by the client (e.g. Browser is closed) Django views seem to raise a IOError exception.
What's the proper way to detect such an aborted request if I just want to ignore them? Just catching IOError seems too wide.. might accidentally ignore other IO problems.

In django 1.3 and up, you can use a logging filter class to suppress the exceptions which you aren't interested in. Here's the logging filter class I'm using to narrowly suppress IOError exceptions raised from _get_raw_post_data():
import sys, traceback
class _SuppressUnreadablePost(object):
def filter(self, record):
_, exception, tb = sys.exc_info()
if isinstance(exception, IOError):
for _, _, function, _ in traceback.extract_tb(tb):
if function == '_get_raw_post_data':
return False
return True
In Django 1.4, you will be able to do away with most of the complexity and suppress the new exception class UnreadablePostError. (See this patch).

The best way to do it would be to use a custom middleware class that implements process_exception() to return a custom HTTP response, say a rendered errors/request_aborted.html template, if an IOException is caught.

Raven now connects itself to the got_request_exception() signal to catch unhandled exceptions, bypassing the logging system entirely, so the solution proposed by dlowe does not work anymore.
However raven looks for a skip_sentry attribute on the exception instance, so you can use a middleware to set it on the errors you want to ignore:
import sys
import traceback
class FilterPostErrorsMiddleware(object):
"""
A middleware that prevents unreadable POST errors to reach Sentry.
"""
def process_exception(self, request, exception):
if isinstance(exception, IOError):
tb = sys.exc_info()[2]
for _, _, function, _ in traceback.extract_tb(tb):
if function == '_get_raw_post_data':
exception.skip_sentry = True
break
Note: you have to use a recent version of raven (e.g. 1.8.4), as previous versions mistakenly checked for the skip_sentry attribute on the exception type rather than instance.

If you want to ignore the IOError, then just let it be. You don't need to catch it. If you absolutely must catch it, you can do what #Filip Dupanović suggested, and maybe return a django.http.HttpResponseServerError to set the response code to 500.

Related

Django to catch all invalid logins and unauthorized access like 401s and 403s

I want to capture all invalid logins/unauthorized access such as 401s and 403s returned from the site so I can log them to a security logging service, investigating if there is an easy way to catch all of these without putting in much custom logic.
I have tried using middleware approach:
def simple_middleware(get_response):
# One-time configuration and initialization.
def middleware(request):
response = get_response(request)
if response.status_code in [403, 401]:
log.warning('invalid login')
return response
return middleware
Unfortunately an incorrect login to the /admin/ login, it returns status 200, however I think this would work for custom login that explicitly throws 401/403.
I have also tried using the signal approach using request_finished but all I get is just the handler class.
So... looking for ideas.
As you found out, a login attempt doesn't necessarily imply a specific response code, since you may decide to treat the attempt with a redirect or any other type of answer.
In case of Django, the default auth middleware (which I assume you are using) fires a user_login_failed signal which you can handle with your logging logic.
You can see in the documentation how to register a signal handler, so it should be something like
from django.contrib.auth.signals import user_login_failed
from django.dispatch import receiver
#receiver(request_finished)
def handle_login_failed(sender, **kwargs):
print(f"Oops, login failed using these credentials: {kwargs.get('credentials', None)}")
The signal triggering is in the source code for the auth package.

Global Exception Handling in Django-rest-framework

Is there a way to handle all the exceptions globally without using try-except block in django rest framework.
I want to convert html error page that django is throwing to a customised json object response.
I have created an exception.py file in my app
def custom_exception_handler(exc, context=None):
response = exception_handler(exc)
if isinstance(exc, HttpResponseServerError):
custom_response_data = {
'detail': 'Internal Server Error' # custom exception message
}
response.data = custom_response_data
return response
i have configured this in settings.py.
REST_FRAMEWORK = {
'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
'PAGE_SIZE': 10,
'EXCEPTION_HANDLER':'my_project.my_app.exceptions.custom_exception_handler'}
Since I came across with a similar situation that lead me to this question, I'll answer following the original question that is related to Django Rest Framework specifically and not just Django.
I understand that you want to handle raised exceptions from your views, globally, without having to define try/except blocks on each view module.
DRF allows you to define your own Custom Exception Handling mechanism (docs).
Here is an example:
At my_custom_except_handler.py:
import logging
from rest_framework.views import exception_handler
from django.http import JsonResponse
from requests import ConnectionError
def custom_exception_handler(exc, context):
# Call REST framework's default exception handler first
response = exception_handler(exc, context)
# checks if the raised exception is of the type you want to handle
if isinstance(exc, ConnectionError):
# defines custom response data
err_data = {'MSG_HEADER': 'some custom error messaging'}
# logs detail data from the exception being handled
logging.error(f"Original error detail and callstack: {exc}")
# returns a JsonResponse
return JsonResponse(err_data, safe=False, status=503)
# returns response as handled normally by the framework
return response
As stated in the docs, the defined response object refers to:
The exception handler function should either return a Response object, or return None if the exception cannot be handled. If the handler returns None then the exception will be re-raised and Django will return a standard HTTP 500 'server error' response.
In other words, 'response' won't be None only when handling these exceptions docs:
Subclasses of APIException.
Django's Http404 exception.
Django's PermissionDenied exception.
If your custom handler returns None, then the exception will be handled 'normally' by the framework, returning typical 500 server error.
Finally remember to set the required key at settings.py:
REST_FRAMEWORK = {'EXCEPTION_HANDLER':
'my_project.my_app.my_custom_except_handler.custom_exception_handler'}
Hope it helps!
The definite answer to your question is no.
At least I don't know how to do it globally in Django, whereas global includes middleware exceptions).
Further, as request by #Shubham Kumar, the hook you need is process_exception and for an implementation check this SO post with the offical docs on how to activate it.
As stated in the Django docs:
request is an HttpRequest object. exception is an Exception object raised by the view function.
Django calls process_exception() when a view raises an exception. process_exception() should return either None or an HttpResponse object. If it returns an HttpResponse object, the template response and response middleware will be applied and the resulting response returned to the browser. Otherwise, default exception handling kicks in.
Again, middleware are run in reverse order during the response phase, which includes process_exception. If an exception middleware returns a response, the process_exception methods of the middleware classes above that middleware won’t be called at all.
Meaning that you will merely be able to hook into the view function and catch all those exceptions.

Exception handling for missing parameters of a view passed on as json data

Lately, I found out the elementary truth related to exception handling - our server should rarely (preferably, never) through 5XX error.
Having said that, let's consider a REST view with some parameters, and a respective URL.
urlpatterns = [
url(r'^some_view/', some_view),
]
#api_view(['POST'])
def some_view(request):
# ...
param1 = request.data['param1']
In this code, I have to manually handle the exception where the developer calls some_view without assigning value to param1, otherwise - if I don't handle this case explicitly - they will get 500 error (MultipleKeyValueError) which is bad. And we have a dilemma:
Handling it will cause irritating and repetitive try-except blocks, especially when we have multiple params
Not handling will lead to 500 error
The solution is to rewrite the view this way:
urlpatterns = [
url(r'^some_view/(?P<param1>\w+/', some_view),
]
#api_view(['POST'])
def some_view(request, param1):
# ...
Here Django will throw 400 exception - as opposed to 500 - saying that url is not found. But on the other hand, the first option (where I use request.data['param1']) gives the pleasant benefit that I can call the REST resource not only from an outside app, but also from my web app submitting params by serializing HTML form.
So here I am asking about the best practice. How do you, guys, handle this situation? Do you explicitly write try-except blocks watching for missing params, or do you use the url-parameters options, or maybe there's some third option that I didn't mention here ?
If you're writing REST views, you should be using Django Rest Framework. One of the benefits of that framework is that you define serializers in which you declare which fields you accept and which are mandatory, just as you do with forms in vanilla Django. You can therefore return validation errors rather than 500s.
So here I am asking about the best practice. How do you, guys, handle
this situation? Do you explicitly write try-except blocks watching for
missing params, or do you use the url-parameters options, or maybe >there's some third option that I didn't mention here ?
If I am trying to handle a common pattern for exception handling across views, I typically us a “third option” of writing a decorator that handles the exception and returns a JSON response with success = False:
from functools import wraps
def handle_missing_key(func)
#wraps(func)
def _decorator(*args, **kwargs):
try:
func(*args, **kwargs)
except KeyError as ex:
return JsonResponse({
'success': False,
'error': '%s' % ex
})
return _decorator
Then for your methods you can just do:
#handle_missing_key
#api_view(['POST'])
def some_view(request, param1):
pass
This way you're just sending back a 200 response but the JSON defines the error message and lets any calling application know what is missing. You can find the missing key using KeyError.args or the error message in the JsonResponse. If you want to send back a response other than a JsonResponse (i.e., status response 400 or the like), you can, of course, use something like this:
response = HttpResponse(status=400)
response.reason_phrase = 'Key %s is missing' % ex
return response

How to send django exception log manually?

I have a specific view in my app which should return HttpResponse('') if everything was done successfully and smth like HttpResponseBadRequest() otherwise.
This view works with external data so some unexpected exceptions can be raised.
Of course I need to know what happened. So I have smth like that:
def my_view(request):
try:
# a lot of code
return HttpResponse('')
except:
import logging
logger = logging.getLogger('django.request')
# logger.error(extra={'request': request}) or logger.exception()
return HttpResponseBadRequest('')
I have a default logging config if it matters (django 1.5).
logger.exception sends only a small traceback and no request data.
logger.error sends only request data.
So the question is how to get exactly the same traceback as django sends (with the same subject/body)).
I think there must be some clean and simple solution that i just can't find.
UPDATE
So with the patched exception method i ended up with the following code:
logger.exception('Internal Server Error: %s', request.path,
extra={'status_code': 500, 'request': request})
which produces equal email traceback as built-in django logging.
This problem is due to a bug which has been fixed in recent versions of Python 2.7. This bug meant that the exception method did not accept the extra argument.
If you can't upgrade to a suitably recent Python 2.7, then you can perhaps patch your Python with the fix referenced in the issue I've linked to. If you can't do that, you'll need to subclass Logger and override the exception method to do the right thing (which is basically to add a **kwargs to the exception method's signature, which is passed through to its call to the error method.

How can I handle Exceptions raised by dango-social-auth?

In django-social-auth, there are a few instances where a back-end will raise a ValueError (such as when a user cancels a login request or if a user tries to associate with an account that's already been associated with another User). If a User runs into one of these scenarios, they'll be presented with a 500 error on your site.
So, what's the best way to catch these? I'd prefer to be able to display a useful message (via the messages framework) when this happens, but I'm at a loss as to the best way to do this.
I'm thinking about writing my own view (in a separate app) that just wraps social_auth's associate_complete view, but this seems clunky... Any ideas?
I could fork django-social-auth and customize this behavior, but I'd prefer not to maintain a separate fork - especially since I can't assume everone would want to handle these Exceptions in the same manner.
Rather old question but worth mention that recent version of DSA supports a custom exception processor where you can do whatever you want with the exception message. The default version stores them in the messages app.
Also the exceptions are differentiated now instead of the not-useful ValueError used. Check the docs http://django-social-auth.readthedocs.org/en/latest/configuration.html.
Update (13/08/2013):
Since I've posted the above things have changed, now DSA has an exception middleware that when enabled stores the exception message in the jango builtin messages app. It's preferable to subclass the middleware and add the custom behavior to it. Check the doc at http://django-social-auth.readthedocs.org/en/latest/configuration.html#exceptions-middleware
Sample middleware:
# -*- coding: utf-8 -*-
from social_auth.middleware import SocialAuthExceptionMiddleware
from social_auth.exceptions import AuthFailed
from django.contrib import messages
class CustomSocialAuthExceptionMiddleware( SocialAuthExceptionMiddleware):
def get_message(self, request, exception):
msg = None
if (isinstance(exception, AuthFailed) and
exception.message == u"User not allowed"):
msg = u"Not in whitelist"
else:
msg = u"Some other problem"
messages.add_message(request, messages.ERROR, msg)
I've ecountered the same problem and it seems, that creating wrapper view is the best way to handle this situation, at this point, atleast. Here is how I had mine done:
def social_auth_login(request, backend):
"""
This view is a wrapper to social_auths auth
It is required, because social_auth just throws ValueError and gets user to 500 error
after every unexpected action. This view handles exceptions in human friendly way.
See https://convore.com/django-social-auth/best-way-to-handle-exceptions/
"""
from social_auth.views import auth
try:
# if everything is ok, then original view gets returned, no problem
return auth(request, backend)
except ValueError, error:
# in case of errors, let's show a special page that will explain what happened
return render_to_response('users/login_error.html',
locals(),
context_instance=RequestContext(request))
You will have to setup url for it:
urlpatterns = patterns('',
# ...
url(r'^social_auth_login/([a-z]+)$', social_auth_login, name='users-social-auth-login'),
)
And then use it as before in template:
Log in with Google
Hope this helps, even aftern two months after question was asked :)
You need add social auth middleware:
MIDDLEWARE_CLASSES += ('social_auth.middleware.SocialAuthExceptionMiddleware',)
If any error occurs user will be redirected to erorr url(LOGIN_ERROR_URL from settings).
For detailed explanation please see documentation:
http://django-social-auth.readthedocs.org/en/latest/configuration.html#exceptions-middleware
In my app's views.py:
from social_auth.views import associate_complete
def associate_complete_wrapper(request, backend):
try:
return associate_complete(request, backend)
except ValueError, e:
if e and len(e.args) > 0:
messages.error(request, "Failed to Associate: %s" % e.args[0])
return redirect(reverse('pieprofile-edit-account'))
Then in the Root URLconf (notice the order of these url patterns):
url(r'^associate/complete/(?P<backend>[^/]+)/$', 'myapp.views.associate_complete_wrapper'),
url(r'', include('social_auth.urls')),
My associate_complete_wrapper url essentially hijacks social_auth's socialauth_associate_complete url.