Currently my app is working with several external APIs, and those functions can raise some kind of error, and I would like to be aware of all kind of situations and don't crash my app. In the following piece of code you can see my function, using slack api (it's just an example, not real example), and you can see how I'm creating a space giving the name and the emails. That function can returns several errors, like APIerror, TypeError... etc etc
Also you can see, all the functions in that file, can raise some errors so it is using try and except and try to don't break our app. In the except part, I don't want to do anything special, just return the error or I don't know, raise an custom error. In the future would be cool if I can enqueue that task if something goes wrong.
slack_client.py
def create_space():
try:
slack_api.create.space(spaceName='space name', email=emails)
except (...):
...
def delete_space():
try:
slack_api.delete.space(spaceId=space_id)
except (...):
...
I would like to refactor the try and except part, because I don't want to copy and paste try and except during all my slack_client class, but in case the connection is down, or some parameter is wron, the "Slack" API will raise me an error.
I thought about create a class, to run each function inside try and catch, but it's a bit weird and I will loose the arguments part like "spaceName='space name'", which I think it's really cool and clear.
def execute_func(func, *args):
try:
func(args)
except (..):
...
So would like some help for this kind of situation, because handle external API is not always really easy... (at the moment working with 2 external APIs)
In a previous life I used VAX/VMS and still remember great pleasure at discoverting LIB$SIG_TO_RET which converted a signal to a return value. You can do exactly the same in not many lines of Python, along the lines you indicated
def execute_func(func, *args, **kwargs):
try:
result = func( *args, **kwargs)
return (True, result)
except Exception as exc:
return ( False, exc)
Usage
result = execute_func( func, a1, a2, ... k1=v1, k2=v2, ...)
if result[0]:
func_result = result[1] # if it's useful
...
else:
# it went wrong, deal with the exception
caught_exception = result[1]
print( caught_exception)
You can catch any exception that is thrown and capture it in a variable to log or raise. You could even raise your own custom exception. Something like this should work:
First create your exception:
class CustomSlackException(Exception):
pass
Then raise it when the API you called throws some exception:
try:
slack_api.create.space(spaceName='space name', email=emails)
except Exception as e:
print(e)
raise CustomSlackException("Custom error message!")
Now you can catch your exception and handle it as needed.
Related
I'm trying to utilize django's row-level-locking by using the select_for_update utility. As per the documentation, this can only be used when inside of a transaction.atomic block. The side-effect of using a transaction.atomic block is that if my code throws an exception, all the database changes get rolled-back. My use case is such that I'd actually like to keep the database changes, and allow the exception to propagate. This leaves me with code looking like this:
with transaction.atomic():
user = User.objects.select_for_update.get(id=1234)
try:
user.do_something()
except Exception as e:
exception = e
else:
exception = None
if exception is not None:
raise exception
This feels like a total anti-pattern and I'm sure I must be missing something. I'm aware I could probably roll-my-own solution by manually using transaction.set_autocommit to manage the transaction, but I'd have thought that there would be a simpler way to get this functionality. Is there a built in way to achieve what I want?
I ended up going with something that looks like this:
from django.db import transaction
class ErrorTolerantTransaction(transaction.Atomic):
def __exit__(self, exc_type, exc_value, traceback):
return super().__exit__(None, None, None)
def error_tolerant_transaction(using=None, savepoint=True):
"""
Wraps a code block in an 'error tolerant' transaction block to allow the use of
select_for_update but without the effect of automatic rollback on exception.
Can be invoked as either a decorator or context manager.
"""
if callable(using):
return ErrorTolerantTransaction('default', savepoint)(using)
return ErrorTolerantTransaction(using, savepoint)
I can now put an error_tolerant_transaction in place of transaction.atomic and exceptions can be raised without a forced rollback. Of course database-related exceptions (i.e. IntegrityError) will still cause a rollback, but that's expected behavior given that we're using a transaction. As a bonus, this solution is compatible with transaction.atomic, meaning it can be nested inside an atomic block and vice-versa.
I was wondering if you could re-raise a (specific) caught exception and have it caught by a later (general) except in the same try-except. As an example, I want to do something with a specific IOError, but if its not the expected IOError then the exception should be handled like any other error. What I initially tried:
try:
raise IOError()
except IOError as ioerr:
if ioerr.errno == errno.ENOENT:
# do something with the expected err
else:
# continue with the try-except - should be handled like any other error
raise
except Exception as ex:
# general error handling code
However, this doesn't work: the raise re-raises the exception outside the context of the try-except.
What would be the pythonic way of writing this to get the desired exception 'fall-through' behaviour?
(I'm aware there was a proposed 'conditional except' that wasn't implemented, which could've solved this)
If you ultimately want it to catch everything, make it do that. Catch first, sieve later. ;)
try:
raise IOError()
except Exception as ex:
if isinstance(ex, IOError) and ex.errno == errno.ENOENT:
# do something with the expected err
# do the rest
I am not an expert in writing pythonically, but I think one obvious approach (if you know that you're expecting one particular kind of exception), would be to use nested exception handling:
try:
try:
raise IOError()
except IOError as ioerr:
if ioerr.errno == errno.ENOENT:
# do something with the expected err
else:
# pass this on to higher up exception handling
raise
except Exception as ex:
# general error handling code
I know in your comment that you didn't want nested else's -- I don't know if nested exception handling is as bad in your book, but at the very least you can avoid code duplication.
So, I'm working on the same here, and after reviewing the available solutions, I'm going to go with catching the parent exception, and then testing for specifics. In my case I'm working with the dns module.
try:
answer = self._resolver.query(name, 'NS')
except dns.exception.DNSException, e: #Superclass of exceptions tested for
if isinstance(e, dns.resolver.NXDOMAIN):
#Do Stuff
elif isinstance(e, dns.resolver.NoAnswer):
# Do other stuff
else:
# Do default stuff
When I want to select objects with a get() function like
personalProfile = World.objects.get(ID=personID)
If get function doesn't return find a value, a "matching query does not exist." error occurs.
If I don't need this error, I'll use try and except function
try:
personalProfile = World.objects.get(ID=personID)
except:
pass
But I think this is not the best way since I use
except:
pass
Please recommend some idea or code sample to fight with this issue
That depends on what you want to do if it doesn't exist..
Theres get_object_or_404:
Calls get() on a given model manager, but it raises Http404 instead of the model’s DoesNotExist exception.
get_object_or_404(World, ID=personID)
Which is very close to the try except code you currently do.
Otherwise theres get_or_create:
personalProfile, created = World.objects.get_or_create(ID=personID)
Although, If you choose to continue with your current approach, at least make sure the except is localised to the correct error and then do something with that as necessary
try:
personalProfile = World.objects.get(ID=personID)
except MyModel.DoesNotExist:
raise Http404("No MyModel matches the given query.")
The above try/except handle is similar to what is found in the docs for get_object_or_404...
A get_or_none() function has been proposed, multiple times now. The rejection notice is feature creep, which you might or might not agree with. The functionality is present --with slightly different semantics-- in the first() queryset method.
But first things first:
The manager throws World.DoesNotExist, a specialized subclass of ObjectDoesNotExist when a World object was not found:
try:
personalProfile = World.objects.get(ID=personID)
except World.DoesNotExist:
pass
There's also get_object_or_404() which raises a Http404 exception when the object was not found.
You can also roll your own get_or_none(). A possible implementation could be:
def get_or_none(queryset, *args, **kwargs):
try:
return queryset.get(*args, **kwargs)
except ObjectDoesNotExist:
return None
Note that this still raises MultipleObjectsReturned when more than one matching object is found. If you always want the first object regardless of any others, you can simplify using first(), which returns None when the queryset is empty:
def get_or_none(queryset, *args, **kwargs):
return queryset.filter(*args, **kwargs).first()
Note however, for this to work reliably, you need a proper order for objects, because in the presence of multiple objects first() might be non-deterministic (it probably returns the first object from the database index used to filter the query and neither indexes not the underlying tables need be sorted or even have a repeatable order).
Use both, however, only when the use of the object to retrieve is strictly optional for the further program flow. When failure to retrieve an object is an error, use get_object_or_404(). When an object should be created when it does not exist, use get_or_create(). In those cases, both are better suited to simplify program flow.
As alasdair mentioned you could use the built in first() method.
It returns the object if it exists or None if it's not
personalProfile = World.objects.filter(ID=personID).first()
I am basically trying to improve upon Python: try-except as an Expression? with a couple of features
Ability to pass in extra args and kwargs to the "try-except" question. This is because I have a success function which takes in a few parameters
Failure function if callable should be called with the exception argument so it gets a chance to handle it.
Here is a sample code with a test however I am not able to get the last line to work
def method2(exc_class = None):
if exc_class:
raise exc_class()
def method1():
return "Hello world"
def try_except(function, failure, exceptions = [], args = [], kwargs = {}):
"""
Run the given function with args and kwargs. If it throws one of the
exceptions in the list then either return failure or call failure function
"""
try:
return function(*args, **kwargs)
except exceptions or Exception as e:
return failure(e) if callable(failure) else failure
if __name__ == "__main__":
#Prints hello world
print try_except(method1, "Failure")
#Prints Failure great!!
print try_except(method2, "Failure", kwargs = {"exc_class" : ValueError})
# I expect below line to print "Failure" properly but it throws a ValueError
print try_except(method2, "Failure", kwargs = {"exc_class" : ValueError}, exceptions=[ValueError])
My question here is the except exceptions or Exception as e: line does not substitute the list of exceptions properly. I cannot do *exceptions as shown in the original question because I want to take in extra parameters for the function.
I am okay to change the try_except to some extent though.
Note: I have considered doing a catch all except Exception as e and then checking if the exception class in in the list and if it is not then rethrow it. However that won't work because when I rethrow the original stack trace is lost and I do not want to do that.
When catching several exception types with one except clause, you can't use just any iterable object; you must specifically use a tuple. You can still allow try_except to take any iterable as an argument by creating a tuple in the except clause itself.
try:
return function(*args, **kwargs)
except tuple(exceptions or (Exception,)) as e:
return failure(e) if callable(failure) else failure
From the docs:
For an except clause with an expression, that expression is evaluated, and the clause matches the exception if the resulting object is “compatible” with the exception. An object is compatible with an exception if it is the class or a base class of the exception object, or a tuple containing an item compatible with the exception.
Is it necessary to catch errors stemming from HTTP inputs? Is it ever a good idea to let it fail naturally (allow the exception to bubble up)?
I have a Django view for a server side interface for an AJAX call that looks something like this:
def some_view(request):
try:
some_int = int(request.POST.get('some_int')) # May raise ValueError or TypeError
except (ValueError, TypeError):
return HttpResponseBadRequest('some_int must be an int')
# ... Code that assumes some_int is an int
return HttpResponse('The normal response')
Is it ever acceptable in production code to have something like this?
def some_view(request):
some_int = int(request.POST.get('some_int')) # Ignore ValueError or TypeError raised
# ... Code that assumes some_int is an int
return HttpResponse('normal_response')
As I accept more parameters, I find that it is frustrating to maintain so many try/except blocks which are mostly the same and I end up with a ton of boiler plate code.
Of course I tried to refactor this into a separate function but since Django requires an HttpResponse to be returned, not raised as an exception, I can't plug it into a view without a try/except block. Also, conversions to int aren't the only thing I check... there are a lot of business logic sanity checks performed depending on the input as well. For example, I would validate that the JSON passed is of a specific format (i.e. array of objects of int array, etc.).
My views end up being 70+ lines of code just for sanity checks and a few lines of code that actually generate the response. Somehow I feel like there should be a more elegant way but I haven't found one so I'm considering forgoing all checks and just letting Django take care of it. Is this a bad idea?
I'm aware of the following potential problems if I don't catch the exceptions:
The same HTTP 500 is returned for all errors
If logging is enabled in production, it would probably log an error every time an invalid input occurs
Are there other problems I should be aware of? It just feels wrong not to catch exceptions from user inputs even though there's not much I can do about it in terms of recovery logic.
I think the best way to handle this is by writing your own middleware that catches the exceptions and turns them into your desired response.
That might look something like this:
# views.py
def some_view(request):
some_int = int(request.POST.get('some_int'))
# ... Code that assumes some_int is an int
return HttpResponse('normal_response')
# middleware.py
class MyValidationMiddleware(object):
def process_exception(self, request, e):
if isinstance(e, ValueError):
return HttpResponseBadRequest('Input did not validate')
else:
return None # let it bubble up
(Since middleware is site-wide, you might want to explicitly define your own input-validation exceptions to distinguish them from other sorts of exceptions that might occur.)
Alternatively, you could do the same sort of thing with per-view decorators.