How could one handle exceptions globally with Flask? I have found ways to use the following to handle custom db interactions:
try:
sess.add(cat2)
sess.commit()
except sqlalchemy.exc.IntegrityError, exc:
reason = exc.message
if reason.endswith('is not unique'):
print "%s already exists" % exc.params[0]
sess.rollback()
The problem with try-except is I would have to run that on every aspect of my code. I can find better ways to do that for custom code. My question is directed more towards global catching and handling for:
apimanager.create_api(
Model,
collection_name="models",
**base_writable_api_settings
)
I have found that this apimanager accepts validation_exceptions: [ValidationError] but I have found no examples of this being used.
I still would like a higher tier of handling that effects all db interactions with a simple concept of "If this error: show this, If another error: show something else" that just runs on all interactions/exceptions automatically without me including it on every apimanager (putting it in my base_writable_api_settings is fine I guess). (IntegrityError, NameError, DataError, DatabaseError, etc)
I tend to set up an error handler on the app that formats the exception into a json response. Then you can create custom exceptions like UnauthorizedException...
class Unauthorized(Exception):
status_code = 401
#app.errorhandler(Exception)
def _(error):
trace = traceback.format_exc()
status_code = getattr(error, 'status_code', 400)
response_dict = dict(getattr(error, 'payload', None) or ())
response_dict['message'] = getattr(error, 'message', None)
response_dict['traceback'] = trace
response = jsonify(response_dict)
response.status_code = status_code
traceback.print_exc(file=sys.stdout)
return response
You can also handle specific exceptions using this pattern...
#app.errorhandler(ValidationError)
def handle_validation_error(error):
# Do something...
Error handlers get attached to the app, not the apimanager. You probably have something like
app = Flask()
apimanager = ApiManager(app)
...
Put this somewhere using that app object.
My preferred approach uses decorated view-functions.
You could define a decorator like the following:
def handle_exceptions(func):
#wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except ValidationError:
# do something
except HTTPException:
# do something else ...
except MyCustomException:
# do a third thing
Then you can simply decorate your view-functions, e.g.
#app.route('/')
#handle_exceptions
def index():
# ...
I unfortunately do not know about the hooks Flask-Restless offers for passing view-functions.
Related
I have a custom prediction routine in google-ml-engine. Works very well.
I now am doing input checking on the instance data, and want to return error responses from my predict routine.
The example: https://cloud.google.com/ai-platform/prediction/docs/custom-prediction-routines
Raises exceptions on input errors, etc. However, when this happens the response body always has {'error': Prediction failed: unknown error}. I can see the correct errors are being logged in google cloud console, but the https response is always the same unknown error.
My question is:
How to make the Custom prediction routine return a proper error code and error message string?
Instead of returning a prediction, I can return an error string/code in prediction -but it ends up in the prediction part of the response which seems hacky and doesn't get any of the google errors eg based on instance size.
root:test_deployment.py:35 {'predictions': {'error': "('Instance does not include required sensors', 'occurred at index 0')"}}
What's the best way to do this?
Thanks!
David
Please take a look at the following code, I created a _validate function inside predict and use a custom Exception class.
Basically, I validate instances, before I call the model predict method and handle the exception.
There may be some overhead to the response time when doing this validation, which you need to test for your use case.
requests = [
"god this episode sucks",
"meh, I kinda like it",
"what were the writer thinking, omg!",
"omg! what a twist, who would'v though :o!",
99999
]
api = discovery.build('ml', 'v1')
parent = 'projects/{}/models/{}/versions/{}'.format(PROJECT, MODEL_NAME, VERSION_NAME)
parent = 'projects/{}/models/{}'.format(PROJECT, MODEL_NAME)
response = api.projects().predict(body=request_data, name=parent).execute()
{'predictions': [{'Error code': 1, 'Message': 'Invalid instance type'}]}
Custom Prediction class:
import os
import pickle
import numpy as np
import logging
from datetime import date
import tensorflow.keras as keras
class CustomModelPredictionError(Exception):
def __init__(self, code, message='Error found'):
self.code = code
self.message = message # you could add more args
def __str__(self):
return str(self.message)
def isstr(s):
return isinstance(s, str) or isinstance(s, bytes)
def _validate(instances):
for instance in instances:
if not isstr(instance):
raise CustomModelPredictionError(1, 'Invalid instance type')
return instances
class CustomModelPrediction(object):
def __init__(self, model, processor):
self._model = model
self._processor = processor
def _postprocess(self, predictions):
labels = ['negative', 'positive']
return [
{
"label":labels[int(np.round(prediction))],
"score":float(np.round(prediction, 4))
} for prediction in predictions]
def predict(self, instances, **kwargs):
try:
instances = _validate(instances)
except CustomModelPredictionError as c:
return [{"Error code": c.code, "Message": c.message}]
else:
preprocessed_data = self._processor.transform(instances)
predictions = self._model.predict(preprocessed_data)
labels = self._postprocess(predictions)
return labels
#classmethod
def from_path(cls, model_dir):
model = keras.models.load_model(
os.path.join(model_dir,'keras_saved_model.h5'))
with open(os.path.join(model_dir, 'processor_state.pkl'), 'rb') as f:
processor = pickle.load(f)
return cls(model, processor)
Complete code in this notebook.
If it is still relevant to you, I found a way by using google internal libraries (not sure if it would be endorsed by Google though).
AI platform custom prediction wrapping code only returns custom error message if the Exception thrown is a specific one from their internal library.
It might also not be super reliable as you would have very little control in case Google wants to change it.
class Predictor(object):
def predict(self, instances, **kwargs):
# Your prediction code here
# This is an internal google library, it should be available at prediction time.
from google.cloud.ml.prediction import prediction_utils
raise prediction_utils.PredictionError(0, "Custom error message goes here")
#classmethod
def from_path(cls, model_dir):
# Your logic to load the model here
You would get the following message in your HTTP response
Prediction failed: Custom error message goes here
i need a little bit of help understanding a problem that i have with user defined exceptions in python 2.7.11.
I have two files main.py and myErrors.py .in main i post data and receive a response and and in myErrors i handle the errors.
What i'm trying to do is execute the version error in the try:except statement, but it doesn't get executed even thought it should be. what i'm doing is that i pass the response to myErrors and update that data to a dictionary in the errors file.-
my question was badly phrased. What I want to do is, is pass the response to the error handler, but i don't want to execute it, until we get to the Try:except clause in on_response method. So when we get the response and if it's not successful, then check the error code and raise the exception. Now what i'm doing is checking first for errors and then executing the check for success (error code)
Here is the main
def send_messages(self):
response = cm.postData(url=simulateSasServer, jsondata=json_data)
self.on_response(response)
def on_response(self, response):
myERRORS.myERRORS(response)
# if registration is succesful change state to REGISTERED.
if 'registrationResponse' in response:
try:
responseObjects = response['registrationResponse']
for responseObject in responseObjects:
if responseObject['error']['errorCode'] == 0:
do_action
except myErrors.Version():
raise ("version_message")
Here is the myErrors
class myERRORS(Exception):
error_code = {'SUCCESS': 0,
'VERSION': 100,
}
response_data = {}
def __init__(self, response):
self.response_data.update(response)
class Version(myERRORS):
def __init__(self):
self.name = "VERSION"
self.err_code = self.error_code['VERSION']
self.msg = "SAS protocol version used by CBSD is not supported by SAS"
self.version_error()
if self.version_error() is True:
print (self.name, self.err_code, self.msg)
raise Exception(self.name, self.err_code, self.msg)
def version_error(self):
response_objects = self.response_data.values()[0]
if 'registrationResponse' in self.response_data:
for r_object in response_objects:
if r_object['error']['errorCode'] == self.error_code['VERSION']:
return True
Any help is much appreciated.
There isn't really anything special about exceptions. They are classes. What you did is create an instance of a class. You did not raise it. Change:
myERRORS.myERRORS(response)
to:
raise myERRORS.myERRORS(response)
I am calling tastypie api from normal django views.
def test(request):
view = resolve("/api/v1/albumimage/like/user/%d/" % 2 )
accept = request.META.get("HTTP_ACCEPT")
accept += ",application/json"
request.META["HTTP_ACCEPT"] = accept
res = view.func(request, **view.kwargs)
return HttpResponse(res._container)
Using tastypie resource in view
Call an API on my server from another view
achieve the same thing but seems harder.
Is my way of calling api acceptable?
Besides, it would be awesome if I could get the result in python dictionary instead of json.
Is it possible?
If you need a dictionary, it means that you must design your application better. Don't do important stuff in your views, nor in the Tastypie methods. Refactor it to have common funcionality.
As a general rule, views must be small. No more than 15 lines. That makes the code readable, reusable and easy to test.
I'll provide an example to make it clearer, suppose in that Tastypie method you must be creating a Like object, maybe sending a signal:
class AlbumImageResource(ModelResource):
def like_method(self, request, **kwargs):
# Do some method checking
Like.objects.create(
user=request.user,
object=request.data.get("object")
)
signals.liked_object(request.user, request.data.get("object"))
# Something more
But, if you need to reuse that behavior in a view, the proper thing would be to factorize that in a different function:
# myapp.utils
def like_object(user, object):
like = Like.objects.create(
user=request.user,
object=request.data.get("object")
)
signals.liked_object(request.user, request.data.get("object"))
return like
Now you can call it from your API method and your view:
class AlbumImageResource(ModelResource):
def like_method(self, request, **kwargs):
# Do some method checking
like_object(request.user, request.data.get("object")) # Here!
And in your view...
# Your view
def test(request, object_id):
obj = get_object_or_404(Object, id=object_id)
like_object(request.user, obj)
return HttpResponse()
Hope it helps.
What I meant exactly was, I would like to have JSON response when I modify the obj_create(). I've implemented the UserSignUpResource(ModelResource) and inside the obj_create(), I did some validation and when it fails, I raise BadRequest(). However, this doesn't throw out JSON. It throws out String instead.
Any idea if I can make it throw out {'error': 184, 'message': 'This username already exists'} format? Or am I not suppose to modify obj_create()? Or what should I do instead?
Many help, thanks.
Cheers,
Mickey
easy that, i've just created a little helper method in tastypies http module:
import json
#tastypies HttpResponse classes here...
def create_json_response(data, http_response_class):
return http_response_class(content=json.dumps(data), content_type="application/json; charset=utf-8")
then you can simply say:
from tastypie.http import HttpNotFound, create_json_response
#choose HttpNotFound, HttpCreated whatever...
raise ImmediateHttpResponse(create_json_response({"error":"resource not found"}, HttpNotFound))
You should use the error_response method from the resource.
Something like:
def obj_create(self, bundle, **kwargs):
# Code that finds Some error
my_errors = {"error": ["Some error"]}
raise ImmediateHttpResponse(response=self.error_response(bundle.request, my_errors))
Usually you would call super and the errors should arise from the tastypie validation process. The exception would be automatically thrown (with the errors dictionary being saved on the bundle.errors property).
In my django piston API, I want to yield/return a http response to the the client before calling another function that will take quite some time. How do I make the yield give a HTTP response containing the desired JSON and not a string relating to the creation of a generator object?
My piston handler method looks like so:
def create(self, request):
data = request.data
*other operations......................*
incident.save()
response = rc.CREATED
response.content = {"id":str(incident.id)}
yield response
manage_incident(incident)
Instead of the response I want, like:
{"id":"13"}
The client gets a string like this:
"<generator object create at 0x102c50050>"
EDIT:
I realise that using yield was the wrong way to go about this, in essence what I am trying to achieve is that the client receives a response right away before the server moves onto the time costly function of manage_incident()
This doesn't have anything to do with generators or yielding, but I've used the following code and decorator to have things run in the background while returning the client an HTTP response immediately.
Usage:
#postpone
def long_process():
do things...
def some_view(request):
long_process()
return HttpResponse(...)
And here's the code to make it work:
import atexit
import Queue
import threading
from django.core.mail import mail_admins
def _worker():
while True:
func, args, kwargs = _queue.get()
try:
func(*args, **kwargs)
except:
import traceback
details = traceback.format_exc()
mail_admins('Background process exception', details)
finally:
_queue.task_done() # so we can join at exit
def postpone(func):
def decorator(*args, **kwargs):
_queue.put((func, args, kwargs))
return decorator
_queue = Queue.Queue()
_thread = threading.Thread(target=_worker)
_thread.daemon = True
_thread.start()
def _cleanup():
_queue.join() # so we don't exit too soon
atexit.register(_cleanup)
Perhaps you could do something like this (be careful though):
import threading
def create(self, request):
data = request.data
# do stuff...
t = threading.Thread(target=manage_incident,
args=(incident,))
t.setDaemon(True)
t.start()
return response
Have anyone tried this? Is it safe? My guess is it's not, mostly because of concurrency issues but also due to the fact that if you get a lot of requests, you might also get a lot of processes (since they might be running for a while), but it might be worth a shot.
Otherwise, you could just add the incident that needs to be managed to your database and handle it later via a cron job or something like that.
I don't think Django is built either for concurrency or very time consuming operations.
Edit
Someone have tried it, seems to work.
Edit 2
These kind of things are often better handled by background jobs. The Django Background Tasks library is nice, but there are others of course.
You've turned your view into a generator thinking that Django will pick up on that fact and handle it appropriately. Well, it won't.
def create(self, request):
return HttpResponse(real_create(request))
EDIT:
Since you seem to be having trouble... visualizing it...
def stuff():
print 1
yield 'foo'
print 2
for i in stuff():
print i
output:
1
foo
2