I want to defer some processing load from my Django app to an AWS Lambda.
I'm calling my code from the Lambda like this:
lambda.py:
#bc_lambda(level=logging.INFO, service=LAMBDA_SERVICE)
def task_handler(event, context):
message = event["Records"][0]["body"]
renderer = get_renderer_for(message)
result = renderer.render()
return result
get_renderer_for is a factory method that returns an instance of the class Renderer:
from myproject.apps.engine.documents import (
DocumentsLoader,
SourceNotFound,
source_from_version,
)
from myproject.apps.engine.environment import Environment
class Renderer:
def __init__(self, message):
self.message = message
def render(self):
ENVIRONMENT = Environment(DocumentsLoader())
version_id = self.message.get("version_id")
try:
source = source_from_version(version_id)
except SourceNotFound:
source = None
template = ENVIRONMENT.from_string(source)
if template:
return template.render(self.message)
return None
def get_renderer_for(message):
"""
Factory method that returns an instance of the Renderer class
"""
return Renderer(message)
In CloudWatch, I see I'm getting this error: module initialization error. Apps aren't loaded yet.
I understand that Django is not available for the Lambda function, right? How can I fix this? How can I make the rest of the project available to the lambda function?
The only two libraries that Lambda supports out of the box are the standard library and boto3.
There are several ways to install external Python libraries for use in Lambda. I recommend uploading them as a Lambda layer. This is a good guide: https://medium.com/#qtangs/creating-new-aws-lambda-layer-for-python-pandas-library-348b126e9f3e
Related
I am trying to invoke my SageMaker endpoint and pass CustomAttributes argument specified here.
what I want to know is how to retrieve the CustomAttributes in the model endpoint?
I created an inference.py file for my endpoint that has the following structure:
imports
def get_device():
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
return device
def model_fn(model_dir):
return model
def transform_fn(model, request_body, content_type, accept)
return json.dumps(predictions)
just as request_body, content_type, accept are passed to transform_fn, i want to pass the CustomAttributes. Is this possible and if so how can i do it ?
Thanks in advance!
It is convenient to structure the inference code by having full control of all the underlying steps.
As the documentation "Adapting Your Own Inference Container" suggests, you can arrange 4 functions: model_fn, input_fn, predict_fn and output_fn.
Beyond these, you can create your own handler to handle all the attributes you pass when you invoke an endpoint ("How to implement the pre- and/or post-processing handler(s)").
You can have the input_handler / output_handler pair of functions or a single handler function.
Below is an example of code for a generic inference script.
Inside your block of endpoint's invocation:
import boto3
import json
runtime = boto3.Session().client('sagemaker-runtime')
runtime_client.invoke_endpoint(
EndpointName = your_endpoint_name,
Body = your_data,
CustomAttributes = json.dumps(your_attributes_dict),
ContentType = your_content_type
)
and inside your inference.py:
import json
def handler(data, context):
processed_input = _process_input(data, context)
custom_attrs = json.loads(context.custom_attributes)
# here place your function to parse and use your custom_attrs json
response = requests.post(context.rest_uri, data=processed_input)
return _process_output(response, context)
def _process_input(data, context):
# your _process_input to decode the request_content_type
if context.request_content_type == YOUR_CONTEXT_TYPE:
return your_process_func(data)
raise ValueError('{{"error": "unsupported content type {}"}}'.format(
context.request_content_type or "unknown"))
def _process_output(data, context):
if data.status_code != 200:
raise ValueError(data.content.decode('utf-8'))
response_content_type = context.accept_header
prediction = data.content
return prediction, response_content_type
Pay attention to this note:
Note that if handler function is implemented, input_handler and
output_handler are ignored.
It means that if you use a framework like TensorFlow or PyTorch, you will have to see how to override these methods starting from their default handlers.
I have a custom prediction routine in google-ml-engine. Works very well.
I now am doing input checking on the instance data, and want to return error responses from my predict routine.
The example: https://cloud.google.com/ai-platform/prediction/docs/custom-prediction-routines
Raises exceptions on input errors, etc. However, when this happens the response body always has {'error': Prediction failed: unknown error}. I can see the correct errors are being logged in google cloud console, but the https response is always the same unknown error.
My question is:
How to make the Custom prediction routine return a proper error code and error message string?
Instead of returning a prediction, I can return an error string/code in prediction -but it ends up in the prediction part of the response which seems hacky and doesn't get any of the google errors eg based on instance size.
root:test_deployment.py:35 {'predictions': {'error': "('Instance does not include required sensors', 'occurred at index 0')"}}
What's the best way to do this?
Thanks!
David
Please take a look at the following code, I created a _validate function inside predict and use a custom Exception class.
Basically, I validate instances, before I call the model predict method and handle the exception.
There may be some overhead to the response time when doing this validation, which you need to test for your use case.
requests = [
"god this episode sucks",
"meh, I kinda like it",
"what were the writer thinking, omg!",
"omg! what a twist, who would'v though :o!",
99999
]
api = discovery.build('ml', 'v1')
parent = 'projects/{}/models/{}/versions/{}'.format(PROJECT, MODEL_NAME, VERSION_NAME)
parent = 'projects/{}/models/{}'.format(PROJECT, MODEL_NAME)
response = api.projects().predict(body=request_data, name=parent).execute()
{'predictions': [{'Error code': 1, 'Message': 'Invalid instance type'}]}
Custom Prediction class:
import os
import pickle
import numpy as np
import logging
from datetime import date
import tensorflow.keras as keras
class CustomModelPredictionError(Exception):
def __init__(self, code, message='Error found'):
self.code = code
self.message = message # you could add more args
def __str__(self):
return str(self.message)
def isstr(s):
return isinstance(s, str) or isinstance(s, bytes)
def _validate(instances):
for instance in instances:
if not isstr(instance):
raise CustomModelPredictionError(1, 'Invalid instance type')
return instances
class CustomModelPrediction(object):
def __init__(self, model, processor):
self._model = model
self._processor = processor
def _postprocess(self, predictions):
labels = ['negative', 'positive']
return [
{
"label":labels[int(np.round(prediction))],
"score":float(np.round(prediction, 4))
} for prediction in predictions]
def predict(self, instances, **kwargs):
try:
instances = _validate(instances)
except CustomModelPredictionError as c:
return [{"Error code": c.code, "Message": c.message}]
else:
preprocessed_data = self._processor.transform(instances)
predictions = self._model.predict(preprocessed_data)
labels = self._postprocess(predictions)
return labels
#classmethod
def from_path(cls, model_dir):
model = keras.models.load_model(
os.path.join(model_dir,'keras_saved_model.h5'))
with open(os.path.join(model_dir, 'processor_state.pkl'), 'rb') as f:
processor = pickle.load(f)
return cls(model, processor)
Complete code in this notebook.
If it is still relevant to you, I found a way by using google internal libraries (not sure if it would be endorsed by Google though).
AI platform custom prediction wrapping code only returns custom error message if the Exception thrown is a specific one from their internal library.
It might also not be super reliable as you would have very little control in case Google wants to change it.
class Predictor(object):
def predict(self, instances, **kwargs):
# Your prediction code here
# This is an internal google library, it should be available at prediction time.
from google.cloud.ml.prediction import prediction_utils
raise prediction_utils.PredictionError(0, "Custom error message goes here")
#classmethod
def from_path(cls, model_dir):
# Your logic to load the model here
You would get the following message in your HTTP response
Prediction failed: Custom error message goes here
I'm currently running on one python Gui development based on Pyqt5. There is quite a lot of file system related operations such as copy, move and delete. I tried to enhance the performance of these operations by utilizing concurrent.futures module and experimented on both Threadpoolexecutor and Processpoolexecutor. However, neither of them could give me a unblocking gui which would be able to go on with user interaction after I submitted the file operations to the pool. These submitted operations are simply static methods of native python classes. I also added callbacks through the Future objects returned by submissions, which are the instance methods of one qt class and which did get called when the corresponding submitted job had completed.
Just unable to figure out a way to solve the problem of freezed ui(even just for seconds' duration) Could anybody help?
from concurrent.futures import ProcessPoolExecutor
class ArchiveResManager:
def __init__(self, resource_dir):
self._path_reference_dir = resource_dir
def get_resource_list(self):
return tuple(
de.path for de in os.scandir(self._path_reference_dir) if de.is_file()
)
def add_callback_resource_added(fn):
self._callback_on_resource_added = fn
def add_resources(self, resources: iter):
src_paths = tuple(p for p in resources)
dest_paths = tuple(self._get_dest_path(p) for p in resources)
with ProcessPoolExecutor(2) as executor:
exe.submit(
ArchiveResManager._do_resource_operations,
src_paths,
dest_paths
).add_done_callback(self._on_resource_added)
#staticmethod
def _do_resource_operations(src_paths, dest_paths):
added_items = list()
for src_path, dest_path in zip(src_paths, dest_paths):
shutil.copyfile(
src_path, dest_path
)
added_items.append(dest_path)
return added_items
def _on_resource_added(self, future):
if future.done():
if self._callback_on_resource_added:
self._callback_on_resource_added(future.result())
# This model class is bound to a instance of QListView in the QDialog of my project
class ArchiveModel(QAbstractListModel):
def __init__(self, archive_dir, parent):
super().__init__(parent)
self._archive_manager = ArchiveResManager(archive_dir)
self._archive_manager.add_callback_resource_added(self._on_archive_operations_completed)
self._archive_items = self._archive_manager.get_resource_list()
def _on_archive_operations_completed(self, result_list):
last_row = self.rowCount() - 1
self.beginInsertRows(QModelIndex(), last_row, last_row + len(result_list))
self._archive_items.extend(result_list)
self.endInsetRows()
I've a multi-tenant setup where I'd like to pass certain customer specific information, specifically request.host to the celery task, where ideally it should be available in a global variable. Is there a way to set this up, in a manner transparent to the application?
the task would be called the same way:
my_background_func.delay(foo, bar)
the task is defined the same way, except that it has access to a global variable called 'request' having an attribute 'host':
#celery_app.task
def my_background_func(foo, bar):
print "running the task for host:" + request.host
here's how I solved it ...
class MyTask(Task):
abstract = True
def delay(self, *args, **kwargs):
return self.apply_async(args, kwargs, headers={'host': request.host})
on the client:
#celery_app.task(base=MyTask, bind=True)
def hellohost(task):
return "hello " + task.request.headers['host']
it works, but strangely hellohost.delay().get() hangs on the client
I am trying to access access application configuration inside a blueprint authorisation.py which in a package api. I am initializing the blueprint in __init__.py which is used in authorisation.py.
__init__.py
from flask import Blueprint
api_blueprint = Blueprint("xxx.api", __name__, None)
from api import authorisation
authorisation.py
from flask import request, jsonify, current_app
from ..oauth_adapter import OauthAdapter
from api import api_blueprint as api
client_id = current_app.config.get('CLIENT_ID')
client_secret = current_app.config.get('CLIENT_SECRET')
scope = current_app.config.get('SCOPE')
callback = current_app.config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
#api.route('/authorisation_url')
def authorisation_url():
url = auth.get_authorisation_url()
return str(url)
I am getting RuntimeError: working outside of application context
I understand why that is but then what is the correct way of accessing those configuration settings?
----Update----
Temporarily, I have done this.
#api.route('/authorisation_url')
def authorisation_url():
client_id, client_secret, scope, callback = config_helper.get_config()
auth = OauthAdapter(client_id, client_secret, scope, callback)
url = auth.get_authorisation_url()
return str(url)
Use flask.current_app in place of app in the blueprint view.
from flask import current_app
#api.route("/info")
def get_account_num():
num = current_app.config["INFO"]
The current_app proxy is only available in the context of a request.
Overloading record method seems to be quite easy:
api_blueprint = Blueprint('xxx.api', __name__, None)
api_blueprint.config = {}
#api_blueprint.record
def record_params(setup_state):
app = setup_state.app
api_blueprint.config = dict([(key,value) for (key,value) in app.config.iteritems()])
To build on tbicr's answer, here's an example overriding the register method example:
from flask import Blueprint
auth = None
class RegisteringExampleBlueprint(Blueprint):
def register(self, app, options, first_registration=False):
global auth
config = app.config
client_id = config.get('CLIENT_ID')
client_secret = config.get('CLIENT_SECRET')
scope = config.get('SCOPE')
callback = config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
super(RegisteringExampleBlueprint,
self).register(app, options, first_registration)
the_blueprint = RegisteringExampleBlueprint('example', __name__)
And an example using the record decorator:
from flask import Blueprint
from api import api_blueprint as api
auth = None
# Note there's also a record_once decorator
#api.record
def record_auth(setup_state):
global auth
config = setup_state.app.config
client_id = config.get('CLIENT_ID')
client_secret = config.get('CLIENT_SECRET')
scope = config.get('SCOPE')
callback = config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
Blueprints have register method which called when you register blueprint. So you can override this method or use record decorator to describe logic which depends from app.
The current_app approach is fine but you must have some request context. If you don't have one (some pre-work like testing, e.g.) you'd better place
with app.test_request_context('/'):
before this current_app call.
You will have RuntimeError: working outside of application context , instead.
You either need to import the main app variable (or whatever you have called it) that is returned by Flask():
from someplace import app
app.config.get('CLIENT_ID')
Or do that from within a request:
#api.route('/authorisation_url')
def authorisation_url():
client_id = current_app.config.get('CLIENT_ID')
url = auth.get_authorisation_url()
return str(url)
You could also wrap the blueprint in a function and pass the app as an argument:
Blueprint:
def get_blueprint(app):
bp = Blueprint()
return bp
Main:
from . import my_blueprint
app.register_blueprint(my_blueprint.get_blueprint(app))
I know this is an old thread. But while writing a flask service, I used a method like this to do it. It's longer than the solutions above but it gives you the possibility to use customized class yourself. And frankly, I like to write services like this.
Step 1:
I added a struct in a different module file where we can make the class structs singleton. And I got this class structure from this thread already discussed. Creating a singleton in Python
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
else:
cls._instances[cls].__init__(*args, **kwargs)
return cls._instances[cls]
Step 2:
Then I created a Singleton EnvironmentService class from our Singleton class that we defined above, just for our purpose. Instead of recreating such classes, create them once and use them in other modules, routes, etc. import. We can access the class with the same reference.
from flask import Config
from src.core.metaclass.Singleton import Singleton
class EnvironmentService(metaclass=Singleton):
__env: Config = None
def initialize(self, env):
self.__env = env
return EnvironmentService()
def get_all(self):
return self.__env.copy()
def get_one(self, key):
return self.__env.get(key)
Step 3:
Now we include the service in the application in our project root directory. This process should be applied before the routes.
from flask import Flask
from src.services.EnvironmentService import EnvironmentService
app = Flask(__name__)
# Here is our service
env = EnvironmentService().initialize(app.config)
# Your routes...
Usage:
Yes, we can now access our service from other routes.
from src.services.EnvironmentService import EnvironmentService
key = EnvironmentService().get_one("YOUR_KEY")