I have a Sagemaker model trained and deployed and looking to run a Batch Transform on multiple files.
I have a lambda function configured to run when a new file is uploaded to S3.
Currently, I have only seen ways to use the Invoke Endpoint function on lambda
i.e
runtime= boto3.client('runtime.sagemaker')
message = ["On Wed Sep PDT Brent Welch said Hacksaw said W"]
response = runtime.invoke_endpoint(EndpointName="sagemaker-scikit-learn-2020-02-05-13-44-45-011",
ContentType='text/plain',
Body=message)
json.loads(response['Body'].read())
However, i have multiple files that need to processed by the Sagemaker model.
I have code to create a transformer and run batch transform on multiple files
import sagemaker
training_job = sagemaker.estimator.Estimator.attach("{model_name")
transformer = training_job.transformer(instance_count=1,instance_type='ml.m4.xlarge',strategy='MultiRecord',assemble_with='Line')
#batch_input_s3
transformer.transform(batch_input_s3)
print('Waiting for transform job: ' + transformer.latest_transform_job.job_name)
transformer.wait()
however, im unable to use this transformer code in a lambda since it requires the sagemaker library, which is over the size limit for zip file is 50mb
Related
I have created a docker image which has Entrypoint as processing.py. This script is taking data from /opt/ml/processing/input and after processing putting it /opt/ml/processing/output folder.
For processing the data I should put the file in /opt/ml/processing/input from s3 and then pick processed file from /opt/ml/processing/output into S3.
Following script in sagemaker is doing it properly:
from sagemaker.processing import Processor, ProcessingInput, ProcessingOutput
import sagemaker
input_data = 's3://sagemaker-ap-south-1-057036842446/sagemaker/Data/Training/Churn_Modelling.csv'
output_dir = 's3://sagemaker-ap-south-1-057036842446/sagemaker/Outputs/'
image_uri = '057036842446.dkr.ecr.ap-south-1.amazonaws.com/aws-docker-repo:latest'
aws_role = sagemaker.get_execution_role()
processor = Processor(image_uri= image_uri, role=aws_role, instance_count=1, instance_type="ml.m5.xlarge")
processor.run(
inputs=[
ProcessingInput(
source=input_data,
destination='/opt/ml/processing/input'
)
],
outputs=[
ProcessingOutput(
source='/opt/ml/processing/output',
destination=output_dir
)
]
)
Could someone please guide how this can be executed with lambda function? It is not recognizing sagemaker package, second there is a challenge in placing file before the script execution and pick processed files.
I am trying codepipeline to automate this operation. However got no success on that.
Not sure how to put image from S3 into folders internally used by script
I need to know how S3 processing step which pick data from /opt/ml/processing/input
If you want to kick off a Processing Job from Lambda you can use boto3 to make the CreateProcessingJob API call:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_processing_job
I would suggest creating the Job as you have been doing using the SageMaker SDK. Once created, you can describe the Job using the DescribeProcessingJob API call:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.describe_processing_job
You can then use the information from the DescribeProcessingJob API call output to fill out the CreateProcessingJob in Lambda.
I'm having trouble executing VertexAI's batch inference, despite endpoint deployment and inference working perfectly. My TensorFlow model has been trained in a custom Docker container with the following arguments:
aiplatform.CustomContainerTrainingJob(
display_name=display_name,
command=["python3", "train.py"],
container_uri=container_uri,
model_serving_container_image_uri=container_uri,
model_serving_container_environment_variables=env_vars,
model_serving_container_predict_route='/predict',
model_serving_container_health_route='/health',
model_serving_container_command=[
"gunicorn",
"src.inference:app",
"--bind",
"0.0.0.0:5000",
"-k",
"uvicorn.workers.UvicornWorker",
"-t",
"6000",
],
model_serving_container_ports=[5000],
)
I have a Flask endpoint defined for predict and health essentially defined below:
#app.get(f"/health")
def health_check_batch():
return 200
#app.post(f"/predict")
def predict_batch(request_body: dict):
pred_df = pd.DataFrame(request_body['instances'],
columns = request_body['parameters']['columns'])
# do some model inference things
return {"predictions": predictions.tolist()}
As described, when training a model and deploying to an endpoint, I can successfully hit the API with JSON schema like:
{"instances":[[1,2], [1,3]], "parameters":{"columns":["first", "second"]}}
This also works when using the endpoint Python SDK and feeding in instances/parameters as functional arguments.
However, I've tried performing batch inference with a CSV file and a JSONL file, and every time it fails with an Error Code 3. I can't find logs on why it failed in Logs Explorer either. I've read through all the documentation I could find and have seen other's successfully invoke batch inference, but haven't been able to find a guide. Does anyone have recommendations on batch file structure or the structure of my APIs? Thank you!
I would like to place a csv file in an S3 bucket and get predictions from a Sagemaker model using batch transform job automatically. I would like to do that by using s3 event notification (upon csv upload) to trigger a Lambda function which would do a batch transform job. The lambda function I have written so far is this:
import boto3
sagemaker = boto3.client('sagemaker')
input_data_path = 's3://yeex/upload/examples.csv'.format(default_bucket, 's3://yeex/upload/', 'examples.csv')
output_data_path = 's3://nooz/download/'.format(default_bucket, 's3://nooz/download')
transform_job = sagemaker.transformer.Transformer(
model_name = y_xgboost_21,
instance_count = 1,
instance_type = 'ml.m5.large',
strategy = 'SingleRecord',
assemble_with = 'Line',
output_path = output_data_path,
base_transform_job_name='y-test-batch',
sagemaker_session=sagemaker.Session(),
accept = 'text/csv')
transform_job.transform(data = input_data_path,
content_type = 'text/csv',
split_type = 'Line')
The error it returns is that object sagemaker does not have module transform
What is the syntax I should use in Lambda function?
While Boto3 (boto3.client("sagemaker")) is the general-purpose AWS SDK for Python across different services, examples that you might see referencing classes like Estimator, Transformer, Predictor and etc are referring to the SageMaker Python SDK (import sagemaker).
In general I'd say (almost?) anything that can be done in one can also be done in the other as they use the same underlying service APIs - but the purpose of the SM Python SDK is to provide higher-level abstractions and useful utilities: For example transparently zipping and uploading a source_dir to S3 to deliver "script mode" training.
As far as I'm aware, the SageMaker Python SDK is still not pre-installed in AWS Lambda Python runtimes by default: But it is an open-source and pip-installable package.
So you have 2 choices here:
Continue using boto3 and create your transform job via the low-level create_transform_job API
Install sagemaker in your Python Lambda bundle (Tools like AWS SAM or CDK might make this process easier) and instead import sagemaker so you can use the Transformer and other high-level Python APIs.
I have my own trained TF Object Detection model. If I try to deploy/implement the same model in AWS Sagemaker. It was not working.
I have tried TensorFlowModel() in Sagemaker. But there is an argument called entrypoint- how to create that .py file for prediction?
entrypoint is a argument which contains the file name inference.py,which means,once you create a endpoint and try to predict the image using the invoke endpoint api. the instance will be created based on you mentioned and it will go to the inference.py script and execute the process.
Link : Documentation for tensor-flow model deployment in amazon sage-maker
.
The inference script must contain a methods input_handler and output_handler or handler which will cover both the function in inference.py script, this for pre and post processing of your image.
Example for Deploying the tensor flow model
In the above link, i have mentioned a medium post, this will be helpful for your doubts.
Hi I am trying out opencv in AWS lambda. I want to save a SVM model in txt file so that I can load it again. Is it possible to save it in tmp directory and load it from there whenever I need it or will I have to use s3?
I am using python and trying to do something like this:
# saving the model
svm.save("/tmp/svm.dat")
# Loading the model
svm = cv2.ml.SVM_load("/tmp/svm.dat")
Its not possible as Lambda execution environment is distributed and therefore the same function might run on several different instances.
The alternative is to save your svm.dat to S3 and then download it every time you start your lambda function.