After successfully training object detection model with AWS SageMaker, how do I use this model to perform real time object detection on RTSP video?
One solution for real-time object detection is as follows.
After training your model, upload your model to S3. You can check this using
$aws sagemaker list-training-jobs --region us-east-1
Then you need to your trained model on an Amazon SageMaker endpoint. You can do this using this:
object_detector = estimator.deploy(initial_instance_count = 1,
instance_type = 'ml.g4dn.xlarge')
After your model is attached to an endpoint you will need to create an API that will allow users to pass input to your trained model for inference. You can do this using Serverless. With Serverless, you can create a template which will create a Lambda function as handler.py and serverless.yml which needs to be configured on how your application will operate. Make sure that in serverless.yml you specify your endpoint name, SAGEMAKER_ENDPOINT_NAMEas well as Resource: ${ssm:sagemakerarn}. This is an allow policy resource (AWS Systems Manager Agent) parameter that needs to be passed in. In your lambda function, make sure you invoke your SageMaker endpoint.
From here you can now deploy your API for real-time detection:
serverless deploy -v
Finally, you can use curl to invoke your API.
See here for a detailed walk-through.
You haven't specified where you will host the model: on AWS, a mobile device or something else. However, the general approach is that your model, assuming a CNN that processes images, will consume one frame at a time. You haven't specified a programming language or libraries so here is the general process in psuedocode:
while True:
video_frame = get_next_rtsp_frame()
detections = model.predict(video_frame)
# There might be multiple objects detected, handle each one:
for detected_object in detections:
(x1, y1, x2, y2, score) = detected_object # bounding box
# use the bounding box information, say to draw a box on the image
One challenge with a real-time video stream requirement is avoiding latency, depending on your platform and what kind of processing you do in your loop. You can skip frames, or don't buffer missed frames, to address this.
Related
While training a model on AWS Sagemaker(let us assume training takes 15 hours or more). If our laptop lose internet connection in between, the Kernal on which it is training will die. But the model continues to train (I confirmed this with model.save command, and the model did save in the s3 bucket).
I want to know if there is a way, to track the status/progress of our model training when Kernel dies at Sagemaker environment.
Note: I know we can create a training job under Training - Training Jobs - Create Training Jobs. I just wanted to know if there is any other approach to track if we are not creating the Training Job.
Could you specify the 'Job Name' of the sagemaker training job? You can get the status using an API call if you have the job name. https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeTrainingJob.html
Another note: you can specify the job name of a training job using the 'TrainingJobName' parameter of training requests: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html
Simply check of status
When you run a training job, a log tracker is automatically created in CloudWatch within the "/aws/sagemaker/TrainingJobs" group with the name of your job and in turn one or more sub-logs, based on the number of instances selected.
This already ensures you can track the status of the job even if the kernel dies or if you simply turn off the notebook instance.
Monitor metrics
For sagemaker's built-in algorithms, no configuration action is required since the monitorable metrics are already prepared.
Custom model
On custom models, on the other hand, to have a monitoring graph of metrics, you can configure the log group related to them in CloudWatch (Metrics) as the official documentation explains. at "Monitor and Analyze Training Jobs Using Amazon CloudWatch Metrics" and "Define Metrics".
Basically, you just need to add the parameter metric_definitions to your Estimator (or a subclass of it):
metric_definitions=[
{'Name': 'train:error', 'Regex': 'Train_error=(.*?);'},
{'Name': 'validation:error', 'Regex': 'Valid_error=(.*?);'}
]
this will capture from the print/logger output of your training script the text identified by the regexes you set (which you can clearly change to your liking) and create a tracking within cloudwatch metrics.
A complete code example from doc:
import sagemaker
from sagemaker.estimator import Estimator
estimator = Estimator(
image_uri="your-own-image-uri",
role=sagemaker.get_execution_role(),
sagemaker_session=sagemaker.Session(),
instance_count=1,
instance_type='ml.c4.xlarge',
metric_definitions=[
{'Name': 'train:error', 'Regex': 'Train_error=(.*?);'},
{'Name': 'validation:error', 'Regex': 'Valid_error=(.*?);'}
]
)
I have created a custom container for prediction and successfully uploaded the model to Vertex AI. I was also able to deploy the model to an endpoint and successfully request predictions from the endpoint. Within the custom container code, I use the parameters field as described here, which I then supply later on when making an online prediction request.
My questions are regarding requesting batch predictions from a custom container for prediction.
I cannot find any documentation that describes what happens when I request a batch prediction. Say, for example, I use the my_model.batch_predict function from the Python SDK and set the instances_format to "csv" and provide the gcs_source. Now, I have setup my custom container to expect prediction requests at /predict as described in this documentation. Does Vertex AI make a POST request to this path, converting the cvs data into the appropriate POST body?
How do I specify the parameters field for batch prediction as I did for online prediction?
Yes vertex AI makes a POST request your custom containers in batch prediction.
No, there is no way for batch prediction to pass a parameter since we don't know which column is "parameter". We put everything into "instances".
I have an endpoint running a trained SageMaker model on AWS, which expects the data on a specific format.
Initially, the data has been processed on the client side of the application, it means, the API Gateway (which receives the POST API calls on AWS) used to receive pre-processed data, but now there's a change, the API Gateway will receive raw data from the client, and the job of pre-processing this data before sending to our SageMaker model is up to our workflow.
What is the best way to create a pre-processing job on this workflow, without needing to re-train the model? My pre-process is just a bunch of dataframe transformations, no standardization or calculation with the training set required (it would not need to save any model file).
Thanks!
After some research, this is the solution I've followed:
First I have created a SKLearn sagemaker model to do all the preprocess setup (I've built a Scikit-Learn custom class to handle all the preprocess steps, following this AWS code)
Trained this preprocess model on my training data. My model, in specific, didn't need to be trained (it does not have any standardization or anything that would need to store training data parameters), but sagemaker requires the model to be trained.
Loaded the trained legacy model that we had using the Model parameter.
Created a PipelineModel with the preprocessing model and legacy model in cascade:
pipeline_model = PipelineModel(name=model_name,
role=role,
models=[
preprocess_model,
trained_model
])
Create a new endpoint, calling the PipelineModel and then changed the Lambda function to call this new endpoint. With this I could send the raw data directly for the same API Gateway and it would call only one endpoint, without needing to pay two endpoints 24/7 to perform the entire process.
I've found this to be a good and "economic" way to perform the preprocess outside the trained model, without having to do hard processing jobs on a Lambda function.
I would create a Lambda, which is getting invoked by the API-Gateway, processing the data and sending it to your SageMaker endpoint.
I have my own trained TF Object Detection model. If I try to deploy/implement the same model in AWS Sagemaker. It was not working.
I have tried TensorFlowModel() in Sagemaker. But there is an argument called entrypoint- how to create that .py file for prediction?
entrypoint is a argument which contains the file name inference.py,which means,once you create a endpoint and try to predict the image using the invoke endpoint api. the instance will be created based on you mentioned and it will go to the inference.py script and execute the process.
Link : Documentation for tensor-flow model deployment in amazon sage-maker
.
The inference script must contain a methods input_handler and output_handler or handler which will cover both the function in inference.py script, this for pre and post processing of your image.
Example for Deploying the tensor flow model
In the above link, i have mentioned a medium post, this will be helpful for your doubts.
I'm finding the right way to use AWS Rekognition service.
My problem is How to verify a person image on multi collections, I'm reading Build Your Own Face Recognition Service Using Amazon Rekognition | AWS Machine Learning Blog from Amazon but cannot find the implementation document for it. My point is Face verification title.
Update 1:
My target is: Using AWS Rekognition to get person's info by their face.
My problem is: How to make AWS Rekognition improves its accuracy when recognizing a face.
What I tried:
Upload multi captured portraits of a person with same ExternalImageID but I'm not sure it works or not.
Finding a way to create Collection for each person, then upload person's portraits to their Collection but I don't how to search a face through multiple Collections.
I'm trying use S3 for storage people's images then using Lambda function to do something that I've not got yet.
Update 2:
What is your input material: Input materials are some people's portrait photo with ExternalImageID is their name (eg: my portrait photo will have ExternalImageID is "Long").
What are you trying to do: I'm trying to get ExternalImageID when I send a portrait photo of a registered person. (eg: with my other portrait photo, AWS has to response ExternalImageID is "Long").
Do you have it working, but it is not recognizing some people? Yes, it's work but sometimes it cannot recognize exactly people.
Please tell us your use-case / scenario and what you are trying to accomplish:
Create an AWS Rekognition collection with sample name (eg facetest).
Register some people with their name is ExternalImageID.
Submit an image to AWS Rekognition API to get ExternalImageID - his name.
Okay, so basically you have it working but it doesn't always recognise the person. I'll assume it does not even list the person in the response, even with a low percentage.
I would recommend adding multiple images of the same person to the Face Collection, specifying the same ExternalImageId for each image. (Use one Face Collection with all people in it, including multiple images of the same person.)
Please note that "If you provide the same image, specify the same collection, and use the same external ID in the IndexFaces operation, Amazon Rekognition doesn't save duplicate face metadata." However, adding different images with the same ExternalImageId should be fine.