I am looking for the simplest solution for updating a Sagemaker Endpoint. The only thing I want to change is the docker image (to update the code).
I am building the new Docker image on my computer, and then I upload it on ECR (I plan to do this in a CI/CD in the near future).
From my understanding, it seems that the simplest way is to create a new EndpointConfig, then call the UpdateEndpoint API to switch the endpoint to the new config, then delete the old EndpointConfig. Does anyone know a simpler way? Or can anyone confirm that this is the simplest approach to doing this?
Related
What are the ways how one can roll back the App Runner service to the previous revision?
We use a simple GitHub Actions workflow to build and upload a new docker image to the ECR registry.
How can I select an old image in ECR and upload this instead (rolling back revision)?
I'm also interested in this issue. It seems that when you create an app runner app with a specific image tag and that's it you cannot choose which version to deploy. You either have to amend the service change the Container image URI to the previous image
I am able to train a model on Sagemaker and then deploy a model endpoint out of it.
Now, I want to retrain my model every week with the new data that is coming in. My question is - when I retrain the model how do I update my existing endpoint to use the latest model. (I don't want to deploy a new endpoint)
From some exploration, I think I can do it in 2 ways -
Near the end of the training job, I create a new EndpointConfig and later use UpdateEndpoint - The downside of this would be - I would end up with a lot of unnecessary Endpoint Configurations in my AWS Account? Or am I thinking about it wrongly?
Near the end of the training job, I deploy the trained model using .deploy() and set update_endpoint=True as illustrated in Sagemaker SDK Doc
I am not sure which is the better solution to accomplish this? Is there an even better way to do this?
If you are interested in doing this programmatically, use an AWS SDK (I will answer this assuming you are using Java.
Look the AWS SDK for Java V2 Javadocs. You can use the UpdateEndpoint to perform this use case. This method deploys the new EndpointConfig specified in the request, switches to using newly created endpoint, and then deletes resources provisioned for the endpoint using the previous EndpointConfig (there is no availability loss).
I aim to copy a gcr image from one project to another as soon as the image lands in the container registry of the first project. I am aware of the gcloud container images add-tag command, looking for a more automated option. Also the second project where the image has to be copied is protected by VPC-SC. Any leads will be appreciated...
I understand that you are looking for the best way to mirror the GCR images between two projects. Currently, you can follow the workaround in this document click to copy the container images for your use case. At the moment, the only way to move between two registries is by pulling from one and pushing to another, if you have the right permission. There is currently a tool on github that can automate this for you, gcrane click . However, for mirroring the container images between two projects, a feature request has already been submitted but there is no ETA.
According to the GCP documentation click , If the project is protected by VPC-SC, the container registry does not use googleapis.com domain. To achieve this, container registry need to configured via private DNS or BIND to map to the restricted VIP separately from other APIs.
When a change is made to a container registry that you own, a Pub/Sub message can be published. You can use this Pub/Sub message as a trigger to perform work. My immediate thought would be to create a Cloud Function that is triggered by the arrival of a message which then fires off a Cloud Build recipe. The Cloud Build would perform a docker pull of your original image and then a tag rename and then a docker push. It feels like this would be 100% automated and use components that are designed for CI/CD pipelines.
References:
Configuring Pub/Sub notifications
Cloud Build documentation
I have started exploring AWS SageMaker starting with these examples provided by AWS. I then made some modifications to this particular setup so that it uses the data from my use case for training.
Now, as I continue to work on this model and tuning, after I delete the inference endpoint once, I would like to be able to recreate the same endpoint -- even after stopping and restarting the notebook instance (so the notebook / kernel session is no longer valid) -- using the already trained model artifacts that gets uploaded to S3 under /output folder.
Now I cannot simply jump directly to this line of code:
bt_endpoint = bt_model.deploy(initial_instance_count = 1,instance_type = 'ml.m4.xlarge')
I did some searching -- including amazon's own example of hosting pre-trained models, but I am a little lost. I would appreciate any guidance, examples, or documentation that I could emulate and adapt to my case.
Your comment is correct - you can re-create an Endpoint given an existing EndpointConfiguration. This can be done via the console, the AWS CLI, or the SageMaker boto client.
https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-endpoint.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint
Is it possible to auto update image used by already deployed container post auto build by trigger?
One solution I know is to add command in cloud build.yaml for restarting the server.
Is there any better approach?
you will need to restart the container with an updated image, as you cannot modify an image when the container is already deployed. If you need the latest image version to be consistently deployed, then just omit a specific version number for the image, the client defaults to latest. More details here.
If this is not the issue, please specify and provide some examples.