I pushed a Docker image into an AWS ECR. Is there a way to also make it available in the EC2 instance associated with this repository (i.e. usable with the various docker CLI commands)?
As an example when I run docker images I don't view a new image I just pushed using docker push <Repository URI> in the list of images
Once you have pushed a Image to ECS , you will have to pull the image to run it as a container. This can be done on any instance or your local machine. For doing this you can follow :
1. aws ecr get-login --no-include-email --region <region> --profile <profile_name>
2. Cope the output from above and paste + enter
3. Pull the image finally as --> docker pull 501429058813.dkr.ecr.us-east-1.amazonaws.com/main:v1.0.0
Hope this helps
Related
I have a Docker image in AWS ECR and I open my Sagemaker Notebook instance--->go to terminal-->docker run....
This is how I start my Docker container.
Now, I want to automate this process(running my docker image on Sagemaker Notebook Instance) instead of typing the docker run commands.
Can I create a cron job on Sagemaker? or Is there any other approach?
Thanks
For this you can create an inline Bash shell in your SageMaker notebook as follows. This will take your Docker container, create the image, ECR repo if it does not exist and push the image.
%%sh
# Name of algo -> ECR
algorithm_name=your-algo-name
cd container #your directory with dockerfile and other sm components
chmod +x randomForest-Petrol/train #train file for container
chmod +x randomForest-Petrol/serve #serve file for container
account=$(aws sts get-caller-identity --query Account --output text)
# Region, defaults to us-west-2
region=$(aws configure get region)
region=${region:-us-west-2}
fullname="${account}.dkr.ecr.${region}.amazonaws.com/${algorithm_name}:latest"
# If the repository doesn't exist in ECR, create it.
aws ecr describe-repositories --repository-names "${algorithm_name}" > /dev/null 2>&1
if [ $? -ne 0 ]
then
aws ecr create-repository --repository-name "${algorithm_name}" > /dev/null
fi
# Get the login command from ECR and execute it directly
aws ecr get-login-password --region ${region}|docker login --username AWS --password-stdin ${fullname}
# Build the docker image locally with the image name and then push it to ECR
# with the full name.
docker build -t ${algorithm_name} .
docker tag ${algorithm_name} ${fullname}
docker push ${fullname}
I am contributing this on behalf of my employer, AWS. My contribution is licensed under the MIT license. See here for a more detailed explanation
https://aws-preview.aka.amazon.com/tools/stackoverflow-samples-license/
SageMaker Notebook instance lifecycle configuration script can be used to run a script when you create a notebook or at start time. In this script, you access other AWS resources from your notebook at create time or start time, say access your ECR images and automate starting docker container using a shell script. This script show an example of how you can use cron to schedule certain actions, can be modified per your usecase
Refer more lifecycle config samples in this github page
Can this be done and would I need to login twice in the buildspec?
Looking at completing a build, then on success, to push this image to the same account, as well as two other accounts. Anyone have or know if this can be done within the buildspec?
if this can be done within the buildspec
it lets you define any commands, and all you need console commands. So i would say - yes.
To push image into the AWS ECR you need to execute command docker login .....
which uses token. but there is a aws ecr get-login command which can help you with it.
so you need to have installed and configured aws ecr and then you can do this to login you docker into ecr:
$(aws ecr get-login --no-include-email --region {your ECR region})
after that you can do docker push imagename:latest
repeat 1-2 steps as many times as you need.
cleanup local mess.
I got a question - I have a docker image running locally on my Mac. - I'm trying to export that local image and deploy on AWS elasticbean stalk env.
Should I use docker export command which outputs it as a tar file then upload to AWS? or should it be in a different non compressed format?
I already tried the above and docker export it as a tar file but AWS didn't like that so what approach should I take here?
You can create a repository in your aws ECR (Amazon Elastic Container Registry) and push your local image to that repo
aws ecr get-login --no-include-email --region us-east-2
docker tag test-pod:latest 24533xxxxx.dkr.ecr.us-east-2.amazonaws.com/test:latest
docker push 24533xxxxx.dkr.ecr.us-east-2.amazonaws.com/test:latest
I'm trying to pull the pre-built docker images for SageMaker. I am able to successfully docker login to ECR (my AWS credentials). When I try to pull the image I get the standard no basic auth credentials.
Maybe I'm misunderstanding... I assumed those ECR URLs were public.
$(aws ecr get-login --region us-west-2 --no-include-email)
docker pull 246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-scikit-learn
As of 29th August 2021, get-login is deprecated and the command in the answer won't work. so, with AWS CLI v2, here's what has worked for me:
You would need to login to AWS CLI on your machine, then pipe the password to your docker login like this:
$ sudo aws ecr get-login-password --region <region> | sudo docker login --username AWS --password-stdin <account-id>.dkr.ecr.<region>.amazonaws.com
find the account IDs of the repo in the aws region nearest to you here; and available images with tags here by region.
Then you should be able pull images like this:
$ sudo docker pull 720646828776.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3
Could you show your ECR login command and pull command in the question?
For SageMaker pre-built image 520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-mxnet:1.3.0-cpu-py3
What I do is:
Log in ECR
$(aws ecr get-login --no-include-email --registry-ids 520713654638 --region us-west-2)
Pull the image
docker pull 520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-mxnet:1.3.0-cpu-py3
These images are public readable so you can pull them from any AWS account. I guess the reason you failed is that you did not specify --registry-ids in your login. But it's better if you can provide your scripts for others to identify what's wrong.
When I am trying to pull docker image from ECR, I am getting the below error:
Get https://3242344.dkr.ecr.ap-south-1.amazonaws.imagename/latest: no basic auth credentials
Docker service is running fine and I am able to list the repositories.
First, you need to Authenticate your Docker logins to the Amazon ECR:
aws ecr get-login --region <<region>> --no-include-email
Refer below link for Amazon ECR Registries authentication:
https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html#registry_auth
Describe your image within a repository by using below command:
aws ecr describe-images --repository-name amazonlinux
Pull the image by using below command:
docker pull aws_account_id.dkr.ecr.us-west-2.amazonaws.com/amazonlinux:latest
For more information please refer below link:
https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-pull-ecr-image.html
You need to retrieve the docker login command using AWS CLI
$(aws ecr get-login --no-include-email --region <your region>)
More info in Getting Started with Amazon ECR.