when I try to push a docker image to aws ecr it fails giving the following
sudo docker push xxxxxxx.dkr.ecr.us-east-2.amazonaws.com/my-app:1.0
7d9a9c94af8d: Preparing
f77d412f54b5: Preparing
629960860aca: Preparing
f019278bad8b: Preparing
8ca4f4055a70: Preparing
3e207b409db3: Waiting
no basic auth credentials
although logging in is done successfully
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin xxxx.dkr.ecr.us-east-2.amazonaws.com
Login Succeeded
And the /home/[my user]/.docker/config.json file has the following data
{
"auths": {
"xxxx.dkr.ecr.us-east-2.amazonaws.com": {
"auth": "QVsVkhaRT...."
}
}
}
I am using aws cli version 2.3.5
aws --version
aws-cli/2.3.5 Python/3.8.8 Linux/5.8.0-63-generic exe/x86_64.ubuntu.20 prompt/off
I am using docker version 20.10.10
docker --version
Docker version 20.10.10, build b485636
How can I solve this problem?
You're running sudo docker push.
This means that the credentials in your account won't be used. Instead, Docker is trying to use (nonexistent) credentials in the root user account.
Changing your command to docker push should suffice.
I have a CodeBuild Project which runs a docker build command with a buildspec.yml like this
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
However during the docker build process I have a sh script that runs the aws s3 cp command. I have given the service role permissions to the bucket and the error I get is.
+ aws s3 cp s3://bucket/file /var/www/html/filelocal
fatal error: Unable to locate credentials
Do roles not propagate through to Docker on Codebuild?
You are building an isolated file system. If you ran the docker build locally with the credentials on your local machine, you would see the same behavior. You would have to add credentials to your container to run those same operations. With that said, you could add credentials to your container via build-args or you could just use the codebuild role to gather the files you need and then copy those into the container during the build process. I would vote for the second way so you don't have to worry about cleaning up the credentials before publishing the container. Although you can query the environment to get the role's temporary credentials, which means you probably wouldn't have to worry about cleaning those up, you could just as easily remove those concerns by just letting the codebuild role handle gathering the files you need to build the container.
I have a line in my Dockerfile like this:
FROM 6*********.dkr.ecr.ap-southeast-1.amazonaws.com/*************:ff03401
This ECR is owned by another user.
As recommended in this question, I am trying to log in by using these commands in the build section of my buildspec.yml, and then immediately pull this docker image:
- aws configure set aws_access_key_id $ECR_ACCESS_KEY
- aws configure set aws_secret_access_key $ECR_SECRET_KEY
- eval aws ecr get-login --no-include-email --region ap-southeast-1 --registry-ids 6***********
- docker pull 6***********.dkr.ecr.ap-southeast-1.amazonaws.com/****************:ff03401
When I look at the Codebuild logs, I see that eval aws ecr get-login... outputs a docker login ... command which, if I run it on my local machine, logs me in successfully, and lets me do the docker pull 6******....
In Codebuild, however, docker pull says:
Error response from daemon: Get https://6**********.dkr.ecr.ap-southeast-1.amazonaws.com/v2/******************/manifests/ff03401: no basic auth credentials
I have also tried adding --profile ecrproduction to the first three commands, without success.
I am trying to set up Bitbucket pipelines to deploy to ECS as here: https://confluence.atlassian.com/bitbucket/deploy-to-amazon-ecs-892623902.html
These instructions say how to push to Docker hub, but I want to push the image to Amazon's image repo. I have set AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID in my Bitbucket parameters list and I can run these command locally with no problems (the keys defined in ~/.aws/credentials). However, I keep getting the error 'no basic auth credentials'. I am wondering if it is not recognising the variables somehow. The docs here: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html say that:
The AWS CLI uses a provider chain to look for AWS credentials in a number of different places, including system or user environment variables and local AWS configuration files. So I am not sure why it isn't working. My bitbucket pipelines configuration is as so (I have not included anything unnecessary):
- export IMAGE_NAME=$AWS_REPO_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/my/repo-name:$BITBUCKET_COMMIT
# build the Docker image (this will use the Dockerfile in the root of the repo)
- docker build -t $IMAGE_NAME .
# authenticate with the AWS repo (this gets and runs the docker login command)
- eval $(aws ecr get-login --region $AWS_DEFAULT_REGION)
# push the new Docker image to the repo
- docker push $IMAGE_NAME
Is there a way of specifying the credentials for aws ecr get-login to use? I even tried this, but it doesn't work:
- mkdir -p ~/.aws
- echo -e "[default]\n" > ~/.aws/credentials
- echo -e "aws_access_key_id = $AWS_ACCESS_KEY_ID\n" >> ~/.aws/credentials
- echo -e "aws_secret_access_key = $AWS_SECRET_ACCESS_KEY\n" >> ~/.aws/credentials
Thanks
I use an alternative method to build and push Docker images to AWS ECR that requires no environment variables:
image: amazon/aws-cli
options:
docker: true
oidc: true
aws:
oidc-role: arn:aws:iam::123456789012:role/BitBucket-ECR-Access
pipelines:
default:
- step:
name: Build and push to ECR
script:
- aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
- docker build -t 123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage:0.0.1 .
- docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage:0.0.1
You will need to update the role ARN to match a Role you have created in your AWS IAM console with sufficient permissions.
Try this:
bitbucket-pipeline.yml
pipelines:
custom:
example-image-builder:
- step:
image: python:3
script:
- export CLONE_ROOT=${BITBUCKET_CLONE_DIR}/../example
- export IMAGE_LOCATION=<ENTER IMAGE LOCATION HERE>
- export BUILD_CONTEXT=${BITBUCKET_CLONE_DIR}/build/example-image-builder/dockerfile
- pip install awscli
- aws s3 cp s3://example-deployment-bucket/deploy-keys/bitbucket-read-key .
- chmod 0400 bitbucket-read-key
- ssh-agent bash -c 'ssh-add bitbucket-read-key; git clone --depth 1 git#bitbucket.org:example.git -b master ${CLONE_ROOT}'
- cp ${CLONE_ROOT}/requirements.txt ${BUILD_CONTEXT}/requirements.txt
- eval $(aws ecr get-login --region us-east-1 --no-include-email)
- docker build --no-cache --file=${BUILD_CONTEXT}/dockerfile --build-arg AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} --build-arg AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} --tag=${IMAGE_LOCATION} ${BUILD_CONTEXT}
- docker push ${IMAGE_LOCATION}
options:
docker: true
dockerfile
FROM python:3
MAINTAINER Me <me#me.me>
COPY requirements.txt requirements.txt
ENV DEBIAN_FRONTEND noninteractive
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
RUN apt-get update && apt-get -y install stuff
ENTRYPOINT ["/bin/bash"]
I am running out of time, so for now I included more than just the answer to your question. But this would be a good enough template to work from. Ask questions in the comments if there is any line you don't understand and I will edit the answer.
i had the same issue. the error is mainly due to an old version of awscli.
you need to use a docker image with a more recent awscli.
for my project i use linkmobility/maven-awscli
You need to set the Environnment variables in Bitbucket
small changes to your pipeline
image: Docker-Image-With-awscli
eval $(aws ecr get-login --no-include-email --region ${AWS_DEFAULT_REGION} )
Im new to AWS. I want to set up a private docker repository on an AWS ECS container instance. I created a repository named name. The example push commands shown by AWS are working.
aws ecr get-login --region us-west-2
docker build -t name .
docker tag name:latest ############.dkr.ecr.us-west-2.amazonaws.com/name:latest
docker push ############.dkr.ecr.us-west-2.amazonaws.com/name:latest
But with this commands I build and pushed an image named name and I want to build an image named foo. So I altered the commands to:
docker build -t foo .
docker tag foo ###########.dkr.ecr.us-west-2.amazonaws.com/name/foo
docker push ###########.dkr.ecr.us-west-2.amazonaws.com/name/foo
This should work, but it doesn't. After a period of retrys I get the error:
The push refers to a repository [###########.dkr.ecr.us-west-2.amazonaws.com/name/foo]
8cc63cf4528f: Retrying in 1 second
...
name unknown: The repository with name 'name/foo' does not exist in the registry with id '############'
Does AWS really require a dedicated repository for every image i want to push?
The EC2 Container Registry requires an image Repository to be setup for each image "name" or "namespace/name" you want to publish to the registry.
You can publish any :tags you want in each Repository though (The default limit is 100 tags).
I haven't seen anywhere in the AWS documentation that specifically states the repository -> image name mapping but it's implied by Creating a Repository - Section 6d in the ECR User Guide
The Docker Image spec includes it's definition of a Repository
Repository
A collection of tags grouped under a common prefix (the name component before :). For example, in an image tagged with the name
my-app:3.1.4, my-app is the Repository component of the name. A
repository name is made up of slash-separated name components,
optionally prefixed by a DNS hostname. The hostname must comply with
standard DNS rules, but may not contain _ characters. If a hostname is
present, it may optionally be followed by a port number in the format
:8080. Name components may contain lowercase characters, digits, and
separators. A separator is defined as a period, one or two
underscores, or one or more dashes. A name component may not start or
end with a separator.
You need to create a repository for each image name, but the image name can be of the form "mycompanyname/helloworld". So you create mycompanyname/app1, mycompanyname/app2, etc
aws ecr create-repository --repository-name mycompanyname/helloworld
aws ecr create-repository --repository-name mycompanyname/app1
aws ecr create-repository --repository-name mycompanyname/app2
docker tag helloworld:latest xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/mycompanyname/helloworld:latest
docker push xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/mycompanyname/helloworld:latest
docker tag app1:latest xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/mycompanyname/app1:latest
docker push xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/mycompanyname/app1:latest
I tried the following steps and confirmed working for me:
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com
aws ecr create-repository --repository-name test
docker build -t test .
docker tag test:latest xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/test:latest
docker push xxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/test:latest
Addition to the above answer, I came across here today, as the login command change with aws-cli v2, posting as an answer might help others.
as aws-cli v1 login command no longer work.
V1
$(aws ecr get-login --no-include-email)
To push image to ECR using aws-cli v2 you need
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 123456789.dkr.ecr.us-west-2.amazonaws.com
Then you are okay to build and push
docker build -t myrepo .
docker tag myrepo:latest 123456789.dkr.ecr.us-west-2.amazonaws.com/myrepo
docker push 123456789.dkr.ecr.us-west-2.amazonaws.com/myrepot
Typically One image per registry is a clean approach, that why AWS increase image per repository and repository per region from 1000 to 10,000.
For this i automated the script that can read your public images from csv file and pull them. After that it will try to create repository in ECR and push to registry.
Prepare CSV file ecr-images.csv
docker.io/amazon/aws-for-fluent-bit,2.13.0
docker.io/couchdb,3.1
docker.io/bitnami/elasticsearch,7.13.1-debian-10-r0
k8s.gcr.io/kube-state-metrics/kube-state-metrics,v2.0.0
k8s.gcr.io/metrics-server-amd64,v0.3.6
--------------------KEEP THIS LINE AT END-------------------------
Automated script ecr.sh that will copy images to ecr
#!/bin/bash
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
assert_value() {
if [ -z "$1" ]; then
echo "No args: $2"
exit 1
fi
}
repository_uri=$1
assert_value "$repository_uri" "repository_uri"
create_repo() {
## try to create & failure will ignored by <|| true>
aws ecr create-repository --repository-name "$1" --output text || true
}
## Copy Docker Images to ECR
COUNTER=0
while IFS=, read -r dockerImage tag; do
outputImage=$(echo "$dockerImage" | sed -E 's/(\w+?\.)+\w+?\///')
outputImageUri="$repository_uri/$outputImage"
# shellcheck disable=SC2219
let COUNTER=COUNTER+1
echo "--------------------------------------------------------------------------"
echo "$COUNTER => $dockerImage:$tag pushing to $outputImageUri:$tag"
echo "--------------------------------------------------------------------------"
docker pull "$dockerImage:$tag"
docker tag "$dockerImage:$tag" "$outputImageUri:$tag"
create_repo "$outputImage"
docker push "$outputImageUri:$tag"
done <"$SCRIPT_DIR/ecr-images.csv"
Run
repository_uri=<ecr_account_id>.dkr.ecr.<ecr_region>.amazonaws.com
aws ecr get-login-password --region us-east-1 | \
docker login --username AWS --password-stdin $repository_uri
./ecr.sh $repository_uri
# Build an image from azure devops pipeline to aws eks
parameters:
- name: succeed
displayName: Succeed or fail
type: boolean
default: false
trigger:
- main
- releases/*
pool:
vmImage: "windows-latest"
stages:
- stage: init
jobs:
- job: init
continueOnError: false
steps:
- task: Docker#2
inputs:
containerRegistry: 'docker'
repository: 'ecr-name'
command: 'build'
Dockerfile: '**/Dockerfile'
tags: 'latest'
- task: ECRPushImage#1
inputs:
awsCredentials: 'aws credentilas'
regionName: 'us-east-1'
imageSource: 'any-name'
sourceImageName: 'ecr-name'
sourceImageTag: 'latest'
repositoryName: 'ecr-name'
pushTag: 'latest'
Create a repo per application:
aws ecr create-repository --repository-name worker --region us-east-1
aws ecr create-repository --repository-name gateway --region us-east-1
Login to registry
The AWS usr name is fixed for all registry logins
aws ecr get-login-password \
--region us-east-1 \
| docker login \
--username AWS \
--password-stdin <aws_12_digit_account_number>.dkr.ecr.us-east-1.amazonaws.com
Push image
docker build -f Dockerfile -t <123456789012>.dkr.ecr.us-east-1.amazonaws.com/worker:v1.0.0
docker push <123456789012>.dkr.ecr.us-east-1.amazonaws.com/worker:v1.0.0