I am trying to push a docker image that i build in one Jenkins steps, i have read some tutorials that use
Amazon ECR plugin (but it seems is not been develop anymore/adoption, the Jenkins official AWS plugin from amazon does come with ecrLogin. but not sure how to use it.
Do i need to put this code into a script{} ?
withAWS(credentials: 'my_credentials'){
my_loging = ecrLogin()
sh 'docker --login ${my_loging}'
sh "docker push my_image_tag"
}
or just pretend that i am doing it like from my local computer
withAWS(credentials: 'my_credentials'){
sh "aws ecr get-login-password --region my_region | docker login --username AWS --password-stdin ecr_url"
sh "docker push ${docker_full_tag}"
}
The 2nd approach is what I've been using, and it works great.
Just make sure that you've properly setup AWSCLI on the user which Jenkins uses to execute it's pipeline/shell commands.
Related
when I try to push a docker image to aws ecr it fails giving the following
sudo docker push xxxxxxx.dkr.ecr.us-east-2.amazonaws.com/my-app:1.0
7d9a9c94af8d: Preparing
f77d412f54b5: Preparing
629960860aca: Preparing
f019278bad8b: Preparing
8ca4f4055a70: Preparing
3e207b409db3: Waiting
no basic auth credentials
although logging in is done successfully
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin xxxx.dkr.ecr.us-east-2.amazonaws.com
Login Succeeded
And the /home/[my user]/.docker/config.json file has the following data
{
"auths": {
"xxxx.dkr.ecr.us-east-2.amazonaws.com": {
"auth": "QVsVkhaRT...."
}
}
}
I am using aws cli version 2.3.5
aws --version
aws-cli/2.3.5 Python/3.8.8 Linux/5.8.0-63-generic exe/x86_64.ubuntu.20 prompt/off
I am using docker version 20.10.10
docker --version
Docker version 20.10.10, build b485636
How can I solve this problem?
You're running sudo docker push.
This means that the credentials in your account won't be used. Instead, Docker is trying to use (nonexistent) credentials in the root user account.
Changing your command to docker push should suffice.
I have a Docker image in AWS ECR and I open my Sagemaker Notebook instance--->go to terminal-->docker run....
This is how I start my Docker container.
Now, I want to automate this process(running my docker image on Sagemaker Notebook Instance) instead of typing the docker run commands.
Can I create a cron job on Sagemaker? or Is there any other approach?
Thanks
For this you can create an inline Bash shell in your SageMaker notebook as follows. This will take your Docker container, create the image, ECR repo if it does not exist and push the image.
%%sh
# Name of algo -> ECR
algorithm_name=your-algo-name
cd container #your directory with dockerfile and other sm components
chmod +x randomForest-Petrol/train #train file for container
chmod +x randomForest-Petrol/serve #serve file for container
account=$(aws sts get-caller-identity --query Account --output text)
# Region, defaults to us-west-2
region=$(aws configure get region)
region=${region:-us-west-2}
fullname="${account}.dkr.ecr.${region}.amazonaws.com/${algorithm_name}:latest"
# If the repository doesn't exist in ECR, create it.
aws ecr describe-repositories --repository-names "${algorithm_name}" > /dev/null 2>&1
if [ $? -ne 0 ]
then
aws ecr create-repository --repository-name "${algorithm_name}" > /dev/null
fi
# Get the login command from ECR and execute it directly
aws ecr get-login-password --region ${region}|docker login --username AWS --password-stdin ${fullname}
# Build the docker image locally with the image name and then push it to ECR
# with the full name.
docker build -t ${algorithm_name} .
docker tag ${algorithm_name} ${fullname}
docker push ${fullname}
I am contributing this on behalf of my employer, AWS. My contribution is licensed under the MIT license. See here for a more detailed explanation
https://aws-preview.aka.amazon.com/tools/stackoverflow-samples-license/
SageMaker Notebook instance lifecycle configuration script can be used to run a script when you create a notebook or at start time. In this script, you access other AWS resources from your notebook at create time or start time, say access your ECR images and automate starting docker container using a shell script. This script show an example of how you can use cron to schedule certain actions, can be modified per your usecase
Refer more lifecycle config samples in this github page
I'm trying to log in to AWS ECR with the Docker login command. I can get a password with the AWS CLI with the command aws ecr get-login-password but when piping this into the docker login command I get the following error:
Error saving credentials: error storing credentials - err: exit status 1, out: `not implemented`
The command I am running is the one recommended in the AWS ECR documentation:
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin account_id_redacted.dkr.ecr.us-east-1.amazonaws.com/blog-project
I'm running the latest version of AWS CLI as of this question, 2.0.57.
I'm running Docker version 2.4.0 on macOS 10.14.6
Has anyone else run into this issue, and if so have they found a solution?
I've definitely achieved this in the past, but I wonder if there is an issue between the latest versions of Docker and the AWS CLI...
I'm not 100% sure what the issue was here, but it was something to do with the Docker credentials helper.
I installed the Docker credentials helper for macOS, changed the credsStore parameter in ~/.docker/config.json to osxkeychain. That fixed the issues.
I had similar issue, seems like my ~/.docker/config.json was totally messed after work with multiple repos / hubs.
So I just wiped out all the content in this file leaving it empty and rerun aws ecr get-login-password | docker login ... which automatically populated config with appropriate values.
I had this issue on macOS from
.docker/config.json
remove
"credsStore" : "ecr-login"
This resolved the issue for me
if anybody has the same problem on windows then go to C:\Users folder and in the .docker folder remove the config.json file.
it might fix your problem
I believe this is the intended result (sorta). The point of using amazon-ecr-credential-helper is to not need to use docker login. You should instead configure the AWS CLI with your profile credentials (mine: myprofile). Then, you would just need to slightly modify your scripts.
For example, in ECR the AWS given steps to upload a docker image are:
Retrieve an authentication token and authenticate your Docker client
to your registry. Use the AWS CLI:
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com
Note: If you receive an error using the AWS CLI, make sure that you have the latest version of
the AWS CLI and Docker installed.
Build your Docker image using the
following command. For information on building a Docker file from
scratch see the instructions here . You can skip this step if your
image is already built:
docker build -t toy_project .
After the build completes, tag your
image so you can push the image to this repository:
docker tag toy_project:latest XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest
Run the following command to push this image to your newly created AWS
repository:
docker push XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest
However, you would want to skip step 1. The reason is that if you configured aws cli (i.e. aws configure --profile myprofile) then your credentials will be stored. So you can skip to step 2.
On the 4th step, you simply need to add AWS_PROFILE, just like below
AWS_PROFILE=myprofile docker push XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest`
With amazon-ecr-credential-helper, you no longer need to use docker login or worry about storing credentials, that is the point of amazon-ecr-credential-helper. However, this may not be the best solution for you if you need to actively use docker login in your scripts.
Note: my ~/.docker/config.json looks like
{
"credsStore": "ecr-login"
}
I was getting the same error while running this command on MacOS.
Error possibly occurred because that particular location didn't have the appropriate permissions for users read/write/execute.
Also while I was doing
% docker ps
It was giving an error as: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
What I did:
% sudo chmod 777 /var/run/docker.sock
This gave all the required permissions to that location.
Hope it would help!
I am trying to set up docker image of amazon ECR on ubuntu18.04 machine of AWS,using commands provided by view push commands of Amazon Container Services
,please note i have already set up docker on my ubuntu18.04 and also output of docker -v is as below
ubuntu#ip-172-31-0-143:~$ docker -v
Docker version 19.03.7, build 7141c199a2
When i execute the command provided by amazon container services on aws cli on ubuntu18.04 i get error as
Error: Cannot perform an interactive login from a non TTY device
The command which i am using is
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 8233251134332.dkr.ecr.us-east-2.amazonaws.com/gatling-lots
please note i have successfully configured awscli and i can see the
detailed from aws s3 ls
Here is detailed error log
ubuntu#ip-172-31-0-143:~$ aws ecr get-login-password --region us-
east-2 | docker login --username AWS --password-stdin
823443336.dkr.ecr.us-west-2.amazonaws.com/gatling-lots
usage: aws [options] <command> <subcommand> [<subcommand> ...]
[parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument operation: Invalid choice, valid choices are:
batch-check-layer-availability | batch-delete-image
batch-get-image | complete-layer-upload
create-repository | delete-lifecycle-policy
delete-repository | delete-repository-policy
describe-images | describe-repositories
get-authorization-token | get-download-url-for-layer
get-lifecycle-policy | get-lifecycle-policy-preview
get-repository-policy | initiate-layer-upload
list-images | put-image
put-lifecycle-policy | set-repository-policy
start-lifecycle-policy-preview | upload-layer-part
get-login | help
Error: Cannot perform an interactive login from a non TTY device
output of
ubuntu#ip-172-31-0-143:~$ (aws ecr get-login --no-include-email --region us-east-2)
docker login -u AWS -p
MzQxL2c0Yks4RjVxeDg9IiwidmVyc2lvbiI6IjIiLCJ0eXBlIjoiREFUQV9LRVkiLCJleHBpcmF0aW9uIjoxNTgzNjgzNDY5fQ== https://825251119036.dkr.ecr.us- east-2.amazonaws.com
The problem is not aws but docker. The solution is on docker to use the -p parameter, and wrap the aws login call to the -p parameter as such:
docker login -u AWS -p $(aws ecr get-login-password --region the-region-you-are-in) xxxxxxxxx.dkr.ecr.the-region-you-are-in.amazonaws.com
And this requires AWS CLI version 2.
docker login prints this error message when you use --password-stdin, but don't actually send a password to the command's stdin.
For example:
$ echo "" | docker login --password-stdin --username jorendorff
Error: Cannot perform an interactive login from a non TTY device
Therefore, almost any kind of problem with the command before the | pipe symbol will result in this unhelpful error message.
it took me forever to figure out that the issue was that I forgot to run aws configure and enter the right details. That solved my issue.
You need to install AWS CLI version 2.
Follow the instructions in this Installing or updating the latest version of the AWS CLI
This command does the trick in bash and linux at 2020/10/06:
linux#host:~$ $(aws ecr get-login --no-include-email)
That's because
$ aws ecr get-login --no-include-email
Gives the following output:
docker login -u AWS -p xxxxxxxxxxxxx== https://xxx.dkr.ecr.eu-west-1.amazonaws.com
Devin's answer is correct.
But there is one more way.
The updated version of docker requires this parameter --password-stdin.
aws ecr get-login-password --region <YOUR_REGION> | docker login --username AWS --password-stdin <ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com
Below steps are resolve that issue.
$curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
$aws --version
aws-cli/2.0.30 Python/3.7.3 Linux/4.14.181-142.260.amzn2.x86_64 botocore/2.0.0dev34
$aws ecr get-login-password --region your_region | docker login --username AWS --password-stdin Account_ID.dkr.ecr.your_region.amazonaws.com
Replace your Account ID and Region.
I know this question is answered already, but, this was my experience.
This didn't work for me initially.
aws ecr get-login-password --region <your-region>| docker login --username AWS --password-stdin <your-container>
I had the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY saved under variables in GitLab.
But the solution was to uncheck the Protected flag from the variables saved on GitLab. I don't know how secure this approach is, but, it did work for me.
I hope this would help someone one day.
You need to authorize your EC2 machine to access AWS services either by
running aws configure and providing the right details
OR
Give your EC2 machine a role to enable it access ECR
Also if you run your docker commands with sudo, then add sudo before the docker command as shown below
aws ecr get-login-password --region us-west-2 | sudo docker login --username AWS --password-stdin 8233251134332.dkr.ecr.us-east-2.amazonaws.com/gatling-lots
Cheers.
Also remember you cannot log into partitioned regions (cn-* or gov) while using a non-partitioned AWS profile. Add --profile foo to specify a profile with your designated region.
In my case, I forgot to add ECR related policy in my AWS IAM.
To add a policy follow these steps.
The issue I found is AWS CLI v1 vs AWS CLI v2. I resolved this by uninstalling v1 and installing AWS CLI v2.
No worries in this case. Just type 'aws configure' in your terminal and paste the security credentials such as 'aws_access_key_id' and 'aws_secret_access_key'and then type the region of the repository and the output format as 'json'.
It worked for me.
I got this error on Ubuntu 18.04 after my AWS CLI was automatically updated.
I solved it by reverting it back to the previous version using this command:
sudo apt-get install awscli=1.14.44-1ubuntu1 -V
I faced this error after re-starting Docker.
It was solved when I did docker login initially.
Then aws ecr get-login-password --region <your_region> | docker login --username AWS --password-stdin <your_uri>/<your_image> command worked again.
All of the above did not work for me on a windows OS. However, windows (10) was suggesting updates. I applied the Update & Restart and when I executed the login command
aws ecr get-login-password --region **your_region_code** | sudo docker login --username AWS --password-stdin **numeric-account-id**.dkr.ecr.**your-region-code**.amazonaws.com*
Everything worked again normally.
I had the same problem with Atlassian Bamboo, and logging into AWS ECR from an SSH task in a build plan.
I could not run aws configure because of insufficient permissions.
So I solved this by setting the AWS credential variables and then the docker login as proposed by one of the other answers:
export AWS_ACCESS_KEY_ID=<value>
export AWS_SECRET_ACCESS_KEY=<value>
docker login -u AWS -p $(aws ecr get-login-password --region <region>) <accountid>.dkr.ecr.<region>.amazonaws.com
The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY can be created in your AWS profile, Security Credentials section.
Hope this helps someone, and a future me when I forget and come back to find help.
This answer is for similar error getting for github actions. Please try this and let me know if this works
- name: Docker login
uses: docker/login-action#v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}```
From OSX, I'm just following the guide you can find here (detailed repro steps below): https://console.aws.amazon.com/ecs/home?region=us-east-1#/repositories/create
aws-cli/1.10.24 Python/2.7.10 Darwin/15.4.0 botocore/1.4.15
Docker version 1.11.1, build 5604cbe (The new beta)
Everything goes fine until the push command, which fails with:
no basic auth credentials
I saw someone w/ the same error here: Can't push image to Amazon ECR - fails with "no basic auth credentials" Unfortunately, my issue doesn't seem to have to do with mismatched access keys. I've reset them several times to check.
Here are the steps I've taken:
➜ eval $(aws ecr get-login --region us-east-1)
Warning: '-e' is deprecated, it will be removed soon. See usage.
Login Succeeded
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker build -t testing .
Sending build context to Docker daemon 38.91 kB
Step 1 : FROM wordpress:latest
---> 81aa77247862
...etc...
Removing intermediate container c5849505c95d
Successfully built 63b304c8227a
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker tag testing:latest MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker push MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
The push refers to a repository [MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing]
d80ff78e2dbe: Preparing
...etc...
23b28a7c4771: Waiting
6eb35183d3b8: Waiting
no basic auth credentials
To be able to push to ecr, you need to log into the ecr docker repo.
For that you will need to setup ur access key (with privilege to ecr) using aws configure.
and then execute
eval $(aws ecr get-login --region us-east-1)
once successful, you should see
Login Succeeded
As a complement to Shibashis, you can try adding -no-include-email:
eval $(aws ecr get-login --no-include-email | sed 's|https://||')
For more clarity,
Before using the push command did you do docker login to AWS from your terminal?
If you are wondering how you can get the login cmd, did you notice that AWS itself generates this command by aws ecr get-login this command?
Do docker login -u AWS -p <hashpassword-from-aws-ecr-cmd>
and do
docker push <ecr-repo-url>
Cheers!