Docker unable to connect AWS EC2 cloud - amazon-web-services

Hi I am able to deploy my spring boot application in my local docker container(1.11.2) in Windows-7.I followed the below steps to run the docker image in AWS EC2(Free Account:eu-central-1) but getting error
Step 1
Generated Amazon "AccessKeyID" and "SecretKey".
Step 2
Created new repository and it shows 5 Steps to push my docker image in AWS EC2.
Step 3
Installed Amazon CLI and run "aws configure" and configured all the details.
While running aws iam list-users --output table it shows all the user list
Step 4
Run the following command in Docker Container aws ecr get-login --region us-west-2
It returns the docker login.
While running the docker login it returns the following error :
XXXX#XXXX MINGW64 ~
$ docker login -u AWS -p <accessKey>/<secretKey>
Uwg
Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized:
incorrect username or password
XXXX#XXXX MINGW64 ~
$ gLBBgkqhkiG9w0BBwagggKyMIICrgIBADCCAqcGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQME8
Zei
bash: gLBBgkqhkiG9w0BBwagggKyMIICrgIBADCCAqcGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQ
ME8Zei: command not found
XXXX#XXXX MINGW64 ~
$ lJnpBND9CwzAgEQgIICeLBms72Gl3TeabEXDx+YkK9ZlbyGxPmsuVI/rq81tDeIC68e0Ma+ghg3Dt
Bus
bash: lJnpBND9CwzAgEQgIICeLBms72Gl3TeabEXDx+YkK9ZlbyGxPmsuVI/rq81tDeIC68e0Ma+ghg
3DtBus: No such file or directory
I didn't get proper answer in google.It would be great if some one guide me to resolve this issue.Thanks in advance.

Your command is not pointing to your ECR endpoint, but to DockerHub. Using Linux, normally I would simply run:
$ eval $(aws ecr get-login --region us-west-2)
This is possible because the get-login command is a wrapper that retrieves a new authorization token and formats the docker login command. You only need to execute the formatted command (in this case with eval)
But if you really want to run the docker login manually, you'll have to specify the authorization token and the endpoint of your repository:
$ docker login -u AWS -p <password> -e none https://<aws_account_id>.dkr.ecr.<region>.amazonaws.com
Where <password> is actually the authorization token (which can be generated by the aws ecr get-authorization-token command).
Please refer to the documentation for more details: http://docs.aws.amazon.com/cli/latest/reference/ecr/index.html

Related

Different result after run command between shell script and terminal

Situation
Use shell script to push docker images (Nodejs app) to ECR
Command
aws ecr get-login-password --region ap-southeast-1 | docker login --username AWS --password-stdin xxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com
File shell script
#!/bin/sh
docker build -t abc/abc-api ../../abc/abc-api
docker build -t abc/abc-fe ../../abc/abc-fe
aws ecr get-login-password --region ap-southeast-1 | docker login --username AWS --password-stdin xxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com
docker tag abc/abc-api xxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/abc:api-latest
docker push xxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/abc:api-latest
docker tag abc/abc-fe xxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/abc:fe-latest
docker push xxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/abc:fe-latest
Current
When run file shell script
Unable to locate credentials. You can configure credentials by running "aws configure".
Error: Cannot perform an interactive login from a non TTY device
When run command on terminal
WARNING! Your password will be stored unencrypted in /home/***/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
What I want
Shell script login succeeded
Environment
Linux mint 20
aws-cli 2.0.46
Configured aws authentication by aws configure
Based on the comments.
The reason why the script does not work is that it is executed under root user. The root user does not have aws profile defined, which explain why the script errors out with "You can configure credentials by running "aws configure" message.
To rectify the issue, there are some posibilites:
Don't run the script under root user. Run it as your own user, because you have aws profile configure for yourself (preferred).
Create new aws profile for the root user.

`Authorization Token has expired` issue AWS-CLI on MacOS Sierra

I'm trying to push a docker image to the AWS ECR repository using the aws-cli.
I just run the get-login command
execute the output (which returns login succeeded)
then try to push a docker image then I get the
message:
denied: Your Authorization Token has expired. Please run
'aws ecr get-login' to fetch a new one.
I don't know whats going wrong, I'm pushing to the right repo, the time on my mac is correct.
This was working before, but since I reinstalled my mac and upgraded to macOS Sierra it's not working anymore, so probably related to that.
My aws --version output:
aws-cli/1.11.34 Python/2.7.10 Darwin/16.3.0 botocore/1.4.91
The complete output of the commands I run:
$ aws ecr get-login --region eu-west-1
docker login -u AWS -p AQECAHh....b6Wk -e none https://1234567890.dkr.ecr.eu-west-1.amazonaws.com
$ docker login -u AWS -p AQECAHh....b6Wk -e none https://1234567890.dkr.ecr.eu-west-1.amazonaws.com
Flag --email has been deprecated, will be removed in 1.13.
Login Succeeded
$ docker push 1234567890.dkr.ecr.eu-west-1.amazonaws.com/service-web:latest
The push refers to a repository [1234567890.dkr.ecr.eu-west-1.amazonaws.com/service-web]
c1f87971dfa9: Preparing
2eb644aea3de: Preparing
9c8843ffe48e: Preparing
39bb58d049d4: Preparing
f053bc969599: Preparing
7169084246b8: Waiting
bb134a1936fd: Waiting
184e76848a1c: Waiting
75c8fcf65748: Waiting
eb9b9ee1ea58: Waiting
f4bf35723edd: Waiting
ddffe1a64b3c: Waiting
fd1a1154db16: Waiting
b542e946067a: Waiting
d49ed2a5e1ed: Waiting
bb39b980367a: Waiting
25b8358d062f: Waiting
997eee521fc7: Waiting
50b5447183a8: Waiting
4339b5cb0e1d: Waiting
3dbd4a53b21b: Waiting
2bec16216500: Waiting
b9fd8e264df6: Waiting
b6ca02dfe5e6: Waiting
denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.
Neither of solutions above worked for my but I found that when I set region in ecr login command it worked.
aws ecr get-login --region us-west-2
You might just be running the command and not pasting the command that is echo'd out from that command back into the terminal. Easy mistake to make. Once you run:
aws ecr get-login --no-include-email --region us-east-1
It will print out another command to run, you'll need to copy that command and run it in your terminal to authenticate fully.
Or a cool shortcut is to just pipe the echo'd command back into the shell with:
aws ecr get-login --no-include-email --region us-east-1 | sh
Latest versions of Docker use a new credentials storage feature which has a bug where doing a docker login with a URL that specifies a protocol will result in token expiration errors. This issue will be fixed in Docker 1.13.
For the time being, the workaround is to execute your login commands without specifying the protocol.
So in the command blob returned by aws ecr get-login:
docker login -u AWS -p AQECAHh....b6Wk -e none https://1234567890.dkr.ecr.eu-west-1.amazonaws.com
Should be replaced with this:
docker login -u AWS -p AQECAHh....b6Wk -e none 1234567890.dkr.ecr.eu-west-1.amazonaws.com
Omitting the https://should make docker work for the time being.
This answer worked for me using the AWS CLI v2.0.26
https://github.com/aws/aws-cli/issues/4962#issuecomment-592064025
aws --region us-west-2 ecr get-login-password | docker login --username AWS --password-stdin xxxxxxxxxxxxxx.dkr.ecr.us-west-2.amazonaws.com
where us-west-2 is your region and the xxxxxxxxxxxxxx is your account ID found at the beginning of the line below "Repository Name" here: https://us-west-2.console.aws.amazon.com/ecr/create-repository?region=us-west-2
You need to refresh your authorization token every 12 hours try:
$(aws ecr get-login --no-include-email --region us-east-1) - change region according to your configuration
Simple Command:
password=$(aws ecr get-login-password --region us-east-1)
echo $password | docker login --username AWS --password-stdin 787566098823.dkr.ecr.us-east-1.amazonaws.com
Just had the same issue on Linux Mint 18.1 (Ubuntu 16.04) with AWS ECR and latest Docker 17.06.1-ce used via latest Python Docker client 2.5.1. Login worked, push failed.
Removal of ~/.docker/config.json helped. It only contained, probably stale, authorisation token.
I don't think it has something to do with underlying OS. In my case it worked previously and the only change I can recall was upgrade from Ubuntu repo's docker.io 1.12 to Docker repo's docker-ce 17.06.
You get also the message "Your Authorization Token has expired" if you have more than one credentials in ~/.aws/credentials (path depending on your os) and forget to add the --p flag.
Use this command to get login:
aws ecr get-login --region eu-west-1 -p <yourprofilename>
I've had luck using eval. For example,
$ aws ecr get-login --region us-east-1 --no-include-email --profile username_env
Didn't work.
$ eval $(aws ecr get-login --region us-east-1 --no-include-email --profile username_env)
Did work.
The following steps worked for me. First, run
aws ecr get-login --region us-west-2
You will get an output which returns:
docker login -u AWS -p AQECAHh....b6Wk -e none 1234567890.dkr.ecr.eu-west-1.amazonaws.com
Now, remove "-e none" from the above result and run the command again.
You will be able to login successfully.
Now, try pushing your docker image and it will work!
In my case the issue was multiple credentials in ~/.aws/credentails so I used --profile
aws ecr get-login --no-include-email --region us-east-2 --profile xxxx
This worked for me.
I just wanted to post the official migration link as I believe it'll be most up to date if things change:
https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html#cliv2-migration-ecr-get-login
It states
$(aws ecr get-login --no-include-email)
should be replaced by
aws ecr get-login-password | docker login --username AWS --password-stdin MY-REGISTRY-URL
This is due to potential password exposure in the CLI. It's worth mentioning you can migrate to the new method from CLI version 1.17.10 for a smooth migration to 2.X
I was also getting the same error, below is the solution I have tried and it is working:
1. Run command:
aws ecr get-login --no-include-email --region ap-southeast-1 (change region as per your repository)
2. you will get output something like:
docker login -u AWS -p xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx== https://youraccountid.dkr.ecr.ap-southeast-1.amazonaws.com
Remove "https://" and then run the command as
docker login -u AWS -p xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx== youraccountid.dkr.ecr.ap-southeast-1.amazonaws.com
And it will work and you will be able to push the image.
This happened when I was trying to push/pull from a registry in another AWS account. I needed to run get-login with the --registry-ids flag, passing in the ID of the registry I wanted to log into.
Most of the above solutions won't be working if you aws-cli/2.0.0
For me, I have aws-cli/2.0.0 Python/3.8.1 Darwin/19.4.0 botocore/2.0.0dev4
What worked for was to do re-login.
If you are on aws-cli/2.0.0 then the following might work for you as well.
aws ecr get-login-password |docker login --username AWS --password-stdin $IMAGE_PATH
I faced the same issue when I tryed to push one of my App docker image to ECR. I was able to solve it by applying the following steps:
Generate access keys and secret keys to make programmatic calls to
AWS from the AWS CL. You can generate access keys and secret keys from Identity and Access Management(IAM). Store those keys for future use.
Run aws configure in your console utilizing those access keys and
secret keys also provide the correct region.
Run the following command to retrieve an authentication token and
authenticate your Docker client to your registry.
aws ecr get-login-password --region ap-south-1 | docker login --username AWS --password-stdin #####.dkr.ecr.ap-south-1.amazonaws.com
Now build and push the docker image to ECR
I was using the stable version of docker for mac Version 1.12
I just upgraded to the beta version Version 1.13.0-rc4-beta34.1 (14853)
and now it all works as intended.
So if there are people with the same issue, make sure you use docker for mac version 1.13 or higher, if 1.13 isn't released yet, switch to the beta version.
This is the current format I believe, assuming you're trying to get access for Docker:
aws ecr get-login-password \
--region REGION \
| docker login \
--username AWS \
--password-stdin ACCESS_ID.dkr.ecr.REGION.amazonaws.com
I know the post is related to MacOS Sierra, but for those who have the problem on Windows, I performed the following:
1) aws ecr get-login, this command will output a long string
docker login -u AWS -p eyJwYXlsb2FkIjoiUXBnQ2FTV1B6Q1JqZGlH......(Omitted the whole line for better understanding) -e none https://xxxxxxx.dkr.ecr.us-east-1.amazonaws.com.
2) Copy and paste the above line (perhaps -e none won't work, so remove it too). The output will show a warning followed by a success:
WARNING! Using --password via the CLI is insecure. Use --password-stdin
Login Succeeded
If you need to use a secure way, use the --password-stdin
3) Now you can safely push the image
-docker push xxxxxxx.dkr.ecr.us-east-1.amazonaws.com/ecfs-test
0429f33dd264: Pushed
48accfb13167: Pushed
f3bb6dd29c05: Pushed
e58ae65fa4eb: Pushed
3c6037fae296: Pushed
3efd1f7c01f6: Pushed
73b4683e66e8: Pushed
ee60293db08f: Pushed
9dc188d975fd: Pushed
58bcc73dcf40: Pushed
latest: digest: sha256:4354d137733c98a1bc8609d2d2f8e97316373904e size: 2404
Maybe this solution will work on Mac too.
The problem is because the aws ecr get-login command retrieves a token that is valid for a specified registry for 12 hours, and then it prints a docker login command with that authorization token and we are not executing that command that we get back.
We need to execute this printed command to log in to your registry with Docker. In my case , I am using eval to execute the printed command that I get back from the aws ecr get-login like this:
eval $(aws ecr get-login --region eu-west-1 --profile )
This issue usually happens when you take a lot of time without accessing your CLI terminal. For this reason when you come back to your CLI, you need to login again.
For your case MacOs/Linux, Please use the following command to establish a fresh login session.
aws ecr get-login-password --region [Your Region] | sudo docker login --username AWS --password-stdin [IAM User Id].dkr.ecr.[Your Region].amazonaws.com
Please replace the placeholders with your relevant values.
I did this and it works:
first, run this command:
aws configure
in order to obtain your
Access key ID:
and
Secret access key:
2- Go to IAM->Users->"your user"->Security credentials-> Create Access Key
and chose your region
then click enter
now run this command again
aws ecr get-login-password | docker login --username AWS --password-stdin `Your repositoryUri`
When performing an unauthenticated pull from an Amazon ECR Public repository, you receive an authentication token expired response. This is likely due to the fact that you've previously requested an authentication token from Amazon ECR Public and that token has expired. When the new Amazon ECR Public image pull is performed, the expired token is used and the error is received.
To resolve this, log your Docker CLI out of the Amazon ECR Public registry and re-attempt your unauthenticated image pull like:
docker logout public.ecr.aws
https://docs.aws.amazon.com/AmazonECR/latest/public/public-troubleshooting.html
A warning: aws ecr get-login does not appear to connect to AWS servers and appears to work even if you have bad AWS access/secret keys or even if you have forgotten to enter your AWS access/secret keys as environmental variables.
It will still happily give you a long password without providing an error. The message, then, you get from AWS is an expiration error instead of a more correct and helpful "authorization incorrect."
Note: Using aws-cli version 1.11.112.
Another solution variant for this particular error is a missing --registry-ids argument to the aws ecr get-login invocation.
The full get-login invocation would be something like:
eval "$(aws ecr get-login --no-include-email \
--region us-east-1 \
--registry-ids 11223344 \
)"
Please substitute your own region and registry ID values.
The question mentions that login had succeeded but docker push had failed.
The two possible reasons for the above condition are:
The AWS credentials are expired. Go to the AWS console or use aws-cli to generate a new pair. Store them in the environment or in ~/.aws/credentials file.
You might be using the wrong AWS credentials from a different account. Temporarily set AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY, and AWS_REGION with credentials of account where ECR repository exists.
ECR repositories which are associated with an account works only with those account's credentials
Always make sure which AWS credentials are being used for the operation.
Check environment variables and ~/.aws/credentials to confirm it.
This is what worked for me. I was using Docker for Windows. The problem appeared to be with the docker configuration. In particular with how the credentials were stored. If you look in ~/.docker/config.json, it might look something like this:
{
"auths": {
"XXXX.dkr.ecr.us-east-1.amazonaws.com": {}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (windows)"
},
"credsStore": "desktop",
"stackOrchestrator": "swarm"
}
if you delete credStore line and try login in again with
docker login -u AWS -p "XXX...the really long password ehre..XXX" https://XXXX.dkr.ecr.us-east-1.amazonaws.com
, you will should see something like this
{
"auths": {
"XXXX.dkr.ecr.us-east-1.amazonaws.com": {
"auth": "XXX...the really long password ehre..XXX"
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (windows)"
},
"stackOrchestrator": "swarm"
}
Annoyingly, I have to do this each time, as docker adds the credStore line back in again
I was getting this error because I have multiple profiles. The profile flagged solved it for me:
$(aws ecr get-login --no-include-email --region us-west-2 --profile xxxx)
In my case the bellow script worked for aws version aws-cli/2.0.8
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ${aws_account_id}.dkr.ecr.${region}.amazonaws.com
aws ecr get-login seems not to be supported anymore.
I had to use get-login-password instead:
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <ACCESS_ID>.dkr.ecr.<REGION>.amazonaws.com

How to pull a docker template image on a Jenkins slave (AWS/ECR)?

In our current setup we have the following Jenkins config.
Jenkins Master <-- ssh --> Jenkins Slave
So the Jenkins Master is able to connect successfully to the slave. I would like to provide a way so that the slave gets a docker image so we can build using a prebuilt Docker slave. When building the Docker slave locally i can use it, but i seem to hit a wall when i want to pull this Docker build slave from an AWS ECR repository. I seem to be unable to find a way to provide the credentials.
We are using the AWS ECR plugin but this not help in providing details for the ECR pull. (See post http://getmesh.io/Blog/Jenkins+2+Pipeline+101)
Any clue where i can configure the AWS ECR credentials so the Docker Template can be pulled?
As far as I am aware your Jenkins docker slave server should have awscli installed with a valid AWS secret and key. Once this is completed you can run below command on the Jenkins docker slave server to authenticate:
aws ecr get-login --region YOUR_REGION --no-include-email | xargs -n 1 -P 10 -I {} bash -c {}
The command will take the output from the awscli and login on the AWS ECR.
As AWS ECR token expires every 12 hours I have added a cronjob to renew the token every 6 hours.
0 */6 * * * aws ecr get-login --region YOUR_REGION --no-include-email | xargs -n 1 -P 10 -I {} bash -c {}
Or as an alternative you can create an internal AWS ECR anonymous proxy from where everyone on your organisation can pull containers. Check this project for more details

Can't push an image to ECS Private Registry - no basic auth credentials

From OSX, I'm just following the guide you can find here (detailed repro steps below): https://console.aws.amazon.com/ecs/home?region=us-east-1#/repositories/create
aws-cli/1.10.24 Python/2.7.10 Darwin/15.4.0 botocore/1.4.15
Docker version 1.11.1, build 5604cbe (The new beta)
Everything goes fine until the push command, which fails with:
no basic auth credentials
I saw someone w/ the same error here: Can't push image to Amazon ECR - fails with "no basic auth credentials" Unfortunately, my issue doesn't seem to have to do with mismatched access keys. I've reset them several times to check.
Here are the steps I've taken:
➜ eval $(aws ecr get-login --region us-east-1)
Warning: '-e' is deprecated, it will be removed soon. See usage.
Login Succeeded
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker build -t testing .
Sending build context to Docker daemon 38.91 kB
Step 1 : FROM wordpress:latest
---> 81aa77247862
...etc...
Removing intermediate container c5849505c95d
Successfully built 63b304c8227a
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker tag testing:latest MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker push MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
The push refers to a repository [MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing]
d80ff78e2dbe: Preparing
...etc...
23b28a7c4771: Waiting
6eb35183d3b8: Waiting
no basic auth credentials
To be able to push to ecr, you need to log into the ecr docker repo.
For that you will need to setup ur access key (with privilege to ecr) using aws configure.
and then execute
eval $(aws ecr get-login --region us-east-1)
once successful, you should see
Login Succeeded
As a complement to Shibashis, you can try adding -no-include-email:
eval $(aws ecr get-login --no-include-email | sed 's|https://||')
For more clarity,
Before using the push command did you do docker login to AWS from your terminal?
If you are wondering how you can get the login cmd, did you notice that AWS itself generates this command by aws ecr get-login this command?
Do docker login -u AWS -p <hashpassword-from-aws-ecr-cmd>
and do
docker push <ecr-repo-url>
Cheers!

Docker Login for AWS ECR failing with "Cannot connect to the Docker deamon..."

I'm having trouble getting started with ECR because the login command generated by
aws ecr get-login
returns:
Warning: '-e' is deprecated, it will be removed soon. See usage.
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
Unfortunately, the documentation for ECR doesn't acknowledge this as possible issue. Has anyone here experienced this before and if so what did you do to fix it? For reference I've setup AWS CLI with an IAM user that has policies for full EC2/ECR permissions attached. I've also created a permissions statement on the ECR registry that allows my IAM user full rights.
Docker service is not running on your machine.
You will be able to login/connect to ECR, only when the docker process is running.
You need to add --no-include-email, like:
$ aws ecr get-login --no-include-email --region us-east-1
As i updgraded Docker to version 17.06.2-ce, this deprecation notice moved to be an error :
unknown shorthand flag: 'e' in -e
See 'docker login --help'.
The cause is the "aws ecr get-login" command returing an invalid parameter ("-e none"). Before this docker version, it was a warning / depreciation error, now docker failed with a return code of 125.
To prevent this, I log on ECR with this command :
$> $(aws ecr get-login | sed -e "s/-e none//g")