When I try to run sagemaker locally for tensorflow in script mode. It seems like I cannot pull the docker container. I have ran the code below from a sagemaker notebook instance and everything ran fine. But when running it on my machine it doesn't work.
How can I download the container, so I can debug things locally?
import os
import sagemaker
from sagemaker.tensorflow import TensorFlow
hyperparameters = {}
role = 'arn:aws:iam::xxxxxxxx:role/yyyyyyy'
estimator = TensorFlow(
entry_point='train.py',
source_dir='.',
train_instance_type='local',
train_instance_count=1,
hyperparameters=hyperparameters,
role=role,
py_version='py3',
framework_version='1.12.0',
script_mode=True)
estimator.fit()
I get this output
INFO:sagemaker:Creating training-job with name: sagemaker-tensorflow-
scriptmode-2019-01-28-18-51-57-787
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: pull access denied for 520713654638.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-tensorflow-scriptmode, repository does not exist or may require 'docker login'
subprocess.CalledProcessError: Command 'docker pull 520713654638.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-tensorflow-scriptmode:1.12.0-cpu-py3' returned non-zero exit status 1.
The warning looks like the output you get when using the docker login stuff here. If I follow these steps to register to the directory with tensorflow container it says login success
Invoke-Expression -Command (aws ecr get-login --no-include-email --registry-ids 520713654638 --region eu-west-2)
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded
But then I still cannot pull it
docker pull 520713654638.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-tensorflow-scriptmode:1.11.0-cpu-py3
Error response from daemon: pull access denied for 520713654638.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-tensorflow-scriptmode, repository does not exist or may require 'docker login'
the same sequence works for me locally : 'aws ecr get-login', 'docker login', 'docker pull'.
Does your local IAM user have sufficient credentials to pull from ECR? The 'AmazonEC2ContainerRegistryReadOnly' policy should be enough: https://docs.aws.amazon.com/AmazonECR/latest/userguide/ecr_managed_policies.html
Alternatively, you can grab the container from Github and build it: https://github.com/aws/sagemaker-tensorflow-container
Related
I'm trying to run a Docker container on AWS Lambda.
Specifically I'm following this official tutorial
I have the following Dockerfile
FROM public.ecr.aws/lambda/nodejs:12
COPY app app.js package.json /var/task/
RUN npm install
CMD [ "app.handler" ]
But, when I try to build this I get the following error:
docker build -t hello-world .
Sending build context to Docker daemon 4.608kB
Step 1/4 : FROM public.ecr.aws/lambda/nodejs:12
pull access denied for public.ecr.aws/lambda/nodejs, repository does not exist or may require 'docker login': denied: Your authorization token has expired. Reauthenticate and try again.
When I login with docker login command I still get the same error.
Does anyone know how to resolve this?
Works fine here. You shouldn't need credentials for Public ECR (you can use auth for specific cases) but if you just want to consume it, remove the existing credentials
docker logout public.ecr.aws
and then try the build again.
That said, if you still want to make use of the authentication, you need to re-auth as described in the doc
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
I'm trying to log in to AWS ECR with the Docker login command. I can get a password with the AWS CLI with the command aws ecr get-login-password but when piping this into the docker login command I get the following error:
Error saving credentials: error storing credentials - err: exit status 1, out: `not implemented`
The command I am running is the one recommended in the AWS ECR documentation:
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin account_id_redacted.dkr.ecr.us-east-1.amazonaws.com/blog-project
I'm running the latest version of AWS CLI as of this question, 2.0.57.
I'm running Docker version 2.4.0 on macOS 10.14.6
Has anyone else run into this issue, and if so have they found a solution?
I've definitely achieved this in the past, but I wonder if there is an issue between the latest versions of Docker and the AWS CLI...
I'm not 100% sure what the issue was here, but it was something to do with the Docker credentials helper.
I installed the Docker credentials helper for macOS, changed the credsStore parameter in ~/.docker/config.json to osxkeychain. That fixed the issues.
I had similar issue, seems like my ~/.docker/config.json was totally messed after work with multiple repos / hubs.
So I just wiped out all the content in this file leaving it empty and rerun aws ecr get-login-password | docker login ... which automatically populated config with appropriate values.
I had this issue on macOS from
.docker/config.json
remove
"credsStore" : "ecr-login"
This resolved the issue for me
if anybody has the same problem on windows then go to C:\Users folder and in the .docker folder remove the config.json file.
it might fix your problem
I believe this is the intended result (sorta). The point of using amazon-ecr-credential-helper is to not need to use docker login. You should instead configure the AWS CLI with your profile credentials (mine: myprofile). Then, you would just need to slightly modify your scripts.
For example, in ECR the AWS given steps to upload a docker image are:
Retrieve an authentication token and authenticate your Docker client
to your registry. Use the AWS CLI:
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com
Note: If you receive an error using the AWS CLI, make sure that you have the latest version of
the AWS CLI and Docker installed.
Build your Docker image using the
following command. For information on building a Docker file from
scratch see the instructions here . You can skip this step if your
image is already built:
docker build -t toy_project .
After the build completes, tag your
image so you can push the image to this repository:
docker tag toy_project:latest XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest
Run the following command to push this image to your newly created AWS
repository:
docker push XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest
However, you would want to skip step 1. The reason is that if you configured aws cli (i.e. aws configure --profile myprofile) then your credentials will be stored. So you can skip to step 2.
On the 4th step, you simply need to add AWS_PROFILE, just like below
AWS_PROFILE=myprofile docker push XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/toy_project:latest`
With amazon-ecr-credential-helper, you no longer need to use docker login or worry about storing credentials, that is the point of amazon-ecr-credential-helper. However, this may not be the best solution for you if you need to actively use docker login in your scripts.
Note: my ~/.docker/config.json looks like
{
"credsStore": "ecr-login"
}
I was getting the same error while running this command on MacOS.
Error possibly occurred because that particular location didn't have the appropriate permissions for users read/write/execute.
Also while I was doing
% docker ps
It was giving an error as: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
What I did:
% sudo chmod 777 /var/run/docker.sock
This gave all the required permissions to that location.
Hope it would help!
I have a line in my Dockerfile like this:
FROM 6*********.dkr.ecr.ap-southeast-1.amazonaws.com/*************:ff03401
This ECR is owned by another user.
As recommended in this question, I am trying to log in by using these commands in the build section of my buildspec.yml, and then immediately pull this docker image:
- aws configure set aws_access_key_id $ECR_ACCESS_KEY
- aws configure set aws_secret_access_key $ECR_SECRET_KEY
- eval aws ecr get-login --no-include-email --region ap-southeast-1 --registry-ids 6***********
- docker pull 6***********.dkr.ecr.ap-southeast-1.amazonaws.com/****************:ff03401
When I look at the Codebuild logs, I see that eval aws ecr get-login... outputs a docker login ... command which, if I run it on my local machine, logs me in successfully, and lets me do the docker pull 6******....
In Codebuild, however, docker pull says:
Error response from daemon: Get https://6**********.dkr.ecr.ap-southeast-1.amazonaws.com/v2/******************/manifests/ff03401: no basic auth credentials
I have also tried adding --profile ecrproduction to the first three commands, without success.
Hi I am able to deploy my spring boot application in my local docker container(1.11.2) in Windows-7.I followed the below steps to run the docker image in AWS EC2(Free Account:eu-central-1) but getting error
Step 1
Generated Amazon "AccessKeyID" and "SecretKey".
Step 2
Created new repository and it shows 5 Steps to push my docker image in AWS EC2.
Step 3
Installed Amazon CLI and run "aws configure" and configured all the details.
While running aws iam list-users --output table it shows all the user list
Step 4
Run the following command in Docker Container aws ecr get-login --region us-west-2
It returns the docker login.
While running the docker login it returns the following error :
XXXX#XXXX MINGW64 ~
$ docker login -u AWS -p <accessKey>/<secretKey>
Uwg
Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized:
incorrect username or password
XXXX#XXXX MINGW64 ~
$ gLBBgkqhkiG9w0BBwagggKyMIICrgIBADCCAqcGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQME8
Zei
bash: gLBBgkqhkiG9w0BBwagggKyMIICrgIBADCCAqcGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQ
ME8Zei: command not found
XXXX#XXXX MINGW64 ~
$ lJnpBND9CwzAgEQgIICeLBms72Gl3TeabEXDx+YkK9ZlbyGxPmsuVI/rq81tDeIC68e0Ma+ghg3Dt
Bus
bash: lJnpBND9CwzAgEQgIICeLBms72Gl3TeabEXDx+YkK9ZlbyGxPmsuVI/rq81tDeIC68e0Ma+ghg
3DtBus: No such file or directory
I didn't get proper answer in google.It would be great if some one guide me to resolve this issue.Thanks in advance.
Your command is not pointing to your ECR endpoint, but to DockerHub. Using Linux, normally I would simply run:
$ eval $(aws ecr get-login --region us-west-2)
This is possible because the get-login command is a wrapper that retrieves a new authorization token and formats the docker login command. You only need to execute the formatted command (in this case with eval)
But if you really want to run the docker login manually, you'll have to specify the authorization token and the endpoint of your repository:
$ docker login -u AWS -p <password> -e none https://<aws_account_id>.dkr.ecr.<region>.amazonaws.com
Where <password> is actually the authorization token (which can be generated by the aws ecr get-authorization-token command).
Please refer to the documentation for more details: http://docs.aws.amazon.com/cli/latest/reference/ecr/index.html
From OSX, I'm just following the guide you can find here (detailed repro steps below): https://console.aws.amazon.com/ecs/home?region=us-east-1#/repositories/create
aws-cli/1.10.24 Python/2.7.10 Darwin/15.4.0 botocore/1.4.15
Docker version 1.11.1, build 5604cbe (The new beta)
Everything goes fine until the push command, which fails with:
no basic auth credentials
I saw someone w/ the same error here: Can't push image to Amazon ECR - fails with "no basic auth credentials" Unfortunately, my issue doesn't seem to have to do with mismatched access keys. I've reset them several times to check.
Here are the steps I've taken:
➜ eval $(aws ecr get-login --region us-east-1)
Warning: '-e' is deprecated, it will be removed soon. See usage.
Login Succeeded
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker build -t testing .
Sending build context to Docker daemon 38.91 kB
Step 1 : FROM wordpress:latest
---> 81aa77247862
...etc...
Removing intermediate container c5849505c95d
Successfully built 63b304c8227a
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker tag testing:latest MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
~/projects/pw/docker/aws-wordpress ‹master ✗› (ruby-2.3.1) (5.11.0) ()
➜ docker push MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing:latest
The push refers to a repository [MYACCOUNT.dkr.ecr.us-east-1.amazonaws.com/testing]
d80ff78e2dbe: Preparing
...etc...
23b28a7c4771: Waiting
6eb35183d3b8: Waiting
no basic auth credentials
To be able to push to ecr, you need to log into the ecr docker repo.
For that you will need to setup ur access key (with privilege to ecr) using aws configure.
and then execute
eval $(aws ecr get-login --region us-east-1)
once successful, you should see
Login Succeeded
As a complement to Shibashis, you can try adding -no-include-email:
eval $(aws ecr get-login --no-include-email | sed 's|https://||')
For more clarity,
Before using the push command did you do docker login to AWS from your terminal?
If you are wondering how you can get the login cmd, did you notice that AWS itself generates this command by aws ecr get-login this command?
Do docker login -u AWS -p <hashpassword-from-aws-ecr-cmd>
and do
docker push <ecr-repo-url>
Cheers!