name: Deploy to AWS S3
command: |
echo aws --version
if [ "${CURRENT_BRANCH}" == "main" ]
then
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://${AWS_BUCKET_PRODUCTION} --delete
elif [ "${CURRENT_BRANCH}" == "staging" ]
then
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://${AWS_BUCKET_STAGING} --delete
else
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://${AWS_BUCKET_DEV} --delete
fi
Giving on these commands no idea, I ran command on my window machine seems to run fine, giving problem on CI machine.
Stupid mistake I had misspelled REIGION as this in environment variables.
Related
I'm trying to push an image to ECR on GitLab and the docker login command keeps failing. I tried looking around online and tried 3 different variations and they all failed. Here's my gitlab-ci.yml file
image: docker:latest
services:
- docker:dind
before_script:
- apk add --update --no-cache jq py-pip
- pip install awscli
- aws ecr get-login --no-include-email --region us-west-2
- docker login -u AWS -p $(aws ecr get-login-password --region us-west-2) https://775362094965.dkr.ecr.us-west-2.amazonaws.com
#Denied: not authorized
- docker login --username AWS --password-stdin https://775362094965.dkr.ecr.us-west-2.amazonaws.com
# Error: Cannot perform an interactive login from a non TTY device
- docker login --username AWS --password-stdin public.ecr.aws/u1c1h9j4
# Error: Cannot perform an interactive login from a non TTY device
stages:
- build
- deploy
build-job:
stage: build
script:
- docker build -t $REPOSITORY_URL:$IMAGE_TAG .
- docker push $REPOSITORY_URL:$IMAGE_TAG
only:
- main
deploy-job:
stage: deploy
script:
- echo `aws ecs describe-task-definition --task-definition $CI_AWS_ECS_TASK_DEFINITION --region us-west-2` > input.json
- echo $(cat input.json | jq '.taskDefinition.containerDefinitions[].image="'$REPOSITORY_URL':'$IMAGE_TAG'"') > input.json
- echo $(cat input.json | jq '.taskDefinition') > input.json
- echo $(cat input.json | jq 'del(.taskDefinitionArn)' | jq 'del(.revision)' | jq 'del(.status)' | jq 'del(.requiresAttributes)' | jq 'del(.compatibilities)' | jq 'del(.registeredAt)' | jq 'del(.registeredBy)') > input.json
- aws ecs register-task-definition --cli-input-json file://input.json --region us-west-2
- revision=$(aws ecs describe-task-definition --task-definition $CI_AWS_ECS_TASK_DEFINITION --region us-west-2 | egrep "revision" | tr "/" " " | awk '{print $2}' | sed 's/"$//' | cut -d "," -f 1)
- aws ecs update-service --cluster $CI_AWS_ECS_CLUSTER --service $CI_AWS_ECS_SERVICE --task-definition $CI_AWS_ECS_TASK_DEFINITION:$revision --region us-west-2
I inserted all 3 docker login variations that I tried and listed the error I received in my pipeline right underneath them. I tried them all individually
aws ecr get-login-password | docker login --username AWS --password-stdin https://775362094965.dkr.ecr.us-west-2.amazonaws.com
Generally password-stdin option reads password from user input, but as you want to implement CI/CD, you can pipe the password as given above.
I found a detailed description to solve the issue #
https://medium.com/devops-with-valentine/gitlab-ci-build-push-docker-image-to-aws-ecr-elastic-container-registry-b63b91a58728
Either you have not specified in the post, but maybe your setup does not have aws-cli setup of its own ! You may need to add aws-cli image to get your pipeline working
For modern versions of AWS CLI v1 or v2:
script:
# ... install awscli
- aws sts get-caller-identity # make sure AWS CLI is properly configured
- docker login -U AWS -p `aws ecr get-login-password` --region us-west-2 https://775362094965.dkr.ecr.us-west-2.amazonaws.com
For older versions of AWS CLI v1 that don't have get-login-password:
script:
# ... install awscli
- aws sts get-caller-identity # make sure AWS CLI is properly configured
- $(aws ecr get-login --no-include-email --region us-west-2)
You must also make sure your configured identity has permission to ECR.
#Denied: not authorized
This would imply to me you might have an issue with your IAM permissions.
I'm trying to run the following command from gitlab CI:
$ aws ecr get-login-password --region eu-west-3
Here's how the job in the .gitlab-ci.yml looks like this
publish-job:
stage: publish
image:
name: amazon/aws-cli:latest
entrypoint: [""]
script:
- aws configure set aws_access_key_id MY_ACCESS_KEY_ID
- aws configure set aws_secret_access_key MY_SECRET_ACCESS_KEY
- aws configure set default.region eu-west-3
- aws ecr get-login-password --region eu-west-3
And at the last step I get the following error:
$ aws ecr get-login-password --region eu-west-3
An error occurred (UnrecognizedClientException) when calling the GetAuthorizationToken operation: The security token included in the request is invalid.
I know there's a similar question on stack overflow but I think it's not the same problem. In that question it's an issue that has to do with permissions. In my case I'm pretty sure it isn't for 2 reasons:
I gave the user associated with the access key AdministratorAccess (temporarily in order to rule out the possibility that I'm dealing with an permissions issue)
I performed the exact same steps (by copying and pasting) with docker and it works, so it's not the credentials. Here's the Dockerfile:
FROM amazon/aws-cli:latest
RUN aws configure set aws_access_key_id THE_SAME_ACCESS_KEY_ID
RUN aws configure set aws_secret_access_key THE_SAME_SECRET_ACCESS_KEY
RUN aws configure set default.region eu-west-3
RUN aws ecr get-login-password --region eu-west-3
Then I ran $ docker build --progress=plain . and the last step returned a hash
Any Idea why those steps give inconsistent results? And how to fix the CI?
I declared an AWS_DEFAULT_REGION environment variable that was preventing the cli from executing the command (even though I hardcoded the credentials at this stage). When I removed the environment variable, everything started working properly.
situation
I did success build on circle-ci and I tried deploying to AWS S3 but
something problem occurs
Goal
Build on ciecle-ci →AWS S3 → AWS cloudfront
this is my repo
error
!/bin/bash -eo pipefail
if ["${develop}"=="master"]
then
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://{AWS_BUCKET_PRODUCTION} --delete
elif ["${develop}" == "staging"]
then
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://{AWS_BUCKET_STAGING} --delete
else
aws --region ${AWS_REGION} s3 sync ~/repo/build s3://{AWS_BUCKET_DEV} --delete
fi
/bin/bash: [==master]: command not found
/bin/bash: line 3: [: missing `]'
fatal error: Parameter validation failed:
Invalid bucket name "{AWS_BUCKET_DEV}": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
Variable name
AWS_BUCKET_PRODUCTION
AWS_BUCKET_STAGING
AWS_DEV_BUCKET
I used this site for checking my name
https://regex101.com/r/2IFv8B/1
You need to put a $ in front of {AWS_BUCKET_DEV}, {AWS_BUCKET_STAGING}, and {AWS_BUCKET_PRODUCTION} respectively. Otherwise the shell doesn’t replace the variable names with their values.
... s3://${AWS_BUCKET_DEV} ...
I am trying to run the following command:
aws s3 cp --region ap-south-1 --acl public-read my.exe s3://bucket/binaries/my.exe
upload failed: ./my.exe to s3://bucket/binaries/my.exe A client error
(InvalidRequest) occurred when calling the PutObject operation: You
are attempting to operate on a bucket in a region that requires
Signature Version 4. You can fix this issue by explicitly providing
the correct region location using the --region argument, the
AWS_DEFAULT_REGION environment variable, or the region variable in the
AWS CLI configuration file. You can get the bucket's location by
running "aws s3api get-bucket-location --bucket BUCKET".
How do I fix this error? I also tried the
AWS_DEFAULT_REGION=ap-south-1 aws s3 cp --acl public-read my.exe s3://bucket/binaries/my.exe
but with no luck.
# aws --version
aws-cli/1.10.28 Python/2.7.9 Linux/3.16.0-4-amd64 botocore/1.4.19
It seems to be working after upgrading awscli.
pip install --upgrade awscli
aws --version
aws-cli/1.10.43 Python/2.7.9 Linux/3.16.0-4-amd64 botocore/1.4.33
I want to deploy a project in AWS using :
$ aws --region eu-central-1 deploy push --application-name DemoApp --s3-location s3://paquirrin-codedeploy/Project1.zip --ignore-hidden-file --source .
But I got this error:
A client error (ApplicationDoesNotExistException) occurred when calling the RegisterApplicationRevision operation: Applications not found for 289558260222
but the application exists:
$ aws deploy list-applications
{
"applications": [
"DemoApp"
]
}
and CodeDeploy agent is running
[root#ip-171-33-54-212 ~]# /etc/init.d/codedeploy-agent status
The AWS CodeDeploy agent is running as PID 2649
but I haven't found the folder deployment-root inside /opt/codedeploy-agent !
You are deploying to region eu-central-1. But you may not be listing the applications in eu-central-1 using following command:
aws deploy list-applications
Instead, use following command to ensure that application exists:
aws deploy list-applications --region eu-central-1