I am trying to deploy a Windows container to AWS ECR using Gitlab CI:
Here is the Gitlab yaml file:
variables:
AWS_REGISTRY: ****************.amazonaws.com/devops
AWS_DEFAULT_REGION: *****
APP_NAME: devops
windows:
stage: build
tags:
- prod
before_script:
./docker_install.sh > /dev/null
script:
- docker build -t ${AWS_REGISTRY}/${CI_PROJECT_PATH}
- docker push ${AWS_REGISTRY}/${CI_PROJECT_PATH}
Docker file is
FROM mcr.microsoft.com/windows/servercore:ltsc2019
CMD [ "cmd" ]
Error is :
Running with gitlab-runner 13.8.0 (*****)
on *********-aws-gitlab-runner-prod ******
Resolving secrets
00:00
Preparing the "docker" executor
00:02
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:*************** for alpine:latest with digest alpine#sha256:********************* ...
Preparing environment
00:01
Running on runner-***************** via ******************.compute.internal...
Getting source from Git repository
00:02
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/jostens/devops/ci-images/docker-base-windows-2019-std-core/.git/
Checking out 76498ebe as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
/bin/sh: eval: line 110: docker: not found
$ docker build -t ${AWS_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_REF_SLUG} .
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 127
Please Help/advice
Related
I am trying to use Gitlab CI/CD with Cloud Build and Cloud Run to deploy a Flask application.
I am getting an error
starting build "Edited"
FETCHSOURCE
Fetching storage object: gs://Edited
Copying gs://Edited
\ [1 files][ 2.1 GiB/ 2.1 GiB] 43.5 MiB/s
Operation completed over 1 objects/2.1 GiB.
BUILD
Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/docker
Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile: no such file or directory
Finished Step #0
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
--------------------------------------------------------------------------------
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1
My .gitlab-ci.yml
image: aft/ubuntu-py-dvc
stages:
- deploy
deploy:
stage: deploy
tags:
- fts-cicd
image: aft/ubuntu-py-gcloudsdk-dvc
services:
- docker:dind
script:
- echo $dvc > CI_PIPELINE_ID.json
- echo $GCP_LOGIN > gcloud-service-key.json
- dvc remote modify --local view-model-weights credentialpath CI_PIPELINE_ID.json
- dvc pull
- gcloud auth activate-service-account --key-file gcloud-service-key.json
- gcloud config set project $PROJECT_ID
- gcloud builds submit . --config=cloudbuild.yaml
cloudbuild.yaml
steps:
# Build the container image
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/fts-im', '.']
# Push the container image to Container Registry
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/fts-im']
# Deploy container image to Cloud Run
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: gcloud
args: ['run', 'deploy', 'fts_im', '--image', 'gcr.io/$PROJECT_ID/fts_im', "--platform", "managed", "--region","asia-northeast1", "--port", "8000","--memory", "7G", "--cpu", "2", "--allow-unauthenticated"]
images:
- gcr.io/$PROJECT_ID/fts-im
Dockerfile
FROM python:3.9.16-slim
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ADD . /app
COPY .* app/
WORKDIR /app
ADD . .secrets
COPY CI_PIPELINE_ID.json .secrets/CI_PIPELINE_ID.json
RUN ls -la .
RUN ls -la data/
RUN pwd
RUN ls -la .secrets
RUN pip install -r requirements.txt
CMD ["gunicorn" , "-b", "0.0.0.0:8000", "wsgi:app"]
Trying other solutions I tried to prune dockers from the VM which was used for Runner in the CICD settings, I have experimented from a test repo and it worked completely, I am getting this error while replicating it on a new repo. with changed the name to fts_im.
I haven't deleted the previous build and deployed app from cloud build and cloud run, because while using the previous repo I experimented build multiple time all successful.
As per this document Dockerfile should present in the same directory where the build config file is,
Run below command check if Dockerfile present in current directory or not
docker build -t docker-whale
If Dockerfile is present in the same directory where the build config file is, then review this documentation to ensure the correct working directory has been set in the build config file.
Make sure GitLab CI/CD is set up correctly and configured to run on the current branch.
Also you have to specify the full path of the Dockerfile in cloudbuild.yaml file
The name of the file should be Dockerfile and not **.**Dockerfile. The file should not have any extension. check the Dockerfile is named correctly .
Check you have not misspelled image name, I can see 2 different image names gcr.io/$PROJECT_ID/fts-im and gcr.io/$PROJECT_ID/fts_im, I’m not sure whether they are 2 different image or you misplaced _(underscore) with -(Hyphen).
I'm using AWS Code Build to build a Docker image from ECR. This is the Code Build configuration.
Here is the buidspec.yml
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws --version
- aws ecr get-login-password --region my-region | docker login --username AWS --password-stdin my-image-uri
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t pos-drf .
- docker tag pos-drf:latest my-image-uri/pos-drf:latest
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push my-image-uri/pos-drf:latest
Now it's working up until the build command docker build -t pos-drf .
the error message I get is the following
[Container] 2022/12/30 15:12:39 Running command docker build -t pos-drf .
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /codebuild/output/src696881611/src/Dockerfile: no such file or directory
[Container] 2022/12/30 15:12:39 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t pos-drf .. Reason: exit status 1
Now quite sure this is not a permission related issue.
Please let me know if I need to share something else.
UPDATE:
This is the Dockerfile
# base image
FROM python:3.8
# setup environment variable
ENV DockerHOME=/home/app/webapp
# set work directory
RUN mkdir -p $DockerHOME
# where your code lives
WORKDIR $DockerHOME
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install dependencies
RUN pip install --upgrade pip
# copy whole project to your docker home directory.
COPY . $DockerHOME
RUN apt-get dist-upgrade
# RUN apt-get install mysql-client mysql-server
# run this command to install all dependencies
RUN pip install -r requirements.txt
# port where the Django app runs
EXPOSE 8000
# start server
CMD python manage.py runserver
My mistake was that I had the Dockerfile locally but hadn't pushed it.
CodeBuild worked successfully after pushing the Dockerfile.
I am getting the following error from the BUILD stage of my CodeBuild build process:
"Error while executing command: docker build -t ..." Reason: exit status 1
I have a code build service role set up with permissions for ecr, the aws ecr login stage has succeeded, and my buildspec.yml is really simple - pretty much just the standard template. Runtime is the Amazon-managed ubuntu image, standard.
Is there any reason why the Docker build could be failing and anything anyone would suggest to troubleshoot?
Thank you
Full buildspec.yml file:
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email --region eu-west-1)
build:
commands:
- echo Building the Docker image...
- docker build -t maxmind:latest .
- docker tag maxmind:latest 381475286792.dkr.ecr.eu-west-1.amazonaws.com/maxmind:latest
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push 381475286792.dkr.ecr.eu-west-1.amazonaws.com/maxmind:latest
Full error message (BUILD stage):
COMMAND_EXECUTION_ERROR: Error while executing command docker build -t maxmind:latest .. Reason: exit status 1
Full error message (POST_BUILD stage):
COMMAND EXECUTION_ERROR: Error while executing command: docker push 381475286792.dkr.ecr.eu-west-1.amazonaws.com/maxmind:latest. Reason: exit status 1
Full error message (logstream):
[Container] 2020/05/20 09:28:54 Running command docker build -t maxmind:latest .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[Container] 2020/05/20 09:28:54 Command did not exit successfully docker build -t maxmind:latest . exit status 1
[Container] 2020/05/20 09:28:54 Phase complete: BUILD State: FAILED
Things I have tried
Attached AmazonEC2ContainerRegistryPowerUser policy to the codebuild-service-role created by my build process
Based on the comments.
There were two issues. The first one was not using PrivilegedMode mode in the CodeBuild project. The mode is required when building a docker image inside a docker container.
The second issue was missing permission iam:DeletePolicyVersion.
Enabling the mode and adding the missing permissions, solved the issue.
Just want to share this in case anyone still has this issue.
This issue can be caused of 3 reasons:
Not having PrivilegedMode enabled id the CodeBuild project
Not having enough permissions for the IAM role
An error with your dockerfile build
In my case it was the 3rd reason.
I activated s3 logs which helped me see better error messages as it turned out to be that I was missing a folder in my project which my build dockerfile tried to COPY.
But it can be any error, like running an npm command that doesn't exists.
I spent many hours to solve my problem. I use CodePipeline : CodeSource, CodeBuild that produces docker container (code from Bitbucket) and stores the image in ECR.
In CodeDeploy I want to deploy that image from ECR to Elastic Beanstalk:
Errors in Elastic Beanstalk:
Environment health has transitioned from Info to Degraded. Command failed on all instances. Incorrect application version found on all instances. Expected version "Sample Application" (deployment 6). Application update failed 15 seconds ago and took 59 seconds.
During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, re-deploy the appropriate application version.
Failed to deploy application.
Unsuccessful command execution on instance id(s) 'i-04df549361597208a'. Aborting the operation.
Another error from EB:
Incorrect application version "code-pipeline-1586854202535-MyflashcardsBuildOutput-ce0d6cd7-8290-40ad-a95e-9c57162b9ff1"
(deployment 9). Expected version "Sample Application" (deployment 8).
Error in CodeDeploy:
Action execution failed
Deployment completed, but with errors: During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, re-deploy the appropriate application version. Failed to deploy application. Unsuccessful command execution on instance id(s) 'i-04df539061522208a'. Aborting the operation. [Instance: i-04df549333582208a] Command failed on instance. An unexpected error has occurred [ErrorCode: 0000000001].
Does anyone know what happens here?
I use Dockerfile:
### STAGE 1: Build ###
FROM node:12.7-alpine AS build
WORKDIR /usr/src/app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build
### STAGE 2: Run ###
FROM nginx:1.17.1-alpine
EXPOSE 80
COPY --from=build /usr/src/app/dist /usr/share/nginx/html
and buildspec.yml:
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws --version
- $(aws ecr get-login --region eu-west-1 --no-include-email)
- REPOSITORY_URI=176901363719.dkr.ecr.eu-west-1.amazonaws.com/myflashcards
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=myflashcards
build:
commands:
- echo Build started on `date`
- echo Building the Docker image
- docker build --tag $REPOSITORY_URI:latest .
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- echo Writing image definitions file...
- printf '[{"name":"eagle","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
# - echo Deleting old artifacts
# - aws s3 sync dist/ s3://$BUCKET_NAME --delete
artifacts:
files: imagedefinitions.json
The third step (CodeDeploy) fails:(
Ran into the same issue. The first fix worked for me. Listing down all possible fixes which can resolve this issue:
Reason: some bug with elasticbeanstalk, which is making the multi-stage builder step to fail. AWS logs would show you a message like docker pull requires exactly one argument
Solution: Use unnamed builder. By default, the stages are not named, and you refer to them by their integer number, starting with 0 for the first FROM instruction. Make changes in your docker file as below:
### STAGE 1: Build ###
FROM node:12.7-alpine
WORKDIR /usr/src/app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build
### STAGE 2: Run ###
FROM nginx:1.17.1-alpine
EXPOSE 80
COPY --from=0 /usr/src/app/dist /usr/share/nginx/html
Reason: Incase using t2.micro as instance type. npm install command sometimes times out on the t2.micro instance.
Solution: Change the instance type that Elastic Beanstalk is using something other than t2.micro(say t2.small)
If none of the above two fixes work, try changing the COPY line of your Dockerfile as below:
COPY package*.json ./
As AWS sometimes prefer ./ over '.'
I'm trying to get a simple docker app to build using AWS codebuild, but I am coming across an error where the aws command is not found:
[Container] 2016/12/10 04:29:17 Build started on Sat Dec 10 04:29:17 UTC 2016
[Container] 2016/12/10 04:29:17 Running command echo Building the Docker image...
[Container] 2016/12/10 04:29:17 Building the Docker image...
[Container] 2016/12/10 04:29:17 Running command docker build -t aws-test .
[Container] 2016/12/10 04:29:17 sh: 1: docker: not found
[Container] 2016/12/10 04:29:17 Command did not exit successfully docker build -t aws-test . exit status 127
[Container] 2016/12/10 04:29:17 Phase complete: BUILD Success: false
[Container] 2016/12/10 04:29:17 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t aws-test .. Reason: exit status 127
I've got a super simple docker file which builds a simple express app:
FROM node:6.2.0
# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Install app dependencies
COPY package.json /usr/src/app/
# Bundle app source
COPY . /usr/src/app
EXPOSE 3000
CMD npm install && npm start
And I've got a super simple buildspec.yml which is suppose to build the docker container and push it to the aws registry:
version: 0.1
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --region us-west-2)
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t <CONTAINER_NAME> .
- docker tag <CONTAINER_NAME>:latest <ID>.dkr.ecr.us-west-2.amazonaws.com/<CONTAINER_NAME>:latest
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push <ID>.dkr.ecr.us-west-2.amazonaws.com/<CONTAINER_NAME>:latest
However once ran, it throws the error posted above ^^ I'm not sure why the aws cli utils aren't found? This guide here:
http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html
Suggests I don't need to do anything to setup the aws cli utils anywhere?
Also one other thing I noticed, I removed $(aws ecr get-login --region us-west-2) step from the buildspec file, built it again and it then said that the docker command was not found?! Have I missed a step somewhere (I don't think I have).
So it turned out I was using the wrong environment. Here is what I'm using now:
I was trying to specify my own docker image, which was ultimately not setup with any of the AWS cli utils!
Thanks to #Clare Liguori for tipping me off!