we wanna try CodePipeline with a image that we already have on ECR.
So we follow the steps on the documentation.
We have buildspec.yml like this:
phases:
install:
runtime-versions:
nodejs: 10
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws --version
- $(aws ecr get-login --no-include-email --region us-east-1)
- REPOSITORY_URI=OUR_URL_FROM_ECR
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=${COMMIT_HASH:=latest}
- echo $REPOSITORY_URI
- echo $COMMIT_HASH
- echo $IMAGE_TAG
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $REPOSITORY_URI:latest .
- docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo Writing image definitions file...
- printf '[{"name":"Petr","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
artifacts:
files: imagedefinitions.json
We created a new pipeline flow, but when we push some changes we get this log:
[Container] 2019/11/07 23:30:49 Waiting for agent ping
[Container] 2019/11/07 23:30:51 Waiting for DOWNLOAD_SOURCE
[Container] 2019/11/07 23:30:52 Phase is DOWNLOAD_SOURCE
[Container] 2019/11/07 23:30:52 CODEBUILD_SRC_DIR=/codebuild/output/src386464501/src
[Container] 2019/11/07 23:30:52 YAML location is /codebuild/output/src386464501/src/buildspec.yml
[Container] 2019/11/07 23:30:52 No commands found for phase name: INSTALL
[Container] 2019/11/07 23:30:52 Processing environment variables
[Container] 2019/11/07 23:30:52 Moving to directory /codebuild/output/src386464501/src
[Container] 2019/11/07 23:30:52 Registering with agent
[Container] 2019/11/07 23:30:52 Phases found in YAML: 4
[Container] 2019/11/07 23:30:52 POST_BUILD: 6 commands
[Container] 2019/11/07 23:30:52 INSTALL: 0 commands
[Container] 2019/11/07 23:30:52 PRE_BUILD: 9 commands
[Container] 2019/11/07 23:30:52 BUILD: 4 commands
[Container] 2019/11/07 23:30:52 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
[Container] 2019/11/07 23:30:52 Phase context status code: Message:
[Container] 2019/11/07 23:30:52 Entering phase INSTALL
[Container] 2019/11/07 23:30:52 Running command echo "Installing Node.js version 10 ..."
Installing Node.js version 10 ...
[Container] 2019/11/07 23:30:52 Running command n 10.16.3
installed : v10.16.3 (with npm 6.9.0)
[Container] 2019/11/07 23:31:02 Phase complete: INSTALL State: SUCCEEDED
[Container] 2019/11/07 23:31:02 Phase context status code: Message:
[Container] 2019/11/07 23:31:02 Entering phase PRE_BUILD
[Container] 2019/11/07 23:31:02 Running command echo Logging in to Amazon ECR...
Logging in to Amazon ECR...
[Container] 2019/11/07 23:31:02 Running command aws --version
aws-cli/1.16.242 Python/3.6.8 Linux/4.14.143-91.122.amzn1.x86_64 exec-env/AWS_ECS_EC2 botocore/1.12.232
[Container] 2019/11/07 23:31:07 Running command $(aws ecr get-login --no-include-email --region us-east-1)
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Container] 2019/11/07 23:31:10 Running command REPOSITORY_URI=***********
[Container] 2019/11/07 23:31:10 Running command COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
[Container] 2019/11/07 23:31:10 Running command IMAGE_TAG=${COMMIT_HASH:=latest}
[Container] 2019/11/07 23:31:10 Running command echo $REPOSITORY_URI
***********
[Container] 2019/11/07 23:31:10 Running command echo $COMMIT_HASH
88f8cfc
[Container] 2019/11/07 23:31:10 Running command echo $IMAGE_TAG
88f8cfc
[Container] 2019/11/07 23:31:10 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2019/11/07 23:31:10 Phase context status code: Message:
[Container] 2019/11/07 23:31:10 Entering phase BUILD
[Container] 2019/11/07 23:31:10 Running command echo Build started on `date`
Build started on Thu Nov 7 23:31:10 UTC 2019
[Container] 2019/11/07 23:31:10 Running command echo Building the Docker image...
Building the Docker image...
[Container] 2019/11/07 23:31:10 Running command docker build -t $REPOSITORY_URI:latest .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[Container] 2019/11/07 23:31:10 Command did not exit successfully docker build -t $REPOSITORY_URI:latest . exit status 1
[Container] 2019/11/07 23:31:10 Phase complete: BUILD State: FAILED
[Container] 2019/11/07 23:31:10 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t $REPOSITORY_URI:latest .. Reason: exit status 1
[Container] 2019/11/07 23:31:10 Entering phase POST_BUILD
[Container] 2019/11/07 23:31:10 Running command echo Build completed on `date`
Build completed on Thu Nov 7 23:31:10 UTC 2019
[Container] 2019/11/07 23:31:10 Running command echo Pushing the Docker images...
Pushing the Docker images...
[Container] 2019/11/07 23:31:10 Running command docker push $REPOSITORY_URI:latest
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[Container] 2019/11/07 23:31:10 Command did not exit successfully docker push $REPOSITORY_URI:latest exit status 1
[Container] 2019/11/07 23:31:10 Phase complete: POST_BUILD State: FAILED
[Container] 2019/11/07 23:31:10 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker push $REPOSITORY_URI:latest. Reason: exit status 1
[Container] 2019/11/07 23:31:10 Expanding base directory path: .
[Container] 2019/11/07 23:31:10 Assembling file list
[Container] 2019/11/07 23:31:10 Expanding .
[Container] 2019/11/07 23:31:10 Expanding file paths for base directory .
[Container] 2019/11/07 23:31:10 Assembling file list
[Container] 2019/11/07 23:31:10 Expanding imagedefinitions.json
[Container] 2019/11/07 23:31:10 Skipping invalid file path imagedefinitions.json
[Container] 2019/11/07 23:31:10 Phase complete: UPLOAD_ARTIFACTS State: FAILED
[Container] 2019/11/07 23:31:10 Phase context status code: CLIENT_ERROR Message: no matching artifact paths found
We wanna know if we are missing something, we follow some steps from here:
https://aws.amazon.com/es/blogs/devops/build-a-continuous-delivery-pipeline-for-your-container-images-with-amazon-ecr-as-source/
Any advice?
I ran into similar error. The fix is, this build project needs to build a Docker image therefore set Privilege Mode to true.
Privileged mode grants a build project's Docker container access.
There will be two possibilities:-
one is that you didn't add ecr and ecs in your roles which you created for ec2 instance or if you are using elastic beanstalk. first verify that
otherwise look into this second possibility:-
Use following commands in your phases:-
phases:
install:
commands:
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2&
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
for more details use this link
https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker-custom-image.html#sample-docker-custom-image-files
https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html#troubleshooting-cannot-connect-to-docker-daemon
For error
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is
the docker daemon running?
I found this helpful:
https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html
Specifically point 5.d.
Follow the steps in Run CodeBuild directly to create a build project, run the build, and view build information.
If you use the console to create your project:
a. For Operating system, choose Ubuntu.
b. For Runtime, choose Standard.
c. For Image, choose aws/codebuild/standard:4.0.
d. Because you use this build project to build a Docker image, select Privileged.
I have had the same problem as you and the way I fixed it was by following: try going to CodeBuild and then to its IAM Role. AmazonEC2ContainerRegistryFullAccess role and now click on 'Edit' for that code build and select 'Environment' and click on Allow AWS CodeBuild to modify this service role so it can be used with this building project. Now try again.
Cheers
Related
I'm struggling pushing a docker image to amazon registry (ECR) through my aws codepipeline.
Despite I'm able to successfully login into the registry docker push fails with error no basic auth credentials
Here the buildspec.yml used by my codepipeline
`
version: 0.2
phases:
install:
runtime-versions:
java: corretto8
pre_build:
commands:
- echo -e "[INFO] \t\t In the pre_build phase..."
- echo -e "[INFO] \t\t Logging in to Amazon ECR..."
- aws --version
- aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com
- cat ~/.docker/config.json
- REPOSITORY_URI=<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library
build:
commands:
- echo -e "[INFO] \t\t In the build phase..."
- echo -e "[INFO] \t\t Listing build folder directory..."
- ls -lrtha
- which docker
- ./build_script.sh
- docker build --build-arg JAR_FILE=build/libs/*.jar -t javaapp-aws:0.0.1 .
- docker tag <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1
post_build:
commands:
- echo -e "[INFO] \t\t In the post_build phase..."
- echo -e "[INFO] \t\t Build completed"
- docker images
- docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1
- printf '{"ImageURI":"%s"}' <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 > imagedefinitions.json
artifacts:
files: imagedefinitions.json
`
Here the logs from the build phase
[Container] 2022/10/28 21:08:38 going inside waitForAgent
[Container] 2022/10/28 21:08:38 Waiting for agent ping
[Container] 2022/10/28 21:08:41 Waiting for DOWNLOAD_SOURCE
[Container] 2022/10/28 21:08:41 Phase is DOWNLOAD_SOURCE
[Container] 2022/10/28 21:08:41 finished waitForAgent
[Container] 2022/10/28 21:08:41 inside CopySrc
[Container] 2022/10/28 21:08:41 CODEBUILD_SRC_DIR=/codebuild/output/src628607031/src
[Container] 2022/10/28 21:08:41 finished CopySrc
[Container] 2022/10/28 21:08:41 YAML location is /codebuild/readonly/buildspec.yml
[Container] 2022/10/28 21:08:41 No commands found for phase name: install
[Container] 2022/10/28 21:08:41 Setting HTTP client timeout to higher timeout for S3 source
[Container] 2022/10/28 21:08:41 Processing environment variables
[Container] 2022/10/28 21:08:41 Selecting 'java' runtime version 'corretto8' based on manual selections...
[Container] 2022/10/28 21:08:41 Running command echo "Installing corretto(OpenJDK) version 8 ..."
Installing corretto(OpenJDK) version 8 ...
[Container] 2022/10/28 21:08:41 Running command export JAVA_HOME="$JAVA_8_HOME"
[Container] 2022/10/28 21:08:41 Running command export JRE_HOME="$JRE_8_HOME"
[Container] 2022/10/28 21:08:41 Running command export JDK_HOME="$JDK_8_HOME"
[Container] 2022/10/28 21:08:41 Running command for tool_path in "$JAVA_8_HOME"/bin/* "$JRE_8_HOME"/bin/*;
do tool=`basename "$tool_path"`;
if [ $tool != 'java-rmi.cgi' ];
then
rm -f /usr/bin/$tool /var/lib/alternatives/$tool \
&& update-alternatives --install /usr/bin/$tool $tool $tool_path 20000;
fi;
done
[Container] 2022/10/28 21:08:44 Moving to directory /codebuild/output/src628607031/src
[Container] 2022/10/28 21:08:44 Configuring ssm agent with target id: codebuild:e2e7f1b3-7716-4764-90cf-8ca556103f3a
[Container] 2022/10/28 21:08:44 Successfully updated ssm agent configuration
[Container] 2022/10/28 21:08:44 Registering with agent
[Container] 2022/10/28 21:08:44 Phases found in YAML: 4
[Container] 2022/10/28 21:08:44 INSTALL: 0 commands
[Container] 2022/10/28 21:08:44 PRE_BUILD: 6 commands
[Container] 2022/10/28 21:08:44 BUILD: 8 commands
[Container] 2022/10/28 21:08:44 POST_BUILD: 5 commands
[Container] 2022/10/28 21:08:44 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
[Container] 2022/10/28 21:08:44 Phase context status code: Message:
[Container] 2022/10/28 21:08:44 Entering execCommands
[Container] 2022/10/28 21:08:44 Entering phase INSTALL
[Container] 2022/10/28 21:08:44 Phase complete: INSTALL State: SUCCEEDED
[Container] 2022/10/28 21:08:44 Phase context status code: Message:
[Container] 2022/10/28 21:08:44 Entering phase PRE_BUILD
[Container] 2022/10/28 21:08:44 Running command echo -e "[INFO] \t\t In the pre_build phase..."
[INFO] In the pre_build phase...
[Container] 2022/10/28 21:08:44 Running command echo -e "[INFO] \t\t Logging in to Amazon ECR..."
[INFO] Logging in to Amazon ECR...
[Container] 2022/10/28 21:08:44 Running command aws --version
aws-cli/1.25.1 Python/3.8.10 Linux/4.14.281-212.502.amzn2.x86_64 exec-env/AWS_ECS_EC2 botocore/1.27.1
[Container] 2022/10/28 21:08:44 Running command aws ecr get-login-password --region <ACCOUNTID> | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Container] 2022/10/28 21:08:45 Running command cat ~/.docker/config.json
{
"auths": {
"<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com": {
"auth": "HEREISSTOREDTHELONGAUTHTOCKEN"
}
}
}
[Container] 2022/10/28 21:08:45 Running command REPOSITORY_URI=<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library
[Container] 2022/10/28 21:08:45 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2022/10/28 21:08:45 Phase context status code: Message:
[Container] 2022/10/28 21:08:45 Entering phase BUILD
[Container] 2022/10/28 21:08:45 Running command echo -e "[INFO] \t\t In the build phase..."
[INFO] In the build phase...
[Container] 2022/10/28 21:08:45 Running command echo -e "[INFO] \t\t Listing build folder directory..."
[INFO] Listing build folder directory...
[Container] 2022/10/28 21:08:45 Running command ls -lrtha
total 52K
-rw-rw-r-- 1 root root 14 Oct 28 09:06 version.txt
-rw-rw-r-- 1 root root 34 Oct 28 09:06 settings.gradle
-rwxrwxr-x 1 root root 1.2K Oct 28 09:06 release.sh
-rw-rw-r-- 1 root root 25 Oct 28 09:06 info.txt
-rw-rw-r-- 1 root root 2.7K Oct 28 09:06 gradlew.bat
-rwxrwxr-x 1 root root 5.7K Oct 28 09:06 gradlew
-rwxrwxr-x 1 root root 120 Oct 28 09:06 get_tag.sh
-rwxrwxr-x 1 root root 920 Oct 28 09:06 build_script.sh
-rw-rw-r-- 1 root root 912 Oct 28 09:06 build.gradle
-rw-rw-r-- 1 root root 0 Oct 28 09:06 application.yml
-rw-rw-r-- 1 root root 692 Oct 28 09:06 README.md
-rw-rw-r-- 1 root root 113 Oct 28 09:06 Dockerfile
-rw-rw-r-- 1 root root 293 Oct 28 09:06 .gitignore
drwxr-xr-x 3 root root 18 Oct 28 21:08 src
drwxr-xr-x 3 root root 21 Oct 28 21:08 gradle
drwxr-xr-x 4 root root 278 Oct 28 21:08 .
drwxr-xr-x 3 root root 17 Oct 28 21:08 ..
[Container] 2022/10/28 21:08:45 Running command which docker
/usr/local/bin/docker
[Container] 2022/10/28 21:08:45 Running command ./build_script.sh
[INFO] Building component image for javaapp-aws, version 0.0.1
[INFO] Building java...
Downloading https://services.gradle.org/distributions/gradle-7.2-bin.zip
..........10%...........20%...........30%...........40%...........50%...........60%...........70%...........80%...........90%...........100%
Welcome to Gradle 7.2!
Here are the highlights of this release:
- Toolchain support for Scala
- More cache hits when Java source files have platform-specific line endings
- More resilient remote HTTP build cache behavior
For more details see https://docs.gradle.org/7.2/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileJava
> Task :processResources NO-SOURCE
> Task :classes
> Task :bootJarMainClassName
> Task :bootJar
> Task :jar SKIPPED
> Task :assemble
> Task :compileTestJava NO-SOURCE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test NO-SOURCE
> Task :check UP-TO-DATE
> Task :build
BUILD SUCCESSFUL in 1m 3s
3 actionable tasks: 3 executed
[Container] 2022/10/28 21:09:49 Running command docker build --build-arg JAR_FILE=build/libs/*.jar -t javaapp-aws:0.0.1 .
Sending build context to Docker daemon 38.05MB
Step 1/5 : FROM openjdk:8-jdk-alpine
8-jdk-alpine: Pulling from library/openjdk
e7c96db7181b: Pulling fs layer
f910a506b6cb: Pulling fs layer
c2274a1a0e27: Pulling fs layer
f910a506b6cb: Verifying Checksum
f910a506b6cb: Download complete
e7c96db7181b: Verifying Checksum
e7c96db7181b: Download complete
e7c96db7181b: Pull complete
f910a506b6cb: Pull complete
c2274a1a0e27: Verifying Checksum
c2274a1a0e27: Download complete
c2274a1a0e27: Pull complete
Digest: sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3
Status: Downloaded newer image for openjdk:8-jdk-alpine
---> a3562aa0b991
Step 2/5 : ARG JAR_FILE
---> Running in 5f2f64973a5f
Removing intermediate container 5f2f64973a5f
---> 3323b6a04548
Step 3/5 : COPY ${JAR_FILE} app.jar
---> 5f2e1367eabe
Step 4/5 : ENTRYPOINT ["java","-jar","/app.jar"]
---> Running in d75a0dcd812b
Removing intermediate container d75a0dcd812b
---> 69e20eacbcbe
Step 5/5 : EXPOSE 8080
---> Running in acb0e7e5dcae
Removing intermediate container acb0e7e5dcae
---> 275d152d55fc
Successfully built 275d152d55fc
Successfully tagged javaapp-aws:0.0.1
[Container] 2022/10/28 21:09:55 Running command docker tag javaapp-aws:0.0.1 <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1
[Container] 2022/10/28 21:09:55 Phase complete: BUILD State: SUCCEEDED
[Container] 2022/10/28 21:09:55 Phase context status code: Message:
[Container] 2022/10/28 21:09:55 Entering phase POST_BUILD
[Container] 2022/10/28 21:09:55 Running command echo -e "[INFO] \t\t In the post_build phase..."
[INFO] In the post_build phase...
[Container] 2022/10/28 21:09:55 Running command echo -e "[INFO] \t\t Build completed"
[INFO] Build completed
[Container] 2022/10/28 21:09:55 Running command docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws 0.0.1 275d152d55fc Less than a second ago 142MB
javaapp-aws 0.0.1 275d152d55fc Less than a second ago 142MB
openjdk 8-jdk-alpine a3562aa0b991 3 years ago 105MB
[Container] 2022/10/28 21:09:55 Running command docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1
The push refers to repository [<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws]
5c8eac8a352b: Preparing
ceaf9e1ebef5: Preparing
9b9b7f3d56a0: Preparing
f1b5933fe4b5: Preparing
no basic auth credentials
[Container] 2022/10/28 21:09:55 Command did not exit successfully docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 exit status 1
[Container] 2022/10/28 21:09:55 Phase complete: POST_BUILD State: FAILED
[Container] 2022/10/28 21:09:55 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1. Reason: exit status 1
[Container] 2022/10/28 21:09:55 exiting execCommands
[Container] 2022/10/28 21:09:55 Expanding base directory path: .
[Container] 2022/10/28 21:09:55 Assembling file list
[Container] 2022/10/28 21:09:55 Expanding .
[Container] 2022/10/28 21:09:55 Expanding file paths for base directory .
[Container] 2022/10/28 21:09:55 Assembling file list
[Container] 2022/10/28 21:09:55 Expanding imagedefinitions.json
[Container] 2022/10/28 21:09:55 Skipping invalid file path imagedefinitions.json
[Container] 2022/10/28 21:09:55 Phase complete: UPLOAD_ARTIFACTS State: FAILED
[Container] 2022/10/28 21:09:55 Phase context status code: CLIENT_ERROR Message: no matching artifact paths found
What is strage is that the login command works as per above log. Paste here a snipped to focus on
`
[Container] 2022/10/28 21:08:44 Running command aws --version
aws-cli/1.25.1 Python/3.8.10 Linux/4.14.281-212.502.amzn2.x86_64 exec-env/AWS_ECS_EC2 botocore/1.27.1
[Container] 2022/10/28 21:08:44 Running command aws ecr get-login-password --region <ACCOUNTID> | docker login --username AWS --password-stdin <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
`
but at the end get error
[Container] 2022/10/28 21:09:55 Running command docker push <ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws:0.0.1 The push refers to repository [<ACCOUNTID>.dkr.ecr.<AWSREGION>.amazonaws.com/library/javaapp-aws] 5c8eac8a352b: Preparing ceaf9e1ebef5: Preparing 9b9b7f3d56a0: Preparing f1b5933fe4b5: Preparing no basic auth credentials
any suggestion?
tks
I've also tried locally on my laptop but got the same issue I'm experiencing within the pipeline.
I've been trying to figure this out for a couple days and the issue is compounded by the fact that I'm not getting a useful error message.
I'm using the following buildspec.yml file in codebuild to build docker containers and then send to AWS ECR.
version: 0.2
env:
parameter-store:
AWS_DEFAULT_REGION: "/docker_test/region"
IMAGE_REPO_NAME: "/docker_test/repo_name"
IMAGE_TAG: "/docker_test/img_tag"
AWS_ACCOUNT_ID: "account_id"
phases:
install:
runtime-versions:
docker: 18
pre_build:
commands:
- echo Logging in to Amazon ECR and DockerHub...
- docker login -u AWS -p $(aws ecr get-login-password --region $AWS_DEFAULT_REGION) $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker-compose -f docker-compose.yml -f docker-compose.prod.yml build
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
artifacts:
files:
- 'Dockerrun.aws.json'
I've tried docker 19, slightly different versions of the docker login line and made sure my roles were set. I get "login succeeded" so I assume the login line is good.
[Container] 2021/12/22 16:19:20 Running command docker login -u AWS -p $(aws ecr get-login-password --region $AWS_DEFAULT_REGION) $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Container] 2021/12/22 16:19:26 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2021/12/22 16:19:26 Phase context status code: Message:
[Container] 2021/12/22 16:19:26 Entering phase BUILD
The post_build phase fails however with the following:
Successfully built d6878cbb68ba
Successfully tagged ***.dkr.ecr.***.amazonaws.com/***:latest
[Container] 2021/12/22 16:21:58 Phase complete: BUILD State: SUCCEEDED
[Container] 2021/12/22 16:21:58 Phase context status code: Message:
[Container] 2021/12/22 16:21:58 Entering phase POST_BUILD
[Container] 2021/12/22 16:21:58 Running command echo Build completed on `date`
Build completed on Wed Dec 22 16:21:58 UTC 2021
[Container] 2021/12/22 16:21:58 Running command echo Pushing the Docker image...
Pushing the Docker image...
[Container] 2021/12/22 16:21:58 Running command docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
Pushing myapp (***.dkr.ecr.***.amazonaws.com/***:latest)...
The push refers to repository [***.dkr.ecr.***.amazonaws.com/***]
EOF
[Container] 2021/12/22 16:22:49 Command did not exit successfully docker-compose -f docker-compose.yml -f docker-compose.prod.yml push exit status 1
[Container] 2021/12/22 16:22:49 Phase complete: POST_BUILD State: FAILED
[Container] 2021/12/22 16:22:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose -f docker-compose.yml -f docker-compose.prod.yml push. Reason: exit status 1
[Container] 2021/12/22 16:22:49 Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED
[Container] 2021/12/22 16:22:49 Phase context status code: Message:
I'm just not sure how to get more information on this error - that would be ideal.
EDIT:
I'm adding the docker-compose.prod.yml file for additional context:
version: "3.2"
services:
myapp:
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_TAG}
command: bash -c "
python manage.py migrate
&& gunicorn --bind :8000 --workers 3 --threads 2 --timeout 240 project.wsgi:application"
restart: always
ports:
- "80:80"
celery_worker:
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_TAG}
command: celery -A project worker --loglevel=${CELERY_LOG_LEVEL:-WARNING}
restart: always
OK, so I figured it out. Your question about making sure the repo exists pointed me in the right direction #mreferre. I was confused about the use of IMAGE_TAG and IMAGE_REPO_NAME in the code samples I referenced when trying to build this. They were essentially supposed to be the same thing so the push was failing because I was trying to push to an ECR repo named "proj-name" which didn't exist. I just needed to change it to "repo-name" so the image in docker-compose.prod.yml becomes:
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_REPO_NAME}
yaml executed successfully in AWS code build but image not send to aws ecr.
buildspec.yml file output is given below
`[Container] 2020/10/26 09:50:07 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2020/10/26 09:50:07 Phase context status code: Message:
[Container] 2020/10/26 09:50:07 Entering phase BUILD
[Container] 2020/10/26 09:50:07 Phase complete: BUILD State: SUCCEEDED
[Container] 2020/10/26 09:50:07 Phase context status code: Message:
[Container] 2020/10/26 09:50:07 Entering phase POST_BUILD
[Container] 2020/10/26 09:50:07 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2020/10/26 09:50:07 Phase context status code: Message: `
Every phase is executed successfully with SUCCEEDED message.
Below is the buildspec.yml file code snippet
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $REPOSITORY_URI:latest .
- docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo Writing image definitions file...
- printf '[{"name":"ui","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
- cat imagedefinitions.json
No command is being run due to bad indentation. Please rectify the indentation using the buildspec reference as guide:
https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
Also I do not see a docker login before the push:
- aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
A buildspce sample for Docker is here:
https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html#sample-docker-files
===============
I'm new to AWS CodeBuild. We are trying it for the first time. I set up my Buildspec.yml file and when I run it I get errors indicating that it cd: can't cd to /codebuild/output/src.... throughout the build run, however it progresses to the next step each time disregarding the errors. However, when it gets to the artifacts step it can't find the artifacts path. I've tried many iterations of the artifacts: files: statements with no luck. I'm not sure if the cd errors and the artifact issue are related.
Here is the Buildspec file currently. I've also attempted several different versions of the "files:" statement including - '**/*'
buildspec.yml
version: 0.2
phases:
install:
commands:
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- echo $CODEBUILD_SRC_DIR
- uServices/hello-world-java/gradlew build
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- uServices/hello-world-java/build/distributions/*
output from the codebuild
[Container] 2019/01/31 14:19:42 Waiting for agent ping
[Container] 2019/01/31 14:19:43 Waiting for DOWNLOAD_SOURCE
[Container] 2019/01/31 14:19:44 Phase is DOWNLOAD_SOURCE
[Container] 2019/01/31 14:19:44 CODEBUILD_SRC_DIR=/codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
[Container] 2019/01/31 14:19:44 YAML location is /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git /buildspec.yml
[Container] 2019/01/31 14:19:44 Processing environment variables
[Container] 2019/01/31 14:19:44 Moving to directory /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
[Container] 2019/01/31 14:19:44 Registering with agent
[Container] 2019/01/31 14:19:44 Phases found in YAML: 4
[Container] 2019/01/31 14:19:44 POST_BUILD: 1 commands
[Container] 2019/01/31 14:19:44 INSTALL: 1 commands
[Container] 2019/01/31 14:19:44 PRE_BUILD: 1 commands
[Container] 2019/01/31 14:19:44 BUILD: 3 commands
[Container] 2019/01/31 14:19:44 Phase complete: DOWNLOAD_SOURCE Success: true
[Container] 2019/01/31 14:19:44 Phase context status code: Message:
[Container] 2019/01/31 14:19:44 Entering phase INSTALL
[Container] 2019/01/31 14:19:44 Running command echo Nothing to do in the install phase...
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Nothing to do in the install phase...
[Container] 2019/01/31 14:19:44 Phase complete: INSTALL Success: true
[Container] 2019/01/31 14:19:44 Phase context status code: Message:
[Container] 2019/01/31 14:19:45 Entering phase PRE_BUILD
[Container] 2019/01/31 14:19:45 Running command echo Nothing to do in the pre_build phase...
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Nothing to do in the pre_build phase...
[Container] 2019/01/31 14:19:45 Phase complete: PRE_BUILD Success: true
[Container] 2019/01/31 14:19:45 Phase context status code: Message:
[Container] 2019/01/31 14:19:45 Entering phase BUILD
[Container] 2019/01/31 14:19:45 Running command echo Build started on `date`
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Build started on Thu Jan 31 14:19:45 UTC 2019
[Container] 2019/01/31 14:19:45 Running command echo $CODEBUILD_SRC_DIR
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
/codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
[Container] 2019/01/31 14:19:45 Running command uServices/hello-world-java/gradlew build
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Downloading https://services.gradle.org/distributions/gradle-3.5-bin.zip
..........................................................................................................................................................................................................................................................................................
Unzipping /root/.gradle/wrapper/dists/gradle-3.5-bin/daoimhu7k5rlo48ntmxw2ok3e/gradle-3.5-bin.zip to /root/.gradle/wrapper/dists/gradle-3.5-bin/daoimhu7k5rlo48ntmxw2ok3e
Set executable permissions for: /root/.gradle/wrapper/dists/gradle-3.5-bin/daoimhu7k5rlo48ntmxw2ok3e/gradle-3.5/bin/gradle
Starting a Gradle Daemon (subsequent builds will be faster)
:buildEnvironment
------------------------------------------------------------
Root project
------------------------------------------------------------
classpath
No dependencies
BUILD SUCCESSFUL
Total time: 8.74 secs
[Container] 2019/01/31 14:19:56 Phase complete: BUILD Success: true
[Container] 2019/01/31 14:19:56 Phase context status code: Message:
[Container] 2019/01/31 14:19:56 Entering phase POST_BUILD
[Container] 2019/01/31 14:19:56 Running command echo Build completed on `date`
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Build completed on Thu Jan 31 14:19:56 UTC 2019
[Container] 2019/01/31 14:19:56 Phase complete: POST_BUILD Success: true
[Container] 2019/01/31 14:19:56 Phase context status code: Message:
[Container] 2019/01/31 14:19:56 Expanding base directory path: .
[Container] 2019/01/31 14:19:56 Assembling file list
[Container] 2019/01/31 14:19:56 Expanding /codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
.
[Container] 2019/01/31 14:19:56 Expanded to /codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
.
[Container] 2019/01/31 14:19:56 Skipping invalid artifact path /codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
/codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
.
[Container] 2019/01/31 14:19:56 Phase complete: UPLOAD_ARTIFACTS Success: false
[Container] 2019/01/31 14:19:56 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for /codebuild/output/tmp/script.sh: 1: cd: can't cd to /codebuild/output/src590503358/src/github.com/SomosEngineering/tss-modern.git
Summary
In my development area I can run Gradle and see the .zip artifact
sitting in the following directory as below. I would expect codebuild
to find it and load it to S3. [clipford#NVA-1a-TSS-DEV01
distributions]$ pwd
/home/clipford/work/tss-modern/uServices/hello-world-java/build/distributions
[clipford#NVA-1a-TSS-DEV01 distributions]$ ls
tss-modern-clipford.zip
I ended up attempting to build this from an amazon S3 container the build worked completely. Then I went back to the GitHub version and the issue associated with not being able to cd to the codebuild directories magically went away. (I don't believe in magic either but assume that it was an amazon issue).
The Issue with the artifact not loading was related to the fact that the artifact in our project sits in a directory that is not part of the GitHub repo. It's actually ignored in .gitignore. The artifact directory is created at build time and it appears that since it's ignored in the .gitignore and not part of the repo codebuild doesn't find the artifact and can't load it to S3.
I am new to AWS CodePipeline. When trying to deploy an application using docker images and ECS services, I am getting the following error using my buildspec.yml file:
[Container] 2018/08/14 06:20:36 Phase context status code: Message:
[Container] 2018/08/14 06:20:36 Entering phase INSTALL
[Container] 2018/08/14 06:20:36 Phase complete: INSTALL Success: true
[Container] 2018/08/14 06:20:36 Phase context status code: Message:
[Container] 2018/08/14 06:20:36 Entering phase PRE_BUILD
[Container] 2018/08/14 06:20:36 Running command echo Logging in to Amazon ECR... Logging in to Amazon ECR...
[Container] 2018/08/14 06:20:36 Running command aws --version aws-cli/1.15.41 Python/2.7.6 Linux/4.14.47-56.37.amzn1.x86_64 botocore/1.10.41
[Container] 2018/08/14 06:20:39 Running command $(aws ecr get-login --no-include-email --region us-east-xxx) WARNING! Using --password via the CLI is insecure. Use --password-stdin. Login Succeeded
[Container] 2018/08/14 06:20:40 Running command REPOSITORY_URI= xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python /codebuild/output/tmp/script.sh: 4: /codebuild/output/tmp/script.sh: xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python: not found
[Container] 2018/08/14 06:20:40 Command did not exit successfully REPOSITORY_URI= xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python exit status 127
[Container] 2018/08/14 06:20:40 Phase complete: PRE_BUILD Success: false
[Container] 2018/08/14 06:20:40 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: REPOSITORY_URI= xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python. Reason: exit status 127
Any assistance is appreciated. If it helps, I can attach my buildspec.yml and taskdefinition.json files. Is there a recommended way to do that?
It seems you have a space between = and xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python.
This means you're evaluating xxxx.xxxx.ecr.us-east-xxx.amazonaws.com/python as a command, and since this URI is not a valid command, it exits with status code 127.
Simply deleting the space should help.