I'm building a CI/CD pipeline using git, codebuild and elastic beanstalk.
During codebuild execution when build fails due to syntax error of a test case, I see codebuild progress to next stage and ultimately go on to produce the artifacts.
My understanding was if the build fails, execution should stop. is this a correct behavior ?
Please see the buildspec below.
version: 0.2
phases:
install:
commands:
- echo Installing package.json..
- npm install
- echo Installing Mocha...
- npm install -g mocha
pre_build:
commands:
- echo Installing source NPM placeholder dependencies...
build:
commands:
- echo Build started on `date`
- echo Compiling the Node.js code
- mocha modules/**/tests/*.js
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- modules/*
- node_modules/*
- package.json
- config/*
- server.js
CodeBuild detects build failures by exit codes. You should ensure that your test execution returns a non-zero exit code on failure.
POST_BUILD will always run as long as BUILD was also run (regardless of BUILD's success or failure.) The same goes for UPLOAD_ARTIFACTS. This is so you can retrieve debug information/artifacts.
If you want to do something different in POST_BUILD depending on the success or failure of BUILD, you can test the builtin environment variable CODEBUILD_BUILD_SUCCEEDING, which is set to 1 if BUILD succeeded, and 0 if it failed.
CodeBuild uses the environment variable CODEBUILD_BUILD_SUCCEEDING to show if the build process seems to go right.
the best way I found right now is to create a small script in the install secion and then alway use this like:
phases:
install:
commands:
- echo '#!/bin/bash' > /usr/local/bin/ok; echo 'if [[ "$CODEBUILD_BUILD_SUCCEEDING" == "0" ]]; then exit 1; else exit 0; fi' >> /usr/local/bin/ok; chmod +x /usr/local/bin/ok
post_build:
commands:
- ok && echo Build completed on `date`
The post_build section is run even if the build section might fail. Expanding on the previous answers, you can use the variable CODEBUILD_BUILD_SUCCEEDING in the post_build section of the buildspec.yml file. You can make the post_build section to run if and only if the build section completed successfully. Below is an example of how this can be achieved:
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION)
- CODEBUILD_RESOLVED_SOURCE_VERSION="${CODEBUILD_RESOLVED_SOURCE_VERSION:-$IMAGE_TAG}"
- IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_URI="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG"
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $IMAGE_URI .
post_build:
commands:
- bash -c "if [ /"$CODEBUILD_BUILD_SUCCEEDING/" == /"0/" ]; then exit 1; fi"
- echo Build stage successfully completed on `date`
- docker push $IMAGE_URI
- printf '[{"name":"clair","imageUri":"%s"}]' "$IMAGE_URI" > images.json
artifacts:
files: images.json
add this in build section
build:
on-failure: ABORT
I just wanted to point out that if you want the whole execution to stop when a command fails, you may specify the -e option:
When running a bash file
- /bin/bash -e ./commands.sh
Or when running a set of commands/bash file
#!/bin/bash
set -e
# ... commands
The post_build stage will be executed and the artifacts will be produced. The post_build is good to properly shut down the build environment, if necessary, and the artifacts could be useful even if the build failed. E.g. extra logs, intermediate files, etc.
I would suggest to use post_build only for commands what are agnostic to the result of your build, and properly de-initialise the build environment. Otherwise you can just exclude that step.
Related
I'm using AWS Codebuild to run the automated tests. However, regardless of the test results, CodeBuild always indicates success.
Even CodeBuild indicating success, when checking the logs, I identified that several tests are failing.
How to make CodeBuild also fail when tests fail?
Thank you for your help.
This is my buildspec:
phases:
install:
runtime-versions:
java: corretto11
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn test
post_build:
commands:
- echo Build completed on `date`
I changed to false
<testFailureIgnore>false</testFailureIgnore>
I'm using AWS CodeBuild to deploy the function to AWS lambda using serverless-framework.
Here is my buildspec.yml,
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- echo installing Mocha...
- npm install -g mocha
- echo installing Serverless...
- npm install -g serverless
pre_build:
commands:
- echo running npm install for global project...
- npm install
- echo running npm install for each function...
- folders=src/*
- for value in $folders;
do
echo $value
npm --prefix $value install $value;
done
build:
commands:
- sls package
- serverless deploy --stage $STAGE --region $AWS_DEFAULT_REGION | tee deploy.out
post_build:
commands:
- echo done
# - . ./test.sh
The problem is even when the serverless deploy --stage $STAGE --region $AWS_DEFAULT_REGION | tee deploy.out command fails, the build project is shown as success by AWS codebuild in the codepipeline.
I want the build status as failure when serverless deploy command fails.
This happens because post_build executes whether build fails or succeeds. Thus it does not meter that build fails, post_build will run anyway. This is explained in the build phase transitions.
You can rectify this by "manually" checking if build failed in post_build by checking CODEBUILD_BUILD_SUCCEEDING env variable:
CODEBUILD_BUILD_SUCCEEDING: Whether the current build is succeeding. Set to 0 if the build is failing, or 1 if the build is succeeding.
Thus in your post_build you can check ifCODEBUILD_BUILD_SUCCEEDING == 0 and exit 1 if is true.
post_build:
commands:
- if [[ $CODEBUILD_BUILD_SUCCEEDING == 0 ]]; then exit 1; fi
- echo done
# - . ./test.sh
Your command:
- serverless deploy --stage $STAGE --region $AWS_DEFAULT_REGION | tee deploy.out
... is not returning a non-zero code on failure which is required to fail the build. The command tee is masking the return code from serverless deploy as it itself is responding with a '0' return code.
I would recommend to re-write the command as:
- serverless deploy --stage $STAGE --region $AWS_DEFAULT_REGION > deploy.out
- cat deploy.out
I usually name artifacts based on the commits they have been build from.
Based on this documentation, CODEBUILD_WEBHOOK_PREV_COMMIT is what I am looking for in AWS Code Build
Here is the buildspec.yml
phases:
install:
commands:
- apt-get update -y
build:
commands:
- export $CODEBUILD_WEBHOOK_PREV_COMMIT
- echo Entered the build phase...
- echo Build started on `date`
- mvn clean install -Dmaven.test.skip=true
- for f in ./target/*.car;do mv -- "$f" $(echo $f | sed -E "s/.car$/_${CODEBUILD_WEBHOOK_PREV_COMMIT}.car/") ;done
artifacts:
files:
- ./target/*.car
Build works but the commit does not show in the final .car name. I would like to understand why.
Hypothesis n°1: VARs needs to be explicitly sourced
I tried the following without much success
env:
variable:
- COMMIT="${CODEBUILD_WEBHOOK_PREV_COMMIT}"
phases:
install:
commands:
- apt-get update -y
build:
commands:
- echo Entered the build phase...
- echo Build started on `date`
- mvn clean install -Dmaven.test.skip=true
- carpath=./*_CA/target/*.car
- for f in $carpath;do mv -- "$f" $(echo $f | sed -E "s/.car$/_${COMMIT}.car/") ;done
VARs are only available to AWS default build container
I am using Maven's official image maven:3.6.3-jdk-8 instead of Amazon's general purpose build image. Are VARs available for custom images? I can't find any clear indication they are not.
I lost entire afternoon on this, for anyone that is coming here with the same problem, here is how I solved it:
First, I've put printenv in commands to see whats going on and $CODEBUILD_WEBHOOK_PREV_COMMIT env variable was completely missing. But you can use $CODEBUILD_RESOLVED_SOURCE_VERSION instead, which was there!
When I run the following command: - go build -o app I get the following error (for multiple dependencies) : main.go:21:2: cannot find package "github.com/gorilla/mux" in any of:
/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
/go/src/github.com/gorilla/mux (from $GOPATH)
/codebuild/output/src324986171/src/github.com/gorilla/mux
Meaning the codebuild fails. Any idea how I can fix this, or in general where the problem is ?
Thanks for your help.
EDIT:
After adding go get ./... to the build I get the following error for all my local packages:# cd .; git clone https://github.com/aristotle/dbhelper /go/src/github.com/aristotle/dbhelper
Cloning into '/go/src/github.com/aristotle/dbhelper'...
My buildspec.yml looks like this:
version: 0.2
phases:
install:
commands:
- echo CODEBUILD_SRC_DIR - $CODEBUILD_SRC_DIR
- echo GOPATH - $GOPATH
- echo GOROOT - $GOROOT
build:
commands:
- echo Build started on `date`
- echo Getting packages
- go get ./...
- echo Compiling the Go code...
- go build -o app main.go
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- app
According to this article you need to add it to the install section of your buildspec.yml file.
install:
commands:
- go get github.com/gorilla/mux
maybe it does also work to include go get ./... this will resolve all dependencies... but if you do not have too many it is a good practise to list them explicitly.
this is the source article: https://www.contributing.md/2017/06/30/golang-with-aws-codebuild/
I tried to follow this doc ( https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-nginx.html ) but couldn't build the custom nginx conf.
I am able to deploy an application and environment and it works. After testing a working environment I wanted to modify some nginx configurations and I followed the steps as:
cd WS
mkdir -p .ebextensions/nginx/conf.d
cp ~/dozee.conf .ebextensions/nginx/conf.d
eb deploy
WS is a directory from where eb deploy works perfectly. After logging(ssh) into the instance created by eb environment I could see the dozee.conf present at /var/app/current/.ebextensions/nginx/conf.d/ but was not present at /etc/nginx/conf.d/.
What I might be missing here? Any help is appreciated :)
The most likely problem is that your .ebextensions folder is not being included in your build. Can you post your buildspec.yml? To give you an idea what needs to happen, here is one of mine:
version: 0.2
phases:
install:
commands:
- echo Entering install phase...
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Entering pre_build phase...
- echo Running tests...
- mvn test
build:
commands:
- echo Entering build phase...
- echo Build started on `date`
- mvn package -Dmaven.test.skip=true
post_build:
commands:
- echo Entering post_build phase...
- echo Build completed on `date`
- mv target/app.war app.war
artifacts:
type: zip
files:
- app.war
- .ebextensions/**/*