I have the following buildspec.yml:
version: 0.2
phases:
install:
commands:
- curl -L -o sbt-0.13.6.deb http://dl.bintray.com/sbt/debian/sbt-0.13.6.deb && \
- dpkg -i sbt-0.13.6.deb && \
- rm sbt-0.13.6.deb && \
- apt-get update && \
- apt-get install sbt && \
pre_build:
commands:
- echo Entered the pre_build phase...
- docker login -u user -p pass
build:
commands:
- echo Build started on `date`
- sbt test
- echo test completed on `date`
- sbt docker:publishLocal
- docker tag image repo
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push repo
cache:
paths:
- $HOME/.ivy2/cache
- $HOME/.sbt
and fails with
/codebuild/output/tmp/script.sh: 4: /codebuild/output/tmp/script.sh: docker: not found
in the console. As far as I see in the examples provided in the doc, docker should be already given.
How can I avoid this?
Thanks
On your CodeBuild project select the "privileged" flag to enable Docker in your build container. If you are using a CodeBuild managed image, then selecting this flag is all that's needed. If you are using a custom image then ensure the Docker is started as explained in https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker-custom-image.html
Related
I am trying to install SonarQube using AWS CodeBuild. I am using a Nodejs: 10 as the run time environment. I am getting the below error when I run the below script as the build spec? As I understood, the issue is the NodeJS env does not contain Maven inbuilt. If that is the case, How can I proceed with Maven with in the Node JS Env. Thanks in advance.
[Container] 2020/07/26 18:16:43 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: mvn test. Reason: exit status 1
Issue occurs when it starts to execute -mvn test
buildspec.yml
version: 0.2
env:
secrets-manager:
LOGIN: SonarCloud:sonartoken
HOST: SonarCloud:HOST
Organization: SonarCloud:Organization
Project: prod/sonar:Project
phases:
install:
runtime-versions:
nodejs: 10
pre_build:
commands:
- npm install
- apt-get update
- apt-get install -y jq
- wget http://www-eu.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
- tar xzf apache-maven-3.5.4-bin.tar.gz
- ln -s apache-maven-3.5.4 maven
- wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip
- unzip ./sonar-scanner-cli-3.3.0.1492-linux.zip
- export PATH=$PATH:/sonar-scanner-3.3.0.1492-linux/bin/
build:
commands:
- mvn test
- mvn sonar:sonar -Dsonar.login=$LOGIN -Dsonar.host.url=$HOST -Dsonar.projectKey=$Project -Dsonar.organization=$Organization
- sleep 5
- curl https://sonarcloud.io/api/qualitygates/project_status?projectKey=$Project >result.json
- cat result.json
- if [ $(jq -r '.projectStatus.status' result.json) = ERROR ] ; then $CODEBUILD_BUILD_SUCCEEDING -eq 0 ;fi
- echo Build started on `date`
- echo Compiling the Node.js code
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- server.js
- package.json
- controller/*
Maven is available in java : openjdk8.
You need to add the same to your yml.
Sample format :
phases:
install:
runtime-versions:
java: openjdk8
build:
commands:
- mvn test
Add either java: corretto11 or java: openjdk8 or java: openjdk11 under runtime-versions: and maven will start executing.
Probably you might need to use your project specific settings.xml for maven build, which you can easily provide in a S3 bucket and then refer it under build commands of buildspec.yml
I'm using corretto rather than openjdk in my aws configuration, as aws provide LTS for corretto. Reference - https://aws.amazon.com/corretto/faqs/
I am trying to run a code pipeline with github as the source, codeBuild as the builder and elastic beanstalk as the server infrastructure. I am using a docker image amazonlinux:2018.03 which works perfectly locally but during the codebuild in the pipeline i get the following error:
docker-compose: command not found
I have tried to install docker, docker-compose etc. but it keeps giving me this error. I've set the build to use a file buildspec.yaml:
version: 0.2
phases:
install:
commands:
- echo "installing"
- sudo yum install -y yum-utils
- sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
- sudo curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
- sudo chmod +x /usr/local/bin/docker-compose
- sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
- docker-compose --version
build:
commands:
- bash compose-local.sh
compose-local.sh:
#!/bin/bash
sudo docker-compose up
I have tried for a couple of days. And i am not sure if i am overseeing something with codeBuild i dont know?
Run /usr/local/bin/docker-compose up instead.
If using Ubuntu 2.0+ or Amazon Linux 2 image, we need to specify docker as the runtime-versions in install phase at buildspec.yml file, e.g.:
version: 0.2
phases:
install:
runtime-versions:
docker: 18
build:
commands:
- echo Build started on `date`
- echo Building the Docker image with docker-compose...
- docker-compose -f docker-compose.yml build
Also please make sure to enable privilege mode: https://docs.aws.amazon.com/codebuild/latest/userguide/create-project.html#create-project-console
I'm using zappa to deploy on aws. And I wanted to implement CI/CD on AWS.
So, I created a pipeline and successfully did Aws COMMIT and AWS BUILD.
I'm unable to deploy the same using AWS CODE DEPLOY.
The Buildspec.yaml looks like this:
version: 0.2
phases:
install:
commands:
- echo Setting up virtualenv
- python -m venv venv
- source venv/bin/activate
- echo Installing requirements from file
- pip install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python tests.py
- flask db upgrade
post_build:
commands:
- echo Build completed on `date`
- echo Starting deployment
- zappa update dev
- echo Deployment completed
How should I execute zappa deploy or zappa update on AWS?
I'm not sure how to add create appspec.yaml file.
Please HELP! Stuck!!
Here's a buildspec.yml file that I use. You could adjust this to suit your needs (for example, including the DB upgrade command).
version: 0.2
phases:
install:
commands:
- mkdir /tmp/src/
- mv $CODEBUILD_SRC_DIR/* /tmp/src/
- cd /tmp/src/
- python3 -m venv docker_env && source docker_env/bin/activate && pip install --upgrade pip==9.0.3 && pip install -r requirements.txt && zappa update production && deactivate && rm -rf docker_env
post_build:
commands:
- cd $CODEBUILD_SRC_DIR
- rm -rf /tmp/src/
- echo Build completed on `date`
Note that this is using the Docker image danielwhatmuff/zappa:python3.6 in CodeBuild. I use this image as it's based on AWS Lambda and has been tuned for Zappa.
Zappa update to Code Deploy:
Your Buildspec.yaml looks fair good but there is one important point to consider.
Postbuild will always run regardless of success/failure. Debug information can be pulled from a failed build.
Either check the reason for failure from build log, or modify your yml to look like below (caution: this is only draft change, test before using in systems):
version: 0.2
phases:
install:
commands:
- yum -y groupinstall development
- yum -y install zlib-devel
- yum -y install openssl-devel
- wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tar.xz
- tar xJf Python-3.6.0.tar.xz
- cd Python-3.6.0
- ./configure
- make
- make install
- ln -s /usr/local/bin/python3.6 /usr/bin/python3
- curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
- python3 get-pip.py
- pip3 install virtualenv
- virtualenv -p /usr/bin/python3 venv
- source venv/bin/activate
- pip3 install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python3 tests.py
- flask db upgrade
post_build:
commands:
- if [ $CODEBUILD_BUILD_SUCCEEDING = 1 ]; then echo Build completed on `date`; echo Starting deployment; zappa update dev; else echo Build failed ignoring deployment; fi
- echo Deployment completed
Hope it answers.
Zappa update to AWS
Below are the steps to do Zappa update on AWS
Configure AWS with IAM user
Configure AWS cli in the local host using command
a. pip install awscli
b. aws configure
Call "Zappa init", it will generate zappa_settings.json based on details provided
Zappa deploy <name provided for environment in step3>
Now your application will be deployed to AWS. Whenever you need to update call
Zappa update <name provided for environment in step3>
I am trying to build a docker image whenever there is push to my source code and move the docker image to the ECR( EC2 Container Registry).
I have tried with the following build-spec file
version: 0.2
env:
variables:
IMG: "app"
REPO: "<<zzzzzzzz>>.dkr.ecr.us-east-1.amazonaws.com/app"
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws ecr get-login --region us-east-1
- TAG=echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8
build:
commands:
- echo $TAG
- docker build -t $IMG:$TAG .
- docker tag $IMG:$TAG $REPO:$TAG
post_build:
commands:
- docker push $REPO:$TAG
- printf Image":"%s:%s" $REPO $TAG > build.json
artifacts:
files: build.json
discard-paths: yes
when I build this I am receiving the error invalid reference format at docker build -t
I looked into the document and found no help.
TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)
you can use $()
version: 0.2
phases:
install:
commands:
- echo Entered the install phase...
- TAG=$(echo "This is test")
pre_build:
commands:
- echo $TAG
build:
commands:
- echo Entered the build phase...
- echo Build started on $TAG
Logs:
[Container] 2018/03/17 16:15:31 Running command TAG=$(echo "This is test")
[Container] 2018/03/17 16:15:31 Entering phase PRE_BUILD
[Container] 2018/03/17 16:15:31 Running command echo $TAG
This is test
So after of lots of retries, i finally found my mistake.
Env CODEBUILD_RESOLVED_SOURCE_VERSION should be replaced with CODEBUILD_SOURCE_VERSION environment variable because I am using codebuild to build directly from source repo in GitHub.
To log in to ecr, need to add --no-include-email option and wrap the command with $(). This will allow you to run docker login. My updated buildspec file would be similar below
version: 0.2
env:
variables:
REPO: "184665364105.dkr.ecr.us-east-1.amazonaws.com/app"
phases:
pre_build:
commands:
- echo $CODEBUILD_SOURCE_VERSION
- TAG=$(echo $CODEBUILD_SOURCE_VERSION | head -c 8)
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email --region us-east-1)
build:
commands:
- echo $TAG
- echo $REPO
- docker build --tag $REPO:$TAG .
post_build:
commands:
- docker push $REPO:$TAG
I tried to follow this doc ( https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-nginx.html ) but couldn't build the custom nginx conf.
I am able to deploy an application and environment and it works. After testing a working environment I wanted to modify some nginx configurations and I followed the steps as:
cd WS
mkdir -p .ebextensions/nginx/conf.d
cp ~/dozee.conf .ebextensions/nginx/conf.d
eb deploy
WS is a directory from where eb deploy works perfectly. After logging(ssh) into the instance created by eb environment I could see the dozee.conf present at /var/app/current/.ebextensions/nginx/conf.d/ but was not present at /etc/nginx/conf.d/.
What I might be missing here? Any help is appreciated :)
The most likely problem is that your .ebextensions folder is not being included in your build. Can you post your buildspec.yml? To give you an idea what needs to happen, here is one of mine:
version: 0.2
phases:
install:
commands:
- echo Entering install phase...
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Entering pre_build phase...
- echo Running tests...
- mvn test
build:
commands:
- echo Entering build phase...
- echo Build started on `date`
- mvn package -Dmaven.test.skip=true
post_build:
commands:
- echo Entering post_build phase...
- echo Build completed on `date`
- mv target/app.war app.war
artifacts:
type: zip
files:
- app.war
- .ebextensions/**/*