I am trying to install SonarQube using AWS CodeBuild. I am using a Nodejs: 10 as the run time environment. I am getting the below error when I run the below script as the build spec? As I understood, the issue is the NodeJS env does not contain Maven inbuilt. If that is the case, How can I proceed with Maven with in the Node JS Env. Thanks in advance.
[Container] 2020/07/26 18:16:43 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: mvn test. Reason: exit status 1
Issue occurs when it starts to execute -mvn test
buildspec.yml
version: 0.2
env:
secrets-manager:
LOGIN: SonarCloud:sonartoken
HOST: SonarCloud:HOST
Organization: SonarCloud:Organization
Project: prod/sonar:Project
phases:
install:
runtime-versions:
nodejs: 10
pre_build:
commands:
- npm install
- apt-get update
- apt-get install -y jq
- wget http://www-eu.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
- tar xzf apache-maven-3.5.4-bin.tar.gz
- ln -s apache-maven-3.5.4 maven
- wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip
- unzip ./sonar-scanner-cli-3.3.0.1492-linux.zip
- export PATH=$PATH:/sonar-scanner-3.3.0.1492-linux/bin/
build:
commands:
- mvn test
- mvn sonar:sonar -Dsonar.login=$LOGIN -Dsonar.host.url=$HOST -Dsonar.projectKey=$Project -Dsonar.organization=$Organization
- sleep 5
- curl https://sonarcloud.io/api/qualitygates/project_status?projectKey=$Project >result.json
- cat result.json
- if [ $(jq -r '.projectStatus.status' result.json) = ERROR ] ; then $CODEBUILD_BUILD_SUCCEEDING -eq 0 ;fi
- echo Build started on `date`
- echo Compiling the Node.js code
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- server.js
- package.json
- controller/*
Maven is available in java : openjdk8.
You need to add the same to your yml.
Sample format :
phases:
install:
runtime-versions:
java: openjdk8
build:
commands:
- mvn test
Add either java: corretto11 or java: openjdk8 or java: openjdk11 under runtime-versions: and maven will start executing.
Probably you might need to use your project specific settings.xml for maven build, which you can easily provide in a S3 bucket and then refer it under build commands of buildspec.yml
I'm using corretto rather than openjdk in my aws configuration, as aws provide LTS for corretto. Reference - https://aws.amazon.com/corretto/faqs/
Related
I am trying to setup a s3 bucket and github account to create a pipeline in aws codepipeline service. I am getting an error I can't seem to find. It seems to be related to the NPM Install command, but not sure why. Can someone help please?
COMMAND_EXECUTION_ERROR: Error while executing command: npm i. Reason: exit status 1
BuildSpec:
version: 0.2
env:
variables:
CACHE_CONTROL: "86400"
S3_BUCKET: "{{s3_bucket_url}}"
BUILD_FOLDER: "dist"
phases:
install:
runtime-versions:
nodejs: 16
commands:
- echo Installing source NPM dependencies...
- npm install
- npm install -g #angular/cli
build:
commands:
- echo Build started
- ng build
artifacts:
files:
- '**/*'
base-directory: 'dist*'
discard-paths: yes
I am new to create a pipeline in aws. I want to create a ci-cd pipeline for my nuxt project.
I create a yml file in which I want to install nvm and then install node version 12.18.3
The problem is I am getting the nvm not found error.
Can you please check and let me know if there is any error in my yml file:
version: 0.2
phases:
install:
commands:
- echo Installing nvm...
- curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
- export NVM_DIR="$HOME/.nvm"
- '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"'
- '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"'
pre_build:
commands:
#install dependencies
- echo Installing node...
- nvm install 12.18.3
- echo Installing npm...
- npm install
build:
commands:
#build
- echo building...
- npm run generate
artifacts:
files:
- '**/*'
base-directory: dist
cache:
paths:
- node_modules/**/*
Thank you.
You don't actually need to use nvm to install specific node.js version on AWS CodeBuild.
You can use runtime-versions option which would install some version, but you don't have much control over this.
phases:
install:
runtime-versions:
nodejs: 12.x
But AWS standard 5 image comes with n preinstalled (haven't checked 4 but it should be there as well), so you can use it like:
phases:
install:
commands:
- n 12.18.3
and it would install that version same as nvm.
Working Solution
For some reason, It doesn't recognize nvm in the next line. I did not get a chance to investigate it further. The following configuration works. The idea is to set the nvm configurations and install node in the same line.
version: 0.2
phases:
install:
commands:
- echo Installing nvm...
- curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
- export NVM_DIR="$HOME/.nvm"
- '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"'
- '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"'
pre_build:
commands:
#install dependencies
- . "$NVM_DIR/nvm.sh" && nvm install 12.18.3 && echo "node installed by arun"
- echo Installing node...
#- nvm install 12.18.3
- echo Installing npm...
- npm install
build:
commands:
#build
- echo building...
- npm run generate
cache:
paths:
- node_modules/**/*
Troubleshoot in local environment.
In order to troubleshoot the buildspec.yaml, you can run the build locally. here is how to run the buildspec locally.
Trying to use CodeBuild for the first time, pulling data from CodeCommit. But I'm having issues with my buildspec. This is the code I have on it so far:
version: 0.2
phases:
INSTALL:
runtime-versions:
nodejs: 10
commands:
- npm install
PRE_BUILD:
commands:
- npm install --quiet --global expo-cli
- >
if [ -f yarn.lock ]; then
yarn
elif [ -f package-lock.json ] || [ -f npm-shrinkwrap.json ]; then
npm ci
else
npm install
fi
BUILD:
commands:
- expo build:web
artifacts:
baseDirectory: web-build
files:
- '**/*'
name:
myname-$(date +%Y-%m-%d)
cache:
paths:
- node_modules/**/*
- $(npm root --global)/**/*
I have already added the runtime for nodejs 10, it had stopped to trigger this error, but now it kicked again. Does anyone know how to properly tweak it for React-Native web projects?
I believe the phase names are case sensitive, so change them to install, pre_build and build.
I'm using zappa to deploy on aws. And I wanted to implement CI/CD on AWS.
So, I created a pipeline and successfully did Aws COMMIT and AWS BUILD.
I'm unable to deploy the same using AWS CODE DEPLOY.
The Buildspec.yaml looks like this:
version: 0.2
phases:
install:
commands:
- echo Setting up virtualenv
- python -m venv venv
- source venv/bin/activate
- echo Installing requirements from file
- pip install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python tests.py
- flask db upgrade
post_build:
commands:
- echo Build completed on `date`
- echo Starting deployment
- zappa update dev
- echo Deployment completed
How should I execute zappa deploy or zappa update on AWS?
I'm not sure how to add create appspec.yaml file.
Please HELP! Stuck!!
Here's a buildspec.yml file that I use. You could adjust this to suit your needs (for example, including the DB upgrade command).
version: 0.2
phases:
install:
commands:
- mkdir /tmp/src/
- mv $CODEBUILD_SRC_DIR/* /tmp/src/
- cd /tmp/src/
- python3 -m venv docker_env && source docker_env/bin/activate && pip install --upgrade pip==9.0.3 && pip install -r requirements.txt && zappa update production && deactivate && rm -rf docker_env
post_build:
commands:
- cd $CODEBUILD_SRC_DIR
- rm -rf /tmp/src/
- echo Build completed on `date`
Note that this is using the Docker image danielwhatmuff/zappa:python3.6 in CodeBuild. I use this image as it's based on AWS Lambda and has been tuned for Zappa.
Zappa update to Code Deploy:
Your Buildspec.yaml looks fair good but there is one important point to consider.
Postbuild will always run regardless of success/failure. Debug information can be pulled from a failed build.
Either check the reason for failure from build log, or modify your yml to look like below (caution: this is only draft change, test before using in systems):
version: 0.2
phases:
install:
commands:
- yum -y groupinstall development
- yum -y install zlib-devel
- yum -y install openssl-devel
- wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tar.xz
- tar xJf Python-3.6.0.tar.xz
- cd Python-3.6.0
- ./configure
- make
- make install
- ln -s /usr/local/bin/python3.6 /usr/bin/python3
- curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
- python3 get-pip.py
- pip3 install virtualenv
- virtualenv -p /usr/bin/python3 venv
- source venv/bin/activate
- pip3 install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python3 tests.py
- flask db upgrade
post_build:
commands:
- if [ $CODEBUILD_BUILD_SUCCEEDING = 1 ]; then echo Build completed on `date`; echo Starting deployment; zappa update dev; else echo Build failed ignoring deployment; fi
- echo Deployment completed
Hope it answers.
Zappa update to AWS
Below are the steps to do Zappa update on AWS
Configure AWS with IAM user
Configure AWS cli in the local host using command
a. pip install awscli
b. aws configure
Call "Zappa init", it will generate zappa_settings.json based on details provided
Zappa deploy <name provided for environment in step3>
Now your application will be deployed to AWS. Whenever you need to update call
Zappa update <name provided for environment in step3>
I tried to follow this doc ( https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-nginx.html ) but couldn't build the custom nginx conf.
I am able to deploy an application and environment and it works. After testing a working environment I wanted to modify some nginx configurations and I followed the steps as:
cd WS
mkdir -p .ebextensions/nginx/conf.d
cp ~/dozee.conf .ebextensions/nginx/conf.d
eb deploy
WS is a directory from where eb deploy works perfectly. After logging(ssh) into the instance created by eb environment I could see the dozee.conf present at /var/app/current/.ebextensions/nginx/conf.d/ but was not present at /etc/nginx/conf.d/.
What I might be missing here? Any help is appreciated :)
The most likely problem is that your .ebextensions folder is not being included in your build. Can you post your buildspec.yml? To give you an idea what needs to happen, here is one of mine:
version: 0.2
phases:
install:
commands:
- echo Entering install phase...
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Entering pre_build phase...
- echo Running tests...
- mvn test
build:
commands:
- echo Entering build phase...
- echo Build started on `date`
- mvn package -Dmaven.test.skip=true
post_build:
commands:
- echo Entering post_build phase...
- echo Build completed on `date`
- mv target/app.war app.war
artifacts:
type: zip
files:
- app.war
- .ebextensions/**/*