NVM not found in AWS CI-CD Pipeline - amazon-web-services

I am new to create a pipeline in aws. I want to create a ci-cd pipeline for my nuxt project.
I create a yml file in which I want to install nvm and then install node version 12.18.3
The problem is I am getting the nvm not found error.
Can you please check and let me know if there is any error in my yml file:
version: 0.2
phases:
install:
commands:
- echo Installing nvm...
- curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
- export NVM_DIR="$HOME/.nvm"
- '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"'
- '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"'
pre_build:
commands:
#install dependencies
- echo Installing node...
- nvm install 12.18.3
- echo Installing npm...
- npm install
build:
commands:
#build
- echo building...
- npm run generate
artifacts:
files:
- '**/*'
base-directory: dist
cache:
paths:
- node_modules/**/*
Thank you.

You don't actually need to use nvm to install specific node.js version on AWS CodeBuild.
You can use runtime-versions option which would install some version, but you don't have much control over this.
phases:
install:
runtime-versions:
nodejs: 12.x
But AWS standard 5 image comes with n preinstalled (haven't checked 4 but it should be there as well), so you can use it like:
phases:
install:
commands:
- n 12.18.3
and it would install that version same as nvm.

Working Solution
For some reason, It doesn't recognize nvm in the next line. I did not get a chance to investigate it further. The following configuration works. The idea is to set the nvm configurations and install node in the same line.
version: 0.2
phases:
install:
commands:
- echo Installing nvm...
- curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
- export NVM_DIR="$HOME/.nvm"
- '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"'
- '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"'
pre_build:
commands:
#install dependencies
- . "$NVM_DIR/nvm.sh" && nvm install 12.18.3 && echo "node installed by arun"
- echo Installing node...
#- nvm install 12.18.3
- echo Installing npm...
- npm install
build:
commands:
#build
- echo building...
- npm run generate
cache:
paths:
- node_modules/**/*
Troubleshoot in local environment.
In order to troubleshoot the buildspec.yaml, you can run the build locally. here is how to run the buildspec locally.

Related

Error while executing command: mvn test. Reason: exit status 1 - AWS

I am trying to install SonarQube using AWS CodeBuild. I am using a Nodejs: 10 as the run time environment. I am getting the below error when I run the below script as the build spec? As I understood, the issue is the NodeJS env does not contain Maven inbuilt. If that is the case, How can I proceed with Maven with in the Node JS Env. Thanks in advance.
[Container] 2020/07/26 18:16:43 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: mvn test. Reason: exit status 1
Issue occurs when it starts to execute -mvn test
buildspec.yml
version: 0.2
env:
secrets-manager:
LOGIN: SonarCloud:sonartoken
HOST: SonarCloud:HOST
Organization: SonarCloud:Organization
Project: prod/sonar:Project
phases:
install:
runtime-versions:
nodejs: 10
pre_build:
commands:
- npm install
- apt-get update
- apt-get install -y jq
- wget http://www-eu.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
- tar xzf apache-maven-3.5.4-bin.tar.gz
- ln -s apache-maven-3.5.4 maven
- wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip
- unzip ./sonar-scanner-cli-3.3.0.1492-linux.zip
- export PATH=$PATH:/sonar-scanner-3.3.0.1492-linux/bin/
build:
commands:
- mvn test
- mvn sonar:sonar -Dsonar.login=$LOGIN -Dsonar.host.url=$HOST -Dsonar.projectKey=$Project -Dsonar.organization=$Organization
- sleep 5
- curl https://sonarcloud.io/api/qualitygates/project_status?projectKey=$Project >result.json
- cat result.json
- if [ $(jq -r '.projectStatus.status' result.json) = ERROR ] ; then $CODEBUILD_BUILD_SUCCEEDING -eq 0 ;fi
- echo Build started on `date`
- echo Compiling the Node.js code
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- server.js
- package.json
- controller/*
Maven is available in java : openjdk8.
You need to add the same to your yml.
Sample format :
phases:
install:
runtime-versions:
java: openjdk8
build:
commands:
- mvn test
Add either java: corretto11 or java: openjdk8 or java: openjdk11 under runtime-versions: and maven will start executing.
Probably you might need to use your project specific settings.xml for maven build, which you can easily provide in a S3 bucket and then refer it under build commands of buildspec.yml
I'm using corretto rather than openjdk in my aws configuration, as aws provide LTS for corretto. Reference - https://aws.amazon.com/corretto/faqs/

CodeBuild + ReactNative + Expo Web - This build image requires selecting at least one runtime version

Trying to use CodeBuild for the first time, pulling data from CodeCommit. But I'm having issues with my buildspec. This is the code I have on it so far:
version: 0.2
phases:
INSTALL:
runtime-versions:
nodejs: 10
commands:
- npm install
PRE_BUILD:
commands:
- npm install --quiet --global expo-cli
- >
if [ -f yarn.lock ]; then
yarn
elif [ -f package-lock.json ] || [ -f npm-shrinkwrap.json ]; then
npm ci
else
npm install
fi
BUILD:
commands:
- expo build:web
artifacts:
baseDirectory: web-build
files:
- '**/*'
name:
myname-$(date +%Y-%m-%d)
cache:
paths:
- node_modules/**/*
- $(npm root --global)/**/*
I have already added the runtime for nodejs 10, it had stopped to trigger this error, but now it kicked again. Does anyone know how to properly tweak it for React-Native web projects?
I believe the phase names are case sensitive, so change them to install, pre_build and build.

how to deploy to aws using ci/cd for zappa(python)

I'm using zappa to deploy on aws. And I wanted to implement CI/CD on AWS.
So, I created a pipeline and successfully did Aws COMMIT and AWS BUILD.
I'm unable to deploy the same using AWS CODE DEPLOY.
The Buildspec.yaml looks like this:
version: 0.2
phases:
install:
commands:
- echo Setting up virtualenv
- python -m venv venv
- source venv/bin/activate
- echo Installing requirements from file
- pip install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python tests.py
- flask db upgrade
post_build:
commands:
- echo Build completed on `date`
- echo Starting deployment
- zappa update dev
- echo Deployment completed
How should I execute zappa deploy or zappa update on AWS?
I'm not sure how to add create appspec.yaml file.
Please HELP! Stuck!!
Here's a buildspec.yml file that I use. You could adjust this to suit your needs (for example, including the DB upgrade command).
version: 0.2
phases:
install:
commands:
- mkdir /tmp/src/
- mv $CODEBUILD_SRC_DIR/* /tmp/src/
- cd /tmp/src/
- python3 -m venv docker_env && source docker_env/bin/activate && pip install --upgrade pip==9.0.3 && pip install -r requirements.txt && zappa update production && deactivate && rm -rf docker_env
post_build:
commands:
- cd $CODEBUILD_SRC_DIR
- rm -rf /tmp/src/
- echo Build completed on `date`
Note that this is using the Docker image danielwhatmuff/zappa:python3.6 in CodeBuild. I use this image as it's based on AWS Lambda and has been tuned for Zappa.
Zappa update to Code Deploy:
Your Buildspec.yaml looks fair good but there is one important point to consider.
Postbuild will always run regardless of success/failure. Debug information can be pulled from a failed build.
Either check the reason for failure from build log, or modify your yml to look like below (caution: this is only draft change, test before using in systems):
version: 0.2
phases:
install:
commands:
- yum -y groupinstall development
- yum -y install zlib-devel
- yum -y install openssl-devel
- wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tar.xz
- tar xJf Python-3.6.0.tar.xz
- cd Python-3.6.0
- ./configure
- make
- make install
- ln -s /usr/local/bin/python3.6 /usr/bin/python3
- curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
- python3 get-pip.py
- pip3 install virtualenv
- virtualenv -p /usr/bin/python3 venv
- source venv/bin/activate
- pip3 install -r requirements.txt
build:
commands:
- echo Build started on `date`
- echo Building and running tests
- python3 tests.py
- flask db upgrade
post_build:
commands:
- if [ $CODEBUILD_BUILD_SUCCEEDING = 1 ]; then echo Build completed on `date`; echo Starting deployment; zappa update dev; else echo Build failed ignoring deployment; fi
- echo Deployment completed
Hope it answers.
Zappa update to AWS
Below are the steps to do Zappa update on AWS
Configure AWS with IAM user
Configure AWS cli in the local host using command
a. pip install awscli
b. aws configure
Call "Zappa init", it will generate zappa_settings.json based on details provided
Zappa deploy <name provided for environment in step3>
Now your application will be deployed to AWS. Whenever you need to update call
Zappa update <name provided for environment in step3>

AWS CodeBuild - docker: not found

I have the following buildspec.yml:
version: 0.2
phases:
install:
commands:
- curl -L -o sbt-0.13.6.deb http://dl.bintray.com/sbt/debian/sbt-0.13.6.deb && \
- dpkg -i sbt-0.13.6.deb && \
- rm sbt-0.13.6.deb && \
- apt-get update && \
- apt-get install sbt && \
pre_build:
commands:
- echo Entered the pre_build phase...
- docker login -u user -p pass
build:
commands:
- echo Build started on `date`
- sbt test
- echo test completed on `date`
- sbt docker:publishLocal
- docker tag image repo
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker image...
- docker push repo
cache:
paths:
- $HOME/.ivy2/cache
- $HOME/.sbt
and fails with
/codebuild/output/tmp/script.sh: 4: /codebuild/output/tmp/script.sh: docker: not found
in the console. As far as I see in the examples provided in the doc, docker should be already given.
How can I avoid this?
Thanks
On your CodeBuild project select the "privileged" flag to enable Docker in your build container. If you are using a CodeBuild managed image, then selecting this flag is all that's needed. If you are using a custom image then ensure the Docker is started as explained in https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker-custom-image.html

Is there a way to change directory on AWS codebuild

With Snap-CI going away I've been trying to get our builds working on AWS CodeBuild. I have my buildspec.yml built out, but changing directories doesn't seem to work.
version: 0.1
phases:
install:
commands:
- apt-get update -y
- apt-get install -y node
- apt-get install -y npm
build:
commands:
- cd MyDir //Expect to be in MyDir now
- echo `pwd` //Shows /tmp/blablabla/ instead of /tmp/blablabla/MyDir
- npm install //Fails because I'm not in the right directory
- bower install
- npm run ci
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- MyDir/MyFile.war
discard-paths: yes
It seems like this should be fairly simple, but so far I haven't had any luck getting this to work.
If you change the buildspec.yml version to 0.2 then the shell keeps its settings.
In version: 0.1 you get a clean shell for each command.
Each command in CodeBuild runs in a separate shell against the root of your source (access root of your source from CODEBUILD_SRC_DIR environment variable).
Your possible options are
Short circuit the commands to run under the same shell: Works when you have relatively simple buildspec (like yours).
commands:
- cd MyDir && npm install && bower install
- cd MyDir && npm run ci
Move your commands from buildspec to a script and have more control (useful for more complicated build logic).
commands:
- ./mybuildscipt.sh
Let me know if any of these work for you.
-- EDIT --
CodeBuild has since launched buildspec v0.2 where this work around is no longer required.