Bitbucket pipelines with different build variables - build

I have a simple Vue.js project that utilizes Vite to build the distribution. I am utilizing dotenv to target specific environments for my deployment via different .env files such as .env and .env.dev where .env may contain
VITE_APP_TITLE=My Site (local)
and .env.dev might contain
VITE_APP_TITLE=My Site (dev)
Running vite build and vite build --mode dev generates the correct distribution with the appropriate substitutions, however, I cannot seem to get a similar behavior from Bitbucket pipelines.
My build pipeline currently looks like this
image: node:14
pipelines:
branches:
develop:
- step:
name: Build and Test
caches:
- node
script:
- npm install
# - npm test
- step:
name: Run build
script:
- npm install
- npm run build:dev
artifacts:
- dist/**
caches:
- node
- step:
name: Deploy to dev
deployment: dev
script:
- echo "Deploying to dev environment"
- pipe: atlassian/aws-s3-deploy:0.3.8
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
S3_BUCKET: $AWS_BUCKET
LOCAL_PATH: 'dist'
ACL: 'public-read'
- step:
name: Invalidate Cloudfront Cache
script:
- pipe: atlassian/aws-cloudfront-invalidate:0.6.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
DISTRIBUTION_ID: $AWS_DISTRIBUTION_ID
PATHS: "/index.html"
I am utilizing the repository "Deployments" setting to add variables for the deployment stage but there does not appear to be a way for me to access these for the build stage as the deployment: setting can only be used during one stage of the pipeline. Has anyone figured out a way to account for different build environment variables during the build stage of the pipeline that could point me in the right direction?

If you want full control, you would need to go with custom steps and variables. But this doesn't allow for easy automation.
So for automation it is about question on when the deployment end would change? If it's based on the branch/tag, then you would just do a separate pipeline with the settings hardcoded into it per different deployment. If it's all going thru the same way, then you are more limited on how to control it.

Related

Next js on AWS amplify, how to set Environment variables

During development, everything works, the application has .env.local file. After installing to amazon amplify, the application does not see variables, I added my keys and values in Environment variables, and I also tried to add them to the console, but then the application breaks
version: 1
frontend:
phases:
preBuild:
commands:
- npm ci
build:
commands:
- EMAIL=${EMAIL}
- EMAIL_PASS=${EMAIL_PASS}
- NEXT_PUBLIC_GOOGLE_ANALYTICS=${NEXT_PUBLIC_GOOGLE_ANALYTICS}
- npm run build
artifacts:
baseDirectory: .next
files:
- '**/*'
cache:
paths:
- node_modules/**/*

Github actions can't seem to find private package in monorepo with yarn workspaces and lerna

I've created a monorepo based on yarn workspaces and lerna which contains the following package types:
UI website package (Vue Vite application)
DTO package (private npm package)
n backend packages (AWS serverless)
And my project structure looks something like this
root
-- package.json
-- packages/
--- ui/
---- package.json
---- dist/
--- dto/
---- package.json
---- dist/
--- serverlessBackend1/
---- package.json
---- build/
--- serverlessBackend2/
---- package.json
---- build/
--- serverlessBackendN/
---- package.json
---- build/
The DTO package contains mostly types, which are used within every other package, therefore it's listed as dependency in every package.json of my packages.
In my root package.json I have the following three basic lerna scripts:
{
[...]
"workspaces": [
"packages/*"
],
"scripts": {
"build": "lerna run build",
"publish": "lerna publish --conventional-commits --yes",
"deploy": "lerna run deploy"
},
"dependencies": {
[...]
},
"devDependencies": {
[...]
}
}
Now I wanted to create a github actions pipeline, which takes care of distributing the different packages to their destinations. Ftp upload for the website bundle, publishing the dto package to npm and deploying all serverless projects to AWS.
As I'm pretty new to Github actions, I've dug my way through offical documentation, readmes, other projects, stackoverflow questions and managed to set up a pipeline, which works in two of three cases.
Unfortunately the step, where I want to deploy all serverless packages to AWS seems to have a weird issue. First, this is how the Job is configured:
Deploy-to-AWS:
runs-on: ubuntu-latest
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
steps:
- name: Checkout
uses: actions/checkout#v2
with:
submodules: recursive
token: ${{ secrets.GITHUB_TOKEN }}
- name: Installing dependencies
run: yarn
- name: Add AWS credentials with profile
run: |
aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY }} --profile ${{ secrets.PROFILE_NAME }}
aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }} --profile ${{ secrets.PROFILE_NAME }}
- name: Deploy to AWS dev
run: yarn deploy
When I execute yarn deploy locally from within my root dir, everything works as expected and the deploy script in each serverless package is executed and all packages are deployed correctly. This is how the package.json does look like in the serverless packages:
{
[...]
"scripts": {
"build": "tsc",
"runDeployment": "serverless deploy -v --aws-profile my-profile-name",
"deploy": "npm run build && npm run runDeployment"
},
"dependencies": {
"#userName/my-private-dto-package": "^0.3.2",
[...]
},
"devDependencies": {
[...]
}
}
But when I try the same within the Github actions workflow, I receive an error that the my private package module cannot be found:
2nd-serverless-package: path/to/file/where/dto/is/imported.ts(1,88): error TS2307: Cannot find module '#userName/my-private-dto-package' or its corresponding type declarations.
This seem to happen to every package but the first. So perhaps the dependency is just resolved for the first package?
I've searched the internet up and down but to no avail.
I think it might has something to do with the dependencies being symlinked and therefore the DTO package is just available on root level and not directly inside the serverless package.
But I solved it by separating the workflows for each serverless package and installing the dependencies directly.
name: Serverless deployment package-name
on:
push:
branches:
- main
paths:
- 'packages/serverlessBackend1/**'
jobs:
Deploy-to-AWS:
runs-on: ubuntu-latest
env:
NODE_AUTH_TOKEN: ${{ secrets.NPMRC_AUTH_TOKEN }}
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup up node
uses: actions/setup-node#v2
- name: Provide profile credentials
run: |
aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY }} --profile my-profile-name
aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }} --profile my-profile-name
- name: Install dependencies
run: cd packages/serverlessBackend1 && npm install
- name: Deploy event backend to AWS dev
run: cd packages/serverlessBackend1 && npm run deploy
This solved it for me entirely. Not the solution to my initial question, therefore not marking it as answer but I thought my findings could perhaps help somebody else.

Expo Web Build in Vercel - Command "npx expo-cli build:web" exited with 1

I want to build and deploy my expo app in vercel. I know that I can build it locally (expo build:web), cd into the build folder and run vercel, but I would like it to be done automatically with source control integration.
So I have connected my github repository, npm install seems to be working ok. The problem is with the build command. I tried expo build:web but this failed because the expo cli is not installed in vercel, so I tried npx expo-cli build:web and got the folloowing output: Command "npx expo-cli build:web" exited with 1.
Error: Could not find MIME for Buffer <null>
at /vercel/.npm/_npx/727/lib/node_modules/expo-cli/node_modules/xdl/src/Webpack.ts:294:23
at finalCallback (/vercel/.npm/_npx/727/lib/node_modules/expo-cli/node_modules/webpack/lib/Compiler.js:257:39)
Does anyone know how I can run expo build:web in vercel? Many thanks
From my perspective, I think you need to use something like GitHub actions for this task.
I wrote a post on How I automated the releases with the expo-cli and I think we need to follow the same logic.
// .github/workflows/staging.yml
name: Expo Publish on Staging
on:
push:
branches:
- develop
jobs:
publish:
name: Install and publish
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- uses: actions/setup-node#v1
with:
node-version: 12.x
- uses: expo/expo-github-action#v5
with:
expo-version: 3.x
expo-username: ${{ secrets.EXPO_CLI_USERNAME }}
expo-password: ${{ secrets.EXPO_CLI_PASSWORD }}
expo-cache: true
- run: yarn
- run: expo build:web <------- not sure if it works tho
- run: <------- here we need a way to upload to vercel
I am in the same situation as you —trying to automate the react-native-web to vercel release.
I had the same issue, but found a solution. expo build:web creates a production ready static bundle in the web-build/ directory, which Vercel can deploy. Thus, to deploy the Expo Web-app to Vercel, one can first have a GHA-step that runs expo build:web and then deploy the bundle it produces to Vercel.
Note that one either has to specify the path to the web-build directory as the root-directory in Vercel (which is done in the code below) or has to set the working-directory to the web-build-path in the Vercel step in the code.
Full Github Actions code for job:
deploy-web-staging:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Setup .npmrc and use node 14.15.1
uses: actions/setup-node#v1
with:
node-version: 14.15.1
- name: Expo Web
uses: expo/expo-github-action#v5
with:
expo-username: ${{ secrets.EXPO_CLI_USERNAME }}
expo-password: ${{ secrets.EXPO_CLI_PASSWORD }}
expo-cache: true
- name: Install dependencies
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn
- name: Build Expo Web
working-directory: ./packages/app
run: expo build:web
- name: Vercel Deploy
uses: amondnet/vercel-action#v20.0.0
with:
vercel-token: ${{ secrets.VERCEL_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
vercel-org-id: VERCEL_ORG_ID
vercel-project-id: VERCEL_PROJECT_ID
scope: TEAM

I'm trying to add env var to django secret_key on github action but showing error

name: MoneyTracker Test
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up python 3.7
uses: actions/setup-python#v2
with:
python-version: 3.7
- name: Install dependency
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Lint with flake8
run: |
pip install flake8
flake8
- name: Coverage report
env:
secret_key: ${{secrets.SECRET_KEY}}
debug: ${{secrets.DEBUG}}
db: ${{secrets.DB}}
run: |
pip install coverage
coverage run MoneyTracker/manage.py test
coverage report
- name: Django testing
run: |
python3 MoneyTracker/manage.py test MoneyTracker
Project link is in here.
How should I add secret key to my project on GitHub action?
Environment variables are case-sensitive. Commands run in actions or steps can create, read, and modify environment variables. To set custom environment variables, you need to specify the variables in the workflow file. You can define environment variables for a step, job, or entire workflow using the jobs.
1. (Recommended way for Secrets) The preferred and secure way is to add the Secret env variables in your GitHub repo settings (See this [Link] for more info. Then you can use those variables with the below syntax in your actions/django.yml file.
env:
SECRET_KEY: ${{ secrets.SECRET_KEY }}
See below image or this Link
Add Secrets in Github Repo
1. (Not recommended for Secrets) You can set the env variables using the below syntax, or you can follow Official Docs Here. But if your repo is public then this method will still expose your SECRET_KEY so I wouldn't recommend this for secrets. However, this method can be used if you want to set env variables like PATH.
env:
SECRET_KEY: your_django_secret_key

How to use github action to deploy a serverless mono repo with multiple packages.json?

I'm trying to deploy micro services that are part of a mono repo, for this I'm using github actions but I'm having an issue related to the plugins in the package.json files. This is the structure of my project:
--repo
---package.json
---resources
----package.json
---services
----Service A
-----package.json
----Service B
-----package.json
First I'm trying to deploy the resources folder that basically creates S3 buckets, cognito user pool, etc... and I have added the plugin called "serverless-cognito-add-custom-attributes" as part of this project, this plugin only exists on the package.json that is inside the "resources" folder.
I'm getting this error when trying to deploy from github actions :
Serverless plugin "serverless-cognito-add-custom-attributes" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
this is the .yml file I'm using on github actions:
name: Deploy Resources to Dev
on:
push:
branches:
- dev
tags:
- RC-*
paths:
- './resources'
jobs:
InstallActions:
name: deploy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Download Node
uses: actions/setup-node#v1
with:
node-version: "12.x"
- name: Install NPM Global Packages
run: |
npm install --global
npm install "./resources" --global
- name: Serverless Deploy
uses: serverless/github-action#master
with:
args: deploy --stage dev --config "./resources/serverless.yml"
env:
AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY_DEV}}
AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_ACCESS_KEY_DEV}}
When the above .yml file run I can see this on the console:
+ ----#1.0.0
added 1 package in 2.935s
+ resources#1.0.0
added 3 packages from 3 contributors in 0.654s
for some reason it seems that
uses: serverless/github-action#master
is unable to find the packages when installed from a sub folder, but doing all manually seems to work fine:
name: Deploy Resources to Dev
on:
push:
branches:
- dev
tags:
- RC-*
paths:
- './resources'
jobs:
Deploy:
name: deploy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Download Node
uses: actions/setup-node#v1
with:
node-version: "12.x"
- name: Install Serverless Framework
run: npm install -g serverless
- name: Serverless Authentication
run: sls config credentials --provider aws --key ${{secrets.AWS_ACCESS_KEY_DEV}} --secret ${{secrets.AWS_SECRET_ACCESS_KEY_DEV}}
- name: Install NPM dependencies
run: |
npm install
npm install "./resources"
- name: Deploy to AWS
run: serverless deploy -v -s dev
working-directory: "./resources"
I had this problem for around 17 hours and then decided to go all manual instead of using the package serverless/github-action#master