Dockerfile
FROM node:lts-alpine as build-stage
ENV VUE_APP_BACKEND_SERVER=${_VUE_APP_BACKEND_SERVER}
RUN echo "server env is:"
RUN echo $VUE_APP_BACKEND_SERVER
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run gcpbuild
Cloudbuild config
steps:
- name: gcr.io/cloud-builders/docker
args:
- build
- '--no-cache'
- '-t'
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- front
- '-f'
- front/Dockerfile
- '--build-arg=ENV=$_VUE_APP_BACKEND_SERVER'
id: Build
...
...
options:
substitutionOption: ALLOW_LOOSE
substitutions:
_VUE_APP_BACKEND_SERVER: 'https://backend.url'
I have also set the variable in the substitutions in the 'Advanced' section. However during the build the echo prints a blank and the variable is not available in the app as expected.
What you need is:
FROM node:lts-alpine as build-stage
ARG VUE_APP_BACKEND_SERVER
...
Also, fix build-arg line in your cloud build config:
- '--build-arg',
- 'VUE_APP_BACKEND_SERVER=${_VUE_APP_BACKEND_SERVER}'
Check out the docs.
Read more about ARG directive in Dockerfiles.
Related
I have a circleCI configuration to run my tests before merge to the master, I start my server to do my tests and the I should connect to my RDS database and its protected with security groups I tried to whitelist circleci ip to allow this happen but with no luck
version: 2.1
orbs:
aws-white-list-circleci-ip: configure/aws-white-list-circleci-ip#1.0.0
aws-cli: circleci/aws-cli#0.1.13
jobs:
aws_setup:
docker:
- image: cimg/python:3.11.0
steps:
- aws-cli/install
- aws-white-list-circleci-ip/add
build:
docker:
- image: cimg/node:18.4
steps:
- checkout
- run: node --version
- restore_cache:
name: Restore Npm Package Cache
keys:
# Find a cache corresponding to this specific package-lock.json checksum
# when this file is changed, this key will fail
- v1-npm-deps-{{ checksum "package-lock.json" }}
# Find the most recently generated cache used from any branch
- v1-npm-deps-
- run: npm install
- run:
name: start the server
command: npm start
background: true
- save_cache:
name: Save Npm Package Cache
key: v1-npm-deps-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: run tests
command: npm run test
- aws-white-list-circleci-ip/remove
workflows:
build-workflow:
jobs:
- aws_setup:
context: aws_context
- build:
requires:
- aws_setup
context: aws_context
my context environment
AWS_ACCESS_KEY_ID
AWS_DEFAULT_REGION
AWS_SECRET_ACCESS_KEY
GROUPID
the error
the orbs I am using
https://circleci.com/developer/orbs/orb/configure/aws-white-list-circleci-ip
I figure it out
version: 2.1
orbs:
aws-cli: circleci/aws-cli#0.1.13
jobs:
build:
docker:
- image: cimg/python:3.11.0-node
steps:
- checkout
- run: node --version
- restore_cache:
name: Restore Npm Package Cache
keys:
# Find a cache corresponding to this specific package-lock.json checksum
# when this file is changed, this key will fail
- v1-npm-deps-{{ checksum "package-lock.json" }}
# Find the most recently generated cache used from any branch
- v1-npm-deps-
- run: npm install
- aws-cli/install
- run:
command: |
public_ip_address=$(wget -qO- http://checkip.amazonaws.com)
echo "this computers public ip address is $public_ip_address"
aws ec2 authorize-security-group-ingress --region $AWS_DEFAULT_REGION --group-id $GROUPID --ip-permissions "[{\"IpProtocol\": \"tcp\", \"FromPort\": 22, \"ToPort\": 7000, \"IpRanges\": [{\"CidrIp\": \"${public_ip_address}/32\",\"Description\":\"CircleCi\"}]}]"
- save_cache:
name: Save Npm Package Cache
key: v1-npm-deps-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: run tests
command: npm run test
# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
build-workflow:
jobs:
- build:
context: aws_context
I have a Dockerfile:
FROM public.ecr.aws/bitnami/node:15 AS stage-01
COPY package.json /app/package.json
COPY package-lock.json /app/package-lock.json
WORKDIR /app
RUN npm ci
FROM stage-01 AS stage-02
COPY src /app/src
COPY public /app/public
COPY tsconfig.json /app/tsconfig.json
WORKDIR /app
RUN PUBLIC_URL=/myapp/web npm run build
FROM public.ecr.aws/bitnami/nginx:1.20
USER 1001
COPY --from=stage-02 /app/build /app/build
COPY nginx.conf /opt/bitnami/nginx/conf/server_blocks/nginx.conf
COPY ./env.sh /app/build
COPY window.env /app/build
EXPOSE 8080
WORKDIR /app/build
CMD ["/bin/sh", "-c", "/app/build/env.sh && nginx -g \"daemon off;\""]
If I build this image locally it starts normally and does what it has to do.
My local docker version:
Client: Docker Engine - Community
Version: 20.10.7
API version: 1.41
Go version: go1.13.15
Git commit: f0df350
Built: Wed Jun 2 11:56:40 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:52:16 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b63
docker-init:
Version: 0.19.0
GitCommit: de40ad0
If I build it in Codebuild it does not starts:
/app/build/env.sh: 4: /app/build/env.sh: cannot create ./env-config.js: Permission denied
This is the image I am using in codebuild: aws/codebuild/amazonlinux2-x86_64-standard:3.0
I have also run the same script in local and still no error.
What could be the cause of this? If you have something in mind please let me know, otherwise I will post more code
This is my env.sh
#!/usr/bin/env sh
# Add assignment
echo "window._env_ = {" > ./env-config.js
# Read each line in .env file
# Each line represents key=value pairs
while read -r line || [ -n "$line" ];
do
echo "$line"
# Split env variables by character `=`
if printf '%s\n' "$line" | grep -q -e '='; then
varname=$(printf '%s\n' "$line" | sed -e 's/=.*//')
varvalue=$(printf '%s\n' "$line" | sed -e 's/^[^=]*=//')
fi
# Read value of current variable if exists as Environment variable
eval value=\"\$"$varname"\"
# Otherwise use value from .env file
[ -z "$value" ] && value=${varvalue}
echo name: "$varname", value: "$value"
# Append configuration property to JS file
echo " $varname: \"$value\"," >> ./env-config.js
done < window.env
echo "}" >> ./env-config.js
buildspec:
version: 0.2
env:
git-credential-helper: yes
secrets-manager:
GITHUB_TOKEN: "github:GITHUB_TOKEN"
phases:
install:
runtime-versions:
nodejs: 12
commands:
- npm install
build:
commands:
- echo Build started on `date`
- GITHUB_USERNAME=${GITHUB_USERNAME} GITHUB_EMAIL=${GITHUB_EMAIL} GITHUB_TOKEN=${GITHUB_TOKEN} AWS_REGION=${AWS_DEFAULT_REGION} GITHUB_REPOSITORY_URL=${GITHUB_REPOSITORY_URL} ECR_REPOSITORY_URL=${ECR_REPOSITORY_URL} ENV=${ENV} node release.js
My build project terraform configuration:
resource "aws_codebuild_project" "dashboard_image" {
name = var.project.name
service_role = var.codebuild_role_arn
artifacts {
type = "CODEPIPELINE"
}
environment {
compute_type = "BUILD_GENERAL1_SMALL"
image = "aws/codebuild/amazonlinux2-x86_64-standard:3.0"
type = "LINUX_CONTAINER"
privileged_mode = true
environment_variable {
name = "GITHUB_REPOSITORY_URL"
value = "https://github.com/${var.project.github_organization_name}/${var.project.github_repository_name}.git"
}
environment_variable {
name = "ECR_REPOSITORY_URL"
value = var.project.ecr_repository_url
}
environment_variable {
name = "ECR_IMAGE_NAME"
value = var.project.ecr_image_name
}
environment_variable {
name = "ENV"
value = "prod"
}
}
source {
type = "CODEPIPELINE"
buildspec = "buildspec.yml"
}
}
It's all about your Dockerfile and user permissions in it. Try to run docker run public.ecr.aws/bitnami/nginx:1.20 whoami - you will see that this image has not default user. It will be the same if you exec something inside this container. You have to add --user root to run or exec commands. See section "Why use a non-root container?" in Bitnami Nginx image documentation
That's why you don't have permission to create file inside the /app folder. The owner of this folder is root from the first public.ecr.aws/bitnami/node:15 image (which has root user by default).
In order to make it work in your case you have to change the line from USER 1001 to USER root (or someone with proper permissions) and double check that env.sh file has execute permission chmod +x env.sh.
This is the change I had to make to my Dockerfile in order to make it work:
FROM public.ecr.aws/bitnami/node:15 AS stage-01
COPY package.json /app/package.json
COPY package-lock.json /app/package-lock.json
WORKDIR /app
RUN npm ci
FROM stage-01 AS stage-02
COPY src /app/src
COPY public /app/public
COPY tsconfig.json /app/tsconfig.json
WORKDIR /app
RUN PUBLIC_URL=/myapp/web npm run build
FROM public.ecr.aws/bitnami/nginx:1.20
USER root
COPY --from=stage-02 /app/build /app/build
COPY nginx.conf /opt/bitnami/nginx/conf/server_blocks/nginx.conf
COPY ./env.sh /app/build
COPY window.env /app/build
RUN chmod 777 /app/build/env-config.js
EXPOSE 8080
WORKDIR /app/build
USER 1001
CMD ["/bin/sh", "-c", "/app/build/env.sh && nginx -g \"daemon off;\""]
It is probably due to the codebuild permissions when cloning the repository
777 is just temporary, later I will probably test if I can restrict the permissions.
I want to Do CI/CD with CircleCI to ECR, ECS.
Dockerfiles works correctly in local with docker-compose.
but, I am getting the following error in CircleCI.
COPY failed: stat /var/lib/docker/tmp/docker-builder505910231/b-plus-app/build: no such file or directory
Here is the relevant code where the error occurred.
↓Dockerfile(react)↓
FROM node:14.17-alpine
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN rm -r -f b-plus-app/build && cd b-plus-app \
&& rm -r -f node_modules && npm i && npm run build
↓Dockerfile(nginx)↓
FROM nginx:1.15.8
RUN rm -f /etc/nginx/conf.d/*
RUN rm -r -f /usr/share/nginx/html
#Stop Here
COPY b-plus-app/build /var/www
COPY prod_conf/ /etc/nginx/conf.d/
CMD /usr/sbin/nginx -g 'daemon off;' -c /etc/nginx/nginx.conf
↓.circleci/config.yml↓
version: 2.1
orbs:
aws-ecr: circleci/aws-ecr#6.15
aws-ecs: circleci/aws-ecs#2.0.0
workflows:
react-deploy:
jobs:
- persist_to_workspace:
- aws-ecr/build-and-push-image:
account-url: AWS_ECR_ACCOUNT_URL
region: AWS_REGION
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
create-repo: true
path: 'front/'
repo: front
tag: "${CIRCLE_SHA1}"
filters:
branches:
only: main
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build-and-push-image
family: 'b_plus_service'
cluster-name: 'b-plus'
service-name: 'b-plus'
container-image-name-updates: "container=front,tag=${CIRCLE_SHA1}"
nginx-deploy:
jobs:
- aws-ecr/build-and-push-image:
account-url: AWS_ECR_ACCOUNT_URL
region: AWS_REGION
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
create-repo: true
dockerfile: Dockerfile.prod
path: 'front/'
repo: nginx
tag: "${CIRCLE_SHA1}"
#requires:
# - react-deploy:
# - rails-deploy:
filters:
branches:
only: main
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build-and-push-image
family: 'b_plus_service'
cluster-name: 'b-plus'
service-name: 'b-plus'
container-image-name-updates: "container=nginx,tag=${CIRCLE_SHA1}"
If you know how to fix the problem, please let me know. Thank you for reading my question.
I want to deploy aws lamda .net core project using bit bucket pipeline
I have created bitbucket-pipelines.yml like below but after build run getting error -
MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.
file code -
image: microsoft/dotnet:sdk
pipelines:
default:
- step:
caches:
- dotnetcore
script: # Modify the commands below to build your repository.
- export PROJECT_NAME=TestAWS/AWSLambda1/AWSLambda1.sln
- dotnet restore
- dotnet build $PROJECT_NAME
- pipe: atlassian/aws-lambda-deploy:0.2.1
variables:
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
AWS_DEFAULT_REGION: 'us-east-1'
FUNCTION_NAME: 'my-lambda-function'
COMMAND: 'update'
ZIP_FILE: 'code.zip'
project structure is like this -
The problem is here:
PROJECT_NAME=TestAWS/AWSLambda1/AWSLambda1.sln
This is the incorrect path. Bitbucket Pipelines will use a special path in the Docker image, something like /opt/atlassian/pipelines/agent/build/YOUR_PROJECT , to do a Git clone of your project.
You can see this when you click on the "Build Setup" step in the Pipelines web console:
Cloning into '/opt/atlassian/pipelines/agent/build'...
You can use a pre-defined environment variable to retrieve this path: $BITBUCKET_CLONE_DIR , as described here: https://support.atlassian.com/bitbucket-cloud/docs/variables-in-pipelines/
Consider something like this in your yml build script:
script:
- echo $BITBUCKET_CLONE_DIR # Debug: Print the $BITBUCKET_CLONE_DIR
- pwd # Debug: Print the current working directory
- find "$(pwd -P)" -name AWSLambda1.sln # Debug: Show the full file path of AWSLambda1.sln
- export PROJECT_NAME="$BITBUCKET_CLONE_DIR/AWSLambda1.sln"
- echo $PROJECT_NAME
- if [ -f "$PROJECT_NAME" ]; then echo "File exists" ; fi
# Try this if the file path is not as expected
- export PROJECT_NAME="$BITBUCKET_CLONE_DIR/AWSLambda1/AWSLambda1.sln"
- echo $PROJECT_NAME
- if [ -f "$PROJECT_NAME" ]; then echo "File exists" ; fi
I have integrated my github project with circleci 2.0. but when i run build from circleci dashboard, i am getting this error.
Could not find a version that satisfies the requirement apturl==0.5.2
(from -r requirements.txt (line 1)) (from versions: )
No matching distribution found for apturl==0.5.2 (from -r requirements.txt (line 1))
Here is my config.yml
# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more
details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
# use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
- image: circleci/python:3.6.1
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/Amazon_customers
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "requirements.txt" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run:
name: install dependencies
command: |
pipenv install
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "requirements.txt" }}
# run tests!
# this example uses Django's built-in test-runner
# other common Python testing frameworks include pytest and nose
# https://pytest.org
# https://nose.readthedocs.io
- run:
name: run tests
command: |
. venv/bin/activate
python manage.py test
- store_artifacts:
path: test-reports
destination: test-reports
And this is my requirements.txt file:
coverage==4.5.1
Django==2.0.6
djangorestframework==3.8.2
pkg-resources==0.0.0
pytz==2018.4
I don't have any apturl==0.5.2 in requirements.txt.How can i resolve this error.
version: 2
jobs:
build:
working_directory: ~/tt-server
docker:
- image: circleci/python3.5
environment:
# Enviroment Variables
steps:
- checkout
- run:
command: pipenv install
- run:
command: "echo mkdir /tmp/artifacts"
- run:
command: |
pipenv run "coverage run manage.py test --parallel=4"
pipenv run "coverage combine"
pipenv run "coverage report -m"
pipenv run "coverage html -d /tmp/artifacts"
pipenv run "coveralls"
- store_artifacts:
path: /tmp/artifacts
replace your congig.yml with this code.Also remove pkg-resources==0.0.0 from requiremnets.txt