Django/React - Azure App Service can't find static files - django

I've deployed my Django React app previously through a dedicated server and now I am trying to achieve the same with Azure Web App function so I can use CI/CD easier. I've configured my project as below but only my django appears to deploy as I get a '404 main.js and index.css not found'.
This makes me think there is an issue with my static file configuration but I'm unsure.
.yml file:
name: Build and deploy Python app to Azure Web App - test123
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
working-directory: ./frontend
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: '3.8'
- name: Create and start virtual environment
run: |
python -m venv venv
source venv/bin/activate
- name: Install dependencies
run: |
pip install -r requirements.txt
python manage.py collectstatic --noinput
- name: Zip artifact for deployment
run: zip pythonrelease.zip ./* -r
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: python-app
path: pythonrelease.zip
# Optional: Add step to run tests here (PyTest, Django test suites, etc.)
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: python-app
path: .
- name: unzip artifact for deployment
run: unzip pythonrelease.zip
- name: 'Deploy to Azure Web App'
uses: azure/webapps-deploy#v2
id: deploy-to-webapp
with:
app-name: 'test123'
slot-name: 'Production'
publish-profile: ${{ secrets.secret}}
settings.py
STATIC_URL = '/static/'
STATIC_ROOT = 'staticfiles'
STATICFILES_DIRS = (
os.path.join(BASE_DIR, "static"),
)
Repo Structure:
Any advice would be greatly appreciated.
Cheers

To host static files in your web app, add the whitenoise package to requirements.txt and the configuration for it to settings.py. as mentioned here : Django Tips
requirements.txt | whitenoise==4.1.2

Related

whitelist AWS RDS on CircleCI

I have a circleCI configuration to run my tests before merge to the master, I start my server to do my tests and the I should connect to my RDS database and its protected with security groups I tried to whitelist circleci ip to allow this happen but with no luck
version: 2.1
orbs:
aws-white-list-circleci-ip: configure/aws-white-list-circleci-ip#1.0.0
aws-cli: circleci/aws-cli#0.1.13
jobs:
aws_setup:
docker:
- image: cimg/python:3.11.0
steps:
- aws-cli/install
- aws-white-list-circleci-ip/add
build:
docker:
- image: cimg/node:18.4
steps:
- checkout
- run: node --version
- restore_cache:
name: Restore Npm Package Cache
keys:
# Find a cache corresponding to this specific package-lock.json checksum
# when this file is changed, this key will fail
- v1-npm-deps-{{ checksum "package-lock.json" }}
# Find the most recently generated cache used from any branch
- v1-npm-deps-
- run: npm install
- run:
name: start the server
command: npm start
background: true
- save_cache:
name: Save Npm Package Cache
key: v1-npm-deps-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: run tests
command: npm run test
- aws-white-list-circleci-ip/remove
workflows:
build-workflow:
jobs:
- aws_setup:
context: aws_context
- build:
requires:
- aws_setup
context: aws_context
my context environment
AWS_ACCESS_KEY_ID
AWS_DEFAULT_REGION
AWS_SECRET_ACCESS_KEY
GROUPID
the error
the orbs I am using
https://circleci.com/developer/orbs/orb/configure/aws-white-list-circleci-ip
I figure it out
version: 2.1
orbs:
aws-cli: circleci/aws-cli#0.1.13
jobs:
build:
docker:
- image: cimg/python:3.11.0-node
steps:
- checkout
- run: node --version
- restore_cache:
name: Restore Npm Package Cache
keys:
# Find a cache corresponding to this specific package-lock.json checksum
# when this file is changed, this key will fail
- v1-npm-deps-{{ checksum "package-lock.json" }}
# Find the most recently generated cache used from any branch
- v1-npm-deps-
- run: npm install
- aws-cli/install
- run:
command: |
public_ip_address=$(wget -qO- http://checkip.amazonaws.com)
echo "this computers public ip address is $public_ip_address"
aws ec2 authorize-security-group-ingress --region $AWS_DEFAULT_REGION --group-id $GROUPID --ip-permissions "[{\"IpProtocol\": \"tcp\", \"FromPort\": 22, \"ToPort\": 7000, \"IpRanges\": [{\"CidrIp\": \"${public_ip_address}/32\",\"Description\":\"CircleCi\"}]}]"
- save_cache:
name: Save Npm Package Cache
key: v1-npm-deps-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: run tests
command: npm run test
# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
build-workflow:
jobs:
- build:
context: aws_context

How can I solve syntax error in yaml file when pushing to github?

I'm using postgresql with django. I set a github action that verifies my code whenever I push or pull, and I get the following error:
You have an error in your yaml syntax on line 19
Here is my yaml:
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: Python application
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:14
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: github_actions
ports:
- 5433:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.9.7
uses: actions/setup-python#v2
with:
python-version: "3.9.7"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Test with Unittest
env:
SECRET_KEY: ${{secrets.SECRET_KEY}}
EMAIL_FROM_USER: ${{secrets.EMAIL_FROM_USER}}
EMAIL_HOST_PASSWORD: ${{secrets.EMAIL_HOST_PASSWORD}}
DB_NAME: ${{secrets.DB_NAME}}
DB_USER: ${{secrets.DB_USER}}
DB_PASSWORD: ${{secrets.DB_PASSWORD}}
DB_HOST: ${{secrets.DB_HOST}}
DB_ENGINE: ${{secrets.DB_ENGINE}}
DB_PORT: ${{secrets.DB_PORT}}
run: |
python3 manage.py test
line 19 corresponds to image: postgres:14 but I can't see any syntax error here. I've looked at some examples and it looks exactly the same.
For GitHub actions, configuring a Django web app service container using the Docker Hub for postgres images works fine with this code only.
image: postgres
For your particular case, you can check if it works for you.
To answer my question, I followed these two posts that are up to date:
https://www.hacksoft.io/blog/github-actions-in-action-setting-up-django-and-postgres
https://www.digitalocean.com/community/tutorials/how-to-use-postgresql-with-your-django-application-on-ubuntu-14-04
Make sure you install all the dependencies.
I also set the port to 5432 and image to postgres:14.2
(To know your postrgesql version you can enter /usr/lib/postgresql/14/bin/postgres -V)
See final yml file:
name: Python application
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:14.2
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: github_action
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.10
uses: actions/setup-python#v2
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Test with Unittest
env:
SECRET_KEY: ${{secrets.SECRET_KEY}}
EMAIL_FROM_USER: ${{secrets.EMAIL_FROM_USER}}
EMAIL_HOST_PASSWORD: ${{secrets.EMAIL_HOST_PASSWORD}}
DB_NAME: ${{secrets.DB_NAME}}
DB_USER: ${{secrets.DB_USER}}
DB_PASSWORD: ${{secrets.DB_PASSWORD}}
DB_HOST: ${{secrets.DB_HOST}}
DB_ENGINE: ${{secrets.DB_ENGINE}}
DB_PORT: ${{secrets.DB_PORT}}
run: |
python3 manage.py test

How to use zappa in gitlab CI/CD to deploy app to AWS Lambda?

I am trying to deploy a flask application on aws lambda via zappa through gitlab CI. Since inline editing isn't possible via gitlab CI, I generated the zappa_settings.json file on my remote computer and I am trying to use this to do zappa deploy dev.
My zappa_settings.json file:
{
"dev": {
"app_function": "main.app",
"aws_region": "eu-central-1",
"profile_name": "default",
"project_name": "prices-service-",
"runtime": "python3.7",
"s3_bucket": -MY_BUCKET_NAME-
}
}
My .gitlab-ci.yml file:
image: ubuntu:18.04
stages:
- deploy
before_script:
- apt-get -y update
- apt-get -y install python3-pip python3.7 zip
- python3.7 -m pip install --upgrade pip
- python3.7 -V
- pip3.7 install virtualenv zappa
deploy_job:
stage: deploy
script:
- mv requirements.txt ~
- mv zappa_settings.json ~
- mkdir ~/forlambda
- cd ~/forlambda
- virtualenv -p python3 venv
- source venv/bin/activate
- pip3.7 install -r ~/requirements.txt -t ~/forlambda/venv/lib/python3.7/site-packages/
- zappa deploy dev
The CI file, upon running, gives me the following error:
Any suggestions are appreciated
zappa_settings.json is commited to the repo and not created on the fly. What is created on the fly is AWS credentials file. Values required are being read from Gitlab env vars set in the web UI of the project.
zappa_settings.json
{
"prod": {
"lambda_handler": "main.handler",
"aws_region": "eu-central-1",
"profile_name": "default",
"project_name": "dummy-name",
"s3_bucket": "dummy-name",
"aws_environment_variables": {
"STAGE": "prod",
"PROJECT": "dummy-name"
}
},
"dev": {
"extends": "prod",
"debug": true,
"aws_environment_variables": {
"STAGE": "dev",
"PROJECT": "dummy-name"
}
}
}
.gitlab-ci.yml
image:
python:3.6
stages:
- test
- deploy
variables:
AWS_DEFAULT_REGION: "eu-central-1"
# variables set in gitlab's web gui:
# AWS_ACCESS_KEY_ID
# AWS_SECRET_ACCESS_KEY
before_script:
# adding pip cache
- export PIP_CACHE_DIR="/home/gitlabci/cache/pip-cache"
.zappa_virtualenv_setup_template: &zappa_virtualenv_setup
# `before_script` should not be overriden in the job that uses this template
before_script:
# creating virtualenv because zappa MUST have it and activating it
- pip install virtualenv
- virtualenv ~/zappa
- source ~/zappa/bin/activate
# installing requirements in virtualenv
- pip install -r requirements.txt
test code:
stage: test
before_script:
# installing testing requirements
- pip install -r requirements_testing.txt
script:
- py.test
test package:
<<: *zappa_virtualenv_setup
variables:
ZAPPA_STAGE: prod
stage: test
script:
- zappa package $ZAPPA_STAGE
deploy to production:
<<: *zappa_virtualenv_setup
variables:
ZAPPA_STAGE: prod
stage: deploy
environment:
name: production
script:
# creating aws credentials file
- mkdir -p ~/.aws
- echo "[default]" >> ~/.aws/credentials
- echo "aws_access_key_id = "$AWS_ACCESS_KEY_ID >> ~/.aws/credentials
- echo "aws_secret_access_key = "$AWS_SECRET_ACCESS_KEY >> ~/.aws/credentials
# try to update, if the command fails (probably not even deployed) do the initial deploy
- zappa update $ZAPPA_STAGE || zappa deploy $ZAPPA_STAGE
after_script:
- rm ~/.aws/credentials
only:
- master
I haven't used zappa in a while, but I remember that a lot of errors that were caused by bad AWS credentials, but zappa reporting something else.

While building project in circleci 2.0 getting apturl==0.5.2 missing error

I have integrated my github project with circleci 2.0. but when i run build from circleci dashboard, i am getting this error.
Could not find a version that satisfies the requirement apturl==0.5.2
(from -r requirements.txt (line 1)) (from versions: )
No matching distribution found for apturl==0.5.2 (from -r requirements.txt (line 1))
Here is my config.yml
# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more
details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
# use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
- image: circleci/python:3.6.1
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/Amazon_customers
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "requirements.txt" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run:
name: install dependencies
command: |
pipenv install
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "requirements.txt" }}
# run tests!
# this example uses Django's built-in test-runner
# other common Python testing frameworks include pytest and nose
# https://pytest.org
# https://nose.readthedocs.io
- run:
name: run tests
command: |
. venv/bin/activate
python manage.py test
- store_artifacts:
path: test-reports
destination: test-reports
And this is my requirements.txt file:
coverage==4.5.1
Django==2.0.6
djangorestframework==3.8.2
pkg-resources==0.0.0
pytz==2018.4
I don't have any apturl==0.5.2 in requirements.txt.How can i resolve this error.
version: 2
jobs:
build:
working_directory: ~/tt-server
docker:
- image: circleci/python3.5
environment:
# Enviroment Variables
steps:
- checkout
- run:
command: pipenv install
- run:
command: "echo mkdir /tmp/artifacts"
- run:
command: |
pipenv run "coverage run manage.py test --parallel=4"
pipenv run "coverage combine"
pipenv run "coverage report -m"
pipenv run "coverage html -d /tmp/artifacts"
pipenv run "coveralls"
- store_artifacts:
path: /tmp/artifacts
replace your congig.yml with this code.Also remove pkg-resources==0.0.0 from requiremnets.txt

Selenium tests fail on CircleCI

I have a Django app I am trying to Selenium tests on CircleCI, but even though they run fine locally on my test environment they keep failing with a NoSuchElementException from Selenium on CircleCI.
At the beginning of most of my browser tests, I run the following method, which is what is making the tests fail:
def login():
driver.get(self.live_server_url + reverse("login"))
# FAILURE HAPPENS HERE: Not able to find the `id_email` element
driver.find_element_by_id("id_email").send_keys(u.email)
driver.find_element_by_id("id_password").send_keys("12345678")
driver.find_element_by_id("submit-login").click()
config.yml
version: 2
jobs:
build:
docker:
- image: circleci/python:3.6.5-node-browsers
environment:
CI_TESTING: 1
- image: redis
working_directory: ~/repo
steps:
- checkout
# Selenium setup
- run: mkdir test-reports
- run:
name: Download Selenium
command: |
curl -O http://selenium-release.storage.googleapis.com/3.5/selenium-server-standalone-3.5.3.jar
- run:
name: Start Selenium
command: |
java -jar selenium-server-standalone-3.5.3.jar -log test-reports/selenium.log
background: true
- restore_cache:
name: Restore Pip Package Cache
keys:
- v1-dependencies-{{ checksum "requirements.txt" }}
- v1-dependencies-
- run:
name: Install Pip Dependencies
command: |
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
- save_cache:
name: Save Pip Package Cache
key: v1-dependencies-{{ checksum "requirements.txt" }}
paths:
- ./venv
- restore_cache:
name: Restore Yarn Package Cache
keys:
- yarn-packages-{{ .Branch }}-{{ checksum "yarn.lock" }}
- yarn-packages-{{ .Branch }}
- yarn-packages-master
- yarn-packages-
- run:
name: Install Yarn Dependencies
command: |
yarn install
- save_cache:
name: Save Yarn Package Cache
key: yarn-packages-{{ .Branch }}-{{ checksum "yarn.lock" }}
paths:
- node_modules/
- run:
name: Run Django Tests
command: |
. venv/bin/activate
./test.sh
- store_artifacts:
path: test-reports
destination: test-reports
Driver definition:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("headless")
drive = webdriver.Chrome(chrome_options=chrome_options)
Is my CircleCI setup wrong? I have looked into multiple pages in documentation and it all seems right to me.
https://circleci.com/docs/2.0/project-walkthrough/#install-and-run-selenium-to-automate-browser-testing
https://github.com/CircleCI-Public/circleci-demo-python-flask/blob/master/.circleci/config.yml#L16:7
https://circleci.com/docs/2.0/browser-testing/