How deploy Django Project with Bitbucket Pipelines and AWS Elastic Beanstalk? - django

I have a django project running perfect in AWS Elastic Beanstalk in awsgi mode.
I want add an pipeline from Bitbucket to simplify deploy process.
bitbucket-pipelines.yml
image: atlassian/default-image:2
pipelines:
default:
- step:
name: "Build and Test"
script:
- echo "Everything is awesome!"
- apt-get update
- apt-get install -y zip
- zip application.zip application/*
- pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
S3_BUCKET: $S3_BUCKET
APPLICATION_NAME: $APPLICATION_NAME
ENVIRONMENT_NAME: $APPLICATION_ENVIRONMENT
COMMAND: 'upload-only'
ZIP_FILE: 'application.zip'
artifacts:
- application.zip
- step:
name: "Deploy to Production"
deployment: production
script:
- echo "Deployment to Production!"
- pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
APPLICATION_NAME: $APPLICATION_NAME
ENVIRONMENT_NAME: $APPLICATION_ENVIRONMENT
COMMAND: 'deploy-only'
WAIT: 'true'
I make a new folder called application and put all my Django project inside.
Bitbucket pipelines works fine but show 502 error.
#edit
/var/log/web.stdout.log
web: Traceback (most recent call last):
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
web: worker.init_process()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
web: super().init_process()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
web: self.load_wsgi()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
web: self.wsgi = self.app.wsgi()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
web: self.callable = self.load()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
web: return self.load_wsgiapp()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
web: return util.import_app(self.app_uri)
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
web: mod = importlib.import_module(module)
web: File "/usr/lib64/python3.7/importlib/__init__.py", line 127, in import_module
web: return _bootstrap._gcd_import(name[level:], package, level)
web: File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web: File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web: File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
web: ModuleNotFoundError: No module named 'application'
.ebextensions/django_config.conf
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: 'myproject.wsgi:application'

Related

Error In Deploying Fastapi api to aws elastic beanstalk

I am trying to deploy my fastapi api to aws elastic beanstalk, I zipped my code file app.py with the requirements.txt and .ebextensions folder that contains config file with the following: option_settings:
aws:elasticbeanstalk:application:environment:
PYTHONPATH: "/var/app/current:$PYTHONPATH"
aws:elasticbeanstalk:container:python:
WSGIPath: "app:app"
but I am getting the following error:
Feb 1 16:48:27 ip-172-31-45-195 web: [2023-02-01 16:48:27 +0000] [14516] [ERROR] Error handling request /
Feb 1 16:48:27 ip-172-31-45-195 web: Traceback (most recent call last):
Feb 1 16:48:27 ip-172-31-45-195 web: File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 271, in handle
Feb 1 16:48:27 ip-172-31-45-195 web: keepalive = self.handle_request(req, conn)
Feb 1 16:48:27 ip-172-31-45-195 web: File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 323, in handle_request
Feb 1 16:48:27 ip-172-31-45-195 web: respiter = self.wsgi(environ, resp.start_response)
Feb 1 16:48:27 ip-172-31-45-195 web: TypeError: call() missing 1 required positional argument: 'send'
any help
I tried to play with the config file but nothing worked

SAM build from GitHub actions suddenly failing today: module 'lib' has no attribute 'OpenSSL_add_all_algorithms

Yesterday, my SAM build was working with the below GitHub actions. Today it suddenly started failing with an error:
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
The SAM template doesn't have any error which I have validated; are any others facing this issue?
name: SAM deploy
on:
push:
branches:
- main
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
- uses: aws-actions/setup-sam#v2
- uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.MY_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.MY_AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: SAM Build
run: sam build --use-container --template-file source/deploy-template.yml
- name: SAM Deploy
run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name my-stack --resolve-s3 --capabilities CAPABILITY_IAM --region us-west-2 --parameter-overrides Environment=npd
GitHub actions error details for the SAM build
Traceback (most recent call last):
496
File "/home/runner/work/_temp/setup-sam-PFBc7r/bin/sam", line 8, in <module>
497
sys.exit(cli())
498
^^^^^
499
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
500
return self.main(*args, **kwargs)
501
^^^^^^^^^^^^^^^^^^^^^^^^^^
502
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
503
rv = self.invoke(ctx)
504
^^^^^^^^^^^^^^^^
505
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1651, in invoke
506
cmd_name, cmd, args = self.resolve_command(ctx, args)
507
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
508
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1698, in resolve_command
509
cmd = self.get_command(ctx, cmd_name)
510
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
511
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/cli/command.py", line 133, in get_command
512
mod = importlib.import_module(pkg_name)
513
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
514
File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
515
return _bootstrap._gcd_import(name[level:], package, level)
516
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
517
File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
518
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
519
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
520
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
521
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
522
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
523
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/build/__init__.py", line 6, in <module>
524
from .command import cli # noqa
505
^^^^^^^^^^^^^^^^^^^^^^^^
506
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/build/command.py", line 12, in <module>
507
from samcli.commands._utils.options import (
508
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/_utils/options.py", line 21, in <module>
509
from samcli.commands._utils.template import get_template_data, TemplateNotFoundException
510
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/_utils/template.py", line 10, in <module>
511
from botocore.utils import set_value_from_jmespath
512
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/botocore/utils.py", line 37, in <module>
513
import botocore.httpsession
514
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/botocore/httpsession.py", line 46, in <module>
515
from urllib3.contrib.pyopenssl import (
516
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
517
import OpenSSL.crypto
518
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/OpenSSL/__init__.py", line 8, in <module>
519
from OpenSSL import crypto, SSL
520
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/OpenSSL/crypto.py", line 3268, in <module>
521
_lib.OpenSSL_add_all_algorithms()
522
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
523
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
524
Error: Process completed with exit code 1.
I validated the SAM template
I also tried sudo apt install python3-openssl in the GitHub actions, but it didn't work.
Running this command worked for me:
pip install pyopenssl --upgrade
It started working again after downgrading the version of aws-actions/setup-sam#v2. The issue is with the cryptography version 39.0.0 which was released 13hrs ago; the latest version SAM uses this version, and during the SAM build we are noticing an error.
Refer to the PyPi release history: https://pypi.org/project/cryptography/38.0.4/#history
So whoever using aws-actions/setup-sam#v2 should point back to the SAM version to 1.59.0 which could be a temporary fix:
name: SAM deploy
on:
push:
branches:
- main
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
- uses: aws-actions/setup-sam#v2
with:
version: 1.59.0
- uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.MY_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.MY_AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: SAM Build
run: sam build --use-container --template-file source/deploy-template.yml
- name: SAM Deploy
run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name my-stack --resolve-s3 --capabilities CAPABILITY_IAM --region us-west-2 --parameter-overrides Environment=npd
Please run this command:
pip3 install --force-reinstall cryptography==38.0.4
or
pip3 install --upgrade cryptography==38.0.4
cryptography 39.0.0 no longer supports openssl 1.1.0 or older and thus is causing this issue.
https://cryptography.io/en/latest/changelog/#v39-0-0

Run cron job with curl to execute django command management

I have a command management that I should run daily at midnight, I usually execute it from my pc, I want to create a cronjob that can do this automatically
here the command that I execute from my local environment:
env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- ./manage.py daily_job
and here the corn job yml file that I wrote:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: update-client-tasks
labels:
environment: application
spec:
schedule: "0 0 * * *"
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: update-client-tasks
image: gcr.io/application-156608/application:v0.5.32
args:
- /usr/bin/curl --data "key=$CRON_KEY" http://application/code/ | env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- ./manage.py daily_job
env:
- name: CRON_KEY
valueFrom:
secretKeyRef:
name: application-secrets
key: cronKey
when running this corn job I got this error:
2021-06-22 09:00:24.184 JSTTraceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line utility.execute() File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 377, in execute django.setup() File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate app_config = AppConfig.create(entry) File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 90, in create module = import_module(entry) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked ModuleNotFoundError: No module named 'django_extensions'
please any help will be very useful
N.B: update
and to resolve this problem of django extensions
I changed the args in the yaml file
args:
- /usr/bin/curl --data "key=$CRON_KEY" http://application/code/ | env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- pip3 install django-extensions && python3 manage.py daily_job
I got this error:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 90, in create
module = import_module(entry)
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'django_extensions'
some have downvoted my question due to lack of explication and unclear details,
I really apologize about that, as myself, do not understand well kubernetes and having troubles to understand the documentation

Error while building a simple project. Writes could not find the specified file

./project
/site_ts
/web
Dockerfile
requirements.txt
docker-compose.yml
my docker-compose.yml:
version: '3'
services:
web:
build: ./web
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/site_ts
ports:
- '8000:8000'
my Dockerfile:
FROM python:3.8
ENV PYTHONUNBUFFERED 1
RUN mkdir /site_ts
WORKDIR /site_ts
COPY requirements.txt /site_ts/
RUN pip install --upgrade pip && pip install -r requirements.txt
ADD . /site_ts/
i write docker-compose up and take this Error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose\cli\main.py", line 67, in main
File "compose\cli\main.py", line 123, in perform_command
File "compose\cli\command.py", line 69, in project_from_options
File "compose\cli\command.py", line 132, in get_project
File "compose\cli\docker_client.py", line 43, in get_client
File "compose\cli\docker_client.py", line 170, in docker_client
File "site-packages\docker\api\client.py", line 188, in __init__
File "site-packages\docker\api\client.py", line 213, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: (2, 'CreateFile', 'The specified file cannot be found.')
[9356] Failed to execute script docker-compose

Docker-compose can't find .env file

I have 3 containers with postgresql, nginx and django. The last two have their own Dockerfile, I run whole system with docker-compose. I have multiple problems:
I can't run my custom nginx container, named nginx with build: ./nginx/. only image: nginx. Yes, I try docker-compose build as was suggested in thread.
Strange behavior with volumes, I mentioned theese one: - ./web:/code and nothing work, i commented it, but now Django said me:
/usr/local/lib/python3.5/site-packages/environ/environ.py:609: UserWarning: /code/translation_microservice/.env doesn't exist - if you're not configuring your environment separately, create one.
"environment separately, create one." % env_file)
Traceback (most recent call last):
File "manage.py", line 22, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 337, in execute
django.setup()
File "/usr/local/lib/python3.5/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.5/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models()
File "/usr/local/lib/python3.5/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/usr/local/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 986, in _gcd_import
File "", line 969, in _find_and_load
File "", line 958, in _find_and_load_unlocked
File "", line 673, in _load_unlocked
File "", line 673, in exec_module
File "", line 222, in _call_with_frames_removed
File "/code/api_controller/models.py", line 8, in
from web.translation import TranslatedLesson, TranslatedStep
ImportError: No module named 'web'
My tree of files:
.
├── docker-compose.yml
├── nginx
│   ├── Dockerfile
│   └── nginx.conf
├── __pycache__
├── README.md
├── venv
│   ├── bin
│   ├── include
│   ├── lib
│   └── pip-selfcheck.json
└── web
├── .env
├── Dockerfile
├── manage.py
├── requirements.txt
└── translation
My docker-compose file:
version: '3'
services:
site:
build: ./web
env_file: ./web/.env
# wanna do this
#volumes:
# - ./web:/code
command: python manage.py runserver 0.0.0.0:8000
proxy:
# wanna build: ./nginx/
image: nginx
ports:
- "80:80"
- "443:443"
volumes:
- /www/static
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- /var/lib/postgresql/data/
And finally my both Dockerfiles. Django Dockerfile:
FROM python:3.5
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD ./requirements.txt /code/
RUN pip install --no-cache-dir -r requirements.txt
ADD . /code/
EXPOSE 8000
CMD ["/usr/local/bin/gunicorn", "translation_microservice.wsgi", "-w", "2", "-b", ":8000", "--access-logfile", "-", "--capture-output"]
Nginx Dockerfile:
FROM nginx:latest
COPY ./nginx.conf /etc/nginx/nginx.conf