I have a command management that I should run daily at midnight, I usually execute it from my pc, I want to create a cronjob that can do this automatically
here the command that I execute from my local environment:
env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- ./manage.py daily_job
and here the corn job yml file that I wrote:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: update-client-tasks
labels:
environment: application
spec:
schedule: "0 0 * * *"
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: update-client-tasks
image: gcr.io/application-156608/application:v0.5.32
args:
- /usr/bin/curl --data "key=$CRON_KEY" http://application/code/ | env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- ./manage.py daily_job
env:
- name: CRON_KEY
valueFrom:
secretKeyRef:
name: application-secrets
key: cronKey
when running this corn job I got this error:
2021-06-22 09:00:24.184 JSTTraceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line utility.execute() File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 377, in execute django.setup() File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate app_config = AppConfig.create(entry) File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 90, in create module = import_module(entry) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked ModuleNotFoundError: No module named 'django_extensions'
please any help will be very useful
N.B: update
and to resolve this problem of django extensions
I changed the args in the yaml file
args:
- /usr/bin/curl --data "key=$CRON_KEY" http://application/code/ | env=application; kubectl exec -ti $(kubectl get pods| grep $env| awk '{print $1}') -c "application" -- pip3 install django-extensions && python3 manage.py daily_job
I got this error:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 90, in create
module = import_module(entry)
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'django_extensions'
some have downvoted my question due to lack of explication and unclear details,
I really apologize about that, as myself, do not understand well kubernetes and having troubles to understand the documentation
Related
Yesterday, my SAM build was working with the below GitHub actions. Today it suddenly started failing with an error:
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
The SAM template doesn't have any error which I have validated; are any others facing this issue?
name: SAM deploy
on:
push:
branches:
- main
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
- uses: aws-actions/setup-sam#v2
- uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.MY_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.MY_AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: SAM Build
run: sam build --use-container --template-file source/deploy-template.yml
- name: SAM Deploy
run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name my-stack --resolve-s3 --capabilities CAPABILITY_IAM --region us-west-2 --parameter-overrides Environment=npd
GitHub actions error details for the SAM build
Traceback (most recent call last):
496
File "/home/runner/work/_temp/setup-sam-PFBc7r/bin/sam", line 8, in <module>
497
sys.exit(cli())
498
^^^^^
499
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
500
return self.main(*args, **kwargs)
501
^^^^^^^^^^^^^^^^^^^^^^^^^^
502
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
503
rv = self.invoke(ctx)
504
^^^^^^^^^^^^^^^^
505
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1651, in invoke
506
cmd_name, cmd, args = self.resolve_command(ctx, args)
507
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
508
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/click/core.py", line 1698, in resolve_command
509
cmd = self.get_command(ctx, cmd_name)
510
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
511
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/cli/command.py", line 133, in get_command
512
mod = importlib.import_module(pkg_name)
513
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
514
File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
515
return _bootstrap._gcd_import(name[level:], package, level)
516
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
517
File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
518
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
519
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
520
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
521
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
522
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
523
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/build/__init__.py", line 6, in <module>
524
from .command import cli # noqa
505
^^^^^^^^^^^^^^^^^^^^^^^^
506
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/build/command.py", line 12, in <module>
507
from samcli.commands._utils.options import (
508
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/_utils/options.py", line 21, in <module>
509
from samcli.commands._utils.template import get_template_data, TemplateNotFoundException
510
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/samcli/commands/_utils/template.py", line 10, in <module>
511
from botocore.utils import set_value_from_jmespath
512
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/botocore/utils.py", line 37, in <module>
513
import botocore.httpsession
514
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/botocore/httpsession.py", line 46, in <module>
515
from urllib3.contrib.pyopenssl import (
516
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
517
import OpenSSL.crypto
518
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/OpenSSL/__init__.py", line 8, in <module>
519
from OpenSSL import crypto, SSL
520
File "/home/runner/work/_temp/setup-sam-PFBc7r/.venv/lib/python3.11/site-packages/OpenSSL/crypto.py", line 3268, in <module>
521
_lib.OpenSSL_add_all_algorithms()
522
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
523
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'
524
Error: Process completed with exit code 1.
I validated the SAM template
I also tried sudo apt install python3-openssl in the GitHub actions, but it didn't work.
Running this command worked for me:
pip install pyopenssl --upgrade
It started working again after downgrading the version of aws-actions/setup-sam#v2. The issue is with the cryptography version 39.0.0 which was released 13hrs ago; the latest version SAM uses this version, and during the SAM build we are noticing an error.
Refer to the PyPi release history: https://pypi.org/project/cryptography/38.0.4/#history
So whoever using aws-actions/setup-sam#v2 should point back to the SAM version to 1.59.0 which could be a temporary fix:
name: SAM deploy
on:
push:
branches:
- main
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
- uses: aws-actions/setup-sam#v2
with:
version: 1.59.0
- uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.MY_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.MY_AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: SAM Build
run: sam build --use-container --template-file source/deploy-template.yml
- name: SAM Deploy
run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name my-stack --resolve-s3 --capabilities CAPABILITY_IAM --region us-west-2 --parameter-overrides Environment=npd
Please run this command:
pip3 install --force-reinstall cryptography==38.0.4
or
pip3 install --upgrade cryptography==38.0.4
cryptography 39.0.0 no longer supports openssl 1.1.0 or older and thus is causing this issue.
https://cryptography.io/en/latest/changelog/#v39-0-0
I have a django project running perfect in AWS Elastic Beanstalk in awsgi mode.
I want add an pipeline from Bitbucket to simplify deploy process.
bitbucket-pipelines.yml
image: atlassian/default-image:2
pipelines:
default:
- step:
name: "Build and Test"
script:
- echo "Everything is awesome!"
- apt-get update
- apt-get install -y zip
- zip application.zip application/*
- pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
S3_BUCKET: $S3_BUCKET
APPLICATION_NAME: $APPLICATION_NAME
ENVIRONMENT_NAME: $APPLICATION_ENVIRONMENT
COMMAND: 'upload-only'
ZIP_FILE: 'application.zip'
artifacts:
- application.zip
- step:
name: "Deploy to Production"
deployment: production
script:
- echo "Deployment to Production!"
- pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
APPLICATION_NAME: $APPLICATION_NAME
ENVIRONMENT_NAME: $APPLICATION_ENVIRONMENT
COMMAND: 'deploy-only'
WAIT: 'true'
I make a new folder called application and put all my Django project inside.
Bitbucket pipelines works fine but show 502 error.
#edit
/var/log/web.stdout.log
web: Traceback (most recent call last):
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
web: worker.init_process()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
web: super().init_process()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
web: self.load_wsgi()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
web: self.wsgi = self.app.wsgi()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
web: self.callable = self.load()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
web: return self.load_wsgiapp()
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
web: return util.import_app(self.app_uri)
web: File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
web: mod = importlib.import_module(module)
web: File "/usr/lib64/python3.7/importlib/__init__.py", line 127, in import_module
web: return _bootstrap._gcd_import(name[level:], package, level)
web: File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
web: File "<frozen importlib._bootstrap>", line 983, in _find_and_load
web: File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
web: ModuleNotFoundError: No module named 'application'
.ebextensions/django_config.conf
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: 'myproject.wsgi:application'
I am trying to deploy a website using elastic beanstalk and connect it up to a Postgres Database. When trying to makemigrations in django I am getting an error.
I have been able to successfully set up a local Postgres database.
I have been trying for a while with no luck.
This is the Error Log
(venv) C:\Users\dania\Development\my_website>python manage.py makemigrations
Traceback (most recent call last):
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 217, in ensure_connection
self.connect()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\postgresql\base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\psycopg2\__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection timed out (0x0000274C/10060)
Is the server running on host "my_website.c8lfjtthzko9.ap-southeast-2.rds.amazonaws.com" (13.239.177.165) and accepting
TCP/IP connections on port 5432?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\base.py", line 364, in execute
output = self.handle(*args, **options)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\core\management\commands\makemigrations.py", line 101, in handle
loader.check_consistent_history(connection)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\migrations\loader.py", line 283, in check_consistent_history
applied = recorder.applied_migrations()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\migrations\recorder.py", line 73, in applied_migrations
if self.has_table():
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\migrations\recorder.py", line 56, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 256, in cursor
return self._cursor()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 233, in _cursor
self.ensure_connection()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 217, in ensure_connection
self.connect()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 217, in ensure_connection
self.connect()
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\base\base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\django\db\backends\postgresql\base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "C:\Users\dania\Development\my_website\venv\lib\site-packages\psycopg2\__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not connect to server: Connection timed out (0x0000274C/10060)
Is the server running on host "my_website.c8lfjtthzko9.ap-southeast-2.rds.amazonaws.com" (13.239.177.165) and accepting
TCP/IP connections on port 5432?
This is the databases section of my project .settings.py file:
from .base import *
DEBUG = config('DEBUG', cast=bool)
ALLOWED_HOSTS = ['localhost', '127.0.0.1','http://my_website-dev.ap-southeast-2.elasticbeanstalk.com','*******']
AUTH_PASSWORD_VALIDATORS = [
{'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator'},
{'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator'},
{'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator'},
{'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator'}
]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'my_website',
'USER': '******',
'PASSWORD': '*********',
'HOST': 'my_website.*********.ap-southeast-2.rds.amazonaws.com',
'PORT': '5432'
}
}
This is my Django Config File
Container_commands:
01_makemigrations:
command: "source /opt/python/run/venv/bin/activate && python manage.py makemigrations"
leader_only: true
02_migrate:
command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput"
leader_only: true
03_createsu:
command: "source /opt/python/run/venv/bin/activate && python manage.py createsu"
leader_only: true
04_collectstatic:
command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: my_website.settings.base
aws:elasticbeanstalk:container:python:
WSGIPath: "my_website/wsgi.py"
packages:
yum:
httpd24-devel: []
And my requirements.txt
appdirs==1.4.4
autopep8==1.4.4
awsebcli==3.18.2
botocore==1.15.49
cassandra-driver==3.24.0
cement==2.8.2
certifi==2019.3.9
chardet==3.0.4
click==7.1.2
colorama==0.4.3
defusedxml==0.6.0
distlib==0.3.1
Django==2.2
django-allauth==0.39.1
django-countries==5.3.3
django-crispy-forms==1.7.2
django-debug-toolbar==1.10.1
docutils==0.15.2
filelock==3.0.12
future==0.16.0
geomet==0.2.1.post1
idna==2.7
importlib-metadata==1.7.0
importlib-resources==3.0.0
jmespath==0.10.0
mod-wsgi==4.7.1
oauthlib==3.0.1
pathspec==0.5.9
pep8==1.7.1
Pillow==6.2.2
psycopg2-binary==2.8.5
pycodestyle==2.5.0
python-dateutil==2.8.0
python-decouple==3.1
python-slugify==4.0.1
python3-openid==3.1.0
pytz==2018.5
PyYAML==5.3.1
requests==2.20.1
requests-oauthlib==1.2.0
semantic-version==2.5.0
six==1.11.0
slugify==0.0.1
sqlparse==0.2.4
stripe==2.27.0
termcolor==1.1.0
text-unidecode==1.3
urllib3==1.24.2
virtualenv==20.0.30
wcwidth==0.1.9
zipp==3.1.0
I have tried fixing up my Database credentials and creating a new database instance on Amazon RDS but unfortunately no luck.
Thanks for your help in advance!
EDIT: For more information i've added the error log from when I try to deploy this code to elastic beanstalk
Printing Status:
2020-08-16 10:33:18 INFO createEnvironment is starting.
2020-08-16 10:33:19 INFO Using elasticbeanstalk-ap-southeast-2-369458984841 as Amazon S3 storage bucket for environment data.
2020-08-16 10:33:40 INFO Created security group named: sg-049f4a122d881069e
2020-08-16 10:33:43 INFO Created load balancer named: awseb-e-f-AWSEBLoa-UWLA7I0LF6WN
2020-08-16 10:33:58 INFO Created security group named: awseb-e-f9zthb6x58-stack-AWSEBSecurityGroup-UXKL3CPFGSVJ
2020-08-16 10:33:58 INFO Created Auto Scaling launch configuration named: awseb-e-f9zthb6x58-stack-AWSEBAutoScalingLaunchConfiguration-UB2HYII1KHCR
2020-08-16 10:34:46 INFO Created Auto Scaling group named: awseb-e-f9zthb6x58-stack-AWSEBAutoScalingGroup-1EGDGJYCTFOFQ
2020-08-16 10:34:46 INFO Waiting for EC2 instances to launch. This may take a few minutes.
2020-08-16 10:35:01 INFO Created Auto Scaling group policy named: arn:aws:autoscaling:ap-southeast-2:369458984841:scalingPolicy:5681f42e-9672-4f68-8566-fe465044ea90:autoScalingGroupName/awseb-e-f9zthb6x58-stack-AWSEBAutoScalingGroup-1EGDGJYCTFOFQ:policyName/awseb-e-f9zthb6x58-stack-AWSEBAutoScalingScaleUpPolicy-TLZXCZ8CV6Q2
2020-08-16 10:35:01 INFO Created Auto Scaling group policy named: arn:aws:autoscaling:ap-southeast-2:369458984841:scalingPolicy:13bc6e9d-fd8f-4a12-a2bd-74f036335481:autoScalingGroupName/awseb-e-f9zthb6x58-stack-AWSEBAutoScalingGroup-1EGDGJYCTFOFQ:policyName/awseb-e-f9zthb6x58-stack-AWSEBAutoScalingScaleDownPolicy-JUL92X7VVS7T
2020-08-16 10:35:01 INFO Created CloudWatch alarm named: awseb-e-f9zthb6x58-stack-AWSEBCloudwatchAlarmHigh-KVUMT41RNBO5
2020-08-16 10:35:01 INFO Created CloudWatch alarm named: awseb-e-f9zthb6x58-stack-AWSEBCloudwatchAlarmLow-131ZV7XJVRICW
2020-08-16 10:36:03 ERROR [Instance: i-0bf7de5ced912896c] Command failed on instance. Return code: 1 Output: (TRUNCATED)... in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
container_command 02_migrate in .ebextensions/django.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2020-08-16 10:36:03 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2020-08-16 10:37:06 ERROR Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.
And Now included a screenshot of the security group:
[1]: https://i.stack.imgur.com/4LEfH.jpg
Based on the comments and additional info.
The inbound rule for 5432 port is set to security group sg-597. This allows for inbound traffic only from other instances that have the same sg, not from the internet.
To enable access to the rds from the internet have to use either 0.0.0.0/0 as the source (i.e. all source locations), or which is better, specific IP range or address (e.g. 1.2.3.4/32). The range can be the range of your work or home public network, and IP can be the specific address of your workstation.
The following docker-compose.yml works:
version: "3.8"
services:
my_db:
container_name: my_db
image: postgres
environment:
- POSTGRES_DB=ArborMetrix
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=753951456852
# volumes:
# - type: volume
# source: my_db_volume
# target: /my_project
web_app:
container_name: web_app
build: .
command: bash -c "python manage.py makemigrations &&
python manage.py migrate &&
python manage.py runserver 0.0.0.0:8000"
# volumes:
# - type: volume
# source: web_app_volume
# target: /my_project
ports:
- "8000:8000"
depends_on:
- my_db
#volumes:
# my_db_volume:
# web_app_volume:
When I un-comment the comments above to implement the volumes, I get the following error:
Starting my_db ... done
Starting web_app ... done
Attaching to my_db, web_app
my_db |
my_db | PostgreSQL Database directory appears to contain a database; Skipping initialization
my_db |
my_db | 2020-06-26 18:47:07.281 UTC [1] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
my_db | 2020-06-26 18:47:07.282 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
my_db | 2020-06-26 18:47:07.283 UTC [1] LOG: listening on IPv6 address "::", port 5432
my_db | 2020-06-26 18:47:07.314 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
my_db | 2020-06-26 18:47:07.448 UTC [25] LOG: database system was shut down at 2020-06-26 18:46:48 UTC
my_db | 2020-06-26 18:47:07.484 UTC [1] LOG: database system is ready to accept connections
web_app | Traceback (most recent call last):
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
web_app | self.connect()
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
web_app | self.connection = self.get_new_connection(conn_params)
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
web_app | connection = Database.connect(**conn_params)
web_app | File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
web_app | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
web_app | psycopg2.OperationalError: could not connect to server: Connection refused
web_app | Is the server running on host "127.0.0.1" and accepting
web_app | TCP/IP connections on port 5432?
web_app |
web_app |
web_app | The above exception was the direct cause of the following exception:
web_app |
web_app | Traceback (most recent call last):
web_app | File "manage.py", line 21, in <module>
web_app | main()
web_app | File "manage.py", line 17, in main
web_app | execute_from_command_line(sys.argv)
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
web_app | utility.execute()
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
web_app | self.fetch_command(subcommand).run_from_argv(self.argv)
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
web_app | self.execute(*args, **cmd_options)
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
web_app | output = self.handle(*args, **options)
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
web_app | res = handle_func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/makemigrations.py", line 101, in handle
web_app | loader.check_consistent_history(connection)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 283, in check_consistent_history
web_app | applied = recorder.applied_migrations()
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 76, in applied_migrations
web_app | if self.has_table():
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 56, in has_table
web_app | return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 260, in cursor
web_app | return self._cursor()
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 236, in _cursor
web_app | self.ensure_connection()
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
web_app | self.connect()
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
web_app | raise dj_exc_value.with_traceback(traceback) from exc_value
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
web_app | self.connect()
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
web_app | self.connection = self.get_new_connection(conn_params)
web_app | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_app | return func(*args, **kwargs)
web_app | File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
web_app | connection = Database.connect(**conn_params)
web_app | File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
web_app | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
web_app | django.db.utils.OperationalError: could not connect to server: Connection refused
web_app | Is the server running on host "127.0.0.1" and accepting
web_app | TCP/IP connections on port 5432?
web_app |
web_app exited with code 1
References:
In Dockerfile:
FROM python
ENV PYTHONUNBUFFERED 1
ENV COMPOSE_CONVERT_WINDOWS_PATHS=1
RUN mkdir /my_project
WORKDIR /my_project
COPY . /my_project
RUN pip install -r requirements.txt
In settings.py:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql_psycopg2",
"NAME": "ArborMetrix",
"USER": "postgres",
"PASSWORD": "753951456852",
"HOST": "my_db",
"PORT": "5432",
}
}
Problem:
I put the volume in the same directory as the one containing all the source code for web_app.
Fellow user Iain Shelvington commented,
The volume for web_app is overwriting the working directory
/my_project with a named volume.
I didn't know volumes overwrites the ENTIRE directory for itself (selfish!).
Solution:
I changed the target to an empty directory in the container.
target: /my_project/my_db_volume
target: /my_project/web_app_volume
I am following the Django Channels tutorial available at https://channels.readthedocs.io/en/latest/tutorial/part_2.html
I am using docker-compose to build the redis server:
docker-compose.yml
redis:
image: "redis:alpine"
command: redis-server --requirepass letmein
ports:
- "6379:6379"
volumes:
- ./redis.conf:/usr/local/etc/redis/redis.conf
environment:
- REDIS_REPLICATION_MODE=master
networks:
node_net:
ipv4_address: 127.0.0.2
networks:
node_net:
ipam:
driver: default
config:
- subnet: 127.0.0.0/16
When attempting to test the connection with
async_to_sync(channel_layer.send)('test_channel', {'type': 'hello'})
I get the error
`File "<console>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 120, in __call__
return call_result.result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/lib/python3.6/site-packages/asgiref/sync.py", line 180, in main_wrap
result = await self.awaitable(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 299, in send
async with self.connection(index) as connection:
File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 835, in __aenter__
self.conn = await self.pool.pop()
File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 73, in pop
conns.append(await aioredis.create_redis(**self.host, loop=loop))
File "/usr/local/lib/python3.6/site-packages/aioredis/commands/__init__.py", line 175, in create_redis
loop=loop)
File "/usr/local/lib/python3.6/site-packages/aioredis/connection.py", line 113, in create_connection
timeout)
File "/usr/local/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
return (yield from fut)
File "/usr/local/lib/python3.6/site-packages/aioredis/stream.py", line 24, in open_connection
lambda: protocol, host, port, **kwds)
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 794, in create_connection
raise exceptions[0]
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 781, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 439, in sock_connect
return (yield from fut)
File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 469, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.2', 6379)`
The advice available at: Docker [Errno 111] Connect call failed ('127.0.0.1', 6379) describes changing 127.0.0.1:6379 to redis:6379. However it is unclear where this change should occur:
in my settings.py ?
`CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
"hosts": [('127.0.0.2', 6379)],
},
},
}`
or in my docker-compose.yml? or another place? Thank you in advance :)
You have '127.0.0.2' instead of '127.0.0.1' for localhost