I've setted up a Django project using settings modules, a base.py, development.py, and production.py but I'm a little confused on how to run the project locally and remotely.
So, I can run python manage.py runserver --settings=<project>.settings.development and everything goes alright but my doubt is: How can I run it through heroku local giving the settings option? It will also help to run it on remotely with production settings.
You can make use of Environment variables
In __init__.py of your settings directory add:
import os
from .base import *
environment = os.environ.get("ENV")
if environment == development:
from .development import *
else:
from .production import *
this will load the development settings module if the value of ENV is set to development, otherwise by default it will load the production settings module.
Configuring environment variables in Heroku
Related
I'm building a flask application, located in a subdirectory within my project called myapp. Running gunicorn --bind 0.0.0.0:$PORT myapp.app:app works fine, no errors, regardless of what FLASK_APP is set to. When I try to use Flask's CLI, however, it fails to find my app (reasonable), but when I set FLASK_APP to myapp.app., it appears to be doubling up the import path, resulting in an import error:
FLASK_APP=myapp.app flask run
* Serving Flask app 'myapp.app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Try 'flask run --help' for help.
Error: While importing 'myapp.myapp.app', an ImportError was raised.
How can I solve this? Is this a bug in Flask?
There was a __init__.py in my project directory that was causing this.
I dug into the source code of flask in cli.py and found the following code in prepare_import:
# move up until outside package structure (no __init__.py)
while True:
path, name = os.path.split(path)
module_name.append(name)
if not os.path.exists(os.path.join(path, "__init__.py")):
break
Because I had an __init__.py in my project directory (also called myapp), this made flask try to import myapp.app from the ancestor of my project, resulting in myapp.myapp.app.
I want to make specific settings for each environment (local vs staging). I set up Config Vars in my heroku staging app and set DEBUG setting to false to try it out, but it didn't work. Am I missing something or making it wrong?
My seetings.py file
Config Vars in the staging app
Result when I tried somthing wrong
You should create a directory where your current settings.py file is located and name it settings. Then create a base.py, dev.py, and prod.py file in this directory.
Also create an __init__.py in the same location as these 3 settings files and inside that __init__.py put from your_project_name.settings.base import *. In base.py you'll have all the shared settings between prod and dev, and in prod.py and dev.py you would just from .base import * to 'inherit' the settings from the base.py file. This is one of the only cases where it's recommended to import like this.
Then you can set the DJANGO_SETTINGS_MODULE environment variable in production to use my_project_name.settings.prod instead of the default settings variable.
DEBUG in the settings file needs to be set via the environment variable, if available.
So change DEBUG = True to DEBUG = os.environ.get('DEBUG', True) and you should be fine. This is usually called a feature flag (pattern).
Responding:
If you are using a "two scoops" pattern, #wjh18 is on the right path.
The pattern I outlined is solid, in use for years.
Can you see what the python terminal grabs on Heroku via heroku run bash --app APPNAME, then python then import os then os.environ.get('DEBUG'). The should match your settings on Heroku. If so, there may be something in the stack that is inhibiting settings (lazy load) from working correct.
A number of gotcha exist in Django is you deviate from established patterns.
Just in case, the env var is ONLY for the Django settings page, otherwise access the Django DEBUG via proper import of settings (from django.conf import settings).
I have a Django 2.2 project and all secrets are in .env file.
I'm using a library dotenv to load .env to the Django application in the manage.py file
import dotenv
def main():
# Read from .env file
env_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), '.env')
dotenv.read_dotenv(env_file)
....
The environment file is working and is loaded well when running locally.
On the server, I'm using the supervisor to run the application with the following configuration.
[supervisord]
[program:myapp]
command=/var/www/html/app/start_gunicorn.sh
directory=/var/www/html/app/
autostart=true
autorestart=true
stopasgroup=true
stopsignal=QUIT
logfile=/home/ubuntu/log/supervisor/supervisor.log
logfile_maxbytes=5MB
logfile_backups=10
loglevel = info
stderr_logfile=/home/ubuntu/log/supervisor/qcg-backend.err.log
stdout_logfile_maxbytes=5MB
stdout_logfile_backups=10
stdout_logfile=/home/ubuntu/log/supervisor/qcg-backend.out.log
stderr_logfile_maxbytes=5MB
stderr_logfile_backups=10
But the environment variables are not loaded and not working in Django.
Running following command from SSH console is working.
python manage.py shell
import os
os.environ.get('DEBUG')
> True
But running the application, environment variables are not accessible and not applied in the application.
manage.py is not invoked when running Django in production. From the dotenv docs, it says that you should add the loader code to the top of wsgi.py as well.
I think putting it on settings.py is more convenient. No need to add it to both manage.py and wsgi.py
My Django project has multiple settings file for development, production and testing. And I am using supervisor to manage the celery worker. My question how to load the settings file for celery based on the environment I am in.
By using environment variables. Let's say you have the following setting files at the root of your repository.
config.settings.development.py
config.settings.production.py
...
The recommended way to have your celery instance defined is in your config like celery.py module:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
# os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings.production')
app = Celery('proj')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
Instead of setting the DJANGO_SETTINGS_MODULE variable within the module (I have commented that out) make sure that those are present in the environment at the time that supervisord is started.
To set those variables in your staging, testing, and production system you can execute the following bash command.
E.g. on your production system:
$ export DJANGO_SETTINGS_MODULE=config.settings.production
$ echo $DJANGO_SETTINGS_MODULE
I would also suggest you load them from an .env file. In my opinion thats more convenient. You could do that with for example python-dotenv.
Update
The .env file is mostly unique on your different systems and is usually not under source/version control. By unique I mean for development you may have a more verbose LOG_LEVEL or a different SECRET_KEY, because you don't want them to show up in your source control or want to be able to adjust them without modifying your source files.
So, in your base.py (production.py and development.py are inheriting) you can load the variables from the file with for example:
import os
from dotenv import load_dotenv
load_dotenv() # .env file has to be in the same directory
# ...
import os
DJANGO_SETTINGS_MODULE = os.getenv("DJANGO_SETTINGS_MODULE")
print(DJANGO_SETTINGS_MODULE)
# ...
I personally don't use the package since I use docker, which has a declarative way of defining an .env file but the code above should give you an idea how it could work. There are similar packages out there like django-environ which is featured in the book Two Scoops of Django. So I would tend to use this instead of python-dotenv, a matter of taste.
You likely want to configure different settings files. From here you have two options. You can use the django-admin settings param at runtime
django-admin runserver --settings=thecelery.settings
Also, you may have the option to configure settings in the settings.py. If you have a 1 settings file currently, this would require you to setup additional settings files, and set environment variables on the instance. Then in your base settings file, you can do soemthing like this
import os
environment your_env = os.environ["environment"]
if your_env == "celery":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "thecelerysettings")
else:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "defaultsettings")
I am using cookiecutter-django .env design to load different settings depending on environment. Running locally should use "local.py" settings and wunning in aws elatic beanstalk, it should load "dev.py". Both import from "common.py".
Running the server in AES with dev settings works, but collectstatic fails, because it tries importing the local settings instead of dev settings.
How can the EC2 instance run collectstatic and load the (appropriate) dev.py settings?
OK, found it. The manage.py file looked like this
if __name__ == '__main__':
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings.local')
forcing all commands to run with local settings, instead of loading from the .env file.
I have changet it to
import environ
ROOT_DIR = environ.Path(__file__) - 1
env = environ.Env()
env.read_env(ROOT_DIR.file('config/settings/.env'))
if __name__ == '__main__':
os.environ.setdefault('DJANGO_SETTINGS_MODULE', env('DJANGO_SETTINGS_MODULE', default='config.settings.local'))
Which allows manage.py commands to run using whatever settings I have actually specified.