I'm trying to run a Django application from a cloned repository and I noticed that it has environment variables stored in the settings.py file(namely: the SECRET_KEY and DEBUG).
When running the application, it gives me the following error:
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable
I understand that Django cannot run without it but I have the following doubts regarding this problem.
Should I provide my own SECRET_KEY and declare it inside a .env file.
Also, is it necessary to have the same SECRET_KEY as the original project file did?
Should I provide my own SECRET_KEY and declare it inside a .env file.
Yes, you can either set the required environment variable in the shell where you run the django server or put it in a .env file.
Also, is it necessary to have the same SECRET_KEY as the original project file did?
No, you don't need the same value as the original project did. SECRET_KEY is used to salt. SECRET_KEY is used for cryptographic siging in sessions, password reset tokens, etc. For more detials see the documentation. The only restriction is that you must maintain the same SECRET_KEY for an instance of the django app. Otherwise sessions and other signed data will be invaliated.
First, welcome to SO!
Second, it's best practice to store the secret key and other sensitive information (database password and so on) in environment variables. However, if you're just cloning a repo to practice in your local machine, you can use the cloned one until you think about deploying and version control.
One way you can do that quite easy in my opinion is django-environ. Check it out: https://django-environ.readthedocs.io/en/latest/
Regarding your question about the secret key, there is a reason why it is called a secret key: keep it a secret! In development, it's somewhat fine to temporarily use the cloned one but always make sure to keep the key secret for production. If you use version control (such as Git) the .env file should not be included to avoid incidents.
python-decouple is the alternative for django-environ , in case if you get any error using django-environ.
STEP 1 :- pip install python-decouple
STEP 2 :- open settings.py file
STEP 3 :- Import the config object:
from decouple import config
STEP 4 :- Retrieve the configuration parameters:
SECRET_KEY = config('SECRET_KEY')
STEP 5 :- create a .env text file in your repository’s root directory in the form:
SECRET_KEY=YOUR_SECRET_KEY
Note :- Remember , in .env file don't give spaces .
Visit :- https://pypi.org/project/python-decouple/
for more detailed explaination & documentation.
I'm having trouble understanding the switchover from local to production settings for deploying Django projects. I'm using an Ubuntu virtual machine (VM) if it matters.
I understand how to configure my settings. I understand best practices for creating settings files (base.py, local.py, production.py, blah, blah). I know that in local development DEBUG=True, in production DEBUG=False, blah, blah.
But how do I implement this switchover in deployment? Do I get rid of the local.py? Do I create some kind of logic so that my VM only reads base.py and production.py?
What's the best approach?
I'm not sure about the best approach but what I do works...
At the foot of my settings.py, I have:
try:
from local_settings import *
except ImportError, e:
pass
I keep all my local development settings in local_settings.py which overide any production settings. I also make sure not to upload my local_settings.py file!
What you could do is to check in your settings py which environment is used at the moment.
To do this you can set an environment variable on your system that would have diffrent values on your development environment and the production environment.
you can set these environment variables by
sudo -H gedit /etc/environment
and add the following line in the file:
DEBUG="true"
(to make this changes available you will have to log out and log back in into your system)
in the production environment you would then set DEBUG="false".
then you can do this in your settings.py:
DEBUG = os.environ.get('DEBUG', 'true') != 'false'
and then you can set every setting that would be different depending on the environment that is used like this:
if DEBUG:
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
else:
STATICFILES_STORAGE = STATICFILES_STORAGE = 'custom_storages.StaticStorage'
(the setting above uses the local django server to serve staticfiles if it is in the development environment, and amazon s3 with boto if in the production environment (which is defined in the custom_storages module)
This way you can push your updates and always the right settings should be picked up depending on the environment.
I have looked everywhere and all I could find was outdated or bits and pieces from several different sources, but none of them had a complete step of how to deploy Django to a production environment.
What I would like to see is the recommended way to deploy Django utilizing git, local and production settings.py, .gitignore specifically for Django.
How would I implement the settings to accommodate both environments and what should be added to the .gitignore file so that only necessary files are sent to the git repository?
Please note that I want to know best practices and up to date methods of using local and production settings.
Such questions arise when deploying Django apps.
Where should the settings be stored?
How can I properly implement the settings to accommodate both environments?
What sensitive information should be separate from the main settings?
Update 25/09/2016:
Thanks andreas for clarifying this and now I completely agree that there is no standard way of deploying Django. I went on a long research and found the most common used methods to deploy Django settings. I will share my findings with everyone who's having the same doubt.
Answering
Where should the settings be stored?
The settings are better accommodated within a settings module rather than a single settings.py file.
How can I properly implement the settings to accommodate both environments?
This is where the settings module comes in very handy because you can simply have environment independent settings and they are maintainable with git or any other version control.
What sensitive information should be separate from the main settings?
Sensitive information such as database, email, secret key and anything with authentication related information and as to how you will serve this private information secretly, I do recommend to use andreas suggestion.
You could either create a separate file at /etc/app_settings and make it very restrict or simply use environment variables from within the system.
.gitignore part
For the .gitignore part I found an awesome repo where you can get all sort of examples at https://github.com/github/gitignore.
I ended up using:
# OSX Finder turds
.DS_Store
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# Distribution / packaging
.Python
env/
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Django stuff:
*.log
my_app/settings/dev.py
bower_components
Because the answer ended up being so long I've created an article explaining every step of the way at Django Settings Deployment.
There is not one recommended way of deploying a Django app. The official documentation provides a guide for deploying with Apache and mod_wsgi. Some other options are a PaaS like Heroku/Dokku or deployment with Docker
It's common to divide your settings in different files. You could for example divide settings in four different files:
base file (base.py) - Common settings for all environments (every file below imports from this with from .base import *
development file (development.py) - settings specific for development, (DEBUG = True etc...)
production file (production.py) - settings specific for the production environment. (DEBUG = False, whitenoise configuration etc)
testing file (testing.py)
You can control what settings file is loaded with the environment variable DJANGO_SETTINGS_MODULE
It is also common recommended practice that you store secrets like the SECRET_KEY in environment variables. SECRET_KEY = os.environ.get('SECRET_KEY', None). Or check out the django-environ package.
Check out this repo for an example setup.
I'm setting up a Django project with different files for local and production settings. I can confirm that my Django secret key is successfully in an environment variable in virtualenv and when I do runserver I get no error. However when I try manage.py syncdb I get
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
I don't understand why I can successfully browse to the site after runserver but I can't sync the database. When I run env I can see that the secret key is there and in my base settings file (imported into local settings) I am doing this:
SECRET_KEY = os.environ.get('MY_SECRET_KEY')
Any help debugging this would be greatly appreciated.
Euan
I'm not sure why the runserver command is working while syncdb isn't, but you can sort it out by adding a environment variable for DJANGO_SETTINGS_MODULE in the same way you did for the SECRET_KEY. The only difference is that you don't need to reference DJANGO_SETTINGS_MODULE within the django code anywhere. I'm running my own setup in exactly that way and the only problem I run into is forgetting to change the settings module when I switch between projects :-)
EDIT: I didn't realise that you were adding --settings=myapp.settings.local to runserver as well as syncdb. The reason you need to do this is that you are using settings on a different path from the default so python can't find them. Also, although you set the DJANGO_SETTINGS_MODULE in the wsgi file, this is only fired when the site is accessed via your webserver. When running a manage command the wsgi file is ignored (AFAIK) so adding DJANGO_SETTINGS_MODULE to your environment variables in the same way as SECRET_KEY makes your settings file available to the manage command.
Hope that helps
Somewhat similar situation here, using virtualenv running an outer script which involves django models.
To make this work please make sure:
Your sys.path list has a path to your virtualenv site-packages. For me it's: sys.path.append('/home/user/.virtualenvs/Project/local/lib/python2.7/site-packages')
Your django settings variable is added to os.environ. Eg: os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Project.settings")
For the life of me, I have been looking for this everywhere and have not found the answer. I hope I am not posting a duplicate.
It is advised everywhere that you should keep your secret keys in a separate file from your general settings.py. Also, that you should never commit your "secret.py" file that contains keys such as SECRET_KEY, AWS_SECRET_KEY and so on.
My question is: In your production server, you need to reference your secret keys, that means that your "secret.py" settings file, should live somewhere around the server right? If so, how do you protect your secret keys in production?
I wanted to add a new answer because, as a beginner, the previous accepted answer didn't make a lot of sense to me (it was only one part of the puzzle).
So here's how I store my keys both LOCALLY and in PRODUCTION (Heroku, and others).
Note: You really only have to do this if you plan on putting your project online. If it's just a local project, no need.
I also made a video tutorial for people who prefer that format.
1) Install python-dotenv to create a local project environment to store your secret key.
pip install python-dotenv
2) Create a .env file in your base directory (where manage.py is).
YourDjangoProject
├───project
│ ├───__init__.py
│ ├───asgi.py
│ ├───settings.py
│ ├───urls.py
│ └───wsgi.py
├───.env
├───manage.py
└───db.sqlite3
If you have a Heroku project, it should look something like this:
YourDjangoProject
├───.git
├───project
│ ├───__init__.py
│ ├───asgi.py
│ ├───settings.py
│ ├───urls.py
│ └───wsgi.py
├───venv
├───.env
├───.gitignore
├───manage.py
├───Procfile
├───requirements.txt
└───runtime.txt
3) Add .env to your .gitignore file.
echo .env > .gitignore # Or just open your .gitignore and type in .env
This is how you keep your secret key more secure because you don't upload your .env file to git or heroku (or wherever else).
4) Add your SECRET_KEY from your settings.py file into the .env file like so (without quotes)
**Inside of your .env file**
SECRET_KEY=qolwvjicds5p53gvod1pyrz*%2uykjw&a^&c4moab!w=&16ou7 # <- Example key, SECRET_KEY=yoursecretkey
5) Inside of your settings.py file, add the following settings:
import os
import dotenv # <- New
# Add .env variables anywhere before SECRET_KEY
dotenv_file = os.path.join(BASE_DIR, ".env")
if os.path.isfile(dotenv_file):
dotenv.load_dotenv(dotenv_file)
# UPDATE secret key
SECRET_KEY = os.environ['SECRET_KEY'] # Instead of your actual secret key
or, thanks to #Ashkay Chandran's answer:
from dotenv import load_dotenv, find_dotenv
load_dotenv(find_dotenv())
SECRET_KEY = os.environ['SECRET_KEY']
And now your secret key is successfully stored locally.
Update: I found out you can also use the config method from the package python-decouple that seems to be a bit easier:
from decouple import config
SECRET_KEY = config('SECRET_KEY')
Now you don't need to import os or use dotenv because it takes care of those parts for you AND will still use the .env file. I started using this in all of my projects.
6) Add the SECRET_KEY environment variable on your host (such as Heroku).
I work mostly with Heroku sites, so if you're wanting to use Heroku for a Django project, this part is for you.
This assumes that you already have a Heroku project setup and have Heroku CLI downloaded on your computer.
You have 2 options:
From Command Line / Terminal, you can enter the following command in your project's directory:
heroku config:set SECRET_KEY=yoursecretkey # Again, no quotes.
You can go to your Heroku dashboard, click on your app, go to your apps settings, and see the "Config Vars" section and click "Reveal Vars" or "Add Vars" and add your SECRET_KEY there.
Then, when you push your project to Heroku through git, it should be working properly without any issue.
and that's it! 🙂
This answer was targeted towards total beginners / intermediates to hopefully cut through any confusion (because it was definitely confusing for me).
See the Django deployment docs for a discussion on this.
There's quite a few options for production. The way I do it is by setting my sensitive data variables as environmental variables on the production environments. Then I retrieve the variables in the settings.py via os.environ like so:
import os
SECRET_KEY = os.environ['SECRET_KEY']
Another possible option is to copy in the secret.py file via your deploy script.
I'm sure there are also other specific options for different web servers.
You should store your settings in a modular way. By that I mean to spread your settings across multiple files.
For example, you can have base_settings.py to store all your base settings; dev_settings.py for your development server settings; and finally prod_base_settings.py for all production settings. All non-base settings files will import all the base settings and then only change whatever is necessary:
# base_settings.py
...
# dev_settings.py
from base_settings import *
DEBUG = TRUE
...
# prod_base_settings.py
from base_settings import *
DEBUG = FALSE
...
This approach allows you to have different settings from different setups. You can also commit all these files except then on the production server you can create the actual production settings file prod_settings.py where you will specify all the sensitive settings. This file should not be committed anywhere and its content kept secure:
# prod_settings.py
from prod_base_settings import *
SECRET_KEY = 'foo'
As for the file names you can use whatever filenames you feel are appropriate. Personally I actually create a Python package for the settings and then keep the various settings inside the package:
project/
project/
settings/
__init__.py
base.py
dev.py
...
app1/
models.py
...
app2/
models.py
...
Storing secrets in the environment still places them in the environment; which can be exploited if an unauthorized user gains access to the environment. It is a trivial effort to list environment variables, and naming one SECRET makes is all the more helpful and obvious to a bad actor an unwanted user.
Yet secrets are necessary in production, so how to access them while minimizing attack surface? Encrypt each secret in a file with a tool like git-secret, then allow authorized users to read in the file, as mentioned in django's docs. Then "tell" a non-root user the secret so it can be read-in during initialization.
(Alternatively, one could also use Hashicorp's Vault, and access the secrets stored in Vault via the HVAC python module.)
Once this non-root user is told, something like this is easy:
# Remember that './secret_key.txt' is encrypted until it's needed, and only read by a non-root user
with open('./secret_key.txt') as f:
SECRET_KEY = f.read().strip()
This isn't perfect, and, yes, an attacker could enumerate variables and access it -- but it's very difficult to do so during run-time, and Django does a good job of protecting its keys from such a threat vector.
This is a much safer approach than storing secrets in the environment.
I know it has been a long time, but I just opensourced a small Django app I am using to generate a new secret key if it does not exist yet. It is called django-generate-secret-key.
pip install django-generate-secret-key
Then, when provisioning / deploying a new server running my Django project, I run the following command (from Ansible):
python manage.py generate_secret_key
It simply:
checks if a secret key needs to be generated
generates it in a secretkey.txt file (can be customized)
All you need then is to have in your settings file:
with open('/path/to/the/secretkey.txt') as f:
SECRET_KEY = f.read().strip()
You can now benefit from a fully automated provisioning process without having to store a static secret key in your repository.
Instead of if/then logic you should use a tool designed for factoring out sensitive data. I use YamJam https://pypi.python.org/pypi/yamjam/ . It allows all the advantages of the os.environ method but is simpler -- you still have to set those environ variables, you'll need to put them in a script somewhere. YamJam stores these config settings in a machine config store and also allows a project by project ability to override.
from YamJam import yamjam
variable = yamjam()['myproject']['variable']
Is the basic usage. And like the os.environ method, it is not framework specific, you can use it with Django or any other app/framework. I've tried them all, multiple settings.py files, brittle logic of if/then and environment wrangling. In the end, I switched to yamjam and haven't regretted it.
Adding to zack-plauch's answer,
To get the path to the .env file, when using python-dotenv module, the find_dotenv method can be used,
from dotenv import load_dotenv, find_dotenv
load_dotenv(find_dotenv())
SECRET_KEY = os.environ['SECRET_KEY']
The find_dotenv() looks for a ".env" file in the path, so it can be saved inside the same directory too,
Also, if a name is used for the .env file like "django-config.env", load_dotenv(find_dotenv("django-config.env"), will fetch and load that to host-machine environment variable mappings.
I am surprised that noone has talked about django-environ.
I usually create a .env file like this:
SECRET_KEY=blabla
OTHER_SECRET=blabla
This file should be added in .gitignore
You can checkin in git, an example file named .env.example just for others to know which env var they need. The content of .env.example file will look like this (just keys without any values)
SECRET_KEY=
OTHER_SECRETS=
Where to store SECRET_KEY DJANGO
Store your django SECRET_KEY in an environmental variable or separate file, instead of directly encoding In your configuration module settings.py
settings.py
#from an environment variable
import os
SECRET_KEY = os.environ.get('SECRET_KEY')
#from an file
with open('/etc/secret_key.txt') as f:
SECRET_KEY = f.read().strip()
How to generate Django SECRET_KEY manually:
$ python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
7^t+3on^bca+t7#)w%2pedaf0m&$_gnne#^s4zk3a%4uu5ly86
import string
import secrets
c = string.ascii_letters + string.digits + string.punctuation
secret_key = ''.join(secrets.choice(c) for i in range(67))
print(secret_key)
df&)ok{ZL^6Up$\y2*">LqHx:D,_f_of#P,~}n&\zs*:y{OTU4CueQNrMz1UH*mhocD
Make sure the key used in production is not used elsewhere and avoid sending it to source control.