I added the settings.py file of my django project to .gitignore before pushing to Github.
Now when I open the project in PyCharm, the file name settings.py is highlighted in a kind of lime green color.
Is this telling me something else or are there any other implications for this other than having added it to .gitignore, maybe in terms of later on when I want to put my site on a remote server for instance?
I will probably push new commits to Github in the future so I had planned to keep it in .gitignore, but would this present any other problems?
I have looked everywhere and all I could find was outdated or bits and pieces from several different sources, but none of them had a complete step of how to deploy Django to a production environment.
What I would like to see is the recommended way to deploy Django utilizing git, local and production settings.py, .gitignore specifically for Django.
How would I implement the settings to accommodate both environments and what should be added to the .gitignore file so that only necessary files are sent to the git repository?
Please note that I want to know best practices and up to date methods of using local and production settings.
Such questions arise when deploying Django apps.
Where should the settings be stored?
How can I properly implement the settings to accommodate both environments?
What sensitive information should be separate from the main settings?
Update 25/09/2016:
Thanks andreas for clarifying this and now I completely agree that there is no standard way of deploying Django. I went on a long research and found the most common used methods to deploy Django settings. I will share my findings with everyone who's having the same doubt.
Answering
Where should the settings be stored?
The settings are better accommodated within a settings module rather than a single settings.py file.
How can I properly implement the settings to accommodate both environments?
This is where the settings module comes in very handy because you can simply have environment independent settings and they are maintainable with git or any other version control.
What sensitive information should be separate from the main settings?
Sensitive information such as database, email, secret key and anything with authentication related information and as to how you will serve this private information secretly, I do recommend to use andreas suggestion.
You could either create a separate file at /etc/app_settings and make it very restrict or simply use environment variables from within the system.
.gitignore part
For the .gitignore part I found an awesome repo where you can get all sort of examples at https://github.com/github/gitignore.
I ended up using:
# OSX Finder turds
.DS_Store
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# Distribution / packaging
.Python
env/
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Django stuff:
*.log
my_app/settings/dev.py
bower_components
Because the answer ended up being so long I've created an article explaining every step of the way at Django Settings Deployment.
There is not one recommended way of deploying a Django app. The official documentation provides a guide for deploying with Apache and mod_wsgi. Some other options are a PaaS like Heroku/Dokku or deployment with Docker
It's common to divide your settings in different files. You could for example divide settings in four different files:
base file (base.py) - Common settings for all environments (every file below imports from this with from .base import *
development file (development.py) - settings specific for development, (DEBUG = True etc...)
production file (production.py) - settings specific for the production environment. (DEBUG = False, whitenoise configuration etc)
testing file (testing.py)
You can control what settings file is loaded with the environment variable DJANGO_SETTINGS_MODULE
It is also common recommended practice that you store secrets like the SECRET_KEY in environment variables. SECRET_KEY = os.environ.get('SECRET_KEY', None). Or check out the django-environ package.
Check out this repo for an example setup.
I've tried to find a concrete answer for this but failed. I'm rather new to openshift, django and git so I hope it's not too trivial and not a duplicate.
The situation is as follows:
1. I'm developing a Django based web application and source control it using git on a private bitbucket repo. I have my regular django application source tree as needed.
2. I wish to deploy my app on openshift which comes with a different directory tree. I've been able to successfully run this quickstart:
git://github.com/rancavil/django-openshift-quickstart.git
Basically what I try to achieve is a wrapper directory (and git project) which is based on the repo specified in #2. Within this wrapper one of the subdirectories (wsgi/my_project) should be based on my private repo specified in #1.
I wish to be able to pull recent changes from the private repo and then push and deploy to openshift these changes.
Any help would be greatly appreciated!
I was also struggling with openshift structure. But there is no need for doing the staff like you have proposed.
You can have any structure of a project you want. Openshift will need just wsgi/application or wsgi.py as an entry point. I think that having one more file in your project is hardly a problem.
My current structure for a project (got it running recently) is following:
|- .openshift (directory for build and deployment hooks like syncdb or collectstatic commands)
|- custom_project (directory that is you django project)
\- templates
|- settings.py
|- urls.py
|- manage.py
|- requirements.txt
|- wsgi.py
There is really no need to have project under wsgi folder. However there are few more pitfalls to be aware of. Like creating static folder or setting .htaccess for static files.
Including my wsgi.py:
#!/usr/bin/python
import os
import logging
try:
virtenv = os.environ['OPENSHIFT_PYTHON_DIR'] + '/virtenv/'
virtualenv = os.path.join(virtenv, 'bin/activate_this.py')
execfile(virtualenv, dict(__file__=virtualenv))
except (IOError, KeyError):
pass
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'custom_project.settings')
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I have used a lot of resources to make my project running on openshift so I'm also including them for other.
bitbucket and openshit repos: Can I use my existing git repo with openshift?
Slides with necesary steps: http://appsembler.com/blog/paas-bakeoff-comparing-stackato-openshift-dotcloud-and-heroku-for-django-hosting-and-deployment/
And the most important one, the really well written step-by-step tutorial: http://www.paascheatsheet.com/#Introduction
My problem/issue
We are working on an opensource project which we have hosted on github. The project is written in Django and so we have our settings in a settings.py file. We have it open source not because we are too cheap for a subscription, but because we intentionally want it to be open source.
Anyways, we also want to run this code ourselves and so we have a site_settings.py which contains all sensitive data like the db password etc. Since we don't want this on github for everyone to see, this is in the .gitignore file.
Normally we clone the repo and add this file locally and on the server, this way we all have our personal settings and sensitive data isn't stored on git.
The problem is that we now want to run our code on Heroku, which doesn't support the creation of files on the server, since heroku doesn't work like a normal server ( it works with a distributed file system).
Of course I have thought about it myself, but I am not sure if this is the right approach. Please keep in mind that I am both new to Heroku and Git. The following is my current solution:
My current idea on how to fix this
There are two remotes:
origin == Github
heroku == Heroku
leaving out all the development and feature branches, we only have both the master branches (on Heroku and Github) to worry about.
My idea was to have the branches set up like this locally:
master -> remotes/origin/master
heroku -> remotes/heroku/master
On the master branch we put the site_settings.py in the .gitignore so git will ignore it when committing to Github, the same goes for all the dev branches etc. We collect all our changes in the master branch until we have a new release.
On the heroku branch we edited the .gitignore file to stop ignoring the site_settings.py, so it will be committed to Heroku once we push to the heroku branch.
Once we have a new release we switch to heroku branch and merge master into heroku like so:
git checkout heroku
git pull
git merge master
Once we have sorted out merge conflicts from the merge, we commit to Heroku and we have a new release up and running.. =D
My question
Is my idea acceptable to solve this (will it even work with the .gitignore like this?) and if not how do we solve this in the cleanest/best way possible? If you need more info don't hesitate to ask :)
For starters, storing your config in files is the wrong way to go about this, so don't do it.
From the 12factor app:
An app’s config is everything that is likely to vary between deploys
(staging, production, developer environments, etc). This includes:
Resource handles to the database, Memcached, and other backing
services
Credentials to external services such as Amazon S3 or Twitter
Per-deploy values such as the canonical hostname for the deploy
Apps
sometimes store config as constants in the code. This is a violation
of twelve-factor, which requires strict separation of config from
code. Config varies substantially across deploys, code does not.
A litmus test for whether an app has all config correctly factored out
of the code is whether the codebase could be made open source at any
moment, without compromising any credentials.
Note that this definition of “config” does not include internal
application config, such as config/routes.rb in Rails, or how code
modules are connected in Spring. This type of config does not vary
between deploys, and so is best done in the code.
Another approach to config is the use of config files which are not
checked into revision control, such as config/database.yml in Rails.
This is a huge improvement over using constants which are checked into
the code repo, but still has weaknesses: it’s easy to mistakenly check
in a config file to the repo; there is a tendency for config files to
be scattered about in different places and different formats, making
it hard to see and manage all the config in one place. Further, these
formats tend to be language- or framework-specific.
The twelve-factor app stores config in environment variables (often
shortened to env vars or env). Env vars are easy to change between
deploys without changing any code; unlike config files, there is
little chance of them being checked into the code repo accidentally;
and unlike custom config files, or other config mechanisms such as
Java System Properties, they are a language- and OS-agnostic standard.
Another aspect of config management is grouping. Sometimes apps batch
config into named groups (often called “environments”) named after
specific deploys, such as the development, test, and production
environments in Rails. This method does not scale cleanly: as more
deploys of the app are created, new environment names are necessary,
such as staging or qa. As the project grows further, developers may
add their own special environments like joes-staging, resulting in a
combinatorial explosion of config which makes managing deploys of the
app very brittle.
In a twelve-factor app, env vars are granular controls, each fully
orthogonal to other env vars. They are never grouped together as
“environments,” but instead are independently managed for each deploy.
This is a model that scales up smoothly as the app naturally expands
into more deploys over its lifetime.
For Python, your config can be found in os.environ. For specific config, particularly the database you want to be using something like:
import os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
For more info on config vars, see here: https://devcenter.heroku.com/articles/config-vars
Rather than developing a complex git-based deployment of settings files and variables, on Heroku, I'd use the environment variables for all of your sensitive stuff. That way you don't have to keep them in code. See database docs and environment variables docs for information - put all this stuff in there, and then access via os.getenv() python docs.
To see your environment variables for the app, using the heroku cli tool:
heroku run env
Will print out all the variables. To add new variables (e.g. S3 keys etc) use:
heroku config:add VAR_NAME=value
I'm new to Git. I need to setup Git to deploy a Django website to the production server. My question here is to know what is the best way of doing this.
By now I only have a Master branch. My problem here is that Development environment is not equal to the Production environment. How can I have the two environments(Development and Production) in Git? Should I use two new Branches(Development and Production). Please give me a clue on this.
Other question... when I finish to upload/push the code to the Production server I need to restart the Gunicorn(serves Django website). How can I do this?
And the most important question... Should I use Git to do this or I have better options?
Best Regards,
The first question you must solve is your project structure. Usually the difference between development and the production environment is setting.py and url.py. So why you firstly separate those? :) For example you can have one main settings.py where you define all the default settings which are in common. Then at the end of the file you just import the settings_dev.py and settting_prod.py for exemple:
try:
from settings_prod import *
except ImportError:
pass
try:
from settings_dev import *
except ImportError:
pass
Then simply you can overload all the setting you want and have custom settings of the project (for example installed apps). The same logic you can use for urls.py file.
Then you can simply ignore adding the *_dev files to repo and on the server side you can just checkout the code from repo and restart http server. To automatize this for now I can't give the right name of app to use. Sometimes simple python script could be solution like: watching if the file datetime changed and if yes, just run restart command for http.
Hope that helped.
Ignas
You can follow this brunching model - http://nvie.com/posts/a-successful-git-branching-model/
And, git is ok but use Fabric for deployment.