Separate custom settings variable between development, staging and production - django

I'm following the project structure as laid out by Zachary Voase, but I'm struggling with one specific issue.
I'd very much like to have a custom settings boolean variable (let's call it SEND_LIVE_MAIL) that I would be using in the project. Basically, I'd like to use this settings variable in my code and if SEND_LIVE_MAIL is True actually send out a mail, whereas when it is set to False just print its contents out to the console. The latter would apply to the dev environment and when running unittests.
What would be a good way of implementing this? Currently, depending on the environment, the django server uses dev, staging or prd settings, but for custom settings variables I believe these need to be imported 'literally'. In other words, I'd be using in my views something like
from settings.development import SEND_LIVE_MAIL
which of course isn't what I want. I'd like to be able to do something like:
from settings import SEND_LIVE_MAIL
and depending on the environment, the correct value is assigned to the SEND_LIVE_MAIL variable.
Thanks in advance!

You shouldn't be importing directly from your settings files anyways. Use:
>>> from django.conf import settings
>>> settings.SEND_LIVE_MAIL
True

The simplest solution is to have this at the bottom of your settings file:
try:
from local_settings import *
except ImportError:
pass
And in local_settings.py specify all your environment-specific overrides. I generally don't commit this file to version control.
There are more advanced ways of doing it, where you end up with a default settings file and a per-environment override.
This article by David Cramer covers the various approaches, including both of the ones I've mentioned: http://justcramer.com/2011/01/13/settings-in-django/

import os
PROJECT_PATH = os.path.dirname(__file__)
try:
execfile(os.path.join(PROJECT_PATH, local_settings.py'))
except IOError:
pass
Then you can have your local_settings.py behave as if it was pasted directly into your settings.py:
$ cat local_settings.py
INSTALLED_APPS += ['foo']

You can do something like this for a wide variety of environment based settings, but here's an example for just SEND_LIVE_MAIL.
settings_config.py
import re
import socket
class Config:
def __init__(self):
fqdn = socket.getfqdn()
env = re.search(r'(devhost|stagehost|prodhost)', fqdn)
env = env and env.group(1)
env = env or 'devhost'
if env == 'devhost':
self.SEND_LIVE_MAIL = # whatever
elif env == 'stagehost':
self.SEND_LIVE_MAIL = # whatever
elif env == 'prodhost':
self.SEND_LIVE_MAIL = # whatever
config = Config()
settings.py
from settings_config import config
SEND_LIVE_MAIL = config.SEND_LIVE_MAIL

Related

Access\Read Flask Env Variable within app

I can't seem to access my env variables' values inside my app. I'm loading my env variables into my app with a .env file and the dotenv package; I learned about it here.
My .env
FLASK_ENV=development
FLASK_APP=app.py
DEBUG=ON
TESTING=False
I want to use the value of the TESTING variable inside my app and run certain code based on whether it is True or False.
How can I get these values? The docs say
Certain configuration values are also forwarded to the Flask object so
you can read and write them from there: app.testing = True
But I get module 'app' has no attribute 'testing'
When I log app by itself I see a Flask object. When I log this out like app.Flask, I see the env variables, but these appear like this, with no refernce to the current value.
{'__name__': 'TESTING', 'get_converter': None}
I want to be able to do something like:
app.testing => False
app.FLASK_ENV => development
and then eventually:
if app.testing == True:
<do something>
PS - I know the app loads this .env file okay because if I remove the values the environment changes back to production, the default.
#settings.py
from pathlib import Path # python3 only
from dotenv import load_dotenv
load_dotenv(verbose=True)
env_path = Path('.') / '.env'
load_dotenv(dotenv_path=env_path)
import os
print(os.environ['TESTING'])
Equivalent in JS is process.env

Importing variables from other py files

I have a .py file that contains database settings (call it connection.py). I am trying to import the variables contained in my connection.py into my views.py. However, the variables are not being imported.
In my views.py I have:
from connection import *
def location_where_connection_variables_needed(request, some_id):
conn = psycopg2.connect(conn_string)
cursor = conn.cursor()
I then use the variables in connection.py in my views.py script. But the variables are not being imported into my views.py. I placed connection.py in the main folder (ie, connection.py is in the same folder as my app's folder). Is this the correct place to put this? Or am I doing something else fundamentally wrong?
Edit:
What I mean by not being imported:
I am not getting any error at all but the variables are not being imported because the code in views.py is not executing. If I add the connection variable manually in my views.py, everything works fine.
Connection.py
conn_string = "host='localhost' dbname='test' user='postgres' password='pwd'"
With constant values, I typically put those into a file called constants.py and specify those values in all caps, which is a Python convention. So, I would replace "connection.py" with "constants.py" as such:
# constants.py
CONN_STRING = "host='localhost' dbname='test' user='postgres' password='pwd'"
# views.py
from constants import CONN_STRING
def location_where_connection_variables_needed(request, some_id):
conn = psycopg2.connect(CONN_STRING)
cursor = conn.cursor()
Also, make sure to clear out any .pyc files that might not be getting cleared out by manage.py. That can sometimes cause things not to work as expected. You might also stay away from import * since it's not as implicit (you can't see what you're importing) and it's easy to run into naming collisions.
You can try a global variable instead of a local one. Use it as
global conn_string = "host='localhost' dbname='test' user='postgres' password='pwd'"

Importing CSV to Django and settings not recognised

So i'm getting to grips with Django, or trying to. I have some code that isn't dependent on being called by the webpage - it's designed to populate the database with information. Eventually it will be set up as a cron job to run overnight. This is the first crack at it, which is to do an initial population (once I have that working, I'll move to an add structure, where only new records are pushed.) I'm using Python 2.7, Django 1.5 and Sqlite3. When I run this code, I get
Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
That seems fairly obvious, but I've spent a couple of hours now trying to work out how to adjust that setting. How do I call / open a connection / whatever the right terminology is here? I have a number of functions like this that will be scheduled jobs, and this has been frustrating me all afternoon.
import urllib2
import csv
import requests
from django.db import models
from gmbl.models import Match
master_data_file = urllib2.urlopen("http://www.football-data.co.uk/mmz4281/1213/E0.csv", "GET")
data = list(tuple(rec) for rec in csv.reader(master_data_file, delimiter=','))
for row in data:
current_match = Match(matchdate=row[1],
hometeam=row[2],
awayteam = row [3],
homegoals = row [4],
awaygoals = row[5],
homeshots = row[10],
awayshots = row[11],
homeshotsontarget = row[12],
awayshotsontarget = row[13],
homecorners = row[16],
awaycorners = row[17])
current_match.save()
I had originally started out with http://django-csv-importer.readthedocs.org/en/latest/ but I had the same error, and the documentation doesn't make much sense trying to debug it. When I tried calling settings.configure in the function, it said it didn't exist; presumably I had to import it, but couldn't make that work.
Make sure Django, and your project are in PYTHONPATH then you can do:
import urllib2
import csv
import requests
from django.core.management import setup_environ
from django.db import models
from yoursite import settings
setup_environ(settings)
from gmbl.models import Match
master_data_file = urllib2.urlopen("http://www.football-data.co.uk/mmz4281/1213/E0.csv", "GET")
data = list(tuple(rec) for rec in csv.reader(master_data_file, delimiter=','))
# ... your code ...
Reference: http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/
Hope it helps!

django with twisted web - wgi and vhost

I have a project which has a directory setup like:
myproject
someapp
sites
foo
settings.py - site specific
settings.py - global
I am using twisted web.wsgi to serve this project. The problem am I running into is setting up the correct environment.
import sys
import os
from twisted.application import internet, service
from twisted.web import server, resource, wsgi, static, vhost
from twisted.python import threadpool
from twisted.internet import reactor
from django.core.handlers.wsgi import WSGIHandler
from django.core.management import setup_environ,ManagementUtility
sys.path.append(os.path.abspath("."))
sys.path.append(os.path.abspath("../"))
DIRNAME= os.path.dirname(__file__)
SITE_OVERLOADS = os.path.join(DIRNAME,'sites')
def import_module(name):
mod = __import__(name)
components = name.split('.')
for comp in components[1:]:
mod = getattr(mod,comp)
return mod
def buildServer():
hosts = [d for d in os.listdir(SITE_OVERLOADS) if not os.path.isfile(d) and d != ".svn"]
root = vhost.NameVirtualHost()
pool = threadpool.ThreadPool()
pool.start()
reactor.addSystemEventTrigger('after', 'shutdown', pool.stop)
for host in hosts:
settings = os.path.join(SITE_OVERLOADS,"%s/settings.py" % host)
if os.path.exists(settings):
sm = "myproject.sites.%s.settings" % host
settings_module = import_module(sm)
domain = settings_module.DOMAIN
setup_environ(settings_module)
utility = ManagementUtility()
command = utility.fetch_command('runserver')
command.validate()
wsgi_resource = wsgi.WSGIResource(reactor,pool,WSGIHandler())
root.addHost(domain,wsgi_resource)
return root
root = buildServer()
site = server.Site(root)
application = service.Application('MyProject')
sc = service.IServiceCollection(application)
i = internet.TCPServer(8001, site)
i.setServiceParent(sc)
I am trying to setup vhosts for each site which has a settings module in the subdirectory "sites". However, it appears that the settings are being shared for each site.
Django projects within the same Python process will share the same settings. You will need to spawn them as separate processes in order for them to use separate settings modules.
Since your goal is a bunch of shared-nothing virtual hosts, you probably won't benefit from trying to set up your processes in anything but the simplest way. So, how about changing your .tac file to just launch a server for a single virtual host, starting up a lot of instances (manually, with a shell script, with another simple Python script, etc), and then putting a reverse proxy (nginx, Apache, even another Twisted Web process) in front of all of those processes?
You could do this all with Twisted, and it might even confer some advantages, but for just getting started you would probably rather focus on your site than on minor tweaks to your deployment process. If it becomes a problem that things aren't more integrated, then that would be the time to revisit the issue and try to improve on your solution.

Django paths, developing in windows, deploying on linux

I'm developing Django apps on my local windows machine then deploying to a hosted linux server. The format for paths is different between the two and manually replacing before deployment is consuming more time than it should. I could code based on a variable in my settings file and if statements but I was wondering if anyone had best practices for this scenario.
The Django book suggests using os.path.join (and to use slashes instead of backslashes on Windows):
import os.path
TEMPLATE_DIRS = (
os.path.join(os.path.dirname(__file__), 'templates').replace('\\','/'),
)
I think this is the best solution as you can easily create relative paths like that. If you have multiple relative paths, a helper function will shorten the code:
def fromRelativePath(*relativeComponents):
return os.path.join(os.path.dirname(__file__), *relativeComponents).replace("\\","/")
If you need absolute paths, you should use an environment variable (with os.environ["MY_APP_PATH"]) in combination with os.path.join.
We have a situation very similar to yours, and we've been using different paths in settings, basing on sys.platform.
Something like this:
import os, sys
DEVELOPMENT_MODE = sys.platform == 'win32'
if DEVELOPMENT_MODE:
HOME_DIR = 'c:\\django-root\\'
else:
HOME_DIR = '/home/django-root/'
It works quite OK - assumed all development is being done on Windows.
Add
import os.path
BASE_PATH = os.path.dirname(__file__)
at the top of your settings file, and then use BASE_PATH everywhere you want to use a path relative to your Django project.
For example:
MEDIA_ROOT = os.path.join(BASE_PATH, 'media')
(You need to use os.path.join(), instead of simply writing something like MEDIA_ROOT = BASE_PATH+'/media', because Unix joins directories using '/', while windows prefers '\')
in your settings.py add the following lines
import os.path
SETTINGS_PATH = os.path.abspath(os.path.dirname(__file__))
head, tail = os.path.split(SETTINGS_PATH)
#add some directories to the path
import sys
sys.path.append(os.path.join(head, "apps"))
#do what you want with SETTINGS_PATH