Access Apache SetEnv variable from Django wsgi.py file - django

Question

I'm trying to separate out Django's secret key and DB pass into environmental variables, as widely suggested, so I can use identical code bases between local/production servers.
The problem I am running into is correctly setting and then reading the environmental vars on the production server running Apache + mod_wsgi.
Vars set in my user profile aren't available because Apache isn't run as that user. Vars set in the Virtual Hosts file with SetEnv are not available because the scope is somehow different.
I've read a couple 1,2 of SO answers, which leads to this blog with a solution.
I can't figure out how to apply the solution to current versions of Django which use a wsgi.py file, which looks like:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
How can I apply that blog solution to the wsgi.py file, or is there a better place to store env-vars where Django can get at them?

Answer 1

If anyone else is frustrated by Graham's answer, here is a solution that actually works for the original question. I personally find setting environmental variables from Apache to be extremely useful and practical, especially since I configure my own hosting environment and can do whatever I want.
wsgi.py (tested in Django 1.5.4)
from django.core.handlers.wsgi import WSGIHandler
class WSGIEnvironment(WSGIHandler):
def __call__(self, environ, start_response):
os.environ['SETTINGS_CONFIG'] = environ['SETTINGS_CONFIG']
return super(WSGIEnvironment, self).__call__(environ, start_response)
application = WSGIEnvironment()
Of minor note, you lose the future-proof method of django.core.wsgi.get_wsgi_application, which currently only returns WSGIHandler(). If the WSGIHandler.__call__ method is ever updated and you update Django also, you may have to update the WSGIEnvironment class if the arguments change. I consider this a very small penalty to pay for the convenience.

Answer 2

FWIW. Relying on environment variables for fine grained configuration settings are in general not a good idea. This is because not all WSGI hosting environments or commercial PaaS offerings support the concept. Using environment variables for fine grained settings can also effectively lock you into a specific PaaS offering where you have directly embedded a lookup of a specifically named environment variable directly into your code, where the naming convention of that environment variable is specific to that hosting service. So although use of environment variables is pushed by certain services, always be careful of being dependent on environment variables as it will reduce the portability of your WSGI application and make it harder to move between deployment mechanisms.
That all said, the blog post you mention will not usually help. This is because it is using the nasty trick of setting the process environment variables on each request based on the per request WSGI environ settings set using SetEnv in Apache. This can cause various issues in a multi threading configuration if the values of the environment variables can differ based on URL context. For the case of Django, it isn't helpful because the Django settings module would normally be imported before any requests had been handled, which means that the environment variables would not be available at the time required.
This whole area of deployment configuration is in dire need of a better way of doing things, but frankly it is mostly a lost cause because hosting services will not change things to accommodate a better WSGI deployment strategy. They have done their work, have their customers locked into the way they have done it already and aren't about to create work for themselves and change things even if a better way existed.
Anyway, 'all problems in computer science can be solved by another level of indirection'. (http://en.wikipedia.org/wiki/Indirection) and that is what you can do here.
Don't have your application lookup environment variables. Have it import a deployment specific Python configuration module which contains a means of using an API to get the configuration settings. This configuration module would implement different ways of getting the actual settings based on the deployment mechanism. In some cases it could grab the values from environment variables. For other such as with Apache/mod_wsgi, the values could be in that configuration module, or read from a separate configuration file which could be an ini, json or yaml format. In providing a API, it can also map names of configuration settings to different names used by different PaaS offerings.
This configuration module doesn't need to be a part of your application code, but could be manually place into a subdirectory of '/etc/' on the target system. You just then need to set the Python module search path so your application can see it. The whole system could be made quite elegant as part of a wider better standard for WSGI deployment, but as I said, little incentive to do the hard work of creating such a thing when existing PaaS offerings are highly unlikely to change to use such a standard.

Answer 3

Here's an alternative solution that's as future-proof as get_wsgi_application. It even lets you set environment variables to use in your Django initialization.
# in wsgi.py
KEYS_TO_LOAD = [
# A list of the keys you'd like to load from the WSGI environ
# into os.environ
]
def loading_app(wsgi_environ, start_response):
global real_app
import os
for key in KEYS_TO_LOAD:
try:
os.environ[key] = wsgi_environ[key]
except KeyError:
# The WSGI environment doesn't have the key
pass
from django.core.wsgi import get_wsgi_application
real_app = get_wsgi_application()
return real_app(wsgi_environ, start_response)
real_app = loading_app
application = lambda env, start: real_app(env, start)
I'm not 100% clear how mod_wsgi manages its processes, but I assume it doesn't re-load the WSGI app very often. If so, the performance penalty from initializing Django will only happen once, inside the first request.
Alternatively, if you don't need to set the environment variables before initializing Django, you can use the following :
# in wsgi.py
KEYS_TO_LOAD = [
# A list of the keys you'd like to load from the WSGI environ
# into os.environ
]
from django.core.wsgi import get_wsgi_application
django_app = get_wsgi_application()
def loading_app(wsgi_environ, start_response):
global real_app
import os
for key in KEYS_TO_LOAD:
try:
os.environ[key] = wsgi_environ[key]
except KeyError:
# The WSGI environment doesn't have the key
pass
real_app = django_app
return real_app(wsgi_environ, start_response)
real_app = loading_app
application = lambda env, start: real_app(env, start)

Answer 4

For Django 1.11:
Apache config:
<VirtualHost *:80 >
...
SetEnv VAR_NAME VAR_VALUE
</VirtualHost>
wsgi.py:
import os
import django
from django.core.handlers.wsgi import WSGIHandler
class WSGIEnvironment(WSGIHandler):
def __call__(self, environ, start_response):
os.environ["VAR_NAME"] = environ.get("VAR_NAME", "")
return super(WSGIEnvironment, self).__call__(environ, start_response)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
django.setup(set_prefix=False)
application = WSGIEnvironment()

Answer 5

I ran into the same problem of separating production and development code while tracking both of them into version control. I solved the problem using different wsgi scripts, one for production server and one for development server. Create two different setting files as mentioned here. And reference them in wsgi scripts. For example the following is wsgi_production.py file
...
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.production")
...
and in wsgi_development.py file
...
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.development")
...