Deploying Django project using mod_wsgi and virtualenv - django

Trying to deploy a Django 1.4 project using mod_wsgi and virtualenv, i'm running into a 500. The Apache error_log reveals:
mod_wsgi (pid=30452): Exception occurred processing WSGI script '/path/to/project/site-packages/projectapp/wsgi.py'.
[...] Traceback (most recent call last):
[...] File "/path/to/project/env/myenv/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 219, in __call__
[...] self.load_middleware()
[...] File "/path/to/project/env/myenv/lib/python2.6/site-packages/django/core/handlers/base.py", line 47, in load_middleware
[...] raise exceptions.ImproperlyConfigured('Error importing middleware %s: "%s"' % (mw_module, e))
[...] ImproperlyConfigured: Error importing middleware projectapp.content.middleware: "cannot import name SomeModel"
From the error message i would expect that this is some kind of a path issue. However, when the offending middleware is removed from the Django settings, the site launches just fine, and there are other modules loaded from projectapp.content, SomeModel is also loaded in this case, otherwise the whole site wouldn't be able to run.
The import error raised doesn't come directly from the middleware, as it doesn't import the model. SomeModel is defined in a speparate app which is actually checked out into the src directory of the virtualenv. But the directory containing this app is also in the path.
The wsgi.py file i'm using:
import os
import sys
sys.stdout = sys.stderr
sys.path.insert(0, '/path/to/project/env/myenv/lib/python2.6/site-packages/')
# The module inside the following directory
# defines SomeModel from the error message
sys.path.insert(0, '/path/to/project/env/myenv/src/some-app/')
sys.path.insert(0, '/path/to/project/site-packages/')
import django.core.handlers.wsgi
os.environ['DJANGO_SETTINGS_MODULE'] = 'projectapp.settings'
application = django.core.handlers.wsgi.WSGIHandler()
Printing sys.path after inserting the module paths shows everything in the expected order, /path/to/project/site-packages/ is listed first and /path/to/project/env/myenv/src/some-app/ (which defines SomeModel) second.
I've also tried a different variant, based on the example from the mod_wsgi docs:
import os
import sys
import site
ALLDIRS = [
'/path/to/project/site-packages/',
'/path/to/project/env/myenv/lib/python2.6/site-packages/',
'/path/to/project/env/myenv/src/some-app/',
]
# Remember original sys.path
prev_sys_path = list(sys.path)
sys.stdout = sys.stderr
# Add each new site-packages directory
for directory in ALLDIRS:
site.addsitedir(directory)
# Reorder sys.path so new directories are at the front
new_sys_path = []
for item in list(sys.path):
if item not in prev_sys_path:
new_sys_path.append(item)
sys.path.remove(item)
sys.path[:0] = new_sys_path
#activate_this = '/path/to/project/env/myenv/bin/activate_this.py'
#execfile(activate_this, dict(__file__=activate_this))
import django.core.handlers.wsgi
os.environ['DJANGO_SETTINGS_MODULE'] = 'projectapp.settings'
application = django.core.handlers.wsgi.WSGIHandler()
The error logged by Apache is exactly the same.
I have to add that the middleware is loaded just fine when i'm running the development server on the same machine, so i have no idea what's going wrong. Is there a way to get a better traceback from mod_wsgi?
EDIT:
Fogot to mention that i'm using mod_wsgi in daemon mode. Here are the relevant parts from my vhost config:
<VirtualHost x.x.x.x:80>
WSGIDaemonProcess foo user=foo threads=10 umask=0002
WSGIProcessGroup foo
WSGIScriptAlias / /path/to/project/site-packages/projectapp/wsgi.py
</VirtualHost>

Okay after hours of debugging this turned out to be a race condition caused by the middleware. The middleware i'm using is similiar to the FlatpageFallbackMiddleware from Django contrib and actually the import of the view caused the problem.
from projectapp.content.views import generic_content_detail
class ContentFallbackMiddleware(object):
def process_response(self, request, response):
[...]
Moving the import statement inside the process_response method solved the problem for me:
class ContentFallbackMiddleware(object):
def process_response(self, request, response):
from projectapp.content.views import generic_content_detail
[...]

Related

How to specify environment variables in Apache site config for Django?

How do you specify environment variables for an Apache site?
I have a multi-tenant Django site being served by Apache+ModWSGI. Each site uses the same Django code, but their settings differ slightly. An environment variable tells Django which settings to load. Currently, I'm getting a separate wsgi_<site_name>.py file, containing the variable appropriate for each site, but this violate the DRY rule. I'd like to, instead, put this variable in the Apache site config, and use a single wsgi.py file for all sites.
I can't find much info on how to do this. I found this one old blog post suggesting I could use the Apache config syntax:
SetEnv SITE somename
along with a wsgi file like:
import os, sys
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
PROJECT_DIR = os.path.abspath(os.path.join(BASE_DIR, '..'))
sys.path.append(PROJECT_DIR)
os.environ['DJANGO_SETTINGS_MODULE'] = 'app.settings'
from django.core.wsgi import get_wsgi_application
_application = get_wsgi_application()
def application(environ, start_response):
os.environ['SITE'] = environ['SITE']
return _application(environ, start_response)
but this has no effect. If I print out the SITE variable, Django never receives the value "somename" and displays the default value.
As far as I can tell the mod_env component is still supported, and I've confirmed it's both installed and enabled on my server.
The problem was that I was calling get_wsgi_application() too early. The correct code is:
from django.core.wsgi import get_wsgi_application
def application(environ, start_response):
os.environ['SITE'] = environ['SITE']
_application = get_wsgi_application()
return _application(environ, start_response)

Passing environment variables from apache via mod_wsgi to use in django 1.11 settings

Found a few versions of this question, such as Django get environment variables from apache, however the advice I've found so far doesn't seem to work with the latest LTS django (1.11).
I have an apache configuration which holds a number of environment variables, not limited to connection credentials for the DB. Im using this to make my code portable between dev/prod etc.
My apache conf just uses SetEnv to pass in some variables.
I've tried two different styles of approach to use these variables, both seem to suffer from the same issue; it needs to read the settings file before we can write to the environment, and the settings file requires values from the environment.
My two variants are;
import os
import django
from django.core.handlers.wsgi import WSGIHandler
from django.core.wsgi import get_wsgi_application
_application = get_wsgi_application()
def application(environ, start_response):
for key in [keys...]:
if environ.get(key):
os.environ[key] = environ.get(key)
return _application(environ, start_response)
and
import os
import django
from django.core.handlers.wsgi import WSGIHandler
class WSGIEnvironment(WSGIHandler):
def __call__(self, environ, start_response):
for key in [keys...]:
if environ.has_key(key):
print "Key: %s = %s" % (key,environ[key])
os.environ[key] = environ[key]
return super(WSGIEnvironment, self).__call__(environ, start_response)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'apiClient.settings')
django.setup(set_prefix=False)
application = WSGIEnvironment()
Either way im trying to use the values in settings as;
KEY = "thing"
if os.environ.has_key("KEY"):
KEY = os.environ["KEY"]
You can't use SetEnv as the settings file is evaluated before the first request is handled. Setting environment variables from per request WSGI environ values was always a bad idea and can cause problems, so you shouldn't do that anyway.
Result is that you cannot set environment variables from the Apache configuration file. Set them from the WSGI script file. If they are things that should not be added to a Git repository, create a file on the specific host with the values in some format, and have the WSGI script file read that file from the local host to set them when first loaded and before any Django code is executed.
For Django > 1.7 get_wsgi_application calls django.setup() which initializes the settings. So if your env vars aren't set at that point you won't see them in the settings.
To get around it, don't actually call get_wsgi_application until you're ready. This works for me in wsgi.py:
def application(environ, start_response):
os.environ['APPLICATION_ENV'] = environ.get('APPLICATION_ENV', None)
return get_wsgi_application()(environ, start_response)
You have to defer initialising the Django app until the first request. Something like this in your wsgi.py:
from django.core.wsgi import get_wsgi_application
_application = None
def application(environ, start_response):
global _application
if _application == None:
for key in environ:
if key.startswith('ENV_'):
os.environ[key[4:]] = environ[key]
_application = get_wsgi_application()
return _application(environ, start_response)
It's a pity that there appears to be no option in mod_wsgi to set the initial environment when starting the daemon process, as you can with mod_fastcgi.

Getting an error in django-admin.py

I have installed django 1.6 using pip on Windows 7. When I try to run django-admin.py, I get this error
C:\django-admin.py startproject test007
Traceback (most recent call last):
File "C:\Python27\Scripts\django-admin.py", line 2 in ?
from django.core import management
File "C:\Python27\Lib\site-packages\django\core\management\__init__.py", line 55
except ImportError as e:
^
SyntaxError: invalid syntax
in the \management__init__.py, the imports are
import collections
import os
import sys
from optparse import OptionParser, NO_DEFAULT
import imp
from django.core.exceptions import ImproperlyConfigured
from django.core.management.base import BaseCommand, CommandError, handle_default_options
from django.core.management.color import color_style
from django.utils.importlib import import_module
from django.utils import six
# For backwards compatibility: get_version() used to be in this module.
from django import get_version
Here's the relevant block from the same file
parts = app_name.split('.')
parts.append('management')
parts.reverse()
part = parts.pop()
path = None
# When using manage.py, the project module is added to the path,
# loaded, then removed from the path. This means that
# testproject.testapp.models can be loaded in future, even if
# testproject isn't in the path. When looking for the management
# module, we need look for the case where the project name is part
# of the app_name but the project directory itself isn't on the path.
try:
f, path, descr = imp.find_module(part, path)
except ImportError as e:
if os.path.basename(os.getcwd()) != part:
raise e
else:
if f:
f.close()
while parts:
part = parts.pop()
f, path, descr = imp.find_module(part, [path] if path else None)
if f:
f.close()
return path
in which the same line 55 from the traceback is in the try/except block towards the bottom. I've uninstalled and reinstalled but, to no avail.
It works when I give the/full/path/to/django-admin.py but it shouldn't be required.

Pandas + Django + mod_wsgi + virtualenv

Pandas is producing 'module' object has no attribute 'core' when being imported under django and mod_wsgi inside a virtual environment. It works fine running under the django development server inside the virtual environment.
Other modules e.g.: numpy have no problems so I assume this means the virtual environment is set up correctly with mod_wsgi. Any advice would be appreciated.
staging.wsgi
import os
import sys
import site
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
site_packages = os.path.join(PROJECT_ROOT, 'env/openportfolio/lib/python2.7/site-packages')
site.addsitedir(os.path.abspath(site_packages))
sys.path.insert(0, PROJECT_ROOT)
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
os.environ['DJANGO_SETTINGS_MODULE'] = 'openportfolio.settings_staging'
import pandas #triggers error
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Error
Traceback (most recent call last):
File "/usr/local/web/django/www/staging/openportfolio/apache/staging.wsgi", line 22, in <module>
import pandas
File "/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pandas/__init__.py", line 12, in <module>
from pandas.core.api import *
File "/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pandas/core/api.py", line 6, in <module>
import pandas.core.datetools as datetools
AttributeError: 'module' object has no attribute 'core'
Python Path
['/usr/local/web/django/www/staging/openportfolio',
'/usr/local/web/django/www/staging',
'/Library/Python/2.7/site-packages/pip-1.0.2-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC',
'/Library/Python/2.7/site-packages',
'/usr/local/web/django/www/staging/env/openportfolio/lib/python2.7/site-packages']
So it turns out this was a Python path order issue. By running sys.path.reverse() in my wsgi config file, the code now runs.
Due to the order of Python path, the built in OS X numpy library must have been imported first over the virtual environment one causing the issue.
'RuntimeError: module compiled against API version 6 but this version of numpy is 4' was the error line I missed in my original post which could have helped debug the answer.

Could not import/No module named Django Error with Apache

I had a small proof of concept set up on a development server on a local machine. I'm now trying to move it over to django on a production server, which I'm using webfaction for. However, now that I'm switched over to apache from the built in django server I get the following:
ViewDoesNotExist: Could not import orgDisplay.views. Error was: No module named orgDisplay.views
But when check my orgDisplay apps folder there is a view.py in it. What am I doing wrong? I've tried adding the following to my settings.py by suggestion of the django IRC room.
import sys
sys.path.append(r"/home/user/webapps/django_project/myproject/orgDisplay")
which is the path to my app.
any ideas on how to even begin to trouble shoot this?
Thanks in advance.
I assume you're using mod_wsgi (which is recommended by Django authors), not mod_python. This is the way you should use your sys.path:
django.wsgi:
import os, sys
sys.path.append(r"/home/user/webapps/django_project/myproject/")
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
sys.stdout = sys.stderr # Prevent crashes upon print
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
urls.py:
from django.conf.urls.defaults import *
urlpatterns = (
("", include("orgDisplay.urls")),
# ...
)
orgDisplay/urls.py:
import views
urlpatterns = (
(r'^some_view/$', views.some_view), # It is actually orgDisplay.views.some_view
# many more records ...
)
It is a bad idea to add project dir itself to path since you're be getting name conflicts between multiple projects.
I think you're appending the wrong directory to sys.path. I think Python is looking in the .../myproject/orgDisplay folder for the orgDisplay package. Try removing the orgDisplay from your string, like this:
import sys
sys.path.append(r"/home/user/webapps/django_project/myproject")
The other option would be to simply add myproject (or whatever your project is actually called) in the import statement.
# instead of "from orgDisplay import views"
from myproject.orgDisplay import views
Also, make sure to restart Apache after every edit.
looking at manage.py, it does it like so:
import sys
from os.path import abspath, dirname, join
from django.core.management import setup_environ
# setup the environment before we start accessing things in the settings.
setup_environ(settings_mod)
sys.path.insert(0, join(PINAX_ROOT, "apps"))
sys.path.insert(0, join(PROJECT_ROOT, "apps"))
Provided your WSGI file is in your project directory, a slightly more flexible way is this:
import os, sys
sys.path.append(os.path.dirname(__file__))
This will enable you to change your project location later without having to modify your WSGI file.