I'm working on a Django site, using the Django 1.4 official release. My site has a few apps. One of the apps has a model named Campaign with FKs to models in the other apps. As suggested in the Django reference (https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey), I chose to define the FK fields using a string instead of the related model classes themselves since I expect to have circular references in the next version or so, and this approach avoids circular import issues.
When I deployed the site on AWS (Amazon Web Services) using the BitNami djangostack 1.4 (Apache, mod_wsgi, MySQL), my deployed site worked correctly for the most part. On pages displaying forms for the Campaign model, Django raised an exception when trying to create a form field relying on a foreign key field of the Campaign model, complaining the related model was not loaded. The funny/scary thing is that when I set settings.DEBUG to True (definitely not something we'll want once the site goes live), the issue no longer occurs!
The site worked perfectly when I tested it on my local Django development server, and it also worked perfectly using the same BitNami djangostack deployed on my Windows workstation.
Here's the relevant Apache error output I see on the AWS console.
[error] ERROR :: Internal Server Error: /campaigns/
[error] Traceback (most recent call last):
[error] File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/core/handlers/base.py", line 101, in get_response
[error] request.path_info)
... (django/wsgi blah blah)
[error] File "/opt/bitnami/apps/django/django_projects/Project/campaign/views.py", line 5, in <module>
[error] from forms import CampaignForm
[error] File "/opt/bitnami/apps/django/django_projects/Project/campaign/forms.py", line 12, in <module>
[error] class CampaignForm(forms.ModelForm):
[error] File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/forms/models.py", line 206, in __new__
[error] opts.exclude, opts.widgets, formfield_callback)
[error] File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/forms/models.py", line 160, in fields_for_model
[error] formfield = f.formfield(**kwargs)
[error] File "/opt/bitnami/apps/django/lib/python2.6/site-packages/django/db/models/fields/related.py", line 1002, in formfield
[error] (self.name, self.rel.to))
[error] ValueError: Cannot create form field for 'reward' yet, because its related model 'reward.Reward' has not been loaded yet
So, here's a quick recap:
My site works on my local Django development server, regardless of settings.DEBUG value
With the BitNami stack on my local Windows machine, it works, regardless of settings.DEBUG value
With the BitNami stack on AWS (Ubuntu), it works with DEBUG = True but not with DEBUG = False
I understand what the error means, but I don't understand why it's occurring, i.e. why the dependent model is not loaded. Has anyone ever encountered a similar issue, or has advice that could help me fix it?
Note: I tried to Google the error message but all I found was the Django source code where that error was raised. I also tried searching for more general queries like mod_wsgi django related model, but I couldn't find anything that seemed relevant to my problem.
Alright, I found a solution to my problem on a blog post by Graham Dumpleton (http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html).
In short, the Django development server validates the models (which resolves string-based relations) when starting, and that operation probably wasn't done when using mod_wsgi under the BitNami djangostack on Ubuntu with DEBUG = False. Very specific conditions, I know - but G. Dumpleton's 'fixed' mod_wsgi code solved the issue for me.
This is what my wsgi.py looks like now:
#wsgi.py
#make sure the folder containing the apps and the site is at the front of sys.path
#maybe we could just define WSGIPythonPath in django.conf file...?
project_root = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
if project_root not in sys.path:
sys.path.insert(0, project_root)
#set the DJANGO_SETTINGS_MODULE environment variable (doesn't work without this, despite what G. Dumpleton's blog post said)
site_name = os.path.basename(os.path.dirname(__file__))
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "%s.settings" % site_name)
#new code - fixes inter-app model dependencies
from my_site import settings
import django.core.management
django.core.management.setup_environ(settings) # mimic manage.py
utility = django.core.management.ManagementUtility()
command = utility.fetch_command('runserver')
command.validate() # validate the models - *THIS* is what was missing
#setup WSGI application object
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I had just this problem and was looking at modifying wsgi.py as suggested. Then a colleague suggested simply reordering the apps in INSTALLED_APPS and, hey presto, it's working without having to touch the wsgi.py.
Related
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
I want to read environment variables from Apache vhost config file and store them into Django settings.
Before updating to Django 1.7 everything was fine but now it is broken.
It seems the problem is in my wsgi.py script, when I call
_application = get_wsgi_application()
because it reads config file before the environment variable is set.
Is there another way to do this in Django 1.7?
In my /etc/apache2/sites-enabled/mysyte.conf I have:
<VirtualHost *:80>
...
SetEnv SECRET_KEY ...
SetEnv EMAIL_HOST ...
SetEnv EMAIL_HOST_PASSWORD ...
SetEnv EMAIL_HOST_USER ...
SetEnv EMAIL_PORT 25
...
In my wsgi.py:
import os
from os.path import abspath, dirname
from sys import path
SITE_ROOT = dirname(dirname(abspath(__file__)))
path.append(SITE_ROOT)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "locacle.settings.production")
_application = get_wsgi_application()
def application(environ, start_response):
for key, value in environ.items():
if isinstance(environ[key], str):
os.environ[key] = environ[key]
return _application(environ, start_response)
In my settings.py I have:
from os import environ
from base import *
def get_env_setting(setting):
""" Get the environment setting or return exception """
try:
return environ[setting]
except KeyError:
error_msg = "Set the %s env variable" % setting
raise ImproperlyConfigured(error_msg)
EMAIL_HOST = get_env_setting('EMAIL_HOST')
...
This is what log file reports:
...
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/www/.../settings/production.py", line 34, in <module>
EMAIL_HOST = get_env_setting('EMAIL_HOST')
File "/home/www/...settings/production.py", line 21, in get_env_setting
raise ImproperlyConfigured(error_msg)
ImproperlyConfigured: Set the EMAIL_HOST env variable
...
I'm afraid that what you are trying to do here is inherently fragile, only worked by lucky accident on previous versions of Django, and will not work with Django 1.7 or any future version of Django. (Update: it also would make you potentially vulnerable to the "shellshock" bash bug, whereas WSGI normally isn't.)
The basic problem is that that WSGI environ is only available on a per-request basis, but you are trying to set global configuration for your Django process based on it. This is inefficient and conceptually broken (why are you re-setting OS environment variables again and again every single time a request comes in? what if different requests have a different WSGI environ?), and it can only work at all if Django waits to configure itself until the first request arrives.
But the unpredictable timing and ordering of Django's startup sequence in previous versions caused problems. For instance, when using runserver in local development Django would eagerly configure itself, due to validation checks, but under a production environment it would only configure itself lazily (which you were relying on), meaning that sometimes imports would happen in a different order and circular imports would crop up in production that weren't reproducible under runserver.
Django 1.7 includes a revamped startup sequence in order to address these problems, make the startup sequence predictable and consistent between development and production, and allow users to explicitly register code to run at startup time (via AppConfig.ready()). A side effect of this is that settings are configured when your process starts up (specifically, with the call to django.setup() in get_wsgi_application()), not waiting until the first request comes in.
If you're only running one Django site on this server, I would simply move your configuration into normal environment variables rather than SetEnv in your Apache config, and avoid the whole problem.
If you're running multiple Django sites which require different config through a single Apache server, that won't work. In that case, perhaps someone more familiar with Apache and mod_wsgi can give you advice on how to pass environment variables from your Apache config to your Django process in a reliable way; I've personally found the process model (actually, process models, since there's more than one depending how you configure it) of mod_wsgi confusing and error-prone when trying to run multiple independently-configured sites through one server. I find it simpler to use a dedicated WSGI server like gunicorn or uwsgi and have Apache (or nginx) proxy to it. Then it's simple to run multiple gunicorn/uwsgi processes with separate OS environments.
I use django-haystack 2.0.0 to index my site, and it has been working great until I upgraded to Django 1.5 and started using the WSGI interface. If I just use the django_gunicorn command it works great, but the Django documentation "highly recommends" I use the gunicorn command.
When I start my site with the gunicorn command, Haystack throws the following error on any page load:
ImportError: cannot import name signals
I have no problems importing signals from the Django or Python shells. I use virtualenv and install all packages locally inside that environment. My wsgi.py file looks just like the default one in the django admin, except that I add the local path to the python path as such:
path = os.sep.join(os.path.abspath(__file__).split(os.sep)[:-2])
if path not in sys.path:
sys.path.append(path)`
Any help you could provide would be very appreciated, thank you!
I don't use gunicorn, but I had the same problem when I used the HAYSTACK_SIGNAL_PROCESSOR setting to point to a custom class that I wrote. That class imported one of my models, which eventually propagated up the import chain, to import my settings module, thus causing a circular import.
When using a setting such as HAYSTACK_SIGNAL_PROCESSOR that points to a class, make sure that class standsalone, and doesn't import either directly or indirectly the Django settings file.
This is a follow-up question for Google App Engine and Django support:
The tutorial works great for an empty project, however when I try to deploy an existing Django app to Google App Engine, it starts throwing errors:
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 223, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/utils/functional.py", line 184, in inner
self._setup()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." % ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
And a second, which might be related:
Traceback (most recent call last):
File "/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 223, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/utils/functional.py", line 184, in inner
self._setup()
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/python27_runtime/python27_lib/versions/third_party/django-1.4/django/conf/__init__.py", line 95, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'settings' (Is it on sys.path?): No module named settings
I have modified my settings.py as per the tutorial. The app deploys without problems and the syncdb works as well; the database and all required tables are in place.
I found this reference to the error message as shown in the log but its suggested fix did not help.
Any ideas what might be causing this?
EDIT:
To shorten this already lengthy question I removed the previously posted wsgi.py file. I had nothing to do with this issue.
EDIT 2:
I think I made a small improvement and GAE is now complaining:
ImportError: No module named properties
I presume there might be an entry missing in my app.yaml file but I have no clue as to which file that may be. I found some references regarding missing modules but none that reflect this error message...
After much trial and error I managed to solve this, albeit partially:
I did not manage to get my existing PyCharm project deployed (yet). As I was keen to have a version running on GAE I went for a different approach and created a new empty Django project (project 2) on my local machine.
I first deployed the blank project using this tutorial which ran fine. I then copied my models.py file from my existing PyCharm project to project 2 and added an app called properties which uses the models.py file. Initially this went wrong as GAE kept complaining it could not find my properties app. After fiddling about with it, it suddenly appeared in my admin. I'm not sure what caused the initial problem but it's working now.
Things crucial to success for me:
I added the following to the very top of my settings.py:
import os
ROOT_PATH = os.path.dirname(__file__)
... and then replaced the default STATIC_ROOT with:
STATIC_ROOT = ROOT_PATH + os.sep + 'static'
Then I ran:
python manage.py collectstatic
from the dir of project 2. This collects the required static files which Django admin uses (image files, css files etc)
So, I currently have a stripped down version of my original project running on GAE. Things that are missing:
The original project used grappelli for a more styled admin interface. I'm not sure whether this will run on GAE. EDIT: Deploying Grappelli on GAE is reasonbly easy. Just copy the Grappelli package to your project dir and run manage.py collectstatic. Also add grappelli urls as per documention. Redeploy. Done.
The original project uses a module called templated_email to send out templated emails from my properties app. Same as grappelli: I'm not sure whether this is supported by GAE. Hopefully it will run on GAE as it is crucial for my specific app. EDIT: I bumped into this little gem which makes templated_email obsolete and lets me plug into GAE mail system.
To be honest, this approach partially defies the benefits of PyCharm which should fully support Django and GAE out of the box. It will probably require some additional research and tweaking to get that up and running.
Any comments or insights re the above are obviously appreciated!
Being a beginner of using Django, i am trying to add some module for the purpose of testing Django, but I've got a problem regarding the importError which I've googled for solution with no success. Below is my situation
The project is created to my PC J:\ while the python package is installed on C:. According to guardian's installation guide following code have to be added in django's backend:
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend', # default
'guardian.backends.ObjectPermissionBackend',)
The problem's come, when I put the guardian's app under the directory of mysite, configurate the install_apps setting with 'guardian' and start syncdb, the error claim as below which i believe that it is because the django's filw do not understand what is "guradian" really is
File "J:\mysite\guardian\conf\settings.py", line 6, in
raise ImproperlyConfigured("In order to use django-guardian's "
django.core.exceptions.ImproperlyConfigured: In order to use django-guardian's O
bjectPermissionBackend authorization backend you have to configure ANONYMOUS_USE
R_ID at your settings module
So, I move the 'guardian' folder under Django's contrib folder, adding the sys path and configurate the install_app setting with 'django.contrib.guardian'. However, I end up with the importerror.
As it seems from the error message, you need to add the user id for anonymous user for you site.
Create a user (named maybe anonymous) and put the id of user in the settings.py file.
Obtain the user id from the database using the shell.
Put the id in the settings file:
ANONYMOUS_USER_ID = <Your anonymous USER_ID>
EDIT:
Just looked through the documentation of the django-guardian app. It also specifies this:
http://packages.python.org/django-guardian/configuration.html