WSGI loads settings of the wrong project: how to debug? - django

I have two django based web applications on the same server.
One of them i'll call CORRECT_PROJECT and the other one WRONG_PROJECT
The last one, CORRECT_PROJECT, is installed using a virtual environment and uses a different version of django (1.4). There's a very strange problem: sometimes, usually after a log out or an email confirmation (but sometimes looks just random!), the server returns a 500 internal server error and the error log says
"Could not import settings 'WRONG_PROJECT.settings' (Is it on sys.path?): No module name WRONG_PROJECT.settings, refer: CORRECT_PROJECT/URL"
That is, by loading CORRECT_PROJECT, sometimes the system (WSGI? Apache? Django?) tries to load the settings from WRONG_PROJECT.
By hitting refresh aggressively the error disappears.
What could be wrong? How can I debug?
CORRECT_PROJECT uses WSGI in deamon mode.
Solution
Use deamon mode: http://modwsgi.readthedocs.org/en/latest/configuration-directives/WSGIDaemonProcess.html

You are using wsgi.py from Django 1.4. That will not work when hosting multiple web apps in the same process.
Best solution is to use daemon mode and delegate each to a distinct daemon process group.
If can't do that, change the wsgi.py files of both so they do not use:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
but instead use:
os.environ["DJANGO_SETTINGS_MODULE"] = "mysite.settings"
Change mysite.settings as necessary.

Related

Django Webfaction 'Timeout when reading response headers from daemon process'

My Django app on my production server hosted on Webfaction was working fine until I just tried to restart it after pushing a change to the settings.py file. I ran
apache2/bin/restart
as usual. Then I tried to access my app on my browser, and I got a 504 Gateway timeout. I looked into the mod_wsgi logs and saw this:
[Thu Nov 03 23:46:53.605625 2016] [wsgi:error] [pid 8027:tid 139641332168448]
[client 127.0.0.1:34570] Timeout when reading response headers from daemon
process 'myapp' : /home/<me>/webapps/<myapp>/<ProjectName>/<myapp>/wsgi.py
What does this mean and how do I fix it? The only thing I changed in the settings.py file was moving some variable names around. I can still successfully interact with the app with
python2.7 manage.py shell
But I can't get to it on the web, nor use the API.
EDIT: Here's my wsgi.py file:
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<myapp>.settings")
application = get_wsgi_application()
Python C extension modules, like numpy, are known to cause timeouts when used under mod_wsgi. There's a clear explanation of the problem (direct from the author of mod_wsgi) available at https://serverfault.com/a/514251/109598
If that sounds like it might be the cause of your problem, then the solution is probably simple - add the following to your httpd.conf:
WSGIApplicationGroup %{GLOBAL}
Be sure to restart your Apache instance after making that change.
Try increasing Timeout directive in httpd.conf, which defaults to 60 seconds in Apache 2.4. For example:
TimeOut 600
Here is how I was able to find the root cause of my issue.
python manage.py showmigrations
My app could not reach the database server, so it would eventually time out. Running manage.py I could see see the error message on the console.
In my case (Python 3.6), the mimetypes module caused this problem. I did not further investigate this, but removing a call to mimetypes.guess_type solved the problem. The call was made in the related Django view function.
I hit the same problem because the home directory of the user under which the wsgi process was running had became unavailable at some point during the server upgrade.
This might help someone.
Thank you to lenhhoxung who in the comments to one of the other solutions mentioned upgrading server capabilities. I had been successfully running a demo site on an AWS EC2 Nano instance for a long time, but for some reason it suddenly started erroring out on one page that has some complex computations. I upgraded it to an AWS EC2 Micro instance, problem solved. I think this is worthy of its own answer here considering this took a good chunk of a day. All credit to lenhhoxung, though! Thanks!

Django and loading config file before server starts

My django app is communicating with external server and before running django server i would like load some config file. Variables from this file are going to be used by some module while app is running
Problem is that config file can be located in many places.
My dream would be to run manage.py --cfg "/path/to/cfg/file.cfg" or
manage.py runserver --cfg "/path/to/cfg/file.cfg"
and some variables (like globals?) are going to be loaded and they are going to be avaible for django modules to be used. After django server shutdown those variables can dissapear
Is there some nice way to accomplish this?
There seem to be two parts to your problem:
How do I support changing which set of variables (as defined in a config file) are used for a given run
How can I load these variables such that they are visible to all the modules of my application.
The standard mechanism for doing the 2nd is to put stuff in settings.py.
If you do FOO="bar" in settings.py, in your module you can do:
from django.conf import settings
if settings.FOO == "bar":
# Do something
As far as how you can support multiple configurations, the first thing I could come up with is to rename your real settings.py to be real_settings.py and then create a series of config1_settings.py, config2_settings.py, config3_settings.py ... which look like:
from real_settings.py import *
from path_to_configX.py import *
where configX.py has all the values for whatever variables you want for configuration X.
You would then start django's built in server via:
manage.py runserver --settings=configX_settings.py
Note that doing this for a production server (where you can't as easily just pass something on the command line to kick it off) may be a bit trickier, but you're going to need to provide more use case details for that.

Django tutorial on remote server: how to view in my browser?

I'm getting started with a Django tutorial, and I've run into a snag. Having created the sample "mysite" on my usual domain, I want to be able to display it in my browser. The tutorial points me to http://127.0.0.1:8000. However, that's not going to work, as I'm doing this remotely.
[background information]
What I have done, apparently successfully, is
django-admin.py startproject mysite
(created mysite directory containing four files)
python manage.py runserver
(Validating models... 0 errors found, etc.)
The absolute path is
/home/toewsweb/public_html/pythonlab/mysite
What URL should I be able to use to bring this up in my browser?
I also put mysite at
/home/toewsweb/mysite (since it's not supposed to go in a publicly accessible directory)
What URL should I be able to use in this case?
This is a virtual private server, so I have access to httpd.conf. I have downloaded and installed mod_wsgi and have added it to the Apache configuration. I actually did set a subdomain with a DocumentRoot of /home/toewsweb/public_html/pythonlab/mysite; however, when I point the browser to that subdomain, I just get the directory listing.
[/background information]
Right now, I just want to know how to view what I'm working on in my browser.
Thanks!
For development purposes, there's no need to mess about with configuring WSGI (although it's useful to know, as you will need to do it for production). Just start the dev server so that it listens to an external port:
./manage.py runserver 0:8000
This binds to the external IP address, so now you can access your Django site via port 8000 on that server:
http://whatever.my.ip.is:8000
You need to setup the apache WSGIScriptAlias directive in your VirtualHost to properly load python and your site. Django's docs have a great explanation on what you need to do.
Basic configuration
Once you’ve got mod_wsgi installed and activated, edit your httpd.conf file and add:
WSGIScriptAlias / /path/to/mysite/apache/django.wsgi
The first bit above is the url you want to be serving your application at (/ indicates the root url), and the second is the location of a "WSGI file" -- see below -- on your system, usually inside of your project. This tells Apache to serve any request below the given URL using the WSGI application defined by that file.
Next we'll need to actually create this WSGI application, so create the file mentioned in the second part of WSGIScriptAlias and add:
import os
import sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
If your project is not on your PYTHONPATH by default you can add:
path = '/path/to/mysite'
if path not in sys.path:
sys.path.append(path)
just below the import sys line to place your project on the path. Remember to replace 'mysite.settings' with your correct settings file, and '/path/to/mysite' with your own project's location.
OR
The other option is to run the dev server so it's accesible externally like so:
python manage.py runserver 0.0.0.0:80
though please DO NOT use this in production. The dev server is single-threaded, and has not been auditing for security.

Django Caught an exception while rendering: No module named registration

I seem to have run into a bit of an issue.
I am busy creating an app, and over the last few weeks setup my server to use Git, mod_wsgi to host this app.
Since deploying it, everything seems to be running smoothly however, I had to go through all my files and insert the absolute url of the project to make sure it works fine.
on my local machine
from registration.models import UserRegistration
on server
from myapp.registration.models import UserRegistration
Am I doing something wrong?
And this has also caused an issue for me where I cannot access my django admin interface.
All i get is this: Caught an exception while rendering: No module named registration
Exception Value: Caught an exception while rendering: No module named registration
As far as I am concerned my app has all the relevant urls, but it does not seem to work.
Thank you in advance
The problem is occurring because somehow your local machine is adding the myapp directory to the PYTHONPATH, as well as its parent directory. The way to fix this is to modify your .wsgi script to add both these directories to sys.path:
import sys
sys.path.insert(0, '/path/to/parent')
sys.path.insert(0, '/path/to/parent/myapp')
Read and use improved WSGI script in:
http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
This will set up environment to better match Django built in development server and you shoud hopefully not see a difference between the two, especially in respect of how Python module search path is handled.

Works using Django development server, but throws import error with Apache using mod_wsgi

I have a Django project that works fine with the development server that comes with it.
No errors are produced at all when I use "django manage.py runserver" and the app works fine, but when I try to use it with mod_wsgi and Apache the browser displays "Internal Server Error" with a 500 error code and it generates an import error in the Apache error log.
Here's the error in the log:
ImportError: No module named registration
I'm using the Django registration module which is located in a path like this:
/opt/raj/photos/registration
I know that the registration app is in the path because I can fire up a Python shell, import sys, and get a list of paths using sys.path.
Here are some of the paths output from Python shell:
sys.path
['', '/opt/raj/pyamf', '/opt/raj', '/opt/raj/pictures', '/opt/raj/pictures /registration', '/usr/lib/python2.6',....]
Any thoughts would be appreciated.
Is it in the pythonpath for the webserver? All those '/opt' paths are typically not in the standard python path, so something is adding those for you I would guess. Are you sure it also gets added for the webserver process, or is PYTHONPATH set in some shell configfile somewhere for your user only?
There is a PythonPath directive when using mod_python, is there something similar for mod_wsgi?
This is almost certainly a case of the path not being the same for the webserver as it is for you, so I would focus my search in those areas.