It is working fine under Django's runserver with the monkey patch:
if __name__ == "__main__":
import gevent
from gevent import monkey
monkey.patch_all()
execute_manager(settings)
However, in production we are using Apache with mod_wsgi, and a wsgi file. Putting the above in the WSGI file has no effect. It seem that when the wsgi file is called, it is not as __main__, but removing the if also does nothing.
I found gevent.wsgi.WSGIHandler() and tried to replace django.core.handlers.wsgi with it, but it requires request and application as parameters, which I don't have in my wsgi file.
This is what my wsgi file looks like:
import os,sys
import django.core.handlers.wsgi
from gevent import wsgi
sys.path.append('/app/src')
sys.path.append('/app/src/webInterface')
os.environ['DJANGO_SETTINGS_MODULE'] = 'WebInterface.settings'
#application = django.core.handlers.wsgi.WSGIHandler()
application = wsgi.WSGIHandler()
You are correct that __name__ is not '__main__' in mod_wsgi. Even without the if(), where in the WSGI script file did you place the monkey patch call? You don't show that in what the WSGI script file looks like.
Overall, using gevent monkey patching in mod_wsgi is probably a bad idea anyway. This is because using gevent usually gives people a false sense of security that they no longer have to deal with thread locking because greenlets will to some degree order execution so for simple stuff it isn't needed. It is most definitely a bad idea to rely on that under mod_wsgi because all the request handler threads will still be real threads and not greenlets because the threads are created as external threads using Apache thread APIs. Thus very much still need to handle thread locking properly.
One last thing. You may want to add to your question what you are trying to achieve in doing this, because your attempts at replacing the application with the WSGIHandler from gevent make no sense.
Related
I m new to django. I created a web with dajngo,and successfully deployed it in the server
The python app has been successfully setup and virtual environment has been setup.
but while running the web it gives me "Server Error (500)" I don't know whats the problem.
I think error is in "wsgi.py" file but i'm unable to idenify it.
My wsgi file:
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'karan_web.settings')
application = get_wsgi_application()
my "passenger_wsgi.py" file is:
import imp
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))
wsgi = imp.load_source('wsgi', 'karan_web/wsgi.py')
application = wsgi.application
can someone help me with it;
Sorry for the late answer, I had figured out the as to this just forgot to post it.
As I stated in my question, the actual problem was in passenger_wsgi.py, the django server starts with the wsgi.py and act as the gate way to the Django server.
So whenever a Django project is uploaded on the hosting server, It creates a passenger_wsgi.py file and one default wsgi.py, address of which is provided in passenger_wsgi.py by default.
So we just need to change that address and provide the address of our own wsgi.py in the project
In my case it was
import os
import sys
from karan_web import wsgi
application = wsgi.application
just edit in passenger_wsgi.py the following code.
from karan_web.wsgi import application
You need to check if your code syntax is correct and running properly. If its still doesn't work try to delete and recreate your database in cpanel and check if you have made all necessary migrations, don't forget to restart your python app. If after all these it still doesn't work check if all your files have the correct file permission(766).
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
I want to read environment variables from Apache vhost config file and store them into Django settings.
Before updating to Django 1.7 everything was fine but now it is broken.
It seems the problem is in my wsgi.py script, when I call
_application = get_wsgi_application()
because it reads config file before the environment variable is set.
Is there another way to do this in Django 1.7?
In my /etc/apache2/sites-enabled/mysyte.conf I have:
<VirtualHost *:80>
...
SetEnv SECRET_KEY ...
SetEnv EMAIL_HOST ...
SetEnv EMAIL_HOST_PASSWORD ...
SetEnv EMAIL_HOST_USER ...
SetEnv EMAIL_PORT 25
...
In my wsgi.py:
import os
from os.path import abspath, dirname
from sys import path
SITE_ROOT = dirname(dirname(abspath(__file__)))
path.append(SITE_ROOT)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "locacle.settings.production")
_application = get_wsgi_application()
def application(environ, start_response):
for key, value in environ.items():
if isinstance(environ[key], str):
os.environ[key] = environ[key]
return _application(environ, start_response)
In my settings.py I have:
from os import environ
from base import *
def get_env_setting(setting):
""" Get the environment setting or return exception """
try:
return environ[setting]
except KeyError:
error_msg = "Set the %s env variable" % setting
raise ImproperlyConfigured(error_msg)
EMAIL_HOST = get_env_setting('EMAIL_HOST')
...
This is what log file reports:
...
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/www/.../settings/production.py", line 34, in <module>
EMAIL_HOST = get_env_setting('EMAIL_HOST')
File "/home/www/...settings/production.py", line 21, in get_env_setting
raise ImproperlyConfigured(error_msg)
ImproperlyConfigured: Set the EMAIL_HOST env variable
...
I'm afraid that what you are trying to do here is inherently fragile, only worked by lucky accident on previous versions of Django, and will not work with Django 1.7 or any future version of Django. (Update: it also would make you potentially vulnerable to the "shellshock" bash bug, whereas WSGI normally isn't.)
The basic problem is that that WSGI environ is only available on a per-request basis, but you are trying to set global configuration for your Django process based on it. This is inefficient and conceptually broken (why are you re-setting OS environment variables again and again every single time a request comes in? what if different requests have a different WSGI environ?), and it can only work at all if Django waits to configure itself until the first request arrives.
But the unpredictable timing and ordering of Django's startup sequence in previous versions caused problems. For instance, when using runserver in local development Django would eagerly configure itself, due to validation checks, but under a production environment it would only configure itself lazily (which you were relying on), meaning that sometimes imports would happen in a different order and circular imports would crop up in production that weren't reproducible under runserver.
Django 1.7 includes a revamped startup sequence in order to address these problems, make the startup sequence predictable and consistent between development and production, and allow users to explicitly register code to run at startup time (via AppConfig.ready()). A side effect of this is that settings are configured when your process starts up (specifically, with the call to django.setup() in get_wsgi_application()), not waiting until the first request comes in.
If you're only running one Django site on this server, I would simply move your configuration into normal environment variables rather than SetEnv in your Apache config, and avoid the whole problem.
If you're running multiple Django sites which require different config through a single Apache server, that won't work. In that case, perhaps someone more familiar with Apache and mod_wsgi can give you advice on how to pass environment variables from your Apache config to your Django process in a reliable way; I've personally found the process model (actually, process models, since there's more than one depending how you configure it) of mod_wsgi confusing and error-prone when trying to run multiple independently-configured sites through one server. I find it simpler to use a dedicated WSGI server like gunicorn or uwsgi and have Apache (or nginx) proxy to it. Then it's simple to run multiple gunicorn/uwsgi processes with separate OS environments.
I'm trying to separate out Django's secret key and DB pass into environmental variables, as widely suggested, so I can use identical code bases between local/production servers.
The problem I am running into is correctly setting and then reading the environmental vars on the production server running Apache + mod_wsgi.
Vars set in my user profile aren't available because Apache isn't run as that user. Vars set in the Virtual Hosts file with SetEnv are not available because the scope is somehow different.
I've read a couple 1,2 of SO answers, which leads to this blog with a solution.
I can't figure out how to apply the solution to current versions of Django which use a wsgi.py file, which looks like:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
How can I apply that blog solution to the wsgi.py file, or is there a better place to store env-vars where Django can get at them?
If anyone else is frustrated by Graham's answer, here is a solution that actually works for the original question. I personally find setting environmental variables from Apache to be extremely useful and practical, especially since I configure my own hosting environment and can do whatever I want.
wsgi.py (tested in Django 1.5.4)
from django.core.handlers.wsgi import WSGIHandler
class WSGIEnvironment(WSGIHandler):
def __call__(self, environ, start_response):
os.environ['SETTINGS_CONFIG'] = environ['SETTINGS_CONFIG']
return super(WSGIEnvironment, self).__call__(environ, start_response)
application = WSGIEnvironment()
Of minor note, you lose the future-proof method of django.core.wsgi.get_wsgi_application, which currently only returns WSGIHandler(). If the WSGIHandler.__call__ method is ever updated and you update Django also, you may have to update the WSGIEnvironment class if the arguments change. I consider this a very small penalty to pay for the convenience.
FWIW. Relying on environment variables for fine grained configuration settings are in general not a good idea. This is because not all WSGI hosting environments or commercial PaaS offerings support the concept. Using environment variables for fine grained settings can also effectively lock you into a specific PaaS offering where you have directly embedded a lookup of a specifically named environment variable directly into your code, where the naming convention of that environment variable is specific to that hosting service. So although use of environment variables is pushed by certain services, always be careful of being dependent on environment variables as it will reduce the portability of your WSGI application and make it harder to move between deployment mechanisms.
That all said, the blog post you mention will not usually help. This is because it is using the nasty trick of setting the process environment variables on each request based on the per request WSGI environ settings set using SetEnv in Apache. This can cause various issues in a multi threading configuration if the values of the environment variables can differ based on URL context. For the case of Django, it isn't helpful because the Django settings module would normally be imported before any requests had been handled, which means that the environment variables would not be available at the time required.
This whole area of deployment configuration is in dire need of a better way of doing things, but frankly it is mostly a lost cause because hosting services will not change things to accommodate a better WSGI deployment strategy. They have done their work, have their customers locked into the way they have done it already and aren't about to create work for themselves and change things even if a better way existed.
Anyway, 'all problems in computer science can be solved by another level of indirection'. (http://en.wikipedia.org/wiki/Indirection) and that is what you can do here.
Don't have your application lookup environment variables. Have it import a deployment specific Python configuration module which contains a means of using an API to get the configuration settings. This configuration module would implement different ways of getting the actual settings based on the deployment mechanism. In some cases it could grab the values from environment variables. For other such as with Apache/mod_wsgi, the values could be in that configuration module, or read from a separate configuration file which could be an ini, json or yaml format. In providing a API, it can also map names of configuration settings to different names used by different PaaS offerings.
This configuration module doesn't need to be a part of your application code, but could be manually place into a subdirectory of '/etc/' on the target system. You just then need to set the Python module search path so your application can see it. The whole system could be made quite elegant as part of a wider better standard for WSGI deployment, but as I said, little incentive to do the hard work of creating such a thing when existing PaaS offerings are highly unlikely to change to use such a standard.
Here's an alternative solution that's as future-proof as get_wsgi_application. It even lets you set environment variables to use in your Django initialization.
# in wsgi.py
KEYS_TO_LOAD = [
# A list of the keys you'd like to load from the WSGI environ
# into os.environ
]
def loading_app(wsgi_environ, start_response):
global real_app
import os
for key in KEYS_TO_LOAD:
try:
os.environ[key] = wsgi_environ[key]
except KeyError:
# The WSGI environment doesn't have the key
pass
from django.core.wsgi import get_wsgi_application
real_app = get_wsgi_application()
return real_app(wsgi_environ, start_response)
real_app = loading_app
application = lambda env, start: real_app(env, start)
I'm not 100% clear how mod_wsgi manages its processes, but I assume it doesn't re-load the WSGI app very often. If so, the performance penalty from initializing Django will only happen once, inside the first request.
Alternatively, if you don't need to set the environment variables before initializing Django, you can use the following :
# in wsgi.py
KEYS_TO_LOAD = [
# A list of the keys you'd like to load from the WSGI environ
# into os.environ
]
from django.core.wsgi import get_wsgi_application
django_app = get_wsgi_application()
def loading_app(wsgi_environ, start_response):
global real_app
import os
for key in KEYS_TO_LOAD:
try:
os.environ[key] = wsgi_environ[key]
except KeyError:
# The WSGI environment doesn't have the key
pass
real_app = django_app
return real_app(wsgi_environ, start_response)
real_app = loading_app
application = lambda env, start: real_app(env, start)
For Django 1.11:
Apache config:
<VirtualHost *:80 >
...
SetEnv VAR_NAME VAR_VALUE
</VirtualHost>
wsgi.py:
import os
import django
from django.core.handlers.wsgi import WSGIHandler
class WSGIEnvironment(WSGIHandler):
def __call__(self, environ, start_response):
os.environ["VAR_NAME"] = environ.get("VAR_NAME", "")
return super(WSGIEnvironment, self).__call__(environ, start_response)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
django.setup(set_prefix=False)
application = WSGIEnvironment()
I ran into the same problem of separating production and development code while tracking both of them into version control. I solved the problem using different wsgi scripts, one for production server and one for development server. Create two different setting files as mentioned here. And reference them in wsgi scripts. For example the following is wsgi_production.py file
...
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.production")
...
and in wsgi_development.py file
...
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.development")
...
I'm trying to use server push with my Django app. Of all the different solutions I've seen, django-socketio seemed to be the easiest to implement, and I got it working when launching the server through manage.py. However, when it goes to production, I'd like it to be served through Apache. In the past, I've done this with wsgi, but django-socketio isn't playing nicely with the default wsgi script. Is there something simple I can just change in my django.wsgi that lets apache do the right thing? If not, what would be the suggested way to handle this?
EDIT: Here's the WSGI script I was normally using (without any kind of push server), plus a bit more explanation.
import os, sys
locale = os.path.realpath(__file__)
ROOT_DIR = locale[:locale.find('/server/apache/')]
sys.path.append(ROOT_DIR)
sys.path.append(ROOT_DIR+'/server')
os.environ['DJANGO_SETTINGS_MODULE'] = 'server.settings'
os.environ['PYTHON_EGG_CACHE'] = '/var/www/.python-eggs'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
sys.stdout = sys.stderr
The sys.stdout = sys.stderr is just to redirect any test print statements to apache's error log. The problem is, when I use this, I get an error complaining that request.environ doesn't have the key "socketio" or "DJANGO_SOCKETIO_PORT". I can add DJANGO_SOCKETIO_PORT easily enough (os.environ['DJANGO_SOCKETIO_PORT']="9000"), but from what I can tell, request.environ['socketio'] is set to an instance of SocketIOProtocol somewhere in django-socketio's internals. Also, after looking at the command that django-socketio added to manage.py, I noticed that it creates an instance of SocketIOServer, and calls serve_forever on it, but I have no idea where to put that in my code. Hopefully this will make it easier to see what I'm trying to get done.