I have recently set up a small personal website that I host on a little machine at home. The machine is running Windows 7, Apache 2.4.12, Mod_WSGI 4.4.6, and Python 3.4.2. I noticed that if the site has not been accessed recently, it takes a long time to load (20 - 30 seconds). I believe part of the problem is that Apache is lazily loading WSGI when it is accessed. This Stack Overflow post suggests that I can have Apache force load WSGI at startup, but then shows an example for daemon mode, which is a Linux only thing(?).
I am not very familiar with the syntax required for the httpd.conf file, and every time I try adding a line such as
WSGIImportScript "c:/.../wsgi.py"
Apache will not start. Is WSGIImportScript supported on windows? Am I not linking to the correct thing?
I was also under the impression that I could set WSGILazyInitialization to Off, and accomplish the same thing, but I can't figure out the correct syntax for that either.
Along the same lines, but a little off topic, due to the small nature of my site, is there a way to have Apache store all of the static files in RAM (<60MB) for faster access?
Thanks!
The WSGILazyInitialization is completely unrelated.
The WSGIImportScript is described in the documentation at:
http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIImportScript
But you are better off not using it. You can do the same by specifying both process-group and application-group arguments to WSGIScriptAlias. Have both defined and it will have a side effect or preloading the script file.
WSGIScriptAlias /some/url /some/path/wsgi.py process-group=%{GLOBAL} application-group=%{GLOBAL}
The bigger question is why is your Apache instance killing the process off handling the requests anyway. A normal Apache configuration would not kill off the process and it should stay in memory.
BTW, you can set the Apache LogLevel to info to cause mod_wsgi to log more information about when it is loading the script file etc.
Related
I'm running django web application on AWS ecs.
I'd like to warm up the server (hit the first request and it takes some time for django to load up) when deploying a new version.
Is there a way for warming up the server before registering it to the Application load balancer?
Edit
I'm using nginx + uwsgi
I asumed that you use mod_wsgi , because that is the behavior described here:
Q: Why do requests against my application seem to take forever, but then after a bit they all run much quicker?
A: This is because mod_wsgi by default performs lazy loading of any application. That is, an application is only loaded the first time
that a request arrives which targets that WSGI application. This means
that those initial requests will incur the overhead of loading all the
application code and performing any startup initialisation.
This startup overhead can appear to be quite significant, especially if using Apache prefork MPM and embedded mode. This is
because the startup cost is incurred for each process and with prefork
MPM there are typically a lot more processes that if using worker MPM
or mod_wsgi daemon mode. Thus, as many requests as there are processes
will run slowly and everything will only run full speed once code has
all been loaded.
Note that if recycling of Apache child processes or mod_wsgi daemon processes after a set number of requests is enabled, or for
embedded mode Apache decides itself to reap any of the child
processes, then you can periodically see these delayed requests
occurring.
Some number of the benchmarks for mod_wsgi which have been posted do not take into mind these start up costs and wrongly try to compare
the results to other systems such as fastcgi or proxy based systems
where the application code would be preloaded by default. As a result
mod_wsgi is painted in a worse light than is reality. If mod_wsgi is
configured correctly the results would be better than is shown by
those benchmarks.
For some cases, such as when WSGIScriptAlias is being used, it is actually possible to preload the application code when the processes
first starts, rather than when the first request arrives. To preload
an application see the WSGIImportScript directive.
I think you may try to use WSGIScriptAlias see more here
I just changed health check from nginx based ones to uwsgi related ones,
create an endpoint in django, and let the ELB use that as health check
I'm running a Django site on Apache2.2 with mod_wsgi 3.3 in daemon mode (on Debian Wheezy).
When I use the touch wsgi.py command on my WSGI script from my shell, the process is reloaded and everything works fine. But, when I modify the wsgi.py file from the web-application code, the modification time is correctly changed (checked by stat) but WSGI process is not restarted. I'm using the same user account when manually touching the file as WSGI-daemon runs under.
I've tried the following two methods for "touching" the file from the web-app code, none of which works:
os.system('touch /abs/path/to/wsgi.py')
using django-rosetta's ROSETTA_WSGI_AUTO_RELOAD setting to do the job for me
Both options above actually do seem to work exactly as when I execute touch manually from the shell. They update all Access, Modify and Change file attributes (I'm using ext4, if that could matter).
I know it's really strange behavior and after I've read all docs I'm hopeless. Does anyone have at least a clue what could be the reason?
Do be aware that the reload only occurs on the next request received by the web application. It is not instantaneous at the point that you touch the wsgi.py file. Although you seem to suggest this isn't the issue, Apache would also normally run as a special user and would not have the ability to modify the WSGI script file unless you had specifically set up permissions to allow it to.
To better determine what is going on, ensure that LogLevel is set to info in Apache configuration and look at what log messages mod_wsgi generates in the Apache error log.
For more details and other ways of triggering reload see:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
Alright, finally found the solution. I was using a custom middleware changing the locale for some URLs. The middleware, however, wasn't calling translation.deactivate() at the end of the request (process_response method) which obviously led to some weird inter-thread sharing of locale selection, thus affecting also all further requests. This was happening only when the middleware changed the locale on the first request after process start.
More info here: set language within a django view
My understanding is that mod_python loads the python process into apache, avoiding the overhead of doing that on each call. My expectation was that this would mean that my django stack would also only be loaded once.
What I am observing, however, is that every request is running the entire django stack from the beginning, as though it were the first request. The settings are re-imported. Middleware __init__'s, which are supposed to be run once at django startup, are run each time. And so forth. It seems to be essentially like I would expect CGI to be.
Is this expected behavior? I have mostly worked with mod_wsgi, which I believe does not work this way, but I have to use mod_python for my current client.
Thanks!
Apache on UNIX systems is a multiprocess system as pointed out by someone else. Also make sure the MaxRequestsPerChild hasn't been set to be 1 in Apache configuration for some reason. Ideally that directive should be set to 0, meaning keep processes around and not recycle them based on number of requests.
It loads Django once per httpd process. Since multiple processes start (each child being a process), multiple instances of Django are started.
I'm reading http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode but it seems like way too much work, I've been restarting my apache2 server gracefully whenever I make tweaks to Django code as it inconsistently picks up the right files and probably tries to rely on cached .pycs.
I setup Django using mod_wsgi using the steps outlined at this blog post.
It automatically reflects updates (although every now and then, there will be a delay for a few minutes - never figure out why nor is it that much of an inconvenience).
If you are having to restart your Apache server then you can't be using mod_wsgi daemon mode. Use daemon mode and then simply touching the WSGI script file when an atomic set of changes have been completed isn't that hard and certainly safer than a system which restarts arbitrarily when it detects any single change. If you do want automatic restart based on code changes, then that is described in that document as well. For a Django slant on it, read:
http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html
http://blog.dscpl.com.au/2009/02/source-code-reloading-with-modwsgi-on.html
What is it about what is documented there which is 'way too much work'?
I'm developing a Django site. I'm making all my changes on the live server, just because it's easier that way. The problem is, every now and then it seems to like to cache one of the *.py files I'm working on. Sometimes if I hit refresh a lot, it will switch back and forth between an older version of the page, and a newer version.
My set up is more or less like what's described in the Django tutorials: http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/#howto-deployment-modwsgi
I'm guessing it's doing this because it's firing up multiple instances of of the WSGI handler, and depending on which handler the the http request gets sent to, I may receive different versions of the page. Restarting apache seems to fix the problem, but it's annoying.
I really don't know much about WSGI or "MiddleWare" or any of that request handling stuff. I come from a PHP background, where it all just works :)
Anyway, what's a nice way of resolving this issue? Will running the WSGI handler is "daemon mode" alleviate the problem? If so, how do I get it to run in daemon mode?
Running the process in daemon mode will not help. Here's what's happening:
mod_wsgi is spawning multiple identical processes to handle incoming requests for your Django site. Each of these processes is its own Python Interpreter, and can handle an incoming web request. These processes are persistent (they are not brought up and torn down for each request), so a single process may handle thousands of requests one after the other. mod_wsgi is able to handle multiple web requests simultaneously since there are multiple processes.
Each process's Python interpreter will load your modules (your custom Python files) whenever an "import module" is executed. In the context of django, this will happen when a new view.py is needed due to a web request. Once the module is loaded, it resides in memory, and so any changes you make to the file will not be reflected in that process. As more web requests come in, the process's Python interpreter will simply use the version of the module that is already loaded in memory. You are seeing inconsistencies between refreshes since each web request you are making can be handled by different processes. Some processes may have loaded your Python modules during earlier revisions of your code, while others may have loaded them later (since those processes had not received a web request).
The simple solution: Anytime you modify your code, restart the Apache process. Most times that is as simple as running as root from the shell "/etc/init.d/apache2 restart". I believe a simple reload works as well, which is faster, "/etc/init.d/apache2 reload"
The daemon solution: If you are using mod_wsgi in daemon mode, then all you need to do is touch (unix command) or modify your wsgi script file. To clarify scrompt.com's post, modifications to your Python source code will not result in mod_wsgi reloading your code. Reloading only occurs when the wsgi script file has been modified.
Last point to note: I only spoke about wsgi as using processes for simplicity. wsgi actually uses thread pools inside each process. I did not feel this detail to be relevant to this answer, but you can find out more by reading about mod_wsgi.
Because you're using mod_wsgi in embedded mode, your changes aren't being automatically seen. You're seeing them every once in a while because Apache starts up new handler instances sometimes, which catch the updates.
You can resolve this by using daemon mode, as described here. Specifically, you'll want to add the following directives to your Apache configuration:
WSGIDaemonProcess example.com processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup example.com
Read the mod_wsgi documentation rather than relying on the minimal information for mod_wsgi hosting contained on the Django site. In partcular, read:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
This tells you exactly how source code reloading works in mod_wsgi, including a monitor you can use to implement same sort of source code reloading that Django runserver does. Also see which talks about how to apply that to Django.
http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html
http://blog.dscpl.com.au/2009/02/source-code-reloading-with-modwsgi-on.html
You can resolve this problem by not editing your code on the live server. Seriously, there's no excuse for it. Develop locally using version control, and if you must, run your server from a live checkout, with a post-commit hook that checks out your latest version and restarts Apache.