Flask + mod_wsgi automatic reload on source code change - flask

Does anyone know how to make a mod_wsgi automatically reload a Flask app when any of the modules changes? I've tried WSGIScriptReloading On, but no luck. The official documentation is kind of a bear ... I guess I'll give it a stab if no one knows. Thanks in advance!
Also, if it could not permanently crash on syntax errors (like the the Flask reloader), that'd be awesome.

With mod_wsgi, WSGIScriptReloading looks for changes to the .wsgi config file, rather than the code.
My workflow is to upload my code changes then just
$ touch MyWebApp.wsgi
which causes the last modified file timestamp to change and mod_wsgi to reload the code.
You can do this 'remotely' by saving the .wsgi file on your local machine and then uploading it again, or I just do it via SSH.
Not a lot you can do about syntax errors, the code is either running or it isn't, but a fix plus a touch will get it running again.
One gotcha to look out for if you're working via FTP: make sure you upload the 'touched' .wsgi file last otherwise it'll try and start with the wrong code.

I think it's a very realistic situation to want to reload source code automatically in production. Think of an environment where sources are deployed per version and a 'production' symlink points to one of those versions. Whenever you want release a newer version, you just point the symlink to another path. But apache and mod_wsgi still collect the files from the symlinked directory and therefor need to have a reloading mechanism in place based on timestamps, size or w/e. Sure, one application might not be a problem, but what about hosting 15-20 applications that are all undergoing active development? Not automatically reloading sources is a pure loss in such a situation compared to restarting apache every single time.
Back to the question: if the framework you're using (in this case flask) does not have a plugin or tool in place for automatic source code reloading, then the two options described by Graham and Malphas are your best options. Either trigger the wsgi process to restart or implement a monitoring system.

You're right about adding the WSGIScriptReloading directive. The Flask docs don't make it 100% clear, but Apache looks for changes to your .wsgi file. The recommended solution is to just execute a touch command on your your .wsgi file, as part of your release process.

For production I prefer apache mod_wsgi too, while for development I use the flask's built-in server. I separate prod and dev config files and I set the debug directive True in the dev config so that Flask can detect and reload code changes automatically.

The correct answer to this question is that the WSGIScriptReloading On needs to be added into 000-default.conf present under /etc/apache2/sites-enabled folder.
See below example
<VirtualHost *:80>
# The ServerName directive sets the request scheme, hostname and port that
# the server uses to identify itself. This is used when creating
# redirection URLs. In the context of virtual hosts, the ServerName
# specifies what hostname must appear in the request's Host: header to
# match this virtual host. For the default virtual host (this file) this
# value is not decisive as it is used as a last resort host regardless.
# However, you must set it for any further virtual host explicitly.
#ServerName www.example.com
ServerName sentiments.live
ServerAdmin admin#sentiments.live
DocumentRoot /var/www/html
WSGIDaemonProcess flaskapp threads=5
WSGIScriptAlias / /var/www/html/sentiments/flaskapp.wsgi
<Directory sentiments>
WSGIScriptReloading On
WSGIProcessGroup sentiments
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
</Directory>

What do you mean 'the official documentation is kind of a bear'? What is wrong with the included recipe:
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Monitoring_For_Code_Changes
That document also explains why WSGIScriptReloading doesn't do what you expect.
And no it is not possible to permanently crash on syntax errors. It is embedded in Apache and the whole point of Apache is to keep stuff running.
Sounds like you should not be using Apache/mod_wsgi for development. Everyones knows one should not use automatic source code reloading in production so can't imagine you would want to do that.

Related

How do I give permission for wsgi to operate in my directory?

Apache and wsgi now have permissions problems accessing the directory in which my Django project is stored. I think I need to get back to having wsgi run as user (or maybe just group) kevin.
I had this working, and I don't know what I did to stop it working, but now I get an permissions error as soon as I get past logging into my app. Unfortunately, I don't have the apache config under version control. I'm pretty sure that's where the problem is because restoring a backup of the project from when it was working didn't change anything, but tweaking permissions on the database and the root directory of the project has altered the particulars of the error message. I wasn't going to give o+w to the whole works, nor did I feel like giving an ACL to every single file or directory.
Background: apache 2 as delivered with Ubuntu 16.04 LTS. Django 1.11. Python 3.5. libsqlite3.0 3.11.0
I thought that what I had in the tail of /etc/apache2/apache2.conf took care of this, in particular the line
WSGIProcessGroup kevin
because all the files and directories are owned by kevin.kevin, and my evidence seems to indicate that when it was working, it did so because it was running with at least effective group 'kevin' and not the current 'www-data'.
The whole tail of stuff I added to apache2.conf for this project now looks like this:
# KOSMANOR:
# In accordance with https://www.mindchasers.com/dev/apache-install
WSGIDaemonProcess kevin processes=4 threads=12 python-path=/build/comprosloco
WSGIProcessGroup kevin
WSGIRestrictEmbedded On
WSGILazyInitialization On
WSGIScriptAlias / /build/comprosloco/comprosloco/wsgi.py
<Directory /build/comprosloco/comprosloco>
Require all granted
</Directory>
# 29 Jan 2018 on kosmanor.com
# 6 Mar 2018 revised reference to 1.11, but it's so modified as to be unrecognizable. Not sure what reference was used.
# 6 Mar 2018 In particular the stuff about virtual environments is ignored here.
# WSGIScriptAlias / is set above in accordance with the mindchaser page
# WSGIPythonPath not set (semms to find my stuff without it)
# WSTIPythonHome is not set (not using a vitual environment yet)
# Added in accordance (somewhat modified) with https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/modwsgi/
#
Alias /static/ /usr/local/lib/python3.5/dist-packages/django/contrib/admin/static/
<Directory /usr/local/lib/python3.5/dist-packages/django/contrib/admin/static>
Require all granted
</Directory>
The documentation for WSGIDaemonProcess can be found at:
http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html
You possibly can use the user and group options.
Otherwise the directories and files need to at least be readable by the Apache user on your system. If needing to write files, then the Apache user would need write access to the files and in cases like SQLite also the directory so it can create a database lock file.
Very helpful comments. I did get it solved, but the details should probably be stated, as something of the sort could happen to others.
Unless you do something about it, WSGI is run as the user that Apache normally starts child processes. On my Ubuntu 16.04, that was 'www-data'. That user does not normally have permissions on the directories I build with my non-privileged user account.
Django is going to need r and x permissions on all directories, and w permission on the database AND THE DIRECTORY WHERE THE DATABASE RESIDES. The latter is because it creates its transaction and locking files there.
If WSGI is failing on account of permissions, the apache error log is your friend. The normal access log can be helpful too.
You can make the WSGI child run as some other user with the WSGIDaemonProcess entry in the apache config file (/etc/apache2/apache2.conf on my system). I added "user=me group=me' to the entry, where 'me' was my account's and group's names. That gave it all the permissions it needed.

Django Admin Static Files on Apache

There are quite a few posts on here regarding the same subject, I've tried as many as I thought applicable. Yet, I still have this error. Everything is being served fine for a django site I've built except the static files for admin.
Versions: Django 1.6, Apache 2.4.7, Ubuntu 14.04
Ok - I have ran manage.py collectstatic and here is the static folder:
/srv/bahai-site/bahai-site/soul/static
This contains four folders: 1) admin/ 2) bootstrap/ 3) core/ and 4) flags/
Within these folders, I can see all expected static files and that they contain the code I'd expect.
When looking at a static file on the site itself, something like /static/core/css/desktop_992.css loads correctly while /static/admin/css/base.css throws a 404.
Apache directives in http.conf:
Alias /static/ /srv/bahai-site/bahai-site/soul/static/
<Directory /srv/bahai-site/bahai-site/soul/static/>
Allow from all
</Directory>
WSGIScriptAlias / /srv/bahai-site/soul/soul/wsgi.py
WSGIPythonPath /srv/bahai-site/soul/soul:/srv/envelope/bahai-site/lib/python2.7/site-packages
<Directory /srv/bahai-site/soul/soul>
<Files wsgi.py>
Allow from all
</Files>
</Directory>
Side question: notice the WSGIScriptAlias and WSGIPythonPath only contains one "bahai-site/" directory level, instead of the two that actually exist after "/srv/". Why is the site running fine? Shouldn't that be causing a problem?
Anyway, what am I doing wrong here? How do I get these admin static files up and running?
as far as the pythonpath is concerned I personally don't include that in my apache virtual host config file and it works. Maybe that's not needed? Honestly I'm not all that well versed in server installations. It defintely seems like a permission issue.
I would run collectstatic using sudo, then I would also make sure the static folder has www-data group permission which is the permission apache uses out of the box.
If you need help setting permissions on a folder in Ubuntu check out this link here. https://askubuntu.com/questions/244406/how-do-i-give-www-data-user-to-a-folder-in-my-home-folder
The problem was that I had updated two different configuration files for Apache with competing directives.
Note to Other Beginners:
Directives (i.e. things like ...) can be placed in any file in the apache2 root -- in my case that was any file within /etc/apache2/. (Question to advanced users: is this only for Ubuntu? Or pretty much any Linux/Unix?) This will confuse you because, if you solve a problem using information from multiple online posts/documents, you'll likely have a smorgasbord of various people's Apache configuration preferences/styles. Hence my problem.
As a beginner, you're probably expecting to see some kind of single location you can put your configurations/customizations (in this case directives) into. That's not the case on Apache. You can put any of these anywhere. This is probably good once you advance your knowledge of Apache but it sucks when you're starting out.
My fix here was to go through all the files I touched in the /etc/apache2/ folder and consolidate any rules into one place. This lead to me removing duplicate/conflicting rules, which solved the problem.
Now I think I need to talk to a therapist....

403 not found in EC2 - apache&WSGI error

I have editted the httpd.conf inside apache by adding:
<Directory /home/ec2-user/hqlocal>
Order deny,allow
Allow from all
</Directory>
WSGIScriptAlias / /home/ec2-user/hqlocal/hq_local/apache/django.wsgi
WSGIPythonPath /usr/lib/python2.7/site-packages
# Serving Django admin static files
Alias /static/admin /usr/lib/python2.7/site-packages/django/contrib/admin/static/admin
# Serving Django static files (our own files use in our app), assuming in settings.py, STATIC_ROOT is “/var/www/static/” and our STATIC_URL is “/static/”
Alias /static /var/www/static/
I have tried many solutions that fix the 403 problem like swapping deny and allow, and so on. But none of them works, I still get 403 cannot access '/'
Therefore, I tried commenting line by line.
The result is that the line
WSGIScriptAlias / /home/ec2-user/hqlocal/hq_local/apache/django.wsgi
causing the problem.
By adding this line, the system becomes 403 but without it, when entering the public DNS, the server will point to Apache default page.
So, I assume that Apache may not have permission to read this file. Therefore, i changed the permission to be global. However, it still didn't work properly.
May I know how to fix this?
Thank you very much.
The directory '/home/ec2-user' is likely not readable to the Apache user. Watch:
http://code.google.com/p/modwsgi/wiki/WhereToGetHelp?tm=6#Conference_Presentations
for common mod_wsgi setup issues including this one.
Also, you should not need:
WSGIPythonPath /usr/lib/python2.7/site-packages
if mod_wsgi is actually compiled against the system Python 2.7. If it isn't compiled against the system Python 2.7, but another version, you should not be forcing it to use modules installed into another Python version as that will cause problems. You should reinstall mod_wsgi with version compiled against correct Python version you want to use.
There are two possible reasons.
Make sure you set the permission correctly for the directory '/home/ec2-user/hqlocal', you can do this:
chmod -R 755 /home/ec2-user/hqlocal
Disable selinux (if using enforcing mode, you need get permissions for apache in selinux). You can configure the /etc/selinux/config:
SELINUX:disabled
Then reboot the machine.

How to setup Django/Apache for a designer's dev environment

I've been developing in my own django environment for a while now using the manage.py runserver with no problems, but now that we've got a designer and a front-end developer needing to work on the project, I find myself at a loss as to what is the Best Practise for their environments.
I could ask them to setup their own python environment, but that's asking an awful lot since they're not Python people and they're running Windows (my dev and the production environment are both Linux).
So instead, I've set them up on a remote server, the disk of which they can mount locally. However in this setup, I'm actually using different instances of manage.py runserver ip:port running in a screen instance. It doesn't handle things like constant reloads very well (common for our designer) and it hangs from time to time due to the single-threaded nature of the dev server. I'd like to know how to set this up with Apache.
The problem with this of course is the staticfiles. Every time either of the aforementioned parties want to add or change a static file, they'd have to run manage.py collectstatic which just isn't practical. I just don't know any other way to do it though. All of the documentation I've found for using Apache is for a production environment, so... that's why I'm here.
Source control? Have them check in changes and then set up a post commit hook to collectstatic and restart the server. With nice windows GUIs I've never had a designer who couldn't grasp the basic concepts. If you're using a dcvs you can always have them in their own fork so you have to merge into the main repos to prevent them from breaking other things by mistake.
The answer to this one was a lot simpler than I thought it would be and I apologise for confusing those who responded. Basically all I wanted was a way to host our designer's dev environment in something more stable than ./manage.py runserver ip:port in a screen session. I figured that there had to be a way to set something like this up for Apache but had no idea what it was.
Here's what I got to work:
In your settings.py set your STATIC_URL and MEDIA_URL variables to relative URLs. In my case I used /static/ and /media/.
MEDIA_ROOT = PROJECT_ROOT + "/htdocs/media/"
MEDIA_URL = "/media/"
SERVE_STATIC = True
STATIC_ROOT = PROJECT_ROOT + "/htdocs/public/"
STATIC_URL = "/static/"
Configure Apache as you would if you didn't have any static files at all. In other words, ignore the recommendations of the docs to use SetHandler None in a <Locaiton> block.
<VirtualHost *:80>
WSGIScriptReloading On
WSGIDaemonProcess someprocessname
WSGIProcessGroup somegroupname
WSGIApplicationGroup somegroupname
WSGIPassAuthorization On
WSGIScriptAlias / /path/to/config.wsgi
ServerName somewhere.awesome.ca
<Location "/">
Order Allow,Deny
Allow from all
</Location>
ErrorLog /var/log/apache2/somewhere.awesome.ca.err
CustomLog /var/log/apache2/somewhere.awesome.ca.log combined
</VirtualHost>
Lastly, you just have to follow the Django howto for serving staticfiles through Python at https://docs.djangoproject.com/en/1.3/howto/static-files/#serving-other-directories
I hope this helps to point someone in the right direction in the future.

multiple django projects, mod_wsgi, single domain

I have two distinct Django "projects", which I want to run on a single domain using mod_wsgi. With mod_python, I believe there was a way to do this, where certain url paths would be mapped to one Django project, and other paths mapped to the other project, all at the server level.
Is it possible to do this with mod_wsgi, and if so, how?
Things I have considered: what goes in the Apache virtual host description, what goes into application.wsgi files, etc. But I haven't figured out exactly how to do this.
Thanks!
This shouldn't be complicated. It's just a matter of setting the WSGIScriptAlias directive - you'll need two of these, one for each path, each pointing to a separate .wsgi file which contains your project settings.
I am also working with Apache and I am running multiple Django projects with one domain. There are only two things you have to do:
Modify your Virtual Host files
Since I am using Debian I have one vhost file for each domain I am hosting. In your vhost file you should have multiple vhost sections. One for each project. Inside these sections you can define WSGIScriptAlias.
<VirtualHost *:80>
...
WSGIScriptAlias / /path/to/project1.wsgi
...
</VirtualHost>
<VirtualHost *:80>
...
WSGIScriptAlias / /path/to/project2.wsgi
...
</VirtualHost>
Of course you have to add all the other necessary information. Project 1 and 2 certainly will have different sub-domains. For example project1.yourdomain.com and project2.yourdomain.com.
Write your *.wsgi files
There are many ways to write and store *.wsgi files. I don't know any best practices. In my case I store them in my project folder.
This is an example:
import os
import sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
sys.path.append('/path/to/your/project')
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
I've seen a lot of other *.wsgi files with more "magic". But this should get you started. You can find a lot of examples all over the internet.
Hope that answers your question. Don't be afraid to ask more questions.