I have a django website, that uses an environment variable, DJANGO_MODE to decide which settings to use - development or staging. The environment variable is in the bashrc and when running the app using the development server, everything works fine.
But when I run the app using uWSGI, it doesn't seem to notice the environment variable and uses the default(development) settings instead of production.
I run uWSGI in Emperor mode, and other than the environment variable ignoring, everything seems to be running fine. And yes, the user running uWSGI is the same for which the bashrc has DJANGO_MODE set.
The command used to run uWSGI is -
exec uwsgi --emperor /etc/uwsgi/vassals --uid web_user --gid --web_user
And the ini file for the vassal -
[uwsgi]
processes = 2
socket = /tmp/uwsgi.sock
wsgi-file = /home/web_user/web/project_dir/project/wsgi.py
chdir = /home/web_user/web/project_dir
virtualenv = /home/web_user/.virtualenvs/production_env
logger = syslog
chmod-socket = 777
It cannot work as bash config files are read by the bash. You have to set var in the emperor or in the vassal (the second one is a better approach). Just add
env=DJANGO_MODE=foobar
to your config (do not use whitespace).
Related
I have a nginx server setup with django and uwsgi.
There are multiple django apps run simultaneously on different virualenvs, separate uwsgi processes on separate ports on which nginx listens to.
The setup works alright.
I use pidfile flag to stop uwsgi processes. When I start the any uwsgi server, pidfile is created and I can immediately restart without errors with the script provided below.
After while when I come back to update the server, the pidfile is gone and the uwsgi process is still running. This makes this approach useless.
project_A_uwsgi.ini:
[uwsgi]
master = true
socket = /tmp/stripe_test_uwsgi.sock
chmod-socket = 666
chdir = /home/ubuntu/TestProjects/ProjectA
wsgi-file = /home/ubuntu/TestProjects/ProjectA/ProjectA/wsgi.py
virtualenv = /home/ubuntu/TestProjects/ProjectA/venv
vacuum = true
enable-threads = true
daemonize= /home/ubuntu/TestProjects/ProjectA/uwsgi.log
project_A_server_restart.sh:
#!/bin/sh
DEPLOYMENT_PATH='/home/ubuntu/TestProjects/ProjectA'
. ${DEPLOYMENT_PATH}/venv/bin/activate
uwsgi --stop /home/ubuntu/TestProjects/project_A_uwsgi.pid
uwsgi --ini /home/ubuntu/TestProjects/project_A_uwsgi.ini --pidfile /home/ubuntu/TestProjects/project_A_uwsgi.pid
sudo service nginx restart
What could be the reason for the pidfile to get deleted? how will I prevent that?
I have a wsgi.ini file in my project, and I use uwsgi wsgi.ini to run my project.But when I change the django code,I want to restart the project instead kill uwsgi then reload it. The uwsgi official document provide the following methods:
# using kill to send the signal
kill -HUP `cat /tmp/project-master.pid`
# or the convenience option --reload
uwsgi --reload /tmp/project-master.pid
# or if uwsgi was started with touch-reload=/tmp/somefile
touch /tmp/somefile
But I don't have a project-master.pid file in /tmp catalog in my system(centOS).
my question:
how to use uwsgi restart django instead of kill it then start it?
if use uwsgi official document provided method,how to create a .pid file and what content should in this file?
I find the anwser. project-master.pid is set in wsgi.ini file, you should set pidfile=/tmp/project-master.pid first. Then use uwsgi to start server: uwsgi wsgi.ini.After you start it, you can see a project-master.pid file in /tmp catalog. When you want to reload uwsgi server, you can use such command to restart server: uwsgi --reload /tmp/project-master.pid.
I found simplier answer in my opinion, you can just kill your uwsgi process and then spawn it again:
killall uwsgi
And then just run your uwsgi command again.
You don't need to use uWSGI server for your local development needs. Apache/uWSGI are meant for production, and having them restarted implicitly at every code change is not often desirable. In fact, production server not restarting even after the code is changed often acts as a safety net, so that you don't end up restarting the server without finalising the deployment.
Just use inbuild server django provides with itself.
python manage.py runserver 8000
I am trying to deploy a Django app using nginx + uwsgi.
I created a virtual environment (virtualenv), and installed both uwsgi and Django inside the virtual env (i.e.local to the virtual environment). I have no global Django and uwsgi. When I run the uwsgi --ini project.ini, I am having an 'ImportError: No module named django.core.wsgi' exception:
from django.core.wsgi import get_wsgi_application
ImportError: No module named django.core.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 5987)
spawned uWSGI worker 1 (pid: 5988, cores: 1)
spawned uWSGI worker 2 (pid: 5989, cores: 1)
spawned uWSGI worker 3 (pid: 5990, cores: 1)
spawned uWSGI worker 4 (pid: 5991, cores: 1)
Based on my search, it's recommended to put env and pythonpath variables in the ini if you are using Django1.5 or lower. However, I am using Django 1.7, so I did not place them anymore. Here's my project.ini:
#project.ini file
[uwsgi]
# Django-related settings
# the base directory (full path)
chdir = /root/virtualenv/project
# Django wsgi file
module = project.wsgi:application
# the virtualenv (full path)
home = /root/virtualenv
# process-related settings
# master
master = true
# maximum number of worker processes
processes = 4
# the socket (use the full path to be safe
socket = /root/virtualenv/project/project.sock
# ... with appropriate permissions - may be needed
chmod-socket = 666
chown-socket = root:root
# clear environment on exit
vacuum = true
# other config options
uid = root
gid = root
processes = 4
daemonize = /var/log/uwsgi/project.log
no-site = True
How will i fix this? I'm quite stuck on this for a day already.
Any ideas are greatly appreciated.
Thanks in advance!
your module is pointed to your project, shouldn't it be pointed to your projects main app that way it can find the wsgi file?
so on my INI file looks like this.
In my particular case I'm using a virtual environment, django 1.7 and uwsgi.
vhost = true
plugins = python
socket = /tmp/noobmusic.sock
master = true
enable-threads = true
processes = 2
wsgi-file = /home/myname/webapps/music/music/music/wsgi.py
virtualenv = /home/myname/webapps/music/musicenv/
chdir = /home/myname/webapps/music/music/
this is the only site I've ever setup uwsgi as I typically use mod-wsgi and unfortunately do not remember all the steps.
I had similar issue. Solved it -after many hours- by making sure that uwsgi is installed using same python version (2 / 3) as the python version of your virtualenv. Otherwise it will not use your virtualenv and thus start throwing 'can not find module xyz' kind of errors. To install uwsgi under python3 you have to use pip3 (which in turn might need to be installed with something like 'apt-get install python-pip3'). When calling uwsgi on cli or via .ini file you need to reference your virtualenv mentioning the full path (which ends one folderlevel above the folder in which the /bin/ is; so /example/myvenv/bin/activate means the full path is /example/myvenv.
I made the uwsgi install global so outside of my virtualenv. I suppose above applies/would work as well when installing uwsgi within the virtualenv, but have not tried (yet).
Keep the system-wide uwsgi the same version as your virtual environment python.
In my environment, my virtual environment is python3.7, but the system default python is python3.6.
After I uninstall uWSGI, and re-install the system-wide uWSGI with python3.7, the problem has been resolved.
sudo pip uninstall uwsgi
sudo -H python3.7 -m pip install uwsgi
I can't see any problem in your configuration (though I'm not very good at these topics). I can try to advice some steps to localize the problem.
Test uwsgi without using virtualenv. Note that the virtual directory is just a directory, so add it to your PYTHONPATH and run uwsgi.
Before that you can try
python -c 'import django.core.wsgi'
If that works, then the problem is in uwsgi virtualenv configuration.
Test virtualenv. Run it and check that the module can be imported.
If that works, then the problem is in uwsgi. Go to the previous case.
I have a Django app working with uWSGI. My uWSGI config is:
[uwsgi]
http = 127.0.0.1:8000
env = DJANGO_SETTINGS_MODULE=my_django_app.settings
module = my_django_app.wsgi:application
home = /var/www/myapp/venv
chdir = /var/www/myapp/app
pidfile = /tmp/myapp.pid
logger = syslog
logfile-chown = True
logfile-chmod = 644
touch-logreopen = /var/www/uwsgi/log/rotate_monitor
uid = myapp
gid = myapp
master = True
vacuum = True
harakiri = 20
max-requests = 5000
processes = 4
threads = 1
post-buffering=4096
touch-reload = /var/www/myapp/conf/uwsgi.ini
route = ^/test log:someone called /test
But route directive doesn't work, I get regular uwsgi log entry. I played around it a lot, but can't find a way to make it work. Any ideas? My uWSGI version is 2.0.5.1
I had to install uWSGI with pcre support. It solved problem with routing.
To do it I had to install pcre lib(for Ubuntu the package called libpcre3) and reinstall uWSGI then. Since PCRE is already in your system uWSGI will be automatically compiled with pcre support.
There are also points from #tobias.mcnulty comment that also might be useful:
libpcre3-dev package on Ubuntu also needs to be installed
afterwards you may want to run pip install -I --no-cache-dir uwsgi to reinstall it, otherwise you might just get the same cached wheel.
the error in the logs is !!! no internal routing support, rebuild with pcre support !!!
For my current flask deployment, I had to set up a uwsgi server.
This is how I have created the uwsgi daemon:
sudo vim /etc/init/uwsgi.conf
# file: /etc/init/uwsgi.conf
description "uWSGI server"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /myproject/myproject-env/bin/uwsgi --uid www-data --gid www-data --home /myproject/myproject-env/site/F11/Engineering/ --socket /tmp/uwsgi.sock --chmod-socket --module F11 --callable app --pythonpath /myproject/myproject-env/site/F11/Engineering/ -H /myproject/myproject-env
However after running this successfully: sudo start uwsgi
uwsgi start/running, process 1286
And trying to access the application via browser:
I get a 502 Bad Gateway
and an error entry in nginx error.log:
2013/06/13 23:47:28 [error] 743#0: *296 upstream prematurely closed
connection while reading response header from upstream, client:
xx.161.xx.228, server: myproject.com, request: "GET /show_records/2013/6 HTTP/1.1", upstream:
"uwsgi://unix:///tmp/uwsgi.sock:", host: "myproject.com"
But the sock file has the permission it needs:
srw-rw-rw- 1 www-data www-data 0 Jun 13 23:46 /tmp/uwsgi.sock
If I run the exec command from above in the command line as a process, it works perfectly fine. Why is the daemon not working correctly please?
btw Nginx is running as
vim /etc/nginx/nginx.conf
user www-data;
and vim /etc/nginx/sites-available/default
location / {
uwsgi_pass unix:///tmp/uwsgi.sock;
include uwsgi_params;
}
and it is started as sudo service nginx start
I am running this on Ubuntu 12.04 LTS.
I hope I have provided all the necessary data, hope someone can guide me into the right direction. Thanks.
Finally I have solved this problem after working nearly 2 days on it. I hope this solution will help other flask/uwsgi users that are experiencing a similar problem.
I had two major issues that caused this.
1) The best way to find the problems with a daemon is obviously a log file and a cleaner structure.
sudo vim /etc/init/uwsgi.conf
Change the daemon script to the following:
# file: /etc/init/uwsgi.conf
description "uWSGI server"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /home/ubuntu/uwsgi-1.9.12/uwsgi -c /myproject/uwsgi.ini
vim /myproject/uwsgi.ini
[uwsgi]
socket = /tmp/uwsgi.sock
master = true
enable-threads = true
processes = 5
chdir= /myproject/F11/Engineering
module=F11:app
virtualenv = /myproject/myproject-env/
uid = www-data
gid = www-data
logto = /myproject/error.log
This is much cleaner way of setting up the daemon. Also notice the last line how to setup the log file. Initially I had set the log file to /var/log/uwsgi/error.log. After a lot of sweat and tears I realized the daemon is running as www-data and hence can not access the /var/log/uwsgi/error.log since the error.log was owned by root:root. This made the uwsgi fail silently.
I found it much more efficient to just point the log file to my own /myproject, where the daemon has guaranteed access as www-data. And also don't forget to make the whole project accessible to www-data or the daemon will fail with an Internal Server error message. -->
sudo chown www-data:www-data -R /myproject/
Restart uwsgi daemon:
sudo service uwsgi restart
2) Now you have three log files to lookout for:
tail -f /var/log/upstart/uwsgi.log --> Shows problems with your daemon upon start
tail -f /var/log/nginx/error.log --> Shows permission problems when wsgi access is refused, often because /tmp/uwsgi.sock file is owned by root instead of www-data. In that case simply delete the sock file sudo rm /tmp/uwsgi.sock
tail -f /myproject/error.log --> Shows errors thrown by uwsgi in your application
This combination of log files helped me to figure out that I also had a bad import with Flask-Babel in my Flask application. Bad in that sense, that the way I utilized the library was falling back to the system's locale to determine the datetime format.
File "/myproject/F11/Engineering/f11_app/templates/show_records.html", line 25, in block "body"
<td>{{ record.record_date|format_date }}</td>
File "./f11_app/filters.py", line 7, in format_date
day = babel_dates.format_date(value, "EE")
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 459, in format_date
return pattern.apply(date, locale)
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 702, in apply
return self % DateTimeFormat(datetime, locale)
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 699, in __mod__
return self.format % other
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 734, in __getitem__
return self.format_weekday(char, num)
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 821, in format_weekday
return get_day_names(width, context, self.locale)[weekday]
File "/myproject/myproject-env/local/lib/python2.7/site-packages/babel/dates.py", line 69, in get_day_names
return Locale.parse(locale).days[context][width]
AttributeError: 'NoneType' object has no attribute 'days'
This is the way I was using the Flask filter:
import babel.dates as babel_dates
#app.template_filter('format_date')
def format_date(value):
day = babel_dates.format_date(value, "EE")
return '{0} {1}'.format(day.upper(), affix(value.day))
The strangest part is that this code is working perfectly fine within the dev environment (!). It works even fine when running the uwsgi as a root process from the command line. But it fails when ran by the www-data daemon. This must have something to do with how the locale is set, which Flask-Babel is trying to fall back to.
When I changed the import like this, it all worked finally with the daemon:
from flask.ext.babel import format_date
#app.template_filter('format_date1')
def format_date1(value):
day = format_date(value, "EE")
return '{0} {1}'.format(day.upper(), affix(value.day))
Hence be careful when using Eclipse/Aptana Studio that is trying to pick the right namespace for your classes in code. It can really turn ugly.
It is now working perfectly fine as a uwsgi daemon on an Amazon Ec2 (Ubuntu 12.04) since 2 days. I hope this experience helps fellow python developers.
I gave up, there was no uwsgi.log generated, and nginx just kept complaining about :
2014/03/06 01:06:28 [error] 23175#0: *22 upstream prematurely closed connection while reading response header from upstream, client: client.IP, server: my.server.IP, request: "GET / HTTP/1.1", upstream: "uwsgi://unix:/var/web/the_gelatospot/uwsgi.sock:", host: "host.ip"
for every single request. This only happened if running uwsgi as a service, as a process it started fine under any user. So this would work from command line (page responds):
$exec /var/web/the_gelatospot/mez_server.sh
This didn't (/etc/init/site_service.conf):
description "mez sites virtualenv and uwsgi_django" start on runlevel
[2345] stop on runlevel [06] respawn respawn limit 10 5 exec
/var/web/the_gelatospot/mez_server.sh
The process would start but on each request nginx would complain about the closed connection. Strangely I have this same config working just fine for 2 other apps using the same nginx version and same uwsgi version as well as both apps are mezzanine CMS apps. I tried everything I could think of and what was suggested. In the end I switched to gunicorn which works fine:
#!/bin/bash
NAME="the_gelatospot" # Name of the application
DJANGODIR=/var/web/the_gelatospot # Django project directory
SOCKFILE=/var/web/the_gelatospot/gunicorn.sock # we will communicte using this unix socket
USER=ec2-user
GROUP=ec2-user # the user to run as, the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=settings
#DJANGO_SETTINGS_MODULE=the_gelatospot.settings # which settings file should Django use
#DJANGO_WSGI_MODULE=the_gelatospot.wsgi # WSGI module name
DJANGO_WSGI_MODULE=wsgi
echo "Starting $NAME as `the_gelatospot`"
# Activate the virtual environment
cd $DJANGODIR
source ../mez_env/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
cd ..
# Start your Django GUnicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec gunicorn -k eventlet ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--log-level=debug \
--bind=unix:$SOCKFILE
And here is the one that wouldn't work as a service (nginx complaining of prematurely closed connection and no app log data coming through).
#!/bin/bash
DJANGODIR=/var/web/the_gelatospot/ # Django project directory
cd $DJANGODIR
source ../mez_env/bin/activate
uwsgi --ini uwsgi.ini
And the uwsgi.ini:
[uwsgi]
uid = 222
gid = 500
socket = /var/web/the_gelatospot/uwsgi.sock
virtualenv = /var/web/mez_env
chdir = /var/web/the_gelatospot/
wsgi-file = /var/web/the_gelatospot/wsgi.py
pythonpath = ..
env = DJANGO_SETTINGS_MODULE=the_gelatospot.settings
die-on-term = true
master = true
chmod-socket = 666
;experiment using uwsgitop
worker = 1
;gevent = 100
processes = 1
daemonize = /var/log/nginx/uwsgi.log
logto = /var/log/nginx/uwsgi.logi
log-maxsize = 10000000
enable-threads = true
I went from gunicorn to uWSGI last year and I had no issues with it till now, it also seemed a bit faster than gunicorn. Right now I'm thinking of sticking to gunicorn. Its getting better, puts up much nicer numbers with eventlet installed, and it is easier to configure.
Hope this workaround helps. I would still like to know the issue with uWSGI and nginx but I'm stumped.
Update:
So using gunicorn allowed me to run the server as a service and while toying in mezzanine I ran into this error: FileSystemEncodingChanged
To fix this I found the solution here:
https://groups.google.com/forum/#!msg/mezzanine-users/bdln_Y99zQw/9HrhNSKFyZsJ
And I had to modify it a bit since I don't use supervisord, I only use upstart and a shell script. I added this to right before I execute gunicorn in my mez_server.sh file:
export LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8
exec gunicorn -k eventlet ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--log-level=debug \
--bind=unix:$SOCKFILE
I also tried this fix using uWSGI like so ( a lot shorter since using uwsgi.ini ):
export LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8
exec uwsgi --ini uwsgi.ini
I am still sticking to gunicorn since it still worked with the problem and lead me in the right direction to resolve it. I was very disappointed that uWSGI provided no output in the log file even with these params, I only seen the server start process and that's it:
daemonize = /var/log/nginx/uwsgi.log
logto = /var/log/nginx/uwsgi.logi
While nginx kept throwing the disconnect error, uWSGI sat there like nothing is happening.
As a single line running with daemon true command is
gunicorn app.wsgi:application -b 127.0.0.1:8000 --daemon
bind your app with 127.0.0.1:8000 and --deamon force it to run as daemon
but define a gunicorn_config.cfg file and run with -c flag is good practice
for more:
https://gunicorn-docs.readthedocs.org/en/develop/configure.html