Django + uWSGI + nginx requests hang - django

Question

I'm running a Django web application using Nginx and uWSGI. I'm having problems with the requests hanging for no apparent reason.
I have added a bunch of logging in the application, and this snippet is where it seems to hang. There are two log lines at the start of the try block, and the first one gets printed, but not he second one, so it would seem that it hangs in the middle of the code. This code is from a middleware class that I added in the Django configuration.
def process_request(self, request):
if 'auth' not in request.session:
try:
log.info("Auth not found") # this line is logged
log.info("another log line") # this line is never logged
if request.is_ajax():
return HttpResponse(status=401)
...
I managed to get a backtrace from the uWSGI thread and this is where it's stuck:
*** backtrace of 76 ***
/usr/bin/uwsgi(uwsgi_backtrace+0x2e) [0x45121e]
/usr/bin/uwsgi(what_i_am_doing+0x30) [0x451350]
/lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f8a4b2b8c30]
/lib/x86_64-linux-gnu/libc.so.6(epoll_wait+0x33) [0x7f8a4b37d653]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(+0x27625) [0x7f8a44092625]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(ev_run+0x29b) [0x7f8a4409d11b]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(+0x32bc0) [0x7f8a4409dbc0]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4bd4) [0x7f8a4a0c30d4]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x80d) [0x7f8a4a0c517d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0x162310) [0x7f8a4a0c5310]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7f8a4a08ce23]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0x7d30d) [0x7f8a49fe030d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7f8a4a08ce23]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7f8a4a04b837]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/greenlet.so(+0x375c) [0x7f8a49b1c75c]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/greenlet.so(+0x30a6) [0x7f8a49b1c0a6]
[0x7f8a42f26f38]
*** end of backtrace ***
SIGUSR2: --- uWSGI worker 3 (pid: 76) is managing request /login?next=/&token=45092ca6-c1a0-4c23-9d44-4d171fc561b8 since Wed Dec 2 09:52:44 2015 ---
The Nginx error log prints out [error] 619#0: *55 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: vdr
There are no errors in the printouts from uWSGI, so I'm a bit at a loss. Has anyone seen anything similar? All this is running within a Docker container if that makes any difference.
Nginx conf:
upstream uwsgi {
server unix:///tmp/vdr.sock;
}
server {
listen 80;
charset utf-8;
client_max_body_size 500M;
server_name localhost 172.17.0.2;
location /static {
alias /home/vdr/vdr-ui/static;
}
location / {
include uwsgi_params;
uwsgi_pass uwsgi;
uwsgi_read_timeout 200s;
}
}
uWSGI conf:
[uwsgi]
chdir = %d
module = alft_ui.wsgi:application
uid=1000
master=true
pidfile=/tmp/vdr.pid
vacuum=true
max-requests=5000
processes=4
env=DJANGO_SETTINGS_MODULE=alft_ui.settings.prod-live
home=/home/vdr/vdr-ui/env
socket=/tmp/vdr.sock
chmod-socket=666

Answer 1

So I finally found the cause for this. It turns out that my setup script added some logstash settings to the Django configuration. These settings pointed to the IP 10.8.0.1 which wasn't reachable from this environment. This would explain why the app got stuck on a logging line. Removing these settings made everything work again.
Always good to know that it was your own fault all along :)