Could anybody please help me, how to add Username and Clinet IP Address in the Django manage.py runserver default output.
currently I am seeing:
[01/Jul/2019 11:34:27] "GET / HTTP/1.1" 200 237
expected result:
[pid: 10|app: 0|req: 1/2] 10.176.123.254 (Logged in username/email) {86 vars in 4942 bytes} [Mon Jul 1 06:08:37 2019] GET / => generated 291 bytes in 1160 msecs (HTTP/1.1 200) 7 headers in 250 bytes (1 switches on core 0)
I have a Django application that is served within docker container via uwsgi. I have prepared a custom view just to reproduce the issue I'm mentioning. It looks exactly like below:
def get(self, request):
logger = logging.getLogger('ReleaseReport')
logger.critical('Entering and sleeping')
time.sleep(180)
logger.critical('Awaking')
return Response({'response': 'anything'})
The only thing it does (intentionally) is to log message, sleep for 3 minutes, and log another message afterwards.
Here is how the log file works after I try to visit the view from Firefox / Chrome / PyCharm's rest API client:
spawned uWSGI worker 9 (pid: 14, cores: 1)
spawned uWSGI worker 10 (pid: 15, cores: 1)
spawned uWSGI http 1 (pid: 16)
CRITICAL 2018-08-31 12:10:37,658 views Entering and sleeping
CRITICAL 2018-08-31 12:11:37,742 views Entering and sleeping
CRITICAL 2018-08-31 12:11:38,687 views Awaking
[pid: 10|app: 0|req: 1/1] 10.187.133.2 () {36 vars in 593 bytes} [Fri Aug 31 12:10:37 2018] GET /api/version/ => generated 5156 bytes in 61229 msecs (HTTP/1.1 200) 4 headers in 137 bytes (1 switches on core 0)
CRITICAL 2018-08-31 12:12:37,752 views Entering and sleeping
CRITICAL 2018-08-31 12:12:38,784 views Awaking
[pid: 15|app: 0|req: 1/2] 10.187.133.2 () {36 vars in 593 bytes} [Fri Aug 31 12:11:37 2018] GET /api/version/ => generated 5156 bytes in 61182 msecs (HTTP/1.1 200) 4 headers in 137 bytes (1 switches on core 0)
CRITICAL 2018-08-31 12:13:38,020 views Entering and sleeping
CRITICAL 2018-08-31 12:13:38,776 views Awaking
[pid: 10|app: 0|req: 2/3] 10.187.133.2 () {36 vars in 593 bytes} [Fri Aug 31 12:12:37 2018] GET /api/version/ => generated 5156 bytes in 61034 msecs (HTTP/1.1 200) 4 headers in 137 bytes (1 switches on core 0)
After one minute, the view seems to be executed again, and after another minute the third time is executed. Moreover, the log says it is HTTP 200, but the client doesn't receive the data and just says it cannot load it after few more minutes (depends on client). However the first HTTP 200 in log occurs much earlier before client gives up.
Any clues what may be causing that issue? Here is my uwsgi.ini:
[uwsgi]
http = 0.0.0.0:8000
chdir = /app
module = web_server.wsgi:application
pythonpath = /app
static-map = /static=/app/static
master = true
processes = 10
vacuum = true
The Dockerfile command is as following:
/usr/local/bin/uwsgi --ini /app/uwsgi.ini
In my real application, it causes that client thinks the request failed, but since it was actually executed and finished 3 times, there are 3 records in a database. Changing worker processes number to 1 doesn't change much. Instead of waitin one minute to spawn view again, it is spawned after previous one finishes.
What's wrong with my configuration?
EDIT:
I have changed my view a bit, it now accepts sleep time parameter and looks like this:
def get(self, request, minutes=None):
minutes = int(minutes)
original_minutes = minutes
logger = logging.getLogger(__name__)
while minutes > 0:
logger.critical(f'Sleeping, {minutes} more minutes...')
time.sleep(60)
minutes -= 1
logger.critical(f'Slept for {original_minutes} minutes...')
return Response({'slept_for': original_minutes})
Now, curling:
> curl http://test-host/api/test/0
{"slept_for":0}
> curl http://test-host/api/test/1
{"slept_for":1}
> curl http://test-host/api/test/2
curl: (52) Empty reply from server
In log:
CRITICAL 2018-08-31 14:23:36,200 views Slept for 0 minutes...
[pid: 10|app: 0|req: 1/14] 10.160.43.172 () {28 vars in 324 bytes} [Fri Aug 31 14:23:35 2018] GET /api/test/0 => generated 15 bytes in 265 msecs (HTTP/1.1 200) 4 headers in 129 bytes (1 switches on core 0)
CRITICAL 2018-08-31 14:23:42,878 views Slept for 0 minutes...
[pid: 10|app: 0|req: 2/15] 10.160.43.172 () {28 vars in 324 bytes} [Fri Aug 31 14:23:42 2018] GET /api/test/0 => generated 15 bytes in 1 msecs (HTTP/1.1 200) 4 headers in 129 bytes (1 switches on core 0)
CRITICAL 2018-08-31 14:23:46,370 views Sleeping, 1 more minutes...
CRITICAL 2018-08-31 14:24:46,380 views Slept for 1 minutes...
[pid: 10|app: 0|req: 3/16] 10.160.43.172 () {28 vars in 324 bytes} [Fri Aug 31 14:23:46 2018] GET /api/test/1 => generated 15 bytes in 60011 msecs (HTTP/1.1 200) 4 headers in 129 bytes (1 switches on core 0)
CRITICAL 2018-08-31 14:27:06,903 views Sleeping, 2 more minutes...
CRITICAL 2018-08-31 14:28:06,963 views Sleeping, 1 more minutes...
CRITICAL 2018-08-31 14:29:06,995 views Slept for 2 minutes...
[pid: 9|app: 0|req: 1/17] 10.160.43.172 () {28 vars in 324 bytes} [Fri Aug 31 14:27:06 2018] GET /api/test/2 => generated 15 bytes in 120225 msecs (HTTP/1.1 200) 4 headers in 129 bytes (1 switches on core 0)
If I use the same command to test server running with manage.py runserver, it does answer every time - no matter if I sleep 2 minutes or 10. So it is not the client fault.
I've changed harakiri to 3600, no change.
EDIT2 (my Dockerfile):
FROM python:3.7.0-alpine
ADD . /app
RUN set -ex \
&& apk add mysql-dev \
pcre-dev \
&& apk add --no-cache --virtual .build-deps \
gcc \
make \
libc-dev \
musl-dev \
linux-headers \
libffi-dev \
&& pip install --no-cache-dir -r /app/requirements.txt \
&& runDeps="$( \
scanelf --needed --nobanner --recursive /venv \
| awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
| sort -u \
| xargs -r apk info --installed \
| sort -u \
)" \
&& apk add --virtual .python-rundeps $runDeps \
&& apk del .build-deps
WORKDIR /app
RUN mkdir -p static
RUN python manage.py collectstatic --clear --noinput
EXPOSE 8000
CMD ["/usr/local/bin/uwsgi", "--ini", "/app/uwsgi.ini"]
It actually was the Dockerfile issue. Previously I had uWSGI in my requirements.txt, so it was installed by pip install.
When I've removed it and added uwsgi-python3 to apk add, now everything is fine.
No idea why it matters (everything else was working fine), but it solved my issue.
My uWSGI logs are full of entries like this for every request:
localhost [pid: 4029|app: 0|req: 1/1] 127.0.0.1 () {48 vars in 906
bytes} [Wed Mar 23 18:35:38 2016] GET / => generated 94847 bytes in
1137 msecs (HTTP/1.1 200) 4 headers in 224 bytes (1 switches on core
0)
I'm not sure if it's uWSGI spamming them or Django. How can I disable the request logging?
What does your uwsgi.ini look like? That's where you'd want to disable logging.
Set: disable-logging=True
Here's the docs:
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#disable-logging
Or from the console with:
uwsgi --disable-logging
I use nginx and uwsgi deploy my django webapp, the start few minutes all things well but after some time the ngnix will report 502 error and I check the uwsgi stderr log. there are some error marked => generated & SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request such as blow error message:
>[pid: 27427|app: 0|req: 1/1] 10.209.78.30 () {38 vars in 1030 bytes} [Mon Sep 29 15:46:00 2014] GET blabla...(just request url address) => generated 16018 bytes in 1428 msecs (HTTP/1.1 200) 6 headers in 157 bytes (1 switches on core 0)
Mon Sep 29 15:46:03 2014 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request blabla...(just request url address) (ip 10.209.78.30) !!!
Mon Sep 29 15:46:03 2014 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 287] during GET blabla...(just request url address) (10.209.78.30)
IOError: write error
[pid: 27425|app: 0|req: 1/2] 10.209.78.30 () {38 vars in 2050 bytes} [Mon Sep 29 15:46:00 2014] GET blabla...(just request url address) => generated 0 bytes in 3578 msecs (HTTP/1.1 200) 6 headers in 0 bytes (0 switches on core 0)
Mon Sep 29 15:46:04 2014 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request blabla...(just request url address) (ip 10.217.95.130) !!!
Mon Sep 29 15:46:04 2014 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 287] during GET blabla...(just request url address) (10.217.95.130)
IOError: write error
[pid: 27426|app: 0|req: 1/3] 10.217.95.130 () {38 vars in 1871 bytes} [Mon Sep 29 15:46:00 2014] GET blabla...(just request url address) => generated 0 bytes in 4415 msecs (HTTP/1.1 200) 6 headers in 0 bytes (0 switches on core 0)
Mon Sep 29 15:46:05 2014 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request blabla...(just request url address) (ip 10.217.95.130) !!!
Mon Sep 29 15:46:05 2014 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 287] during GET blabla...(just request url address) (10.217.95.130)
IOError: write error
I replace the request url by blabla...
My nginx.conf is bleow:
location / {
add_header "Access-Control-Allow-Origin" "*";
add_header "Access-Control-Allow-Methods" "GET, OPTIONS";
add_header "Access-Control-Allow-Headers" "origin, authorization, accept";
uwsgi_pass unix:///tmp/djangoapp.sock;
include uwsgi_params;
}
uWSGI launch command:
uwsgi --socket /tmp/djangoapp.sock -p 10 --wsgi-file /opt/graphite/conf/graphite_wsgi.py --chmod-socket=666
I am at the end of my rope here... and plan to start digging into Django source code to figure this out.
I have a Django app that has been created in the standard fashion via start project. So far I have been testing it via "python manage.py runserver 8081".
Now I am trying to put this behind Nginx using mod_uwsgi.
So I test it using the excellent instructions over here.
My ini file looks as follows:
uwsgi]
chdir=<path_to_my_project>
module=<application>.wsgi:application
pidfile=/tmp/<pid_file_name>
max-requests=5000
daemonize=/var/log/uwsgi/<log_file_name>
env=DJANGO_SETTINGS_MODULE=<application>.settings
http-socket=127.0.0.1:50000
So I start the app using:
uwsgi --ini uwsgi.ini
All seems to look good:
Log file at "/var/log/uwsgi/" created: Looks good.
Directory is changed: Looks good. Log file indicates so.
Module is loaded: Assume that it has... I don't see any error message in the logs.
pidfile is created: Looks good. I do see the correct PID number in the file.
Tested the closing of the process via SIGINT.
Listening at http-socket: Looks good.
PROBLEM:
When I hit the site via http://site_url; I get the HTML page in my browser. But none of the CSS styles are reflected. Also, none of my java-script functions are executed. So not sure what is amiss.
FYI: I noticed the following lines in the log:
[pid: 15730|app: 0|req: 7/7] 127.0.0.1 () {36 vars in 696 bytes} [Thu Apr 25 16:40:50 2013] GET / => generated 11774 bytes in 2 msecs (HTTP/1.1 200) 1 headers in 59 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 8/8] 127.0.0.1 () {36 vars in 710 bytes} [Thu Apr 25 16:40:50 2013] GET /static/jquery/jquery-1.9.1.min.js => generated 2874 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 9/9] 127.0.0.1 () {36 vars in 727 bytes} [Thu Apr 25 16:40:50 2013] GET /static/bootstrap/css/bootstrap.css => generated 2877 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 10/10] 127.0.0.1 () {36 vars in 716 bytes} [Thu Apr 25 16:40:50 2013] GET /static/bootstrap/js/bootstrap.min.js => generated 2883 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 11/11] 127.0.0.1 () {32 vars in 570 bytes} [Thu Apr 25 16:40:50 2013] GET /favicon.ico => generated 2808 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 12/12] 127.0.0.1 () {36 vars in 696 bytes} [Thu Apr 25 16:40:51 2013] GET / => generated 11774 bytes in 2 msecs (HTTP/1.1 200) 1 headers in 59 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 13/13] 127.0.0.1 () {36 vars in 710 bytes} [Thu Apr 25 16:40:51 2013] GET /static/jquery/jquery-1.9.1.min.js => generated 2874 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 14/14] 127.0.0.1 () {36 vars in 727 bytes} [Thu Apr 25 16:40:51 2013] GET /static/bootstrap/css/bootstrap.css => generated 2877 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 15/15] 127.0.0.1 () {36 vars in 716 bytes} [Thu Apr 25 16:40:51 2013] GET /static/bootstrap/js/bootstrap.min.js => generated 2883 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
[pid: 15730|app: 0|req: 16/16] 127.0.0.1 () {32 vars in 570 bytes} [Thu Apr 25 16:40:51 2013] GET /favicon.ico => generated 2808 bytes in 3 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
Is there some additional config that I seem to have missed?
Would greatly appreciate your time and input. I will leave a solution in case I find one.
Best...
You may want to check the section on the django docs about serving static files. The rules for mod_wsgi apply to mod_uwsgi. A more easy approach is configuring uWSGI to serve static files:
http://uwsgi-docs.readthedocs.org/en/latest/StaticFiles.html
Check the part about offloading if your site is pretty loaded
Generally avoid using django for serving static files
I am not using nginx but Apache, running mod_wsgi.
There I have the possibility to use
Alias /static/ /var/www/django-project/static/
All files are in my directory and accesaible at http://example.com/static/.
So maybe you can do the same with nginx.
Edit:
This skips the process of wsgi/cgi/uwsgi and allows the http daemon to handle the static files directly. There is no need for python to parse these kind of requests - because they're only accessing static files.