Since upgrading to django 1.5 my logs show several SuspiciousOperation exceptions with the text:
Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): <my server's ip>
Is this genuinely a 'suspicious' request, or should I always be including my server's IP address in the ALLOWED_HOSTS setting in addition to my domain name? Any idea what would be making requests with HTTP_HOST = "ip address" rather than HTTP_HOST = "domain name"?
Here is the request environment:
{'HTTP_ACCEPT_ENCODING': "'identity'",
'HTTP_CONNECTION': "'close'",
'HTTP_HOST': "'168.62.208.14'",
'HTTP_X_FORWARDED_PROTOCOL': "'https'",
'HTTP_X_REAL_IP': "'176.10.35.241'",
'HTTP_X_SCHEME': "'https'",
'PATH_INFO': "u'/'",
'QUERY_STRING': "''",
'RAW_URI': "'/'",
'REMOTE_ADDR': "'127.0.0.1'",
'REMOTE_PORT': "'45068'",
'REQUEST_METHOD': "'GET'",
'SCRIPT_NAME': "u''",
'SERVER_NAME': "'168.62.208.14'",
'SERVER_PORT': "'80'",
'SERVER_PROTOCOL': "'HTTP/1.0'",
'SERVER_SOFTWARE': "'gunicorn/0.14.6'",
'gunicorn.socket': "'<socket._socketobject object at 0x7ab3b40>'",
'wsgi.errors': '"<open file \'<stderr>\', mode \'w\' at 0x7f0c94810270>"',
'wsgi.file_wrapper': "'<class gunicorn.http.wsgi.FileWrapper at 0x34eec80>'",
'wsgi.input': "'<gunicorn.http.body.Body object at 0x2a0bf10>'",
'wsgi.multiprocess': 'False',
'wsgi.multithread': 'False',
'wsgi.run_once': 'False',
'wsgi.url_scheme': "'http'",
'wsgi.version': '[1, 0]'}
No, it shouldn't
By default, there are no reasons why IP address should be accepted as a valid HOST header. This message is a sign of a misconfigured production environment: such requests shouldn't reach the back-end.
Here's a post on security.stackexchange.com on Host header poisoning & ALLOWED_HOSTS.
What to do
Filter out all requests with an invalid HOST header before they reach django back-end.
How to
Most likely you're using nginx as a reverse proxy in front of django. If you don't use any reverse proxy at all (or you're using runserver), you have to (otherwise you're risking your security).
Add a default server block returning 444 at the top of your configuration. It should be the first server block in the configuration:
# File: /etc/nginx/sites-available/domain.com
upstream django_server {
server 127.0.0.1:8000;
}
# Catch all requests with an invalid HOST header
server {
server_name "";
listen 80;
return 444;
}
# Your config goes there
server {
server_name domain.com;
listen 80;
location / {
proxy_pass http://django_server;
}
}
NO, IT SHOULDN'T.
Usually it's not a secure way to configure your Django server. Sometimes, e.g., when testing your application, you may access it via direct IP address, but in there's no reason to disable log warnings.
My old answer was wrong, thanks to Max Malysh for pointing that out.
Old answer (INSECURE):
Short answer is: YES (according to provided headers).
Long answer:
According to documentation:
If the Host header (or X-Forwarded-Host if USE_X_FORWARDED_HOST is enabled) does not match any value in this list, the django.http.HttpRequest.get_host() method will raise SuspiciousOperation.
In other words: if your requests pass your server ip address as Host header (and apparently they do), and you think it's okay, then YES, you should add server ip to ALLOWED_HOSTS.
Also, ip address could be in HTTP_HOST for many reasons, also someone could directly ask for ip address.
In practice, just edit the file MyProjectName/settings.py and add the host IP (IP address to the machine in which you're running Django) to the list ALLOWED_HOSTS, which by default is empty.
So, in your, case we would have before the changes:
...
ALLOWED_HOSTS = []
...
After changes:
...
ALLOWED_HOSTS = ['168.62.208.14'] #Make sure your host IP is a string
...
Run the server again and you should be good. Here's an example using the port 8000:
python manage.py runserver 168.62.208.14:8000
. Now if you go to your browser and enter the address http://168.62.208.14:8000, you should find yourself in the page "Congratulations on your first Django-powered page".
Related
I can normally open up my web in the front several hours after deployment,but later , it occurred 502 bad gateway ,it is so wired, my web uses Django and Nginx and Uwsgi, i do research a lot on google,but failed with nothing
Here is my configuration:
1.Nginx configuration
# mysite_nginx.conf
upstream django {
server 127.0.0.1:8004; # for a web port socket (we'll use this first)
}
server {
listen 80;
server_name www.example.com # substitute your machine's IP address or FQDN
charset utf-8;
client_max_body_size 75M; # adjust to taste
location /media {
alias /home/blender_font_project/django_file/Blender_website/media;
}
location /static {
alias /home/blender_font_project/django_file/Blender_website/static;
}
location / {
uwsgi_pass 127.0.0.1:8003;
include /etc/nginx/uwsgi_params;
}
}
2.uwsgi configuration
# mysite_uwsgi.ini file
[uwsgi]
chdir = /home/blender_font_project/django_file/Blender_website
module = djangoTest5.wsgi
master = true
processes = 10
socket = :8003
vacuum = true
harakiri=60
daemonize=/home/blender_font_project/uwsgi_file/real3dfont_logfile
3.this is my Nginx error log
231 connect() failed (111: Connection refused) while connecting to upstream
BTW , i have set Django to DEBUG Ture and i can access my resource by www.example.com/static/example.jpg,but the web page shows 502
I really dont know why , thanks if you offer any help!
(...After million years struggle and strive,with inspiration from a super hero in comment named #Atul Mishra , i finally figure it out...)
It is the matter Django itself,i forget to download mysql module in View , i would have expect a Django error html if it is the django problem, but no , so i mistakenly attribute it to Nginx or Uwsgi
But the wired thing is that Django should have report the error , but no ! what an irresponsible dude!!
so , 1.remember to add Django error log function ,it saves your life , and
2.test Django with runserver before Nginx enter the stage even when comet is striking the earth!!
My django webapp runs properly using gunicorn, proxied through nginx; but I keep getting error emails: [Django] ERROR (EXTERNAL IP): Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS. Each time the IP changes, and I suspect this is a security issue.
I've only started to notice this issue a week ago, and the only significant change I've made was to set up using Cloudflare as CDN
Report at /
Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS.
Request Method: HEAD
Request URL: http://112.124.42.80/
Django Version: 2.0.6
Python Executable: /xxx/xxxx/xxxx/.virtualenv/bin/python3.6
Python Version: 3.6.8
META:
HTTP_ACCEPT = 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2'
HTTP_ACCEPT_ENCODING = 'gzip'
HTTP_CONNECTION = 'close'
HTTP_HOST = '112.124.42.80'
HTTP_PROXY_CONNECTION = 'keep-alive'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'
HTTP_X_FORWARDED_FOR = '60.191.52.254'
HTTP_X_FORWARDED_PROTO = 'http'
HTTP_X_REAL_IP = '60.191.52.254'
PATH_INFO = '/'
QUERY_STRING = ''
RAW_URI = '/'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '49562'
REQUEST_METHOD = 'HEAD'
SCRIPT_NAME = ''
SERVER_NAME = '0.0.0.0'
SERVER_PORT = '3030'
SERVER_PROTOCOL = 'HTTP/1.0'
SERVER_SOFTWARE = 'gunicorn/19.9.0'
gunicorn.socket = <gevent._socket3.socket object, fd=13, family=2, type=2049, proto=0>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f81a34dccc0>
wsgi.file_wrapper = ''
wsgi.input = <gunicorn.http.body.Body object at 0x7f81a34dc9e8>
wsgi.multiprocess = True
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version = '(1, 0)'
I assume your web app is usually accessed through a fully qualified domain name. It sounds like different IP addresses are pointing to your server, and nginx is routing these through to gunicorn, and so to Django. Django is doing the right thing in blocking them, but it would be better if nginx was not passing them through. You should change your nginx configuration so it only proxies the addresses you expect. You might want to look at the default_server settings.
My Django website is in HTTPS. When I am trying to POST data to the website from a script I get this error : "referer checking failed - no Referer". It seems to be a CSRF issue but I do not know how to solve it.
Example :
import requests
r = requests.post('https://mywebsite/mypage', data = {'key':'value'})
print r.text
gives me this output :
[...]
<p>Reason given for failure:</p>
<pre>
Referer checking failed - no Referer.
</pre>
<p>In general, this can occur when there is a genuine Cross Site Request Forgery, or when
<a
href="https://docs.djangoproject.com/en/1.8/ref/csrf/">Django's
CSRF mechanism</a> has not been used correctly. For POST forms, you need to
ensure:</p>
<ul>
<li>Your browser is accepting cookies.</li>
<li>The view function passes a <code>request</code> to the template's <code>render</code>
method.</li>
<li>In the template, there is a <code>{% csrf_token
%}</code> template tag inside each POST form that
targets an internal URL.</li>
<li>If you are not using <code>CsrfViewMiddleware</code>, then you must use
<code>csrf_protect</code> on any views that use the <code>csrf_token</code>
template tag, as well as those that accept the POST data.</li>
</ul>
[...]
Do I need to pass a referer to my headers before sending the POST data - which would not be convenient ? Or should I disable CSRF for this page ?
Thanks
AFAIK, This is the purpose of CSRF, to avoid posting data from unknown strange sources. You need csrf token to post this which django generates dynamically.
Upgrading Django might fix the missing Referer error.
As of Django 4.0 (release notes), the backend will first check the Origin header before falling back to the Referer header (source):
CsrfViewMiddleware verifies the Origin header, if provided by the browser, against the current host and the CSRF_TRUSTED_ORIGINS setting. This provides protection against cross-subdomain attacks.
In addition, for HTTPS requests, if the Origin header isn’t provided, CsrfViewMiddleware performs strict referer checking. This means that even if a subdomain can set or modify cookies on your domain, it can’t force a user to post to your application since that request won’t come from your own exact domain.
It's possible you have a reverse proxy running, for example an nginx proxy_pass to 127.0.0.1:8000?
In this case, Django expects the Cross-Site Forgery Protection tokens to match hostname 127.0.0.1, but they will be coming from a normal domain (for example example.com).
Expected Source
Actual Source
http://127.0.0.1
https://example.com
HTTP reverse proxy (example.com:80 -> localhost:3000) is a common way to use nginx with NodeJS applications, but it doesn't work well with Django
Client-Facing URL
Server Proxy URL
https://example.com
http://127.0.0.1:3000
It is better to run Django through a Unix socket rather than a port (example.com:80 -> <socket>). You can do this with Gunicorn:
Client-Facing URL
Server Proxy URL
https://example.com
unix:/run/example.com.sock
Here's how to do this with Django, Gunicorn, and nginx:
Let's say you've got a Django project root, which contains a system folder (the one where settings.py and wsgi.py are):
export DJANGO_PROJECT_PATH=/path/to/django/root
export DJANGO_SETTING_FOLDER=system
First, make sure you have Gunicorn installed and that you are using a virtual environment:
cd $DJANGO_PROJECT_PATH
source .venv/bin/activate # <- Use a virtual environment
pip3 install gunicorn # <- install Gunicorn in the venv
Run Gunicorn. This will start the Django project similar to running python3 manage.py runserver, except that you can listen for requests on a Unix socket:
$DJANGO_PROJECT_PATH/.venv/bin/gunicorn \
--workers=3 \
--access-logfile - \
--bind unix:/run/example.com.sock \ # <- Socket
--chdir=$DJANGO_PROJECT_PATH/ \
$DJANGO_SETTING_FOLDER.wsgi:application
Then create an HTTP proxy using nginx that passes HTTP requests from clients through the gunicon-created socket:
/etc/nginx/sites-enabled/example.com:
server {
listen 80;
listen [::]:80;
server_name example.com;
# serve static files directly through nginx
location /static/ {
autoindex off;
root /path/to/django/root;
}
# serve user-uploaded files directly through nginx
location /media/ {
autoindex off;
root /path/to/django/root;
}
# You can do fun stuff like aliasing files from other folders
location /robots.txt {
alias /path/to/django/root/static/robots.txt;
}
# here is the proxy magic
location / {
include proxy_params;
proxy_pass http://unix:/run/example.com.sock; # <- the socket!
}
}
Make sure to restart nginx:
sudo service restart nginx
After all this, your csrf tokens should match the domain name of your site and you'll be able to log in and submit forms.
I'm running a Django web application using Nginx and uWSGI. I'm having problems with the requests hanging for no apparent reason.
I have added a bunch of logging in the application, and this snippet is where it seems to hang. There are two log lines at the start of the try block, and the first one gets printed, but not he second one, so it would seem that it hangs in the middle of the code. This code is from a middleware class that I added in the Django configuration.
def process_request(self, request):
if 'auth' not in request.session:
try:
log.info("Auth not found") # this line is logged
log.info("another log line") # this line is never logged
if request.is_ajax():
return HttpResponse(status=401)
...
I managed to get a backtrace from the uWSGI thread and this is where it's stuck:
*** backtrace of 76 ***
/usr/bin/uwsgi(uwsgi_backtrace+0x2e) [0x45121e]
/usr/bin/uwsgi(what_i_am_doing+0x30) [0x451350]
/lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f8a4b2b8c30]
/lib/x86_64-linux-gnu/libc.so.6(epoll_wait+0x33) [0x7f8a4b37d653]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(+0x27625) [0x7f8a44092625]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(ev_run+0x29b) [0x7f8a4409d11b]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/gevent/core.so(+0x32bc0) [0x7f8a4409dbc0]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4bd4) [0x7f8a4a0c30d4]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x80d) [0x7f8a4a0c517d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0x162310) [0x7f8a4a0c5310]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7f8a4a08ce23]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0x7d30d) [0x7f8a49fe030d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7f8a4a08ce23]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7f8a4a04b837]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/greenlet.so(+0x375c) [0x7f8a49b1c75c]
/home/vdr/vdr-ui/env/local/lib/python2.7/site-packages/greenlet.so(+0x30a6) [0x7f8a49b1c0a6]
[0x7f8a42f26f38]
*** end of backtrace ***
SIGUSR2: --- uWSGI worker 3 (pid: 76) is managing request /login?next=/&token=45092ca6-c1a0-4c23-9d44-4d171fc561b8 since Wed Dec 2 09:52:44 2015 ---
The Nginx error log prints out [error] 619#0: *55 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: vdr
There are no errors in the printouts from uWSGI, so I'm a bit at a loss. Has anyone seen anything similar? All this is running within a Docker container if that makes any difference.
Nginx conf:
upstream uwsgi {
server unix:///tmp/vdr.sock;
}
server {
listen 80;
charset utf-8;
client_max_body_size 500M;
server_name localhost 172.17.0.2;
location /static {
alias /home/vdr/vdr-ui/static;
}
location / {
include uwsgi_params;
uwsgi_pass uwsgi;
uwsgi_read_timeout 200s;
}
}
uWSGI conf:
[uwsgi]
chdir = %d
module = alft_ui.wsgi:application
uid=1000
master=true
pidfile=/tmp/vdr.pid
vacuum=true
max-requests=5000
processes=4
env=DJANGO_SETTINGS_MODULE=alft_ui.settings.prod-live
home=/home/vdr/vdr-ui/env
socket=/tmp/vdr.sock
chmod-socket=666
So I finally found the cause for this. It turns out that my setup script added some logstash settings to the Django configuration. These settings pointed to the IP 10.8.0.1 which wasn't reachable from this environment. This would explain why the app got stuck on a logging line. Removing these settings made everything work again.
Always good to know that it was your own fault all along :)
Preface: I'm running nginx + gunicorn + django on an amazon ec2 instance using s3boto as a default storage backend. I am free tier. The ec2 security group allows: http, ssh, & https.
I'm attempting to send a multipart/form-data request containing a single element: a photo. When attempting to upload the photo, the iPhone (where the request is coming from) hangs. The photo is around 9.5 MB in size.
When I check the nginx-access.logs:
"POST /myUrl/ HTTP/1.1" 400 5 "-""....
When I check the nginx-error.logs:
[error] 5562#0: *1 readv() failed (104: Connection reset by peer) while reading upstream, client: my.ip.addr.iphone, server: default, request: "POST /myUrl/ HTTP/1.1", upstream: "http://127.0.0.1:8000/myUrl/", host: "ec2-my-server-ip-addr.the-location-2.compute.amazonaws.com"
[info] 5562#0: *1 client my.ip.addr.iphone closed keepalive connection
I really cannot figure out why this is happening... I have tried changing the /etc/nginx/sites-available/default timeout settings...
server { ...
client_max_body_size 20M;
client_body_buffer_size 20M;
location / {
keepalive_timeout 300;
proxy_read_timeout 300;
}
}
Any thoughts?
EDIT: After talking on IRC a little more, his problem is the 403 itself, not the nginx error. Leaving my comments on the nginx error below, in case anyone else stumbles into it someday.
I ran into this very problem last week and spent quite a while trying to figure out what was going on. See here: https://github.com/benoitc/gunicorn/issues/872
Basically, as soon as django sees the headers, it knows that the request isn't authenticated. It doesn't wait for the large request body to finish uploading; it responds immediately, and gunicorn closes the connection right after. nginx keeps sending data, and the end result is that gunicorn sends a RST packet to nginx. Once this happens, nginx cannot recover and instead of sending the actual response from gunicorn/django, it sends a 502 Bad Gateway.
I ended up putting in a piece of middleware that acecsses a couple fields in the django request, which ensures that the entire request body is downloaded before Django sends a response:
checker = re.compile(feed_url_regexp)
class AccessPostBodyMiddleware:
def process_request(self, request):
if checker.match(request.path.lstrip('/')) is not None:
# just need to access the request info here
# not sure which one of these actually does the trick.
# This will download the entire request,
# fixing this random issue between gunicorn and nginx
_ = request.POST
_ = request.REQUEST
_ = request.body
return None
However, I do not have control of the client. Since you do (in the form of your iphone app), maybe you can find a way to handle the 502 Bad Gateway. That will keep your app from having to send the entire request twice.