My django webapp runs properly using gunicorn, proxied through nginx; but I keep getting error emails: [Django] ERROR (EXTERNAL IP): Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS. Each time the IP changes, and I suspect this is a security issue.
I've only started to notice this issue a week ago, and the only significant change I've made was to set up using Cloudflare as CDN
Report at /
Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS.
Request Method: HEAD
Request URL: http://112.124.42.80/
Django Version: 2.0.6
Python Executable: /xxx/xxxx/xxxx/.virtualenv/bin/python3.6
Python Version: 3.6.8
META:
HTTP_ACCEPT = 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2'
HTTP_ACCEPT_ENCODING = 'gzip'
HTTP_CONNECTION = 'close'
HTTP_HOST = '112.124.42.80'
HTTP_PROXY_CONNECTION = 'keep-alive'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'
HTTP_X_FORWARDED_FOR = '60.191.52.254'
HTTP_X_FORWARDED_PROTO = 'http'
HTTP_X_REAL_IP = '60.191.52.254'
PATH_INFO = '/'
QUERY_STRING = ''
RAW_URI = '/'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '49562'
REQUEST_METHOD = 'HEAD'
SCRIPT_NAME = ''
SERVER_NAME = '0.0.0.0'
SERVER_PORT = '3030'
SERVER_PROTOCOL = 'HTTP/1.0'
SERVER_SOFTWARE = 'gunicorn/19.9.0'
gunicorn.socket = <gevent._socket3.socket object, fd=13, family=2, type=2049, proto=0>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f81a34dccc0>
wsgi.file_wrapper = ''
wsgi.input = <gunicorn.http.body.Body object at 0x7f81a34dc9e8>
wsgi.multiprocess = True
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version = '(1, 0)'
I assume your web app is usually accessed through a fully qualified domain name. It sounds like different IP addresses are pointing to your server, and nginx is routing these through to gunicorn, and so to Django. Django is doing the right thing in blocking them, but it would be better if nginx was not passing them through. You should change your nginx configuration so it only proxies the addresses you expect. You might want to look at the default_server settings.
Related
I am trying to serve a Django 3 build from nginx through wsgi and I cannot seem to get the last stage. I can browse to the Django site by a runserver and launching the uwsgi.ini, but, when I try to browse to the site through nginx I get a 502 error (bad gateway)(firefox). If I try from a remote site it renders the nginx home page.
I have a build of Apache on the same server. I had to spoof an IP and use a unique port to get them running side by side.
the nginx/error.log does not register any problem.
Below is the uwsgi.ini.
[uwsgi]
# variables
projectname = website
base = /opt/website/
# configuration
master = true
http = :8000
uid = nginx
virtualenv = /opt/website/djangoprojectenv
pythonpath = %(base)
chdir = %(base)
env = DJANGO_SETTINGS_MODULE=%(projectname).settings.pro
#module = %(projectname).wsgi:application
module = website.wsgi:application
socket = /tmp/%(projectname).new.sock
chown-socket = %(uid):nginx
chmod-socket = 666
And below is the conf file from nginx/conf.d
server {
listen 192.168.1.220:81;
server_name willdoit.com;
access_log off;
error_log /var/log/nginx_error.log;
location / {
uwsgi_pass unix:/tmp/website.sock;
include /etc/nginx/uwsgi_params;
uwsgi_read_timeout 300s;
client_max_body_size 32m;
}
}
The /tmp/website.sock file is owned by nginx:nginx.
If there are additional details I need to post, please advise. Any help you can provide would be welcome.
The socket was defined incorrectly. Closing.
My setup is ELB --https--> traefik --https--> service
I get back a 500 Internal Server Error from traefik on every request. It doesn't appear the request ever makes it to the service. The service is running Apache with access logging and I see no incoming requests logged. I am able to curl the service directly and receive an expected response. Both traefik and the service are running in Docker containers. I am also able to use port 80 all the way through with success, and I can use https to traefik and port 80 to the service. I get an error from apache, but it does go all the way through.
traefik.toml
logLevel = "DEBUG"
RootCAs = [ "/etc/certs/ca.pem" ]
#InsecureSkipVerify = true
defaultEntryPoints = ["https"]
[entryPoints]
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/etc/certs/cert.pem"
keyFile = "/etc/certs/key.pem"
[entryPoints.http]
address = ":80"
[web]
address = ":8080"
[traefikLog]
[accessLog]
[consulCatalog]
endpoint = "127.0.0.1:8500"
domain = "consul.localhost"
exposedByDefault = false
prefix = "traefik"
The tags used for the consul service:
"traefik.enable=true",
"traefik.protocol=https",
"traefik.frontend.passHostHeader=true",
"traefik.frontend.redirect.entryPoint=https",
"traefik.frontend.entryPoints=https",
"traefik.frontend.rule=Host:hostname"
The debug output from traefik for each request:
time="2018-04-08T02:46:36Z"
level=debug
msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request"
Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Cache-Control":["max-age=0"],"Cookie":["__utmc=80117009; PHPSESSID=64c928bgf265fgqdqqbgdbuqso; _ga=GA1.2.573328135.1514428072; messagesUtk=d353002175524322ac26ff221d1e80a6; __hstc=27968611.cbdd9ce39324304b461d515d0a8f4cb0.1523037648547.1523037648547.1523037648547.1; __hssrc=1; hubspotutk=cbdd9ce39324304b461d515d0a8f4cb0; __utmz=80117009.1523037658.5.2.utmcsr=|utmccn=(referral)|utmcmd=referral|utmcct=/; __utma=80117009.573328135.1514428072.1523037658.1523128344.6"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.81 Safari/537.36"],"X-Amzn-Trace-Id":["Root=1-5ac982a8-b9615451a35258e3fd2a825d"],"X-Forwarded-For":["76.105.255.147"],"X-Forwarded-Port":["443"],"X-Forwarded-Proto":["https"]},"ContentLength":0,"TransferEncoding":null,"Host”:”hostname”,”Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.200.20.130:4880","RequestURI":"/","TLS":null}"
time="2018-04-08T02:46:36Z" level=debug
msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL"
Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Cache-Control":["max-age=0"],"Cookie":["__utmc=80117009; PHPSESSID=64c928bgf265fgqdqqbgdbuqso; _ga=GA1.2.573328135.1514428072; messagesUtk=d353002175524322ac26ff221d1e80a6; __hstc=27968611.cbdd9ce39324304b461d515d0a8f4cb0.1523037648547.1523037648547.1523037648547.1; __hssrc=1; hubspotutk=cbdd9ce39324304b461d515d0a8f4cb0; __utmz=80117009.1523037658.5.2.utmcsr=|utmccn=(referral)|utmcmd=referral|utmcct=/; __utma=80117009.573328135.1514428072.1523037658.1523128344.6"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.81 Safari/537.36"],"X-Amzn-Trace-Id":["Root=1-5ac982a8-b9615451a35258e3fd2a825d"],"X-Forwarded-For":["76.105.255.147"],"X-Forwarded-Port":["443"],"X-Forwarded-Proto":["https"]},"ContentLength":0,"TransferEncoding":null,"Host”:”hostname”,”Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"10.200.20.130:4880","RequestURI":"/","TLS":null}" ForwardURL="https://10.200.115.53:443"
assume "hostname" is the correct host name. Any assistance is appreciated.
I think your problem come from "traefik.protocol=https", remove this tag.
Also you can remove traefik.frontend.entryPoints=https because it's useless: this tag create a redirection to https entrypoint but your frontend is already on the https entry point ("traefik.frontend.entryPoints=https")
Firstly, I am aware of the solution presented here: JIRA REST API to get work log - "You do not have the permission to see the specified issue" ... As well the linked blog post (penned by #Nick Josevski), which whilst useful, doesn't address my specifc problem, which could be something trivial...
With following Python 2.7.3 code...
import requests
import getpass
import json
jira_user = raw_input("Username: ")
jira_pass = getpass.getpass()
session = requests.Session()
session.verify = jira_ca_certs # Our internal certs
auth_info = {"username": jira_user, "password": jira_pass}
login_url = 'http://JIRA_SERVER.com/login.jsp'
session.post(login_url, data=auth_info)
I generate the cookies post basic authenticatation to jira (note: I am using "http" without specifying the PORT to authenticate with the login page).. As session automatically holds the returned cookies, I can use session.cookies to set the header:
cookies = requests.utils.dict_from_cookiejar(session.cookies)
headers = {'Content-type': 'application/json', 'cookie': cookies}
Following which, I test the captured cookies with a basic get to the secure JIRA url, using https + PORT:
base = session.get('https://JIRA_SERVER.com:1234', headers=headers)
print 'base: ', base
The above, as expected, returns (though this might not be a valid test?) ...
base: <Response [200]>
Now to test the code for it's intended purpose. I extend the url for a specific JIRA issue, using the same approach:
jira = session.get('https://JIRA_SERVER.com:1234/rest/api/latest/issue/KRYP-6207', headers=headers)
print 'issue: ', jira
print jira.json()
With JSON output, I get a response stating I do not have the permission:
issue: <Response [401]>
{u'errorMessages': [u'You do not have the permission to see the specified issue.', u'Login Required'], u'errors': {}}
The cookies returned, that I use in the header, are:
headers: {'cookie': 'atlassian.xsrf.token=XXXXXXXXXXXXXXXX|lout; Path=/, JSESSIONID=XXXXXXXXXXXXXXXX; Path=/'}
I don't know why this works for the base url, but not the issue url.. I have used Chrome POSTMAN to check the cookies being returned, and they are the same as those listed above i.e. atlassian.xsrf.token, and JSESSIONID.
Hoping someone here can tell me what I am doing wrong! Thanks in advance ...
Maybe you are not correctly logged in? based on the requests I saw in webbrowser developer tools, I created following requests and it worked with our JIRA:
import requests
import getpass
import json
jira_user = raw_input("Username: ")
jira_pass = getpass.getpass()
session = requests.Session()
headers = {
'Host': 'our.jira.com',
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
'Referer': 'https://our.jira.com/login.jsp',
'Content-Type': 'application/x-www-form-urlencoded',
'Connection': 'keep-alive',
'Upgrade-Insecure-Requests': '1',
}
data = [
('os_username', jira_user),
('os_password', jira_pass),
('os_destination', ''),
('user_role', ''),
('atl_token', ''),
('login', 'Anmelden'),
]
loginPost = session.post('https://our.jira.com/login.jsp', headers=headers, data=data)
xsrf_token = session.cookies.get_dict()['atlassian.xsrf.token']
jsessionid = session.cookies.get_dict()['JSESSIONID']
cookies = {
'atlassian.xsrf.token': xsrf_token,
'JSESSIONID': jsessionid,
}
headers = {
'Host': 'our.jira.com',
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
'Connection': 'keep-alive',
'Upgrade-Insecure-Requests': '1',
}
r = session.get('https://our.jira.com/rest/api/latest/issue/SDN-206', headers=headers, cookies=cookies)
print r.json()
I'm under Apache + fastCGI with Django 1.8
My folder on the shared host looks like:
/home/username/
/home/username/MyProject_SRC
/home/username/MyProject_SRC/MyProject/manage.py
and:
/home/username/public_html/.htaccess
/home/username/public_html/index.fcgi
I've edited .htaccess and index.fcgi to look like this:
.htaccess :
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_URI} !=/index.fcgi
RewriteRule ^(.*)$ index.fcgi/$1 [QSA,L]
index.fcgi:
#!/home/username/djangoenv/bin/python3
import sys, os
sys.path.insert(0, "/home/username/MyProject_SRC")
os.chdir("/home/username/MyProject_SRC")
os.environ['DJANGO_SETTINGS_MODULE'] = "MyProject.settings"
from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")
Now, when I do on the server:
(djangoenv)username#hostname [~/public_html]# ./index.fcgi
Status: 200 OK
Vary: Cookie
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
<!DOCTYPE html>
<html lang="en">
.... // Normal webpage
</html>
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
Going on my Home webpage returns a 500 internal error without more in the logs.
EDIT I've got the same result when doing:
(djangoenv)username#hostname [~/MyProject_SRC]# ./manage.py runfcgi
I had a similar problem, in my case I did not find a specific source, but I tried to add this in my index.fcgi and doing ./index.fcgi no longer asked me, if it keeps getting 500 check the version of flup you are using, which be 1.0.2 preferably.
os.environ['REQUEST_METHOD'] = "GET"
os.environ['SERVER_NAME'] = "yourdomain.com"
os.environ['SERVER_PORT'] = "80"
os.environ['SERVER_PROTOCOL'] = "HTTP/1.1"
Since upgrading to django 1.5 my logs show several SuspiciousOperation exceptions with the text:
Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): <my server's ip>
Is this genuinely a 'suspicious' request, or should I always be including my server's IP address in the ALLOWED_HOSTS setting in addition to my domain name? Any idea what would be making requests with HTTP_HOST = "ip address" rather than HTTP_HOST = "domain name"?
Here is the request environment:
{'HTTP_ACCEPT_ENCODING': "'identity'",
'HTTP_CONNECTION': "'close'",
'HTTP_HOST': "'168.62.208.14'",
'HTTP_X_FORWARDED_PROTOCOL': "'https'",
'HTTP_X_REAL_IP': "'176.10.35.241'",
'HTTP_X_SCHEME': "'https'",
'PATH_INFO': "u'/'",
'QUERY_STRING': "''",
'RAW_URI': "'/'",
'REMOTE_ADDR': "'127.0.0.1'",
'REMOTE_PORT': "'45068'",
'REQUEST_METHOD': "'GET'",
'SCRIPT_NAME': "u''",
'SERVER_NAME': "'168.62.208.14'",
'SERVER_PORT': "'80'",
'SERVER_PROTOCOL': "'HTTP/1.0'",
'SERVER_SOFTWARE': "'gunicorn/0.14.6'",
'gunicorn.socket': "'<socket._socketobject object at 0x7ab3b40>'",
'wsgi.errors': '"<open file \'<stderr>\', mode \'w\' at 0x7f0c94810270>"',
'wsgi.file_wrapper': "'<class gunicorn.http.wsgi.FileWrapper at 0x34eec80>'",
'wsgi.input': "'<gunicorn.http.body.Body object at 0x2a0bf10>'",
'wsgi.multiprocess': 'False',
'wsgi.multithread': 'False',
'wsgi.run_once': 'False',
'wsgi.url_scheme': "'http'",
'wsgi.version': '[1, 0]'}
No, it shouldn't
By default, there are no reasons why IP address should be accepted as a valid HOST header. This message is a sign of a misconfigured production environment: such requests shouldn't reach the back-end.
Here's a post on security.stackexchange.com on Host header poisoning & ALLOWED_HOSTS.
What to do
Filter out all requests with an invalid HOST header before they reach django back-end.
How to
Most likely you're using nginx as a reverse proxy in front of django. If you don't use any reverse proxy at all (or you're using runserver), you have to (otherwise you're risking your security).
Add a default server block returning 444 at the top of your configuration. It should be the first server block in the configuration:
# File: /etc/nginx/sites-available/domain.com
upstream django_server {
server 127.0.0.1:8000;
}
# Catch all requests with an invalid HOST header
server {
server_name "";
listen 80;
return 444;
}
# Your config goes there
server {
server_name domain.com;
listen 80;
location / {
proxy_pass http://django_server;
}
}
NO, IT SHOULDN'T.
Usually it's not a secure way to configure your Django server. Sometimes, e.g., when testing your application, you may access it via direct IP address, but in there's no reason to disable log warnings.
My old answer was wrong, thanks to Max Malysh for pointing that out.
Old answer (INSECURE):
Short answer is: YES (according to provided headers).
Long answer:
According to documentation:
If the Host header (or X-Forwarded-Host if USE_X_FORWARDED_HOST is enabled) does not match any value in this list, the django.http.HttpRequest.get_host() method will raise SuspiciousOperation.
In other words: if your requests pass your server ip address as Host header (and apparently they do), and you think it's okay, then YES, you should add server ip to ALLOWED_HOSTS.
Also, ip address could be in HTTP_HOST for many reasons, also someone could directly ask for ip address.
In practice, just edit the file MyProjectName/settings.py and add the host IP (IP address to the machine in which you're running Django) to the list ALLOWED_HOSTS, which by default is empty.
So, in your, case we would have before the changes:
...
ALLOWED_HOSTS = []
...
After changes:
...
ALLOWED_HOSTS = ['168.62.208.14'] #Make sure your host IP is a string
...
Run the server again and you should be good. Here's an example using the port 8000:
python manage.py runserver 168.62.208.14:8000
. Now if you go to your browser and enter the address http://168.62.208.14:8000, you should find yourself in the page "Congratulations on your first Django-powered page".