I'm all over the ALLOWED_HOSTS setting in Django. It has been set on all my production sites for a while now but one site in particular has been throwing out weird debug emails. The site works fine (for me) but about once a week I'll get an email along the lines of:
SuspiciousOperation: Invalid HTTP_HOST header
(you may need to set ALLOWED_HOSTS): thepcspy.com.
<WSGIRequest
path:/,
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{},
META:{...
'HTTP_HOST': 'thepcspy.com.',
'SERVER_NAME': 'thepcspy.com',
...
Notice the full stop after the domain in the error and in HTTP_HOST. As far as I can see in my nginx config, there isn't anything that could add an extra dot after the name on HTTP_HOST (note that SERVER_NAME is correct). What on earth is going on here?
Should I just write this off as somebody intentionally trying to break my server?
The final dot makes it a fully qualified domain name (FQDN). This issue is noted precisely in the Django docs on the ALLOWED_HOSTS setting: https://docs.djangoproject.com/en/stable/ref/settings/#allowed-hosts
If you want to also allow the fully qualified domain name (FQDN), which some browsers can send in the Host header, you must explicitly add another ALLOWED_HOSTS entry that includes a trailing period.
Related
My original question was how to enable HTTPS for a Django login page, and the only response, recommended that I - make the entire site as HTTPS-only.
Given that I'm using Django 1.3 and nginx, what's the correct way to make a site HTTPS-only?
The one response mentioned a middleware solution, but had the caveat:
Django can't perform a SSL redirect while maintaining POST data.
Please structure your views so that redirects only occur during GETs.
A question on Server Fault about nginx rewriting to https, also mentioned problems with POSTs losing data, and I'm not familiar enough with nginx to determine how well the solution works.
And EFF's recommendation to go HTTPS-only, notes that:
The application must set the Secure attribute on the cookie when
setting it. This attribute instructs the browser to send the cookie
only over secure (HTTPS) transport, never insecure (HTTP).
Do apps like Django-auth have the ability to set cookies as Secure? Or do I have to write more middleware?
So, what is the best way to configure the combination of Django/nginx to implement HTTPS-only, in terms of:
security
preservation of POST data
cookies handled properly
interaction with other Django apps (such as Django-auth), works properly
any other issues I'm not aware of :)
Edit - another issue I just discovered, while testing multiple browsers. Say I have the URL https://mysite.com/search/, which has a search form/button. I click the button, process the form in Django as usual, and do a Django HttpResponseRedirect to http://mysite.com/search?results="foo". Nginx redirects that to https://mysite.com/search?results="foo", as desired.
However - Opera has a visible flash when the redirection happens. And it happens every search, even for the same search term (I guess https really doesn't cache :) Worse, when I test it in IE, I first get the message:
You are about to be redirected to a connection that is not secure - continue?
After clicking "yes", this is immediately followed by:
You are about to view pages over a secure connection - continue?
Although the second IE warning has an option to turn it off - the first warning does not, so every time someone does a search and gets redirected to a results page, they get at least one warning message.
For the 2nd part of John C's answer, and Django 1.4+...
Instead of extending HttpResponseRedirect, you can change the request.scheme to https.
Because Django is behind Nginx's reverse proxy, it doesn't know the original request was secure.
In your Django settings, set the SECURE_PROXY_SSL_HEADER setting:
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
Then, you need Nginx to set the custom header in the reverse proxy. In the Nginx site settings:
location / {
# ...
proxy_set_header X-Forwarded-Proto $scheme;
}
This way request.scheme == 'https' and request.is_secure() returns True.
request.build_absolute_uri() returns https://... and so on...
Here is the solution I've worked out so far. There are two parts, configuring nginx, and writing code for Django. The nginx part handles external requests, redirecting http pages to https, and the Django code handles internal URL generation that has an http prefix. (At least, those resulting from a HttpResponseRedirect()). Combined, it seems to work well - as far as I can tell, the client browser never sees an http page that the users didn't type in themselves.
Part one, nginx configuration
# nginx.conf
# Redirects any requests on port 80 (http) to https:
server {
listen 80;
server_name www.mysite.com mysite.com;
rewrite ^ https://mysite.com$request_uri? permanent;
# rewrite ^ https://mysite.com$uri permanent; # also works
}
# django pass-thru via uWSGI, only from https requests:
server {
listen 443;
ssl on;
ssl_certificate /etc/ssl/certs/mysite.com.chain.crt;
ssl_certificate_key /etc/ssl/private/mysite.com.key;
server_name mysite.com;
location / {
uwsgi_pass 127.0.0.1:8088;
include uwsgi_params;
}
}
Part two A, various secure cookie settings, from settings.py
SERVER_TYPE = "DEV"
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True # currently only in Dev branch of Django.
SESSION_EXPIRE_AT_BROWSER_CLOSE = True
Part two B, Django code
# mysite.utilities.decorators.py
import settings
def HTTPS_Response(request, URL):
if settings.SERVER_TYPE == "DEV":
new_URL = URL
else:
absolute_URL = request.build_absolute_uri(URL)
new_URL = "https%s" % absolute_URL[4:]
return HttpResponseRedirect(new_URL)
# views.py
def show_items(request):
if request.method == 'POST':
newURL = handle_post(request)
return HTTPS_Response(request, newURL) # replaces HttpResponseRedirect()
else: # request.method == 'GET'
theForm = handle_get(request)
csrfContext = RequestContext(request, {'theForm': theForm,})
return render_to_response('item-search.html', csrfContext)
def handle_post(request):
URL = reverse('item-found') # name of view in urls.py
item = request.REQUEST.get('item')
full_URL = '%s?item=%s' % (URL, item)
return full_URL
Note that it is possible to re-write HTTPS_Response() as a decorator. The advantage would be - not having to go through all your code and replace HttpResponseRedirect(). The disadvantage - you'd have to put the decorator in front of HttpResponseRedirect(), which is in Django at django.http.__init__.py. I didn't want to modify Django's code, but that's up to you - it's certainly one option.
if you stick your entire site behind https, you don't need to worry about it on the django end. (assuming you don't need to protect your data between nginx and django, only between users and your server)
In my Django's settings.py I have
SESSION_COOKIE_HTTPONLY = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_SSL_REDIRECT = True
SECURE_HSTS_SECONDS = 15768000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SESSION_COOKIE_AGE = 2 * 24 * 3600
However https://detectify.com has found that this flag isn't set for csrftoken cookie. I checked what Chrome tells about the cookie, and if I understand correctly, the empty HTTP column confirms that the two cookies are not HTTP-only:
Also, if I do document.cookie in chrome's console, the csrftoken value is shown.
I wonder why this could be the case. I have Django running on uwsgi and nginx. The nginx configuration is as follows, and the site in question is https://rodichi.net:
server {
listen 443 ssl http2 default_server;
server_name rodichi.net;
ssl_certificate /etc/letsencrypt/live/rodichi.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/rodichi.net/privkey.pem;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
charset utf-8;
... # location settings follow here
```
You have only configured it to have the CSRF token be set to Secure (i.e. only sent over https requests) and not to be HttpOnly (i.e. not available to Javascript).
Looking at the Django documentation you also need to set CSRF_COOKIE_HTTPONLY. However the documentation rightly points out:
Designating the CSRF cookie as HttpOnly doesn’t offer any practical
protection because CSRF is only to protect against cross-domain
attacks. If an attacker can read the cookie via JavaScript, they’re
already on the same domain as far as the browser knows, so they can do
anything they like anyway. (XSS is a much bigger hole than CSRF.)
Although the setting offers little practical benefit, it’s sometimes
required by security auditors.
It also depends how you have implemented CSRF. There are basically two methods for forms:
Set a hidden CSRF field for each form and make this field generate a unique value each time the form is loaded. Therefore if the form submission includes a valid code, then you know the request came from your domain. This is complicated on the server side, as it requires keeping track of valid tokens and also means each form must be dynamically generated to include a random token, but is easier on client side as uses standard form requests rather than JavaScript. For this protection the CSRF cookie is not needed and is not used even if it is present.
The other method invokes setting a CSRF cookie, and having the Javascript read this and send it in a HTTP header (usually X-CSRF-TOKEN). A CSRF request from another domain won't have access to this CSRF cookie so won't be able to set the header correctly. As the cookies will be also be sent on all requests it's easy for the server to check the cookie in the HTTP Request matches the header set in the request. Which means the request came from a somewhere that has access to the cookies, which means it came from same domain. Which means it's not a CSRF attack. This is easier to implement on the server side (as no need to keep a list of active Tokens) but requires Javascript at the front end and requires a CSRF token not to be HttpOnly - precisely because the Token is supposed to be read by Client side Javascript!
Again the Django documentation warns against this:
If you enable this and need to send the value of the CSRF token with
an AJAX request, your JavaScript must pull the value from a hidden
CSRF token form input on the page instead of from the cookie.
So, all in all, it is not recommended to set the HttpOnly attribute for this cookie. It limits you, adds no real protection, and makes the cookie itself meaningless.
You will get it highlighted in and Pen Test reports on your site (including https://detectify.com by the looks of things) but should accept that as you are comfortable that this is correct. Not sure if it's possible to whitelist this cookie in https://detectify.com so it doesn't alert each time?
I am developing a ExtJS application that uses a Django-rest-framework service. I am using CORS headers to allow fetching the data from the service (https://github.com/OttoYiu/django-cors-headers).
What happens is that at a point in time I want to change the URL from the store. And when I do that I get the following error:
XMLHttpRequest cannot load http://10.98.0.241:8000/reacsearch/as?_dc=1418831884352&page=1&start=0&limit=25. The request was redirected to 'http://10.98.0.241:8000/reacsearch/as/?_dc=1418831884352&page=1&start=0&limit=25', which is disallowed for cross-origin requests that require preflight.
In the settings.oy I define the following properties for the CORS
CORS_ALLOW_METHODS = (
'GET',
'OPTIONS'
)
CORS_ORIGIN_ALLOW_ALL = True
This works fine when I use URLs to list all the elements in my database, however when I change the store for another URL I get the error above. Also the link works fine in the browser.
The store url change is made this way:
var store = Ext.getStore(storeName);
store.getProxy().setUrl(newURL);
store.load();
The difference between the views, is that the two that work on the application are viewsets, while the other is just a generic list
class Example1viewset(viewsets.ModelViewSet):
"""
API endpoing that allows metabolites to be viewed.
"""
queryset = examples1.objects.all()
serializer_class = Example1Serializer
class Example1SearchList(generics.ListAPIView):
serializer_class = Example1Serializer
def get_queryset(self):
queryset = Example.objects.all()
if 'attr' in self.kwargs:
queryset = queryset.filter(Q(attribute1__contains=self.kwargs['attr']) | Q(attribute2__contains=self.kwargs['abbr']))
return queryset
Like I mentioned both examples work fine in the browser (even accessing through other computers in the network), however in the application when changing the URL of the store I get the CORS error. Does anyone has any idea why this is happening?
Thank you.
Edit:
Just for clarification, the problem is not in changing the url of the store. As I tried to set those urls as defaults, but they are not working when accessing from the application.
My urls.py file:
router = routers.DefaultRouter()
router.register(r'example', views.Example1ViewSet)
# Wire up our API using automatic URL routing.
# Additionally, we include login URLs for the browsable API.
urlpatterns = [
url(r'^', include(router.urls)),
url(r'^reacsearch/(?P<attr>.+)/$', Example1SearchList.as_view()),
url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework'))
Can it be that the problem is related with the fact that I am not adding the search list to the router?
Edit2
Problem solved since I was trying to fetch data from a different domain. I changed the type of store to jsonp in Extjs, and I also allowed my rest service to render data as jsonp.
Just a reminder if anyone comes accross this same problem, it is necessary to add ?format=jsonp to the store url:
http://my/url/?format=jsonp
Since it looks like an alternate solution was found, I'll explain what the issue appeared to be as well as why the alternative works.
XMLHttpRequest cannot load first url. The request was redirected to 'second url', which is disallowed for cross-origin requests that require preflight.
The issue here is that you are telling Django to enforce the trailing slash, which makes it automatically redirect urls without a trailing slash to urls with a trailing slash, assuming that one exists. This is why, as stated in the error, the request was redirected to the second url, which you can tell has the missing trailing slash. This is controlled by the APPEND_SLASH Django setting which is True by default.
The problem is that when CORS is doing a preflight request, which is what allows it to determine if the request can be made, there must be a valid response at the requested URL. Because you are redirecting the request, the preflight request fails and you're stuck without your information.
You can fix this by adding the trailing slash in your code. There appear to be a few solutions for doing this with ext, but I personally can't recommend a specific one. You can also manually set the url to use the trailing slash, which sounds like what you were doing previously.
Or you can use JSONP...
You've found the alternative solution, which is to use JSONP to make the request instead of relying on CORS. This gets around the preflight issue and works in all major browsers, but there are some drawbacks to consider. You can find more information on CORS vs JSONP by looking around.
You're going to need CORS if you want to push any changes to your API, as JSONP only supports GET requests. There are other advantages, such as the ability to abort requests, that also comes with CORS.
Using the Django-auth application (Django version 1.3), I want to have my login page go to https://mysite.com/login/. Currently, I'm using:
# urls.py
from django.contrib.auth.views import login
urlpatterns = patterns('', url(r'^login/$', login, name='login-view'),)
# navbar.html
<li id="nav-login"><a href="{% url login-view %}" ><b>Login</b></a></li>
which works nicely, but goes to http://mysite.com/login/.
Is there some way to tell Django-auth what prefix (https) to use, when it reverses the view name? I've read the entire manual page, and haven't found anything that covers it. Or maybe some way to tell the url tag to go to https?
Or is the only option to specify the entire URL manually? I hope not :) And given how powerful Django has been so far, I can't believe it wouldn't have that ability - I must be overlooking it. :)
Set OS environmental variable HTTPS to on
You need to enable the OS environmental variable HTTPS to 'on' so django will prepend https to fully generated links (e.g., like with HttpRedirectRequests). If you are using mod_wsgi, you can add the line:
os.environ['HTTPS'] = "on"
to your wsgi script. You can see the need for this by reading django/http/__init__.py:
def build_absolute_uri(self, location=None):
"""
Builds an absolute URI from the location and the variables available in
this request. If no location is specified, the absolute URI is built on
``request.get_full_path()``.
"""
if not location:
location = self.get_full_path()
if not absolute_http_url_re.match(location):
current_uri = '%s://%s%s' % (self.is_secure() and 'https' or 'http',
self.get_host(), self.path)
location = urljoin(current_uri, location)
return iri_to_uri(location)
def is_secure(self):
return os.environ.get("HTTPS") == "on"
Secure your cookies
In settings.py put the lines
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
and cookies will only be sent via HTTPS connections. Additionally, you probably also want SESSION_EXPIRE_AT_BROWSER_CLOSE=True. Note if you are using older versions of django (less than 1.4), there isn't a setting for secure CSRF cookies. As a quick fix, you can just have CSRF cookie be secure when the session cookie is secure (SESSION_COOKIE_SECURE=True), by editing django/middleware/csrf.py:
class CsrfViewMiddleware(object):
...
def process_response(self, request, response):
...
response.set_cookie(settings.CSRF_COOKIE_NAME,
request.META["CSRF_COOKIE"], max_age = 60 * 60 * 24 * 7 * 52,
domain=settings.CSRF_COOKIE_DOMAIN,
secure=settings.SESSION_COOKIE_SECURE or None)
Direct HTTP requests to HTTPS in the webserver
Next you want a rewrite rule that redirects http requests to https, e.g., in nginx
server {
listen 80;
rewrite ^(.*) https://$host$1 permanent;
}
Django's reverse function and url template tags only return relative links; so if you are on an https page your links will keep you on the https site.
As seen in other StackOverflow questions, you could implement middleware that would automatically redirect the login page to a secure version.
If you are really serious about security, you should probably migrate the entire website to SSL. From the EFF's How to Deploy HTTPS Correctly:
You must serve the entire application domain over HTTPS. Redirect HTTP requests with HTTP 301 or 302 responses to the equivalent HTTPS resource.
Some site operators provide only the login page over HTTPS, on the theory that only the user’s password is sensitive. These sites’ users are vulnerable to passive and active attack.
I have a django project running on my localhost and it is working very well, however when I uploaded it to real server, some problem started happening with the url. it happens every time HttpResponseRedirect or any redirect gets called
a page on my local host
http://127.0.0.1:8000/signin
while on the server it becomes
http://xyz.com,%20xyz.com/signin
in firebug i see
GET signin 301 MOVED PERMANENTLY
GET signin http://xyz.com,%20xyz.com/signin
I belive this happens because the urls.py has ^signin/$^ and APPEND_SLASH = True in settings.py because when I visit /signin/ it works!
404 page on my local host
Request URL: http://127.0.0.1:8000/test
on the server
Request URL: http://xyz.com,%20xyz.com/test
for some reason it is adding [comma][space] to url and redirects it.
home page is working without issues
The issue is tracked in the following ticket:
https://code.djangoproject.com/ticket/11877
It has to do with how Django handles proxy redirection. The following middleware will help you out.
class MultipleProxyMiddleware(object):
FORWARDED_FOR_FIELDS = [
'HTTP_X_FORWARDED_FOR',
'HTTP_X_FORWARDED_HOST',
'HTTP_X_FORWARDED_SERVER',
]
def process_request(self, request):
"""
Rewrites the proxy headers so that only the most
recent proxy is used.
"""
for field in self.FORWARDED_FOR_FIELDS:
if field in request.META:
if ',' in request.META[field]:
parts = request.META[field].split(',')
request.META[field] = parts[-1].strip()
If, for example, your Django site is sitting behind a proxy which includes proxy information in the X-Forwarded-For header, and then your web server also does proxying, the header will contain a list (comma separated) of the proxied addresses. By using this middleware, it will strip all but one of the proxied addresses in the headers.
It might not be an answer since I'm working with you on the same application, I fixed it.
It has something to do with nginx to apache redirection, we had proxy_set_header Host $host; and when I disabled it the redirection worked without errors.