I have a Django Rest Framework running at: https://dev-example.domain.com in kubernetes behind a kubernetes Ingress with http traffic disabled.
Note this is not NGINX nor Traefik. It's an Ingress controller setup on GCP.
This concept is explained here
Therefore, trying to go to http://dev-example.domain.com returns a 404. Rightfully.
In the Browsable api, however, all links are prefixed with http:://
Therefore, when one of these links is clicked on, the redirect returns a 404.
Is there a setting that will allow that prefix to be https?
This means django doesnt know your request was over HTTPS, There are several methods you can use to tell it you are using HTTPS.
For example Django will respect the X-Forwarded-Proto Header so have Nginx or whatever you have in front of Django forward that to your application.
i.e
proxy_set_header X-Forwarded-Proto $scheme;
Related
My server runs Django + Gunicorn + nginx.
I have added an SSL certificate and configured nginx to redirect http to https. When an https request is received, nginx passes it to Gunicorn as http.
My program sometimes returns HttpResponseRedirect, and the browser gets a redirect response and re-requests as http, so nginx redirects to https.
How can I avoid this? How can I configure the server so that the first redirection points directly to an https URL?
In the nginx configuration (inside the location block), specify this:
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
The proxy_redirect tells nginx that, if the backend returns an HTTP redirect, it should leave it as is. By default, nginx assumes the backend is stupid and tries to be smart; if the backend returns an HTTP redirect that says "redirect to http://localhost:8000/somewhere", nginx replaces it with something similar to http://yourowndomain.com/somewhere". But Django isn't stupid (or it can be configured to not be stupid).
Django does not know whether the request has been made through HTTPS or plain HTTP; nginx knows that, but the request it subsequently makes to the Django backend is always plain HTTP. We tell nginx to pass this information with the X-Forwarded-Proto HTTP header, so that related Django functionality such as request.is_secure() works properly. You will also need to set SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') in your settings.py.
I've been working on a django app recently and it is finally ready to get deployed to a qa and production environment. Everything worked perfectly locally, but since adding the complexity of the real world deployment I've had a few issues.
First my tech stack is a bit complicated. For deployments I am using aws for everything with my site deployed on multiple ec2's backed by a load balancer. The load balancer is secured with ssl, but the connections to the load balancer are forwarded to the ec2's over standard http on port 80. After hitting an ec2 on port 80 they are forwarded to a docker container on port 8000 (if you are unfamiliar with docker just consider it to be a standard vm). Inside the container nginx listens on port 8000, it handles a redirection for the static files in django and for web requests it forwards the request to django running on 127.0.0.1:8001. Django is being hosted by uwsgi listening on port 8001.
server {
listen 8000;
server_name localhost;
location /static/ {
alias /home/library/deploy/thelibrary/static/;
}
location / {
proxy_set_header X-Forwarded-Host $host:443;
proxy_pass http://127.0.0.1:8001/;
}
}
I use X-Forwarded host because I was having issues with redirects from google oauth and redirects to prompt the user to login making the browser request the url 127.0.0.1:8001 which will obviously not work. Within my settings.py file I also included
USE_X_FORWARDED_HOST = True
to force django to use the correct host for redirects.
Right now general browsing of the site works perfectly, static files load, redirects work and the site is secured with ssl. The problem however is that CSRF verification fails.
On a form submission I get the following error
Referer checking failed - https://qa-load-balancer.com/projects/new does not match https://qa-load-balancer.com:443/.
I'm really not sure what to do about this, its really through stackoverflow questions that I got everything working so far.
Rather than doing an HTTP proxy, I would use Nginx's built-in capacity to communicate with uWSGI. (This will still work if you are using separate Docker containers for Nginx and uWSGI since the communication is done over TCP)
A typical configuration (mine) looks like this:
location / {
uwsgi_pass http://127.0.0.1:8001;
include uwsgi_params;
}
You will have to remove the --http argument (or config-file equivalent) from your uWSGI invocation.
Additionally, in uwsgi_params (found in /etc/nginx or a custom location you specify) there are several directives to pass meta data through. Here's an excerpt from mine that looks like it could be related to your problem:
...
uwsgi_param REQUEST_URI $request_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param HTTPS $https if_not_empty;
Relevant docs: http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html#putting-behind-a-full-webserver
For users who cannot use Nginx's built-in facility, here's the root cause:
Starting in ~Djagno 1.9, the CSRF check requires that the Referer and Host match unless you specify a CSRF_TRUSTED_ORIGINS (see the code around REASON_BAD_REFERER here)
If you don't specify CSRF_TRUSTED_ORIGINS, the system falls back on request.get_host()
request.get_host() uses request._get_raw_host()
request._get_raw_host() checks sequentially HTTP_X_FORWARDED_HOST (if USE_X_FORWARDED_HOST is set), HTTP_HOST, and SERVER_NAME
Most recommended Nginx configurations suggest an entry like proxy_set_header X-Forwarded-Host $host:$server_port;
Eventually, the referrer (e.g. <host>) is compared to X-Forwarded-Host (e.g. <host>:<port>). These do not match so CSRF fails.
There isn't a lot of discussion about this, but Django ticket #26037 references RFC2616. The ticket states that a host without a port is "against spec", but that's not true as the spec actually says:
A "host" without any trailing port information implies the default port for the service requested
This leads to (at minimum) the following options (safest first):
include host and port in CSRF_TRUSTED_ORIGINS
remove port from X-Forwarded-Host in nginx configuration (on the assumption that the non-spec X-Forwarded-Host follows the same semantics as Host)
To avoid hard-coding domains in CSRF_TRUSTED_ORIGINS, the second option is attractive, but it may come with security caveats. Speculatively:
X-Forwarded-Proto should be used to clarify the protocol (since the absence of a port implies a default protocol)
The reverse proxy MUST use port 443 for HTTPS (i.e. the default for the protocol) and disallow non-HTTPS connection types (X-Forwarded-Proto might fix this).
I had the same issue running a Django project on GitPod: the X-Forwarded-Host was in the form hostname:443, causing the CSRF error.
I solved it with a custom middleware that strips the port from the header:
# myproject/middleware.py
from django.utils.deprecation import MiddlewareMixin
class FixForwardedHostMiddleware(MiddlewareMixin):
def process_request(self, request):
forwarded_host = request.META.get('HTTP_X_FORWARDED_HOST')
if forwarded_host:
forwarded_host = forwarded_host.split(':')[0]
request.META['HTTP_X_FORWARDED_HOST'] = forwarded_host
To use this middleware, you need to edit your settings.py, and insert the new middleware before the CSRF one, like so:
# myproject/settings.py
MIDDLEWARE = [
...
'myproject.middleware.FixForwardedHostMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
...
]
See clayton's answer to understand why this fixes the CSRF error.
I don't think this middleware introduces any security issue; please comment if you think otherwise.
Intro
Cloudflare's providing SSL for free now, and I would be a fool to not take advantage of this on my site, and a downright dickhead to break everything in the process of trying to.
I can code apps just fine, but when it comes to setting up or configuring https/nginx/gunicorn/etc/idon'tknowtheterminology, I know barely enough to follow Googled instructions.
Question
I would like to use django-sslify to force https on my Django web app. How may I achieve this without upsetting the balance in my life, given the following known facts?
Known facts
I'm using Django 1.7, running on a DigitalOcean server hooked up
to a (free) Cloudflare DNS. Django is fitted (served?) with nginx
and gunicorn. Basically followed this guide to get it all set up.
Accessing my website currently defaults to a regular http://example.com header.
Manually accessing https://example.com works with
the green lock and all, but this breaks all form submissions with
the error "(403) CSRF verification failed. Request aborted.".
In my Cloudflare site settings, the domain is currently configured to "Flexible SSL".
Trying to use django-sslify with my existing setup totally breaks everything, and the browser is unable to return a response.
This info nugget tells me that I should use the "Full SSL" configuration setting when using django-sslify with Cloudflare's SSL.
Cause for hesitation found here where it is mentioned that a $20/mo Pro Cloudflare account is needed to handle SSL termination. So I really don't want to screw this up :/
There was only 1 mention of "http" or "https" anywhere in my nginx and gunicorn configuration, specifically in my nginx config:
location / {
proxy_pass http://127.0.0.1:8001; ... }
Ok I think that's all I have
Also, my server is providing an Django Rest Framework api for a Phonegap app, does that need to be taken in to consideration? If I need to provide addtional information do let me know and I'll get back to you. Thank you for taking a look at this! :)
CloudFlare allows you to enable specific page rules, one of which is to force SSL (by doing a hard redirect). This is a great thing to use in addition to django-sslify or django-secure
In addition to setting up your SSL redirect, you also need to tell Django to handle secure requests. Luckily, Django provides a decent guide for doing this, but there are a few things that it doesn't mention but I've had to do with nginx.
In your Django settings, you need to tell Django how to detect a secure request
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTOCOL', 'https')
In your nginx configuration you need to set up the X-Forwarded-Protocol header (and the X-Forwarded-For/X-Scheme headers are also useful).
proxy_set_header X-Scheme $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
You also need to proxy the Host header down, so Django is able to read the correct host and port, which is used in generating absolute urls and CSRF, among other things.
proxy_set_header Host $http_host;
Note that I used the $http_host variable instead of $host or $host:$server_port. This will ensure that Django will still respect CSRF requests on non-standard ports, while still giving you the correct absolute urls.
As with most things related to nginx and gunicorn, YMMV and it gets easier after you do it a few times.
Using django-rest-framework. I'm getting HTTP 403 errors when running in production behind nginx. When I call a particular view, which inherits from APIView to support a GET operation, I get:
{"detail": "Invalid username/password"}
But... I only get this in a browser. I don't get it when I use curl for the very same URL. I get this error whether I hit the URL directly, or load the URL via AJAX, in both Chrome and Firefox.
I do not get the error if I log in via the Django Admin with an admin account first.
Also, I only get this is I'm running from behind nginx. If I run with either the Django dev server, or gunicorn, and hit the port directly, I'm fine, and can happily hit the URL anonymously. If I then put nginx in front of this, to forward to the same gunicorn/runserver I get this error.
Maybe it's something to do with my nginx proxy_pass settings?
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
I'm running django rest framework 2.2.6, Django 1.5 and nginx 1.2.7.
I set throttling to a silly high number in rest framework, and looked at the permissions which all seemed open by default (but set as so explicitly as well).
Can anyone point me in the right direction?
Thanks!
Ludo.
In my case, I had HTTP basic authentication configured in Apache, but not in my Django project. Because the request had authentication headers, the Django app thought I wanted to authenticate with it. I disabled authentication in the Django REST framework using these settings:
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': []
}
If it's a basic http auth (like in #Sjoerd's case) and you don't need a django authentication, you can also remove Authorization header in your nginx reverse proxy config:
location / {
...
proxy_set_header Authorization "";
}
When you send a request to the site, which is behind basic authentication,
client (browser) sends 'Authorization' header (as it should - see: "Basic access authentication" on wikipedia).
Ngnix passes it to your application.
Django Rest Framework also supports: BasicAuthentication, and it is enabled by default.
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.SessionAuthentication',
)
}
source: http://www.django-rest-framework.org/api-guide/authentication/
So django-rest-framework sees 'Authorization' header, and thinks that is should be django user with password.
If such user does not exist, then you got: "HTTP 401 Unauthorized".
See: http://www.django-rest-framework.org/api-guide/authentication/#basicauthentication
Soultions
Pick one:
a) Add to your ngnix site config
location / {
...
proxy_set_header Authorization "";
...
}
so DRF will not get 'Authorization' header, so it won't try to match django user.
After this change - basic auth can not be used on django side (empty header!
b) Get rid of "rest_framework.authentication.BasicAuthentication" from REST_FRAMEWORK (in your django settings).
If not defined, add to your django settings:
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.SessionAuthentication',
)
}
Probbably best solution.
c) Use your django user/password for basic auth?
Realy? - Don't!
You will get 403 FORBIDDEN if you have throttling enabled. Based on this issue: https://github.com/tomchristie/django-rest-framework/issues/667, turning off the lazy evaluation of request.user/request.auth is probably the best thing to do.
I have a django project set up with nginx+apache. The http port for outside access is 20111 which is then forwarded to the server machine (which has an internal IP) to port 80. So nginx listens on port 80 (and passes relevant requests to apache on port 5000).
Now the initial login can be reached from the outside via http://externalip:20111 - but when I complete an admin action, like saving an entry, I get redirected to http://externalip/path/to/model -- without the port 20111. The result is a timeout. How can I tell django to use a specific hostname/port (i.e. http://externalip:20111) for all admin redirects?
When deploying applications behind a proxy or load balancer, it is common to rely on the X-Forwarded-Host header. Django has support for it
First of all, you have to setup nginx to send the proper headers. Add to your nginx host configuration (inside your location section):
proxy_set_header X-Forwarded-Host $host:20111;
Second, add to your settings.py:
USE_X_FORWARDED_HOST = True
It will allow django to trust X-Forwarded-Host headers from a request.
It should make it work for you. For security reasons, you should not trust every value sent in X-Forwarded-Host, so add your trusted domains/IPs to ALLOWED_HOSTS in settings.py