Django CSRF "Referer Malformed"... but it isn't - django

I'm trying to test a deployment config for a Django setup that works fine in development mode.
I have name-based routing via Nginx's ssl_preread module on a load balancer, and SSL terminates at another Nginx instance on the server itself where the requests are proxied to uwsgi by socket.
server {
server_name dev.domain.net;
listen 80 proxy_protocol;
listen [::]:80 proxy_protocol;
location / {
return 301 https://$host$request_uri;
}
}
server {
server_name dev.domain.net;
listen 443 ssl;
listen [::]:443 ssl;
location / {
include uwsgi_params;
uwsgi_pass unix:/run/uwsgi/website.sock;
}
location /favicon.ico {
access_log off; log_not_found off;
}
}
I have uwsgi set to log %(host) and %(referer), they match in the logs.
In my uwsgi_params I'm passing $host and $referer like so, since I'm using name-based routing I pick up the $server_name variable that triggered the Nginx response...
uwsgi_param HTTP_REFERER $server_name;
uwsgi_param HTTP_HOST $host;
Adding (or taking away) protocols and ports to these makes no difference. Taking them away predictably generates a Django ALLOWED_HOSTS debug error.
I've confirmed that my ALLOWED_HOSTS includes the $host. I've tried adding CSRF_TRUSTED_ORIGINS for the same $host variable. I've tried setting CSRF_COOKIE_DOMAIN for the same $host variable. I have CSRF_COOKIE_SECURE set to True per the docs recommendation.
No matter what combination of the above settings are used, I get:
Referer checking failed - Referer is malformed. on all POST requests.

Short answer: don't use the uwsgi unix socket, but rather use http-socket and send the proxy request to localhost over unencrypted http (in uwsgi ini file):
http-socket = 127.0.0.1:8001
In nginx, get rid of uwsgi proxy params and simply proxy_pass with proxy_protocol headers enabled:
server {
server_name dev.domain.net;
listen 443 ssl proxy_protocol;
listen [::]:443 ssl proxy_protocol;
location / {
proxy_pass http://127.0.0.1:8001;
}
location /favicon.ico {
access_log off; log_not_found off;
}
}
At that point you can enable all of the recommended deployment settings in the Django docs, explicitly declare your ALLOWED_HOSTS and everything works fine.
These are a quite silly series of hoops with no apparent correct set of answers, especially considering referers are client headers that are easily forged.
The better answer is Django needs to get rid of a client referer check in its CSRF mechanism, it's pointless and makes no sense...

Related

Django CSRF verification failed with nginx reverse proxy

I am desperate.
I have a https server that redirects traffic to my local https server.
My university runs a reverse proxy that forwards the requests my nginx:
The local https server is only visible from the vpn. They do that so that I do not have to care about the keys management and I can just plug some generic ones.
When I on the VPN the external server works as expected but when I am out of the VPN the external server returns CSRF token missing or incorrect.. Sometimes i receive CSRF cookie not set.
I have checked basically everything and read all the CSRF verification failed. Request aborted. posts here.
On the form I try to submit I can see csrfmiddlewaretoken <input type='hidden' name='csrfmiddlewaretoken' value="jdwjwjefjwdjqwølksqøwkop2j3ofje" />
The url in the external server is (anonymized):
https://project.uni.edu/
The url of the internal server (anonymized):
https://project_internal.uni.edux/
The configuration of the nginx:
server {
listen 443 ssl;
server_name project.uni.edu localhost;
ssl_certificate /code/staticfiles/cert/project_name.pem;
ssl_certificate_key /code/staticfiles/cert/project_name_key.pem;
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
}
location /static/ {
alias /code/staticfiles/;
}
location /media/ {
alias /code/mediafiles/;
}
}
My settings in the django file are:
CSRF_TRUSTED_ORIGINS = ["https://project.uni.edu/","https://project_internal.uni.edux/"]
# SESSION_COOKIE_SECURE= True
CSRF_COOKIE_HTTPONLY = False
SESSION_COOKIE_DOMAIN= "project_internal.uni.edux"
I have no idea what's happening. Thanks

Bad Gateway python-telegram-bot with webhook and Nginx

I'm trying to set up a webhook with python-telegram-bot and Nginx. I am faced with a problem, my bot doesn't get messages from telegram. I also tried to make GET/POST queries from the postman and I always get a "502 Bad Gateway" error. I also launched the netstat to monitor port 5000 where my telegram bot connects but it is always empty. It seems like webhook doesn't launch at all.
My Nginx default.conf file looks like the following:
upstream django {
server gunicorn:8000;
}
server {
listen 80;
server_name example.com www.example.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl;
ssl on;
server_name example.com www.example.com;
ssl_certificate fullchain.pem;
ssl_certificate_key privkey.pem;
access_log /var/log/nginx/nginx.vhost.access.log;
error_log /var/log/nginx/nginx.vhost.error.log;
location /TELEGRAM_TOKEN {
proxy_pass http://0.0.0.0:5000/TELEGRAM_TOKEN/;
}
location /static/ {
alias /static/;
}
location / {
proxy_pass http://example.com:8000;
}
}
And my telegram client code:
updater = Updater(api_token)
updater.dispatcher.add_handler(CommandHandler('start', start))
updater.dispatcher.add_handler(CallbackQueryHandler(button))
updater.dispatcher.add_handler(MessageHandler(Filters.text & ~Filters.command, custom_command))
jq = updater.job_queue
job_minute = jq.run_repeating(callback_minute, interval=5)
#updater.start_polling()
updater.start_webhook(listen="0.0.0.0", port=5000, url_path=api_token,
webhook_url=f'https://example.com/{api_token}')
updater.idle()
I also have Django options for the Nginx server but I've never seen any tutorial or documentation on how to tune the webhook with Django and it can be the reason for my problems.
Have anyone any idea about solving my problem?
CallMeStag, your advice to use the following code helped me solve the issue:
print(Bot(api_token).get_webhook_info()))
I had 2 problems with the Nginx config file
First: ssl on
It gave me an error connection refused and I deleted it
Second
I changed this
location /TELEGRAM_TOKEN {
proxy_pass http://0.0.0.0:5000/TELEGRAM_TOKEN/;
}
to this
location /TELEGRAM_TOKEN {
proxy_pass http://example.com:5000/TELEGRAM_TOKEN/;
}
and the webhook is now working.
Thank you.

Nginx 2 different domains on one server

I'd like to know how to configure nginx to get 2 domains working on one server (1 ip address).
I want to setup a Keycloak SSO next to a bookstack instance.
My issue is that when I want to access bookstack.domain.com it redirects to keycloak.domain.com.
Here's my /etc/nginx/conf.d/keycloak.conf :
upstream keycloak {
# Use IP Hash for session persistence
ip_hash;
# List of Keycloak servers
server 127.0.0.1:8080;
}
server {
listen 80;
server_name keycloak.domain.com;
# Redirect all HTTP to HTTPS
location / {
return 301 https://\$server_name\$request_uri;
}
}
server {
listen 443 ssl http2;
server_name keycloak.domain.com;
ssl_certificate /path/to/certificate.crt;
ssl_certificate_key /path/to/certificate_key.key;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://keycloak;
}
}
Here's my /etc/nginx/conf.d/bookstack.conf :
server {
listen 3480;
access_log /var/log/nginx/bookstack_access.log;
error_log /var/log/nginx/bookstack_error.log;
server_name bookstack.domain.com;
root /var/www/bookstack/public;
#
# redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
#
return 301 https://$host$request_uri;
}
server {
listen 5443 ssl http2;
ssl_certificate /path/to/certificate.crt;
ssl_certificate_key /path/to/certificate_key.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AE;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/nginx/dhparam.pem;
server_name bookstack.domain.com;
#HSTS
add_header Strict-Transport-Security "max-age=63072000" always;
root /var/www/bookstack/public;
access_log /var/log/nginx/bookstack_access.log;
error_log /var/log/nginx/bookstack_error.log;
client_max_body_size 1G;
fastcgi_buffers 64 4K;
index index.php;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ ^/(?:\.htaccess|data|config|db_structure\.xml|README) {
deny all;
}
location ~ \.php(?:$|/) {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_pass unix:/var/run/php-fpm.sock;
}
location ~* \.(?:jpg|jpeg|gif|bmp|ico|png|css|js|swf)$ {
expires 30d;
access_log off;
}
}
Please let me know :)
This is exactly expected nginx behavior for the given configuration. One of the server blocks always act as default server for any request arriving on some IP/port combination no matter what is the Host HTTP header value. Here is an official documentation on this subject. You can use default_server parameter for the listen directive to explicitly specify server block that should act as the default server or it will be the first server block that listen on those IP/port otherwise. On multihomed servers things can be more complicated, as discussed here.
Now back to the question. You have four server blocks in your configuration: first one listen on TCP port 80 (default port for http:// scheme), second one listen on TCP port 443 (default port for https:// scheme), one listen on port 3480 and the last one listen on port 5443. Since there is only one server block listening each port, each server block will act as default server for any request coming to that port. So if you type http://bookstack.domain.com in your browser address bar, default port 80 for http:// scheme will be used and your request will be redirected to https://keycloak.domain.com. You are using
return 301 https://\$server_name\$request_uri;
for redirection, and the $server_name variable will be always keycloak.domain.com for that server block (read this answer to understand the difference between $host, $http_host and $server_name variables). If you explicitly specify the port and type http://bookstack.domain.com:3480, your request will be served by the third server block thus being redirected to https://bookstack.domain.com (here your are using $host variable which is right). Default TCP port https:// scheme is 443. But the only server block that listen on that port is for keycloak.domain.com! Oops. The only way you can reach your bookstack.domain.com is to type https://bookstack.domain.com:5443 in your browser. And if you correctly understand all the above information, you can type https://keycloak.domain.com:5443 too, it won't made any difference.
Well, I tried to explain what happened here with your nginx configuration. Get rid of non-standard ports as #Evil_skunk recommends you in his answer. Don't forget to clear your browser cache before trying new configuration - permanent HTTP 301 redirects are often cached by the browsers, unlike temporary HTTP 302 redirects.
Your keycloak config seems ok
It listen on port 80 (http) and port 443 (https) and all requests to 80 (http) are redirected to 443 (https)
Your bookstack config looks wrong for me
It does not listen to port 80 or 443 (instead it listen to 5443 and 3480). If you don't have some kind of special port forwarding then I think request to bookstack.domain.com will never reach the nginx-server defined in bookstack.conf and as a result the only matching server will serve the request => keycloak
You should change bookstack.conf's Listen ports:
server {
listen 80;
#... redirect to https
}
server {
listen 443 ssl;
#ssl config, webroot, ...
}

Nginx Django after add SSL get too many redirects error

Fist of all sorry for my bad english.
I'm having a problem configuring LetsEncrypt in my webapp, i make it work now i can access using https://www.myproject.com but if i try to use www.myproject.com, myproject.com or even https://myproject.com without the www i always get the error ERR_TOO_MANY_REDIRECTS.
This is my nginx config in /etc/nginx/sites-available/myproject
server {
listen 80;
listen [::]:80;
server_name myproject.com www.myproject.com;
return 301 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
include snippets/ssl-myproject.com.conf;
include snippets/ssl-params.conf;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/user;
}
location /media/ {
root /home/user;
}
location /.well-known {
alias /home/user/myproject/.well-known;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/user/myproject.sock;
}
}
I check a lot of questions like mine but in php projects try the solutions but still not found one to solve my problem.
if helps i have to say that i have cloudflare free configure for my domain
Thanks!
FIX: If you use cloudflare in your web when you install SSL certificates have to put the SSL cloudflare configuration in Full or Full(strict).

How to deploy django on VPS with external subdomain.?

Good day.
I have a web app that I have developed using django. I tested fine on my local, and I'm happy with how it works.
However I'm facing an issue bringing it online I used those two guides to reach my deployment:
https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04
and
http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/
However my page is giving me a forbidden page.
I suspect my issue is with the way I'm handling the subdomain. So the site . has been developed using php, and I have worked on my part with django and been provided with a subdomain which is member.domain.com, So I'm deploying it on the VPS and have to make it use the subdomain.
This is how my allowed hosts looks in the settings.py
ALLOWED_HOSTS = ['member.domain.com']
and
in my nginx:
upstream app_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/home/path/project/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name member.domain.com;
client_max_body_size 4G;
access_log /home/path/project/logs/nginx-access.log;
error_log /home/path/project/logs/nginx-error.log;
location /static/ {
alias /home/path/project/src/static/;
}
location /media/ {
alias /home/path/project/src/media/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set "proxy_buffering off" *only* for Rainbows! when doing
# Comet/long-poll stuff. It's also safe to set if you're
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://app_server;
break;
}
}
# Error pages
error_page 502 503 504 /500.html;
location = /500.html {
root /home/path/project/src/static/;
}
}
I'm not sure what I am doing wrong.
I will appreciate any help
To respond to 'example.com' and any subdomains, start the domain with a dot
ALLOWED_HOSTS = ['.example.com', '203.0.113.5']
I didn't even try how to run django on subdomains, but from article link you shared, you missed some configuration in your settings.py
ALLOWED_HOSTS = ['member.domain.com']
Changed
ALLOWED_HOSTS = ['.domain.com']
Hope this will solve your problem