Handling dynamic IP in Django ALLOWED_HOSTS with nginx - django

I'm having issues with what i believe is my nginx.conf which is causing the instance to be restarted again and again, as the health-checks fail on my managed container service.
I'm running my setup in AWS Lightsail Containers, where I have three containers running:
nginx
django
nextjs
When publishing a new release on my AWS Lightsail instance it runs fine for a few minutes, then I hit a 503 error, which causes the instance to reboot - run a few minutes then reboot again.
Looking at the logs I can see that the health-check failed, and django throws and error saying that I should add the request IP to the allowed hosts:
[28/Aug/2021:13:56:23] Invalid HTTP_HOST header: 'x.x.x.x'. You may need to add 'x.x.x.x' to ALLOWED_HOSTS.
[28/Aug/2021:13:56:23] Bad Request: /health.txt
The problem is that my lightsail container service does not have a static IP (nor do I believe I can get a static IP).
My current nginx.conf is below (feedback is appreciated). My question here is how should I deal with this issue? I feel like setting ALLOWED_HOSTS = ['*'] is not a great approach. Can I hardcode the host for the healthcheck or similar?
nginx.conf:
upstream backend {
server ${BACKEND_HOST}:${BACKEND_PORT};
}
upstream frontend {
server ${FRONTEND_HOST}:${FRONTEND_PORT};
}
server {
listen 80 default_server;
server_name example.com;
server_tokens off;
gzip on;
gzip_proxied any;
gzip_comp_level 4;
gzip_types text/css application/javascript image/svg+xml;
location /robots.txt {
include proxy_params;
proxy_pass http://backend;
}
location /health.txt {
include proxy_params;
proxy_pass http://backend;
}
location /api {
include proxy_params;
proxy_pass http://backend;
}
location /admin {
include proxy_params;
proxy_pass http://backend;
}
location / {
proxy_pass http://frontend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}

I use AWS EC2, to pass the health check I get the ip of the instance dynamically and then I insert it into ALLOWED_HOSTS (I think it should work also for Lightsail Containers):
import requests
def get_instance_ip():
try:
ip = requests.get('http://169.254.169.254/latest/meta-data/local-ipv4').text
except requests.exceptions.ConnectionError:
return None
return ip
AWS_IP = get_ec2_instance_ip()
if AWS_IP is not None:
ALLOWED_HOSTS += [AWS_IP]
You can also create a middleware that always returns a 200 status code for the path used by health check (insert the custom middleware before django.middleware.security.SecurityMiddleware in MIDDLEWARE to avoid Invalid HTTP_HOST header error).

Related

site cannot reached after nginx restart - AWS EC2

I am following the tutorial and completed it without any error, when I start nginx it showed me welcome page but after I restart nginx chrome is showing
This site can’t be reached.
I am accessing it by my public IP.
Here is my security group of instance (Inbound)
(Outbound)
Here is my nginx conf file:
upstream sample_project_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/home/ubuntu/django_env/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name <my public ip>;
client_max_body_size 4G;
access_log /home/ubuntu/logs/nginx-access.log;
error_log /home/ubuntu/logs/nginx-error.log;
location /static/ {
alias /home/ubuntu/static/;
}
location /media/ {
alias /home/ubuntu/media/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set "proxy_buffering off" *only* for Rainbows! when doing
# Comet/long-poll stuff. It's also safe to set if you're
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://sample_project_server;
break;
}
}
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/ubuntu/static/;
}
}
any suggestions ? I am really frustrated, because every thing done perfectly but after restarting nginx, site is not reachable.

Nginx always falls back to default config

My stack is django, gunicorn, nginx and supervisor running on a VPS from DigitalOcean.
Supervisor is running the program correctly but I always get the NGINX welcome page. If I delete the default nginx config file everything works and I get the website.
Here are my custom settings:
upstream maet_app_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/webapps/maet/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name maet.bg www.maet.com;
client_max_body_size 4G;
access_log /webapps/maet/logs/nginx-access.log;
error_log /webapps/maet/logs/nginx-error.log;
location /static/ {
alias /webapps/maet/website/static/;
}
location /media/ {
alias /webapps/maet/website/static/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set "proxy_buffering off" *only* for Rainbows! when doing
# Comet/long-poll stuff. It's also safe to set if you're
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://maet_app_server;
break;
}
}
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /webapps/maet/website/static/;
}
}
How do I change it to use this config instead of the default one?
I don't want to delete the default file because I need it for let's encrypt ssl.

How to deploy django on VPS with external subdomain.?

Good day.
I have a web app that I have developed using django. I tested fine on my local, and I'm happy with how it works.
However I'm facing an issue bringing it online I used those two guides to reach my deployment:
https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04
and
http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/
However my page is giving me a forbidden page.
I suspect my issue is with the way I'm handling the subdomain. So the site . has been developed using php, and I have worked on my part with django and been provided with a subdomain which is member.domain.com, So I'm deploying it on the VPS and have to make it use the subdomain.
This is how my allowed hosts looks in the settings.py
ALLOWED_HOSTS = ['member.domain.com']
and
in my nginx:
upstream app_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/home/path/project/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name member.domain.com;
client_max_body_size 4G;
access_log /home/path/project/logs/nginx-access.log;
error_log /home/path/project/logs/nginx-error.log;
location /static/ {
alias /home/path/project/src/static/;
}
location /media/ {
alias /home/path/project/src/media/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set "proxy_buffering off" *only* for Rainbows! when doing
# Comet/long-poll stuff. It's also safe to set if you're
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://app_server;
break;
}
}
# Error pages
error_page 502 503 504 /500.html;
location = /500.html {
root /home/path/project/src/static/;
}
}
I'm not sure what I am doing wrong.
I will appreciate any help
To respond to 'example.com' and any subdomains, start the domain with a dot
ALLOWED_HOSTS = ['.example.com', '203.0.113.5']
I didn't even try how to run django on subdomains, but from article link you shared, you missed some configuration in your settings.py
ALLOWED_HOSTS = ['member.domain.com']
Changed
ALLOWED_HOSTS = ['.domain.com']
Hope this will solve your problem

Django Rest Framework app, served by Gunicorn, proxied by Nginx (SSL)

Goal: I want to serve django-rest-framework(drf) responses over https.
I'm trying to understand why my setup isn't working, whether it can be fixed by a simple settings change in one of the components, or if I should try a different approach.
I have a drf application served by gunicorn. Gunicorn is behind an NGINX proxy. I also have some static content served by NGINX. This setup has worked fine for plain old http.
Then I set NGINX to listen over ssl (,and installed certs, etc.). The static content works over https. I can still get to drf via http, but I get nothing / timed out over https. The gunicorn logs haven't been that helpful to me, but django dev server gives "dev server doesn't work over https" when sent a request.
Here is my first attempt at the nginx config (anonymized). This was mostly from reading the gunicorn and nginx manuals.
server {
server_name example.com;
listen 443 ssl;
ssl_certificate /path/to/cert;
ssl_certificate_key /path/to/key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ...
location /static/{
alias /path/to/static/content/;
}
location / {
return 301 $scheme://example.com/static;
}
location /drf/{
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Real_IP $remote_addr;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://127.0.0.1:6565; # bound to gunicorn, can be reached from http://example.com:6565 - ideally want to be available from https://example.com:6565
}
}
server {
listen 80;
server_name example.com;
return 301 https://example.com
}
I don't think there is an additional step for gunicorn. On the DRF side, I read the relevant Django security section and tried using the SECURE_SSL_REDIRECT = True (new in 1.8 - not addressed in a lot of existing questions) and SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https').
I also tried with a redirect, but this was sort of a shot in the dark:
...after `proxy_pass` above, and commenting out proxy_redirect off:
proxy_redirect http://127.0.0.1:6565 https://example.com/drf
Ubuntu 14, Gunicorn 19, Nginx 1.1,
Django 1.8 (DRF 3.2)
Update:
NGINX Error logging:
I get nothing if I go to https://example.com:6565. Here is an error from nginx's error log for a request to https://example.com/drf/endpoint.
2015/09/08 13:53:52 [error] 12564#0: *14 connect() failed (111: Connection refused while connecting to upstream, client: 155.xxx.xxx.xx, server:example.com, request:"GET /drf/endpoint", upstream: "http://127.0.0.1:6565/drf/endpoint", host: "example.com"

Django #login_required dropping https

I'm trying to test my Django app locally using SSL. I have a view with the #login_required decorator. So when I hit /locker, I get redirected to /locker/login?next=/locker. This works fine with http.
However, whenever I use https, the redirect somehow drops the secure connection, so I get something like https://cumulus.dev/locker -> http://cumulus.dev/locker/login?next=/locker
If I go directly to https://cumulus.dev/locker/login?next=locker the page opens fine over a secure connection. But once I enter the username and password, I go back to http://cumulus.dev/locker.
I'm using Nginx to handle the SSL, which then talks to runserver. My nginx config is
upstream app_server_djangoapp {
server localhost:8000 fail_timeout=0;
}
server {
listen 80;
server_name cumulus.dev;
access_log /var/log/nginx/cumulus-dev-access.log;
error_log /var/log/nginx/cumulus-dev-error.log info;
keepalive_timeout 5;
# path for static files
root /home/gaurav/www/Cumulus/cumulus_lightbox/static;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}
}
}
server {
listen 443;
server_name cumulus.dev;
ssl on;
ssl_certificate /etc/ssl/cacert-cumulus.pem;
ssl_certificate_key /etc/ssl/privkey.pem;
access_log /var/log/nginx/cumulus-dev-access.log;
error_log /var/log/nginx/cumulus-dev-error.log info;
keepalive_timeout 5;
# path for static files
root /home/gaurav/www/Cumulus/cumulus_lightbox/static;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}
}
}
Django is running on plain HTTP only behind the proxy, so it will always use that to construct absolute URLs (such as redirects), unless you configure it how to see that the proxied request was originally made over HTTPS.
As of Django 1.4, you can do this using the SECURE_PROXY_SSL_HEADER setting. When Django sees the configured header, it will treat the request as HTTPS instead of HTTP: request.is_secure() will return true, https:// URLs will be generated, and so on.
However, note the security warnings in the documentation: you must ensure that the proxy replaces or strips the trusted header from all incoming client requests, both HTTP and HTTPS. Your nginx configuration above does not do that with X-Forwarded-Ssl, making it spoofable.
A conventional solution to this is to set X-Forwarded-Protocol to http or https, as appropriate, in each of your proxy configurations. Then, you can configure Django to look for it using:
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTOCOL', 'https')