Can't connect to Website - Connection refused - Nginx - SSL - django

I'm working with Docker, Nginx and Django. I would like to secure my application with ssl but it won't work.
I got a valid certificate using certbot
This is my nginx.conf file:
upstream app {
server app:80;
}
server {
listen 80;
listen [::]:80;
server_name mydomain.de;
return 301 https://$server_name$request_uri;
location ~ /.well-known/acme-challenge {
allow all;
root /var/www/certbot;
}
}
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name mydomain.de;
ssl_certificate /etc/nginx/ssl/live/mydomain.de/fullchain.pem;
ssl_certificate_key /etc/nginx/ssl/live/mydomain.de/privkey.pem;
location / {
proxy_pass https://app;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header Host $host;
# proxy_redirect off;
}
location /staticfiles/ {
alias /app/staticfiles/;
add_header Access-Control-Allow-Origin *;
location ~ /.well-known/acme-challenge {
allow all;
root /var/www/certbot;
}
}
}
That's my docker-compose file:
version: '3.4'
services:
app:
image: django
build:
context: ./app
dockerfile: Dockerfile
env_file:
- ./.env
volumes:
- ./app/:/app/
- ./app/staticfiles/:/app/staticfiles
command: gunicorn --bind 0.0.0.0:8000 --chdir /app/ Webserver.wsgi
nginx:
build: ./nginx
ports:
- 80:80
- 433:433
depends_on:
- app
volumes:
- ./app/staticfiles/:/app/staticfiles
- ./certbot/conf:/etc/nginx/ssl
- ./certbot/data:/var/www/certbot
db:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
POSTGRES_DB_PORT: "5432"
POSTGRES_DB_HOST: "myhost"
POSTGRES_PASSWORD: "mypw"
POSTGRES_USER: myname
POSTGRES_DB: dev_db
volumes:
postgres_data:
If I try to access my Website I just see the browser message "Connection refused"
I renamed sensitive information like domain name and passwords

Below I'm providing a working certbot nginx configuration example:
server {
# show half the users an optimized site, half the regular site
listen 80;
gzip on;
gzip_http_version 1.0;
gzip_min_length 1100;
gzip_buffers 4 32k;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 9;
gzip_disable "MSIE [1-6]\.";
gzip_types text/plain text/xml text/css
text/comma-separated-values
text/javascript
application/x-javascript
application/atom+xml;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# side note: only use TLS since SSLv2 and SSLv3 have had recent vulnerabilities
access_log /var/www/vhosts/mydomain.de/logs/access_log;
error_log /var/www/vhosts/mydomain.de/logs/error_log;
server_name 3dact.com www.mydomain.de;
location ~* .+.>(xml|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf) {
access_log off;
expires 30d;
break;
}
charset utf-8;
root /var/www/vhosts/mydomain.de/public/dist;
index index.html index.htm;
location / {
try_files $uri $uri/ /index.html;
}
# what to serve if upstream is not available or crashes
error_page 500 502 503 504 /media/50x.html;
location ~* .+.>(xml|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf) {
root /var/www/vhosts/mydomain.de/public/dist;
access_log off;
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
break;
}
location /dist {
alias /var/www/vhosts/mydomain.de/public/dist;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/nginx/ssl/live/mydomain.de/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/nginx/ssl/live/mydomain.de/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.mydomain.de) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = mydomain.de) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name mydomain.de www.mydomain.de;
return 404; # managed by Certbot
}
The first server block provides the actual locations and certbot configurations, the second one is used by certbot for domain redirections (www.). Given you properly map the volumes in docker-compose.yml, that should preserve the consistency when you're connecting. Also, make sure ports 80 and 443 get proper exposure outside of the container.
In your docker-compose.yml:
nginx:
build: ./nginx
ports:
- 80:80
- 433:433
depends_on:
- app
volumes:
- ./app/staticfiles/:/app/staticfiles
- ./certbot/conf:/etc/nginx/ssl # Make sure it maps into /etc/nginx/ssl/live/mydomain.de
- ./certbot/data:/var/www/certbot
- ./letsencrypt:/etc/letsencrypt # This is where options-ssl-nginx.conf and ssl-dhparams.pem are located
If you don't have a local ./letsencrypt dir or the files are located in a different place, please create any dirs, copy the files there and configure the mapping accordingly.

Related

Hosting two different django project in the same droplet with different subdomains (NGINX, Gunicorn, ubuntu)

As the title says I want to host two different django project in the same droplet (NGINX, Gunicorn, ubuntu) with different subdomains. One will be our main site example.com. which is up and running and working perfectly. We want to host the staging site staging.example.com in the same droplet.
We have created new sockets and service files for the staging site and activated and enabled them but the issue is nginx still points to the files in main domain directory rather than the staging directory and hence we get this error below even though these domains have been added in the allowed hosts of settings.py of the staging site
DisallowedHost at /
Invalid HTTP_HOST header: 'staging.example.com'. You may need to add 'staging.example.com' to ALLOWED_HOSTS
Here is our staging.guinicorn.service file
[Unit]
Description=staging.gunicorn daemon
Requires=staging.gunicorn.socket
After=network.target
[Service]
User=admin
Group=www-data
WorkingDirectory=/home/admin/example1staging
ExecStart=/home/admin/example1staging/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/staging.gunicorn.sock djangoproject.wsgi:application
[Install]
WantedBy=multi-user.target
Here is our staging.guicorn.socket file
[Unit]
Description=staging.gunicorn socket
[Socket]
ListenStream=/run/staging.gunicorn.sock
[Install]
WantedBy=sockets.target
Lastly here is our nginx config
server {
listen 80;
listen [::]:80;
server_name example.com www.example.com;
return 302 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
ssl_client_certificate /etc/ssl/cloudflare.crt;
ssl_verify_client on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/admin/example1;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
server {
listen 80;
listen [::]:80;
server_name staging.example.com www.staging.example.com;
return 302 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
ssl_client_certificate /etc/ssl/cloudflare.crt;
ssl_verify_client on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/admin/example1staging;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/staging.gunicorn.sock;
}
}
Some help here would be extremely welcome.

Django, DRF, Docker, nginx - Server Configuration - Base URL for mediafiles

I am trying to set up my Django Rest Framework application. Unfortunately I still can't manage to get the media files url right.
I tried adding the lines from this answer to my nginx configuration ending up with "Bad Request (400)": Django Rest framework swagger base url
The application is running on my-domain.com.
https://my-domain.com/user/
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"id": "512e0f9f-f08a-4a2d-8e1d-b933a16c0a1f",
"date_joined": "2021-10-10T11:41:05.077828Z",
"name": "Maxine Mustermann",
"user_picture": "**http://127.0.0.1:8337**/mediafiles/images/usr1.jpeg"
}
]
}
Expected: https://my-domain.com/mediafiles/images/usr1.jpeg (Eventhough the picture can be accessed with this url)
Returned: http://127.0.0.1:8337/mediafiles/images/usr1.jpeg
docker-compose file:
version: "3.8"
services:
db:
image: postgres:13-alpine
volumes:
- postgres_data_staging:/var/lib/postgresql/data/
env_file:
- ./.env.staging.db
expose:
- 5432
web:
build:
context: ./api
dockerfile: Dockerfile.staging
command: gunicorn config.wsgi:application --workers 3 --bind 0.0.0.0:8000
volumes:
- static_volume_staging:/home/api/web/staticfiles
- media_volume_staging:/home/api/web/mediafiles
expose:
- 8000
env_file:
- ./.env.staging
depends_on:
- db
nginx:
build: ./nginx
volumes:
- static_volume_staging:/home/api/web/staticfiles
- media_volume_staging:/home/api/web/mediafiles
ports:
- 8337:80
depends_on:
- web
volumes:
postgres_data_staging:
static_volume_staging:
media_volume_staging:
nginx.conf:
upstream ae_backend {
server web:8000;
}
server {
listen 80;
client_max_body_size 75M;
server_name $DOMAIN www.$DOMAIN;
location / {
proxy_pass http://ae_backend;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
location /staticfiles/ {
alias /home/api/web/staticfiles/;
}
location /mediafiles/ {
alias /home/api/web/mediafiles/;
}
}
Server configuration nginx/sites-enabled/my-api:
server {
server_name my-domain.com;
location = /favicon.ico { access_log off; log_not_found off; }
# add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Authorization';
location / {
proxy_pass http://127.0.0.1:8337;
}
listen [::]:443 ssl; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/my-domain.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/my-domain.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = my-domain.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name my-domain.com;
return 404; # managed by Certbot
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Authorization';
}
Any help and reference will be appreciated.
Thank you!!
Regards,
Florian
btw: This is my first post on stackoverflow. Please be nice ;-)
I edited this part into my Server nginx config:
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8337;
}
and made sure that my-domain.com ist listed in the "ALLOWED_HOSTS" (settings.py).

User uploads with nginx and docker

I have a Django app where users can upload files containing data they want to be displayed in the app. The app is containerised using docker. In production I am trying to configure nginx to make this work and as far as I can tell it is working to some extent.
As far as I can tell the file does actually get uploaded as I can see it in the container, and I can also download it from the app. The problem I am having is that once the form has been submitted it is supposed to redirect to another form, where the user can assign stuff to the data in the app (not really relevant to the question). However, I am getting a 500 error instead.
I have taken a look at the nginx error logs and I am seeing:
[info] 8#8: *11 client closed connection while waiting for request, client: 192.168.0.1, server: 0.0.0.0:443
and
[info] 8#8: *14 client timed out (110: Operation timed out) while waiting for request, client: 192.168.0.1, server: 0.0.0.0:443
when the operation is performed.
I also want the media files to be persisted so they are in a docker volume.
I suspect the first log message may be the culprit but is there a way to prevent this from happening, or is it just a poor connection on my end?
Here is my nginx conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log debug;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
proxy_headers_hash_bucket_size 52;
client_body_buffer_size 1M;
client_max_body_size 10M;
gzip on;
upstream app {
server django:5000;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name dali.vpt.co.uk;
location / {
return 301 https://$server_name$request_uri;
}
}
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name dali.vpt.co.uk;
ssl_certificate /etc/nginx/ssl/cert.crt;
ssl_certificate_key /etc/nginx/ssl/cert.key;
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
# cookiecutter-django app
location #proxy_to_app {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app;
}
location /media/ {
autoindex on;
alias /app/tdabc/media/;
}
}
}
and here is my docker-compose file:
version: '2'
volumes:
production_postgres_data: {}
production_postgres_backups: {}
production_media: {}
services:
django: &django
build:
context: .
dockerfile: ./compose/production/django/Dockerfile
image: production_django:0.0.1
depends_on:
- postgres
- redis
volumes:
- .:/app
- production_media:/app/tdabc/media
env_file:
- ./.envs/.production/.django
- ./.envs/.production/.postgres
command: /start.sh
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: production_postgres:0.0.1
volumes:
- production_postgres_data:/var/lib/postgresql/data
- production_postgres_backups:/backups
env_file:
- ./.envs/.production/.postgres
nginx:
build:
context: .
dockerfile: ./compose/production/nginx/Dockerfile
image: production_nginx:0.0.1
depends_on:
- django
volumes:
- production_media:/app/tdabc/media
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
Any help or insight into this problem would be much appreciated.
Thanks for your time.
Update
Another thing that I should mention is that when I run the app with my production settings but set DEBUG to True it works perfectly but this is only happening when DEBUG is set to false.

HTTP request 408 error(Client request timeout) over mobile network. Django + Gunicorn + Nginx

I have faced with strange issue. My API has endpoint on which i am using PATCH(tried also PUT and POST - same result) to send json with jsonarray(request body size typically is 40KB) from mobile phone. When i am doing this using Wi-Fi - all works fine. But if i am using mobile network, i am facing with SocketConnection Timeout(in Retrofit/Kotlin) with some phones(iOS works perfect).
From nginx access.log :
domain - - [21/Nov/2017:16:33:01 +0000] "PATCH /endpoint/ HTTP/1.1" 408 0 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Nexus 5 Build/M4B30Z)"
Nginx config :
server {
listen port;
server_name domain;
client_max_body_size 32m;
proxy_connect_timeout 75s;
proxy_read_timeout 300s;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/user/apps/api;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/user/apps/api/api.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/domain/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
Gunicorn config :
respawn
setuid root
setgid www-data
chdir /home/user/apps/api
exec /home/user/virtualenvs/apienv/bin/gunicorn --workers 3 --timeout=90 --bind unix://api.sock api.wsgi:application
The problem was in network connection. We changed location of our clouds from Frankfurt to Amsterdam and now all works perfect

Elasticbeanstalk - Force HTTPs on Docker container with Nginx

I have a single-container Docker running a React environment on Elasticbeanstalk with Nginx. I pointed a subdomain to the ELB URL, and want to force a HTTPS redirection if you visit the subdomain (i.e. you type subdomain.domain.com and it should redirect you to HTTPS).
Now, if I visit the default ELB URL (something.eu-central-1.elasticbeanstalk.com), it will be redirected to HTTPS. But I want my custom domain (which is parked somewhere else but points to something.eu-centralblabla with a CNAME) to be forced to use HTTPS as well, but it doesn't happen. It allows regular HTTP requests.
I've tried several guides and followed AWS documentation, but I cannot seem to force it to redirect to HTTPS on my custom subdomain.
These are my files:
/.ebextensions folder
http-instance.config
files:
/etc/nginx/conf.d/https.conf:
mode: "000644"
owner: root
group: root
content: |
# HTTPS Server
server {
listen 443;
server_name localhost;
ssl on;
ssl_certificate /etc/pki/tls/certs/server.crt;
ssl_certificate_key /etc/pki/tls/certs/server.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_prefer_server_ciphers on;
location / {
proxy_pass http://docker;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
#SSL CRT and KEY below
https-instance-single.config
Resources:
sslSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
IpProtocol: tcp
ToPort: 443
FromPort: 443
CidrIp: 0.0.0.0/0
/nginx folder
default.conf
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html?/$request_uri;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
error_page 500 504 /500.html;
error_page 502 /502.html;
error_page 503 /503.html;
client_max_body_size 4G;
keepalive_timeout 10;
location ~ ^/(favicon|static)/ {
gzip_static on;
expires max;
add_header Cache-Control public;
# add_header Last-Modified "";
# add_header ETag "";
open_file_cache max=1000 inactive=500s;
open_file_cache_valid 600s;
open_file_cache_errors on;
break;
}
}
What am I doing wrong? Thanks for your help!
You should be able to manage this in your nginx config by adding this within the server context:
set $redirect_to_https 0;
if ($http_x_forwarded_proto != 'https') {
set $redirect_to_https 1;
}
if ($redirect_to_https = 1) {
rewrite ^ https://$host$request_uri? permanent;
}
Or something to that effect.
Route all http traffic to https:
server {
listen 80;
return 301 https://$host$request_uri;
}
Then hangle the proxy stuff in the 443 block