How to setup nginx to site to a Django app over https? - django

I have an application that I am running with multiple docker containers with a docker-compose file. Two main containers are django_container and nginx_container.
I want to force using https for my application but unfortunately, my Nginx config doesn't seem to work. Basically, my gunicorn server seems to be running OK.
I can see the http://django:8001 from inside the nginx_container, but I can't see http://example.com from inside it or outside.
Here is the content of my Nginx config
server {
listen 80;
server_name example.com;
rewrite ^ https://example.com$request_uri? permanent;
}
server {
listen 443 ssl;
server_name example.com;
ssl_certificate /etc/nginx/certs/example_com.crt;
ssl_certificate_key /etc/nginx/certs/cert.key;
access_log /log/nginx.access.log main;
location / {
proxy_pass http://django:8001;
proxy_set_header X-Forwarded-Proto $scheme;
}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
To create the containers I use the following docker-compose.yml file, which read the values from the corresponding .env file
version: "3.9"
services:
postgres:
image: postgres:13.0
container_name: postgres_container
restart: always
ports:
- "5432:5432"
volumes:
- $P_VOLUME:/initdb.d/
environment:
P_USER: $P_USER
P_PASSWORD: $P_PASSWORD
P_DB: $P_DB
django:
build:
context: .
dockerfile: ./Dockerfile
container_name: django_container
depends_on:
- postgres
ports:
- "8001:8001"
volumes:
- $ROOT_DIR:/code/
environment:
SECRET_KEY: $SECRET_KEY
ROOT_DIR: $ROOT_DIR
nginx:
image: nginx
container_name: nginx_container
depends_on:
- django
ports:
- "80:80"
- "443:443"
environment:
NGINX_HOST: $NGINX_HOST
NGINX_PORT: 80
volumes:
- $ROOT_DIR/config/$NGINX_CONF:/etc/nginx/conf.d/default.conf
- $ROOT_DIR/log/:/log/
- $ROOT_DIR/certs/:/etc/nginx/certs/
I checked the .env file and it seems the paths and configs are correct.
What am I missing here? Thanks a ton!

Related

Incorrect redirect of NGINX with Docker

I'm building my first project with Django, NGINX and Docker. Below the nginx.conf:
upstream project {
server website:8000;
}
server {
listen 80;
server_name MY-DOMAIN;
location / {
proxy_pass http://project;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
client_max_body_size 4G;
}
location /static/ {
alias /app/static-folder/;
}
location /media/ {
alias /app/media-folder/;
}
}
And the docker-compose:
version: '3.7'
services:
website:
container_name: web_project
image: project/django
build: ./djangodocker
restart: always
env_file: prod.env
command: sh -c "cd djangodocker/ &&
gunicorn djangodocker.wsgi:application --bind 0.0.0.0:8000"
volumes:
- static-folder:/app/static-folder
- media-folder:/app/media-folder
expose:
- 8000
nginx:
container_name: nginx_web_project
image: project/nginx
build: ./nginx
volumes:
- static-folder:/app/static-folder
- media-folder:/app/media-folder
ports:
- 8000:80
depends_on:
- website
volumes:
static-folder:
media-folder:
I can build the image but I can't see the website into the correct url. I see the website at MY-DOMAIN:8000 instead of MY-DOMAIN and this is my problem.
You map the nginx port to port 8000 on the line
- 8000:80
in your docker-compose file. Change that to
- 80:80
That way, nginx listens on port 80 on the host machine and you don't need to specify the port number.

How do I run multiple sites on the same server using docker and nginx?

I'm trying to run two sites on django on the same server under different ip, an error occurs that the port is busy, I fixed the ports, but the site does not start. Tell me where is the error please? Ip work, when I go to the second ip I get redirects to the first site. All settings were specified for the second site. At the end, I added the nginx setting of the first site
This is the second docker-compose file and its settings. I would be very grateful for your help
.env
#Django
# Should be one of dev, prod
MODE=prod
PORT=8008
#postgres
DB_NAME=xxx
DB_USER=xxx
DB_HOST=xxx
DB_PASSWORD=xxxx
DB_PORT=5432
POSTGRES_PASSWORD=mysecretpassword
#WSGI
WSGI_PORT=8008
WSGI_WORKERS=4
WSGI_LOG_LEVEL=debug
# Celery
CELERY_NUM_WORKERS=2
# Email
EMAIL_HOST_USER=xxxx
EMAIL_HOST_PASSWORD=xxxx
docker-compose.yml
version: '3'
services:
backend:
build: ./
container_name: site_container
restart: always
command: ./commands/start_server.sh
ports:
- "${PORT}:${WSGI_PORT}"
volumes:
- ./src:/srv/project/src
- ./commands:/srv/project/commands
- static_content:/var/www/site
env_file:
- .env
depends_on:
- postgres
postgres:
image: postgres:12
volumes:
- pg_data:/var/lib/postgresql/data
env_file:
- .env
# environment:
# - DJANGO_SETTINGS_MODULE=app.settings.${MODE}
nginx:
image: nginx:1.19
volumes:
- ./nginx:/etc/nginx/conf.d
- static_content:/var/www/site
ports:
- 81:80
- 444:443
env_file:
- .env
depends_on:
- backend
volumes:
pg_data: {}
static_content: {}
default.conf
server {
listen 80 default_server;
server_name 183.22.332.12;
location /static/ {
root /var/www/site;
}
location /media/ {
root /var/www/site;
}
location / {
proxy_set_header Host $host;
proxy_pass http://backend:8010;
}
}
default.conf for first site
server {
#listen 80 default_server;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name site1 ip_site1;
ssl_certificate /etc/letsencrypt/live/site1/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/site1/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/site1/chain.pem;
location /static/ {
root /var/www/artads;
}
location /media/ {
root /var/www/artads;
}
location / {
proxy_set_header Host $host;
proxy_pass http://backend:8008;
}
}
server {
listen 80 default_server;
server_name ip_site2 site2;
location /static/ {
root /var/www/gdr_mr;
}
location /media/ {
root /var/www/gdr_mr;
}
location / {
proxy_set_header Host $host;
proxy_pass http://backend:8013;
}
}
server {
listen 80;
listen [::]:80;
server_name www.site1 site1;
location / {
return 301 https://site1$request_uri;
}
}
Thanks to #Roman Tokaren and #Oleksandr
Here the english translated version submitted by #Roman Tokaren here
You can always argue a lot about the "correct" launch - after all, how many people, so many opinions, but I will describe an example + - of a "convenient" and scalable configuration. For the "convenience" of working in such a configuration, I would suggest installing nginxproxymanager as a reverse proxy and combining containers and nginxproxymanager into one network - after which it will be possible to forward container ports via http (s), tcp, udp to an external interface using the GUI as well as a number of other goodies, such as the generation of SSL certificates and their auto renewal
First, let's create the network itself
docker network create --driver bridge --subnet 172.26.0.0/24 testnet
Let's configure NPM (nginxproxymanager) - by default we will consider the reverse proxy as the last network node, as a result we will get
version: "3"
services:
app:
image: 'jc21/nginx-proxy-manager:latest'
networks:
testnet:
ipv4_address: 172.26.0.254
restart: always
ports:
# Public HTTP Port:
- '80:80'
# Public HTTPS Port:
- '443:443'
# Admin Web Port:
- '81:81'
environment:
# These are the settings to access your db
DB_MYSQL_HOST: "172.26.0.253"
DB_MYSQL_PORT: 3306
DB_MYSQL_USER: "user"
DB_MYSQL_PASSWORD: "pwd"
DB_MYSQL_NAME: "npm"
volumes:
- ./data/nginx-proxy-manager:/data
- ./letsencrypt:/etc/letsencrypt
depends_on:
- db
db:
image: yobasystems/alpine-mariadb:latest
restart: always
networks:
testnet:
ipv4_address: 172.26.0.253
environment:
MYSQL_ROOT_PASSWORD: "pwd"
MYSQL_DATABASE: "npm"
MYSQL_USER: "user"
MYSQL_PASSWORD: "pwd"
volumes:
- ./data/mariadb:/var/lib/mysql
networks:
testnet:
external: true
And configure the container itself
version: '3'
services:
backend:
build: ./
container_name: site_container
restart: always
command: ./commands/start_server.sh
networks:
testnet:
ipv4_address: 172.26.0.2
volumes:
- ./src:/srv/project/src
- ./commands:/srv/project/commands
- static_content:/var/www/site
env_file:
- .env
depends_on:
- postgres
postgres:
image: postgres:12
volumes:
- pg_data:/var/lib/postgresql/data
env_file:
- .env
# environment:
# - DJANGO_SETTINGS_MODULE=app.settings.${MODE}
networks:
testnet:
external: true
volumes:
pg_data: {}
static_content: {}
After that, we carry out the initial configuration of NPM according to the instructions and add the host
If you want run more than one site in a server, you can
use different ip
use different port
use different domain
use different path and rewrite it
...
You can choose one of top list tips.
In your config, you choose different ip and same port, but you set all the site to default and not listen the different ip
server{
listen ip:port;
}
Usually IP is just omitted.
Or you can one ip and different port.
server{
listen port1;
}
server{
listen port2;
}
Or you can one ip and one port but different domain.
server{
listen port;
server_name 1.a.com;
}
server{
listen port;
server_name 2.a.com;
}
If you're running two virtual servers with different IPs on the same machine, you'd want to specify the IP address in the listen directive:
server {
listen 192.168.1.1:80;
server_name example.net www.example.net;
...
}
server {
listen 192.168.1.2:80;
server_name example.com www.example.com;
...
}
More on how nginx processes requests can be found here: http://nginx.org/en/docs/http/request_processing.html

NGINX and TRAEFIK staticfiles are not working with HTTPS in production

I'm developing a website for my association Here. It uses Django 3.0.7 and PostgreSQL. I follow this tutorial to make it works in good condition in development and in production.
In development mode all is good, site is working perfectly, static files and media files are served by the Django built-in web server.
I test the "vanilla" (coming from the tutorial) production on my local machine it works too. But the tutorial is not complete for me so after completing it, I decided to adapt the code to fit my needs. As I follow the tutorial I created a new docker-compose and a new dockerfile for production. But there are 2 differences between the tutorial and the site I want to set in production:
I want to Use TREAFIK to route the traffic on different URL (appli.amis-simserhof.fr) because I will have other projects in the future on the same server with others subdomains..
I want to secure the website using HTTPS. So I use CERTBOT (let's encrypt) to generate certificates. I add this certificates to TREAFFIK in order to use HTTPS and it works.
So I adapted the docker-compose file with my new stuff :
version: '3.7'
services:
**traefik:
image: traefik:v1.7.12
command: [
"--loglevel=INFO",
"--api",
"--docker",
"--docker.domain=amis-simserhof.fr",
"--entrypoints=name:https address::443 tls:/etc/certs/fullchain.pem,/etc/certs/privkey.pem",
"--entrypoints=name:http address::80 redirect.entrypoint:https",
]
ports:
- 80:80 # http
- 443:443 # https
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /etc/letsencrypt/live/amis-simserhof.fr/fullchain.pem:/etc/certs/fullchain.pem
- /etc/letsencrypt/live/amis-simserhof.fr/privkey.pem:/etc/certs/privkey.pem**
web:
image: registry.gitlab.com/guillaumekoenigtncy/aas-web:latest
command: gunicorn aas.wsgi:application --bind 0.0.0.0:8000
expose:
- 8000
volumes:
- static_volume:/home/app/web/staticfiles
- media_volume:/home/app/web/mediafiles
environment:
- DEBUG=0
- SECRET_KEY=change_me
- DJANGO_ALLOWED_HOSTS=localhost appli.amis-simserhof.fr
- SQL_ENGINE=django.db.backends.postgresql
- SQL_DATABASE=postgres
- SQL_USER=postgres
- SQL_PASSWORD=postgres
- SQL_HOST=db
- SQL_PORT=5432
- DATABASE=postgres
labels:
- traefik.frontend.rule=HostRegexp:appli.amis-simserhof.fr
- traefik.frontend.entryPoints=https
depends_on:
- db
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
**nginx:
image: registry.gitlab.com/guillaumekoenigtncy/aas-nginx:latest
volumes:
- static_volume:/home/app/web/staticfiles
- media_volume:/home/app/web/mediafiles
expose:
- 80
- 443
depends_on:
- web**
volumes:
postgres_data:
static_volume:
media_volume:
I also created a dockerfile and a config file for nginx :
FROM nginx:1.19.0
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
upstream website {
server web:8000;
}
server {
listen 80;
listen [::]:80;
access_log off;
server_name appli.amis-simserhof.fr;
location / {
proxy_pass http://website;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /staticfiles/ {
alias /home/app/web/staticfiles/;
}
location /mediafiles/ {
alias /home/app/web/mediafiles/;
}
}
The problem is that now static files are not served in production (I got a 404). When I set the DEBUG to True in the setting.py of the project files are served sot that means static files are present in the correct folder that I have set in the settings.
STATIC_URL = "/staticfiles/"
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
MEDIA_URL = "/mediafiles/"
MEDIA_ROOT = os.path.join(BASE_DIR, "mediafiles")
In production, I have correctly run the python manage.py collectstatic command. I look inside both containers, ngnix and web, and in both the files are present in the correct directory.
So I conclude my configuration is not working but when I made changes in it, I saw this changes in the nginx logs of the containers. There are no errors...
I try everything during the last 2 days: set the ssl in the nginx config and remove it from traefik (site is not accessible), expose or not ports 80 and 443 in nginx or traefik (conflicts on open ports or too many redirect error), add or remove / at the end or at the beginning of blocks or alias in nginx configuration (change nothing), etc.
If you have any hint or tips I would be really grateful...
Have a nice day :)
I'am was able do define static path with following nginx service configuration:
version: '3.7'
services:
server:
container_name: dj
build:
context: ./server/project/
restart: unless-stopped
expose:
- 8000
volumes:
- static_volume:/app/staticfiles
- media_volume:/app/media
command: gunicorn project.wsgi:application --bind 0.0.0.0:8000
networks:
- web
nginx:
container_name: nginx
build:
context: ./
dockerfile: Dockerfile
restart: unless-stopped
volumes:
- static_volume:/app/staticfiles
- media_volume:/app/media
labels:
- "traefik.enable=true"
- "traefik.http.routers.${SERVICE}.rule=(Host(`${DOMAIN_NAME}`) && PathPrefix(`/static`)) || (Host(`${DOMAIN_NAME}`) && PathPrefix(`/media`))"
- "traefik.http.routers.${SERVICE}.tls.certresolver=letsEncrypt"
- "traefik.http.routers.${SERVICE}.entrypoints=web-secure"
- "traefik.http.services.${SERVICE}.loadbalancer.server.port=80"
depends_on:
- server
networks:
- web
volumes:
static_volume:
media_volume:
networks:
web:
external: true
Note this line:
- "traefik.http.routers.static-http.rule=Host(`ex.example.com`) && PathPrefix(`/static`)"
change the ex.example.com to you domain
Here is part of my configuration, The "server" part is not completed yet
UPDATE
To use the same static files between django and nginx use following nxing configuration:
#default.conf
server {
listen 80;
server_name _;
ignore_invalid_headers on;
location / {
root /usr/share/nginx/html;
index index.html;
default_type application/javascript;
try_files $uri $uri/ /index.html =404;
}
location /static {
autoindex on;
alias /app/staticfiles/;
}
location /media {
autoindex on;
alias /app/media/;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
}
Also note that I've updated docker-compose file and added Dockerfile below:
#Dockerfile
FROM nginx:1.19.1-alpine
COPY nginx/default.conf /etc/nginx/conf.d/default.conf
So your Project-Structure should look like:
Project-root/
|-server/
|-nginx/
| |-default.conf
|-Dockerfile
|-docker-compose.yml

Serving static files with Ngnix, Django and Docker

I am fairly new to Nginx and Docker and am currently facing an issue regarding a docker container setup. The setup consists of three containers: Nginx, Django and Postgres. It works as expected for the most part, however, I am not able to access static files through Nginx.
Here is the nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local]'
'"$request" $status $body_bytes_sent'
'"$http_referer" "$http_user_agent"'
'"$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
upstream server {
server server:8000;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name localhost;
charset utf-8;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ #rewrites;
}
location #rewrites {
rewrite ^(.+)$ /index.html last;
}
location ^~ /static/ {
autoindex on;
alias /usr/share/nginx/html/static/;
}
location ~ ^/api {
proxy_pass http://server;
}
location ~ ^/admin {
proxy_pass http://server;
}
}
}
I would expect Nginx to serve /usr/share/nginx/html/static/ when I access the address localhost:8000/static. I did check the container fs at /usr/share/nginx/html/static/, and the static files are present.
Here is the docker-compose.yml:
version: "3"
services:
nginx:
container_name: nginx
build:
context: .
dockerfile: ./nginx/Dockerfile
image: nginx
restart: always
volumes:
- ./server/static:/usr/share/nginx/html/static
ports:
- 80:80
depends_on:
- server
command: nginx -g 'daemon off';
server:
container_name: server
build:
context: ./server
dockerfile: Dockerfile
hostname: server
ports:
- 8000:8000
volumes:
- ./server:/src/project
depends_on:
- "db"
restart: on-failure
env_file: .env
command: >
bash -c '
python manage.py makemigrations &&
python manage.py migrate &&
gunicorn project.wsgi -b 0.0.0.0:8000'
db:
container_name: postgres
image: postgres:latest
hostname: postgres
ports:
- 5432:5432
volumes:
- /var/lib/postgresql/data
The folder ./server/static contains all static files assembled trough python manage.py collectstatic and adds them to the volume /usr/share/nginx/html/static. However, when I try to access the static files, f.e. at localhost:8000/admin, I receive warnings for missing css files (base.css, login.css, ..).
Update:
For anyone wondering, I had to change to nginx port in the docker-compose file to 8000, so that requests from client to localhost:8000 will be processed by nginx and not the server directly. With that in my mind I also changed the port of the server in the docker-compose file to expose so that it is only internally accessible. This will however prevent you from accessing the admin part of django as well.
Here is the nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local]'
'"$request" $status $body_bytes_sent'
'"$http_referer" "$http_user_agent"'
'"$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
upstream server {
server server:8000;
}
server {
listen 8000 default_server;
listen [::]:8000 default_server;
server_name localhost;
charset utf-8;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ #rewrites;
}
location #rewrites {
rewrite ^(.+)$ /index.html last;
}
location ^~ /static/ {
autoindex on;
alias /usr/share/nginx/html/static/;
}
location ~ ^/api {
proxy_pass http://server;
}
location ~ ^/admin {
proxy_pass http://server;
}
}
}
Here is the docker-compose.yml:
version: "3"
services:
nginx:
container_name: nginx
build:
context: .
dockerfile: ./nginx/Dockerfile
image: nginx
restart: always
volumes:
- ./server/static:/usr/share/nginx/html/static
ports:
- 8000:8000
- 80:8000
depends_on:
- server
command: nginx -g 'daemon off';
server:
container_name: server
build:
context: ./server
dockerfile: Dockerfile
hostname: server
expose:
- "8000"
volumes:
- ./server:/src/project
depends_on:
- "db"
restart: on-failure
env_file: .env
command: >
bash -c '
python manage.py makemigrations &&
python manage.py migrate &&
gunicorn project.wsgi -b 0.0.0.0:8000'
db:
container_name: postgres
image: postgres:latest
hostname: postgres
ports:
- 5432:5432
volumes:
- /var/lib/postgresql/data
You're finding solutions based on false assumptions, because you have not emerged yourself in the material. That's fine if you just want to have a setup that works and then understand it later. You're not the first person to use Docker for Django development, so look around:
There is what Docker has already done. It's not the only way to do it and certainly not the best way on several fronts.
If you really want an Nginx based setup, then Real Python has a really good example.
The short points of why your setup was not working and your fixes are not an improvement:
You use http protocol and WSGI is much better suited for this
Your solution assumed static files must be served by Django. They must not. Nginx is much better at it, but serving them with Nginx is in fact a bit of a slow down if you're in the early stage of a project where you might be adding a lot of new static files.
And so...you should read the document that virtually anyone seems to skip.

Cabot - 502 bad gateway error

Mine nginx.conf file
worker_processes 1;
error_log stderr notice;
events {
worker_connections 1024;
}
http {
types_hash_max_size 2048;
upstream cabot_app {
server cabot:5000 max_fails=3 fail_timeout=3s;
}
server {
listen 8088;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://cabot_app;
}
location /static/ {
include /etc/nginx/mime.types;
default_type application/octet-stream;
root /code/;
}
}
}
and docker-compose yaml file
redis:
image: redis
restart: always
db:
image: postgres
restart: always
ports:
- "5432:5432"
restart: always
environment:
POSTGRES_PASSWORD: test
POSTGRES_USER: test
POSTGRES_DB: tests
cabot:
build: .
restart: always
links:
- redis:celerybroker
- db
- redis
env_file: cabot_env
ports:
- "5000:5000"
nginx:
image: nginx
ports:
- "9999:8088"
links:
- cabot
volumes_from:
- cabot
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./cabot/static/:/code/static:ro
Temporarily I am getting 502 bad gateway. its goes off when I reload the page.
Nginx logs files gives:
/code/static/CACHE/css/base.36481a0991d5.css" failed (2: No such file or directory)
I'm not familiar with cabot. But from the error log it looks like cabot is using django-compressor (not sure). And the CACHE files by django-compressor is not generated yet.
You may need to add an additional step to generate the compressed css/js files.
python manage.py compress