I'm developing a website for my association Here. It uses Django 3.0.7 and PostgreSQL. I follow this tutorial to make it works in good condition in development and in production.
In development mode all is good, site is working perfectly, static files and media files are served by the Django built-in web server.
I test the "vanilla" (coming from the tutorial) production on my local machine it works too. But the tutorial is not complete for me so after completing it, I decided to adapt the code to fit my needs. As I follow the tutorial I created a new docker-compose and a new dockerfile for production. But there are 2 differences between the tutorial and the site I want to set in production:
I want to Use TREAFIK to route the traffic on different URL (appli.amis-simserhof.fr) because I will have other projects in the future on the same server with others subdomains..
I want to secure the website using HTTPS. So I use CERTBOT (let's encrypt) to generate certificates. I add this certificates to TREAFFIK in order to use HTTPS and it works.
So I adapted the docker-compose file with my new stuff :
version: '3.7'
services:
**traefik:
image: traefik:v1.7.12
command: [
"--loglevel=INFO",
"--api",
"--docker",
"--docker.domain=amis-simserhof.fr",
"--entrypoints=name:https address::443 tls:/etc/certs/fullchain.pem,/etc/certs/privkey.pem",
"--entrypoints=name:http address::80 redirect.entrypoint:https",
]
ports:
- 80:80 # http
- 443:443 # https
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /etc/letsencrypt/live/amis-simserhof.fr/fullchain.pem:/etc/certs/fullchain.pem
- /etc/letsencrypt/live/amis-simserhof.fr/privkey.pem:/etc/certs/privkey.pem**
web:
image: registry.gitlab.com/guillaumekoenigtncy/aas-web:latest
command: gunicorn aas.wsgi:application --bind 0.0.0.0:8000
expose:
- 8000
volumes:
- static_volume:/home/app/web/staticfiles
- media_volume:/home/app/web/mediafiles
environment:
- DEBUG=0
- SECRET_KEY=change_me
- DJANGO_ALLOWED_HOSTS=localhost appli.amis-simserhof.fr
- SQL_ENGINE=django.db.backends.postgresql
- SQL_DATABASE=postgres
- SQL_USER=postgres
- SQL_PASSWORD=postgres
- SQL_HOST=db
- SQL_PORT=5432
- DATABASE=postgres
labels:
- traefik.frontend.rule=HostRegexp:appli.amis-simserhof.fr
- traefik.frontend.entryPoints=https
depends_on:
- db
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
**nginx:
image: registry.gitlab.com/guillaumekoenigtncy/aas-nginx:latest
volumes:
- static_volume:/home/app/web/staticfiles
- media_volume:/home/app/web/mediafiles
expose:
- 80
- 443
depends_on:
- web**
volumes:
postgres_data:
static_volume:
media_volume:
I also created a dockerfile and a config file for nginx :
FROM nginx:1.19.0
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
upstream website {
server web:8000;
}
server {
listen 80;
listen [::]:80;
access_log off;
server_name appli.amis-simserhof.fr;
location / {
proxy_pass http://website;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /staticfiles/ {
alias /home/app/web/staticfiles/;
}
location /mediafiles/ {
alias /home/app/web/mediafiles/;
}
}
The problem is that now static files are not served in production (I got a 404). When I set the DEBUG to True in the setting.py of the project files are served sot that means static files are present in the correct folder that I have set in the settings.
STATIC_URL = "/staticfiles/"
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
MEDIA_URL = "/mediafiles/"
MEDIA_ROOT = os.path.join(BASE_DIR, "mediafiles")
In production, I have correctly run the python manage.py collectstatic command. I look inside both containers, ngnix and web, and in both the files are present in the correct directory.
So I conclude my configuration is not working but when I made changes in it, I saw this changes in the nginx logs of the containers. There are no errors...
I try everything during the last 2 days: set the ssl in the nginx config and remove it from traefik (site is not accessible), expose or not ports 80 and 443 in nginx or traefik (conflicts on open ports or too many redirect error), add or remove / at the end or at the beginning of blocks or alias in nginx configuration (change nothing), etc.
If you have any hint or tips I would be really grateful...
Have a nice day :)
I'am was able do define static path with following nginx service configuration:
version: '3.7'
services:
server:
container_name: dj
build:
context: ./server/project/
restart: unless-stopped
expose:
- 8000
volumes:
- static_volume:/app/staticfiles
- media_volume:/app/media
command: gunicorn project.wsgi:application --bind 0.0.0.0:8000
networks:
- web
nginx:
container_name: nginx
build:
context: ./
dockerfile: Dockerfile
restart: unless-stopped
volumes:
- static_volume:/app/staticfiles
- media_volume:/app/media
labels:
- "traefik.enable=true"
- "traefik.http.routers.${SERVICE}.rule=(Host(`${DOMAIN_NAME}`) && PathPrefix(`/static`)) || (Host(`${DOMAIN_NAME}`) && PathPrefix(`/media`))"
- "traefik.http.routers.${SERVICE}.tls.certresolver=letsEncrypt"
- "traefik.http.routers.${SERVICE}.entrypoints=web-secure"
- "traefik.http.services.${SERVICE}.loadbalancer.server.port=80"
depends_on:
- server
networks:
- web
volumes:
static_volume:
media_volume:
networks:
web:
external: true
Note this line:
- "traefik.http.routers.static-http.rule=Host(`ex.example.com`) && PathPrefix(`/static`)"
change the ex.example.com to you domain
Here is part of my configuration, The "server" part is not completed yet
UPDATE
To use the same static files between django and nginx use following nxing configuration:
#default.conf
server {
listen 80;
server_name _;
ignore_invalid_headers on;
location / {
root /usr/share/nginx/html;
index index.html;
default_type application/javascript;
try_files $uri $uri/ /index.html =404;
}
location /static {
autoindex on;
alias /app/staticfiles/;
}
location /media {
autoindex on;
alias /app/media/;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
}
Also note that I've updated docker-compose file and added Dockerfile below:
#Dockerfile
FROM nginx:1.19.1-alpine
COPY nginx/default.conf /etc/nginx/conf.d/default.conf
So your Project-Structure should look like:
Project-root/
|-server/
|-nginx/
| |-default.conf
|-Dockerfile
|-docker-compose.yml
Related
I am using nginx as a proxy to pass to my django app container. I can successfully access the main url but when it passes control back to the django app and logins in the django app container gives it the local url and it doesn't route back to the correct page.
The redirect uri comes in as http://app
site-enabled
upstream app{
server 0.0.0.0:8888;
}
server {
listen 8080;
server_name app-dev.company.net;
location / {
# django running in uWSGI
proxy_pass http://app;
include uwsgi_params;
uwsgi_read_timeout 300s;
client_max_body_size 320m;
sendfile on;
proxy_read_timeout 1800;
proxy_connect_timeout 1800;
proxy_send_timeout 1800;
send_timeout 1800;
}
}
docker-compose.yml
version: "3.9"
services:
app:
image: ecr/app
container_name: django_app
ports:
- 8888
env_file:
- dev.env
volumes:
- staticfiles:/opt/app/static/
nginx:
build: ./nginx
container_name: django_app
volumes:
- staticfiles:/opt/app/static/
ports:
- 8080:8080
depends_on:
- app
volumes:
staticfiles:
Your docker-compose / nginx configuration file is full with little errors that could cause this kind of problem - so lets try to remove them.
delete the container names if not needed. This will make it easier to understand how to link from one container in another.
Where is your NGINX Dockerfile you need to do build ./nginx.
docker-compose
version: "3.9"
services:
app:
image: ecr/app
ports:
- 8888
env_file:
- dev.env
volumes:
- staticfiles:/opt/app/static/
nginx:
build: ./nginx
volumes:
- staticfiles:/opt/app/static/
ports:
- 8080:8080
depends_on:
- app
volumes:
staticfiles:
NGINX Configuration
You can not use 0.0.0.0 in your upstream block. Normally I use the service name. In your case app. So please change that to server app:8888; and test it one more time.
Location Configuration
You are proxying http traffic to your django app container. There is no need to use uwsgi_read_timeout or include uwsgi_params. In your case a simple http proxy configuration would be enough. For example in all the things you have randomly added to the nginx configuration, one important proxy configuration is missing.
proxy_pass http://django_app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
Make sure your understand the directives you are using in the NGINX configuration as well as in the docker-compose file, clean up the config files and try again.
I have a dockerized app that works fine in development mode on my host machine. I'm trying to figure out how I can host my app on ec2 using the default ip address created when I launch my instance.
My folder structure is as follows.
backend
|---projectname
|---Dockerfile
|---requirements.txt
|---wait-for-it.sh
config/nginx
|---app.conf
frontend
|---nuxt folders
|---Dockerfile
This is my current docker compose file I'm using
docker-compose.yml
version: '3.4'
services:
db:
restart: always
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
env_file: .env
ports:
- "5432:5432"
expose:
- 5432
redis:
restart: always
image: redis
volumes:
- redisdata:/data
django:
build:
context: ./backend
env_file: .env
command: >
sh -c "./wait-for-it.sh db:5432 &&
cd autobets && python manage.py collectstatic --noinput &&
gunicorn --workers=2 --bind=0.0.0.0:8000 autobets.wsgi:application"
ports:
- "8000:8000"
volumes:
- ./backend:/app
depends_on:
- db
restart: on-failure
nuxt:
build:
context: ./frontend
environment:
- API_URI=http://django:8000/api
command: bash -c "npm install && npm run dev"
volumes:
- ./frontend:/app
ports:
- "3000:3000"
depends_on:
- django
- redis
volumes:
pgdata:
redisdata:
config/nginx/app.config
upstream django {
ip_hash;
server django:8000;
}
upstream nuxt {
ip_hash;
server nuxt:3000;
}
server {
location ~ /(api|admin|static)/ {
proxy_pass http://django;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
location / {
proxy_pass http://nuxt;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
listen 8000;
server_name localhost;
}
Say if my ec2 domain name pointer is ec2-52-204-122-132.compute-1.amazonaws.com
How do I set nginx up in my app to accept http connections to the frontend of my app?
On the backend localhost:8000/admin is my admin page I'd like to access this also using the ec2 domain name too.
What's the best way to alter my config so when I push my app after add the domain name pointer I can access my app hosted on ec2?
I've been reading documentation but can't find any helpful info for a dockerized django vue type app running on ec2.
firstly you need to make sure the security group attach to your box is open for incoming connection on port that NGinx listen
For each container you want to put on NGinx config you will need to find their do to so do:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
NGinx is not inside a container,docker-compose so, nginx.conf
upstream django {
ip_hash;
server <container IP>:8000; # <-- Change this, with container IP
}
upstream nuxt {
ip_hash;
server <container IP>:3000; # <-- Change this, with container IP
}
server {
location ~ /(api|admin|static)/ {
proxy_pass #django # <-- Change this, with container IP
proxy_pass <container IP>:8000; # <-- OR Change this, with container IP
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
location / {
proxy_pass #nuxt # <-- Change this, add port
proxy_pass <container IP>:3000 # <-- OR Change this, add port
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
listen 8000; # <-- Change port number, django already use this host port
server_name localhost ec2-52-204-122-132.compute-1.amazonaws.com; # <-- change this line, add EC2 public domain
}
docker-compose.yml
version: '3.4'
services:
db:
restart: always
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
env_file: .env
ports:
- "5432:5432"
expose:
- 5432
networks: # <-- Add this
- random_name # <-- Add this
redis:
restart: always
image: redis
volumes:
- redisdata:/data
networks: # <-- Add this
- random_name # <-- Add this
django:
build:
context: ./backend
env_file: .env
command: >
sh -c "./wait-for-it.sh db:5432 &&
cd autobets && python manage.py collectstatic --noinput &&
gunicorn --workers=2 --bind=0.0.0.0:8000 autobets.wsgi:application"
ports:
- "8000:8000"
volumes:
- ./backend:/app
depends_on:
- db
restart: on-failure
networks: # <-- Add this
- random_name # <-- Add this
nuxt:
build:
context: ./frontend
environment:
- API_URI=http://django:8000/api # <-- Wrong
# From you Javascript on the client point of view,
# you will request the public server, not the internal name of
# you backend inside a container, that the public will never see
- API_URI=http://ec2-52-204-122-132.compute-1.amazonaws.com/api # <-- Right
command: bash -c "npm install && npm run dev"
volumes:
- ./frontend:/app
ports:
- "3000:3000"
depends_on:
- django # <-- will become useless if change API_URI=
- redis # <-- bad design
networks: # <-- Add this
- random_name # <-- Add this
volumes:
pgdata:
redisdata:
networks: # <-- Add this
- random_name: # <-- Add this wanto make sure container can communicate
Security group on AWS should be open on the port you'll use for listening on NGinx, host 8000 is already used by django, so use another on
From app architecture POV, your backend should do the cache stuff with Redis, not the Frontend. Complexity or backend response caching should be cache on the backend somewhere on the controllers. You client should cache only statis assets. but you here to make you have working on a server, not to speak archi.
[SOLVED] This is a duplicate. Please see this question.
I have a basic Django API working in development when I launch via the runserver command. I am returning a list of objects including the URL of an image in my media folder. In development, this image URL includes the port as shown below. The link works fine when I click it in the browser.
"image_url": "http://0.0.0.0:1337/mediafiles/publisher/sample-image4.jpg",
In production (gunicorn, nginx, docker) everything works the same except the URLs returned by the API do not include the port, so the links are broken. How can I ensure the port is included even in production?
"image_url": "http://0.0.0.0/mediafiles/publisher/sample-image4.jpg",
My guess is it could be an nginx config issue since it works in development server, but I don't really know where the problem is so my searches are not really helping. I'm still quite new to nginx, django and docker.
settings.py
...
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_URL = '/mediafiles/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'mediafiles')
...
docker-compose.yml
version: '3.7'
services:
web:
build: ./app
command: gunicorn hello_django.wsgi:application --bind 0.0.0.0:8000
volumes:
- ./app/:/usr/src/app/
- static_volume:/usr/src/app/staticfiles
- media_volume:/usr/src/app/mediafiles
ports:
- "8000"
env_file: ./app/.env
environment:
- DB_ENGINE=django.db.backends.postgresql
- DB_USER
- DB_PASSWORD
- DB_HOST=db
- DB_PORT=5432
- DATABASE=postgres
depends_on:
- db
networks:
- backend
nginx:
build: ./nginx
volumes:
- static_volume:/usr/src/app/staticfiles
- media_volume:/usr/src/app/mediafiles
ports:
- "1337:80"
depends_on:
- web
networks:
- backend
networks:
backend:
driver: bridge
volumes:
postgres_data:
static_volume:
media_volume:
nginx.conf
upstream hello_django {
server web:8000;
}
server {
listen 80;
location / {
proxy_pass http://hello_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /staticfiles/ {
alias /usr/src/app/staticfiles/;
}
location /mediafiles/ {
alias /usr/src/app/mediafiles/;
}
location /favicon.ico {
access_log off;
log_not_found off;
}
}
I am fairly new to Nginx and Docker and am currently facing an issue regarding a docker container setup. The setup consists of three containers: Nginx, Django and Postgres. It works as expected for the most part, however, I am not able to access static files through Nginx.
Here is the nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local]'
'"$request" $status $body_bytes_sent'
'"$http_referer" "$http_user_agent"'
'"$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
upstream server {
server server:8000;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name localhost;
charset utf-8;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ #rewrites;
}
location #rewrites {
rewrite ^(.+)$ /index.html last;
}
location ^~ /static/ {
autoindex on;
alias /usr/share/nginx/html/static/;
}
location ~ ^/api {
proxy_pass http://server;
}
location ~ ^/admin {
proxy_pass http://server;
}
}
}
I would expect Nginx to serve /usr/share/nginx/html/static/ when I access the address localhost:8000/static. I did check the container fs at /usr/share/nginx/html/static/, and the static files are present.
Here is the docker-compose.yml:
version: "3"
services:
nginx:
container_name: nginx
build:
context: .
dockerfile: ./nginx/Dockerfile
image: nginx
restart: always
volumes:
- ./server/static:/usr/share/nginx/html/static
ports:
- 80:80
depends_on:
- server
command: nginx -g 'daemon off';
server:
container_name: server
build:
context: ./server
dockerfile: Dockerfile
hostname: server
ports:
- 8000:8000
volumes:
- ./server:/src/project
depends_on:
- "db"
restart: on-failure
env_file: .env
command: >
bash -c '
python manage.py makemigrations &&
python manage.py migrate &&
gunicorn project.wsgi -b 0.0.0.0:8000'
db:
container_name: postgres
image: postgres:latest
hostname: postgres
ports:
- 5432:5432
volumes:
- /var/lib/postgresql/data
The folder ./server/static contains all static files assembled trough python manage.py collectstatic and adds them to the volume /usr/share/nginx/html/static. However, when I try to access the static files, f.e. at localhost:8000/admin, I receive warnings for missing css files (base.css, login.css, ..).
Update:
For anyone wondering, I had to change to nginx port in the docker-compose file to 8000, so that requests from client to localhost:8000 will be processed by nginx and not the server directly. With that in my mind I also changed the port of the server in the docker-compose file to expose so that it is only internally accessible. This will however prevent you from accessing the admin part of django as well.
Here is the nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local]'
'"$request" $status $body_bytes_sent'
'"$http_referer" "$http_user_agent"'
'"$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
upstream server {
server server:8000;
}
server {
listen 8000 default_server;
listen [::]:8000 default_server;
server_name localhost;
charset utf-8;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ #rewrites;
}
location #rewrites {
rewrite ^(.+)$ /index.html last;
}
location ^~ /static/ {
autoindex on;
alias /usr/share/nginx/html/static/;
}
location ~ ^/api {
proxy_pass http://server;
}
location ~ ^/admin {
proxy_pass http://server;
}
}
}
Here is the docker-compose.yml:
version: "3"
services:
nginx:
container_name: nginx
build:
context: .
dockerfile: ./nginx/Dockerfile
image: nginx
restart: always
volumes:
- ./server/static:/usr/share/nginx/html/static
ports:
- 8000:8000
- 80:8000
depends_on:
- server
command: nginx -g 'daemon off';
server:
container_name: server
build:
context: ./server
dockerfile: Dockerfile
hostname: server
expose:
- "8000"
volumes:
- ./server:/src/project
depends_on:
- "db"
restart: on-failure
env_file: .env
command: >
bash -c '
python manage.py makemigrations &&
python manage.py migrate &&
gunicorn project.wsgi -b 0.0.0.0:8000'
db:
container_name: postgres
image: postgres:latest
hostname: postgres
ports:
- 5432:5432
volumes:
- /var/lib/postgresql/data
You're finding solutions based on false assumptions, because you have not emerged yourself in the material. That's fine if you just want to have a setup that works and then understand it later. You're not the first person to use Docker for Django development, so look around:
There is what Docker has already done. It's not the only way to do it and certainly not the best way on several fronts.
If you really want an Nginx based setup, then Real Python has a really good example.
The short points of why your setup was not working and your fixes are not an improvement:
You use http protocol and WSGI is much better suited for this
Your solution assumed static files must be served by Django. They must not. Nginx is much better at it, but serving them with Nginx is in fact a bit of a slow down if you're in the early stage of a project where you might be adding a lot of new static files.
And so...you should read the document that virtually anyone seems to skip.
I have a Docker + Gunicorn + Nginx + Django setup on AWS EC2 and Route 53. Right now I want to redirect mydomain.com to www.mydomain.com.
Is it appropriate to do a redirect in a Nginx configuration? Or are there are better solutions.
Here is docker-compose-yml, using gunicorn to start the Django server.
version: '2'
services:
nginx:
image: nginx:latest
container_name: dj_nginx
ports:
- "80:8000"
- "443:443"
volumes:
- ./src/my_project/static:/static
- ./src:/src
- ./config/nginx:/etc/nginx/conf.d
depends_on:
- web
web:
build: .
container_name: dj_web
command: bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn my_project.wsgi -b 0.0.0.0:8000"
depends_on:
- db
volumes:
- ./src:/src
- ./apps/django_rapid:/src/my_project/django_rapid
expose:
- "8000"
db:
image: postgres:latest
container_name: dj_db
Here is my Nginx Conf
upstream web {
ip_hash;
server web:8000;
}
# portal
server {
listen 8000;
location / {
proxy_pass http://web/;
}
location /media {
alias /media; # your Django project media files - amend as required
}
location /static {
alias /static; # your Django project static files - amend as required
}
server_name localhost;
}
# portal (https)
server {
listen 443;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/conf.d/mynginx.crt;
ssl_certificate_key /etc/nginx/conf.d/mynginx.key;
location /media {
alias /media; # your Django project media files - amend as required
}
location /static {
alias /static; # your Django project static files - amend as required
}
location / {
proxy_pass http://web/;
}
}
Yes, it's appropriate to do these kinds of redirects in the webserver. If it's https your certificate needs to cover both domains.
Yes it's appropriate to do nginx redirections, but i find doing PREPEND_WWW more simple.
Add this in your settings.py
PREPEND_WWW = True