Deploy to docker with nginx, django, daphne - django

Question

I want to deploy my service to docker.
and my service is developed using python+django and django-channels
── myproject
├── myproject
│ ├── settings.py
│ ├── urls.py
│ ├── asgi.py
│ ├── ...
├── collected_static
│ ├── js
│ ├── css
│ ├── ...
├── nginx
│ ├── Dockerfile
│ ├── service.conf
├── requirements.txt
├── manage.py
├── Dockerfile
└── docker-compose.yml
myproject/Dockerfile :
FROM python
ENV PYTHONUNBURRERED 1
RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject
RUN pip install -r requirements.txt
RUN python manage.py migrate
myproject/docker-compose.yml:
version: '2'
services:
nginx:
build: ./nginx
networks:
- front
- back
ports:
- "80:80"
depends_on:
- daphne
redis:
image: redis
networks:
- "back"
ports:
- "6379:6379"
worker:
build: .
working_dir: /opt/myproject
command: bash -c "python manage.py runworker"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
daphne:
build: .
working_dir: /opt/myproject
command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
ports:
- "8000:8000"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
networks:
front:
back:
myproject/nginx/Dockerfile
FROM nginx
COPY service.conf /etc/nginx/sites-enabled/
myproject/nginx/service.conf
server {
listen 80;
server_name example.com #i just want to hide domain name..
charset utf-8;
client_max_body_size 20M;
location /static/ {
alias /opt/myproject/collected_static/;
}
location / {
proxy_pass http://0.0.0.0:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
and i write a command docker-compose up -d, nginx and daphne work well.
but when i connected to example.com:80, i just can see nginx default page.
and when i connected to example.com:8000, i just can see myproject's service page. (but cannot see static files)
I want to link nginx and daphne service! what should I do? please help me.
when i just deploy with nginx+daphne+django without docker, my service works well.

Answer 1

TLDR;
Nginx is not configured correctly, but also your docker-compose needs some correction:
Nginx
The Nginx website has some helpful tips for deploying with Docker that you should read, including a sample, very simple Dockerfile:
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY content /usr/share/nginx/html
COPY conf /etc/nginx
which points to some improvements you need to make (see the Docker Compose section for further help with Docker).
Bearing in mind the updates to deployment that we will make below, you will also need to change your Nginx config:
rename service.conf -> service.template
change listen ${NGINX_PORT};
change server_name ${NGINX_HOST};
change proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};
Docker Compose
Now your Nginx configuration is correct, you need to setup the docker compose directives correctly, thankfully, the Docker Hub Nginx page has an example for docker compose:
Here is an example using docker-compose.yml:
web:
image: nginx
volumes:
- ./mysite.template:/etc/nginx/conf.d/mysite.template
ports:
- "8080:80"
environment:
- NGINX_HOST=foobar.com
- NGINX_PORT=80
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
The mysite.template file may then contain variable references like this:
listen ${NGINX_PORT};
From r00m's answer
You can make all those improvements, and in fact, without sharing the volumes your static files won't be served correctly.
Create an image for the project and re-use it
Add the Volume references to allow static files to be shared
OPTIONAL: you should also follow the advice about collecting the static files, but your project structure kind of suggests that you've already done that.
Bringing it all together
Finally, we can merge those three improvements to give us the following setup:
myproject/Dockerfile:
FROM python
ENV PYTHONUNBUFFERED 1
RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject
RUN pip install -r requirements.txt
RUN python manage.py migrate # Can this be done during build? i.e. no link to the DB?
VOLUME ["/opt/myproject/collected_static"]
myproject/docker-compose.yml:
version: '2'
services:
nginx:
build: ./nginx
networks:
- front
- back
ports:
- "80:80"
volumes_from:
- "daphne"
environment:
- NGINX_HOST=example.com
- NGINX_PORT=80
- DAPHNE_HOST=daphne
- DAPHEN_PORT=8000
depends_on:
- daphne
links:
- daphne
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/service.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
redis:
image: redis
networks:
- "back"
ports:
- "6379:6379"
daphne:
build: .
image: "myproject:latest"
working_dir: /opt/myproject
command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
ports:
- "8000:8000"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
worker:
image: "myproject:latest"
working_dir: /opt/myproject
command: bash -c "python manage.py runworker"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
networks:
front:
back:
myproject/nginx/Dockerfile
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY service.template /etc/nginx/conf.d
myproject/nginx/service.template
server {
listen ${NGINX_PORT};
server_name ${NGINX_HOST}
charset utf-8;
client_max_body_size 20M;
location /static/ {
alias /opt/myproject/collected_static/;
}
location / {
proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
Final thoughts
I'm not sure what you're trying to achieve with your network directives, but it almost certainly doesn't achieve it, for example nginx shouldn't connect into your backend network (I think...).
You need to consider whether "migrate" should be done at build time or run time.
Do you need to be able to change your nginx configuration easily? If so, you should remove the COPY from the nginx build and add in the volumes directive from the Docker Compose section.

Answer 2

You have misconfigured NGINX. Try proxy_pass http://127.0.0.1:8000;
As for the static files, it's because you haven't made the files available to the container. I would suggest the following modifications:
myproject/Dockerfile:
[...]
ADD . /opt/myproject
VOLUME ["/opt/myproject/collected_static"]
[..]
# may I also suggest automatic static file collection?
RUN python manage.py collectstatic --noinput
myproject/docker-compose.yml:
[...]
build: ./nginx
volumes_from:
- "worker" # or daphne
I would also consider adding image option to daphne and worker services. This will tag the image and allow to reuse it, thus it will be only built once (instead of twice).
myproject:
build: .
image: "myproject:latest"
[..]
worker:
image: "myproject:latest"
[..]
daphne:
image: "myproject:latest"