serving Django static files with Docker, nginx and gunicorn - django

I am setting us a Django 2.0 application with Docker, nginx and gunicorn.
It's running the server but static files are not working.
Here is the settings.py content
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static_my_project')
]
STATIC_ROOT = os.path.join(BASE_DIR, 'static_cdn', 'static_root')
While developing, I put my static files inside static_my_project, which on running collectstatic copies to static_cdn/static_root
The directory structure is like
app
|- myapp
|- settings.py
|- static_my_project
|- static_cdn
|- static_root
|- config
|- nginx
|- nginx.conf
|- manage.py
|- Docker
|- docker-compose.yml
on running
docker-compose up --build
on running collectstatic, it gives path where static files will be copied
koober-dev | --: Running collectstatic
koober-dev |
koober-dev | You have requested to collect static files at the destination
myapp-dev | location as specified in your settings:
myapp-dev |
myapp-dev | /app/static_cdn/static_root
myapp-dev |
myapp-dev | This will overwrite existing files!
myapp-dev | Are you sure you want to do this?
myapp-dev |
myapp-dev | Type 'yes' to continue, or 'no' to cancel:
myapp-dev | 0 static files copied to '/app/static_cdn/static_root', 210 unmodified.
the config/nginx/nginx.conf file contains following settings
upstream web {
ip_hash;
server web:9010;
}
server {
location /static {
autoindex on;
alias /static/;
}
location / {
proxy_pass http://web;
}
listen 10080;
server_name localhost;
}
docker-compose.yml
version: '3'
services:
nginx:
image: nginx:latest
container_name: "koober-nginx"
ports:
- "10080:80"
- "10443:43"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn/static_root/:/static
depends_on:
- web
web:
build: .
container_name: "koober-dev"
command: ./start.sh
volumes:
- .:/app
- ./static_cdn/static_root/:/app/static_cdn/static_root
ports:
- "9010:9010"
depends_on:
- db
db:
image: postgres
container_name: "koober-postgres-db"
Dockerfile
FROM ubuntu:18.04
# -- Install Pipenv:
FROM python:3
ENV PYTHONUNBUFFERED 1
ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
# -- Install Application into container:
RUN set -ex && mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
# -- Adding dependencies:
ADD . /app/
But it is not loading static files.

You need to have a shared volume to the STATIC_ROOT directory so that your nginxcontainer can reverse proxy to both web server and static files generated by your web server.
In docker-compose.yml:
services:
nginx:
image: nginx:alpine
volumes:
- ./static_cdn/static_root/:/static
ports:
- 80:80
web:
build: .
volumes:
- ./static_cdn/static_root/:/app/static_cdn/static_root
Now in your nginx.conf add:
location /static/ {
alias /static/;
}

Nginx Dockerfile:
FROM nginx:stable-alpine
COPY default.conf /etc/nginx
COPY default.conf /etc/nginx/conf.d
EXPOSE 80
default.conf referred to in the above Dockerfile:
server {
listen 80 default_server;
server_name _;
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /app/static/;
}
location /media/ {
alias /app/static/;
}
}
Note: The default.conf and Dockerfile mentioned above is under the same folder, build that image and use it as the Nginx image in the below docker-compose file.
The docker-compose file would look like this:
version: '3.8'
services:
web:
image: <django-image-name>
command: gunicorn --bind 0.0.0.0:8000 licensing_platform.wsgi --workers=4
volumes:
- static_volume:/app/static
- media_volume:/app/media
expose:
- "8000"
networks:
- django-network
nginx:
image: <nginx-image-name>
restart: always
volumes:
- static_volume:/app/static
- media_volume:/app/media
ports:
- "80:80"
depends_on:
- web
networks:
- django-network
networks:
django-network:
name: django-network
volumes:
media_volume:
static_volume:
The app/ path referenced depends on the working directory:
The django application Dockerfile would start with:
FROM ubuntu:20.04
ADD . /app
WORKDIR /app
EXPOSE 8000
Code reference: https://github.com/addu390/licensing-as-a-platform
The code reference mentioned above is an open-source project I'm working on with no commercial benefits.

Related

Getting forbidden error while accessing a directory in using nginx inside docker

I am using Docker version 4.14 to build a Django application and Nginx as a web service.
I have two Dockerfiles, one for nginx and one for the Django app, The Django workflow and API serving are fine with nginx, and even static files are served well through nginx.
I use Django logging to log the exceptions in the app to a directory named logs, but when I try to access the directory through nginx, I get a 403 Forbidden error.
**nginx docker file**
FROM nginx:1.21-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
**nginx conf**
server {
listen 80;
include mime.types;
types {
text/plain log txt;
}
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /code/static/;
}
location /logs {
alias /code/logs/;
}
}
\*\*root dockerfile
\*\*
FROM python:3.10.2-slim-bullseye
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project
COPY . .
\*\*docker compose
\*\*
version: "3.9"
services:
web:
build: .
ports:
\- "8000:8000"
command: gunicorn tech_plotz.wsgi:application --bind 0.0.0.0:8000
volumes:
\- static_volume:/code/static
\- logs_volume:/code/logs
expose:
\- 8000
depends_on:
\- db
db:
image: postgres:13
volumes:
\- postgres_data:/var/lib/postgresql/data/
environment:
\- "POSTGRES_HOST_AUTH_METHOD=trust"
nginx:
build: ./nginx
ports:
\- 1337:80
volumes:
\- static_volume:/code/static
\- logs_volume:/code/logs
depends_on:
\- web
volumes:
postgres_data:
static_volume:
logs_volume:
how can i access the logs directory from the nginx?
In a shared Docker compose, shared directories need to be served by nginx.

Nginx frontend not calling Nginx in backend

So I am using Django + react with nginx both on backend and frontend, containerized in docker. The following image will clarify how I want to serve the whole application:
Having been googling but couldn't make sense of the solutions. Issue is that Nginx in frontend not connecting with nginx on backend on port 8082.
Following are docker, nginx and docker-compose files.
Nginx configurations for frontend:
upstream react {
server reactapp:3000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
root /usr/share/nginx/html;
}
location /add-to-waiting/ {
proxy_pass http://0.0.0.0:8082;
}
}
Dockerfile for react and nginx for frontend:
# build environment
FROM node as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
RUN npm i --silent
RUN npm install react-scripts#3.4.1 -g --silent
COPY . ./
RUN npm run build
# production environment
FROM nginx:stable-alpine
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
EXPOSE 443
CMD ["nginx", "-g", "daemon off;"]
docker-compose.yml for frontend:
services:
frontend:
build: .
ports:
- "8090:80"
container_name: irisfrontend
Nginx configurations for backend
upstream django {
server website:8000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
proxy_pass http://django;
}
location /media/ {
alias /app/media/;
}
location /static/ {
alias /app/forex/static/admin/;
}
}
Dockerfile for nginx in backend:
FROM nginx:1.19.0
COPY ./default.conf /etc/nginx/conf.d/default.conf
Dockerfile for gunicorn in backend:
FROM python:3
ADD requirements.txt /app/requirements.txt
ADD . /app
WORKDIR /app
EXPOSE 8000:8000
RUN pip install --upgrade pip && pip install -r /app/requirements.txt
RUN python manage.py collectstatic --no-input --settings=forex.settings.production
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "3", "forex.wsgi:application", "DJANGO_SETTINGS_MODULE=forex.settings.production"]
docker-compose.yml for backend:
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
- static:/app/forex/static/admin/
depends_on:
- db
nginx:
build: ./nginx
volumes:
- static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
volumes:
static:
What changes do I need to make to successfully do a post request from frontend nginx to backend nginx?
Include a network for both containers so that they can communicate.
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
-
static:/app/forex/static/admin/
depends_on:
- db
networks:
- nettest
nginx:
build: ./nginx
volumes:
-
static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
networks:
- nettest
volumes:
static:
networks:
nettest:

how to make static files works using django docker nginx and postgresql since its not serving them

When i try to access http://127.0.0.1:8000/admin, i get this.
my folder structure is :
django-react-nginx
|
|_ _ docker-compose.yml
|
> backend
|
|_ Dockerfile
|
|_ entrypoint.sh
> languages
|
|_ settings.py
> media
> static # This folder appears after running docker-compose -d --build
> nginx
|
|_ default.conf
|
|_ Dockerfile
now
here is the files
Django
settings.py
DEBUG = True
ALLOWED_HOSTS = ['*']
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
Docker file
FROM python:3.8
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /backend
COPY requirements.txt /backend/
RUN pip install -r requirements.txt && \
pip install --upgrade pip
COPY ./entrypoint.sh /
ENTRYPOINT ["sh", "/entrypoint.sh"]
entrypoint.sh
#!/bin/sh
python manage.py migrate --no-input
python manage.py collectstatic --no-input
gunicorn languages.wsgi:application --bind 0.0.0.0:8000
Nginx
default.conf
upstream django {
server backend:8000;
}
server {
listen 80;
location / {
proxy_pass http://django;
}
location /static/ {
autoindex on;
alias /backend/static;
}
location /media/ {
autoindex on;
alias /backend/static;
}
}
Dockerfile
FROM nginx:1.19.8-alpine
COPY ./default.conf /etc/nginx/conf.d/default.conf
Root Folder
Docker-compose
version: "3.7"
services:
backend:
build: ./backend/languages
stdin_open: true # interactive
tty: true # interactive
restart: "on-failure"
env_file:
.env
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/backend/static
ports:
- 8000:8000
networks:
- nginx_network
- db_network
depends_on:
- db
db:
image: postgres:11
restart: "on-failure"
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
networks:
- db_network
nginx:
build: ./nginx
restart: always
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
ports:
- 80:80
networks:
- nginx_network
depends_on:
- backend
networks:
nginx_network:
driver: bridge
db_network:
driver: bridge
volumes:
postgres_data:
As mentioned above i am not sure how to make the static files work, they been copied to the folder mentioned above because when i can see the folder created and when i check the logs i get this 165 static files copied to '/backend/static'.
Nginx config
Nginx config looks fine. Container is mapped to /backend/languages/static with /static and the alias points to the same folder inside container - /static.
nginx.conf
location /static/ {
autoindex on;
alias /static;
}
upd issue detected: an alias must have the same ending slash to work properly. so it has to be alias /static/
nginx in compose
nginx:
...
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
Django config
But Django config looks broken. Here you've configured to collect static to static folder within BASE_DIR
settings.py
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
and this is exactly what collectstatic is reporting to you:
165 static files copied to '/backend/static'
files are in the /backend/static, not /static. Hovewer the container is configured to root folder /static:
django in compose
backend
...
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
looks like the issue can be fixed by pointing to
volumes:
- ./backend/languages/static:/backend/static
Nevertheless there is still some work to do: media files are not supposed to be served by Django as well, so it is recommended to configure Nginx to serve media files too.
Django Dockerfile
I believe the mapping ./backend/languages:/backend works but what's the point of dockerizing then? There is no app in this docker image, just dependencies. It is "better" to include sources in the image so the deployment in the end would require to update images and restart containers only.
So, as a side note, I'd suggest to (at least try to):
update configs to serve media files with Nginx
include django app sources inside your docker image
collect static files during the build and not to include them into docker image
remove collectstatic from the entrypoint
supply static files and treat them as a separate product, think of them if you were going to deliver them to a CDN, a separate hosting - it is a very common solution
whilst they are still on the same hosting, keep mapping them to the container as you do now, but in a "reversed" way: deliver static files to the host folder, access them from the container in ro mode
You can use whitenoise by configuring it in your settings file. Set up Whitenoise with Django here

Nginx not serving media files. Dockerizing django/nginx/gunicorn/postgresql

I have a project running in 3 docker containers. One is for django project itself, another one for postgres and the third one for nginx. My static files are served just fine while all media files are not found. However every file which was uploaded is properly saved in web container in media folder.
Here is the docker-compose file:
version: '3.8'
volumes:
postgres_data:
static:
services:
db:
image: postgres:latest
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./.env
web:
build: .
restart: always
command: gunicorn foodgram.wsgi:application --bind 0.0.0.0:8000
volumes:
- ./static:/static
- ./media:/media
ports:
- "8000:8000"
depends_on:
- db
env_file:
- ./.env
nginx:
build:
context: .
dockerfile: nginx/Dockerfile
ports:
- "8080:80"
volumes:
- ./static:/etc/nginx/html/static
- ./media:/etc/nginx/html/media
depends_on:
- web
Here is the dockerfile:
FROM python:3.8.5
WORKDIR /code
COPY . /code
RUN pip install -r /code/requirements.txt
Here is the nginx.conf:
events {}
http {
include mime.types;
server {
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
location /static/ {
autoindex on;
root /etc/nginx/html/;
}
location /media/ {
autoindex on;
root /etc/nginx/html/;
}
}
}
and finally nginx dockerfile:
FROM nginx:latest
COPY nginx/nginx.conf /etc/nginx/nginx.conf
Error returned:
19#19: *1 open() "/etc/nginx/html/media/4.jpg" failed (2: No such file or directory)
There are no nested folders in media/static folders.

docker ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on

I'm new to Docker and setting up my first Django application using Docker
My application path looks like
app
|- helloworld
|- __init__.py
|- manage.py
|- static_cdn
|- static_root
|- config
|- nginx
|- nginx.conf
|- Dockerfile
|- docker-compose.yml
|- requirements.txt
|- start.sh
the contents of Docerfile
FROM ubuntu:18.04
# -- Install Pipenv:
FROM python:3
ENV PYTHONUNBUFFERED 1
ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
# -- Install Application into container:
RUN set -ex && mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
# -- Adding dependencies:
ADD . /app/
contents of docker-compose.yml
version: '3'
services:
nginx:
image: nginx:latest
ports:
- "9010:9010"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn:/static
depends_on:
- web
web:
build: .
command: ./start.sh
volumes:
- .:/app
- ./static_cdn:/static
ports:
- "9010:9010"
depends_on:
- db
expose:
- "9010"
db:
image: postgres
contents of config/nginx/nginx.conf
upstream web {
ip_hash;
server web:9010;
}
server {
location /static {
autoindex on;
alias /static/
}
location / {
proxy_pass http://127.0.0.1;
}
listen 9011;
server_name localhost;
}
contents of start.sh
#!/usr/bin/env bash
# Start Gunicorn processes
echo --: Starting application build
echo --: Creating migration
exec python3 manage.py makemigrations
echo ------: makemigrations complete
echo --: Running migration
exec python3 manage.py migrate
echo ------: migrate complete
echo --: Running collectstatic
exec python3 manage.py collectstatic
echo ------: collectstatic complete
echo Starting Gunicorn.
exec gunicorn helloworld.wsgi:application \
--bind 0.0.0.0:9010 \
--workers 3
Now, when I build using docker
docker-compose up --build
It gives error as
ERROR: for nginx Cannot start service nginx: driver failed
programming external connectivity on endpoint koober_nginx_1
(8ea5c084a7283a16afbf136a73dc4b27d9cae35fe14d735b83199ad5d0e03431):
Bind for 0.0.0.0:9010 failed: port is already allocated
I have followed few tutorials to create those Docker files and nginx conf file.
1. How can I solve above issue.
2. Do I need to use FROM ubuntu:18.04 with above configuration?
Edit 2
Now, it stuck after creating migration from start.sh commands
You can't allocate port 9010 of your host for both services.
This is what you're doing in the ports section of declaration of service nginx and web.
Moreover, by default nginx will listen to port 80 and 443 for https.
You can keep it like that and publish to a different port on your host. See how to use port keyword in docker-compose :
https://docs.docker.com/compose/compose-file/#ports
Maybe you want something more like that:
version: '3'
services:
nginx:
image: nginx:latest
ports:
- "10080:80"
- "10443:443"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn:/static
depends_on:
- web
web:
build: .
command: ./start.sh
container_name: "web-app"
volumes:
- .:/app
- ./static_cdn:/static
expose:
- "9010"
depends_on:
- db
db:
image: postgres
contents of config/nginx/nginx.conf
upstream web {
ip_hash;
server web-app:9010;
}
server {
location /static {
autoindex on;
alias /static/
}
location / {
proxy_pass http://web;
}
listen 80;
server_name localhost;
}
Concerning your last question, you could go for an official Python image from the Docker hub Python repository or start from any other base image like debian:jessie-slim from Debian official repository or keep the Ubuntu 18.04 image
For me, stopping/restarting the NGINX server worked.
To stop Nginx, use the following command:
sudo systemctl stop nginx
To start Nginx when it is stopped, use the following command:
sudo systemctl start nginx
To restart Nginx, use the following command:
sudo systemctl restart nginx
To reload Nginx after making configuration changes, use the following command:
sudo systemctl reload nginx