Getting forbidden error while accessing a directory in using nginx inside docker - django

I am using Docker version 4.14 to build a Django application and Nginx as a web service.
I have two Dockerfiles, one for nginx and one for the Django app, The Django workflow and API serving are fine with nginx, and even static files are served well through nginx.
I use Django logging to log the exceptions in the app to a directory named logs, but when I try to access the directory through nginx, I get a 403 Forbidden error.
**nginx docker file**
FROM nginx:1.21-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
**nginx conf**
server {
listen 80;
include mime.types;
types {
text/plain log txt;
}
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /code/static/;
}
location /logs {
alias /code/logs/;
}
}
\*\*root dockerfile
\*\*
FROM python:3.10.2-slim-bullseye
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project
COPY . .
\*\*docker compose
\*\*
version: "3.9"
services:
web:
build: .
ports:
\- "8000:8000"
command: gunicorn tech_plotz.wsgi:application --bind 0.0.0.0:8000
volumes:
\- static_volume:/code/static
\- logs_volume:/code/logs
expose:
\- 8000
depends_on:
\- db
db:
image: postgres:13
volumes:
\- postgres_data:/var/lib/postgresql/data/
environment:
\- "POSTGRES_HOST_AUTH_METHOD=trust"
nginx:
build: ./nginx
ports:
\- 1337:80
volumes:
\- static_volume:/code/static
\- logs_volume:/code/logs
depends_on:
\- web
volumes:
postgres_data:
static_volume:
logs_volume:
how can i access the logs directory from the nginx?
In a shared Docker compose, shared directories need to be served by nginx.

Related

Nginx frontend not calling Nginx in backend

So I am using Django + react with nginx both on backend and frontend, containerized in docker. The following image will clarify how I want to serve the whole application:
Having been googling but couldn't make sense of the solutions. Issue is that Nginx in frontend not connecting with nginx on backend on port 8082.
Following are docker, nginx and docker-compose files.
Nginx configurations for frontend:
upstream react {
server reactapp:3000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
root /usr/share/nginx/html;
}
location /add-to-waiting/ {
proxy_pass http://0.0.0.0:8082;
}
}
Dockerfile for react and nginx for frontend:
# build environment
FROM node as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
RUN npm i --silent
RUN npm install react-scripts#3.4.1 -g --silent
COPY . ./
RUN npm run build
# production environment
FROM nginx:stable-alpine
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
EXPOSE 443
CMD ["nginx", "-g", "daemon off;"]
docker-compose.yml for frontend:
services:
frontend:
build: .
ports:
- "8090:80"
container_name: irisfrontend
Nginx configurations for backend
upstream django {
server website:8000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
proxy_pass http://django;
}
location /media/ {
alias /app/media/;
}
location /static/ {
alias /app/forex/static/admin/;
}
}
Dockerfile for nginx in backend:
FROM nginx:1.19.0
COPY ./default.conf /etc/nginx/conf.d/default.conf
Dockerfile for gunicorn in backend:
FROM python:3
ADD requirements.txt /app/requirements.txt
ADD . /app
WORKDIR /app
EXPOSE 8000:8000
RUN pip install --upgrade pip && pip install -r /app/requirements.txt
RUN python manage.py collectstatic --no-input --settings=forex.settings.production
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "3", "forex.wsgi:application", "DJANGO_SETTINGS_MODULE=forex.settings.production"]
docker-compose.yml for backend:
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
- static:/app/forex/static/admin/
depends_on:
- db
nginx:
build: ./nginx
volumes:
- static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
volumes:
static:
What changes do I need to make to successfully do a post request from frontend nginx to backend nginx?
Include a network for both containers so that they can communicate.
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
-
static:/app/forex/static/admin/
depends_on:
- db
networks:
- nettest
nginx:
build: ./nginx
volumes:
-
static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
networks:
- nettest
volumes:
static:
networks:
nettest:

configuring django with nginx gunicorn on docker

Good day, I am new to docker and I have a Django app I will like to dockerize, I have searched for tutorials to help me set up my Django app with docker, I followed this article here on test-driven https://testdriven.io/blog/dockerizing-django-with-postgres-gunicorn-and-nginx/. I get issues making nginx work with my app. here is my code.
my apps docker file:
FROM python:3.8-alpine
ENV PATH="/scripts:${PATH}"
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN mkdir /app
COPY ./testdocker /app
WORKDIR /app
COPY ./scripts /scripts
RUN chmod +x /scripts/*
RUN mkdir -p /vol/web/media
RUN mkdir -p /vol/web/static
RUN adduser -D user
RUN chown -R user:user /vol
RUN chmod -R 755 /vol/web
USER user
nginx docker file:
FROM nginx:1.19.3-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY ./nginx.conf /etc/nginx/conf.d
RUN mkdir -p /vol/static
my docker-compose file:
version: '3.7'
services:
app:
build:
context: .
command: sh -c "gunicorn testdocker.wsgi:application --bind 0.0.0.0:8000"
volumes:
- static_data:/vol/web
expose:
- "8000"
environment:
- SECRET_KEY=MANME1233
- ALLOWED_HOSTS=127.0.0.1, localhost
nginx:
build:
context: ./nginx
volumes:
- static_data:/vol/static
ports:
- "8080:80"
depends_on:
- app
volumes:
static_data:
my nginx conf file:
upstream testapp {
server app:8000;
}
server {
listen 8080;
server_name app;
location / {
proxy_pass http://testapp;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static {
alias /vol/static;
}
}
I can't seem to get nginx to reverse proxy to my web app, upon opening the URL on the browser I get a 404 bad request or address not found. please what am I doing wrong or not doing right?.
#victormazeli It looks like you missed placing your services within the same docker network and I see some misconfiguration in nginx conf file. Try updating your docker-compose.yml as follows:
version: '3.7'
services:
app:
build:
context: .
command: sh -c "gunicorn testdocker.wsgi:application --bind 0.0.0.0:8000"
volumes:
- static_data:/vol/web
expose:
- "8000"
environment:
- SECRET_KEY=MANME1233
- ALLOWED_HOSTS=127.0.0.1, localhost
networks:
- main
nginx:
build:
context: ./nginx
volumes:
- static_data:/vol/static
ports:
- "8080:80"
depends_on:
- app
networks:
- main
volumes:
static_data:
networks:
main:
Then, update your nginx config as follows:
server {
server_name nginx;
listen 80;
location /static {
alias /vol/static;
}
location / {
proxy_pass http://app:8000/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
}
Another thing to keep in mind here is that you have 2 targets that are being served by the NGINX reverse-proxy:
Django project located in testdocker which should be accessible via localhost:8080
Static file data which is accessible via localhost:8080/static/[relative_path]
To access the static data, you will need the path relative to /vol/static in nginx service (which is a docker volume mount also mounted to /vol/web in app service). According to app's Dockerfile, the static_data volume should contain 2 directories: media and static. Therefore, if you have say an index.html located in directory /vol/web/static in app service, it should be accessible via localhost:8080/static/static/index.html.
Give this a try and let me know how it works out for you.

serving Django static files with Docker, nginx and gunicorn

I am setting us a Django 2.0 application with Docker, nginx and gunicorn.
It's running the server but static files are not working.
Here is the settings.py content
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static_my_project')
]
STATIC_ROOT = os.path.join(BASE_DIR, 'static_cdn', 'static_root')
While developing, I put my static files inside static_my_project, which on running collectstatic copies to static_cdn/static_root
The directory structure is like
app
|- myapp
|- settings.py
|- static_my_project
|- static_cdn
|- static_root
|- config
|- nginx
|- nginx.conf
|- manage.py
|- Docker
|- docker-compose.yml
on running
docker-compose up --build
on running collectstatic, it gives path where static files will be copied
koober-dev | --: Running collectstatic
koober-dev |
koober-dev | You have requested to collect static files at the destination
myapp-dev | location as specified in your settings:
myapp-dev |
myapp-dev | /app/static_cdn/static_root
myapp-dev |
myapp-dev | This will overwrite existing files!
myapp-dev | Are you sure you want to do this?
myapp-dev |
myapp-dev | Type 'yes' to continue, or 'no' to cancel:
myapp-dev | 0 static files copied to '/app/static_cdn/static_root', 210 unmodified.
the config/nginx/nginx.conf file contains following settings
upstream web {
ip_hash;
server web:9010;
}
server {
location /static {
autoindex on;
alias /static/;
}
location / {
proxy_pass http://web;
}
listen 10080;
server_name localhost;
}
docker-compose.yml
version: '3'
services:
nginx:
image: nginx:latest
container_name: "koober-nginx"
ports:
- "10080:80"
- "10443:43"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn/static_root/:/static
depends_on:
- web
web:
build: .
container_name: "koober-dev"
command: ./start.sh
volumes:
- .:/app
- ./static_cdn/static_root/:/app/static_cdn/static_root
ports:
- "9010:9010"
depends_on:
- db
db:
image: postgres
container_name: "koober-postgres-db"
Dockerfile
FROM ubuntu:18.04
# -- Install Pipenv:
FROM python:3
ENV PYTHONUNBUFFERED 1
ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
# -- Install Application into container:
RUN set -ex && mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
# -- Adding dependencies:
ADD . /app/
But it is not loading static files.
You need to have a shared volume to the STATIC_ROOT directory so that your nginxcontainer can reverse proxy to both web server and static files generated by your web server.
In docker-compose.yml:
services:
nginx:
image: nginx:alpine
volumes:
- ./static_cdn/static_root/:/static
ports:
- 80:80
web:
build: .
volumes:
- ./static_cdn/static_root/:/app/static_cdn/static_root
Now in your nginx.conf add:
location /static/ {
alias /static/;
}
Nginx Dockerfile:
FROM nginx:stable-alpine
COPY default.conf /etc/nginx
COPY default.conf /etc/nginx/conf.d
EXPOSE 80
default.conf referred to in the above Dockerfile:
server {
listen 80 default_server;
server_name _;
location / {
proxy_pass http://web:8000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /app/static/;
}
location /media/ {
alias /app/static/;
}
}
Note: The default.conf and Dockerfile mentioned above is under the same folder, build that image and use it as the Nginx image in the below docker-compose file.
The docker-compose file would look like this:
version: '3.8'
services:
web:
image: <django-image-name>
command: gunicorn --bind 0.0.0.0:8000 licensing_platform.wsgi --workers=4
volumes:
- static_volume:/app/static
- media_volume:/app/media
expose:
- "8000"
networks:
- django-network
nginx:
image: <nginx-image-name>
restart: always
volumes:
- static_volume:/app/static
- media_volume:/app/media
ports:
- "80:80"
depends_on:
- web
networks:
- django-network
networks:
django-network:
name: django-network
volumes:
media_volume:
static_volume:
The app/ path referenced depends on the working directory:
The django application Dockerfile would start with:
FROM ubuntu:20.04
ADD . /app
WORKDIR /app
EXPOSE 8000
Code reference: https://github.com/addu390/licensing-as-a-platform
The code reference mentioned above is an open-source project I'm working on with no commercial benefits.

Deploy to docker with nginx, django, daphne

I want to deploy my service to docker.
and my service is developed using python+django and django-channels
── myproject
├── myproject
│ ├── settings.py
│ ├── urls.py
│ ├── asgi.py
│ ├── ...
├── collected_static
│ ├── js
│ ├── css
│ ├── ...
├── nginx
│ ├── Dockerfile
│ ├── service.conf
├── requirements.txt
├── manage.py
├── Dockerfile
└── docker-compose.yml
myproject/Dockerfile :
FROM python
ENV PYTHONUNBURRERED 1
RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject
RUN pip install -r requirements.txt
RUN python manage.py migrate
myproject/docker-compose.yml:
version: '2'
services:
nginx:
build: ./nginx
networks:
- front
- back
ports:
- "80:80"
depends_on:
- daphne
redis:
image: redis
networks:
- "back"
ports:
- "6379:6379"
worker:
build: .
working_dir: /opt/myproject
command: bash -c "python manage.py runworker"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
daphne:
build: .
working_dir: /opt/myproject
command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
ports:
- "8000:8000"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
networks:
front:
back:
myproject/nginx/Dockerfile
FROM nginx
COPY service.conf /etc/nginx/sites-enabled/
myproject/nginx/service.conf
server {
listen 80;
server_name example.com #i just want to hide domain name..
charset utf-8;
client_max_body_size 20M;
location /static/ {
alias /opt/myproject/collected_static/;
}
location / {
proxy_pass http://0.0.0.0:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
and i write a command docker-compose up -d, nginx and daphne work well.
but when i connected to example.com:80, i just can see nginx default page.
and when i connected to example.com:8000, i just can see myproject's service page. (but cannot see static files)
I want to link nginx and daphne service! what should I do? please help me.
when i just deploy with nginx+daphne+django without docker, my service works well.
TLDR;
Nginx is not configured correctly, but also your docker-compose needs some correction:
Nginx
The Nginx website has some helpful tips for deploying with Docker that you should read, including a sample, very simple Dockerfile:
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY content /usr/share/nginx/html
COPY conf /etc/nginx
which points to some improvements you need to make (see the Docker Compose section for further help with Docker).
Bearing in mind the updates to deployment that we will make below, you will also need to change your Nginx config:
rename service.conf -> service.template
change listen ${NGINX_PORT};
change server_name ${NGINX_HOST};
change proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};
Docker Compose
Now your Nginx configuration is correct, you need to setup the docker compose directives correctly, thankfully, the Docker Hub Nginx page has an example for docker compose:
Here is an example using docker-compose.yml:
web:
image: nginx
volumes:
- ./mysite.template:/etc/nginx/conf.d/mysite.template
ports:
- "8080:80"
environment:
- NGINX_HOST=foobar.com
- NGINX_PORT=80
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
The mysite.template file may then contain variable references like this:
listen ${NGINX_PORT};
From r00m's answer
You can make all those improvements, and in fact, without sharing the volumes your static files won't be served correctly.
Create an image for the project and re-use it
Add the Volume references to allow static files to be shared
OPTIONAL: you should also follow the advice about collecting the static files, but your project structure kind of suggests that you've already done that.
Bringing it all together
Finally, we can merge those three improvements to give us the following setup:
myproject/Dockerfile:
FROM python
ENV PYTHONUNBUFFERED 1
RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject
RUN pip install -r requirements.txt
RUN python manage.py migrate # Can this be done during build? i.e. no link to the DB?
VOLUME ["/opt/myproject/collected_static"]
myproject/docker-compose.yml:
version: '2'
services:
nginx:
build: ./nginx
networks:
- front
- back
ports:
- "80:80"
volumes_from:
- "daphne"
environment:
- NGINX_HOST=example.com
- NGINX_PORT=80
- DAPHNE_HOST=daphne
- DAPHEN_PORT=8000
depends_on:
- daphne
links:
- daphne
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/service.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
redis:
image: redis
networks:
- "back"
ports:
- "6379:6379"
daphne:
build: .
image: "myproject:latest"
working_dir: /opt/myproject
command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
ports:
- "8000:8000"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
worker:
image: "myproject:latest"
working_dir: /opt/myproject
command: bash -c "python manage.py runworker"
environment:
- REDIS_HOST=redis
networks:
- front
- back
depends_on:
- redis
links:
- redis
networks:
front:
back:
myproject/nginx/Dockerfile
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY service.template /etc/nginx/conf.d
myproject/nginx/service.template
server {
listen ${NGINX_PORT};
server_name ${NGINX_HOST}
charset utf-8;
client_max_body_size 20M;
location /static/ {
alias /opt/myproject/collected_static/;
}
location / {
proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
Final thoughts
I'm not sure what you're trying to achieve with your network directives, but it almost certainly doesn't achieve it, for example nginx shouldn't connect into your backend network (I think...).
You need to consider whether "migrate" should be done at build time or run time.
Do you need to be able to change your nginx configuration easily? If so, you should remove the COPY from the nginx build and add in the volumes directive from the Docker Compose section.
You have misconfigured NGINX. Try proxy_pass http://127.0.0.1:8000;
As for the static files, it's because you haven't made the files available to the container. I would suggest the following modifications:
myproject/Dockerfile:
[...]
ADD . /opt/myproject
VOLUME ["/opt/myproject/collected_static"]
[..]
# may I also suggest automatic static file collection?
RUN python manage.py collectstatic --noinput
myproject/docker-compose.yml:
[...]
build: ./nginx
volumes_from:
- "worker" # or daphne
I would also consider adding image option to daphne and worker services. This will tag the image and allow to reuse it, thus it will be only built once (instead of twice).
myproject:
build: .
image: "myproject:latest"
[..]
worker:
image: "myproject:latest"
[..]
daphne:
image: "myproject:latest"

Django and nginx with docker compose - static files are not loaded

I know that several similar questions have been asked, but I have not managed to find a proper answer anywhere.
I want to dockerize my Django app. The app works fine in the docker container but the static files (images, css etc.) are not loaded.
My docker-compose.yml:
version: '2'
services:
web:
build: .
command: ./start.sh
volumes:
- .:/app
- /app/www/static
ports:
- "8000:8000"
env_file: .env
nginx:
build: ./nginx
links:
- web
ports:
- "0.0.0.0:80:80"
volumes_from:
- web
The start.sh script
python manage.py makemigrations --noinput
python manage.py migrate --noinput
python manage.py collectstatic --noinput
exec gunicorn audiobridge.wsgi:application --bind 0.0.0.0:8000 --workers 3
The Dockerfile for the web container:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
ADD . /app/
Then I have an nginx/ directory in my project which has the following Dockerfile:
FROM nginx
COPY sites-enabled/audiobridge /etc/nginx/sites-enabled/
and inside nginx/sites-enabled there is the nginx configuration file:
server {
listen 80;
server_name 127.0.0.1;
charset utf-8;
location /static {
alias /app/www/static/;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
enter image description here
volumes:
- "../nginx/conf.d:/etc/nginx/conf.d/templates"
- "/static:/static"
Did you try add this code to nginx and django app?