docker ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on - django

I'm new to Docker and setting up my first Django application using Docker
My application path looks like
app
|- helloworld
|- __init__.py
|- manage.py
|- static_cdn
|- static_root
|- config
|- nginx
|- nginx.conf
|- Dockerfile
|- docker-compose.yml
|- requirements.txt
|- start.sh
the contents of Docerfile
FROM ubuntu:18.04
# -- Install Pipenv:
FROM python:3
ENV PYTHONUNBUFFERED 1
ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
# -- Install Application into container:
RUN set -ex && mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
# -- Adding dependencies:
ADD . /app/
contents of docker-compose.yml
version: '3'
services:
nginx:
image: nginx:latest
ports:
- "9010:9010"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn:/static
depends_on:
- web
web:
build: .
command: ./start.sh
volumes:
- .:/app
- ./static_cdn:/static
ports:
- "9010:9010"
depends_on:
- db
expose:
- "9010"
db:
image: postgres
contents of config/nginx/nginx.conf
upstream web {
ip_hash;
server web:9010;
}
server {
location /static {
autoindex on;
alias /static/
}
location / {
proxy_pass http://127.0.0.1;
}
listen 9011;
server_name localhost;
}
contents of start.sh
#!/usr/bin/env bash
# Start Gunicorn processes
echo --: Starting application build
echo --: Creating migration
exec python3 manage.py makemigrations
echo ------: makemigrations complete
echo --: Running migration
exec python3 manage.py migrate
echo ------: migrate complete
echo --: Running collectstatic
exec python3 manage.py collectstatic
echo ------: collectstatic complete
echo Starting Gunicorn.
exec gunicorn helloworld.wsgi:application \
--bind 0.0.0.0:9010 \
--workers 3
Now, when I build using docker
docker-compose up --build
It gives error as
ERROR: for nginx Cannot start service nginx: driver failed
programming external connectivity on endpoint koober_nginx_1
(8ea5c084a7283a16afbf136a73dc4b27d9cae35fe14d735b83199ad5d0e03431):
Bind for 0.0.0.0:9010 failed: port is already allocated
I have followed few tutorials to create those Docker files and nginx conf file.
1. How can I solve above issue.
2. Do I need to use FROM ubuntu:18.04 with above configuration?
Edit 2
Now, it stuck after creating migration from start.sh commands

You can't allocate port 9010 of your host for both services.
This is what you're doing in the ports section of declaration of service nginx and web.
Moreover, by default nginx will listen to port 80 and 443 for https.
You can keep it like that and publish to a different port on your host. See how to use port keyword in docker-compose :
https://docs.docker.com/compose/compose-file/#ports
Maybe you want something more like that:
version: '3'
services:
nginx:
image: nginx:latest
ports:
- "10080:80"
- "10443:443"
volumes:
- .:/app
- ./config/nginx:/etc/nginx/conf.d
- ./static_cdn:/static
depends_on:
- web
web:
build: .
command: ./start.sh
container_name: "web-app"
volumes:
- .:/app
- ./static_cdn:/static
expose:
- "9010"
depends_on:
- db
db:
image: postgres
contents of config/nginx/nginx.conf
upstream web {
ip_hash;
server web-app:9010;
}
server {
location /static {
autoindex on;
alias /static/
}
location / {
proxy_pass http://web;
}
listen 80;
server_name localhost;
}
Concerning your last question, you could go for an official Python image from the Docker hub Python repository or start from any other base image like debian:jessie-slim from Debian official repository or keep the Ubuntu 18.04 image

For me, stopping/restarting the NGINX server worked.
To stop Nginx, use the following command:
sudo systemctl stop nginx
To start Nginx when it is stopped, use the following command:
sudo systemctl start nginx
To restart Nginx, use the following command:
sudo systemctl restart nginx
To reload Nginx after making configuration changes, use the following command:
sudo systemctl reload nginx

Related

Django static files are not found inside docker container

I am building a Django app with Docker. I run the command collectstatic in my entrypoint when database is ready. When I check my container, the /static/ folder is empty. Thus, Nginx cannot load the static files.
# settings.py
STATIC_URL = '/static/'
STATIC_ROOT = '/static/'
Here is my docker-compose file
version: "3.9"
services:
db:
image: postgis/postgis:14-3.3
container_name: db
volumes:
- ./data/db:/var/lib/postgresql/data
env_file:
- prod.env
backend:
container_name: backend
build:
dockerfile: ./django/Dockerfile
command: gunicorn api.wsgi:application --bind 0.0.0.0:8000
volumes:
- static:/usr/src/app/static
ports:
- "8000:8000"
env_file:
- prod.env
depends_on:
- db
nginx:
container_name: nginx
build:
dockerfile: ./nginx/Dockerfile
volumes:
- static:/usr/src/app/static
ports:
- "80:80"
depends_on:
- backend
restart: always
redis:
container_name: redis
restart: unless-stopped
image: redis:alpine
expose:
- 6379
worker:
container_name: worker
build:
dockerfile: ./django/Dockerfile
command: celery -A api worker -l INFO
volumes:
- static:/usr/src/app/static
env_file:
- prod.env
depends_on:
- db
- backend
- redis
volumes:
static:
My Nginx configuration:
upstream api {
server backend:8000;
}
server {
listen 80;
location / {
proxy_pass http://api;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /usr/src/app/static/;
}
}
backend Dockerfile:
# syntax=docker/dockerfile:1
FROM python:3
WORKDIR /usr/src/app
RUN apt-get update
RUN apt-get install -y libgdal-dev gdal-bin netcat
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
COPY /django/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY /django/django-entrypoint.sh /django-entrypoint.sh
RUN chmod +x /django-entrypoint.sh
COPY django /usr/src/app
ENTRYPOINT ["/django-entrypoint.sh"]
And the entrypoint:
#!/bin/sh
if [ "$POSTGRES_NAME" = "postgres" ]
then
echo "Waiting for Postgres..."
while ! nc -z $POSTGRES_HOST $POSTGRES_PORT; do
sleep 0.1
done
echo "PostgreSQL started"
fi
ls
# python manage.py flush --no-input
python manage.py migrate --no-input
python manage.py collectstatic --no-input
exec "$#"
In my files (local), I do not seem to see the '/static/' folder to be generated. How is this? I have check that static in backend and nginx by ssh in the container and the static folders were empty. In the logs, collectstatic was executed without an error with this message:
backend | 173 static files copied to '/static'.
You use a Docker named volume to hold the static files
volumes:
- static:/usr/src/app/static
# ^^^^^^
# a volume name, not a host path
This named volume only exists inside Docker's storage; you will not see its content on your host system or in your local source tree.
This isn't a problem for the setup you're describing here: since you're re-running collectstatic every time the container starts up, and the volume contents hide the image contents in this directory, there's no particular need for the files to exist in source control or your host filesystem. If you did need them, you could presumably run manage.py collectstatic in a non-Docker virtual environment.
Try by adding the following command in Dockerfile and re-build image.
RUN python manage.py collectstatic --noinput
You can place it after RUN pip install --no-cache-dir -r requirements.txt

Nginx frontend not calling Nginx in backend

So I am using Django + react with nginx both on backend and frontend, containerized in docker. The following image will clarify how I want to serve the whole application:
Having been googling but couldn't make sense of the solutions. Issue is that Nginx in frontend not connecting with nginx on backend on port 8082.
Following are docker, nginx and docker-compose files.
Nginx configurations for frontend:
upstream react {
server reactapp:3000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
root /usr/share/nginx/html;
}
location /add-to-waiting/ {
proxy_pass http://0.0.0.0:8082;
}
}
Dockerfile for react and nginx for frontend:
# build environment
FROM node as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
RUN npm i --silent
RUN npm install react-scripts#3.4.1 -g --silent
COPY . ./
RUN npm run build
# production environment
FROM nginx:stable-alpine
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
EXPOSE 443
CMD ["nginx", "-g", "daemon off;"]
docker-compose.yml for frontend:
services:
frontend:
build: .
ports:
- "8090:80"
container_name: irisfrontend
Nginx configurations for backend
upstream django {
server website:8000;
}
server {
listen 80;
client_max_body_size 100M;
proxy_set_header X-Forwarded-Proto $scheme;
location / {
proxy_pass http://django;
}
location /media/ {
alias /app/media/;
}
location /static/ {
alias /app/forex/static/admin/;
}
}
Dockerfile for nginx in backend:
FROM nginx:1.19.0
COPY ./default.conf /etc/nginx/conf.d/default.conf
Dockerfile for gunicorn in backend:
FROM python:3
ADD requirements.txt /app/requirements.txt
ADD . /app
WORKDIR /app
EXPOSE 8000:8000
RUN pip install --upgrade pip && pip install -r /app/requirements.txt
RUN python manage.py collectstatic --no-input --settings=forex.settings.production
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "3", "forex.wsgi:application", "DJANGO_SETTINGS_MODULE=forex.settings.production"]
docker-compose.yml for backend:
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
- static:/app/forex/static/admin/
depends_on:
- db
nginx:
build: ./nginx
volumes:
- static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
volumes:
static:
What changes do I need to make to successfully do a post request from frontend nginx to backend nginx?
Include a network for both containers so that they can communicate.
services:
db:
build: ./db
website:
build:
context: .
dockerfile: Dockerfile.app
env_file:
- env
container_name: website_container_8
volumes:
-
static:/app/forex/static/admin/
depends_on:
- db
networks:
- nettest
nginx:
build: ./nginx
volumes:
-
static:/app/forex/static/admin/
ports:
- "8082:80"
depends_on:
- website
networks:
- nettest
volumes:
static:
networks:
nettest:

nginx: [emerg] host not found in upstream when dockerizing a django/react project

I'm trying to dockerize a django/react project but i'm running into this error when running docker-compose up.
i don't understand where the error come from.
i'm new to docker.
my goal is to deploy the frontend and the backend separately in one server.
nginx: [emerg] host not found in upstream
[emerg] 1#1: host not found in upstream "backend:8000" in /etc/nginx/conf.d/default.conf:2
nginx_1 | nginx: [emerg] host not found in upstream "backend:8000" in /etc/nginx/conf.d/default.conf:2
here is my code
django's Dockerfile
ENV DJANGO_SECRET_KEY $DJANGO_SECRET_KEY
ENV DJANGO_CORS_ORIGIN_WHITELIST $DJANGO_CORS_ORIGIN_WHITELIST
RUN mkdir /backend
WORKDIR /backend
COPY requirements.txt /backend/
EXPOSE 8000
RUN pip install -r requirements.txt
COPY . /backend/
RUN python manage.py makemigrations
RUN python manage.py migrate
react's Dockerfile
FROM node
USER root
WORKDIR /frontend
COPY . /frontend
ARG API_URL
ENV REACT_APP_HOST_IP_ADDRESS $API_URL
RUN yarn
RUN yarn config set ignore-engines true
RUN yarn build
server's config
upstream api {
server backend:8000;
}
server {
listen 8080;
location /api/ {
proxy_pass http://api$request_uri;
}
# ignore cache frontend
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend;
try_files $uri $uri/ /index.html;
}
}
docker-copose.yml
version: '3'
services:
backend:
build:
context: ./ruelle_backend
args:
DJANGO_ALLOWED_HOSTS: http://ruelle-online.fr
DJANGO_SECRET_KEY: anis1807
DJANGO_CORS_ORIGIN_WHITELIST: http://ruelle-online.fr
command: gunicorn ivan_backend.wsgi --bind 0.0.0.0:8000
ports:
- "8000:8000"
frontend:
build:
context: ./ruelle_frontend
args:
API_URL: http://ruelle-online.fr
volumes:
- build_folder:/frontend/build
nginx:
image: nginx:latest
ports:
- 80:8080
volumes:
- ./webserver/nginx-proxy.conf:/etc/nginx/conf.d/default.conf:ro
- build_folder:/var/www/frontend
depends_on:
- backend
- frontend
volumes:
build_folder:

Django Nginx Docker Gunicorn cant reach

Sorry if it the answer seems obvious, but I've been bashing my head for the past couple of hours.
I've been following multiple tutorials trying to dockerize my application. No matter what combination of url:port or just url I tried I can't access the pages.
I have zero clue what am I doing wrong. I am assuming the following:
NGINX config plain wrong. How can upstream web know what's web? I assume docker exposes it, but unsure whether this is correct.
web container not exposing address and port properly?
I tried multipled settings, but non work.
I have the following:
Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
ADD ./django.conf /etc/nginx/conf.d/default.conf
#COPY ./django.conf /etc/nginx/sites-available/
#RUN ln -s /etc/nginx/sites-available/django.conf /etc/nginx/sites-enabled
docker-compose.yml
version: '3'
services:
db:
image: postgres:9.6
ports:
- "5432:5432"
environment:
- POSTGRES_USER=random_user
- POSTGRES_PASSWORD=random_password
redis:
restart: always
image: redis:4.0
expose:
- "6379"
nginx:
image: nginx:1.14.0
container_name: nginx01
ports:
- "8000:8000"
volumes:
- ./code
depends_on:
- web
web:
build: .
container_name: backend-api
command: bash -c "python manage.py migrate && python manage.py collectstatic --noinput && gunicorn ops4_backend.wsgi -b 0.0.0.0:8000"
depends_on:
- db
volumes:
- ./code
expose:
- "8000"
restart: always
django.conf
upstream web {
ip_hash;
server web:8000;
}
# portal
server {
location / {
proxy_pass http://web:8000;
}
listen 8000;
server_name localhost;
location /static {
autoindex on;
alias /src/static/;
}
}
docker ps -a
I was doing almost the same task yesterday, and the only valuable difference I see is that you include django.conf in Django container, not in nginx one. I have following volume in nginx section of docker-compose.yml:
- ./nginx:/etc/nginx/conf.d (where ./nginx is the folder with django.conf)
EDIT: here is my docker-compose.yml:
version: '3'
services:
nginx:
image: nginx
restart: always
ports:
- "80:8000"
volumes:
- ./nginx:/etc/nginx/conf.d
- ./static_cdn:/static
depends_on:
- web
db:
image: postgres
restart: always
web:
build: .
restart: always
command: bash -c "python3 manage.py makemigrations && python3 manage.py migrate && python3 manage.py collectstatic --no-input && gunicorn core.wsgi -b 0.0.0.0:8000"
volumes:
- .:/code
expose:
- "8000"
depends_on:
- db
django.conf:
upstream web {
ip_hash;
server web:8000;
}
server {
location /static/ {
alias /static/;
}
location / {
proxy_pass http://web/;
}
listen 8000;
server_name localhost;
}
With this setup nginx is available at 127.0.0.1:80 or just 127.0.0.1 as 80 is default http port.
There are so many wrong things, please use this configuration. and then change it step by step toward your specific needs.
also checkout this page that talks about Nginx Docker Configurations because one of your main problems is that you are not exposing django nginx conf that you make:
ADD ./django.conf /etc/nginx/conf.d/default.conf
to nginx container.
so nginx does not know how to map your configs.

How to setup nginx and django with docker-compose?

I'm having a situation while setting up Django and all the dependencies I need with Docker (docker-toolbox, docker-compose).
I'm encountering an error while I'm trying to access to my url http://192.168.99.100:8000/ which says 502 Bad Gateway (nginx/1.13.1). For this error I don't really understand from where it comes since it's the first time I'm using Django with nginx on Docker.
docker-compose.yml :
version: '2'
services:
nginx:
image: nginx:latest
container_name: nz01
ports:
- "8000:8000"
volumes:
- ./src:/src
- ./config/nginx:/etc/nginx/conf.d
- /static:/static
depends_on:
- web
web:
...
...
Dockerfile :
FROM python:latest
ENV PYTHONUNBUFFERED 1
#ENV C_FORCE_ROOT true
ENV APP_USER myapp
ENV APP_ROOT /src
RUN mkdir /src;
RUN groupadd -r ${APP_USER} \
&& useradd -r -m \
--home-dir ${APP_ROOT} \
-s /usr/sbin/nologin \
-g ${APP_USER} ${APP_USER}
WORKDIR ${APP_ROOT}
RUN mkdir /config
ADD config/requirements.pip /config/
RUN pip install -r /config/requirements.pip
USER ${APP_USER}
ADD . ${APP_ROOT}
config/nginx/... .conf
upstream web {
ip_hash;
server web:8000;
}
server {
location /static/ {
autoindex on;
alias /static/;
}
location / {
proxy_pass http://web/;
}
listen 8000;
server_name localhost;
}
Is there something I'm doing wrong ?
The issue here is with gunicorn command line argument, missing -b flag. It needs to be launched with -b flag for binding with the address specified. In your case gunicorn binds itself to default 127.0.0.1:8000 which isn't accessible to other containers. So, just change the command for web to following:
command: bash -c 'python manage.py makemigrations && python manage.py migrate && gunicorn oqtor.wsgi -b 0.0.0.0:8000'
That should make your web app accessible via nginx.