host not found in upstream when i use nginx and docker-compose - django

this is my docker-compose.yml file
i want to make a nginx to reverseproxy and django to webserver
and i also separate them each container
version: '2'
services:
django:
build: ./django
container_name: django
nginx:
restart: always
build: ./nginx
container_name: reversproxy
ports:
- "7891:7891"
depends_on:
- django
and follwoing is my nginx.conf
i set the uwsgi_pass to uwsgicluster, upstream to django containter
but nginx container doesn't work with error
nginx: [emerg] host not found in upstream "django:7893"
worker_processes 1;
events { worker_connections 1024; }
http {
sendfile on;
upstream uwsgicluster {
server django:7893;
}
server {
listen 7891;
location / {
include /etc/nginx/uwsgi_params;
uwsgi_pass uwsgicluster;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
}
how can i solve that problem??

You should define links in docker compose file to link django to nginx, or nginx will not be able to access django container.
links
Link to containers in another service. Either specify both the service
name and a link alias ("SERVICE:ALIAS"), or just the service name.
Official document explain

Related

Cannot access Strapi in a container using AWS Lightsail, Ubuntu, Docker and NGINX

I'm trying to practice and I have created an instance using AWS Lightsail with Ubuntu 20.04 LTS, also I've installed Docker and Docker Compose.
I would like to run this applications:
MariaDB
Strapi CMS
Let's assume that my public IP is: 1.20.20.20
When I try to access "http://1.20.20.20:1337", I receives "This site can’t be reached".
I thought that maybe I needed a web server and I have included NGINX in my docker-compose configuration.
My folders in the instance:
envs/strapi.env
nginx/strapi.conf
docker-compose.yml
docker-compose.yml
version: '3.8'
services:
nginx:
image: nginx:stable
container_name: nginx
volumes:
- ./nginx/strapi.conf:/etc/nginx/conf.d/default.conf
ports:
- 80:80
restart: on-failure
depends_on:
- strapi
db:
image: mariadb:10.6.4
container_name: db
restart: always
env_file:
- envs/db.env
volumes:
- db-data:/var/lib/mysql
networks:
- apps
strapi:
image: strapi/strapi:3.6.8
container_name: strapi
env_file:
- envs/strapi.env
restart: always
depends_on:
- db
ports:
- 1337:1337
networks:
- apps
volumes:
db-data:
networks:
apps:
driver: bridge
nginx/strapi.conf (I have copied the configuration from Strapi docs):
server {
# Listen HTTP
listen 1337;
server_name 1.20.20.20;
# Static Root
location / {
root /var/www/html;
}
# Strapi API
location /api/ {
rewrite ^/api/?(.*)$ /$1 break;
proxy_pass http://strapi;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
# Strapi Admin
location /admin {
proxy_pass http://strapi/admin;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
}
Still cannot access the CMS using my browser.
For some reason, when I do "docker ps -a", the status of the nginx container is always Restarting....
I have tried to follow and read steps that I have found on a few articles without luck. I don't have a lot of experience using NGINX and Ubuntu servers.
My goal is to be able to access the web app (Strapi CMS) through my browser.
Any help would be appreciated.

wsgi.url_scheme http in docker using nginx

I am using Apache on CentOS. On this server I have a Django project with docker. There are two containers in docker (nginx and python).
In the Apache I have .conf that have proxy to nginx container that is exposed on port 803. SSL is set in the Apache conf as well.
ProxyPreserveHost On
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Scheme "https"
ProxyPass / http://127.0.0.1:803/
ProxyPassReverse / http://127.0.0.1:803/
On the docker I have app.conf for nginx that looks like this:
upstream project {
server project-python:5000;
}
server {
listen 80;
server_name _;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
client_max_body_size 64M;
location / {
gzip_static on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Scheme "https";
proxy_set_header X-Forwarded-Proto "https";
proxy_set_header X-Forwarded-Protocol "ssl";
proxy_set_header X-Forwarded-Ssl=on;
proxy_set_header Host $host;
proxy_pass http://project;
proxy_redirect off;
}
}
In the Dockerfile Python is exposed on port 5000 and in the docker-compose.prod.yml file for production the python is started with gunicorn with this command:
gunicorn project.wsgi:application --preload --bind 0.0.0.0:5000
So I have two issues.
In the Django when I dump request.META I got wsgi.url_scheme that is http.
The second one is that I don't even understand how nginx is communicating with gunicorn because when I set app.conf to be just like below it is working also. How the nginx know that Python is exposed on port 5000.
server {
listen 80;
server_name _;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
client_max_body_size 64M;
location / {
proxy_pass http://project;
proxy_redirect off;
}
}
docker-compose.yml
version: '3'
services:
project-python:
build:
context: .
dockerfile: docker/python/Dockerfile
container_name: project-python
volumes:
- .:/var/www:rw
- .aws:/home/www/.aws
project-nginx:
build:
context: docker/nginx
dockerfile: Dockerfile
container_name: project-nginx
ports:
- "127.0.0.1:803:80"
depends_on:
- project-python
docker-compose.prod.yml
version: '3'
services:
project-python:
restart: unless-stopped
env_file:
- ./.env.prod
command: gunicorn project.wsgi:application --preload --bind 0.0.0.0:5000
expose:
- 5000
project-nginx:
restart: unless-stopped
environment:
APP_ENV: "production"
APP_NAME: "project-nginx"
APP_DEBUG: "False"
SERVICE_NAME: "project-nginx"

NGINX doesn't work when proxy_set_header is set to $host

I've been setting a simple docker-compose for a Django application, in which I have 3 containers: the Django app, a Postgres container, and NGINX. I successfully set up both Django and Postgres and tested connecting directly to their containers, so now the only thing left was to set up NGINX on the docker-compose file. I used the following NGINX default.conf file, from another template repository:
upstream django {
server app:8000;
}
server {
listen 80;
server_name localhost;
location / {
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_pass http://django;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location /static/ {
autoindex on;
alias /static/;
}
location /media/ {
autoindex on;
alias /media/;
}
}
And this was my docker-compose file:
version: "2"
services:
nginx:
image: nginx:latest
container_name: NGINX
ports:
- "80:80"
- "443:443"
volumes:
- ./test:/djangoapp/test
- ./config/nginx:/etc/nginx/conf.d
- ./test/static:/static
depends_on:
- app
app:
build: .
container_name: DJANGO
command: bash -c "./wait-for-it.sh db:5432 && python manage.py makemigrations && python manage.py migrate && gunicorn test.wsgi -b 0.0.0.0:8000"
depends_on:
- db
volumes:
- ./djangoapp/test:/djangoapp/test
- ./test/static:/static
expose:
- "8000"
env_file:
- ./config/djangoapp.env
db:
image: postgres:latest
container_name: POSTGRES
env_file:
- ./config/database.env
But for some reason I wasn't able to connect on the Django app at all via localhost:80 (the browser always threw me a 502 error, and the container wasn't logging anything when I tried). After a lot of troubleshooting, I found out that the offending line was proxy_set_header Host $host;, and commenting it out made me successfully connect to the Django app via localhost. So the problem was that my NGINX configuration had to use the proxy_host variable instead.
The problem is that I have no idea why that happened in the first place, because looking at this other question (Nginx: when to use proxy_set_header Host $host vs $proxy_host), I was suppose to use $host to proxy from my Django application, and other NGINX configuration examples also sets up the Host like that.
I may be missing something as NGINX is a tad bit confusing for me, but I don't understand why I wasn't able to connect and NGINX wasn't logging anything before I commented that line.

No connection with dockerized nginx gunicorn

I've created simple example for dockerized django with nginx and gunicorn. But for some reason connection doesn't go through nginx.
Here is nginx.conf contents
worker_processes 1;
error_log /var/log/nginx/error.log info;
events {
worker_connections 1024;
}
http {
gzip on;
gzip_http_version 1.1;
gzip_comp_level 2;
include /etc/nginx/mime.types;
default_type application/octet-stream;
server {
listen 80;
server_name www.example.dj example.dj;
location = /favicon.ico {
alias /app/static_root/favicon.ico;
}
location /static/ {
alias /app/static_root/;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/app/example.sock;
}
}
}
docker-compose.yml file
version: '3'
services:
web:
build: .
hostname: web
command: bash -c "python manage.py migrate && gunicorn example.wsgi:application --workers 3 --bind unix:/app/example.sock"
volumes:
- ./src:/app
expose:
- "8000"
nginx:
image: nginx:latest
ports:
- "80:8000"
volumes:
- ./src:/app
- ./config/nginx/nginx.conf:/etc/nginx/nginx.conf
depends_on:
- web
In /etc/hosts I have
127.0.0.1 www.example.dj example.dj
So when I run docker-compose up I expect to see django start page by http://example.dj url, but there is no connection. Can you guys help me with it?
Code available on the github

Can not connect to websocket using django-channels , nginx on docker as services

I'm using docker compose to build a project with django, nginx as services. When I launch the daphne server, and a client tries to connect to the websocket server, I get this error:
*1 recv() failed (104: Connection reset by peer) while reading response header from upstream
Client-side shows this
failed: Error during WebSocket handshake: Unexpected response code: 502
Here is my docker-compose.yml
version: '3'
services:
nginx:
image: nginx
command: nginx -g 'daemon off;'
ports:
- "1010:80"
volumes:
- ./config/nginx/nginx.conf:/etc/nginx/nginx.conf
- .:/makeup
links:
- web
web:
build: .
command: /usr/local/bin/circusd /makeup/config/circus/web.ini
environment:
DJANGO_SETTINGS_MODULE: MakeUp.settings
DEBUG_MODE: 1
volumes:
- .:/makeup
expose:
- '8000'
- '8001'
links:
- cache
extra_hosts:
"postgre": 100.73.138.65
Nginx:
server {
listen 80;
server_name thelab518.cloudapp.net;
keepalive_timeout 15;
root /makeup/;
access_log /dev/stdout;
error_log /dev/stderr;
location /api/stream {
proxy_pass http://web:8001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location / {
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://web:8000;
}
And the circusd's web.ini file:
[watcher:web]
cmd = /usr/local/bin/gunicorn MakeUp.wsgi:application -c config/gunicorn.py
working_dir = /makeup/
copy_env = True
user = www-data
[watcher:daphne]
cmd = /usr/local/bin/daphne -b 0.0.0.0 -p 8001 MakeUp.asgi:channel_layer
working_dir = /makeup/
copy_env = True
user = root
[watcher:worker]
cmd = /usr/bin/python3 manage.py runworker
working_dir = /makeup/
copy_env = True
user = www-data
As quite explicitly stated in the fine manual, to successfully run Channels you need to have a dedicated application server implementing the ASGI protocol, such as the supplied daphne
The entire Django execution model has been changed with Channels, so that there are separate "interface servers" taking care of receiving and sending messages over, for example, WebSockets or HTTP or SMS, and "worker servers" that run the actual code (potentially on a different server or VM or container or...). The two are connected by a "Channel layer" that carries messages and replies back and forth.
The current implementation supplies 3 channel layers that talk ASGI between an interface server and a worker server:
An In-memory channel layer, used mainly for running the test server (it's single process)
An IPC based channel layer, usable to run different workers on the same server
A redis based channel layer, that should be used for heavy production sites, able to connect interface servers to multiple worker servers.
You configure them like you do for DATABASES::
CHANNEL_LAYERS = {
"default": {
"BACKEND": "asgi_redis.RedisChannelLayer",
"ROUTING": "my_project.routing.channel_routing",
"CONFIG": {
"hosts": [("redis-channel-1", 6379), ("redis-channel-2", 6379)],
},
},
}
Of course this means that your docker config has to change and add one or more interface servers instead of, or in addition to, nginx (even if, in that case, you'll need to accept websocket connections on a different port with all the connected possible problems) and, quite likely, an instance of redis connectin all them.
This in turn means that until circus and nginx can support ASGI, it won't be possible to use them with django-channels, or that this support will only be for the regular http part of your system.
You can find more info in the Deploying section of the official documentation.
It looks that you stared daphne on port 8001, and trying to expose port 8000 and 8001 in docker-compose. The port 8000 is not pointing to any server (daphne is on 8001). In your nginx please set proxy to 8001 ports and expose only port 8001 in docker-compose.
I have created a simple example how it can be set on github where I have proxy to asgi and wsgi servers, but you can go with only asgi server:
The nginx:
upstream app {
server wsgiserver:8000;
}
upstream ws_server {
server asgiserver:9000;
}
server {
listen 8000 default_server;
listen [::]:8000;
client_max_body_size 20M;
location / {
try_files $uri #proxy_to_app;
}
location /tasks {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_pass http://ws_server;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app;
}
}
The docker-compose.yml:
version: '2'
services:
nginx:
extends:
file: docker-common.yml
service: nginx
ports:
- 8000:8000
volumes:
- ./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
volumes_from:
- asgiserver
asgiserver:
extends:
file: docker-common.yml
service: backend
entrypoint: /app/docker/backend/asgi-entrypoint.sh
links:
- postgres
- redis
- rabbitmq
expose:
- 9000
wsgiserver:
extends:
file: docker-common.yml
service: backend
entrypoint: /app/docker/backend/wsgi-entrypoint.sh
links:
- postgres
- redis
- rabbitmq
expose:
- 8000