Deploy django channels to dcoker and use nginx has a proxy server - django

Im trying to deploy django channels using docker, and im using the daphne protocol to serve the application. when i run that its starts successfully but i'm unable to connect with the websocket.
i can connect with the application in other routes but cannot connect with the websocket
the websocket is served in HTTPS://EXAMPLE.COM/WS/SOCK
this is my asgi.py
application = ProtocolTypeRouter({
'http': django_asgi_app,
'websocket': AllowedHostsOriginValidator(
AuthMiddlewareStack(
URLRouter(websocket_urlpatterns)
))})
Dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED 1
WORKDIR /myproject
ADD . /myroject
RUN pip install -r /myprjoect/requirements.txt
EXPOSE 8000
CMD ["daphne", "-b", "0.0.0.0", "-p", "8000", "myproject.asgi:application"]
nginx.conf
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forward-Proto https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://localhost:8443/;
}
#path to proxy my WebSocket requests
location /ws/ {
proxy_pass http://localhost:8443;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
and im using the docker-compose to up the application
anf its configuration is
version: "3.8"
services:
myservice:
image: myimage
container_name: my_container
ports:
- 8443:8000

Now my websocket is woking fine.
To resolve this problem i had to change proxy_set_header Connection “upgrade”; to proxy_set_header Connection Upgrade;

Related

Django channels connection failed (nginx + daphne + certbot + docker-compose)

I'm using Django channels using docker-compose
and nginx outside it
websockets were working just fine but after installing certbot, it stopped working
it returns 404 as if it is treated as just normal http request
VM253:1 WebSocket connection to 'wss://example.com/ws/results/tjrtudvzanxxqbyt' failed:
daphne : "GET /ws/results/tjrtudvzanxxqbyt" 404 2618
my nginx file for the server is
upstream language_transcriber{
server localhost:8000;
}
# the nginx server instance
server {
listen 443 default_server;
server_name example.com;
access_log /var/log/nginx/example.com.log;
ssl_certificate_key /etc/nginx/ssl/example.com.private_key.pem;
ssl_certificate /etc/nginx/ssl/fullchain.crt;
root /var/www;
ssl on;
# pass the request to the node.js server with the correct headers
# and much more can be added, see nginx config options
location /ws/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP$remote_addr;
proxy_set_header X-Forwarded-For$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto$scheme;
proxy_set_header X-Url-Scheme $scheme; proxy_redirect off;
proxy_set_header X-NginX-Proxy true;
proxy_http_version 1.1;proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_pass http://127.0.0.1:8000;
}
location /static/ {
alias /var/www/example.com/static/;
autoindex off;
}
location / {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://language_transcriber/;
proxy_redirect off;
}
}
and here's my docker-compose.yml
version: "3.8"
services:
django:
build: .
container_name: django
command: daphne -b 0.0.0.0 -p 8001 project.asgi:application --access-log=/code/media/daphne.access.log
ports:
- "8000:8001"
volumes:
- ./media:/code/media
environment:
- redis_host=redis://redis:6379/0
celery:
build: .
command: celery -A project worker --concurrency=10 --loglevel=info --pool=gevent -E -Ofair --logfile=/code/media/celery.log
restart: always
stdin_open: true # docker run -i
tty: true # docker run -t
environment:
- redis_host=redis://redis:6379/0
depends_on:
- django
- redis
volumes:
- ./media:/code/media
redis:
image: "redis:alpine"
worker_channels:
build: .
command: python manage.py runworker thumbnails-generate thumbnails-delete
depends_on:
- django
- redis
environment:
- redis_host=redis://redis:6379/0
volumes:
waves-volume:
the app works just fine on my localhost but on production it just serves http not ws

django + nginx custom location block not working

I want to deploy django as backend server with nginx.
I use daphne as asgi server due to django channel
location /api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:10131;
}
as you can see, http://127.0.0.1:10131 is django which should be connected to http://my_domain.com/api
but django can't recognize requested uri.
surely, I set FORCE_SCRIPT_NAME to /api
What should I do further?
please help.
First check whether your daphne service is running on server or not & then try this
server {
listen 80; # can change to 10131
server_name http://my_domain.com;
location /api {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:10131;
}
}
You can change listen port if you want your api accessible to specific port eg. 10131
If you're facing any issues then try to check your daphne logs like this
sudo journalctl -u daphne
and to check your api is running or not run this command
curl http://127.0.0.1:10131/api
you may need to install curl run this to install it
sudo apt install curl

Websocket 502 Bad Gateway

I've two container wsgi and asgi.
wsgi server is running on 127.0.0.8000:
gunicorn app.wsgi:application --bind 127.0.0.1:8000
Also asgi server is running on 127.0.0.1:8001 using of daphne:
daphne -b 127.0.0.1 -p 8001 app.asgi:application
I have a websocket request like this:
wss://app.example.com/ws/chat/f770eef/
But unfortunately these errors occur in the system:
i) nginx log says:
2022/05/22 13:15:29 [error] 463129#463129: *9 upstream prematurely
closed connection while reading response header from upstream, client:
11.198.111.11, server: app.example.com, request: "GET /ws/chat/f770eef/
ii) Requests do not reach daphne.
asgi.py
import os
from django.core.asgi import get_asgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')
application = get_asgi_application()
nginx cofing:
server {
listen 443;
listen [::]:443;
server_name app.example.com;
root /var/www/html;
...
location /ws/ {
proxy_pass http://127.0.0.1:8001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
}
...
}
I had the same problem with my setup. changing the the proxy_pass setting to HTTPS instead of http fixed it for me.
...
proxy_pass https://127.0.0.1:8001;
...
Since you're using WSS, thats WS over SSL protocol so HTTP doesn't cut it and I guess Nginx or some other service terminates the connection

Cannot access Strapi in a container using AWS Lightsail, Ubuntu, Docker and NGINX

I'm trying to practice and I have created an instance using AWS Lightsail with Ubuntu 20.04 LTS, also I've installed Docker and Docker Compose.
I would like to run this applications:
MariaDB
Strapi CMS
Let's assume that my public IP is: 1.20.20.20
When I try to access "http://1.20.20.20:1337", I receives "This site can’t be reached".
I thought that maybe I needed a web server and I have included NGINX in my docker-compose configuration.
My folders in the instance:
envs/strapi.env
nginx/strapi.conf
docker-compose.yml
docker-compose.yml
version: '3.8'
services:
nginx:
image: nginx:stable
container_name: nginx
volumes:
- ./nginx/strapi.conf:/etc/nginx/conf.d/default.conf
ports:
- 80:80
restart: on-failure
depends_on:
- strapi
db:
image: mariadb:10.6.4
container_name: db
restart: always
env_file:
- envs/db.env
volumes:
- db-data:/var/lib/mysql
networks:
- apps
strapi:
image: strapi/strapi:3.6.8
container_name: strapi
env_file:
- envs/strapi.env
restart: always
depends_on:
- db
ports:
- 1337:1337
networks:
- apps
volumes:
db-data:
networks:
apps:
driver: bridge
nginx/strapi.conf (I have copied the configuration from Strapi docs):
server {
# Listen HTTP
listen 1337;
server_name 1.20.20.20;
# Static Root
location / {
root /var/www/html;
}
# Strapi API
location /api/ {
rewrite ^/api/?(.*)$ /$1 break;
proxy_pass http://strapi;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
# Strapi Admin
location /admin {
proxy_pass http://strapi/admin;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
}
Still cannot access the CMS using my browser.
For some reason, when I do "docker ps -a", the status of the nginx container is always Restarting....
I have tried to follow and read steps that I have found on a few articles without luck. I don't have a lot of experience using NGINX and Ubuntu servers.
My goal is to be able to access the web app (Strapi CMS) through my browser.
Any help would be appreciated.

Can not connect to websocket using django-channels , nginx on docker as services

I'm using docker compose to build a project with django, nginx as services. When I launch the daphne server, and a client tries to connect to the websocket server, I get this error:
*1 recv() failed (104: Connection reset by peer) while reading response header from upstream
Client-side shows this
failed: Error during WebSocket handshake: Unexpected response code: 502
Here is my docker-compose.yml
version: '3'
services:
nginx:
image: nginx
command: nginx -g 'daemon off;'
ports:
- "1010:80"
volumes:
- ./config/nginx/nginx.conf:/etc/nginx/nginx.conf
- .:/makeup
links:
- web
web:
build: .
command: /usr/local/bin/circusd /makeup/config/circus/web.ini
environment:
DJANGO_SETTINGS_MODULE: MakeUp.settings
DEBUG_MODE: 1
volumes:
- .:/makeup
expose:
- '8000'
- '8001'
links:
- cache
extra_hosts:
"postgre": 100.73.138.65
Nginx:
server {
listen 80;
server_name thelab518.cloudapp.net;
keepalive_timeout 15;
root /makeup/;
access_log /dev/stdout;
error_log /dev/stderr;
location /api/stream {
proxy_pass http://web:8001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location / {
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://web:8000;
}
And the circusd's web.ini file:
[watcher:web]
cmd = /usr/local/bin/gunicorn MakeUp.wsgi:application -c config/gunicorn.py
working_dir = /makeup/
copy_env = True
user = www-data
[watcher:daphne]
cmd = /usr/local/bin/daphne -b 0.0.0.0 -p 8001 MakeUp.asgi:channel_layer
working_dir = /makeup/
copy_env = True
user = root
[watcher:worker]
cmd = /usr/bin/python3 manage.py runworker
working_dir = /makeup/
copy_env = True
user = www-data
As quite explicitly stated in the fine manual, to successfully run Channels you need to have a dedicated application server implementing the ASGI protocol, such as the supplied daphne
The entire Django execution model has been changed with Channels, so that there are separate "interface servers" taking care of receiving and sending messages over, for example, WebSockets or HTTP or SMS, and "worker servers" that run the actual code (potentially on a different server or VM or container or...). The two are connected by a "Channel layer" that carries messages and replies back and forth.
The current implementation supplies 3 channel layers that talk ASGI between an interface server and a worker server:
An In-memory channel layer, used mainly for running the test server (it's single process)
An IPC based channel layer, usable to run different workers on the same server
A redis based channel layer, that should be used for heavy production sites, able to connect interface servers to multiple worker servers.
You configure them like you do for DATABASES::
CHANNEL_LAYERS = {
"default": {
"BACKEND": "asgi_redis.RedisChannelLayer",
"ROUTING": "my_project.routing.channel_routing",
"CONFIG": {
"hosts": [("redis-channel-1", 6379), ("redis-channel-2", 6379)],
},
},
}
Of course this means that your docker config has to change and add one or more interface servers instead of, or in addition to, nginx (even if, in that case, you'll need to accept websocket connections on a different port with all the connected possible problems) and, quite likely, an instance of redis connectin all them.
This in turn means that until circus and nginx can support ASGI, it won't be possible to use them with django-channels, or that this support will only be for the regular http part of your system.
You can find more info in the Deploying section of the official documentation.
It looks that you stared daphne on port 8001, and trying to expose port 8000 and 8001 in docker-compose. The port 8000 is not pointing to any server (daphne is on 8001). In your nginx please set proxy to 8001 ports and expose only port 8001 in docker-compose.
I have created a simple example how it can be set on github where I have proxy to asgi and wsgi servers, but you can go with only asgi server:
The nginx:
upstream app {
server wsgiserver:8000;
}
upstream ws_server {
server asgiserver:9000;
}
server {
listen 8000 default_server;
listen [::]:8000;
client_max_body_size 20M;
location / {
try_files $uri #proxy_to_app;
}
location /tasks {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_pass http://ws_server;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app;
}
}
The docker-compose.yml:
version: '2'
services:
nginx:
extends:
file: docker-common.yml
service: nginx
ports:
- 8000:8000
volumes:
- ./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
volumes_from:
- asgiserver
asgiserver:
extends:
file: docker-common.yml
service: backend
entrypoint: /app/docker/backend/asgi-entrypoint.sh
links:
- postgres
- redis
- rabbitmq
expose:
- 9000
wsgiserver:
extends:
file: docker-common.yml
service: backend
entrypoint: /app/docker/backend/wsgi-entrypoint.sh
links:
- postgres
- redis
- rabbitmq
expose:
- 8000