Issue connecting to secure websocket using Django/Nginx/Daphne - django

Having an issue using secure websocket (wss) with django, Nginx, Gunicorn, and daphne. My site is hosted through cloudflare which provides the SSL/TLS certificate. I'm using a linux socket in /run/daphne/daphne.sock, where I gave the user 'ubuntu' ownership of the daphne folder.
The websockets work fine locally when it is not secured. When I tried hosting on my EC2 instance, I get the error-
sockets.js:16 WebSocket connection to 'wss://www.mywebsite.com/ws/sheet/FPIXX8/' failed:
Then it keeps trying to reconnect and fail again. It never sets up an initial connection since I don't get a ping.
Here are a few relevant files and snippets of code-
settings.py
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer', # asgi_redis.RedisChannelLayer ?
'CONFIG': {
'hosts': [('localhost', 6379)],
},
}
}
I think there may be an issue with the 'hosts' but I haven't been able to figure out through tutorials/googling, nor am I sure exactly what it does besides set a port. Since I'm using sockets, I imagine this would need to be different (or maybe it's ignored?) in deployment.
Routing.py (in main project dir)
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
import app.routing
application = ProtocolTypeRouter({
# (http->django views is added by default)
'websocket': AuthMiddlewareStack(
URLRouter(
app.routing.websocket_urlpatterns
)
),
})
Routing.py (in app)
from django.urls import re_path
from django.conf.urls import url
from app import consumers
websocket_urlpatterns = [
re_path(r'ws/sheet/(?P<gameID>\w+)/$', consumers.GameConsumer.as_asgi()),
]
I'm pretty sure neither routing.py files don't cause an issue because I still get redirected properly.
asgi.py
import os
from django.core.asgi import get_asgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
application = get_asgi_application()
I haven't touched this file.
Javascript
if (window.location.protocol == 'https:') {
wsProtocol = 'wss://'
} else {wsProtocol = 'ws://'}
updateSocket = new WebSocket(
wsProtocol + window.location.host +
'/ws/sheet/' + game_ID + '/');
This also seems to work fine, since I get redirected and it attempts to connect to proper URL with wss://.
systemd/system/daphne.service
[Unit]
Description=project Daphne Service
After=network.target
[Service]
User=ubuntu
Type=simple
WorkingDirectory=/home/ubuntu/django/project/project
ExecStart=/home/ubuntu/django/project/project/virtualenv/bin/daphne \
-u /run/daphne/daphne.sock \
project.asgi:application
[Install]
WantedBy=multi-user.target
systemd/system/daphne.socket
[Unit]
Description=daphne socket
[Socket]
ListenStream=/run/daphne/daphne.sock
[Install]
WantedBy=sockets.target
nginx
server {
listen 80;
server_name www.mywebsite.com mywebsite.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/django/project/project/;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
location /ws/ {
proxy_pass http://unix:/run/daphne/daphne.sock;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
WantedBy=sockets.target
I should probably add running 'sudo nginx -t' to get errors does not return any errors. Running 'sudo journalctl -u daphne' usually would not return errors, but I just checked again and I got another Permission Denied error at '/run/daphne/daphne.sock.lock' which I thought was fixed, so I tried to give permission again. Then, I got a 'Address already in use' for the daphne.sock. So, I restarted all services and now it just says 'Configuring endpoint unix:/run/daphne/daphne.sock' as the latest message, so I don't know if that is good or bad.

Related

Connect via a secure websocket WSS from Django project to Redis/Daphne

I am trying to connect a secure websocket WSS with Redis/Daphne from my Django-Project:
new WebSocket('wss://myproject.com:9002/ws/myChat/')
But it is not possible to connect. In the Browser-Console I always get the following error:
myCode.js:36 WebSocket connection to 'wss://myproject.com:9002/ws/myChat/' failed
This is the only error I see, e.g. Nginx (sudo tail -F /var/log/nginx/error.log) shows no related error.
The Daphne and Redis-Services seem to work:
Redis Status:
redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-04-07 08:44:07 CEST; 55min ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Process: 281 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCCESS)
Main PID: 381 (redis-server)
Tasks: 4 (limit: 60)
Memory: 4.1M
CGroup: /system.slice/redis-server.service
└─381 /usr/bin/redis-server 127.0.0.1:6379
Daphne-Service Config:
[Unit]
Description=WebSocket Daphne Service
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/home/django_user/appdir/myProject
ExecStart=/home/django_user/appdir/env/bin/python3 /home/django_user/appdir/env/bin/daphne -e ssl:9002:privateKey=/etc/letsencrypt/live/myproject.com/privkey.pem:certKey=/etc/letsencrypt/myproject.com/fullchain.pem myproject.asgi:application
Restart=on-failure
[Install]
WantedBy=multi-user.target
Daphne-Service Status:
daphne_myproject.service - WebSocket Daphne Service
Loaded: loaded (/etc/systemd/system/daphne_myproject.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-04-07 08:44:04 CEST; 50min ago
Main PID: 277 (python3)
Tasks: 1 (limit: 60)
Memory: 74.8M
CGroup: /system.slice/daphne_myproject.service
└─277 /home/django_user/appdir/env/bin/python3 /home/django_user/appdir/env/bin/daphne -e ssl:9002:privateKey=/etc/letsencrypt/live/myproject.com/privkey.pem:certKey=/etc/letsencrypt/live/myproject.com/fullchain.pem myproject.asgi:application
Nginx-Config:
upstream websocket{
server 127.0.0.1:6379;
}
server {
server_name myproject.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/django_user/appdir/myProject;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/myproject.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/myproject.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
location /ws {
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
server {
if ($host = myproject.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name myproject.com;
return 404; # managed by Certbot
}
And in my Django-Project:
settings.py
ASGI_APPLICATION = "myproject.routing.application"
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
"hosts": [('127.0.0.1', 6379)],
},
},
}
routing.py: (in the chat-app, which is part of my Django-project, the wss://myproject... websocket is opened)
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
import chat.routing
application = ProtocolTypeRouter({
'websocket': AuthMiddlewareStack(
URLRouter(
chat.routing.websocket_urlpatterns
)
),
})
asgi.py:
import os
from django.core.asgi import get_asgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
application = get_asgi_application()
Did I miss something here? I am trying now for days but can't figure out where the problem could be. Could it be because I use the root user for the Daphne service and not django_user? I did that because django_user could not access my certificates in /etc/letsencrypt/... (though he has sudo rights).
Update:
Just saw this error (at sudo systemctl status daphne_myproject.service):
Apr 07 19:42:07 myproject.com python3[268]: 2022-04-07 19:42:07,215 ERROR Exception inside application: Django can only handle ASGI/HTTP connections, not websocket.
I just solved my problem by modifying asgi.py as follows. Maybe it will help someone else with the same problem:
import os
import django
from channels.routing import get_default_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
django.setup()
application = get_default_application()

Websocket Connection Failed on AWS EC2 Ubuntu Instance(Django Channels)

I am trying to deploy my django chammels asgi app in ubuntu aws ec2 instance, my wsgi app is working fine also the asgi on ubuntu is working fine (tested with python websocket library) there is no error in code and daphne is running perfectly
I enabled all ports in the security group of EC2 as well
There is no error on the codebase and also daphne is running on localhost of EC2 ubuntu Instance perfectly
server {
listen 80;
server_name MY_SERVER_IP;
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
location /ws/{
proxy_pass http://0.0.0.0:4001;
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
[Unit]
Description=project Daphne Service
After=network.target
[Service]
User=root
Type=simple
WorkingDirectory=/home/ubuntu/myproject
ExecStart=/home/ubuntu/myproject/venv/bin/python /home/ubuntu/myproject/venv/bin/d>
[Install]
WantedBy=multi-user.target
I created this python script to determine if my daphne is working or not, it is providing me the perfect result, no error
import asyncio
import websockets
import json
async def hello():
uri = "ws://127.0.0.1:4001/ws/chat/person/?token=<CHAT_TOKEN>
async with websockets.connect(uri) as websocket:
await websocket.send(json.dumps({"message":"Hey there"}))
g = await websocket.recv()
print(g)
asyncio.get_event_loop().run_until_complete(hello())
This following script returns
{
"id": 1,
"message": "Hey There",
"time_stamp": "TIME",
"receiver": {
"username": "John Doe",
"id": 2",
"profile_picture": "LINK"
}
}
Now in my frontend application when I try to connect with the websocket
socket = new WebSocket("ws://<SERVER_IP>:4001/ws/chat/person/?token=<MY_TOKEN>")
socket.onopen = functon(){
...
}
...
But the following script returns
WebSocket connection to 'ws://<SERVER_IP>/ws/chat/person/?token=<TOKEN> failed:
I fixed the problem by using supervisor
sudo apt install nginx supervisor
Now, you will need to create the supervisor configuration file (often located in /etc/supervisor/conf.d/ - here, we’re making Supervisor listen on the TCP port and then handing that socket off to the child processes so they can all share the same bound port:
[fcgi-program:asgi]
# TCP socket used by Nginx backend upstream
socket=tcp://localhost:8000
# Directory where your site's project files are located
directory=/my/app/path
# Each process needs to have a separate socket file, so we use process_num
# Make sure to update "mysite.asgi" to match your project name
command=daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers mysite.asgi:application
# Number of processes to startup, roughly the number of CPUs you have
numprocs=4
# Give each process a unique name so they can be told apart
process_name=asgi%(process_num)d
# Automatically start and recover processes
autostart=true
autorestart=true
# Choose where you want your log to go
stdout_logfile=/your/log/asgi.log
redirect_stderr=true
Create the run directory for the sockets referenced in the supervisor configuration file.
sudo mkdir /run/daphne/
sudo chown <user>.<group> /run/daphne/
d /run/daphne 0755 <user> <group>
sudo supervisorctl reread
sudo supervisorctl update

Uvicorn + Django + NGinx - Error 404 while handling websockets

I am setting up a server with Nginx and Uvicorn to run a Django app with websockets.
Everything goes well with normal http request and I can get my webpage but my websockets handshakes always end up with a 404 error.
Everything goes well running with runserver.
here is my asgi.py file
import os
import django
django.setup()
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application
import MYAPP.routing
import Stressing.routing
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'MYAPP.settings')
application = ProtocolTypeRouter({
"http": get_asgi_application(),
"websocket": AuthMiddlewareStack( URLRouter( Stressing.routing.websocket_urlpatterns ) ),
# Just HTTP for now. (We can add other protocols later.)
})
my settings.py
ASGI_APPLICATION = 'MYAPP.asgi.application'
redis_host = os.environ.get('REDIS_HOST', '127.0.0.1')
CHANNEL_LAYERS = {
'default': {
'CONFIG': {
'hosts': [(redis_host, 6379)],
},
'BACKEND': 'channels_redis.core.RedisChannelLayer',
},
}
my nginx config file
server {
listen 80;
server_name MYAPP.box 10.42.0.1;
location = /favicon.ico { access_log off; log_not_found off; }
location ~ ^/static {
autoindex on;
root /home/MYAPP;
}
location ~ ^/ {
include proxy_params;
proxy_pass http://unix:/home/MYAPP/MYAPP.sock;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://unix:/home/MYAPP/MYAPP.sock;
}
location ~ ^/ws/ {
proxy_pass http://unix:/home/MYAPP/MYAPP.sock;
proxy_http_version 1.1;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
The request seems to reach to socket as I can see in the gunicorn log
not found /ws/0
Does anyone has an idea of the origin of the problem?
I have removed the regex expression from my Nginx file and it is working now
location ~ ^/ {
...
location ~ ^/ws/
became
location / {
...
location /ws/
so it is solved. But if someone can explain me the background of it?
I thought that Nginx always prioritizes the most specific path. Is it only working without regex? (I guess there is no ways to determine which regex is more specific?!)

Django deployment with Daphne

UPDATE:
It may be closed, remember to always check code copied from outside... The problem was with quote marks in Daphne Service
I want to deploy my Django app (Rest API + React on frontend) on AWS. I use nginx, gunicorn (for http handling) and daphne (for async websockets - Im using django channels for chat application). I was following THIS tutorial.
It looks like I configured well nginx and gunicorn (page is normally loading, I can handle sync request to rest API) but I guess there are some problems with daphne and/or asgi. I use systemctl services for server. All 3 (nginx, gunicorn, daphne) statuses show 'active'. Everything works fine on my development server on local.
On deployment server when I enter website, I see in console
Firefox can’t establish a connection to the server at ws://PATH_TO_WEBSOCKET
Is Daphne connected well with Nginx?
I use Redis for CHANNEL_LAYERS. I have installed channels-redis and redis-server on Ubuntu server. I think it works fine as I get respond
> redis-cli
> ping
**PONG**
project tree
.
├── frontend #react files
├── checkers #project dir
│ ├── settings.py
│ ├── urls.py
│ └── asgi.py
├── chat #chat app
│ ├── consumers.py
│ └── routing.py
└── checkers.sock
asgi.py
import os
import django
from channels.routing import get_default_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'checkers.settings')
django.setup()
application = get_default_application()
settings.py
ROOT_URLCONF = 'checkers.urls'
ASGI_APPLICATION = "checkers.routing.application"
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
"hosts": [('127.0.0.1', 6379)],
},
},
}
daphne service worker
[Unit]
Description=My Daphne Service
After=network.target
[Service]
Type=simple
User=ams
Group=www-data
WorkingDirectory=/home/ams/production_app
ExecStart=/home/ams/production_app/production_env/bin/daphne --access-log /home/ams/production_app/log/daphne-access.log -b 0.0.0.0 -p 9001 checkers.asgi:application
[Install]
WantedBy=multi-user.target
Nginx config file at /etc/nginx/sites-available/ (our websockets endpoints start with /ws/)
upstream channels-backend {
server 0.0.0.0:9001;
}
server {
listen 80;
server_name MY_SERVER;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ams/production_app;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ams/production_app/checkers.sock;
}
location /ws/ {
proxy_pass http://channels-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}

Django channels work on local but not on server, Error during WebSocket handshake

Website is loading fine but the channels are not working. In console I get:
WebSocket connection to 'ws://fortests.ovh/8' failed: Error during WebSocket handshake: Unexpected response code: 404
Server: Ubuntu 16.04 on Digital Ocean
nginx version: nginx/1.10.0 (Ubuntu)
Redis server v=3.2.8
my settings.py:
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'asgi_redis.RedisChannelLayer',
'CONFIG': {
'hosts': [('localhost', 6379)],
},
'ROUTING': 'slist.routing.channel_routing',
}
}
wsgi.py
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "slist.settings")
application = get_wsgi_application()
consumers.py
import json
from channels import Group
from channels.auth import channel_session_user_from_http, channel_session_user
from .models import Item
# Connected to websocket.connect
#channel_session_user_from_http
def ws_add(message):
# Accept the connection
message.reply_channel.send({"accept": True})
# Add to the users group
Group("users").add(message.reply_channel)
routing.py
from channels.routing import route
from tobuy.consumers import ws_add, ws_receive, ws_disconnect
channel_routing = [
route("websocket.connect", ws_add),
route("websocket.receive", ws_receive),
route("websocket.disconnect", ws_disconnect),
]
js
var socket = new WebSocket('ws://' + window.location.host + '/' + {{ active_list.id }});
nginx settings
server {
listen 80;
server_name server_name fortests.ovh;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/kuba1/slistproject/slistvenv/src;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/kuba1/slistproject/slistvenv/src/slist.sock;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
I also had above 404 error when using daphne (channels 2) in a docker setting. In ended up with below nginx conf:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream websocket {
server daphne:8000;
}
server {
listen 80 default_server;
server_name localhost;
root /var/www;
location /static {
index index.html index.htm;
}
location / {
root /var/www;
try_files $uri $uri/index.html #daphne;
expires max;
access_log off;
}
location #daphne {
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
where daphne (on line 5) was the name of the container running the daphne process.