I am currently facing an issue with the Django Channels and Websockets. I have an application that somehow works with files and Sharepoint. Issued code sample where defect occurs:
#shared_task(bind=True)
def upload_to_sharepoint_task(self, user_id: str, document_type: str, upload_file: str, filename: str) -> None:
"""Uploads a file to SharePoint."""
task_id = self.request.id
try:
upload_to_sharepoint(user_id, document_type, upload_file, filename)
print(f"Task {task_id} completed successfully.")
async_to_sync(get_channel_layer().group_send)(task_id, {
"type": "send_task_status",
"message": {'task_id': task_id, 'status': "SUCCESS", }
})
except BackendError as e:
print(f"Task {task_id} failed with error: {str(e)}")
async_to_sync(get_channel_layer().group_send)(task_id, {
"type": "send_task_status",
"message": {'task_id': task_id, 'status': "FAILURE", 'error': str(e)}
})
If everything the method upload_to_sharepoint does not throw an exception, everything works just fine - the message is sent to the group and, via WebSocket, propagated to the Client. However, if I try to simulate a situation when the exception is thrown (for instance, by throwing the BackendException explicitly), the error message gets printed in the console, but WebSocket does not receive any message nor the Client.
My TaskConsumer:
class TaskConsumer(WebsocketConsumer):
def connect(self):
self.task_id = self.scope['url_route']['kwargs']['task_id']
# Join the task_id group
async_to_sync(self.channel_layer.group_add)(self.task_id, self.channel_name)
self.accept()
def disconnect(self, close_code):
# Leave the task_id group
async_to_sync(self.channel_layer.group_discard)(
self.task_id, self.channel_name
)
def receive(self, text_data):
text_data_json = json.loads(text_data)
message = text_data_json["message"]
print(f"Task {self.task_id} received message: {message}")
async_to_sync(self.channel_layer.group_send)(
self.task_id, {"type": "send_task_status", "message": message}
)
def send_task_status(self, event):
message = event['message']
self.send(text_data=json.dumps({"message": message}))
Does anyone have an idea what could be wrong and how I can propagate the information about unsuccessful upload back to the Client via used WebSocket?
Related
I want to connect my Django WebSocket with a third-party web socket. This program is one I wrote, and it functions properly. To avoid having to re-login with the third-party API, I have now added the code to check whether the same room is present in my database. if we use the same API KEY to re-connect to the third-party API. It gives the following error:
{"event":"login","status":401,"message":"Connected from another location"}
I want to see if the same cryptocurrency coin is already connected or not. We don't want to logon with the same API KEY once we're connected. I have two issues here:
Don't send the login request to that web socket again.
Don't send the subscribe request, if the same coin already exists. Let's say BTCUSD already connected and giving me the data. I want to just connect to the next user to same room and get the data on next request.
import websocket
import time
import ssl
from channels.generic.websocket import AsyncWebsocketConsumer
from .models import Room
login = {
"event": "login",
"data": {
"apiKey": "API_KEY",
},
}
class CryptoConsumer(AsyncWebsocketConsumer):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.ws = websocket.WebSocket(sslopt={"cert_reqs": ssl.CERT_NONE})
self.ws.connect("wss://crypto.financialmodelingprep.com")
async def connect(self):
self.room_name = self.scope["url_route"]["kwargs"]["room_name"]
self.room_group_name = "crypto_%s" % self.room_name
# ****** Code Block ******
subscribe = {
"event": "subscribe",
"data": {
"ticker": self.room_name,
},
}
room = await Room.add(self.room_name) # Method in models.py to add the user and return True
if room is False:
self.ws.send(json.dumps(login))
print("The group with the name %s doesn't exist" % self.room_group_name)
time.sleep(1)
self.ws.send(json.dumps(subscribe))
# ****** End Code Block ******
# Join room group
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
async def disconnect(self, close_code):
unsubscribe = {
"event": "unsubscribe",
"data": {
"ticker": self.room_name,
},
}
self.ws.send(json.dumps(unsubscribe))
self.ws.close()
# Leave room group
await self.channel_layer.group_discard(self.room_group_name, self.channel_name)
# Receive message from WebSocket
async def receive(self, text_data="{'text': 'Dummy Text'}"):
text_data_json = json.loads(text_data)
message = text_data_json["message"]
message = str(self.ws.recv())
# Send message to room group
await self.channel_layer.group_send(
self.room_group_name, {"type": "chat_message", "message": message}
)
# Receive message from room group
async def chat_message(self, event):
message = event["message"]
# Send message to WebSocket
await self.send(text_data=json.dumps({"message": message}))
Note: Why I want to do this entire step because we don't want the API KEY made public.
So, the code coming from the front end will connect to our Django web socket, and then we'll connect to the third-party web socket and return the data that was sent by them.
I am using Django Channels and RabbitMQ pika, for the first time. I am trying to consume from RabbitMQ queue. I am using Django Channels AsyncConsumer to group send it to everyone connected in the websocket.
User type 1 : Can create a task
User type 2 : Can accept the task.
Use case : When user type 1 creates the task it is published in the rabbitmq. When it is consumed from the queue, it has to be group-sent to frontend. And when the user type 2 accepts the task other instances of user type 2 cannot accept the same and we consume from the queue again and send the next task in the queue to everyone.
I have created the connection in a different thread using sync_to_async I am appending it to an in-memory list from the callback function.
And whenever someone accepts I just pop it out of the list and acknowledge the queue.
class AcceptTaskConsumer(AsyncConsumer):
body = [] #IN MEMORY LIST
delivery = {} #To store ack delivery_tag
async def websocket_connect(self, event):
print("AcceptTaskConsumer connected", event)
AcceptTaskConsumer.get_task() #STARTS Queue listener in new thread
self.room_group_name = "user_type_2"
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.send({
"type": "websocket.accept"
})
async def websocket_receive(self, event):
if event["text"] == "Hi": #If connecting first time
if AcceptTaskConsumer.body:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": AcceptTaskConsumer.body[0]["body"]
}
)
else:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": "No New Tasks"
}
)
else: #When someone accepts a task-> ack and send next task in queue
print(json.loads(event["text"])["id"])
AcceptTaskConsumer.channel.basic_ack(delivery_tag=AcceptTaskConsumer.delivery[json.loads(event["text"])["id"]])
AcceptTaskConsumer.delivery.pop(json.loads(event["text"])["id"])
AcceptTaskConsumer.body.pop(0)
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": "No New Tasks"
}
)
if AcceptTaskConsumer.body:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": AcceptTaskConsumer.body[0]["body"]
}
)
async def message(self, event):
await self.send({
"type": "websocket.send",
"text": event["text"]
})
#classmethod
#sync_to_async
def get_task(cls): #pika consumer
cls.connection = pika.BlockingConnection(
pika.ConnectionParameters(host='localhost'))
cls.channel = cls.connection.channel()
cls.channel.queue_declare(queue='task_', arguments={"x-max-priority": 3})
cls.channel.basic_consume(
queue='task_', on_message_callback=AcceptTaskConsumer.callback, auto_ack=False)
cls.channel.start_consuming()
#classmethod
def callback(cls, ch, method, properties, body):
task_obj = {"body": json.dumps(body.decode("utf-8")),
"delivery_tag": method.delivery_tag}
AcceptTaskConsumer.body.append(task_obj)
AcceptTaskConsumer.delivery[json.loads(json.loads(task_obj["body"]))["id"]] = method.delivery_tag
cls.channel.stop_consuming()
async def websocket_disconnect(self, event):
print(event)
await self.send({
"type": "websocket.close"
})
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
I am pretty sure this is not the right way to do it, because it's not working as expected
I run into frequent errors like.
39 of 169 channels over capacity in group delivery
pika.exceptions.StreamLostError: Stream connection lost: BrokenPipeError(32, 'Broken pipe')
I tried running the queue listener like this answer as well. Nothing working.
Any one experienced has any thoughts about this? Is there a better way to approach this problem.?
you should move the rabitMQ cosumering logic out of the websocket consumer.
Just have a django command that runs the Rabbit Consumer, that consumer can take messages from RabbitMQ and then use send_group to send them over groups to channels.
if your django command you will need to call send_group see https://channels.readthedocs.io/en/latest/topics/channel_layers.html#using-outside-of-consumers
from channels.layers import get_channel_layer
channel_layer = get_channel_layer()
async_to_sync(
channel_layer.group_send
)(
"user_type_2",
{"type": "message", "msg": 123}
)
Then in the websocket consumer you should subscribe to the groups that the user wants/has permition to get.
I am using a combination of DRF 3.11.0 and Channels 2.4.0 to implement a backend, and it is hosted on Heroku on 1 dyno with a Redis resource attached. I have a socket on my React frontend that successfully sends/received from the backend server.
I am having an issues where any message sent back to the front end over the socket is being sent twice. I have confirmed through console.log that the front end is only pinging the back end once. I can confirm through print() inside of the API call that the function is only calling async_to_sync(channel_layer.group_send) once as well. The issue is coming from my consumer - when I use print(self.channel_name) inside of share_document_via_videocall(), I can see that two instances with different self.channel_names are being called (specific.AOQenhTn!fUybdYEsViaP and specific.AOQenhTn!NgtWxuiHtHBw. It seems like the consumer has connected to two separate channels, but I'm not sure why. When I put print() statements in my connect() I only see it go through the connect process once.
How do I ensure that I am only connected to one channel?
in settings.py:
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
#"hosts": [('127.0.0.1', 6379)],
"hosts": [(REDIS_HOST)],
},
},
}
Consumer:
import json
from asgiref.sync import async_to_sync
from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from rest_framework.authtoken.models import Token
from django.contrib.auth.models import AnonymousUser
from .exceptions import ClientError
import datetime
from django.utils import timezone
class HeaderConsumer(AsyncWebsocketConsumer):
async def connect(self):
print("connecting")
await self.accept()
print("starting")
print(self.channel_name)
await self.send("request_for_token")
async def continue_connect(self):
print("continuing")
print(self.channel_name)
await self.get_user_from_token(self.scope['token'])
await self.channel_layer.group_add(
"u_%d" % self.user['id'],
self.channel_name,
)
#... more stuff
async def disconnect(self, code):
await self.channel_layer.group_discard(
"u_%d" % self.user['id'],
self.channel_name,
)
async def receive(self, text_data):
text_data_json = json.loads(text_data)
if 'token' in text_data_json:
self.scope['token'] = text_data_json['token']
await self.continue_connect()
async def share_document_via_videocall(self, event):
# Send a message down to the client
print("share_document received")
print(event)
print(self.channel_name)
print(self.user['id'])
await self.send(text_data=json.dumps(
{
"type": event['type'],
"message": event["message"],
},
))
#database_sync_to_async
def get_user_from_token(self, t):
try:
print("trying token" + t)
token = Token.objects.get(key=t)
self.user = token.user.get_profile.json()
except Token.DoesNotExist:
print("failed")
self.user = AnonymousUser()
REST API call:
class ShareViaVideoChat(APIView):
permission_classes = (permissions.IsAuthenticated,)
def post(self, request, format=None):
data = request.data
recipient_list = data['recipient_list']
channel_layer = get_channel_layer()
for u in recipient_list:
if u['id'] != None:
print("sending to:")
print('u_%d' % u['id'])
async_to_sync(channel_layer.group_send)(
'u_%d' % u['id'],
{'type': 'share_document_via_videocall',
'message': {
'document': {'data': {}},
'sender': {'name': 'some name'}
}
}
)
return Response()
with respect to you getting to calls with different channel names are you sure your frontend has not connected twice to the consumer? Check in the debug console in your browser.
i get same problem with nextjs as a frontend of Django channels WebSocket server.
and after searching i found the problem related with tow things:
1- react strict mode (the request sending twice) :
to disable react strict mode in next.js , go to module name "next.config.js" , and change the value for strict mode to false , as the following :
/** #type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: false,
}
module.exports = nextConfig
2- in nextjs the code run twice (outside useEffect Hook) , one on server side and the second on the client side (which means each user will connect to websocket server twice, and got two channels name , and join to same group twice each time with different channel name . ) ,
so i changed my codes to connect with Django channels server only from client side , if you like see my full code / example , kindly visit the following URL , and note the checking code about "typeof window === "undefined":
frontend nextjs code :
https://stackoverflow.com/a/72288219/12662056
i don't know if my problem same your problem , but i hope that helpful.
Django Channels docs has following basic example of a Server Sent Events. AsyncHttpConsumer
from datetime import datetime
from channels.generic.http import AsyncHttpConsumer
class ServerSentEventsConsumer(AsyncHttpConsumer):
async def handle(self, body):
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
while True:
payload = "data: %s\n\n" % datetime.now().isoformat()
await self.send_body(payload.encode("utf-8"), more_body=True)
await asyncio.sleep(1)
I want to accept messages sent via channel_layer and send them as events.
I changed the handle method, so it subscribes the new channel to a group. And I'm planning to send messages to the channel layer via channel_layer.group_send
But I couldn't figure out how to get the messages sent to the group, within handle method. I tried awaiting for the channel_layer.receive, it doesn't seem to work.
class ServerSentEventsConsumer(AsyncHttpConsumer):
group_name = 'my_message_group'
async def myevent(self, event):
# according to the docs, this method will be called \
# when a group received a message with type 'myevent'
# I'm not sure how to get this event within `handle` method's while loop.
pass
async def handle(self, body):
await self.channel_layer.group_add(
self.group_name,
self.channel_name
)
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
while True:
payload = "data: %s\n\n" % datetime.now().isoformat()
result = await self.channel_receive()
payload = "data: %s\n\n" % 'received'
I'm sending the messages to channel_layer like below: ( from a management command)
def send_event(event_data):
group_name = 'my_message_group'
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
group_name,
{
'type': 'myevent',
'data': [event_data]
}
)
I had the same issue and I even went to dig into Django Channels code but without success.
... Until I found this answer in this (still opened) issue: https://github.com/django/channels/issues/1302#issuecomment-508896846
That should solve your issue.
In your case the code would be (or something quite similar):
class ServerSentEventsConsumer(AsyncHttpConsumer):
group_name = 'my_message_group'
async def http_request(self, message):
if "body" in message:
self.body.append(message["body"])
if not message.get("more_body"):
await self.handle(b"".join(self.body))
async def myevent(self, event):
# according to the docs, this method will be called \
# when a group received a message with type 'myevent'
# I'm not sure how to get this event within `handle` method's while loop.
pass
async def handle(self, body):
await self.channel_layer.group_add(
self.group_name,
self.channel_name
)
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
TL;DR
Want this flow:
ws://...
websocket client 1 <-----------> websocket client 2
^
|
server (send messages via views)
So I have the following:
views.py
def alarm(request):
layer = get_channel_layer()
async_to_sync(layer.group_send)('events', {
'type': 'events.alarm',
'content': 'triggered'
})
return HttpResponse('<p>Done</p>')
consumers.py
class EventConsumer(JsonWebsocketConsumer):
def connect(self):
print('inside EventConsumer connect()')
async_to_sync(self.channel_layer.group_add)(
'events',
self.channel_name
)
self.accept()
def disconnect(self, close_code):
print('inside EventConsumer disconnect()')
print("Closed websocket with code: ", close_code)
async_to_sync(self.channel_layer.group_discard)(
'events',
self.channel_name
)
self.close()
def receive_json(self, content, **kwargs):
print('inside EventConsumer receive_json()')
print("Received event: {}".format(content))
self.send_json(content)
def events_alarm(self, event):
print('inside EventConsumer events_alarm()')
self.send_json(
{
'type': 'events.alarm',
'content': event['content']
}
)
in routing.py
application = ProtocolTypeRouter({
'websocket': AllowedHostsOriginValidator(
AuthMiddlewareStack(
URLRouter(
chat.routing.websocket_urlpatterns,
)
)
),
})
where websocket_urlpatterns is
websocket_urlpatterns = [
url(r'^ws/chat/(?P<room_name>[^/]+)/$', consumers.ChatConsumer),
url(r'^ws/event/$', consumers.EventConsumer),
]
urls.py
urlpatterns = [
url(r'^alarm/$', alarm, name='alarm'),
]
when I call /alarm/ , only the HTTP request is made and the message is not sent to the websocket
The following are the logs:
[2018/09/26 18:59:12] HTTP GET /alarm/ 200 [0.07, 127.0.0.1:60124]
My intention is to make django view send to a group (use case would be for a server to send notification to all connected members in the group).
What setting am I missing here.
I am running django channels 2.1.3 with redis as backend. The CHANNELS_LAYERS etc. have all been setup.
Reference Links:
sending messages to groups in django channels 2
github issue
channels docs
EDIT:
I could send the message using the websocket-client from the view
from websocket import create_connection
ws = create_connection("ws://url")
ws.send("Hello, World")
But is it possible to send without using the above (donot wan't to create a connection)?
Source code: chat-app
credits to #Matthaus Woolard for making the concept pretty clear.
So this was the problem:
The client had disconnected when I tried to send the message from the django view. This happened as the server restarted upon code change. I refreshed the browser and it started to work.
Silly mistake
So to summarize:
Add the following in connect() in case of Synchronous consumer:
async_to_sync(self.channel_layer.group_add)('events', self.channel_name)
or add this incase of Async Consumer:
await self.channel_layer.group_add('events', self.channel_name)
create a view as follows:
def alarm(request):
layer = get_channel_layer()
async_to_sync(layer.group_send)('events', {
'type': 'events.alarm',
'content': 'triggered'
})
return HttpResponse('<p>Done</p>')