I'm new to Django channels. I created a WebSocket and I want to send some info to every user periodically (broadcast periodically).
Each user connects to the WebSocket when goes to my website and see those info on top of each page.
I have no idea whether it is necessary to create a group or not, and if it is necessary, how to create it...
So how can I send info to every visitor periodically(in an efficient way)?
May be something like this:
class Consumer(AsyncConsumer):
async def websocket_connect(self, event):
await self.send({
"type": "websocket.accept",
})
while True:
await self.channel_layer.group_send(
{
"text": 'Hello user!'
}
)
# OR
await self.send({
"type": "websocket.send",
"text": 'Hello user!'
})
await asyncio.sleep(5)
Thanks in advance...
Related
I want to connect my Django WebSocket with a third-party web socket. This program is one I wrote, and it functions properly. To avoid having to re-login with the third-party API, I have now added the code to check whether the same room is present in my database. if we use the same API KEY to re-connect to the third-party API. It gives the following error:
{"event":"login","status":401,"message":"Connected from another location"}
I want to see if the same cryptocurrency coin is already connected or not. We don't want to logon with the same API KEY once we're connected. I have two issues here:
Don't send the login request to that web socket again.
Don't send the subscribe request, if the same coin already exists. Let's say BTCUSD already connected and giving me the data. I want to just connect to the next user to same room and get the data on next request.
import websocket
import time
import ssl
from channels.generic.websocket import AsyncWebsocketConsumer
from .models import Room
login = {
"event": "login",
"data": {
"apiKey": "API_KEY",
},
}
class CryptoConsumer(AsyncWebsocketConsumer):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.ws = websocket.WebSocket(sslopt={"cert_reqs": ssl.CERT_NONE})
self.ws.connect("wss://crypto.financialmodelingprep.com")
async def connect(self):
self.room_name = self.scope["url_route"]["kwargs"]["room_name"]
self.room_group_name = "crypto_%s" % self.room_name
# ****** Code Block ******
subscribe = {
"event": "subscribe",
"data": {
"ticker": self.room_name,
},
}
room = await Room.add(self.room_name) # Method in models.py to add the user and return True
if room is False:
self.ws.send(json.dumps(login))
print("The group with the name %s doesn't exist" % self.room_group_name)
time.sleep(1)
self.ws.send(json.dumps(subscribe))
# ****** End Code Block ******
# Join room group
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
async def disconnect(self, close_code):
unsubscribe = {
"event": "unsubscribe",
"data": {
"ticker": self.room_name,
},
}
self.ws.send(json.dumps(unsubscribe))
self.ws.close()
# Leave room group
await self.channel_layer.group_discard(self.room_group_name, self.channel_name)
# Receive message from WebSocket
async def receive(self, text_data="{'text': 'Dummy Text'}"):
text_data_json = json.loads(text_data)
message = text_data_json["message"]
message = str(self.ws.recv())
# Send message to room group
await self.channel_layer.group_send(
self.room_group_name, {"type": "chat_message", "message": message}
)
# Receive message from room group
async def chat_message(self, event):
message = event["message"]
# Send message to WebSocket
await self.send(text_data=json.dumps({"message": message}))
Note: Why I want to do this entire step because we don't want the API KEY made public.
So, the code coming from the front end will connect to our Django web socket, and then we'll connect to the third-party web socket and return the data that was sent by them.
I am using Django Channels and RabbitMQ pika, for the first time. I am trying to consume from RabbitMQ queue. I am using Django Channels AsyncConsumer to group send it to everyone connected in the websocket.
User type 1 : Can create a task
User type 2 : Can accept the task.
Use case : When user type 1 creates the task it is published in the rabbitmq. When it is consumed from the queue, it has to be group-sent to frontend. And when the user type 2 accepts the task other instances of user type 2 cannot accept the same and we consume from the queue again and send the next task in the queue to everyone.
I have created the connection in a different thread using sync_to_async I am appending it to an in-memory list from the callback function.
And whenever someone accepts I just pop it out of the list and acknowledge the queue.
class AcceptTaskConsumer(AsyncConsumer):
body = [] #IN MEMORY LIST
delivery = {} #To store ack delivery_tag
async def websocket_connect(self, event):
print("AcceptTaskConsumer connected", event)
AcceptTaskConsumer.get_task() #STARTS Queue listener in new thread
self.room_group_name = "user_type_2"
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.send({
"type": "websocket.accept"
})
async def websocket_receive(self, event):
if event["text"] == "Hi": #If connecting first time
if AcceptTaskConsumer.body:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": AcceptTaskConsumer.body[0]["body"]
}
)
else:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": "No New Tasks"
}
)
else: #When someone accepts a task-> ack and send next task in queue
print(json.loads(event["text"])["id"])
AcceptTaskConsumer.channel.basic_ack(delivery_tag=AcceptTaskConsumer.delivery[json.loads(event["text"])["id"]])
AcceptTaskConsumer.delivery.pop(json.loads(event["text"])["id"])
AcceptTaskConsumer.body.pop(0)
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": "No New Tasks"
}
)
if AcceptTaskConsumer.body:
await self.channel_layer.group_send(
self.room_group_name,
{
"type": "message",
"text": AcceptTaskConsumer.body[0]["body"]
}
)
async def message(self, event):
await self.send({
"type": "websocket.send",
"text": event["text"]
})
#classmethod
#sync_to_async
def get_task(cls): #pika consumer
cls.connection = pika.BlockingConnection(
pika.ConnectionParameters(host='localhost'))
cls.channel = cls.connection.channel()
cls.channel.queue_declare(queue='task_', arguments={"x-max-priority": 3})
cls.channel.basic_consume(
queue='task_', on_message_callback=AcceptTaskConsumer.callback, auto_ack=False)
cls.channel.start_consuming()
#classmethod
def callback(cls, ch, method, properties, body):
task_obj = {"body": json.dumps(body.decode("utf-8")),
"delivery_tag": method.delivery_tag}
AcceptTaskConsumer.body.append(task_obj)
AcceptTaskConsumer.delivery[json.loads(json.loads(task_obj["body"]))["id"]] = method.delivery_tag
cls.channel.stop_consuming()
async def websocket_disconnect(self, event):
print(event)
await self.send({
"type": "websocket.close"
})
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
I am pretty sure this is not the right way to do it, because it's not working as expected
I run into frequent errors like.
39 of 169 channels over capacity in group delivery
pika.exceptions.StreamLostError: Stream connection lost: BrokenPipeError(32, 'Broken pipe')
I tried running the queue listener like this answer as well. Nothing working.
Any one experienced has any thoughts about this? Is there a better way to approach this problem.?
you should move the rabitMQ cosumering logic out of the websocket consumer.
Just have a django command that runs the Rabbit Consumer, that consumer can take messages from RabbitMQ and then use send_group to send them over groups to channels.
if your django command you will need to call send_group see https://channels.readthedocs.io/en/latest/topics/channel_layers.html#using-outside-of-consumers
from channels.layers import get_channel_layer
channel_layer = get_channel_layer()
async_to_sync(
channel_layer.group_send
)(
"user_type_2",
{"type": "message", "msg": 123}
)
Then in the websocket consumer you should subscribe to the groups that the user wants/has permition to get.
I am using a combination of DRF 3.11.0 and Channels 2.4.0 to implement a backend, and it is hosted on Heroku on 1 dyno with a Redis resource attached. I have a socket on my React frontend that successfully sends/received from the backend server.
I am having an issues where any message sent back to the front end over the socket is being sent twice. I have confirmed through console.log that the front end is only pinging the back end once. I can confirm through print() inside of the API call that the function is only calling async_to_sync(channel_layer.group_send) once as well. The issue is coming from my consumer - when I use print(self.channel_name) inside of share_document_via_videocall(), I can see that two instances with different self.channel_names are being called (specific.AOQenhTn!fUybdYEsViaP and specific.AOQenhTn!NgtWxuiHtHBw. It seems like the consumer has connected to two separate channels, but I'm not sure why. When I put print() statements in my connect() I only see it go through the connect process once.
How do I ensure that I am only connected to one channel?
in settings.py:
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
#"hosts": [('127.0.0.1', 6379)],
"hosts": [(REDIS_HOST)],
},
},
}
Consumer:
import json
from asgiref.sync import async_to_sync
from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from rest_framework.authtoken.models import Token
from django.contrib.auth.models import AnonymousUser
from .exceptions import ClientError
import datetime
from django.utils import timezone
class HeaderConsumer(AsyncWebsocketConsumer):
async def connect(self):
print("connecting")
await self.accept()
print("starting")
print(self.channel_name)
await self.send("request_for_token")
async def continue_connect(self):
print("continuing")
print(self.channel_name)
await self.get_user_from_token(self.scope['token'])
await self.channel_layer.group_add(
"u_%d" % self.user['id'],
self.channel_name,
)
#... more stuff
async def disconnect(self, code):
await self.channel_layer.group_discard(
"u_%d" % self.user['id'],
self.channel_name,
)
async def receive(self, text_data):
text_data_json = json.loads(text_data)
if 'token' in text_data_json:
self.scope['token'] = text_data_json['token']
await self.continue_connect()
async def share_document_via_videocall(self, event):
# Send a message down to the client
print("share_document received")
print(event)
print(self.channel_name)
print(self.user['id'])
await self.send(text_data=json.dumps(
{
"type": event['type'],
"message": event["message"],
},
))
#database_sync_to_async
def get_user_from_token(self, t):
try:
print("trying token" + t)
token = Token.objects.get(key=t)
self.user = token.user.get_profile.json()
except Token.DoesNotExist:
print("failed")
self.user = AnonymousUser()
REST API call:
class ShareViaVideoChat(APIView):
permission_classes = (permissions.IsAuthenticated,)
def post(self, request, format=None):
data = request.data
recipient_list = data['recipient_list']
channel_layer = get_channel_layer()
for u in recipient_list:
if u['id'] != None:
print("sending to:")
print('u_%d' % u['id'])
async_to_sync(channel_layer.group_send)(
'u_%d' % u['id'],
{'type': 'share_document_via_videocall',
'message': {
'document': {'data': {}},
'sender': {'name': 'some name'}
}
}
)
return Response()
with respect to you getting to calls with different channel names are you sure your frontend has not connected twice to the consumer? Check in the debug console in your browser.
i get same problem with nextjs as a frontend of Django channels WebSocket server.
and after searching i found the problem related with tow things:
1- react strict mode (the request sending twice) :
to disable react strict mode in next.js , go to module name "next.config.js" , and change the value for strict mode to false , as the following :
/** #type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: false,
}
module.exports = nextConfig
2- in nextjs the code run twice (outside useEffect Hook) , one on server side and the second on the client side (which means each user will connect to websocket server twice, and got two channels name , and join to same group twice each time with different channel name . ) ,
so i changed my codes to connect with Django channels server only from client side , if you like see my full code / example , kindly visit the following URL , and note the checking code about "typeof window === "undefined":
frontend nextjs code :
https://stackoverflow.com/a/72288219/12662056
i don't know if my problem same your problem , but i hope that helpful.
Take a look at this example.
As you can see, some sort of event is constantly being sent to the client. I want to imitate this using Django-Channels, inside consumers.py. Here's a simplified version of what I have:
class ChatConsumer(AsyncConsumer):
async def ws_connect(self, event):
self.send = get_db_object()
....
await self.send({
"type": "websocket.accept"
})
# I need to CONSTANTLY receive & send data
async def ws_receive(self, event):
obj = ...# query DB and get the newest object
json_obj = {
'field_1': obj.field_1,
'field_2': obj.field_2,
}
await self.send({
"type": "websocket.send",
"text": json.dumps(json_obj)
})
#database_sync_to_async
def get_db_object(self, **kwargs):
return Some_Model.objects.get(**kwargs)[0]
Here, I want my Django backend to constantly:
Query DB
Receive obj from DB
Send the received obj to Front-End WebSocket as event
How can I achieve this? The important thing is that I need to CONSTANTLY send data to the client.
Most of the Django-Channels resources on the internet cover only Chat Apps, which don't necessarily constantly send data to the client. I couldn't find any working code that does this job.
Please, no more recommendation for Redis or channels documentation... or some random 3rd party libraries that lacks good documentation... It's easy to recommend but hard to implement. For example, I found someone recommending Snorky, but it really lacks documentation on how to implement it.
However, if there's a website that specifically does this job, I might take a look at it, even if it doesn't use Django-Channels.
consumers.py
import asyncio
from channels.consumer import AsyncConsumer
class ChatConsumer(AsyncConsumer):
async def websocket_connect(self, event):
self.connected = True
print("connected", event)
await self.send({
"type": "websocket.accept"
})
while self.connected:
await asyncio.sleep(2)
obj = # do_something (Ex: constantly query DB...)
await self.send({
'type': 'websocket.send',
'text': # obj,
})
async def websocket_receive(self, event):
print("receive", event)
async def websocket_disconnect(self, event):
print("disconnected", event)
self.connected = False
Javascript
var loc = window.location;
var wsStart = 'ws://';
if (loc.protocol == 'https:') {
wsStart = 'wss://'
}
var endpoint = wsStart + loc.host + loc.pathname;
var socket = new WebSocket(endpoint);
socket.onmessage = function(e){
console.log("message", e);
};
socket.onopen = function(e){
console.log("open", e);
};
socket.onerror = function(e){
console.log("error", e)
};
socket.onclose = function(e){
console.log("close", e)
};
All you need to do is just modify obj and send it. You can extend this function as much as you want. So, right now I'm interested in getting the latest inserted row in my PostgreSQL and injecting that row into my WebSocket. I can query my DB every 2 seconds as it was specified by await asyncio.sleep(2), and inject it into the Front-End socket.
Using channels==1.* and Django==1.* you can use the threading module for example:
# Some view.py
import threading
import time
class Publisher(threading.Thread):
def __init__(self, reply_channel, frequency=0.5):
super(Publisher, self).__init__()
self._running = True
self._reply_channel = reply_channel
self._publish_interval = 1.0 / frequency
def run(self):
while self._running:
self._reply_channel.send({'text': 'some data'})
time.sleep(self._publish_interval)
def stop(self):
self._running = False
publishers = {}
def ws_connect(message):
message.reply_channel.send({'accept': True})
publisher = Publisher(reply_channel=message.reply_channel)
publisher.start()
publishers[message.reply_channel] = publisher
def ws_disconnect(message):
publisher = publishers[message.reply_channel]
publisher.stop()
del publishers[message.reply_channel]
A little late to the party here, but when it comes to Django you should always try to do things "their way" first...
So, to do this I simply used Django Channels. My client sends a message to the server, which the server then responds to with the needed database info. It looks as follows:
class SettingsConsumer(WebsocketConsumer):
def connect(self):
self.accept()
def disconnect(self, close_code):
pass
def receive(self, text_data):
settings = get_settings(self.scope["user"])
self.send(
text_data=json.dumps(
{
"playMode": settings.play_mode,
"isRecording": settings.is_recording,
}
)
)
Now, as for the JS to trigger constant events... I simply use SetInterval to request updates from the consumer every .25s!
My logic is that it's the client so if it's doing a little extra work no biggie, as the server is gonna be responding anyways. The JS looks as follows...
const chatSocket = new WebSocket(
'ws://'
+ window.location.host
+ '/ws/macros/update/'
);
chatSocket.onmessage = function(e) {
const data = JSON.parse(e.data);
if(data["playMode"] == true) {
$('#playmode-checkbox').attr('checked', true);
} else {
$('#playmode-checkbox').attr('checked', false);
}
if(data["isRecording"] == true) {
$('#recording-checkbox').attr('checked', true);
} else {
$('#recording-checkbox').attr('checked', false);
}
};
chatSocket.onclose = function(e) {
console.error('Chat socket closed unexpectedly');
};
window.setInterval(function() {
chatSocket.send(JSON.stringify({
'message': "start"
}));
}, 250);
And yes, you can do more to make this more async-friendly and optimized. But, you asked for simple and working so I hope this helps!
I need to postpone sending channels message. Here is my code:
# consumers.py
class ChatConsumer(WebsocketConsumer):
def chat_message(self, event):
self.send(text_data=json.dumps(event['message']))
def connect(self):
self.channel_layer.group_add(self.room_name, self.channel_name)
self.accept()
def receive(self, text_data=None, bytes_data=None):
send_message_task.apply_async(
args=(
self.room_name,
{'type': 'chat_message',
'message': 'the message'}
),
countdown=10
)
# tasks.py
#shared_task
def send_message_task(room_name, message):
layer = get_channel_layer()
layer.group_send(room_name, message)
The task is being executed and I can't see any errors but message is not being sent. It works only if I send it from consumer class method.
I also tried using AsyncWebsocketConsumer and sending with AsyncToSync(layer.group_send). It errors with "You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly."
Then I tried declaring send_message_task as async and using await. Nothing happens again (with no errors) and I'm not sure if the task is executed at all.
Here are versions:
Django==1.11.13
redis==2.10.5
django-celery==3.2.2
channels==2.1.2
channels_redis==2.2.1
Settings:
REDIS_HOST = os.getenv('REDIS_HOST', '127.0.0.1')
BROKER_URL = 'redis://{}:6379/0'.format(REDIS_HOST)
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": ['redis://{}:6379/1'.format(REDIS_HOST)],
},
},
}
Any ideas?
UPD: Just found out that redis channel layer is retreived but it's group_send method is not called and just skipped.
UPD 2: Sending using AsyncToSync(layer.group_send) from console works. Calling task without apply_async also works. But running it with apply_async causes an error You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly. Defining task as async and using await also breaks everything of course.
Maybe this is not direct answer to a starting question but this might help.
If you get exception "You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly" then you probably makes some of this:
event loop is created somewhere
some ASYNC code is started
some SYNC code is called from ASYNC code
SYNC code is trying to call ASYNC code with AsyncToSync that prevents this
Seems that AsyncToSync detects outer event loop and makes decision to not interfere with it.
Solution is to directly include your async call in outer event loop.
Example code is below, but best is to check your situation and that outer loop is running ...
loop = asyncio.get_event_loop()
loop.create_task(layer.group_send(room_name, {'type': 'chat_message', 'message': message}))
You need the async_to_sync() wrapper on connect when using channel layers because all channel layer methods are asynchronous.
def connect(self):
async_to_sync(self.channel_layer.group_add(
self.room_name, self.channel_name)
self.accept()
Same deal with sending the message from your celery task.
#shared_task
def send_message_task(room_name, message):
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
room_name,
{'type': 'chat_message', 'message': message}
)
Also you can just call your celery task from your consumer's receive() like this:
send_message_task.delay(self.room_name, 'your message here')
Regarding the AsyncToSync error you need to upgrade channels and daphne to a newer version as explained in this thread.
I found an ugly and inefficient decision, but it works:
#shared_task
def send_message_task(room_name, message):
def sender(room_name, message):
channel_layer = get_channel_layer()
AsyncToSync(channel_layer.group_send)(
room_name,
{'type': 'chat_message', 'message': message}
)
thread = threading.Thread(target=sender, args=(room_name, message,))
thread.start()
thread.join()
If someone can improve it, I will appreciate.
The problem in your code is that you used underscore in your type chat_message. I believe you missed it in the documentation:
The name of the method will be the type of the event with periods
replaced by underscores - so, for example, an event coming in over the
channel layer with a type of chat.join will be handled by the method
chat_join.
So in your case, the type will be chat.message
{
'type': 'chat.message',
'message': 'the message'
}