Hi i'm fiddling with django-channels 2 and want to grab the URL parameter and use it in a function, but it seems like it contains a space or something similiar which causes issues with my function. My function takes 1 argument but when I try to pass the URL parameter in it gives me the following error task_status() takes 1 positional argument but 2 were given. I can see when I try to print the URL it prints the correct value but a new line is also created.
Is there any way to grab ONLY the URL parameter and be able to use it directly in a function?
consumers.py
class ChatConsumer(AsyncConsumer):
async def websocket_connect(self, event):
await self.send({
"type": "websocket.accept"
})
user = self.scope['user']
get_task_id = self.scope['url_route']['kwargs']['task_id']
print(get_task_id)
get_info = await self.task_status(get_task_id)
print(get_info)
await self.send({
"type": "websocket.send",
"text": "hey"
})
async def websocket_receive(self, event):
print("receive", event)
async def websocket_disconnect(self, event):
print("disconnected", event)
def task_status(task_id):
command = "golemcli tasks show {}".format(task_id)
taskoutput = subprocess.getoutput(command)
print(taskoutput)
routing.py
from django.urls import re_path
from . import consumers
websocket_urlpatterns = [
re_path(r'ws/dashboard/task/(?P<task_id>[0-9a-f-]+)', consumers.ChatConsumer),
]
website URL http://localhost:8000/dashboard/task/aa3c6c12-5446-11ea-b237-1e0f691c9a55
Your task_status is a method so its first augment should be self, python always adds this augment to the function when you call it that is why you are getting you error. It is calling the method with self and with task_id
To fix this you should define your method like this:
def task_status(self, task_id):
....
on another point you are waiting synchronously in your task_status method this will block the entire server from handling any other trafic.
You should make your task_status an async method and then use an async subprocess comand to start and await for the output https://docs.python.org/3/library/asyncio-subprocess.html#examples
Related
The SQL update does not seem to be being enacted upon, and no errors are being thrown either. Below is a simplified version of my code. For context, the "choice" field in the model is a Boolean Field with a default of False, and a user may (ideally) change this by sending a JSON package with the "CHOICE" event and "Yes" message.
consumers.py
import json
from channels.generic.websocket import AsyncJsonWebsocketConsumer
from asgiref.sync import sync_to_async
from .models import Room
class Consumer(AsyncJsonWebsocketConsumer):
async def connect(self):
self.room_code = self.scope['url_route']['kwargs']['room_code']
#Websockets connection code
async def disconnect(self):
#Websockets disconnect code
async def receive(self, text_data):
response = json.loads(text_data)
event = response.get("event", None)
message = response.get("message", None)
if event == "CHOICE":
room_set = await sync_to_async(Room.objects.filter)(room_code=self.room_code)
room = await sync_to_async(room_set.first)()
if (not room.choice) and message["choice"] == 'Yes':
sync_to_async(room_set.update)(choice=True) #line seems to not be working
elif room.choice and message["choice"] == 'No':
sync_to_async(room_set.update)(choice=False)
#code to send message to group over Websockets
#code regarding other events
async def send_message(self, res):
#Websockets send message code
I've tried to only include the relevant code here, but if more is needed please let me know. Thanks in advance!
I fixed this issue by adding await before the sync_to_async(room.update)(choice=True) lines. It seems without an await it will move onto the next line of code before completing the SQL update, causing the update to not go through.
Problem Summary
I am sending data to a front-end (React component) using Django and web-sockets. When I run the app and send the data from my console everything works. When I use a button on the front-end to trigger a Django view that runs the same function, it does not work and generates a confusing error message.
I want to be able to click a front-end button which begins sending the data to the websocket.
I am new to Django, websockets and React and so respectfully ask you to be patient.
Overview
Django back-end and React front-end connected using Django Channels (web-sockets).
User clicks button on front-end, which does fetch() on Django REST API end-point.
[NOT WORKING] The above endpoint's view begins sending data through the web-socket.
Front-end is updated with this value.
Short Error Description
The error Traceback is long, so it is included at the end of this post. It begins with:
Internal Server Error: /api/run-create
And ends with:
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
What I've Tried
Sending Data Outside The Django View
The function below sends data to the web-socket.
Works perfectly when I run it in my console - front-end updates as expected.
Note: the same function causes the attached error when run from inside the Django view.
import json
import time
import numpy as np
import websocket
def gen_fake_path(num_cities):
path = list(np.random.choice(num_cities, num_cities, replace=False))
path = [int(num) for num in path]
return json.dumps({"path": path})
def fake_run(num_cities, limit=1000):
ws = websocket.WebSocket()
ws.connect("ws://localhost:8000/ws/canvas_data")
while limit:
path_json = gen_fake_path(num_cities)
print(f"Sending {path_json} (limit: {limit})")
ws.send(path_json)
time.sleep(3)
limit -= 1
print("Sending complete!")
ws.close()
return
Additional Detail
Relevant Files and Configuration
consumer.py
class AsyncCanvasConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.group_name = "dashboard"
await self.channel_layer.group_add(self.group_name, self.channel_name)
await self.accept()
async def disconnect(self, close_code):
await self.channel_layer.group_discard(self.group_name, self.channel_name)
async def receive(self, text_data=None, bytes_data=None):
print(f"Received: {text_data}")
data = json.loads(text_data)
to_send = {"type": "prep", "path": data["path"]}
await self.channel_layer.group_send(self.group_name, to_send)
async def prep(self, event):
send_json = json.dumps({"path": event["path"]})
await self.send(text_data=send_json)
Relevant views.py
#api_view(["POST", "GET"])
def run_create(request):
serializer = RunSerializer(data=request.data)
if not serializer.is_valid():
return Response({"Bad Request": "Invalid data..."}, status=status.HTTP_400_BAD_REQUEST)
# TODO: Do run here.
serializer.save()
fake_run(num_cities, limit=1000)
return Response(serializer.data, status=status.HTTP_200_OK)
Relevant settings.py
WSGI_APPLICATION = 'evolving_salesman.wsgi.application'
ASGI_APPLICATION = 'evolving_salesman.asgi.application'
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels.layers.InMemoryChannelLayer"
}
}
Relevant routing.py
websocket_url_pattern = [
path("ws/canvas_data", AsyncCanvasConsumer.as_asgi()),
]
Full Error
https://pastebin.com/rnGhrgUw
EDIT: SOLUTION
The suggestion by Kunal Solanke solved the issue. Instead of using fake_run() I used the following:
layer = get_channel_layer()
for i in range(10):
path = list(np.random.choice(4, 4, replace=False))
path = [int(num) for num in path]
async_to_sync(layer.group_send)("dashboard", {"type": "prep", "path": path})
time.sleep(3)
Rather than creating a new connection from same server to itself , I'd suggest you to use the get_channel_layer utitlity .Because you are in the end increasing the server load by opening so many connections .
Once you get the channel layer , you can simply do group send as we normally do to send evnets .
You can read more about here
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync
def media_image(request,chat_id) :
if request.method == "POST" :
data = {}
if request.FILES["media_image"] is not None :
item = Image.objects.create(owner = request.user,file=request.FILES["media_image"])
message=Message.objects.create(item =item,user=request.user )
chat = Chat.objects.get(id=chat_id)
chat.messages.add(message)
layer = get_channel_layer()
item = {
"media_type": "image",
"url" : item.file.url,
"user" : request.user.username,
'caption':item.title
}
async_to_sync(layer.group_send)(
'chat_%s'%str(chat_id),
#this is the channel group name,which is defined inside your consumer"
{
"type":"send_media",
"item" : item
}
)
return HttpResponse("media sent")
In the error log, I can see that the handshake succeded for the first iteration and failed for 2nd . You can check that by printing something in the for loop . If that's the case the handshake most probably failed due to mulitple connections . I don't know how many connections the Inmemrorycache supports from same origin,but that can be reason that the 2nd connection is getting diconnected . You can get some idea in channel docs.Try using redis if you don't want to change your code,its pretty easy if you are using linux .
I am using a combination of DRF 3.11.0 and Channels 2.4.0 to implement a backend, and it is hosted on Heroku on 1 dyno with a Redis resource attached. I have a socket on my React frontend that successfully sends/received from the backend server.
I am having an issues where any message sent back to the front end over the socket is being sent twice. I have confirmed through console.log that the front end is only pinging the back end once. I can confirm through print() inside of the API call that the function is only calling async_to_sync(channel_layer.group_send) once as well. The issue is coming from my consumer - when I use print(self.channel_name) inside of share_document_via_videocall(), I can see that two instances with different self.channel_names are being called (specific.AOQenhTn!fUybdYEsViaP and specific.AOQenhTn!NgtWxuiHtHBw. It seems like the consumer has connected to two separate channels, but I'm not sure why. When I put print() statements in my connect() I only see it go through the connect process once.
How do I ensure that I am only connected to one channel?
in settings.py:
CHANNEL_LAYERS = {
'default': {
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG': {
#"hosts": [('127.0.0.1', 6379)],
"hosts": [(REDIS_HOST)],
},
},
}
Consumer:
import json
from asgiref.sync import async_to_sync
from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from rest_framework.authtoken.models import Token
from django.contrib.auth.models import AnonymousUser
from .exceptions import ClientError
import datetime
from django.utils import timezone
class HeaderConsumer(AsyncWebsocketConsumer):
async def connect(self):
print("connecting")
await self.accept()
print("starting")
print(self.channel_name)
await self.send("request_for_token")
async def continue_connect(self):
print("continuing")
print(self.channel_name)
await self.get_user_from_token(self.scope['token'])
await self.channel_layer.group_add(
"u_%d" % self.user['id'],
self.channel_name,
)
#... more stuff
async def disconnect(self, code):
await self.channel_layer.group_discard(
"u_%d" % self.user['id'],
self.channel_name,
)
async def receive(self, text_data):
text_data_json = json.loads(text_data)
if 'token' in text_data_json:
self.scope['token'] = text_data_json['token']
await self.continue_connect()
async def share_document_via_videocall(self, event):
# Send a message down to the client
print("share_document received")
print(event)
print(self.channel_name)
print(self.user['id'])
await self.send(text_data=json.dumps(
{
"type": event['type'],
"message": event["message"],
},
))
#database_sync_to_async
def get_user_from_token(self, t):
try:
print("trying token" + t)
token = Token.objects.get(key=t)
self.user = token.user.get_profile.json()
except Token.DoesNotExist:
print("failed")
self.user = AnonymousUser()
REST API call:
class ShareViaVideoChat(APIView):
permission_classes = (permissions.IsAuthenticated,)
def post(self, request, format=None):
data = request.data
recipient_list = data['recipient_list']
channel_layer = get_channel_layer()
for u in recipient_list:
if u['id'] != None:
print("sending to:")
print('u_%d' % u['id'])
async_to_sync(channel_layer.group_send)(
'u_%d' % u['id'],
{'type': 'share_document_via_videocall',
'message': {
'document': {'data': {}},
'sender': {'name': 'some name'}
}
}
)
return Response()
with respect to you getting to calls with different channel names are you sure your frontend has not connected twice to the consumer? Check in the debug console in your browser.
i get same problem with nextjs as a frontend of Django channels WebSocket server.
and after searching i found the problem related with tow things:
1- react strict mode (the request sending twice) :
to disable react strict mode in next.js , go to module name "next.config.js" , and change the value for strict mode to false , as the following :
/** #type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: false,
}
module.exports = nextConfig
2- in nextjs the code run twice (outside useEffect Hook) , one on server side and the second on the client side (which means each user will connect to websocket server twice, and got two channels name , and join to same group twice each time with different channel name . ) ,
so i changed my codes to connect with Django channels server only from client side , if you like see my full code / example , kindly visit the following URL , and note the checking code about "typeof window === "undefined":
frontend nextjs code :
https://stackoverflow.com/a/72288219/12662056
i don't know if my problem same your problem , but i hope that helpful.
Django Channels docs has following basic example of a Server Sent Events. AsyncHttpConsumer
from datetime import datetime
from channels.generic.http import AsyncHttpConsumer
class ServerSentEventsConsumer(AsyncHttpConsumer):
async def handle(self, body):
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
while True:
payload = "data: %s\n\n" % datetime.now().isoformat()
await self.send_body(payload.encode("utf-8"), more_body=True)
await asyncio.sleep(1)
I want to accept messages sent via channel_layer and send them as events.
I changed the handle method, so it subscribes the new channel to a group. And I'm planning to send messages to the channel layer via channel_layer.group_send
But I couldn't figure out how to get the messages sent to the group, within handle method. I tried awaiting for the channel_layer.receive, it doesn't seem to work.
class ServerSentEventsConsumer(AsyncHttpConsumer):
group_name = 'my_message_group'
async def myevent(self, event):
# according to the docs, this method will be called \
# when a group received a message with type 'myevent'
# I'm not sure how to get this event within `handle` method's while loop.
pass
async def handle(self, body):
await self.channel_layer.group_add(
self.group_name,
self.channel_name
)
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
while True:
payload = "data: %s\n\n" % datetime.now().isoformat()
result = await self.channel_receive()
payload = "data: %s\n\n" % 'received'
I'm sending the messages to channel_layer like below: ( from a management command)
def send_event(event_data):
group_name = 'my_message_group'
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
group_name,
{
'type': 'myevent',
'data': [event_data]
}
)
I had the same issue and I even went to dig into Django Channels code but without success.
... Until I found this answer in this (still opened) issue: https://github.com/django/channels/issues/1302#issuecomment-508896846
That should solve your issue.
In your case the code would be (or something quite similar):
class ServerSentEventsConsumer(AsyncHttpConsumer):
group_name = 'my_message_group'
async def http_request(self, message):
if "body" in message:
self.body.append(message["body"])
if not message.get("more_body"):
await self.handle(b"".join(self.body))
async def myevent(self, event):
# according to the docs, this method will be called \
# when a group received a message with type 'myevent'
# I'm not sure how to get this event within `handle` method's while loop.
pass
async def handle(self, body):
await self.channel_layer.group_add(
self.group_name,
self.channel_name
)
await self.send_headers(headers=[
(b"Cache-Control", b"no-cache"),
(b"Content-Type", b"text/event-stream"),
(b"Transfer-Encoding", b"chunked"),
])
I need to postpone sending channels message. Here is my code:
# consumers.py
class ChatConsumer(WebsocketConsumer):
def chat_message(self, event):
self.send(text_data=json.dumps(event['message']))
def connect(self):
self.channel_layer.group_add(self.room_name, self.channel_name)
self.accept()
def receive(self, text_data=None, bytes_data=None):
send_message_task.apply_async(
args=(
self.room_name,
{'type': 'chat_message',
'message': 'the message'}
),
countdown=10
)
# tasks.py
#shared_task
def send_message_task(room_name, message):
layer = get_channel_layer()
layer.group_send(room_name, message)
The task is being executed and I can't see any errors but message is not being sent. It works only if I send it from consumer class method.
I also tried using AsyncWebsocketConsumer and sending with AsyncToSync(layer.group_send). It errors with "You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly."
Then I tried declaring send_message_task as async and using await. Nothing happens again (with no errors) and I'm not sure if the task is executed at all.
Here are versions:
Django==1.11.13
redis==2.10.5
django-celery==3.2.2
channels==2.1.2
channels_redis==2.2.1
Settings:
REDIS_HOST = os.getenv('REDIS_HOST', '127.0.0.1')
BROKER_URL = 'redis://{}:6379/0'.format(REDIS_HOST)
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": ['redis://{}:6379/1'.format(REDIS_HOST)],
},
},
}
Any ideas?
UPD: Just found out that redis channel layer is retreived but it's group_send method is not called and just skipped.
UPD 2: Sending using AsyncToSync(layer.group_send) from console works. Calling task without apply_async also works. But running it with apply_async causes an error You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly. Defining task as async and using await also breaks everything of course.
Maybe this is not direct answer to a starting question but this might help.
If you get exception "You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly" then you probably makes some of this:
event loop is created somewhere
some ASYNC code is started
some SYNC code is called from ASYNC code
SYNC code is trying to call ASYNC code with AsyncToSync that prevents this
Seems that AsyncToSync detects outer event loop and makes decision to not interfere with it.
Solution is to directly include your async call in outer event loop.
Example code is below, but best is to check your situation and that outer loop is running ...
loop = asyncio.get_event_loop()
loop.create_task(layer.group_send(room_name, {'type': 'chat_message', 'message': message}))
You need the async_to_sync() wrapper on connect when using channel layers because all channel layer methods are asynchronous.
def connect(self):
async_to_sync(self.channel_layer.group_add(
self.room_name, self.channel_name)
self.accept()
Same deal with sending the message from your celery task.
#shared_task
def send_message_task(room_name, message):
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
room_name,
{'type': 'chat_message', 'message': message}
)
Also you can just call your celery task from your consumer's receive() like this:
send_message_task.delay(self.room_name, 'your message here')
Regarding the AsyncToSync error you need to upgrade channels and daphne to a newer version as explained in this thread.
I found an ugly and inefficient decision, but it works:
#shared_task
def send_message_task(room_name, message):
def sender(room_name, message):
channel_layer = get_channel_layer()
AsyncToSync(channel_layer.group_send)(
room_name,
{'type': 'chat_message', 'message': message}
)
thread = threading.Thread(target=sender, args=(room_name, message,))
thread.start()
thread.join()
If someone can improve it, I will appreciate.
The problem in your code is that you used underscore in your type chat_message. I believe you missed it in the documentation:
The name of the method will be the type of the event with periods
replaced by underscores - so, for example, an event coming in over the
channel layer with a type of chat.join will be handled by the method
chat_join.
So in your case, the type will be chat.message
{
'type': 'chat.message',
'message': 'the message'
}