I have a frontend consumer that handles HTTP requests (webhooks)
from django.views.decorators.csrf import csrf_exempt
from channels.generic.http import AsyncHttpConsumer
from django.http import JsonResponse
import json
import myapp.models
import redis
#csrf_exempt
class frontEndConsumer(AsyncHttpConsumer):
async def http_request(self, request):
await self.channel_layer.group_add("link", self.channel_name)
await self.channel_layer.group_send("link",
{
"type": "websocket.reg",
"where": "webhook",
"message": "test message",
})
#channel layer functions
async def webhook_reg(self, event):
print("WH: message from " + event["where"] + ": " + event["message"]
# make response
fulfillmentText = "device connected"
fulfillmentText = {'fulfillmentText': fulfillmentText}
fulfillmentText = json.dumps(fulfillmentText).encode('utf-8')
await self.send_response(200, fulfillmentText,
headers=[(b"Content-Type", b"text/plain"),])
and I have a backend consumer that handles websocket connections.
from channels.generic.websocket import AsyncJsonWebsocketConsumer
import redis
import myapp.models
account = redis.Redis(db=0)
class backEndConsumer(AsyncJsonWebsocketConsumer):
async def websocket_connect(self, type):
await self.channel_layer.group_add("link", self.channel_name)
print("channel name: " + self.channel_name)
await self.accept()
#channel layer functions
async def websocket_reg(self, event):
print("WS: message from " + event["where"] + ": " + event["message"])
await self.channel_layer.group_send("link",
{
"type": "webhook.reg",
"where": "websocket",
"message": "msg from websocket",
})
await self.send(event["message"])
I start them in procfile with:
web: daphne APbackend.asgi:application --port $PORT --bind 0.0.0.0
I am using django channels to have the front and back talk to each other. In my code, I'm trying to get both sides to send messages to each other.
I would establish websocket connection
have webhook send a http request to my front end.
front end would send a message to the backend (via channel messaging)
backend would send a message to the frontend.
front end message would finish the http request.
But this doesn't work, step 4, with the message going back to the front end saying the handler is missing. Why is that?
You are using the same group to talk in both directions this means when you send webhook.reg from the backend it will end up trying to call the function webhook_reg on each backEndConsumer instance but there is no such method there.
You should create 2 groups.
backend_to_frontend
frontend_to_backend
the frontend should group_add the backend_to_frontend
the backend should group_add the frontend_to_backend
&
in the backend you should group_send to the backend_to_frontend
in the frontend you should group_send to the frontend_to_backend
I was able to get this to work using either two group names or two channel names. Also knowing http_reponse would end the http_request context helped me debug why I wasn't able to get message back to dialogflow.
Thanks!
Related
I'm using Google Scheduler to send message to the Pub/Sub topic. I want to listen those messages continuously. My code executes only once and it doesn't listen.
main.py
from fastapi import FastAPI, Depends
from typing import List
from core.config import get_db
from sqlalchemy.orm import Session
app = FastAPI()
from concurrent.futures import TimeoutError
from google.cloud import pubsub_v1
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path("project_id", "subscription_id")
def callback(message: pubsub_v1.subscriber.message.Message) -> None:
print(f"Received {message}.")
message.ack()
streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback)
print(f"Listening for messages on {subscription_path}..\n")
with subscriber:
try:
streaming_pull_future.result(timeout=5)
except TimeoutError:
streaming_pull_future.cancel() # Trigger the shutdown.
streaming_pull_future.result() # Block until the shutdown is complete.
#app.get("/")
def home(db: Session = Depends(get_db)):
return {
"message": "Welcome!"
}
Is there a way to listen pubsub messages continuously in FastAPI?
Thanks!
https://cloud.google.com/pubsub/docs/publish-receive-messages-client-library
https://fastapi.tiangolo.com/advanced/websockets/
I've solved problem just by deleting:
with subscriber:
try:
streaming_pull_future.result(timeout=5)
except TimeoutError:
streaming_pull_future.cancel() # Trigger the shutdown.
streaming_pull_future.result() # Block until the shutdown is complete.
Environment:
Ubuntu 16.04.6
conda 4.12.0
Apache/2.4.18 (Ubuntu)
python==3.8.1
Django==4.0.3
channels==3.0.5
asgi-redis==1.4.3
asgiref==3.4.1
daphne==3.0.2
I am attempting to create a websocket service that only relays messages from redis to an authenticated user. Users do not communicate with each other, therefore I don't need Channel layers and my understanding is that Channel layers are an entirely optional part of Channels.
I'm simply trying to broadcast messages specific to a user that has been authenticated through custom middleware. I have custom auth middleware that takes an incoming session id and returns the authenticated user dictionary along with a user_id. I'm also attaching this user_id to the scope.
I have elected to route all traffic through daphne via Apache2 using ProxyPass on port 8033. This is preferred since I'm using a single domain for this service.
However, I'm really struggling to maintain a connection to the websocket server, particularly if I refresh the browser. It will work on the first request, and fail after with the following message in journalctl:
Application instance <Task pending name='Task-22' coro=<ProtocolTypeRouter.__call__() running at /root/miniconda2/lib/python3.8/site-packages/channels/routing.py:71> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f658c03f670>()]>> for connection <WebSocketProtocol client=['127.0.0.1', 46010] path=b'/ws/userupdates/'> took too long to shut down and was killed.
After spending hours on the github for channels, and trying many of the suggestions (particularly found on https://github.com/django/channels/issues/1119), I'm still at a loss. The version of the code below is the best working version so far that at least establishes an initial connection, sends back the connection payload {"success": true, "user_id": XXXXXX, "message": "Connected"} and relays all redis messages successfully. But, if I refresh the browser, or close and reopen, it fails to establish a connection and posts the above journalctl message until I restart apache and daphne.
My hunch is that I'm not properly disconnecting the consumer or I'm missing proper use of async await. Any thoughts?
Relevant apache config
RewriteEngine On
RewriteCond %{HTTP:Connection} Upgrade [NC]
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteRule /(.*) ws://127.0.0.1:8033/$1 [P,L]
<Location />
ProxyPass http://127.0.0.1:8033/
ProxyPassReverse /
</Location>
app/settings.py
[...]
ASGI_APPLICATION = 'app.asgi.application'
ASGI_THREADS = 1000
CHANNEL_LAYERS = {}
[...]
app/asgi.py
import os
from django.core.asgi import get_asgi_application
from django.urls import path
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')
import websockets.routing
from user.models import UserAuthMiddleware
application = ProtocolTypeRouter({
'http': get_asgi_application(),
"websocket": AllowedHostsOriginValidator(
UserAuthMiddleware(
URLRouter(websockets.routing.websocket_urlpatterns)
)
),
})
user/models.py::UserAuthMiddleWare
Pulls user_id from custom authentication layer and attaches user_id to scope.
class UserAuthMiddleware(CookieMiddleware):
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
# Check this actually has headers. They're a required scope key for HTTP and WS.
if "headers" not in scope:
raise UserSessionError(
"UserAuthMiddleware was passed a scope that did not have a headers key "
+ "(make sure it is only passed HTTP or WebSocket connections)"
)
# Go through headers to find the cookie one
for name, value in scope.get("headers", []):
if name == b"cookie":
cookies = parse_cookie(value.decode("latin1"))
break
else:
# No cookie header found - add an empty default.
cookies = {}
# now gather user data from session
try:
req = HttpRequest()
req.GET = QueryDict(query_string=scope.get("query_string"))
setattr(req, 'COOKIES', cookies)
setattr(req, 'headers', scope.get("headers")),
session = UserSession(req)
scope['user_id'] = session.get_user_id()
except UserSessionError as e:
raise e
return await self.app(scope, receive, send)
websockets/routing.py
from django.urls import re_path
from . import consumers
websocket_urlpatterns = [
re_path(r'ws/userupdates/', consumers.UserUpdatesConsumer.as_asgi())
]
websockets/consumers.py::UserUpdatesConsumer
from channels.generic.websocket import JsonWebsocketConsumer
import json, redis
class UserUpdatesConsumer(JsonWebsocketConsumer):
def connect(self):
self.accept()
self.redis = redis.Redis(host='127.0.0.1', port=6379, db=0, decode_responses=True)
self.p = self.redis.pubsub()
if 'user_id' not in self.scope:
self.send_json({
'success': False,
'message': 'No user_id present'
})
self.close()
else:
self.send_json({
'success': True,
'user_id': self.scope['user_id'],
'message': 'Connected'
})
self.p.psubscribe(f"dip_alerts")
self.p.psubscribe(f"userupdates_{self.scope['user_id']}*")
for message in self.p.listen():
if message.get('type') == 'psubscribe' and message.get('data') in [1,2]:
continue
if message.get('channel') == "dip_alerts":
self.send_json({
"key": "dip_alerts",
"event": "dip_alert",
"data": json.loads(message.get('data'))
})
else:
self.send(message.get('data'))
I'm currently working on a custom bot for a Slack workspace to send messages when specific entries are given on a Google Sheet. I'm using a simple Flask setup with the Slack SDK and pydrive. I'm running into an issue where each message that is sent "asynchronusly" (i.e. not triggered by a specific Slack event) gets sent twice when running the Flask application. If I disable the Flask app, I only get one instance of the "Restarting Money Bot" sent. If the server is running, I get two instances, back to back. If the server is running and I respond to a slash command (test_command()), I only get the single response as required. I've attempted to alter the send method (using a raw API call rather than the SDK method) to no avail; I'm not sure where else to go with this issue.
Here's the code that's causing the issues. slack_handle just has simple wrapper functions for the API calls so I can keep Slack tokens and signatures local to one file:
from flask import Flask, request, redirect, make_response, render_template, send_from_directory
import os
import logging
import threading
import time
import json
from flask.wrappers import Response
import slack_handle
import drive_handle
logging.basicConfig(level=logging.DEBUG)
bot = Flask(__name__)
drive_access = drive_handle.gdrive()
#bot.route('/slack/test', methods=["POST"])
def test_command():
if not slack_handle.verifier.is_valid_request(request.get_data(), request.headers):
return make_response("Invalid request", 403)
td_test = threading.Thread(target = test_command_thread)
td_test.start()
return Response(status=200)
def test_command_thread():
slack_handle.sendMoneyBot("Money Bot is active :heavy_dollar_sign:")
slack_handle.sendMoneyBot("Last refreshed " + str(drive_access.access_time) + " minute(s) ago")
slack_handle.sendMoneyBot("Good connection: (" + str(drive_access.access_status) + ")")
# To run prior to server start
slack_handle.sendTreasurer("Restarting Money Bot")
td = threading.Thread(target = drive_access.mainDaemon)
td.daemon = True
td.start()
# Local server debug
if __name__ == "__main__":
port = int(os.environ.get('PORT', 5000))
bot.run(host='0.0.0.0', port=port, debug=True)
Any help would be awesome
I am using django channel in my current project. From one of my django app, I am sending notification to the channel layer, so that websocket can broadcast the message. But problem is
consumer is not getting my message.
Utils in django app for sending notification to channel:
from asgiref.sync import AsyncToSync
from channels.layers import get_channel_layer
import json
def async_send(group_name, text):
channel_layer = get_channel_layer()
AsyncToSync(channel_layer.group_send)(
group_name,
{
'type': 'notify',
'text': json.dumps(text)
}
)
My consumer file is:
from channels.generic.websocket import AsyncWebsocketConsumer
class InformationConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_layer.group_add(str(self.scope['user']), self.channel_name)
await self.accept()
async def notify(self, event):
await self.send(
{
"message": event['text'],
},
)
print(event['text'])
I am supposed to get the output of event['text'], but getting nothing :(
change from
self.channel_layer.group_add(str(self.scope['user']), self.channel_name)
to
await self.channel_layer.group_add(str(self.scope['user']), self.channel_name)
I am trying to set up an app which uses django2.0.2 and channels2.1.1. What I would like to achieve is using a background/worker task to perform some work that will produce data, that should dynamically appear on the website. My problem, related primarily to channels, is: how do I correctly establish communication between the worker, and the consumer connected to a websocket?
Below is a minimal example highlighting the issue: The idea is that the user triggers the worker, the worker produces some data and sends it, via the channel layer, to a consumer that is connected to the websocket.
#routing.py
from channels.routing import ChannelNameRouter, ProtocolTypeRouter, URLRouter
from channels.auth import AuthMiddlewareStack
from django.urls import path
from testApp.consumers import *
application = ProtocolTypeRouter({
"websocket":AuthMiddlewareStack(
URLRouter([
path("wspath",TestConsumer),
]),
),
"channel":ChannelNameRouter({
"test_worker": TestWorker,
}),
})
The consumers:
#consumers.py
from channels.consumer import SyncConsumer
from channels.generic.websocket import WebsocketConsumer
from asgiref.sync import async_to_sync
class TestConsumer(WebsocketConsumer):
def websocket_connect(self,message):
async_to_sync(self.channel_layer.group_add)("testGroup",self.channel_name)
self.connect()
#I understand this next part is a bit weird, but I figured it
#is the most concise way to explain my problem
async_to_sync(self.channel_layer.group_send)(
"testGroup",
{
'type':"echo_msg",
'msg':"sent from WebsocketConsumer",
})
def echo_msg(self, message):
print("Message to WebsocketConsumer", message)
class TestWorker(SyncConsumer):
def triggerWorker(self, message):
async_to_sync(self.channel_layer.group_add)("testGroup",self.channel_name)
async_to_sync(self.channel_layer.group_send)(
"testGroup",
{
'type':"echo_msg",
'msg':"sent from worker",
})
def echo_msg(self, message):
print("Message to worker ", message)
The view
#views.py
from django.shortcuts import render
import channels.layers
from asgiref.sync import async_to_sync
def index(request):
if request.method == "POST":
channel_layer = channels.layers.get_channel_layer()
async_to_sync(channel_layer.send)('test_worker',{
'type':'triggerWorker',
})
return render(
request,
"index.html",
{})
and the html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<script>
console.log('ws://' + window.location.host)
var socket = new WebSocket(
'ws://' + window.location.host + "/wspath"
);
</script>
</head>
<div>Click to run worker</div>
<body>
<form action="" method="POST">
{% csrf_token %}
<button type="submit">Start</button>
</form>
</body>
Now, when I run this by executing (in separate consoles)
python3 manage.py runserver
and
python3 manage.py runworker test_worker
and then trigger the worker, the runserver console outputs:
Message to WebsocketConsumer {'type': 'echo_msg', 'msg': 'sent from WebsocketConsumer'}
where as the runworker console outputs:
Message to worker {'type': 'echo_msg', 'msg': 'sent from worker'}
Message to worker {'type': 'echo_msg', 'msg': 'sent from WebsocketConsumer'}
So I can send stuff worker -> worker, WebsocketConsumer -> WebsocketConsumer, WebsocketConsumer -> worker.
From what I understand (which is evidently wrong) there should have also been a message worker -> WebsocketConsumer, because I added both to the "testGroup".
So, my question is why didn't the WebsocketConsumer receive anything from the worker (which is what I am interested in, to eventually establish communication with the javaScript)? Or, in other words, why can I only send stuff from the WebsocketConsumer to the worker, and not vice versa?
I'm running your code. I can see everything is working.
You have that initial message going on POST - that is working - you add it to the group. When the websocket connects it sends a message to the worker. You can see that being received in your runworker terminal. That is bouncing it back to your consumer and is being printed out in the terminal where you run runserver. To get it back to your web browser you need to write:
def echo_msg(self, message):
print("Message to WebsocketConsumer", message)
self.send(json.dumps(message))
Open your developer tools in Chrome to see it coming back. Go to networking > select the websocket connection > then click frames.
BTW You don't need to add test worker to the same group over and over again. The worker is always running.
Another BTW: If your groups are going to be named the same for all users, you don't need to add your worker to a group. You can send messages directly to your worker by it's route name (send instead of group_send). The worker can send messages back to the group without it being added to the group. You only need to add the websocket consumers to groups.
Also, if you don't want multiple users seeing the same messages you don't need groups at all. Just send the messages to the worker with a channel name (self.channel_name) to send it back to.
Also you'll probably want to work with json consumers rather than parsing the messages yourself but that is up to you.
Your websocket is never being used. You are sending a message from your view to the worker - not from your consumer to your worker. The view is sending that message when you post data to it.
The URL you have in your JavaScript is /wspath/ and the URL you have registered on your consumer is chat/stream. The websocket never connects.
Also, your worker will be added to the same group over and over again with your current setup. You only need to add the worker to the group once.
If this answer helped you please mark it correct.