django channel set up routing - django

I'm learning how to setup a websocket connection with django-channel, i've got this setup in routing.py
from channels import route
from gallery import socket as sock
channel_routing = [
# Wire up websocket channels to our consumers:
route("websocket.connect", sock.ws_connect, path=r"^/render-status/$"),
route("websocket.receive", sock.ws_receive, , path=r"^/render-status/$"),
]
and the following javascript
// When we're using HTTPS, use WSS too.
var ws_scheme = window.location.protocol == "https:" ? "wss" : "ws";
var ws_path = ws_scheme + '://' + window.location.host + '/render-status/';
console.log("Connecting to " + ws_path)
var socket = new ReconnectingWebSocket(ws_path);
socket.onmessage = function(message) {
console.log("Got message: " + message.data);
var data = JSON.parse(message.data);
// if action is started, add new item to table
if (data.action == "started") {
}
// if action is completed, just update the status
else if (data.action == "completed"){
}
};
var message = {
action: "incomplete",
job_name: "10",
};
socket.send(JSON.stringify(message));
tried it out and there's a failure in connecting (from the console)
colorpicker2.js:565 Connecting to ws://127.0.0.1:8000/render-status/
reconnecting-websocket.min.js:1 Uncaught DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state.
at a.send (http://127.0.0.1:8000/static/js/reconnecting-websocket.min.js:1:2611)
at HTMLDocument.<anonymous> (http://127.0.0.1:8000/static/js/colorpicker2.js:584:11)
at k (http://127.0.0.1:8000/static/js/jquery.js:15:16962)
at Object.fireWith [as resolveWith] (http://127.0.0.1:8000/static/js/jquery.js:15:17720)
at Function.ready (http://127.0.0.1:8000/static/js/jquery.js:15:12435)
at HTMLDocument.D (http://127.0.0.1:8000/static/js/jquery.js:15:9840)
send # reconnecting-websocket.min.js:1
(anonymous) # colorpicker2.js:584
k # jquery.js:15
fireWith # jquery.js:15
ready # jquery.js:15
D # jquery.js:15
reconnecting-websocket.min.js:1 WebSocket connection to 'ws://127.0.0.1:8000/render-status/' failed: Error during WebSocket handshake: Unexpected response code: 404
open # reconnecting-websocket.min.js:1
a # reconnecting-websocket.min.js:1
(anonymous) # colorpicker2.js:566
k # jquery.js:15
fireWith # jquery.js:15
ready # jquery.js:15
D # jquery.js:15
22reconnecting-websocket.min.js:1 WebSocket connection to 'ws://127.0.0.1:8000/render-status/' failed: Error during WebSocket handshake: Unexpected response code: 404
open # reconnecting-websocket.min.js:1
(anonymous) # reconnecting-websocket.min.js:1
I've also checked the documentation https://channels.readthedocs.io/en/stable/routing.html just to be sure this is how you set a path.

Your routing is correct, but you are sending the message too soon, while the socket is still trying to connect. Use the onopen callback to ensure the message is sent only after the connection was established:
socket.onopen = function() {
var message = {
action: "incomplete",
job_name: "10",
};
socket.send(JSON.stringify(message));
}

Related

AWS API Gateway Invoke URL link works in browser but not with XMLHttpRequest

Whenever I use the following to try and load the json:
'''
<script>
function shutDown() {
let request = new XMLHttpRequest();
request.open("GET", "https://er2uqo9cyc.execute-api.us-east-1.amazonaws.com/testing/ec2");
request.send();
request.onload = () => {
console.log(request);
if (request.status == 200) {
console.log(JSON.parse(request.response));
} else {
console.log('error ${request.status} ${request.statusText}');
}
}
}
</script>
'''
I get the following error in the console of my browser, and no JSON:
'''
[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (styles.css, line 0)
[Error] Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. Status code: 200
[Error] XMLHttpRequest cannot load https://er2uqo9cyc.execute-api.us-east-1.amazonaws.com/testing/ec2 due to access control checks.
[Error] Failed to load resource: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. Status code: 200 (ec2, line 0)
'''
Why isn't it working with XMLHttpRequest?

Communication between Flask and ESP8266 via SocketIO (Updated 2x)

I have a small web app for which the back-end is a Flask+SocketIO server. I would like to get some data from an ESP8266 into my app. The most simple way to achieve this I could think of was to have the micro controller connected directly to the back-end.
I am using the timum-viw library with this example code to implement the client on the micro controller.
The problem is that on trying to run the example I get
(12765) accepted ('192.168.0.11', 59848)
192.168.0.11 - - [06/Jul/2020 18:15:25] "GET /socket.io/?transport=websocket HTTP/1.1" 400 122 0.000265
192.168.0.11 - - [06/Jul/2020 18:15:31] code 400, message Bad request syntax ('This is a webSocket client!')
192.168.0.11 - - [06/Jul/2020 18:15:31] "This is a webSocket client!" 400 -
in the terminal window of the dev server. (The IP belongs to the ESP8266.)
I have the same experience with the arduinoWebSockets library and the WebSocketClientSocketIO example.
Can you help me figure out what the problem is?
Update
Everything is hosted locally at this point. I am running the flask dev server with python3 flask_main.py, eventlet is installed.
The minimal code that manifests the problem:
Arduino:
#include <SocketIoClient.h>
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <Hash.h>
#define USE_SERIAL Serial
#define SSID_primary "**********"
#define WIFI_PWD_primary "**********"
#define SERVER_IP "192.168.0.7"
#define SERVER_PORT 5005
ESP8266WiFiMulti wifiMulti;
SocketIoClient socketIOClient;
void setup() {
//// set up serial communication
USE_SERIAL.begin(115200);
USE_SERIAL.setDebugOutput(true);
for(uint8_t t = 4; t > 0; t--) {
USE_SERIAL.printf("[SETUP] BOOT WAIT %d...\n", t);
USE_SERIAL.flush();
delay(1000);
}
//// connect to some access point
wifiMulti.addAP(SSID_primary, WIFI_PWD_primary);
while(wifiMulti.run() != WL_CONNECTED) {
delay(500);
USE_SERIAL.print("Looking for WiFi ");
}
USE_SERIAL.printf("Connected to %s\n", WiFi.SSID().c_str());
USE_SERIAL.printf("My local IP address is %s\n", WiFi.localIP().toString().c_str());
//// set up socket communication
socketIOClient.begin(SERVER_IP, SERVER_PORT);
}
void loop() {
socketIOClient.emit("message", "\"hi there :)\"");
socketIOClient.loop();
delay(1000);
}
Flask minimal code:
from flask import Flask, render_template, request
from flask_socketio import SocketIO
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
#socketio.on('message')
def handle_message_event(msg):
print('received msg from {} : {}'.format(request.remote_addr, str(msg)))
if __name__ == '__main__':
socketio.run(app, host="0.0.0.0", port=5005, debug=True)
The code below is for debugging only. I do not wish to use them in any form later.
Weirdly enough the Arduino code works fine with a node.js server:
var app = require('express')();
var http = require('http').createServer(app);
var io = require('socket.io')(5005);
io.attach(http, {
pingInterval: 10000,
pingTimeout: 5000,
cookie: false
});
io.on('connection', function (socket) {
console.log('user connected');
socket.on('disconnect', function () {
console.log('user disconnected');
});
socket.on('message', function (msg) {
console.log("message: "+msg);
});
timeout();
});
http.listen();
Could there be something wrong with my Flask? It responds to connections from this:
from socketIO_client import SocketIO, LoggingNamespace
socketIO = SocketIO('localhost', 5005, LoggingNamespace)
while True:
_ = raw_input("> ")
socketIO.emit('message', "hello 2")
But the node server does not!
Update 2
So I went ahead and looked at the communication with wire shark:
Python client & Flask server (works)
The payload of frame 27:
Hypertext Transfer Protocol
GET /socket.io/?EIO=3&transport=websocket&sid=8f47e9a521404b66b23cd985cdee049d HTTP/1.1\r\n
Upgrade: websocket\r\n
Host: localhost:5005\r\n
Origin: http://localhost:5005\r\n
Sec-WebSocket-Key: TQ589ew7EgwDILWb50Eu9Q==\r\n
Sec-WebSocket-Version: 13\r\n
Connection: upgrade\r\n
Connection: keep-alive\r\n
Accept-Encoding: gzip, deflate\r\n
Accept: */*\r\n
User-Agent: python-requests/2.18.4\r\n
\r\n
[Full request URI: http://localhost:5005/socket.io/?EIO=3&transport=websocket&sid=8f47e9a521404b66b23cd985cdee049d]
[HTTP request 1/1]
[Response in frame: 29]
Doing the same with the arduino & flask (does not work)
The payload of frame 34:
Hypertext Transfer Protocol
GET /socket.io/?transport=websocket HTTP/1.1\r\n
Host: 192.168.0.7:5005\r\n
Connection: Upgrade\r\n
Upgrade: websocket\r\n
Sec-WebSocket-Version: 13\r\n
Sec-WebSocket-Key: D9+/7YOHoA8lW7a/0V8vsA==\r\n
Sec-WebSocket-Protocol: arduino\r\n
Origin: file://\r\n
User-Agent: arduino-WebSocket-Client\r\n
\r\n
[Full request URI: http://192.168.0.7:5005/socket.io/?transport=websocket]
[HTTP request 1/1]
[Response in frame: 36]
So it turns out that Flask freaks out about the
Origin: file://\r\n
part because it thinks it is CORS. This is why this answer actually works, however I think it is the wrong fix. Removing this extra header entry is the right way to go about this. This is most simply done by editing this line to match this:
_client.extraHeaders = WEBSOCKETS_STRING("");
in your local library.
There goes hours of research :D

Channels websocket AsyncJsonWebsocketConsumer disconnect not reached

I have the following consumer:
class ChatConsumer(AsyncJsonWebsocketConsumer):
pusher = None
async def connect(self):
print(self.scope)
ip = self.scope['client'][0]
print(ip)
self.pusher = await self.get_pusher(ip)
print(self.pusher)
await self.accept()
async def disconnect(self, event):
print("closed connection")
print("Close code = ", event)
await self.close()
raise StopConsumer
async def receive_json(self, content):
#print(content)
if 'categoryfunctionname' in content:
await cellvoltage(self.pusher, content)
else:
print("ERROR: Wrong data packets send")
print(content)
#database_sync_to_async
def get_pusher(self, ip):
p = Pusher.objects.get(auto_id=1)
try:
p = Pusher.objects.get(ip=ip)
except ObjectDoesNotExist:
print("no pusher found")
finally:
return p
Connecting, receiving and even getting stuff async from the database works perfectly. Only disconnecting does not work as expected. The following Terminal Log explains what's going on:
2018-09-19 07:09:56,653 - INFO - server - HTTP/2 support not enabled (install the http2 and tls Twisted extras)
2018-09-19 07:09:56,653 - INFO - server - Configuring endpoint tcp:port=1111:interface=192.168.1.111
2018-09-19 07:09:56,653 - INFO - server - Listening on TCP address 192.168.1.111:1111
[2018/09/19 07:11:25] HTTP GET / 302 [0.02, 10.171.253.112:35236]
[2018/09/19 07:11:25] HTTP GET /login/?next=/ 200 [0.05, 10.111.253.112:35236]
{'type': 'websocket', 'path': '/ws/chat/RP1/', 'headers': [(b'upgrade', b'websocket'), (b'connection', b'Upgrade'), (b'host', b'10.111.111.112:1111'), (b'origin', b'http://10.111.253.112:1111'), (b'sec-websocket-key', b'vKFAnqaRMm84AGUCxbAm3g=='), (b'sec-websocket-version', b'13')], 'query_string': b'', 'client': ['10.111.253.112', 35238], 'server': ['192.168.1.111', 1111], 'subprotocols': [], 'cookies': {}, 'session': <django.utils.functional.LazyObject object at 0x7fe4a8d1ba20>, 'user': <django.utils.functional.LazyObject object at 0x7fe4a8d1b9e8>, 'path_remaining': '', 'url_route': {'args': (), 'kwargs': {'room_name': 'RP1'}}}
10.111.253.112
[2018/09/19 07:11:25] WebSocket HANDSHAKING /ws/chat/RP1/ [10.111.253.112:35238]
[2018/09/19 07:11:25] WebSocket CONNECT /ws/chat/RP1/ [10.111.111.112:35238]
no pusher found
1 - DEFAULT - 0.0.0.0
ERROR: Wrong data packets send
{'hello': 'Did I achieve my objective?'}
[2018/09/19 07:11:46] WebSocket DISCONNECT /ws/chat/RP1/ [10.111.253.112:35238]
2018-09-19 07:11:56,792 - WARNING - server - Application instance <Task pending coro=<SessionMiddlewareInstance.__call__() running at /home/pi/PycharmProjects/LOGGER/venv/lib/python3.6/site-packages/channels/sessions.py:175> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.6/asyncio/futures.py:403, <TaskWakeupMethWrapper object at 0x7fe4a82e6fd8>()]>> for connection <WebSocketProtocol client=['10.171.253.112', 35238] path=b'/ws/chat/RP1/'> took too long to shut down and was killed.
After the 10 seconds timeout it gives a warning that the connection was killed:
WARNING - server - Application instance taskname running at location
at linenumber for connection cxn-name took too long to shut down and
was killed.
The disconnect method was thus also not reached.
What could this be?
Am I using the correct method?
Could I expand the timeout period?
if you are intending to run some custom logic during connection close then you should override websocket_disconnect and then call super (rather than rais the exception yourself)
https://github.com/django/channels/blob/507cb54fcb36df63282dd19653ea743036e7d63c/channels/generic/websocket.py#L228-L241
The code linked in the other answer was very helpful. Here it is for reference (as of May 2022):
async def websocket_disconnect(self, message):
"""
Called when a WebSocket connection is closed. Base level so you don't
need to call super() all the time.
"""
try:
for group in self.groups:
await self.channel_layer.group_discard(group, self.channel_name)
except AttributeError:
raise InvalidChannelLayerError(
"BACKEND is unconfigured or doesn't support groups"
)
await self.disconnect(message["code"])
raise StopConsumer()
async def disconnect(self, code):
"""
Called when a WebSocket connection is closed.
"""
pass
I'd note that there is no need to use super() on websocket_disconnect. They have a hook disconnect which was perhaps added since the original answer that can be used to add any teardown code. You can simply override the disconnect method in your own class and it will be called.

Django channels redis channel layer opens a lot of connections

We ported a part of our application to django channels recently, using the redis channel layer backend. A part of our setup still runs on python2 in a docker which is why we use redis pub/sub to send messages back to the client. A global listener (inspired by this thread) catches all messages and distributes them into the django channels system. It all works fine so far but I see a lot of debug messages Creating tcp connection... passing by. The output posted below corresponds to one event. Both the listener as well as the consumer seem to be creating two redis connections. I have not enough knowledge about the underlying mechanism to be able to tell if this the expected behavior, thus me asking here. Is this to be expected?
The Listener uses a global channel layer instance:
# maps the publish type to a method name of the django channel consumer
PUBLISH_TYPE_EVENT_MAP = {
'state_change': 'update_client_state',
'message': 'notify_client',
}
channel_layer = layers.get_channel_layer()
class Command(BaseCommand):
help = u'Opens a connection to Redis and listens for messages, ' \
u'and then whenever it gets one, sends the message onto a channel ' \
u'in the Django channel system'
...
def broadcast_message(self, msg_body):
group_name = msg_body['subgrid_id'].replace(':', '_')
try:
event_name = PUBLISH_TYPE_EVENT_MAP[msg_body['publish_type']]
# backwards compatibility
except KeyError:
event_name = PUBLISH_TYPE_EVENT_MAP[msg_body['type']]
async_to_sync(channel_layer.group_send)(
group_name, {
"type": event_name,
"kwargs": msg_body.get('kwargs'),
})
The consumer is a JsonWebsocketConsumer that is initialized like this
class SimulationConsumer(JsonWebsocketConsumer):
def connect(self):
"""
Establishes the connection with the websocket.
"""
logger.debug('Incoming connection...')
# subgrid_id can be set dynamically, see property subgrid_id
self._subgrid_id = self.scope['url_route']['kwargs']['subgrid_id']
async_to_sync(self.channel_layer.group_add)(
self.group_name,
self.channel_name
)
self.accept()
And the method that is called from the listener:
def update_client_state(self, event):
"""
Public facing method that pushes the state of a simulation back
to the client(s). Has to be called through django channels
```async_to_sync(channel_layer.group_send)...``` method
"""
logger.debug('update_client_state event %s', event)
current_state = self.redis_controller.fetch_state()
data = {'sim_state': {
'sender_sessid': self.session_id,
'state_data': current_state}
}
self.send_json(content=data)
A single event gives me this output
listener_1 | DEBUG !! data {'publish_type': 'state_change', 'subgrid_id': 'subgrid:6d1624b07e1346d5907bbd72869c00e8'}
listener_1 | DEBUG !! event_name update_client_state
listener_1 | DEBUG !! kwargs None
listener_1 | DEBUG !! group_name subgrid_6d1624b07e1346d5907bbd72869c00e8
listener_1 | DEBUG Using selector: EpollSelector
listener_1 | DEBUG Parsing Redis URI 'redis://:#redis-threedi-server:6379/13'
listener_1 | DEBUG Creating tcp connection to ('redis-threedi-server', 6379)
listener_1 | DEBUG Parsing Redis URI 'redis://:#redis-threedi-server:6379/13'
listener_1 | DEBUG Creating tcp connection to ('redis-threedi-server', 6379)
threedi-server_1 | DEBUG Parsing Redis URI 'redis://:#redis-threedi-server:6379/13'
threedi-server_1 | DEBUG Creating tcp connection to ('redis-threedi-server', 6379)
threedi-server_1 | DEBUG update_client_state event {'type': 'update_client_state', 'kwargs': None}
threedi-server_1 | DEBUG Parsing Redis URI 'redis://:#redis-threedi-server:6379/13'
threedi-server_1 | DEBUG Creating tcp connection to ('redis-threedi-server', 6379)

http error 400 with "meetup" API

I'm trying to hook my website to Meetup.com.
Instructions
Everything works when I request authorization. However when I try to get the access token, I keep getting:
http error 400: bad request.
Here is my code:
def meetupauth(request):
r =request.REQUEST
code = r['code']
state = r['state']
consumer = OAuthConsumer.objects.filter(apiservice_id=LINKEDINAPI)[0]
url_str = {'client_id' : consumer.token, 'client_secret' : consumer.tokensecret, 'grant_type' : 'authorization_code', 'redirect_uri' : 'http://127.0.0.1:8000/apiservice/meetupauth', 'code' : code}
#url_str = (('client_id' , consumer.token), ('client_secret' , consumer.tokensecret), ('grant_type' , 'authorization_code'), ('redirect_uri' , 'http://127.0.0.1:8000/apiservice/meetupauth'), ('code' , code))
url_string = urllib.urlencode(url_str)
req = urllib2.Request(meetup_access_token_url, url_string)
resp = urllib2.urlopen(req)
return redirect("/dashboard/")
This is the error:
HTTPError at /apiservice/meetupauth
HTTP Error 400: Bad Request
Request Method: GET
Request URL: http://127.0.0.1:8000/apiservice/meetupauth?code=acd62b4e1f28e3454c322d6b00136443&state=
Django Version: 1.4
Exception Type: HTTPError
Exception Value: HTTP Error 400: Bad Request
Dumb mistake.
the code is:
consumer = OAuthConsumer.objects.filter(apiservice_id=LINKEDINAPI)[0]
it should have been
consumer = OAuthConsumer.objects.filter(apiservice_id=MEETUPAPI)[0]