realtime with django web sockets - django

I have a question for you please, I have an app with django channels:
THE FIRST QUESTION
the idea is to get all transaction in real time, but the transaction is growing in each moment, example the app is beginning to 10 transactions and before is 100 transactions and more, the app realtime is a little slow, how to do the app to get transactions? maybe one by one transaction if I decided to do one by one, how to implement this function:
my function to get all transaction is:
class TransactionConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.room_name = self.scope['url_route']['kwargs']['room_name']
self.room_group_name = 'transaction_%s' % self.room_name
# Join room group
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave room group
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data):
text_data_json = json.loads(text_data)
message = text_data_json['message']
# Send the message to room group
await self.channel_layer.group_send(
self.room_group_name,
{
'type': 'list_transaction',
'message': message
}
)
# Receive message from room group
async def chat_message(self, event):
message = event['message']
# Send message to WebSocket
await self.send(text_data=json.dumps({
'message': message
}))
async def list_transaction(self, event):
token = event['message']
message = self.list_trans_all(token)
await self.send(text_data=json.dumps({
'massage': message
}))
# HERE RETURN ALL TRANSACTIONS BY JWT TOKEN USER LOGGED
def list_trans_all(self, token):
end_point = 'http://127.0.0.1:8000/api/v1/transactions/list-trans-users/'
headers = {'Authorization': 'Bearer ' + token}
listrans = requests.get(end_point, headers=headers)
return listrans.json()
and my client is:
<script>
var typeTransaction = 'buy';
var transaction_Socket = new WebSocket(
'ws://127.0.0.1:8001/ws/transaction/' + typeTransaction + '/');
export default {
name: 'orders',
data(){
return {
// transaction_datas: '',
}
},
mounted(){
// console.log(transaction_Socket);
// recive msg
transaction_Socket.onmessage = function(e) {
var transaction_datas = JSON.parse(e.data);
console.log(transaction_datas);
};
},
methods:{
getTransaction(){
const token_auth = window.localStorage.token_auth;
// console.log(array_msg.msg);
transaction_Socket.send(JSON.stringify({
'message': token_auth
}));
}
}
};
the code above is working well, but my problem is a load of the transaction is to begging a little slow, every time the data is an increment
THE SECOND QUESTIONS
the code above to get all transaction and give it to all users, my question is how to give each transaction for one user like this example:
|user = 2
buy = 23
product = tv
agent_id = 5
|user = 3
buy = 4
product = fp
agent_id = 7
|agent = 7
sell = 4
product = somnting
and each user and agent has to dashboard, when I click in button in the real-time the code above, the transaction gives it all users, I have a SQL with a filter by agent_id, how to gives transactions for one agent the user and agent has different JWT tokens like this
|agent = 7 // DASHBOARD AGENT ID = 7
user = 3
buy = 4
product = fp
agent_id = 7
|agent = 5 // DASHBOARD AGENT ID = 5
user = 2
buy = 34
product = somenting
agent_id = 2
I guess you understand me, thank you attention
please help me.

Related

How to connect Django web socket with the third-Party Web Sockets?

I want to connect my Django WebSocket with a third-party web socket. This program is one I wrote, and it functions properly. To avoid having to re-login with the third-party API, I have now added the code to check whether the same room is present in my database. if we use the same API KEY to re-connect to the third-party API. It gives the following error:
{"event":"login","status":401,"message":"Connected from another location"}
I want to see if the same cryptocurrency coin is already connected or not. We don't want to logon with the same API KEY once we're connected. I have two issues here:
Don't send the login request to that web socket again.
Don't send the subscribe request, if the same coin already exists. Let's say BTCUSD already connected and giving me the data. I want to just connect to the next user to same room and get the data on next request.
import websocket
import time
import ssl
from channels.generic.websocket import AsyncWebsocketConsumer
from .models import Room
login = {
"event": "login",
"data": {
"apiKey": "API_KEY",
},
}
class CryptoConsumer(AsyncWebsocketConsumer):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.ws = websocket.WebSocket(sslopt={"cert_reqs": ssl.CERT_NONE})
self.ws.connect("wss://crypto.financialmodelingprep.com")
async def connect(self):
self.room_name = self.scope["url_route"]["kwargs"]["room_name"]
self.room_group_name = "crypto_%s" % self.room_name
# ****** Code Block ******
subscribe = {
"event": "subscribe",
"data": {
"ticker": self.room_name,
},
}
room = await Room.add(self.room_name) # Method in models.py to add the user and return True
if room is False:
self.ws.send(json.dumps(login))
print("The group with the name %s doesn't exist" % self.room_group_name)
time.sleep(1)
self.ws.send(json.dumps(subscribe))
# ****** End Code Block ******
# Join room group
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
async def disconnect(self, close_code):
unsubscribe = {
"event": "unsubscribe",
"data": {
"ticker": self.room_name,
},
}
self.ws.send(json.dumps(unsubscribe))
self.ws.close()
# Leave room group
await self.channel_layer.group_discard(self.room_group_name, self.channel_name)
# Receive message from WebSocket
async def receive(self, text_data="{'text': 'Dummy Text'}"):
text_data_json = json.loads(text_data)
message = text_data_json["message"]
message = str(self.ws.recv())
# Send message to room group
await self.channel_layer.group_send(
self.room_group_name, {"type": "chat_message", "message": message}
)
# Receive message from room group
async def chat_message(self, event):
message = event["message"]
# Send message to WebSocket
await self.send(text_data=json.dumps({"message": message}))
Note: Why I want to do this entire step because we don't want the API KEY made public.
So, the code coming from the front end will connect to our Django web socket, and then we'll connect to the third-party web socket and return the data that was sent by them.

WebSocket connection failed (Django channels)

I'm trying to establish a websocket connection, however, I'm getting this error:
WebSocket connection to 'wss://oasis.yiaap.com/ws/question/aaass-bc6f6f2e-4a96-4174-a8dd-a3b8f03645a7/' failed:
I think everything is set up correctly.
#routes.py
websocket_urlpatterns = [
#re_path(r"ws/questions/", consumers.QuestionsConsumer.as_asgi()),
path("ws/question/<question_slug>/", consumers.QuestionConsumer.as_asgi())
]
#consumers.py
class QuestionConsumer(AsyncJsonWebsocketConsumer):
async def connect(self):
#print("DEBUG")
self.question_slug = self.scope['url_route']['kwargs']['question_slug']
self.room_name = self.question_slug
self.room_group_name = 'question_%s' % self.room_name
# In order to detect who is the owner of the room
# We'll setup the user
# Join room group
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.accept()
# Check if the question exists in the database otherwise close the connection
# and send a message says 'question not found' so client can act on it
is_questions_exists = await questions_exists(question_slug=self.question_slug)
if is_questions_exists == False:
await self.send_json({"message": "question_not_found"})
await self.close()
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
async def disconnect(self, close_code):
user_id = self.scope["client"][1]
owner_user_id = cache.get(f"{self.room_name}_owner")
if user_id == owner_user_id:
# Check if the user is the question owner then delete the question from database
await delete_question(self.question_slug)
cache.delete_pattern(f"{self.room_name}_owner")
# Leave room group
await self.channel_layer.group_discard(
self.room_group_name,
self.channel_name
)
# Receive answer from WebSocket
async def receive_json(self, content):
answer = content.get('answer', None)
is_owner = content.get("is_owner", None)
if answer:
# Send answer to room group
await self.channel_layer.group_send(
self.room_group_name,
{
'type': 'question_answer',
'answer': answer
}
)
elif is_owner:
# Set a websocket client port as user id in cache
user_id = self.scope["client"][1]
cache.set(f"{self.room_name}_owner", user_id)
# Receive answer from room group
async def question_answer(self, event):
answer = event['answer']
# Send answer to WebSocket
await self.send_json({
'answer': answer
})
#asgi.py
application = ProtocolTypeRouter({
"http": get_asgi_application(),
"websocket":
URLRouter(
websocket_urlpatterns
)
})
//this is in react, however, I think everything is okay here
const wsfunc = () => {
// Subscribe to anychanges to `questions` in the backend/database
const ws = new WebSocket(
`${HOST_WEBSOCKET_URL}question/${question_data.slug}/`
)
ws.onopen = (e) => {
ws.onmessage = (e) => {
const data = JSON.parse(e.data)
if (data) {
switch (data.message) {
case "question_not_found":
router.push("/")
break
default:
console.log(data)
setAnswers((state) => [data.answer, ...state])
break
}
}
}
// Send a message and say I am the owner
ws.send(
JSON.stringify({
is_owner: true,
})
)
ws.onclose = () => {
window.location.href = "/"
}
}
}
I tried adding a debug message in consumers.py on connect, but it doesn't get triggered, so probably something is wrong before that.
I'm not really an expert on websockets, in fact this is my first time using it.
Basically, I can't establish a websocket connection.
EDIT:
this is in production server. Also, I'm using Daphne and Nginx
Thanks in advance.

How to know if you are connecting two sockets to same group of channel-layer in django channels

I am actually trying to build a system wherein two entities(being doctor and patient) can share the same data over websockets using django.
The way I have setup my channels is sending the auth-token through query_string in the websocket protocol
The models are configured in the following fashion
the patient model also has an attribute called "grp_id"
grp_id = models.UUIDField(default=uuid.uuid4, editable=False)
The consumers.py file is
• Working for patient
make a connection request by sending auth token through query-string which will authenticate the user
since the user is patient, the grp_id of the patient is fetched
A channel is created using the grp_id value
The patient triggers the start_sepsis function which would receive a set of sepsis-attribute, then serialize it and store it it in DB
The same serialize data is broadcasted over the channel
• Working for doctor
authentication like above
the doctor fetches patients associated to them using _get_patient_of_doctor helper function
doctor will try to connect all the patient's grp_id associated to it
Once connected broadcast a message called "doc is connected"
class SepsisDynamicConsumer(AsyncJsonWebsocketConsumer):
groups = ['test']
#database_sync_to_async
def _get_user_group(self, user):
return user.user_type
#database_sync_to_async
def _get_user_grp_id(self, user):
#************** THE grp function initiated **************
print("THE USER obj", user)#output below
print("email", user.email)#output below
if "PATIENT" == user.user_type:
return str(user.patient_set.all()[0].grp_id)
elif "DOCTOR" == user.user_type:
return map(str, user.doctor_set.all()[0].patient_set.all().values_list('grp_id', flat=True))
else:
print("THE USER IS SOMETHING ELSE")
#database_sync_to_async
def _get_patient_of_doctor(self, user):
x = user.doctor_set.all()
doc_obj = x[0]
pat_ids = doc_obj.patient_set.all().values_list('grp_id', flat=True)
return pat_ids
#database_sync_to_async
def _created_sepsis_data(self, data):
"""[summary]
This helper function would generate and return the
"""
x = get_user_model().objects.get(id=data['patient'])
x = x.patient_set.values('id')[0]['id']
data.update({'patient': x})
serializer = SepsisPatientSerializer(data=data)
serializer.is_valid(raise_exception=True)
x = serializer.create(serializer.validated_data)
return SepsisPatientSerializer(x).data
async def connect(self):
user = self.scope['user']
print("THE USER IS ------>", user)#output below
if user.is_anonymous:
print("user was unknown")
await self.close()
else:
if user.user_type == 'PATIENT':
pat_grp_id = await self._get_user_grp_id(user)
await self.channel_layer.group_add(
group=pat_grp_id,
channel=self.channel_name
)
print("CONNECT TO ---------> ", pat_grp_id)
elif user.user_type == 'DOCTOR':
for doc_pat_grp_id in await self._get_user_grp_id(user):
print("Doc connected --------->", doc_pat_grp_id)
await self.channel_layer.group_add(
group=doc_pat_grp_id,
channel=self.channel_name
)
print("Doc connected ", doc_pat_grp_id)
await self.accept()
async def start_sepsis(self, message):
data = message.get('data')
sepsis_generated_and_saved_data = await self._created_sepsis_data(data)
await self.send_json({
'type': 'echo.message',
'data': sepsis_generated_and_saved_data
})
async def disconnect(self, code):
user = self.scope['user']
if user.is_anonymous:
await self.close()
else:
if user.user_type == 'PATIENT':
pat_grp_id = await self._get_user_grp_id(user)
await self.channel_layer.group_discard(
group=pat_grp_id,
channel=self.channel_name
)
await super().disconnect(code)
async def echo_message(self, message):
await self.send_json(message)
async def receive_json(self, content, **kwargs):
message_type = content.get('type')
if message_type == 'start.sepsis':
await self.start_sepsis(content)
if message_type == 'echo.message':
await self.send_json({
'type': message_type,
'data': content.get('data'),
})
The problem is:
When I run a test which goes in a following order
create a patient
send a websocket connection request to server
in patient's DB, I have added a "grp_id" attribute to have a unique ASCII unicode for django-channel-layer group which is where the patient would be connected
connect a doctor
send a websocket connection request to the server
send a "echo.message" through websocket from patient with data:"Patient got connected"
send a "echo.message" through websocket from doctor with data:"doctor got connected"
The thing is whenever the test runs the patient receives only those data which the patient has sent and doctor receives the same data that a doctor sent even though I using a django-channel broadcast function which would send data to all the connected users.
Also address something like (what if there are multiple patients associated to doctors and the doctor is getting the last patient in the query_set because of the "for loop") well right now only one doctor is associated to every patient.
My test
async def test_patient_doctor_on_same_channel(self, settings):
settings.CHANNEL_LAYERS = TEST_CHANNEL_LAYERS
# create doctor
doctor_user, doctor_access = await create_user(
'test_Doctor.user#example.com', 'pAssw0rd', 'DOCTOR', 'testDoctor_username'
)
# create patient
user, access = await create_user(
'test.user#example.com', 'pAssw0rd', 'PATIENT', 'test_username'
)
# connect patient
communicator = WebsocketCommunicator(
application=application,
path=f'/sepsisDynamic/?token={access}'
)
connected, _ = await communicator.connect()
# connect doctor
doctor_communicator = WebsocketCommunicator(
application=application,
path=f'/sepsisDynamic/?token={doctor_access}'
)
# the doctor connects to the online patient
doctor_connected, _ = await doctor_communicator.connect()
# Simply echo an message
message = {
'type': 'echo.message',
'data': 'This is a test message.'
}
await communicator.send_json_to(message)
# checking if patient received the data on their end
response = await communicator.receive_json_from()
assert response == message
# checking if doctor received the patient's data on their end
response_doc = await doctor_communicator.receive_json_from()
response_doc == message
await communicator.disconnect()
await doctor_communicator.disconnect()
My intuition says that it is not connected to the same websocket but I don't know why.
The output of the test
THE USER IS ------> test_username
************** THE grp function initiated **************
THE USER obj test_username
email test.user#example.com
CONNECT TO ---------> 1b2a455c-28b0-4c4d-9d26-40f6a4634fa9
**********************************************************
THE USER IS ------> testDoctor_username
THE DOC CONDITION RAN
************** THE grp function initiated **************
THE USER obj testDoctor_username
email test_Doctor.user#example.com
THE patient's unicode <QuerySet [UUID('1b2a455c-28b0-4c4d-9d26-40f6a4634fa9')]>
Doc connected ---------> 1b2a455c-28b0-4c4d-9d26-40f6a4634fa9
Doc connected 1b2a455c-28b0-4c4d-9d26-40f6a4634fa9
THE RECEIVE FUNCTION RAN
THE MESSAGE TYPE echo.message
******The error******
TestWebSocket::test_patient_doctor_on_same_channel - asyncio.exceptions.TimeoutError
response_doc = await doctor_communicator.receive_json_from()
Please help.
Thank you.
You're using send_json which sends to the same channel. To send to a group, you have to use the channel_layer.group_send method as documented in the docs chat example. You specify the handler in each channel in the group that will then push each message down to the client. One issue with your architecture is that the doctor's channel can be connected to many patient groups and there is no way of knowing which particular patient he is sending a message to. A way to solve that could be to forward the patient group numbers to the frontend so that it can specify the patient group name when sending the message. Then you can send the message like this
async def receive_json(self, content, **kwargs):
message_type = content.get('type')
if message_type == 'start.sepsis':
await self.start_sepsis(content)
if message_type == 'echo.message':
group_name = await self._get_user_grp_id(user)
if "DOCTOR" == user.user_type:
group_name = content['group_name']
await self.channel_layer.group_send(
'type': 'chat.message',
'data': content.get('data'),
})
async def chat_message(self, event):
"""
Called when someone has messaged our chat.
"""
# Send a message down to the client
await self.send_json(
{
"message": event["data"],
}
)

Django channel visitor counter realtime

I am trying to show visitor counter on realtime basis with django. like how many visitor are online at my webiste.
I written a websocket consumer but it alwasy give me 0 even if i open the site in multiple browser.
this is my django channel consume:
class VisitorConsumer(WebsocketConsumer):
user_count = 0
def connect(self):
self.room_name = 'visitors'
self.room_group_name = 'counter_%s' % self.room_name
# Join room group
async_to_sync(self.channel_layer.group_add)(
self.room_group_name,
self.channel_name
)
# Send message to room group
async_to_sync(self.channel_layer.group_send)(
self.room_group_name,
{
'type': 'send_visitor_count',
'count': self.user_count
}
)
self.accept()
def disconnect(self, close_code):
# Leave room group
async_to_sync(self.channel_layer.group_discard)(
self.room_group_name,
self.channel_name
)
# Receive message from room group
def send_visitor_count(self, event):
count = event['count']
# Send message to WebSocket
self.send(text_data=json.dumps({
'count': count
}))
adn this is the routing:
websocket_urlpatterns = [
re_path(r'ws/visitors/$', consumers.VisitorConsumer),
]
I am not getting why it's always firing 0.
Can anyone help to fix this?
I don't see where you are incrementing the user_count but even that may not work if you incremented it because the different instances of the consumer running in different workers won't have access to the same user_count variable. So you should store it in a cache like Redis or DB and don't forget to actually increment it

Constantly send data to client from server

Take a look at this example.
As you can see, some sort of event is constantly being sent to the client. I want to imitate this using Django-Channels, inside consumers.py. Here's a simplified version of what I have:
class ChatConsumer(AsyncConsumer):
async def ws_connect(self, event):
self.send = get_db_object()
....
await self.send({
"type": "websocket.accept"
})
# I need to CONSTANTLY receive & send data
async def ws_receive(self, event):
obj = ...# query DB and get the newest object
json_obj = {
'field_1': obj.field_1,
'field_2': obj.field_2,
}
await self.send({
"type": "websocket.send",
"text": json.dumps(json_obj)
})
#database_sync_to_async
def get_db_object(self, **kwargs):
return Some_Model.objects.get(**kwargs)[0]
Here, I want my Django backend to constantly:
Query DB
Receive obj from DB
Send the received obj to Front-End WebSocket as event
How can I achieve this? The important thing is that I need to CONSTANTLY send data to the client.
Most of the Django-Channels resources on the internet cover only Chat Apps, which don't necessarily constantly send data to the client. I couldn't find any working code that does this job.
Please, no more recommendation for Redis or channels documentation... or some random 3rd party libraries that lacks good documentation... It's easy to recommend but hard to implement. For example, I found someone recommending Snorky, but it really lacks documentation on how to implement it.
However, if there's a website that specifically does this job, I might take a look at it, even if it doesn't use Django-Channels.
consumers.py
import asyncio
from channels.consumer import AsyncConsumer
class ChatConsumer(AsyncConsumer):
async def websocket_connect(self, event):
self.connected = True
print("connected", event)
await self.send({
"type": "websocket.accept"
})
while self.connected:
await asyncio.sleep(2)
obj = # do_something (Ex: constantly query DB...)
await self.send({
'type': 'websocket.send',
'text': # obj,
})
async def websocket_receive(self, event):
print("receive", event)
async def websocket_disconnect(self, event):
print("disconnected", event)
self.connected = False
Javascript
var loc = window.location;
var wsStart = 'ws://';
if (loc.protocol == 'https:') {
wsStart = 'wss://'
}
var endpoint = wsStart + loc.host + loc.pathname;
var socket = new WebSocket(endpoint);
socket.onmessage = function(e){
console.log("message", e);
};
socket.onopen = function(e){
console.log("open", e);
};
socket.onerror = function(e){
console.log("error", e)
};
socket.onclose = function(e){
console.log("close", e)
};
All you need to do is just modify obj and send it. You can extend this function as much as you want. So, right now I'm interested in getting the latest inserted row in my PostgreSQL and injecting that row into my WebSocket. I can query my DB every 2 seconds as it was specified by await asyncio.sleep(2), and inject it into the Front-End socket.
Using channels==1.* and Django==1.* you can use the threading module for example:
# Some view.py
import threading
import time
class Publisher(threading.Thread):
def __init__(self, reply_channel, frequency=0.5):
super(Publisher, self).__init__()
self._running = True
self._reply_channel = reply_channel
self._publish_interval = 1.0 / frequency
def run(self):
while self._running:
self._reply_channel.send({'text': 'some data'})
time.sleep(self._publish_interval)
def stop(self):
self._running = False
publishers = {}
def ws_connect(message):
message.reply_channel.send({'accept': True})
publisher = Publisher(reply_channel=message.reply_channel)
publisher.start()
publishers[message.reply_channel] = publisher
def ws_disconnect(message):
publisher = publishers[message.reply_channel]
publisher.stop()
del publishers[message.reply_channel]
A little late to the party here, but when it comes to Django you should always try to do things "their way" first...
So, to do this I simply used Django Channels. My client sends a message to the server, which the server then responds to with the needed database info. It looks as follows:
class SettingsConsumer(WebsocketConsumer):
def connect(self):
self.accept()
def disconnect(self, close_code):
pass
def receive(self, text_data):
settings = get_settings(self.scope["user"])
self.send(
text_data=json.dumps(
{
"playMode": settings.play_mode,
"isRecording": settings.is_recording,
}
)
)
Now, as for the JS to trigger constant events... I simply use SetInterval to request updates from the consumer every .25s!
My logic is that it's the client so if it's doing a little extra work no biggie, as the server is gonna be responding anyways. The JS looks as follows...
const chatSocket = new WebSocket(
'ws://'
+ window.location.host
+ '/ws/macros/update/'
);
chatSocket.onmessage = function(e) {
const data = JSON.parse(e.data);
if(data["playMode"] == true) {
$('#playmode-checkbox').attr('checked', true);
} else {
$('#playmode-checkbox').attr('checked', false);
}
if(data["isRecording"] == true) {
$('#recording-checkbox').attr('checked', true);
} else {
$('#recording-checkbox').attr('checked', false);
}
};
chatSocket.onclose = function(e) {
console.error('Chat socket closed unexpectedly');
};
window.setInterval(function() {
chatSocket.send(JSON.stringify({
'message': "start"
}));
}, 250);
And yes, you can do more to make this more async-friendly and optimized. But, you asked for simple and working so I hope this helps!