I have a Flask application that sends data through websocket with emit.
My Python code looks like this. I want the emit to be triggerd in AnotherClass at some conditions.
class MainClass:
def __init__(self):
app = Flask(__name__)
app.secret_key = 'any random string'
self.socketio = SocketIO(app, logger=True, binary=True, engineio_logger=True,
always_connect=True, async_mode='eventlet')
self.socketio.init_app(app, cors_allowed_origins='*')
self.socketio.on_event('on-record', self.record, namespace='/audio')
self.AC = AnotherClass(socketio=self.socketio)
def run_app(self):
self.socketio.run(self.app, debug=True, port=8000)
def record(self, some_other_data):
# This function gets triggered when client emit
# I inserted the following emit to test if this works
self.socketio.emit('send-data', {'data': data}, namespace='/audio')
class AnotherClass:
def __init__(self, socketio):
self.socketio = socketio
def send_data(self, data):
self.socketio.emit('send-data', {'data': data}, namespace='/audio')
When I trigger the emit function from AnotherClass.send_data, I get the following log that says the emit works and sending the data, however, the client-side application does not receive the data. On the contrary, the same emit works in the MainClass.record which is the event gets triggered when the client emits something. (I inserted the emit to test in which case works, ideally, it should not be in record function.)
cedb89204e80438da86361a97f6f1bac: Sending packet MESSAGE data 51-/audio,["send-data",{"data":{"_placeholder":true,"num":0}}]. cedb89204e80438da86361a97f6f1bac: Sending packet MESSAGE data <binary>
This is the JavaScript code that receives the emit from the Python script above.
var socketio = io.connect('0.0.0.0:8000' + '/audio', {transports: ['websocket']});
socketio.on("send-data", function(data) {
// Log when client receives the data from server-side
console.log("Data received");
}
})
Could anyone please give me any insights on what I am doing wrongly?
Related
I'm using Channels v2.
I want to integrate long-polling into my project.
The only consumer I see in the documentation for http long polling is the AsyncHttpConsumer.
The code I need to run in my handle function is not asynchronous. It connects to another device on the network using a library that is not asynchronous. From what I understand, this will cause the event loop to block, which is bad.
Can I run my handler synchronously, in a thread somehow? There's a SyncConsumer, but that seems to have something to do with Web Sockets. It doesn't seem applicable to Long Polling.
Using AsyncHttpConsumer as a reference, I was able to write an almost exact duplicate of the class, but subclassing SyncConsumer instead of AsyncConsumer as AsyncHttpConsumer does.
After a bit of testing, I soon realized that since my code was all running in a single thread, until the handle() method finished running, which presumably runs until done, the disconnect() method wouldn't be triggered, so there was no way to interrupt a long running handle() method, even if the client disconnects.
The following new version runs handle() in a thread, and gives the user 2 ways to check if the client disconnected:
from channels.consumer import SyncConsumer
from channels.exceptions import StopConsumer
from threading import Thread, Event
# We can't pass self.client_disconnected to handle() as a reference if it's
# a regular bool. That means if we use a regular bool, and the variable
# changes in this thread, it won't change in the handle() method. Using a
# class fixes this.
# Technically, we could just pass the Event() object
# (self.client_disconnected) to the handle() method, but then the client
# needs to know to use .is_set() instead of just checking if it's True or
# False. This is easier for the client.
class RefBool:
def __init__(self):
self.val = Event()
def set(self):
self.val.set()
def __bool__(self):
return self.val.is_set()
def __repr__(self):
current_value = bool(self)
return f"RefBool({current_value})"
class SyncHttpConsumer(SyncConsumer):
"""
Sync HTTP consumer. Provides basic primitives for building synchronous
HTTP endpoints.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.handle_thread = None
self.client_disconnected = RefBool()
self.body = []
def send_headers(self, *, status=200, headers=None):
"""
Sets the HTTP response status and headers. Headers may be provided as
a list of tuples or as a dictionary.
Note that the ASGI spec requires that the protocol server only starts
sending the response to the client after ``self.send_body`` has been
called the first time.
"""
if headers is None:
headers = []
elif isinstance(headers, dict):
headers = list(headers.items())
self.send(
{"type": "http.response.start", "status": status, "headers": headers}
)
def send_body(self, body, *, more_body=False):
"""
Sends a response body to the client. The method expects a bytestring.
Set ``more_body=True`` if you want to send more body content later.
The default behavior closes the response, and further messages on
the channel will be ignored.
"""
assert isinstance(body, bytes), "Body is not bytes"
self.send(
{"type": "http.response.body", "body": body, "more_body": more_body}
)
def send_response(self, status, body, **kwargs):
"""
Sends a response to the client. This is a thin wrapper over
``self.send_headers`` and ``self.send_body``, and everything said
above applies here as well. This method may only be called once.
"""
self.send_headers(status=status, **kwargs)
self.send_body(body)
def handle(self, body):
"""
Receives the request body as a bytestring. Response may be composed
using the ``self.send*`` methods; the return value of this method is
thrown away.
"""
raise NotImplementedError(
"Subclasses of SyncHttpConsumer must provide a handle() method."
)
def disconnect(self):
"""
Overrideable place to run disconnect handling. Do not send anything
from here.
"""
pass
def http_request(self, message):
"""
Sync entrypoint - concatenates body fragments and hands off control
to ``self.handle`` when the body has been completely received.
"""
if "body" in message:
self.body.append(message["body"])
if not message.get("more_body"):
full_body = b"".join(self.body)
self.handle_thread = Thread(target=self.handle, args=(full_body, self.client_disconnected), daemon=True)
self.handle_thread.start()
def http_disconnect(self, message):
"""
Let the user do their cleanup and close the consumer.
"""
self.client_disconnected.set()
self.disconnect()
self.handle_thread.join()
raise StopConsumer()
The SyncHttpConsumer class is used very similarly to how you would use the AsyncHttpConsumer class - you subclass it, and define a handle() method. The only difference is that the handle() method takes an extra arg:
class MyClass(SyncHttpConsumer):
def handle(self, body, client_disconnected):
while not client_disconnected:
...
Or you could, just like with the AsyncHttpConsumer class, override the disconnect() method instead if you prefer.
I'm still not sure if this is the best way to do this, or why Django Channels doesn't include something like this in addition to AsyncHttpConsumer. If anyone knows, please let us know.
So I wrote a dedicated flask app for handling emails for my application and deployed it on heroku. In which I have set up a route to send emails:
#app.route('/send', methods=['POST'])
def send_now():
with app.app_context():
values = request.get_json()
email = values['email']
code = values['code']
secret_2 = str(values['secret'])
mail = Mail(app)
msg = Message("Password Recovery",sender="no*****#gmail.com",recipients=[email])
msg.html = "<h1>Your Recovery Code is: </h1><p>"+str(code)+"</p>"
if secret == secret_2:
mail.send(msg)
response = {'message': 'EmailSent'}
return jsonify(response), 201
It works fine for a single user at a time, however when multiple users send a POST request, the client user needs to wait till the POST returns a 201. Thus the wait period keeps increasing (it may not even send). So how do I handle this so accommodate multiple simultaneous users. Threads? Buffer? I have no idea
You need to send mail via Asynchronous thread calls in Python. Have a look at this code sample and implement in your code.
from threading import Thread
from app import app
def send_async_email(app, msg):
with app.app_context():
mail.send(msg)
def send_email(subject, sender, recipients, text_body, html_body):
msg = Message(subject, sender=sender, recipients=recipients)
msg.body = text_body
msg.html = html_body
thr = Thread(target=send_async_email, args=[app, msg])
thr.start()
This will allow to send the mail in background.
Env: Python 3.6, and Django 2.1
I have created a Django website and a socket server, and files are organized like this:
web
...
user (a Django app)
__init__.py
views.py
...
server.py
Actually I want to build a umbrella rental system by using django, and server connects to umbrella shelf via multi-thread socket (sending some messages). Like I press the borrow button, and views.py can call the server test_function and send some messages to the connected umbrella shelf.
I can import server variables or functions in views.py, but I cannot get the right answer while server.py is running. So I want to ask you if you could give me some advice. Thanks a lot!
By the way, I tried to import the global variable clients directly in views.py, but still got [].
server.py defines a multi-thread server, which is basically as below:
clients = []
class StuckThread(threading.Thread):
def __init__(self, **kwargs):
self.name = kwargs.get('name', '')
def run(self):
while True:
# do something
def func1(self):
# do something
def test_function(thread_name):
# if the function is called by `views.py`, then `clients = []` and return 'nothing', but if I call this function in `server.py`, then I can get a wanted result, which is `got the thread`
for client in clients:
if client['thread'].name == thread_name:
return 'got the thread'
return 'nothing'
if __name__ == '__main__':
ip_port = ('0.0.0.0', 65432)
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ip_port)
server.listen(max_listen_num)
while True:
client, address = socket.accept()
param = {'name': 'test name'}
stuck_thread = StuckThread(**param)
clients.append({"client": client, "address": address, "thread": stuck_thread})
stuck_thread.start()
and I have a Django views.py like this
def view_function(request):
from server import clients
print(clients) # got []
form server import test_function
print(test_function('test name')) # got 'nothing'
return render(request, 'something.html')
I have solve this problem by socket communication between django views.py and server.py. I open another port to receive messages from views.py. Once the borrow button is pressed, a socket client in views.py will build up and send arguments and other messages to the server.
I am pretty new to Twisted, so I am sure this is a rookie mistake. I have built a simple server which receives a message from the client and upon receipt of message the server fires a callback which prints the message to the console.
At first instance, the server works as expected. Unfortunately, when I start up a second client I get the follow error "twisted.internet.defer.AlreadyCalledError." It was my understanding that the factory would make a new instance of the deferred i.e. the new deferred wouldn't have been called before?
Please see the code below. Any help would be very appreciated.
import sys
from twisted.internet.protocol import ServerFactory, Protocol
from twisted.internet import defer
class LockProtocol(Protocol):
lockData = ''
def dataReceived(self, data):
self.lockData += data
if self.lockData.endswith('??'):
self.lockDataReceived(self.lockData)
def lockDataReceived(self, lockData):
self.factory.lockDataFinished(lockData)
class LockServerFactory(ServerFactory):
protocol = LockProtocol
def __init__(self):
self.deferred = defer.Deferred() # Initialise deferred
def lockDataFinished(self, lockData):
self.deferred.callback(lockData)
def clientConnectionFailed(self, connector, reason):
self.deferred.errback(reason)
def main():
HOST = '127.0.0.1' # localhost
PORT = 10001
def got_lockData(lockData):
print "We have received lockData. It is as follows:", lockData
def lockData_failed(err):
print >> sys.stderr, 'The lockData download failed.'
errors.append(err)
factory = LockServerFactory()
from twisted.internet import reactor
# Listen for TCP connections on a port, and use our factory to make a protocol instance for each new connection
port = reactor.listenTCP(PORT,factory)
print 'Serving on %s' % port.getHost()
# Set up callbacks
factory.deferred.addCallbacks(got_lockData,lockData_failed)
reactor.run() # Start the reactor
if __name__ == '__main__':
main()
Notice that there is only one LockServerFactory ever created in your program:
factory = LockServerFactory()
However, as many LockProtocol instances are created as connections are accepted. If you have per-connection state, the place to put it is on LockProtocol.
It looks like your "lock data completed" event is not a one-off so a Deferred is probably not the right abstraction for this job.
Instead of a LockServerFactory with a Deferred that fires when that event happens, perhaps you want a multi-use event handler, perhaps custom built:
class LockServerFactory(ServerFactory):
protocol = LockProtocol
def __init__(self, lockDataFinished):
self.lockDataFinished = lockDataFinished
factory = LockServerFactory(got_lockData)
(Incidentally, notice that I've dropped clientConnectionFailed from this implementation: that's a method of ClientFactory. It will never be called on a server factory.)
Scrapy application, but the question is really about the Python language - experts can probably answer this immediately without knowing the framework at all.
I've got a class called CrawlWorker that knows how to talk to so-called "spiders" - schedule their crawls, and manage their lifecycle.
There's a TwistedRabbitClient that has-one CrawlWorker. The client only knows how to talk to the queue and hand off messages to the worker - it gets completed work back from the worker asynchronously by using the worker method connect_to_scrape below to connect to a signal emitted by a running spider:
def connect_to_scrape(self, callback):
self._connect_to_signal(callback, signals.item_scraped)
def _connect_to_signal(self, callback, signal):
if signal is signals.item_scraped:
def _callback(item, response, sender, signal, spider):
scrape_config = response.meta['scrape_config']
delivery_tag = scrape_config.delivery_tag
callback(item.to_dict(), delivery_tag)
else:
_callback = callback
dispatcher.connect(_callback, signal=signal)
So the worker provides a layer of "work deserialization" for the Rabbit client, who doesn't know about spiders, responses, senders, signals, items (anything about the nature of the work itself) - only dicts that'll be published as JSON with their delivery tags.
So the callback below isn't registering properly (no errors either):
def publish(self, item, delivery_tag):
self.log('item_scraped={0} {1}'.format(item, delivery_tag))
publish_message = json.dumps(item)
self._channel.basic_publish(exchange=self.publish_exchange,
routing_key=self.publish_key,
body=publish_message)
self._channel.basic_ack(delivery_tag=delivery_tag)
But if I remove the if branch in _connect_to_signal and connect the callback directly (and modify publish to soak up all the unnecessary arguments), it works.
Anyone have any ideas why?
So, I figured out why this wasn't working, by re-stating it in a more general context:
import functools
from scrapy.signalmanager import SignalManager
SIGNAL = object()
class Sender(object):
def __init__(self):
self.signals = SignalManager(self)
def wrap_receive(self, receive):
#functools.wraps(receive)
def wrapped_receive(message, data):
message = message.replace('World', 'Victor')
value = data['key']
receive(message, value)
return wrapped_receive
def bind(self, receive):
_receive = self.wrap_receive(receive)
self.signals.connect(_receive, signal=SIGNAL,
sender=self, weak=False)
def send(self):
message = 'Hello, World!'
data = {'key': 'value'}
self.signals.send_catch_log(SIGNAL, message=message, data=data)
class Receiver(object):
def __init__(self, sender):
self.sender = sender
self.sender.bind(self.receive)
def receive(self, message, value):
"""Receive data from a Sender."""
print 'Receiver received: {0} {1}.'.format(message, value)
if __name__ == '__main__':
sender = Sender()
receiver = Receiver(sender)
sender.send()
This works if and only if weak=False.
The basic problem is that when connecting to the signal, weak=False needs to be specified. Hopefully someone smarter than me can expound on why that's needed.