I have a basic TCP server implemented in twisted, to which a client is connected. The client connects and sends data necessary to start a websocket resource. Using these details sent by the TCP client, I want to add an autobahn websocket resource as a child under a twisted web resource. and when the client disconnects, I want to remove this child from the twisted web resource. please suggest, what should be the best way to implement this? Can I use resource.delEntity(child)?
so far, the code looks like this:
from twisted.internet.protocol import Protocol, Factory
from autobahn.twisted.websocket import WebSocketServerFactory, WebSocketServerProtocol
from autobahn.twisted.resource import WebSocketResource
class ProtoPS(WebSocketServerProtocol):
def onMessage(self,payload,isBinary):
if not isBinary:
print("Text message received: {}".format(payload.decode('utf8')))
self.sendMessage(payload,isBinary)
class BaseResource:
def __init__(self,proto,vid):
self.vid = vid
self.factory = WebSocketServerFactory()
self.factory.protocol = proto
self.resource = WebSocketResource(self.factory)
wsroot.putChild(self.vid,self.resource)
class BackendProto(Protocol):
def __init__(self):
self.SERVICEMAP = {}
def dataReceived(self,data):
msg = json.loads(data)
if ('cmd' in msg) and (msg['cmd'] == "create"):
self.vid = msg['client']['id']
self.SERVICEMAP[self.vid] = BaseResource(ProtoPS,self.vid)
def connectionLost(self,reason):
wsroot.delEntity(self.vid)
del self.SERVICEMAP[self.vid]
class BackendFactory(Factory):
protocol = BackendProto
if __name__ == '__main__':
reactor.listenTCP(8081,BackendFactory())
wsroot = Data("","text/plain")
wssite = Site(wsroot)
reactor.listenTCP(9000,wssite)
Related
Env: Python 3.6, and Django 2.1
I have created a Django website and a socket server, and files are organized like this:
web
...
user (a Django app)
__init__.py
views.py
...
server.py
Actually I want to build a umbrella rental system by using django, and server connects to umbrella shelf via multi-thread socket (sending some messages). Like I press the borrow button, and views.py can call the server test_function and send some messages to the connected umbrella shelf.
I can import server variables or functions in views.py, but I cannot get the right answer while server.py is running. So I want to ask you if you could give me some advice. Thanks a lot!
By the way, I tried to import the global variable clients directly in views.py, but still got [].
server.py defines a multi-thread server, which is basically as below:
clients = []
class StuckThread(threading.Thread):
def __init__(self, **kwargs):
self.name = kwargs.get('name', '')
def run(self):
while True:
# do something
def func1(self):
# do something
def test_function(thread_name):
# if the function is called by `views.py`, then `clients = []` and return 'nothing', but if I call this function in `server.py`, then I can get a wanted result, which is `got the thread`
for client in clients:
if client['thread'].name == thread_name:
return 'got the thread'
return 'nothing'
if __name__ == '__main__':
ip_port = ('0.0.0.0', 65432)
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ip_port)
server.listen(max_listen_num)
while True:
client, address = socket.accept()
param = {'name': 'test name'}
stuck_thread = StuckThread(**param)
clients.append({"client": client, "address": address, "thread": stuck_thread})
stuck_thread.start()
and I have a Django views.py like this
def view_function(request):
from server import clients
print(clients) # got []
form server import test_function
print(test_function('test name')) # got 'nothing'
return render(request, 'something.html')
I have solve this problem by socket communication between django views.py and server.py. I open another port to receive messages from views.py. Once the borrow button is pressed, a socket client in views.py will build up and send arguments and other messages to the server.
I have a django application and I want to consume messages from a rabbit mq. I want the listener to start consuming when I start the django server.I am using pika library to connect to rabbitmq.Proving some code example will really help.
First you need to somehow run your application at the start of the django project
https://docs.djangoproject.com/en/2.0/ref/applications/#django.apps.AppConfig.ready
def ready(self):
if not settings.IS_ACCEPTANCE_TESTING and not settings.IS_UNITTESTING:
consumer = AMQPConsuming()
consumer.daemon = True
consumer.start()
Further in any convenient place
import threading
import pika
from django.conf import settings
class AMQPConsuming(threading.Thread):
def callback(self, ch, method, properties, body):
# do something
pass
#staticmethod
def _get_connection():
parameters = pika.URLParameters(settings.RABBIT_URL)
return pika.BlockingConnection(parameters)
def run(self):
connection = self._get_connection()
channel = connection.channel()
channel.queue_declare(queue='task_queue6')
print('Hello world! :)')
channel.basic_qos(prefetch_count=1)
channel.basic_consume(self.callback, queue='queue')
channel.start_consuming()
This will help
http://www.rabbitmq.com/tutorials/tutorial-six-python.html
I am pretty new to Twisted, so I am sure this is a rookie mistake. I have built a simple server which receives a message from the client and upon receipt of message the server fires a callback which prints the message to the console.
At first instance, the server works as expected. Unfortunately, when I start up a second client I get the follow error "twisted.internet.defer.AlreadyCalledError." It was my understanding that the factory would make a new instance of the deferred i.e. the new deferred wouldn't have been called before?
Please see the code below. Any help would be very appreciated.
import sys
from twisted.internet.protocol import ServerFactory, Protocol
from twisted.internet import defer
class LockProtocol(Protocol):
lockData = ''
def dataReceived(self, data):
self.lockData += data
if self.lockData.endswith('??'):
self.lockDataReceived(self.lockData)
def lockDataReceived(self, lockData):
self.factory.lockDataFinished(lockData)
class LockServerFactory(ServerFactory):
protocol = LockProtocol
def __init__(self):
self.deferred = defer.Deferred() # Initialise deferred
def lockDataFinished(self, lockData):
self.deferred.callback(lockData)
def clientConnectionFailed(self, connector, reason):
self.deferred.errback(reason)
def main():
HOST = '127.0.0.1' # localhost
PORT = 10001
def got_lockData(lockData):
print "We have received lockData. It is as follows:", lockData
def lockData_failed(err):
print >> sys.stderr, 'The lockData download failed.'
errors.append(err)
factory = LockServerFactory()
from twisted.internet import reactor
# Listen for TCP connections on a port, and use our factory to make a protocol instance for each new connection
port = reactor.listenTCP(PORT,factory)
print 'Serving on %s' % port.getHost()
# Set up callbacks
factory.deferred.addCallbacks(got_lockData,lockData_failed)
reactor.run() # Start the reactor
if __name__ == '__main__':
main()
Notice that there is only one LockServerFactory ever created in your program:
factory = LockServerFactory()
However, as many LockProtocol instances are created as connections are accepted. If you have per-connection state, the place to put it is on LockProtocol.
It looks like your "lock data completed" event is not a one-off so a Deferred is probably not the right abstraction for this job.
Instead of a LockServerFactory with a Deferred that fires when that event happens, perhaps you want a multi-use event handler, perhaps custom built:
class LockServerFactory(ServerFactory):
protocol = LockProtocol
def __init__(self, lockDataFinished):
self.lockDataFinished = lockDataFinished
factory = LockServerFactory(got_lockData)
(Incidentally, notice that I've dropped clientConnectionFailed from this implementation: that's a method of ClientFactory. It will never be called on a server factory.)
I am currently working on developing a Twitch.tv chat and moderation bot(full code can be found on github here: https://github.com/DarkElement75/tecsbot; might not be fully updated to match the problem I describe), and in doing this I need to have many different Twisted TCP connections to various channels. I have (because of the way Twitch's Whisper system works) one connection for sending / receiving whispers, and need the ability for any connection to any channel can reference this whisper connection and send data on this connection, via the TwitchWhisperBot's write() function. However, I have yet to find a method that allows my current global function to reference this write() function. Here is what I have right now:
#global functions
def send_whisper(self, user, msg):
whisper_str = "/w %s %s" % (user, msg)
print dir(whisper_bot)
print dir(whisper_bot.transport)
whisper_bot.write("asdf")
def whisper(self, user, msg):
'''global whisper_user, whisper_msg
if "/mods" in msg:
thread.start_new_thread(get_whisper_mods_msg, (self, user, msg))
else:
whisper_user = user
whisper_msg = msg'''
if "/mods" in msg:
thread.start_new_thread(get_whisper_mods_msg, (self, user, msg))
else:
send_whisper(self, user, msg)
#Example usage of these (inside a channel connection):
send_str = "Usage: !permit add <user> message/time/<time> <message count/time duration/time unit>/permanent"
whisper(self, user, send_str)
#Whisper classes
class TwitchWhisperBot(irc.IRCClient, object):
def write(self, msg):
self.msg(self.channel, msg.encode("utf-8"))
logging.info("{}: {}".format(self.nickname, msg))
class WhisperBotFactory(protocol.ClientFactory, object):
wait_time = 1
def __init__(self, channel):
global whisper_bot
self.channel = channel
whisper_bot = TwitchWhisperBot(self.channel)
def buildProtocol(self, addr):
return TwitchWhisperBot(self.channel)
def clientConnectionLost(self, connector, reason):
# Reconnect when disconnected
logging.error("Lost connection, reconnecting")
self.protocol = TwitchWhisperBot
connector.connect()
def clientConnectionFailed(self, connector, reason):
# Keep retrying when connection fails
msg = "Could not connect, retrying in {}s"
logging.warning(msg.format(self.wait_time))
time.sleep(self.wait_time)
self.wait_time = min(512, self.wait_time * 2)
connector.connect()
#Execution begins here:
#main whisper bot where other threads with processes will be started
#sets variables for connection to twitch chat
whisper_channel = '#_tecsbot_1444071429976'
whisper_channel_parsed = whisper_channel.replace("#", "")
server_json = get_json_servers()
server_arr = (server_json["servers"][0]).split(":")
server = server_arr[0]
port = int(server_arr[1])
#try:
# we are using this to make more connections, better than threading
# Make logging format prettier
logging.basicConfig(format="[%(asctime)s] %(message)s",
datefmt="%H:%M:%S",
level=logging.INFO)
# Connect to Twitch IRC server, make more instances for more connections
#Whisper connection
whisper_bot = ''
reactor.connectTCP(server, port, WhisperBotFactory(whisper_channel))
#Channel connections
reactor.connectTCP('irc.twitch.tv', 6667, BotFactory("#darkelement75"))
The simplest solution here would be using the Singleton pattern, since you're guaranteed to only have a single connection of each type at any given time. Personally, with Twisted, I think the simplest solution is to use the reactor to store your instance (since reactor itself is a singleton).
So what you want to do is, inside TwitchWhisperBot, at sign in:
def signedOn(self):
reactor.whisper_instance = self
And then, anywhere else in the code, you can access that instance:
reactor.whisper_instance = self
Just for sanity's sake, you should also check if it's been set or not:
if getattr(reactor, 'whisper_instance'):
reactor.whisper_instance.write("test_user", "message")
else:
logging.warning("whisper instance not set")
Background
The purpose of this project is to create a SMS based kill switch for a program I have running locally. The plan is to create web socket connection between the local program and an app hosted on Heroku. Using Twilio, receiving and SMS will trigger a POST request to this app. If it comes from a number on my whitelist, the application should send a command to the local program to shut down.
Problem
What can I do to find a reference to the namespace so that I can broadcast a message to all connected clients from a POST request?
Right now I am simply creating a new web socket client, connecting it and sending the message, because I can't seem to figure out how to get access to the namespace object in a way that I can call an emit or broadcast.
Server Code
from gevent import monkey
from flask import Flask, Response, render_template, request
from socketio import socketio_manage
from socketio.namespace import BaseNamespace
from socketio.mixins import BroadcastMixin
from time import time
import twilio.twiml
from socketIO_client import SocketIO #only necessary because of the hack solution
import socketIO_client
monkey.patch_all()
application = Flask(__name__)
application.debug = True
application.config['PORT'] = 5000
# White list
callers = {
"+15555555555": "John Smith"
}
# Part of 'hack' solution
stop_namespace = None
socketIO = None
# Part of 'hack' solution
def on_connect(*args):
global stop_namespace
stop_namespace = socketIO.define(StopNamespace, '/chat')
# Part of 'hack' solution
class StopNamespace(socketIO_client.BaseNamespace):
def on_connect(self):
self.emit("join", 'server#email.com')
print '[Connected]'
class ChatNamespace(BaseNamespace, BroadcastMixin):
stats = {
"people" : []
}
def initialize(self):
self.logger = application.logger
self.log("Socketio session started")
def log(self, message):
self.logger.info("[{0}] {1}".format(self.socket.sessid, message))
def report_stats(self):
self.broadcast_event("stats",self.stats)
def recv_connect(self):
self.log("New connection")
def recv_disconnect(self):
self.log("Client disconnected")
if self.session.has_key("email"):
email = self.session['email']
self.broadcast_event_not_me("debug", "%s left" % email)
self.stats["people"] = filter(lambda e : e != email, self.stats["people"])
self.report_stats()
def on_join(self, email):
self.log("%s joined chat" % email)
self.session['email'] = email
if not email in self.stats["people"]:
self.stats["people"].append(email)
self.report_stats()
return True, email
def on_message(self, message):
message_data = {
"sender" : self.session["email"],
"content" : message,
"sent" : time()*1000 #ms
}
self.broadcast_event_not_me("message",{ "sender" : self.session["email"], "content" : message})
return True, message_data
#application.route('/stop', methods=['GET', 'POST'])
def stop():
'''Right here SHOULD simply be Namespace.broadcast("stop") or something.'''
global socketIO
if socketIO == None or not socketIO.connected:
socketIO = SocketIO('http://0.0.0.0:5000')
socketIO.on('connect', on_connect)
global stop_namespace
if stop_namespace == None:
stop_namespace = socketIO.define(StopNamespace, '/chat')
stop_namespace.emit("join", 'server#bayhill.com')
stop_namespace.emit('message', 'STOP')
return "Stop being processed."
#application.route('/', methods=['GET'])
def landing():
return "This is Stop App"
#application.route('/socket.io/<path:remaining>')
def socketio(remaining):
try:
socketio_manage(request.environ, {'/chat': ChatNamespace}, request)
except:
application.logger.error("Exception while handling socketio connection",
exc_info=True)
return Response()
I borrowed code heavily from this project chatzilla which is admittedly pretty different because I am not really working with a browser.
Perhaps Socketio was a bad choice for web sockets and I should have used Tornado, but this seemed like it would work well and this set up helped me easily separate the REST and web socket pieces
I just use Flask-SocketIO for that.
from gevent import monkey
monkey.patch_all()
from flask import Flask
from flask.ext.socketio import SocketIO
app = Flask(__name__)
socketio = SocketIO(app)
#app.route('/trigger')
def trigger():
socketio.emit('response',
{'data': 'someone triggered me'},
namespace='/global')
return 'message sent via websocket'
if __name__ == '__main__':
socketio.run(app)