Python BaseHTTPServer and Tornado - python-2.7

I'm running a BaseHTTPServer, passed through ThreadedHTTPServer so I get threading.
server = ThreadedHTTPServer(('', int(port)), MainHandler)
Next I fork according to the info here: Daemonizing python's BaseHTTPServer
Then I do:
server.serve_forever()
What I am trying to do is have the same Python script run a Tornado WebSocket server as well, I tried creating the second handler and in my main creating the second server similar to above, but then the serve_forever() blocks (I assume) and I can't start the Tornado WebSocket server.
I had considered using Tornado to serve my general web stuff too but performance was aweful and unusable, so I'd prefer to run it alongside, unless there is a simpler alternative to adding WebSockets to the BaseHTTPServer.
Can anyone offer a solution please?

Yes, serve_forever() blocks it all. You can use handle_request to serve one request at a time. To assure it won't block you have to set timeout. To run it periodically you can use tornado.ioloop.PeriodicCallback. Example:
#!/usr/bin/python
# -*- coding: utf-8 -*-
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import threading
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
message = threading.currentThread().getName()
self.wfile.write(message)
self.wfile.write('\n')
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
class IndexHandler(tornado.web.RequestHandler):
def get(self):
greeting = self.get_argument('greeting', 'Hello')
self.write(greeting + ', friendly user!\n')
if __name__ == '__main__':
# create Tornado Server
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
# create BaseHTTPServer
server = ThreadedHTTPServer(('localhost', 8080), Handler)
server.timeout = 0.01
tornado.ioloop.PeriodicCallback(server.handle_request, 100).start() # every 100 milliseconds
tornado.ioloop.IOLoop.instance().start()
Running:
$ curl http://localhost:8080/
Thread-1
$ curl http://localhost:8080/
Thread-2
$ curl http://localhost:8000/
Hello, friendly user!
$ curl http://localhost:8080/
Thread-3
$ curl http://localhost:8000/
Hello, friendly user!
$ curl http://localhost:8080/
Thread-4
$ curl http://localhost:8000/
Hello, friendly user!
$ curl http://localhost:8000/
Hello, friendly user!
I used here timeout attribute to set timeout. I'm not sure if it's proper way to do it. Other method: http://code.activestate.com/recipes/499376/
Another solution: running every server in its own thread:
#!/usr/bin/python
# -*- coding: utf-8 -*-
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import threading
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
message = threading.currentThread().getName()
self.wfile.write(message)
self.wfile.write('\n')
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
class IndexHandler(tornado.web.RequestHandler):
def get(self):
greeting = self.get_argument('greeting', 'Hello')
self.write(greeting + ', friendly user!\n')
def run_tornado():
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
def run_base_http_server():
server = ThreadedHTTPServer(('localhost', 8080), Handler)
server.serve_forever()
if __name__ == '__main__':
threading.Thread(target=run_tornado).start()
threading.Thread(target=run_base_http_server).start()

Related

How to run two threads ? So that the RunBot function is async?

How to run two threads ? So that the RunBot function is async?
After startup, it works fine - but #client.event does not work due to the fact that the web server thread took
`
import discord
from ast import literal_eval
import aiohttp
import aiosqlite
from quart import Quart, render_template, request, session, redirect, url_for, make_response, websocket
from quart_discord import DiscordOAuth2Session, requires_authorization, Unauthorized
import asyncio
from threading import Thread
import multiprocessing as mp
TOKEN = "token"
client = discord.Client(command_prefix='-=-=-=', intents=discord.Intents.all())
app = Quart(__name__)
#client.event
async def on_ready():
print(f'{client.user} Bot Content')
#app.before_serving
async def before_serving():
async def RunBot()
#don't' work
#create new Thread
await client.run(True)
# loop = asyncio.get_event_loop()
# await client.login(TOKEN)
# loop.create_task(client.connect())
#client.event
async def on_message(message):
print(f'New msg {message.content}, Server: {message.guild}')
#app.route("/")
async def index():
return render_template('index.html')
if __name__ == "__main__":
from hypercorn.config import Config
from hypercorn.asyncio import serve
asyncio.run(serve(app, Config()))
`
I tried a simple launch via task but it didn't work
It is necessary that the web server does not block the client stream

Telebot not responding to requests with FLask server

I am trying to deploy my telegram bot to Heroku using Flask. ALthough I followed the tutorials and set up everything the same I am not getting any result and my bot is not responding.
import requests
import pandas as pd
import telebot
from telebot import types
import random
import os
from flask import Flask, request
from requests.models import MissingSchema
URL_HEROKU = f"some_url"
TOKEN = os.environ['TOKEN']
bot = telebot.TeleBot(TOKEN, parse_mode=None)
app = Flask(__name__)
#bot.message_handler(commands=['start'])
def start(message):
#Init keyboard markup
msg = ''' Hello, how are you?'''
bot.reply_to(message, msg)
#bot.message_handler(commands=['random'])
#bot.message_handler(regexp=r'random')
def send_some(message):
bot.send_message(message.chat.id, text='Hello')
#app.route('/' + TOKEN, methods=['GET'])
def getMessage():
bot.process_new_updates([telebot.types.Update.de_json(request.stream.read().decode("utf-8"))])
return "!", 200
#app.route("/")
def webhook():
bot.remove_webhook()
bot.set_webhook(url= URL_HEROKU + TOKEN)
return "!", 200
if __name__ == "__main__":
app.run(host="0.0.0.0", port=int(os.environ.get('PORT', 5000)))
My Procfile is:
web: python telebreed.py
and my requirements.txt:
Flask==2.0.2
gunicorn==20.1.0
pandas==1.2.2
pyTelegramBotAPI==4.6.0
requests==2.26.0
do you see any mistake ? When I open my app in Heroku I only see "!" which is the character defined in getMessage() method.

Run RASA with flask

I want to run RASA with --enable-api inside the python code rather than the command line. Below is my code which is not working. Let me know how can i do that. The issue is once i hit the service because the channel is 'cmdline' it comes to the command line. I don't know how to resolve this.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import logging
import rasa_core
from rasa_core.agent import Agent
from rasa_core.policies.keras_policy import KerasPolicy
from rasa_core.policies.memoization import MemoizationPolicy
from rasa_core.interpreter import RasaNLUInterpreter
from rasa_core.utils import EndpointConfig
from rasa_core.run import serve_application
from rasa_core import config
from rasa_core.policies.fallback import FallbackPolicy
from rasa_core.policies.keras_policy import KerasPolicy
from flask import Flask
from flask_cors import CORS, cross_origin
app = Flask(__name__)
CORS(app)
logger = logging.getLogger(__name__)
#app.route("/conversations/default/respond",methods=['POST'])
def run_weather_bot(serve_forever=True):
logging.basicConfig(level="ERROR")
interpreter = RasaNLUInterpreter('C:\\xxxx_nlu\\models\\nlu\\default\\weathernlu')
action_endpoint = EndpointConfig(url="http://xxx.xx.xx.xxx:5055/webhook")
agent = Agent.load('C:\\xxxx_nlu\\models\\dialogue', interpreter=interpreter, action_endpoint=action_endpoint)
rasa_core.run.serve_application(agent,channel='cmdline')
return agent
if __name__ == '__main__':
app.run("xxx.xx.xx.xxx",5005,debug=True)
You're calling rasa bot in the command line in your run_weather_bot function using below command.
rasa_core.run.serve_application(agent,channel='cmdline')
As you can see its serving as command line application.
I have made some changes in your code for a conversation with rasa chatbot. You can refer AGENT documentation and Weather bot article for connection of RASA agent and how RASA agent handles the input message.
def rasa_agent():
interpreter = RasaNLUInterpreter("Path for NLU")
action_endpoint = EndpointConfig(url="Webhook URL")
agent = Agent.load('Path to Dialogue', interpreter=interpreter, action_endpoint=action_endpoint)
## Next line runs the rasa in commandline
# rasa_core.run.serve_application(agent,channel='cmdline')
return agent
#app.route("/conversations/default/respond",methods=['POST'])
def run_weather_bot(serve_forever=True):
agent = rasa_agent() # calling rasa agent
## Collect Query from POST request
## Send Query to Agent
## Get Response of BOT
output = {} ## Append output
return jsonify(output)

How to run a Flask app on CherryPy WSGI server (Cheroot) using HTTPS?

I am running a Python 2.7 Flask app on CherryPy Cheroot WSGI server usinh HTTP now as below.
from cheroot.wsgi import Server as WSGIServer
from cheroot.wsgi import PathInfoDispatcher as WSGIPathInfoDispatcher
from MyFlaskApp import app
d = WSGIPathInfoDispatcher({'/': app})
server = WSGIServer(('0.0.0.0', 80), d)
if __name__ == '__main__':
try:
server.start()
except KeyboardInterrupt:
server.stop()
What would I have to to move to HTTPS from here?
I found below instruction, but it does not seem to applicable to my application.
from cheroot.server import HTTPServer
from cheroot.ssl.builtin import BuiltinSSLAdapter
HTTPServer.ssl_adapter = BuiltinSSLAdapter(
certificate='cert/domain.crt',
private_key='cert/domain.key')
Can I apply above sample to my Flask app on Cheroot? If not, what would be a simple example for Flask app on Cheroot for HTTPS?
I figured out the necessary modification.
Not much information on Flask app on Cheroot with https, so I thought I'd share it.
from cheroot.wsgi import Server as WSGIServer
from cheroot.wsgi import PathInfoDispatcher as WSGIPathInfoDispatcher
from cheroot.ssl.builtin import BuiltinSSLAdapter
from MyFlaskApp import app
my_app = WSGIPathInfoDispatcher({'/': app})
server = WSGIServer(('0.0.0.0', 443), my_app)
ssl_cert = "[path]/myapp.crt"
ssl_key = "[path]/myapp.key"
server.ssl_adapter = BuiltinSSLAdapter(ssl_cert, ssl_key, None)
if __name__ == '__main__':
try:
server.start()
except KeyboardInterrupt:
server.stop()

Running a python subprocess in Flask

I have a Flask web app running a crawling process in this fashion:
on terminal tab 1:
$ cd /path/to/scraping
$ scrapyrt
http://scrapyrt.readthedocs.io/en/latest/index.html
on terminal tab 2:
$ python app.pp
and in app.py:
params = {
'spider_name': spider,
'start_requests':True
}
response = requests.get('http://localhost:9080/crawl.json', params)
print ('RESPONSE',response)
data = json.loads(response.text)
which works.
now I'de like to move everthing into app.py, and for that I've tried:
import subprocess
from time import sleep
try:
subprocess.check_output("scrapyrt", shell=True, cwd='path/to/scraping')
except subprocess.CalledProcessError as e:
raise RuntimeError("command '{}' return with error (code {}): {}".format(e.cmd, e.returncode, e.output))
sleep(3)
params = {
'spider_name': spider,
'start_requests':True
}
response = requests.get('http://localhost:9080/crawl.json', params)
print ('RESPONSE',response)
data = json.loads(response.text)
this starts twisted, like so:
2018-02-03 17:29:35-0200 [-] Log opened.
2018-02-03 17:29:35-0200 [-] Site starting on 9080
2018-02-03 17:29:35-0200 [-] Starting factory <twisted.web.server.Site instance at 0x104effa70>
but crawling process hangs and does not go through.
what am I missing here?
Have you considered using a scheduler, such as APScheduler or similar?
You can run code using crons or in intervals, and it integrates well with Flask.
Take a look here:
http://apscheduler.readthedocs.io/en/stable/userguide.html
Here is an example:
from flask import Flask
from apscheduler.schedulers.background import BackgroundScheduler
app = Flask(__name__)
cron = BackgroundScheduler()
cron.start()
#app.route('/')
def index():
return render_template("index.html")
#cron.scheduled_job('interval', minutes=3)
def my_scrapper():
# Do the stuff you need
return True