I am trying to run a flask web app and a flask restful api on the same domain.
They both run fine, when they are run alone.
I am not using venv.
However, when I try to run them under a same domain, Nginx throws an error - 404, not found. Also, localhost/ runs the web-app fine without errors but localhost/web and localhost/rest gives a 404.
The request.url prints http://localhost/web and http://localhost/rest but request is not being sent to flask server responsible for `/rest', it seems, as no data is logged for the second server (the restful api). All logged data is for the 1st server (the web app) only.
#Nginx.conf:
server {
listen 80;
server_name localhost;
location /web {
include uwsgi_params;
uwsgi_pass unix:/var/www/webproject/web_uwsgi.sock;
}
location /api {
include uwsgi_params;
uwsgi_pass unix:/var/www/restproject/rest_api_uwsgi.sock;
}
}
#webproject.ini #restproject.ini
[uwsgi] [uwsgi]
vhost = true vhost = true
project = webproject project = restproject
base = /var/www/ base = /var/www/
chdir = %(base)%(project) chdir = %(base)%(project)
wsgi-file = run.py wsgi-file = api.py
callable = app callable = ap
mount = /webproject = app mount = /restproject = ap
manage-script-name = true manage-script-name = true
plugins = python plugins = python
socket = /var/www/webproject/%n. socksocket = /var/www/restproject/%n.sock
chmod-socket = 666 chmod-socket = 666
logto = /var/log/uwsgi/%n.log logto = /var/log/uwsgi/%n.log
Any suggestion would be helpful.
Related
I am using telegram bot api (telebot), flask and gunicorn.
When I use command python app.py everything is work fine but when I use python wsgi.py flask is stated on http://127.0.0.1:5000/ and bot doesn't answer and if I am using gunicorn --bind 0.0.0.0:8443 wsgi:app webhook is setting but telegram bot doesn't answer. I tried to add app.run from app.py to wsgi.py but it doesn't work
app.py
import logging
import time
import flask
import telebot
API_TOKEN = '111111111:token_telegram'
WEBHOOK_HOST = 'droplet ip'
WEBHOOK_PORT = 8443 # 443, 80, 88 or 8443 (port need to be 'open')
WEBHOOK_LISTEN = '0.0.0.0' # In some VPS you may need to put here the IP addr
WEBHOOK_SSL_CERT = 'webhook_cert.pem' # Path to the ssl certificate
WEBHOOK_SSL_PRIV = 'webhook_pkey.pem' # Path to the ssl private key
WEBHOOK_URL_BASE = "https://%s:%s" % (WEBHOOK_HOST, WEBHOOK_PORT)
WEBHOOK_URL_PATH = "/%s/" % (API_TOKEN)
logger = telebot.logger
telebot.logger.setLevel(logging.DEBUG)
bot = telebot.TeleBot(API_TOKEN)
app = flask.Flask(__name__)
# Empty webserver index, return nothing, just http 200
#app.route('/', methods=['GET', 'HEAD'])
def index():
return ''
# Process webhook calls
#app.route(WEBHOOK_URL_PATH, methods=['POST'])
def webhook():
if flask.request.headers.get('content-type') == 'application/json':
json_string = flask.request.get_data().decode('utf-8')
update = telebot.types.Update.de_json(json_string)
bot.process_new_updates([update])
return ''
else:
flask.abort(403)
# Handle all other messages
#bot.message_handler(func=lambda message: True, content_types=['text'])
def echo_message(message):
bot.reply_to(message, message.text)
# Remove webhook, it fails sometimes the set if there is a previous webhook
bot.remove_webhook()
#
time.sleep(1)
# Set webhook
bot.set_webhook(url=WEBHOOK_URL_BASE + WEBHOOK_URL_PATH,
certificate=open(WEBHOOK_SSL_CERT, 'r'))
if __name__ == "__main__":
# Start flask server
app.run(host=WEBHOOK_LISTEN,
port=WEBHOOK_PORT,
ssl_context=(WEBHOOK_SSL_CERT, WEBHOOK_SSL_PRIV),
debug=True)
wsgi.py
from app import app
if __name__ == "__main__":
app.run()
It's a bad practice to show python webservers to the world. It's not secure.
Good practice - using a reverse proxy, e.g. nginx
Chain
So the chain shoud be:
api.telegram.org -> your_domain -> your_nginx -> your_webserver -> your_app
SSL
Your ssl certificates shoud be checked on nginx level. On success just pass request to your webserver (flask or something else). It's also a tip about how to use infitity amount of bots on one host/port :)
How to configure nginx reverse proxy - you can find in startoverflow or google.
Telegram Webhooks
telebot is so complicated for using webhooks.
Try to use aiogram example. It's pretty simple:
from aiogram import Bot, Dispatcher, executor
from aiogram.types import Message
WEBHOOK_HOST = 'https://your.domain'
WEBHOOK_PATH = '/path/to/api'
bot = Bot('BOT:TOKEN')
dp = Dispatcher(bot)
#dp.message_handler()
async def echo(message: Message):
return message.answer(message.text)
async def on_startup(dp: Dispatcher):
await bot.set_webhook(f"{WEBHOOK_HOST}{WEBHOOK_PATH}")
if __name__ == '__main__':
executor.start_webhook(dispatcher=dp, on_startup=on_startup,
webhook_path=WEBHOOK_PATH, host='localhost', port=3000)
app.run(host=0.0.0.0, port=5000, debug=True)
Change port number and debug arguments based on your case.
I'm serving my django app behind a reverse proxy
The internet -> Nginx -> Gunicorn socket -> Django app
In the nginx config:
upstream my_server {
server unix:/webapps/my_app/run/gunicorn.sock fail_timeout=0;
}
The SSL is set up with certbot at the nginx level.
request.build_absolute_uri in views.py generates http links. How can I force it to generate https links?
By default Django ignore all X-Forwarded headers, base on Django docs.
Force read the X-Forwarded-Host header by setting USE_X_FORWARDED_HOST = True and set SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https'). So in settings.py:
USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
I use django behind apache2 so my solution was to put this on apache2
<VirtualHost *:443>
RequestHeader set X-Forwarded-Proto 'https' env=HTTPS
After adding headers mod:
a2enmod headers
And this on django setting.py:
USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
With this all my build_absolute_uri started with https
There's a note in the django documentation here https://docs.djangoproject.com/en/3.0/ref/request-response/#django.http.HttpRequest.build_absolute_uri:
Mixing HTTP and HTTPS on the same site is discouraged, therefore build_absolute_uri() will always generate an absolute URI with the same scheme the current request has. If you need to redirect users to HTTPS, it’s best to let your Web server redirect all HTTP traffic to HTTPS.
The following is my python code
from gevent import monkey;monkey.patch_all()
from flask import Flask,render_template, url_for, request, redirect, flash,jsonify,session,Markup
from socketio import socketio_manage
from socketio.namespace import BaseNamespace
from socketio.server import SocketIOServer
app=Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
class ChatNamespace(BaseNamespace):
def recv_connect(self):
print "successfully connected"
self.emit('show_result','successfully connect')
def on_receive_message(self,msg):
print "message is "+msg["data"]
self.emit('show_result2',msg["data"])
#app.route('/')
def index():
#print "in the function"
return render_template('index.html')
#app.route("/socket.io/<path:path>")
def run_socketio(path):
socketio_manage(request.environ, {'/test': ChatNamespace})
return 'ok'
if __name__=='__main__':
#app.run(debug=True, port=80, host='0.0.0.0')
app.debug=True
#app.run()
SocketIOServer(('0.0.0.0', 5000), app,resource="socket.io").serve_forever()
print "successfull listen to socket"
and the following is nginx configuration
server {
location / {
proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
try_files $uri #proxy;
}
location #proxy {
proxy_pass http://127.0.0.1:8000;
}
location /templates {
alias /home/www/flask_project/templates/;
}
location /script {
alias /home/www/flask_project/script/;
}
location /static {
alias /home/www/flask_project/static/;
}
}
Each time I run the app ,I use the following command
gunicorn main2:app -b localhost:5000
I know that I am missing a lot of information to run this gevent-socketio app on a live server.
Can anyone help me out, I am totally new to this web-socket technology
Have you tried FlaskSocketio??
If you are trying to do it without the extension, you will need the socketio gunicorn worker to run your app.
gunicorn --worker-class socketio.sgunicorn.GeventSocketIOWorker module:app
Previously I was working on my Django app on a local server and all my settings work with django-allauth (I was using manage.py runserver rather than nginx + gunicorn for staging/production)
Now, I'm implementing django-allauth to my staging website, but I can't make it work on my staging server (Note: In this example I've replaced my domain name to mydomain.com).
Basically after I clicked the link to login with Facebook at http://staging.mydomain.com/accounts/login/, it redirects to https://www.facebook.com/dialog/oauth?response_type=code&state=YXdAxg2WiIBo&redirect_uri=http%3A%2F%2Fstaging.mydomain.com%2Faccounts%2Ffacebook%2Flogin%2Fcallback%2F&client_id=1600059933550804&scope=email&auth_type=reauthenticate and I get the following errors:
Given URL is not allowed by the Application configuration.: One or
more of the given URLs is not allowed by the App's settings. It must
match the Website URL or Canvas URL, or the domain must be a subdomain
of one of the App's domains.
Here are my setting files:
nginx conf file
server {
listen 80;
server_name staging.mydomain.me;
location /static {
alias /home/myusername/sites/staging.mydomain.me/static;
}
location / {
proxy_set_header Host $host;
proxy_pass http://unix:/tmp/staging.mydomain.me.socket;
}
}
settings.py
...
SITE_ID = 1
LOGIN_REDIRECT_URL = '/'
ACCOUNT_EMAIL_REQUIRED = True
SOCIALACCOUNT_PROVIDERS = {
'facebook': {
'SCOPE': ['email'],
'AUTH_PARAMS': {'auth_type': 'reauthenticate'},
'METHOD': 'oauth2',
'VERIFIED_EMAIL': False,
}
}
AUTH_USER_MODEL = 'users.User'
ACCOUNT_ADAPTER = 'users.adapters.AccountAdapter'
ACCOUNT_SIGNUP_FORM_CLASS = 'users.forms.UserApplicationForm'
ACCOUNT_LOGOUT_ON_GET = True
...
Settings in Facebook app
Settings in Sites [http://staging.mydomain.com/admin/sites/site/1/]
Settings in Social application [http://staging.mydomain.com/admin/socialaccount/socialapp/2/]
Try one of theese 2 things:
create a test app for your facebook app, and set the staging.mydomain.com domain for it, configure a brand new social app (allauth) with the new api key and secret (for your new test app), a new django site, and so on...
change your facebook app settings to have staging.mydomain.com as its app domain.
I think both solutions will work, but I prefer the first one, as your staging app perfectly fits in definition of 'test app' and therefore, I think is a more elegant solution.
Hope this helps you.
I am serving my django project using cherrypy's wsgi server. The server serves all static and media files from django project. The code is the following
#create basic django WSGI app
app = WSGIHandler()
path = {'/': app}
#create app that handles static files from static root and put it in path
path[static_url] = StaticFileWsgiApplication(static_root)
#create app that handles /media and put it in path
path[media_url] = StaticFileWsgiApplication(media_root)
#dispatch the applications
dispatcher = wsgiserver.WSGIPathInfoDispatcher(path)
if __name__ == '__main__':
print 'Close window to exit'
#create wsgi cherrypy server
server = wsgiserver.CherryPyWSGIServer(
('0.0.0.0', 8000),
dispatcher,
server_name='cherrypy-django',
numthreads=20
)
try:
server.start()
except KeyboardInterrupt:
server.stop()
and the static file app
class StaticFileWsgiApplication(object):
#init function to get the directory of collectstatic (STATIC_ROOT)
def __init__(self, static_file_root):
self.static_file_root = os.path.normpath(static_file_root)
#Every wsgi app must be callable. It needs 2 arguments
#eviron: wsgi environ path
#start_response:Function creates the response
def __call__(self, environ, start_respone):
def done(status, headers, output):
#usefull for debugging
#returns the output(actual static file)
#also it produces the response using the start_response()
start_respone(status, headers.items())
return output
#get the path_info see PEP 333
path_info = environ['PATH_INFO']
#remove leading '/' from path (URI)
if path_info[0]=='/':
path_info = path_info[1:]
#actual file path in filesystem
file_path = os.path.normpath((os.path.join(self.static_file_root, path_info)))
#prevent escaping out of paths bellow media root (e.g via '..')
if not file_path.startswith(self.static_file_root):
status = '401 UNAUTHORIZED'
headers = {'Content_type':'text/plain'}
output = ['Permission denied. Illegal Path']
return done(status, headers, output)
#Only allow GET and HEAD requests not PUT, POST, DELETE
if not (environ['REQUEST_METHOD'] == 'GET' or environ['REQUEST_METHOD'] == 'HEAD'):
status = '405 METHOD NOT ALLOWED'
headers = {'Content_type':'text/plain'}
output = SimpleResponse(['405:Method not allowed'])
output.status_code = 405
return done(status, headers, output)
if not (os.path.exists(file_path)):
status = "404 NOT FOUND"
headers = {'Content_type':'text\plain'}
output = SimpleResponse(['Page not found %s' %file_path])
output.status_code = 404
return done(status, headers, output)
try:
fp = open(file_path, 'rb')
except IOError:
status = '401 UNAUTHORIZED'
headers = {'Content_type':'text/plain'}
output = SimpleResponse(['Permission denied %s' %file_path])
output.status_code = 401
return done(status, headers, output)
#This is a very simple implementation of conditional GET with
#the Last-Modified header. It makes media files a bit speedier
#because the files are only read off disk for the first request
#(assuming the browser/client supports conditional GET).
#mimetype needs to be ascii not uinicode, as django is all unicode, need to do conversion
mtime = http_date(os.stat(file_path)[stat.ST_MTIME]).encode('ascii','ignore')
if environ.get('HTTP_IF_MODIFIED_SINCE', None) == mtime:
headers = {'Last-Modified':mtime}
status = '304 NOT MODIFIED'
output = SimpleResponse()
output.status_code = 304
else:
status = '200 OK'
mime_type = mimetypes.guess_type(file_path)[0]
if mime_type:
headers = {'Content_type':mime_type}
output = BlockIteratorResponse(fp)
return done(status, headers,output)
I get an error in call about not initialising headers before using it...probably because all headers are inside if's. The error is
UnboundLocalError:Local variable "headers" referenced before assignment
The funny thing is that i get this error only on Mozilla but web works fine, everything is served correctly, and after i sign in to my webpage it doesn't show the error anymore.
Should I initialize headers variable in beginning of call like
headers = {}
And why is it happening only in firefox?
If you want to serve static content using CherryPy, you should use built-in tools -- see Serving Static Content. I guess you need to create a CherryPy app serving static content only and then combine it with your Django app with WSGIPathInfoDispatcher(...).
BUT what you should really consider is using nginx + uWSGI -- see Setting up Django and your web server with uWSGI and nginx.