Auto reloader (on code change) works fine using app.run/flask run --host=0.0.0.0 --port 8080.
However as soon as I add flask_socketio to the mix it stops working.
E.g. bringing in:
from flask_socketio import SocketIO, send, emit
socketio = SocketIO(app, path='/kpi/socket.io')
socketio.run(app, host='0.0.0.0', port=8080, debug=True, use_reloader=True, use_debugger=True)
requirements.txt
Flask==1.0.2
Flask-Bootstrap==3.3.7.1
Gunicorn==19.8.1
inotify==0.2.9
pymysql==0.7.2
flask-socketio==3.0.1
eventlet==0.23.0
socketIO-client==0.7.2
I do get "* Restarting with stat" in the logs, same as I would if I was typically running flask.
Extraneous info: this is running in a docker container where I have a compose file for dev where I run the dev mode, and have an env variable for FLASK_DEBUG=1. Using nginx to proxy the rest to port 80 then I server that up to another local port. This all works fine and dandy until I add socketio.
Ending up doing this:
Running this in dev (this brings in the werkzeug debugger for wsgi)
# TODO run this only in dev
from werkzeug.debug import DebuggedApplication
app.debug = True
app.wsgi_app = DebuggedApplication(app.wsgi_app, evalex=True)
Then use supervisord and run uwsgi instead including the flag
--py-autoreload 1
Those both return the functionality I want in development. Just need to make them only run in dev mode now and I'm good.
Related
Locally, it works. Socketio upgrades to websocket instead of resorting to polling.
This is obvious from the logs:
...
FYnWEW0ufWGO7ExdAAAA: Received request to upgrade to websocket
FYnWEW0ufWGO7ExdAAAA: Upgrade to websocket successful
...
Upon deploying the application, it partially works when I create a procfile with the content:
web: gunicorn app:app
The issue here is that socketio fails to upgrade to websocket and therefore resorts to polling.
Here is a gif showcasing that it in production doesn't upgrade to websockets and resorts to spamming pollings instead
My file structure is
wsgi.py
app.py
Procfile
requirements.txt
This is how I initialize socketio
app = ...
socketio = SocketIO(app,
logger=True,
engineio_logger=True,
cors_allowed_origins="*"
)
if __name__ == "__main__":
socketio.run(app, debug=False, port=5000)
Notice Im not setting async_mode, which was the issue for this SO-question
How do I deploy my flask app with socketio to Heroku and have it upgrade to websockets?
I think the issue is that Im just not using the right procfile command to start the application in deployment.
Having a procfile with the content
web: gunicorn --worker-class eventlet -w 1 wsgi:app
Did the job.
Also, it is important that your dyno is set to "ON"
I am trying to set up my django project to work with Celery and Redis. I have no issues running it locally, but I can't get it working in the production server.
My hosting recommends to setup Redis using unixsocket and run redis in a screen:
port 0
unixsocket /path/redis.sock
This all works, when I run redis I get:
* The server is now ready to accept connections at /here-comes-my-path/redis.sock
Now I have issues:
How do I verify the connection? redis-cli -p 0 returns
Could not connect to Redis at 127.0.0.1:0: Can't assign requested address
not connected>
How do I start celery worker? Running celery -A rbwpredictor worker -l info
returns (I've xed sensitive data):
Traceback (most recent call last):
File "/usr/home/xxxx/.virtualenvs/xxxx/bin/celery", line 6, in
from celery.main import main
File "/home/xxx/domains/xxxx/public_python/xxxx/celery.py", line 6, in
from celery import Celery
ImportError: cannot import name 'Celery'
My Celery settings in settings.py:
CELERY_RESULT_BACKEND = 'django-db'
CELERY_STATUS_PENDING = 'PENDING'
CELERY_STATUS_STARTED = 'STARTED'
CELERY_STATUS_RETRY = 'RETRY'
CELERY_STATUS_FAILURE = 'FAILURE'
CELERY_STATUS_SUCCESS = 'SUCCESS'
As mentioned above locally everything works fine, it's the configuration on the server I struggle with.
You have configured redis to communicate through unix socket not through standard port
To connect with redis-cli you can use
redis-cli -s /here-comes-my-path/redis.sock
And you should reconfigure redis.conf or just set BROKER_URL
BROKER_URL = 'redis+socket:///here-comes-my-path/redis.sock'
Hello I have a django app. My whole system configuration is the following: python 3, django 1.11, eventlet 0.21.0.
1) Nginx as an upstream server:
upstream proj_server {
server unix:///tmp/proj1.sock fail_timeout=0;
server unix:///tmp/proj2.sock fail_timeout=0;
}
2) Supervisor that controls workers. There is a gunicorn worker:
[program:proj]
command=/home/vagrant/.virtualenvs/proj/bin/gunicorn -c /vagrant/proj/proj/proj/deploy/gunicorn.small.conf.py proj.wsgi:application
directory=/vagrant/proj/proj/proj/deploy
user=www-data
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/proj.log
3) This is a gunicorn.small.conf content:
bind = ["unix:///tmp/proj1.sock", "unix:///tmp/proj2.sock"]
pythonpath = "/vagrant/proj/proj/proj/deploy"
workers = 2
worker_class = "eventlet"
worker_connections = 10
timeout = 60
graceful_timeout = 60
4) And this is proj.wsgi content:
"""
WSGI config for proj project.
This module contains the WSGI application used by Django's development server
and any production WSGI deployments. It should expose a module-level variable
named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
this application via the ``WSGI_APPLICATION`` setting.
Usually you will have the standard Django WSGI application here, but it also
might make sense to replace the whole Django WSGI application with a custom one
that later delegates to the Django one. For example, you could introduce WSGI
middleware here, or combine a Django application with an application of another
framework.
"""
import eventlet
eventlet.monkey_patch()
from eventlet import wsgi
import django.core.handlers.wsgi
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")
# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
# Apply WSGI middleware here.
# from helloworld.wsgi import HelloWorldApplication
# application = HelloWorldApplication(application)
So, as you can see there is a chain: nginx as an upstream server calls one of the gunicorn eventlet workers using two sockets proj1.sock or proj2.sock.
Note that according with the eventlet documentation I try to use eventlet.monkey_patch() as early as possible. The most appropriate place for that is proj.wsgi that is called by gunicorn in the first place.
However it seems that library isn't monkey patched.
To check this I added to the proj/proj/proj/__init__.py (the first module that called by the django application) the following code:
import eventlet
import os
print("monkey patched os is: " + str(eventlet.patcher.is_monkey_patched('os')))
print("monkey patched select is: " + str(eventlet.patcher.is_monkey_patched('select')))
print("monkey patched socket is: " + str(eventlet.patcher.is_monkey_patched('socket')))
print("monkey patched time is: " + str(eventlet.patcher.is_monkey_patched('time')))
print("monkey patched subprocess is: " + str(eventlet.patcher.is_monkey_patched('subprocess')))
then i issued **./manage.py check** and got that answer:
monkey patched os is: false
monkey patched select is: false
monkey patched socket is: false
monkey patched time is: false
monkey patched subprocess is: false
What am I doing wrong?
What if you change proj.wsgi file content to one line raise Exception? That should eliminate eventlet from suspects.
I'm not good with Django, here's a pure speculation:
based on name, proj.wsgi is executed when WSGI server is about to start
manage.py check doesn't seem to be related to remote network service (WSGI), seems to be a general management command, so it shouldn't execute WSGI related code
One possible solution, taken from your question text:
proj/proj/proj/init.py (the first module that called by the django application
Try to put monkey_patch call in there.
P.S.: you don't need supervisor for gunicorn, its master process (arbiter) is designed to run forever in spite of problems with workers.
I'm working on a Django app that's using the django-websocket-redis app to publish events from the server side.
I've successfully configured a test server using nginx and uWSGI to route both HTTP and WS requests correctly to my Django app, but on my local development environment I cannot get this working.
According to the django-websocket-redis documentation, it's enough to start the Django development server and everything should work fine, but it seems like this is far away from reality.
Here is what I've checked:
redis is running on localhost:6379 and is responding to PING requests
tried to run the server on different ports (80, 8080, 8000) to check if the django-websocket-redis makes any assumption about the development server's port, but nothing changed
searched for solution online, but there is nothing about this topic
On my local environment on the client side I see a 404 error when my app tries to connect to the local WebSocket. My settings.py sets the WEBSOCKET_URL to the correct URL (on test server it's working, but locally isn't).
Anyone has an idea what am I doing wrong? Thanks!
I've found a workaround to this.
I've modified my wsgi.py to this and it works now:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
from django.conf import settings
from ws4redis.django_runserver import WebsocketRunServer
if settings.DEBUG:
_django_app = get_wsgi_application()
_websocket_app = WebsocketRunServer()
def application(environ, start_response):
if environ.get('PATH_INFO').startswith(settings.WEBSOCKET_URL):
return _websocket_app(environ, start_response)
return _django_app(environ, start_response)
else:
application = get_wsgi_application()
UPDATE: it's not needed to create a custom management command as I mentioned previously, so I've deleted that part of the answer.
I have a VirtualBox machine managed by Vagrant. On this machine I run a Django development web server:
./manage.py runserver 0.0.0.0:8080
The codebase is big so for faster code reload I installed pyinotify. Django supports it since 1.7. I'm using 1.7.
The codebase sits in a synced folder (NFS) and I'm using Sublime 3 to edit files on my host machine (OS X).
However inotify doesn't go well with NFS and the code autoreload do not work.
How do I restart a dev server in the VM?
I have a rudimentary web server running next to a dev server. This simple web server listens for a request (on a different port) and touches the project's manage.py. This in turn triggers a restart:
reloader.py
#!/usr/bin/env python
import os
import logging
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(message)s',
datefmt='%m/%d/%Y %I:%M:%S %p')
PORT = 9001
FILE_TO_TOUCH = '/path/to/myenv/manage.py'
def touch(fname, times=None):
with open(fname, 'a'):
os.utime(fname, times)
class HandleTouch(BaseHTTPRequestHandler):
def do_GET(self):
logging.info("Touching %s", FILE_TO_TOUCH)
touch(FILE_TO_TOUCH)
self.send_response(200)
self.send_header("Content-Length", '0')
self.end_headers()
httpd = SocketServer.TCPServer(("", PORT), HandleTouch)
logging.info("Server listening on port %s", PORT)
httpd.serve_forever()
On the host machine I use a simple shell command to watch for file changes and hit the reloader's endpoint:
watcher.sh
#!/bin/sh
watchmedo shell-command -c 'curl http://192.168.10.10:9001' -R -i '/path/to/myenv/manage.py' -p '*.py' ./
Where watchmedo is a utility from watchdog.