Following code approximates real code. Flask app on starts creates a worker thread. Routing function use data processing done by worker function .
app = Flask(__name__)
timeStr=""
def loop ():
global timeStr
while True:
time.sleep (2)
timeStr =datetime.now().replace(microsecond=0).isoformat()
print (timeStr)
ThreadID = Thread (target=loop)
ThreadID.daemon = True
ThreadID.start()
#app.route('/')
def test():
return os.name + " " + platform.platform() + " " + timeStr
application=app
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8080, debug=True)
The above app works beautifully for days when started likes this:
python3 app.py
However in uwsgi, even though I have enabled threads, app is not working. It's not updating global timeStr
sudo /usr/local/bin/uwsgi --wsgi-file /home/pi/pyTest/app.py --http :80 --touch-reload /home/pi/pyTest/app.py --enable-threads --stats 127.0.0.1:9191
What do I need to do for app to function correctly under UWSGI, so I have create systemd service proper way?
Bad news, good news.
I have an app that starts a worker thread in much the same way. It uses a queue.Queue to let routes pass work on the worker thread. The app has been running happily on my home intranet (on a Rapsberry Pi) using the Flask development server. I tried putting my app behind uwsgi, and observed the same failure -- the worker thread didn't appear to get scheduled. The thread reported as _is_alive = True, but I couldn't find a uwsgi switch combination that let it actually run.
Using gunicorn resolved the issue.
virtualenv venv --python=python3
. venv/bin/activate
pip install flask gunicorn
gunicorn -b 0.0.0.0:5000 demo:app
was enough to get my app to work (meaning the worker thread actually ran, and side-effects were noticeable).
Related
I'm building a server with Flask/Gunicorn and Nginx. My script (Flask server) does two things with 'threading':
connect to MQTT broker
run a flask server
But when i try using gunicorn: gunicorn --bind 0.0.0.0:5000 wsgi:app, the first thread doens't run.
Here is the code (not complet):
import threading
def run_mqtt():
while True:
mqtt_client.connect(mqtt_server, port=mqtt_port)
def run_server():
app.run(host='0.0.0.0', port=5000, debug=False)
if __name__ == '__main__':
t1 = threading.Thread(target=run_mqtt)
t2 = threading.Thread(target=run_server)
t1.daemon = True
t2.daemon = True
t1.start()
t2.start()
Please help me, i have to find the solution very fast! Thanks!!
Gunicorn is based on the pre-fork worker model. That means that when it starts, it has a master process and spawns off worker processes as necessary. Most likely the first thread did run, but you lost track of it in the other prefork processes.
If you want to have a background thread that flask controllers can interact with and share memory with, it's unlikely that gunicorn is a good solution for you.
Auto reloader (on code change) works fine using app.run/flask run --host=0.0.0.0 --port 8080.
However as soon as I add flask_socketio to the mix it stops working.
E.g. bringing in:
from flask_socketio import SocketIO, send, emit
socketio = SocketIO(app, path='/kpi/socket.io')
socketio.run(app, host='0.0.0.0', port=8080, debug=True, use_reloader=True, use_debugger=True)
requirements.txt
Flask==1.0.2
Flask-Bootstrap==3.3.7.1
Gunicorn==19.8.1
inotify==0.2.9
pymysql==0.7.2
flask-socketio==3.0.1
eventlet==0.23.0
socketIO-client==0.7.2
I do get "* Restarting with stat" in the logs, same as I would if I was typically running flask.
Extraneous info: this is running in a docker container where I have a compose file for dev where I run the dev mode, and have an env variable for FLASK_DEBUG=1. Using nginx to proxy the rest to port 80 then I server that up to another local port. This all works fine and dandy until I add socketio.
Ending up doing this:
Running this in dev (this brings in the werkzeug debugger for wsgi)
# TODO run this only in dev
from werkzeug.debug import DebuggedApplication
app.debug = True
app.wsgi_app = DebuggedApplication(app.wsgi_app, evalex=True)
Then use supervisord and run uwsgi instead including the flag
--py-autoreload 1
Those both return the functionality I want in development. Just need to make them only run in dev mode now and I'm good.
I have a test Django site using a mod_wsgi daemon process, and have set up a simple Celery task to email a contact form, and have installed supervisor.
I know that the Django code is correct.
The problem I'm having is that when I submit the form, I am only getting one message - the first one. Subsequent completions of the contact form do not send any message at all.
On my server, I have another test site with a configured supervisor task running which uses the Django server (ie it's not using mod_wsgi). Both of my tasks are running fine if I do
sudo supervisorctl status
Here is my conf file for the task I've described above which is saved at
/etc/supervisor/conf.d
the user in this instance is called myuser
[program:test_project]
command=/home/myuser/virtualenvs/test_project_env/bin/celery -A test_project worker --loglevel=INFO --concurrency=10 -n worker2#%%h
directory=/home/myuser/djangoprojects/test_project
user=myuser
numprocs=1
stdout_logfile=/var/log/celery/test_project.out.log
stderr_logfile=/var/log/celery/test_project.err.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
stopasgroup=true
; Set Celery priority higher than default (999)
; so, if rabbitmq is supervised, it will start first.
priority=1000
My other test site has this set as the command - note worker1#%%h
command=/home/myuser/virtualenvs/another_test_project_env/bin/celery -A another_test_project worker --loglevel=INFO --concurrency=10 -n worker1#%%h
I'm obviously doing something wrong in that my form is only submitted. If I look at the out.log file referred to above, I only see the first task, nothing is visible for the other form submissions.
Many thanks in advance.
UPDATE
I submitted the first form at 8.32 am (GMT) which was received, and then as described above, another one shortly thereafter for which a task was not created. Just after finishing the question, I submitted the form again at 9.15, and for this a task was created and the message received! I then submitted the form again, but no task was created again. Hope this helps!
use ps auxf|grep celery to see how many worker you started,if there is any other worker you start before and you don't kill it ,the worker you create before will consume the task,result in you every two or three(or more) times there is only one task is received.
and you need to stop celery by:
sudo supervisorctl -c /etc/supervisord/supervisord.conf stop all
everytime, and set this in supervisord.conf:
stopasgroup=true ; send stop signal to the UNIX process group (default false)
Otherwise it will causes memory leaks and regular task loss.
If you has multi django site,here is a demo support by RabbitMQ:
you need add rabbitmq vhost and set user to vhost:
sudo rabbitmqctl add_vhost {vhost_name}
sudo rabbitmqctl set_permissions -p {vhost_name} {username} ".*" ".*" ".*"
and different site use different vhost(but can use same user).
add this to your django settings.py:
BROKER_URL = 'amqp://username:password#localhost:5672/vhost_name'
some info here:
Using celeryd as a daemon with multiple django apps?
Running multiple Django Celery websites on same server
Run Multiple Django Apps With Celery On One Server With Rabbitmq VHosts
Run Multiple Django Apps With Celery On One Server With Rabbitmq VHosts
Hello I have a django app. My whole system configuration is the following: python 3, django 1.11, eventlet 0.21.0.
1) Nginx as an upstream server:
upstream proj_server {
server unix:///tmp/proj1.sock fail_timeout=0;
server unix:///tmp/proj2.sock fail_timeout=0;
}
2) Supervisor that controls workers. There is a gunicorn worker:
[program:proj]
command=/home/vagrant/.virtualenvs/proj/bin/gunicorn -c /vagrant/proj/proj/proj/deploy/gunicorn.small.conf.py proj.wsgi:application
directory=/vagrant/proj/proj/proj/deploy
user=www-data
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/proj.log
3) This is a gunicorn.small.conf content:
bind = ["unix:///tmp/proj1.sock", "unix:///tmp/proj2.sock"]
pythonpath = "/vagrant/proj/proj/proj/deploy"
workers = 2
worker_class = "eventlet"
worker_connections = 10
timeout = 60
graceful_timeout = 60
4) And this is proj.wsgi content:
"""
WSGI config for proj project.
This module contains the WSGI application used by Django's development server
and any production WSGI deployments. It should expose a module-level variable
named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
this application via the ``WSGI_APPLICATION`` setting.
Usually you will have the standard Django WSGI application here, but it also
might make sense to replace the whole Django WSGI application with a custom one
that later delegates to the Django one. For example, you could introduce WSGI
middleware here, or combine a Django application with an application of another
framework.
"""
import eventlet
eventlet.monkey_patch()
from eventlet import wsgi
import django.core.handlers.wsgi
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")
# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
# Apply WSGI middleware here.
# from helloworld.wsgi import HelloWorldApplication
# application = HelloWorldApplication(application)
So, as you can see there is a chain: nginx as an upstream server calls one of the gunicorn eventlet workers using two sockets proj1.sock or proj2.sock.
Note that according with the eventlet documentation I try to use eventlet.monkey_patch() as early as possible. The most appropriate place for that is proj.wsgi that is called by gunicorn in the first place.
However it seems that library isn't monkey patched.
To check this I added to the proj/proj/proj/__init__.py (the first module that called by the django application) the following code:
import eventlet
import os
print("monkey patched os is: " + str(eventlet.patcher.is_monkey_patched('os')))
print("monkey patched select is: " + str(eventlet.patcher.is_monkey_patched('select')))
print("monkey patched socket is: " + str(eventlet.patcher.is_monkey_patched('socket')))
print("monkey patched time is: " + str(eventlet.patcher.is_monkey_patched('time')))
print("monkey patched subprocess is: " + str(eventlet.patcher.is_monkey_patched('subprocess')))
then i issued **./manage.py check** and got that answer:
monkey patched os is: false
monkey patched select is: false
monkey patched socket is: false
monkey patched time is: false
monkey patched subprocess is: false
What am I doing wrong?
What if you change proj.wsgi file content to one line raise Exception? That should eliminate eventlet from suspects.
I'm not good with Django, here's a pure speculation:
based on name, proj.wsgi is executed when WSGI server is about to start
manage.py check doesn't seem to be related to remote network service (WSGI), seems to be a general management command, so it shouldn't execute WSGI related code
One possible solution, taken from your question text:
proj/proj/proj/init.py (the first module that called by the django application
Try to put monkey_patch call in there.
P.S.: you don't need supervisor for gunicorn, its master process (arbiter) is designed to run forever in spite of problems with workers.
I have a python script written with Flask, which requires some preparation work (connect to databases, acquire some other resources etc) before it can actually accept requests.
I was using it under Apache HTTPD with wsgi. Apache config:
WSGIDaemonProcess test user=<uid> group=<gid> threads=1 processes=4
WSGIScriptAlias /flask/test <path>/flask/test/data.wsgi process-group=test
And it was working fine: Apache would start 4 completely separate processes, each with its own database connection.
I am now trying to switch to uwsgi + nginx. nginx config:
location /flask/test/ {
include uwsgi_params;
uwsgi_pass unix:/tmp/uwsgi.sock;
}
uwsgi:
uwsgi -s /tmp/uwsgi.sock --mount /flask/test=test.py --callable app --manage-script-name --processes=4 --master
The simplified script test.py:
from flask import Flask, Response
app = Flask(__name__)
def do_some_preparation():
print("Prepared!")
#app.route("/test")
def get_test():
return Response("test")
do_some_preparation()
if __name__ == "__main__":
app.run()
What I would expect is to see "Prepared!" 4 times in the output. However, uwsgi does not do that, output:
Python main interpreter initialized at 0x71a7b0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 363800 bytes (355 KB) for 4 cores
*** Operational MODE: preforking ***
mounting test.py on /flask/test
Prepared! <======================================
WSGI app 0 (mountpoint='/flask/test') ready in 0 seconds ...
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1212)
spawned uWSGI worker 1 (pid: 1216, cores: 1)
spawned uWSGI worker 2 (pid: 1217, cores: 1)
spawned uWSGI worker 3 (pid: 1218, cores: 1)
spawned uWSGI worker 4 (pid: 1219, cores: 1)
So, in this simplified example, uwsgi spawned 4 workers, but executed do_some_preparation() only once. In real application, there are several database connections opened, and apparently those are being reused by these 4 processes and cause issues with concurrent request.
Is there a way to tell uwsgi to spawn several completely separate processes?
EDIT: I could, of course, get it working with a workaround like:
from flask import Flask, Response
app = Flask(__name__)
all_prepared = False
def do_some_preparation():
global all_prepared
all_prepared = True
print("Prepared!")
#app.route("/test")
def get_test():
if not all_prepared:
do_some_preparation()
return Response("test")
if __name__ == "__main__":
app.run()
But then I will have to place this "all_prepared" check into every route, which does not seem like a good solution.
By default uWSGI does preforking. So your app is loaded one time, and then forked.
If you want to load the app one time per worker add --lazy-apps to the uWSGI options.
By the way in both cases you are under true multiprocessing :)
It seems like I have found an answer myself. And the answer is: my code should have been redesigned as:
#app.before_first_request
def do_some_preparation():
...
Then Flask will take care of running do_some_preparation() function for each worker separately, allowing each one to have its own database connection (or other concurrency-intolerant resource).