I was reading about CONN_MAX_AGE settings and the documentation says:
Since each thread maintains its own connection, your database must support at least as many simultaneous connections as you have worker threads.
So I wonder, On uWSGI, how does a Django process maintain it's own threads, does it spawn new thread for each request and kill it at the end of request?
If yes, how does a ceased thread maintain the connection?
Django is not in control of any threads (well... maybe in development server, but it's pretty simple), but uWSGI is. uWSGI will spawn some threads, depending on it's configuration and in each thread it will run django request handling.
Spawning threads may be dynamic or static, it can be strictly 4 threads or dynamic from 2 to 12 depending on load.
And no, there is no new thread on each request because that will allow someone to kill your server by making many concurrent connections to it because it will spawn so many threads that no server will take it.
Requests are handled one by one on each thread, main uWSGI process will round-robin requests between threads. If there are more requests than threads, some of them will wait until others are finished
In uWSGI there are also workers - independent processes that can spawn own threads so load can be better spreaded.
Also you can have multiple uWSGI servers and tell your HTTP server (apache, proxy) to spread requests between them. That way you can even serve your uWSGI instances on different machines and it will all look like from the outside as one big server.
Related
I have 2 celery workers which pool via eventlet, config is below:
celery multi start w1 w2 -A proj -l info --time-limit=600 -P eventlet -c 1000
When running more than 100 tasks at a time, I get hit by the error:
OperationalError: FATAL: remaining connection slots are reserved for
non-replication superuser connections
I'm running on PostgreSQL with max. connections set at the default of 100.
From what I read online, I thought worker threads in the pools would share the same DB connection. However, mine seem to try and create one connection per thread, which is why the error occurs.
Any ideas?
Thanks!
Django has (or had?) idle DB connection reuse to avoid overhead of creating new connection for each request. Idle reuse is not relevant in this scenario.
Django never had limiting DB connection pool. (please correct if wrong)
Consider overall design:
how many tasks do you need to execute concurrently? (real numbers are often not nice powers of 10)
how many simultaneous connections from this application can your database sustain?
do you need to place artificial bottlenecks (pools) or do you need to increase limits and use available hardware?
Consider using external [Postgresql connection pool] (google terms in square braces) or include one somewhere in your application.
I am aware that Django is request/response cycle and Django Channels is different, my question is not about this.
We know that uWSGI/gunicorn creates worker processes and can be configured execute each request in threads. So that it can serve 10 requests "concurrently" (not in parallel) in a single uWSGI worker process with 10 threads.
Now let's assume that each web client wants to create a websocket using Django Channels, from my limited understanding (with vanilla implementation), that it will process each message in a single thread, which means, to process x amount of connections concurrently, you need x amount of channel worker processes. I know someone will suggest to increase the number of processes, I am not here to debate on this.
My question is simply are there any existing libraries that does similar job with uWSGI/gunicorn that execute consumer functions in threads?
I think you are asking for daphne. It is mentioned in channels document itself.
Daphne provides an option to scale process using a shared FD. Unfortunately, it is not working as expected.
Rightnow, a better alternative is to use uvicorn. You can run multiple workers with
$ uvicorn project.asgi --workers 4
I have been using this in production and it seems good enough.
I have a web server running Django with gunicorn using synchronous workers (my gunicorn config is straight out of the box, no gevent or anything).
In a gunicorn worker process, there is obviously a main thread. I've also written some custom middleware to send request & response data to Keen.io, which I do from a celery worker. BUT, the call to enqueue to the celery worker requires a remote call to my RabbitMQ queueing service, so I do that in ANOTHER thread, so that Django's http response will be sent to client even if the enqueue call hasn't completed by the time the response is rendered.
If you wonder why I do this, let's just say in my experience on highly-scaled systems, you will inevitably see failures or extreme latencies with enqueueing, or really with any over-the-wire call. I don't want these latencies to mess with my response times.
My question is, will gunicorn WAIT for the spawned threads to complete before accepting a subsequent request? Or could those spawned threads potentially "pile up"? This is concerning to me, because I use a database connection pool which would quickly deplete if these threads keep piling up.
I have a boost asio application with many threads, similar to a web server, handling hundreds of concurrent requests. Every request will need to make calls to both memcached and redis (via libmemcached and redispp respectively). Is the best practice in this situation to make a separate connection to both redis and memcached from each thread (effectively tripling the open sockets on the server, three per request)? Or is there a way for me to build a static object, with a single memcached/redis connection, and allow all threads to share that single connection? I'm a bit confused when it comes to the thread safety of something like this, and everything needs to be asynchronous between the threads, but blocking for each thread's individual request (so each thread has a linear progression, but many threads can be in different places in their own progression at any given time). Does that make sense?
Thanks so much!
Since memcached have syncronous protocol you should not write next request before you got answer to prevous. So, no other thread can chat in same memcached connection. I'd prefer to make thread-local connection if you work with it in "blocking" mode.
Or you can make it work in "async" manner: make pool of connections, pick a connection from it (and lock it). After request is done, return it to pool.
Also, you can make a request queue and process it in special thread(s) (using multigets and callbacks).
We have a significantly complex Django application currently served by
apache/mod_wsgi and deployed on multiple AWS EC2 instances behind a
AWS ELB load balancer. Client applications interact with the server
using AJAX. They also periodically poll the server to retrieve notifications
and updates to their state. We wish to remove the polling and replace
it with "push", using web sockets.
Because arbitrary instances handle web socket requests from clients
and hold onto those web sockets, and because we wish to push data to
clients who may not be on the same instance that provides the source
data for the push, we need a way to route data to the appropriate
instance and then from that instance to the appropriate client web
socket.
We realize that apache/mod_wsgi do not play well with web sockets and
plan to replace these components with nginx/gunicorn and use the
gevent-websocket worker. However, if one of several worker processes
receive requests from clients to establish a web socket, and if the
lifetime of worker processes is controlled by the main gunicorn
process, it isn't clear how other worker processes, or in fact
non-gunicorn processes can send data to these web sockets.
A specific case is this one: A user who issues a HTTP request is
directed to one EC2 instance (host) and the desired behavior is that data is
to be sent to another user who has a web socket open in a completely
different instance. One can easily envision a system where a message
broker (e.g. rabbitmq) running on each instance can be sent a message
containing the data to be sent via web sockets to the client connected
to that instance. But how can the handler of these messages access
the web socket, which was received in a worker process of gunicorn?
The high-level python web socket objects created gevent-websocket and
made available to a worker cannot be pickled (they are instance
methods with no support for pickling), so they cannot easily be shared
by a worker process to some long-running, external process.
In fact, the root of this question comes down to how can web sockets
which are initiated by HTTP requests from clients and handled by WSGI
handlers in servers such as gunicorn be accessed by external
processes? It doesn't seem right that gunicorn worker processes,
which are intended to handle HTTP requests would spawn long-running
threads to hang onto web sockets and support handling messages from
other processes to send messages to the web sockets that have been
attached through those worker processes.
Can anyone explain how web sockets and WSGI-based HTTP request
handlers can possibly interplay in the environment I've described?
Thanks.
I think you've made the correct assesment that mod_wsgi + websockets is a nasty combination.
You would find all of your wsgi workers hogged by the web sockets and an attempt to (massively) increase the size of the worker pool would probably choke the server because of the memory usage and context switching.
If you like to stick with the synchronous wsgi worker architecture (as opposed to the reactive approach implemented by gevent, twisted, tornado etc), I would suggest looking into uWSGI as a application server. Recent versions can handle some URLs in the old way (i.e. your existing django views would still work the same as before), and route other urls to a async websocket handler. This might be a relatively smooth migration path for you.
It doesn't seem right that gunicorn worker processes, which are intended to handle HTTP requests would spawn long-running threads to hang onto web sockets and support handling messages from other processes to send messages to the web sockets that have been attached through those worker processes.
Why not? This is a long-running connection, after all. A long-running thread to take care of such a connection would seem... absolutely natural to me.
Often in these evented situations, writing is handled separately from reading.
A worker that is currently handling a websocket connection would wait for relevant message to come down from a messaging server, and then pass that down the websocket.
You can also use gevent's async-friendly Queues to handle in-code message passing, if you like.