I am currently running a web server through two containers:
NGINX Container: Serves HTTPS requests and redirects HTTP to HTTPS. HTTPS requests are passed through uwsgi to the django app.
Django Container: Runs the necessary Django code.
When running docker-compose up --build, everything compiles correctly until uWSGI raises a Segmentation Fault.
....
django3_1 | Python main interpreter initialized at 0x7fd7bce0d190
django3_1 | python threads support enabled
django3_1 | your server socket listen backlog is limited to 100 connections
django3_1 | your mercy for graceful operations on workers is 60 seconds
django3_1 | mapped 145840 bytes (142 KB) for 1 cores
django3_1 | *** Operational MODE: single process ***
django3_1 | !!! uWSGI process 7 got Segmentation Fault !!!
test_django3_1 exited with code 1
Would appreciate if there's any advice, as I'm not able to see into the container for debugging purposes when it is starting up, therefore I don't know where this segmentation fault is occurring.
The SSL certificates have been correctly set up.
The django3 container was running on a python:3.9-alpine image. This installs Python 3.9.2 on the container. There appear to be some issue between uWSGI and Python dependencies on this version. Rolling the container back to python:3.8-alpine resolved the dependency version mismatch.
Related
I use a Daphne server for my ASGI Django application. When I run htop on my Ubuntu server it shows all the load on only one processor and the app gets slow, so I'd like to know what is the easiest way to speed up my server. App is inside a Docker container.
I run Daphne with command: daphne WBT.asgi:application --bind "0.0.0.0" --port "8000".
I use Nginx as a proxy.
I tried using uvicorn server with multiple workers but then I get problems with layout. Ie. let's say that I have 4 workers when I visit my web app, it loads it on only one worker correctly, so in 25% cases I get no layout.
The reason I asked this question is that every other question regarding the subject that I have seen on Stack Overflow is more complicated than mine. I am not trying to get uwsgi to work with anything else. I am simply trying to get the most basic implementation of it to actually do something.
I am following these instructions:
https://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html
My domain is called "prosaurus.com" and It's registered on Godaddy. I have hosting through Digital Ocean which has provided me with the following IP address: 143.198.142.59
In Digital Ocean I have set up two A records to point to the associated droplet/IP address. One is for "prosuaurus.com" the other is for "*.prosuarus.com"
I have followed the instructions exactly, opting for PIP to install uwsgi (as opposed to using wget and make).
I have created a file called 'foobar.py' as suggested and entered the suggested contents.
I have started the server as suggested on port 9090. This is the output I get:
uwsgi --http :9090 --wsgi-file foobar.py
*** Starting uWSGI 2.0.20 (64bit) on [Wed Sep 7 16:45:00 2022] ***
compiled with version: 11.2.0 on 02 September 2022 19:04:11
os: Linux-5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022
nodename: ubuntu-s-1vcpu-1gb-intel-sfo3-01
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/deployer
detected binary path: /home/deployer/.local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 3721
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :9090 fd 4
spawned uWSGI http 1 (pid: 59211)
uwsgi socket 0 bound to TCP address 127.0.0.1:46629 (port auto-assigned) fd 3
Python version: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x55b300c4fb30
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72920 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55b300c4fb30 pid: 59210 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 59210, cores: 1)
oba
Everything appears to have worked correctly. I saw no error messages in any of the above steps (There was one warning suggesting I should use a virtual environment but this was just a warning right?)
yet, I do not see my hello world page. I don't see it on http://prosaurus.com:9090 or http://www.prosaurus.com:9090 or http://143.198.142.59:9090
What could I have possibly missed?
Side note: This server does have nginx installed on it, however I specifically turned it off before this test.
Firewall...
sudo ufw allow 9090
It would have been nice if they had mentioned this possibility in the docs.
I am working on a flask-socketio server which is getting stuck in a state where only 504s (gateway timeout) are returned. We are using AWS ELB in front of the server. I was wondering if anyone wouldn't mind giving some tips as to how to debug this issue.
Other symptoms:
This problem does not occur consistently, but once it begins happening, only 504s are received from requests. Restarting the process seems to fix the issue.
When I run netstat -nt on the server, I see many entries with rec-q's of over 100 stuck in the CLOSE_WAIT state
When I run strace on the process, I only see select and clock_gettime
When I run tcpdump on the server, I can see the valid requests coming into the server
AWS health checks are coming back succesfully
EDIT:
I should also add two things:
flask-socketio's server is used for production (not gunicorn or uWSGI)
Python's daemonize function is used for daemonizing the app
It seemed that switching to gunicorn as the wsgi server fixed the problem. This legitimately might be an issue with the flask-socketio wsgi server.
I have a problem with uwsgi everytime I restart the server when I have a code updates.
When I restart the uwsgi using "sudo restart accounting", there's a small gap between stop and start instance that results to downtime and stops all the current request.
When I try "sudo reload accounting", it works but my memory goes up (double). When I run the command "ps aux | grep accounting", it shows that I have 10 running processes (accounting.ini) instead of 5 and it freezes up my server when the memory hits the limit.
accounting.ini
I am running
Ubuntu 14.04
Django 1.9
nginx 1.4.6
uwsgi 2.0.12
This is how uwsgi does graceful reload. Keeps old processes until requests are served and creates new ones that will take over incoming requests.
Read Things that could go wrong
Do not forget, your workers/threads that are still running requests
could block the reload (for various reasons) for more seconds than
your proxy server could tolerate.
And this
Another important step of graceful reload is to avoid destroying
workers/threads that are still managing requests. Obviously requests
could be stuck, so you should have a timeout for running workers (in
uWSGI it is called the “worker’s mercy” and it has a default value of
60 seconds).
So i would recommend trying worker-reload-mercy
Default value is to wait 60 seconds, just lower it to something that your server can handle.
Tell me if it worked.
Uwsgi chain reload
This is another try to fix your issue. As you mentioned your uwsgi workers are restarting in a manner described below:
send SIGHUP signal to the master
Wait for running workers.
Close all of the file descriptors except the ones mapped to sockets.
Call exec() on itself.
One of the cons of this kind of reload might be stuck workers.
Additionaly you report that your server crashes when uwsgi maintains 10 proceses (5 old and 5 new ones).
I propose trying chain reload. DIrect quote from documentation explains this kind of reload best:
When triggered, it will restart one worker at time, and the following worker is not reloaded until the previous one is ready to accept new requests.
It means that you will not have 10 processes on your server but only 5.
Config that should work:
# your .ini file
lazy-apps = true
touch-chain-reload = /path/to/reloadFile
Some resources on chain reload and other kinds are in links below:
Chain reloading uwsgi docs
uWSGI graceful Python code deploy
I have a django app which I communicate with, using a python script.
Sometime (about 1/500) I get strange errors.
https <-> nginx <-> uwsgi <-> django
Rare errors in the python script:
http.client.BadStatusLine: ''
urllib.error.URLError:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
I cannot see any trace of those in either:
appLog
uwsgi log
nginx log
How can I begin to trace this issue?