I will use uwsgi on AWS EC2 service.
I structured my server's workflow like this.
Internet - ElasticLoadBalancer-a - webserver-nginx - ElasticLoadBalancer-b - web-application-django_uwsgi_server - RDS
So between webserver-nginx and web-application-django_uwsgi_server, There will be communication through http data.
Also you should know that I run web-application-django_uwsgi_server on a docker container. Like, docker run --name django_uwsgi -p 8080:8000 djangoproject_and_uwsgi:1.0
And I set uwsgi ini file like this.
uwsgi.ini in docker container:
[uwsgi]
chdir = /sampledir
wsgi-file = /sampledir/sample/wsgi.py
master = true
processes = 10
# cron = -5 -1 -1 -1 -1 /path/to/some/script
harakiri=20
max-requests=5000
vacuum = true
enable-threads = true
single-interpreter = true
lazy-apps = true
# From this line, I ask these below options.
http = 0.0.0.0:8000
# the socket (use the full path to be safe
# socket = /path/to/your/project/mysite.sock
# chmod-socket = 664
I'm newbie in linux and network, but heard that socket settings will be better than just setting like http = 8000. Because of http overheads.
But I had webserver-nginx on another EC2 instance. In other words, webserver-nginx and web-application-django_uwsgi_server are not on the same EC2 instance(machine).
QUESTION:
In this case, How can I set socket or port settings appropriately?
Related
Hi I am using AWS EC2 instance (c5.18xlarge (72vcpu and 144 Gib ram)) with Nginx + Django and postgresql.
The problem is it gets to 100% CPU utilisation in 400-500 concurrent users and slows down.
Here are my settings:
postgresql.conf
max_connections = 800
shared_buffers = 50000MB
Nginx.conf
user www-data;
worker_processes 128;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
worker_rlimit_nofile 50000;
events {
worker_connections 10000;
#multi_accept on;
}
sysctl.conf
net.core.somaxconn = 65000
net.core.netdev_max_backlog = 65535
fs.file-max = 90000
net.ipv4.ip_forward = 1
kernel.shmmax=1006632960
gunicorn_start.bash
NAME="django_app" # Name of the application
DJANGODIR=/home/ubuntu/project # Django project directory
SOCKFILE=/home/ubuntu/env/run/gunicorn.sock # we will communicte using this unix socket
USER=ubuntu # the user to run as
GROUP=ubuntu # the group to run as
NUM_WORKERS=128 # how many worker processes should Gunicorn spaw
TIMEOUT=120
DJANGO_SETTINGS_MODULE=project.settings # which settings file should Django use
DJANGO_WSGI_MODULE=project.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
I know the server can't overload on the current usage of the app. Please suggest how can this be solved. Or how can it be debugged to reach the problem.
I am also attaching the top screenshot:
INFO:
Framweork: Django 2.X < 3.x ;
Services: supervisord; uWSGI
Host: CentOS Linux 7
Hello, i am currently testing how to deploy multiple django apps with uWSGI on my host. I set everything up based on manuals provided by my Host & uWsgi and it works. However i would like to customized everything a bit further, so that i can understand everything a bit better.
As far as i understand my uWSGI service uwsgi.ini currently works in an emperor mode and provides vassals for my two different app named baron_app.ini and prince_app.ini to handle my different apps.
Question
I noticed that the err.log is a kind of confusing for debugging with multiple apps.
for instance...
announcing my loyalty to the Emperor...
Sat May 2 21:37:58 2020 - [emperor] vassal baron_app.ini is now loyal....
[pid: 26852|app: 0|req: 2/2].....
Question: Is there a way to give my vassals a name so that it will printed in the Log ? Or a way to tell uWSGI to set some kind of process & app log relation (Emperor - Vassals - Worker etc.) in the Log?
For instance i could imagine something like this, could be easier when it comes to find errors.
#baron_app: announcing my loyalty to the Emperor...
#emperor: Sat May 2 21:37:58 2020 - [emperor] vassal baron_app.ini is now loyal....
#prince_app: [pid: 26852|app: 0|req: 2/2].....
i tried something like procname-prefix and vassal_name but it seems not to work - maybe because i don´t know where to put it, in the uwsgi.ini or vassals*.ini?
my current settings...
...< etc < services.d < uwsgi.ini**
[program:uwsgi]
command=uwsgi --master -- %(ENV_HOME)s/uwsgi/apps-enabled
autostart=true
autorestart=true
stderr_logfile = ~/uwsgi/err.log
stdout_logfile = ~/uwsgi/out.log
stopsignal=INT
vacuum = 1
...< uwsgi < apps-enabled < baron_app.ini**
[uwsgi]
base = /home/kiowa/baron_app/baron_app
chdir = /home/kiowa/baron_app/
static_files = /home/kiowa/baron_app/
http = :8080
master = true
wsgi-file = %(base)/wsgi.py
touch-reload = %(wsgi-file)
static-map = /static=%(static_files)/static_storage/production_static
enable-threads = true
single-interpreter = true
app = wsgi
virtualenv = /home/kiowa/.local/env_baron
plugin = python
uid = kiowa
gid = kiowa
...< uwsgi < apps-enabled < baron_app.ini**
[uwsgi]
base = /home/kiowa/prince_app/baron_app
chdir = /home/kiowa/prince_app/
static_files = /home/kiowa/prince_app/
http = :8000
master = true
wsgi-file = %(base)/wsgi.py
touch-reload = %(wsgi-file)
static-map = /static=%(static_files)/static_storage/production_static
enable-threads = true
single-interpreter = true
app = wsgi
virtualenv = /home/kiowa/.local/prince_app
plugin = python
uid = kiowa
gid = kiowa
Ok i was able to seperate my vassals log files by putting this into my vassals.ini
; set app / error Log - check
logger = file:%(var_logs)/vassal_baron/baron_app.log
; disable default req log and set request Log and - check
req-logger = file:%(var_logs)/vassal_baron/baron_request.log
disable-logging = true
I'm getting a little issue with uWSGI and my Django application on production server. I have a FreeBSD jail which has only one Django application. When I made code improvements, I do a touch on settings file in order to take into account modifications.
However, touch kills my uWSGI service each time. So I need to start uWSGI manually else I get a 502 Bad Gateway issue with my browser.
Environment:
Django version : 1.11.20
uWSGI version : 2.0.15
Python version : 3.6.2
uWSGI.ini file:
This is my uwsgi.ini file :
[uwsgi]
pythonpath=/usr/local/www/app/src/web
virtualenv = /usr/local/www/app/venv
module=main.wsgi:application
env = DJANGO_SETTINGS_MODULE=main.settings.prod
env = no_proxy=*.toto.fr
env = LANG=en_US.UTF-8
master=true
processes=2
vaccum=true
chmod-socket=660
chown-socket=www:www
socket=/tmp/uwsgi.sock
socket-timeout = 60
post-buffering = 8192
max-requests = 5000
buffer-size = 32768
offload-threads = 1
uid=www
gid=www
logdate=true
log-maxsize = 20000000
manage-script-name=true
touch-reload = /usr/local/www/app/src/web/main/settings/prod.py
Issue:
When I make a deployment, once it's done, I do :
touch /usr/local/www/app/src/web/main/settings/prod.py
Then I have my uWSGI service out.
This is the last log I have :
Thank you very much !
I am using emperor mode and noticed a couple of uwsgi worker processes keep using CPU.
Here is the ini config for the particular website
[uwsgi]
socket = /tmp/%n.sock
master = true
processes = 2
env = DJANGO_SETTINGS_MODULE=abc.settings
module = django.core.handlers.wsgi:WSGIHandler()
pythonpath = /var/www/abc/abc
chdir = /var/www/abc/abc
chmod-socket = 666
uid = www-data
virtualenv = /var/www/abc
vacuum = true
procname-prefix-spaced = %n
plugins = python
enable-threads = true
single-interpreter = true
sharedarea = 4
htop shows:
13658 www-data 20 0 204M 59168 4148 S 3.0 3.5 3h03:50 abc uWSGI worker 1
13659 www-data 20 0 209M 65092 4428 S 1.0 3.8 3h02:02 abc uWSGI worker 2
I have checked nginx and uwsgi log and both not showing the site is be accessed.
The question is:
why the workers keep using around 1-5% of the CPU when the site is not being accessed.
I think I have found the cause of this, in development, I am using the timer to monitor code changes then reload the uwsgi processes, and I think it's because the project is using django-cms and it's kind of big, so constantly monitoring for code changes every second is kind of heavy, after changing the timer to 5 seconds the processes actually gone quiet.
I want to create uWsgi socket in my project folder and not in /tmp/
Here's my uWSGI config
[uwsgi]
socket = /tmp/uwsgi.sock #I want this in any other folder
#say in /home/me/Desktop/myDjangoApp/
chmod-socket = 666
processes = 1
master = true
vhost = true
no-site = true
But whenever I restart uWSGI with the socket created in my folder, it [fails].
Can't I create the uwsgi.sock in other folder?
I use the following uwsgi config (reduced to relevant parts):
[uwsgi]
uid = moin
gid = www-data
socket = /var/run/moin/uwsgi.sock
hook-as-root = exec:mkdir -vp /var/run/moin/; chown -v moin:www-data /var/run/moin/
Note the hook which creates the /var/run/wiki/ directory with user permissions before the server drops privileges. Call the directory whatever you want, moin was the name of my wiki engine.
On newer distros, /var/run points to a tmpfs location so that any manually created directory gets dropped after a reboot. The hook definition in this file keeps your configuration compact, compared to adding/modifying another init script.
Well, I found an alternative solution for the same. I created the socket in localhost
Here is my uwsgi file
[uwsgi]
uid = www-data
gid = www-data
master = 1
workers = 2
plugins = python
socket = 127.0.0.1:3100
enable-threads = true
processes = 2
pythonpath = <>
wsgi-file = <>
chdir = <>
unix sockets must obey to file permission schemes.
In the second config you're seting uid=www-data and gid=www-data.
The socket file must be writable by www-data and nginx must be able to read/write /tmp/uwsgi.sock
On the other hand if you find that difficult, using host:port (tcp sockets) will work too and you've seem to be able to do it like that.