uWSGI is not creating Unix socket at specified path - django

I've worked with Nginx and uWSGI numerous times, but I've never seen socket not being created even when specified explicitly.
My uWSGI config (/etc/uwsgi/sites/my_site.ini) is quite simple:
[uwsgi]
project = my_site
base = /root
chdir = %(base)/%(project)
home = %(base)/Env/%(project)
module = %(project).wsgi:application
master = true
processes = 5
socket = /run/uwsgi/%(project).sock
chmod-socket = 664
vacuum = true
As visible, uWSGI must create socket with 644 permissions (although some suggest it should be 666).
Nginx server configuration (/etc/nginx/nginx.conf) is quite simple as well:
server {
listen 80;
server_name mysite.com www.mysite.com;
location /static/ {
root /root/my_site;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/run/uwsgi/my_site.sock;
}
}
uwgsi.service (startup script) (/etc/systemd/system/uwsgi.service on CentOS):
[Unit]
Description=uWSGI Emperor service
After=syslog.target
[Service]
ExecStart=/usr/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all
[Install]
WantedBy=multi-user.target
After making these configurations, I executed following commands:
sudo systemctl daemon-reload
sudo systemctl restart uwsgi
sudo systemctl restart nginx
sudo systemctl status uwsgi
sudo systemctl status nginx
Everything was fine according to commands above, but Nginx returned 502 bad gateway) error when visiting website.
Thus I checked /var/log/nginx/error.log/:
2018/08/25 19:33:10 [crit] 14920#0: *3 connect() to unix:/run/uwsgi/my_site.sock failed (2: No such file or directory) while connecting to upstream, client: xx.xxx.xxx.xx, server: my_site.com, request: "GET / HTTP/1.1", upstream: $
As you can see above, uWSGI doesn't create socket at /run/uwsgi/ directory for some reason...
I'm sure that socket is not being created at /run/uwsgi/, since ls /run/uwsgi/ returns nothing.
Permissions:
sudo usermod -a -G root nginx;chmod 710 /root
Main problem:
uWSGI is not creating a unix socket at the specified path in the configuration above, what could be the problem?
Is there any possible cause?

Related

Not connecting to gunicorn, Gunicorn Exited too quickly

I have a django project named MySite, and this project has some application inside as below:
- MySite
-- app
-- venv
-- media
-- django_project
--- wsgi.py
--- settings.py
--- urls.py
--- asgi.py
To deploy on aws, I am in the phase of gunicorn configuring. However I face with this error:
guni:gunicorn BACKOFF Exited too quickly (process log may have details)
However, first time status is like:
gunicorn STARTING
this is my gunicorn.conf:
[program:gunicorn]
directory=/home/ubuntu/MySite
command=/usr/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/MySite/app.sock django_project.wsgi.application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
program:gunicorn
in gunicorn.err.log it says problem is in:
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: django_project.wsgi.application
when I try this:
gunicorn --bind 0.0.0.0:8000 django_project.wsgi:application
I get this error:
SyntaxError: invalid syntax
[2021-02-10 10:12:40 +0000] [6914] [INFO] Worker exiting (pid: 6914)
[2021-02-10 10:12:40 +0000] [6912] [INFO] Shutting down: Master
[2021-02-10 10:12:40 +0000] [6912] [INFO] Reason: Worker failed to boot.
The entire process to install and run gunicorn which I did:
**********************************START********************************
sudo apt-get upgrade -y
sudo apt-get update -y
1) Clone the git project
git clone https://github.com/XX/MyProj.git
2) cd /MySite ## there is a venv with django installed in
3) Activate venv
source venv/bin/activate
5. Instal NGINX and GUNICORN
pip3 install gunicorn ## install without sudo..
sudo apt-get install nginx -y
pip install psycopg2-binary
6. Connect gunicorn (#Error: Worker failed to boot.)
gunicorn --bind 0.0.0.0:8000 django_project.wsgi:application
7. Install supervisor
sudo apt-get install -y supervisor ## This command holds the website after we logout
8. Config supervisor
cd /etc/supervisor/conf.d
sudo touch gunicorn.conf
9) ##In the file file do following###
[program:gunicorn]
directory=/home/ubuntu/MySite
command=/usr/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/MySite/app.sock django_project.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
Program:gunicorn
####endfile####
10). Config supervisor
cd /etc/supervisor/conf.d
sudo touch gunicorn.conf
11). Connect file to supervisor
sudo mkdir -p /var/log/gunicorn
sudo supervisorctl reread
sudo supervisorctl reread
sudo supervisorctl update
12. Check if gunicorn is running in background
sudo supervisorctl status
**********************************END********************************
Your issue could be a combination of a few things:
Verify you django settings.py is set up properly for deployment. Pay very close attention to the STATIC_ROOT and STATICFILES_DIR variables as they are critical to serving your projects static files.
Make sure your virtualenv is activated and you have ran pip install -r requirements.txt.
Note: At this point try to run your project with your servers public ip python manage.py runserver server_public_ip:8000 a lot of people assume that, because their project ran locally, it will run on the server. Something always goes wrong.
Make sure you run python manage.py collectstatic on your server. This collects your static files and makes a directory for it. Take note of the path it tells you it's going to copy them to, you're going to need it for your /static/ location block in your nginx sites-available configuration file.
Make sure your gunicorn.conf command variable points to your virtualenv path, and that it points to a .sock file that gunicorn and nginx can access (sudo chown user:group). Here is an example:
[program:gunicorn]
command=/home/user/.virtualenvs/djangoproject/bin/gunicorn -w3 --bind unix:/etc/supervisor/socks/gunicorn.sock djangoproject.wsgi:application --log-level=info
directory=/home/user/djangoproject
numprocs=3
process_name=%(program_name)s_%(process_num)d
user=user
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8,HOME="/home/user/djangoproject", USER="user"
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/gunicorn.err.log
stdout_logfile=/var/log/supervisor/gunicorn.out.log
There are a couple ways you can set up your nginx configuration files. Some docs have you do it all in the nginx.conf file, however, you should break it up into sites-available with a symbolic link to sites-enabled. Either way should get you the same result. Here is an example:
upstream django {
server unix:/etc/supervisor/socks/gunicorn.sock;
}
server {
listen 80;
server_name www.example.com;
client_max_body_size 4G;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_headers_hash_max_size 1024;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;
location /media/ {
autoindex off;
alias /home/user/djangoproject/media/;
}
location /static/ {
autoindex off;
alias /home/user/djangoproject/static/;
}
location / {
proxy_pass http://django;
proxy_redirect off;
}
}
Make sure that you run sudo service nginx restart after every nginx configuration change.
Make sure you run sudo supervisorctl reread, sudo supervisorctl update all, and sudo supervisorctl restart all after every supervisor configuration file change. In your case every gunicorn.conf file change.
Lastly, as a general rule, make sure all of your directory paths match to their respected processes. For example: Let's say your gunicorn command variable points to a sock file that is in /path/to/project/gunicorn.sock, but your nginx configuration has a proxy_pass to /etc/supervisor/socks/gunicorn.sock, nginx doesn't know where to pass your requests to, so gunicorn never sees the request. Also, you can add this to your nginx location blocks so you can see where each request gets to in your browser dev tools response header: add_header X-debug-message "The port 80, / location was served from django" always;
Note: If you are getting the "Welcome to Nginx" page, it means nginx doesn't know where to send the request. A lot of times you have a static root directory path problem. However, there are other issues, but situational to how things are set up. You'll have to debug with some trial and error. Also, try adding a location block to a known url like http://example.com/login, if you get there, you know you have an nginx configuration issue. If you get 404 not found, then you most likely have a django project problem or a gunicorn problem.
This guide should work just fine :-)
Install dependencies
[user#machine]$ sudo apt update && sudo apt upgrade -y
[user#machine]$ sudo apt install python3-dev python3-pip supervisor nginx -y
[user#machine]$ sudo systemctl enable nginx.service
[user#machine]$ sudo systemctl restart nginx.service
[user#machine]$ sudo systemctl status nginx.service
Create new site
/etc/nginx/sites-available/<site-name>
server{
listen 80;
server_name <ip or domain>;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /static/ {
root /path/to/static/root;
}
# main django application
location / {
include proxy_params;
proxy_pass http://unix:/path/to/project/root/run.sock;
}
}
Create symbolic link
[user#machine]$ ln -s /etc/nginx/sites-available/<site-name> /etc/nginx/sites-enabled
Setup supervisor
/etc/supervisor/conf.d/<project-name>.conf
[program:web]
directory=/path/to/project/root
user=<user>
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/path/to/logs/gunicorn-error.log
command=/path/to/gunicorn --access-logfile - --workers 3 --bind unix:/path/to/project/root/run.sock <project-name>.wsgi:application
Then
[user#machine]$ sudo supervisorctl reread
[user#machine]$ sudo supervisorctl update
[user#machine]$ sudo supervisorctl status web
[user#machine]$ sudo systemctl restart nginx.service
The error tells you that the way you start gunicorn is wrong.
It looks like you are using supervisor. As per the docs, the correct syntax is:
[program:gunicorn]
command=/path/to/gunicorn main:application -c /path/to/gunicorn.conf.py
directory=/path/to/project
user=nobody
autostart=true
autorestart=true
redirect_stderr=true

nginx fails while the website somehow still works

Description
I have built my Django3, gunicorn, nginx, Ubuntu 18.04, Digital Ocean project based on this guide. I only had 1 problem that it does not shows the CSS and all other static files like images. Before during the whole guide nginx have given the correct answers as the guide says and also currently the html site still online and running
To solve this I was in the process of using this another guide to make my static files displayed on my site.
I have done all the steps what the creator recommended but at the
What I have tried
After each step of the following [1.2.3...] commands I have executed the following commands to refresh:
python3 manage.py collectstatic
sudo nginx -t && sudo systemctl restart nginx
sudo systemctl restart gunicorn
1.RUN: sudo ln -s /etc/nginx/sites-available/ch-project /etc/nginx/sites-enabled
1.RESULT: ln: failed to create symbolic link '/etc/nginx/sites-enabled/ch-project': File exists
2.RUN: /etc/nginx/sites-enabled/my-project
2.RESULT: -bash: /etc/nginx/sites-enabled/my-project: Permission denied
3.RUN: systemctl status nginx.service
3.RESULT:
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2020-03-26 13:27:08 UTC; 13s ago
Docs: man:nginx(8)
Process: 11111 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
Process: 11111 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=1/FAILURE)
Main PID: 11111 (code=exited, status=0/SUCCESS)
4.RUN: sudo nginx -t
4.RESULT:
nginx: [emerg] open() "/etc/nginx/sites-enabled/myproject" failed (2: No such file or directory) in /etc/nginx/nginx.conf:62
nginx: configuration file /etc/nginx/nginx.conf test failed
Nginex should be Ok otherwise because the html on the website loads and works perfectly. This stack overflow post says I should maybe do something with the security of the nginx.conf but in that case they talk about a worldpres site so I don't know how to implement that here.
I have tried this stack overflow post's answer previously, bellow the answer it has a subpost to further configure RUN: sudo nginx -c /etc/nginx/sites-enabled/default -t
6.RESULT:
nginx: [emerg] "server" directive is not allowed here in /etc/nginx/sites-enabled/default:21
nginx: configuration file /etc/nginx/sites-enabled/default test failed
The static files still don't load but nginx has been fixed with the following commands.
deleting accidentally created myproject file from /etc/nginx/sites-enabled/myproject what file name is in the official guide but the_actual_myproject has different name RUN:
cd /etc/nginx/sites-enabled
sudo rm myproject
RUN:
namei -l /run/gunicorn.sock
sudo systemctl restart gunicorn
sudo systemctl daemon-reload
sudo systemctl restart gunicorn.socket gunicorn.service
sudo nginx -t && sudo systemctl restart nginx
RESULT:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
RUN: systemctl status nginx.service
RESULT:
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-03-26 16:32:09 UTC; 7min ago
Docs: man:nginx(8)
Process: 11111 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, sta
Process: 11111 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=ex
Main PID: 11111 (nginx)
Tasks: 2 (limit: 1152)
CGroup: /system.slice/nginx.service

restarting gunicorn and nginx doesn't reflect changes after pull request

Recently I hosted a Django website on digitalocean. I then edited the content of website locally and pushed them to GitHub then pulled it to server and restarted nginx and gunicorn:
sudo systemctl restart nginx
sudo systemctl restart gunicorn
Then changes didn't reflect back to live website, Just to check if I setup gnuicorn and nginx properly, I ran sudo systemctl status gunicorn.socket I get Failed to dump process list, ignoring: No such file or directory
(venv) leptitoxadmin#ubuntu1:~/pyapps/leptx$ sudo systemctl status gunicorn.socket
[sudo] password for leptxadmin:
Failed to dump process list, ignoring: No such file or directory
● gunicorn.socket - gunicorn socket
Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; vendor preset: enabled)
Active: active (running) since Wed 2020-02-12 05:57:58 UTC; 2h 42min ago
Listen: /run/gunicorn.sock (Stream)
CGroup: /system.slice/gunicorn.socket
Feb 12 05:57:58 ubuntu1 systemd[1]: Listening on gunicorn socket.
is this the reason, why the changes don't reflect to live website? How do I solve it? thanks
I think you should also consider reloading daemon as well
sudo systemctl daemon-reload
and then
sudo systemctl restart gunicorn
i have same problem and now i solved!!
./manage.py collectstatic
sudo systemctl restart gunicorn
that code can solve problem!

Python Django Ubuntu Server 16.04 aws Internal Server Error

Hi so I am trying to deploy my portfolio to Ubuntu Server 16.04 but I keep getting internal server error.
To walk through what i have done i created the instance and changed the HTTP and HTTPs security settings to anywhere.
After that i launched the instance then running these commands
ubuntu#ip-172-31-41-27:~ sudo apt-get update
ubuntu#ip-172-31-41-27:~$ sudo apt-get install python-pip python-dev
nginx git
ubuntu#ip-172-31-41-27:~$ sudo apt-get update
ubuntu#ip-172-31-41-27:~$ sudo pip install virtualenv
ubuntu#ip-172-31-41-27:~/portfolio$ virtualenv venv --python=python3
ubuntu#ip-172-31-41-27:~/portfolio$ source venv/bin/activate
(venv)ubuntu#ip-172-31-41-27:~/portfolio$ pip install -r
requirements.txt
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ pip install django bcrypt
django-extensions
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ pip install gunicorn
i edited the settings.py
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
ALLOWED_HOSTS = ['52.14.89.55']
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
then run,
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ python manage.py
collectstatic
followed by,
ubuntu#ip-172-31-41-27:~/portfolio$ sudo vim
/etc/systemd/system/gunicorn.service
where I add
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/portfolio
ExecStart=/home/ubuntu/portfolio/venv/bin/gunicorn --workers 3 --bind
unix:/home/ubuntu/portfolio/portfolio.sock portfolio.wsgi:application
[Install]
WantedBy=multi-user.target
followed by a gunicorn reboot
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl daemon-reload
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl start gunicorn
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl enable gunicorn
finally,
ubuntu#54.162.31.253:~$ sudo vim /etc/nginx/sites-
available/portfolio
adding
server {
listen 80;
server_name 52.14.89.55;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/portfolio;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/portfolio/portfolio.sock;
}
}
creating the link
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo ln -s
/etc/nginx/sites-available/portfolio /etc/nginx/sites-enabled
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
deleting the default
then restarting
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo service nginx
restart
EDIT**
I changed the following codes to match exactly what I have done.

How socket file is created in Gunicorn

I am deploying a Django project using Gunicorn as application server and Nginx as a web server. This is the tutorial http://tutos.readthedocs.io/en/latest/source/ndg.html which I am following. Code for the gunicorn_start.sh (kept in the folder where the project resides).
#!/bin/bash
NAME="visualization"
DJANGODIR=/var/www/dist/adc # Django project directory*
SOCKFILE=/var/www/dist/run/gunicorn.sock
USER=barun
GROUP=barun
NUM_WORKERS=1
DJANGO_SETTINGS_MODULE=adc.settings
DJANGO_WSGI_MODULE=adc.wsgi
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/barun/anaconda3/envs/adcenv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
exec /home/barun/anaconda3/envs/adcenv/bin/gunicorn
${DJANGO_WSGI_MODULE}:application \ #we should put the address
gunicorn address
--name $NAME \
--workers $NUM_WORKERS \
--user $USER \
--bind=unix:$SOCKFILE
Code for gunicorn.service (/lib/systemd/system/gunicorn.service):
[Unit]
Description=visualization gunicorn daemon
[Service]
Type=simple
User=barun
WorkingDirectory=/var/www/dist
ExecStart=/home/barun/anaconda3/envs/adcenv/bin/gunicorn --workers 3 --
bind unix:/var/www/dist/run/gunicorn.sock adc.wsgi:application
[Install]
WantedBy=multi-user.target
Everything is working fine. The location is given in gunicorn_start.sh file for creating a socket file, the run folder is created but gunicorn.sock file is not creating. There is not permission related problem
Apart from this, The error I am getting in Nginx error.log file is:
4783#4783: *1 connect() to unix:/var/www/dist/run/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 172.16.1.213, server: 192.16.1.213, request: "GET / HTTP/1.1", upstream: "http://unix:/var/www/dist/run/gunicorn.sock:/", host: "172.16.1.213:8080"
When I am executing the ./gunicorn_start.sh the socket file should be created but it is not happening.
Solved!
I have done everything from fresh. Uninstall and again install the Gunicorn and Nginx. delete all configuration file and write it again.
Now it's working well.