I was deploying a django app and it failed because for some reason the gunicorn.socket file was not created even though before adding nginx it worked perfectly fine so I searched the internet and found this answer where the guy says that the reason for this is the virtual environment but I'm sure there must be a way around it using venv right?
the log I get from nginx:
connect() to unix:/run/gunicorn.sock failed (11 1: Connection refused) while connecting to upstream,
error from gunicorn:
gunicorn.socket: Failed with result 'service-start-limit-hit'.
I'm 100% sure the problem is with gunicorn not with the setup of nginx becuase I did check for the gunicorn file and it did not exist.
I don't know django, but I am going to assume that this issue is similar to an issue I saw trying to get the MySQL service to start on one of my servers today (see here: https://stackoverflow.com/a/55141733/708323)
Basically, the "start-limit-hit" message is a red herring - service start tries to start which ever service name you provide multiple times, and if after the fifth failed attempt (for me, at least) it crashes out on the sixth attempt with "start-limit-hit". You'll need to investigate the actual syslog (possibly /var/log/syslog) to see what the real errors are that are preventing the service from starting on attempts 1-5.
Check whether you have proper permissions for the user/group mentioned inside the gunicorn.service file.
I had the same issue, my gunicorn file had:
[Service]
User= www-data
Group= www-data
I changed the permission of gunicorn service file, socket file and django directory and it worked fine.
chown www-data:www-data /etc/systemd/system/gunicorn.service
chown www-data:www-data /etc/systemd/system/gunicorn.socket
chown www-data:www-data -R myproject/
Hope, it helps.
Related
I have created a droplet, an ubuntu 18.04 server on digitalocen successfully.
Followed the install digitalocean's guide and ran the gunicorn,nginx, django etc. and ran the website successfully on the browser using my ip address i.e. firewalls, gunicorn and nginx and everything were configured correctly.
During the testing while uploading the images to the media folder I got an error "permissions denied" (POST error).
Found somewhere the below code on google and thought it solves the permissions related problem: https://www.semicolonworld.com/question/55551/django-errno-13-permission-denied-39-var-www-media-animals-user-uploads-39
sudo groupadd varwwwusers
sudo adduser www-data varwwwusers
sudo chgrp -R varwwwusers /var/www/
sudo chmod -R 760 /var/www/
But the above code messed up everything and ran into "502 Bad Gateway".
I am not very familiar with Linux deployments. Can someone help me investigating and resolving this issue. I think I have messed up the users, groups permissions.
How can I get this resolved OR how can I revert what I did OR is there any activity logs I can see and investigate.
I have checked the error.log and able to see
' connect() to unix:/run/gunicorn.sock failed (111: Connection refused) while connecting to upstream'
Thanks for the help in advance.
I guess that the problem is that by running sudo chmod -R 760 /var/www/ you have forbidden non-ownesrs of the folder to read, write and execute files form that directory. Then, since uwsgi processes does not belong to the www-data group (usually), uwsgi connection was refused.
The easiest way to solve this problem is to run sudo chmod -R 766 /var/www/ - this will grant writing and readind priveleges to anyone (chmod -R 764 should also work and is safer - it allows only reading files for non-owners). Alternatively, you may find out the name of the user running uwsgi and grant him permissions for reading files form the directory. See chmod manual (or google it) for reference.
I'm working in a Django project deployment. I'm working in a CentOS 7 server provided ma EC2 (AWS). I have tried to fix this bug by many ways but I cant understand what am I missing.
I'm using ningx and gunicorn to deploy my project. I have created my /etc/systemd/system/myproject.servicefile with the following content:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=centos
Group=nginx
WorkingDirectory=/home/centos/myproject_app
ExecStart=/home/centos/myproject_app/django_env/bin/gunicorn --workers 3 --bind unix:/home/centos/myproject_app/django.sock app.wsgi:application
[Install]
WantedBy=multi-user.target
When I run sudo systemctl restart myproject.serviceand sudo systemctl enable myproject.service, the django.sock file is correctly generated into /home/centos/myproject_app/.
I have created my nginx conf flie in the folder /etc/nginx/sites-available/ with the following content:
server {
listen 80;
server_name my_ip;
charset utf-8;
client_max_body_size 10m;
client_body_buffer_size 128k;
# serve static files
location /static/ {
alias /home/centos/myproject_app/app/static/;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/centos/myproject_app/django.sock;
}
}
After, I restart nginx with the following command:
sudo systemctl restart nginx
If I run the command sudo nginx -t, the reponse is:
nginx: configuration file /etc/nginx/nginx.conf test is successful
When I visit my_ip in a web browser, I'm getting a 502 bad gateway response.
If I check the nginx error log, I see the following message:
1 connect() to unix:/home/centos/myproject_app/django.sock failed (13: Permission denied) while connecting to upstream
I really have tried a lot of solutions changing the sock file permissions. But I cant understand how to fix it. How can I fix this permissions bug?... Thank you so much
If all the permissions under the myproject_app folder are correct, and centos user or nginx group have access to the files, I would say it looks like a Security Enhanced Linux (SELinux) issue.
I had a similar problem, but with RHEL 7. I managed to solve it by executing the following command:
sudo semanage permissive -a httpd_t
It's related to the security policies of SELinux, you have to add the httpd_t to the list of permissive domains.
This post from the NGINX blog may be helpful: NGINX: SELinux Changes when Upgrading to RHEL 6.6 / CentOS 6.6
Motivated by a similar issue, I wrote a tutorial a while ago on How to Deploy a Django Application on RHEL 7. It should be very similar for CentOS 7.
Most probably one of two
1- the directory is not accessible to nginx /home/centos/myproject_app/
$ ls -la /home/centos/myproject_app/
if it is not accessible try to change the path to /etc/nginx
if not then try the command
$ /home/centos/myproject_app/django_env/bin/gunicorn --workers 3 --bind unix:/home/centos/myproject_app/django.sock app.wsgi:application
if still not working then activate the environment and python manage.py runserver 0.0.0.0:8000 go to the browser and go to http://ip:8000 the problem may be here, but it the command of gunicorn worked well, then the problem in directory access for nginx user
Exact same problem here.
Removing Group=www-data fixed the issue for me
I have a wsgi.ini file in my project, and I use uwsgi wsgi.ini to run my project.But when I change the django code,I want to restart the project instead kill uwsgi then reload it. The uwsgi official document provide the following methods:
# using kill to send the signal
kill -HUP `cat /tmp/project-master.pid`
# or the convenience option --reload
uwsgi --reload /tmp/project-master.pid
# or if uwsgi was started with touch-reload=/tmp/somefile
touch /tmp/somefile
But I don't have a project-master.pid file in /tmp catalog in my system(centOS).
my question:
how to use uwsgi restart django instead of kill it then start it?
if use uwsgi official document provided method,how to create a .pid file and what content should in this file?
I find the anwser. project-master.pid is set in wsgi.ini file, you should set pidfile=/tmp/project-master.pid first. Then use uwsgi to start server: uwsgi wsgi.ini.After you start it, you can see a project-master.pid file in /tmp catalog. When you want to reload uwsgi server, you can use such command to restart server: uwsgi --reload /tmp/project-master.pid.
I found simplier answer in my opinion, you can just kill your uwsgi process and then spawn it again:
killall uwsgi
And then just run your uwsgi command again.
You don't need to use uWSGI server for your local development needs. Apache/uWSGI are meant for production, and having them restarted implicitly at every code change is not often desirable. In fact, production server not restarting even after the code is changed often acts as a safety net, so that you don't end up restarting the server without finalising the deployment.
Just use inbuild server django provides with itself.
python manage.py runserver 8000
First of all I am a 100% newbie, so I'm not really even sure what to ask.
I'm trying to build a production environment on Ubuntu 12.04 for Django. I installed Nginx, uWSGI, Python, Django, etc. I followed this guide here: https://www.digitalocean.com/community/tutorials/how-to-serve-django-applications-with-uwsgi-and-nginx-on-ubuntu-14-04.
I'm getting a 502 Bad Gateway error, and when I checked the error.log file I found this at the end:
2015/07/22 17:43:18 [crit] 23919#0: *3 connect() to unix:/home/ubuntu/lsb/lsb.sock failed (2: No such file or directory) while connecting to upstream, client: 180.152.149.63, server: 115.159.36.162, request: "GET / HTTP/1.1", upstream: "uwsgi://unix:/home/ubuntu/lsb/lsb.sock:", host: "115.159.36.162:8000"
According to the guide, I created all the relevant files but I only have 1 project, and replaced all instance of user with ubuntu, and firstproject with lsb.
For the server block configuration file, I just used the server's IP address and port 8000. The one that's in /etc/nginx/sites-available.
server {
listen 8000;
server_name 115.159.36.162;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/lsb;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/home/ubuntu/lsb/lsb.sock;
}
}
Besides this I pretty much followed the guide to the letter. I found a guy who said it was due to the problem of permissions with uwsgi_params, I used sudo chown ubuntu uwsgi_params but that didn't fix it either (I'm not even sure if that's the right command to use...).
In any case I don't even know what I don't know here. ANY suggestions would be greatly appreciated, thanks!!
UPDATE after Daniel Roseman's suggestions
I indeed never started uwsgi, but when I tried to do sudo service uwsgi start, I get an error saying uwsgi: unrecognized service. I then went back up the guide and saw that they suggested I run a uwsgi command right after I install it to test it:
uwsgi --http :8080 --home /home/user/Env/firstsite --chdir /home/user/firstsite -w firstsite.wsgi
This command returned another error,
Fatal Python error: Py_Initialize: Unable to get the locale encoding ImportError: No module named 'encodings'
Some websites say that this has to do with uwsgi not being able to find Python. I installed Python3.4 before, it's accessible when I type python3.4 anywhere in the shell. Besides that I think there's also 2.7 installed and maybe 3.2 as well. Not really sure if this is relevant, and not sure how to help uwsgi "find" Python, or if that's even the problem.
Looking back, here's a list of things that I did differently from the guide (contradicting my previous "to the letter" claim) that might be relevant to the problem.
I installed python3.4, from this Felix Krull depository, mainly because when I tried to do the normal sudo apt-get install python3 it gives me an error, so after I added the depository I did sudo apt-get install python3.4.
The pip I used isn't the pip3 that all these sites keep saying, since, again, I can't seem to install pip3 at all. So what I ended up doing is I got this get-pip.py from pip's website, and executed it using python3.4.
On top of that, in the guide that says install python-dev, sudo apt-get install python-dev, I did instead sudo apt-get install python3.4-dev. This is something the guide says uwsgi needs, so I'm not sure if this is screwing things up.
Once again thanks for all the help Daniel!
I can this setup to work if I start gunicorn manually or if I add gunicorn to my django installed apps. But when I try to start gunicorn with systemd the gunicorn socket and service start fine but they don't serve anything to Nginx; I get a 502 bad gateway.
Nginx is running under the "http" user/group, chroot jail. I used pythonbrew to setup the virtualenvs so gunicorn is installed in my home directory under .pythonbrew. The vitualenv directory is owned by my user and the adm group.
I'm pretty sure there is a permission issue somewhere, because everything works if I start gunicorn but not if systemd starts it. I've tried changing the user and group directives inside the gunicorn.service file, but nothing worked; if root start the server then I get no errors and a 502, if my user starts it I get no errors and 504.
I have checked the Nginx logs and there are no errors, so I'm sure it's a gunicorn issue. Should I have the virtualenv in the app directory? Who should be the owner of the app directory? How can I narrow down the issue?
/usr/lib/systemd/system/gunicorn-app.service
#!/bin/sh
[Unit]
Description=gunicorn-app
[Service]
ExecStart=/home/noel/.pythonbrew/venvs/Python-3.3.0/nlp/bin/gunicorn_django
User=http
Group=http
Restart=always
WorkingDirectory = /home/noel/.pythonbrew/venvs/Python-3.3.0/nlp/bin
[Install]
WantedBy=multi-user.target
/usr/lib/systemd/system/gunicorn-app.socket
[Unit]
Description=gunicorn-app socket
[Socket]
ListenStream=/run/unicorn.sock
ListenStream=0.0.0.0:9000
ListenStream=[::]:8000
[Install]
WantedBy=sockets.target
I realize this is kind of a sprawling question, but I'm sure I can pinpoint the issue with a few pointers. Thanks.
Update
I'm starting to narrow this down. When I run gunicorn manually and then run ps aux|grep gunicorn then I see two processes that are started: master and worker. But when I start gunicorn with systemd there is only one process started. I tried adding Type=forking to my gunicorn.services file, but then I get an error when loading service. I thought that maybe gunicorn wasn't running under the virtualenv or the venv isn't getting activated?
Does anyone know what I'm doing wrong here? Maybe gunicorn isn't running in the venv?
I had a similar problem on OSX with launchd.
The issue was I needed to allow for the process to spawn sub processes.
Try adding Type=forking:
[Unit]
Description=gunicorn-app
[Service]
Type=forking
I know this isn't the best way, but I was able to get it working by adding gunicorn to the list of django INSTALLED_APPS. Then I just created a new systemd service:
[Unit]
Description=hack way to start gunicorn and django
[Service]
User=http
Group=http
ExecStart=/srv/http/www/nlp.com/nlp/bin/python /srv/http/www/nlp.com/nlp/nlp/manage.py run_gunicorn
Restart=always
[Install]
WantedBy=multi-user.target
There must be a better way, but judging by the lack of responses not many people know what that better way is.