Can't run uwsgi .ini file with systemd emperor - django

I am trying to set up uwsgi.service to run on systemd for Django 1.10 on Linode with Fedora 24.
/etc/systemd/system/uwsgi.service
[Unit]
Description=uWSGI Emperor
After=syslog.target
[Service]
ExecStart=/home/ofey/djangoenv/bin/uwsgi --ini /etc/uwsgi/emperor.ini
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all
[Install]
WantedBy=multi-user.target
This should then call,
/etc/uwsgi/emporer.ini
[uwsgi]
emperor = /etc/uwsgi/vassals
uid = www-data
gid = www-data
limit-as = 1024
logto = /tmp/uwsgi.log
I then use a symbolic link,
$ sudo ln -s /home/ofey/djangoForum/django.ini /etc/uwsgi/vassals/
to
/home/ofey/djangoForum/django.ini
[uwsgi]
project = djangoForum
base = /home/ofey
chdir = %(base)/%(project)
home = %(base)/djangoenv
module = crudProject.wsgi:application
master = true
processes = 2
socket = 127.0.0.1:3031
chmod-socket = 664
vacuum = true
I have restarted all with,
$ sudo systemctl daemon-reload
$ sudo systemctl restart nginx.service
$ sudo systemctl retart uwsgi.service
The last command gives,
Job for uwsgi.service failed because the control process exited with error code. See "systemctl status uwsgi.service" and "journalctl -xe" for details.
$ sudo systemctl status uwsgi.service
gives,
● uwsgi.service - uWSGI Emperor
Loaded: loaded (/etc/systemd/system/uwsgi.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Dec 07 23:56:28 ofeyspi systemd[1]: Starting uWSGI Emperor...
Dec 07 23:56:28 ofeyspi uwsgi[7834]: [uWSGI] getting INI configuration from /etc/uwsgi/emperor.ini
Dec 07 23:56:28 ofeyspi systemd[1]: uwsgi.service: Main process exited, code=exited, status=1/FAILURE
Dec 07 23:56:28 ofeyspi systemd[1]: Failed to start uWSGI Emperor.
Dec 07 23:56:28 ofeyspi systemd[1]: uwsgi.service: Unit entered failed state.
Dec 07 23:56:28 ofeyspi systemd[1]: uwsgi.service: Failed with result 'exit-code'.
Dec 07 23:56:28 ofeyspi systemd[1]: uwsgi.service: Service hold-off time over, scheduling restart.
Dec 07 23:56:28 ofeyspi systemd[1]: Stopped uWSGI Emperor.
Dec 07 23:56:28 ofeyspi systemd[1]: uwsgi.service: Start request repeated too quickly.
Dec 07 23:56:28 ofeyspi systemd[1]: Failed to start uWSGI Emperor.
I can not figure out why uwsgi.service will not run.
uwsgi runs when I don't go through systemd and instead use,
$ sudo --ini django.ini

The most likely reason for that is that Emperor is not able to:
create the .pid file to write the processID;
create the unix-socket files for the vassals (seems to be not your case, since you are using :3031 port);
write to the log file specified by --logto option (/tmp/uwsgi.log in your case).
This often happens when any of these files exist and are owned by another user (most likely, root) or are located in a directory to which the user starting the service is not able to write.
Systemd's status log is not very informative on this subject. So, the quickest way to identify the case is to run your ExecStart command from systemd's service not as root and to see the output:
$ /home/ofey/djangoenv/bin/uwsgi --ini /etc/uwsgi/emperor.ini
If the output shows that the problem is permission, try the following.
Since you are going to run your server on behalf of www-data user, as it has been suggested already, make sure you have:
[Service]
User=www-data
Group=www-data
RuntimeDirectory=uwsgi
in your systemd unit config. Then, make sure the .pid files and unix-socket files (if any) are created under that directory (i.e. under /run/uwsgi) by adding this to your vassals .ini files:
runtime_dir = /run/uwsgi
pidfile=%(runtime_dir)/%n.pid
# if you prefer using unix-socket instead of a port
socket = %(runtime_dir)/%n.sock
# trying to chmod-socket is useless with a port, by the way
chmod-socket = 664
The %n variable in the given example stands for the vassal's .ini file name without extension (see the full-list here).
Finally, make sure the --logto file specified in Emperor's and vassals' configs is writable by such.
Please note, if you run uwsgi --ini /etc/uwsgi/emperor.ini as root and then terminate the procecc with ctrl+D, it will leave the above-mentioned temp files existing and owned by root, which will prevent other owners (like www-data) from writing to them, until you delete the files or chown/chmod them again.

The uwsgi systemd docs advise adding RuntimeDirectory=uwsgi to your service file. Try adding that.
Also check /tmp/uwsgi.log to see if any logging was generated there.

Comment out KillSignal = SIGQUIT
It can cause problems, see http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
also causes issues on Centos 7

Related

Permission Denied when attempting to start Daphne systemctl process

I'm deploying a website using Django and Django-Channels, with Channel's daphne ASGI server substituting for the typical Gunicorn WSGI setup. Using this Gunicorn WSGI tutorial as a jumping off guide, I attempted to write a systemctl service for my daphne server, when I hit the below error:
CRITICAL Listen failure: [Errno 13] Permission denied: '27646' -> b'/run/daphne.sock.lock'
I was unfortunately unable to find any answers to why permissions would be denied to the .sock file, (in context to Daphne) so I was hoping I could get some hints on where to begin debugging this problem. Below are my daphne.socket and my daphne.service files.
daphne.service
[Unit]
Description=daphne daemon
Requires=daphne.socket
After=network.target
[Service]
User=brianl
Group=www-data
WorkingDirectory=/home/brianl/autoXMD
ExecStart=/home/brianl/autoXMD/env/bin/daphne -u /run/daphne.sock -b 0.0.0.0 -p 8000 autoXMD.asgi:application
[Install]
WantedBy=multi-user.target
daphne.socket
[Unit]
Description=daphne socket
[Socket]
ListenStream=/run/daphne.sock
[Install]
WantedBy=sockets.target
Based off the linked DigitalOcean tutorial, I start my service with sudo systemctl start daphne.socket.
My guess is that there's some kind of discrepancy between setting up systemctl services for Gunicorn and Daphne that I missed, but I don't know for sure.
(If it helps, I'm planning on using Nginx as the main server, but I haven't reached that point yet)
EDIT:
It would help if I also attached the full output systemd gives:
● daphne.service - daphne daemon
Loaded: loaded (/etc/systemd/system/daphne.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Thu 2019-09-05 22:00:43 UTC; 1min 51s ago
Process: 22041 ExecStart=/home/brianl/autoXMD/env/bin/daphne -u /run/daphne.sock -b 0.0.0.0 -p 8000 autoXMD.asgi:application (code=exited, status=0/SUCCESS)
Main PID: 22041 (code=exited, status=0/SUCCESS)
Sep 05 22:00:43 autoxmd daphne[22041]: warnings.warn('%s. joblib will operate in serial mode' % (e,))
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,013 INFO Starting server at tcp:port=8000:interface=0.0.0.0, unix:/run/daphne.sock
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,017 INFO HTTP/2 support not enabled (install the http2 and tls Twisted extras)
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,020 INFO Configuring endpoint tcp:port=8000:interface=0.0.0.0
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,022 INFO Listening on TCP address 0.0.0.0:8000
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,022 INFO Configuring endpoint unix:/run/daphne.sock
Sep 05 22:00:43 autoxmd daphne[22041]: 2019-09-05 22:00:43,022 CRITICAL Listen failure: [Errno 13] Permission denied: '22041' -> b'/run/daphne.sock.lock'
Sep 05 22:00:43 autoxmd systemd[1]: daphne.service: Start request repeated too quickly.
Sep 05 22:00:43 autoxmd systemd[1]: daphne.service: Failed with result 'start-limit-hit'.
Sep 05 22:00:43 autoxmd systemd[1]: Failed to start daphne daemon.
I think this occurred for the permission issue. By default /run directory is owned by root. So the daphne socket failed to create the daphne.sock.lock file in /run directory.
The solution is that create a folder in /run directory and give permission to your user and group.
For example:
sudo mkdir /run/daphne
sudo chown brianl:www-data /run/daphne
Now change the Unix sock path in the service & socket file.
daphne.service
[Unit]
Description=daphne daemon
Requires=daphne.socket
After=network.target
[Service]
User=brianl
Group=www-data
WorkingDirectory=/home/brianl/autoXMD
ExecStart=/home/brianl/autoXMD/env/bin/daphne -u /run/daphne/daphne.sock -b 0.0.0.0 -p 8000 autoXMD.asgi:application
[Install]
WantedBy=multi-user.target
daphne.socket
[Unit]
Description=daphne socket
[Socket]
ListenStream=/run/daphne/daphne.sock
[Install]
WantedBy=sockets.target
Hopefully, this works for you. For further you can go through a similar issue MySQL Daemon Lock issue

Failed to start Redis In-Memory Data Store. Ubuntu 18.04

I am trying to install redis on my AWS server. I have Ubuntu 18.04 installed on it. I am following steps to install redis from digitalocean article.
When i run sudo systemctl status redis command i am getting below error.
screenshot
I tried to edit /etc/systemd/system/redis.service file and added Type=forking under [Service] section but still getting the same error.
Can anyone suggest me how i can get it fixed?
Thanks in advance.
Based on same digitalocean tutorial, actually it's running fine.
Run this command sudo systemctl restart redis.service, we get (showing "failed" on last line):
● redis.service - Redis In-Memory Data Store
Loaded: loaded (/etc/systemd/system/redis.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-06-28 12:03:11 +03; 1min 0s ago
Process: 20428 ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf (code=exited, status=
Main PID: 20428 (code=exited, status=203/EXEC)
Jun 28 12:03:11 XYZ systemd[1]: redis.service: Service hold-off time over, scheduling restar
Jun 28 12:03:11 XYZ systemd[1]: redis.service: Scheduled restart job, restart counter is at
Jun 28 12:03:11 XYZ systemd[1]: Stopped Redis In-Memory Data Store.
Jun 28 12:03:11 XYZ systemd[1]: redis.service: Start request repeated too quickly.
Jun 28 12:03:11 XYZ systemd[1]: redis.service: Failed with result 'exit-code'.
Jun 28 12:03:11 XYZ systemd[1]: Failed to start Redis In-Memory Data Store.
But if you run sudo service redis-server status, we get (showing "running" on 3rd line):
● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-06-28 11:50:13 +03; 19min ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Process: 19278 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 19371 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCC
Main PID: 19382 (redis-server)
Tasks: 4 (limit: 4915)
CGroup: /system.slice/redis-server.service
└─19382 /usr/bin/redis-server 127.0.0.1:6379
Jun 28 11:50:13 XYZ systemd[1]: Starting Advanced key-value store...
Jun 28 11:50:13 XYZ systemd[1]: redis-server.service: Can't open PID file /var/run/redis/red
Jun 28 11:50:13 XYZ systemd[1]: Started Advanced key-value store.
After searching for hours, it seems like it's some difference between systemctl & service and nothing more, but the actual redis server is running fine. Corrects me if that's not the case. Here's the link: https://askubuntu.com/questions/903354/difference-between-systemctl-and-service-commands
You can even check if redis is working fine, by redis-cli ping, should print PONG
I also encountered this problem, then I tried to check it again.
Finally, I found that when I authorized /var/lib/redis, I entered the wrong command, causing the redis account to have no access to /var/lib/redis.
sudo chown redis:redis /var/lib/redis
sudo systemctl restart redis
succeeded.

Glusterfs-server can't stop in good status

Glusterfs-server can't stop in good status.
I have been trying the following steps to install glusterfs and start service.
yum install centos-release-gluster
yum install glusterfs-server
systemctl start glusterd
and stop it.
systemctl stop glusterd
Then displayed following status "Active: failed".
glusterd.service - GlusterFS, a clustered file-system server
Loaded: loaded (/usr/lib/systemd/system/glusterd.service; disabled)
Active: failed (Result: exit-code) since 火 2017-01-24 18:23:55 JST; 4s ago
Process: 2523 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 2524 (code=exited, status=15)
1月 24 18:23:52 ds009 systemd[1]: Started GlusterFS, a clustered file-system server.
1月 24 18:23:55 ds009 systemd[1]: Stopping GlusterFS, a clustered file-system server...
1月 24 18:23:55 ds009 systemd[1]: glusterd.service: main process exited, code=exited, status=15/n/a
1月 24 18:23:55 ds009 systemd[1]: Stopped GlusterFS, a clustered file-system server.
1月 24 18:23:55 ds009 systemd[1]: Unit glusterd.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
Using environment is"CentOS Linux release 7.2.1511 (Core)"
And installed glusterfs-server version is 3.8.8-1.el7.
Does anyone have an idea what wrong is and to fix this.
I have found a workaround.
Edit /usr/lib/systemd/system/glusterd.service file as following.
[Service]
Type=forking
PIDFile=/var/run/glusterd.pid
LimitNOFILE=65536
Environment="LOG_LEVEL=INFO"
EnvironmentFile=-/etc/sysconfig/glusterd
ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS
ExecStopPost=/usr/bin/systemctl reset-failed glusterd # Add this
KillMode=process
The failed status is clear when the GlusterFS service stop.

how to fix uwsgi services failure to start in centos 7

uwsgi.service - uWSGI Emperor service
Loaded: loaded (/etc/systemd/system/uwsgi.service; disabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Sun 2016-05-22 05:46:09 EDT; 5min ago
Process: 6371 ExecStartPre=/usr/bin/bash -c mkdir -p /run/uwsgi; chown user:nginx /run/uwsgi (code=exited, status=1/FAILURE)
May 22 05:46:09 apxyws systemd[1]: Failed to start uWSGI Emperor service.
May 22 05:46:09 apxyws systemd[1]: Unit uwsgi.service entered failed state.
May 22 05:46:09 apxyws systemd[1]: uwsgi.service failed.
May 22 05:46:09 apxyws systemd[1]: uwsgi.service holdoff time over, scheduling restart.
May 22 05:46:09 apxyws systemd[1]: start request repeated too quickly for uwsgi.service
May 22 05:46:09 apxyws systemd[1]: Failed to start uWSGI Emperor service.
May 22 05:46:09 apxyws systemd[1]: Unit uwsgi.service entered failed state.
May 22 05:46:09 apxyws systemd[1]: uwsgi.service failed.
Does anybody know how to fix something like this, i am still newbie for setup things like this, been searching the answer but nothing similiar to me.
i just started using django and it worked using:
uwsgi --http :8080 --home /root/Env/apxweb --chdir /root/apxweb -w apxweb.wsgi
but when i started using it with nginx,
uwsgi service failure to start.
notes: nginx service worked.
Thanks Luke Dixon,
Yes the problem is within user,
I update the file like this:
/usr/bin/bash -c mkdir -p /run/uwsgi; chown root:nginx /run/uwsgi
uwsgi worked again,
i dont know if this is the correct way to fix uwsgi,
but anyway thank you very much.

Starting uWSGI service fails silently

I'm having a problem which is driving me nuts. I'm trying to create an Ubuntu 15.04 Upstart job, which would start uWSGI server (uWSGI version 2.0.7-debian) running a Django application.
I've configured the service as a follows and try to start the job by issuing command $ sudo service uwsgi start. There is no output in log files, no socket file created and no error.
# file: /etc/init/uwsgi.conf
description "uWSGI server"
start on runlevel [2345]
stop on runlevel [!2345]
exec /usr/bin/uwsgi --ini /srv/configs/uwsgi.ini
service uwsgi status says
● uwsgi.service - LSB: Start/stop uWSGI server instance(s)
Loaded: loaded (/etc/init.d/uwsgi)
Active: active (exited) since Tue 2015-05-05 09:40:08 EEST; 1s ago
Docs: man:systemd-sysv-generator(8)
Process: 21405 ExecStop=/etc/init.d/uwsgi stop (code=exited, status=0/SUCCESS)
Process: 21460 ExecStart=/etc/init.d/uwsgi start (code=exited, status=0/SUCCESS)
May 05 09:40:08 web-2 systemd[1]: Starting LSB: Start/stop uWSGI server instance(s)...
May 05 09:40:08 web-2 uwsgi[21460]: * Starting app server(s) uwsgi
May 05 09:40:08 web-2 uwsgi[21460]: ...done.
May 05 09:40:08 web-2 systemd[1]: Started LSB: Start/stop uWSGI server instance(s).
The uWSGI app is configured as
[uwsgi]
uid = www-data
gid = www-data
socket = /srv/sockets/uwsgi.sock
chmod-socket = 666
master = true
processes = 4
enable-threads = true
plugins = python
chdir = /srv/sites/current
module = wsgi:application
virtualenv = /srv/environments/current
logto = /srv/logs/uwsgi.log
logfile-chown = true
touch-reload = /srv/sites/current/wsgi.py
The service starts fine and everything works ok if the service is started directly, e.g. by issuing command:
sudo -u www-data /usr/bin/uwsgi --ini /srv/configs/uwsgi.ini
For the socket dir and log files directories I've set the most relaxed permissions.
So, I'm at a loss. What to do next?
Thanks to nmgeeks comment, I started looking at other places where uWSGI is configured.
While trying to move everything our own app related under /srv I deleted /run/uwsgi directory, where uWSGI is trying to create a PID file.
You can find the reference at /usr/share/uwsgi/conf/default.ini.
Too bad uWSGI doesn't produce any error in this situation, which left me frustrated for a day.
I just want to put some additional details. In my case I didn't delete anything, but, for some reasons, systemd was not able to create /run/uwsgi directory. So, I did echo "/run/uwsgi 0755 www-data www-data -" > /etc/tmpfiles.d/uwsgi.conf. After that I restarted my VM (I didn't find a way to apply this changes without restarting) and uwsgi started to work!
Yes, it's too bad that uWSGI doesn't produce any error in this situation.