Celery and Django on production machine - django

Question

/opt/fubar/fubar/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'fubar.settings')
app = Celery('fubar')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
/opt/fubar/fubar/init.py
from __future__ import absolute_import, unicode_literals
from .celery import app as celery_app
__all__ = ('celery_app', )
/etc/supervisor/conf.d/celeryworker.conf
[program:fubar-celery]
command=/opt/fubar/env/bin/celery worker -A fubar --loglevel=INFO
directory=/opt/fubar
user=www-data
numprocs=1
stdout_logfile=/var/log/celery/fubar/worker.log
stderr_logfile=/var/log/celery/fubar/worker.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
killasgroup=true
priority=998
$ service rabbitmq-server status
● rabbitmq-server.service - RabbitMQ Messaging Server
Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-07-26 06:19:47 UTC; 19h ago
Main PID: 21884 (rabbitmq-server)
Tasks: 92 (limit: 4915)
CGroup: /system.slice/rabbitmq-server.service
├─21884 /bin/sh /usr/sbin/rabbitmq-server
├─21905 /bin/sh /usr/lib/rabbitmq/bin/rabbitmq-server
├─22073 /usr/lib/erlang/erts-9.2/bin/epmd -daemon
├─22185 /usr/lib/erlang/erts-9.2/bin/beam.smp -W w -A 64 -P 1048576 -t 5000000 -stbt db -zdbbl 32000 -K true -B i
├─22299 erl_child_setup 65536
├─22372 inet_gethost 4
└─22373 inet_gethost 4
$ celery worker -A fubar --loglevel=INFO on localhost returns
[tasks]
. fubar.celery.debug_task
. apps.raptor.tasks.launchraptor
. apps.raptor.tasks.nolaunch
while I see no tasks in the log file in production
Apache error log shows:
mod_wsgi (pid=26854): Exception occurred processing WSGI script '/var/www/fubar/fubar.wsgi'.
...
celery.exceptions.NotRegistered: 'apps.raptor.tasks.launchraptor'
I installed supervisor with pip install supervisor to get v4.2.0
What can I run to test whether things are configured properly?
Why is celery worker started with supervisor not finding the tasks that show up when run as celery worker.

Answer 1

I got rid of RabbitMQ and moved to redis. I had more success installing/configuring redis. Doesn't answer the question asked.
Observation was that supervisor installed with pip install supervisor doesn't work and does when done via apt install supervisor. I don't know why.
tail -f /var/log/celery/fubar/worker.log is the best way to see what's going on with the worker.
celery inspect ... also works for a snapshot of whats happening
I had to change command= in the conf.d/fubar*.conf files to point to a shell script which worked better than calling celery from the conf file itself. Also, shell scripts must be owned by the user=value and set to +x.