Why are my environment variables not detected when starting up celery? - django

Question

I am running django on centos served by apache and mod_wsgi. I followed the instructions to set up celery to be run as a daemon.
I put this init script https://github.com/celery/celery/blob/3.1/extra/generic-init.d/celeryd in /etc/init.d/celeryd
and set up the configuration in
/etc/default/celeryd
I am using environment variables in my django settings.py file so I can use different configurations in my development and production environments. I know these environment variables are set correctly because the app has been working this whole time. I think that celery is just not getting the variable passed to it or something.
I checked by typing the env command. variables are showing fine.
To start up I just do:
service celeryd start
It tries to start up but throws an error saying that I do not have my environment variables set.
I wrote a function to grab environment variables. that is what throws the error.
def get_env_variable(var_name):
try:
return os.environ[var_name]
except KeyError:
error_msg = "Set the %s environment variable" % var_name
raise ImproperlyConfigured(error_msg)
The only way that error is thrown is if the environment variable is not set correctly.
Does anyone know why celery is not detecting the enironment variables that I have set?

Answer 1

I just discovered that I not only had to set my environment variables in the system, but I also had to pass those variables in to the /etc/default/celleryd script.
I just put my variables at the bottom of /etc/default/celleryd:
export MY_SPECIAL_VARIABLE = "my production variable"
export MY_OTHERSPECIAL_VARIABLE = "my other production variable"

Answer 2

if environment variables write on ~/.bashrc, you can add source ~/.bashrc to /etc/init.d/celeryd at first.

Answer 3

Does your /etc/default/celeryd define what user celery should run as?
In mine I have:
CELERYD_USER="celery"
CELERYD_GROUP="celery"
Can you post your /etc/defaults/celeryd config file?

Answer 4

I had the same problem using celery and supervisor, I had supervisord to use a shell script to start celery worker and also source the env variables.
#!/bin/bash
source ~/.profile
CELERY_LOGFILE=/usr/local/src/imbue/application/imbue/log/celeryd.log
CELERYD_OPTS=" --loglevel=INFO --autoscale=10,5 --concurrency=8"
cd /usr/local/src/imbue/application/imbue/conf
exec celery worker -n celeryd#%h -f $CELERY_LOGFILE $CELERYD_OPTS
in ~/.profile:
export C_FORCE_ROOT="true"
export KEY="DEADBEEF"