How to include a Django logging.FileHanler with Celery on ElasticBeanstalk - django

Question

If I do not include logging I have an Django application with Celery working fine on ElasticBeanstalk using AWS SQS.
When I include logging with a 'logging.FileHandler' celery gets permission denied error because it doesn't have permission rights for my log files. This is my error
ValueError: Unable to configure handler 'celery_file': [Errno 13]
Permission denied: '/opt/python/log/django.log'
This is my logging setup
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler'
},
'file': {
'level': log_level,
'class': 'logging.FileHandler',
#'filters': ['require_debug_false'],
'filename': os.environ.get('LOG_FILE_PATH', LOG_FILE_PATH + '/var/log/django.log')
},
'mail_admins': {
'level': 'ERROR',
#'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'cqc_file' :{
'level': log_level,
'class': 'logging.FileHandler',
#'filters': ['require_debug_false'],
'filename': os.environ.get('LOG_FILE_PATH', LOG_FILE_PATH + '/var/log/cqc.log')
},
'null': {
'class': 'logging.NullHandler',
},
'celery_file': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': os.environ.get('LOG_FILE_PATH', LOG_FILE_PATH + '/var/log/celery.log'),
}
},
'loggers': {
'': {
'level': 'WARNING',
'handlers': ['file'],
},
'debug' : {
'level': log_level,
'handlers': ['file'],
},
'django.security.DisallowedHost': {
'handlers': ['null'],
'level' : 'CRITICAL',
'propagate': False,
},
'django.request': {
'handlers': ['file','mail_admins'],
'level': 'DEBUG',
'propagate': True,
},
'cqc_report' : {
'level' : 'INFO',
'handlers' : ['cqc_file']
},
'celery.task' : {
'handlers': ['console', 'celery_file'],
'level': 'INFO',
'propagate': True,
}
}
}
I think I need to give celery access to the django.log, celery.log and cqc.log files through and elastic beanstalk container command. I tired this using:
03_change_permissions:
command: chmod g+s /opt/python/log
04_change_owner:
command: chown root:wsgi /opt/python/log
05_add_celery_to_wsgi:
command: usermod -a -G wsgi celery
But this just gave me an error saying no user celery (or something to that effect).
How do I get the File logging to work?

Answer 1

The error tells you that the user who is running the process does not have the necessary privileges it needs in order to write into the log. Your solution
03_change_permissions:
command: chmod g+s /opt/python/log
04_change_owner:
command: chown root:wsgi /opt/python/log
05_add_celery_to_wsgi:
command: usermod -a -G wsgi celery
seems to be on the right track, because when there are no privileges for a user to access a location, the solution is to provide the necessary privileges. However, your assumption that the user is celery seems to be wrong. A big clue for this is that this user does not exist. So, you need to make sure that
the user to run the process exists
you identify the correct user for the process
the user has the necessary privileges for logging