Django not logging all errors on production - django

My Django app is only properly logging and emailing errors when Debug=True.
On production, I'm only getting 'Invalid HTTP_HOST header' errors in both the log file and my inbox. Every other error is not logged in the django log nor is it emailed to me.
I've tried some things with this test request:
def test(request):
logging.debug('debug test message') // Always appears in log file
from django.core.mail import mail_admins
mail_admins('test error', 'test error via call to mail_admins()') // Always arrives in my inbox just fine
raise Appointment.DoesNotExist // When Debug=False (both locally and in production), does not appear in log file or in inbox.
My logging settings for both local and production:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
'csv_formatter': {
'format': '%(levelname)s;%(asctime)s;%(message)s'
}
},
'handlers': {
'null': {
'level': 'DEBUG',
'class': 'logging.NullHandler',
},
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
},
'default': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'maxBytes': 1024 * 1024 * 50,
'backupCount': 10,
'formatter': 'csv_formatter',
},
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
'formatter': 'verbose',
},
},
'loggers': {
'': {
'handlers': ['default'],
'level': 'DEBUG',
'propagate': True,
},
'asyncio': {
'level': 'WARNING',
},
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'django.security': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
How do I make sure all Django errors are logged and emailed in production (when Debug=False)?

Related

How to exclude 404 error messages from logging to email?

Good afternoon. I have this type of logs in the settings:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'verbose': {
'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
'style': '{',
},
'simple': {
'format': '{levelname} {message}',
'style': '{',
},
},
'handlers': {
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
},
'mail_admins': {
'level': 'WARNING',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler',
'include_html': True,
},
},
'loggers': {
'': {
'handlers': ['console', 'mail_admins'],
'level': 'DEBUG',
'propagate': False,
},
}
}
I receive all messages of the level of warning and above in the email.
Due to various robots, spammers or other users looking for hidden files and folders, I get a lot of 404 (page not found) error messages.
How can I stop receiving information about 404 errors?
This is thanks to the mail_admins handler, that sends you any WARNING level and above error by email, see doc
As suggested in docs, you can subclass this handler and use your new custom handler instead.
class MyCustomHandler(AdminEmailHandler):
"""Custom Handler that ignores 404 errors instead of sending them by email"""
def handle(self, record):
if record.exc_info:
exc_type, exc_value, tb = record.exc_info
if isinstance(exc_value, Http404):
return
super().handle(record)

Can't get Django to send me debug mail

I have a Django site that I have deployed to Heroku, and I'm having trouble getting it to send me error notifications via email. I've logged into the Heroku console and used django.core.mail.send_mail() to send emails, so I know it's configured to send correctly.
I've got ADMINS and MANAGERS configured, and the django.middleware.common.BrokenLinkEmailsMiddleware Middleware so I should be getting emails whenever someone gets a 404 error on the page. I have the following Logging configuration in my settings.py:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': ('%(asctime)s [%(process)d] [%(levelname)s] ' +
'pathname=%(pathname)s lineno=%(lineno)s ' +
'funcname=%(funcName)s %(message)s'),
'datefmt': '%Y-%m-%d %H:%M:%S'
},
'simple': {
'format': '%(levelname)s %(message)s'
}
},
'handlers': {
'mail_admins': {
'level': 'DEBUG',
'class': 'django.utils.log.AdminEmailHandler',
'include_html': True,
},
'null': {
'level': 'DEBUG',
'class': 'logging.NullHandler',
},
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
}
},
'loggers': {
'testlogger': {
'handlers': ['console'],
'level': 'INFO',
}
}
}
What am I missing?

Sentry server not responding on Celery logs

I have a problem related with sending log messages to Sentry server while using Django and Celery async tasks. I utilize Django 1.11, Raven 6.1, Celery 4.0.2.
The log message shows in Sentry server when task is launching synchronous, for example:
#celery.shared_task()
def test_worker():
logger.warning('Logger works!')
raise Exception('---Worker works!')
$ from api.tasks import test_worker
$ test_worker()
When trying launch async task using:
$ res = test_worker.delay()
Sentry server does not contain log about logger.
My settings.py is:
RAVEN_CONFIG = {
'dsn': 'http://my_dsn',
}
SENTRY_AUTO_LOG_STACKS = True
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'root': {
'level': 'DEBUG',
'handlers': ['sentry'],
},
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s - %(message)s'
},
'verbose': {
'format': '%(asctime)s [%(levelname)s] %(module)s:%(funcName)s - %(message)s'
},
},
'handlers': {
# logs directly to Sentry
'sentry': {
'level': 'DEBUG',
'class': 'raven.contrib.django.raven_compat.handlers.SentryHandler',
},
# logs everything to the console
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
},
'loggers': {
'raven': {
'level': 'DEBUG',
'handlers': ['console'],
'propagate': False,
},
'sentry.errors': {
'level': 'DEBUG',
'handlers': ['console'],
'propagate': False,
},
'processing': {
'level': 'DEBUG',
'handlers': ['sentry'],
'propagate': False
},
'celery': {
'level': 'DEBUG',
'handlers': ['sentry'],
'propagate': False
}
},
}
Where I made a mistake, or what I am doing wrong?
Thanks in advance.

Django logging not working in views

I have logging set-up in my Django 1.9 project. I am getting the logs in django_request.log automatically as expected but not in mylog.log when I use logging in views.py. Where am I wrong?
views.py
import logging
logr = logging.getLogger(__name__)
def sample_view(request):
logr.debug('Ran Sample view')
return HttpResponse("Done!")
settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': 'logs/mylog.log',
'maxBytes': 1024 * 1024 * 5, # 5MB
'backupCount': 5,
'formatter': 'standard'
},
'request_handler': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': 'logs/django_request.log',
'maxBytes': 1024 * 1024 * 5, # 5MB
'backupCount': 5,
'formatter': 'standard'
},
},
'loggers': {
'': {
'handlers': ['default'],
'level': 'DEBUG',
'propagate': True
},
'django.request': {
'handlers': ['request_handler'],
'level': 'DEBUG',
'propagate': False
},
}
}
I found out this "If the disable_existing_loggers key in the LOGGING dictConfig is set to True (which is the default) then all loggers from the default configuration will be disabled. Disabled loggers are not the same as removed; the logger will still exist, but will silently discard anything logged to it, not even propagating entries to a parent logger."
on https://docs.djangoproject.com/en/1.7/topics/logging/#configuring-logging
So if you set 'disable_existing_loggers': False, you will be able to see all logs.

Gunicorn Django and logging info to a file

I'm trying to setup my logging settings to send logging.info('any message') to a file through stdout.
This is my gunicorn_django script:
$ gunicorn_django -w $NUM_WORKERS --user=$USER --group=$GROUP --log-level=info --log-file=$LOGFILE &>>$LOGFILE
These are my logging settings:
import sys
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'console': {
'level': 'INFO',
'filters': ['require_debug_false'],
'class': 'logging.StreamHandler',
'stream': sys.stdout,
'formatter': 'simple',
},
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'app.location': {
'handlers': ['console'],
'level': 'INFO',
'propagate': False,
},
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
When I do:
import logging
logging.getLogger('app.location')
logging.info('any message')
It is not logged to the gunicorn_django $LOGFILE. Just print >> sys.stderr messages seem to be showed in the $LOGFILE
How can I log info messages (through stdout) to a file
Your logger is just getting garbage collected as soon as you create it. You need to keep a reference to it as a logger, and then use that logger, like so:
import logging
logger = logging.getLogger('app.location')
logger.info('any message')