simple error log message in django - django

I've got error emails setup via Django's logging mechanism in 1.3. It sends me a nice email when an error happens. However, when I log a simple error message it's being formatted oddly and I'm not sure why.
For example, there's a condition in my app where if something doesn't exist in the DB I want to know about, but I have a suitable default value that will work fine. Thus, I want an email sent to me with some info; it's not necessarily happening on an Exception.
If I do something like this:
logger.error("fee did not exist in the database for action %s", "actionX")
The information in the logfile is fine, but the email is really lacking some information. Here's the subject line:
[Django] ERROR: Test this jazz %s
And then the body:
None
Request repr() unavailable
My question is, how do I get A) the value to show up in the subject and B) get some actual, relevant information in the body....like line number or something like that.

You need to acknowledge two things:
You want to send an email using Python's builtin logging system and
You are not logging a regular exception so the builtin mechanism for sending emails won't work since it's depending on an exception type-like object to be passed and stuff to be stored in a traceback.
Anyways, not impossible!
LOGGING = {
...
'handlers': {
...
'my_special_mail_handler': {
'level': 'ERROR',
'filters': [],
'class': 'myapp.loggers.MyEmailHandler',
'include_html': False,
},
},
'loggers': {
...
'my_special_logger': {
'handlers': ['console', 'my_special_mail_handler'],
'level': 'DEBUG',
'propagate': True,
},
}
}
MY_RECIPIENTS = (("Name of person", "email#example.com"),)
...that stuff is merged into your settings.
Then, there's your special logging class, MyEmailHandler:
from django.utils.log import AdminEmailHandler
from django.core.mail.message import EmailMultiAlternatives
from django.conf import settings
class MyEmailHandler(AdminEmailHandler):
def emit(self, record):
if not getattr(settings, "MY_RECIPIENTS", None):
return
subject = self.format_subject(record.getMessage())
message = getattr(record, "email_body", record.getMessage())
mail = EmailMultiAlternatives(u'%s%s' % (settings.EMAIL_SUBJECT_PREFIX, subject),
message, settings.SERVER_EMAIL, [a[1] for a in settings.MY_RECIPIENTS],)
mail.send(fail_silently=False)
Now you're able to create a special logging entry that's both emailed and output to the terminal this way:
import logging
logger = logging.getLogger("my_special_logger")
error = logger.makeRecord(
logger.name, logging.ERROR, 0, 0,
u"Subject: Some error occured",
None, None, "", None,
)
error.email_body = msg
logger.handle(error)
and to make stuff easy, use a utility function:
import logging
def my_log(msg, body, level=logging.ERROR):
logger = logging.getLogger("my_special_logger")
error = logger.makeRecord(
logger.name, level, 0, 0,
u"Subject: Some error occured",
None, None, "", None,
)
error.email_body = msg
logger.handle(error)

The Django Logging docs state:
Of course, it isn't enough to just put logging calls into your code. You also need to configure the loggers, handlers, filters and formatters to ensure that logging output is output in a useful way.
You need to tweak your settings in your logging.config() dictionary. That allows you to set out exactly what information you want and how to format.
Cutting a small bit from the docs:
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
Gives you a bit of an idea how you can influence the output, and the logging.config docs for python will fill out the available options.

Related

Configuring Sentry handler for Django with new sentry_sdk without raven

The new sentry_sdk for django provides very brief installation (with intergration via raven marked for deprecation).
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
sentry_sdk.init(
dsn="https://<key>#sentry.io/<project>",
integrations=[DjangoIntegration()]
)
Previously, one would configure sentry with raven as the handler class like this.
'handlers': {
'sentry': {
'level': 'ERROR', # To capture more than ERROR, change to WARNING, INFO, etc.
'class': 'raven.contrib.django.raven_compat.handlers.SentryHandler',
'tags': {'custom-tag': 'x'},
},
...
...
'loggers':{
'project.custom':{
'level': 'DEBUG',
'handlers': ['sentry', 'console', ,]
}
See more as defined here
The challenge is that raven does not accept the new format of the SENTRY_DSN. in the format https://<key>#domain.com/project/ The old format is along the lines of https://<key>:<secret>#domain.com/project.
Raven will throw InvalidDSN with the old format. The old DSN Key is marked for deprecation.
The documentation is very silent on how to define handlers. and clearly raven which has been deprecated is not happy with the new key format.
I could rely on the old DSN deprecated format but will appreciate advice on how to configure the handler with the new format.
I have taken some reading and practicing. The new version does not depend on raven in any way. So you have to remove all references to raven. in settings.py and any references to the raven client.
There is no need to worry about the handler for sentry_sdk.
It will suffice to declare the handler for the console only and append that handler to every other logger defined as required.
'handlers':{
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
}
# No need to define a sentry handler, defined by the integration.
}
'loggers':{
'project.custom': {
'level': 'DEBUG',
'handlers': ['console', ], # You dont have to add sentry handler here
}
}
Also NOTE that logger.exception will just be ignored, you will have to use
capture_exception from sentry_sdk or capture_message from sentry_sdk
Apparently, it is possible that an event is not logged into sentry for a number of reasons, however logger.exception should always work

how to log all my database changes being done from the application and not only from the django-admin.

I want to log all my database changes being done from the application and not only from the django-admin. how can i achieve that ? Currently we can only see the history in django admin for the changes done through the admin interface. do i need to define signals for this?
In settings.py, we have to enable logging. Put this code in your settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django.db.backends': {
'handlers': ['console'],
'level': 'DEBUG',
'propagate': True,
},
},
}
Django documentation for logging - https://docs.djangoproject.com/en/1.11/topics/logging/#django-db-backends
Django admin uses LogEntry.objects.log_action to record those history changes. There's nothing stopping you from calling that same method in your own code to record changes made elsewhere.
You can use a pre_save signal to get the object before committing to the database and then fetch the old values from the database to compare and check for changes.
The formatting of the message can be a plain string, but the admin puts it in a JSON format so it can be translated. You can look at the source for construct_change_message is django.contrib.admin.utils to figure out that JSON format if you want to continue using that for the ManyToManyField, etc.
There are two types of changes possible.
If you are concerned with the structural changes in the database, they are anyways saved in the migrations folder inside your app directory.
If you want to log DB changes in terms of entries made in the database, you might find the python package django-audit-log useful. You can install it via pip, and once installed, you can add trackers to your models by doing something like this:
from audit_log.models.managers import AuditLog
class YourModelName(models.Model):
#your model definition here
audit_log = AuditLog()
You can find the docs here
Another alternative is django-reversion which allows you to do version control for model instances.
Hope this helps!

Ignore e-mails about missing ALLOWED_HOSTS entry for one hostname

I have configured my application so it accepts requests with my FQDN in the Host: header (using ALLOWED_HOSTS). However, our backend team monitors the site with a different address (just a HTTP “ping” to check if the application is alive). I don’t need to serve any pages on this second site, however, getting e-mails about a missing ALLOWED_HOSTS entry for this particular case is a bit annoying (it comes every five minutes).
I have found several articles that discuss how to disable such notifications all at once, but I’d like to ignore this for only this one host. Is there a solution for this?
You could solve this by addding a filter to the mail_admins and silencing the specific django.security.DisallowedHost log record. Docs.
When a non-valid host is received a django.core.exceptions.SuspiciousOperation is raised which is logged by django.security.DisallowedHost logger and handled by default by mail_admins logging handler.
By adding a django.utils.log.CallbackFilter filter to the mail_admins handler you could silent the log record you want. In this case silenting the mail_admins handler will avoid sending an email.
An example LOGGING settings would look like this (I will only add the relevant bits, your settings will probably have more entries):
def skip_missing_host(record):
if record.name == 'django.security.DisallowedHost':
# `record.msg` contents are similar to:
# u"Invalid HTTP_HOST header: 'somehost'.
# You may need to add u'somehost' to ALLOWED_HOSTS."
# Add condition to determine when the ERROR should be logged:
if "u'somehost'" in record.msg:
# Returning False indicates that the record shouldn't be handled:
return False
return True
LOGGING = {
'filters': {
'skip_missing_host': {
'()': 'django.utils.log.CallbackFilter',
'callback': skip_missing_host,
},
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false', 'skip_missing_host'],
'class': 'django.utils.log.AdminEmailHandler'
},
},
}
Alternatively you could create your own django.security.DisallowedHost handler, which would need to be configured in the LOGGING setting.

how do I disable a log message that exists in the Django source?

e.g. in get_response() at https://github.com/django/django/blob/master/django/core/handlers/base.py#L133
there is a
logger.warning('Not Found: %s', request.path,
extra={
'status_code': 404,
'request': request
})
...which seems to log something every time a request 404's I think.
This is clogging up my logs as (for instance) RSS bots crawl some old, non-working URLs on my site
I'd like to stop the logging noise, so I've tried something like the below in my LOGGING config in settings.py.
LOGGING = {
...
'loggers': {
...
'django.core.handlers': {
'handlers': ['app_logs'],
'propagate': False,
'level': 'ERROR'
},
}
Here, I'm trying to quiet the logger.warning by setting the module log level of django.core.handlers to ERROR, but it seems to be not working. Anyone know what to check or do?
Maybe I'm missing something obvious or perhaps flat out doing it wrong hmmm
The correct logger name is django.request.

Store Django Log messages in a database?

Django (the python web framework) uses python's logging system to store logs.
Is there an easy way to store log messages in a database, and then allow admin users to look over them via the web? It's the sort of thing I could write myself, but no point re-inventing the wheel. I don't want to log exceptions, but info/debug/notice type messages that I have added to the code.
Ideally I'd like to be able to store metadata about the log message as it's done (like the remote IP address, user agent, wsgi process id, etc.), and then filter / browse based on that (i.e. show me all log messages from this IP address in the last 24 hours). Has anyone done this?
Just use Sentry. Raven, the Django piece of the functionality hooks into the logging framework, so in addition to collecting errors from your app, any custom log messages should show up as well.
Apart from the obvious choice of Sentry, for the sake of exercise, there is a nice blog article titled "Creating custom log handler that logs to database models in django" in the "James Lin Blog", which briefly explains how to do this using a second database, with code samples.
The code is adapted from the standard Python RotatingFileHandler:
...The RotatingFileHandler allows you to specify which file to write to and rotate files, therefore my DBHandler should also allow you to specify which model to insert to and specify expiry in settings, and best of all, on a standalone app.
This could also be easily adapted for using a single db.
check django-db-logger
it takes less than a minute to integrate
https://github.com/CiCiUi/django-db-logger
try django-requests. I've tried it and it basically just puts the request logs in a table called requests.
You can check a good solution that I posted here. You just need a string-connection to connect to your database. For example, if you use a MySQL, the connection-string should be:
# mysqlclient
'mysql+mysqldb://username:password#host:port/database'
or
# PyMySQL
'mysql+pymysql://username:password#host:port/database')
then you can use PhpMyAdmin as a "MySQL web administration tool" to look over the database via web browsers or DataGrip (my preference) to access any database remotely.
for using the handler in Django you just need to add the handler class to the LOGGING variable of setting.py as follow:
level = 'INFO' if DEBUG else 'WARNING' # I prefer INFO in debugging mode and WARNING in production
handler = ['log_db_handler', ] # In production I rarely check the server to see console logs
if DEBUG:
handler.append('console')
LOGGING = {'version': 1,
'disable_existing_loggers': False,
'formatters': {'verbose': {'format': '{levelname} {message}', # {asctime} {module} {process:d} {thread:d}
'style': '{', }, },
'handlers': {'log_db_handler': {'level': level,
'class': 'db_logger.handlers.DBHandler',
'formatter': 'verbose', },
'console': {'class': 'logging.StreamHandler', }},
'loggers': {'db_log': {'handlers': handler,
'level': level,
'propagate': False, },
'django': {'handlers': handler,
'level': level,
'propagate': True, },
'django.request': {'handlers': handler,
'level': level,
'propagate': True, }}}
Pay attention that the 'db_logger.handlers.DBHandler' points to the handler class.