How Can I Add to Django's Default Logging? - django

I'm using Django 1.6 and would like to add to—not replace—Django's default logging. Specifically, I want to make it so that in addition to Django's default logging behavior, any log record with a log level of DEBUG or higher gets written to a log file that gets automatically rotated.
Based on what I've found through searches and my own experimenting, it seems like you have to redefine all of the logging (Django's defaults plus my rotating file logging), even when using 'disable_existing_loggers': False. I figure I'm probably doing something wrong.
I'd like to see an example of what to put in my settings.py file that will allow me to accomplish this.

settings.py is just Python, so we can do:
from django.utils.log import DEFAULT_LOGGING
# Use defaults as the basis for our logging setup
LOGGING = DEFAULT_LOGGING
# We need some formatters. These ones are from the docs.
LOGGING['formatters'] = {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
}
# A log file handler that rotates periodically
LOGGING['handlers'].update({
'rotate': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'formatter': 'verbose',
'filename': '/tmp/debug.log',
}
})
# The default 'django' logger is a catch-all that does nothing. We replace it with
# a rotating file handler.
LOGGING['loggers'].update({
'django': {
'handlers': ['rotate'],
'propagate': True,
'level': 'DEBUG',
}
})
# If you don't want to completely replace the django handler, you could do something
# like this instead:
#LOGGING['loggers']['django']['handlers'] += ['rotate']
This will add your rotating file handler to the existing handlers, define the basic formatters, and replace the catch-all logger (which does nothing) with one that replace the default logger

Based on your comments I tried this. It seems to do what you're asking.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'rotate': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/tmp/debug.log',
}
},
'loggers': {
'django': {
'handlers': ['rotate'],
'propagate': True,
'level': 'DEBUG',
}
},
}

Related

How to disable logging in pyppeteer

I'm using pyppeteer to take screenshots of images to a make a pdf but pyppeteer auto logs everything I take a screenshot at and because of server limitations and the logs is written to a file the logs are crashing my server.
Is there any way to completely disable logging? I tried this already:
'logLevel': logging.NOTSET,
'env': {'DEBUG': 'puppeteer:*,-not_this'},
I also tried to disable logging like this:
logging.getLogger('pyppeteer').setLevel(logging.NOTSET)
And nothing seems to work.
Update
I managed to found a workaround although not a solution by disabling all logging in the application like this:
logging.disable(logging.CRITICAL)
# pyppeteer code...
logging.disable(logging.NOTSET)
Try this:
logging.getLogger("<logger name>").disabled = True
If you want to disable all loggers you can use:
for name in logging.root.manager.loggerDict:
# print("logger", name)
logging.getLogger(name).disabled = True
Or maybe if you want to disable all except some loggers you created:
allowed_loggers = ['some_logger']
for name in logging.root.manager.loggerDict:
# print("logger", name)
if name in allowed_loggers:
continue
logging.getLogger(name).disabled = True
This should log all of your errors to a file.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
'format': '{asctime} {name}] {message}',
'style': '{',
}
},
'handlers': {
'file': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': 'error.log',
'formatter': 'simple',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': True,
},
'': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': False,
},
},
}

Django LOGGING not printing to console and file

I have encountered a strange behavior of Django Loggers.
I am developing a front end application using Django. During the login service, I make some requests to certain components and use log.warning() calls to see the flow of the requests.
The logs worked perfectly, until I decided to add a LOGGING configuration to print the output of the logs in a file, as I want to deploy the application via Docker and I want to periodically check the log files.
When I added the following Django configuration concerning logging:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'detailed': {
'class': 'logging.Formatter',
'format': "[%(asctime)s] - [%(name)s:%(lineno)s] - [%(levelname)s] %(message)s",
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'formatter': 'detailed',
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': "{}/am.log".format(BASE_DIR),
'mode': 'w',
'formatter': 'detailed',
'level': 'INFO',
'maxBytes': 2024 * 2024,
'backupCount': 5,
},
},
'loggers': {
'am': {
'level': 'INFO',
'handlers': ['console', 'file']
},
}
}
The logging stops working. The file specified in the logging configuration, am.log, is indeed created but nothing gets printed to this file. Even the console logging does not take place.
I have taken this logging configuration from one of my Django projects for the backend of this application, and there it works perfectly. I really don't understand what I am doing wrong. Could you please help me or guide me in the right direction. I would be very grateful.
I wish you all a good day!
By using the key "am" in your 'loggers' configuration, you're defining one logger with name "am":
'loggers': {
'am': { # <-- name of the logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}
So to use that logger, you have to get it by that name:
logger = logging.getLogger("am")
logger.warning("This is a warning")
If you name your loggers by the name of the module in which you're running, which is recommended practice, then you need to define each module logger:
logger = logging.getLogger(__name__) # <-- this logger will be named after the module, e.g. your app name.
Then in your logging configuration you can specify logging behavior per module (per app):
'loggers': {
'my_app': { # <-- logging for my app
'level': 'INFO',
'handlers': ['console', 'file']
},
'django': { # <-- logging for Django module
'level': 'WARNING',
'handlers': ['console', 'file']
},
}
Or if you just want to log everything the same, use the root ('') logger, which doesn't have a name, just empty string:
'loggers': {
'': { # <-- root logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}

console not showing print message on server, django and uwsgi with nginx

I am currently using django's back end to post another api which works correctly when I am developing locally. But when I push it to a staging cloud server which uses uwsgi with nginx that it is not working properly anymore. I am trying to use print from django to either save the messages into a file or somehow show it in terminal so I can know the post response to start debugging but I have no luck trying to find a way to log any print
I have already did something like this in the settings
LOGGING = {
'version': 1,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/path/to/your/file.log',
'formatter': 'simple'
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
}
}
it does work perfectly and show messages normally but how can I also print messages into the file too?
Can someone give me an idea or advice?
Thanks in advance
So it looks like you already have file logging set up properly. The only thing is that it does not log prints, you have to use django's logging system. Like this:
import logging
logger = logging.getLogger(__name__)
# In your code
logger.debug('Some message') # Logs as debug message
logger.error('Error message') # Logs as error message
Your logger should log all of these to the file since it's level is DEBUG
More info here: https://docs.djangoproject.com/en/2.0/topics/logging/#using-logging

Logging output running django tests under Pycharm

When running/debugging individual tests using django.test.TestCase under PyCharm logging.logger messages are not shown. I've tried setting logging.basicConfig(level=logging.DEBUG) as suggested by How can I see log messages when unit testing in PyCharm? but that didn't help either. I suspect it might be django's TestCase setup interferring.
Is there some other way in test setup or runner configuration that I can turn on debug logging for the test run?
The logging I have set up in my settings.py right now is:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'verbose'
},
'file': {
'level': 'DEBUG',
'class': 'logging.handlers.TimedRotatingFileHandler',
'filename': '/var/log/em/mcqueen-dev.log',
'when': 'midnight',
'formatter': 'verbose',
},
},
'formatters': {
'verbose': {
'format': '%(asctime)s.%(msecs).03d - %(process)d - %(thread)d - %(levelname)8s - %(filename)s:%(lineno)d - %(funcName)s - %(message)s'
},
'simple': {
'format': '%(asctime)s - %(levelname)s %(message)s'
},
},
'loggers': {
'mcqueen_api': {
'handlers': ['console', 'file'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'DEBUG')
},
'mcqueen_app': {
'handlers': ['console', 'file'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'DEBUG')
},
'mcqueen_base': {
'handlers': ['console', 'file'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'DEBUG')
},
},
}
When I want to see logs while working on tests for Django project in PyCharm I'm putting this snippet of code in the file with test:
import logging
logger = logging.getLogger(__name__)
logging.disable(logging.NOTSET)
logger.setLevel(logging.DEBUG)
While running Django tests level for disabling logs is set high (to 50), lowering it (as in line #3 in code above) will cause that logs will be displayed (or saved to file - depending on log handler that is in use).
This thread on stackoverflow explains the probable reason your logging output not showing to console. Apparently, django's unittest runner replaces the global sys.stdout/sys.stderr, but the StreamHandler specified from the django settings is still bound up with the original sys.stdout/sys.stderr. The fix is to add a stream handler to your logger in the test module, based on the values of sys.stdout/sys.stderr during execution.
If you want the logger to log to console for all methods of your test case, then you're probably best using a custom base class (see linked to thread for more details) to wrap the logic for adding/removing at setUp/tearDown.
I prefer to use decorators over the individual test methods for wrapping. For example (using the 'django_test' logger config provided by Sơn Lâm's answer):
import logging
import sys
from contextlib import contextmanager
from django.test import TestCase
#contextmanager
def streamhandler_to_console(lggr):
# Use 'up to date' value of sys.stdout for StreamHandler,
# as set by test runner.
stream_handler = logging.StreamHandler(sys.stdout)
lggr.addHandler(stream_handler)
yield
lggr.removeHandler(stream_handler)
def testcase_log_console(lggr):
def testcase_decorator(func):
def testcase_log_console(*args, **kwargs):
with streamhandler_to_console(lggr):
return func(*args, **kwargs)
return testcase_log_console
return testcase_decorator
logger = logging.getLogger('django_test')
class SomeTestCase(TestCase):
#testcase_log_console(logger)
def test_something(self):
logger.info('show something to console.')
I think it will be work
Log configuration on settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': "[%(asctime)s] %(levelname)s %(message)s",
'datefmt': "%d/%b/%Y %H:%M:%S"
}
},
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/var/log/django_practices.log',
'formatter': 'verbose'
},
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'stream': sys.stdout,
'formatter': 'verbose'
},
},
'loggers': {
'django_test': {
'handlers': ['file', 'console'],
'level': 'DEBUG',
},
'name_your_app': {
'handlers': ['file', 'console'],
'level': 'DEBUG',
}
}
}
In UnitTest file
import logging
logger = logging.getLogger('django_test')
logger.info('test_log')
And Log will be appearance.
I think this is to do with the test runner that you're using - assuming you're using the inbuilt Django tests setup in PyCharm then it should already be setting an environment variable PYTHONUNBUFFERED=1 which I think is what makes the output print directly without being buffered and only showing at the end (which is what I presume is happening). Check that this is set in the test configuration and if it isn't then try that.
See also: Pycharm unit test interactive debug command line doesn't work (particularly if you're using a different test runner)

Where the logging errors located in a django website

I am a newbie in Django. I have a website created with Django + Apache. I imported logging module and print some intermediate values with logging error function. But I don't know where the logging file locates. The OS is Ubuntu.
I searched internet and could not find a good answer (Maybe I did not do it properly). Could someone tell me where can I find the logging file that contains the values I logged in the code?
Thanks.
Keep this in your settings and the filename specifies the location of the file
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'handlers': {
'file_userlogins': { # define and name a handler
'level': 'DEBUG',
'class': 'logging.FileHandler', # set the logging class to log to a file
'formatter': 'verbose', # define the formatter to associate
'filename': os.path.join(VENV_ROOT, 'log', 'userlogins.log') # log file
},
'loggers': {
'logview.userlogins': { # define a logger - give it a name
'handlers': ['file_userlogins'], # specify what handler to associate
'level': 'INFO', # specify the logging level
'propagate': True,
},
}
}