I setup logging and it's working great. What I want to do now is log all debug info for one particular app called 'my app' in addition to the other logs and am not sure how to achieve this.
Working set up as it is now:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
#
'formatters': {
'default': {
'format': '\n%(levelname)s - %(asctime)s\n%(message)s\n'
},
},
#
'handlers': {
#
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
'include_html': True,
},
#
'debug_log': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/var/log/django/debug.log',
'maxBytes': 1024*1024*5,
'backupCount': 5,
'formatter': 'default',
},
#
'error_log': {
'level': 'ERROR',
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/var/log/django/error.log',
'maxBytes': 1024*1024*5,
'backupCount': 5,
'formatter': 'default',
},
},
#
'loggers': {
'': {
'handlers': ['mail_admins', 'error_log',],
'level': 'INFO',
'propagate': True,
},
'django': {
'handlers': ['debug_log',],
'level': 'DEBUG',
'propagate': True,
},
}
}
I'd like to set up DEBUG logging that captures everything for the one particular app. I add the following handler to LOGGING:
'app_log': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/var/log/django/app.log',
'maxBytes': 1024*1024*5,
'backupCount': 5,
'formatter': 'default',
},
And the following logger:
'my_app': {
'handlers': ['app_log',],
'level': 'DEBUG',
'propagate': True,
},
Add the following to my_app.views:
import logging
logger = logging.getLogger(__name__)
And here's where I'm not sure what I need to do next. I can add something like the following to a certain view
...
if blahblahblah:
logger.debug('My Message Here')
...
and things work great. However, what I really want is all debug info for the entire app, not just 'My Message Here'. I want all the query info etc... ie: same info as the standard debug logging but just for that one app.
How do I achieve this?
You haven't explicitly enabled the SQL query logging but I am uncertain if it can be enabled in app scope. AFAIK it can only be enabled or disabled for the entire project.
If you want to do this at the app level, or view level you can try this:
from django.db import connection
logger.debug(connection.queries)
Place this code at the end of each view where you want to see the sql logs.
Related
I'm using pyppeteer to take screenshots of images to a make a pdf but pyppeteer auto logs everything I take a screenshot at and because of server limitations and the logs is written to a file the logs are crashing my server.
Is there any way to completely disable logging? I tried this already:
'logLevel': logging.NOTSET,
'env': {'DEBUG': 'puppeteer:*,-not_this'},
I also tried to disable logging like this:
logging.getLogger('pyppeteer').setLevel(logging.NOTSET)
And nothing seems to work.
Update
I managed to found a workaround although not a solution by disabling all logging in the application like this:
logging.disable(logging.CRITICAL)
# pyppeteer code...
logging.disable(logging.NOTSET)
Try this:
logging.getLogger("<logger name>").disabled = True
If you want to disable all loggers you can use:
for name in logging.root.manager.loggerDict:
# print("logger", name)
logging.getLogger(name).disabled = True
Or maybe if you want to disable all except some loggers you created:
allowed_loggers = ['some_logger']
for name in logging.root.manager.loggerDict:
# print("logger", name)
if name in allowed_loggers:
continue
logging.getLogger(name).disabled = True
This should log all of your errors to a file.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
'format': '{asctime} {name}] {message}',
'style': '{',
}
},
'handlers': {
'file': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': 'error.log',
'formatter': 'simple',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': True,
},
'': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': False,
},
},
}
I have encountered a strange behavior of Django Loggers.
I am developing a front end application using Django. During the login service, I make some requests to certain components and use log.warning() calls to see the flow of the requests.
The logs worked perfectly, until I decided to add a LOGGING configuration to print the output of the logs in a file, as I want to deploy the application via Docker and I want to periodically check the log files.
When I added the following Django configuration concerning logging:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'detailed': {
'class': 'logging.Formatter',
'format': "[%(asctime)s] - [%(name)s:%(lineno)s] - [%(levelname)s] %(message)s",
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'formatter': 'detailed',
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': "{}/am.log".format(BASE_DIR),
'mode': 'w',
'formatter': 'detailed',
'level': 'INFO',
'maxBytes': 2024 * 2024,
'backupCount': 5,
},
},
'loggers': {
'am': {
'level': 'INFO',
'handlers': ['console', 'file']
},
}
}
The logging stops working. The file specified in the logging configuration, am.log, is indeed created but nothing gets printed to this file. Even the console logging does not take place.
I have taken this logging configuration from one of my Django projects for the backend of this application, and there it works perfectly. I really don't understand what I am doing wrong. Could you please help me or guide me in the right direction. I would be very grateful.
I wish you all a good day!
By using the key "am" in your 'loggers' configuration, you're defining one logger with name "am":
'loggers': {
'am': { # <-- name of the logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}
So to use that logger, you have to get it by that name:
logger = logging.getLogger("am")
logger.warning("This is a warning")
If you name your loggers by the name of the module in which you're running, which is recommended practice, then you need to define each module logger:
logger = logging.getLogger(__name__) # <-- this logger will be named after the module, e.g. your app name.
Then in your logging configuration you can specify logging behavior per module (per app):
'loggers': {
'my_app': { # <-- logging for my app
'level': 'INFO',
'handlers': ['console', 'file']
},
'django': { # <-- logging for Django module
'level': 'WARNING',
'handlers': ['console', 'file']
},
}
Or if you just want to log everything the same, use the root ('') logger, which doesn't have a name, just empty string:
'loggers': {
'': { # <-- root logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}
I'm trying to configure django logging in the django settings file so that it logs django info and info for my application to a custom file for easy viewing. Here's my logging config:
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'console': {
# exact format is not important, this is the minimum information
'format': '%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
},
'file': {
# exact format is not important, this is the minimum information
'format': '%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
},
},
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'formatter': 'file',
'filename': 'logs/django_log.log',
'backupCount': 10, # keep at most 10 log files
'maxBytes': 5242880, # 5*1024*1024 bytes (5MB)
},
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'console',
},
},
'loggers': {
'django': {
'handlers': ['file', 'console'],
'level': 'INFO',
'propagate': True,
},
'py.warnings': {
'handlers': ['console'],
},
'my_application': {
'level': 'INFO',
'handlers': ['file', 'console'],
# required to avoid double logging with root logger
'propagate': False,
},
},
}
This works on my local manage.py test server, both with django events appearing and events that I log, initialized with my_application as the logger name. However, on my web server, the logging file is created and, oddly, only populated with occasional django WARNING messages. So, there is no permissions error or inability to access the log file. Since the same config works on my local, the config can't be the issue, and it clearly has only INFO level logs.
My server setup is taken from this guide: https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-18-04 and uses Gunicorn with Nginx as a reverse proxy. Could the issue be with the configs for these? I'm stumped.
Additionally, where's a good best practice place to store this django log file?
Also, one related bonus question: What's a good best practice free/cheap service that can notify you if a specific error is logged? It seems like a good idea to set something like that up, but I don't think the django emailer is necessarily the most elegant or best.
I am performing a simple Python Django TestCase. The question is simple: I want to use loggers to redirect information (stdout) to one file and failures/errors (stderr) to another.
I know I can print stdout to a file simply by defining the logger and using it to print the messages (i.e. logger.info('my debug message')); however, I have not found a way to log a failure/error. I can redirect the output of the entire test run using Bash (of which I am semi-successful), but I know there is a way to simplify it and use loggers to do all that backend work for me.
Here is an example of my logger:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s, %(module)s at line %(lineno)d:\n%(message)s'
},
},
'handlers': {
'app': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'logs/app.log',
'formatter': 'verbose'
},
'test': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'logs/test.log',
'formatter': 'verbose'
},
'test-fail': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': 'logs/test-fail.log',
'formatter': 'verbose',
}
},
'loggers': {
'app': {
'handlers': ['app'],
'level': 'DEBUG',
'propagate': True,
},
'test': {
'handlers': ['test', 'test-fail'],
'level': 'DEBUG',
'propagate': True,
},
},
}
And let's just say I'm trying to log the following test case to the logs/test-fail.log file specified in my logger:
logger = logging.getLogger('test')
def test_this(self):
logger.debug('This message will print to the logs/test.log file')
self.assertTrue(False) # will fail
In short, I want to print errors/failures from a Django TestCase using the logger and not using Bash. Also note that I'm fine with using the default logger, but I want to define an error logger case for only test cases, and I don't want the test error case being used by default.
why not try changing the logger and handler level to ERROR ? (more info in case https://docs.djangoproject.com/en/2.2/topics/logging/#topic-logging-parts-loggers)
In my project i deployed my django-project with tornado server, and my tornado main function is:
def main():
tornado.options.options.logging = None
tornado.options.parse_command_line()
os.environ['DJANGO_SETTINGS_MODULE'] = 'Zero.settings'
application = get_wsgi_application()
container = tornado.wsgi.WSGIContainer(application)
http_server = tornado.httpserver.HTTPServer(container, xheaders=True)
http_server.listen(tornado.options.options.port)
tornado.ioloop.IOLoop.current().start()
When i use tornado.options.options.logging = None to disabled tornado logging output, but it still output the log message in my console with twice, my django logging config is:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(threadName)s] [%(name)s:%(funcName)s] [%(levelname)s]- %(message)s'}
},
'filters': {
},
'handlers': {
'error': {
'level': 'ERROR',
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'log', 'error.log'),
'maxBytes': 1024*1024*5,
'backupCount': 5,
'formatter': 'standard',
},
'console':{
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'standard'
},
},
'loggers': {
'django': {
'handlers': ['console'],
'level': 'INFO',
'propagate': True
},
}
}
The final result is:
2018-06-15 17:40:55,724 [MainThread] [base_views:get] [INFO]- get message correct
INFO:base_views:get message correct
so what can i do to solve this problem.
Thank U.
You've only configured the django logger, not the root logger. When Tornado sees that the root logger is not configured, it adds its own last-resort handler (using logging.basicConfig instead of tornado.log). Because of the way python loggers propagate, this results in any other loggers being duplicated.
When you use tornado.options.options.logging = None, you should make sure that you configure the root logger yourself, or configure all of your other handlers with propagate=False. In this case, move the loggers.django section of your config to a new root section:
'handlers': {...}
'root': {
'handlers': ['console'],
'level': 'INFO',
},