Django Logger with User info - django

I'm just curious. Is it possible to put User info within the formatters info in LOGGING config in setting.py?
Right now I just put that info in the message to log but maybe there's a way to set it in formatters argument.
This is my LOGGING configuration right now:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '[%(asctime)s] %(levelname)s [%(funcName)s] %(message)s'
}
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': BASE_DIR + '/logs/uca_{:%d_%m_%Y}.log'.format(time.now()),
'formatter': 'simple'
}
},
'loggers': {
'ucalog': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True
}
}
}
I haven't seen anything similar in django's documentation and I think it would be usefull to get records of WHO did WHAT.

try this :
'format': '[%(asctime)s] %(levelname)s [%(funcName)s] - %(username)s: %(message)s'
logging.basicConfig(format=format)
logger.info(message, extra={'username' : request.user.username})

Related

Django logs print statemeents are not writing to file

In django i am adding logging and storing logs to file. It is writing to files the infor and errors
But not print statements.
No print statements are coming.
I wants to write all including print statements also.
LOGGING = {
'version': 1,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'logs.log',
'formatter': 'simple'
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
}
}
Please have a look, how i can achive this.
If you want to have a proper logging system, you should change all the print to logger.info to log your statements.
Have a look here.

Using dictConfig() with variable log file name

I have configuration dictionary for python scripts that I import from another module like below
LOG_CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'loggers': {
'': {
'handlers': ['consoleHandler', 'fileHandler'],
'level': 'DEBUG'
}
},
'handlers': {
'consoleHandler': {
'level': 'INFO',
'formatter': 'consoleFormatter',
'class': 'logging.StreamHandler',
'stream': 'ext://sys.stdout',
},
'fileHandler': {
'level': 'INFO',
'formatter': 'fileFormatter',
'class': 'logging.FileHandler',
'mode': 'w',
}
},
'formatters': {
'fileFormatter': {
'format': '%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S',
'class': 'logging.Formatter'
},
'consoleFormatter': {
'format': 'UAC_API>> %(levelname)s - %(message)s'
},
},
}
I want log filename to be variable. When loading configuration from a file, I used:
logging.config.fileConfig(config_file,
defaults={'logfilename': logfullpath},
disable_existing_loggers=False)
Where logfullpath is path to the log file.
But defaults={'logfilename': logfullpath} does not work with dictConfig (comparing to logging.config.fileConfig has just one agument). Would anybody help me with that please?
Found a workaround if anyone has the same problem. Since the LOG_CONFIG is the dictionary I expanded it like can be seen below (added filename:logfullpath as key value pair for fileHandler). Then I used it as argument for dictConfig.
LOG_CONFIG['handlers']['fileHandler']['filename'] = logfullpath
logging.config.dictConfig(LOG_CONFIG)

Django server not working with AzureLogHandler (opencensus)

I'm trying to connect my django project logs to Azure Application Insights using OpenCensus. The middleware for montirong requests works well but I also want to send telemetry logs (not just requests) to Azure. Here is my django LOGGING configuration :
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(asctime)s %(levelname).3s %(process)d %(name)s : %(message)s',
},
'simple': {
'format': '%(asctime)s %(levelname)-7s : %(message)s',
},
},
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse',
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
'azure': {
'formatter': 'simple',
'class': 'opencensus.ext.azure.log_exporter.AzureLogHandler',
'connection_string': 'InstrumentationKey=XXXX-XXXX-XXXX-XXXX'
},
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'': {
'level': os.environ.get('LOGLEVEL', 'INFO'),
'handlers': ['console', 'azure'],
},
'devdebug': {
'handlers': ['console'],
'level': 'INFO',
'propagate': False,
},
'django': {
'handlers': ['console', 'mail_admins'],
'level': os.environ.get('LOGLEVEL', 'INFO'),
'propagate': False,
}
},
}
Without 'azure' handler in my root logger config, everything works fine. With 'azure' handler, the server starts but doesn't work : I am unable to connect to it. I really don't know what is happening as it doesn't show me unusual logs (even with LOGLEVEL=DEBUG).
My handler configuration should be good as I can receive logs in Azure (when I run any manage command). Even when I run manage runsslserver localhost:53215, I receive logs but it is like my server is not running when I try to reach it.
I have setup the logging via settings and have it working (although not using runsslserver. I have set the connection_string as an environment variable.
The problem I am having is that the 'django' logger does not work in production. All other loggers are sending messages but not that one??? I have a classic case of it works on my machine and our dev servers but not prod... FML!!
INTEGRATIONS = ['postgresql', 'httplib','logging', 'threading']
config_integration.trace_integrations(INTEGRATIONS)
# Set the AppInsights Key as an env variable so it can be used by the logging system
os.environ['APPLICATIONINSIGHTS_CONNECTION_STRING'] = 'InstrumentationKey=XXXXXXXXXXXXXXXXXXXXXXX'
LOGGING = {
'disable_existing_loggers': True, #False, #<-- if true then make sure that you have a 'django' and '' logger
'filters': {
"require_debug_false": {
'()': 'django.utils.log.RequireDebugFalse',
},
'require_debug_true': {
'()': 'django.utils.log.RequireDebugTrue',
},
},
'formatters': {
'simple': {
'format': '[%(asctime)s] %(levelname)s %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
'verbose': {
'format': '[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
'azure_verbose': {
'format': '[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s traceId=%(traceId)s spanId=%(spanId)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'filters': ['require_debug_true'],
'formatter': 'verbose'
},
'azure':{
'level': 'INFO',
'filters': ['require_debug_false'],
'class': 'opencensus.ext.azure.log_exporter.AzureLogHandler',
'formatter': 'azure_verbose'
},
},
'loggers': {
'mylogger': {
"handlers": [
"azure",
"console",
],
},
'django': {
'handlers': [
'azure',
'console',
],
},
'py.warnings': {
'handlers': [
'azure',
'console',
],
},
'': {
'handlers': [
'azure',
'console',
],
},
},
'version': 1
}

Elastic APM: How to turn off logging for django

flushing due to time since last flush 9.060s > max_flush_time 9.060s
I 'm getting tone of those message in django debug.
I tried changing to their default setting
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
},
'handlers': {
'elasticapm': {
'level': 'WARNING',
'class': 'elasticapm.contrib.django.handlers.LoggingHandler',
},
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
}
},
'loggers': {
'django.db.backends': {
'level': 'ERROR',
'handlers': ['console'],
'propagate': False,
},
'mysite': {
'level': 'WARNING',
'handlers': ['elasticapm'],
'propagate': False,
},
# Log errors from the Elastic APM module to the console (recommended)
'elasticapm.errors': {
'level': 'ERROR',
'handlers': ['console'],
'propagate': False,
},
},
}
Still getting lots of logs.
How to turn this off?

Why does my Django 1.3 logging setup cause all messages to be output twice?

My Django 1.3 logging setup causes all log messages to be output twice. I've read that importing settings.py two or more times has caused this problem in the past, but Django 1.3 has new logging features, and I don't think I'm importing settings.py twice anywhere.
settings.py config:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
'format': '%(levelname)s: %(message)s'
},
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple',
},
},
'loggers': {
'custom': {
'handlers': ['console'],
'level': 'DEBUG',
},
}
}
Code:
import logging
log = logging.getLogger('custom')
log.debug('message1')
log.debug('message2')
Output:
DEBUG: message1
DEBUG:custom:message1
Thanks for your help.
Have you tried setting propagate = False? Along with disable_existing_loggers = True?
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'simple': {
'format': '%(levelname)s: %(message)s'
},
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple',
},
},
'loggers': {
'custom': {
'handlers': ['console'],
'level': 'DEBUG',
'propagate': False,
},
}
}
You might want to try 'disable_existing_loggers': True?
I've been suffered by the same issue. And I fixed it by redirecting the root logs to a file, and project logs to both the file and the console.
I grep my code, and could not find anywhere basicConfig() exists, also tried to set disable_existing_loggers to True, It doesn't help, finally solved the problem by set a file logger. I guess it maybe a problem by design in some cases.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
# exact format is not important, this is the minimum information
'format': '%(asctime)s %(name)-12s %(lineno)d %(levelname)-8s %(message)s',
},
},
'handlers': {
'console': {
#'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'simple',
},
# Add Handler for mail_admins for `warning` and above
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
},
'file': {
#'level': 'INFO',
'class': 'logging.FileHandler',
'formatter': 'simple',
'filename': os.path.join(os.path.dirname(PROJECT_ROOT), 'crawler.admin.log'),
},
},
'loggers': {
# root logger
'': {
'level': 'INFO',
'handlers': ['file', 'mail_admins'],
},
'scrapy': {
'level': 'WARNING',
'handlers': ['console', 'mail_admins'],
'propagate': False,
},
'crawleradmin': {
'level': 'INFO',
'handlers': ['console', 'file', 'mail_admins'],
# required to avoid double logging with root logger
'propagate': False,
},
},
}