Django: logging only for my project's apps - django

By default I can enable logging in settings.py in the LOGGING configuration by creating a logger "" which will catch all logs. But what if I only want to see logging from my project's apps as opposed to Django internals?
I can imagine explicitly getting a logger in each of my Django app modules and naming it by some convention, e.g. logging.getLogger("myproject." + __file__). Then I can create a logger called 'myproject' (in SETTINGS) which picks up all of these to output. I'd prefer not to hardcode my project name, so I'd do some os.path logic on ___file___ to extract the full namespace up to the file at any arbitrary depth.
At this point, I stop and wonder is there an easier way?

You could use a scheme like the following to create identical loggers for all of your local apps without having to manually add them all to the logging configuration.
First, split out your local apps:
LOCAL_APPS = [
'myapp1',
'myapp2',
...
]
THIRD_PARTY_APPS = [
'django. ...',
...
]
INSTALLED_APPS = LOCAL_APPS + THIRD_PARTY_APPS
Next create the logger configuration for your local apps:
local_logger_conf = {
'handlers': ['my_handler',],
'level': 'DEBUG',
}
Finally, define your loggers as follows:
'loggers': { app: copy.deepcopy(local_logger_conf) for app in LOCAL_APPS }

Not sure if I fully understood your question, because the answer seems too simple.
Assuming you have defined in LOGGING a handler for your project's apps, for example like this:
'handlers': {
'handler_for_my_apps': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'debug.log',
},
and given your apps app1, app2, and so, you could have all the logs from those apps without any Django's internal logs by defining the loggers:
'loggers': {
'app1': {
'handlers': ['handler_for_my_apps'],
'level': 'DEBUG',
},
'app2': {
'handlers': ['handler_for_my_apps'],
'level': 'DEBUG',
},
There will be no Django logs in the same file, unless of course you defined a logger named django with a handler handler_for_my_apps.
In your apps you can get the logger using logging.getLogger(__name__) as recommended by the docs.
Unless I misunderstood your question...

Thank you for sharing, reading this post helped solve logging for my project. I'll share the solution I opted for, hoping it can help other people.
Define a list of Dictionaries:
LOCAL_APPS = [
{'app_name': 'app1', 'level': 'INFO'},
{'app_name': 'app2', 'level': 'DEBUG'},
]
Now create a function to modify settings' LOGGING:
def create_app_logger(app_name, level):
app_handler = app_name + '_handler'
LOGGING['handlers'][app_handler] = {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': f'../logs/{app_name}_logs.log',
'formatter': 'custom',
}
LOGGING['loggers'][app_name] = {
'handlers': ['console', app_handler],
'level': level,
'propagate': False,
}
Finally loop through the list:
for dictionary in LOCAL_APPS:
create_app_logger(dictionary['app_name'], dictionary['level'])
Since an app can be a world on it's own, this way you'll have a log file for each app, plus you have control over the logging level you want. It could be further personalized of course.
Cheers

Related

Django (IIS) deployed, logging issue

I deployed django app on IIS, however my logging code that was working perfectly on local host, caused server 500 error...
Can I get any help please?
LOGGING = {
'version': 1,
'loggers': {
'django': {
'handlers': ['debuglog'],
'level': 'DEBUG'
},
'django.server': {
'handlers': ['errorlog'],
'level': 'ERROR'
}
},
'handlers': {
'debuglog': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': './logs/debug.log',
'formatter': 'simple',
},
'errorlog': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': './logs/error.log',
'formatter': 'simple',
}
},
'formatters': {
'simple': {
'format': '{levelname} {message}',
'style': '{',
}
}
}
Maybe IIS does not allow django to create log files and it needs permission to do so? If this is the case, how would I do that?
you can follow the below steps to deploy the Django site in iis:
1)Add the site with your project root path in iis.
2)Select site-> Doble clcik handler mappings. Click “Add Module Mapping…” from the actions selection on the right.
3)Set request path to * and Module to FastCgiModule.
set executable:
C:\Python39\Scripts\python.exeC:\Python39\Lib\site-packages\wfastcgi.py
4)Now back to the server node and select the fast cgi module.
add below environment variable:
DJANGO_SETTINGS_MODULE = “path to your settings”
WSGI_HANDLER = django.core.wsgi.get_wsgi_application()
PYTHONPATH = “path to your repo-root e.g. C:\inetpub\wwwroot\djangosite”
Do not forget to assign iis_iusrs and iusr permission to the site root folder and python folder.
check the log folder permission too.
https://learn.microsoft.com/en-us/visualstudio/python/configure-web-apps-for-iis-windows?view=vs-2019

Django LOGGING not printing to console and file

I have encountered a strange behavior of Django Loggers.
I am developing a front end application using Django. During the login service, I make some requests to certain components and use log.warning() calls to see the flow of the requests.
The logs worked perfectly, until I decided to add a LOGGING configuration to print the output of the logs in a file, as I want to deploy the application via Docker and I want to periodically check the log files.
When I added the following Django configuration concerning logging:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'detailed': {
'class': 'logging.Formatter',
'format': "[%(asctime)s] - [%(name)s:%(lineno)s] - [%(levelname)s] %(message)s",
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'formatter': 'detailed',
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': "{}/am.log".format(BASE_DIR),
'mode': 'w',
'formatter': 'detailed',
'level': 'INFO',
'maxBytes': 2024 * 2024,
'backupCount': 5,
},
},
'loggers': {
'am': {
'level': 'INFO',
'handlers': ['console', 'file']
},
}
}
The logging stops working. The file specified in the logging configuration, am.log, is indeed created but nothing gets printed to this file. Even the console logging does not take place.
I have taken this logging configuration from one of my Django projects for the backend of this application, and there it works perfectly. I really don't understand what I am doing wrong. Could you please help me or guide me in the right direction. I would be very grateful.
I wish you all a good day!
By using the key "am" in your 'loggers' configuration, you're defining one logger with name "am":
'loggers': {
'am': { # <-- name of the logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}
So to use that logger, you have to get it by that name:
logger = logging.getLogger("am")
logger.warning("This is a warning")
If you name your loggers by the name of the module in which you're running, which is recommended practice, then you need to define each module logger:
logger = logging.getLogger(__name__) # <-- this logger will be named after the module, e.g. your app name.
Then in your logging configuration you can specify logging behavior per module (per app):
'loggers': {
'my_app': { # <-- logging for my app
'level': 'INFO',
'handlers': ['console', 'file']
},
'django': { # <-- logging for Django module
'level': 'WARNING',
'handlers': ['console', 'file']
},
}
Or if you just want to log everything the same, use the root ('') logger, which doesn't have a name, just empty string:
'loggers': {
'': { # <-- root logger
'level': 'INFO',
'handlers': ['console', 'file']
},
}

Django loggin in AWS

I was just wondering what is the current best way to perform logging within Django and AWS.
I was looking at this article, which suggested writing to the following location, but I have found that this doesn't work:
/opt/python/log
Then use commands such as
command: chmod g+s /opt/python/log
command: chown root:wsgi /opt/python/log
I have also seen articles that suggest using watchtower, but I don't like the idea of adding my secret access keys into the code:
https://medium.com/#zoejoyuliao/plug-your-django-application-logging-directly-into-aws-cloudwatch-d2ec67898c0b
What is the current and best way to do this ?
Thanks
You can customise your logging in settings.py file
Add this to your loggers section.
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'APPNAME': {
'handlers': ['applogfile',],
'level': 'DEBUG',
},
}
add this to handlers section you can specify filename path.
'applogfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DJANGO_ROOT, 'APPNAME.log'),
'maxBytes': 1024*1024*15, # 15MB
'backupCount': 10,
},
With this call you are using your customised logger.
logger = logging.getLogger('APPNAME')

django logging in script near manage.py - messages are just ignored

I'm using Django 1.4.1.
There's a script tmp.py in the same direcotry as manage.py, it does some routime operatons and is started from cron with command like python /path/to/project/tmp.py
Here is tmp.py:
import os
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "MYPROJECT.settings")
if __name__ == "__main__":
import logging
logger = logging.getLogger('parser.test')
logger.info('Hello, world')
This script makes a log entry, but this entry is just ignored by logger.
When I put same 3 lines into any of my views, log entry appears in my log file as expected.
Logger config from setings.py:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format' : "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
'datefmt' : "%d/%b/%Y %H:%M:%S"
},
},
'handlers': {
'null': {
'level':'DEBUG',
'class':'django.utils.log.NullHandler',
},
'logfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': "/hosting/MYPROJECTS/logs/logfile",
'maxBytes': 50000,
'backupCount': 2,
'formatter': 'standard',
},
'console':{
'level':'DEBUG',
'class':'logging.StreamHandler',
'formatter': 'standard'
},
},
'loggers': {
'django': {
'handlers':['console'],
'propagate': True,
'level':'WARN',
},
'django.db.backends': {
'handlers': ['console'],
'level': 'DEBUG',
'propagate': False,
},
'parser': {
'handlers': ['console', 'logfile'],
'level': 'DEBUG',
},
}
}
Why this logger works only in views? Maybe logger is still not configured after os.environ.setdefault("DJANGO_SETTINGS_MODULE", "MYPROJECT.settings") in my tmp.py?
I can't solve this problem but I found alternative solution: write a custom admin command instead of separate script near manage.py
https://docs.djangoproject.com/en/dev/howto/custom-management-commands/
It logs works fine in admin commands.
I managed to get Django logging settings to work for my non-Django script.
First, let's orient to the Django directory: <myproject>/<src>/<myproject>/settings.py & <myproject>/<src>/manage.py
# FILE: <myproject>/<src>/scripts/save_report_2_mongo.py
[...]
import logging
logger = logging.getLogger(__name__)
[...]
At the top of my script I import logging. Then I create the logging object. In my case __name__ == scripts.save_report_2_mongo. If the OP's setup is anywhere near to mine, then by this example name != "main", and the logger is never instantiated. Right?
Finally, inside settings.py
# FILE: <myproject>/<src>/<myproject>/settings.py
[...]
LOGGING = {
[...]
'formatters' : {...},
'handlers' : {...},
'loggers': {
'django': {
'handlers': ['console', 'fileInfo', 'fileDebug'],
'level': 'DEBUG',
'propagate': True,
},
'scripts.save_report_2_mongo': {
'handlers': ['sr2m_handler'],
'level': 'DEBUG',
},
}
I believe this works because of this passage in Python docs: docs.python.org > Logging HOWTO > Advanced Logging Tutorial
Advanced Logging Tutorial
The logging library takes a modular approach and offers several
categories of components: loggers, handlers, filters, and formatters.
Loggers expose the interface that application code directly uses.
[...]
Logging is performed by calling methods on instances of the Logger
class (hereafter called loggers). Each instance has a name, and they
are conceptually arranged in a namespace hierarchy using dots
(periods) as separators. For example, a logger named ‘scan’ is the
parent of loggers ‘scan.text’, ‘scan.html’ and ‘scan.pdf’. Logger
names can be anything you want, and indicate the area of an
application in which a logged message originates.
A good convention to use when naming loggers is to use a module-level
logger, in each module which uses logging, named as follows:
logger = logging.getLogger(__name__)
This means that logger names track the package/module hierarchy, and
it’s intuitively obvious where events are logged just from the logger
name.
My emphasis--logger names track the package/module hierarchy. In this case I obtained the value of __name__ inside my script and then named the logger in the loggers section of settings.py.

How to write to a specific log file from management command?

I have a Django 1.3 application. I have several management commands. I want each command to write, to a different logfile, instead of default filename defined in settings.py. I run these commands as part of cron everyday. Our current logfile looks like the example given here https://docs.djangoproject.com/en/dev/topics/logging/. And, we use
logger = logging.getLogger(__name__)
Thanks
You'll need to do this by adding a logger and a handler for each package:
'handlers': {
'my_command_handler': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/path/to/my_command_log',
},
...
},
'loggers': {
'my_pkg.management.commands.my_command': {
'level': 'DEBUG',
'handlers': ['my_command_handler'],
},
...
}
You may also want to consider adding 'propagate': False to the command loggers, if you don't want the messages to get to other loggers.