I try to implement logging in my django project (Django/Postgresql/Docker/Celery) but got an error when I deploy my project in linux server (but it work locally)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/src/app/logs/debug.log'
I've read about a solution in this SO post: Django: Unable to configure handler 'syslog'
but fisrt I do not even understand why this solution should works and I do not manage to implement it in my project.
I use Docker/docker-compose so I install netcat with Dockerfile and then try to RUN nc -lU /logs/debug.log /logs/info.log but it failed (no such file or directory)
Dockerfile
# Pull the official base image
FROM python:3.8.3-alpine
# Set a work directory
WORKDIR /usr/src/app
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update && apk add postgresql-dev gcc g++ python3-dev musl-dev netcat-openbsd
# create UNIX socket for logs files
#RUN nc -lU /logs/debug.log /logs/info.log
...
settings.py
...
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'loggers': {
'django': {
'handlers':['files_info','files_debug'],
'propagate': True,
'level':'INFO',
},
},
'handlers': {
'files_info': {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': './logs/info.log',
'formatter': 'mereva',
},
'files_debug': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': './logs/debug.log',
'formatter': 'mereva',
},
},
'formatters': {
'mereva': {
'format': '{levelname} {asctime} {module} {message}',
'style': '{',
}
},
}
...
In fact, logs folder and .log files were excluded with my .gitignore files.
Changing to logsfiles folder and debug and info (without extension) files, resolve the problem and I was able to run container
neverthless, nothing is written in theses files...
Related
edit: Added in Django logging, still does not push errors to gunicorn from Django.
I have a django project deployed to Heroku in a worker using gunicorn. I have included all flags/configurations that I am using below. The issue is that the logging in Heroku does not display any output from Django, and my project is running into some 502 errors, that I am unsure of how to debug. Gunicorn just is not displaying any logs whatsoever from Django/python.
Procfile:
release: python manage.py migrate backend
web: bin/boot
worker: gunicorn backend.wsgi:application -b 0.0.0.0:$PORT -c /app/gunicorn.conf.py --log-file -
gunicorn.conf.py
# gunicorn.conf.py
# Non logging stuff
workers = 3
# Whether to send Django output to the error log
capture_output = True
# How verbose the Gunicorn error logs should be
loglevel = "debug"
Logging in settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'verbose': {
'format': '%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(module)s %(process)d %(thread)d %(message)s'
}
},
'handlers': {
'gunicorn': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'formatter': 'verbose',
'filename': '/opt/djangoprojects/reports/bin/gunicorn.errors',
'maxBytes': 1024 * 1024 * 100, # 100 mb
}
},
'loggers': {
'gunicorn.errors': {
'level': 'DEBUG',
'handlers': ['gunicorn'],
'propagate': True,
},
}
}
I deployed django app on IIS, however my logging code that was working perfectly on local host, caused server 500 error...
Can I get any help please?
LOGGING = {
'version': 1,
'loggers': {
'django': {
'handlers': ['debuglog'],
'level': 'DEBUG'
},
'django.server': {
'handlers': ['errorlog'],
'level': 'ERROR'
}
},
'handlers': {
'debuglog': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': './logs/debug.log',
'formatter': 'simple',
},
'errorlog': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': './logs/error.log',
'formatter': 'simple',
}
},
'formatters': {
'simple': {
'format': '{levelname} {message}',
'style': '{',
}
}
}
Maybe IIS does not allow django to create log files and it needs permission to do so? If this is the case, how would I do that?
you can follow the below steps to deploy the Django site in iis:
1)Add the site with your project root path in iis.
2)Select site-> Doble clcik handler mappings. Click “Add Module Mapping…” from the actions selection on the right.
3)Set request path to * and Module to FastCgiModule.
set executable:
C:\Python39\Scripts\python.exeC:\Python39\Lib\site-packages\wfastcgi.py
4)Now back to the server node and select the fast cgi module.
add below environment variable:
DJANGO_SETTINGS_MODULE = “path to your settings”
WSGI_HANDLER = django.core.wsgi.get_wsgi_application()
PYTHONPATH = “path to your repo-root e.g. C:\inetpub\wwwroot\djangosite”
Do not forget to assign iis_iusrs and iusr permission to the site root folder and python folder.
check the log folder permission too.
https://learn.microsoft.com/en-us/visualstudio/python/configure-web-apps-for-iis-windows?view=vs-2019
I was just wondering what is the current best way to perform logging within Django and AWS.
I was looking at this article, which suggested writing to the following location, but I have found that this doesn't work:
/opt/python/log
Then use commands such as
command: chmod g+s /opt/python/log
command: chown root:wsgi /opt/python/log
I have also seen articles that suggest using watchtower, but I don't like the idea of adding my secret access keys into the code:
https://medium.com/#zoejoyuliao/plug-your-django-application-logging-directly-into-aws-cloudwatch-d2ec67898c0b
What is the current and best way to do this ?
Thanks
You can customise your logging in settings.py file
Add this to your loggers section.
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'APPNAME': {
'handlers': ['applogfile',],
'level': 'DEBUG',
},
}
add this to handlers section you can specify filename path.
'applogfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DJANGO_ROOT, 'APPNAME.log'),
'maxBytes': 1024*1024*15, # 15MB
'backupCount': 10,
},
With this call you are using your customised logger.
logger = logging.getLogger('APPNAME')
I am using python logging in django and I want to log errors to var/log folder. I am getting this error :
Unable to configure handler 'exception_logs': [Errno 2] No such file or directory: '/var/log/exceptions.log'
Below is my code snippet in settings.py
'handlers': {'exception_logs': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'formatter': 'simple',
'filename': '/var/log/exceptions.log',
},
},
First, make sure the file has been created
sudo touch /var/log/exceptions.log
Then change the ownership of the file to the user that the server runs as. Assuming this is www-data:
sudo chown www-data /var/log/exceptions.log
Alternatively, you could change the group owner of the file, and add the user to that group.
My Django Logger works fine on my local machine but does not work when I deploy it to the server.
Local: OS X 10.9.5, Python 2.7.5, Django 1.6.2
Server: Ubuntu 12.04, Apache 2.2.22, mod_wsgi Version: 3.3-4ubuntu0.1, Python 2.7.3, Django 1.6
On my local setup I run python manage.py syncdb, this creates the survey.log file which I then follow with tail -f survey.log so I can see the error messages as they are created.
On my server I run python manage.py syncdb, this creates the survey.log file which I then follow with tail -f survey.log. However I can not see any of my debug messages and when I inspect the file with nano it is empty.
Why is no logging data being recorded into survey.log on my production environment? What am I missing?
views.py
import logging
logger = logging.getLogger(__name__)
logger.debug('This is your images list in 7: %s', images)
settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'applogfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
#'filename': os.path.join(DJANGO_ROOT, 'survey.log'),
'filename': 'survey.log',
'maxBytes': 1024*1024*15, # 15MB
'backupCount': 10,
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'survey': {
'handlers': ['applogfile',],
'level': 'DEBUG',
},
}
}
EDIT
I have now discovered that my Django error logs are getting written to my Apache error logs. Is this in any way normal?
Running
sudo tail /var/log/apache2/error.log
Provides me with the expected print out that I should be getting in my above Django error log file. e.g.
[15/Dec/2014 21:36:07] DEBUG [survey:190] This is your images list in 7: ['P3D3.jpg', 'P1D1.jpg', 'P5D5.jpg']
You aren't using the correct logger in your views.py. Try this:
import logging
logger = logging.getLogger('survey')
logger.debug('This is a debug message.')
The logger you get must match the loggers defined in LOGGING.
My guess is that the logfile actually gets created but in a directory that you don't except it to be. Try to put a full path in your configuration. Something like 'filename': '/tmp/survey.log',, restart apache and check if it's there. Of course first you must apply a solution that Derek posted.
The reason you see the log message in your apache logs is probably because mod_wsgi configures somehow a default (root) logger and your configuration doesn't disable it (you have 'disable_existing_loggers': False,).
if you don't pass the full path of the filename, it is created in the directory the running process is started ( or make a sys/chdir), it depends on the operating system which dir is used