django.request logger to find "Synchronous middleware … adapted" for Django async - django

I've set up a trial async view in my Django app but the view continues to render in sync. As per Django docs, I'm checking that my Middleware isn't causing the issue:
Middleware can be built to support both sync and async contexts. Some of Django’s middleware is built like this, but not all. To see what middleware Django has to adapt, you can turn on debug logging for the django.request logger and look for log messages about “Synchronous middleware … adapted”.
There's already been a Stack Overflow question to elaborate on using the logger to find which middleware is prevent async from working, but the answer is incomplete.
This is what I have in my settings, as per the above Stack Overflow answer:
settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django.request': {
'handlers': ['console'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'DEBUG'),
'propagate': False,
},
},
}
And my view:
views.py
from time import sleep
import asyncio
import logging
logger = logging.getLogger("info")
# --- Views
from django.db import transaction
#transaction.non_atomic_requests
async def async0(request):
#loop = asyncio.get_event_loop()
#loop.create_task(lets_sleep())
await asyncio.sleep(1)
logger.debug('in index')
logger.info('something')
return HttpResponse("Hello, async Django!")
I've restarted the built in Django server, but I don't see a log output anywhere. Where should I be looking ?

although you posted your view, the log is about the middleware, so you should have a sync only middleware to get the log you want. after this, you need to add root logger as well:
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"console": {
"class": "logging.StreamHandler",
},
},
"root": {
"level": "DEBUG",
},
"loggers": {
"django.request": {
"handlers": ["console"],
"level": "DEBUG",
},
},
}
this was enough for me to get the Synchronous middleware someMiddlewareOfYours adapted log msg

Related

Sending Logs to GCP from Django DRF application

I am trying to access the logs from my django app in GCP logging. I have thus far been unsuccessful.
Here is my logging config:
client = gcp_logging.Client.from_service_account_json(
json_credentials_path='logging_service_account.json')
client.setup_logging()
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {'format': '%(levelname)s : %(message)s - [in %(pathname)s:%(lineno)d]'},
'short': {'format': '%(message)s'}
},
'handlers': {
'stackdriver': {
'formatter': 'standard',
'class': 'google.cloud.logging.handlers.CloudLoggingHandler',
'client': client
},
'requestlogs_to_stdout': {
'class': 'logging.StreamHandler',
'filters': ['request_id_context'],
}
},
'filters': {
'request_id_context': {
'()': 'requestlogs.logging.RequestIdContext'
}
},
'loggers': {
'StackDriverHandler': {
'handlers': ['stackdriver'],
'level': "DEBUG"
},
'django.request': {
'handlers': ['stackdriver']
},
'requestlogs': {
'handlers': ['requestlogs_to_stdout'],
'level': 'INFO',
'propagate': False,
},
},
}
I invoke the logs along the lines of:
import logging
logger = logging.getLogger('StackDriverHandler')
class OrganisationDetail(generics.RetrieveUpdateDestroyAPIView):
///
def patch(self, request, pk, format=None):
try:
///
if serializer.is_valid():
serializer.save()
logger.info(f"PATCH SUCCESSFUL: {serializer.data}")
return Response(serializer.data)
logger.warning(f"PATCH Failed: {serializer.errors}")
return JsonResponse(serializer.errors, status=400)
except Exception as e:
logger.error(f"PATCH Failed with exception: {e}")
return JsonResponse({'error': str(e)}, status=500)
In GCP, I set up a service account, enabled logging api and gave the SA write logs and monitor metrics permissions.
I then made a secret to contain my service_account key, and in my cloud-build.yaml file I run a step like this:
- name: gcr.io/cloud-builders/gcloud
entrypoint: 'bash'
args: [ '-c', "gcloud secrets versions access latest --secret=<secret_name> --format='get(payload.data)' | tr '_-' '/+' | base64 -d > logging_service_account.json" ]
The above step should:
Fetch the secret
Write it to a json file in the app instance container that can be accessed by my settings.py file with gcp_logging.Client.from_service_account_json( json_credentials_path='logging_service_account.json')
Perhaps there is a more straight forward way to achieve this, but it feels like it should work. Any help would be much appreciated. Thanks
After all the steps above, when I visit the logging service on my gcp console, I only see the one log under my logging service account that says it is created, none of the logs from my actual django app.

Django.request is not showing synchronous middleware as docs suggest

I've set up a very simple asynchronous view but it's not working. As per the Django instructions I want to check that it's not my middleware causing the issue. The Django docs say that the django.request logger will disclose which middleware is not working in async. Below is the quote from the official docs. I've set up the django.request logger and it logs an 4xx or 5xx errors (as expected) but that's all. Is this a mistake in the Django docs?
https://docs.djangoproject.com/en/4.0/topics/async/
You will only get the benefits of a fully-asynchronous request stack if you have no synchronous middleware loaded into your site. If there is a piece of synchronous middleware, then Django must use a thread per request to safely emulate a synchronous environment for it.
Middleware can be built to support both sync and async contexts. Some of Django’s middleware is built like this, but not all. To see what middleware Django has to adapt, you can turn on debug logging for the django.request logger and look for log messages about “Synchronous middleware … adapted”.
settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
},
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'logs/debug.log',
},
},
'root': {
'handlers': ['console'],
'level': 'WARNING',
},
'loggers': {
'django.request': {
#'handlers': ['file','console'],
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}

How configure Django logging to file for app

I am struggling with Django logging configuration. I have one app called "api" and I want to save to file all logs from this app. When I set up a logger to django everything works fine but when I change it to my app_name it doesn't.
Here is my configuration:
File structure:
email_api
api
tasks.py
email_api
celery.py
settings
logs
email.log
My logging configuration:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': 'logs/email.log',
},
},
'loggers': {
'api': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
tasks.py file where I logging:
import logging
logger = logging.getLogger(__name__)
#app.task(bind=True, default_retry_delay=3, max_retries=3,)
def send_email(self, data, email_id):
message = create_message(data, email)
try:
logger.debug("Log Message Here")
message.send()
Keys in the LOGGING['loggers'][...] dict are names of loggers. You have configured logging with api as a name of the logger.
In order to write to this logger, you should request it by that name:
logger = logging.getLogger('api')
...
logger.debug("Log Message Here")

How to setup django logging to console

I know there is a very similar question. That one is six years old and the answer in that one doesn't help me. All I want is to know how to configure django so that it can log to the console.
This are my settings:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django': {
'handlers': ['console'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
},
},
}
And in my view I have this:
class Home(TemplateView):
template_name = "inicio/magic_py.html"
def get_context_data(self, **kwargs):
logger = logging.getLogger("django")
logger.debug("home!!!!!!")
print("home?")
The console doesn't show the log.debug, it only shows the print. What am I missing? please help.
I use django 1.10
DEBUG log level is lower than INFO so your logs are being filtered out, you'll need to either lower your log level to DEBUG or you need to log using logger.info() or higher.

django: How can I create a custom Logging Filter for SuspiciousOperation exception?

After migrate to 1.11 ( from 1.8 ) I'm receiving some SuspiciousOperation errors from logging.
It seems it comes from JS request who keeps session alive if user move their mouse. But this is not important.
How can I filter just this exception?
What I tried:
I just created a filter somewhere:
import logging
from django.core.exceptions import SuspiciousOperation
class StopSuspiciousOperation(logging.Filter):
def filter(self, record):
if record.exc_info:
exc_value = record.exc_info[1]
return isinstance(exc_value, SuspiciousOperation)
return True
Then I added this filter to my configuration:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse',
},
'stop_suspicious_operation': {
'()': 'aula.utils.loggingFilters.StopSuspiciousOperation',
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false',
'stop_suspicious_operation',], #<-- here
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
But I'm still receiving the error:
Internal Server Error: /keepalive
SuspiciousOperation at /keepalive
The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example.
Request Method: GET
Request URL: https://XXXXXX/YYYYYY
Django Version: 1.11.9
Python Executable: /usr/bin/python
Python Version: 2.7.3
I am not sure about the correct answer, but I think django is catching the SuspiciousOperation at WSGI level and is logging an ERROR. See:
https://docs.djangoproject.com/en/dev/ref/exceptions/#suspiciousoperation
If a SuspiciousOperation exception reaches the WSGI handler level it
is logged at the Error level and results in a HttpResponseBadRequest.
You maybe just want to filter out the bad requests like this:
from logging import LogRecord
def filter_400(record: LogRecord) -> bool:
'''Filter out HTTP Error Code 400 Bad Request'''
return record.status_code != 400