Logstash conf file for parsing django exceptions - django

Question

I have been trying to use logstash, elastic search, and Kibana for monitoring my django server.
I have set the conf file as given below
input {
tcp { port => 5000 codec => json }
udp { port => 5000 type => syslog }
}
output {
elasticsearch_http {
host => "127.0.0.1"
port => 9200
}
stdout { codec => rubydebug }
}
But the messages logged are too lengthy and could not find a method to parse it.
Any help is appreciated

Answer 1

As far as I can tell, there is not a pattern or built-in that will directly parse Django exceptions.
You need to tell the forwarding agent to target the Django log files that you're generating, marking them as "type": "django".
Then, on the Logstash server, you can use the following:
pattern:
DJANGO_LOGLEVEL (DEBUG|INFO|ERROR|WARNING|CRITICAL)
DJANGO_LOG %{DJANGO_LOGLEVEL:log_level}\s+%{TIMESTAMP_ISO8601:log_timestamp}\s+%{TZ:log_tz}\s+%{NOTSPACE:logger}\s+%{WORD:module}\s+%{POSINT:proc_id}\s+%{GREEDYDATA:content}
filter:
filter {
if [type] == "django" {
grok {
match => ["message", "%{DJANGO_LOG}" ]
}
date {
match => [ "timestamp", "ISO8601", "YYYY-MM-dd HH:mm:ss,SSS"]
target => "#timestamp"
}
}
}
if you don't want to add the pattern file, you can expand the DJANGO_LOGLEVEL pattern into the %{DJANGO_LOGLEVEL:log_level} field and place the targeting rule that follows DJANGO_LOG into the grok match placeholder.