Amazon CloudWatch Insights Query - amazon-web-services

Question

I have logs like this:
I, [2020-06-17T09:32:48.100103 #9] INFO -- : [54b35e04-9c19-443d-adff-b2c3192b5590] Completed 500 Internal Server Error in 7ms (ActiveRecord: 2.3ms | Allocations: 1705)
I, [2020-06-17T10:37:27.169909 #9] INFO -- : [c800e9ce-fba3-4e1a-a19f-526f32746925] Completed 401 Unauthorized in 0ms (ActiveRecord: 0.0ms | Allocations: 115)
So as you can see in message it is always fallowing the pattern: Completed [THE ERROR CODE] [ERROR MESSAGE] ...
I'm using this query to retrieve logs with certain error codes:
fields #timestamp, #message
| filter #message like /401/
| sort #timestamp desc
| limit 20
But how can I parse the message to get separate fields for error code and message?

Answer 1

You can use parse function with the regex syntax: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html
Like this for example:
parse #message /Completed (?<errorCode>\d+) (?<errorMessage>.+) in (?<timeMilis>\d+)ms /
| filter isPresent(errorCode)
Result will be something like this
-------------------------------------------------
| errorCode | errorMessage | timeMilis |
|-----------|-----------------------|-----------|
| 500 | Internal Server Error | 7 |
| 401 | Unauthorized | 0 |
-------------------------------------------------
This will just filter and extract the fields, you can do further processing from there.