I have cloudwatch log message "LogMonitor" from lambda log group which has all properties to apply filter. It have two timestamps, startTime and endTime and I want to find the difference (endTime - startTime) to know the response time of the lambda.
Query in cloudwatch log insights :
filter #message like /"LogMonitor"/
| LogMonitor.Id as Id,
LogMonitor.StartTime as startTime,
LogMonitor.EndTime as endTime
by Id
What is the best approach to find difference of timestamps in query without using custom metrics?
I think you should be able to run something like this:
filter #message like /"LogMonitor"/
| LogMonitor.Id as Id,
LogMonitor.StartTime as startTime,
LogMonitor.EndTime as endTime
| stats avg(endTime - startTime) by Id
Related
I have two different log groups and am retrieving details in different ways.
Once per:
fields #timestamp, message.event.detail.myIdentifier as placeA.myIdentifier
And once with
fields #timestamp
| parse message.event.Records.0.body '"myIdentifier ": "*"' as placeB.myIdentifier
Is there a way to join these log entries by myIdentifier?
Ideally the goal would be to compare the timestamps of all places per identifier.
So e.g.:
myIdentifier
placeA-timestamp
placeB-timestamp
First
12:00:00
13:00:00
Second
12:05:00
13:05:00
Is there a way to achieve this with cloud watch log insights?
Thanks for your help!
Hope you're well. I've been trying to put together a CloudWatch Query that returns the first event in each contactId.
I thought I'd add a count stat, and then exclude all events that were equal to or greater than 2. I'm clearly not doing something right though. Although I am being provided with the count, it seems for some reason that the count is excluding other information from the query. The query returns almost no information on the event that it is counting. I'd like the count to be added, and also INCLUDE the information from the query.
Here is the query I am using:
fields #timestamp, #message
| sort number asc
| stats count(ContactId) as number by ContactId
| filter ContactFlowModuleType = 'SetLoggingBehavior' and Parameters.LoggingBehavior = 'Enable'
| fields #message
| display Results, ContactId, #timestamp, ContactFlowModuleType, number
With this query, it says that 'time stamp' is invalid. I believe the stats clause has something to do with it.
I'm looking to determine the sequence of events on a contactId basis, so that I can exclude all logged events after the initial event. For now, I'd just like to see a count on the basis of ContactId, so I can perform the exclusion myself.
Steve
I'm running AWS lambda. And I should find some informations from the Cloudwatch logs.
And What am I doing seems to too inefficient. But I don't know how to work.
I want to know more efficient way.
What am I doing is...
I have some ids
1111-1111-111
2222-2222-222
3333-3333-333
...
Search for specific messages with id in AWS log insight conolse
fields #timestamp, #logStream ,#message
| filter #message like /myId/
| sort #timestamp desc
| parse #message '"myId" : "*"' as my_id
| filter my_id like /1111-1111-111/
Download result csv file.
Parse #logStream with python
with open('1111-1111-111.csv') as csvfile:
reader = csv.DictReader(csvfile, delimiter=',')
for row in reader:
print(str(i) +": " +row['#logStream'])
get logStreams and search again in logInsight console
2021/06/05/[$LATEST]1111111111111
2021/06/05/[$LATEST]1111111111111
2021/06/05/[$LATEST]1111111111111
2021/06/05/[$LATEST]2222222222222
2021/06/05/[$LATEST]3333333333333
2021/06/05/[$LATEST]3333333333333
2021/06/05/[$LATEST]3333333333333
2021/06/05/[$LATEST]3333333333333
2021/06/05/[$LATEST]4444444444444
...
Search again with logStreams and get what I really want.
fields #timestamp, #logStream ,#message
| filter #logStream='2021/06/05/[$LATEST]1111111111111'
| filter #message like /file_name/
| parse #message "'file_name': '*'" as file_name
After getting file_name, I should search again inside file with myId. Because I'm not sure because of same logStreams.
If I do this manually, This is too hard. And If I do this with aws boto3 it's also hard for me because I'm not familiar with boto3 logs client wait process and result. Also I think there would be better way.
Could you suggest to me better workflow?
I'm trying to create a CloudWatch Insights query for Amazon Connect that will give me call counts by date. I'm able to get the number of log messages by date, however, I need to only count unique ContactId's. The query I have has many duplicated ContactId's since each time Connect logs to CloudWatch, it uses ContactId to tie all of the events related to a contact together. Is there a way to modify this query to only show the count of the unique ContactId?
filter #message like /ContactId/
| stats count(*) as callCount by toMillis(datefloor(1d))
| sort callCount desc
Embarassingly enough, almost immediately after posting this, I found my answer. count_distinct() gets me what I needed.
filter #message like /ContactId/
| stats count_distinct(ContactId) as callCount by toMillis(datefloor(1d))
| sort callCount desc
I have a startTime and endTime field in a stream table. My scan parameter is:
{"TableName":"streams","Limit":"10","FilterExpression":"startTime >= :start_time AND endTime <= :end_time","ExpressionAttributeValues":{":start_time":{"N":"1449446488"},":end_time":{"N":"1449453688"}}}
My data startTime is 1449448560 and endTime is 1449452160.
I don't understand why it does not work when running in AWS Lambda function.
Thanks,
It works after I removed the "Limit" field.