I have a use case where I'm expecting some write volume to my Dynamodb table everyday around 3pm from a kinesis stream, the streaming takes about 20 mins.
So in order to make sure the kinesis successfully put data to my Dynamodb table, I want to use CloudWatch to monitor such behavior on my Dynamodb and configure a alarm saying, if everyday around 3pm to 4pm, there's no write traffic to my Dynamodb table, alarm.
Does Cloudwatch support such use case?
I have tried to set up an alarm saying I'm expecting write volume to my table on a daily basis, if not, alarm. However, I want to shorten this period since I know if there's no write volume around 3pm, something must be wrong and actions need to be taken. I don't want to wait till the day is over.
I know you can use math on metric, but after reading the document: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html, I don't think my use case is supported, unless I missed something?
Appreciate any suggestions here! Thanks!
Related
I'm trying to monitor the change in traffic on our application between days. We currently have a metric plotting the sum of 1 day between sequential days, however this comes with a 24 hr delay as we need to wait for the current day's traffic.
I'm trying to figure out a way to produce a graph that sums data over a 1hr period, and subtracts the same datapoint from 24 hrs in the past.
I can't figure out a way to do this with metric math. Anyone have any suggestions? Preferably within AWS, thanks!
You can set up an export of the CloudWatch logs into S3. Then you can use Athena to write SQL queries of arbitrary complexity, which you can then visualize using QuickSight.
It is not going to be as near-realtime as CloudWatch: there may be about a few hours' lag in S3 export. For a real-time solution, you may consider streaming logs into a Kinesis Datastream and then create a custom Lambda to process the logs and push them into a QuickSight dataset.
Is there a quick way to check how many data (volume wise, GBs, TBs etc) did my specific DMS task transfered for example within last month?
I can't find any note in the documentation on that, I could probably try with boto3 but want to double check first. Thanks for help!
Even with Boto3, you can check the API - DescribeReplicationTasks but likely, there is no information about your data transfer.
Reference: https://docs.aws.amazon.com/dms/latest/APIReference/API_DescribeReplicationTasks.html
If you have only 1 data replication task that is associated with only 1 replication instance, you can check that replication instance's network metric via CloudWatch metric. From CloudWatch metrics, AWS DMS namespace, there will be several network metrics such as NetworkTransitThroughput or NetworkReceiveThroughput. You can choose one and try as below:
Statistic: Sum
Period: 30 days (or up to you)
And you have a 30DAYS_THROUGHPUT.
I'm trying to identify the initial creation date of a metric on CloudWatch using the AWS CLI but don't see any way of doing so in the documentation. I can kind of identify the start date if there is a large block of missing data but that doesn't work for metrics that have large gaps in data.
CloudWatch metrics are "created" with the first PutMetricData call that includes the metric. I use quotes around created, because the metric doesn't have an independent existence, it's simply an entry in the time-series database. If there's a gap in time with no entries, the metric effectively does not exist for that gap.
Another caveat to CloudWatch metrics is that they only have a lifetime of 455 days, and individual metric values are aggregated as they age (see specifics at link).
All of which begs the question: what's the real problem that you're trying to solve.
DynamoDB with Autoscaling enabled will not scale down when there is no activity to the table. This is because it uses a CloudWatch alarm that requires there to be activity to the read or write units consumed (depending on which type of scaling you want)
So how do I get DynamoDB to scale down when there is no activity to the table?
The workaround I came up with is to schedule a Lambda function to run once a minute that performs one trivial read or write (or both) to the table for a key that I don't care about.
I'm hoping Amazon will have a better solution for this use case in the future.
Need to call a function at specific times without having a server up and running all the time
In particular, the challenge I'm facing is that we only use AWS Lambda and DynamoDB to - among other things - send a reminder to users at a time of their choice. That means we have to call a lambda function at the time the user wants to be reminded.
The time changes dynamically (depending on each user's choice) so the question is, what is a good way to set this up?
We are considering setting up a server if there's no way around it but even if we go for this solution, I lack the experience to see a good way to set this up. Any suggestions are greatly appreciated.
You can use AWS DynamoDB TTL event stream to trigger Lambda to achieve this. The approach is as follows.
Create a DynamoDB table to store User alarms.
When user setup an alarm, calculate the difference between the alarm timestamp and current timestamp.
Then store the difference as the TTL value of the alarm record, along with alarm information.
Configure DynamoDB streams to trigger a Lambda when TTL exceeds
You can call your Lambda function on a scheduled event:
http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html
So set up your Lambda function with cron like event to wake on any interval you need, retrieve the list of alarms you need to send next, send them, mark completed alarms so they won't be triggered again.