I'm wanting to apply a CloudWatch alarm to stop instances which aren't being used in our pre-production environment. We often have instances being spun up, used and then left turned on which is really starting to cost us a fair amount of money.
CloudWatch alarms have a handy feature whereby we can stop based on some metrics - this is awesome and what I'd like to use to constantly keep an eye on the servers with but let it tidy up the instances for me.
The problem with this is that it appears that the CloudWatch alarms need to be created individually against each instance. Is there a way in which I can create one alarm which would share values across all current and future instances which will be started?
ETA - Alternatively, tell me that these options are better than CloudWatch and I'll be happy at that.
AWS EC2 stop all through PowerShell/CMD tools
Add a startup script that creates the CloudWatch alarm to the base image you use to generate your VMs.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CLIReference.html
I don't believe this is possible - CloudWatch seems designed to be 'very manual' or 'very automated'. i.e. You can't setup one alarm which would go off if any one instance is idle, you have to setup individual alarms for each instance.
A couple of possible solutions, which are probably not what you want to hear:
Script your instance creation, and add a call to cloudwatch to create an alarm for each instance.
Run a service continually, which looks for instances and checks to ensure that there is an alarm for the instance, create alarms for the new instances, and remove alarms for instances which have been terminated.
I think what you are actually looking for would be auto-scaling:
https://aws.amazon.com/documentation/autoscaling/
Related
A common scenario is to have instances automatically created in EC2 via autoscaling groups or via custom CLI/SDK calls. I have configurations of both.
I wish to monitor either the individual memory usages or the maximum memory within the group of instances to trigger a CloudWatch alarm if it goes over a threshold. Previously I tried to create a metric maths search function to solve this, to no avail. The idea being that the function would amalgamate all instances that matched X and find the maximum.
How can custom metrics such as memory or others collected via the CloudWatch Agent be monitored via an alarm? Is dynamic creation/deletion upon autoscale trigger possible (hopefully avoiding orphans), or is there an alternative best practice to this?
I'm trying to setup the Cloudwatch Alarm for memory on all instances of an AWS Elastic Beanstalk environment. I've setup capability to get Memory usage on Cloudwatch using the following tutorial:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-cw.html
Now I want to setup an alarm that would trigger if the MemoryUtilization of any of these instances go beyond a certain threshold. I can select all and setup alert on each of those separately, but I want to make sure that even if Beanstalk scales up the cluster or swaps an instance, the alert doesn't have to be reconfigured.
Is there a way I can setup alarm for a condition where Instance Name = "env-name" and Metric is MemoryUtilization?
What I understand from your question are the following requirements:
You have multiple metrics and want to use a logical OR condition when configuring an alarm, e.g. (avg metric1 > x || avg metric2 > y) ==> set alarm state to ALARM
You want the alarm to consider new metrics as they become available when new instances are launched by elastic beanstalk during scale out.
You want old metrics to not be considered as soon as elastic beanstalk scales in.
I think this is currently not possible.
There is an ongoing discussion on aws discussion forums [1] which reveals that at least (1) is possible using Metric Math. The Metric Math feature supports max. 10 metrics.
Solution
What you need to do is, to create a single metric which transports the information whether the alarm should be triggered or not ('computed metric'). There are multiple ways to achieve this:
For complex metrics you could write a bash script and run it on an EC2 instance using cron. The script would first query existing metrics using a dimension filter ('list-metrics'), then gather each metric ('get-metric-data'), aggregate it and then push the computed metric data point ('put-metric-data').
If the metric is rather simple, you could try the aggregate option of the AWS put-metric-data script [2]:
option_settings:
"aws:elasticbeanstalk:customoption" :
CloudWatchMetrics : "--mem-util --mem-used --mem-avail --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --auto-scaling --aggregated"
The documentation for the aggregated option says:
Adds aggregated metrics for instance type, AMI ID, and overall for the region.
References
[1] https://forums.aws.amazon.com/thread.jspa?threadID=94984
[2] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html#put-metric-data
In the Elastic Beanstalk console for your environment:
Click the Monitoring link in the left-hand side navigation links.
Underneath the Overview, in the Monitoring section, click the Edit button.
Choose AWSEBAutoScalingGroup for the Resource.
Choose MemoryUtilization under CloudWatch Metric.
Modify Statistic and Description as desired.
Click the Add button, and then click the Save button in the Monitoring section.
Scroll down to find the new panel that was added. Click the bell icon in the upper right hand corner of the panel. This will take you to the settings to set up a new alarm.
If you do not see the MemoryUtilization metric available, verify that you have correctly set up the collection of the memory metrics.
Cloudwatch cannot create alarms in a generic way. There are only 2 ways to accomplish the task.
1) Create a startup script in your AMI. When a new instance is launched, it is responsible for its own Cloudwatch alarms. I used this a long time ago, and the approach is solid. However, running scripts on termination isn't reliable, so you'll have to periodically clean out the old alarms.
2) Use a tool that has decent capabilities (ahem.... not Cloudwatch). I recommend Blue Matador. With them, you don't even have to setup the alarms or thresholds, the machine learning automatically baselines your resources and creates alerts for you.
If you got here and don't know Beanstalk or Cloudwatch well enough to contribute, start here: How to Monitor AWS Elastic Beanstalk with CloudWatch
I want to create a web app for my organization where users can schedule in advance at what times they'd like their EC2 instances to start and stop (like creating events in a calendar), and those instances will be automatically started or stopped at those times. I've come across four different options:
AWS Datapipeline
Cron running on EC2 instance
Scheduled scaling of Auto Scaling Group
AWS Lambda scheduled events
It seems to me that I'll need a database to store the user's scheduled times for autostarting and autostopping an instance, and that I'll have to pull that data from the database regularly (to make sure that's the latest updated schedule). Which would be the best of the four above options for my use case?
Edit: Auto Scaling only seems to be for launching and terminating instances, so I can rule that out.
Simple!
Ask users to add a tag to their instance(s) indicating when they should start and stop (figure out some format so they can easily specify Mon-Fri or Every Day)
Create an AWS Lambda function that scans instances for their tags and starts/stops them based upon the tag content
Create an Amazon CloudWatch Event rule that triggers the Lambda function every 15 minutes (or whatever resolution you want)
You can probably find some sample code if you search for AWS Stopinator.
Take a look at ParkMyCloud if you're looking for an external SaaS app that can help your users easily schedule (or override that schedule) your EC2, RDS, and ASG instances. It also connects to SSO, provides an API, and shows you all of your resources across regions/accounts/clouds. There's a free trial available if you want to test it out.
Disclosure: I work for ParkMyCloud.
We have configured an Auto Scaling group in AWS. And it works fine. We have configured some alarms for the group, such as: send alarm if the average CPUUtilization > 60 for 2 minutes ... use AWS CLI.
The only problem is, if we want to monitoring each instance in the group. We have to configure them manually. Are they any way to do it automatically like config, template?
Amazon CloudWatch alarms can be created on the Auto Scaling group as a whole, such as Average CPUUtilization. This is because alarms are used to tell Auto Scaling when to add/remove instances and such decisions would be based upon the group as a whole. For example, if one machine is 100% busy but another is 0% busy, then on average the group is only 50% busy.
There should be no reason for placing an alarm on the individual instances in an auto-scaling group, at least as far as triggering a scaling action.
There is no in-built capability to specify an alarm that will be applied against each auto-scaled instance individually. You could do it programmatically by responding to an Amazon SNS notification whenever an instance is added/removed by Auto Scaling, but this would require your own code to be written.
You can accomplish this with lifecycle hooks and a little lambda glue. When you have lifecycle events for adding or terminating an instance, you can create an alarm on that individual instance or remove it (depending on the event) via a lambda function.
To John's point, this is a little bit of an anti-pattern with horizontal scaling and load balancing. However, theory and practice sometimes diverge.
Hi I am new to AWS and I started learning now. In my environment I noticed all the issues are getting raised by autoscaling only. Whenever new instances comes up there are some small issues happening in environment. So I thought if we have a script to trigger a alert whenever any ec2 instance get reboots or terminates or any new instance spins up but I do not have a idea how to achieve it.
While checking with AWS support they are pointing cloudwatch where we need to trigger alarm for each instance. When we have huge number of instances and multiple environments, setting up a alarm for individual instance is hectic job. Can anyone suggest how can I get information about a instance reboot, termination and new instance spin up. Thanks in advance.
You can create SNS notification whenever auto scaling scales.
Read AWS documentation
You can write your script in your favourite language which can subscribe to the particular SNS topic.