I'm setting up an auto-scaling group and to dynamically tag instances, is it possible to auto-increment name tag?
for eg:-
Key: Name Value: Instance-1
Key: Name Value: Instance-2
.
.
.
Key: Name Value: Instance-N
Yes, it possible. But not directly using any AWS functionality.
You have to embed a function where in as the instance is launched by AutoScaling Group a notification (AWS SNS) is triggered and is stored in SQS.
You will have to use a Program/Script which runs on a server and keeps a constant watch on the SQS, get the latest notification and Information regarding the Instance. Then change the Tag Name of the instance using a incremental functionality.
Related
In the AWS Cloud Environment i have to the get list of all Elastic IPs with instance name and in all region/AZs like eu-west-1 , eu-west-3, ap-south-1, us-east-1 etc..
How this can be achieved via AWS System manager if possible. If not, then do i have to write any other function/code or use other AWS functionality.
I would do it by writing a program (eg in Python) with these steps:
Loop through each (applicable) Region using describe_regions()
Loop through each Elastic IP address using describe_addresses()
Extract the InstanceId and PublicIp from the response
Call describe_instances() for the given InstanceId to obtain a list of Tags. The "instance name" is stored as a Tag with Key = 'Name'
I am faced with the following situation:
There is an EC2 instance on say eu-west-1.
When selecting Snapshots on the EC2 service, I see that periodically, every 7 days on the exact same time, a snapshot is taken from the particular image.
The problem is I cannot find:
any related policy on Lifecycle Manager service
any relevant Lambda function that could carry out such a task.
Via what other (managed) means could such a process be carried out periodically with such an accuracy on time?
edit: The corresponding CloudTrail log entry is:
(actual values regarding user, event and request id have been scrambled of course)
AWS access key:
AWS region: eu-west-1
Error code:
Event ID: 454g0236-x4e6-43c1-3565-4xb6d541c2h1
Event name: CreateSnapshot
Event source: ec2.amazonaws.com
Event time: 2019-11-23, 05:00:44 AM
Read only: false
Request ID: zedfbc42-2513-459e-3241-ffcb8442ba44
Source IP address: events.amazonaws.com
User name: g45tg34m3l53mmm53333421knbb43
There are multiple other options,
Check Cloudwatch events, if there is any event triggering. Most probably this one is in your case.
Cronjob on an EC2 instance.
If i understood you question you are looking for a way to know if Lifecycle Manager is available for EC2 snapshots.
Below given links should be able to help you on the same.
For enabling a custom Snapshot Lifecycle policy manually refer Snapshot Lifecycle
For automating a solution for the same please referautomation of snapshot lifecycle
I would like to perform the following operations in order with CloudFormation.
Start up an EC2 instance.
Give it privileges to access the full internet using security group A.
Download particular versions of Java and Python
Remove its internet privileges by removing security group A and adding a security group B.
I observe that there is a DependsOn attribute for specifying the order in which to create resources, but I was unable to find a feature that would allow me to update the security groups on the same EC2 instance twice over the course of creating a stack.
Is this possible with CloudFormation?
Not in CloudFormation natively, but you could launch the EC2 instance with a configured userdata script that itself downloads Java/Python and the awscli, as necessary, and then uses the awscli to switch security groups for the current EC2 instance.
However, if all you need is Java and Python pre-loaded then why not simply create an AMI with them already installed and launch from that AMI?
The best way out is to utilise a Cloudformation custom resource here. You can create a lambda function that does exactly what you need. This lambda function can then be called as a custom resource function in the cloud formation template.
You can pass your new security group ID and instance ID to the lambda function and code the lambda function to use AWS SDK and do the modifications that you need.
I have leveraged it to post an update to my web server about the progress of the cloud formation template. Below is the sample code of the template.
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
MarkInstanceProfileComplete:
Type: 'Custom::EC2InstanceProfileDone'
Version: '1.0'
DependsOn: EC2InstanceProfile
Properties:
ServiceToken: !Ref CustomResourceArn
HostURL: !Ref Host
LoginType: !Ref LoginType
SecretId: !Ref SecretId
WorkspaceId: !Ref WorkspaceId
Event: 2
Total: 3
Here the resource MarkInstanceProfileComplete is a custom resource that calls a Lambda function. It takes the event count and total count as input and processes them to calculate percentage progress. Based on that it sends out a request to my web server. For all we care, this Lambda function can do potentially anything you want it to do.
It is possible to configure an autoscaling group to use Launch Template instead of Launch Configuration. I want to use it because when I configure an autoscaling group using Launch Configuration I can not set mixed instances policy (ondemand + spot instances).
I have created launch template (name 'test') with such configuration:
AMI Ubuntu 16,
some security groups and IAM Role with EC2, AutoScaling and CodeDeploy full access
and all other settings in default *Not specified* state.
Launch Template ID lt-0e1111
Default Version Number 1
Launch Template Name test
Owner arn:aws:iam::11111111:user/itsp
Version 1 (Default)
Name test
Created by arn:aws:iam::179961111910:user/itsp
Description Not specified
Date Created February 8, 2019 at 12:47:54 AM UTC+3
AMI ID ami-0bf1924ed30c48a6a
Availability Zone Not specified
Security Groups Not specified
Instance Type t2.nano
Key pair name itsp
Security Group Ids sg-07e746cf0e4935b84,sg-0a45f7e6ac321614d,sg-27bf6d4b
Advanced Details:
Purchasing option Not specified
Spot request type Not specified
Spot interruption behavior Not specified
IAM Instance Profile Not specified
Stop - Hibernate behavior Not specified
Monitoring Not specified
EBS-optimized instance Not specified
Tenancy Host ID Not specified
RAM Disk ID Not specified
User Data Not specified
Elastic Inference Accelerator Not specified
Maximum Spot price Not specified
Spot expiry date -
Spot block duration Not specified
Shutdown behavior Not specified
Termination Protection Not specified
Placement Group Name Not specified
Tenancy Not specified
Tenancy Affinity Not specified
Kernel ID Not specified
Elastic Graphics Not specified
Then I am creating an AutoScaling group with that launch template. Here is my autoscaling group configuration:
Launch Template test
Launch Template Version 1
Launch Template Description -
Instance Types -
Spot Diversity 0
Optional On-Demand Base 0
On-Demand Percentage 0%
Desired Capacity 1
Min 1
Max 1
Availability Zone(s) eu-central-1a
Subnet(s) subnet-53075e38
Classic Load Balancers -
Target Groups -
Health Check Type EC2
Health Check Grace Period 300
Instance Protection -
Termination Policies Default
Suspended Processes -
Placement Groups -
Default Cooldown 300
Enabled Metrics -
Creation Time Fri Feb 08 00:50:16 GMT+300 2019
Service-Linked Role arn:aws:iam::11111111111:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling
Such configuration appears to be set when I choose
*Fleet Composition - Adhere to the launch template
The launch template determines the instance type and purchase option (On-Demand or Spot).*
option and my lauch template configuration is described above.
So there are no spots in my autoscaling group.
Then I configure a CodeDeploy application 'Application' and DeploymentGroup 'DG_TEST' inside of it with such configuration:
Deployment type Blue/green
Service role ARN arn:aws:iam::11111111111111:role/CodeDeployRole
Deployment configuration CodeDeployDefault.AllAtOnce
Rollback enabled -
Environment configuration: Amazon EC2 Auto Scaling groups -> Name 'test'
Ok, after everything is set up I set desired instances count to 1 in my Autoscaling group and new on-demand instance spawns in my 'test' group.
I am now starting a blue-green deployment with AWS CLI command:
aws deploy create-deployment --application-name Application --s3-location bucket=codedeploybucket,key=application,bundleType=zip --deployment-group-name DG_TEST
And I can see new autoscaling group with same lauch template appears in my green fleet.
So we just checked all our services are working as expected.
Now I want to tweak my autoscaling group configuration a little to mix ondemand and spot instances in it:
Fleet Composition - Combine purchase options and instances
Choose a mix of On-Demand Instances and Spot Instances and multiple instance types. Spot Instances are automatically launched at the lowest price available.
Instance Types:
t2.nano (1vCPUs, 0.5GiB)
t2.small (1vCPUs, 2GiB)
t2.micro (1vCPUs, 1GiB)
Maximum Spot Price - Use default (recommended)
Spot Allocation Strategy - Diversify Spot Instances across your 2 lowest priced instance types per Availability Zone
Optional On-Demand Base - Designate the first 0 instances as On-Demand
On-Demand Percentage 50% On-Demand and 50% Spot
Desired Capacity 1
Min 1
Max 1
I save it and try same AWS CLI command
aws deploy create-deployment --application-name Application --s3-location bucket=codedeploybucket,key=application,bundleType=zip --deployment-group-name DG_TEST
And now I have my deployment failed with such comment:
The following validation error occurred: Valid requests must contain either LaunchTemplate, LaunchConfigurationName, InstanceId or MixedInstancesPolicy parameter. (Service: AmazonAutoScaling; Status Code: 400; Error Code: ValidationError; Request ID: 11111114-2b2b-11e9-93b7-11111111111)
Another strange thing is that if I configure my Launch Templae to use spots only:
Launch Template ID lt-00dc4c187d1111159
Default Version Number 1
Launch Template Name test2
Owner
arn:aws:iam::179961111910:user/itsp
Version 1 (Default)
Name test2
Created by arn:aws:iam::179961111910:user/itsp
Description Not specified
Date Created February 8, 2019 at 1:14:13 AM UTC+3
AMI ID ami-0bf1924ed30c48a6a
Availability Zone Not specified
Security Groups Not specified
Instance Type t2.micro
Key pair name itsp
Security Group Ids sg-0a45f7e6ac321614d,sg-07e746cf0e4935b84,sg-27bf6d4b
Advanced details
Purchasing option spot
Spot request type one-time
Spot interruption behavior Not specified
IAM Instance Profile CodeDeployRole (arn:aws:iam::1111111111:instance-profile/CodeDeployRole)
Stop - Hibernate behavior Not specified
Monitoring Not specified
EBS-optimized instance Not specified
Tenancy Host ID Not specified
RAM Disk ID Not specified
User Data Not specified
Elastic Inference Accelerator Not specified
Maximum Spot price Not specified
Spot expiry date -
Spot block duration Not specified
Shutdown behavior Not specified
Termination Protection Not specified
Placement Group Name Not specified
Tenancy Not specified
Tenancy Affinity Not specified
Kernel ID Not specified
Elastic Graphics Not specified
and use that option in my autoscaling group:
*Fleet Composition - Adhere to the launch template
The launch template determines the instance type and purchase option (On-Demand or Spot).*
Then after same deployment command I receive another error in CodeDeploy (failed deployment reason):
The IAM role arn:aws:iam::179966034910:role/CodeDeployRole does not give you permission to perform operations in the following AWS service: AmazonAutoScaling. Contact your AWS administrator if you need help. If you are an AWS administrator, you can grant permissions to your users or groups by creating IAM policies.
And that is definitely not the case because my role for CodeDeploy have full EC2, Autoscaling and CodeDeploy access.
Am I doing something wrong or that are bugs? How can I configure mixed autoscaling group (ondemand+spot instances) with codedeploy using launch templates? Thanks in advance!
I faced a similar issue when working with CodeDeploy and LaunchTemplates.
It looks like LaunchTemplates are still not supported by CodeDeploy.
We have to stick to LaunchConfigurations. They still don't support versioning, but for the most part, they are pretty much the same :/
I could't find where AWS has documented that little incompatibility issue when migrating LaunchConfigurations to LaunchTemplates... but could find here more info.
Sheers!
I have a Cloud Formation to set up an EC2 instance. I'm currently using the Parameters to specify the Subnet Id for the EC2 instance as well as the VPC Id for the Security Group (to be used in turn by the EC2 instance).
In my situation the Subnet Id specified is required to be part of the VPC and I'd like to only have to specify the Subnet Id in the Parameters. But I can't find a way to derive the VPC from the Subnet Id (http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet.html)
I see the GetAtt function can be used to return resource attributes. Is there something equivalent to return resource properties?
From the documentation Fn:GetAtt, you can only retrieve AvailabilityZone and Ipv6CidrBlocks details of the Subnet. There is no inbuilt support to get VpcId of the given subnet in CFn Templates.
There is a work-around though. If you are using the aws-cli documentation, you can use the describe-subnets method to fetch the VpcId of the required subnet and pass it as input to the Cloudformation template create_stack call.
This method works even if you are using any SDK. for example, in Java.
//pseudo code only!
DescribeSubnetsRequest request = new DescribeSubnetsRequest();
request.withSubnetIds("subnet-abcdefgh");
DescribeSubnetsResult result = awsClient.describeSubnets(request);
String myVpc = result.getSubnets().get(0).getVpcId();
// add the above VPC Id to the parameters of your Cloud formation template create stack request.
Hope this helps.
I created a small project called cli2cloudformation. Using that you're able to execute cli commands inside your cloudformation stack and use the results of the commands.
Just check it here. I hope it helps you.