Create Cloudwatch Event for EBS Snapshot using Cloudformation - amazon-web-services

I am trying to create cloudwatch scheduled event for taking snapshot of my ebs. I am new to cloudformation not much familiar with it that's why having complexity in achieving this. I am attaching my current template which spawns my ec2 instance and override the default volume from 10gb to 20gb. I want to create a cloudwatch event on exactly the same created volume to take the snapshot of this volume that has been created from this template. I would be glad if anyone can help me in setting an event with target using the cloudformation syntax.
Parameters:
KeyName:
Description: The EC2 Key Pair to allow SSH access to the instance
Type: 'AWS::EC2::KeyPair::KeyName'
Resources:
Ec2Instance:
Type: 'AWS::EC2::Instance'
DependsOn:
- InstanceSecurityGroup
- CWIAMRole
- EC2CWInstanceProfile
Properties:
KeyName: !Ref KeyName
ImageId: ami-057a963e8be173b19
InstanceType: t3a.micro
IamInstanceProfile: !Ref EC2CWInstanceProfile
NetworkInterfaces:
- AssociatePublicIpAddress: 'True'
DeleteOnTermination: 'True'
DeviceIndex: '0'
# Add subnet id below
SubnetId: subnet-031c6fb8172d780aa
GroupSet:
- !Ref InstanceSecurityGroup
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeType: gp2
DeleteOnTermination: 'true'
VolumeSize: '20'
LambdaSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Enable SSH access via port 22
# Add you vpc id below
VpcId: vpc-02e91d5d082e3a097
GroupName: DS Lambda Security Group
InstanceSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
DependsOn:
- LambdaSecurityGroup
Properties:
GroupDescription: Enable SSH access via port 22
# Add you vpc id below
VpcId: vpc-02e91d5d082e3a097
GroupName: DS DB Instance Security Group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
# Add vpn ip below for e.g 192.168.78.2/32
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '5432'
ToPort: '5432'
SourceSecurityGroupId: !Ref LambdaSecurityGroup
CWIAMRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/CloudWatchAgentAdminPolicy'
RoleName: DS_CW_AGENT_ROLE
EC2CWInstanceProfile:
Type: 'AWS::IAM::InstanceProfile'
Properties:
InstanceProfileName: EC2CWInstanceProfile
Roles:
- !Ref CWIAMRole
S3VPCEndpoint:
Type: 'AWS::EC2::VPCEndpoint'
Properties:
RouteTableIds:
- 'rtb-031f3057458433643'
ServiceName: com.amazonaws.ap-southeast-1.s3
VpcId: vpc-02e91d5d082e3a097

Sadly, you can't do this easily. The reason is that the Instance resource does not return the id of its root volume.
What's more, you can't create an independent AWS::EC2::Volume resource and use it as a root volume in your instance. This is only for additional volumes.
The only way to get the volume id of your root device would be through development of a custom resource. This would be in the form of lambda function, which would take the instance id, and use AWS SDK to find the volume id and return to cloud formation. With that volume id you could create CloudWatch Event rules.

Related

Cannot connect to EC2 via SSH | AWS Cloudformation Template

I have the following CloudFormation template that I use to create an EC2 instance in a single public subnet in a single availability zone. I have attach the internet gateway to the VPC and created ingress and egress routes to allow SSH connection to the EC2 instance.
Below is my CF template
AWSTemplateFormatVersion: "2010-09-09"
Description: "CF template for test website. v1.0.0. DEV Env"
Metadata:
Instances:
Description: "This is the dev environment architecture. Use the dev settings when setting up this environment"
Parameters:
ECommKeyPair:
Type: AWS::EC2::KeyPair::KeyName
Description: Select the dev key pair for the region
Resources:
DevEnvInternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
DevEnvVpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.1.1/16
EnableDnsHostnames: 'true'
EnableDnsSupport: 'true'
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
DevEnvVpcIgwAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId:
Ref: DevEnvVpc
InternetGatewayId:
Ref: DevEnvInternetGateway
DevEnvPublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: DevEnvVpc
CidrBlock: 10.0.1.1/16
AvailabilityZone: "us-west-2a"
MapPublicIpOnLaunch: 'true'
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
DevEnvSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Allow all inbound (ingress) and outbound (egress) traffic for port 22
GroupName: test-website-sec-group
VpcId:
Ref: DevEnvVpc
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
Description: allow all inbound traffic
IpProtocol: tcp
FromPort: 22
ToPort: 22
SecurityGroupEgress:
- CidrIp: 0.0.0.0/0
Description: allow all outbound traffic
IpProtocol: tcp
FromPort: 22
ToPort: 22
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
DevEnvRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: DevEnvVpc
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
DevEnvRoute:
Type: AWS::EC2::Route
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: DevEnvInternetGateway
RouteTableId:
Ref: DevEnvRouteTable
DevEnvEc2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
ImageId: ami-00f7e5c52c0f43726
AvailabilityZone: "us-west-2a"
KeyName:
Ref: ECommKeyPair
SecurityGroupIds:
- !GetAtt "DevEnvSecurityGroup.GroupId"
SubnetId:
Ref: DevEnvPublicSubnet
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeSize: 20
VolumeType: gp2
Tags:
- Key: Environment
Value: Dev
- Key: WebsiteName
Value: test
I am using Putty to connect to the EC2 instance with the private key file(ppk) that I associated with the EC2 instance. When tried to connect to instance with Putty, it is receiving the "Network error: Connection timed out" error message.
I even cannot connect to the instance using the AWS inbuilt "EC2 Instance Connect" through the web browser as well.
Greatly appreciate if you could point out to me the issue in my CF template.
You forgot to create AWS::EC2::SubnetRouteTableAssociation:
DevRouteAssos:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref DevEnvRouteTable
SubnetId: !Ref DevEnvPublicSubnet

AutoScaling ec2 instance with cloudformation

I am trying to autoScale this ec2 instance please guide me how to do it. Any template that might be helpful so that I can get started with the autoscaling. I am attaching only ec2 instance template which I want to autoScale.
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SourceStackName:
Description: "Source stack name"
Type: String
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: "shifa-vpc"
Resources:
webserver:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: webserver-sg
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-VpcID"
EC2Instance:
Type: AWS::EC2::Instance
Properties:
AvailabilityZone: us-east-1a
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-09d95fab7fff3776c # ami-0bdcc6c05dec346bf
InstanceType: t2.micro
#IamInstanceProfile: !Ref ListS3BucketsInstanceProfile
#KeyName: ky-webserver
NetworkInterfaces:
- Description: Primary network interface
DeviceIndex: 0
SubnetId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-PublicSubnet"
GroupSet:
- !Ref webserver
Outputs:
ec2:
Description: ec2
Value: !Ref EC2Instance
Export:
Name:
Fn::Sub: "${AWS::StackName}-server"
sgGroupId:
Description: ec2
Value: !GetAtt webserver.GroupId
Export:
Name:
Fn::Sub: "${AWS::StackName}-sgid"
I am new to cloudformation and I am in training.
Amazon have some examples for AutoScaling instances.
Importantly EC2 instance resources are not part of the autoscaling configuration within CloudFormation.
Instead you would use either a Launch Template or a Launch Configuration resource. The Launch Template is newer so preferably you should use this. These will define the instance configuration such as volumes, instance type etc.
The other component is the Autoscaling Group this will reference either one of previous components and define how the instance should scale.
If you're trying to scale an existing instance you will need to make an AMI from it first and the reference it.
AWS has an example template with autoscaling groups here.

Get "Encountered unsupported property KeyName" error when start a stack to launch a ec2 instance

I use cloudformation to launch a ec2 instance. Below is the cloudformation template:
Parameters:
KeyName:
Description: The EC2 Key Pair to allow SSH access to the instance
Type: 'AWS::EC2::KeyPair::KeyName'
Resources:
Ec2Instance:
Type: 'AWS::EC2::SpotFleet'
Properties:
SecurityGroups:
- !Ref InstanceSecurityGroup
- MyExistingSecurityGroup
KeyName: !Ref KeyName
ImageId: ami-07d0cf3af28718ef8
InstanceType: p2.8xlarge
AllocationStrategy: lowestPrice
SpotPrice: 1
InstanceSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Enable SSH access via port 22
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
I created a stack in cloudformation and specify the Key Name from a drop list of key pair. After than the stack rolled back and I see this error message Encountered unsupported property KeyName. I wonder what wrong with my configuration?
Check documentation on AWS::EC2::SpotFleet. It only supports SpotFleetRequestConfigData as property.
You will probably need to specify something like:
Ec2Instance:
Type: 'AWS::EC2::SpotFleet'
Properties:
SpotFleetRequestConfigData:
SpotPrice: 1
AllocationStrategy: lowestPrice
LaunchSpecifications:
- InstanceType: p2.8xlarge
SecurityGroups:
- !Ref InstanceSecurityGroup
- MyExistingSecurityGroup
KeyName: !Ref KeyName
ImageId: ami-07d0cf3af28718ef8
Check the AWS::EC2::SpotFleet documentation, it has a quite elaborate example.

The DB instance and EC2 security group are in different VPCs, cloudFormation error

I want to automate the process of creating RDS. I would like to create RDS Aurora.
When deploying the application, stack cloudFormation is validated and I have an error:
An error occurred: DatabaseCluster - The DB instance and EC2 security group are in different VPCs.
Can you say what's wrong?
I followed this post Issue with creating a Postgres RDS in Cloudformation Template but this doesn't work.
Here is part of my serverless.yml file
resources:
Resources:
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
ServerlessRDS:
Type: AWS::RDS::DBInstance
Properties:
Engine: aurora
DBClusterIdentifier: !Ref "DatabaseCluster"
DBInstanceIdentifier: db-name-${opt:stage, self:provider.stage}
DBInstanceClass: db.t2.medium
VPCSecurityGroups:
- !Ref VpcSecurityGroup
DBSubnetGroupName: !Ref myDBSubnetGroup
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
ServerlessVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: "10.0.0.0/16"
myDBSubnetGroup:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "description"
SubnetIds:
- !Ref ServerlessSubnetA
- !Ref ServerlessSubnetB
ServerlessSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1b"
CidrBlock: "10.0.0.0/24"
ServerlessSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1a"
CidrBlock: "10.0.1.0/24"
You need to add DBSubnetGroupName parameter to AWS::RDS::DBCluster Resource.
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
DBSubnetGroupName:
Ref: myDBSubnetGroup
Also you may want to add explicit dependencies on ServerlessSubnetA and ServerlessSubnetB in VpcSecurityGroup Resource to kind of group resource creation by the service and to avoid any race conditions.
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- ServerlessSubnetA
- ServerlessSubnetB
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
I encountered this situation and found someone had changed the dBs VPC.
Someone had taken a snapshot and restored it to a different VPC and that obviously didn't have the Security Group needed.
This probably happened because the website talking to the database was running on a different VPC.
To resolve it, you need to make a tough decision, because at this point you may end up having to blow the whole thing away and if its part of a group of CFN templates then that could spell even more problems with domino effects.
It's probably worth you try restore the dB back to the original VPC (or change Subnet Group/VPC) where the CFN template thinks it lives and where the security group was created by the CFN template and then re-run the CFN template.
If this fails, for example the database version has recently been updated you might get stuck with:
Cannot upgrade postgres from 9.6.22 to 9.6.11
At this point, the only recourse I know of is to delete the CFN stack and re-run it.

Create EC2 in default public subnet only using cloudformation

I have CF template which is creating EC2 machine.
AWSTemplateFormatVersion: 2010-09-09
Mappings:
InstanceAMI:
# ubuntu 18.04
us-west-2:
ami: 'ami-0bbe6b35405ecebdb'
us-east-1:
ami: 'ami-0ac019f4fcb7cb7e6'
Parameters:
Endpoint:
Type: String
# TODO edit the default value
Description:
Resources:
NodeInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: "/"
Roles:
- !Ref NodeInstanceRole
NodeInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AdministratorAccess
CdpDeplSvcSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access Deployment service
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 8080
ToPort: 8080
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Tags:
- Key: Name
Value: 'Access Deployment'
DeploymentMachine:
Type: AWS::EC2::Instance
Properties:
# AvailabilityZone: us-east-1a
ImageId: !FindInMap [InstanceAMI, !Ref "AWS::Region", ami]
InstanceType: 't2.small'
KeyName: 'key'
Tags:
- Key: Name
Value: 'Deployment'
BlockDeviceMappings:
- DeviceName: "/dev/sda1"
Ebs:
# VolumeType: "io1"
# Iops: "200"
DeleteOnTermination: "true"
VolumeSize: "30"
NetworkInterfaces:
- DeviceIndex: 0
AssociatePublicIpAddress: 'true'
DeleteOnTermination: 'true'
GroupSet:
- !GetAtt CdpDeplSvcSecurityGroup.GroupId
IamInstanceProfile: !Ref NodeInstanceProfile
It executing correctly. But the problem I am facing is sometimes it creates in default private subnet, sometimes in default public subnet.
I want to deploy this machine in the default public subnet only. I don't want to pass VPC id or subnet id as parameter. For that, what I have change here.
This will just put it into a random subnet - you need to hardcode subnet, or specify subnet through a parameter and then reference the parameter - this can provide you with some flexibility for varying the subnet per customer.
Potentially, during deployment of your stack, you could script the deployment, using the AWS CLI to get all public subnets, and pass one in as a parameter into your cloudformation stack.