I am trying to autoScale this ec2 instance please guide me how to do it. Any template that might be helpful so that I can get started with the autoscaling. I am attaching only ec2 instance template which I want to autoScale.
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SourceStackName:
Description: "Source stack name"
Type: String
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: "shifa-vpc"
Resources:
webserver:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: webserver-sg
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-VpcID"
EC2Instance:
Type: AWS::EC2::Instance
Properties:
AvailabilityZone: us-east-1a
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-09d95fab7fff3776c # ami-0bdcc6c05dec346bf
InstanceType: t2.micro
#IamInstanceProfile: !Ref ListS3BucketsInstanceProfile
#KeyName: ky-webserver
NetworkInterfaces:
- Description: Primary network interface
DeviceIndex: 0
SubnetId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-PublicSubnet"
GroupSet:
- !Ref webserver
Outputs:
ec2:
Description: ec2
Value: !Ref EC2Instance
Export:
Name:
Fn::Sub: "${AWS::StackName}-server"
sgGroupId:
Description: ec2
Value: !GetAtt webserver.GroupId
Export:
Name:
Fn::Sub: "${AWS::StackName}-sgid"
I am new to cloudformation and I am in training.
Amazon have some examples for AutoScaling instances.
Importantly EC2 instance resources are not part of the autoscaling configuration within CloudFormation.
Instead you would use either a Launch Template or a Launch Configuration resource. The Launch Template is newer so preferably you should use this. These will define the instance configuration such as volumes, instance type etc.
The other component is the Autoscaling Group this will reference either one of previous components and define how the instance should scale.
If you're trying to scale an existing instance you will need to make an AMI from it first and the reference it.
AWS has an example template with autoscaling groups here.
Related
I have enabled a Local Zone (us-east-1-nyc-1a) in my region (us-east-1), then deployed an EC2 instance into that Local Zone. The public IP that was allocated to my instance, indicates it is geo-located in the parent region (N.Virginia), while I would expect it to be in the Local Zone (NY/NJ).
My CloudFormation template configuration:
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
VpcId:
Type: AWS::EC2::VPC::Id
Resources:
Ec2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-0b0dcb5067f052a63
InstanceType: t3.medium
SubnetId: !Ref Ec2Subnet
SecurityGroupIds:
- !GetAtt WebSecurityGroup.GroupId
Ec2Subnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcId
CidrBlock: 172.31.96.0/24
AvailabilityZone: us-east-1-nyc-1a
MapPublicIpOnLaunch: true
WebSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
I also tried allocating EIP in the Local Zone (and assigning it to the EC2 instance), but it also being geo-located in the parent region.
Will appreciate your suggestions.
I am trying to EC2 instance (new), Security group (new) and VPC(existing). Here is my cloudformation template.
When I run the template in Stack, I got error as *"Value () for parameter groupId is invalid. The value cannot be empty"*. How to solve this?
Template:
Parameters:
VPCID:
Description: Name of an existing VPC
Type: AWS::EC2::VPC::Id
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.medium
AllowedValues:
- t2.medium
- t2.large
AccessLocation:
Description: The IP address range that can be used to access to the EC2 instances
Type: String
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SecurityGroups:
- !Ref 'InstanceSecurityGroup'
KeyName: !Ref 'KeyName'
ImageId: !Ref 'ImageId'
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPCID
GroupDescription: Enable SSH
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: !Ref 'AccessLocation'
SecurityGroups can only be used for default VPC. Since you are explicitly assigning VPCID to InstanceSecurityGroup, this will be considered as non-default, resulting in failed deployment.
You must use SecurityGroupIds (not SecurityGroups) in your case as your VPC use will be considered as non-default:
SecurityGroupIds:
- !GetAtt 'InstanceSecurityGroup.GroupId'
The error in EC2Instance resource in SecurityGroups attribute. SecurityGroups needs an array of GroupId but when you use !Ref InstanceSecurityGroup this returns ResourceId. So you need to use GetAtt instead to get GroupId.
Parameters:
VPCID:
Description: Name of an existing VPC
Type: AWS::EC2::VPC::Id
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.medium
AllowedValues:
- t2.medium
- t2.large
AccessLocation:
Description: The IP address range that can be used to access to the EC2 instances
Type: String
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SecurityGroups:
- !GetAtt InstanceSecurityGroup.GroupId
KeyName: !Ref 'KeyName'
ImageId: !Ref 'ImageId'
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPCID
GroupDescription: Enable SSH
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: !Ref 'AccessLocation'
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-security-group.html
I am unable to connect from a Fargate container to an RDS instance when its ingress is limited through security groups. I can connect with lambdas though.
The container has no issue hitting SQS, or the internet. Only has issues hitting the RDS endpoint.
Here is an excerpt from the template, where the database ingress is open. Fargate can connect without issue.
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Ref ServiceName
Cluster: !Ref Cluster
TaskDefinition: !Ref TaskDefinition
PlatformVersion: 1.3.0
DeploymentConfiguration:
MinimumHealthyPercent: 100
MaximumPercent: 200
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
Subnets:
- !Ref PrivateSubnet1
SecurityGroups:
- !Ref DatabaseAccessSecurityGroup
DatabaseInstance:
Type: AWS::RDS::DBInstance
Properties:
Engine: mysql
EngineVersion: 8.0.16
AvailabilityZone: !GetAtt PrivateSubnet1.AvailabilityZone
PubliclyAccessible: false
...
VPCSecurityGroups:
- !Ref DatabaseSecurityGroup
DatabaseSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Cloudformation managed Db subnet group
SubnetIds:
- !Ref PrivateSubnet1
- !Ref PrivateSubnet2
DatabaseSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPC
GroupName: database-sg
GroupDescription: Database security group
SecurityGroupIngress:
- Description: Access to RDS
# allowing all works with Fargate
CidrIp: 0.0.0.0/0
FromPort: 3306
ToPort: 3306
IpProtocol: tcp
DatabaseAccessSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: database-access-sg
GroupDescription: Security group for accessing db
VpcId: !Ref VPC
But if I change the DatabaseSecurityGroup Group ingress to only allow ingress through DatabaseAccessSecurityGroup I get errors when trying to connect through Fargate. Lambdas using the same security group have no issue.
SecurityGroupIngress:
- Description: Access to RDS
CidrIp: 0.0.0.0/0
SourceSecurityGroupId: !GetAtt DatabaseAccessSecurityGroup.GroupId
FromPort: 3306
ToPort: 3306
IpProtocol: tcp
Is there any way to get the Fargate Service to respect security group rules?
This issue was occurring because I was using a Service in the Cloudformation template, but spinning up the tasks via ecs.runTask, which overrode the security groups in the Service.
I am trying to create cloudwatch scheduled event for taking snapshot of my ebs. I am new to cloudformation not much familiar with it that's why having complexity in achieving this. I am attaching my current template which spawns my ec2 instance and override the default volume from 10gb to 20gb. I want to create a cloudwatch event on exactly the same created volume to take the snapshot of this volume that has been created from this template. I would be glad if anyone can help me in setting an event with target using the cloudformation syntax.
Parameters:
KeyName:
Description: The EC2 Key Pair to allow SSH access to the instance
Type: 'AWS::EC2::KeyPair::KeyName'
Resources:
Ec2Instance:
Type: 'AWS::EC2::Instance'
DependsOn:
- InstanceSecurityGroup
- CWIAMRole
- EC2CWInstanceProfile
Properties:
KeyName: !Ref KeyName
ImageId: ami-057a963e8be173b19
InstanceType: t3a.micro
IamInstanceProfile: !Ref EC2CWInstanceProfile
NetworkInterfaces:
- AssociatePublicIpAddress: 'True'
DeleteOnTermination: 'True'
DeviceIndex: '0'
# Add subnet id below
SubnetId: subnet-031c6fb8172d780aa
GroupSet:
- !Ref InstanceSecurityGroup
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeType: gp2
DeleteOnTermination: 'true'
VolumeSize: '20'
LambdaSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Enable SSH access via port 22
# Add you vpc id below
VpcId: vpc-02e91d5d082e3a097
GroupName: DS Lambda Security Group
InstanceSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
DependsOn:
- LambdaSecurityGroup
Properties:
GroupDescription: Enable SSH access via port 22
# Add you vpc id below
VpcId: vpc-02e91d5d082e3a097
GroupName: DS DB Instance Security Group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
# Add vpn ip below for e.g 192.168.78.2/32
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '5432'
ToPort: '5432'
SourceSecurityGroupId: !Ref LambdaSecurityGroup
CWIAMRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/CloudWatchAgentAdminPolicy'
RoleName: DS_CW_AGENT_ROLE
EC2CWInstanceProfile:
Type: 'AWS::IAM::InstanceProfile'
Properties:
InstanceProfileName: EC2CWInstanceProfile
Roles:
- !Ref CWIAMRole
S3VPCEndpoint:
Type: 'AWS::EC2::VPCEndpoint'
Properties:
RouteTableIds:
- 'rtb-031f3057458433643'
ServiceName: com.amazonaws.ap-southeast-1.s3
VpcId: vpc-02e91d5d082e3a097
Sadly, you can't do this easily. The reason is that the Instance resource does not return the id of its root volume.
What's more, you can't create an independent AWS::EC2::Volume resource and use it as a root volume in your instance. This is only for additional volumes.
The only way to get the volume id of your root device would be through development of a custom resource. This would be in the form of lambda function, which would take the instance id, and use AWS SDK to find the volume id and return to cloud formation. With that volume id you could create CloudWatch Event rules.
I want to automate the process of creating RDS. I would like to create RDS Aurora.
When deploying the application, stack cloudFormation is validated and I have an error:
An error occurred: DatabaseCluster - The DB instance and EC2 security group are in different VPCs.
Can you say what's wrong?
I followed this post Issue with creating a Postgres RDS in Cloudformation Template but this doesn't work.
Here is part of my serverless.yml file
resources:
Resources:
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
ServerlessRDS:
Type: AWS::RDS::DBInstance
Properties:
Engine: aurora
DBClusterIdentifier: !Ref "DatabaseCluster"
DBInstanceIdentifier: db-name-${opt:stage, self:provider.stage}
DBInstanceClass: db.t2.medium
VPCSecurityGroups:
- !Ref VpcSecurityGroup
DBSubnetGroupName: !Ref myDBSubnetGroup
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
ServerlessVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: "10.0.0.0/16"
myDBSubnetGroup:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "description"
SubnetIds:
- !Ref ServerlessSubnetA
- !Ref ServerlessSubnetB
ServerlessSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1b"
CidrBlock: "10.0.0.0/24"
ServerlessSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1a"
CidrBlock: "10.0.1.0/24"
You need to add DBSubnetGroupName parameter to AWS::RDS::DBCluster Resource.
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
DBSubnetGroupName:
Ref: myDBSubnetGroup
Also you may want to add explicit dependencies on ServerlessSubnetA and ServerlessSubnetB in VpcSecurityGroup Resource to kind of group resource creation by the service and to avoid any race conditions.
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- ServerlessSubnetA
- ServerlessSubnetB
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
I encountered this situation and found someone had changed the dBs VPC.
Someone had taken a snapshot and restored it to a different VPC and that obviously didn't have the Security Group needed.
This probably happened because the website talking to the database was running on a different VPC.
To resolve it, you need to make a tough decision, because at this point you may end up having to blow the whole thing away and if its part of a group of CFN templates then that could spell even more problems with domino effects.
It's probably worth you try restore the dB back to the original VPC (or change Subnet Group/VPC) where the CFN template thinks it lives and where the security group was created by the CFN template and then re-run the CFN template.
If this fails, for example the database version has recently been updated you might get stuck with:
Cannot upgrade postgres from 9.6.22 to 9.6.11
At this point, the only recourse I know of is to delete the CFN stack and re-run it.