I need to create subnet per availability zone in particular region using cloudformation.
For example if the region is Mumbai, it is with three availability zones and the CF template should create a public and a private subnet in each availability zone: 1a,1b and 1c. Is it really possible? I have done the same using terraform but have no idea how can I achieve this in CF.
It would be great if someone could help on this.
Thanks in advance.
Sadly, there are no loops in plain CloudFormation. Thus you can't create any constructs that would loop over AZs, get their IDs and create a pair of private-public subnets in each AZ.
If you really want to keep everything in CloudFormation than you would have to look at custom resources or marcros.
Both of them would require you to write your own lambda function that would use AWS API to get the number of AZs, their names and perform iteration to create the subnets.
If you already are using terraform successful, maybe its worth considering to keep using it, as it has loops useful in your use-case.
You might be able to write it in cloudformation. For example with Fn::If and then using Fn::GetAZs and creating the resources only if enough azs a parameter you can hardcode.
PrivateSubnet6:
Condition: Has6AZs
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: MyVPC
CidrBlock: 10.0.20.0/22
AvailabilityZone:
Fn::Select:
- 5
- Fn::GetAZs: !Ref 'AWS::Region'
Tags:
- Key: "Name"
Value: "PrivateSubnet6"
Has6AZs:
Fn::Equals: [!Ref AWS::Region, "us-east-1"]
Related
I created a cluster in ECS with basic settings, nothing specific about the configuration except that I am using 1 On Demand t2.micro EC2 instance for the cluster.
I wanted to see what exactly was created and took a look at the CloudFormation template the cluster created.
I noticed in the template it has a configuration for EcsSpotFleet
EcsSpotFleet:
Condition: CreateWithSpot
Type: AWS::EC2::SpotFleet
Properties:
SpotFleetRequestConfigData:
AllocationStrategy: !Ref SpotAllocationStrategy
IamFleetRole: !Ref IamSpotFleetRoleArn
TargetCapacity: !Ref AsgMaxSize
SpotPrice: !If [ CreateWithSpotPrice, !Ref SpotPrice, !Ref 'AWS::NoValue' ]
TerminateInstancesWithExpiration: true
LaunchSpecifications:
....
I am wondering why is this created? Because I know the Cluster instances are created with ASG + LC. My only explanation is this fleet is used for running the CloudFormation stack. I cannot find an explanation to this in the documentation, not even sure if instances are needed for CloudFormation stack run.
p.s. I am very new to AWS, also have very little knowledge on CloudFormation.
Not all code in CloudFormation will be executed. It still depends on the "Condition" flag.
AWS usually create a template that covers most of the user cases and enables/disable parts of the template using the "Condition"
You can read more about Condition in AWS documentation here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html
In my CloudFormation file, I created an instance, a Launch Configuration and an Auto Scaling Group.
LogstashInstance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile:
Ref: LogstashInstanceProfile
InstanceType: t2.micro
KeyName: chuongtest
ImageId: ami-0cd31be676780afa7
UserData:
SecurityGroupIds:
- Ref: LogstashSecurityGroup
SubnetId: subnet-0e5691582096fe1e6
Tags:
- Key: Name
Value: Logstash Instance
LogstashLaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
EbsOptimized: false
IamInstanceProfile:
Ref: LogstashInstanceProfile
ImageId: ami-0cd31be676780afa7
InstanceMonitoring: true
InstanceType: t2.micro
KeyName: chuongtest
LaunchConfigurationName: LogstashLaunchConfiguration
SecurityGroups:
- Ref: LogstashSecurityGroup
UserData:
LogstashAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AutoScalingGroupName: LogstashAutoScalingGroup
AvailabilityZones:
- ap-southeast-1b
DesiredCapacity: 1
LaunchConfigurationName:
Ref: LogstashLaunchConfiguration
MaxSize: 1
MinSize: 1
Tags:
- Key: Name
PropagateAtLaunch: "false"
Value: Logstash ASG
- Key: Instances
PropagateAtLaunch: "true"
Value: Logstash
My idea is to create an instance, attach it to the ASG and use the ASG to keep the work continue.
But this code will launch 2 instances.
The first instances has userdata different from the later instances so I can not delete the instance.
I have looked in the documentation but I can not find anything make sense. Is there a way to configure in the template, or the only way is scripting?
I have looked in the documentation but I can not find anything make sense. Is there a way to configure in the template, or the only way is scripting?
It does not make sense, because its not practical. ASG will be launching instances only from the associated AWS::AutoScaling::LaunchConfiguration. Thus, there is not much sense on "manually" attaching instances in your cause the the ASG. ASG will not re-launch it when it fails.
If the "manually" attached instances gets terminated for some reason, e.g. hardware failure, ASG will launch a replacement based on the launch configuration.
userdata different from the later instances
In this case the best would be to provide some condition in your UserData which would determine which version of your UserData to run.
Alternatively, just have two ASGs. One for the first instance, while the other group for the remaining instances.
Edit
Having discussed via the comments I would highly recommend the approach of using EFS as it is designed for the workload design you're doing.
In addition it is multi AZ vs the approach of rotating a single EBS volume as instances fail in an autoscaling group which would not persist should the AZ fail and potentially lead to loss of data.
Original
If that first instance is different from the rest you should be careful about adding it to the autoscaling group (as you want it to persist), especially in how it handles the replacement of instances.
Autoscaling groups are designed to scale similar instances that are generally immutable, so by having your host in this autoscaling group it will be treated as if it is one of the other hosts.
An Auto Scaling group contains a collection of Amazon EC2 instances that are treated as a logical grouping for the purposes of automatic scaling and management.
If you must add it to the autoscaling group here is the general approach:
Create a Custom Resource which involves a Lambda attaching the EC2 to the ASG using attach_instances
Add set_instance_protection to your instance to prevent it being replaced during a scale in event (or the instance failing)
Also ensure you set both the MinSize and DesiredCapacity to 0 to remove any extra instance that gets launched.
As I stated earlier you should not add this instance to the ASG if it can be avoided, tt will probably lead to confusion.
I have already created AWS network resources(VPC,subnets,IGW etc) through AWS console.
Now, I am trying to create a cloudformation stack consisting of those existing & new resources.
To implement existing resources in the stack, I am using Parameters
Parameters:
VpcId:
Description: VPC Id
Type: AWS::EC2::VPC::Id
PublicSubnetId1:
Description: Public Subnet 1
Type: AWS::EC2::Subnet::Id
PublicSubnetId2:
Description: Public Subnet 2
Type: AWS::EC2::Subnet::Id
InternetGateway:
Type: AWS::EC2::InternetGateway::Id
Question 1: Is this the right approach ? (As we have diff envs so to handle dynamic AWS resources, I am doing this)
Question 2: I am able to set parameter with VPC,subnets but not with Internet gateway. How to put internetgateway as a parameter ?
Thanks
Question 1: Is this the right approach ? (As we have diff envs so to handle dynamic AWS resources, I am doing this)
Its one way of doing this. If you want to be able to select different subnet or vpc at template creation then its fine. However, if the VPC and subnets are fixed and do not change, probably better would be to export them from their own stacks, end them import them in other stacks. This way other stacks can easily refer to them, and you have simplified deployment your stacks as you don't need to provide all these parameters for each new stack.
Question 2: I am able to set parameter with VPC,subnets but not with Internet gateway. How to put internetgateway as a parameter ?
Unfortunatly, there is no such parameter like AWS::EC2::InternetGateway::Id. In this case you would have to use String:
InternetGateway:
Type: String
Creating a layered stack set in cloudformation. Network Stack and App Stack
Just need the syntax to Output and Export two Availability Zones that are captured when a cfn user chooses them in the network template parameters dialogue.
eg, a user chooses two AZ's in a region via the usual mechanism.
AZoneNames:
Type: 'List<AWS::EC2::AvailabilityZone::Name>'
Description: Availability Zones (choose two zones)
That captures the az's, and i assume, cfn indexes them [0,1] to an array in the background. That part works.
So I need to output the two az's and export them for the app stack but not sure how. I've attempted with the below snippet but it doesnt work
StackAvailabilityZone1:
Description: The first az that was chosen at network stack creation
Value: !Ref AvailabilityZone 0
Export:
Name: !Sub 'AZ1'
I'm sure its probably staring me in the face. Thanks so much for any ideas.
You can try the following, using Select:
StackAvailabilityZone1:
Description: The first az that was chosen at network stack creation
Value: !Select [0, !Ref AZoneNames]
Export:
Name: AZ1
I have
Parameters
Zookeeper1SubnetParam:
Description: Subnet where Zookeeper 1 should run
Type: AWS::EC2::Subnet::Id
Zookeeper1AZ:
Description: Availability Zone of the Subnet
Type: AWS::EC2::AvailabilityZone::Name
From this I'm creating an ENI (which requires a subnet) and an EBS Volume (which requires an availability zone).
Here's the ENI:
Zookeeper1IPResource:
Properties:
Description: Zookeeper1-IP
GroupSet:
- Fn::GetAtt:
- ZookeeperSecurityGroup
- GroupId
PrivateIpAddress:
Ref: Zookeeper1IPParam
SubnetId:
Ref: Zookeeper1SubnetParam
Type: AWS::EC2::NetworkInterface
And here's the EBS:
Zookeeper1EBSVolume:
Properties:
AvailabilityZone:
Ref: Zookeeper1AZ
Size: 8
VolumeType: gp2
Type: AWS::EC2::Volume
I find it really bad for user experience, to also ask as a parameter for an availability zone, because it can be deducted from the selected subnet
Now, the million dollar question, how do I get the Availability Zone from the Subnet in CloudFormation? As far as I can tell, I can't do a GetAtt for AZ on my ENI.
Any solution welcome!
To answer your question, you can't retrieve the Availability Zone from the Subnet.
But if you have total control of the template or resources that supplies the parameter to your template there are workarounds
If have control over the source the provides you the Subnet parameter, you can return also the Availability Zone from that source as an Outputs and supply it in your template as a parameter where you create ENI and EBS.
In addition, you could also create the Subnet in the same template you will create the ENI and EBS and use the { "Fn::GetAtt" : [ "mySubnet", "AvailabilityZone" ] }
Question(sorry, my rep can't allow me to comment yet)
Do you happen to have dynamic values or resources to be created that depends on availability zones? If yes, you can create Mappings and if that is not enough, you could add Conditions in your template.
I don't know if it is something new, but according to the documentation you can get the AZ of a subnet with GetAttr.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet.html#aws-resource-ec2-subnet-returnvalues
Quoting the documentation:
{ "Fn::GetAtt" : [ "mySubnet", "AvailabilityZone" ] }
UPDATE:
This suggestion is wrong, let me quote another documentation from AWS:
Supported Functions
For the Fn::GetAtt logical resource name, you cannot use functions. You must specify a string that is a resource's logical ID.
For the Fn::GetAtt attribute name, you can use the Ref function.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html#getatt-supported-functions