I have
Parameters
Zookeeper1SubnetParam:
Description: Subnet where Zookeeper 1 should run
Type: AWS::EC2::Subnet::Id
Zookeeper1AZ:
Description: Availability Zone of the Subnet
Type: AWS::EC2::AvailabilityZone::Name
From this I'm creating an ENI (which requires a subnet) and an EBS Volume (which requires an availability zone).
Here's the ENI:
Zookeeper1IPResource:
Properties:
Description: Zookeeper1-IP
GroupSet:
- Fn::GetAtt:
- ZookeeperSecurityGroup
- GroupId
PrivateIpAddress:
Ref: Zookeeper1IPParam
SubnetId:
Ref: Zookeeper1SubnetParam
Type: AWS::EC2::NetworkInterface
And here's the EBS:
Zookeeper1EBSVolume:
Properties:
AvailabilityZone:
Ref: Zookeeper1AZ
Size: 8
VolumeType: gp2
Type: AWS::EC2::Volume
I find it really bad for user experience, to also ask as a parameter for an availability zone, because it can be deducted from the selected subnet
Now, the million dollar question, how do I get the Availability Zone from the Subnet in CloudFormation? As far as I can tell, I can't do a GetAtt for AZ on my ENI.
Any solution welcome!
To answer your question, you can't retrieve the Availability Zone from the Subnet.
But if you have total control of the template or resources that supplies the parameter to your template there are workarounds
If have control over the source the provides you the Subnet parameter, you can return also the Availability Zone from that source as an Outputs and supply it in your template as a parameter where you create ENI and EBS.
In addition, you could also create the Subnet in the same template you will create the ENI and EBS and use the { "Fn::GetAtt" : [ "mySubnet", "AvailabilityZone" ] }
Question(sorry, my rep can't allow me to comment yet)
Do you happen to have dynamic values or resources to be created that depends on availability zones? If yes, you can create Mappings and if that is not enough, you could add Conditions in your template.
I don't know if it is something new, but according to the documentation you can get the AZ of a subnet with GetAttr.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet.html#aws-resource-ec2-subnet-returnvalues
Quoting the documentation:
{ "Fn::GetAtt" : [ "mySubnet", "AvailabilityZone" ] }
UPDATE:
This suggestion is wrong, let me quote another documentation from AWS:
Supported Functions
For the Fn::GetAtt logical resource name, you cannot use functions. You must specify a string that is a resource's logical ID.
For the Fn::GetAtt attribute name, you can use the Ref function.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html#getatt-supported-functions
Related
In my CloudFormation file, I created an instance, a Launch Configuration and an Auto Scaling Group.
LogstashInstance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile:
Ref: LogstashInstanceProfile
InstanceType: t2.micro
KeyName: chuongtest
ImageId: ami-0cd31be676780afa7
UserData:
SecurityGroupIds:
- Ref: LogstashSecurityGroup
SubnetId: subnet-0e5691582096fe1e6
Tags:
- Key: Name
Value: Logstash Instance
LogstashLaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
EbsOptimized: false
IamInstanceProfile:
Ref: LogstashInstanceProfile
ImageId: ami-0cd31be676780afa7
InstanceMonitoring: true
InstanceType: t2.micro
KeyName: chuongtest
LaunchConfigurationName: LogstashLaunchConfiguration
SecurityGroups:
- Ref: LogstashSecurityGroup
UserData:
LogstashAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AutoScalingGroupName: LogstashAutoScalingGroup
AvailabilityZones:
- ap-southeast-1b
DesiredCapacity: 1
LaunchConfigurationName:
Ref: LogstashLaunchConfiguration
MaxSize: 1
MinSize: 1
Tags:
- Key: Name
PropagateAtLaunch: "false"
Value: Logstash ASG
- Key: Instances
PropagateAtLaunch: "true"
Value: Logstash
My idea is to create an instance, attach it to the ASG and use the ASG to keep the work continue.
But this code will launch 2 instances.
The first instances has userdata different from the later instances so I can not delete the instance.
I have looked in the documentation but I can not find anything make sense. Is there a way to configure in the template, or the only way is scripting?
I have looked in the documentation but I can not find anything make sense. Is there a way to configure in the template, or the only way is scripting?
It does not make sense, because its not practical. ASG will be launching instances only from the associated AWS::AutoScaling::LaunchConfiguration. Thus, there is not much sense on "manually" attaching instances in your cause the the ASG. ASG will not re-launch it when it fails.
If the "manually" attached instances gets terminated for some reason, e.g. hardware failure, ASG will launch a replacement based on the launch configuration.
userdata different from the later instances
In this case the best would be to provide some condition in your UserData which would determine which version of your UserData to run.
Alternatively, just have two ASGs. One for the first instance, while the other group for the remaining instances.
Edit
Having discussed via the comments I would highly recommend the approach of using EFS as it is designed for the workload design you're doing.
In addition it is multi AZ vs the approach of rotating a single EBS volume as instances fail in an autoscaling group which would not persist should the AZ fail and potentially lead to loss of data.
Original
If that first instance is different from the rest you should be careful about adding it to the autoscaling group (as you want it to persist), especially in how it handles the replacement of instances.
Autoscaling groups are designed to scale similar instances that are generally immutable, so by having your host in this autoscaling group it will be treated as if it is one of the other hosts.
An Auto Scaling group contains a collection of Amazon EC2 instances that are treated as a logical grouping for the purposes of automatic scaling and management.
If you must add it to the autoscaling group here is the general approach:
Create a Custom Resource which involves a Lambda attaching the EC2 to the ASG using attach_instances
Add set_instance_protection to your instance to prevent it being replaced during a scale in event (or the instance failing)
Also ensure you set both the MinSize and DesiredCapacity to 0 to remove any extra instance that gets launched.
As I stated earlier you should not add this instance to the ASG if it can be avoided, tt will probably lead to confusion.
I need to create subnet per availability zone in particular region using cloudformation.
For example if the region is Mumbai, it is with three availability zones and the CF template should create a public and a private subnet in each availability zone: 1a,1b and 1c. Is it really possible? I have done the same using terraform but have no idea how can I achieve this in CF.
It would be great if someone could help on this.
Thanks in advance.
Sadly, there are no loops in plain CloudFormation. Thus you can't create any constructs that would loop over AZs, get their IDs and create a pair of private-public subnets in each AZ.
If you really want to keep everything in CloudFormation than you would have to look at custom resources or marcros.
Both of them would require you to write your own lambda function that would use AWS API to get the number of AZs, their names and perform iteration to create the subnets.
If you already are using terraform successful, maybe its worth considering to keep using it, as it has loops useful in your use-case.
You might be able to write it in cloudformation. For example with Fn::If and then using Fn::GetAZs and creating the resources only if enough azs a parameter you can hardcode.
PrivateSubnet6:
Condition: Has6AZs
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: MyVPC
CidrBlock: 10.0.20.0/22
AvailabilityZone:
Fn::Select:
- 5
- Fn::GetAZs: !Ref 'AWS::Region'
Tags:
- Key: "Name"
Value: "PrivateSubnet6"
Has6AZs:
Fn::Equals: [!Ref AWS::Region, "us-east-1"]
I have already created AWS network resources(VPC,subnets,IGW etc) through AWS console.
Now, I am trying to create a cloudformation stack consisting of those existing & new resources.
To implement existing resources in the stack, I am using Parameters
Parameters:
VpcId:
Description: VPC Id
Type: AWS::EC2::VPC::Id
PublicSubnetId1:
Description: Public Subnet 1
Type: AWS::EC2::Subnet::Id
PublicSubnetId2:
Description: Public Subnet 2
Type: AWS::EC2::Subnet::Id
InternetGateway:
Type: AWS::EC2::InternetGateway::Id
Question 1: Is this the right approach ? (As we have diff envs so to handle dynamic AWS resources, I am doing this)
Question 2: I am able to set parameter with VPC,subnets but not with Internet gateway. How to put internetgateway as a parameter ?
Thanks
Question 1: Is this the right approach ? (As we have diff envs so to handle dynamic AWS resources, I am doing this)
Its one way of doing this. If you want to be able to select different subnet or vpc at template creation then its fine. However, if the VPC and subnets are fixed and do not change, probably better would be to export them from their own stacks, end them import them in other stacks. This way other stacks can easily refer to them, and you have simplified deployment your stacks as you don't need to provide all these parameters for each new stack.
Question 2: I am able to set parameter with VPC,subnets but not with Internet gateway. How to put internetgateway as a parameter ?
Unfortunatly, there is no such parameter like AWS::EC2::InternetGateway::Id. In this case you would have to use String:
InternetGateway:
Type: String
Creating a layered stack set in cloudformation. Network Stack and App Stack
Just need the syntax to Output and Export two Availability Zones that are captured when a cfn user chooses them in the network template parameters dialogue.
eg, a user chooses two AZ's in a region via the usual mechanism.
AZoneNames:
Type: 'List<AWS::EC2::AvailabilityZone::Name>'
Description: Availability Zones (choose two zones)
That captures the az's, and i assume, cfn indexes them [0,1] to an array in the background. That part works.
So I need to output the two az's and export them for the app stack but not sure how. I've attempted with the below snippet but it doesnt work
StackAvailabilityZone1:
Description: The first az that was chosen at network stack creation
Value: !Ref AvailabilityZone 0
Export:
Name: !Sub 'AZ1'
I'm sure its probably staring me in the face. Thanks so much for any ideas.
You can try the following, using Select:
StackAvailabilityZone1:
Description: The first az that was chosen at network stack creation
Value: !Select [0, !Ref AZoneNames]
Export:
Name: AZ1
I am looking to create a Spot Fleet in Cloudformation which runs a single game server at a time; if prices spike and the server needs to be terminated it will use the 2 minute heads-up to gracefully shutdown and store anything to be persisted on an EBS Volume. The next instance started by the fleet will then mount the volume and restart the game server from where the previous one left off.
SpotFleet:
Type: "AWS::EC2::SpotFleet"
Properties:
SpotFleetRequestConfigData:
IamFleetRole: !Sub arn:aws:iam::${AWS::AccountId}:role/aws-ec2-spot-fleet-tagging-role
TargetCapacity: 1
LaunchSpecifications:
- InstanceType: "m5.large"
ImageId: "ami-abcd1234"
IamInstanceProfile: !GetAtt InstanceProfile.Arn
WeightedCapacity: 1
Now I'm stuck on defining the persisted volume in the cf template. Initially I would just add it as a resource:
Volume:
Type: "AWS::EC2::Volume"
Properties:
Size: 10
AvailabilityZone: !Ref AWS::Region
But then how do I reference it in the fleet? You can define BlockDeviceMappings on LaunchSpeficiations within a fleet as per
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-spotfleet-spotfleetrequestconfigdata-launchspecifications-blockdevicemappings.html
but from the attributes available I can't seem to reference existing volumes and as such am getting the idea that these volumes are not persisted.
Alternatively I thought of attaching the volume to the spot instance via a VolumeAttachment:
VolumeAttachment:
Type: "AWS::EC2::VolumeAttachment"
Properties:
Device: "dev/server"
InstanceId: !Ref SpotFleet
VolumeId: !Ref Volume
but obviously the SpotFleet reference here returns the fleet name, not the id of any created instances. And neither !Ref nor !GetAtt seem to be able to extract those ids from a fleet.
Am I overlooking anything cruicial as to how to accomplish the above in CloudFormation or should I be looking at adding the EC2:AttachVolume and EC2:DetachVolume permissions to the InstanceProfile and simply attaching the volume manually from within the EC2 instance?
Many thanks,
EC2 Spot instances now support the option of setting the "Interruption behavior" to stop instead of terminate.
When this option is selected, a spot instance retains its instance ID, EBS volumes, its private and elastic IP address, and its EBS volumes, which remain in place and attached.
Some instance types also support a "hybernate" option that writes a snapshot of the entire system state to EBS to allow the instance to "resume" rather than reboot when capacity becomes available again.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html
What you are looking for is the BlockDeviceMappings property, found in the SpotFleet SpotFleetRequestConfigData LaunchSpecifications, which is a property of SpotFleetRequestConfigData, which is a property of the AWS::EC2::SpotFleet resource type.
The BlockDeviceMappings property will allow you to define additional EBS volumes to attach to your launch specification. This is the specification that controls device mappings at launch time.
For exmample:
"BlockDeviceMappings" : [{
"DeviceName" : "/dev/sdf",
"Ebs" : {"VolumeSize": "10", "VolumeType" : "gp2", "DeleteOnTermination" : "true"}
}],
will specify a 10GB volume on the /dev/sdf device of your spot fleet instance.