Reusing AWS::CloudFormation::Init (and userdata?) for multiple instances - amazon-web-services

Question

Is it possible to reuse the same bootstrapping config from AWS::CloudFormation::Init (and/or userdata) for multiple EC2::Instances in a template?
I need to set the content of 3 files and then run 3 commands to bootstrap all servers, but the Metadata block is about 30 lines long (and is likely to grow).
Each server instance has a different set of Tags, some have more tags than others.
Ideally, I think that you should be able to declare the AWS::CloudFormation::Init as a resource, and refer to it from multiple EC2::Instances, but I don't think that this is possible.
I initially thought (as a newbie) that AWS::CloudFormation::CustomResource might be appropriate, but I don't think that it is.
I'm currently thinking of using a AWS::CloudFormation::Stack to import a shared instance template, but I would need to somehow pass the Tags parameter for each resource in the stack template into the instance template. The question is - if this is the best approach, what do I enter in the 3 locations that currently have ?????
(Bonus credit - what's the difference between userdata and this init block?)
stack.template
...
"Resources" : {
"Server1" : {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"InstanceType": "m1.medium",
...
"Tags": { ???? }
},
"TemplateURL": "https://s3.amazonaws.com/mybucket/instance.template"
}
instance.template
...
"Parameters" : {
"InstanceType" : {...}
"KeyName": {...}
...
"Tags": {
????
}
},
"Resources" : {
"Instance" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"files" : {
"/etc/apt/apt.conf.d/99auth" : {
"content" : "APT::Get::AllowUnauthenticated yes;"
},
"/etc/apt/sources.list.d/my-repo.list" : {
"content" : "deb http://my-repo/repo apt/"
},
},
"commands" : {
"01-apt-get update" : {
"command" : "apt-get update"
},
"02-apt-get install puppet" : {
"command" : "apt-get install puppet my-puppet-config"
},
"03-puppet apply" : {
"command" : "puppet apply"
}
}
}
}
},
"Properties" : {
"InstanceType" : {"Ref" : "InstanceType"},
"ImageId" : "ami-84a333be",
"KeyName" : {"Ref" : "KeyName"},
"SubnetId" : {"Ref" : "SubnetId"},
"SecurityGroupIds" : [ { "Ref" : "SecurityGroupId"] } ],
"Tags" : [
????
]
}
}
}

Answer 1

Is it possible to reuse the same bootstrapping config from
AWS::CloudFormation::Init (and/or userdata) for multiple
EC2::Instances in a template?
No, this is not possible with the AWS::EC2::Instance resource in a single template. However, there is a resource type AWS::AutoScaling::LaunchConfiguration, but today this resource is only applicable to Auto Scaling groups. Ideally, AWS would provide a similar resource type that can be applied to multiple AWS::EC2::Instance resources. That being said, there is a lot of value in using Auto Scaling groups.
Here is a simple example that would allow you to do this with a single Launch Configuration and multiple Auto Scaling group resources in a single template. Auto scaling is typically configured with more than one instance, but I am using a MinSize and MaxSize of 1 to mirror the configuration you were going after with the AWS::EC2::Instance resource type. Even though we are using a single instance with an Auto Scaling group, we still gain the benefits of Auto Scaling with single instance resiliency. If the instance becomes unhealthy, Auto Scaling will automatically replace the instance.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources" : {
"LaunchConfig" : {
"Type" : "AWS::AutoScaling::LaunchConfiguration",
"Properties" : {
"InstanceType" : { "Ref" : "InstanceType" },
"ImageId" : "ami-84a333be",
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroupIds" : [{"Ref" : "SecurityGroupId"}],
"UserData" : { "Fn::Base64" : { "Fn::Join" : [ "", [
"#!/bin/bash -v\n",
"# Run cfn-init\n",
"/opt/aws/bin/cfn-init -v ",
" -stack ", { "Ref": "AWS::StackName" },
" -resource LaunchConfig ",
" --region ", { "Ref" : "AWS::Region" }, "\n",
"# Signal success\n",
"/opt/aws/bin/cfn-signal -e $? '", { "Ref" : "WaitConditionHandle" }, "'\n"
]]}}
},
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"files" : {
"/etc/apt/apt.conf.d/99auth" : {
"content" : "APT::Get::AllowUnauthenticated yes;"
},
"/etc/apt/sources.list.d/my-repo.list" : {
"content" : "deb http://my-repo/repo apt/"
}
},
"commands" : {
"01-apt-get update" : {
"command" : "apt-get update"
},
"02-apt-get install puppet" : {
"command" : "apt-get install puppet my-puppet-config"
},
"03-puppet apply" : {
"command" : "puppet apply"
}
}
}
}
}
},
"ASG1" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : [ { "Ref" : "AZ" } ],
"VPCZoneIdentifier" : [ { "Ref" : "SubnetId" } ],
"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
"MaxSize" : "1",
"MinSize" : "1",
"Tags" : [
{ "Key" : "Name", "Value": "Server1", "PropagateAtLaunch" : "true" },
{ "Key" : "Version", "Value": "1.0", "PropagateAtLaunch" : "true" }
]
}
},
"ASG2" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : [ { "Ref" : "AZ" } ],
"VPCZoneIdentifier" : [ { "Ref" : "SubnetId" } ],
"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
"MaxSize" : "1",
"MinSize" : "1",
"Tags" : [
{ "Key" : "Name", "Value": "Server2", "PropagateAtLaunch" : "true" },
{ "Key" : "Version", "Value": "1.0", "PropagateAtLaunch" : "true" }
]
}
},
"WaitConditionHandle" : {
"Type" : "AWS::CloudFormation::WaitConditionHandle"
},
"WaitCondition" : {
"Type" : "AWS::CloudFormation::WaitCondition",
"Properties" : {
"Handle" : { "Ref" : "WaitConditionHandle" },
"Timeout" : "300"
}
}
}
}
This is an incomplete example and there is much more you can do with Auto Scaling, but I just wanted to give you a simple example sharing a Launch Configuration with multiple instances. Each Auto Scaling group does define its own set of tags.
I'm currently thinking of using a AWS::CloudFormation::Stack to import
a shared instance template, but I would need to somehow pass the Tags
parameter for each resource in the stack template into the instance
template. The question is - if this is the best approach, what do I
enter in the 3 locations that currently have ?????
I would recommend the solution I described above, but I would also like to answer your questions on how use tags with the nested stack approach.
stack.template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources" : {
"Server1" : {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/instance.template",
"Parameters": {
"InstanceType": "m1.medium",
"TagName": "Server1"
"TagVersion": "1.0"
}
}
},
"Server2" : {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/instance.template",
"Parameters": {
"InstanceType": "m1.medium",
"TagName": "Server2"
"TagVersion": "1.0"
}
}
}
}
}
instance.template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters" : {
"InstanceType" : {...},
"KeyName": {...}
"TagName": {
"Description" : "The name tag to be applied to each instance",
"Type" : "String"
},
"TagVersion": {
"Description" : "The version tag to be applied to each instance",
"Type" : "String"
}
},
"Resources" : {
"Instance" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
...
}
},
"Properties" : {
"InstanceType" : { "Ref" : "InstanceType" },
"ImageId" : "ami-84a333be",
"KeyName" : { "Ref" : "KeyName" },
"SubnetId" : { "Ref" : "SubnetId" },
"SecurityGroupIds" : [ { "Ref" : "SecurityGroupId"] } ],
"Tags" : [
{ "Key" : "Name", "Value": { "Ref" : "TagName" } },
{ "Key" : "Version", "Value": { "Ref" : "TagVersion" } },
]
}
}
}
}
There is not a standard for passing an entire array of tags as a parameter, so you can see I simply broke each tag out into its own parameter and passed these on to the nested stacks.
(Bonus credit - what's the difference between userdata and this init
block?)
UserData allows you to pass arbitrary data to the instance at first boot. Often this is a shell script that can automate tasks when the instance starts. For example, you could simply run a yum update.
"UserData" : { "Fn::Base64" : { "Fn::Join" : [ "", [
"#!/bin/bash\n"
"yum update -y", "\n"
]]}}
UserData becomes even more useful when combined with the AWS::CloudFormation::Init metadata which allows you to structure your bootstrapping configuration. In this case, UserData is simply used to invoke the cfn-init script that executes the AWS::CloudFormation::Init metadata. I've included this pattern in my first example above using a Launch Configuration. It is important to note that the UserData section is executed only once during the first boot of the instance. This is important to keep in mind when thinking about how you want to handle updates to your instances.