Writing reusable CloudFormation snippets with AWS::Include and Nested Stacks - amazon-web-services

I have been using nested stacks in CloudFormation for several months and they are very useful. So I thought I should spend sometime to make each nested stack reusable to other teams in the org.
I saw the use case of AWS::Include in several places like here and here and it makes good sense to me.
One approach I have in mind is one snippet for each resource, like an AWS::EC2::Subnet or AWS::EC2::InternetGateway which can be included zero or more times into a vpc.json template, which itself can be used as a nested stack in a larger application.
The snippet does not take any parameters, but can reference a parameter that exists in the parent template.
At first glance this doesn't seem enough to me. Consider this example:
"PublicSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {"Ref": "VPC"},
"AvailabilityZone": {
"Fn::Select" : [ "0", { "Fn::GetAZs" : {"Ref": "AWS::Region"} }]
},
"CidrBlock": {
"Fn::FindInMap": ["AZSubnetMap", {
"Fn::Select" : [ "0", { "Fn::GetAZs" : {"Ref": "AWS::Region"} }]},
"PublicSubnet"]},
"MapPublicIpOnLaunch": "true",
"Tags": [..]
}
}
How can I avoid hard coding that "0" for the AZ in a Subnet snippet for example?

Unfortunately, AWS doesn't provide a way to dynamically update the template as per the requirement.
I have solved a similar problem using Mustache Templates using Java Library Handle Bars. Using this library you can generate template on the fly based on the requirements.
Hope this helps.

You will have to use two AWS::Include files located in:
s3://yourname/PublicSubnetA.yaml
s3://yourname/PublicSubnetB.yaml
And call them from your MAIN Template:
Fn::Transform:
Name: AWS::Include
Parameters:
Location : "s3://yourname/PublicSubnetA.yaml"
Fn::Transform:
Name: AWS::Include
Parameters:
Location : "s3://yourname/PublicSubnetB.yaml"
I 'am trying to find way to send additional parameters or override parameter to AWS::include, as you see it has
Parameters:
Location:
Why ti's not understanding more parameters and not just Location, I would be glad to have something like this:
Fn::Transform:
Name: AWS::Include
Parameters:
MySubnetIndex: 0
Location : "s3://yourname/PublicSubnetB.yaml"

I tried this way to send additional parameters:
Fn::Transform:
Name: AWS::Include
Parameters:
Location : "s3://my.test/create-ec2.yaml"
EC2Size :
Type: String
Default: "t2.micro"
And got interesting error:
The value of parameter EC2Size under transform Include must resolve to a string, number, boolean or a list of any of these
Looks like it understand what this is additional parameters, probably it need to be configured little bit different. I was not able to find way to fix this error yet.

Related

Pseudo parameter in cloudformation

I'm using the below simple template and trying to list the region in parameter section using Pseudo parameter AWS::Region, however getting the below error while trying to update the stack :
Failed to retrieve external values
Image posted here [1]: https://i.stack.imgur.com/kUcoo.png
Nothing reporting on cloudtrail :
"responseElements": {
"parameters": [
{
"parameterType": "List<AWS::Region>",
"noEcho": false,
"parameterKey": "S3"
}
Below are my template details :
Parameters:
Region:
Type: List<AWS::Region>
Default: us-east-1
Resources:
###
Can someone give an advice. Thanks in advance.
Its not possible. You have to use CommaDelimitedList.

serverless-aws-documentation model definitions with optional fields?

I want to define request and response models. I use the Serverless Framework with AWS and everything I'm seeing recommends using serverless-aws-documentation
The README says that I need to have this line in custom.documentation.models.MODELNAME
schema: ${file(models/error.json)}
But they don't have an example file of models/error.json to use as a baseline.
In the actual example serverless.yml they have a definition like this:
-
name: DoSomethingRequest
contentType: "application/json"
schema:
type: array
items:
type: string
This doesn't provide enough detail for what I'm trying to do.
My goal is to have a schema defined for an array of string objects, a message and a status code. The message and status code, though, are optional. These could also be part of other models and if possible I'd like to not repeat their definition for each model.
My current attempt is:
-
name: ReturnArrayResponse
contentType: "application/json"
schema:
type: array
itemsArray:
type: string
message:
type: string
statusCode:
type: number
I think this is going to do what I want, but how can I have message and statusCode be optional and repeat these two items in my other models?
I'd be happy with either a yml solution I can put in my serverless.yml file or a json file that I can reference.
Including a file
In the example given, error.json can contain any valid schema. So something as simple as this is fine:
{"type":"object","properties":{"message":{"type":"string"}}}
It's also fine to include attributes like $schema and title:
{
"$schema" : "http://json-schema.org/draft-04/schema#",
"title" : "Error Schema",
"type" : "object",
"properties" : {
"message" : { "type" : "string" },
"statusCode": { "type": "number" },
"itemsArray": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
This is especially handy when you have models already defined in AWS, but you don't have the serverless yaml to build them. You can simply copy the schema out of the AWS console, paste the json into a file, and use the schema: ${file()} syntax mentioned in the question. As far as I can tell anything that you can get the AWS console to accept will work.
DRY
I don't know of a way to reference models from within other models in a serverless file, but you could use the same approach as the plugin authors, and just put anything you need to reuse outside of the models and somewhere it's easier to reuse. The plugin authors use commonModelSchemaFragments.
So if you have some fragments like so:
commonModelSchemaFragments:
# defining common fragments means you can reference them with a single line
StringArrayFragment:
type: array
items:
type: string
HttpResponse:
type: object
properties:
message:
type: string
statusCode:
type: number
You can reference those fragments in models like this:
-
name: HttpStatusResponse
contentType: "application/json"
schema:
type: object
properties:
serverResponse:
${self:custom.commonModelSchemaFragments.HttpResponse}
messageArray:
${self:custom.commonModelSchemaFragments.StringArrayFragment}
Marking attributes optional
You can accomplish this by marking attributes as required. Simply provide a list of all attributes, except the ones you want to be optional. The json schema for that looks like this:
{
"type": "object",
"required": ["message"],
"properties": {
"optionalMessage": {
"type": "string"
},
"message": {
"type": "string"
}
}
}
which you would build by using yaml like this in your serverless file:
-
name: OptionalResponse
contentType: "application/json"
schema:
type: object
required:
- "message"
properties:
message:
type: string
optionalMessage:
type: string
Note on request validation
Marking attributes required or optional only matters if request body validation is turned on:
I don't know of a way to turn on request validation using any special serverless syntax. It looks like you can do this in the resources section, but I haven't tried it. Source.
Just a guess (posting it as an answer to preserve formatting) - your top-level entity in the schema should be an object, not an array, something like this:
schema:
type: object
properties:
items:
type: array
items:
type: string
message:
type: string
statusCode:
type: number

Invalid domain name identifier specified

When trying to create an AWS::ApiGateway::BasePathMapping through CloudFormation, I am given the following error:
Invalid domain name identifier specified
Below is the portion(s) of my CloudFormation template that should create the AWS::ApiGateway::BasePathMapping:
{
"Parameters": {
"ApiDomainName": {
"Description": "The domain name for the API",
"Type": "String"
}
},
"Resources": {
"ApiBasePathMapping": {
"Type": "AWS::ApiGateway::BasePathMapping",
"Properties": {
"DomainName": {
"Ref": "ApiDomainName"
},
"RestApiId": {
"Ref": "RepositoryApi"
},
"Stage": {
"Ref": "ApiProductionStage"
}
},
"DependsOn": [
"ApiProductionStage"
]
}
}
}
The documentation makes no mention that it needs to be anything special for the DomainName, but the documentation for this resource seems to be lacking some information (It doesn't list outputs for example even though there is a Distribution Domain Name created as an example).
The remainder of the stack works as expected. I am trying to add this resource in as a Change Set. I do own the domain I am trying to use, and I have created a certificate in ACM for this domain.
Quoting from AWS forums:
You can only create or modify base path mappings after the domain name
has been added to API Gateway. This "Invalid domain name identifier
specified" error message is returned when the domain name given in the
base path mapping is not found, indicating that it has not been added
yet.
Also, as of March 2017, the only way to add domain name to the API Gateway via CloudFormation is via custom resources that CloudFormation offers.
Ref: https://forums.aws.amazon.com/message.jspa?messageID=769627
It is now possible to just do that. You just have to explicit state on your CFN template that there is a dependency (DependsOn):
...
ApiDevMapping:
Type: 'AWS::ApiGateway::BasePathMapping'
Properties:
BasePath: v1.0
Stage: dev
DomainName: my-api.example.com
RestApiId: !Ref MobileApiDev
DependsOn:
- MobileApiDevDomain
...

How to pass a VPC's ID as a parameter to the Cluster.template to set as a default value?

I am trying to find a way to set default VPCs, Subnets and Security Groups in the Cluster.template JSON file.
Is there a way to pass an existing VPC ( or Subnet/Security group) as a parameter to the template using the "Ref" built-in?
This Obviously dones't work:
"Parameters": {
"VpcId": {
"Type": "AWS::EC2::VPC::Id",
"Default": { "Ref" : "vpc-123456789" },
....
}
To inject a VPC id into your template I would do the following. First remove your default value.
"Parameters": {
"VpcId": {
"Type": "AWS::EC2::VPC::Id"
....
}
Next place the value you want to set VpcId to inside a parameters.json file and when you perform a create-stack or update-stack using your cloudformation use the parameters file as the input.
parameters.json
[
{
"ParameterKey": "VpcId",
"ParameterValue": "vpc-123456789"
}
]
Multi-valued Parameters
If you had a parameter that takes a list of values you could represent it as follows
"PrivateEC2Subnets": {
"Type": "CommaDelimitedList",
"Description": "List of private subnets to run your EC2 instances inside. Note that they must be in the same availability zone that your ELB is configured for. May require you to manually create a private subnet with a specific AZ if your VPC isnt auto-configured."
},
Then in your external parameters file pass in a comma separated list like so
{
"ParameterKey": "PrivateEC2Subnets",
"ParameterValue": "subnet-9934670a544,subnet-d74ea349f"
},
For more information on the different parameter types see the AWS doc http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html but beware, people have reported issues when trying to represent lists of complex datatypes in external parameters files. To my knowledge, only CommaDelimitedList works if you want to pass the values in from another json file outside your cloudformation template.
I found out that it's really much simpler than I thought... this worked:
"Parameters": {
"VpcId": {
"Type": "List<AWS::EC2::VPC::Id>",
"Default": "vpc-123456789,vpc-987654123" ,
....
}

Referencing Resources between CloudFormation stacks

If I have two cloudformation stacks, how do I references a resource in one stack from the other stack?
In the example below I have a stack that creates an EBS volume and want to reference that via the Ref: key in the second stack for my EC2 instance but I keep getting a rollback since it can't see that resource from the first stack:
"Template format error: Unresolved resource dependencies"
I already tried the DependsOn clause but it didn't work. Do I need to pass information via Parameters?
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"CubesNetworking": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/cf_network.json"
}
},
"CubesInstances": {
"DependsOn": ["CubesNetworking"],
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/mybucket/cf_instances.json"
}
}
}
}
In each of your nested stacks, you should have an output section. Then you can get those values in your calling stack (the one you have listed above) with syntax like:
{ "Fn::GetAtt" : [ "CubesNetworking", "Outputs.VolumeID" ] }
You then pass the values into your other nested stacks via Parameters:
"Parameters" : {
"VolumeId" : { "Fn::GetAtt" : [ "CubesNetworking", "Outputs.VolumeID" ] }
You still want the DependsOn since you need the volume created before the instance.
Edit, Mid-2017:
CloudFormation has introduced the ability to export values from one stack, and reference them in other stacks that do not have to be nested.
So your output can specify an export:
Outputs:
Desc:
Value: !Ref CubesNetworking.VolumeID
Export:
Name: some-unique-name
Then in another stack:
Fn::ImportValue: some-unique-name