I'm facing a decision to Use Cross-Stack References to Export Shared Resources or to Use Nested Stacks to Reuse Common Template Patterns following AWS CloudFormation best practices.
However, they seem the same to me apart from a few differences:
cross-stack uses Fn::ImportValue, templates are in one folder.
nested-stack must be in S3, uses type AWS::CloudFormation::Stack and TemplateURL.
There's no clear pros and cons between them as far as I could search.
My goal is to create a parent stack that passes some core variables like stackName to the child stacks, then the child stacks create the resources sharing some variables between them like ARN or Policies, using the stackName to name their resources like stackNameDynamoDBTable.
You should use cross-stack references as it was created for your use case of passing between stacks.
Whereas nested stacks would work, it’s primary purpose is for reuse of modular components, like a template of a resource you use in lots of stacks to save copy pasting and updating the stacks independently.
Nested stacks: if you need to manage your stacks from a single point, you should use nested stacks.
example: assume that you have load balancer configuration that you use for most of your stacks. Instead of copying and pasting the same configurations into your templates you can create a dedicated template for load balancer.
cross-stack : Alternatively, if you need to manage your stacks as separate entities, you should use cross-stack references.(AWS limits the number of VPCs you can create in an AWS region to five.)
example : You might have a network stack that includes a VPC, a security group, and a subnet. You want all public web apps to use these resources. By exporting the resources, you allow all stacks with public web applications to use them.
There is a way to get the best of both worlds. The trick is to use cross-stack resource sharing but make it depend on a parameter that is passed using Nested stack.
Here's an example from how I used this, consider two stacks IAMRoleStack and ComputeStack. The former contains all the necessary IAM roles and the latter contains a bunch of Lambda functions that those roles are applied to.
Resources:
IAMCustomAdminRoleForLambda:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Policies:
Output:
IAMRoleArnForLambda:
Description: Returns the Amazon Resource Name for the newly created IAM Custom
Role for Lambda function
Value: !GetAtt 'IAMCustomAdminRoleForLambda.Arn'
Export:
Name: !Sub '${AWS::StackName}-IAMRoleArnForLambda'
StackName:
Description: Returns name of stack after deployment
Value: !Sub ${AWS::StackName}
As you can see I've exported the IAM role but it's Name depends on the stack name that is calculated once the stack is deployed. You can read more about exporting outputs in the docs.
In the ComputeStack, I use this role by importing it.
Resources:
LambdaForCompute:
Type: AWS::Lambda::Function
Properties:
Role: !ImportValue
Fn::Sub: ${StackNameOfIAMRole}-IAMRoleArnForLambda
The parent stack that "nests" both ComputeStack and IAMRoleStack orchestrates passing the stack name parameter.
Resources:
IAMRoleStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Ref IAMRoleStackURL
ComputeStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Ref ComputeStackURL
Parameters:
StackNameOfIAMRole: !GetAtt IAMRoleStack.Outputs.StackName
I can't attest to best practice but this style allows me to pick and choose where I want orchestrated deployment and where I want to do the deployments individually.
I also want to point out that this kind of modularization based on type of resources is not very feasible for nested stacks. For e.g. in this scenario, if I had 10 different roles for 10 different Lambda functions, I would have to pass each of those 10 roles through parameters. Using this hybrid style, I only need to pass one parameter the stack name.
With cross stacks, you pass a reference to a bunch existing components X to stacks A and B when you want A and B to reuse these very same existing components. With nested stacks, when you nest a nested stack Y in stacks C and D, Y shall create a new set of components Y is describing individually for C and for D.
It is similar to concepts 'passing by reference' and 'passing by value' in programming.
Related
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: hello
Resources:
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
StageName: stage
TracingEnabled: true
FunctionA:
...
Environment:
Variables:
TEST: !Ref ApiGatewayApi
Events:
GetUsers:
Type: Api
Properties:
Path: /account
Method: get
RestApiId:
Ref: ApiGatewayApi
FunctionB:
...
Environment:
Variables:
API_URL: !GetAtt ApiGatewayApi.RootResourceId
Events:
OrderEvent:
Type: SQS
Properties:
Queue: !GetAtt OrderServiceQueue.Arn
This leads to a circular dependency. IF I do !Ref in a function that does not have an event with API, it does not complain about it. I read the premium support article from aws, blogs and other stack overflow questions but they are not similar to my question.
FunctionB successfully refers to the API gateway id while FunctionA does not.
I create the api outside the function, so I think it SHOULD !Ref the endpoint in it. Is there something else?
The circular reference is created by how AWS SAM uses the events in order to create the definition of the API. This basically means that it needs the ARNs of the lambda functions to construct this definition before it can create the API. But since you are needing IDs of the API in order to create the lambda, you end up with a circular reference since neither can be created without the other one already existing.
The first way to solve this problem is by deploying your stacks in multiple steps. You could first deploy an empty API, which would allow you to reference the API IDs when adding the lambdas. The significant drawback of this approach is of course if you want to easily replicate this stack on another account or redeploying the API for some reason, which means you'd have to use this trick again each time.
Another way, if you really want to have this value as an environment variable, would be to manually create the definition body for the API (in which you construct the ARNs of the lambda, not reference them) and presumably, you'll also need to manually create the permissions in order to allow your API Gateway resource to execute the lambda functions.
However, a better way I feel would be to use the lambda proxy integration (which is used by default I think, but I could not find any documentation to verify this). When using the lambda proxy integration, the incoming event in lambda contains all the information about the API. You could easily extract the information you need from that event instead of having it as an environment variable, but this depends on your exact use case.
I am using Terraform for most of my infrastructure, but at the same time I'm using the serverless framework to define some Lambda functions. Serverless uses CloudFormation under the hood where I need access to some ARNs for resources created by Terraform.
My idea was to create a CloudFormation stack in Terraform and export all of the value that I need, but it complains that it cannot create a stack without any resources. I don't want to define any resources in CloudFormation, only the outputs, so I though maybe there is a way to define some dummy resource, but I couldn't find any.
Is there a way to work around this issue? If not, I'm also open to other suggestions for getting parameters passed from Terraform to CloudFormation.
You can use AWS::CloudFormation::WaitConditionHandle for this. Example:
Resources:
NullResource:
Type: AWS::CloudFormation::WaitConditionHandle
The Resource section is required, but you can create non-resource type of resource.
For example, minimalist template with only a non-resource would be:
Conditions:
Never:
!Equals [ "A", "B" ]
Resources:
NonResource:
Type: Custom::NonResource
Condition: Never
Outputs:
MyOutput:
Value: some-value
You can use create AWS SSM parameter using Terraform and reference them in your serverless framework. That would do the job easily.
https://www.serverless.com/blog/definitive-guide-terraform-serverless/
I would like to create a custom resource in cloudformation that will contain sub resources so that I can prevision application resources for high value clients.
For example. I have an application that has an SNS::Subscription, SQS::Queue, ElasticBeanstalk::Environment, ElasticBeanstalk::Application and ElasticBeanstalk::ApplicationVersion.
Copying this for any client I want to provision dedicated resources for is a hassle. Keeping them all up to date is a problem.
I would like to define these once, and use a custom resource to generate them with custom variables provided. Like-
GeneralPurpose:
Type: COM::MyApplication
Properties:
QueueName: general
InstanceType: t3.micro
Instances: 30
AcmeClient:
Type: COM::MyApplication
Properties:
QueueName: acme
InstanceType: t5.medium
Instances: 10
SnsFilterValue: acme
These custom properties would be used in the custom resource template I defined and generate all the resources required for both the general and acme application.
If I need to change the default visibility timeout for the application queue, I can do it in one place and update all resources that used this custom resource.
Is this possible?
Consider using a nested stacks instead of a custom resource. You can define parameterized templates to use as the nested stacks. This will be much easier to maintain.
Is there a way to access auto-generated URLs for deployed resources before the deployment is finished? (like db host, lambda function URL, etc.)
I can access them after the deployment is finished, but sometimes I need to access them while building my stack. (E.g. use them in other resources).
What is a good solution to handle this use-case? I was thinking about outputting them into the SSM parameter store from CloudFormation template, but I'm not sure if this is even possible.
Thanks for any suggestion or guidance!
If "use them in other resources" means another serverless service or another CloudFormation stack, then use CloudFormation Outputs to export the values you are interested in. Then use CloudFormation ImportValue function to reference that value in another stack.
See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html and https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
Within Serverless Framework, you can access a CloudFormation Output value using https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-cloudformation-outputs
If you want to use the autogenerated value within the same stack, then just use CloudFormation GetAtt function. See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html.
For example, I have a CloudFormation stack that outputs the URL for an ElasticSearch cluster.
Resources:
Search:
Type: AWS::Elasticsearch::Domain
Properties: <redacted>
Outputs:
SearchUrl:
Value: !GetAtt Search.DomainEndpoint
Export:
Name: myapp:search-url
Assuming that the CloudFormation stack name is "mystack", then in my Serverless service, I can reference the SearchUrl by:
custom:
searchUrl: ${cf:mystack.SearchUrl}
To add to bwinant's answer, ${cf:<stack name>.<output name>} does not work if you want to reference a variable in another stack which is located in another region. There is a plugin to achieve this called serverless-plugin-cloudformation-cross-region-variables. You can use it like so
plugins:
- serverless-plugin-cloudformation-cross-region-variables
custom:
myVariable: ${cfcr:ca-central-1:my-other-stack:MyVariable}
I have a yml cloudformation template (A) for an AWS codepipeline build that I want to make a variation of it in an another template (B).
The original (A) has a repository as one of its resources which it created when initially run thru cloudformation. I'd like the variation (B) template to use the same ECR repository generated in the original (A), for the codebuild.
Is there a way I can have (B) template use the ECR resource created in A by passing in the repository resource value as a parameter or something?
For example the resource in A that I want to reuse (not recreate) in B is something like :
Repository:
Type: AWS::ECR::Repository
Properties:
RepositoryName: !Sub comp/${ServiceName}
RepositoryPolicyText:
Version: 2012-10-17
Statement:
...
I am not sure from your question what resources you are referring to. But in general you can export any value from one stack into another, using the Export property of the Output section
From Exporting Stack Output Values
To share information between stacks, export a stack's output values.
Other stacks that are in the same AWS account and region can import
the exported values. For example, you might have a single networking
stack that exports the IDs of a subnet and security group for public
web servers. Stacks with a public web server can easily import those
networking resources. You don't need to hard code resource IDs in the
stack's template or pass IDs as input parameters.