Is it possible or any way around to deploy multiple stack from a single stack? For example, I define three stacks namely DynamoDBStack, IAMRoleStack and LambdaStack, now I want these stacks to be deployed under, lets say DeployAllStacks, so that it's identified as single stack in AWS CloudFormation stack list? I realised the following approach is wrong and cdk ls recognise as 4 stacks instead of one, I'm trying something like this:
public DeployAllStacks(Construct scope, string id, IStackProps props = null):base(scope,id, props)
{
new DynamoDBStack(this, "DataBase");
new IAMRoleStack(this, "IAMRole");
new LambdaStack(this, "Lambda")
}
I understand that if I write all the resources in one single stack then it's doable, however I want the code to be neater and write separate stack to reused elsewhere.
You can't. Stacks are the unit of deployment in the CDK. They can be grouped, but deploy separately in the cloud.
No matter. If code reuse is your goal, construct subclasses are the idiomatic CDK approach to composition. This is how the CDK itself is built. Multi-stack Apps also have a role for bigger projects, although this adds friction to deploys and cross-stack references*. See the CDK best practices guidance on Constructs and Stacks for design considerations. If I am interpreting your stack names correctly, one stack seems a good option, with construct subclasses for code org/reuse if needed.
* The CDK also supports CloudFormation nested stacks, which is a non-CDK native approach to reuse. But the child stacks are still stacks that "show up in the list".
Related
I'm trying to understand the use case of Nested Stacks vs. Constructs specifically in CDK. The AWS docs say the following:
Stacks are the unit of deployment: everything in a stack is deployed together. So when building your application's higher-level logical units from multiple AWS resources, represent each logical unit as a Construct, not as a Stack. Use stacks only to describe how your constructs should be composed and connected for your various deployment scenarios.
This makes sense when evaluating whether to use a Construct or Stack, but what about Nested Stacks? Both Constructs and Nested Stacks solve the problems of:
reusability of component architectures
controlled information sharing between components / mitigating import and export (deadly embrace) issues
and both Constructs and Nested Stacks are deployed together from the root Stack (from what I understand, NestedStacks are rarely deployed alone and are intended to be deployed as part of a group of NestedStacks under one parent Stack)
So what's the benefit of using Nested Stacks over Constructs besides working around the resource limitations of a single Stack (i.e. when should I use one over the other)?
It's instructive how differently the CloudFormation docs and CDK docs present Nested Stacks. The CloudFormation docs focus on their role in component resuse. The CDK docs don't mention reuse, instead presenting them as a workaround for the per-stack resource limit. Of course, Nested Stacks do both things.
CDK Constructs are more composable and portable than the Nested Stacks it inherited from CloudFormation. The CDK docs recommend Constructs for composition here and here.
Apart from overcoming the per-stack resource limit, there is a backwards compatability rationale for using Nested Stacks when including existing CloudFormation templates into CDK apps.
I use CDK to deploy a lambda function (along some IAM role & queue) and monitoring resources about the lambda, lambda log group and queue earlier. What i have right now is basically 2 class, 1 class to create all the lambda related resource and another to create monitoring resource and they are added all into 1 deployment stack.
Recently im deploying this to a new account and i realized my stack fail to create because some of the monitoring stuff is looking for the lambda log group and cant find it since its not created yet.
So what is the better option:
have 2 deployment group, 1 for lambda related resource and 1 for monitoring resource
use dependencies to create some ordering in my stack.
seems like both possible solution but what is a better long term solution?
Assuming you mean a Stack for your two classes, then you are better off making them both cdk.NestedStacks and instantiating them in a single common stack. You can then expose constructs as class attributes in one stack and pass them into the other as parameters to the second. Of course, this only works one way - if you have to go both ways you need to re-evaluate how you have your stacks organized.
The advantage of doing this is great: exposing constructs as an attribute is the best practice as it gives you direct access to that construct before it creates the CloudFormation data for it. you have complete access to every part of that construct from various arns (like dynamodb stream arns which are difficult to import) and automatically know the layer versions for lamdba layers - among many other things.
In addition, you never run into a stack dependency - if they are different top level stacks and you share constructs between them you can very run into lock situations where attempting to change something in one stack creates a dependency lock and prevents the stack from deploying.
The downside is that they all are part of the deployment. So there is a potential for something to be updated when you didnt expect it too - though CDK does use the Cloudformation Changeset system so it should not update things that have no changes applied to them (but sometimes, changes occur because of the way CDK generates tokens and such that you may not be aware of)
IF you do not go this route you are stuck using the various from* methods in cdk constructs to import the existing construct into your stack. This causes some issues, as it it can't import everything about a given construct at synth time (layer version and dynamo stream arns are two notable ones i mentioned already). Plus, you need to know the name of the construct - and Best Practices says you shouldn't deliberately name your constructs so you can easily spin up adhoc versions of your app without naming issues.
In CloudFormation both nested stacks and modules encourage reusability by creating common components.
From the AWS documentation it is not coming out when to use what. Any suggestions?
Here is good experimentation with the following conclusions:
CloudFormation Modules are a good evolution of the nested stacks concept that arguably should have arrived a few years ago. And unfortunately, in their current state, they aren’t suitable for use in an enterprise—or any—environment.
So I think its better to stick with nested stacks, unless you have specific requirements to use modules.
The fundamental difference is that with a module you deploy a single stack for your productive resources whereas nested stack deploy 1 or many stacks at once, as the term "nested" suggests. I say "productive resources" to refer to the resources you actually want to deploy, excluding the overhead that comes with modules and nested stacks respectively.
The one thing modules and nested stacks have in common is that in both cases the underlying basic unit is just a regular stack.
Some key differences are:
Modules are instantiated via a user defined resource type MyOrganization::MyNamespace::MyApp::MODULE. A nested stack is created via resource type AWS::CloudFormation::Stack.
Modules are native to CFN. It is an abstraction of a modularized stack stored in S3 as a .zip file. A nested stack is a plain CFN stack as YAML/JSON stored in S3. You can view your modules in the console under CloudFormation/Registry/Activated extensions/Tab "Modules" but a nested stack can only be viewed at its storage path in S3.
Modules offer the advantage that you don't need to make your modularized stack file publicly accessible. A nested stack file must be exposed to its parent stack via a public http link to the S3 object (TemplateURL).
Modules are versionized via resource type AWS::CloudFormation::ModuleVersion, nested stacks are just regular stacks referencing one another and therefore there is no concept of versioning. A module version must be published first as CFN stack before deploying the module.
The official docs of AWS for CFN modules are quite sparse. The most comprehensible explanation of modules I found is this AWS blog post.
I am a new AWS CDK user. I am trying to understand the point of a CDK construct. I know that a CDK stack maps directly to a CloudFormation stack. I also know that a construct's purpose is to encapsulate multiple resources that work together. But since we are using real programming languages, why can't we just use a regular class to encapsulate related resources?
Problem:
Recently we encountered a problem with respect to maximum number of resources that you can declare in a single CloudFormation template.
A template can support a maximum of 200 resources and we are very close to reach that limit.
To specify more resources, we need to separate our template into multiple templates by using Nested Stacks,
We are evaluating a best approach to breakdown the template.
Our approach:
We have created a nested stack from our main stack and removed some of the resources from main stack and added them into new nested stack.
Error:
We encountered an error in nested stack e.g.
Resource already exists in Root stack
1. Is it possible to break our main template into nested stacks with existing resources? i.e move our existing resources under nested stack?
or we just need to add new resources into nested stack and keep our main stack as it is with existing resources ?
2. We already reached 200 resource limit so it's difficult to add more nested stacks as well.
Resources in template,
We roughly have 100 CloudWatch alarms, DynamoDB tables, Lambda's, ES, KMS, S3 and other resources, we wan't to separate them into nested stacks specific to resource types.
You can't 'incorporate' existing resources into new CloudFormation stacks. If you move resources into a nested stack and add the nested stack to your root template, it will try to create new resources in the nested stack first, then delete the old resources in the root stack as part of the cleanup process. As such if the new resources have the same names (and duplicate names are not allowed) then the update will fail.
The solution to this is to deploy the stack in two stages, first removing the resources from the root stack, and then adding them to the nested stack. This will result in a short time when those resources are missing from your environment, but if you keep the migrations small it should only be for a few minutes.
Resources which contain state (like DynamoDB, KMS, S3, etc) are much harder to migrate in this fashion because the data is obviously lost when you delete them. You either need to do a full migration process, creating a new resource with a different name in the nested stack, migrating data over, updating your application to use the new resource, and finally deleting the old resource, or accept that that's probably far too much work for an internal refactor and leave those resources in the root template.
AWS CloudFormation now supports increased limits on five service quotas. It is now 500 instead of 200. See if that help you out.
AWS increased limit to 500