I have two CDK/Cfn stacks which instantiate application load balancers with SSL certificates. I'm using DNS validation which the CDK manages by creating a Lambda function which requests and validates the certificates.
Unfortunately, those Lambda functions were manually deleted and now when I try to update my CDK resources, CloudFormation attempts to replace these Lambdas but fails because they no longer exist.
I wish that CloudFormation would behave like Terraform and just say "oh that thing I need to replace isn't there, nbd I needed to replace it anyway, so let's carry on" but it does not.
Not sure how to get out of this jam. Any help is appreciated.
The easiest fix to this drift is redeploying your CDK app with the deleted resource temporarily removed (e.g. commented out). CloudFormation will "delete" the already deleted resource, bringing the template back into sync with the deployed configuration. Then add back the resource to your app and deploy again. Problem solved.
There's a complication in your case. The missing Lambda function is being constructed indirectly by a higher-level CDK construct. Removing the L2/L3 parent will destroy more resources than just the Lambda. If you want to avoid this collateral damage, you can use escape hatch syntax and the node.tryRemoveChild method to surgically remove the missing Lambda only.
You have to import them back to CloudFormation. In TF it is same, and you also import resources into TF.
Related
I am required to modify pre-existing resources using cdk. I understand that there are methods to call upon that allows you to import a given resource, but how can I go about modifying that resource? For example, I have an RDS that was manually created and I want to change the instance type after creation. How do I go about doing that using cdk/cloue formation?
If you'd like to 'take ownership' of a resource in AWS with CloudFormation you can follow the steps outline here. In short:
Create a CloudFormation template that only has the one RDS resource you'd like to take ownership of. Have the template match the resource as much as possible. You should use the CDK to do this, just make sure your CDK code ONLY includes those resources you want to import. That can sometimes be tricky since L2 constructs often create more than just one CloudFormation resource. Trim back the synthesized CloudFormation template as much as needed to get just the one RDS resource you want to import.
Create a new Stack using the 'import' option. It's important that the only resources in the template are resources you are trying to import and 'take ownership of'.
Run Drift Detection and correct anything that is out of sync by updating your template and then running additional Update Stack steps.
You can, of course, have the CDK generate this template. Same rules apply, though. You need to make sure you have only the RDS instance.
Please refer to this post as I go into more detail there.
Additionally, there is a command link option, cdk import which can help do this (not detailed here or in the blog, though).
Once you have the resource imported into a stack you can continue making future changes using the CDK.
As described in the documentation, I think CloudFormation will update with no interruption just by changing to add a tag.
In my case, another team uses terraform to add tag resources with generic tags, and my team uses CloudFormation to update the application-specific tags. Does this cause replacement...?
Also, this only occurred for limited resources such as security groups.
Does anyone know anything about this issue?
Cloudformation is a Jealous Tool - in otherwords, if it doesn't control the Resource in entirety it will overwrite changes, assuming that the changes are in a location it thinks it has to update.
In order for CloudFormation stacks not to do work that is not needed they generate ChangeSets - indicating what resources need updating/rebuilding/ect. If you don't change anything in the template (or your cdk stack) then the ChangeSet won't update that resource cause nothing needs to change. Depending on the resource (and it varies from service to service) sometimes changes made outside of CloudFormation control (by either other services like Terraform or by manual changes in the console - both of which are referred to as 'Drift') dont get overwritten by the stack updating (most common I can think of is API gateway - adding / deleting resources or methods doesn't always get re done by a redeploy of the stack.
However, for things like tags, yes - it will overwrite any changes made in there at any time and reapply the tags as of the moment it deploys - (again, depending on the Service) often even if there are no other changes to deploy for that given resource. I suspect the reason for this is because of the way AWS uses tags on the backend to sort and search resources.
Your best bet is to pick one service and do everything with it.
Each of these two tools will overwrite the tags the other created. You can configure Terraform to ignore certain tags However I'm not sure you can configure CloudFormation to ignore any tags. I think CloudFormation will always delete tags that it doesn't manage. In general it's not going to work well using both CloudFormation and Terraform to manage the same resources.
I followed all the steps mentioned in the official blog post here:
Remediate drift via resource import with AWS CloudFormation | AWS Management & Governance Blog
Instead of using "import resource" at step 4, if I use "update" button, I get the same results.
Select our stack and Import resource into stack, from the Stack actions menu.
What is the difference between Update and Import?
Update is normally used to tell CloudFormation that the template has changed. This can result in new resources being created, modified or deleted.
CloudFormation is not aware of any changes made outside of itself, so it is preferable to always change resources through CloudFormation so that it remains in-sync with the resource configuration.
That article seems to pushing the idea that, if something has changed outside of CloudFormation, it is best to "forget" about the resource, then Import it into the stack. An Import adds an existing resource to the stack without CloudFormation attempting to create it.
I guess Updating the DynamoDB table works because only the BILLING_MODE has changed. It is possible that CloudFormation can become confused when it tries to make a change that has already been made. For example, if a subnet needs to change AZs, it will need to delete and recreate the subnet. This would lead to resources with different IDs. In such a case, the recommended process of Forget + Import would need to be used.
Bottom line: The article is showing the general process, but in some situations, and Update might suffice.
I have a aws cognito user group configured to my serverless.yml. Whenever I do a serverless deploy, it will try to create the same user pool domain even though it already exist, hence returning me the error of:
[aws-cognito-idp-userpool] domain already exist
The only workaround is for me to delete the user pool domain every time I want to do a serverless deploy from the AWS UI. Anyone faced this issue before?
I believe there's no way to skip it,
Check this - https://github.com/serverless/serverless/issues/3183
You can try to break the serverless.yaml file into multiple files and deploy them separately for easier management,
So use the file only to create/deploy resources you need to freshly create.
The serverless.yaml will get converted into the vendor-specific Code to Infra service file,
eg. CloudFormation for AWS
Hope this helps
This is actually a CloudFormation issue vs. a Serverless issue. I ran into it in my Serverless app, BUT had my UserPool* resources independently defined in the resources section of the serverless.yml file. I changed the Domain Prefix and that requires the resource to be recreated. Here's the issue: CloudFormation always creates a resource first before deleting the old one, which blocks the new domain from being associated with the User Pool.
I've seen this behavior with other resources and the recommended behavior is to:
1. Blank out the resource from the template
2. Update the stack (deletes resource)
3. Restore the resource in template
4. Update the stack (creates a new one vs. replace).
This way you still leverage your automation tools without going to the console. Not perfect, and it'd be more preferable if there was a way to force the replacement sequence in CloudFormation. If your setup has Serverless generating the resource, then deleting via the console may be your only option.
I would like to make an automated call to a custom program API as soon as CloudFormation has completed the entire stack creation (deployment of instances, setup of VPC, Puppet scripts, etc.).
What is the correct way to go about this?
After some research, it seems a good option would be to launch an AWS Lambda function triggered by the event that stack creation has been completed successfully, but I have no idea how to approach this.
Any ideas or advice would be appreciated.
You can provision and coordinate a lot of what you're talking about (setup of VPC, etc.) with CloudFormation, the DependsOn attribute and nested CloudFormation stacks. This way you can order the execution of the CloudFormation stacks so that, for example, your VPC is created first followed by launching your EC2 instance(s) followed by the deployment of the software on the instance(s).
You can also coordinate the execution of the other behavior you mentioned (deployment [on] instances, [calling] Puppet scripts, etc.) using AWS::CloudFormation::Init. This way, you can call out to your Puppet scripts from your EC2 instance within the CloudFormation template. The actual execution of your Puppet scripts occurs on the EC2 instance(s).
If you want to see an example of calling out to a configuration management tool from CloudFormation (in this case, we're using Chef Solo), see app-instance.json.
If you'd like to see an example of using nested stacks, see dromedary-master.json.
There's also some examples of using Lambda on our blog as well (Stelligent), but it doesn't seem like you need to use Lambda in this case based on the problem you're trying to solve.
P.S. You don't have to use nested stacks either, but it can make things a little cleaner. But, you do want to control the creation order of the resources so the DependsOn attribute will help you in doing so.