I have a lambda deployed in a stack, that is referenced via ARN by resources that I do not manage. It is problematic that certain updates to other resources can trigger this lambda to be redeployed. So therefore I would like to put a stack policy in place like what is described here, but from within the CDK project itself. I've looked through the stack construct's docs but don't see any API's exposed. I've also looked here at the actual Stack CFN resource, and don't see it as a property you can supply.
Any help or advice would be appreciated!
Related
Is there any way to rollback all the changes made by an AWS Lambda by using the SDK? E.g. the lambda created and launched by a CloudFormation template creates a bucket via the AWS SDK. In case something fails it would be great to have a 'stack rollback' for the same stack that deployed the lambda as well (and all the resources created by the lambda reverted as well).
Or alternatively: how can I 'remember' from my lambda which resources were created so that I can rollback them and delete them when the lambda is called afterwards with a 'Delete' event?
I'm assuming you mean custom resources, as that's the only way you can run scripts in cloudformation.
Custom resources have a property called pysicalReourceId. You can use it after your create event to provide info over the resource you've created. When updating or deleting the resource, the id is provided to the lambda event so you can use it. A guide can also be found here: https://advancedweb.hu/how-to-use-the-physicalresourceid-for-cloudformation-custom-resources/
If for some reason it's not possible to use the resource ID I'd use tagging. When creating, tag your resources and when deleting, fetch the resources based on their tag and delete them.
Goal:
I need to create an AWS ManagedPolicy that contains ALLOW permissions for API actions on resources created in a pre existing stack. No I cannot modify the existing stack template and simply add a policy to it. I need to create a new stack that deploys a policy that enables actions on the existing stacks resources
Solution:
Create a CDK project to generate and deploy this policy stack. Within this CDK project I want to load the existing stack and iterate over its resources adding permissions to my new stack's policy.
Problem:
I don't see any way to load an existing stack in CDK. I was hunting around for a "Stack.fromArn(...)" but don't see anything even similar.
Question:
Is there some obsucre way to do this? Or is it simply not supported?
I did not tried it, however it looks like if you can access/lookup at least one construct from the existing stack, you can use the method Stack.of(construct) https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_core.Stack.html#static-ofconstruct to lookup the first stack scope in which the construct is defined. Not sure however how you could iterate resources in the looked up stack construct.
It might be not be the best answer, however one option could be to export the outputs for resources in existing stack which you want to include in the policy, and import these values in the new stack where you create the policy.
Is there a way to incorporate existing AWS resources that were created outside of CloudFormation into an existing CloudFormation stack? I'd like to do this without having to add a new resource in the CloudFormation stack and migrate the existing resource's data over to that new resource. I see that AWS now has drift detection for CloudFormation stacks. I'm wondering if that might be able to be leveraged to incorporate existing resources into a stack.
The ability to import/adopt resources into an existing CloudFormation stack is the #1 ask from CloudFormation customers. We've been thinking about ways to do it for a while, but haven't hit upon the mechanism that both fits customer needs and works at the scale the service operates.
Since we don't expose stack state info anywhere outside the service for you to modify, the only approach you can take until we offer an adoption feature is to either store metadata about the resources in a parameter store, or use a custom resource as a wrapper to retrieve the information about the underlying resource and then surface it to your stack via Fn::GetAtt.
Now you finally can do it with Resource Import feature, references:
https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md
https://twitter.com/shortjared/status/1193985448164691970?s=21
You can do this by passing existing resource information to your stack via Parameters. Here is an example of how to pass these parameters to the stack.
Check out this blog post from Eric Hammond describing how you can incorporate these parameters into the rest of the stack. The use-case described is a bit different in that they are optionally creating new resources if they aren't passed in, but the overall structure applies to the case you've described.
In this case I don't think Drift Detection will help you, since it will show differences between deployed resources and the configuration described in a stack. Resources defined/created outside of the stack won't be checked.
Amazons CDK (currently in the stage of developer preview as of writing) offers a way to do that:
If you need to reference a resource, such as an Amazon S3 bucket or VPC, that's defined outside of your CDK app, you can use the Xxxx.import(...) static methods that are available on AWS constructs. For example, you can use the Bucket.import() method to obtain a BucketRef object, which can be used in most places where a bucket is required. This pattern enables treating resources defined outside of your app as if they are part of your app.
Source: https://docs.aws.amazon.com/CDK/latest/userguide/aws_construct_lib.html
It also allows to import existing CloudFormation templates:
https://docs.aws.amazon.com/CDK/latest/userguide/use_cfn_template.html
Importing existing resources to stacks is now supported by CloudFormation :
Announcement from AWS : AWS CloudFormation Launches Resource Import
Instructions Via an example : HERE
Cloudformer might help you to create a new stack from existing resources and then you can add more resources to the stack. But don't know of a way to "merge" an existing stack with existing resources outside the stack.
Im my case I needed to import an ARN value from an existing SAM output in my account, so that I could add the proper invoke policy in my new stack.
I was looking for an equivalent of SAM's Fn::ImportValue, and found out that the core module has a static Fn.importValue method you can use as such:
const cdk = require('#aws-cdk/core');
const lambda = require('#aws-cdk/aws-lambda')
class MyStack extends cdk.Stack {
constructor(scope, id, props) {
super(scope, id, props);
// The below line did the trick
const arn = cdk.Fn.importValue(`your-sam-function-export-name`)
const myLambda = lambda.Function.fromFunctionArn(this, 'myLambda', arn)
// ...
}
}
Reference: https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_core.Fn.html
I have a cloudformation template that creates an S3 bucket as part of a cloudformation stack. On the new version of my template, I 'm planning to migrate my application from S3 to EFS.
Is there a way to remove the S3 bucket resource from the template, without having it deleted? Ideally, I would like my older users to have the s3 bucket available after they upgrade, but for the new users to not have it at all. It looks like DeletionPolicies could help here, but the documentation on it says that it only applies to stack deletion, but not upgrades.
Going to elaborate on user3470009's answer.
The main, advertised purpose of the DeletionPolicy is to keep a resource when a stack is deleted. It's mentioned almost as an afterthought in the AWS docs for DeletionPolicy that it also functions during resource removal from a stack:
Note that this capability also applies to stack update operations that
lead to resources being deleted from stacks. For example, if you
remove the resource from the stack template, and then update the stack
with the template.
So the workflow to remove a resource from a stack without deleting the actual resource is:
Add "DeletionPolicy" : "Retain" to the resource declaration in your CF template
Apply changes by either saving in the UI or running aws cloudformation on the CLI or whatever other tool you use
Check in the UI that your resource has the correct changes. There are some gotchas about when CF doesn't update the metadata. See the docs link above
Remove the resource from your template
Apply changes. Watch the events log to see that it says DELETE_SKIPPED:
2018-10-15T15:32:32.956Z HostedZone DELETE_SKIPPED
Setting a DeletionPolicy of "Retain" will cause the bucket itself to remain after a stack update that deletes the resource.
I came across this question requiring a slight variation. I needed to extract my bucket to another stack and can not delete it in the move. This method worked well:
create a new stack with the bucket in question. (note: you now have 2 stacks referencing the same bucket)
remove the bucket from the original stack. The resource is deleted from the original stack but not from S3 since it is still referenced in your new stack.
I also tested Houser's response above and confirmed the bucket will not be deleted if it contains files. While this works, it does attempt to delete the bucket 3 times before it completes (and reports errors each time). migrating to a new stack will not throw any errors.
When you remove a resource from your template, and update a stack from this template, the resources will be deleted. There is no way to avoid that.
Since your existing users will continue using the S3 bucket, I would recommend preserving the bucket in your template. Remove it when the bucket has been removed from your product completely.
If needed, you could version your template (old vs. new).
If you absolutely need to remove the bucket from your template, you may be able to use a loophole. When CloudFormation deletes a bucket, the bucket must be empty. If it's not empty, then the bucket should be preserved and removed from your stack. You could experiment and see if it works for you. If it works in testing, then you can try using it in production.
I have a large template for CloudFormation that has hundreds of resources. All are successfully updated, during an update, except one: an SNS alarm topic.
When deploying the stack, I get no errors, but even if the topic is non-existent the topic is never created.
I'm not expecting anyone to be able to provide me with a solution, but I would simply like to know how to troubleshoot the problem. It would be helpful to get output from the deployment, but the events are so few and really don't reflect the amount of resources being updated/created that they rarely help finding out what goes wrong.
Validation of the template is also successful, but that's almost a given since deploying also succeeds.
Regarding troubleshooting live CloudFormation stacks in general, CloudFormation just recently added support for Change Sets, which should help you preview changes and troubleshoot potential issues with updates before you attempt to apply them to your running stack.
Regarding SNS topics specifically, creating an SNS topic from scratch using the AWS::SNS::Topic resource works correctly. However, if you are using a TopicName property in your SNS resource, make sure that the name is unique across your entire AWS Account, as noted in the documentation:
Resource names must be unique across all of your active stacks. If you reuse templates to create multiple stacks, you must change or remove custom names from your template.
So reusing a constant TopicName in a stack deployed multiple times could cause the issue you're describing.
Also, if you're attempting to update existing CloudFormation-created topics with added/removed subscriptions, note the following important notice in the documentation:
Important
After you create an Amazon SNS topic, you cannot update its properties by using AWS CloudFormation. You can modify an Amazon SNS topic by using the AWS Management Console.
As a potential workaround for adding/removing subscriptions to existing SNS topics via CloudFormation, there is a cloudformation-helpers library containing a Custom::SnsSubscription resource (example) that might help.