I am creating aws resources via cloudformation template.
I need to call a lambda after the creation of a DynamoDB table resource which will populate certain default records in it. After populating the default records only the rest of the resources should be created.
Sounds like you should create a custom resource, see docs custom resources in cfn. Let this lambda create what ever records you need and then call back with a success signal once done. What ever resources you need to put on hold can have a "DependsOn" on your custom resource.
Related
Is there any way to rollback all the changes made by an AWS Lambda by using the SDK? E.g. the lambda created and launched by a CloudFormation template creates a bucket via the AWS SDK. In case something fails it would be great to have a 'stack rollback' for the same stack that deployed the lambda as well (and all the resources created by the lambda reverted as well).
Or alternatively: how can I 'remember' from my lambda which resources were created so that I can rollback them and delete them when the lambda is called afterwards with a 'Delete' event?
I'm assuming you mean custom resources, as that's the only way you can run scripts in cloudformation.
Custom resources have a property called pysicalReourceId. You can use it after your create event to provide info over the resource you've created. When updating or deleting the resource, the id is provided to the lambda event so you can use it. A guide can also be found here: https://advancedweb.hu/how-to-use-the-physicalresourceid-for-cloudformation-custom-resources/
If for some reason it's not possible to use the resource ID I'd use tagging. When creating, tag your resources and when deleting, fetch the resources based on their tag and delete them.
I'm designing an AWS Lambda function for a Cloudformation Custom Resource as per this documentation.
Upon creation of the Custom Resource, the associated Lambda function should (a) create an external resource, and (b) return the ID associated with that external resource.
Upon the update or deletion of the Custom Resource, the associated Lambda function should take the ID of the existing external resource as a parameter/property, and update/delete that external resource accordingly.
To do so, the CloudFormation stack needs to store that ID returned by the Lambda upon the creation of the Custom Resource. Ideally, it should also associate that ID with the Custom Resource itself.
How can this be achieved?
Example code for the scenario in question:
Resources:
CustomResource:
Type: AWS::CloudFormation::CustomResource
Properties:
ServiceToken:
Fn::GetAtt: Lambda.Arn
ID: <see-below>
Lambda:
Type: AWS::Lambda::Function
Properties:
...
(where ID doesn't exist before the creation of CustomResource, but it needs to be populated by the result of Lambda upon creation)
You are confusing several things.
Upon the update or deletion of the Custom Resource, the associated Lambda function should take the ID of the existing external resource as a parameter/property, and update/delete that external resource accordingly.
This is handled via the PhysicalResourceId property. When you create a resource, you are required to return this in the response. CloudFormation will then pass this value to the Lambda for an update or delete.
The contents of that ID are opaque to CloudFormation, and you can store up to 1KB in it. So you can get quite creative with what you store in it. For example, you could store a URL that you use to invoke some third-party web-service.
ID: <see-below>
In this case, you're specifying a property that you pass to the custom resource Lambda. You will find it in the ResourceProperties property of the event. The example that you linked to doesn't show any custom properties, but if you log the invocation you'll see ID there.
return the ID associated with that external resource
You probably shouldn't return the ID of the external resource; it should remain an opaque identifier. But if you do want to return it, then you'll need to add a Data element to the custom resource response. This is a JSON object that contains resource "attributes" that can be retrieved using Fn::GetAtt.
For more information on custom resources, I recommend this doc, which specifies all of the fields in the request and response object.
Also, if you're using the cfnresponse module (which appears any many of the AWS official docs), you shouldn't: it relies on a "vendored" copy of requests in the boto3 library, and that's been deprecated for at least two years. At some point you'll find that your templates stop working (and failed custom resources can be very difficult to recover from).
I'm new to this and I'd like to get some ideas in terms of a code that can dynamically tag AWS resources. I'm confuse as to what will trigger the execution of the code that will tag it. Can someone please point me to right resources and sample codes?
You need to monitor CloudTrail events for creation of resources you would like to tag and invoke a Lambda function for the matching events, which tags
the resources accordingly.
CloudWatch Event Rule is setup to monitor :create* API calls via CloudTrail.
This rule triggers the lambda function whenever a matching event found.
The Lambda function fetches the resource identifier and principal information from the event and tags the resources accordingly.
I've devised a solution to tag EC2 resources for governance. It is developed in CDK Python and uses Boto3 to attach tags.
You can further extend this code to cover other resource types or maintain a DynamoDb table to store additional tags per principal
such as Project, Team, Cost Center. You can then simply fetch the tags of a principal and apply them all at once.
You can write lambda functions and use Cloudwatch events to trigger that function which will assign tag to your resources.
You can use AWS nodejs-sdk or boto3 for Python.
I am trying to more effectively manage the resources we create in our AWS accounts and I would like to start by attaching a lambda or many lambdas whenever a resource is created. At a minimum, I need to tag the resources because we simply can't count on people to do it at all, much less correctly.
For example: I can get an event anytime an object is deleted in S3 but that isn't what I want; I want to know when a bucket it created; either through the console of a CFT or the CLI
The closest thing I can see is CloudFormation events man be monitored. We do a lot of stack creation of resources but not always so this isn't good enough. And, in any event, I would need to know all the resources that were created with that stack which the documentation doesn't make clear if I could even get
Can this be done? If so, how?
CloudTrail tracks user activity and API usage and generates CloudTrail Trails. Trails have Data Events.
CloudTrail Data Events can be passed directly to a Lambda function for processing (and/or S3 and/or CloudWatch).
In my own case we store CloudTrail Events in a S3 bucket but also pass them to a Lambda function that applies a billing tag to all new resources created in the account.
GorillaStack Autotag might be a good starting point as a reference for the function.
Is it possible to send a SNS notification after the CFT completion in AWS ? Is there any way to get the progress of the launching CFT in AWS.
When create resources using a CF template there is an Advanced section of the Options menu. From there you can set Notification options using SNS and Topics.
When you start the CF process you can also view the status and importantly where the template might have failed.
You cannot specify notification ARNs via a CloudFormation template itself. You can specify them if you use the console to create the stack creation. But you cannot use the console to update the ARNs once the stack has been created. You can, however, use aws-cli to update the stack with notifications ARNs once it has been created, eg:
aws cloudformation update-stack --stack-name stack-name --use-previous-template --notification-arns "arn:aws:sns:us-east-1:${ACCOUNT_ID}:${TOPIC_NAME}"
Replace the variable ${VARIABLE} with the literal values from your account.
There's also knowledge center article from AWS where you can replace ROLLBACK_IN_PROGRESS statement with any other state of CloudFormation to get SNS Notification.
You can trick CloudFormation into sending SNS messages from inside the template:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html
Custom resources enable you to write custom provisioning logic in
templates that AWS CloudFormation runs anytime you create, update (if
you changed the custom resource), or delete stacks. For example, you
might want to include resources that aren't available as AWS
CloudFormation resource types. You can include those resources by
using custom resources. That way you can still manage all your related
resources in a single stack.
Use the AWS::CloudFormation::CustomResource or Custom::String resource
type to define custom resources in your templates. Custom resources
require one property: the service token, which specifies where AWS
CloudFormation sends requests to, such as an Amazon SNS topic.