I am creating aws resources via cloudformation template.
I need to call a lambda after the creation of a DynamoDB table resource which will populate certain default records in it. After populating the default records only the rest of the resources should be created.
Sounds like you should create a custom resource, see docs custom resources in cfn. Let this lambda create what ever records you need and then call back with a success signal once done. What ever resources you need to put on hold can have a "DependsOn" on your custom resource.
I'm new to this and I'd like to get some ideas in terms of a code that can dynamically tag AWS resources. I'm confuse as to what will trigger the execution of the code that will tag it. Can someone please point me to right resources and sample codes?
You need to monitor CloudTrail events for creation of resources you would like to tag and invoke a Lambda function for the matching events, which tags
the resources accordingly.
CloudWatch Event Rule is setup to monitor :create* API calls via CloudTrail.
This rule triggers the lambda function whenever a matching event found.
The Lambda function fetches the resource identifier and principal information from the event and tags the resources accordingly.
I've devised a solution to tag EC2 resources for governance. It is developed in CDK Python and uses Boto3 to attach tags.
You can further extend this code to cover other resource types or maintain a DynamoDb table to store additional tags per principal
such as Project, Team, Cost Center. You can then simply fetch the tags of a principal and apply them all at once.
You can write lambda functions and use Cloudwatch events to trigger that function which will assign tag to your resources.
You can use AWS nodejs-sdk or boto3 for Python.
Is there a way to incorporate existing AWS resources that were created outside of CloudFormation into an existing CloudFormation stack? I'd like to do this without having to add a new resource in the CloudFormation stack and migrate the existing resource's data over to that new resource. I see that AWS now has drift detection for CloudFormation stacks. I'm wondering if that might be able to be leveraged to incorporate existing resources into a stack.
The ability to import/adopt resources into an existing CloudFormation stack is the #1 ask from CloudFormation customers. We've been thinking about ways to do it for a while, but haven't hit upon the mechanism that both fits customer needs and works at the scale the service operates.
Since we don't expose stack state info anywhere outside the service for you to modify, the only approach you can take until we offer an adoption feature is to either store metadata about the resources in a parameter store, or use a custom resource as a wrapper to retrieve the information about the underlying resource and then surface it to your stack via Fn::GetAtt.
Now you finally can do it with Resource Import feature, references:
https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md
https://twitter.com/shortjared/status/1193985448164691970?s=21
You can do this by passing existing resource information to your stack via Parameters. Here is an example of how to pass these parameters to the stack.
Check out this blog post from Eric Hammond describing how you can incorporate these parameters into the rest of the stack. The use-case described is a bit different in that they are optionally creating new resources if they aren't passed in, but the overall structure applies to the case you've described.
In this case I don't think Drift Detection will help you, since it will show differences between deployed resources and the configuration described in a stack. Resources defined/created outside of the stack won't be checked.
Amazons CDK (currently in the stage of developer preview as of writing) offers a way to do that:
If you need to reference a resource, such as an Amazon S3 bucket or VPC, that's defined outside of your CDK app, you can use the Xxxx.import(...) static methods that are available on AWS constructs. For example, you can use the Bucket.import() method to obtain a BucketRef object, which can be used in most places where a bucket is required. This pattern enables treating resources defined outside of your app as if they are part of your app.
Source: https://docs.aws.amazon.com/CDK/latest/userguide/aws_construct_lib.html
It also allows to import existing CloudFormation templates:
https://docs.aws.amazon.com/CDK/latest/userguide/use_cfn_template.html
Importing existing resources to stacks is now supported by CloudFormation :
Announcement from AWS : AWS CloudFormation Launches Resource Import
Instructions Via an example : HERE
Cloudformer might help you to create a new stack from existing resources and then you can add more resources to the stack. But don't know of a way to "merge" an existing stack with existing resources outside the stack.
Im my case I needed to import an ARN value from an existing SAM output in my account, so that I could add the proper invoke policy in my new stack.
I was looking for an equivalent of SAM's Fn::ImportValue, and found out that the core module has a static Fn.importValue method you can use as such:
const cdk = require('#aws-cdk/core');
const lambda = require('#aws-cdk/aws-lambda')
class MyStack extends cdk.Stack {
constructor(scope, id, props) {
super(scope, id, props);
// The below line did the trick
const arn = cdk.Fn.importValue(`your-sam-function-export-name`)
const myLambda = lambda.Function.fromFunctionArn(this, 'myLambda', arn)
// ...
}
}
Reference: https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_core.Fn.html
I am trying to more effectively manage the resources we create in our AWS accounts and I would like to start by attaching a lambda or many lambdas whenever a resource is created. At a minimum, I need to tag the resources because we simply can't count on people to do it at all, much less correctly.
For example: I can get an event anytime an object is deleted in S3 but that isn't what I want; I want to know when a bucket it created; either through the console of a CFT or the CLI
The closest thing I can see is CloudFormation events man be monitored. We do a lot of stack creation of resources but not always so this isn't good enough. And, in any event, I would need to know all the resources that were created with that stack which the documentation doesn't make clear if I could even get
Can this be done? If so, how?
CloudTrail tracks user activity and API usage and generates CloudTrail Trails. Trails have Data Events.
CloudTrail Data Events can be passed directly to a Lambda function for processing (and/or S3 and/or CloudWatch).
In my own case we store CloudTrail Events in a S3 bucket but also pass them to a Lambda function that applies a billing tag to all new resources created in the account.
GorillaStack Autotag might be a good starting point as a reference for the function.
Lets say I have a CloudFormation stack running, which creates and deploys an Lambda function. In the AWS Console, if I connect my Lambda function to an API in API Gateway, will my CloudFormation Template be updated immediately if the Lambda function successfully integrates with the API?
It's a one way traffic from Cloudformation to resources.
Meaning if you modify your Cloudformation template and update the stack then the resources that were created by Cloudformation get modified/updated. However the other way is not true. Meaning if you modify your resources the Cloudformation template does not get updated.
Moreover, as a good practice you should avoid modifying the resources directly because you may end up breaking the Cloudformation's update stack functionality for that that stack