Context
I have created a AWS Logs SubscriptionFilter using CDK. I am now trying to create a metric/alarm for some of the metrics for this resource.
Problem
All the metrics I am interested in (see ForwardedLogEvents, DeliveryErrors, DeliveryThrottling in the Monitoring AWS Logs with CloudWatch Metrics docs) requires these dimensions to be specified:
LogGroupName
DestinationType
FilterName
The first two are easy to specify since the LogGroupName is also required while creating the construct and DestinationType in my case is just Lambda. However, I see no way to get FilterName using CDK.
Using CloudWatch, I see that the FilterName is like MyStackName-MyLogicalID29669D87-GCMA0Q4KKALH. So I can't directly specify it using a Fn.ref (since I don't know the logical id). Using CloudFormation, I could have directly done Ref: LogicalId.
I also don't see any properties on the SubscriptionFilter object that will return this (unlike most other CDK constructs this one seems pretty bare and returns absolutely no information about the resource).
There are also no metric* methods on SubscriptionFilter object (unlike other standard constructs like Lambda functions, S3 buckets etc.), so I have to manually specify the Metric object. See for example: CDK metric objects docs.
The CDK construct (and the underlying CloudFormation resource: AWS::Logs::SubscriptionFilter) does not let me specify the FilterName - so I can't use a variable to specify it also and the name is dynamically generated.
Example code that is very close to what I need:
const metric = new Metric({
namespace: 'AWS/Logs',
metricName: 'ForwardedLogEvents',
dimensions: {
DestinationType: 'Lambda',
// I know this value since I specified it while creating the SubscriptionFilter
LogGroupName: 'MyLogGroupName',
FilterName: Fn.ref('logical-id-wont-work-since-it-is-dynamic-in-CDK')
}
})
Question
How can I figure out how to acquire the FilterName property to construct the Metric object?
Or otherwise, is there another way to go about this?
I was able to work around this by using Stack#getLogicalId method.
Example code
In Kotlin, as an extension function for any Construct):
fun Construct.getLogicalId() = Stack.of(this).getLogicalId(this.node.defaultChild as CfnElement)
... and then use it with any Construct:
val metric = Metric.Builder.create()
.namespace("AWS/Logs")
.metricName("ForwardedLogEvents")
.dimensions(mapOf(
"DestinationType" to "Lambda",
"LogGroupName" to myLogGroup.logGroupName,
"FilterName" to mySubscriptionFilter.getLogicalId()
))
.statistic("sum")
.build()
Related
I am creating two Stack using AWS CDK. I use the first Stack to create an S3 bucket and upload lambda Zip file to the bucket using BucketDeployment construct, like this.
//FirstStack
const deployments = new BucketDeployment(this, 'LambdaDeployments', {
destinationBucket: bucket,
destinationKeyPrefix: '',
sources: [
Source.asset(path)
],
retainOnDelete: true,
extract: false,
accessControl: BucketAccessControl.PUBLIC_READ,
});
I use the second Stack just to generate CloudFormation template to my clients. In the second Stack, I want to create a Lambda function with parameters S3 bucket name and key name of the Lambda zip I uploaded in the 1st stack.
//SecondStack
const lambdaS3Bucket = "??"; //TODO
const lambdaS3Key = "??"; //TODO
const bucket = Bucket.fromBucketName(this, "Bucket", lambdaS3Bucket);
const lambda = new Function(this, "LambdaFunction", {
handler: 'index.handler',
runtime: Runtime.NODEJS_16_X,
code: Code.fromBucket(
bucket,
lambdaS3Key
),
});
How do I refer the parameters automatically from 2nd Lambda?
In addition to that, the lambdaS3Bucket need to have AWS::Region parameters so that my clients can deploy it in any region (I just need to run the first Stack in the region they require).
How do I do that?
I had a similar usecase to this one.
The very simple answer is to hardcode the values. The bucketName is obvious.
The lambdaS3Key You can look up in the synthesized template of the first stack.
More complex answer is to use pipelines for this. I've did this and in the build step of the pipeline I extracted all lambdaS3Keys and exported them as environment variable, so in the second stack I could reuse these in the code, like:
code: Code.fromBucket(
bucket,
process.env.MY_LAMBDA_KEY
),
I see You are aware of this PR, because You are using the extract flag.
Knowing that You can probably reuse this property for Lambda Key.
The problem of sharing the names between the stacks in different accounts remains nevertheless. My suggestion is to use pipelines and the exported constans there in the different steps, but also a local build script would do the job.
Do not forget to update the BucketPolicy and KeyPolicy if You use encryption, otherwise the customer account won't have the access to the file.
You could also read about the AWS Service Catalog. Probably this would be a esier way to share Your CDK products to Your customers (CDK team is going to support the out of the box lambda sharing next on)
Is there anyway to make CloudFormation parameter dynamic? I know about the System Manager Parameter, but again I have to change its value manually. I want to use somehow the result of the API call or script(Bash, python) in my CloudFormation resources
for example, as part of the parameter, run a API call to get back some data (any data) and then use/reference the result into the resources, and all in one template.
You can use Cloudformation Custom resource to achieve similar effect, with some caveats.
As an example we can use AWS CDK, which provides a module to create custom resources, and even has a wrapper specifically designed to call AWS API and return the results: https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_custom-resources.AwsSdkCall.html
Some things to remember:
Custom resource needs to return value in form {'PhysicalResourceId': ..., Data: {"MyAttribute": ...}} in order to support using !GetAtt MyResource.MyAttribute style of reference
Like any other CF resource, Custom resource is not triggered on every update, only if one of the parameters of the resource has changed. So if you supplied some parameter to your API call on stack creation, unless you change value, no update will happen and API call will not be triggered.
I am quite new to the CDK, but I'm adding a LogQueryWidget to my CloudWatch Dashboard through the CDK, and I need a way to add all LogGroups ending with a suffix to the query.
Is there a way to either loop through all existing LogGroups and finding the ones with the correct suffix, or a way to search through LogGroups.
const queryWidget = new LogQueryWidget({
title: "Error Rate",
logGroupNames: ['/aws/lambda/someLogGroup'],
view: LogQueryVisualizationType.TABLE,
queryLines: [
'fields #message',
'filter #message like /(?i)error/'
],
})
Is there anyway I can add it so logGroupNames contains all LogGroups that end with a specific suffix?
You cannot do that dynamically (i.e. you can't make this work such that if you add a new LogGroup, the query automatically adjusts), without using something like AWS lambda that periodically updates your Log Query.
However, because CDK is just a code, there is nothing stopping you from making an AWS SDK API call inside the code to retrieve all the log groups (See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CloudWatchLogs.html#describeLogGroups-property) and then populate logGroupNames accordingly.
That way, when CDK compiles, it will make an API call to fetch LogGroups and then generated CloudFormation will contain the log groups you need. Note that this list will only be updated when you re-synthesize and re-deploy your stack.
Finally, note that there is a limit on how many Log Groups you can query with Log Insights (20 according to https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html).
If you want to achieve this, you can create a custom resource using AwsCustomResource and AwsSdkCall classes to do the AWS SDK API call (as mentioned by #Tofig above) as part of the deployment. You can read data from the API call response as well and act on it as you want.
I apologize if this question is unclear in any way - I will do my best to add detail if it is difficult to understand. I have an AWS Lambda, from which I would like to access the tags for that same lambda. I have found the listTags method for AWS Lambda, which appears to be what I am looking for. It can be called as follows:
var params = {
Resource: "arn:aws:lambda:us-west-2:123456789012:function:my-function"
};
lambda.listTags(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
However, in order to use this function, we have to create a new instance of the lambda using the lambda constructor:
var lambda = new AWS.Lambda({apiVersion: '2015-03-31'});
I don't think that this is what I want to do. Instead, I want to have access to the tags for this particular lambda whenever the lambda is run. So, if I invoke the lambda, I want that invocation to be able to look and see that the lambda, itself, has a tag with the key "environment" and value "production," for example. I wouldn't think I would want to construct a new instance from within it... of itself.
Surely there has to be a way to do this? I may be missing something obvious. I've tried the code I've provided above using the context object in place of the lambda, but to no avail.
You should consider Lambda Environment Variables.
AWS Tags simply is metadata used to organise resources, aws documents indicates their usage as:
- Tags for resource organization
- Tags for cost allocation
- Tags for automation
- Tags for access control
Ref: https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html
I have few lambdas created by automation using Ansible.
- lambda:
name: 'NAME'
state: present
zip_file: 'index.js.zip'
tags:
createdBy: 'ansible'
And few more lambdas which are created manually.
I would like to delete all lambdas which are created by "Ansible", so I added "tag" attribute to all the automated lambdas.
I know, we can delete lambda if we have its name, but I would like to get all lambdas and filter the lambda which has tags['createdBy']='ansible.
lambda_facts are a way to get all lambda configuration, but it doesn't give me tag details.
How do I delete lambdas by filtering tags ?
I suggest to get all your functions with help of lambda.listFunctions and run lambda.listTags for each Lambda and build your own array with all functions which includes ansible as a tag.
In the end iterate over this array and call lambda.deleteFunction for each entry.
(I dont know which language you prefer. All my examples are using the JavaScript SDK)