Getting generated name in AWS Cdk - amazon-web-services

When creating a WebAcl in CDK and letting CDK generate the name, I want to use that generated name as a variable in CDK, i.e. when generating the WebAcl like this (no explicit name set in the properties) ...
const webAcl = new CfnWebACL(this, "webAcl", {
defaultAction: {
allow: {},
},
scope: myScope,
visibilityConfig: {
cloudWatchMetricsEnabled: true,
metricName: "webACL",
sampledRequestsEnabled: true,
}
});
... the webAcl will have a generated Name like webAcl-7xtQ0oTU473X after deployment (the id with an appended hash). The problem is that I do not know how to reference this name as a variable in CDK.
I found the following possibilities to get some type of ids/names but none of them resolve to webAcl-7xtQ0oTU473X after deployment (some, however, include the value I want):
webAcl.name // is undefined in CDK
webAcl.logicalId // resolves to 'webAcl' (no hash)
webAcl.attrId // resolves to the id
webAcl.attrLabelNamespace // resolves to 'awswaf:<accountNumber>:webacl:webACL-7xtQ0oTU473X:'
webAcl.attrArn // resolves to the full Arn
Names.uniqueId(webAcl) // resolves to '<stackName><nestedStackName>webACL<someOtherHash>'
Is there some other way to get the desired value as a variable?

L1 construct's properties
L1 constructs are exactly the resources defined by AWS CloudFormation—no more, no less. You must provide the resource's required configuration
yourself.
From the Developer Guide
The library does not define any values for us but uses only the values we have provided. So, for example, if we do not set any value for the Name, then there is no value in the synthesized template.
Below is our resource definition from the synthesized template.
Resources:
webAcl:
Type: AWS::WAFv2::WebACL
Properties:
DefaultAction:
Allow: {}
Scope: my-scope
VisibilityConfig:
CloudWatchMetricsEnabled: true
MetricName: webACL
SampledRequestsEnabled: true
The resource definition does not have the Name property. CloudFormation generates the property when it deploys the template.
There is no way to access properties we have not set. Okay, without hacks.
L1 construct's return values
In the CloudFormation template, we can use the resource's return values. Please, find the list in the CloudFormation user guide for the WebACL. Every resource has this section.
We can access the values in the CDK script as tokens.
console.log({
ref: webAcl.ref,
attrArn: webAcl.attrArn,
attrCapacity: webAcl.attrCapacity,
attrId: webAcl.attrId,
attrLabelNamespace: webAcl.attrLabelNamespace
})
And these are the only values generated by the CloudFormation during the deployment, which we can access in our script. The CDK library maps them nicely to the CloudFormation GetAtt, or Ref function calls in the synthesized template.
For example, we get this for the webAcl.attrCapacity. I have no idea why it looks weird in the console log.
Fn::GetAtt:
- webAcl
- Capacity
Maybe a workaround
We can use intrinsic functions to extract our value from the resource's values.
Fn.select(3, Fn.split(':', webAcl.attrLabelNamespace))
In the synthesized template, we get the chain of function calls.
Fn::Select:
- 3
- Fn::Split:
- ":"
- Fn::GetAtt:
- webAcl
- LabelNamespace
This looks fragile to me, but it might work. Please keep in mind that you can not apply any JavaScript operations to this value but use it as a construct property only. Because the CDK substitutes it with function calls in the template.
Code for reference
I use the following code to test the answer.
import { App, Fn, Stack } from 'aws-cdk-lib'
import { env } from 'process'
import { CfnBucket } from 'aws-cdk-lib/aws-s3'
import { CfnWebACL } from 'aws-cdk-lib/aws-wafv2'
function createStack (scope, id, props) {
const stack = new Stack(scope, id, props)
const webAcl = new CfnWebACL(stack, 'webAcl', {
defaultAction: {
allow: {},
},
scope: 'my-scope',
visibilityConfig: {
cloudWatchMetricsEnabled: true,
metricName: 'webACL',
sampledRequestsEnabled: true,
}
})
new CfnBucket(stack, 'bucket', {
bucketName: Fn.select(3, Fn.split(':', webAcl.attrLabelNamespace))
})
console.log({
ref: webAcl.ref,
attrArn: webAcl.attrArn,
attrCapacity: webAcl.attrCapacity,
attrId: webAcl.attrId,
attrLabelNamespace: webAcl.attrLabelNamespace
})
return stack
}
const app = new App()
createStack(app, 'WebAclName', {
env: { account: env.CDK_DEFAULT_ACCOUNT, region: env.CDK_DEFAULT_REGION }
})

Related

Terraform / Cloudformation - Pass parameter as YAML

I use Terraform to launch a Cloudformation stack to create Glue Databrew resources that don't exist yet on Terraform.
The thing is that I've a variable in Terraform that corresponds to the list of my data sources and in order to create the databrew resources associated to this data, I loop over my list to create one instance of my Cloudformation template for each data source.
Inside this template, I've a resource that I want to be different per data source. It correspond to the AWS::DataBrew::Ruleset resource.
It looks like this :
DataBrewDataQualityRuleset:
Type: AWS::DataBrew::Ruleset
Properties:
Name: !Ref RuleSetName
Description: Data Quality ruleset
Rules:
- Name: Check columns for missing values
Disabled: false
CheckExpression: AGG(MISSING_VALUES_PERCENTAGE) == :val1
SubstitutionMap:
- ValueReference: ":val1"
Value: '0'
ColumnSelectors:
- Regex: ".*"
- Name: Check two
Disabled: false
CheckExpression: :col IN :list
SubstitutionMap:
- ValueReference: ":col"
Value: "`group`"
- ValueReference: ":list"
Value: "[\"Value1\", \"Value2\"]"
TargetArn: !Sub SomeArn
What I want to do is, extract the Rules part of the component and create one file where I will put all my rules per data sources. In fact having something like below :
DataBrewDataQualityRuleset:
Type: AWS::DataBrew::Ruleset
Properties:
Name: !Ref RuleSetName
Description: Data Quality ruleset
Rules: !Ref Rules
TargetArn: !Sub SomeArn
And in my terraform, my Rules parameter would be my actual set of rules for one particular data source.
I've thought about having one YAML file from which I would loop on terraform but I'm not sure it's doable and if cloudformation would accept YAML as parameter type.
Below you'll also find my terraform component :
resource "aws_cloudformation_stack" "databrew_jobs" {
for_each = var.data_sources
name = "datachecks-${each.value.stack_name}"
parameters = {
Bucket = "test_bucket"
DataSetKey = "raw/${each.value.job_name}"
DataSetName = "dataset-${each.value.stack_name}"
RuleSetName = "ruleset-${each.value.stack_name}"
JobName = "profile-job-${each.value.stack_name}"
DataSourceName = "${each.value.stack_name}"
JobResultKey = "databrew-results/${each.value.job_name}"
RoleArn = iam_role_test.arn
}
template_body = file("${path.module}/databrew-job.yaml")
}
Do you have any idea how could I achieve this ?
Thanks in advance !

import an existing SSM Parameter and add as an Task Image environment variable in ApplicationLoadBalancedFargateService

I have parameters in SSM that is saved looking something like /dbUrl/prod , /dbUrl/dev and so on where it is in the format /dbUrl/${stage}.
I want to get this parameter and set as an environment variable for the task definition so I have a different value for the variable based on different environment. Right now stuck on how to import an existing parameter
Also is it possible to access the DB_URL value as process.env.DB_URL inside my node.js code after I manage to set the environment variable here
Docs I was following
ApplicationLoadBalancedFargateService
ApplicationLoadBalancedTaskImageOptions#secrets
const socketService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, `socketService${props.stage}`, {
cluster: cluster,
loadBalancer: loadBalancer ,
memoryLimitMiB: 2048,
cpu: 1024,
desiredCount: 2,
listenerPort: 1111,
taskImageOptions: {
image: ContainerImage.fromAsset("../socket"),
environment: {
},
secrets: {
DB_URL: //how to import existing /dbUrl/${props.stage} from SSM
}
},
});
You linked to documentation on the secrets prop of the ApplicationLoadBalancedTaskImageOptions. The docs specify that the type of the secrets prop is { [string]: Secret }.
You can follow the link to the documentation of Secret to see how it can be obtained. You will see that it has a fromSsmParameter method, which is what you need. That method takes in an IParameter.
To import an existing String Parameter, you can use one of the from* methods defined in the StringParameter class. For example, you can use fromSecureStringParameterAttributes

AWS CDK - Possible to access individual (JSON) value within a Secrets Manager secret when specifying secrets for a container?

I'm trying to put together a relatively simple stack on AWS CDK that involves an ApplicationLoadBalancedFargateService from aws-ecs-patterns.
My problem involves secrets. I have a secret in Secrets Manager that has several key/values (I think technically it's stored as a JSON doc, but AWS provides a key/val interface), and I need to pass them to my containers individually. I do this currently in an equivalent non-cdk (made in the console) stack by simply specifying the key, like this: arn:aws:secretsmanager:us-west-2:[acct]:secret/name-??????:KEY::, where `KEY is the secret key, and the correct value is inserted into the container as an env var.
When I try to do that with CDK, I get an error when I cdk synth:
`secretCompleteArn` does not appear to be complete; missing 6-character suffix
If I remove the last bit (:KEY::), it successfully synths, but my container isn't actually getting what I want.
This is how I'm trying to use it in my cdk (typescript) code:
new ApplicationLoadBalancedFargateService(this, 'Service', {
...
taskImageOptions: {
image: containerImage, // defined elsewhere
...
secrets: {
'DB_DATABASE': ecs.Secret.fromSecretsManager(
Secret.fromSecretCompleteArn(this, 'secret-DB_DATABASE',
'arn:aws:secretsmanager:us-west-2:[acct]:secret:secret/name-??????:KEY::')),
//there's really a few more, pulling keys from the same secret. Omitting for brevity
},
},
});
Is there a way to to make this work? Or do I need to change the way I store/use my secrets?
This is how you pass a specific key as environment variable to your container:
const mySecret = secretsmanager.Secret.fromSecretCompleteArn('<your arn>');
taskDefinition.addContainer('MyContainer', {
// ... other props ...
secrets: {
SECRET_KEY: ecs.Secret.fromSecretsManager(mySecret, 'specificKey'),
},
});
or with the ApplicationLoadBalancedFargateService:
new ApplicationLoadBalancedFargateService(this, 'Service', {
...
taskImageOptions: {
image: containerImage, // defined elsewhere
...
secrets: {
'DB_DATABASE': ecs.Secret.fromSecretsManager(mySecret, 'specificKey'),
},
},
});

aws-cdk LambdaRestApi: The final policy size is bigger than the limit

Hi i have been trying many possibilities, but now i would need some help.
I am using aws-cdk to create architecture by code and so far things have going well. Now i am running into this issue:
The final policy size is bigger than the limit (20480)
In understand what it means, but i have no idea how to solve it.
I am creating a lambdafunction to handle all requests:
const router = new lambda.Function(this, apiName + '-handler-temp', {
runtime: LambdaRuntime, // execution environment
code: lambda.Code.fromAsset('bin/lambda'), // code loaded from "lambda" directory
handler: 'index.handler', // file is "index", function is "handler"
vpc: vpc,
environment: {
DB_HOST: props?.rdsEndpoint as string,
DB_USER: props?.rdsDbUser as string,
DB_PASS: props?.rdsDBPass as string,
DB_PORT: props?.rdsPort as string,
DB_DIALECT: props?.rdsDbSchema as string,
DB_DATABASE: props?.rdsDBName as string,
},
layers: [layer],
timeout: Duration.seconds(30),
memorySize: 1024,
})
and the LambdaRestApi is defined like this:
const api = new LambdaRestApi(this, apiName, {
handler: router,
proxy: false,
cloudWatchRole: false,
description: 'API for Backend',
deployOptions: {
stageName: 'prod',
},
domainName: domainProperties,
})
I am creating Endpoints where i am using 23 times addMethod.
e.g.
const user = api.root.addResource('user')
user.addMethod(HttpMethod.POST)
user.addMethod(HttpMethod.GET)
user.addMethod(HttpMethod.PATCH)
since only one lambda is used to be invoked from apigateway, i am curious, how i can get control of only one policy to be used for lambda execution and it is not creating a new one every time.
I also tried to add property
role: role to the lambda function with this role definition:
const role = new Role(this, apiName + 'ApiGWPermissions', {
assumedBy: new ServicePrincipal('apigateway.amazonaws.com'),
})
role.addToPolicy(
new PolicyStatement({
resources: ['*'],
actions: ['lambda:InvokeFunction'],
})
)
but then i am running into different errors.
Has someone solved this riddle?
Cheers
As suggested in the CDK issue which Ian Walters mentioned, stripping the generated method permissions solved this for me. I'm using .Net but I'd expect that the approach should work for all language implementations. This function removes the permissions:
public void StripMethodPermissions(ConstructNode node) {
foreach (var child in node.Children) {
if (child is Amazon.CDK.AWS.APIGateway.Method) {
var method = ((Amazon.CDK.AWS.APIGateway.Method)child);
var permissions = method.Node.Children.Where(c => c is Amazon.CDK.AWS.Lambda.CfnPermission);
foreach (var permission in permissions) {
child.Node.TryRemoveChild(permission.Node.Id);
}
}
if (child.Node.Children.Length > 0) StripMethodPermissions(child.Node);
}
}
I'm using the technique like this:
var api = new RestApi(this, "MyRestApi", new RestApiProps {
RestApiName = "MyRestApiV1"
});
var handlerLambda = new Function(this, "RequestHandler", new FunctionProps {
Runtime = Runtime.DOTNET_CORE_3_1,
...
});
// Add resources and methods which use the handlerLambda here
// Remove all generated permissions
StripMethodPermissions(api.Root.Node);
// Add a single invoke permission for the lambda
handlerLambda.GrantInvoke(new ServicePrincipal("apigateway.amazonaws.com"));
Thanks to nija-at for showing the way
#Elliveny has the correct answer. Here is a code snitbit for Python which does the same thing (as I cannot post formatted code in comments):
from aws_cdk import (
aws_lambda as _lambda,
aws_events as events,
aws_iam as iam,
core,
)
for child in self.node.children:
if isinstance(child, events.Rule):
for eventChild in child.node.children:
if isinstance(eventChild, _lambda.CfnPermission):
child.node.try_remove_child(eventChild.node.id)
Remember that if you do this, you still need to grant invoke on your lambda for the "events.amazonaws.com" ServicePrincipal. Something like:
my_lambda.add_permission(
"RuleInvoke",
principal=iam.ServicePrincipal("events.amazonaws.com"),
action="lambda:InvokeFunction",
source_arn=f"arn:aws:events:{core.Aws.REGION}:{core.Aws.ACCOUNT_ID}:rule/your-rule-name-here*",
)
I hit a similar issue. There is a CDK issue that might help resolve it if addressed,
https://github.com/aws/aws-cdk/issues/9327
Its also worth noting that by default lambda integrations have allowTestInvoke set to true, which pretty much is going to double the policy document size.
I'm not sure if you can alter the integration options for the lambda with LambdaRestApi though, I'm using RestApi directly.
A short term fix might be to use RestApi rather than LambdaRestApi and create the lambda integration directly with the allowTestInvoke option set to false.
The other thing I did that helped was to just create more than one lambda that worked the same way, but got attached to different routes (e.g. same code, permissions etc, just different logical id) to also reduce the policy document size a bit.
I'm a bit pressed for development time hence the work-arounds. Personally, I think the right solution would be to fix it in the CDK and propose a PR such that LambdaRestApi just did what the user would expect, wild-card permission for the lambda for all
You can increase the memory size of lambda by doing this
Type: AWS::Serverless::Function
Properties:
CodeUri: src/promoCodes/
Role: !GetAtt FunctionRole.Arn
Handler: promoCodesListCsvPDF.promoCodesListCsvPDF
Policies:
- AWSLambdaExecute # Managed Policy
- AWSLambdaBasicExecutionRole # Managed Policy
MemorySize: 512
Layers:
- !Ref NodeDependenciesLayer
Events:
promoCodesListCsvPDFEvent:
Type: Api
Properties:
Path: /api/promo-codes/csv-pdf
Method: POST

AWS CloudFormation & Service Catalog - Can I require tags with user values?

Our problem seems very basic and I would expect common.
We have tags that must always be applied (for billing). However, the tag values are only known at the time the stack is deployed... We don't know what the tag values will be when developing the stack, or when creating the product in the Service Catalog...
We don't want to wait until AFTER the resource is deployed to discover the tag is missing, so as cool as AWS config may be, we don't want to rely on its rules if we don't have to.
So things like Tag Options don't work, because it appears that they expect we know the tag value months prior to some deployment (which isn't the case.)
Is there any way to mandate tags be used for a cloudformation template when it is deployed? Better yet, can we have service catalog query for a tag value when deploying? Tags like "system" or "project", for instance, come and go over time and are not known up-front for many types of cloudformation templates we develop.
Isn't this a common scenario?
I am worried that I am missing something very, very simple and basic which mandates tags be used up-front, but I can't seem to figure out what. Thank you in advance. I really did Google a lot before asking, without finding a satisfying answer.
I don't know anything about service catalog but you can create Conditions and then use it to conditionally create (or even fail) your resource creation. Conditional Resource Creation e.g.
Parameters:
ResourceTag:
Type: String
Default: ''
Conditions:
isTagEmpty:
!Equals [!Ref ResourceTag, '']
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Condition: isTagEmpty
Properties:
DBInstanceClass: <DB Instance Type>
Here RDS DB instance will only be created if tag is non-empty. But cloudformation will still return success.
Alternatively, you can try & fail the resource creation.
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: !If [isTagEmpty, !Ref "AWS::NoValue", <DB instance type>]
I haven't tried this but it should fail as DB instance type will be invalid if tag is null.
Edit: You can also create your stack using the createStack CFN API. Write some code to read & validate the input (e.g. read from service catalog) & call the createStack API. I am doing the same from Lambda (nodejs) reading some input from Parameter Store. Sample code -
module.exports.create = async (event, context, callback) => {
let request = JSON.parse(event.body);
let subnetids = await ssm.getParameter({
Name: '/vpc/public-subnets'
}).promise();
let securitygroups = await ssm.getParameter({
Name: '/vpc/lambda-security-group'
}).promise();
let params = {
StackName: request.customerName, /* required */
Capabilities: [
'CAPABILITY_IAM',
'CAPABILITY_NAMED_IAM',
'CAPABILITY_AUTO_EXPAND',
/* more items */
],
ClientRequestToken: 'qwdfghjk3912',
EnableTerminationProtection: false,
OnFailure: request.onfailure,
Parameters: [
{
ParameterKey: "SubnetIds",
ParameterValue: subnetids.Parameter.Value,
},
{
ParameterKey: 'SecurityGroupIds',
ParameterValue: securitygroups.Parameter.Value,
},
{
ParameterKey: 'OpsPoolArnList',
ParameterValue: request.userPoolList,
},
/* more items */
],
TemplateURL: request.templateUrl,
};
cfn.config.region = request.region;
let result = await cfn.createStack(params).promise();
console.log(result);
}
Another option: add a AWS Custom Resource backed by Lambda. Check for tags in this section & return failure if it doesn't satisfy the constraints. Make all other resource creation depend on this resource (so that they all create if your checks pass). Link also contains example. You will also have to add handling for stack update & deletion (like a default success). I think this is your best bet as of now.