How to get logical ID of resource with CDK? - amazon-web-services

I'm attempting to write some tests for a CDK Construct that validates security group rules defined as part of the construct.
The Construct looks something like the following.
export interface SampleConstructProps extends StackProps {
srcSecurityGroupId: string;
}
export class SampleConstruct extends Construct {
securityGroup: SecurityGroup;
constructor(scope: Construct, id: string, props: SampleConstructProps) {
super(scope, id, props);
// const vpc = Vpc.fromLookup(...);
this.securityGroup = new SecurityGroup(this, "SecurityGroup", {
vpc: vpc,
allowAllOutbound: true,
});
const srcSecurityGroupId = SecurityGroup.fromSecurityGroupId(stack, "SrcSecurityGroup", props.srcSecurityGroupId);
this.securityGroup.addIngressRule(srcSecurityGroup, Port.tcp(22));
}
}
And I want to write a test that looks something like the following.
test("Security group config is correct", () => {
const stack = new Stack();
const srcSecurityGroupId = "id-123";
const testConstruct = new SampleConstruct(stack, "TestConstruct", {
srcSecurityGroupId: srcSecurityGroupId
});
expect(stack).to(
haveResource(
"AWS::EC2::SecurityGroupIngress",
{
IpProtocol: "tcp",
FromPort: 22,
ToPort: 22,
SourceSecurityGroupId: srcSecurityGroupId,
GroupId: {
"Fn::GetAtt": [testConstruct.securityGroup.logicalId, "GroupId"], // Can't do this
},
},
undefined,
true
)
);
});
The issue here is that the test is validated against the synthesized CloudFormation template, so if you want to verify that the security group created by this construct has a rule allowing access from srcSecurityGroup, you need the Logical ID of the security group that was created as part of the Construct.
You can see this in the generated CloudFormation template here.
{
"Type": "AWS::EC2::SecurityGroupIngress",
"Properties": {
"IpProtocol": "tcp",
"FromPort": 22,
"GroupId": {
"Fn::GetAtt": [
"TestConstructSecurityGroup95EF3F0F", <-- This
"GroupId"
]
},
"SourceSecurityGroupId": "id-123",
"ToPort": 22
}
}
That Fn::GetAtt is the crux of this issue. Since these tests really just do an object comparison, you need to be able to replicate the Fn::Get invocation, which requires the CloudFormation Logical ID.
Note that the CDK does provide a handful of identifiers for you.
Unique ID provides something very close, but it's not same identifier used in the CloudFormation stack. For example, securityGroup.uniqueId returns TestStackTestConstructSecurityGroup10D493A7 whereas the CloudFormation template displays TestConstructSecurityGroup95EF3F0F. You can note the differences are the uniqueId prepends the Construct ID to the logical identifier and the appended hash is different in each.
Construct ID is just the identifier that you provide when instantiating a construct. It is not the logical ID either, though it is used as part of the logical ID. I also have not seen a way of programmatically retrieving this ID from the construct directly. You can of course define the ID somewhere and just reuse it, but this still doesn't solve the problem of it not fully matching the logical ID. In this case it's a difference of SecurityGroup as the construct ID and TestConstructSecurityGroup95EF3F0F as the logical ID in the synthesized template.
Is there a straightforward way getting the logical ID of CDK resources?

After writing up this whole post and digging through the CDK code, I stumbled on the answer I was looking for. If anybody has a better approach for getting the logical ID from a higher level CDK construct, the contribution would be much appreciated.
If you need to get the logical ID of a CDK resource you can do the following:
const stack = new Stack();
const construct = new SampleConstruct(stack, "SampleConstruct");
const logicalId = stack.getLogicalId(construct.securityGroup.node.defaultChild as CfnSecurityGroup);
Note that you you already have a CloudFormation resource (eg something that begins with with Cfn) then it's a little easier.
// Pretend construct.securityGroup is of type CfnSecurityGroup
const logicalId = stack.getLogicalId(construct.securityGroup);

From my testing, it seems that stack.getLogicalId will always return the original, CDK allocated logicalId, it won't change if you call overrideLogicalId, so it won't always match the synthed output.
This worked for me, even with a logicalId override set:
stack.resolve((construct.node.defaultChild as cdk.CfnElement).logicalId)
stack.resolve is necessary because .logicalId is a token.

In addition to the excellent answer from jaredready, you can also explicitly set the logical ID using resource.node.default_child.overrideLogicalId("AnyStringHere")
This may make it easier as you can set it once and use hard-coded strings rather than looking up the value for every test.

Related

Getting generated name in AWS Cdk

When creating a WebAcl in CDK and letting CDK generate the name, I want to use that generated name as a variable in CDK, i.e. when generating the WebAcl like this (no explicit name set in the properties) ...
const webAcl = new CfnWebACL(this, "webAcl", {
defaultAction: {
allow: {},
},
scope: myScope,
visibilityConfig: {
cloudWatchMetricsEnabled: true,
metricName: "webACL",
sampledRequestsEnabled: true,
}
});
... the webAcl will have a generated Name like webAcl-7xtQ0oTU473X after deployment (the id with an appended hash). The problem is that I do not know how to reference this name as a variable in CDK.
I found the following possibilities to get some type of ids/names but none of them resolve to webAcl-7xtQ0oTU473X after deployment (some, however, include the value I want):
webAcl.name // is undefined in CDK
webAcl.logicalId // resolves to 'webAcl' (no hash)
webAcl.attrId // resolves to the id
webAcl.attrLabelNamespace // resolves to 'awswaf:<accountNumber>:webacl:webACL-7xtQ0oTU473X:'
webAcl.attrArn // resolves to the full Arn
Names.uniqueId(webAcl) // resolves to '<stackName><nestedStackName>webACL<someOtherHash>'
Is there some other way to get the desired value as a variable?
L1 construct's properties
L1 constructs are exactly the resources defined by AWS CloudFormation—no more, no less. You must provide the resource's required configuration
yourself.
From the Developer Guide
The library does not define any values for us but uses only the values we have provided. So, for example, if we do not set any value for the Name, then there is no value in the synthesized template.
Below is our resource definition from the synthesized template.
Resources:
webAcl:
Type: AWS::WAFv2::WebACL
Properties:
DefaultAction:
Allow: {}
Scope: my-scope
VisibilityConfig:
CloudWatchMetricsEnabled: true
MetricName: webACL
SampledRequestsEnabled: true
The resource definition does not have the Name property. CloudFormation generates the property when it deploys the template.
There is no way to access properties we have not set. Okay, without hacks.
L1 construct's return values
In the CloudFormation template, we can use the resource's return values. Please, find the list in the CloudFormation user guide for the WebACL. Every resource has this section.
We can access the values in the CDK script as tokens.
console.log({
ref: webAcl.ref,
attrArn: webAcl.attrArn,
attrCapacity: webAcl.attrCapacity,
attrId: webAcl.attrId,
attrLabelNamespace: webAcl.attrLabelNamespace
})
And these are the only values generated by the CloudFormation during the deployment, which we can access in our script. The CDK library maps them nicely to the CloudFormation GetAtt, or Ref function calls in the synthesized template.
For example, we get this for the webAcl.attrCapacity. I have no idea why it looks weird in the console log.
Fn::GetAtt:
- webAcl
- Capacity
Maybe a workaround
We can use intrinsic functions to extract our value from the resource's values.
Fn.select(3, Fn.split(':', webAcl.attrLabelNamespace))
In the synthesized template, we get the chain of function calls.
Fn::Select:
- 3
- Fn::Split:
- ":"
- Fn::GetAtt:
- webAcl
- LabelNamespace
This looks fragile to me, but it might work. Please keep in mind that you can not apply any JavaScript operations to this value but use it as a construct property only. Because the CDK substitutes it with function calls in the template.
Code for reference
I use the following code to test the answer.
import { App, Fn, Stack } from 'aws-cdk-lib'
import { env } from 'process'
import { CfnBucket } from 'aws-cdk-lib/aws-s3'
import { CfnWebACL } from 'aws-cdk-lib/aws-wafv2'
function createStack (scope, id, props) {
const stack = new Stack(scope, id, props)
const webAcl = new CfnWebACL(stack, 'webAcl', {
defaultAction: {
allow: {},
},
scope: 'my-scope',
visibilityConfig: {
cloudWatchMetricsEnabled: true,
metricName: 'webACL',
sampledRequestsEnabled: true,
}
})
new CfnBucket(stack, 'bucket', {
bucketName: Fn.select(3, Fn.split(':', webAcl.attrLabelNamespace))
})
console.log({
ref: webAcl.ref,
attrArn: webAcl.attrArn,
attrCapacity: webAcl.attrCapacity,
attrId: webAcl.attrId,
attrLabelNamespace: webAcl.attrLabelNamespace
})
return stack
}
const app = new App()
createStack(app, 'WebAclName', {
env: { account: env.CDK_DEFAULT_ACCOUNT, region: env.CDK_DEFAULT_REGION }
})

AWS CDK - Possible to access individual (JSON) value within a Secrets Manager secret when specifying secrets for a container?

I'm trying to put together a relatively simple stack on AWS CDK that involves an ApplicationLoadBalancedFargateService from aws-ecs-patterns.
My problem involves secrets. I have a secret in Secrets Manager that has several key/values (I think technically it's stored as a JSON doc, but AWS provides a key/val interface), and I need to pass them to my containers individually. I do this currently in an equivalent non-cdk (made in the console) stack by simply specifying the key, like this: arn:aws:secretsmanager:us-west-2:[acct]:secret/name-??????:KEY::, where `KEY is the secret key, and the correct value is inserted into the container as an env var.
When I try to do that with CDK, I get an error when I cdk synth:
`secretCompleteArn` does not appear to be complete; missing 6-character suffix
If I remove the last bit (:KEY::), it successfully synths, but my container isn't actually getting what I want.
This is how I'm trying to use it in my cdk (typescript) code:
new ApplicationLoadBalancedFargateService(this, 'Service', {
...
taskImageOptions: {
image: containerImage, // defined elsewhere
...
secrets: {
'DB_DATABASE': ecs.Secret.fromSecretsManager(
Secret.fromSecretCompleteArn(this, 'secret-DB_DATABASE',
'arn:aws:secretsmanager:us-west-2:[acct]:secret:secret/name-??????:KEY::')),
//there's really a few more, pulling keys from the same secret. Omitting for brevity
},
},
});
Is there a way to to make this work? Or do I need to change the way I store/use my secrets?
This is how you pass a specific key as environment variable to your container:
const mySecret = secretsmanager.Secret.fromSecretCompleteArn('<your arn>');
taskDefinition.addContainer('MyContainer', {
// ... other props ...
secrets: {
SECRET_KEY: ecs.Secret.fromSecretsManager(mySecret, 'specificKey'),
},
});
or with the ApplicationLoadBalancedFargateService:
new ApplicationLoadBalancedFargateService(this, 'Service', {
...
taskImageOptions: {
image: containerImage, // defined elsewhere
...
secrets: {
'DB_DATABASE': ecs.Secret.fromSecretsManager(mySecret, 'specificKey'),
},
},
});

List of Active Directory DNS servers IP addresses in an SSM document

I am converting my 0.11 code to 0.12. Most things seem to be working out well, but I am really lost on the SSM document.
In my 0.11 code, I had this code:
resource "aws_ssm_document" "ssm_document" {
name = "ssm_document_${terraform.workspace}${var.addomainsuffix}"
document_type = "Command"
content = <<DOC
{
"schemaVersion": "1.0",
"description": "Automatic Domain Join Configuration",
"runtimeConfig": {
"aws:domainJoin": {
"properties": {
"directoryId": "${aws_directory_service_directory.microsoftad-lab.id}",
"directoryName": "${aws_directory_service_directory.microsoftad-lab.name}",
"dnsIpAddresses": [
"${aws_directory_service_directory.microsoftad-lab.dns_ip_addresses[0]}",
"${aws_directory_service_directory.microsoftad-lab.dns_ip_addresses[1]}"
]
}
}
}
}
DOC
depends_on = ["aws_directory_service_directory.microsoftad-lab"]
}
This worked reasonably well. However, Terraform 0.12 does not accept this code, saying
This value does not have any indices.
I have been trying to look up different solutions on the web, but I am encountering countless issues with datatypes. For example, one of the solutions I have seen proposes this:
"dnsIpAddresses": [
"${sort(aws_directory_service_directory.oit-microsoftad-lab.dns_ip_addresses)[0]}",
"${sort(aws_directory_service_directory.oit-microsoftad-lab.dns_ip_addresses)[1]}",
]
}
and I am getting
InvalidDocumentContent: JSON not well-formed
which is kinda weird to me, since if I am looking into trace log, I seem to be getting relatively correct values:
{"Content":"{\n \"schemaVersion\": \"1.0\",\n \"description\": \"Automatic Domain Join Configuration\",\n \"runtimeConfig\": {\n \"aws:domainJoin\": {\n \"properties\": {\n \"directoryId\": \"d-9967245377\",\n \"directoryName\": \"012mig.lab\",\n \"dnsIpAddresses\": [\n \"10.0.0.227\",\n
\"10.0.7.103\",\n ]\n }\n }\n }\n}\n \n","DocumentFormat":"JSON","DocumentType":"Command","Name":"ssm_document_012mig.lab"}
I have tried concat and list to put the values together, but then I am getting the datatype errors. Right now, it looks like I am going around in loops here.
Does anyone have any direction to give me here?
Terraform 0.12 has stricter types than 0.11 and less automatic type coercion going on under the covers so here you're running into the fact that the output of the aws_directory_service_directory resource's dns_ip_addresses attribute isn't a list but a set:
"dns_ip_addresses": {
Type: schema.TypeSet,
Elem: &schema.Schema{Type: schema.TypeString},
Set: schema.HashString,
Computed: true,
},
Set's can't be indexed directly and instead must first be converted to a list explicitly in 0.12.
As an example:
variable "example_list" {
type = list(string)
default = [
"foo",
"bar",
]
}
output "list_first_element" {
value = var.example_list[0]
}
Running terraform apply on this will output the following:
Outputs:
list_first_element = foo
However if we use a set variable instead:
variable "example_set" {
type = set(string)
default = [
"foo",
"bar",
]
}
output "set_first_element" {
value = var.example_set[0]
}
Then attempting to run terraform apply will throw the following error:
Error: Invalid index
on main.tf line 22, in output "set_foo":
22: value = var.example_set[0]
This value does not have any indices.
If we convert the set variable into a list with tolist first then it works:
variable "example_set" {
type = set(string)
default = [
"foo",
"bar",
]
}
output "set_first_element" {
value = tolist(var.example_set)[0]
}
Outputs:
set_first_element = bar
Note that sets may have different ordering to what you may expect (in this case it is ordered alphabetically rather than as declared). In your case this isn't an issue but it's worth thinking about when indexing an expecting the elements to be in the order you declared them.
Another possible option here, instead of building the JSON output from the set or list of outputs, you could just directly encode the dns_ip_addresses attribute as JSON with the jsonencode function:
variable "example_set" {
type = set(string)
default = [
"foo",
"bar",
]
}
output "set_first_element" {
value = jsonencode(var.example_set)
}
Which outputs the following after running terraform apply:
Outputs:
set_first_element = ["bar","foo"]
So for your specific example we would want to do something like this:
resource "aws_ssm_document" "ssm_document" {
name = "ssm_document_${terraform.workspace}${var.addomainsuffix}"
document_type = "Command"
content = <<DOC
{
"schemaVersion": "1.0",
"description": "Automatic Domain Join Configuration",
"runtimeConfig": {
"aws:domainJoin": {
"properties": {
"directoryId": "${aws_directory_service_directory.microsoftad-lab.id}",
"directoryName": "${aws_directory_service_directory.microsoftad-lab.name}",
"dnsIpAddresses": ${jsonencode(aws_directory_service_directory.microsoftad-lab.dns_ip_addresses)}
}
}
}
}
DOC
}
Note that I also removed the unnecessary depends_on. If a resource has interpolation in from another resource then Terraform will automatically understand that the interpolated resource needs to be created before the one referencing it.
The resource dependencies documentation goes into this in more detail:
Most resource dependencies are handled automatically. Terraform
analyses any expressions within a resource block to find references to
other objects, and treats those references as implicit ordering
requirements when creating, updating, or destroying resources. Since
most resources with behavioral dependencies on other resources also
refer to those resources' data, it's usually not necessary to manually
specify dependencies between resources.
However, some dependencies cannot be recognized implicitly in
configuration. For example, if Terraform must manage access control
policies and take actions that require those policies to be present,
there is a hidden dependency between the access policy and a resource
whose creation depends on it. In these rare cases, the depends_on
meta-argument can explicitly specify a dependency.

CDK adds random parameters

So I have this function I'm trying to declare and it works and deploys just dandy unless you uncomment the logRetention setting. If logRetention is specified the cdk deploy operation
adds additional parameters to the stack. And, of course, this behavior is completely unexplained in the documentation.
https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#log-group
SingletonFunction.Builder.create(this, "native-lambda-s3-fun")
.functionName(funcName)
.description("")
// .logRetention(RetentionDays.ONE_DAY)
.handler("app")
.timeout(Duration.seconds(300))
.runtime(Runtime.GO_1_X)
.uuid(UUID.randomUUID().toString())
.environment(new HashMap<String, String>(){{
put("FILE_KEY", "/file/key");
put("S3_BUCKET", junk.getBucketName());
}})
.code(Code.fromBucket(uploads, functionUploadKey(
"formation-examples",
"native-lambda-s3",
lambdaVersion.getValueAsString()
)))
.build();
"Parameters": {
"lambdaVersion": {
"Type": "String"
},
"AssetParametersceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40aS3BucketB030C8A8": {
"Type": "String",
"Description": "S3 bucket for asset \"ceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40a\""
},
"AssetParametersceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40aS3VersionKey6A2AABD7": {
"Type": "String",
"Description": "S3 key for asset version \"ceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40a\""
},
"AssetParametersceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40aArtifactHashEDC522F0": {
"Type": "String",
"Description": "Artifact hash for asset \"ceefd938ac7ea929077f2e2f4cf09b5034ebdd14799216b1281f4b28427da40a\""
}
},
It's a bug. They're Working On It™. So, rejoice - we can probably expect a fix sometime within the next decade.
I haven't tried it yet, but I'm guessing the workaround is to manipulate the low-level CfnLogGroup construct, since it has the authoritative retentionInDays property. The relevant high-level Log Group construct can probably be obtained from the Function via its logGroup property. Failing that, the LogGroup can be created from scratch (which will probably be a headache all on its own).
I also encountered the problem described above. From what I can tell, we are unable to specify a log group name and thus the log group name is predictable.
My solution was to simply create a LogGroup with the same name as my Lambda function with the /aws/lambda/ prefix.
Example:
var function = new Function(
this,
"Thing",
new FunctionProps
{
FunctionName = $"{Stack.Of(this).StackName}-Thing",
// ...
});
_ = new LogGroup(
this,
"ThingLogGroup",
new LogGroupProps
{
LogGroupName = $"/aws/lambda/{function.FunctionName}",
Retention = RetentionDays.ONE_MONTH,
});
This does not create unnecessary "AssetParameters..." CF template parameters like the inline option does.
Note: I'm using CDK version 1.111.0 and 1.86.0 with C#

AWS CloudFormation & Service Catalog - Can I require tags with user values?

Our problem seems very basic and I would expect common.
We have tags that must always be applied (for billing). However, the tag values are only known at the time the stack is deployed... We don't know what the tag values will be when developing the stack, or when creating the product in the Service Catalog...
We don't want to wait until AFTER the resource is deployed to discover the tag is missing, so as cool as AWS config may be, we don't want to rely on its rules if we don't have to.
So things like Tag Options don't work, because it appears that they expect we know the tag value months prior to some deployment (which isn't the case.)
Is there any way to mandate tags be used for a cloudformation template when it is deployed? Better yet, can we have service catalog query for a tag value when deploying? Tags like "system" or "project", for instance, come and go over time and are not known up-front for many types of cloudformation templates we develop.
Isn't this a common scenario?
I am worried that I am missing something very, very simple and basic which mandates tags be used up-front, but I can't seem to figure out what. Thank you in advance. I really did Google a lot before asking, without finding a satisfying answer.
I don't know anything about service catalog but you can create Conditions and then use it to conditionally create (or even fail) your resource creation. Conditional Resource Creation e.g.
Parameters:
ResourceTag:
Type: String
Default: ''
Conditions:
isTagEmpty:
!Equals [!Ref ResourceTag, '']
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Condition: isTagEmpty
Properties:
DBInstanceClass: <DB Instance Type>
Here RDS DB instance will only be created if tag is non-empty. But cloudformation will still return success.
Alternatively, you can try & fail the resource creation.
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: !If [isTagEmpty, !Ref "AWS::NoValue", <DB instance type>]
I haven't tried this but it should fail as DB instance type will be invalid if tag is null.
Edit: You can also create your stack using the createStack CFN API. Write some code to read & validate the input (e.g. read from service catalog) & call the createStack API. I am doing the same from Lambda (nodejs) reading some input from Parameter Store. Sample code -
module.exports.create = async (event, context, callback) => {
let request = JSON.parse(event.body);
let subnetids = await ssm.getParameter({
Name: '/vpc/public-subnets'
}).promise();
let securitygroups = await ssm.getParameter({
Name: '/vpc/lambda-security-group'
}).promise();
let params = {
StackName: request.customerName, /* required */
Capabilities: [
'CAPABILITY_IAM',
'CAPABILITY_NAMED_IAM',
'CAPABILITY_AUTO_EXPAND',
/* more items */
],
ClientRequestToken: 'qwdfghjk3912',
EnableTerminationProtection: false,
OnFailure: request.onfailure,
Parameters: [
{
ParameterKey: "SubnetIds",
ParameterValue: subnetids.Parameter.Value,
},
{
ParameterKey: 'SecurityGroupIds',
ParameterValue: securitygroups.Parameter.Value,
},
{
ParameterKey: 'OpsPoolArnList',
ParameterValue: request.userPoolList,
},
/* more items */
],
TemplateURL: request.templateUrl,
};
cfn.config.region = request.region;
let result = await cfn.createStack(params).promise();
console.log(result);
}
Another option: add a AWS Custom Resource backed by Lambda. Check for tags in this section & return failure if it doesn't satisfy the constraints. Make all other resource creation depend on this resource (so that they all create if your checks pass). Link also contains example. You will also have to add handling for stack update & deletion (like a default success). I think this is your best bet as of now.