How do you mock an existing vpc in aws-cdk unit tests? - unit-testing

I am writing an AWS CDK application that creates a ECS Fargate stack. It uses an existing VPC and existing ECR repositories. Simply mocking my Interface, and returning mocks of the IVpc and IRepository interfaces gets me past initial problems, but when the CDK goes to use those constructs, I get more errors.
const mockResources = mock<IExistingResources>(
{
getVpc: (scope: cdk.Construct, vpcId: string) => {
return mock<IVpc>();
},
getElasticContainerRepository: (scope: cdk.Construct, id: string, repositoryName: string) => {
return mock<IRepository>();
}
}
);
I get this error:
TypeError: Cannot destructure property 'subnetIds' of 'baseProps.vpc.selectSubnets(...)' as it is undefined.
This seems like a possible "black hole" where I will need to understand every usage of the mock and account for it. I'm looking for a better way to consistently model my existing resources so that I can test my new code. Any advice would be appreciated.

For anyone that comes across this question, I decided to construct a VPC with known parameters for purposes of my tests. This VPC is returned from the mock fromLookup function.
For example: to build the VPC
function buildVpc(scope: cdk.Construct, vpcId:string): IVpc {
return new Vpc(scope, vpcId, {
cidr: '10.0.0.0/16',
maxAzs: 2,
subnetConfiguration: [{
cidrMask: 26,
name: 'isolatedSubnet',
subnetType: SubnetType.PUBLIC,
}],
natGateways: 0
});
}
And then to use the vpc in the mock of my ExistingResources class,
const mockResources = mock<IExistingResources>(
{
getVpc: (scope: cdk.Construct, vpcId: string) => {
return buildVpc(scope, vpcId);
},
getElasticContainerRepository: (scope: cdk.Construct, id: string, repositoryName: string) => {
return mock<IRepository>();
}
}
);
This allows me to do snapshot testing in a disconnected environment.

Related

How to read parameter store from a different region in CDK?

I am using CDK to deploy AWS resources but need to get some values from the parameter store from a different region. I can see this API in CDK's reference page to read a parameter:
ssm.StringParameter.fromStringParameterAttributes
But it doesn't support passing region. How can I make it work across region?
You can find an implementation here:
import { Construct } from 'constructs';
import { AwsCustomResource, AwsCustomResourcePolicy, AwsSdkCall, PhysicalResourceId } from 'aws-cdk-lib/custom-resources';
interface SSMParameterReaderProps {
readonly parameterName: string;
readonly region: string;
}
export class SSMParameterReader extends AwsCustomResource {
constructor(scope: Construct, name: string, props: SSMParameterReaderProps) {
const { parameterName, region } = props;
super(scope, name, {
onUpdate: {
action: 'getParameter',
service: 'SSM',
parameters: {
Name: parameterName,
},
region,
physicalResourceId: PhysicalResourceId.of(name),
},
policy: AwsCustomResourcePolicy.fromSdkCalls({
resources: AwsCustomResourcePolicy.ANY_RESOURCE,
}),
});
}
public getParameterValue(): string {
return this.getResponseFieldReference('Parameter.Value').toString();
}
}
Source: https://github.com/Idea-Pool/aws-static-site/blob/main/lib/ssm-parameter-reader.ts
(Based on CloudFormation Cross-Region Reference)
This article explains what you are looking for
How to read parameter store from a different region in CDK?
Summary:
Create an AWS custom resource that takes in the parameterName and the regionName as props and returns the value.
It is not currently possible to access SSM Parameters in a different region.
You would have to set up some process to replicate the parameter across the needed regions and keep them in sync.
You could also get the value using a custom resource backed by a lambda.

Using CDK to define ECS Fargate cluster with Service Discovery, without Load Balancer

We use CDK and we are starting using service discovery within our ECS services (hence, server to server not client to server). This means that we don't need ALB (for the moment at least, maybe we can revisit this choice later).
Unfortunately the CDK pattern that exists for building ECS services (ecs_patterns.ApplicationLoadBalancedFargateService) also creates an ALB, so we cannot use it as it is and we need to create those CDK steps ourselves.
The idea is basically to "port" this tutorial from AWS from using AWS CLI to use CDK.
Question is: has anyone done that already and wants to share it or knows the reason why CDK Patterns doesn't have that option?
(If nobody is going to share it, we will do it and then share it of course; I think this option should be present in CDK straight away and it's just a matter of not wasting time with "just" a configuration issue – unless there is something we are not seeing here...).
OK, so at the end it was easier than expected. This is my final solution (and check the comments out, since I got stuck a couple of times):
/*
* ECS Fargate with service discovery but without Load Balancing
*/
import * as cdk from "#aws-cdk/core";
import * as ec2 from "#aws-cdk/aws-ec2";
import * as ecs from "#aws-cdk/aws-ecs";
import * as servicediscovery from "#aws-cdk/aws-servicediscovery";
import * as iam from "#aws-cdk/aws-iam";
import * as logs from "#aws-cdk/aws-logs";
import { DnsRecordType } from "#aws-cdk/aws-servicediscovery";
export class EcsServiceDiscoveryStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const serviceName = "zambulo";
const namespace = "caludio.magic";
const vpc = ec2.Vpc.fromLookup(this, "VPC", {
isDefault: true,
});
const cluster = new ecs.Cluster(this, "EcsServiceDiscovery", {
vpc: vpc,
});
const dnsNamespace = new servicediscovery.PrivateDnsNamespace(
this,
"DnsNamespace",
{
name: namespace,
vpc: vpc,
description: "Private DnsNamespace for my Microservices",
}
);
const taskrole = new iam.Role(this, "ecsTaskExecutionRole", {
assumedBy: new iam.ServicePrincipal("ecs-tasks.amazonaws.com"),
});
taskrole.addManagedPolicy(
iam.ManagedPolicy.fromAwsManagedPolicyName(
"service-role/AmazonECSTaskExecutionRolePolicy"
)
);
/*
* Check the doc for the allowed cpu/mem combiations:
* https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-ecs.FargateTaskDefinition.html
*/
const serviceTaskDefinition = new ecs.FargateTaskDefinition(
this,
`${serviceName}ServiceTaskDef`,
{
cpu: 256,
memoryLimitMiB: 512,
taskRole: taskrole,
}
);
const serviceLogGroup = new logs.LogGroup(
this,
`${serviceName}ServiceLogGroup`,
{
logGroupName: `/ecs/${serviceName}Service`,
removalPolicy: cdk.RemovalPolicy.DESTROY,
}
);
/* Fargate only support awslog driver */
const serviceLogDriver = new ecs.AwsLogDriver({
logGroup: serviceLogGroup,
streamPrefix: `${serviceName}Service`,
});
/*
* If you chose a public image from the registry (like in this case),
* the `assignPublicIp` in the Fargate definition (below) must be true
*/
const serviceContainer = serviceTaskDefinition.addContainer(
`${serviceName}ServiceContainer`,
{
image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
logging: serviceLogDriver,
}
);
serviceContainer.addPortMappings({
containerPort: 80,
});
const serviceSecGrp = new ec2.SecurityGroup(
this,
`${serviceName}ServiceSecurityGroup`,
{
allowAllOutbound: true,
securityGroupName: `${serviceName}ServiceSecurityGroup`,
vpc: vpc,
}
);
serviceSecGrp.connections.allowFromAnyIpv4(ec2.Port.tcp(80));
new ecs.FargateService(this, `${serviceName}Service`, {
cluster: cluster,
taskDefinition: serviceTaskDefinition,
// Must be `true` when using public images
assignPublicIp: true,
// If you set it to 0, the deployment will finish succesfully anyway
desiredCount: 1,
securityGroup: serviceSecGrp,
cloudMapOptions: {
// This will be your service_name.namespace
name: serviceName,
cloudMapNamespace: dnsNamespace,
dnsRecordType: DnsRecordType.A,
},
});
}
}
CDK Patterns is a great resource, however it is limited to only a subset of all of the possible use cases for deploying infrastructure on AWS, hence 'patterns'.
I have no personal experience with AWS Service Discovery but it appears the CDK does offer L2 constructs for it. aws-servicediscovery
In addition to service discovery, you can also create a L2 FargateService construct using the aws-ecs library.

AWS-CDK: Any way to pass vpc cidr through input parameter?

I'm trying to pass vpc cidr as input parameter like below:
import { Stack, StackProps, Construct, CfnParameter } from '#aws-cdk/core';
import { Vpc, SubnetType } from '#aws-cdk/aws-ec2';
export class VpcStructureCdkStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// VPC CIDR as input parameter
const vpcCidr = new CfnParameter(this, 'vpcCidr', {
type: 'String',
description: 'Please enter the IP range (CIDR notation) for this VPC',
allowedPattern: '((\d{1,3})\.){3}\d{1,3}/\d{1,2}'
})
// The code that defines your stack goes here
new Vpc(this, 'VPC', {
maxAzs: 3,
cidr: vpcCidr.valueAsString,
subnetConfiguration: [
{
name: 'App',
subnetType: SubnetType.PRIVATE,
cidrMask: 24
},
...
But getting the following error:
Error: 'cidr' property must be a concrete CIDR string, got a Token (we need to parse it for automatic subdivision)
Same error when using environment variables.
Is there any way to don't hard code vpc cidr?
From the documentation of CDK Parameters:
A CfnParameter instance exposes its value to your AWS CDK app via a token.
Like all tokens, the parameter's token is resolved at synthesis time, but it resolves to a reference to the parameter defined in the AWS CloudFormation template, which will be resolved at deploy time, rather than to a concrete value.
[...] In general, we recommend against using AWS CloudFormation parameters with the AWS CDK.
Especially the last sentence is crucial.
How do you resolve it now?
Well, as you already stated: use environment variables via your programming language.
I don't know about your approach with environment variables, because you didn't show it.
Let me give you an example.
// file: lib/your_stack.ts
export class VpcStructureCdkStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// reading the value from the env.
// Obviously, you have to set it before or pass it before you call any cdk command
const vpcCidr = process.env.VPC_CIDR;
new Vpc(this, 'VPC', {
maxAzs: 3,
// passing it
cidr: vpcCidr,
subnetConfiguration: [
{
name: 'App',
subnetType: SubnetType.PRIVATE,
cidrMask: 24
},
// ...
]
}
}
}
That's only one way of getting a configurable value into CDK though.
A better and debuggable way is to set all your dynamic/user/domain values in the Context.
The best place to do for your own values is within the cdk.json. If it doesn't exist yet, just create it and don't forget to put it into Git (or your VCS of your choice).
{
// ...
context: {
// ...
"VpcCidr": "10.0.0.0/8",
}
}
If the cdk.json approach also isn't sufficient, you have another option:
Pass it to cdk synth/deploy as an argument via -c/--context vpcCidr=10.0.0.0/8.
However, that's harder to debug, because it's not versioned necessarily.
Within your stack (the best place IMHO to do it), you can call the following method to retrieve the actual value from the context:
const vpcCidr = this.node.tryGetContext("VpcCidr");
and pass it to your VPC constructor.
Either create your own VPC either import the existing one. For the second solution there is fromLookup(), which will find proper VPC (by tags, name or other VPCLookupOptions) together with it's CIDR:
https://docs.aws.amazon.com/cdk/api/latest/typescript/api/aws-ec2/vpc.html#aws_ec2_Vpc_fromLookup
I did similar thing by reading external config file.
import * as cdk from "#aws-cdk/core";
import { Vpc, SubnetType } from "#aws-cdk/aws-ec2";
import fs = require("fs");
export class ExampleStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const configJson = fs.readFileSync(
"config/" + this.account + ".json",
"utf8"
);
let config = JSON.parse(configJson);
const vpc = Vpc.fromLookup(this, "vpc", {
vpcId: config.vpcId
});
}
}
const app = new cdk.App();
new ExampleStack(app, 'ExampleStack', {
env: {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
}});
and have a file location at ./config/123123123.json
{
"vpcId" : "123123123"
}

Adding Logic To Check If Infra is In Account and Deploying If Not AWS-CDK

The title may be a big vague so let me clarify. I am currently trying to enable AWSConfig rules and in order to do this the account must have AWSConfigurationRecorder and AWSDeliveryChannel. The issue lies that when an account already has this enabled, it will error out your entire stack when trying to deploy. I am trying to figure out a way to create logic that would essentially check if the AWSConfigurationRecorder or AWSDeliveryChannel are already there and if they are to skip over it and deploy just the rules and visa versa. Here is the code:
export class fullConfigStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const globalConfigRole = new iam.Role(this, 'globalConfigRole', {
assumedBy: new iam.ServicePrincipal('config.amazonaws.com'), // required
});
globalConfigRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSConfigRoleForOrganizations'));
globalConfigRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess'));
const globalConfigRecorder = new config.CfnConfigurationRecorder(this, 'globalConfigRecorder',{
roleArn: globalConfigRole.roleArn,
name: 'globalConfigRecorder',
recordingGroup: {
allSupported: true,
includeGlobalResourceTypes: true
}
});
const globalConfigBucket = new s3.Bucket(this, 'globalConfigBucket',{
accessControl: s3.BucketAccessControl.LOG_DELIVERY_WRITE
});
const cisConfigDeliveryChannel = new config.CfnDeliveryChannel(this,'cisConfigDeliveryChannel',{
s3BucketName: globalConfigBucket.bucketName,
configSnapshotDeliveryProperties: {
deliveryFrequency: 'TwentyFour_Hours'
}
});
const generalConfigRole = new iam.Role(this, 'generalConfigRole',{
assumedBy: new iam.ServicePrincipal('config.amazonaws.com')
});
const cloudTrailEnabledRule = new ManagedRule(this, 'cloudTrailEnabledRule', {
identifier: 'CLOUD_TRAIL_ENABLED'
});
So to clarify again I want to add some if/else logic with the cisConfigDeliveryChannel and globalConfigRecorder as to not error out the entire stack! If there is another way to solve this that I'm not seeing, please let me know!
In your AWS CloudFormation template, you can create a Lambda-backed custom resource with a function that checks whether your resources exist or not. This Lambda function then returns an identifier for CloudFormation to determine if the resources need to be created.

CloudFormation Cross-Region Reference

When you are running multiple CloudFormation stacks within the same region, you are able to share references across stacks using CloudFormation Outputs
However, outputs cannot be used for cross region references as that documentation highlights.
You can't create cross-stack references across regions. You can use the intrinsic function Fn::ImportValue to import only values that have been exported within the same region.
How do you reference values across regions in CloudFormation?
For an example to follow, I have a Route 53 hosted zone deployed in us-east-1. However, I have a backend in us-west-2 that I want to create a DNS-validated ACM certificate which requires a reference to the hosted zone in order to be able to create the appropriate CNAME for prove ownership.
How would I go about referencing that hosted zone id created in us-east-1 from within us-west-2?
The easiest way I have found of doing this is writing the reference you want to share (i.e. your hosted zone id in this case) to the Systems Manager Parameter Store and then referencing that value in your "child" stack in the separate region using a custom resource.
Fortunately, this is incredibly easy if your templates are created using Cloud Development Kit (CDK).
For the custom resource to read from SSM, you can use something like this:
// ssm-parameter-reader.ts
import { Construct } from '#aws-cdk/core';
import { AwsCustomResource, AwsSdkCall } from '#aws-cdk/custom-resources';
interface SSMParameterReaderProps {
parameterName: string;
region: string;
}
export class SSMParameterReader extends AwsCustomResource {
constructor(scope: Construct, name: string, props: SSMParameterReaderProps) {
const { parameterName, region } = props;
const ssmAwsSdkCall: AwsSdkCall = {
service: 'SSM',
action: 'getParameter',
parameters: {
Name: parameterName
},
region,
physicalResourceId: Date.now().toString() // Update physical id to always fetch the latest version
};
super(scope, name, { onUpdate: ssmAwsSdkCall });
}
public getParameterValue(): string {
return this.getData('Parameter.Value').toString();
}
}
To write the hosted zone id to parameter store, you can simply do this:
// route53.ts (deployed in us-east-1)
import { PublicHostedZone } from '#aws-cdk/aws-route53';
import { StringParameter } from '#aws-cdk/aws-ssm';
export const ROUTE_53_HOSTED_ZONE_ID_SSM_PARAM = 'ROUTE_53_HOSTED_ZONE_ID_SSM_PARAM';
/**
* Other Logic
*/
const hostedZone = new PublicHostedZone(this, 'WebsiteHostedZone', { zoneName: 'example.com' });
new StringParameter(this, 'Route53HostedZoneIdSSMParam', {
parameterName: ROUTE_53_HOSTED_ZONE_ID_SSM_PARAM,
description: 'The Route 53 hosted zone id for this account',
stringValue: hostedZone.hostedZoneId
});
Lastly, you can read that value from the parameter store in that region using the custom resource we just created and use that to create a certificate in us-west-2.
// acm.ts (deployed in us-west-2)
import { DnsValidatedCertificate } from '#aws-cdk/aws-certificatemanager';
import { PublicHostedZone } from '#aws-cdk/aws-route53';
import { ROUTE_53_HOSTED_ZONE_ID_SSM_PARAM } from './route53';
import { SSMParameterReader } from './ssm-parameter-reader';
/**
* Other Logic
*/
const hostedZoneIdReader = new SSMParameterReader(this, 'Route53HostedZoneIdReader', {
parameterName: ROUTE_53_HOSTED_ZONE_ID_SSM_PARAM,
region: 'us-east-1'
});
const hostedZoneId: string = hostedZoneIdReader.getParameterValue();
const hostedZone = PublicHostedZone.fromPublicHostedZoneId(this, 'Route53HostedZone', hostedZoneId);
const certificate = new DnsValidatedCertificate(this, 'ApiGatewayCertificate', { 'pdx.example.com', hostedZone });
The cdk library has been updated, the code avove needs to be changed to the following:
import { Construct } from '#aws-cdk/core';
import { AwsCustomResource, AwsSdkCall } from '#aws-cdk/custom-resources';
import iam = require("#aws-cdk/aws-iam");
interface SSMParameterReaderProps {
parameterName: string;
region: string;
}
export class SSMParameterReader extends AwsCustomResource {
constructor(scope: Construct, name: string, props: SSMParameterReaderProps) {
const { parameterName, region } = props;
const ssmAwsSdkCall: AwsSdkCall = {
service: 'SSM',
action: 'getParameter',
parameters: {
Name: parameterName
},
region,
physicalResourceId: {id:Date.now().toString()} // Update physical id to always fetch the latest version
};
super(scope, name, { onUpdate: ssmAwsSdkCall,policy:{
statements:[new iam.PolicyStatement({
resources : ['*'],
actions : ['ssm:GetParameter'],
effect:iam.Effect.ALLOW,
}
)]
}});
}
public getParameterValue(): string {
return this.getResponseField('Parameter.Value').toString();
}
}
CDK 2.x
There is a new Stack property called crossRegionReferences which you can enable to add cross region references. It's as simple as this:
const stack = new Stack(app, 'Stack', {
crossRegionReferences: true,
});
Under the hood, this does something similar to the above answers by using custom resources and Systems Manager. From the CDK docs:
crossRegionReferences?
Enable this flag to allow native cross region stack references.
Enabling this will create a CloudFormation custom resource in both the producing stack and consuming stack in order to perform the export/import
This feature is currently experimental
More details from the CDK core package README:
You can enable the Stack property crossRegionReferences
in order to access resources in a different stack and region. With this feature flag
enabled it is possible to do something like creating a CloudFront distribution in us-east-2 and
an ACM certificate in us-east-1.
When the AWS CDK determines that the resource is in a different stack and is in a different
region, it will "export" the value by creating a custom resource in the producing stack which
creates SSM Parameters in the consuming region for each exported value. The parameters will be
created with the name '/cdk/exports/${consumingStackName}/${export-name}'.
In order to "import" the exports into the consuming stack a SSM Dynamic reference
is used to reference the SSM parameter which was created.
In order to mimic strong references, a Custom Resource is also created in the consuming
stack which marks the SSM parameters as being "imported". When a parameter has been successfully
imported, the producing stack cannot update the value.
CDK 1.x
If you are on CDK 1.x, continue using the workaround that others have shared.
Update 2023-01-16 with cdkv2 version 2.56.0 in a projen generated projects (hence respecting eslint rules and best practices for formatting etc.) :
import {
aws_iam as iam,
custom_resources as cr,
} from 'aws-cdk-lib';
import { Construct } from 'constructs';
interface SSMParameterReaderProps {
parameterName: string;
region: string;
}
export class SSMParameterReader extends cr.AwsCustomResource {
constructor(scope: Construct, name: string, props: SSMParameterReaderProps) {
const { parameterName, region } = props;
const ssmAwsSdkCall: cr.AwsSdkCall = {
service: 'SSM',
action: 'getParameter',
parameters: {
Name: parameterName,
},
region,
physicalResourceId: { id: Date.now().toString() }, // Update physical id to always fetch the latest version
};
super(scope, name, {
onUpdate: ssmAwsSdkCall,
policy: {
statements: [
new iam.PolicyStatement({
resources: ['*'],
actions: ['ssm:GetParameter'],
effect: iam.Effect.ALLOW,
}),
],
},
});
}
public getParameterValue(): string {
return this.getResponseField('Parameter.Value').toString();
}
};
Could not edit the post above ...