I'm using AWS CDK (v1.100.0) to manage API Gateway deployments.
The endpoint definitions come from a swagger file.
The stack looks something like this:
export class MyStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const api = new apigateway.SpecRestApi(this, 'my-api', {
cloudWatchRole: false,
endpointTypes: [ EndpointType.REGIONAL ],
apiDefinition: apigateway.ApiDefinition.fromAsset('path/to/swagger.yaml'),
deployOptions: {
stageName: 'dev',
etc...
}
});
}
}
This correctly creates an API Gateway and stage.
However, whenever I change the swagger file, for example, I change an endpoint method from POST to PUT, when I redeploy my changes, both the old and the new methods are present (same thing if I do other changes, e.g. modify endpoint path, etc).
Any insight into why this might be happening would be greatly appreciated.
You have to re-deploy the stage. You can do this by changing something on the AWS::ApiGatway::Deployment or AWS::ApiGateway::Stage to force it to update or by using a custom lambda function.
There is a Mode property in AWS::ApiGateway::RestApi which used to default to overwrite (see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html#cfn-apigateway-restapi-mode).
However recently the default value was changed from overwrite to merge
If you don't specify this property, a default value is chosen. For REST APIs created before March 29, 2021, the default is overwrite. Otherwise, the default value is merge.
This is what's causing this issue.
Unfortunately, this property has not been exposed in the CDK APIs yet. I have raised a GitHub issue, please see https://github.com/aws/aws-cdk/issues/14436.
In the meantime, a work around (as suggest in the ticket) is:
const api = new apigateway.SpecRestApi(this, 'my-api', {....});
(api.node.defaultChild as CfnRestApi).addPropertyOverride('Mode', 'overwrite');
Related
I have following AWS CDK backed solution:
Static S3 based webpage which communicates with
API Gateway which then sends data to
AWS lambda.
The problem is that S3 page needs to be aware of API gateway endpoint URL.
Obviously this is not achievable within the same CDK stack. So I have defined two stacks:
Backend (API gateway + lambda)
Frontend (S3 based static webpage)
They are linked as dependant in CDK code:
const app = new cdk.App();
const backStack = new BackendStack(app, 'Stack-back', {...});
new FrontendStack(app, 'Stack-front', {...}).addDependency(backStack, "API URL from backend is needed");
I try to share URL as follows.
Code from backend stack definition:
const api = new apiGW.RestApi(this, 'MyAPI', {
restApiName: 'My API',
description: 'This service provides interface towards web app',
defaultCorsPreflightOptions: {
allowOrigins: apiGW.Cors.ALL_ORIGINS,
}
});
api.root.addMethod("POST", lambdaIntegration);
new CfnOutput(this, 'ApiUrlRef', {
value: api.url,
description: 'API Gateway URL',
exportName: 'ApiUrl',
});
Code from frontend stack definition:
const apiUrl = Fn.importValue('ApiUrl');
Unfortunately, instead of URL I get token (${Token[TOKEN.256]}). At the same time, I see URL is resolved in CDK generated files:
./cdk.out/Stack-back.template.json:
"ApiUrlRef": {
"Description": "API Gateway URL",
"Value": {
"Fn::Join": [
"",
[
"https://",
{
"Ref": "MyAPI7DAA778AA"
},
".execute-api.us-west-1.",
{
"Ref": "AWS::URLSuffix"
},
"/",
{
"Ref": "MyAPIDeploymentStageprodA7777A7A"
},
"/"
]
]
},
"Export": {
"Name": "ApiUrl"
}
}
},
What I'm doing wrong?
UPD:
After advice of fedonev to pass data as props, situation did not changed much. Now url looks like that:
"https://${Token[TOKEN.225]}.execute-api.us-west-1.${Token[AWS.URLSuffix.3]}/${Token[TOKEN.244]}/"
I think important part I missed (which was also pointed by
Milan Gatyas) is how I create HTML with URL of gateway.
In my frontend-stack.ts, I use template file. After template is filled, I store it in S3:
const filledTemplatePath: string = path.join(processedWebFileDir,'index.html');
const webTemplate: string = fs.readFileSync(filledTemplatePath, 'utf8')
const Handlebars = require("handlebars")
let template = Handlebars.compile(webTemplate)
const adjustedHtml: string = template({ apiGwEndpoint: apiUrl.toString() })
fs.writeFileSync(filledTemplatePath, adjustedHtml)
// bucket
const bucket: S3.Bucket = new S3.Bucket(this, "WebsiteBucket",
{
bucketName: 'frontend',
websiteIndexDocument: 'index.html',
websiteErrorDocument: 'error.html',
publicReadAccess: true,
})
new S3Deploy.BucketDeployment(this, 'DeployWebsite', {
sources: [S3Deploy.Source.asset(processedWebFileDir)],
destinationBucket: bucket,
});
(I'm new to TS and web, please don't judge much :) )
Am I correct that S3 is populated on synth, deploy does not change anything and this is why I get tokens in html?
Will be grateful for a link or explanation so that I could understand the process better, there are so much new information to me that some parts are still quite foggy.
As #fedonev mentioned, the tokens are just placeholder values in the TypeScript application. CDK app replaces tokens with intrinsic functions when the CloudFormation template is produced.
However, your use case is different. You try to know the information inside the CDK app which is available only at synthesis time, and you can't use the intrinsic function to resolve the URL while being in CDK app to write to file.
If possible you can utilize the custom domain for the API Gateway. Then you can work with beforehand known custom domain in your static file and assign the custom domain to the API Gateway in your CDK App.
[Edit: rewrote the answer to reflect updates to the OP]
Am I correct that S3 is populated on synth, deploy does not change anything and this is why I get tokens in html?
Yes. The API URL will resolve only at deploy-time. You are trying to consume it at synth-time when you write to the template file. At synth-time, CDK represents not-yet-available values as Tokens like ${Token[TOKEN.256]}, the CDK's clever way of handling such deferred values.
What I'm doing wrong?
You need to defer the consumption of API URL until its value is resolved (= until the API is deployed). In most cases, passing constructs as props between stacks is the right approach. But not in your case: you want to inject the URL into the template file. As usual with AWS, you have many options:
Split the stacks into separate apps, deployed separately. Deploy BackendStack. Hardcode the url into FrontendStack. Quick and dirty.
Instead of S3, use Amplify front-end hosting, which can expose the URL to your template as an environment variable. Beginner friendly, has CDK support.
Add a CustomResource construct, which would be backed by a Lambda that writes the URL to the template file as part of the deploy lifecycle. This solution is elegant but not newbie-friendly.
Use a Pipeline to inject the URL variable as a build step during deploy. Another advanced approach.
Based on what's described here and on other pages, I created via CDK a Cognito User Pool and an Identity Pool, and, after manually mapping the custom attributes,
access is granted based on the custom attributes in the User Pool.
Now I'm trying to do everything in CDK, but I can't figure how to do the mapping of the custom attributes. The only thing I found that knows
about attribute mapping is UserPoolIdentityProvider
/ CfnUserPoolIdentityProvider,
but that is of the wrong type, and I cannot use it with
a CfnIdentityPool in cognitoIdentityProviders.
I saw some unanswered posts about the same issue (this,
or this), but
hope dies last, so I thought maybe there will be an answer this time.
I was under the impression that everything is doable via CloudFormation, but this seems mistaken, as
this post and others
suggest.
So can the attribute mapping be done with CDK, or I need to use custom resources and Lambdas (or perhaps something else) if I want to automate this?
Credits to original creator. Found this useful and solves the problem with Custom Resources.
https://github.com/aws-samples/amazon-cognito-abac-authorization-with-react-example/blob/main/lib/cognito_identity_pool_sample-stack.ts
new cognito.CfnIdentityPoolRoleAttachment(this, "defaultRoles", {
identityPoolId: identityPool.ref,
roles: {
'authenticated': authRole.attrArn
}
})
const createParameters = {
"IdentityPoolId": identityPool.ref,
"IdentityProviderName": userPool.userPoolProviderName,
"PrincipalTags": {
"department": "department"
},
"UseDefaults": false
}
const setPrincipalTagAction = {
action: "setPrincipalTagAttributeMap",
service: "CognitoIdentity",
parameters: createParameters,
physicalResourceId: customResources.PhysicalResourceId.of(identityPool.ref)
}
const { region, account } = Stack.of(this)
const identityPoolArn = `arn:aws:cognito-identity:${region}:${account}:identitypool/${identityPool.ref}`
// Creates a Custom resource (https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources-readme.html)
// This is necessary to attach Principal Tag mappings to the Identity Pool after it has been created.
// This uses the SDK, rather than CDK code, as attaching Principal Tags through CDK is currently not supported yet
new customResources.AwsCustomResource(this, 'CustomResourcePrincipalTags', {
onCreate: setPrincipalTagAction,
onUpdate: setPrincipalTagAction,
policy: customResources.AwsCustomResourcePolicy.fromSdkCalls({
resources: [identityPoolArn],
}),
})
I need a component for creating a large number of CodeCommit users. CloudFormation doesn't support adding a public SSH key for an IAM user, so I have to create my own. CDK comes with AwsCustomResource, which does the heavy lifting of creating the Lambda that handles the required CloudFormation events. In other words, my code would be something like:
import { User } from 'aws-cdk-lib/aws-iam';
import { AwsCustomResource } from 'aws-cdk-lib/custom-resources';
import { Construct } from 'constructs';
export interface CodeCommitUserProps {
userName: string;
emailAddress: string;
sshPublicKey: string;
}
export class CodeCommitUser extends Construct {
constructor(scope: Construct, id: string, props: CodeCommitUserProps) {
super(scope, id);
const user = new User(this, 'codecommit-' + props.userName, {
userName: 'codecommit-' + props.userName,
path: '/git/users'
});
}
const custom = new AwsCustomResource(this, ... );
}
Now if I call new CodeCommitUser(...) a few hundred times, I would assume that there will be one CloudFormation event Lambda per user, even if all of them are identical. Is there a way to reuse the Lambdas created by AwsCustomResource if I need multiple copies of the custom resource?
I would assume that there will be one CloudFormation event Lambda per user, even if all of them are identical.
Actually, no. CDK creates a single lambda function, no matter how many times CodeCommitUser is instantiated. How does CDK manage this? Under the hood, CDK uses a SingletonFunction for the AWSCustomResource provider (see the github source). Singleton Functions are guaranteed to be added to the stack "once and only once, irrespective of how many times the construct is declared to be part of the stack"
Is there a way to reuse the Lambdas created by AwsCustomResource if I need multiple copies of the custom resource?
Again, reuse happens automagically. You can prove this to yourself by cdk synth-ing the stack with multiple CodeCommitUsers defined. Then look in the cdk.out directory for the outputted CloudFormation template. The template have only one AWS::Lambda::Function resource defined (assuming your app doesn't use lambdas elsewhere).
You can create your custom resource lambda and deploy it in a separate template. Then you can call on it from what ever template you want.
You can call this resource as many times you want from a single template.
You can either send a list of users in one go, or create a resource for each user (probably not ideal if you talking hundreds).
I wan to verify an email address from my CDK itself so that when my stack is deployed tto some other regions this verification is automatically triggered, rather than going to AWS console and doing it manually.
You can either do that using AwsCustomResource from #aws-cdk/custom-resources and it looks similar to the example that you can find here for validating a domain: Custom Resource Examples.
Verify email using TypeScript
I'm adjusting the example here for your use case:
const verifyDomainIdentity = new AwsCustomResource(this, 'VerifyDomainIdentity', {
onCreate: {
service: 'SES',
action: 'verifyEmailIdentity',
parameters: {
EmailAddress: 'your#example.com'
},
physicalResourceId: PhysicalResourceId.of('verify-email-address')
},
policy: AwsCustomResourcePolicy.fromSdkCalls({resources: AwsCustomResourcePolicy.ANY_RESOURCE}) // This does not work somehow with SES or maybe I did something wrong :-(
});
Unfortunately this does not work out of the box because somehow the generated policy includes an email: prefix instead of ses: and you need to provide your own policy. But there's an alternative below.
Using an existing CDK Construct with TypeScript
The other alternative is to use a CDK Construct which is already doing that for you. I recently ran into the same problem like you and I've published a CDK Construct for that: ses-verify-identities. You can then do it like this:
new VerifySesEmailAddress(this, 'SesEmailVerification', {
emailAddress: 'hello#example.org'
});
You can find the source code of the CDK construct here in case you are interested. The same is possible for verifying domains.
I'm exploring how refactorable CDK applications are. Suppose I defined a custom construct (a stack) to create an EKS cluster. Let's call it EksStack. Ideally, I'd create the role to be associated with the cluster and the EKS cluster itself, as described by the following snippet (I'm using Scala instead of Java, so the snippets are going to be in Scala syntax):
class EksStack (scope: Construct, id: String, props: StackProps) extends Stack(scope, id, props) {
private val role = new Role(this, "eks-role", RoleProps.builder()
.description(...)
.managedPolicies(...)
.assumedBy(...)
.build()
)
private val cluster = new Cluster(this, "eks-cluster", ClusterProps.builder()
.version(...)
.role(role)
.defaultCapacityType(DefaultCapacityType.EC2)
.build()
)
}
When I synthetize the application, I can see that the generated template contains the definition of the VPC, together with the Elastic IPs, NATs, Internet Gateways, and so on.
Now suppose that I want to refactor EksStack and have a different stack, say VpcStack, explicitly create the VPC:
class VpcStack (scope: Construct, id: String, props: StackProps) extends Stack(scope, id, props) {
val vpc = new Vpc(this, VpcId, VpcProps.builder()
.cidr(...)
.enableDnsSupport(true)
.enableDnsHostnames(true)
.maxAzs(...)
.build()
)
}
Ideally, the cluster in EksStack would just be using the reference to the VPC created by VpcStack, something like (note the new call to vpc() in the builder of cluster):
class EksStack (scope: Construct, id: String, props: StackProps, vpc: IVpc) extends Stack(scope, id, props) {
private val role = new Role(this, "eks-role", RoleProps.builder()
.description(...)
.managedPolicies(...)
.assumedBy(...)
.build()
)
private val cluster = new Cluster(this, "eks-cluster", ClusterProps.builder()
.version(...)
.role(role)
.vpc(vpc)
.defaultCapacityType(DefaultCapacityType.EC2)
.build()
)
}
This obviously doesn't work, as CloudFormation would delete the VPC created by EksStack in favor of the one created by VpcStack. I read here and there and tried to add a retain policy in EksStack and to override the logical ID of the VPC in VpcStack, using the ID I originally saw in the CloudFormation template for EksStack:
val cfnVpc = cluster.getVpc.getNode.getDefaultChild.asInstanceOf[CfnVPC]
cfnVpc.applyRemovalPolicy(RemovalPolicy.RETAIN)
and
val cfnVpc = vpc.getNode.getDefaultChild.asInstanceOf[CfnVPC]
cfnVpc.overrideLogicalId("LogicalID")
and then retried the diff. Again, it seems that the VPC is deleted and re-created.
Now, I saw that it is possible to migrate CloudFormation resources (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/refactor-stacks.html) using the "Import resources into stack" action. My question is: can I move the creation of a resource from a stack to another in CDK without re-creating it?
EDIT:
To elaborate a bit on my problem, when I define the VPC in VpcStack, I'd like CDK to think that the resource was created by VpcStack instead ok EksStack. Something like moving the definition of it from one stack to another without having CloudFormation delete the original one to re-create it. In my use case, I'd have a stack define a create initially (either explicitly or implicitly, such as my VPC), but then, after I while, I might want to refactor my application, moving the creation of that resource in a dedicated stack. I'm trying to understand if this moving always leads to the resource being re-created of if there's any way to avoid it.
I think when it comes to refactorability when CDK and CloudFormation in general, especially multi stack configurations, there are a few principles to keep in mind.
The entire app should be able to be completely deleted and recreated. All data management is handled in the app, there is no manual processes that need to occur.
Don't always rely on auto interstack dependency management using Stack exports. I like to classify CloudFormation dependencies into two categories: Hard and soft. Hard dependencies means that you cannot delete the resource because the things using it will prevent it from happening. Soft dependencies are the opposite, the resource could be deleted and recreated without issue even though something else is using it. Hard dependency examples: VPC, Subnets. Soft dependency examples: Topic/Queue/Role.
You'll have a better time passing stack soft dependencies as Stack parameters of SSM Parameter type because you'll be able to update the stack providing the dependencies independent of those using it. Whereas you get into a deadlock when using default stack export method. You can't delete the resources because something else is importing it. So you end up having to do annoying things to make it work like deploying once with it duplicated then deploying again deleting the old stuff. It requires a little extra work to use SSM Parameters without causing stack exports but it is worth it long term for soft dependencies.
For hard dependencies, I disagree with using a lookup because you really do want to prevent deletion if something is using it bc you'll end up with a DELETE_FAILED stack and that is a terrible place to end up. So for things like VPC/Subnets, I think it's really important to actually use stack export/import technique and if you do need to recreate your VPC because of a change, if you followed principle 1, you just need to do a CDK destroy then deploy and all will be good because you built your CDK app to be fully recreatable.
When it comes to recreatability with data, CustomResources are your friend.
I'm not sure if I understand the problem, but if you are trying to reference an existing resource you can use a context query. (e.g. Vpc.fromLookup).
https://docs.aws.amazon.com/cdk/latest/guide/context.html
Additionally, if you would like to use the Vpc created from VpcStack inside of EksStack you can output the vpc id from the VpcStack and use the context query in the eks stack that way.
this is C# code but the principal is the same.
var myVpc = new Vpc(...);
new CfnOutput(this, "MyVpcIdOutput", new CfnOutputProps()
{
ExportName = "VpcIdOutput",
Value = myVpc.VpcId
}
and then when you create the EksStack you can import the vpc id that you previously exported.
new EksStack(this, "MyCoolStack", new EksStackProps()
{
MyVpcId = Fn.ImportValue("VpcIdOutput")
}
where EksStackProps is
public class EksStackProps
{
public string MyVpcId { get; set; }
}