Passing values from CloudFormation to Swagger file - amazon-web-services

I'm creating my infrastructure using CloudFormation.
I'm hoping to create the API using "API Gateway Extensions to OpenAPI"
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions.html
Relevant code segment looks like this.
"MorningApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Description": {
"Fn::Sub": "${Stage}-MorningApi"
},
"Name": {
"Fn::Sub": "${Stage}-MorningApi"
},
"EndpointConfiguration": {
"Types": [
"REGIONAL"
]
},
"BodyS3Location": {
"Bucket" : "cf-morning",
"Key" : "nested-templates-stack/MorningApiStatusStackSwagger.json"
}
}
},
"MorningApiloadBalancer": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"Name": {
"Fn::Sub": "${Stage}-MorningApiloadBalancer"
},
"Subnets": [
{
"Ref": "PrivateASubnet"
},
{
"Ref": "PrivateBSubnet"
},
{
"Ref": "PrivateCSubnet"
}
],
"Type": {
"Ref": "ELBType"
},
"Scheme": "internal",
"IpAddressType": {
"Ref": "ELBIpAddressType"
}
}
}
I need to pass "DNSName" of the "MorningApiloadBalancer" to the swagger file located in the S3 location. I cannot find a way to do this.
Any help is appreciated.

I haven't tried using swagger with API gateway before.
Using Fn::Transform
Using the Swagger file stored in S3 using the Fn::Tranform macro, it basically takes the content of the swagger file and tranforms into cloudformation.
{
"APIGateway": {
"Type": "AWS::ApiGateway::RestApi",
"Name": "myapi",
"Properties": {
"Body": {
"Fn::Transform": {
"Name": "AWS::Include",
"Parameters": {
"Location": {
"Fn::Sub": "s3://${AWS::AccountId}-swagger-files/${SwaggerFile}"
}
}
}
}
}
}
}
Reference:
https://medium.com/#nabtechblog/integrating-swagger-with-aws-lambda-and-api-gateway-using-cloud-formation-macro-functions-7432dec50dd
Inline swagger definition
I saw an example where the swagger definition is embedded into the cloudformation template. If you do so, you can use intrinsic functions inside the swagger definition. In your case, you can use Ref to get the dns name of the load balancer.
"GreetingApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "Greeting API",
"Description": "API used for Greeting requests",
"FailOnWarnings": true,
"Body": {
"swagger": "2.0",
"paths": {
"/greeting": {
"get": {
"x-amazon-apigateway-integration": {
"uri": {
"Fn::Join": [
"",
[
"arn:aws:apigateway:",
{
"Ref": "AWS::Region"
},
":lambda:path/2015-03-31/functions/",
{
"Fn::GetAtt": [
"GreetingLambda",
"Arn"
]
},
"/invocations"
]
]
}
}
}
}
}
}
}
}
Reference:
https://blog.jayway.com/2016/09/18/introduction-swagger-cloudformation-api-gateway/
Getting DNS Name
I think you know this already.
"Fn::GetAtt" : [ "MorningApiloadBalancer" , "DNSName" ]
Hope this helps.

Related

AWS SAM/Cloudformation configure API Gateway to point to lambda function version

I am currently trying to enable provisioned concurrency for my AWS lambda function. I already figured I can only do this on either a Lambda Function Version or a Lambda Function Alias. But I am having a hard time to point my API Gateway to this version, it seems to always point to the base function, not the version.
In the UI I can easily attach my Lambda Function Version to an API Gateway Endpoint, but I cannot figure out how to do it in my SAM Template.
This is what I currently have:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "Desc.",
"Parameters": { },
"Resources": {
"MyLambdaFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Runtime": "dotnetcore3.1",
"CodeUri": "MyCodeUri",
"MemorySize": 1024,
"Timeout": 30,
"Events": {
"HttpEvent1": {
"Type": "Api",
"Properties": {
"Path": "/v1/test",
"Method": "GET",
"RestApiId": {
"Ref": "ApiGateway"
}
}
}
},
"Handler": "MyNamespace::MyNamespace.MyFunc::RunAsync"
}
},
"MyLambdaFunctionConcurrentV1": {
"Type": "AWS::Lambda::Version",
"Properties": {
"FunctionName": {
"Ref": "MyLambdaFunction"
},
"ProvisionedConcurrencyConfig": {
"ProvisionedConcurrentExecutions": 1
}
}
},
"ApiGateway": {
"Type": "AWS::Serverless::Api",
"Properties": {
"StageName": {
"Ref": "ApiStageName"
},
"Cors": {
"AllowCredentials": true,
"AllowHeaders": "'*'",
"AllowMethods": "'*'",
"AllowOrigin": "'*'",
"MaxAge": "'600'"
}
}
}
},
"Outputs": {
"ApiUrl": {
"Description": "API Gateway Endpoint URL",
"Value": {
"Fn::Sub": "https://${ApiGateway}.execute-api.${AWS::Region}.${AWS::URLSuffix}/${ApiStageName}"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-ApiUrl"
}
}
}
}
}
So I am fine to deploy my Lambda function, my API Gateway and my version. But I cannot figure out how to link the API Gateway to my version.
I just figured out I was looking at the wrong documentation. As I have a SAM template and not a Cloudformation template I can use the AutoPublishAlias together with the ProvisionedConcurrencyConfig directly attached to my lambda function.
Knowing this, the solution was way easier - the version is not necessary as the SAM template version of the AWS::Serverless::Function supports ProvisionedConcurrencyConfig directly - as long as AutoPublishAlias is set as well.
This is my working template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "Desc.",
"Parameters": { },
"Resources": {
"MyLambdaFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"AutoPublishAlias": "V1",
"ProvisionedConcurrencyConfig": {
"ProvisionedConcurrentExecutions": 1
},
"Runtime": "dotnetcore3.1",
"CodeUri": "MyCodeUri",
"MemorySize": 1024,
"Timeout": 30,
"Events": {
"HttpEvent1": {
"Type": "Api",
"Properties": {
"Path": "/v1/test",
"Method": "GET",
"RestApiId": {
"Ref": "ApiGateway"
}
}
}
},
"Handler": "MyNamespace::MyNamespace.MyFunc::RunAsync"
}
},
"ApiGateway": {
"Type": "AWS::Serverless::Api",
"Properties": {
"StageName": {
"Ref": "ApiStageName"
},
"Cors": {
"AllowCredentials": true,
"AllowHeaders": "'*'",
"AllowMethods": "'*'",
"AllowOrigin": "'*'",
"MaxAge": "'600'"
}
}
}
},
"Outputs": {
"ApiUrl": {
"Description": "API Gateway Endpoint URL",
"Value": {
"Fn::Sub": "https://${ApiGateway}.execute-api.${AWS::Region}.${AWS::URLSuffix}/${ApiStageName}"
},
"Export": {
"Name": {
"Fn::Sub": "${AWS::StackName}-ApiUrl"
}
}
}
}
}
Side note: I also tried to apply the same AutoPublishAlias to multiple functions in the same stack - it works, so it does not need to be unique.

Cloudformation Property validation failure: Encountered unsupported properties

I'm trying to create a nested stack with the root stack looks like this:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"DynamoDBTable": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"TableName": {
"Fn::Sub": "${AWS::StackName}"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/dynamodb.json"
}
},
"S3WebsiteReact": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"BucketName": {
"Fn::Sub": "${AWS::StackName}-website"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/s3-static-website-react.json"
}
},
"S3UploadBucket": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"Parameters": {
"BucketName": {
"Fn::Sub": "${AWS::StackName}-upload"
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/s3-with-cors.json"
}
},
"Cognito": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": "DynamoDBTable",
"Properties": {
"Parameters": {
"CognitoUserPoolName": {
"Fn::Join" : ["",
{
"Fn::Split": ["-", {
"Ref": "AWS::StackName"
}]
}
]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/cognito.json"
}
},
"ApiGateway": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": ["DynamoDBTable", "Cognito"],
"Properties": {
"Parameters": {
"ApiGatewayName": {
"Fn::Sub": "${AWS::StackName}-api"
},
"CognitoUserPoolArn": {
"Fn::GetAtt": [ "Cognito", "Outputs.UserPoolArn" ]
},
"DynamoDBStack": {
"Fn::GetAtt": [ "DynamoDBTable", "Outputs.DDBStackName" ]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/api-gateway.json"
}
},
"IdentityPool": {
"Description": "Cognito Identity Pool. Must be created after User Pool and API Gateway.",
"Type": "AWS::Cognito::IdentityPool",
"DependsOn": ["Cognito", "ApiGateway", "S3UploadBucket"],
"Properties": {
"Parameters": {
"AppClientId": {
"Fn::GetAtt": [ "Cognito", "Outputs.AppClientId" ]
},
"UserPoolProviderName": {
"Fn::GetAtt": [ "Cognito", "Outputs.ProviderName" ]
},
"UserPoolName": {
"Fn::GetAtt": [ "Cognito", "Outputs.UserPoolName" ]
},
"UploadBucketName": {
"Fn::GetAtt": [ "S3UploadBucket", "Outputs.UploadBucketName" ]
},
"ApiGatewayId": {
"Fn::GetAtt": [ "ApiGateway", "Outputs.ApiGatewayId" ]
}
},
"TemplateURL": "https://s3.amazonaws.com/my-templates-bucket/identity-pool.json"
}
}
},
"Outputs": {
}
}
And I get this error:
2019-06-19 14:45:14 UTC-0400 IdentityPool CREATE_FAILED Property validation failure: [Encountered unsupported properties in {/}: [TemplateURL, Parameters]]
It looks like my identity pool stack has some issues with the parameters. But the identity pool stack parameters look like this:
"Parameters" : {
"AppClientId": {
"Description": "ID of the App Client of the Cognito User Pool passed into this stack.",
"Type": "String"
},
"UserPoolProviderName": {
"Description": "Cognito User Pool Provider name passed into this stack.",
"Type": "String"
},
"UserPoolName": {
"Description": "Cognito User Pool Name passed into this stack.",
"Type": "String"
},
"UploadBucketName": {
"Description": "Name of the bucket that is used to upload files to.",
"Type": "String"
},
"ApiGatewayId": {
"Description": "ID of the API Gateway created for the stack.",
"Type": "String"
}
},
The funny thing is: I tried creating each stack on its own, then passed the outputs from them as parameters to the stacks that need those parameters and every single stack was created successfully without any problems.
I've tried to look for what is unsupported but was unable to find any answers.
The error:
[Encountered unsupported properties in {/}: [TemplateURL, Parameters]]
Says that those two properties are unsupported. Unlike all the rest of the resources declared in your template which also use those two properties, this resource is a AWS::Cognito::IdentityPool, while the rest are all of type AWS::CloudFormation::Stack.
Those two properties are only valid on the AWS::CloudFormation::Stack type, hence the validation error.

AWS Cloudformation "Include" transform issue

AWS Cloudformation defines Transform section on global template level as an array.
See Transform section definition here.
I created two entries in Transform section that include some partial definitions of stack resources.
The goal is to create a few files with definitions/resources grouped by "product domain" per file.
When I create stack based on this template it executes one partial definition only (the last one - other definitions with "AWS::Include" name are ignored or "overridden")
This is the main template definition:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Test of json file inclusion",
"Parameters": {
"Environment": {
"Type": "String",
"Description": "Specify Environment: prod | dev ",
"AllowedValues": ["prod", "dev"],
"Default": "dev"
}
},
"Transform": [
{
"Name": "AWS::Include",
"Parameters": {
"Location": "s3://a1-local/cf-tests/part-1.json"
}
},
{
"Name": "AWS::Include",
"Parameters": {
"Location": "s3://a1-local/cf-tests/part-2.json"
}
}
],
"Outputs": {
}
}
This is part-1 definition
{
"Mappings": {
"MappingForBucket1": {
"eu-west-1": { "AZs": [ "eu-west-1a", "eu-west-1b" ] }
}
},
"Resources": {
"hellobucket1": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": { "Fn::Sub": "as-testbucket1-${Environment}" }
}
}
}
}
This is part-2 definition
{
"Mappings": {
"MappingForBucket2": {
"eu-west-1": { "AZs": [ "eu-west-1a", "eu-west-1b" ] }
}
},
"Resources": {
"hellobucket2": {
"Type": "AWS::S3::Bucket",
"Properties": {
"BucketName": { "Fn::Sub": "as-testbucket2-${Environment}" }
}
}
}
}
How to join/chain multiple transformations in such case?

Unable to set environment variable for aws lambda function executed as AWS::CloudFormation::CustomResource

I am trying to set the environment variable which takes it's value at runtime through my CloudFormation template json for CustomResource. So that later it executes a python lambda and I can read the environment variable in the lambda and process some data.
I want my python lambda to be able to read this variable inside os.environ
Following is my Cloudformation for CustomResource
"TriggerRedshiftSetupLambda": {
"Type": "AWS::CloudFormation::CustomResource",
"Version": 1.0,
"Properties": {
"Environment": {
"Variables": {
"AHost": {
"Fn::GetAtt" : [ "A", "Endpoint.Address" ]
},
"APort": {
"Fn::GetAtt" : [ "A", "Endpoint.Port" ]
}
}
},
"ServiceToken": {
"Fn::GetAtt" : [ "ASetupLambda", "Arn" ]
}
}
}
Here is my lambda code using the variable
def lambda_handler(event, context):
print(os.environ)
print(os.environ['AHost'])
The 1st print statement prints the entire environment variables list but doesn't have any key / value pair for 'AHost'
Am I doing something wrong? How to initialize environment variables through customresource for lambda correctly?
Setting environment variables through the custom resource definition seems not to be supported. What you are setting is the properties section for the actual invocation (so event data).
So taking your template, your configuration should be accessible under the following path.
event['ResourceProperties']['Environment']['Variables']['AHost']
As stated by #jens above it's not possible to set environment variables under os.environ using the CustomResource CloudFormation.
Instead, the Lambda CloudFormation needs to define those values -
"RedshiftSetupLambda": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": { "Fn::Sub": "XYZ-${Branch}" },
"S3Key": { "Fn::Sub": "XYZ-${Commit}.zip" }
},
"Description": "Setup Lambda",
"FunctionName": { "Fn::Sub": "${BlockId}-setup-${Branch}" },
"Handler": "setup.lambda_handler",
"KmsKeyArn": {
"Fn::ImportValue": {
"Fn::Sub": "${BlockId}-Common-RolesKeys-${Branch}-KMSKeyArn"
}
},
"Role": {
"Fn::ImportValue": {
"Fn::Sub": "${BlockId}-Common-RolesKeys-${Branch}-LambdaIAMRoleArn"
}
},
"Runtime": "python2.7",
"Timeout": 30,
"VpcConfig": {
"SecurityGroupIds": [ {"Ref": "SecurityGroup"} ],
"SubnetIds": [
{ "Fn::ImportValue": "VPCCreate-PrivateSubnet1Id" },
{ "Fn::ImportValue": "VPCCreate-PrivateSubnet2Id" },
{ "Fn::ImportValue": "VPCCreate-PrivateSubnet3Id" }
]
},
"Environment": {
"Variables": {
"DB_USERNAME": {
"Ref": "MasterUsername"
},
"AHOST": {
"Fn::GetAtt": ["RedshiftCluster", "Endpoint.Address"]
},
"APORT": {
"Fn::GetAtt": ["RedshiftCluster", "Endpoint.Port"]
},
"CLUSTER_IDENTIFIER": {
"Ref": "RedshiftCluster"
}
}
}
}
}
They can be accessed this way:
print(os.environ['AHOST'])

Can't send parameters from a child CloudFormation template to another child template

I am crafting a monstrosity of a CloudFormation template that uses the AWS::CloudFormation::Stack resource to spawn other nested stacks. The purpose of this is the classic separation of duties and management.
The overall goal is to build an application environment from the ground up (starting with VPC and ending with the application instances).
So far, most of it works. But I've hit a road block with referencing the outputs of the subnet CF template and the securitygroup CF template for use in creating the EC2 instances.
It works like this:
Master template --> Builds VPC --> Calls child template to build subnets --> Calls child template to build security groups --> Calls child template to build EC2 instances
I need to pass the outputs from the subnet and security group templates to the EC2 instance template so the instances can be provisioned into the correct part of the architecture. The VPC ID ref and KeyPairs pass in fine, but the subnetID and securitygroupID do not.
Here's the portion of the master template that calls the security group/subnet templates:
"DevNetworkStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": [
"SNVPC",
"SNIGW"
],
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevNetworkStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"VPCID": {
"Ref": "SNVPC"
},
"SNIGW": {
"Ref": "SNIGW"
}
}
}
},
"DevSecurityGroupsStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": "DevNetworkStack",
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevSecurityGroupsStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"VPCID": {
"Ref": "SNVPC"
}
}
}
},
These work fine. They create and everything is fine. The templates offer outputs like so:
"Outputs": {
"DevAdminSubnetID": {
"Description": "DevAdminSubnetID",
"Value": {
"Ref": "DevAdminSubnet"
}
},
...
"Outputs": {
"DevAdminSecurityGroupID": {
"Description": "DevAdminSecurityGroupID",
"Value": {
"Ref": "DevAdminSecurityGroup"
}
},
I can see the outputs in the CF console.
Now, the next template is trying to use the Security Group ID and the Subnet ID. But it's not working.
Master template calls the next child as:
"DevAdminStack": {
"Type": "AWS::CloudFormation::Stack",
"DependsOn": [
"DevNetworkStack",
"DevSecurityGroupsStack",
"EC2DevRoleInstanceProfile",
"S3DevUserDataBucket",
"S3DevHomeDirsDataBucket"
],
"Properties": {
"TemplateURL": {
"Fn::FindInMap": [
"TemplateURLs",
"DevAdminStack",
"URL"
]
},
"TimeoutInMinutes": "30",
"Parameters": {
"AdminKeyPair": {
"Ref": "AdminServersKeyPair"
},
"VPCID": {
"Ref": "SNVPC"
},
"DevAdminSecurityGroupID": [
{
"Fn::GetAtt": [
"DevSecurityGroupsStack",
"Outputs.DevAdminSecurityGroupID"
]
}
],
"DevAdminSubnetID": [
{
"Fn::GetAtt": [
"DevNetworkStack",
"Outputs.DevAdminSubnetID"
]
}
]
}
}
}
...and the child template looks like this (removed some portions for brevity because I'm just testing right now)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Dev-Admin",
"Mappings": {
"RegionMap": {
"DevAdminServer": {
"AMI": "ami-6fc9770e",
"InstanceType": "t2.micro"
}
}
},
"Parameters": {
"AdminKeyPair": {
"Type": "AWS::EC2::KeyPair::KeyName"
},
"VPCID": {
"Type": "AWS::EC2::VPC::Id"
},
"DevAdminSecurityGroupID": {
"Type": "AWS::EC2::SecurityGroup::Id"
},
"DevAdminSubnetID": {
"Type": "AWS::EC2::Subnet::Id"
}
},
"Resources": {
"DevAdminServer": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"Comment": "Sets up administrative tools for the server",
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"yum": {}
},
"files": {},
"services": {}
}
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": [
"RegionMap",
"DevAdminServer",
"AMI"
]
},
"SecurityGroupIds": [
{
"Ref": "DevAdminSecurityGroupID"
}
],
"SubnetId": {
"Ref": "DevAdminSubnetID"
},
"InstanceType": {
"Fn::FindInMap": [
"RegionMap",
"DevAdminServer",
"InstanceType"
]
},
"KeyName": {
"Ref": "AdminKeyPair"
},
"Tags": [
{
"Key": "Application",
"Value": {
"Ref": "AWS::StackId"
}
}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[]
]
}
}
}
}
}
}
But this resource fails on creation with the error:
Value of property Parameters must be an object with String (or simple type) properties
I know it's the last two variables causing the trouble (subnetID and securitygroupID), because I removed them and the provisioning the child template works just fine.
What am I doing wrong?
The values of DevAdminSecurityGroupID and DevAdminSubnetID are defined as JSON arrays [] in your master template, but they should be Strings (after the Fn::GetAtt intrinsic function expansion):
"DevAdminSecurityGroupID":
{
"Fn::GetAtt": [
"DevSecurityGroupsStack",
"Outputs.DevAdminSecurityGroupID"
]
},
"DevAdminSubnetID":
{
"Fn::GetAtt": [
"DevNetworkStack",
"Outputs.DevAdminSubnetID"
]
}