AWS Cloudformation Link API Key to API Gateway - amazon-web-services

I have the following Cloudformation template I am trying to deploy via SAM. This template correctly creates the DynamoDB table, an API Key, a Lambda function and the API Gateway, but I cannot figure out what I need to specify in the template to associate the API KEY with the API Gateway.
I have found plenty of snippets showing partial examples, but I am struggling to piece it all together.
Thank you in advance,
Denny
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Parameters:
TableName:
Type: String
Default: 'influencetabletest'
Description: (Required) The name of the new DynamoDB table Minimum 3 characters
MinLength: 3
MaxLength: 50
AllowedPattern: ^[A-Za-z-]+$
ConstraintDescription: 'Required parameter. Must be characters only. No numbers allowed.'
CorsOrigin:
Type: String
Default: '*'
Description: (Optional) Cross-origin resource sharing (CORS) Origin. You can specify a single origin, all "*" or leave empty and no CORS will be applied.
MaxLength: 250
Conditions:
IsCorsDefined: !Not [!Equals [!Ref CorsOrigin, '']]
Resources:
ApiKey:
Type: AWS::ApiGateway::ApiKey
DependsOn:
- ApiGetter
Properties:
Name: "TestApiKey"
Description: "CloudFormation API Key V1"
Enabled: "true"
ApiGetter:
Type: AWS::Serverless::Api
Properties:
StageName: prd
DefinitionBody:
swagger: 2.0
info:
title:
Ref: AWS::StackName
paths:
/getdynamicprice:
post:
responses: {}
x-amazon-apigateway-integration:
httpMethod: POST
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaGetter.Arn}/invocations
LambdaGetter:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./index.js
Handler: index.handler
Runtime: nodejs8.10
Environment:
Variables:
TABLE_NAME: !Ref TableName
IS_CORS: IsCorsDefined
CORS_ORIGIN: !Ref CorsOrigin
PRIMARY_KEY: !Sub ${TableName}Id
Policies:
- DynamoDBCrudPolicy:
TableName: !Ref TableName
Events:
Api:
Type: Api
Properties:
Path: /getdynamicprice
Method: POST
RestApiId: !Ref ApiGetter
DynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: !Ref TableName
AttributeDefinitions:
-
AttributeName: !Sub "${TableName}Id"
AttributeType: "S"
KeySchema:
-
AttributeName: !Sub "${TableName}Id"
KeyType: "HASH"
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
Outputs:
ApiKeyID:
Value: !Ref ApiKey
ApiUrl:
Value: !Sub https://${ApiGetter}.execute-api.${AWS::Region}.amazonaws.com/prod/getdynamicprice
Description: The URL of the API Gateway you invoke to get your dynamic pricing result.
DynamoDBTableArn:
Value: !GetAtt DynamoDBTable.Arn
Description: The ARN of your DynamoDB Table
DynamoDBTableStreamArn:
Value: !GetAtt DynamoDBTable.StreamArn
Description: The ARN of your DynamoDB Table Stream

Edit (04/22/2020): there now seems to do all this using AWS SAM. Please see answer below
Here's a sample template where I have connected my API to a API key. But that's only been possible because I am using usage plans. I believe that is the primary purpose of an API key. API gateway usage plan
ApiKey:
Type: AWS::ApiGateway::ApiKey
Properties:
Name: !Join ["", [{"Ref": "AWS::StackName"}, "-apikey"]]
Description: "CloudFormation API Key V1"
Enabled: true
GenerateDistinctId: false
ApiUsagePlan:
Type: "AWS::ApiGateway::UsagePlan"
Properties:
ApiStages:
- ApiId: !Ref <API resource name>
Stage: !Ref <stage resource name>
Description: !Join [" ", [{"Ref": "AWS::StackName"}, "usage plan"]]
Quota:
Limit: 2000
Period: MONTH
Throttle:
BurstLimit: 10
RateLimit: 10
UsagePlanName: !Join ["", [{"Ref": "AWS::StackName"}, "-usage-plan"]]
ApiUsagePlanKey:
Type: "AWS::ApiGateway::UsagePlanKey"
Properties:
KeyId: !Ref <API key>
KeyType: API_KEY
UsagePlanId: !Ref ApiUsagePlan
There does not seem to be a way to do this without a usage plan.

I did try the suggestion from ASR but ended up with a simpler approach.
The AWS SAM (Serverless Application Model) contains prepackaged handling that doesn't necessitate the use of resources of the ApiGateway type.
To create an API Gateway with a stage that requires an authorization token in the header the following simplified code should do it for you :
Resources:
ApiGatewayEndpoint:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Auth:
ApiKeyRequired: true
UsagePlan:
CreateUsagePlan: PER_API
UsagePlanName: GatewayAuthorization [any name you see fit]
LambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: lambda.handler
Runtime: python3.7
Timeout: 30
CodeUri: .
Events:
PostEvent:
Type: Api
Properties:
Path: /content
Method: POST
RequestParameters:
- method.request.header.Authorization:
Required: true
Caching: true
RestApiId:
Ref: ApiGatewayEndpoint [The logical name of your gateway endpoint above]
The elements:
Auth:
ApiKeyRequired: true
UsagePlan:
CreateUsagePlan: PER_API
is what does the trick.
Cloudformation handles the plumbing for you, ie. the Api Key, UsagePlan and UsagePlanKey is automatically created and binded.
Although the docs are definitely not best in class they do provide some additional information: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-resources-and-properties.html

Related

Error understanding Apigateway and nested stack on AWS cloudformation

I working into decouple a big template with lambdas that use apigateway AWS. I resolve multiple errors about nested stack process, but currently error its not clear. Could you check whats its the problem into definitions?
Main stack show general error from api substack create, but api-substack show next error:
Template error: instance of Fn::Sub references invalid resource attribute CreateCatalogFunctionDev.Arn
Next show a code of templates:
Main Template
SubStackAPIDev:
Type: 'AWS::CloudFormation::Stack'
Properties:
TemplateURL: https://s3.awsroute.com/substack-api.yaml
TimeoutInMinutes: 5
Parameters:
CreateCatalogFunctionDev: !GetAtt CreateCatalogFunctionDev.Outputs.CreateCatalogFunctionDev
SubStackCreateCatalogDev:
Type: 'AWS::CloudFormation::Stack'
Properties:
TemplateURL: https://s3.awsroute.com/substack-create-catalog.yaml
TimeoutInMinutes: 5
Parameters:
APIDev: !GetAtt SubStackAPIDev.Outputs.APIGateway
SubStackCreateCatalogDev
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
SAM template for create catalog
Parameters:
SecretsManagerName:
Type: String
Description: "Secrets Manager Name"
SnsTopicArn:
Type: String
Default: arn:aws:sns:us-west-2:XXXXXX:SNS-errorNotification
Description: "Sns topic error handler email notification"
APIDev:
Type: AWS::Serverless::Api
Resources:
LayerDev:
Type: 'AWS::Serverless::LayerVersion'
Properties:
ContentUri: ../../layer
CompatibleRuntimes:
- python3.6
- python3.7
- python3.8
RetentionPolicy: Delete
CreateCatalogFunctionDev:
Type: AWS::Serverless::Function
Properties:
Description: Recieve api request and and process data.
CodeUri: ../../src/catalog/create_catalog/
Handler: create_catalog.lambda_handler
Runtime: python3.8
FunctionName: CreateCatalogFunctionDev
Role: arn:aws:iam::XXXXXXXX:role/lambda-essential-role
Timeout: 480
Environment:
Variables:
CREATE_CATALOG_SECRET_NAME: !Sub '${SecretsManagerName}'
SNS_ARN: !Sub '${SnsTopicArn}'
Layers:
- arn:aws:lambda:us-west-2:XXXXXXX:layer:requests:1
- arn:aws:lambda:us-west-2:XXXXXXX:layer:requests-oauthlib:1
- !Ref LayerDev
Events:
CreateCatalogApiEvent:
Type: Api
Properties:
Path: /api-channel/products/catalog
Method: POST
RestApiId: !Ref APIDev
SubStack API
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
sub_channels_appi
SAM template for API
Parameters:
SwaggerFile:
Type: String
# TODO dejar URL de S3 de gitla-cicd
Default: "s3://cf-templates-1hurrmgzyzoz3-ap-northeast-1/swagger_dev.yaml"
Description: "This swagger file Amazon S3 path"
SecretsManagerName:
Type: String
Default: "/channels/Dev"
Description: "Secrets Manager Name"
StageName:
Type: String
Default: "${stageVariables.alias}"
Description: "This is the alias to the swagger file"
UsaSupplyApiUrlDev:
Type: String
Default: "https://thecornercloud.com/developers/index.php/"
Description: "Corner Cloud Staging"
SnsTopicArn:
Type: String
Default: arn:aws:sns:us-west-2:000365055762:channels-errorNotification
Description: "Sns topic error handler email notification"
CreateCatalogFunctionDev:
Type: String
Resources:
#######################################
# Serverless LayerVersion
#######################################
LayerDev:
Type: 'AWS::Serverless::LayerVersion'
Properties:
ContentUri: ../../layer
CompatibleRuntimes:
- python3.6
- python3.7
- python3.8
RetentionPolicy: Delete
#######################################
# Serverless API
#######################################
APIDev:
Type: AWS::Serverless::Api
Properties:
Auth:
ApiKeyRequired: true
StageName: dev
EndpointConfiguration: REGIONAL
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: !Ref SwaggerFile
Variables:
alias: dev
#######################################
# ApiGateway ApiKey
#######################################
APIKeyDev:
Type: AWS::ApiGateway::ApiKey
Properties:
Name: "APIKeyDev"
Description: "API Key Dev"
Enabled: true
GenerateDistinctId: false
StageKeys:
- RestApiId: !Ref APIDev
StageName: !Ref APIDev.Stage
#######################################
# ApiGateway UsagePlan
#######################################
APIUsagePlanDev:
Type: AWS::ApiGateway::UsagePlan
DependsOn: APIDev
Properties:
ApiStages:
- ApiId: !Ref APIDev
Stage: !Ref APIDev.Stage
Quota:
Limit: 5000
Period: MONTH
Throttle:
BurstLimit: 200
RateLimit: 100
UsagePlanName: APIUsagePlanDev
#######################################
# ApiGateway UsagePlanKey
#######################################
APIUsagePlanKeyDev:
Type: AWS::ApiGateway::UsagePlanKey
Properties:
KeyId: !Ref APIKeyDev
KeyType: API_KEY
UsagePlanId: !Ref APIUsagePlanDev
#######################################
# ApiGateway Deployment
#######################################
DeploymentApiIdDev:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref APIDev
Outputs:
APIGateway:
Description: "API Gateway Reference"
Value: !Ref APIDev
Export:
Name: !Join [":", [!Ref "AWS::StackName", "APIDev"]]
And finally the swagger file (honestly, I didn't define the api with this method and think that I want remove swagger if is possible).
swagger-dev
swagger: "2.0"
info:
version: "1.0.0"
title: "APIDev"
tags:
- name: "Channels"
description: "Manage Channels process."
schemes:
- "https"
x-amazon-apigateway-api-key-source: "HEADER"
securityDefinitions:
APIKey:
type: apiKey
name: X-Api-Key
in: header
paths:
/channels/products/catalog:
post:
tags:
- "Channels"
summary: " products catalog post."
operationId: "ProductsCatalogPostDev"
produces:
- "application/json"
responses:
201:
description: "Successful Operation"
400:
description: "Invalid parameters"
401:
description: "Unauthorized"
405:
description: "Validation exception"
security:
- APIKey: []
x-amazon-apigateway-integration:
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${CreateCatalogFunctionDev.Arn}/invocations
responses:
default:
statusCode: "200"
passthroughBehavior: "when_no_match"
httpMethod: "POST"
contentHandling: "CONVERT_TO_TEXT"
type: "aws_proxy"
Your CreateCatalogFunctionDev is in a different sub-stack then APIDev. You can't reference resources directly across stacks. You either have to export/import their outputs, or pass the references as input parameters.

How to attach authorization/api key to the sam cli generated api?

I used sam cli , to create a project. when I package this and deploy , it creates the lambda and also the api gateway with stage and prod stages, policy, roles e.t.c by default, without having to explicitly define in the cloudformation template ( see code below) . as it generates the api gateway automatically, how do i add/attach say if i wanted to add a api key or some kind of authorization for my api generated by template below?
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
simple-node-api
Sample SAM Template for simple-node-api
Globals:
Function:
Timeout: 3
Resources:
ServerlessHttpApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Auth:
ApiKeyRequired: true # sets for all methods
DefinitionBody:
swagger:2.0
paths:
"/myresource":
post:
x-amazon-apigateway-integration
httpMethod: post
type: aws_proxy
uri: ...
ApiKey:
Type: AWS::ApiGateway::ApiKey
Properties:
Name: !Join ["", [{"Ref": "AWS::StackName"}, "-apikey"]]
Description: "CloudFormation API Key V1"
Enabled: true
GenerateDistinctId: false
Value: abcdefg123456
StageKeys:
- RestApiId: !Ref ServerlessHttpApi
StageName: Prod
ApiUsagePlan:
Type: "AWS::ApiGateway::UsagePlan"
Properties:
ApiStages:
- ApiId: !Ref ServerlessHttpApi
Stage: Prod
Description: !Join [" ", [{"Ref": "AWS::StackName"}, "usage plan"]]
Quota:
Limit: 1000
Period: MONTH
UsagePlanName: !Join ["", [{"Ref": "AWS::StackName"}, "-usage-plan"]]
ApiUsagePlanKey:
Type: "AWS::ApiGateway::UsagePlanKey"
DependsOn:
- ServerlessHttpApi
Properties:
KeyId: !Ref ApiKey
KeyType: API_KEY
UsagePlanId: !Ref ApiUsagePlan
HelloWorldfunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: python3.7
Events:
HelloWorld:
Type: Api
Properties:
RestApiId: !Ref ServerlessHttpApi
Path: /hello
Method: get
Outputs:
ServerlessHttpApi:
Description: API Gateway endpoint URL for Prod stage for Hello World function
Value:
Fn::Sub: https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
HelloWorldfunction:
Description: Express Backend Lambda Function ARN
Value: !Sub HelloWorldfunction.Arn
HelloWorldFunctionIamRole:
Description: Implicit IAM Role created for Hello World function
Value: !Sub HelloWorldFunctionRole.Arn
I modified your code to use the API keys as shown here.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
simple-node-api
Sample SAM Template for simple-node-api
Globals:
Function:
Timeout: 3
Resources:
ServerlessHttpApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Auth:
ApiKeyRequired: true # sets for all methods
ApiKey:
Type: AWS::ApiGateway::ApiKey
DependsOn: [ApiUsagePlan]
Properties:
Name: !Join ["", [{"Ref": "AWS::StackName"}, "-apikey"]]
Description: "CloudFormation API Key V1"
Enabled: true
GenerateDistinctId: false
Value: abcdefg123456665ffghsdghfgdhfgdh4565
StageKeys:
- RestApiId: !Ref ServerlessHttpApi
StageName: Prod
ApiUsagePlan:
Type: "AWS::ApiGateway::UsagePlan"
DependsOn:
- ServerlessHttpApiProdStage
Properties:
ApiStages:
- ApiId: !Ref ServerlessHttpApi
Stage: Prod
Description: !Join [" ", [{"Ref": "AWS::StackName"}, "usage plan"]]
Quota:
Limit: 1000
Period: MONTH
UsagePlanName: !Join ["", [{"Ref": "AWS::StackName"}, "-usage-plan"]]
ApiUsagePlanKey:
Type: "AWS::ApiGateway::UsagePlanKey"
DependsOn:
- ServerlessHttpApi
Properties:
KeyId: !Ref ApiKey
KeyType: API_KEY
UsagePlanId: !Ref ApiUsagePlan
HelloWorldfunction:
Type: AWS::Serverless::Function
Properties:
#CodeUri: hello-world/
CodeUri: ./
Handler: app.lambdaHandler
Runtime: python3.7
Events:
HelloWorld:
Type: Api
Properties:
RestApiId: !Ref ServerlessHttpApi
Path: /hello
Method: get
Outputs:
ServerlessHttpApi:
Description: API Gateway endpoint URL for Prod stage for Hello World function
Value:
Fn::Sub: https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
HelloWorldfunction:
Description: Express Backend Lambda Function ARN
Value: !Sub HelloWorldfunction.Arn
HelloWorldFunctionIamRole:
Description: Implicit IAM Role created for Hello World function
Value: !Sub HelloWorldFunctionRole.Arn
I commented out some parts so that I can run the code, and I can confirm that it deploys and the API auth is set and API key present:
You have to mention it in your AWS SAM template. Below is an example:
Resources:
MyApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Auth:
ApiKeyRequired: true # sets for all methods
MyFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: .
Handler: index.handler
Runtime: nodejs12.x
Events:
ApiKey:
Type: Api
Properties:
RestApiId: !Ref MyApi
Path: /
Method: get
Auth:
ApiKeyRequired: true
You can read more about it here

CloudFront 403 Forbidden Exception (AWS SAM Template)

I'm deploying a serverless application using the AWS SAM Cli and template, but the API Gateway resource is returning a 403 ForbiddenException error when trying to curl / postman it. Tried looking online, but haven't been able to find any answers that solved my issue and wondering if anyone here has experienced this before.
template.yaml:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: nodejs10.x
MemorySize: 256
Api:
Cors:
AllowMethods: "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'"
AllowHeaders: "'Content-Type,X-Amz-Date,X-Amz-Security-Token,Authorization,X-Api-Key,X-Requested-With,Accept,Access-Control-Allow-Methods,Access-Control-Allow-Origin,Access-Control-Allow-Headers'"
AllowOrigin: "'*'"
Parameters:
ApiKey:
Type: String
Default: none
Conditions:
CreateApiKey: !Not [!Equals [!Ref ApiKey, 'none']]
Resources:
# DynamoDB table setup
DyanmoDBStoryTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: Stories
AttributeDefinitions:
- AttributeName: short_id
AttributeType: S
KeySchema:
- AttributeName: short_id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 0
WriteCapacityUnits: 0
BillingMode: PAY_PER_REQUEST
# Log group
DynamoSaveStoryLogGroup:
Type: AWS::Logs::LogGroup
DependsOn: [DynamoSaveStoryLambda]
Properties:
RetentionInDays: 30
LogGroupName: !Sub '/aws/lambda/${DynamoSaveStoryLambda}'
DynamoGetStoryLogGroup:
Type: AWS::Logs::LogGroup
DependsOn: [DynamoGetStoryLambda]
Properties:
RetentionInDays: 30
LogGroupName: !Sub '/aws/lambda/${DynamoGetStoryLambda}'
DynamoUpdateStoryLogGroup:
Type: AWS::Logs::LogGroup
DependsOn: [DynamoUpdateStoryLambda]
Properties:
RetentionInDays: 30
LogGroupName: !Sub '/aws/lambda/${DynamoUpdateStoryLambda}'
# Lambda Fn
DynamoSaveStoryLambda:
Type: AWS::Serverless::Function
Properties:
Policies:
- AmazonDynamoDBFullAccess
Handler: src/lambdas/save-story.handler
Timeout: 10
Events:
SaveStory:
Type: Api
Properties:
RestApiId: !Ref ApiGateway
Path: /story
Method: post
DynamoGetStoryLambda:
Type: AWS::Serverless::Function
Properties:
Policies:
- AmazonDynamoDBFullAccess
Handler: src/lambdas/get-story.handler
Timeout: 10
Events:
SaveStory:
Type: Api
Properties:
RestApiId: !Ref ApiGateway
Path: /story/{shortId}
Method: get
DynamoUpdateStoryLambda:
Type: AWS::Serverless::Function
Properties:
Policies:
- AmazonDynamoDBFullAccess
Handler: src/lambdas/update-story.handler
Timeout: 10
Events:
SaveStory:
Type: Api
Properties:
RestApiId: !Ref ApiGateway
Path: /story/{shortId}
Method: post
# Custom API gateway setup API Keys & usage plans
ApiGateway:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Auth:
ApiKeyRequired: true
UsagePlan:
Type: AWS::ApiGateway::UsagePlan
DependsOn: [ApiGatewayProdStage]
Condition: CreateApiKey
Properties:
ApiStages:
- ApiId: !Ref ApiGateway
Stage: Prod
DynamoLambdasApiKey:
Type: AWS::ApiGateway::ApiKey
DependsOn: [UsagePlan]
Condition: CreateApiKey
Properties:
Value: !Ref ApiKey
Enabled: true
StageKeys:
- RestApiId: !Ref ApiGateway
StageName: Prod
UsagePlanKey:
Type: AWS::ApiGateway::UsagePlanKey
Condition: CreateApiKey
Properties:
KeyId: !Ref DynamoLambdasApiKey
KeyType: API_KEY
UsagePlanId: !Ref UsagePlan
Outputs:
StoryApi:
Description: Serverless api url generated by AWS Cloudformation upon stack deployment
Value: !Sub 'https://${ApiGateway}.execute-api.${AWS::Region}.amazonaws.com/prod'
ApiKey:
Description: Api key to authorize access in API Gateway
Value: !Ref ApiKey
SAM CLI Version: 0.47.0
Error:
Date →Sun, 26 Apr 2020 19:22:02 GMT
Content-Type →application/json
Content-Length →23
Connection →keep-alive
x-amzn-RequestId →01d6b9ec-dcf0-484c-be07-6b629437b305
x-amzn-ErrorType →ForbiddenException
x-amz-apigw-id →Lm_WOF9ZvHcF7nQ=
Testing it directly from the AWS Lambda console works correctly and cloudwatch logs are generated, but not when I curl/postman the request with the API url that is generated during deployment. I have tried the following:
Ensuring the x-api-key header is set correctly and verifying that the API Gateway in AWS console is set with the correct API Key
Configuring CORS in the API in globals of template. Confirming It creates the options endpoints in API Gateway console
Double checking the endpoints are correct
The error states that it is a cloudfront issue so I've confirmed that the S3 bucket has public access. There are no other cloudfront resources in the AWS console. I'm at a loss as to what is blocking the request.
Answer was simpler than I thought, but for anyone else that comes across this issue, the query parameter is case sensitive. The output url from the serverless application model deployment returns https://${serverlessAppId}.execute-api.${region}.amazonaws.com/${StageName}.
In my case, the StageName was Prod and I was making requests as prod

CloudFormation stack Error - Error occurred while GetObject. S3 Error Code: NoSuchKey

I am trying to create a stack on AWS using CloudFormation. The following my cloud Formation script.
I have create the S3 bucket in us-west (Oregon) region. I have been running the cloudformation script in the same region.
During the creation process, I get the following error. Could you please help me how can I address this error?
CloudFormation Script
AWSTemplateFormatVersion: 2010-09-09
Description: The CloudFormation template for AWS resources required by amazon rekognition video analyzer.
Parameters:
SourceS3BucketParameter:
Type: String
MinLength: "1"
Description: "Enter the name of the S3 bucket containing source .zip files."
ImageProcessorSourceS3KeyParameter:
Type: String
MinLength: "1"
Description: "Enter the name of the S3 key of Image Processor lambda function .zip file."
FrameFetcherSourceS3KeyParameter:
Type: String
MinLength: "1"
Description: "Enter the name of the S3 key of Frame Fetcher lambda function .zip file."
FrameFetcherLambdaFunctionName:
Type: String
Default: "framefetcher"
Description: "Name of the Lambda function that fetches frame metadata from DynamoDB."
ImageProcessorLambdaFunctionName:
Type: String
Default: "imageprocessor"
Description: "Name of the Lambda function that receives and processes frame images."
FrameFetcherApiResourcePathPart:
Type: String
Default: "enrichedframe"
Description: "Path part for the API Gateway resource to access FrameFetcher lambda function."
KinesisStreamNameParameter:
Type: String
Default: "FrameStream"
Description: "Name of the Kinesis stream to receive frames from video capture client."
FrameS3BucketNameParameter:
Type: String
MinLength: "1"
Description: "Name of the S3 bucket for storage of captured frames."
DDBTableNameParameter:
Type: String
Default: "EnrichedFrame"
Description: "Name of the DynamoDB table for persistence & querying of captured frames metadata."
DDBGlobalSecondaryIndexNameParameter:
Type: String
Default: "processed_year_month-processed_timestamp-index"
Description: "Name of the DDB Global Secondary Index for querying of captured frames by Web UI."
ApiGatewayRestApiNameParameter:
Type: String
Default: "RtRekogRestApi"
Description: "Name of the API Gateway Rest API."
ApiGatewayStageNameParameter:
Type: String
Default: "development"
Description: "Name of the API Gateway stage."
ApiGatewayUsagePlanNameParameter:
Type: String
Default: "development-plan"
Description: "Name of the API Gateway Usage Plan."
Resources:
FrameS3Bucket:
Type: "AWS::S3::Bucket"
Properties:
BucketName: !Ref FrameS3BucketNameParameter
ImageProcessorLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonKinesisReadOnlyAccess
- arn:aws:iam::aws:policy/AmazonRekognitionReadOnlyAccess
- arn:aws:iam::aws:policy/AmazonS3FullAccess
- arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
- arn:aws:iam::aws:policy/AmazonSNSFullAccess
- arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
Path: "/"
DependsOn:
- FrameS3Bucket
- EnrichedFrameTable
FrameFetcherLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
- arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
- arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
Path: "/"
DependsOn:
- FrameS3Bucket
- EnrichedFrameTable
FrameStream:
Type: "AWS::Kinesis::Stream"
Properties:
Name: !Ref KinesisStreamNameParameter
ShardCount: 1
ImageProcessorLambda:
Type: AWS::Lambda::Function
Properties:
FunctionName: "imageprocessor"
Description: "Function processes frame images fetched from a Kinesis stream."
Handler: "imageprocessor.handler"
Role: !GetAtt ImageProcessorLambdaExecutionRole.Arn
Code:
S3Bucket: !Ref SourceS3BucketParameter
S3Key: !Ref ImageProcessorSourceS3KeyParameter
Timeout: 40 #seconds
MemorySize: 128 #MB
Runtime: python2.7
DependsOn:
- FrameStream
- ImageProcessorLambdaExecutionRole
EventSourceMapping:
Type: "AWS::Lambda::EventSourceMapping"
Properties:
EventSourceArn: !GetAtt FrameStream.Arn
FunctionName: !GetAtt ImageProcessorLambda.Arn
StartingPosition: "TRIM_HORIZON"
DependsOn:
- FrameStream
- ImageProcessorLambda
FrameFetcherLambda:
Type: AWS::Lambda::Function
Properties:
FunctionName: "framefetcher"
Description: "Function responds to a GET request by returning a list of frames up to a certain fetch horizon."
Handler: "framefetcher.handler"
Role: !GetAtt FrameFetcherLambdaExecutionRole.Arn
Code:
S3Bucket: !Ref SourceS3BucketParameter
S3Key: !Ref FrameFetcherSourceS3KeyParameter
Timeout: 10 #seconds
MemorySize: 128 #MB
Runtime: python2.7
DependsOn:
- FrameFetcherLambdaExecutionRole
EnrichedFrameTable:
Type: "AWS::DynamoDB::Table"
Properties:
TableName: !Ref DDBTableNameParameter
KeySchema:
- KeyType: "HASH"
AttributeName: "frame_id"
AttributeDefinitions:
- AttributeName: "frame_id"
AttributeType: "S"
- AttributeName: "processed_timestamp"
AttributeType: "N"
- AttributeName: "processed_year_month"
AttributeType: "S"
ProvisionedThroughput:
WriteCapacityUnits: 10
ReadCapacityUnits: 10
GlobalSecondaryIndexes:
- IndexName: !Ref DDBGlobalSecondaryIndexNameParameter
Projection:
ProjectionType: "ALL"
ProvisionedThroughput:
WriteCapacityUnits: 10
ReadCapacityUnits: 10
KeySchema:
- KeyType: "HASH"
AttributeName: "processed_year_month"
- KeyType: "RANGE"
AttributeName: "processed_timestamp"
# API Gateway Resources
VidAnalyzerRestApi:
Type: "AWS::ApiGateway::RestApi"
Properties:
Description: "The amazon rekognition video analyzer public API."
Name: !Ref ApiGatewayRestApiNameParameter
DependsOn: FrameFetcherLambda
EnrichedFrameResource:
Type: "AWS::ApiGateway::Resource"
Properties:
RestApiId: !Ref VidAnalyzerRestApi
ParentId: !GetAtt VidAnalyzerRestApi.RootResourceId
PathPart: !Ref FrameFetcherApiResourcePathPart
EnrichedFrameResourceGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref VidAnalyzerRestApi
ResourceId: !Ref EnrichedFrameResource
ApiKeyRequired: true
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${FrameFetcherLambda.Arn}/invocations
MethodResponses:
- ResponseModels:
application/json: Empty
StatusCode: 200
ResponseParameters:
"method.response.header.Access-Control-Allow-Origin": true
"method.response.header.Access-Control-Allow-Methods": true
"method.response.header.Access-Control-Allow-Headers": true
# Mock integration to allow Cross-Origin Resource Sharing (CORS)
# for Web UI to invoke API Gateway
EnrichedFrameResourceOPTIONS:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref VidAnalyzerRestApi
ResourceId: !Ref EnrichedFrameResource
ApiKeyRequired: false
HttpMethod: OPTIONS
AuthorizationType: NONE
Integration:
Type: MOCK
IntegrationHttpMethod: OPTIONS
PassthroughBehavior: WHEN_NO_MATCH
RequestTemplates:
"application/json": '{"statusCode": 200 }'
IntegrationResponses:
- StatusCode: 200
ResponseParameters:
"method.response.header.Access-Control-Allow-Origin": "'*'"
"method.response.header.Access-Control-Allow-Methods": "'GET,OPTIONS'"
"method.response.header.Access-Control-Allow-Headers": "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
ResponseTemplates:
"application/json": ''
MethodResponses:
- ResponseModels:
application/json: Empty
StatusCode: 200
ResponseParameters:
"method.response.header.Access-Control-Allow-Origin": true
"method.response.header.Access-Control-Allow-Methods": true
"method.response.header.Access-Control-Allow-Headers": true
VidAnalyzerApiDeployment:
Type: "AWS::ApiGateway::Deployment"
Properties:
Description: "Public API endpoint of video analyzer."
RestApiId: !Ref VidAnalyzerRestApi
DependsOn:
- EnrichedFrameResourceGET
- EnrichedFrameResourceOPTIONS
DevStage:
Type: "AWS::ApiGateway::Stage"
Properties:
DeploymentId: !Ref VidAnalyzerApiDeployment
Description: "API development stage of video analyzer."
RestApiId: !Ref VidAnalyzerRestApi
StageName: !Ref ApiGatewayStageNameParameter
DevUsagePlan:
Type: AWS::ApiGateway::UsagePlan
Properties:
ApiStages:
- ApiId: !Ref VidAnalyzerRestApi
Stage: !Ref DevStage
Description: Development usage plan
UsagePlanName: !Ref ApiGatewayUsagePlanNameParameter
DeletionPolicy: Retain #Had to be added to avoid stack deletion failing due to association with DevStage.
VidAnalyzerApiKey:
Type: "AWS::ApiGateway::ApiKey"
Properties:
Name: "DevApiKey"
Description: "Video Analyzer Dev API Key"
Enabled: true
StageKeys:
- RestApiId: !Ref VidAnalyzerRestApi
StageName: !Ref ApiGatewayStageNameParameter
DependsOn:
- VidAnalyzerApiDeployment
- DevStage
DevUsagePlanKey:
Type: "AWS::ApiGateway::UsagePlanKey"
Properties :
KeyId: !Ref VidAnalyzerApiKey
KeyType: API_KEY
UsagePlanId: !Ref DevUsagePlan
#Give API Gateway permission to invoke FrameFetcher lambda function.
LambdaInvokePermissionSTAR:
Type: "AWS::Lambda::Permission"
Properties:
FunctionName: !GetAtt FrameFetcherLambda.Arn
Action: "lambda:InvokeFunction"
Principal: "apigateway.amazonaws.com"
SourceArn: !Join [ "", ["arn:aws:execute-api:", !Ref "AWS::Region", ':', !Ref "AWS::AccountId", ':', !Ref VidAnalyzerRestApi, '/*/*/', !Ref FrameFetcherLambdaFunctionName]]
DependsOn:
- VidAnalyzerApiDeployment
LambdaInvokePermissionGET:
Type: "AWS::Lambda::Permission"
Properties:
FunctionName: !GetAtt FrameFetcherLambda.Arn
Action: "lambda:InvokeFunction"
Principal: "apigateway.amazonaws.com"
SourceArn: !Join [ "", ["arn:aws:execute-api:", !Ref "AWS::Region", ':', !Ref "AWS::AccountId", ':', !Ref VidAnalyzerRestApi, '/*/GET/', !Ref FrameFetcherApiResourcePathPart]]
DependsOn:
- VidAnalyzerApiDeployment
Outputs:
#API Gateway endpoint Id
VidAnalyzerApiEndpoint:
Description: "Endpoint for invoking video analyzer API."
Value: !Ref VidAnalyzerApiDeployment
#API Key Id
VidAnalyzerApiKey:
Description: "Key for invoking video analyzer API."
Value: !Ref VidAnalyzerApiKey
Error:
The following are the parameters, I have set during the cloudFormation process.
Stack name: streamframerekognition
ApiGatewayRestApiNameParameter : RtRekogRestApi
ApiGatewayStageNameParameter : development
ApiGatewayUsagePlanNameParameter : development-plan
DDBGlobalSecondaryIndexNameParameter : processed_year_month-processed_timestamp-index
DDBTableNameParameter : EnrichedFrame
FrameFetcherApiResourcePathPart : enrichedframe
FrameFetcherLambdaFunctionName : framefetcher
ImageProcessorLambdaFunctionName: imageprocessor
ImageProcessorSourceS3KeyParameter : imagestreamframerekognition
KinesisStreamNameParameter : FrameStream
SourceS3BucketParameter : sourcesstreamframerekognition
FrameFetcherSourceS3KeyParameter : ffstreamframerekognition
FrameS3BucketNameParameter : framesbnpstreamframerekognition
It appears that the object key you are providing as ImageProcessorSourceS3KeyParameter is not a valid key name, or does not exist. Check that a file with that name exists in your S3 bucket.
In the S3 bucket "sourcesstreamframerekognition" you need to have a zip file with your code (S3 Key) for FrameFetcherSourceS3KeyParameter and ImageProcessorSourceS3KeyParameter.

Create an API Key and Usage Plan with SAM

I am learning AWS SAM and having trouble finding information on how to create an API Key and usage plan through the SAM template.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
GetFamilyByIdFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: nodejs8.10
Handler: get-family-by-id.handler
CodeUri: get-family-by-id/
Timeout: 300
Events:
GetFamilyByIdApi:
Type: Api
Properties:
Path: "/family/{id}"
Method: get
I would like to create an API key and associate it with a usage plan for the 'GetFamilyByIdApi' specified above. Any help would be appreciated.
After a bit of digging I figured it out. This solution is when you want to define the value of the api key yourself as opposed to letting API Gateway generate a value. I assume a variation exists for that. Note that 'HvgnApi' refers to my Swagger definition (Type: AWS::Serverless::Api). Enjoy!
ApiKey:
Type: AWS::ApiGateway::ApiKey
Properties:
Name: !Join ["", [{"Ref": "AWS::StackName"}, "-apikey"]]
Description: "CloudFormation API Key V1"
Enabled: true
GenerateDistinctId: false
Value: abcdefg123456
StageKeys:
- RestApiId: !Ref HvgnApi
StageName: prod
ApiUsagePlan:
Type: "AWS::ApiGateway::UsagePlan"
Properties:
ApiStages:
- ApiId: !Ref HvgnApi
Stage: prod
Description: !Join [" ", [{"Ref": "AWS::StackName"}, "usage plan"]]
Quota:
Limit: 1000
Period: MONTH
UsagePlanName: !Join ["", [{"Ref": "AWS::StackName"}, "-usage-plan"]]
ApiUsagePlanKey:
Type: "AWS::ApiGateway::UsagePlanKey"
DependsOn:
- HvgnApi
Properties:
KeyId: !Ref ApiKey
KeyType: API_KEY
UsagePlanId: !Ref ApiUsagePlan
This is much easier to do with the latest version of SAM:
MyAPI:
Type: AWS::Serverless::Api
Properties:
StageName: dev
Auth:
ApiKeyRequired: true # for all methods
UsagePlan:
CreateUsagePlan: PER_API
Description: Usage plan for this API