I have AWS SAM template, part of which looks like this:
# .......
InternalApiKey:
Type: AWS::ApiGateway::ApiKey
Properties:
Enabled: true
Name: !Sub internal_api_key_${Env}
Value: !Ref InternalApiKeyValue
StageKeys:
- RestApiId: !Ref ServerlessRestApi
StageName: Prod
InternalUsagePlan:
Type: AWS::ApiGateway::UsagePlan
Properties:
ApiStages:
- ApiId: !Ref ServerlessRestApi
Stage: Prod
InternalUsagePlanKey:
Type: AWS::ApiGateway::UsagePlanKey
Properties:
KeyId: !Ref InternalApiKey
KeyType: API_KEY
UsagePlanId: !Ref InternalUsagePlan
#......
Deploying this template as two distinct application (CloudFormation stack) with sam deploy --stack-name=stack-a and sam deploy --stack-name=stack-b fails with the following error even when api key names are different:
API Key already exists (Service: AmazonApiGateway; Status Code: 409; Error Code: ConflictException; Request ID:
redacted; Proxy:
null)
How can I deploy two stacks from this template?
A different key value and your problem will go away. So if you are setting the api key value yourself ensure they are unique. Not sure why that's a constraint between different stages.
Do you really have to provide a value for your key? It's an optional parameter. You can remove it from your template and AWS will generate a unique key for you in each deployment.
Related
I have the following parameter in my CloudFormation script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
This parameter covers the CloudFormation scripts in a nested folder of my deployment config.
I use it with a resource like this:
CloudWatchDashboardStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: !Sub "${CloudFormationURL}/cloudwatch-dashboard.cfn.yaml"
Parameters:
AppName: !Ref AppName
DeployPhase: !Ref DeployPhase
DeveloperPrefix: !Ref DeveloperPrefix
Environment: !Ref Environment
Which works fine, and has worked for months.
I needed to add another resource, so I added this:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
CloudFormation throws this error and then fails:
Parameters: [CloudFormationURL] must have values
Checking the changeset for the stack I can see the following value for the CloudFormationURL:
s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS
Checking the application predeploy logs I can see:
upload: deploy/cloudformation/templates/nested/batch-dns.cfn.yaml to s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS/batch-dns.cfn.yaml
And I can see the file in the S3 bucket.
If I remove BatchDNSResource the stack completes successfully.
What the heck am I missing here?
Sometimes, the smallest things will get you.
I had copied the Parameters from the master CloudFormation script, including this one, into the nested script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
If you look closely, you will see that I did not pass the parameter into the nested script when calling the resource:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
Because the CloudFormation console was saying the issue was with the BatchDNSResources I kept looking at the master script for the problem and missing the reference in the other script. There are two ways to solve this problem:
Keep CloudFormationURL as a parameter in the nested script (if you need it for some reason) and pass the value from the master script.
Remove the parameter from the nested script (if it is not needed)
Sometimes just asking for an extra set of eyeballs and getting a little rest will help you to find the issues. I want to leave this question/answer in place here because when I was searching for the error here and elsewhere no one ever mentioned (probably out of embarrassment) that the mistake is simply overlooking something like this. I hope this answer prompts others to check everything when they run across this type of error.
I have attached the sample to give you some clarity of what i am trying to solve.
AWSTemplateFormatVersion: '2010-09-09'
Description: Project Service Catalog get lambda data
Parameters:
Environment:
Type: String
Description: Environment of the SageMaker
ProjectId:
Type: String
Description: Project ID of the SageMaker
SsmRoleLambdaArn:
Type: AWS::SSM::Parameter::Value<String>
Default: '/data-science/role-lambda/arn'
Description: Arn to lookup Role of the Session using project id
Resource:
IdentifyUserRole:
Type: Custom::GetParam
Properties:
ServiceToken: !Ref SsmRoleLambdaArn
pl_role: !Sub '${Environment}-sso-data-science-${ProjectId}-pl-role'
ds_role: !Sub '${Environment}-sso-data-science-${ProjectId}-ds-role'
KmsKey:
Type: AWS::KMS::Key
Properties:
Description: !Sub 'Encryption for ${Environment}-${ProjectId}-${Prid}-${NotebookInstanceNameSuffix}'
EnableKeyRotation: true
Tags:
- Key: Environment
Value: !Ref Environment
- Key: Owner
Value: !Ref Owner
- Key: ProjectId
Value: !Ref ProjectId
- Key: PrincipalId
Value: !Sub
- "${RoleId}:${Prid}"
- RoleId:
Fn::If: [!Equals [!GetAtt IdentifyUserRole.value, true], !GetAtt PORoleId.value, !GetAtt DSRoleId.value]
I am getting error at the IF condition in the PrincipalID tag. Please help solve this condition with some sample templates. I can't use !GetAtt in the Conditions block as well because we are not supposed to use get attributes.
Error Message - During stack validation
An error occurred (ValidationError) when calling the ValidateTemplate operation: Template error: Fn::If requires a list argument with the first element being a condition
You can't hard code the condition in the If like you are attempting:
Fn::If: [!Equals [!GetAtt IdentifyUserRole.value, true], !GetAtt PORoleId.value, !GetAtt DSRoleId.value]
The first argument must be condition from Conditions section (docs):
reference to a condition in the Conditions section.
Subsequently, you can't construct conditions based on GetAtt or any other resources from Resources section.
The same docs also write:
You can only reference other conditions and values from the Parameters and Mappings sections of a template. For example, you can reference a value from an input parameter, but you cannot reference the logical ID of a resource in a condition.
I'm using serverless version 1.29.2
I have a created an initial cloudformation script that creates an API GateWay REST API that will be used by other services. So Here is the cloudformation script responsible for it.
{
"AWSTemplateFormatVersion":"2010-09-09",
"Description":"API",
"Resources":{
"APIGw":{
"Type":"AWS::ApiGateway::RestApi",
"Properties":{
"Name":"API-GW"
}
}
},
"Outputs":{
"ApiGwRestApiId":{
"Value":{
"Ref":"APIGw"
},
"Export":{
"Name":"apigw-restApiId"
}
},
"eyesApiGwRestApiRootResourceId":{
"Value":{
"Fn::GetAtt":[
"APIGw",
"RootResourceId"
]
},
"Export":{
"Name":"apigw-rootResourceId"
}
}
}
}
Here is serverless.yml for the application I was trying to deploy.
service: template-test-service
provider:
name: aws
runtime: python3.6
region: eu-central-1
stage: ${self:custom.environment.stage}
environment:
stage: ${self:custom.environment.stage}
apiGateway:
restApiId:
'Fn::ImportValue': apigw-restApiId
restApiRootResourceId:
'Fn::ImportValue': apigw-rootResourceId
When I perform an sls deploy --stage dev everything works fine, However when I perform another deploy to sls deploy --stage prod
This error shows up.
Another resource with the same parent already has this name
I've struggled with this one for a week now and the issue is to do with the way API Gateway is constructred out of resources and methods. From the documentation
In Amazon API Gateway, you build a REST API as a collection of programmable entities known as API Gateway resources. For example, you use a RestApi resource to represent an API that can contain a collection of Resource entities. Each Resource entity can in turn have one or more Method resources. Expressed in the request parameters and body, a Method defines the application programming interface for the client to access the exposed Resource and represents an incoming request submitted by the client.
Serverless CLI creates all the resource/methods for you when you have a function trigged by an http event.
functions:
GetScenesInGame:
handler: handler.hello
layers: arn:aws:lambda:eu-west-1:xxxxxxxxx:layer:pynamodb-layer:1
events:
- http:
method: GET
path: api/v1/game/{gameId}/scene
From the example above this creates five resources ( api, v1, game, gameIdParam, scene) and finally adds a GET method on the the final resource.
Unfortunately when you have two seperate stacks (as you might in a microservice setup) if they are any part of the above methods then it errors with Another resource with the same parent already has this name
The solution is highlighted in this article from serverless deploy serverless microservice on aws although its not very explict and easily missed.
Firstly there is a top level cloudformation template which configures the required resources.
For the resource you want to add a serverless microservice to you export the id of the resource as an output variable in your stack.
Then in the serverless.yml file you import the api gateway reference and apigateway resource id.
You can then deploy each service without getting a clash of resource names in the api structure.
The templates below show having a top level set of resources.
api/v1/game/{gameId}/page
api/v1/game/{gameId}/scene
And then attaching PageService to the page resource and SceneService to the scene resource.
api-gateway.yml
AWSTemplateFormatVersion: "2010-09-09"
Description: "S3 template for deploying S3 to be used by ACS s3 connector."
Resources:
TestApiGw:
Type: AWS::ApiGateway::RestApi
Properties:
Name: !Sub 'test-apigw-throwaway'
ApiResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !GetAtt
- TestApiGw
- RootResourceId
PathPart: "api"
VersionResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !Ref ApiResource
PathPart: "v1"
GameResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !Ref VersionResource
PathPart: "game"
GameParamResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !Ref GameResource
PathPart: "{gameId}"
SceneResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !Ref GameParamResource
PathPart: "scene"
PageResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref TestApiGw
ParentId: !Ref GameParamResource
PathPart: "page"
Outputs:
ApiRestApiId:
Value: !Ref TestApiGw
Export:
Name: !Sub ${AWS::StackName}-TestApiId
ApiRootResourceId:
Value:
Fn::GetAtt:
- TestApiGw
- RootResourceId
Export:
Name: !Sub ${AWS::StackName}-ApiRootResourceVar
ApiSceneResourceVar:
Value: !Ref SceneResource
Export:
# variable names are global so this will need more work to make it unique across stages.
Name: !Sub ${AWS::StackName}-ApiSceneResourceVar
ApiPageResourceVar:
Value: !Ref PageResource
Export:
# variable names are global so this will need more work to make it unique across stages.
Name: !Sub ${AWS::StackName}-ApiPageResourceVar
serverless.yml ( Serverless Cli file for Page Service)
service: scrap-page-service
provider:
name: aws
runtime: python2.7
apiGateway:
restApiId:
"Fn::ImportValue": throw-stack-1-TestApiId
restApiRootResourceId:
"Fn::ImportValue": throw-stack-1-ApiPageResourceVar
functions:
hello:
handler: handler.hello
events:
- http:
path: ""
method: get
serverless.yml ( Serverless Cli file for Scene Service)
service: scrap-scene-service
provider:
name: aws
runtime: python2.7
apiGateway:
restApiId:
"Fn::ImportValue": throw-stack-1-TestApiId
restApiRootResourceId:
"Fn::ImportValue": throw-stack-1-ApiSceneResourceVar
functions:
hello:
handler: handler.hello
events:
- http:
path: ""
method: get
Hopefully this helps others getting this issue and if somebody has a better way of doing it I'd be keen to know :-)
Did take a look at Serverless documentation on sharing API Gateway ?
It seems that you have to create common resource path components are CloudFormed objects
I create CloudFormation template for my AWS API Gateway and Lambda function and I need to connect specific API Gateway stage to specific Lambda alias. I have two aliases - QA and Prod, and two API stages (QA & Prod too), in CloudFormation template it looks like:
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Lambda function configuration
Resources:
EndpointLambda:
Type: "AWS::Lambda::Function"
Properties:
FunctionName: "endpoint-lambda"
Handler: "com.test.aws.RequestHandler::handleRequest"
Runtime: java8
Code:
S3Bucket: "lambda-functions"
S3Key: "test-endpoint-lambda-0.0.1.jar"
Description: Test Lambda function
MemorySize: 256
Timeout: 60
Environment:
Variables:
ES_HOST: test-es-host-url
ES_ON: true
ES_PORT: 443
ES_PROTOCOL: https
REDIS_URL: test-redis-host-url
QaLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "QA"
Description: "QA alias"
ProdLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "Prod"
Description: "Production alias"
RestApi:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "test-rest-api"
Description: "Test REST API"
RestApiResource:
Type: "AWS::ApiGateway::Resource"
Properties:
RestApiId: !Ref "RestApi"
ParentId: !GetAtt "RestApi.RootResourceId"
PathPart: "/test"
RestApiDeployment:
Type: "AWS::ApiGateway::Deployment"
Properties:
RestApiId: !Ref "RestApi"
QaRestApiStage:
Type: "AWS::ApiGateway::Stage"
Properties:
DeploymentId: !Ref "RestApiDeployment"
RestApiId: !Ref "RestApi"
StageName: "qa"
ProdRestApiStage:
Type: "AWS::ApiGateway::Stage"
Properties:
DeploymentId: !Ref "RestApiDeployment"
RestApiId: !Ref "RestApi"
StageName: "prod"
How can I describe in template that QA API stage should call QA alias of Lambda function, and Prod stage - Prod alias?
To begin with find out how to do it using the GUI. There some documentation about what you want to do here. Theres some extra permissions you'll need to add aswell if this is the first time you've set this up which are included here -
https://docs.aws.amazon.com/apigateway/latest/developerguide/stage-variables.html
But for a quick answer what your looking for is $:{stageVariables.stage} what this does is links the alias of the lambda you want to trigger. In the GUI it'd look something like this:
What this will do is allow your lambda to trigger a certain alias. Once this is entered you'll be able to see a new option when using the Testing feature in the API gateway. So here you'd specify QA.
So, to reflect this in Cloudformation we need to do something similar -
RestApi:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "test-rest-api"
Description: "Test REST API"
paths:
/ExamplePath:
put:
#Here will go all the configuration setting you want
#Such as security, httpMethod, amd passthroughBehavior
#But what you need is
uri: 'arn:aws:apigateway:${AWS:Region}:lambda:path/2-15/03/31/functions/${LambdaARN}:${!stageVariables.stage}/invocations'
More info on this can be found here:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-apitgateway-method-integration.html what you'll want to see is at the very bottom of the page.
Hope this helps!
just like in the title. I can deploy stuff on AWS using only Cloud Formation. Now I try to secure my API Gateway with API Keys and looks like I need a Usage Plan for it. It doesn't seem to be covered by the documentation right here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-apikey.html
Have any of you had a similar problem and if yes, how did you solved it?
AWS has today released the ability to create AWS::ApiGateway::UsagePlan using cloud formation templates
Now AWS::ApiGateway::UsagePlanKey can be created using CloudFormation.
This snippet demonstrates how you might use a UsagePlan and UsagePlanKey, along with an APIKey in a CloudFormation Template
UsagePlan:
Type: 'AWS::ApiGateway::UsagePlan'
Properties:
ApiStages:
- ApiId: !Ref MyRestApi
Stage: !Ref Prod
Description: Customer ABCs usage plan
Quota:
Limit: 5000
Period: MONTH
Throttle:
BurstLimit: 200
RateLimit: 100
UsagePlanName: Plan_ABC
ApiKey:
Type: 'AWS::ApiGateway::ApiKey'
Properties:
Name: TestApiKey
Description: CloudFormation API Key V1
Enabled: 'true'
UsagePlanKey:
Type: 'AWS::ApiGateway::UsagePlanKey'
Properties:
KeyId: !Ref ApiKey
KeyType: API_KEY
UsagePlanId: !Ref UsagePlan
For anyone reading, this is now supported via the AWS::ApiGateway::UsagePlanKey (docs) resource type in CloudFormation. From that page:
The AWS::ApiGateway::UsagePlanKey resource associates an Amazon API Gateway API key with an API Gateway usage plan. This association determines which users the usage plan is applied to.
AWS has provided the CloudFormation template for the API Keys creation with the UsagePlan and the UsagePlan Keys as well, so probably the CFT for the same would be defined as:
ApiKey:
Type: 'AWS::ApiGateway::ApiKey'
DependsOn:
- ApiGatewayDeployment
Properties:
Name: !Sub "you keyName-Apikeys"
Description: Api Keys Description
Enabled: 'true'
StageKeys:
- RestApiId: !Ref ApiGatewayRestApi
StageName: !Sub "your stageName"
usagePlan:
Type: 'AWS::ApiGateway::UsagePlan'
DependsOn:
- ApiGatewayDeployment
Properties:
ApiStages:
- ApiId: !Ref ApiGatewayRestApi
Stage: !Sub "your stageName"
Description: your description of usage plan
Quota:
Limit: 50000
Period: DAY
Throttle:
BurstLimit: 200
RateLimit: 100
UsagePlanName: !Sub "define your name of UsagePlan"
usagePlanKey:
Type: 'AWS::ApiGateway::UsagePlanKey'
DependsOn:
- ApiGatewayDeployment
Properties:
KeyId: !Ref ApiKey
KeyType: API_KEY
UsagePlanId: !Ref usagePlan
I hope this will probably help you.