In our project we are using serverless for AWS. After added serverless-domain-manager plugin we are experiencing strange issue - Error: Unsupported apiGateway.restApiId object
custom:
customDomain:
domainName: test.test.com
stage: ${self:provider.stage}
basePath: plugin
certificateName: test.test.com
createRoute53Record: true
createRoute53IPv6Record: false
endpointType: REGIONAL
securityPolicy: tls_1_2
apiType: rest
autoDomain: true
preserveExternalPathMappings: true
provider:
name: aws
region: eu-central-1
stage: prod
runtime: nodejs16.x
logRetentionInDays: 7
apiGateway:
restApiId: !Ref EmployeeApiGateway
restApiRootResourceId:
Fn::GetAtt:
- EmployeeApiGateway
- RootResourceId
We try to add domain-manager mapping
Related
i'm building a webapp for which i need to utilize AWS resources I am able to create rest of the resources using serverless.yml file using cloudformation template but unable to create Opensearch resource, where am I going wrong ?
This is my serverless.yml file
service: localstack-lambda
plugins:
# - serverless-plugin-warmup
- serverless-localstack
custom:
localstack:
debug: true
stages:
- local
- dev
endpointFile: localstack_endpoints.json
# frameworkVersion: "2"
provider:
name: aws
runtime: nodejs12.x
#functions are pretty important
functions:
uploadFiles:
handler: s3handler.uploadFiles
events:
- http:
path: uploadFiles
method: any
listFiles:
handler: s3handler.listFiles
events:
- http:
path: listFiles
method: any
osHandler:
handler: OpenSearchHandler.osHandler
events:
- http:
path: osHandler
method: any
resources: # CloudFormation template syntax from here on
Resources:
S3Bucket:
Type: "AWS::S3::Bucket"
DeletionPolicy: Retain
Properties:
BucketName: testbucket
OpenSearchServiceDomain:
Type: AWS::OpenSearchService::Domain
Properties:
DomainName: "myopensearch_1"
EngineVersion: "OpenSearch_1.0"
ClusterConfig:
DedicatedMasterEnabled: true
InstanceCount: "2"
ZoneAwarenessEnabled: true
InstanceType: "m3.medium.search"
DedicatedMasterType: "m3.medium.search"
DedicatedMasterCount: "3"
EBSOptions:
EBSEnabled: true
Iops: "0"
VolumeSize: "20"
VolumeType: "gp2"
thank you for your help in advanced :)
I have two AWS Lambda functions. I have 3 stacks dev, test, and PROD.
I want a deploy a specific the Lambda function to only dev and test but not prod.
I want the trial Lambda function to be only in test and dev stages but not in PROD stage.
How can I achieve that? Here is my serverless.yml:
service:
name: demo-app
# Add the serverless-webpack plugin
plugins:
- serverless-webpack
- serverless-offline
provider:
name: aws
runtime: nodejs12.x
timeout: 30
stage: dev
region: us-west-2
profile: serverless-admin
custom:
region: ${self:provider.region}
stage: ${opt:stage, self:provider.stage}
prefix: ${self:service}-${self:custom.stage}
webpack:
webpackConfig: ./webpack.config.js
includeModules: true
functions:
toggle:
handler: src/functions/unleash-toggle/handler.main
timeout: 900
events:
- http:
path: /toggle
method: POST
trial:
handler: src/functions/city/handler.main
timeout: 900
events:
- http:
path: /trial
method: POST
resources:
Resources:
taskTokenTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:service}-${self:custom.stage}-tokenTable
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
After doing some research I found out this can be done.
This can be done using serverless plugin: serverless-plugin-ifelse and defining your conditions under custom block.
you can see the same on Serverless Docs.
The plugin is available on npm
custom:
serverlessIfElse:
- If: '"${self:custom.stage}" == "prod"'
Exclude:
- functions.<functionName>
Complete serverless.yml file
service:
name: demo-app
# Add the serverless-webpack plugin
plugins:
- serverless-webpack
- serverless-plugin-ifelse
- serverless-offline
provider:
name: aws
runtime: nodejs12.x
timeout: 30
stage: dev
region: us-west-2
profile: serverless-admin
custom:
region: ${self:provider.region}
stage: ${opt:stage, self:provider.stage}
prefix: ${self:service}-${self:custom.stage}
webpack:
webpackConfig: ./webpack.config.js
includeModules: true
serverlessIfElse:
- If: '"${self:custom.stage}" == "prod"'
Exclude:
- functions.trail
functions:
toggle:
handler: src/functions/unleash-toggle/handler.main
timeout: 900
events:
- http:
path: /toggle
method: POST
trial:
handler: src/functions/city/handler.main
timeout: 900
events:
- http:
path: /trial
method: POST
The same thing can be achieved by another plugin serverless-plugin-conditional-functions
I want to create a VPC with a DBCluster in it using the serverless-vpc-plugin. If I do it in two steps, first the VPC, and then the cluster, everything works. But if I do it simutaneously serverless fails, complaining that the DBSubnetGroup has not been created yet.
I tried makind the DBCluster DependsOn: VPC but nothing. Here are the relevant parts:
service: vpn
frameworkVersion: '2'
custom:
stage: ${opt:stage, self:provider.stage}
region: ${opt:region, self:provider.region}
vpcConfig:
createNatGateway: 1
createNetworkAcl: true
subnetGroups:
- rds
provider:
name: aws
runtime: nodejs14.x
lambdaHashingVersion: 20201221
resources:
Resources:
ClusterSecret:
Type: AWS::SecretsManager::Secret
Properties:
[...]
AuroraDBCluster:
Type: AWS::RDS::DBCluster
DependsOn: VPC
Properties:
DatabaseName: [...]
DBClusterIdentifier: [...]
DBSubnetGroupName: ${self:service}-${self:custom.stage}
Engine: aurora-postgresql
EngineMode: serverless
EngineVersion: "10.14"
MasterUsername: [...]
MasterUserPassword: [...]
plugins:
- serverless-vpc-plugin
- serverless-offline
DependsOn: RDSSubnetGroup instead of DependsOn: VPC did the job
I have created a Web ACL in my AWS account in the AWS WAF and assigned it some rules. In my serverless.yml file I have used the - serverless-associate-waf plugin.
But when I go to my Web ACLs > my acl > Associated AWS Resources, I do not see the associated API Gateway listed there.
Here is how my serverless.yml file looks:
service: ${opt:product}
plugins:
- serverless-domain-manager
- serverless-apigw-binary
- serverless-associate-waf
custom:
associateWaf:
name: name-of-my-acl
esLogs:
endpoint: link.amazonaws.com
index: "${opt:stage}-logs"
includeApiGWLogs: true
retentionInDays: 30
stage: ${opt:stage, 'dev'}
region: ${opt:region, 'ap-south-1'}
accountId: ${opt:accountId}
awsBucket: ${opt:awsBucket, 'documents'}
awsPermaBucket: ${opt:awsPermaBucket, 'perma-documents-dev'}
cryptoKey: ${opt:cryptoKey}
apigwBinary:
types:
- 'multipart/form-data'
customDomain:
domainName: ${opt:stage}-${opt:product}-api.io
basePath: ""
stage: ${self:custom.stage}
createRoute53Record: true
provider:
vpc:
securityGroupIds:
- sg-1234
subnetIds:
- subnet-1234
- subnet-1234
environment:
region: ${self:custom.region}
stage: ${self:custom.stage}
module: ${opt:product}
awsBucket: ${self:custom.awsBucket}
authToken: ${opt:authToken}
accountId: ${opt:accountId}
awsPermaBucket: ${self:custom.awsPermaBucket}
cryptoKey: ${opt:cryptoKey}
iamRoleStatements:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- logs:DescribeLogStreams
Resource: "*"
- Effect: Allow
Action:
- s3:*
Resource: "*"
- Effect: "Allow"
Action:
- "sqs:*"
Resource: "arn:aws:sqs:${opt:region}:*:${opt:stage}-${opt:product}-sqs-queue"
name: aws
runtime: nodejs12.x
stage: ${self:custom.stage}
region: ${self:custom.region}
memorySize: 256
timeout: 30
package:
exclude:
- "*/**"
include:
- build/**
- node_modules/**
functions:
orgSettingsAPI:
name: ${self:service}-${self:custom.stage}-api
handler: build/src/lambda.handler
events:
- http:
method: any
path: /api/{proxy+}
authorizer:
arn: arn:aws:lambda:${opt:region}:${self:custom.accountId}:function:authenticator-${self:custom.stage}-api
resultTtlInSeconds: 60
identitySource: method.request.header.Authorization
identityValidationExpression: ^Bearer.+
cors:
origins:
- "*"
headers:
- Content-Type
- X-Amz-Date
- Authorization
- X-Api-Key
- X-Amz-Security-Token
allowCredentials: true
maxAge: 86400
- http:
method: any
path: /internal/{proxy+}
vpc:
securityGroupIds:
- sg-1234
subnetIds:
- subnet-1234
- subnet-1234
environment:
SqsQueueName: ${opt:stage}-${opt:product}-sqs-queue
reservedConcurrency: 10
events:
- sqs:
arn:
Fn::GetAtt:
- SqsQueue
- Arn
batchSize: 1
resources:
Resources:
GatewayResponse:
Type: "AWS::ApiGateway::GatewayResponse"
Properties:
ResponseParameters:
gatewayresponse.header.Access-Control-Allow-Origin: "'*'"
gatewayresponse.header.Access-Control-Allow-Headers: "'*'"
ResponseType: EXPIRED_TOKEN
RestApiId:
Ref: "ApiGatewayRestApi"
StatusCode: "401"
AuthFailureGatewayResponse:
Type: "AWS::ApiGateway::GatewayResponse"
Properties:
ResponseParameters:
gatewayresponse.header.Access-Control-Allow-Origin: "'*'"
gatewayresponse.header.Access-Control-Allow-Headers: "'*'"
ResponseType: UNAUTHORIZED
RestApiId:
Ref: "ApiGatewayRestApi"
StatusCode: "401"
When I debugged the deployment process it says:
Serverless: Unable to find WAF named 'name-of-my-acl'. Am I naming it wrong or using it wrong?
I do not understand what is the name I should be using for my WAF in the serverless.yml file.
Found the issue, turns out I need to add
version: V2
just after name since AWS WAF supports V2. Once I added it and redeployed the API Gateway got attached to the created WAF.
PS: the name is the name of the ACL that we want to use.
What is working:
Using the serverless framework:
I have configured an AWS VPC
I have an Amazon Aurora database configured
for my VPC
I have an AWS API Gateway lambda that is configured for my
VPC
When I deploy my lambda, I am able to access it publicly via the AWS
generated URL: XXX.execute-api.us-east-1.amazonaws.com/prod/outages
In my Lambda,I run a very simple query that proves I can connect to
my database.
This all works fine.
What is NOT working:
I have registered a domain with AWS/Route 53 and added a cert (e.g. *.foo.com)
I use the serverless-domain-manager plugin to make my lambda available via my domain (e.g. api.foo.com/outages resolves to XXX.execute-api.us-east-1.amazonaws.com/prod/outages)
This works fine if my lambda is NOT configured for my VPC
But when my lambda IS configured for my VPC, the custom domain api.foo.com/outages does NOT resolve to XXX.execute-api.us-east-1.amazonaws.com/prod/outages
I other words: I can NOT access api.foo.com/outages publicly.
What I need is:
1 - XXX.execute-api.us-east-1.amazonaws.com/prod/outages is available publicly (this works)
2 - My custom domain, api.foo.com/outages points to the SAME lambda as XXX.execute-api.us-east-1.amazonaws.com/prod/outages (in my VPC) and is available publicly (not working. I get: {"message":"Forbidden"})
virtual-private-cloud.yml
service: virtual-private-cloud
provider:
name: aws
region: us-east-1
stage: ${opt:stage, dev}
custom:
appVersion: 0.0.0
VPC_CIDR: 10
resources:
Resources:
ServerlessVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: ${self:custom.VPC_CIDR}.0.0.0/16
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
ServerlessSubnetA:
DependsOn: ServerlessVPC
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: ${self:provider.region}a
CidrBlock: ${self:custom.VPC_CIDR}.0.0.0/24
ServerlessSubnetB:
DependsOn: ServerlessVPC
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: ${self:provider.region}b
CidrBlock: ${self:custom.VPC_CIDR}.0.1.0/24
ServerlessSubnetC:
DependsOn: ServerlessVPC
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: ${self:provider.region}c
CidrBlock: ${self:custom.VPC_CIDR}.0.2.0/24
Outputs:
VPCDefaultSecurityGroup:
Value:
Fn::GetAtt:
- ServerlessVPC
- DefaultSecurityGroup
Export:
Name: VPCDefaultSecurityGroup-${self:provider.stage}
SubnetA:
Description: 'Subnet A.'
Value: !Ref ServerlessSubnetA
Export:
Name: vpc-subnet-A-${self:provider.stage}
SubnetB:
Description: 'Subnet B.'
Value: !Ref ServerlessSubnetB
Export:
Name: vpc-subnet-B-${self:provider.stage}
SubnetC:
Description: 'Subnet C.'
Value: !Ref ServerlessSubnetC
Export:
Name: vpc-subnet-C-${self:provider.stage}
database-service.yml
service: database-service
provider:
name: aws
region: us-east-1
stage: ${opt:stage, dev}
environment:
stage: ${opt:stage, dev}
plugins:
- serverless-plugin-ifelse
custom:
appVersion: 0.0.1
AURORA:
DB_NAME: database${self:provider.stage}
USERNAME: ${ssm:/my-db-username~true}
PASSWORD: ${ssm:/my-db-password~true}
HOST:
Fn::GetAtt: [AuroraRDSCluster, Endpoint.Address]
PORT:
Fn::GetAtt: [AuroraRDSCluster, Endpoint.Port]
serverlessIfElse:
- If: '"${opt:stage}" == "prod"'
Set:
resources.Resources.AuroraRDSCluster.Properties.EngineMode: provisioned
ElseSet:
resources.Resources.AuroraRDSCluster.Properties.EngineMode: serverless
resources.Resources.AuroraRDSCluster.Properties.ScalingConfiguration.MinCapacity: 1
resources.Resources.AuroraRDSCluster.Properties.ScalingConfiguration.MaxCapacity: 4
ElseExclude:
- resources.Resources.AuroraRDSInstanceParameter
- resources.Resources.AuroraRDSInstance
resources:
Resources:
AuroraSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: "Aurora Subnet Group"
SubnetIds:
- 'Fn::ImportValue': vpc-subnet-A-${self:provider.stage}
- 'Fn::ImportValue': vpc-subnet-B-${self:provider.stage}
- 'Fn::ImportValue': vpc-subnet-C-${self:provider.stage}
AuroraRDSClusterParameter:
Type: AWS::RDS::DBClusterParameterGroup
Properties:
Description: Parameter group for the Serverless Aurora RDS DB.
Family: aurora5.6
Parameters:
character_set_database: "utf32"
AuroraRDSCluster:
Type: "AWS::RDS::DBCluster"
Properties:
MasterUsername: ${self:custom.AURORA.USERNAME}
MasterUserPassword: ${self:custom.AURORA.PASSWORD}
DBSubnetGroupName:
Ref: AuroraSubnetGroup
Engine: aurora
EngineVersion: "5.6.10a"
DatabaseName: ${self:custom.AURORA.DB_NAME}
BackupRetentionPeriod: 3
DBClusterParameterGroupName:
Ref: AuroraRDSClusterParameter
VpcSecurityGroupIds:
- 'Fn::ImportValue': VPCDefaultSecurityGroup-${self:provider.stage}
# this is needed for non-serverless mode
AuroraRDSInstanceParameter:
Type: AWS::RDS::DBParameterGroup
Properties:
Description: Parameter group for the Serverless Aurora RDS DB.
Family: aurora5.6
Parameters:
sql_mode: IGNORE_SPACE
max_connections: 100
wait_timeout: 900
interactive_timeout: 900
# this is needed for non-serverless mode
AuroraRDSInstance:
Type: "AWS::RDS::DBInstance"
Properties:
DBInstanceClass: db.t2.small
DBSubnetGroupName:
Ref: AuroraSubnetGroup
Engine: aurora
EngineVersion: "5.6.10a"
PubliclyAccessible: false
DBParameterGroupName:
Ref: AuroraRDSInstanceParameter
DBClusterIdentifier:
Ref: AuroraRDSCluster
Outputs:
DatabaseName:
Description: 'Database name.'
Value: ${self:custom.AURORA.DB_NAME}
Export:
Name: DatabaseName-${self:provider.stage}
DatabaseHost:
Description: 'Database host.'
Value: ${self:custom.AURORA.HOST}
Export:
Name: DatabaseHost-${self:provider.stage}
DatabasePort:
Description: 'Database port.'
Value: ${self:custom.AURORA.PORT}
Export:
Name: DatabasePort-${self:provider.stage}
outage-service.yml
service: outage-service
package:
individually: true
plugins:
- serverless-bundle
- serverless-plugin-ifelse
- serverless-domain-manager
custom:
appVersion: 0.0.12
stage: ${opt:stage}
domains:
prod: api.foo.com
test: test-api.foo.com
dev: dev-api.foo.com
customDomain:
domainName: ${self:custom.domains.${opt:stage}}
stage: ${opt:stage}
basePath: outages
custom.customDomain.certificateName: "*.foo.com"
custom.customDomain.certificateArn: 'arn:aws:acm:us-east-1:395671985612:certificate/XXXX'
createRoute53Record: true
serverlessIfElse:
- If: '"${opt:stage}" == "prod"'
Set:
custom.customDomain.enabled: true
ElseSet:
custom.customDomain.enabled: false
provider:
name: aws
runtime: nodejs12.x
stage: ${opt:stage}
region: us-east-1
environment:
databaseName: !ImportValue DatabaseName-${self:provider.stage}
databaseUsername: ${ssm:/my-db-username~true}
databasePassword: ${ssm:/my-db-password~true}
databaseHost: !ImportValue DatabaseHost-${self:provider.stage}
databasePort: !ImportValue DatabasePort-${self:provider.stage}
functions:
hello:
memorySize: 2048
timeout: 30
handler: functions/handler.hello
vpc:
securityGroupIds:
- 'Fn::ImportValue': VPCDefaultSecurityGroup-${self:provider.stage}
subnetIds:
- 'Fn::ImportValue': vpc-subnet-A-${self:provider.stage}
- 'Fn::ImportValue': vpc-subnet-B-${self:provider.stage}
- 'Fn::ImportValue': vpc-subnet-C-${self:provider.stage}
environment:
functionName: getTowns
events:
- http:
path: outage
method: get
cors:
origin: '*'
headers:
- Content-Type
- authorization
resources:
- Outputs:
ApiGatewayRestApiId:
Value:
Ref: ApiGatewayRestApi
Export:
Name: ${self:custom.stage}-ApiGatewayRestApiId
ApiGatewayRestApiRootResourceId:
Value:
Fn::GetAtt:
- ApiGatewayRestApi
- RootResourceId
Export:
Name: ${self:custom.stage}-ApiGatewayRestApiRootResourceId
I believe you might have missed to add VPC endpoints for API Gateway.
Ensure you create a VPC interface endpoint for API GW and use it in the API you create. This will allow API requests to lambda running within VPC
References:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-apis.html#apigateway-private-api-create-interface-vpc-endpoint
https://aws.amazon.com/blogs/compute/introducing-amazon-api-gateway-private-endpoints/
Hope this helps !!
SOLUTION: We found a solution, in our case the solution is that you can't use wildcard certificates. So if you make a certificate for the exact API URL it does work. Obviously this is not a nice solution for everyone and everything but when you do have this problem and must deploy you can make it work this way.
Also maybe good to mention we did not have any additional problems with the below mentioned case:
I also wanted to add a post I found that might also have lead to this setup not working even if we do get the network to play nice. But again if you or anyone has a working setup like the above mentioned one let me know how you did it.
TLDR
If anyone wants to understand what was going on with API Gateway, take
a look at this thread.
It basically says that API Gateway processes regular URLs (like
aaaaaaaaaaaa.execute-api.us-east-1.amazonaws.com) differently than how
it processes Custom Domain Name URLs (like api.myservice.com). So when
API Gateway forwards your API request to your Lambda Function, your
Lambda Function will receive different path values, depending on which
type of your URL you used to invoke your API.
source:
Understanding AWS API Gateway Custom Domain Names