I have tried to use two different end point for different purpose, but its working fine when i use only one endpoint but deployment got failed with error when i use two endpoints. here is my code in template.yml file
DataConditionApiLambdaFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub ${Environment}-${Application}-condition-api
Description: Lambda function behind the POST, GET, DELETE schedule api endpoints
Timeout: 90
Runtime: nodejs14.x
MemorySize: 128
Handler: !Sub src/handlers/condition/api.eventHandler
Policies:
- AWSLambdaBasicExecutionRole
- DynamoDBCrudPolicy:
TableName:
!Ref DataConditionTable
- SSMParameterReadPolicy:
ParameterName: 'acf/*'
Environment:
Variables:
Environment: !Ref Environment
conditionTableName: !Ref DataConditionTable
GreenhouseApiKeySSMPath: !Ref GreenhouseApiKeySSMPath
ConsensysKeySSMPath: !Ref ConsensysKeySSMPath
ConsensysBaseUri: !Ref ConsensysBaseUri
Tags:
Environment: !Ref Environment
Application: !Ref Application
CodePath: !Ref CodePath
Events:
HttpApiEvent:
Type: HttpApi
Properties:
- Path: /datafilter/{dataConditionId}
Method: GET
- Path: /datafilter/condition
Method: POST
any suggest the better solution to implement multiple endpoints.
Related
Pretty new to AWS Lambda function, and this is my time to get my hands dirty. I got this error in the title when I wanted to docker build my function. And here is how I configured my function:
PitchAiIngest:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub pitch-ai-ingest-${Environment}
Handler: lambda_function.lambda_handler
Runtime: python3.7
CodeUri: pitchai_ingest/
Description: get pitchai information from API and publish to dynamodb
MemorySize: 128
Timeout: 900
Role: !GetAtt LambdaRole.Arn
Environment:
Variables:
LOGGING_LEVEL: INFO
APP_NAME: pitch-ai-ingest
APP_ENV: !Ref Environment
DYNAMO_DB: !Ref PitchAiEventDynamoDBTable
PLAYER_DB: !Ref PitchAiPlayerDynamoDBTable
PITCH_SQS: !Ref PitchAiIngestQueue
Tags:
env: !Ref Environment
service: pitch-ai-service
function_name: !Sub pitch-ai-ingest-${Environment}
Roughly speaking, I post the snippet above in file cfn-tempate.yml under the same directory of folder pitchai_ingest (including Lambda handler).
What should I do to fix it?
I mistakenly set AWS_ACCESS_KEY_ID as AWS_ACCESS_KEY. That's why the credential wasn't found.
I am trying to integrate a custom domain to the HTTP API I am developing with AWS API Gateway and AWS Lambda. I m using the AWS SAM template. There I have a root stack and nested stacks.
For this question I will use a code piece with a one nested stack. There, this is how I want the URL end points to be
root stack - api.example.com
nested stack - api.example.com/nested
Below is my code
Root stack
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
aws-restapi
Sample SAM Template for aws-restapi
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 5
VpcConfig:
SecurityGroupIds:
- sg-041f24xxxx921e8e
SubnetIds:
- subnet-0xxxxx2d
Parameters:
FirebaseProjectId:
Type: String
#Dont create this domain in the AWS Console manually, so it will fail here
DomainName:
Type: String
Default: api.example.com
Resources:
AuthGatewayHttpApi:
Type: AWS::Serverless::HttpApi
Properties:
Domain:
DomainName: !Ref DomainName
EndpointConfiguration: REGIONAL
CertificateArn: arn:aws:acm:us-east-1:xxxx:certificate/xxxx-420d-xxxx-b40d-xxxx
Route53:
HostedZoneId: xxxxxxxxxxx
Auth:
Authorizers:
FirebaseAuthorizer:
IdentitySource: $request.header.Authorization
JwtConfiguration:
audience:
- !Ref FirebaseProjectId
issuer: !Sub https://securetoken.google.com/${FirebaseProjectId}
DefaultAuthorizer: FirebaseAuthorizer
AuthFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: aws-restapi/
Handler: source/testfile.lambdaHandler
Runtime: nodejs14.x
Events:
Gateway:
Type: HttpApi
Properties:
ApiId: !Ref AuthGatewayHttpApi
Path: /hello
Method: get
NestedStackTwo:
DependsOn: AuthGatewayHttpApi
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: nested_stack.yaml
Parameters:
FirebaseProjectId: !Ref FirebaseProjectId
DomainName: !Ref DomainName
Nested stack
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
aws-restapi
Sample SAM Template for aws-restapi
Globals:
Function:
Timeout: 5
VpcConfig:
SecurityGroupIds:
- sg-xxxxxxxxxxxx
SubnetIds:
- subnet-xxxxxxxxxxx
Parameters:
FirebaseProjectId:
Type: String
DomainName:
Type: String
Resources:
AuthGatewayHttpApi2:
Type: AWS::Serverless::HttpApi
Properties:
Domain:
DomainName: !Ref DomainName
BasePath: two
EndpointConfiguration: REGIONAL
CertificateArn: arn:aws:acm:us-east-1:xxxxx:certificate/xxxxx-xxxx-xxxx-xxxx-xxxx
Route53:
HostedZoneId: xxxxxxxxxxxxx
Auth:
Authorizers:
FirebaseAuthorizer:
IdentitySource: $request.header.Authorization
JwtConfiguration:
audience:
- !Ref FirebaseProjectId
issuer: !Sub https://securetoken.google.com/${FirebaseProjectId}
DefaultAuthorizer: FirebaseAuthorizer
GetAllPromotionsFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: aws-restapi/
Handler: source/promotions/promotions-getall.getAllPromotions
Runtime: nodejs14.x
Events:
GetAllPromotionsAPIEvent:
Type: HttpApi # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /promotions/getall
Method: get
ApiId: !Ref AuthGatewayHttpApi2
However I cant get this to work because the nested stack fails to create. Below is the error
CREATE_FAILED AWS::CloudFormation::Stack NestedStackTwo
Embedded stack arn:aws:cloudformation:us-east-1 xxxxx:stack/aws-restapi-NestedStackTwo-8KBISZRAVYBX/a3fcc010-0ce4-11ec-9c90-0e8a861a6983 was not successfully created:
The following resource(s) failed to create: [ApiGatewayDomainNameV234ac706a57]
I believe this is happening because the root stack creates the domain and the nested stack is trying to re-create it instead of reusing the same.
But, here is the fun fact; if i use the AWS API GATEWAY web console, I can do this in no time.
How can I get this to work in aws-sam ?
UPDATE
Following the advice from the user LRutten, I came up with the following code for the nested stack.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
aws-restapi
Sample SAM Template for aws-restapi
Globals:
Function:
Timeout: 5
VpcConfig:
SecurityGroupIds:
- sg-xxxxxx
SubnetIds:
- subnet-xxxxx
Parameters:
FirebaseProjectId:
Type: String
DomainName:
Type: String
Resources:
AuthGatewayHttpApi2:
Type: AWS::Serverless::HttpApi
Properties:
Auth:
Authorizers:
FirebaseAuthorizer:
IdentitySource: $request.header.Authorization
JwtConfiguration:
audience:
- !Ref FirebaseProjectId
issuer: !Sub https://securetoken.google.com/${FirebaseProjectId}
DefaultAuthorizer: FirebaseAuthorizer
MyApiMapping:
Type: 'AWS::ApiGatewayV2::ApiMapping'
Properties:
DomainName: !Ref DomainName
ApiId: !Ref AuthGatewayHttpApi2
Stage: prod
MyDomainName:
Type: 'AWS::ApiGatewayV2::DomainName'
Properties:
DomainName: !Ref DomainName
DomainNameConfigurations:
- EndpointType: REGIONAL
CertificateArn: arn:aws:acm:us-east-1:716460586643:certificate/bac44716-420d-431b-b40d-01378f20432d
GetAllPromotionsFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: aws-restapi/
Handler: source/promotions/promotions-getall.getAllPromotions
Runtime: nodejs14.x
Events:
GetAllPromotionsAPIEvent:
Type: HttpApi # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /promotions/getall
Method: get
ApiId: !Ref AuthGatewayHttpApi2
This ended up with the following error
Embedded stack arn:aws:cloudformation:us-east-1:716460586643:stack/aws-restapi-NestedStackTwo-14SAYLRO1WD1D/62336020-xx-xx-a04e-x was not successfully created:
The following resource(s) failed to create: [MyDomainName, MyApiMapping].
Indeed SAM always creates the domain when you specify it's name there. It stated in the docs as well.
To get around this, you can omit the whole domain configuration in the AWS::Serverless::HttpApi resource and write the resources created by SAM yourself. So add a section with
Type: AWS::ApiGatewayV2::ApiMapping
Properties:
ApiId: String
ApiMappingKey: String
DomainName: String
Stage: String
Docs
Make sure the Stage property in the ApiMapping is also present in the StageName property of the API.
And a
Type: AWS::ApiGatewayV2::DomainName
Properties:
DomainName: String
DomainNameConfigurations:
- DomainNameConfiguration
MutualTlsAuthentication:
MutualTlsAuthentication
Tags: Json
Docs
Manually adding these without having the domain itself defined twice should do the trick.
Edit: woops wasn't really thinking straight. You should of course only have the mapping, not the domain name itself again :P.
I have added one layer to my Lambda function through CloudFormation. Now I have a requirement to add one more layer to my function. Basically, I need two layers in my existing Lambda function. Is it possible? I tried searching the AWS docs but I don't see it.
Resources:
LambdaLayer:
Type: "AWS::Lambda::LayerVersion"
Properties:
CompatibleRuntimes:
- python3.8
Content:
S3Bucket: !Sub "hello-${AWS::Region}"
S3Key: !Sub "myapp/layer1.zip"
LayerName: "layer1"
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Code:
S3Bucket: hello
S3Key: myapp/function.zip"
FunctionName: "hello-function"
Handler: "hello-function.lambda_handler"
Layers:
- !Ref LambdaLayer
Yes, is it possible. Add additional layers in the same way that you did the first layer, only append numbers to the resource names to distinguish them:
Resources:
LambdaLayer1:
Type: "AWS::Lambda::LayerVersion"
Properties:
CompatibleRuntimes:
- python3.8
Content:
S3Bucket: !Sub "hello-${AWS::Region}"
S3Key: !Sub "myapp/layer1.zip"
LayerName: "layer1"
LambdaLayer2:
Type: "AWS::Lambda::LayerVersion"
...
LayerName: "layer2"
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Code:
S3Bucket: hello
S3Key: myapp/function.zip"
FunctionName: "hello-function"
Handler: "hello-function.lambda_handler"
Layers:
- !Ref LambdaLayer1
- !Ref LambdaLayer2
I have created a cloudformation template to create api gateway/resource/method/function on AWS platform, and also associated a lambda function to my api. Once I create the stack I get a url to hit my aws api. Whenever I hit this url, I get internal server error.
I am not sure what could be the reason, but if I toggle between 2 lambda functions from aws console for integration request and deploy, it starts working as expected.
I don't want to do it manually as deployment should be done at the time of stack creation.
Below is the template I used to create resources
Resources:
RestApi:
Type: AWS::ApiGateway::RestApi
Properties:
Name: !Sub 'testing'
EndpointConfiguration:
Types:
- 'EDGE'
ApigwResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref RestApi
ParentId: !GetAtt RestApi.RootResourceId
# lambda function
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Role: 'lambda_role'
Handler: lambda_s3.lambda_handler
Code:
S3Bucket: { 'Fn::ImportValue': !Sub '${S3Bucket}-S3AppsBucketId' }
S3Key: 'lambda_source_code'
Runtime: python3.7
MemorySize: 128
Timeout: 60
FunctionName: !Sub 'lambda_function'
ApiGatewayMethod:
Type: AWS::ApiGateway::Method
Properties:
HttpMethod: PUT
ResourceId: !Ref ApigwResource
RestApiId: !Ref RestApi
AuthorizationType: AWS_IAM
MethodResponses:
- ResponseModels: { 'application/json' : 'Empty' }
StatusCode: 200
Integration:
Type: AWS
IntegrationHttpMethod: PUT
IntegrationResponses:
- StatusCode: 200
Uri: !Sub
- 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations'
- lambdaArn: !GetAtt LambdaFunction.Arn
ApiGatewayDeployment:
Type: AWS::ApiGateway::Deployment
DependsOn:
- 'ApiGatewayMethod'
Properties:
RestApiId: !Ref RestApi
StageName: !Ref Environment
I found the problem, my api gateway didn't have permission to invoke lambda function. When I was toggling lambda functions manually, it used to provide necessary permissions to my api gateway thus this work around was working. I added below piece to my cloudformation template to make this work.
ConfigLambdaPermission:
Type: "AWS::Lambda::Permission"
DependsOn:
- RestApi
- LambdaFunction
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref LambdaFunction
Principal: apigateway.amazonaws.com
The following AWS CloudFormation gives a circular dependency error. My understanding is that the dependencies flow like this: rawUploads -> generatePreview -> previewPipeline -> rawUploads. Although it doesn't seem like rawUploads depends on generatePreview, I guess CF needs to know what lambda to trigger when creating the bucket, even though the trigger is defined in the lambda part of the CloudFormation template.
I've found some resources online that talk about a similar issue, but it doesn't seem to apply here. https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-circular-dependency-cloudformation/
What are my options for breaking this circular dependency chain? Scriptable solutions are viable, but multiple deployments with manual changes are not for my use case.
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
rawUploads:
Type: 'AWS::S3::Bucket'
previewAudioFiles:
Type: 'AWS::S3::Bucket'
generatePreview:
Type: AWS::Serverless::Function
Properties:
Handler: generatePreview.handler
Runtime: nodejs6.10
CodeUri: .
Environment:
Variables:
PipelineId: !Ref previewPipeline
Events:
BucketrawUploads:
Type: S3
Properties:
Bucket: !Ref rawUploads
Events: 's3:ObjectCreated:*'
previewPipeline:
Type: Custom::ElasticTranscoderPipeline
Version: '1.0'
Properties:
ServiceToken:
Fn::Join:
- ":"
- - arn:aws:lambda
- Ref: AWS::Region
- Ref: AWS::AccountId
- function
- aws-cloudformation-elastic-transcoder-pipeline-1-0-0
Name: transcoderPipeline
InputBucket:
Ref: rawUploads
OutputBucket:
Ref: previewAudioFiles
One way is to give the S3 buckets explicit names so that later, instead of relying on Ref: bucketname, you can simply use the bucket name. That's obviously problematic if you want auto-generated bucket names and in those cases it's prudent to generate the bucket name from some prefix plus the (unique) stack name, for example:
InputBucket: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]
Another option is to use a single CloudFormation template but in 2 stages - the 1st stage creates the base resources (and whatever refs are not circular) and then you add the remaining refs to the template and do a stack update. Not ideal, obviously, so I would prefer the first approach.
You can also use the first technique in cases when you need a reference to an ARN, for example:
!Join ['/', ['arn:aws:s3:::logsbucket', 'AWSLogs', Ref: 'AWS:AccountId', '*']]
When using this technique, you may want to also consider using DependsOn because you have removed an implicit dependency which can sometimes cause problems.
This post helped me out in the end: https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/
I ended up configuring an SNS topic in CloudFormation. The bucket would push events on this topic, and the Lambda function listens to this topic. This way the dependency graph is as follows:
S3 bucket -> SNS topic -> SNS topic policy
Lambda function -> SNS topic
Lambda function -> transcoder pipeline
Something along the lines of this (some policies omitted)
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
SNSTopic:
Type: AWS::SNS::Topic
SNSTopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Id: MyTopicPolicy
Version: '2012-10-17'
Statement:
- Sid: Statement-id
Effect: Allow
Principal:
AWS: "*"
Action: sns:Publish
Resource:
Ref: SNSTopic
Condition:
ArnLike:
aws:SourceArn:
!Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
Topics:
- Ref: SNSTopic
rawUploads:
Type: 'AWS::S3::Bucket'
DependsOn: SNSTopicPolicy
Properties:
BucketName: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]
NotificationConfiguration:
TopicConfigurations:
- Topic:
Ref: "SNSTopic"
Event: 's3:ObjectCreated:*'
previewAudioFiles:
Type: 'AWS::S3::Bucket'
generatePreview:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Join ["-", ['generatepreview', Ref: 'AWS::StackName']]
Handler: generatePreview.handler
Runtime: nodejs6.10
CodeUri: .
Environment:
Variables:
PipelineId: !Ref previewPipeline
Events:
BucketrawUploads:
Type: SNS
Properties:
Topic: !Ref "SNSTopic"
previewPipeline:
Type: Custom::ElasticTranscoderPipeline
DependsOn: 'rawUploads'
Version: '1.0'
Properties:
ServiceToken:
Fn::Join:
- ":"
- - arn:aws:lambda
- Ref: AWS::Region
- Ref: AWS::AccountId
- function
- aws-cloudformation-elastic-transcoder-pipeline-1-0-0
Name: transcoderPipeline
InputBucket:
!Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
OutputBucket:
Ref: previewAudioFiles