I have a 6Mb file that I want to upload to S3.
Currently, I use this configuration:
serverless.yml:
lambdaApi:
handler: public/index.php
events:
- httpApi: '*'
resources:
Conditions:
Resources:
ApiMapping:
Type: 'AWS::ApiGatewayV2::ApiMapping'
Condition: HaveCustomDomain
DependsOn: HttpApiIntegrationApi
Properties:
ApiMappingKey: "my-project"
DomainName: my.domain.co
ApiId: !Ref HttpApi
Stage: "$default"
The problem is that because of my lambdaApi Lambda, my POST request cannot exceed 6MB.
I've tried to used this solution (from here: https://theburningmonk.com/2020/04/hit-the-6mb-lambda-payload-limit-heres-what-you-can-do/):
But it does not work with AWS ApiGatewayV2.
When I try the plugin https://github.com/serverless-operations/serverless-apigateway-service-proxy with this custom serverless:
apiGatewayServiceProxies:
- s3:
path: /my/path
method: post
action: PutObject
bucket: dev-connect-files
cors: true
requestParameters:
# if requestParameters has a 'integration.request.path.object' property you should remove the key setting
'integration.request.path.object': 'context.requestId'
'integration.request.header.cache-control': "'public, max-age=31536000, immutable'"
I can create a POST route but it is detached from my real API.
I can't access the PHP code that the /my/path is supposed to execute.
I know I have the option of using a pre-signed URL but I don't want to use it for the moment.
Question:
Is it possible to use this plugin
https://github.com/serverless-operations/serverless-apigateway-service-proxy
with AWS ApiGatewayV2
Maybe it's not really an answer, but why pay for APIGateway, put operations on S3, etc, when you can use presigned URLs to upload large files and avoid timeouts, upload limits, and reduce costs
https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
Additionally, it's more secure as presigned URLs can expire or be limited in usage
Related
I have an existing Lambda function called My-Authorizer. I'm trying to deploy an API Gateway using Serverless, with CloudFormation (CF) resources, one of which is an authorizer that targets this Lambda.
Resources:
ApiGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Name: "${self:service}-test"
# other resources
MyAuthorizer:
Type: AWS::ApiGateway::Authorizer
DependsOn: ApiGateway
Properties:
Name: My-Authorizer
Type: REQUEST
RestApiId:
Ref: ApiGateway
AuthorizerUri: "arn:aws:apigateway:${self:custom.aws_region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${self:custom.aws_region}:${self:custom.aws_account_id}:function:My-Authorizer/invocations"
But CF gives a CREATE_FAILED for MyAuthorizer with the following Status reason:
Resource handler returned message: "Invalid request input (Service: ApiGateway, Status Code: 400, Request ID: <some-request-id>)" (RequestToken: <some-request-token>, HandlerErrorCode: AlreadyExists)
I've checked that MyAuthorizer is not already a resource in this stack.
Question: Why am I getting this error?
It is due to authorization caching is enabled by default and thus IdentitySource property is required by default. Specifying IdentitySource property or disabling authorization caching will fix the issue (see Edit 2 below).
Original (6/29/2022):
I experienced the same issue and after spending hours, I managed to solve it by adding IdentitySource property even though it is marked as required only when authorization caching is enabled. What leads me to the solution is the word "Invalid Request". And my authorizer looks like the following:
Resources:
APIGWAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: apigw-authorizer
IdentitySource: method.request.header.Authorization # customize to your need
RestApiId: !Ref ApiGateway
AuthorizerUri: # arn:aws:apigateway:{region}:lambda:path/2015-03-31/functions/{LambdaFunctionARN}/invocations
Type: REQUEST
Edit 1 (6/29/2022):
It automatically enabled authorization caching for me. The weird thing is I'm able to remove IdentitySource property and successfully update the stack. However, that didn't remove the actual Identity Source nor the Authorization Caching. So, I had to add AuthorizerResultTtlInSeconds property to disable the caching. And thus, to disable the caching, it becomes:
Resources:
APIGWAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: apigw-authorizer
AuthorizerResultTtlInSeconds: 0
IdentitySource: method.request.header.Authorization # customize to your need
RestApiId: !Ref ApiGateway
AuthorizerUri: # arn:aws:apigateway:{region}:lambda:path/2015-03-31/functions/{LambdaFunctionARN}/invocations
Type: REQUEST
Edit 2 (6/30/2022):
Found out that AuthorizerResultTtlInSeconds defaults to 300 is causing the error. So by default, authorization caching is enabled and thus IdentitySource property is actually required by default. I'm able to create a new authorizer without specifying IdentitySource property but it requires disabling the authorization caching.
Resources:
APIGWAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: apigw-authorizer
AuthorizerResultTtlInSeconds: 0 # disable authorization caching
RestApiId: !Ref ApiGateway
AuthorizerUri: # arn:aws:apigateway:{region}:lambda:path/2015-03-31/functions/{LambdaFunctionARN}/invocations
Type: REQUEST
https://docs.aws.amazon.com/apigateway/latest/developerguide/configure-api-gateway-lambda-authorization-with-console.html
I've checked that MyAuthorizer is not already a resource in this stack.
It does not have to be in the stack. It may be in your account somewhere, created manually using AWS console or other CloudFormation stack. You have to change the name of your MyAuthorizer to be unique.
I managed to find a solution that works.
I don't think there is a way to achieve this using AWS::ApiGateway::Authorizer and a Lambda function that already exists in your account. If you are deploying a new function however, then this may work.
Instead, I exported a dummy API Gateway (Export as Swagger + API Gateway Extensions) that I created via the console, which has all the authorisation configuration I require. I then did this:
Resources:
ApiGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Name: "${self:service}-test"
Body:
<exported YAML from above>
Is there some place with samples on how to make a bunch of actions through ApiGateway integration? Looking how to upload object to S3, push item to SQS & SNS queues, make DynamoDB call and many other things, trying to find documentation on how to construct those paths.
I'm using CloudFormation template, which uses integration URI to setup this AWS ApiGateway integration with AWS services.
Can't find documentation talking how to make these URI paths for all kind of services.
When setting up the integration request with another AWS service action, the integration request URI is also an ARN.
For example, for the integration with the GetBucket action of Amazon S3, the integration request URI is an ARN of the following format:
arn:aws:apigateway:api-region:s3:path
See more: https://docs.aws.amazon.com/apigateway/latest/developerguide/integration-request-basic-setup.html
Dynamodb:
A bit more complicated then S3:
https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/
SNS:
https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-proxy-integrate-service/
For SQS I have found cloudformation setup:
PostMethod:
Type: "AWS::ApiGateway::Method"
Properties:
AuthorizationType: "NONE"
ApiKeyRequired: "true"
HttpMethod: "POST"
ResourceId: !Ref "SomeResource"
RestApiId: !Ref "RestApi"
MethodResponses:
- StatusCode: 200
Integration:
Credentials: !GetAtt "RestApiRole.Arn"
IntegrationHttpMethod: "POST"
IntegrationResponses:
- StatusCode: 200
Type: "AWS"
Uri: !Sub "arn:aws:apigateway:${AWS::Region}:sqs:action/SendMessage"
RequestParameters:
integration.request.querystring.QueueUrl: !Sub "'${SomeQueue}'"
integration.request.querystring.MessageBody: "method.request.body"
and here the code for RestApiRole:
RestApiRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "sts:AssumeRole"
Principal:
Service:
- "apigateway.amazonaws.com"
Effect: "Allow"
Policies:
- PolicyName: "InvokeLambda"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "lambda:InvokeFunction"
Resource: !GetAtt "LambdaFunction.Arn"
Effect: "Allow"
From Uri property documentation:
If you specify AWS for the Type property, specify an AWS service that follows this form: arn:aws:apigateway:region:subdomain.service|service:path|action/service_api. For example, a Lambda function URI follows this form: arn:aws:apigateway:region:lambda:path/path. The path is usually in the form /2015-03-31/functions/LambdaFunctionARN/invocations. For more information, see the uri property of the Integration resource in the Amazon API Gateway REST API Reference.
More descriptions and samples from another AWS documentation:
From these documentation samples & descriptions it seems there are 2 type of APIs - action based and path based.
Using Action based API 😍
I think most, if not all support this. While those actions are available in IAM settings and all API documentations, while all AWS services are web services, aka they have API interfaces and those interfaces use Actions. Correct me if that's wrong for some service, but I think by following this structure should be possible to make any call to any service which has integration with API Gateway service.
Sometimes need to use path API 😭
Was trying to upload file to S3 with PutObject and it was giving error:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>MethodNotAllowed</Code>
<Message>The specified method is not allowed against this resource.</Message>
<Method>PUT</Method>
<ResourceType>SERVICE</ResourceType>
<RequestId>....</RequestId>
<HostId>....=</HostId>
</Error>
Replaced with path API format and it worked out. So the learning here is I will continue to try using action APIs first and if can't for that specific action - switch to path API while I feel Action API is more declarative.
Sample structure Action API:
arn:aws:apigateway:us-east-1:SERVICE_NAME:action/ACTION_NAME&Var1=Value1&Var2=Value2
Sample call to S3 service. Action name - GetObject. Documentation for this API Actions says there are 2 required properties - Bucket (bucket name) and Key. So full sample URI:
arn:aws:apigateway:us-east-1:s3:action/GetObject&Bucket=myDemoBucket1&Key=some/path/to/file
Same thing with path API:
arn:aws:apigateway:us-east-1:s3:path/myDemoBucket1/some/path/to/file
I found one way to get samples. Use console UI, make the endpoint, deploy to some stage and go to stage, select Export tab, and export as Swagger + API Gateway Extensions in Yaml format. While i use Yaml with cloudformation. Inside that Yaml there are all you need. If there are no "Stages", go to "Resources" and from dropdown select deploy and create Stage inside dialog.
Here are some different samples I was able to find for main services:
Invoke Lambda docs:
arn:aws:apigateway:api-region:lambda:path//2015-03-31/functions/arn:aws:lambda:lambda-region:account-id:function:lambda-function-name/invocations
The path part seems to map to API action from API docs:
DynamoDB blog post
You need to use HTTP method by API Action documentation + Api Action name + IntegrationRequest template to call DynamoDB.
Sample URI for Query action:
arn:aws:apigateway:us-east-1:dynamodb:action/Query
SNS blog post
Sample URI: arn:aws:apigateway:region:sns:action/Publish
With region: arn:aws:apigateway:us-east-1:sns:action/Publish
You need to pass in TopicArn and Message and other parameters through URL Query String Parameters. There is good thread on the topic: https://stackoverflow.com/a/64268791/1737158
https://docs.aws.amazon.com/sns/latest/api/API_Publish.html#API_Publish_Examples
I am creating website on AWS using serverless framework.
There are 3 parts which I need to serve clients:
endpoints of lambda functions (/getusers)
index.html served throught lambda (I need to insert some constants to it)
static html, js, css, img files, which should be served directly from S3, without going throught costly lambda.
How do I accomplish that? I need all parts of the app to be able to cooperate thus, probably on the same domain. I need index.html and js files to be able to call the lambda endpoints (setted up by API Gateway). Also I would like all files be cached throught CloudFront. I searched google but did not found any example.
The serverless-apigateway-service-proxy plugin will create API Gateway endpoints backed by services other than Lambda, including S3. Here is a complete example serverless.yml that defines an S3 bucket of public static resources and creates an API method connected to it at path /static/{asset}:
service: s3-static-asset-example
frameworkVersion: '2'
plugins:
- serverless-apigateway-service-proxy
provider:
name: aws
custom:
apiGatewayServiceProxies:
- s3:
path: /static/{asset}
method: get
action: GetObject
bucket:
Ref: StaticAssetBucket
key:
pathParam: asset
cors: true
resources:
Resources:
StaticAssetBucket:
Type: AWS::S3::Bucket
Properties:
VersioningConfiguration:
Status: Enabled
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
Outputs:
StaticAssetBucket:
Value:
Ref: StaticAssetBucket
A file app.js placed in the bucket
aws s3 cp app.js s3://s3-static-asset-example-dev-staticassetbucket-xxxxxx
is made available at
https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/static/app.js
Since each service proxy integration has its own path, a Serverless app can include a mix of proxy integrations and normal Lambda method handlers.
You can host you static files on the S3 https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html with the custom domain https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html, you can also use the CloudFront https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-cloudfront-walkthrough.html.
API Gateway can be configured with custom domain via CloudFront - https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html
Using Serverless Framework to
deploy AWS Lambda functions, Serverless creates (or receives) the
specific URL endpoint string. I want to use that string (as a variable)
in another section of the serverless.yml specification file.
Is that URL endpoint available as a variable in serverless.yml?
The Serverless Framework documentation on AWS-related variables
does not seem to answer that case.
Details: my serverless.yml contains a provider: specification
similar to:
provider:
name: aws
runtime: python3.6
memorySize: 512
region: ${opt:region, 'eu-west-1'}
profile: ${opt:profile, 'default'}
stage: ${opt:stage, 'staging'}
and a functions: section starting with:
functions:
round-control:
name: ${self:provider.stage}-round-control
runtime: nodejs8.10
handler: round/control/app.lambdaHandler
events:
- http:
path: round/control
method: get
After a
serverless deploy --profile [my-aws-profile]
the Lambda function sample-experiments-staging-round-control
is reported to be available at endpoint
https://1a234bc5de.execute-api.eu-west-1.amazonaws.com/staging/round/control.
Question: is there a variable in Serverless available that contains
that 1a234bc5de, or 1a234bc5de.execute-api or perhaps even
1a234bc5de.execute-api.eu-west-1.amazonaws.com?
(Obviously, I can also construct the last two if I know the first.)
With that variable, I can construct the full URL endpoint, which I
need in another place in the serverless.yml file.
N.B. That 1a234bc5de isn't a dynamically generated random
string - my current project is (per stage, per region) 'fixed' to
the same string. Perhaps that string is generated at AWS Lambda or
AWS API Gateway?
I was able to pass the URL and unique ID for the API Gateway endpoint to a Lambda function as environment variables as follows:
mylambda:
handler: mylambda.handler
runtime: python3.7
events:
- http:
path: id
cors: true
environment:
APIG_UID: !Ref "ApiGatewayRestApi"
APIG_URL:
!Join
- ''
- - 'https://'
- !Ref ApiGatewayRestApi
- '.execute-api.'
- ${opt:region, self:provider.region}
- '.amazonaws.com/'
- ${opt:stage, self:provider.stage}
Thanks to goingserverless.
Here is a simpler and more modern alternative:
provider:
environment:
API_URL: !Sub 'https://${ApiGatewayRestApi}.execute-api.${aws:region}.amazonaws.com/${sls:stage}'
Note: the code above applies to REST API (API Gateway v1), not HTTP API.
If you are using the new HTTP API (API Gateway v2), the solution is simpler:
provider:
environment:
API_URL: !GetAtt HttpApi.ApiEndpoint
I start study AWS serverless using lambda and API Getway, So I was thinking to create a REST API for a sample project. I noticed that in API Getway, we only can create the http methods which can trigger lambda functions, So I'm not sure if I get this right but, do we need one lambda function for each api route? or we can handle it somehow using one lambda function.
For example, lets say my project need api for below list :
Login
Register
GetUserData
Then if I going to make this by using API Getway and Lambda, I need to have 3 Lambda functions?
Like if I want create API for those mentioned above, it should be something like this for the endpoints ?
https://API_GETWAY_DOMAIN/STAGE/LAMBDA_FUNCTION_FOR_LOGIN
https://API_GETWAY_DOMAIN/STAGE/LAMBDA_FUNCTION_FOR_REGISTER
https://API_GETWAY_DOMAIN/STAGE/LAMBDA_FUNCTION_FOR_GET_USER_DATA
And is there any way to send multi http request to one lambda function, and handle each one of them inside the lambda?
It is possible to have multiple API Gateway routes direct to the same Lambda function. How to set that up depends on how you're maintaining your infrastructure.
For example, if you're using CloudFormation with the Serverless Application Model (SAM), which I recommend since it's probably the most straightforward way to keep track of serverless infrastructure (and infrastructure as code = awesome), you would define your AWS::Serverless::Function with a separate API Gateway source event for each route you define in your AWS::Serverless::Api.
Something like the following:
YourApi:
Type: AWS::Serverless::Api
Properties:
...
DefinitionBody:
swagger: 2.0
...
paths:
'/Login':
post:
x-amazon-apigateway-integration:
# APIG->Lambda requests are always POST
httpMethod: post
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${YourLambda.Arn}/invocations
'/Register':
post:
x-amazon-apigateway-integration:
# APIG->Lambda requests are always POST
httpMethod: post
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${YourLambda.Arn}/invocations
'/GetUserData':
get:
x-amazon-apigateway-integration:
# APIG->Lambda requests are always POST
httpMethod: post
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${YourLambda.Arn}/invocations
YourLambda:
Type: AWS::Serverless::Function
Properties:
...
Events:
login:
Type: Api
Properties:
Path: '/Login'
Method: post
RestApiId: {Ref: YourApi}
register:
Type: Api
Properties:
Path: '/Register'
Method: post
RestApiId: {Ref: YourApi}
getUserData:
Type: Api
Properties:
Path: '/GetUserData'
Method: get
RestApiId: {Ref: YourApi}
Keep in mind, though, that there are pros and cons to consolidating routes into one Lambda function. This StackOverflow question/answer explores that, but I'd like to add a few more benefits to separated Lambda functions:
Clearer metrics on how often your routes are getting called - You get invocations/errors/etc. metrics in CloudWatch for each Lambda function out-of-the-box, so separating them can make it easy to see how often people are registering versus logging in, etc.
More granular alarming - You can set different latency/error/etc. thresholds for different routes, and if an alarm goes off, you know exactly which Lambda function it's for.