Reduced TPS using AWS API Gateway - amazon-web-services

Question

We are conducting several performance tests in our integration environment. Our integration is depicted like this:
AWS API Gateway -> AWS App Load Balancer -> AWS ECS (docker) x 3 (using AutoScaling)
Our initial results show:
TPS = 198, 95% Response Time = 442 ms, Requests = 61790 during 5 minutes performance tests.
When we skip the API Gateway and send all the load to the Application Load Balancer we got:
TPS = 901, 95% Response Time = 433 ms, Requests = 280399 during 5 minutes performance tests.
We have disabled the throttling option in the API Gateway but there's no difference. How can I configure the AWS API Gateway in order to achieve more TPS?
Next you can find CFN code that uses Lambda instead of AWS ECS but suffers from the same TPS issue:
deploy.sh
#!/bin/bash
set -ex
AWS_REGION="us-east-1"
CLOUD_FORMATION_STACK_NAME="APILowTPSTest"
CLOUD_FORMATION_TEMPLATE_FILE="service.yaml"
CLOUD_FORMATION_CAPABILITIES="CAPABILITY_NAMED_IAM"
aws --region "${AWS_REGION}" cloudformation deploy \
--template-file "${CLOUD_FORMATION_TEMPLATE_FILE}" \
--stack-name "${CLOUD_FORMATION_STACK_NAME}" \
--capabilities "${CLOUD_FORMATION_CAPABILITIES}" \
--no-fail-on-empty-changeset \
--tags \
Name="${CLOUD_FORMATION_STACK_NAME}-Stack" \
--parameter-overrides \
EnvironmentName="${CLOUD_FORMATION_STACK_NAME}" \
LambdaFunctionName="API-Tps-Mock-Test"
service.yaml
AWSTemplateFormatVersion: 2010-09-09
Description: >
This template deploys a simple environment using an API Gateway and a Lambda Function for Performances tests purposes
Parameters:
EnvironmentName:
Description: Environment Name (Used to name some resources)
Type: String
LambdaFunctionName:
Description: Lambda function that works as mock
Type: String
Resources:
LambdaIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Policies:
- PolicyDocument:
Version: 2012-10-17
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource:
- !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${LambdaFunctionName}:*
PolicyName: lambda
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Description: Authorization Lambda Function for NCR
FunctionName: !Ref LambdaFunctionName
Handler: index.handler
MemorySize: 128
Timeout: 3
Role: !GetAtt LambdaIAMRole.Arn
Runtime: nodejs14.x
Code:
ZipFile: |
exports.handler = (event, context) => {
return new Promise((resolve) => resolve({
isBase64Encoded: false,
body: JSON.stringify({ Result: "All Good!" }),
statusCode: 200,
}));
}
ApiGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Name: !Join [ "-", [ !Ref EnvironmentName, "API" ] ]
Description: !Join [ "-", [ !Ref EnvironmentName, "API Gateway deployed to test TPS" ] ]
EndpointConfiguration:
Types:
- REGIONAL
Tags:
- Key: Name
Value: !Join [ "-", [ "API", !Ref EnvironmentName ] ]
ApiLambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref LambdaFunction
Action: lambda:InvokeFunction
Principal: apigateway.amazonaws.com
SourceArn:
!Join
- ''
- - 'arn:aws:execute-api:'
- !Ref AWS::Region
- ':'
- !Ref AWS::AccountId
- ':'
- !Ref ApiGateway
- '/*/*/*'
SubRootResource:
Type: 'AWS::ApiGateway::Resource'
Properties:
RestApiId: !Ref ApiGateway
ParentId: !GetAtt ApiGateway.RootResourceId
PathPart: "v1"
SingleRequestResource:
Type: 'AWS::ApiGateway::Resource'
Properties:
RestApiId: !Ref ApiGateway
ParentId: !Ref SubRootResource
PathPart: "test"
ApiGatewaySingleRequestMethod:
Type: AWS::ApiGateway::Method
Properties:
ResourceId: !Ref SingleRequestResource
RestApiId: !Ref ApiGateway
AuthorizationType: "NONE"
HttpMethod: "POST"
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: "POST"
Uri: !Sub
- 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations'
- lambdaArn: !GetAtt "LambdaFunction.Arn"
ApiGatewayDeployment:
Type: AWS::ApiGateway::Deployment
DependsOn:
- ApiGatewaySingleRequestMethod
Properties:
RestApiId: !Ref ApiGateway
APIStage:
Type: AWS::ApiGateway::Stage
Properties:
DeploymentId: !Ref ApiGatewayDeployment
MethodSettings:
- HttpMethod: '*'
LoggingLevel: INFO
ResourcePath: /*
DataTraceEnabled: true
RestApiId: !Ref ApiGateway
StageName: 'dev'