Just like in the title. I try to integrate API Gateway method with a SQS using cloud formation. What I am missing is the correct URI for the SQS. If any of you already did that, what should the URI look like?
I came up with something like that, but have no idea where to put the SQS ARN
"arn:aws:apigateway:${AWS::Region}:sqs:action/SendMessage"
Here is the full configuration for the method:
PostMethod:
Type: "AWS::ApiGateway::Method"
Properties:
ApiKeyRequired: "true"
HttpMethod: "POST"
ResourceId: !Ref "SomeResource"
RestApiId: !Ref "SomeRestApi"
Integration:
IntegrationHttpMethod: "POST"
IntegrationResponses:
- StatusCode: 200
Type: "AWS"
Uri: "arn:aws:apigateway:${AWS::Region}:sqs:action/SendMessage"
And here is an example of URI if you integrate with a lambda function:
arn:aws:apigateway:us-west-2:lambda:path//2015-03-31/functions/arn:aws:lambda:us-west-2:123412341234:function:function_name/invocations
-
To answer my own question. Here is how you integrate SQS as a Service Proxy in API Gateway:
PostMethod:
Type: "AWS::ApiGateway::Method"
Properties:
AuthorizationType: "NONE"
ApiKeyRequired: "true"
HttpMethod: "POST"
ResourceId: !Ref "SomeResource"
RestApiId: !Ref "RestApi"
MethodResponses:
- StatusCode: 200
Integration:
Credentials: !GetAtt "RestApiRole.Arn"
IntegrationHttpMethod: "POST"
IntegrationResponses:
- StatusCode: 200
Type: "AWS"
Uri: !Sub "arn:aws:apigateway:${AWS::Region}:sqs:action/SendMessage"
RequestParameters:
integration.request.querystring.QueueUrl: !Sub "'${SomeQueue}'"
integration.request.querystring.MessageBody: "method.request.body"
I've finally found all answers to my questions in various documentation. RTFM I guess.
EDIT:
and here the code for RestApiRole:
RestApiRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "sts:AssumeRole"
Principal:
Service:
- "apigateway.amazonaws.com"
Effect: "Allow"
Policies:
- PolicyName: "InvokeLambda"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "lambda:InvokeFunction"
Resource: !GetAtt "LambdaFunction.Arn"
Effect: "Allow"
I'm pretty sure the SQS role and policy should look more like this (you seem to have pasted the lambda role instead):
SQSRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action: sts:AssumeRole
Path: /
SQSRolePolicy:
Type: AWS::IAM::Policy
DependsOn: [SQSRole]
Description: IAM policy applied to the service role.
Properties:
PolicyName: send-messages-sqs
PolicyDocument:
Statement:
- Action:
- sqs:SendMessage
Resource:
- !Sub arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:QUEUE_NAME
Effect: Allow
Roles: [!Ref SQSRole]
Related
I am trying to create a CloudFormation template to deploy an API Gateway HTTP API integrated with SQS. I used the below CF template:
AWSTemplateFormatVersion: "2010-09-09"
Resources:
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: ExecutionRole
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- sqs:SendMessage
- sqs:ReceiveMessage
- sqs:DeleteMessage
Resource: !Sub "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:sample-queue"
- Effect: Allow
Action: lambda:InvokeFunction
Resource: "*"
Queue:
Type: AWS::SQS::Queue
Properties:
QueueName: sample-queue
API:
Type: AWS::ApiGatewayV2::Api
Properties:
Body:
openapi: "3.0.1"
info:
title: "HttpApi"
version: "2022-09-29 15:59:08UTC"
paths:
/:
post:
responses:
default:
description: "Default response for POST /"
x-amazon-apigateway-integration:
integrationSubtype: "SQS-SendMessage"
credentials: !Sub "arn:aws:iam::${AWS::AccountId}:role/ExecutionRole"
requestParameters:
MessageBody: "$request.body"
QueueUrl: !Sub "https://sqs.${AWS::Region}.amazonaws.com/${AWS::AccountId}/sample-queue"
payloadFormatVersion: "1.0"
type: "aws_proxy"
connectionType: "INTERNET"
timeoutInMillis: 30000
x-amazon-apigateway-importexport-version: "1.0"
Stage:
Type: AWS::ApiGatewayV2::Stage
Properties:
StageName: v1
ApiId: !Ref API
Deployment:
Type: AWS::ApiGatewayV2::Deployment
Properties:
ApiId: !Ref API
StageName: !Ref Stage
The API this generates looks good in the web console, but not sure why, any POST to this API gives an error:
{
"message": "Not Found"
}
Perhaps I am missing something very silly. But not able to locate it. Can you please help me identify?
I built an cloudFormation template. It perform a simple workflow: client make a request -> api gateway handle it and send to lambda fucntion via proxy integration. There is a role between api gateway an a lambda.
My template consist of:
root resource (AudienceApi)
nested resource (Segment)
method post (PostMethod) with lambda`s integration
lambda fucntion (lambdaFunction)
lambda invoker (it is a permission for api gateway to invoke the lambda)
lambda role
And during testing the whole workflow I have faced with a problem - api responce with an error 500 Internal Server Error. From log I found message:
Invalid permissions on Lambda function.
I continue testing and find out when I remove nested resource (Segment) and connect method post resource (PostMethod) directly to root resource (AudienceApi) my workflow started to working.
Question: What wrong with my template? why it is not working with nested resource path?
Maybe someone look at my template and find an error?
template:
AWSTemplateFormatVersion: "2010-09-09"
Description: "My API Gateway and Lambda function"
Parameters:
apiGatewayStageName:
Type: "String"
AllowedPattern: "^[a-z0-9]+$"
Default: "call"
lambdaFunctionName:
Type: "String"
AllowedPattern: "^[a-zA-Z0-9]+[a-zA-Z0-9-]+[a-zA-Z0-9]+$"
Default: "my-function"
Resources:
AudienceApi:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "my-api"
Description: "My API"
Segment:
Type: 'AWS::ApiGateway::Resource'
Properties:
RestApiId: !Ref AudienceApi
ParentId: !GetAtt
- AudienceApi
- RootResourceId
PathPart: segment
PostMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: POST
# RequestModels:
# application/json: !Ref SegmentationRequestModel
AuthorizationType: NONE
Integration:
IntegrationHttpMethod: POST
Type: AWS_PROXY
Uri: !Sub
- "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations"
- lambdaArn: !GetAtt "lambdaFunction.Arn"
ResourceId: !Ref Segment
RestApiId: !Ref AudienceApi
lambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: |
def handler(event,context):
return {
'body': 'Hello there {0}'.format(event['requestContext']['identity']['sourceIp']),
'headers': {
'Content-Type': 'text/plain'
},
'statusCode': 200
}
Description: "My function"
FunctionName: !Ref "lambdaFunctionName"
Handler: "index.handler"
MemorySize: 128
Role: !GetAtt "lambdaIAMRole.Arn"
Runtime: "python2.7"
Timeout: 10
lambdaApiGatewayInvoke:
Type: "AWS::Lambda::Permission"
Properties:
Action: "lambda:InvokeFunction"
FunctionName: !GetAtt "lambdaFunction.Arn"
Principal: "apigateway.amazonaws.com"
SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${AudienceApi}/*/POST/"
lambdaIAMRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "sts:AssumeRole"
Effect: "Allow"
Principal:
Service:
- "lambda.amazonaws.com"
Policies:
- PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
Effect: "Allow"
Resource:
- !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${lambdaFunctionName}:*"
PolicyName: "lambda"
lambdaLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
LogGroupName: !Sub "/aws/lambda/${lambdaFunctionName}"
RetentionInDays: 90
Based on the comments, the solution was to add * to the ARN after the POST/ in SourceArn:
SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${AudienceApi}/*/POST/*"
I am trying to request parameters using API Gateway in AWS CloudFormation. The parameter that I want to pass down from API gateway to Lambda function is 'action'. I have tried the following code and so far I ran into the error, mention below. Can someone please help me with identifying the issue and a possible resolution?
"Invalid mapping expression specified: Validation Result: warnings : [], errors : [Invalid mapping expression specified: Integration.request.path.action] (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException; Request ID: 037f4753-52b5-4276-979a-131a0f903e63)"
AWSTemplateFormatVersion: "2010-09-09"
Description: "API Gateway and Lambda function"
Resources:
SampleApi:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: Sample
SampleApiMethod:
Type: "AWS::ApiGateway::Method"
Properties:
AuthorizationType: "NONE"
HttpMethod: "GET"
RequestParameters:
method.request.path.action: true
RequestTemplates:
application/yaml
Integration:
IntegrationHttpMethod: "POST"
Type: "AWS_PROXY"
RequestParameters:
Integration.request.path.action: method.request.path.action
Uri: !Sub
- "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations"
- lambdaArn: !GetAtt "SampleLambda.Arn"
CacheKeyParameters:
- method.request.path.action
ResourceId: !GetAtt "SampleApi.RootResourceId"
RestApiId: !Ref "SampleApi"
SampleApiDeployment:
Type: "AWS::ApiGateway::Deployment"
DependsOn: "SampleApiMethod"
Properties:
RestApiId: !Ref "SampleApi"
StageName: test
SampleLambda:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: |
import yaml
import boto3
cf_client = boto3.client('cloudformation')
cf_client.create_stack(
StackName='your-stack',
TemplateURL='Some URL',
Parameters=[
{
'ParameterKey':'action',
'ParameterValue': 'kms:*'
},
]
)
Handler: "index.handler"
Role: !GetAtt "SampleLambdaRole.Arn"
Runtime: python3.7
LambdaApiGatewayInvoke:
Type: "AWS::Lambda::Permission"
Properties:
Action: "lambda:InvokeFunction"
FunctionName: !GetAtt "SampleLambda.Arn"
Principal: "apigateway.amazonaws.com"
SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${SampleApi}/*/GET/"
SampleLambdaRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Action: ["sts:AssumeRole"]
Effect: "Allow"
Principal:
Service: ["lambda.amazonaws.com"]
Policies:
- PolicyDocument:
Version: "2012-10-17"
Statement:
- Action: ["cloudwatch:*", "logs:*"]
Effect: "Allow"
Resource: "*"
PolicyName: "lambdaLogPolicy"
Outputs:
apiGatewayInvokeURL:
Value: !Sub 'https://Sample.execute-api.${AWS::Region}.amazonaws.com/test'
According to the docs, the key for RequestParameters should be like integration.request.<location>.<name>, with a lowercase i for integration. You are using an uppercase I. From the AWS CloudFormation docs:
Specify the destination by using the following pattern integration.request.location.name, where location is query string, path, or header, and name is a valid, unique parameter name.
Also, your template from above contains a RequestTemplates property which is placed in the wrong hierarchy level. It must be placed below Integration as noted in the AWS CloudFormation docs as well. Here is the correct template for AWS::ApiGateway::Method for you:
SampleApiMethod:
Type: "AWS::ApiGateway::Method"
Properties:
AuthorizationType: "NONE"
HttpMethod: "GET"
RequestParameters:
method.request.path.action: true
Integration:
IntegrationHttpMethod: "POST"
Type: "AWS_PROXY"
RequestParameters:
integration.request.path.action: method.request.path.action
RequestTemplates:
"application/yaml": "<define your template here>"
Uri: !Sub
- "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambdaArn}/invocations"
- lambdaArn: !GetAtt "SampleLambda.Arn"
CacheKeyParameters:
- method.request.path.action
ResourceId: !GetAtt "SampleApi.RootResourceId"
RestApiId: !Ref "SampleApi"
More information about defining a request template can be found in the developer reference.
I want to implement a GET method in AWS API Gateway that returns the messages from a AWS SQS. When I test it I get an exception:
<AccessDeniedException>
<Message>Unable to determine service/operation name to be authorized</Message>
</AccessDeniedException>
I have all the stack defined in a Serverless yml file:
functions:
listExportJob:
handler: src/listExportJob.handler
role: listExportJobIAM
environment:
processingqueueUrl: https://xxxxx/processing-exports-queue-eu-local
events:
- processingsqs:
arn: arn:aws:sqs:xxxxx:processing-exports-queue-eu-local
events:
- sqs:ChangeMessageVisibility
- sqs:ChangeMessageVisibilityBatch
- sqs:GetQueueAttributes
- sqs:ReceiveMessage
resources:
Resources:
processingSQSQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: processing-exports-queue-eu-local
ApiGatewayRestApi:
Type: AWS::ApiGateway::RestApi
Properties:
Name: ApiGateway
listExportAPIResource:
Type: "AWS::ApiGateway::Resource"
Properties:
ParentId:
Fn::GetAtt:
- "ApiGatewayRestApi"
- "RootResourceId"
PathPart: "listExport"
RestApiId:
Ref: ApiGatewayRestApi
listExportAPIMethod:
Type: AWS::ApiGateway::Method
DependsOn: processingSQSQueue
Properties:
RestApiId:
Ref: ApiGatewayRestApi
ResourceId:
Ref: listExportAPIResource
HttpMethod: "GET"
MethodResponses:
- StatusCode: "200"
ResponseParameters:
"method.response.header.Access-Control-Allow-Origin": true
AuthorizationType: "NONE"
Integration:
Type: AWS
Credentials:
Fn::GetAtt: [ "APIGatewaySQSIAM", "Arn" ]
IntegrationHttpMethod: POST
IntegrationResponses:
- StatusCode: "200"
ResponseParameters:
"method.response.header.Access-Control-Allow-Origin": "'*'"
ResponseTemplates:
"application/json": ""
Uri: arn:aws:apigateway:xxxxx/processing-exports-queue-eu-local
APIGatewaySQSIAM:
Type: AWS::IAM::Role
Properties:
Path: /app/all/
RoleName: APIGSQSRole
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: APIGATEWAYIAMAll
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Resource: "*"
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- Effect: Allow
Resource:
- "*"
Action:
- "sqs:SendMessage"
listExportJobIAM:
Type: AWS::IAM::Role
Properties:
RoleName: listExportJobRole
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: listExportJobIAMAll
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
- logs:CreateLogGroup
Resource: '*'
- Effect: Allow
Action:
- sqs:ChangeMessageVisibility
- sqs:ChangeMessageVisibilityBatch
- sqs:GetQueueAttributes
- sqs:ReceiveMessage
Resource: arn:aws:sqs:xxxxx:processing-exports-queue-eu-local
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource: '*'
I have defined the GET method with IntegrationHttpMethod POST and I can't see what's wrong with my implementation.
AWS API Gateway has another method that posts a message directly to AWS SQS and works properly.
Finally it works using Lambda-Proxy integration adding a http event to the function:
events:
- http:
path: listExports
method: get
cors: true
I have a lambda function that will handle PUT and GET requests using Amazon API Gateway {proxy+}.
It is working correctly when all the settings are set manually by the Amazon Console. but I want to automate it using AWS Cloudformation.
To inform you, I will write steps to set {proxy+}:
1) create a simple Lambda function and paste this lines of code inside it:
import boto3
def lambda_handler(event, context):
return {
"statusCode": 200,
"headers": {
"Content-Type": 'text/html',
"Access-Control-Allow-Origin": "*"
},
"body": "Hello Reza Amya, Your Lambda is working..!"
}
2) goto Amazon API Gateway and click on Create API.
3) choose New API, fill API name, select Edge optimized from the list for Endpoint Type then click on Create API
4) then your API is created and you should be on it's Resources page, if you are not, go to the Resources page for the created API.
5) from Actions select Create Resource
6) Select Configure as proxy resource (then it should change other fields automatically, if it doesn't, type proxy for Resource Name and {proxy+} for Resource Path) then click on Create Resource
7) Select Lambda Function Proxy for Integration type and select your lambda function from Lambda Function and click on Save
8) on the Add Permission to Lambda Function popup, click on Ok
9) from Actions click on Deploy API
10) Select New Stage from the list for Deployment stage then type a name for Stage name (for me, I have typed 'api') and click on Deploy
11) on the stage on the root page for your deployed API, you can see Invoke URL. click on it, and it will open new tab linked to somewhere like this: https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/api/
12) add a simple segment to end of your URL like this:
https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/api/test
now you should see bellow message in your browser page:
Hello Reza Amya, Your Lambda is working..!
Now the problem is I have written all these steps inside a Yaml file:
AWSTemplateFormatVersion: 2010-09-09
Description: My Lambda Function
Parameters:
S3Bucket:
Description: S3 Bucket where the Lambda code is
Type: String
S3Key:
Description: S3 Key where the Lambda code is
Type: String
S3ObjectVersion:
Description: Version of the S3 Key to use
Type: String
Resources:
apiGateway:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "my-api"
Description: "My API"
EndpointConfiguration:
Types:
- EDGE
Resource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: "apiGateway"
ParentId:
Fn::GetAtt:
- "apiGateway"
- "RootResourceId"
PathPart: "{proxy+}"
ProxyMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: ANY
ResourceId: !Ref Resource
RestApiId: !Ref apiGateway
AuthorizationType: NONE
RequestParameters:
method.request.path.proxy: true
Integration:
CacheKeyParameters:
- 'method.request.path.proxy'
RequestParameters:
integration.request.path.proxy: 'method.request.path.proxy'
Type: AWS_PROXY
IntegrationHttpMethod: ANY
Uri: !Sub
- arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Arn}/invocations
- Arn:
Fn::GetAtt:
- LambdaFunction
- Arn
PassthroughBehavior: WHEN_NO_MATCH
IntegrationResponses:
- StatusCode: 200
apiGatewayDeployment:
Type: "AWS::ApiGateway::Deployment"
DependsOn:
- "ProxyMethod"
Properties:
RestApiId: !Ref "apiGateway"
StageName: "dev"
IAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: 'sts:AssumeRole'
Policies:
- PolicyName: Logging
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
- PolicyName: AccessToDynamoDB
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'dynamodb:CreateTable'
- 'dynamodb:DeleteItem'
- 'dynamodb:DeleteTable'
- 'dynamodb:GetItem'
- 'dynamodb:GetRecords'
- 'dynamodb:UpdateItem'
- 'dynamodb:UpdateTable'
- 'dynamodb:PutItem'
- 'dynamodb:UpdateTable'
Resource: 'arn:aws:dynamodb:*:*:*'
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: {Ref: S3Bucket}
S3Key: {Ref: S3Key}
S3ObjectVersion: {Ref: S3ObjectVersion}
Handler: main.lambda_handler
MemorySize: 128
Role: {'Fn::GetAtt': [IAMRole, Arn]}
Runtime: python3.6
Timeout: 300
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt
- LambdaFunction
- Arn
Action: 'lambda:InvokeFunction'
Principal: apigateway.amazonaws.com
SourceArn: !Sub arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${apiGateway}/*/*
Outputs:
apiGatewayInvokeURL:
Value: !Sub "https://${apiGateway}.execute-api.${AWS::Region}.amazonaws.com/${apiGateway}"
lambdaArn:
Value: !GetAtt "LambdaFunction.Arn"
The above Yaml file will create the Lambda function and will deploy the API, but it will show bellow error when I am trying to test the API:
{"message": "Internal server error"}
Can you please guide me what is wrong and how I can solve the problem?
The issue is related to you IntegrationHttpMethod setting. Although your APIGateway method is ANY, the IntegrationHttpMethod must always be POST for AWS Lambda.
This would lead to the following method declaration.
ProxyMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: ANY
ResourceId: !Ref Resource
RestApiId: !Ref apiGateway
AuthorizationType: NONE
RequestParameters:
method.request.path.proxy: true
Integration:
CacheKeyParameters:
- 'method.request.path.proxy'
RequestParameters:
integration.request.path.proxy: 'method.request.path.proxy'
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub
- arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Arn}/invocations
- Arn:
Fn::GetAtt:
- LambdaFunction
- Arn
PassthroughBehavior: WHEN_NO_MATCH
IntegrationResponses:
- StatusCode: 200