I have tried several variations of using MessageAttributes in my CloudFormation document but I can't get it to work. This is what I've got:
HttpApiSqsIntegration:
Type: AWS::ApiGatewayV2::Integration
Properties:
ApiId: !Ref HttpApiRef
CredentialsArn: !GetAtt MyHttpApiRole.Arn
IntegrationType: AWS_PROXY
IntegrationSubtype: SQS-SendMessage
PayloadFormatVersion: "1.0"
RequestParameters:
QueueUrl: !Ref InputMessageSqs
MessageBody: $request.body
MessageAttributes.entry.1.Name: foo
MessageAttributes.entry.1.Value.StringValue: bar
MessageAttributes.entry.1.Value.DataType: String
I'm following the format specified here, here, and here but it gives the following error message:
Parameter: MessageAttributes.entry.1.Name does not fit schema for Operation:
SQS-SendMessage. (Service: AmazonApiGatewayV2; Status Code: 400;
Error Code: BadRequestException; Request ID: ...; Proxy: null)
Any help is appreciated. Thank you.
You are following the wrong specification. Here is the right one.
RequestParameters For WebSocket APIs, a key-value map specifying
request parameters that are passed from the method request to the
backend. The key is an integration request parameter name and the
associated value is a method request parameter value or static value
that must be enclosed within single quotes and pre-encoded as required
by the backend. The method request parameter value must match the
pattern of method.request.{location}.{name} , where {location} is
querystring, path, or header; and {name} must be a valid and unique
method request parameter name.
For HTTP API integrations with a specified integrationSubtype, request
parameters are a key-value map specifying parameters that are passed
to AWS_PROXY integrations. You can provide static values, or map
request data, stage variables, or context variables that are evaluated
at runtime. To learn more, see Working with AWS service integrations
for HTTP APIs.
For HTTP API integrations without a specified integrationSubtype
request parameters are a key-value map specifying how to transform
HTTP requests before sending them to the backend. The key should
follow the pattern :<header|querystring|path>. where
action can be append, overwrite or remove. For values, you can provide
static values, or map request data, stage variables, or context
variables that are evaluated at runtime. To learn more, see
Transforming API requests and responses.
I haven't done this in CloudFormation but managed to get it working from the web console. I imagine the syntax is equivalent to the one you would use in CloudFormation.
For example, suppose you want to add an attribute called timestamp with value equal to the request header x-timestamp. You would use this syntax in the MessageAttributes field:
{ "timestamp": { "DataType": "String", "StringValue": "${request.header.x-timestamp}" } }
This is (very badly) documented in the SQS API reference.
See the SendMessage and MessageAttributeValue pages.
The correct part of the documentation is where it says that MessageAttributes has to be a "String to MessageAttributeValue object map".
The rest of the documentation where it mentions MessageAttribute.N.Name seems to be wrong.
Here is how you can do within a CloudFormation template:
HttpApiSqsIntegration:
Type: AWS::ApiGatewayV2::Integration
Properties:
ApiId: !Ref HttpApiRef
CredentialsArn: !GetAtt MyHttpApiRole.Arn
IntegrationType: AWS_PROXY
IntegrationSubtype: SQS-SendMessage
PayloadFormatVersion: "1.0"
RequestParameters:
QueueUrl: !Ref InputMessageSqs
MessageBody: $request.body
MessageAttributes: >-
{
"foo": {
"DataType": "String",
"StringValue": "bar"
}
}
If you need a dynamic value you can do:
RequestParameters:
QueueUrl: !Ref InputMessageSqs
MessageBody: $request.body
MessageAttributes: >-
{
"foo": {
"DataType": "String",
"StringValue": "${context.path}"
}
}
Note that MessageAttributes must be a valid JSON string.
See: https://awsteele.com/blog/2021/09/06/api-gateway-http-apis-and-sqs-messageattributes.html
Related
I am writing a SAM template for a business application that, among other things, processes device status updates.
Users post status updates to the API Gateway that, in turn, suitably sets the destSQS message attribute and publishes such requests to an SNS topic.
Several SQS queues are subscribed to the topic and messages are routed depending on the value of destSQS. The following is the AWS::SNS::Subscription resource definition for the SQS queue of interest for this question.
StatusUpdatesQueueSubscription: # accept msg iff destSQS is "StatusUpdates"
Type: AWS::SNS::Subscription
Properties:
Endpoint: !Join [ "-", [ !Sub "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:${AWS::StackName}", !Ref StatusUpdatesQueueName ] ]
Protocol: sqs
FilterPolicy:
destSQS:
- "StatusUpdate"
RawMessageDelivery: True
TopicArn: !Ref DispatcherSNSTopic
So far everything works with "simple" (i.e., non-FIFO) SNS/SQS.
The following is the relevant section of the current OpenAPI (3.0.0) configuration embedded in the SAM template.
/devices/{device_id}/status:
post:
parameters:
- in: path
name: device_id
description: String ID of a device
required: true
schema:
type: string
description: Device unique identifier
security:
- ApiKeyAuth: [ ]
requestBody:
description: A status update for a device
required: true
content:
application/json:
schema:
type: object
responses:
"202":
description: Accepted
x-amazon-apigateway-integration:
type: "aws"
httpMethod: "POST"
uri: !Sub "arn:aws:apigateway:${AWS::Region}:sns:action/Publish"
credentials: !GetAtt DispatcherSNSTopicAPIRole.Arn
requestParameters:
integration.request.querystring.Message: "method.request.body"
integration.request.querystring.TopicArn: !Sub "'${DispatcherSNSTopic}'"
integration.request.querystring.MessageAttributes.entry.1.Name: "'destSQS'"
integration.request.querystring.MessageAttributes.entry.1.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.1.Value.StringValue: "'StatusUpdate'"
responses:
default:
statusCode: 202
Now, I want to enrich this configuration to group messages by device_id (so that status updates for the same device will be processed in order).
I've already modified the template Resources section so that SQS queues and the topic are FIFO and tried to add the following mapping:
integration.request.querystring.MessageAttributes.entry.2.Name: "MessageGroupId"
integration.request.querystring.MessageAttributes.entry.2.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.2.Value.StringValue: method.request.path.device_id
but I get the following error:
Resource handler returned message: "Errors found during import:
Unable to put integration on 'POST' for resource at path '/devices/{device_id}/status': Invalid mapping expression specified: Validation Result: warnings : [], errors : [Invalid mapping expression specified: MessageGroupId]
What am I missing?
After lots of searches and attempts, I've found the solution.
Perhaps sharing is useful for someone in the future.
MessageGroupId has to be set as a child of querystring and not as a MessageAttribute, as follows:
integration.request.querystring.MessageGroupId: "method.request.path.device_id"
the error was due to a syntax error: static strings must be enclosed by single quotes (and possibly, in turn, by double ones)
integration.request.querystring.MessageAttributes.entry.2.Name: "'MyAttributeName'"
I am using a CloudFormation template in YML format.
Depending on the environment, I need to be able to use different URLs for the Allowed Origins attribute of my CorsConfiguration. Ideally, I would like to use a Parameter defined like this:
AllowedOrigins:
Description: Allowed Origins
Type: String
AllowedPattern: '.+'
I have tried to pass in a delimited string (i.e. "http://localhost:4200,http://localhost:4201"), and split the values like this:
OnboardingHttpApi:
Type: AWS::Serverless::HttpApi
Properties:
CorsConfiguration:
AllowOrigins: !Split [ ",", !Ref AllowedOrigins ]
The response in CloudFormation is:
Warnings found during import: CORS Scheme is malformed, ignoring. (Service: AmazonApiGatewayV2; Status Code: 400; Error Code: BadRequestException; Request ID: 21072c02-70c3-473d-9629-784005226bd4; Proxy: null) (Service: null; Status Code: 404; Error Code: BadRequestException; Request ID: null; Proxy: null)
This is the answer I got from AWS Support:
The Split function is for splitting strings into a list, but it is not for referencing an attribute. It is designed to be used with the Select function or other functions. So it is not a stand-alone function to be used for referencing. For this, you can use the CommaDelimitedList parameter type. You can use the CommaDelimitedList parameter type to specify multiple string values in a single parameter. Once you pass the CommaDelimitedList parameter value, you can reference it later on your template. Here is a CloudFormation template that works:
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Parameters:
AllowedOriginsURLs:
Type: CommaDelimitedList
Default: 'https://example.com,https://example2.com'
Description: Please enter your URLs
Resources:
HttpApi:
Type: 'AWS::Serverless::HttpApi'
Properties:
StageName: my-stage-name
Tags:
Tag: MyTag
StageVariables:
StageVar: Value
CorsConfiguration:
AllowOrigins: !Ref AllowedOriginsURLs
AllowHeaders: [ x-apigateway-header ]
AllowMethods: [ GET ]
MaxAge: 600
AllowCredentials: true
The AllowedOriginsURLs parameter is of type CommaDelimitedList, with the default being 'http://localhost:4200,http://localhost:4201'. You can change this parameter on startup, then you can reference AllowedOriginsURLs on AllowOrigins.
I want to integrate a lambda function to an API gateway so that when I do a POST to the path defined on the API gateway the Lambda gets executed and returns a value.
Now, in my current project we currently have 2 Cloudformation templates:
- One that is generic and contains the definition of recourses common to all development environments
- And another non-generic one that deploys different resources depending on the environment passed as a paramater (dev, prod, etc).
In the non generic I have defined the lambda function, as well as the Deployment and Stage.
In the generic CF template I have defined the APIGateway as this:
Resources:
RecargakiApiGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Body:
swagger: "2.0"
info:
version: "VersionTimestamp"
title: "Some API"
host: "some.host"
schemes:
- "https"
paths:
/payment:
post:
produces:
- "application/json"
responses:
"200":
description: "200 response"
schema:
$ref: "#/definitions/Empty"
x-amazon-apigateway-integration:
credentials:
Fn::ImportValue:
"lambda-credentials-outputvalue-in-another-generic-stack"
uri: !Sub
- "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda::${AWS::Region}:${AWS::AccountId}:function:${lambdaName}/invocations"
- lambdaName: "${stageVariables.lambda_name}"
responses:
default:
statusCode: "200"
passthroughBehavior: "when_no_match"
timeoutInMillis: 5000
httpMethod: "POST"
contentHandling: "CONVERT_TO_TEXT"
type: "aws_proxy"
definitions:
Empty:
type: "object"
title: "Empty Schema"
Description: "Desc"
Name: "Name"
EndpointConfiguration:
Types:
- REGIONAL
Now according to the docs on AWS proxy integration it is possible to construct the lambda uri like this:
arn:aws:apigateway:<region>:lambda:path/2015-03-31/functions/arn:aws:lambda::<account_id>:function:${stageVariables.<function_variable_name>}/invocations
I tried to construct the uri according to that format but I get this error:
Unable to put integration on 'POST' for resource at path '/payment': Invalid function ARN or invalid uri (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException
In all the examples I've seen, people assume the Lambda function to be integrated is located in the same CF template/stack as the API Gateway resource, but I have no idea how to reference the function's ARN and/or name in the Swagger definition of the API Gateway when the function is actually located in a different CF stack.
I've also tried referencing a stage variable that would contain the full URI like:
uri: "${stageVariables.functionURI}" # the URI would be constructed in the non generic stack in the format stated by the docs
But it fails with:
Unable to put integration on 'POST' for resource at path '/payment': Invalid ARN specified in the request (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException
Any help is greatly appreciated.
I've been trying to create a method in API Gateway that drops the body of the incoming request into a file/object in an S3 bucket. But I want the method to create a new file each time (so a file with a new name) instead of overwriting the previous one. But I've been struggling to find a way to do it. Anyone has any suggestions/ideas? Something like a timestamp, or a sequence number (or both) to use as a variable in the Path override so that it would become the name of the s3 file. I looked at suggestions to use the X-Amzn-Trace-Id but it doesn't seem to be available in Path override. Anything else I could try? Maybe something in Swagger? I want to achieve it using API Gateway (avoid using a lambda as an extra step) to keep our architecture from getting too complex. Thanks in advance!
You can set the X-Amzn-Trace-Id as the method parameter, then map the parameter to the integration parameter on the path to S3 as the object name .
Example:
---
swagger: "2.0"
info:
version: "2017-12-04T23:03:26Z"
title: "API"
host: "xxxx.execute-api.us-east-1.amazonaws.com"
basePath: "/dev"
schemes:
- "https"
paths:
/:
post:
produces:
- "application/json"
parameters:
- name: "X-Amzn-Trace-Id"
in: "header"
required: false
type: "string"
responses:
200:
description: "200 response"
schema:
$ref: "#/definitions/Empty"
x-amazon-apigateway-integration:
credentials: "arn:aws:iam::178779171625:role/api-gate-way"
responses:
default:
statusCode: "200"
requestParameters:
integration.request.path.traceid: "method.request.header.X-Amzn-Trace-Id"
uri: "arn:aws:apigateway:us-east-1:bucketName.s3:path/{traceid}"
passthroughBehavior: "when_no_match"
httpMethod: "PUT"
type: "aws"
definitions:
Empty:
type: "object"
title: "Empty Schema"
I'm using cloudformation, and have enabled caching like this:
Type: AWS::ApiGateway::Stage
Properties:
CacheClusterEnabled: true
CacheClusterSize: 0.5
My methods are taken from swagger. Here's my swagger file for GET
/v1/myPath/{id}:
get:
tags:
- Books
operationId: getBook
parameters:
- name: id
in: path
description: The ID of the book to retrieve
type: integer
format: int32
required: true
- name: custom-header
in: header
type: string
responses:
'200':
x-amazon-apigateway-integration:
type: aws_proxy
httpMethod: POST
uri: arn:aws:apigateway:us-west-1:lambda:path...
passthroughBehavior: when_no_match
requestParameters:
integration.request.header.custom-header: method.request.header.custom-header
cacheKeyParameters:
- method.request.header.custom-header
How can I set the caching TTL for my GET request in my swagger.yaml? I can't seem to find documentation for this.
Referring to documentation,
CacheTtlInSeconds: Integer
CacheTtlInSeconds
The time-to-live (TTL) period, in seconds, that specifies how long API Gateway caches responses.
Required: No
Type: Integer
Reference:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-apigateway-deployment-stagedescription.html
Hope it helps.
EDIT1:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-apitgateway-method-methodresponse.html#cfn-apigateway-method-methodresponse-responseparameters
Cache-Control Documentation:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
Way to add headers to cloudformation,
ResponseParameters
Response parameters that API Gateway sends to the client that called a method. Specify response parameters as key-value pairs (string-to-Boolean maps), with a destination as the key and a Boolean as the value. Specify the destination using the following pattern: method.response.header.name, where the name is a valid, unique header name. The Boolean specifies whether a parameter is required.
Required: No
Type: Mapping of key-value pairs