I am working on a feature where we are uploading data to aws cloudsearch domain.
also to utilize this aws cloudsearch domain I need to create one api from aws cloudsearch domain.
I have gone through this blog which explains about how we can use aws cloudsearch with api : https://blog.devgenius.io/autocomplete-with-amazon-cloudsearch-8503b606c90d
The problem is I need to create APIgateway from terraform scripts and there is no particular documentation or reference available for me to look into how can we write terraform script to integrate APIgateway with AWS CLOUDSEARCH domain.
so far I have gone through terraform official documentation and have found below steps :
resource "aws_api_gateway_rest_api" "MyDemoAPI" {
name = "MyDemoAPI"
description = "This is my API for demonstration purposes"
}
resource "aws_api_gateway_resource" "MyDemoResource" {
rest_api_id = aws_api_gateway_rest_api.MyDemoAPI.id
parent_id = aws_api_gateway_rest_api.MyDemoAPI.root_resource_id
path_part = "mydemoresource"
}
resource "aws_api_gateway_method" "MyDemoMethod" {
rest_api_id = aws_api_gateway_rest_api.MyDemoAPI.id`enter code here`
resource_id = aws_api_gateway_resource.MyDemoResource.id
http_method = "GET"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "MyDemoIntegration" {
rest_api_id = aws_api_gateway_rest_api.MyDemoAPI.id
resource_id = aws_api_gateway_resource.MyDemoResource.id
http_method = aws_api_gateway_method.MyDemoMethod.http_method
type = "MOCK"
cache_key_parameters = ["method.request.path.param"]
cache_namespace = "foobar"
timeout_milliseconds = 29000
request_parameters = {
"integration.request.header.X-Authorization" = "'static'"
}
# Transforms the incoming XML request to JSON
request_templates = {
"application/xml" = <<EOF
{
"body" : $input.json('$')
}
EOF
}
}
but i am not sure how will it help me in my implementation.
Related
I have to integrate more than 50 Lambda functions with relevant api gateway method. So that I create a terraform map with Lambda function name and api gateway resource path as below.
variable "lambdas" {
description = "Map of Lambda function names and API gateway resource paths."
type = map
default = {
user = {
name = "user-lambda-function"
path = "user"
},
products= {
name = "products-lambda-function"
path = "products"
},
orders= {
name = "orders-lambda-function"
path = "orders"
},
Then I iterate lambda function creation through this map using for_each as below.
resource "aws_lambda_function" "lambda_functions" {
for_each = var.lambdas
filename = "lambda_function_code.zip"
function_name = each.value.name
role = data.aws_iam_role.lambda_execution_role.arn
handler = "index.handler"
source_code_hash = filebase64sha256("lambda_function_code.zip")
runtime = "nodejs14.x"
}
After that I start to create API Gateway, resources and methods as below,
resource "aws_api_gateway_rest_api" "api_gateway" {
name = var.api-gateway-name
}
resource "aws_api_gateway_resource" "resources" {
for_each = var.lambdas
rest_api_id = aws_api_gateway_rest_api.api_gateway.id
parent_id = aws_api_gateway_rest_api.api_gateway.root_resource_id
path_part = each.value.path
}
resource "aws_api_gateway_method" "methods" {
for_each = aws_api_gateway_resource.resources
rest_api_id = aws_api_gateway_rest_api.api_gateway.id
resource_id = each.value.id
http_method = "POST"
authorization = "NONE"
api_key_required = false
}
Then I try to integrate above API Gateway method with relevant Lambda function by iterating above above methods. But here I have to input relevant lambda function invocation uri.
resource "aws_api_gateway_integration" "integration" {
for_each = aws_api_gateway_method.methods
rest_api_id = each.value.rest_api_id
resource_id = each.value.resource_id
http_method = each.value.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = ""
}
I am struggling to input this relevant lambda function uri to integrate with relevant method.
My question is, How do I get relevant lambda function uri to input here with this iteration? Or any solution to achieve this without code every lambdas, resources and methods.
The aws_lambda_function has invoke_arn which is:
ARN to be used for invoking Lambda Function from API Gateway - to be used in aws_api_gateway_integration's uri
So you have to use that in your aws_api_gateway_integration.
I want to add a header (Content-Type) and a mapping template within the integration request of my AWS API Gateway. While following this tutorial, I could easily add those parameters via the console. https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html
However, I am unable to figure out the syntax to add them using Terraform. This is my integration request so far:
resource "aws_api_gateway_integration" "kinesis_integration" {
rest_api_id = aws_api_gateway_rest_api.shippeo_api_kinesis.id
resource_id = aws_api_gateway_resource.shippeo_api_resource_kinesis.id
http_method = aws_api_gateway_method.post_json_files_kinesis.http_method
credentials = aws_iam_role.shippeo_integration_role.arn
type = "AWS"
uri = "arn:aws:apigateway:eu-central-1:kinesis:action/PutRecord"
integration_http_method = "POST"
depends_on = [
aws_api_gateway_resource.shippeo_api_resource_kinesis
]
}
How can I specify the header (Content-Type)+Mapped From and the mapping templates as shown below using Terraform?
Mapping Template:
{
"StreamName": "$input.params('stream-name')",
"Data": "$util.base64Encode($input.json('$.Data'))",
"PartitionKey": "$input.path('$.PartitionKey')"
}
In the documentation, selection_pattern and response_templates is mentioned but I am not sure how to use it in my case.
I have multiple Apigateway trigger a lambda. They can invoke lambda normally but when I want to see triggers of lambda after deployed by terraform or serverless framework, AWS often throws 429 error An error occurred while listing api-gateway relations: Too Many Requests. Can anyone shed any light please how can I fix it?
Sample
Here is my config:
data "aws_lambda_function" "lambda" {
function_name = var.function_name
}
locals {
resource_path = replace(var.resource_path, "/{.*}/", "*")
}
resource "aws_api_gateway_integration" "integration" {
rest_api_id = var.api_id
resource_id = var.resource_id
http_method = aws_api_gateway_method.method.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = data.aws_lambda_function.lambda.invoke_arn
}
# Create Lambda trigger to the method
resource "aws_lambda_permission" "allow_api" {
action = "lambda:InvokeFunction"
function_name = var.function_name
principal = "apigateway.amazonaws.com"
source_arn = "${var.api_arn}/*/${var.http_method}${local.resource_path}"
}
I have the following terraform script which creates an API gateway that passes requests to a lambda function.
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
#
region = "${var.region}"
version = "~> 2.6"
}
resource "aws_api_gateway_rest_api" "MyDemoAPI" {
name = "MyDemoAPI"
description = "This is my API for demonstration purposes"
}
resource "aws_api_gateway_resource" "MyDemoResource" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
parent_id = "${aws_api_gateway_rest_api.MyDemoAPI.root_resource_id}"
path_part = "mydemoresource"
}
resource "aws_api_gateway_method" "MyDemoMethod" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
http_method = "POST"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "MyDemoIntegration" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
http_method = "${aws_api_gateway_method.MyDemoMethod.http_method}"
integration_http_method = "POST"
type = "AWS_PROXY"
uri = "arn:aws:apigateway:ap-southeast-1:lambda:path/2015-03-31/functions/${aws_lambda_function.test_lambda_function.arn}/invocations"
content_handling = "CONVERT_TO_TEXT"
}
resource "aws_api_gateway_method_response" "200" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
http_method = "${aws_api_gateway_method.MyDemoMethod.http_method}"
status_code = "200"
response_models {
"application/json" = "Empty"
}
}
resource "aws_lambda_function" "test_lambda_function" {
filename = "lambda.zip"
description = "test build api gateway and lambda function using terraform"
function_name = "test_lambda_function"
role = "arn:aws:iam::123456789123:role/my_labmda_role"
handler = "gateway.lambda_handler"
runtime = "python3.6"
memory_size = 128
timeout = 60
}
The Method Response section of the API gateway resource display Select an integration response..
But if I create the same API gateway using AWS console, the Method Response section displays something different:
Why does this happen?
The following steps are how I use AWS console to create the API gateway:
Select Create Method under the resource.
Select POST method.
Select the desired options.
I've tried creating the above resources manually first, then execute terraform apply. Then terraform tells me that nothing needs to be changed.
terraform apply
aws_api_gateway_rest_api.MyDemoAPI: Refreshing state... (ID: 1qa34vs1k7)
aws_lambda_function.test_lambda_function: Refreshing state... (ID: test_lambda_function)
aws_api_gateway_resource.MyDemoResource: Refreshing state... (ID: 4xej81)
aws_api_gateway_method.MyDemoMethod: Refreshing state... (ID: agm-1qa34vs1k7-4xej81-POST)
aws_api_gateway_method_response.200: Refreshing state... (ID: agmr-1qa34vs1k7-4xej81-POST-200)
aws_api_gateway_integration.MyDemoIntegration: Refreshing state... (ID: agi-1qa34vs1k7-4xej81-POST)
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
This seems to mean that the manually built structure is the same as the structure built by terraform.
Because API Gateway is a complex AWS component and you can control pretty much everything on it (basically every single part of it is managed independently, giving you a lot of control over what you create but also making things harder to deal with).
See that it says "Select an Integration Response", but since your Terraform code didn't create one, it is therefore empty.
I had come across this very same problem a few weeks ago and I found the solution on Terraform's GitHub. I think Terraform should better document this as you're not the first one nor will you be the last to come up with this question. Well, at least we have this documented in StackOverflow now :)
Long story short, you need to add a aws_api_gateway_integration_response terraform resource to your API Gateway.
resource "aws_api_gateway_integration_response" "MyDemoIntegrationResponse" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
http_method = "${aws_api_gateway_method.MyDemoMethod.http_method}"
status_code = "${aws_api_gateway_method_response.200.status_code}"
response_templates = {
"application/json" = ""
}
}
If you can, however, I suggest you use a proper framework to hook events to your Lambda functions (like the Serverless Framework or AWS SAM) as it's very verbose and error prone to create them in Terraform.
Usually, I combine Terraform and Serverless Framework together: I use Terraform to create infrastructure resources - even if they are serverless - like DynamoDB tables, SQS queues, SNS topics, etc. and the Serverless Framework to create the Lambda functions and their corresponding events.
I had the same issue. What Thales suggested didn't work for me (using terraform 0.11.14 with AWS plugin 2.52.0).
Instead, I used api_gateway_method_response resource and set the response_models parameter as:
response_models = {
"application/json" = "Empty"
}
Then it showed me the default response that all other APIs that I created through the console show!
Not very clear in terraform docs what this should look like, had to figure out myself after many attempts.
In the Terraform documentation for AWS_API_GATEWAY_INTEGRATION, the following parameters are supported:
rest_api_id
resource_id
http_method
type
uri
integration_http_method
They give this example:
resource "aws_api_gateway_integration" "MyDemoIntegration" {
rest_api_id = "${aws_api_gateway_rest_api.MyDemoAPI.id}"
resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
http_method = "${aws_api_gateway_method.MyDemoMethod.http_method}"
type = "MOCK"
}
But I would like to specify a mapping template (as well as a Lambda integration), as you can with the UI:
However I see no way of doing that with Terraform. Is is possible ?
Note: What I am currently doing is applying the rest of the configuration (lambda, s3, iam etc...), and then adding mapping template manually afterwards (as well as the integration type of lambda).
But then every time I terraform apply to apply some other config (eg: s3), Terraform reverts back the mapping template and the integration type.
The "reverting" plan looks like this:
~ aws_api_gateway_integration.post_hit_integration
request_templates.#: "1" => "0"
request_templates.application/json: "{\n \"body\" : $input.json('$'),\n \"headers\": {\n #foreach($param in $input.params().header.keySet())\n \"$param\": \"$util.escapeJavaScript($input.params().header.get($param))\" #if($foreach.hasNext),#end\n \n #end \n },\n \"stage\" : \"$context.stage\"\n}" => ""
uri: "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:000000000000:function:create_hit/invocations" => ""
Based on this issue, here is a config that works:
(You have to use the params uri, type, integration_http_method and request_templates)
# API
resource "aws_api_gateway_rest_api" "my_api" {
name = "my_api"
description = "My Api for adding pets"
}
# Resource
resource "aws_api_gateway_resource" "pets_resource" {
rest_api_id = "${aws_api_gateway_rest_api.my_api.id}"
parent_id = "${aws_api_gateway_rest_api.my_api.root_resource_id}"
path_part = "pets"
}
# Method
resource "aws_api_gateway_method" "post_pet_method" {
rest_api_id = "${aws_api_gateway_rest_api.my_api.id}"
resource_id = "${aws_api_gateway_resource.pets_resource.id}"
http_method = "POST"
authorization = "NONE"
}
# Integration
resource "aws_api_gateway_integration" "post_pet_integration" {
rest_api_id = "${aws_api_gateway_rest_api.my_api.id}"
resource_id = "${aws_api_gateway_resource.pets_resource.id}"
http_method = "${aws_api_gateway_method.post_pet_method.http_method}"
uri = "arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${aws_lambda_function.create_pet.arn}/invocations"
type = "AWS" # Documentation not clear
integration_http_method = "POST" # Not documented
request_templates = { # Not documented
"application/json" = "${file("api_gateway_body_mapping.template")}"
}
}
And contents of api_gateway_body_mapping.template:
{
"body" : $input.json('$'),
"headers": {
#foreach($param in $input.params().header.keySet())
"$param": "$util.escapeJavaScript($input.params().header.get($param))" #if($foreach.hasNext),#end
#end
},
"stage" : "$context.stage"
}
If you want to have it inline rather than in a separate template you can do:
request_templates = {
"application/json" = <<REQUEST_TEMPLATE
{
"body" : $input.json('$'),
"headers": {
#foreach($param in $input.params().header.keySet())
"$param": "$util.escapeJavaScript($input.params().header.get($param))" #if($foreach.hasNext),#end
#end
},
"stage" : "$context.stage"
}
REQUEST_TEMPLATE
}
If you are using a Lambda function as the endpoint, the integration type would be "AWS".
Here is the AWS documentation that explains creating a Lambda integration.
Here is a GitHub post that shows how this can be done using Terraform.
Hope that helps! Let us know if you have any questions.
I had similar issue, I couldn't add the Mapping templates as my Integration type was "AWS_PROXY". I changed it to "AWS" and it worked.
Terraform wouldn't consider about mapping templates if the integration type is "AWS_PROXY"
You can learn more about Integration types here.