Use an existing lambda function as datasource in Terraform - amazon-web-services

I'm trying to use an existing Lambda function as a data source and create an EC2 instance. This Lambda function essentially provides the latest AMI.
I'm looking at this doc:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lambda_invocation
Source Block:
data "aws_lambda_invocation" "example" {
function_name = aws_lambda_function.resource_selector.ResourceSelector
input = <<JSON
{
"key1": "AMIRegexPattern"
}
JSON
}
output "result_entry" {
value = jsondecode(data.aws_lambda_invocation.example.result)["key1"]
}
It throws this error and I'm a little lost:
Error: Reference to undeclared resource
on create-ec2.tf line 26, in data "aws_lambda_invocation" "example":
26: function_name = aws_lambda_function.resource_selector.ResourceSelector
A managed resource "aws_lambda_function" "resource_selector" has not been
declared in the root module.
Here is the function details:
Function Name - ResourceSelector
Function ARN : arn:aws:lambda:us-east-1:xx50:function:ResourceSelector
Any help on what I am missing? Also curious on this line esp and if this is correct:
function_name = aws_lambda_function.resource_selector.ResourceSelector
Thanks

If the lambda is created outside terraform you have to hardcode or pass via parameter, like so:
function_name = "ResourceSelector"
aws_lambda_function.resource_selector doesn't exist, or you can manage the lambda defininng the aws_lambda_function resource.
Also if you just want to get the latest AMI you don't need a lambda. Terraform actually has a data source that can pull that for you: aws_ami
Example:
data "aws_ami" "example" {
most_recent = true
owners = ["self"]
filter {
name = "name"
values = ["myami-*"]
}
}

Related

Terraform Update an existing Lambda function was created on AWS console

I want to update a my code on Lambda function.
I previous creating lambda function on aws console.
I do a creating terraform code for update my function existing but I received an error
I try using this block code
data "archive_file" "stop_ec2" {
type = "zip"
source_file = "src_dir/stop_ec2.py"
output_path = "dest_dir/stop_ec2_upload.zip"
}
data "aws_lambda_function" "existing" {
function_name = MyPocLambda
role = aws_iam_role.iam_for_lambda.arn
filename = "dest_dir/stop_ec2_upload.zip"
source_code_hash ="${data.archive_file.stop_ec2.output_base64sha256}"
}
My error says filename unsupported argument, filename is not expected here
it possible update lambda function using terafform data ?
You have to import your lambda function to TF first. Then you will be able to modify it using TF code using aws_lambda_function resource, not data source.

How to use AWS account_id variable in Terraform

I want access to my AWS Account ID in terraform. I am able to get at it with aws_caller_identity per the documentation. How do I then use the variable I created? In the below case I am trying to use it in an S3 bucket name:
data "aws_caller_identity" "current" {}
output "account_id" {
value = data.aws_caller_identity.current.account_id
}
resource "aws_s3_bucket" "test-bucket" {
bucket = "test-bucket-${account_id}"
}
Trying to use the account_id variable in this way gives me the error A reference to a resource type must be followed by at least one attribute access, specifying the resource name. I expect I'm not calling it correctly?
If you have a
data "aws_caller_identity" "current" {}
then you need to define a local for that value:
locals {
account_id = data.aws_caller_identity.current.account_id
}
and then use it like
output "account_id" {
value = local.account_id
}
resource "aws_s3_bucket" "test-bucket" {
bucket = "test-bucket-${local.account_id}"
}
Terraform resolves the locals based on their dependencies so you can create locals that depend on other locals, on resources, on data blocks, etc.
Any time you create a datasource in terraform , it will export some attributes related to that datasource so that you can reference it somewhere else in your configuration and interpolate it with various ways.
In your case, you are already referencing the value of your account id in output block
So that same way, you can construct the string for the bucket name as follows.
resource "aws_s3_bucket" "test-bucket" {
bucket = "test-bucket-${data.aws_caller_identity.current.account_id}"
}
I would highly recommend you go through the terrraform syntax which can help you better understand the resource, datasource and expressions
https://www.terraform.io/docs/language/expressions/references.html

JSON Syntax error in Terraform aws_iam_role_policy

So with Terraform, I'm creating an IAM policy and attaching it to a role. I am currently running:
Terraform v0.12.16
provider.aws v2.40.0
provider.template v2.1.2
When executing the code, I am able to initialize terraform with no issue. When running terraform plan, I am being presented with the following error:
Error: "policy" contains an invalid JSON: invalid character '}' looking for beginning of value
on ec2-iam.tf line 8, in resource "aws_iam_role_policy" "s3_ec2_policy":
8: resource "aws_iam_role_policy" "s3_ec2_policy" {
I am getting stuck with this error. Any advice would be helpful. Below is my code:
data "template_file" "s3_web_policy" {
template = file("scripts/iam/web-ec2-policy.json")
vars = {
s3_bucket_arn = "arn:aws:s3:::${var.my_app_s3_bucket}/*"
}
}
resource "aws_iam_role_policy" "s3_ec2_policy" {
name = "s3_ec2_policy"
role = aws_iam_role.s3_ec2_role.id
policy = data.template_file.s3_web_policy.rendered
}
resource "aws_iam_role" "s3_ec2_role" {
name = "s3_ec2_role"
assume_role_policy = file("scripts/iam/web-ec2-assume-role.json")
}
It's common to encounter syntax errors when generating JSON from string templates, because the template language isn't aware of JSON syntax and so you as the tempate author must take care to ensure that brackets are all nested correctly, there are no missing or extra commas, etc.
You can usually avoid problems of this type by generating JSON with Terraform's jsonencode function instead:
resource "aws_iam_role_policy" "s3_ec2_policy" {
name = "s3_ec2_policy"
role = aws_iam_role.s3_ec2_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
# etc, etc
]
})
}
If the policy definition seems too large to include inline in your resource block, you can still factor it out into a separate template file if desired:
resource "aws_iam_role_policy" "s3_ec2_policy" {
name = "s3_ec2_policy"
role = aws_iam_role.s3_ec2_role.id
policy = templatefile("${path.module}/scripts/iam/web-ec2-policy.json.tmpl", {
s3_bucket_arn = "arn:aws:s3:::${var.my_app_s3_bucket}/*"
})
}
...but inside the template, rather than using individual template interpolations, just write the entire template as a single call to jsonencode, like this:
${jsonencode({
Version = "2012-10-17"
Statement = [
{
# ...
Resource = s3_bucket_arn
# ...
},
# etc, etc
]
})}
Note that the template_file data source is for Terraform 0.11 and earlier, and is in Terraform 0.12 only for backward compatibility. You should use the templatefile function instead, which serves the same purpose but is integrated directly into the Terraform language.

How to set up a lambda alias with the same event source mapping as the LATEST/Unqualified lambda function in terraform

I'm trying to create a lambda alias for my lambda function using terraform. I've been able to successfully create the alias but the created alias is missing the dynamodb as the trigger.
how the event source is set up
resource "aws_lambda_event_source_mapping" "db_stream_trigger" {
batch_size = 10
event_source_arn = "${data.terraform_remote_state.testddb.table_stream_arn}"
enabled = true
function_name = "${aws_lambda_function.test_lambda.arn}"
starting_position = "LATEST"
}
how the alias is created
resource "aws_lambda_alias" "test_lambda_alias" {
count = "${var.create_alias ? 1 : 0}"
depends_on = [ "aws_lambda_function.test_lambda" ]
name = "test_alias"
description = "alias for my test lambda"
function_name = "${aws_lambda_function.test_lambda.arn}"
function_version = "${var.current_running_version}"
routing_config = {
additional_version_weights = "${map(
"${aws_lambda_function.test_lambda.version}", "0.5"
)}"
}
}
The lambda works with the dynamodb stream as a trigger
The Alias for the lambda is successfully created.
The Alias is using the correct version
The Alias is using the correct weight
The Alias is NOT using the dynamo-db stream as the event source
I had the wrong function_name for the resource "aws_lambda_event_source_mapping". I was providing it the main lambda function's arn as oppose to the alias lambda function's arn. Once i switched it to the alias's arn, I was able to successfully divide the traffic from the stream dependent on the weight!
From aws doc:
Simplify management of event source mappings – Instead of using Amazon Resource Names (ARNs) for Lambda function in event source mappings, you can use an alias ARN. This approach means that you don't need to update your event source mappings when you promote a new version or roll back to a previous version.
https://docs.aws.amazon.com/lambda/latest/dg/aliases-intro.html

Terraform not uploading a new ZIP

I want to use Terraform for deployment of my lambda functions. I did something like:
provider "aws" {
region = "ap-southeast-1"
}
data "archive_file" "lambda_zip" {
type = "zip"
source_dir = "src"
output_path = "build/lambdas.zip"
}
resource "aws_lambda_function" "test_terraform_function" {
filename = "build/lambdas.zip"
function_name = "test_terraform_function"
handler = "test.handler"
runtime = "nodejs8.10"
role = "arn:aws:iam::000000000:role/xxx-lambda-basic"
memory_size = 128
timeout = 5
source_code_hash = "${data.archive_file.lambda_zip.output_base64sha256}"
tags = {
"Cost Center" = "Consulting"
Developer = "Jiew Meng"
}
}
I find that when there is no change to test.js, terraform correctly detects no change
No changes. Infrastructure is up-to-date.
When I do change the test.js file, terraform does detect a change:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
~ aws_lambda_function.test_terraform_function
last_modified: "2018-12-20T07:47:16.888+0000" => <computed>
source_code_hash: "KpnhsytFF0yul6iESDCXiD2jl/LI9dv56SIJnwEi/hY=" => "JWIYsT8SszUjKEe1aVDY/ZWBVfrZYhhb1GrJL26rYdI="
It does zip up the new zip, however, it does not seem to update the function with the new ZIP. It seems like it thinks since the filename has no change, it does not upload ... How can I fix this behaviour?
=====
Following some of the answers here, I tried:
Using null_resource
Using S3 bucket/object with etag
And it does not update ... Why is that?
I ran into the same issue and what solved it for me was publishing the Lambda functions automatically using the publish argument. To do so simply set publish = true in your aws_lambda_function resource.
Note that your function will be versioned after this and each change will create a new one. Therefor you should make sure that you use the qualified_arn attribute reference if you're referring to the function in any of your other Terraform code.
There is a workaround to trigger the resource to be refreshed, if the target lambda file names are src/main.py and src/handler.py. If you have more files to be managed, add them one by one.
resource "null_resource" "lambda" {
triggers {
main = "${base64sha256(file("src/main.py"))}"
handler = "${base64sha256(file("src/handler.py"))}"
}
}
data "archive_file" "lambda_zip" {
type = "zip"
source_dir = "src"
output_path = "build/lambdas.zip"
depends_on = ["null_resource.lambda"]
}
Let me know if this works for you.
There is 2 things you need to take care of:
upload zip file to S3 if its content has changed
update Lambda function if zip file content has changed
I can see you are taking care of the latter with source_code_hash. I don't see how you handle the former. It could look like that:
resource "aws_s3_bucket_object" "zip" {
bucket = "${aws_s3_bucket.zip.bucket}"
key = "myzip.zip"
source = "${path.module}/myzip.zip"
etag = "${md5(file("${path.module}/myzip.zip"))}"
}
etag is the most important option here.
I created this module to help ease some of the issues around deploying Lambda with Terraform: https://registry.terraform.io/modules/rojopolis/lambda-python-archive/aws/0.1.4
It may be useful in this scenario. Basically, it replaces the "archive_file" data source with a specialized lambda archive data source to better manage stable source code hash, etc.