AWS manages a layer called AWSDataWrangler-Python38. How do I import it into my Terraform Code. I tried using the Layer Module
resource "aws_lambda_layer_version" "lambda_layer" {
layer_name = "AWSDataWrangler-Python39"
compatible_runtimes = ["python3.9"]
}
It throws an error to specify filename, but there is no file for this layer since it is managed by AWS and it is not a custom layer
You can not import a resource which is not managed by you.
Since this is a layer managed by AWS, there is a public list with all the ARN numbers available for this layer: https://aws-data-wrangler.readthedocs.io/en/stable/layers.html
If you want to use this layer for a Lambda in your Terraform code, you will have to take an ARN from this list and simply hard-code it (or provide it externally with a variable). For example:
resource "aws_lambda_function" "lambda" {
function_name = "MyFunction"
...
layers = [
"arn:aws:lambda:${var.region}:336392948345:layer:AWSDataWrangler-Python39:6"
]
}
Related
I am working on a Python and AWS course on coursera. However instead of creating s3 , api gateway and others using boto3 I am using terraform. Till now everything is going fine however I am facing a issue.Below is my lambda directory structure
Every lambda has a different directory structure and I have to cd in each directory to apply changes using terraform apply.
Example
Below is my lambda code in terraform for one of the lambda function.<<validate.tf>>
provider "aws" {
region = "us-east-2"
}
terraform {
required_version = "> 0.14"
required_providers {
aws = "~> 3.0"
}
backend "s3" {
bucket = "nyeisterraformstatedata2"
key = "api_gateway/lambda_function/terraform_api_gateway_lambda_validate.tfstate"
region = "us-east-2"
dynamodb_table = "terraform-up-and-running-locks-2"
encrypt = true
}
}
data "archive_file" "zip_file" {
type = "zip"
source_dir = "${path.module}/lambda_dependency_and_function"
output_path = "${path.module}/lambda_dependency_and_function.zip"
}
resource "aws_lambda_function" "get_average_rating_lambda" {
filename = "lambda_dependency_and_function.zip"
function_name = "validate"
role = data.aws_iam_role.lambda_role_name.arn
handler = "validate.lambda_handler"
# The filebase64sha256() function is available in Terraform 0.11.12 and later
# For Terraform 0.11.11 and earlier, use the base64sha256() function and the file() function:
# source_code_hash = "${base64sha256(file("lambda_function_payload.zip"))}"
source_code_hash = filebase64sha256(data.archive_file.zip_file.output_path)
runtime = "python3.8"
depends_on = [data.archive_file.zip_file]
}
<<variable.tf>>
data "aws_iam_role" "lambda_role_name" {
name = "common_lambda_role_s3_api_gateway_2"
}
Based on the comment below I created a main.tf with following code
provider "aws" {
region = "us-east-2"
}
module "test" {
source = "../validate"
}
but I am trying to import using import statement its giving me an error and I am not able to figure out how to solve it
terraform import module.test.aws_lambda_function.test1 get_average_rating_lambda
Warning: Backend configuration ignored
│
│ on ../validate/validate.tf line 10, in terraform:
│ 10: backend "s3" {
│
│ Any selected backend applies to the entire configuration, so Terraform expects provider configurations only in the root module.
│
│ This is a warning rather than an error because it's sometimes convenient to temporarily call a root module as a child module for testing purposes, but this backend configuration block will have no
│ effect.
╵
Error: resource address "module.test.aws_lambda_function.test1" does not exist in the configuration.
Before importing this resource, please create its configuration in module.test. For example:
resource "aws_lambda_function" "test1" {
# (resource arguments)
}
So my question is there a way for terraform to tell which all files have change and apply them in one go rather than one by one.Since I am new to terraform too so if anyone think that this is the wrong way to structing the project please do let me know.Thank you
What you could do is create a new directory with a main.tf file and make it a project that contains your whole cloud environment. Each of these existing folders could be imported as a module. If each of your folders is a running terraform project, it can already be imported as a module without changing it.
You would then use the terraform import command to import each of the resources, in a fashion similar to terraform import module.aws_lambda_function my_lambda_id for each lambda and any other managed resources.
Then, instead of a state file for each lambda, you would have a state file for your whole environment. From there, terraform is smart enough to detect the individual changes and update accordingly.
I created an AWS environment using TERRAFORM.
After that, some resources were created by console (SES, SNS, LAMBDA) they did not was provisioned by TERRAFORM.
I'm writing the TERRAFORM code for these resources (SES, SNS, LAMBDA) that were created by the console.
If I already have these resources running in my account, is it possible to generate this code via TERRAFORM for these resources without removing them?
Or even, how do I have to proceed in this case?
Welcome to the world of IaC, you're in for a treat. :)
You can import all resources that were created without terraform (using a CLI or manually provisioned - resources which are not part of the tf state) to your terraform state. Once these resources are imported you can then start managing their lifecycle using terraform.
Define the resource in your .tf files
Import existing resources
As an example:
In order to import an existing non terraform managed lambda, you first define the resource for it in your .tf files:
main.tf:
resource "aws_lambda_function" "test_lambda" {
filename = "lambda_function_payload.zip"
function_name = "lambda_function_name"
role = "${aws_iam_role.iam_for_lambda.arn}"
handler = "exports.test"
# The filebase64sha256() function is available in Terraform 0.11.12 and later
# For Terraform 0.11.11 and earlier, use the base64sha256() function and the file() function:
# source_code_hash = "${base64sha256(file("lambda_function_payload.zip"))}"
source_code_hash = "${filebase64sha256("lambda_function_payload.zip")}"
runtime = "nodejs12.x"
environment {
variables = {
foo = "bar"
}
}
}
Then you can execute terraform import, in order to import the existing lambda:
terraform import aws_lambda_function.test_lambda my_test_lambda_function
I am trying to run a python function on an AWS Lambda layer, I don't find any documentation on terraform to use an AWS provided lambda layer. How do I use AWS Provided lambda layer AWSLambda-Python27-SciPy1x and runtime Python 2.7?
#----compute/lambda.tf----
data "archive_file" "lambda_zip" {
type = "zip"
source_file = "index.py"
output_path = "check_foo.zip"
}
resource "aws_lambda_function" "check_foo" {
filename = "check_foo.zip"
function_name = "checkFoo"
role = "${aws_iam_role.iam_for_lambda_tf.arn}"
handler = "index.handler"
source_code_hash = "${data.archive_file.lambda_zip.output_base64sha256}"
# i want to use lambda layer - AWSLambda-Python27-SciPy1x and run this function on it
runtime = "python2.7"
}
You have to specify lambda layers as ARNs in terraform using layers parameter:
layers - (Optional) List of Lambda Layer Version ARNs (maximum of 5) to attach to your Lambda Function.
Using the following syntax in terraform:
layers = ["layer-arn"]
For example, the ARN for AWSLambda-Python27-SciPy1x in us-east-1 region is:
arn:aws:lambda:us-east-1:668099181075:layer:AWSLambda-Python27-SciPy1x:24
If you not sure what is your ARN, you can create a dummy a Python 2.7 lambda function, add AWS layer AWSLambda-Python27-SciPy1x layer, and the console will give you its ARN.
I am facing an issue in terraform where I want to read details of some existing resource (r1) created via AWS web console.
I am using those details in creation on new resource (r2) via terraform.
Problem is that it is trying to destroy and recreate that resource which is not desired as it will be failed. How can I manage not to destroy and recreate r1 when I do terraform apply.
Here is how I am doing it :
main.tf
resource "aws_lb" "r1"{
}
...
resource "aws_api_gateway_integration" "r2" {
type = "HTTP"
uri = "${aws_lb.r1.dns_name}}/o/v1/multi/get/m/content"
}
first I import that resource
terraform import aws_lb.r1 {my_arn}
next I apply terraform
terraform apply
error
aws_lb.r1: Error deleting LB: ResourceInUse: Load balancer 'my_arn' cannot be deleted because it is currently associated with another service
The import statement is meant for taking control over existing resources in your Terraform setup.
If your only intention is to derive information on existing resources (outside of your Terraform control), data sources are designed specifically for this need:
data "aws_lb" "r1" {
name = "lb_foo"
arn = "some_specific_arn" #you can use any selector you wish to query the correct LB
}
resource "aws_api_gateway_integration" "r2" {
type = "HTTP"
uri = "${data.aws_lb.r1.dns_name}/o/v1/multi/get/m/content"
}
You can add a lifecycle configuration block in the resource "aws_lb" "r1" (see: https://www.terraform.io/docs/configuration/resources.html#lifecycle) to tell Terraform to ignore changes in the resource.
I guess something like this should work:
resource "aws_lb" "r1"{
lifecycle {
ignore_changes = ["*"]
}
}
I'm trying to import a terraform aws_iam_policy that gets automatically added by automation I don't own. The import seems to work but once I run a terraform plan I get the following error
* aws_iam_policy.mypolicy1: "policy": required field is not set
I'm running the terraform import as follows.
terraform import aws_iam_policy.mypolicy1 <myarn>
Here is my relevant terraform config
resource "aws_iam_policy" "mypolicy1" {
}
resource "aws_iam_role_policy_attachment" "mypolicy1_attachment`" {
role = "${aws_iam_role.myrole1.name}"
policy_arn = "${aws_iam_policy.mypolicy1.arn}"
}
resource "aws_iam_role" "myrole1" {
name = "myrole1"
assume_role_policy = "${file("../policies/ecs-role.json")}"
}
I double checked that the terraform.tfstate included the policy i'm trying to import. Is there something else I'm missing here?
You still need to provide the required fields in the Terraform configuration for the plan to work.
If you remove the aws_iam_policy resource from your configuration and run a plan after importing the policy you should see that Terraform wants to destroy the policy because it is in the state file but not in the configuration.
Simply setup your aws_iam_policy resource to match the imported policy and then a plan should show no changes.
I finally found a relatively elegant, and universal work-around to address Amazon's poor implementation of the import IAM policy capability. The solution does NOT require that you reverse engineer Amazon, or anybody else's, implementation of the "aws_iam_policy" resource that you want to import.
There are two steps.
Create an aws_iam_policy resource definition that has a "lifecycle" argument, with an ignore_changes list. There are three fields in the aws_iam_policy resource that will trigger a replacement: policy, description and path. Add these three fields to the ignore_changes list.
Import the external IAM policy, and attach it to the resource definition that you created in your resource file.
Resource file (ex: static-resources.tf)
resource "aws_iam_policy" "MyLambdaVPCAccessExecutionRole" {
lifecycle {
prevent_destroy = true
ignore_changes = [policy, path, description]
}
policy = jsonencode({})
}
Import Statement: Using the arn of the IAM policy that you want to import, import the policy and attach it to your resource definition.
terraform import aws_iam_policy.MyLambdaVPCAccessExecutionRole arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
The magic is the fields that you need to add to the ignore_changes list, and adding a place-holder for the required "policy" argument. Since this is a required field, Terraform won't let you proceed without it, even though this is one of the fields that you told Terraform to ignore any changes to.
Note: If you use modules, you will need to add "module.." to the front on your resource reference. For example
terraform import module.static.aws_iam_policy.MyLambdaVPCAccessExecutionRole arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole