When I define terraform code for AWS resources I have not used before, I am sometimes unsure about how to define my components. I then often just apply any component and modify it using the AWS GUI. Then I use terraform plan to see my changes and adapt my code, so that it corresponds to the existing AWS infrastructure.
I wonder whether there is a more direct way to use the AWS GUI to define terraform code. Is there any way to just map an existing AWS infrastructure to terraform code?
What you want is terraform import
Terraform is able to import existing infrastructure. This allows you take resources you've created in aws via the GUI and bring it under Terraform management.
The current implementation of Terraform import can only import resources into the state file. It does not generate configuration. This will however enable you to then configure your resources
This is how to import an instance with ID i-abcd1234 to address aws_instance.bar
terraform import aws_instance.bar i-abcd1234
https://www.terraform.io/docs/import/
Related
My team has two completely different environments: a Terraform one (which allow us to create and manage some AWS resources as databases) and a CDK one, with contains API resources and its logics as well.
We would like to use databases resources created with Terraform in the CDK app.
I was looking for some simple way to import outputs or tfstate from Terraform into CDK app, but I've found nothing.
I'd like to know how'd you achieve something like that?
So, I finally solved this issue by using the tfstate file on CDK : our Remote Backend is AWS, so the tfstate is stored on S3. When we run the CDK app we fetch this file from S3 and we inject its outputs into an application service.
It allows to always get the updated outputs from resources generated with Terraform.
Could you please suggest easy way to copy/move already created AWS stuffs from current VPC to another ?
There is no "easy" way but you can use different tools to import existing resources and deploy them to different VPCs.
There is a free tool called Former2 (https://github.com/iann0036/former2) which can be used to scan existing resources and produce outputs. These outputs can be used to deploy new resources. I tested this tool and it seems to be quite intuitive to use to gather information about existing resources and produce outputs in different template languages (Cloudformation, Terraform, CDK, Stroposphere, Pulumi).
Terraform can be used to import existing resources to the current state and after that copy resources to configuration. Future versions of Terraform will be able to also update the configuration. To use terraform you must know every resource you want to import and use import command with their ids. Terraform does not support nested imports so importing vpc does not add subnets or other resources to the state but they have to be imported separately.
I think CloudFormation has also import feature but specific resource template must be written beforehand and submitted during the import. This is not an easy or fast way to copy resources but should work and as an end product there should a template which can be used to deploy resources to another VPCs.
currently I started to work on an old AWS Infrastructure and tried to get know all current service states thru "terraform import" separately, but I would like to import with terraform all service in one time to save time and be sure that I have all aws services.
So is it possible to import all AWS Resources in one time?
Thanks a lot for your help in advance.
Meer
I had to do this recently myself and I would highly recommend terraformer.
A CLI tool that generates tf/json and tfstate files based on existing
infrastructure (reverse Terraform).
It supports quite a lot of Terraform providers. I just tested it for AWS and it works like a charm. ;)
So my is it possible to import all AWS Resources in one time?
Not directly with TF. But you could use a third-party, opensourced tool, called Former2 which can generate TF code from existing resources:
Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources
Suppose I already have a beanstalk deployed using the AWS UI now I want to change some of its setting using terraform Can I do so?
You've to import the beanstalk resource into Terraform using Terraform import
terraform import
For the imported one you must write the Terraform configuration also as Terraform will not create configurations automatically.
Then you can make changes as per your need
I will start a new Terraform project on AWS. The VPC is already created and i want to know what's the best way to integrate it in my code. Do i have to create it again and Terraform will detect it and will not override it ? Or do i have to use Data source for that ? Or is there other best way like Terraform Import ?
I want also to be able in the future to deploy the entire infrastructure in other Region or other Account.
Thanks.
When it comes to integrating with existing objects, you first have to decide between two options: you can either import these objects into Terraform and use Terraform to manage them moving forward, or you can leave them managed by whatever existing system and use them in Terraform by reference.
If you wish to use Terraform to manage these existing objects, you must first write a configuration for the object as if Terraform were going to create it itself:
resource "aws_vpc" "example" {
# fill in here all the same settings that the existing object already has
cidr_block = "10.0.0.0/16"
}
# Can then use that vpc's id in other resources using:
# aws_vpc.example.id
But then rather than running terraform apply immediately, you can first run terraform import to instruct Terraform to associate this resource block with the existing VPC using its id assigned by AWS:
terraform import aws_vpc.example vpc-abcd1234
If you then run terraform plan you should see that no changes are required, because Terraform detected that the configuration matches the existing object. If Terraform does propose some changes, you can either accept them by running terraform apply or continue to update the configuration until it matches the existing object.
Once you have done this, Terraform will consider itself the owner of the VPC and will thus plan to update it or destroy it on future runs if the configuration suggests it should do so. If any other system was previously managing this VPC, it's important to stop it doing so or else this other system is likely to conflict with Terraform.
If you'd prefer to keep whatever existing system is managing the VPC, you can also use the Data Sources feature to look up the existing VPC without putting Terraform in charge of it.
In this case, you might use the aws_vpc data source, which can look up VPCs by various attributes. A common choice is to look up a VPC by its tags, assuming your environment has a predictable tagging scheme that allows you to describe the single VPC you are looking for:
data "aws_vpc" "example" {
tags = {
Name = "example-VPC-name"
}
}
# Can then use that vpc's id in other resources using:
# data.aws_vpc.example.id
In some cases users will introduce additional indirection to find the VPC some other way than by querying the AWS VPC APIs directly. That is a more advanced configuration and the options here are quite broad, but for example if you are using SSM Parameter Store you could place the VPC into a parameter store parameter and retrieve it using the aws_ssm_parameter data source.
If the existing system managing the VPC is CloudFormation, you could also use aws_cloudformation_export or aws_cloudformation_stack to retrieve the information from the CloudFormation API.
If you are happy to manage it via terraform moving forward then you can import existing resources into your terraform state. Here is the usage page for it https://www.terraform.io/docs/import/usage.html
You will have to define a resource block inside of your configuration for the vpc first. You could do something like:
resource "aws_vpc" "existing" {
cidr_block = "172.16.0.0/16"
tags = {
Name = "prod"
}
}
and then on the cli run the command
terraform import aws_vpc.existing <vpc-id>
Make sure you run a terraform plan afterwards, because terraform may try to make changes to it. You kind of have to reverse engineer it a bit, by adding all the necessary configuration to the aws_vpc resource. Once it is aligned, terraform will not attempt to change it. You can then re-use this to deploy to other accounts and regions.
As you suggested, you could use a data source for the vpc. This can be useful if you want to manage it outside of terraform, instead of having the potential to destroy the vpc if it is run by an inexperienced user.
Some customers I've worked with prefer to manage resources like vpcs/subnets (and other core infrastructure) in separate terraform scripts that only senior engineers have access to. This can avoid the disaster scenarios where people destroy the underlying infrastructure by accident.
I personally prefer managing all my terraform code in a git repository that is then deployed using a CI/CD tool, even if it's just myself working on it. Some people may not see the value in spending the time creating the pipeline though and may stick with running it locally.
This post has some great recommendations on running terraform in an an automated environment https://learn.hashicorp.com/terraform/development/running-terraform-in-automation