There is a terraform code to configure an MWAA Environment in AWS. When it runs second time, no need to create IAM role or policy again. So it gives an error.
How to ignore the creation of existing resources in TF?
I assume that you applied a Terraform plan which created resource "MWAA", then you somehow lost the state (locally stored and lost?, or the state wasn't shared with a different client?), then you re-apply the plan again, and Terraform informs you that it created "MWAA", again.
In that case, your main problem is that you lost the state, and you need to make sure that you do persist it, e.g., by storing it in a bucket.
However, if you really need to make Terraform aware about an already created resource, you need to put it in Terraform's state. One tool to do that is "terraform import", about which you can read more here: https://www.terraform.io/cli/import
If you already have the statefile and if terraform is trying to re-install it again, then may be some tag change or modified timestamp value change...
In order to avoid it, you can specify the resource you want to apply using a terraform apply command..
terraform apply --target=resource
Related
I am switching to gitlab and plan to use terraform. I have used cloudformation before and understand , deploying stack to aws, creating change stack and updating resources. how does updating/deleting work in terraform.
Its similar to CFN. TF has a state file (can be local or remote) where it stores information about your currently deployed resources and their configuration.
After any changes to your TF config files, TF would create a plan of how to apply your changes in relation to what it has in the state. The plan is similar to changeset in CFN, it will show what resources have to be deleted, replaced, created or modified.
Just like with changeset you have option to review the plan and if you agree with a proposed actions, you can apply it.
The biggest difference is what happens if there is a failure. Cloudformation will rollback the stack to the previous state whereas Terraform will leave the resources in a partially deployed state.
I am creating AWS infrastructure using Terraform, and using S3 backend configuration. Now the issue is, someone deleted the S3 bucket storing the state, and now every time I run terraform it fails saying the resources already exist. The old tfstate is lost, and new has no information about existing resources.
Note: I do not have write access to the AWS environment. I trigger terraform via Jenkins CD pipeline, so I cannot manually modify the infrastructure or run any terraform command.
Is there a way to cleanup existing resources or force recreating resources(if they already exist) with tf file? This is the only place I can make changes.
You really are in a mess. You need to restore the S3 bucket or make a new one and point your code at that.
You then need to recreate the state you lost, that or delete every object you created via Terraform and start again. Most objects have the ability to import existing objects via the Terraform import command.
This could be a significantly large task.
And you'd be needing write access to the bucket? Terraform refresh is only going to help if you still had the state file. You don't.
If you haven't got permission to do that, then maybe give up that or persist in getting sufficient privilege.
If you can't run Terraform locally then you are also wasting your time.
Good luck.
However....
You don't want to be here again. How did you delete/lose the bucket?
You really need that never to happen again as #ydaetskcoR said some MFA protection on the bucket - definitely do that and adding versioning to it is a REALLY good idea.
Also if you haven't added DynamoDB locking to the bucket do so, its really worth it.
This may also be a good time to think/dwell about tagging your infrastructure, then you might be able to identify what infra belonged to the code. That would also help - well next time.
If you're working in a production account, follow the advice of others and do not mess with anything in the account manually!
If you are just starting out with terraform or terragrunt and you're trying to reset the terraform state:
Ensure that you are logged into the correct AWS account, it is not a production account, and it is not in use by anyone else
Terraform state is saved in two places under the same name: S3 and DynamoDB. In order to reset the state, you will need to delete both.
If there is anything else that was previously created in the account, you will need to delete those manually. Depending on how much you created, this could take a very long time.
Once you have deleted the S3 and DynamoDB named after your terraform state bucket and deleted the infrastructure created by terraform, you should be able to terraform init and terraform plan without further issue.
i had provisioned some resources over AWS which includes EC2 instance as well,but then after that we had attached some extra security groups to these instances which now been detected by terraform and it say's it'll rollback it as per the configuration file.
Let's say i had below code which attaches SG to my EC2
vpc_security_group_ids = ["sg-xxxx"]
but now my problem is how can i update the terraform.tfstate file so that it should not detach manually attached security groups :
I can solve it as below:
i would refresh terraform state file with terraform refresh which will update the state file.
then i have to update my terraform configuration file manually with security group id's that were attached manually
but that possible for a small kind of setup what if we have a complex scenario, so do we have any other mechanism in terraform which would detect the drift and update it
THanks !!
There is no way Terraform will update your source code when detecting a drift on AWS.
The process you mention is right:
Report manual changes done in AWS into the Terraform code
Do a terraform plan. It will refresh the state and show you if there is still a difference
You can use terraform import with the id to import the remote changes to your terraform state file. Later use terraform plan to check if the change is reflected in the code.
This can be achieved by updating terraform state file manually but it is not best practice to update this file manually.
Also, if you are updating your AWS resources (created by Terraform) manually or outside terraform code then it defeats the whole purpose of Infrastructure as Code.
If you are looking to manage complex infrastructure on AWS using Terraform then it is very good to follow best practices and one of them is all changes should be done via code.
Hope this helps.
terraform import <resource>.<resource_name> [unique_id_from_aws]
You may need to temporarily comment out any provider/resource that relies on the output of the manually created resource.
After running the above, un-comment the dependencies and run terraform refresh.
The accepted answer is technically not correct.
As per my testing:
Terraform refresh will update the state file with current live configuration
Terraform plan will only internally update with the live configuration and compare to the code, but not actually update the state file
Terraform apply will update the state file to current live configuration, even if it says no changes to apply (use case = manual change then update TF code to reflect change and now want to update state file)
Recently, we had issues with tfstate being deleted on S3.
As a result, there are a number of EC2 instances still running (duplicates if you will)
Is there a way to query Terraform and list which EC2 instances (and other resources) Terraform has under its control? I want to delete the duplicate AWS resources without messing up Terraform state.
Depending on whether you care about availability you could just delete everything and let Terraform recreate it all.
Or you could use terraform state list and then iterate through that with terraform state show (eg. terraform state list | xargs terraform state show) to show everything.
terraform import is for importing stuff that exists back in to your state which doesn't sound like what you want because it sounds like you've already recreated some things so have duplicates. If you had caught the loss of the resources from your state file before Terraform recreated it (for example by seeing an unexpected creation in the plan and seeing that the resource already existed in the AWS console) then you could have used that to import the resources back into the state file so that Terraform would then show an empty plan for these resources.
Iin the future make sure you use state locking to prevent this from happening again!
Is there a way to create the terraform state file, from the existing infrastructure. For example, an AWS account comes with a some services already in place ( for ex: default VPC).
But terraform, seems to know only the resources, it creates. So,
What is the best way to migrate an existing AWS Infrastructure to Terraform code
Is it possible to add a resource manually and modify the state file manually (bad effects ?)
Update
Terraform 0.7.0 supports importing single resource.
For relatively small things I've had some success in manually mangling a state file to add stubbed resources that I then proceeded to Terraform over the top (particularly with pre-existing VPCs and subnets and then using Terraform to apply security groups etc).
For anything more complicated there is an unofficial tool called terraforming which I've heard is pretty good at generating Terraform state files but also merging with pre-existing state files. I've not given it a go but it might be worth looking into.
Update
Since Terraform 0.7, Terraform now has first class support for importing pre-existing resources using the import command line tool.
As of 0.7.4 this will import the pre-existing resource into the state file but not generate any configuration for the resource. Of course if then attempt a plan (or an apply) Terraform will show that it wants to destroy this orphaned resource. Before running the apply you would then need to create the configuration to match the resource and then any future plans (and applys) should show no changes to the resource and happily keep the imported resource.
Use Terraforming https://github.com/dtan4/terraforming , To date it can generate most of the *.tfstate and *.tf file except for vpc peering.