We create all our AWS resources using Terraform and we are having issues with our S3 bucket.
We are in a development environnement so sometimes we wants to destroy everything and recreate everything except the S3 Bucket.
We tried to implement the following attribute for our S3 Bucket :
resource "aws_s3_bucket" "bucket" {
bucket = "${var.name}-${var.environment}-data"
lifecycle {
prevent_destroy = true
}
}
It seems like the prevent_destroy attribute is not working as we thought it would work. Instead of skipping the deletion of the bucket and terminate the terraform destroy with a success state, it fails instead (as if this attribute tells terraform to fail on purpose).
I've found out similar conclusions already on stackoverflow but what would be the best way to avoid that issue and also the way that would be the best practice please ?
We use also github actions so we have thought of using it to create the bucket but if there's a solution using terraform, it will be easier for us (as there are other resources that are linked to the bucket id).
Thanks in advance !
prevent_destroy is used as a safety measure to ensure that the deletion of the resource does not occur. It is expected that it errors out. From the docs:
This meta-argument, when set to true, will cause Terraform to reject with an error any plan that would destroy the infrastructure object associated with the resource
so you shouldn't be using this meta-argument for your purposes.
Instead, if you want to delete everything except the bucket, you will need to remove the bucket from the terraform state before running the destroy operation, e.g.:
terraform state rm aws_s3_bucket.bucket
terraform destroy
for your next deployment, you will then need to import the bucket back into the terraform state before running the apply:
terraform import aws_s3_bucket.bucket bucket-name
terraform apply
Related
It is very basic at the moment.
connection.tf
provider "aws" {
region = "eu-west-2"
}
main.tf
resource "aws_s3_bucket" "bucket" {
bucket = "mybucket"
acl = "public-read"
provisioner "local-exec" {
command = "aws s3 sync static/ s3://${aws_s3_bucket.bucket.bucket} --acl public-read --delete"
}
website {
index_document = "index.html"
}
}
I have a Github (CI/CD) Action that can rebuild the static/ and it's website contents when updates are pushed to the main branch.
So at the moment the Terraform files (I think) just provision the bucket and push the initial contents of static/
but is there anything else that can be done with Terraform ?
Or how can the inital deployment scripts be improved?
I'm new to Terraform but the static website is up and running on AWS S3.
I've researched online the best way to use Terraform (this is a requirement of the task) to deploy a static website to S3. Having the Github Action to (CI/CD) to update the website was the main suggestion. But not much was mentioned about how the Terraform aspect could be improved - optimised.
It seems very short and I expect there are many other configurations that should be included.
Some suggestions,
Every single names that can be used as variables, put them in a variables.tf file.
store the terraform state file in backend, preferably on an s3 bucket or terraform cloud (fairly new).
create a folder named 'scripts' and write some bash scripts (or powershell if on windows) to do the plan, apply, destroy etc. Why? Well after you proceed with terraform it's not just simply terraform apply, you may and most certainly will have to do more things, pass more parameters etc. So write some bash scripts and run them, let them do the heavy lifting.
I'll give you a situation, if you run terraform destroy you can not delete a bucket with objects in it. But if you can't delete it, it will throw error on CI/CD pipeline. So what you can do is remove the resources from state file then perform destroy. Before applying you can attach again.
I also have a repo on static websites using terraform and AWS, feel free to check out
Best wishes.
The new recommendation from AWS is to disable ACL by default such that Object ownership defaults to Bucket owner. How can I achieve this with aws_s3_bucket resource using Terraform?
I tried doing the following without success
resource "aws_s3_bucket_acl" "example_bucket_acl" {
bucket = aws_s3_bucket.s3-bucket.id
acl = "private"
expected_bucket_owner = data.aws_caller_identity.current.account_id
}
data "aws_caller_identity" "current" {}
This code sets the ACL such that only bucket owner can read and write the bucket and the objects within the bucket, but the object ownership configuration is still set to "object writer". Furthermore, ACL is not disabled as a result of setting this.
From the Terraform's documentation on S3 ACL, it does not state any examples nor provide any arguments that support disabling ACL.
I tried to brute force the solution by running terraform plan after manually changing the settings in AWS to see what differences I would get from the plan, but it says my infrastructure matches the configuration.
Does anyone have any ideas how this can be done? I'm currently using Terraform CLI v1.3.5 and AWS provider v4.40.0.
This is set using aws_s3_bucket_ownership_controls, not with aws_s3_bucket_acl. You can set the control to BucketOwnerEnforced.
I'm currently trying to destroy a workspace, I know that there are some buckets that have a 'do not destroy' type tag applied to them, so when I run terraform destroy for the first time, I got Instance cannot be destroyed error for two buckets:
Resource module.xxx.aws_s3_bucket.xxx_bucket has
lifecycle.prevent_destroy set, but the plan calls for this resource to be
destroyed. To avoid this error and continue with the plan, either disable
lifecycle.prevent_destroy or reduce the scope of the plan using the -target
flag.
so I navigate to the AWS console and delete them manually then tried to run terraform destroy again, then it's complaining about one of the buckets that I've removed manually: Failed getting S3 bucket: NotFound: Not Found, the other one seems fine.
Does anyone know how to resolve this please? Thanks.
If you removed the resource with an action external to a modification in the Terraform state (in this situation a bucket removed manually through the console), then you need to update the Terraform state correspondingly. You can do this with the terraform state subcommand. Given your listed example of a resource named module.xxx.aws_s3_bucket.xxx_bucket, it would appear like:
terraform state rm module.xxx.aws_s3_bucket.xxx_bucket
You can find more info in the documentation.
The resource aws_iam_policy_attachment has the following warning
WARNING: The aws_iam_policy_attachment resource creates exclusive attachments of IAM policies. Across the entire AWS account, all of the users/roles/groups to which a single policy is attached must be declared by a single aws_iam_policy_attachment resource. This means that even any users/roles/groups that have the attached policy via any other mechanism (including other Terraform resources) will have that attached policy revoked by this resource. Consider aws_iam_role_policy_attachment, aws_iam_user_policy_attachment, or aws_iam_group_policy_attachment instead. These resources do not enforce exclusive attachment of an IAM policy.
We changed some of our code from
resource "aws_iam_policy_attachment" "logs" {
name = "${var.function_name}-logs"
roles = [aws_iam_role.lambda.name]
policy_arn = aws_iam_policy.logs[0].arn
}
to
resource "aws_iam_role_policy_attachment" "logs" {
name = "${var.function_name}-logs"
role = aws_iam_role.lambda.name
policy_arn = aws_iam_policy.logs[0].arn
}
The change above is simple but now terraform wants to remove the aws_iam_policy_attachment resource and add the aws_iam_role_policy_attachment. Previously, when we applied the terraform for a module using a shared managed IAM resource, it detached the policy from 30 different IAM roles, forcing us to reattach them by finding and reapplying our terraform modules.
What is a safe strategy to use the less dangerous resource aws_iam_role_policy_attachment?
Our current strategy
Recreate managed IAM policy as an inline policy and add to role
Remove the managed policy manually using AWS console
Possibly easier with this CLI command. It just seems in the console.
aws iam detach-role-policy \
--role-name my-role-name \
--policy-arn arn:aws:iam:1234567890:role/logs
Remove the bad resource from the state
May not be necessary since it was removed in the previous step
terraform state rm aws_iam_policy_attachment.logs
Target apply the new attachment
target apply -target aws_iam_role_policy_attachment.logs
Sanity check
terraform plan
Remove the inline policy from the first step
note on state manipulation
Whenever I am going to do state surgery, I change the state to be a local state. Do all my operations. Then run a plan to ensure my changes have caused no diffs. Then put the state back to your propper backend. This article explains how to do that:
https://medium.com/faun/cleaning-up-a-terraform-state-file-the-right-way-ab509f6e47f3
But at a minimum, at least do: terraform state pull > backup.tfstate
state commands for your task
first, make terraform stop tracking the old way you did this
terraform state rm aws_iam_policy_attachment.logs
Then just do an import on the new associative resource:
terraform import aws_iam_role_policy_attachment.logs lambda-role/arn:aws:iam::xxxxxxxxxxxx:policy/policy-name
per docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment
Do a terraform plan and you should see no diffs.
conclusion
This allows you to not touch your actual AWS configuration. You wont end up deleting any roles or permissions for even a minute. It's safe and verifiable, if you backup your state ahead of time.
I'm finding that cdk tries to recreate S3 buckets every time I deploy. If I don't specify a bucket name, it generates a new junk bucket name every time. If I do specify a name, it refuses to deploy because the bucket already exists. How can I make it "upsert" a bucket?
Here's the code I'm using:
const dataIngestBucket = new Bucket(this, 'data-lake', {
bucketName: `${this.props.environmentName}-my-company-data-lake`
});
As long as I do not see the language you want to use, I give an answer using python. It can be easily traced and converted to any other languages.
Please refer to aws_cdk.aws_s3.Bucket class.
There you will find parameters to specify during class creating which allow you reach your goal, namely auto_delete_objects=True and removal_policy=cdk.RemovalPolicy.DESTROY.
CDK would do an update on your stack resources automatically if CDK code is updated.
For example, when you execute a CDK stack that creates a bucket for the first time, bucket would be created with provided configuration.
When you update your CDK code to say update lifecycle policy of the bucket or add a CORS, as part of the same stack, the update of the stack would automatically update the bucket - it would not recreate the bucket as Cloud Formation knows that there is an update on an existing stack.
In your case, it seems the stack is being re-created after a removal when the stack resources still exists. This causes Cloud Formation to create a new stack and its resources which were not removed when stack was destroyed.
Generally, issues occur when stack update fails and it is in rollback state, for example. In that case, redeploy would try to create the bucket again and fail.
In that case, possible option could be:
Delete the buckets
Delete the stack
Redeploy to re-create
Many times, we do not want to delete the resources, as they contain data; in that case, you can use another library such as boto3 for python in the CDK code, to check if the resource exists - if not create via CDK. This would cause CDK code to be not attempt creating the bucket if it exists ( CDK itself cannot be used to see if say S3 resource exists already - at least have not seen how to achieve this)
Another important point is the removal policy associated with the resource
troubleshooting_resource_not_deleted
My S3 bucket, DynamoDB table, or other resource is not deleted when I
issue cdk destroy
By default, resources that can contain user data have a removalPolicy
(Python: removal_policy) property of RETAIN, and the resource is not
deleted when the stack is destroyed. Instead, the resource is orphaned
from the stack. You must then delete the resource manually after the
stack is destroyed. Until you do, redeploying the stack fails, because
the name of the new resource being created during deployment conflicts
with the name of the orphaned resource.
If you set a resource's removal policy to DESTROY, that resource will
be deleted when the stack is destroyed.
However, even with the removal policy as DESTROY, Cloud formation cannot delete a non-empty bucket. Extract from the same link below -
AWS CloudFormation cannot delete a non-empty Amazon S3 bucket. If you
set an Amazon S3 bucket's removal policy to DESTROY, and it contains
data, attempting to destroy the stack will fail because the bucket
cannot be deleted. You can have the AWS CDK delete the objects in the
bucket before attempting to destroy it by setting the bucket's
autoDeleteObjects prop to true.
Best Practice is to
Design stack resources in such a manner that they have minimal updates being applied which can cause failure. So a stack can be created with say mostly static resources such as ECR, S3 which do not change much and is independent generally of the main application deployment stack which is more likely to fail.
Avoid manually deleting the stack resources which breaks a stack's inconsistency
If a stack is deleted, ensure stack's owned resources are also deleted.
Get rid of having fix names!
With
final IBucket myBucket = Bucket.Builder.create(this, "mybucket")
.bucketName(PhysicalName.GENERATE_IF_NEEDED).build();
(Java, but doesn´t matter)
Do you get a "random-Named" Bucket.
Described here: https://docs.aws.amazon.com/cdk/latest/guide/resources.html
Use it like this in your template (here nested stack)
#Nullable NestedStackProps templateProps = NestedStackProps.builder()
.parameters(new HashMap<String, String>(){{
put("S3Bucket", myBucket.getBucketName());
}})
.build();
Or you still have a fix name (get rid of!!) then get them with:
final IBucket myBucket = Bucket.fromBucketName(this, "mybucket", "my-hold-bucket-name");
But you can not doing things like:
if (!myBucket) then create
(pseudo code)
No ressource-check at compile/runtime!