Lets imagine that i have a cloudformation stack with 2 EC2 instances 1 S3 bucket and 1 dynamoDb table.
Everything is good with this stack until we decide to do some refactoring and split this stack into two stacks. Stack A that has only 2 EC2 instances and Stack B that has 1 S3 bucket and 1 dynamoDb table.
My question is: what is the best way to do that without destroying and re creating the resources or doing migration? (i am looking for a solution similar to terraform move state).
Could you please share how you organise your stacks and how you refactor them.
Moving resources between stacks is now supported for some resources including DynamoDb, EC2 instances and S3 buckets. The aws docs at
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/refactor-stacks.html give the full detail and instructions starting with:
Using the resource import feature, you can move resources between, or refactor, stacks. You need to first add a Retain deletion policy to the resource you want to move to ensure that the resource is preserved when you remove it from the source stack and import it to the target stack.
Important
Not all resources support import operations. See Resources that
Support Import Operations before you remove a resource from your
stack. If you remove a resource that doesn't support import operations
from your stack, you can't import the resource into another stack or
bring it back into the source stack.
Currently, there is no mechanism in CloudFormation to move objects/resources from 1 stack to another.
If you had used Terraform to deploy your resources, then yes, you could use terraform state mv to move resources from one state file to a different state file. A Terraform state file is associated with a single Terraform deployment, so you you could think of it as being analogous to how CloudFormation stores its version of what's deployed in a stack.
Note that Terraform state moves are not the simplest things in the world and AFAIK there is no GUI interface to assist with moves. Make backups and set aside time for testing.
There is, however, a project called Terraboard that is a GUI that can help you inspect the state file.
Related
I managed AWS rds instance on Terraform, but the instance is no longer needed so I removed all .tf files for that instance.
In this case, I think I can perform terraform destroy, but do I need to trigger terraform apply before that, or only one of these is enough? I don't fully understand the difference of those two commands when removing resources.
terraform destroy is used to delete/destroy all resources that you managed by your TF files. In contrast, terraform apply is used to apply changes to your TF controlled infrastructure. This means that only selected resources will be changed (or removed), not all.
In your case, since you removed .tf files associated with some instance, terraform apply will just destroy that instance. Other resources will not be affected. However, if you execute terraform destroy everything will be removed, including the instance and anything else that you have created.
Our team had an issue where someone manually modified an IAM role for an operational event. We hoped that a CloudFormation stack redeployment would revert the state of the IAM role, however, the manual change was still there.
My working theory is that since the IAM role arn is the same, that CloudFormation does not delete and recreate it. Is that accurate? And if so, how do we ensure all relevant resources are torn down during a deployment?
My working theory is that since the IAM role arn is the same, that CloudFormation does not delete and recreate it. Is that accurate?
CloudFormation (CFN) does not check for any changes made outside of its control. You could remove the role, and CFN would still "think" that the role is there.
If you change a resource created by CFN manually outside of CFN (bad practice), you have so called a stack drift. CFN by itself is not aware of any changes made to resources it creates that occurred outside its control. But, CFN provides special tools which you have to explicitly call to detect the drift:
Detect drift on an entire CloudFormation stack
Not all resources support drift detection, but AWS::IAM::Role is one which does.
And if so, how do we ensure all relevant resources are torn down during a deployment?
Not sure what do you mean here. But you have to manually fix the drift. You have four choices:
Change the role back to its original state,
Update template to reflect the external changes,
Use import to import the changed role to stack,
Delete the entire stack, and create new one from the original template.
The last choice ensures that the modified role is also deleted and recreated in its original form.
I am having an interesting problem with AWS. The project I'm working on create's set of various AWS service instances in one predetermined zone. For example, one VPC, one firewall, one S3 bucket, RDS instance..
Now if one of these fails to be created for the lack of the general available resources in the region I have to go through nasty deletion and switch to another region.
Is there something in AWS API that can tell if there is enough resources to create instance of something without actually creating it?
if one of these fails to be created for the lack of the general available resources in the region I have to go through nasty deletion and switch to another region
Not a direct answer to your question (and I would have put it as a comment, but I lack reputation to comment... hint hint), but if you were to use CloudFormation, one of the very nice features of it is that if any one resource fails to create, it will roll back the entire stack. The other nice thing about CloudFormation is it will give you consistent infrastructure across your multiple regions. Have you explores using CloudFormation to deploy your resources.
I have an AWS account where multiple EC2 instances, load balancers, target groups, security groups etc are setup by multiple owners.
We use terraform to set this up but sometimes due to corruption, the state becomes inconsistent. Current mechanism to recover is to manually destroy all resources in that account owned by a particular owner.
Is there an easy way to nuke all resources in an AWS account belonging to a particular owner?
There is no way to delete all resources in an account owned by a particular user but there is a way to delete all resources in an account.
You can use aws-nuke which was created somewhat out of the same use case you described.
At first, you need to set an account alias for your account.
You must create a config file.
Then you can list down all resources that will be deleted using the following command:
aws-nuke -c config/nuke-config.yml --profile aws-nuke-example
Add --no-dry-run option to permanently delete all resources in the same command.
There are also multiple filter options available such as target, resource type, exclude, etc. that you can leverage to suit your needs.
Agree with the other answer that there is no easy way delete orphan resources.
But I see the original issue is that the terraform state is corrupted.
You can checkout the terraform import feature which lets you generate state file from aws resources. In that way you can connect your config to resources again.
Short answer: no.
Longer answer: actually, that's also no. There's no built-in capabillity for this.
The case you're describing is not within the bounds of typical AWS usage... destroying everything in an account -- usually -- should not be easy.
Of course, you could script it, fairly trivially, by wrapping calls to aws-cli to custom code to iterate through the resources and generate additional requests to destroy them... but if you do, lock that code away, since such capability is inherently dangerous.
You can delete all your resources you created, you'll need to automate, see a sample here:
Creation
https://github.com/jouellnyc/AWS/tree/master/create_aws_vpc2
Deletion
https://github.com/jouellnyc/AWS/blob/master/create_aws_vpc2/delete_lb_and_vpc.sh
Other
I've had some success with cloud nuke (played around for a few min; not in depth):
https://github.com/gruntwork-io/cloud-nuke
I dont think there is any state forward way to do it but to check if you have any active resources in your account, do the following:
Open the Billing and Cost Management console.
Choose Bills in the navigation pane.
You can see the charges incurred by different services in the Bill details by service section.
You can see the charges incurred in different AWS Regions in the Bill details by account section.
For each service, identify the Regions where the services have incurred charges.
To terminate the identified active resources under different services, do the following:
Open the AWS Management Console.
For Find services, enter the service name.
After opening the service console, terminate all your active resources. Be sure to check each Region where you have allocated resources.
if the issue is a corrupted terraform state, perhaps storing the state in a versioned S3 bucket would help reduce the impact of that.
Use Terraformer to import all resources into terraform configuration then do whatever you want:
terraformer import aws --resources="*"
https://github.com/GoogleCloudPlatform/terraformer
Take care of your state file lock f.e. by using dynamodb & enable s3 versioning.
Is there an easy way to nuke all resources in an AWS account
belonging to a particular owner?
Since you are using Terraform, you can use Blank Apply.
It will destroy all the Resources in a state file.
We use terraform to set this up but sometimes due to corruption, the
state becomes inconsistent.
It's better to use version control system to avoid drifts and inconsistencies in your state file and use remote states in order to make sure everyone is on the same page.
Lets say an AWS stack was created using CloudFormation.
Now one of those resources was modified outside CloudFormation.
1) Is it possible to have CloudFormation specifically create those resources? Based on my understanding, we can't do that because CloudFormation does not identify a difference, and so does not create the modified resources. Is my observation correct?
2) Also, what options do I have to revert a stack to its original state, if modified outside CloudFormation?
This is one possible hack you could use without deleting the entire stack.
From the template remove the specific resource which got deleted accidentally.
Now update the stack which makes your stack and resources in your account in sync.
Revert the template to its state before step1 and update again which will create the resource which got deleted accidentally.
Unfortunately the answer for both your questions is NO.
If you modify the resources in the stack after stack creation status is COMPLETE, there is nothing CF can do since it doesn't keep track of modification to resources
You have no option other than deleting the current stack and create a new one
First, beware that modifying CloudFormation-created resources outside of CloudFormation is explicitly discouraged, according to AWS CloudFormation Best Practices:
Manage All Stack Resources Through AWS CloudFormation
After you launch a stack, use the AWS CloudFormation console, API, or AWS CLI to update resources in your stack. Do not make changes to stack resources outside of AWS CloudFormation. Doing so can create a mismatch between your stack's template and the current state of your stack resources, which can cause errors if you update or delete the stack.
However, if you've modified a CloudFormation-managed resource accidentally and need to recover, you may have some limited options beyond simply deleting and re-creating the stack altogether (which may not be an acceptable option):
It is not possible for CloudFormation to automatically update its internal state based on the current state of an externally-modified resource.
However, depending on the exact resource type, in some cases you can manually update CloudFormation afterwards by applying a stack update that matches the current state of the resource.
Similarly, it is not possible for CloudFormation to automatically revert an externally-modified resource back to its original unmodified CloudFormation state.
However, depending on the exact resource type, in some cases you can either:
Revert a resource by manually updating the resource back to its original state;
Update the resource by applying a stack update, bringing both the CloudFormation stack and the managed resource to an altogether new state that will once again be in sync.
To force the EC2 re-creating, I do use a simple trick, when I'm deploying, I jump between AMI's IDs (I took two similar AMI's ID), that had helped me when I'm testing user data or things that I want to test during the EC2 bootstrap. Again, it just works for EC2.
Unfortunately, the answer is NO
if you made changes in the stack after the creation, Cloudformation can't track those changes.
if you need to revert those changes, you must delete the stack and rebuild.