We are using Terraform along with a vendor supplied CloudFormation template.
It is all working except for when there are changes to the template. Terraform does not show you what will change as it's all contained within the template.
I was wondering if Terraform could write out the rendered CloudFormation template to disk where we could (in theory) use AWS native CloudFormation to create a change set to attempt to see the changes.
Any ideas if Terraform can write this out?
Terraform works by abstracting different infrastructure types into providers. In the case of AWS, the AWS Provider interprets your Terraform config into a set of AWS API calls. It does not internally create any CloudFormation templates, and has no understanding of or mapping to CloudFormation.
If you really want to use Terraform to describe changes made via CloudFormation you will likely have to create your own tool to convert CF templates into Terraform Variable files. As each new variable file is generated, you could run terraform plan to see what the outcome would be.
Related
Using the CDK is it possible to get an existing CfnDBCluster to make modifications to?
I have an AWS::RDS::DBCluster in CloudFormation whose TimeoutAction I want to change (CloudFormation doesn't support it and I don't want to use the AWS cli).
CDK doesn't natively support importing existing resources for modification.
https://medium.com/#visya/how-to-import-existing-aws-resources-into-cdk-stack-f1cea491e9
This article describes using CDK to generate the template, then use the AWS Management Console to import the resource into the stack.
Here is an issue to track the support within CDK itself: https://github.com/aws/aws-cdk-rfcs/issues/52
In your case, specifically since DB Clusters support this, you could create snapshot of the database then delete it. Then reference the snapshot id when creating recreating the cluster with CDK. Obviously it would require downtime though.
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-rds.DatabaseClusterFromSnapshot.html
I have created an ec2 instance and configured it as a target behind my load balancer. I want to convert this entire environment as something that can be deployed repeatedly in automation. I have looked at cloudformation but don't know if it can help me with converting this environment to a cloudformation template. Is there any other way to achieve this. If there is an approach outside of Cloudformation, that's fine too.
PS: I am new to AWS and it's capabilities
Thanks
Have a look at Former2.
Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account.
You need an IaC (Infrastructure as Code) tool. Cloudformation is one of them, but there are plenty others. Terraform, Pulumi or even the AWS CDK.
Look at Infrastructure as Code try AWS CDK, Terraform, you should also look at methods for replacing existing infrastructure. Spinning a fresh set of infrastructure along side the existing one and swapping out in DNS is the most common of approach.
Would anybody have any clues as to how AWS CloudFormation works under the hood?
Also, would anybody know an open-source equivalent to AWS CF (and I don't mean tooling that may be using CloudFormation)?
It's clearly a powerful orchestrator, but I'd be keen to explore the inner workings of such tools.
AWS Cloudformation has multiple pre-defined set of schemas for each of the components that are supported. When you upload a Cloudformation template for creating resources, it performs the below steps:
It validates the templates against the schema
It generates dynamic form for gathering parameters
It validates the values of parameters
Once it has all it needs, you can click Create to begin with the resource creation
Under the hood, it starts creation of resources using the internal coding for which is keeps echoing the status and progress continuously on the console.
We need to understand here that internally Cloudformation in itself is a product that does use AWS SDK/CLI as needed. However, under the hood, it maintains its own data to compare the attributes and resources when you run an update.
An open source alternative to this is Terraform. Terraform is the most widely used open source replacement of Cloudformation. Terraform is known for its Cloud independent architecture. Terraform works with multiple clouds with minimal changes in the templates.
The under-the-hood working of terraform involves creation of a State file/directory where it stores the current state of any stack identified uniquely by the name provided by the user. Terraform creates resources majorly using Python SDK (boto3) and some other APIs as needed. We need to pass the access key and secrets to the Terraform configuration in order to enable it to access the AWS Cloud environment.
If you are looking to build a smart new alternative, it should be fairly simple considering that AWS strictly follows standard design patterns in its SDK and CLI interface design. This makes it easier to convert template into executable code.
More information about working of Cloudformation can be found here
I've got a big-ass Serverless project and I wonder if matching the cloud formation template schema with CDK would do the trick, or is there something extra to the process.
It is possible to deploy a CDK app to an existing CloudFormation stack, although it would be very difficult to achieve for non-trivial stacks since CDK apps usually involve many resources.
The cdk diff command will be your best friend. You can name your stack in the CDK app using the same name as the existing stack:
MyExistingStack(app, 'my-existing-stack')
Then you can iteratively add/remove resources and run cdk diff to check your success in matching the current deployment. CDK will additionally create metadata resources that will be added to the stack in addition to the currently existing resources.
Matching resource names can be difficult. CDK automatically names many of the resources in a way that will not match you existing stack. Following the instructions on CDK Escape Hatches, you can access lower level CFN Resources directly and modify the name.
If a Construct is missing a feature or you are trying to work around an issue, you can modify the CFN Resource that is encapsulated by the Construct.
All Constructs contain within them the corresponding CFN Resource. For example, the high-level Bucket construct wraps the low-level CfnBucket construct. Because the CfnBucket corresponds directly to the AWS CloudFormation resource, it exposes all features that are available through AWS CloudFormation.
The basic approach to get access to the CFN Resource class is to use construct.node.defaultChild (Python: default_child), cast it to the right type (if necessary), and modify its properties.
I'm new to a large AWS deployment where stuff is mostly deployed through CloudFormation (and some through Terraform). But there are always cases where something has been deployed manually and not through code. Is there a reliable way to quickly figure out if a resource (say, an EC2 instance) already existing in the deployment was deployed through IaC or manually? A CloudFormation-specific answer will be good enough for now.
Going through literally hundreds of CloudFormation stacks manually and looking for the resource is not an option.
You can identify the resources created by cloudformation. Cloudformation applies few default tags as mentioned here
aws:cloudformation:logical-id
aws:cloudformation:stack-id
aws:cloudformation:stack-name
You can run a script to check whether the resource contain one/all of these tags to update your count.
Offical documentation on resource tags
Unfortunately looking at an AWS resource you don't see how it got created. While some resources might have been tagged by CloudFormation indicating that they got created by a CloudFormation stack, that's only valid for a subset of resources.
The only reliable way to figure out whether or not a resource got created via a CloudFormation stack is to go through all CloudFormation stacks and check whether or not the resource in question is a part of it. While that might be cumbersome when doing manually, it's also something you can automate using the AWS CLI.