I have an existing VPC with a few EC2 instances already created and running with security groups, route tables, NACLs all applied where required. Is there a way to convert a setup like this into a cloudformation stack or a terraform equivalent of configuration files?
Update: A modern substitute for CloudFormer is Former2.
You can use CloudFormer, but it doesn't seem to be maintained much anymore.
See: Using CloudFormer (Beta) to Create AWS CloudFormation Templates from Existing AWS Resources - AWS CloudFormation
While that type of capability might sound like a good idea, it is quite difficult to make a template from running infrastructure. It runs into questions, such as:
Should the definition for an Amazon EC2 instance reference the specific Security Group that currently exists, or should it create another one and reference that new one?
Should it create a new VPC, or launch resources in the existing VPC?
Was an instance created directly, or was it launched by Auto Scaling?
In the end, it's probably easier to just write the CloudFormation template yourself, rather than spend time trying to 'fix' one that is automatically generated.
Some tips for writing CloudFormation templates:
Never write them by hand. Always copy the templates or examples from the documentation.
Copy snippets from your existing templates rather than doing them all from scratch again.
Use YAML rather than JSON (less errors due to unbalanced braces)
Related
Had a question regarding infrastructure as code on AWS.
Wondering how to do this (the process of deploying) and also why is this an efficient method for architecture? Also, are there other methods that should be looked at over this?
I am looking to deploy this for a startup I am working for and need assistance in getting this going. Any help is appreciated.
Thank you.
From What is AWS CloudFormation?:
AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you. You don't need to individually create and configure AWS resources and figure out what's dependent on what; AWS CloudFormation handles all of that.
So, instead of manually creating each bit of architecture (network, instances, queues, storage, etc), you can define them in a template and CloudFormation will deploy them. It is smart enough to mostly know the correct order of creation (eg creating the network before creating an Amazon EC2 instance within the network) and it can also remove resources when the 'stack' is no longer required.
Other benefits:
The template effectively documents the infrastructure
Infrastructure can be checked into a source code repository, and versioned
Infrastructure can be repeatedly and consistently deployed (eg Test environment matches Production environment)
Changes can be made to the template and CloudFormation can update the 'stack' by just deploying the changes
There are tools (eg https://former2.com/) that can generate the template from existing infrastructure, or just create it from code snippets taken from the documentation.
Here's an overview: Simplify Your Infrastructure Management Using AWS CloudFormation - YouTube
There are also tools like Terraform that can deploy across multiple cloud services.
I have created an ec2 instance and configured it as a target behind my load balancer. I want to convert this entire environment as something that can be deployed repeatedly in automation. I have looked at cloudformation but don't know if it can help me with converting this environment to a cloudformation template. Is there any other way to achieve this. If there is an approach outside of Cloudformation, that's fine too.
PS: I am new to AWS and it's capabilities
Thanks
Have a look at Former2.
Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account.
You need an IaC (Infrastructure as Code) tool. Cloudformation is one of them, but there are plenty others. Terraform, Pulumi or even the AWS CDK.
Look at Infrastructure as Code try AWS CDK, Terraform, you should also look at methods for replacing existing infrastructure. Spinning a fresh set of infrastructure along side the existing one and swapping out in DNS is the most common of approach.
I was wondering if anybody has had any experience creating a cloudformation template from exisiting AWS resources.
I am currently trying to migrate from a classic elb to a alb using the wizard. However I already have cloudformation templates managed by github. Therefore I would need to add the alb in after it has been created. I tried using cloudformer but it doesn't appear to support alb whereas it does pickup classic.
Has anybody had experience migrating elbs and creating cloudformation templates from existing resources?
Many thanks!
AWS::ElasticLoadBalancingV2::LoadBalancer is one of the resources that can be imported into CloudForamtion. But sadly AWS::ElasticLoadBalancingV2::TargetGroup can't be imported.
Importing is a try-and-see operation. It is not automated as many people expect it to be. The reason is that you have to manually create the template for the resources being imported. What's more, the attributes of the resources in the template created must match exactly existing resources.
CloudFormer is not helpful these days. Its not maintained by AWS anymore and has been in beta for years.
If you haven't tried importing anything before, the best way would be to start with AWS tutorial: Importing Existing Resources Into a Stack
This way you can start with something simple, before you move to ALB with all its listeners and listener rules. Off course you have to create new Target Group as well as it can't be imported sadly.
I´m using AWS CLI and CloudFormation, and I could not find any reference in the documentation.
Does anybody know if it´s possible to create a CloudFormation template from a current configuration.
Let´s say that I want to get a CloudFormation template from my current security group configuration.
Any idea if it´s possible to export that configuration as a template using CLI?
Based on our experience we found 3 possible ways to translate existing manually deployed (from Web Console UI) AWS infra to Cloudformation (CF).
Using a new CloudFormation native introduced feature (since Nov 2019) that allows you to Import existing resources into a CloudFormation stack
Using aws cli execute $aws service_name_here describe for each element that make up your stack eg for RDS Database Stack:
RDS Instance -> Type: AWS::RDS::DBInstance,
RDS (EC2) SG -> Type: AWS::EC2::SecurityGroup,
RDS Subnet Group -> Type: AWS::RDS::DBSubnetGroup and
RDS DB Param Group -> Type: AWS::RDS::DBParameterGroup
And manually translate to CF based on the outputs obtained from the aws cli for each of the components. This approach usually requires more experience in both AWS and CF but the templates that you are creating can be structured and designed under good practices, fully parameterized (Sub, Ref, Join, Fn::GetAtt:, Fn::ImportValue), modular, applying conditions and in a 1st iteration the result would probably be close to the final state of the templates (interesting reference examples: https://github.com/widdix/aws-cf-templates/).
Extra points! :)
Some other new alternatives to export your current deployed AWS infra to Cloudformation / Terraform code:
https://former2.com
https://modules.tf
https://www.brainboard.co/
Related Article: https://medium.com/#exequiel.barrirero/aws-export-configuration-as-code-cloudformation-terraform-b1bca8949bca
It's not possible using the AWS CLI but you can use the CloudFormer [1] tool to create a CloudFormation template from existing resources. I've had decent success with it. The templates aren't as "pretty" as hand-made templates but they provide a good starting point.
[1] http://aws.amazon.com/developertools/6460180344805680
In addition to CloudFormer, you might want to take a look at Bellerophon: https://github.com/arminhammer/bellerophon.
I had some problems getting the tradidtional tools - mentioned above - working in our environment; we have a complicated API Gateway. Former2 didnt' find it at all (although seemed ideal for other resources)
I found another tool, "Terraformer" which extracts AWS into Terraform, which can then be turned into CloudFormation -or used directly as IaC.
https://github.com/GoogleCloudPlatform/terraformer#installation
Maybe that will work for others if the above tools don't.
I would like to make an automated call to a custom program API as soon as CloudFormation has completed the entire stack creation (deployment of instances, setup of VPC, Puppet scripts, etc.).
What is the correct way to go about this?
After some research, it seems a good option would be to launch an AWS Lambda function triggered by the event that stack creation has been completed successfully, but I have no idea how to approach this.
Any ideas or advice would be appreciated.
You can provision and coordinate a lot of what you're talking about (setup of VPC, etc.) with CloudFormation, the DependsOn attribute and nested CloudFormation stacks. This way you can order the execution of the CloudFormation stacks so that, for example, your VPC is created first followed by launching your EC2 instance(s) followed by the deployment of the software on the instance(s).
You can also coordinate the execution of the other behavior you mentioned (deployment [on] instances, [calling] Puppet scripts, etc.) using AWS::CloudFormation::Init. This way, you can call out to your Puppet scripts from your EC2 instance within the CloudFormation template. The actual execution of your Puppet scripts occurs on the EC2 instance(s).
If you want to see an example of calling out to a configuration management tool from CloudFormation (in this case, we're using Chef Solo), see app-instance.json.
If you'd like to see an example of using nested stacks, see dromedary-master.json.
There's also some examples of using Lambda on our blog as well (Stelligent), but it doesn't seem like you need to use Lambda in this case based on the problem you're trying to solve.
P.S. You don't have to use nested stacks either, but it can make things a little cleaner. But, you do want to control the creation order of the resources so the DependsOn attribute will help you in doing so.