I have a lab setup that in aws, with a vpc, an IGW a few different subnets and some ec2 instances, nothing to crazy.
I am trying to export this VPC with everything inside, but can't figure out how to do it.
I tried the cloudformer, but i keep getting errors when trying to create a stack, its keeps saying that i have reached my limit with VPC and IGW.
Is there something better to use that can export this VPC, with all ec2 and everything configured in those ec2 instances.
Terraform, and in particular the TR import tool called Terraforming are very good for this type of work.
Though there is a learning curve associated with Terraform, and it's probably a good idea to start gradually with it.
Unfortunately, there is no such tool or functionality provided by AWS.
CloudFormer is an old, no longer maintained and non-reliable project (in beta for years) as you already experienced.
You may also hear about importing existing resources into CloudFormation. But this is also not useful in your use case, as this works with only some resources and requires manual preparation of templates for resources to be imported.
The only choice is to look for tools outside of AWS. #AbiDembak alread recommended one such tool.
Related
I have created an ec2 instance and configured it as a target behind my load balancer. I want to convert this entire environment as something that can be deployed repeatedly in automation. I have looked at cloudformation but don't know if it can help me with converting this environment to a cloudformation template. Is there any other way to achieve this. If there is an approach outside of Cloudformation, that's fine too.
PS: I am new to AWS and it's capabilities
Thanks
Have a look at Former2.
Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account.
You need an IaC (Infrastructure as Code) tool. Cloudformation is one of them, but there are plenty others. Terraform, Pulumi or even the AWS CDK.
Look at Infrastructure as Code try AWS CDK, Terraform, you should also look at methods for replacing existing infrastructure. Spinning a fresh set of infrastructure along side the existing one and swapping out in DNS is the most common of approach.
I was wondering if anybody has had any experience creating a cloudformation template from exisiting AWS resources.
I am currently trying to migrate from a classic elb to a alb using the wizard. However I already have cloudformation templates managed by github. Therefore I would need to add the alb in after it has been created. I tried using cloudformer but it doesn't appear to support alb whereas it does pickup classic.
Has anybody had experience migrating elbs and creating cloudformation templates from existing resources?
Many thanks!
AWS::ElasticLoadBalancingV2::LoadBalancer is one of the resources that can be imported into CloudForamtion. But sadly AWS::ElasticLoadBalancingV2::TargetGroup can't be imported.
Importing is a try-and-see operation. It is not automated as many people expect it to be. The reason is that you have to manually create the template for the resources being imported. What's more, the attributes of the resources in the template created must match exactly existing resources.
CloudFormer is not helpful these days. Its not maintained by AWS anymore and has been in beta for years.
If you haven't tried importing anything before, the best way would be to start with AWS tutorial: Importing Existing Resources Into a Stack
This way you can start with something simple, before you move to ALB with all its listeners and listener rules. Off course you have to create new Target Group as well as it can't be imported sadly.
I have an existing VPC with a few EC2 instances already created and running with security groups, route tables, NACLs all applied where required. Is there a way to convert a setup like this into a cloudformation stack or a terraform equivalent of configuration files?
Update: A modern substitute for CloudFormer is Former2.
You can use CloudFormer, but it doesn't seem to be maintained much anymore.
See: Using CloudFormer (Beta) to Create AWS CloudFormation Templates from Existing AWS Resources - AWS CloudFormation
While that type of capability might sound like a good idea, it is quite difficult to make a template from running infrastructure. It runs into questions, such as:
Should the definition for an Amazon EC2 instance reference the specific Security Group that currently exists, or should it create another one and reference that new one?
Should it create a new VPC, or launch resources in the existing VPC?
Was an instance created directly, or was it launched by Auto Scaling?
In the end, it's probably easier to just write the CloudFormation template yourself, rather than spend time trying to 'fix' one that is automatically generated.
Some tips for writing CloudFormation templates:
Never write them by hand. Always copy the templates or examples from the documentation.
Copy snippets from your existing templates rather than doing them all from scratch again.
Use YAML rather than JSON (less errors due to unbalanced braces)
I have an AWS website that is running inside a private subnet and I am not sure what the best way is to update it.
I would like something that is non-burdensome, ideally it would be nice to have some EC2 Box (with security groups only allowing select IP's to connect too) running the development page, and then I could simply copy it over to the private EC2 Box with a click of a button.
I am not too familiar with best practices, but the idea of connecting through several EC2 boxes seems burdensome.
Thank You!
Sounds like you might want to make use of AWS CodeDeploy. There are other tools as well, but since you are already on/using AWS this might be a good one to start with:
AWS CodeDeploy is a service that automates code deployments to any
instance, including Amazon EC2 instances and instances running
on-premises. AWS CodeDeploy makes it easier for you to rapidly release
new features, helps you avoid downtime during application deployment,
and handles the complexity of updating your applications. You can use
AWS CodeDeploy to automate software deployments, eliminating the need
for error-prone manual operations, and the service scales with your
infrastructure so you can easily deploy to one instance or thousands.
https://aws.amazon.com/codedeploy/
The purpose is production-level deployment of a 8-container application, using swarm.
It seems (ECS aside) we are faced with 2 options:
Use the so called docker-for-aws that does (swarm) provisioning via a cloudformation template.
Set up our VPC as usual, install docker engines, bootstrap the swarm (via init/join etc) and deploy our application in normal EC2 instances.
Is the only difference between these two approaches the swarm bootstrap performed by docker-for-aws?
Any other benefits of docker-for-aws compared to a normal AWS VPC provisioning?
Thx
If you need to provide a portability across different cloud providers - go with AWS CloudFormation template provided by Docker team. If you only need to run on AWS - ECS should be fine. But you will need to spend a bit of time on figuring out how service discovery works there. Benefit of Swarm is that they made it fairly simple, just access your services via their service name like they were DNS names with built-in load-balancing.
It's fairly easy to automate new environment creation with it and if you need to go let's say Azure or Google Cloud later - you simply use template for them to get your docker cluster ready.
Docker team has put quite a few things into that template and you really don't want to re-create them yourself unless you really have to. For instance if you don't use static IPs for your infra (fairly typical scenario) and one of the managers dies - you can't just restart it. You will need to manually re-join it to the cluster. Docker for AWS handles that through IPs sync via DynamoDB and uses other provider specific techniques to make failover / recovery work smoothly. Another example is logging - they push your logs automatically into CloudWatch, which is very handy.
A few tips on automating your environment provisioning if you go with Swarm template:
Use some infra automation tool to create VPC per environment. Use some template provided by that tool so you don't write too much yourself. Using a separate VPC makes all environment very isolated and easier to work with, less chance to screw something up. Also, you're likely to add more elements into those environments later, such as RDS. If you control your VPC creation it's easier to do that and keep all related resources under the same one. Let's say DEV1 environment's DB is in DEV1 VPC
Hook up running AWS Cloud Formation template provided by docker to provision a Swarm cluster within this VPC (they have a separate template for that)
My preference for automation is Terraform. It lets me to describe a desired state of infrastructure rather than on how to achieve it.
I would say no, there are basically no other benefits.
However, if you want to achieve all/several of the things that the docker-for-aws template provides I believe your second bullet point should contain a bit more.
E.g.
Logging to CloudWatch
Setting up EFS for persistence/sharing
Creating subnets and route tables
Creating and configuring elastic load balancers
Basic auto scaling for your nodes
and probably more that I do not recall right now.
The template also ingests a bunch of information about related resources to your EC2 instances to make it readily available for all Docker services.
I have been using the docker-for-aws template at work and have grown to appreciate a lot of what it automates. And what I do not appreciate I change, with the official template as a base.
I would go with ECS over a roll your own solution. Unless your organization has the effort available to re-engineer the services and integrations AWS offers as part of the offerings; you would be artificially painting yourself into a corner for future changes. Do not re-invent the wheel comes to mind here.
Basically what #Jonatan states. Building the solutions to integrate what is already available is...a trial of pain when you could be working on other parts of your business / application.