I'm new to a large AWS deployment where stuff is mostly deployed through CloudFormation (and some through Terraform). But there are always cases where something has been deployed manually and not through code. Is there a reliable way to quickly figure out if a resource (say, an EC2 instance) already existing in the deployment was deployed through IaC or manually? A CloudFormation-specific answer will be good enough for now.
Going through literally hundreds of CloudFormation stacks manually and looking for the resource is not an option.
You can identify the resources created by cloudformation. Cloudformation applies few default tags as mentioned here
aws:cloudformation:logical-id
aws:cloudformation:stack-id
aws:cloudformation:stack-name
You can run a script to check whether the resource contain one/all of these tags to update your count.
Offical documentation on resource tags
Unfortunately looking at an AWS resource you don't see how it got created. While some resources might have been tagged by CloudFormation indicating that they got created by a CloudFormation stack, that's only valid for a subset of resources.
The only reliable way to figure out whether or not a resource got created via a CloudFormation stack is to go through all CloudFormation stacks and check whether or not the resource in question is a part of it. While that might be cumbersome when doing manually, it's also something you can automate using the AWS CLI.
Related
As described in the documentation, I think CloudFormation will update with no interruption just by changing to add a tag.
In my case, another team uses terraform to add tag resources with generic tags, and my team uses CloudFormation to update the application-specific tags. Does this cause replacement...?
Also, this only occurred for limited resources such as security groups.
Does anyone know anything about this issue?
Cloudformation is a Jealous Tool - in otherwords, if it doesn't control the Resource in entirety it will overwrite changes, assuming that the changes are in a location it thinks it has to update.
In order for CloudFormation stacks not to do work that is not needed they generate ChangeSets - indicating what resources need updating/rebuilding/ect. If you don't change anything in the template (or your cdk stack) then the ChangeSet won't update that resource cause nothing needs to change. Depending on the resource (and it varies from service to service) sometimes changes made outside of CloudFormation control (by either other services like Terraform or by manual changes in the console - both of which are referred to as 'Drift') dont get overwritten by the stack updating (most common I can think of is API gateway - adding / deleting resources or methods doesn't always get re done by a redeploy of the stack.
However, for things like tags, yes - it will overwrite any changes made in there at any time and reapply the tags as of the moment it deploys - (again, depending on the Service) often even if there are no other changes to deploy for that given resource. I suspect the reason for this is because of the way AWS uses tags on the backend to sort and search resources.
Your best bet is to pick one service and do everything with it.
Each of these two tools will overwrite the tags the other created. You can configure Terraform to ignore certain tags However I'm not sure you can configure CloudFormation to ignore any tags. I think CloudFormation will always delete tags that it doesn't manage. In general it's not going to work well using both CloudFormation and Terraform to manage the same resources.
Cloudformation doesn't support tags for event rules, and I heard somewhere that there's a workaround using cloud formation stacks but I haven't been able to find how exactly.
I know I could use AWS CLI, or tag the resource manually after it's created but I would like to know if there is a workaround using CloudFormation only.
Thanks.
You can create a custom resource using AWS CloudFormation.
Here’s a blog post describing the process and a GitHub project that you might want to look into for examples.
I have created an ec2 instance and configured it as a target behind my load balancer. I want to convert this entire environment as something that can be deployed repeatedly in automation. I have looked at cloudformation but don't know if it can help me with converting this environment to a cloudformation template. Is there any other way to achieve this. If there is an approach outside of Cloudformation, that's fine too.
PS: I am new to AWS and it's capabilities
Thanks
Have a look at Former2.
Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account.
You need an IaC (Infrastructure as Code) tool. Cloudformation is one of them, but there are plenty others. Terraform, Pulumi or even the AWS CDK.
Look at Infrastructure as Code try AWS CDK, Terraform, you should also look at methods for replacing existing infrastructure. Spinning a fresh set of infrastructure along side the existing one and swapping out in DNS is the most common of approach.
As part of infra automation we are using cloudformation for automating the AWS infrastructure. We are utilising the service ElaticTranscoder as well, as i understand cloudformation yet does not provide support for ElasticTranscoder, is there any efficient way to automate ElasticTranscoder using cloudformation.
Custom resources provide a way for you to write custom provisioning logic in AWS CloudFormation template and have AWS CloudFormation run it during a stack operation, such as when you create, update or delete a stack.
Check out this example.
Also, a quick google search gives me this result.
Another option is not using CloudFormation. You can use Terraform which does support Elastic Transcoder.
https://www.terraform.io/docs/providers/aws/r/elastic_transcoder_pipeline.html
Ansible also has third-party support for it.
https://github.com/wimnat/ansible-modules/blob/master/elastictranscoder/elastictranscoder.py
Last but not least, you can vote for this feature in AWS wish list by liking or retweeting the request.
https://twitter.com/search?q=%23awswishlist%20transcoder&src=typd
https://awswishlist.com/
I would like to make an automated call to a custom program API as soon as CloudFormation has completed the entire stack creation (deployment of instances, setup of VPC, Puppet scripts, etc.).
What is the correct way to go about this?
After some research, it seems a good option would be to launch an AWS Lambda function triggered by the event that stack creation has been completed successfully, but I have no idea how to approach this.
Any ideas or advice would be appreciated.
You can provision and coordinate a lot of what you're talking about (setup of VPC, etc.) with CloudFormation, the DependsOn attribute and nested CloudFormation stacks. This way you can order the execution of the CloudFormation stacks so that, for example, your VPC is created first followed by launching your EC2 instance(s) followed by the deployment of the software on the instance(s).
You can also coordinate the execution of the other behavior you mentioned (deployment [on] instances, [calling] Puppet scripts, etc.) using AWS::CloudFormation::Init. This way, you can call out to your Puppet scripts from your EC2 instance within the CloudFormation template. The actual execution of your Puppet scripts occurs on the EC2 instance(s).
If you want to see an example of calling out to a configuration management tool from CloudFormation (in this case, we're using Chef Solo), see app-instance.json.
If you'd like to see an example of using nested stacks, see dromedary-master.json.
There's also some examples of using Lambda on our blog as well (Stelligent), but it doesn't seem like you need to use Lambda in this case based on the problem you're trying to solve.
P.S. You don't have to use nested stacks either, but it can make things a little cleaner. But, you do want to control the creation order of the resources so the DependsOn attribute will help you in doing so.