I'm new to Cloud Formation. I want to update the settings of already created a lot of RDS instances using Cloud Formation. I don't have the info about either those resources were created through CF or manually. Is it possible to update such resources with CF?
I can think of another way like I can use AWS SDK (boto3) but doing it with CF is perefrable.
The only way to do this from CloudFormation (CF) is to develop your own CF custom resource. This will be a lambda function which will use AWS SDK to query the state of your RDS databases, and perform any actions you want.
Since its fully custom, you can program any logic which satisfies your requirements.
If the resources were created manually, you can also import them to CF, and then update using CF.
Related
I have a bunch of AWS resources (ec2 instances, rds, s3, etc.)
Those resources were created manually over the years in AWS console.
Now I would like to duplicate this environment using CloudFormation. What is the best approach? Is there a tool, that converts all the resources into a cloudformation stack or template?
I couldn't find anything, or maybe I didn't understand the process correctly...
These days you would use a third party, free and fully open-source tool called former2 developed by renovated AWS Hero. The former2 is used by corporate clients of AWS as explained in the AWS blog post.
You could potentially try the AWS Console Recorder extension for Chrome/Firefox which supposedly could create CloudFormation templates based on your AWS Console clicks.
From their README:
Records actions made in the AWS Management Console and outputs the equivalent CLI/SDK commands and CloudFormation/Terraform templates.
Caveat:
Not all resources are supported.
There was a service named CloudFormer that could perform this action but has since become deprecated.
Functionality does exist within CloudFormation to create a stack from existing resources.
However, to use this you will want to design the stack to use the same options and setup as your resources. Once this is completed you could then manage these resources via CloudFormation.
More information is available in the Import Existing Resources into a CloudFormation Stack blog post.
Using the CDK is it possible to get an existing CfnDBCluster to make modifications to?
I have an AWS::RDS::DBCluster in CloudFormation whose TimeoutAction I want to change (CloudFormation doesn't support it and I don't want to use the AWS cli).
CDK doesn't natively support importing existing resources for modification.
https://medium.com/#visya/how-to-import-existing-aws-resources-into-cdk-stack-f1cea491e9
This article describes using CDK to generate the template, then use the AWS Management Console to import the resource into the stack.
Here is an issue to track the support within CDK itself: https://github.com/aws/aws-cdk-rfcs/issues/52
In your case, specifically since DB Clusters support this, you could create snapshot of the database then delete it. Then reference the snapshot id when creating recreating the cluster with CDK. Obviously it would require downtime though.
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-rds.DatabaseClusterFromSnapshot.html
I am familiar with AWS SDKs (Python/Java) and the need for the same for a lot of services like S3, DDB, KMS etc.
Are there any valid use cases for using the AWS Java SDK e.g. to programmatically spin off an EC2 instance or an EMR cluster or Redshift cluster or RDS instance or for that matter any resource that requires setting up of an infrastructure/cluster?
If AWS console is not an option, and programmatic access is all we have then, don't we always end up using AWS CLI for corresponding services or CloudFormation or Terraform for that matter?
Generally its best practice to manage any infrastructure/service deployments by using an infrastructure as code solutions such as CloudFormation, CDK (which generates CloudFormation stacks under the hood) or Terraform.
Whilst you could use the SDK to create this services (and create a solution that is similar to the solutions above) you will be building a lot of functionality that other services have already created which would put more ownership on you to fix it if you want to support another service etc.
The SDKs simply provide every AWS API interaction in a programmatic way, even under the hood CloudFormation and Terraform will likely be using the SDKs to programmatically create the resources (although I am speculating) but would then add additional functionality such as state management and drift detection on top of this.
I only have seen services being created via the SDKs when that service is not available in the selected tool, and even then it would generally be wrapped within that tool (such as custom resources for CloudFormation).
In summary, yes you could use the SDK to generate these but unless there is a specific usecase to use the SDK I would advise using a tool that already manages this so you can focus more on your infrastructure/applications.
The AWS CLI is built using the AWS SDK for Python. Terraform is built using the AWS SDK for GoLang. You may want to stick with higher level infrastructure-as-code tools, but those tools wouldn't exist without the SDKs, and if you wanted to build a tool like that you would most likely build it on top of one of the SDKs.
There are also quite a few use-cases I've seen discussed here on StackOverflow for performing infrastructure automation through AWS Lambda, for example periodically stopping and starting RDS instances, where using one of the AWS SDKs in the Lambda code would likely be much easier than trying to get Terraform to run inside a Lambda function.
I am creating a EMR cluster through cloud formation .
I have already created security configuration from AWS management console .
I am not able to find any way where i can add this security config while creating EMR from cloud formation.
Alternatively i can leverage lambda function , but how to add a EMR security configuration after cluster is created ?
any help would be appreciated...
Unfortunately this is not yet possible with cloudformation. Security Configurations were released September of last year and Cloudformation has yet to support it. At this time the only way would be to do it manually after the creation or via the lambda or other method.
As of today, this is only possible using Lambda backed custom resources. This is what you'll need to do on a high-level:
Create EMR cluster using CloudFormation supported AWS::EMR::Cluster resource type.
Define a Lambda backed custom resource, say with resource type Custom::EMRSecurityConfiguration.
Define a Lambda function that'll be used in step #2 and do the needful by using AWS SDK (e.g. boto3 if you're writing your Lambda in Python) and actually create/update/delete the security configuration based on the kind of EventType (the value for this will be passed in by CloudFormation to your function).
Lambda function defined in #3 will be triggered by CloudFormation everytime you create/update/delete the custom resource defined in #2. I'd recommend looking at AWS docs for Lambda-backed custom resources and security configuration.
I want to migrate my existing VPC, subnets etc from one amazon account to another amazon account using cloud formation.
How can i do this?
If you have a CloudFormation template for your VPC environment already, then you can simply create a new stack using that same template in another AWS account.
However, this will create a copy of your VPC environment as it was when it was initially created. Any changes done to the VPC since it was created using CloudFormation will not be included. This will include the acquisition of data in a database, for example.
If you do not already have a CloudFormation template, you can try to create one using AWS Cloud Former. Cloud Former can be used to examine your AWS environment and create a CloudFormation template from what it sees.
Instructions for running AWS Cloud Former can be found in the AWS Documentation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html