I'm trying to do the following: the parent Stacks launches the first child Stack which creates a fully configured EC2 instance. Once that is completed, the parent Stack kicks off a second Stack that uses a Lambda function to create an AMI, which is then used for an AutoScaling setup even further downstream. This is working perfectly.
Now the challenge: when I update the metadata for the EC2 instance from the first child Stack I would really like the second Stack to be triggered. In other words: I want to be able to change the seed instance and have the CloudFormation Stack update, creating a new AMI.
I'm able to get the seed instance to update, but the second child Stack isn't triggered :-(
I've Google everything I could think of, but Update Policy doesn't apply, manually kicking off the second child is defying the point of having nested Stacks and I'm pretty sure I'm missing some obvious feature or clever trick, so I'm asking you guys to help me out. Please.
Have you tried using Lambda-backed custom resource? You can have the service token of resource as the Lambda and use DependsOn with the first nested stack. It will kick off whenever the CF script runs or updates.
You can also lookup the stack itself from the Lambda function to determine any changes if you want.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html
Related
I have a script which is going to create a new LaunchConfiguration with new AMI_ID and retaining all the existing metadata like user_data etc. Then I'm triggering a ASG instance refresh to launch instances with the new LC. What I'm concerned is, as I'm doing things outside of cfn what is going to happen when the cfn stack update gonna happen next time? is this change is going to be considered a drift? just wanted to make sure that this LC creation and updating the ASG don't affect things with CFN.
is this change is going to be considered a drift?
Yes. You are causing a drift. When you update your stack, the ASG will be reverted back to the state it is in the template, not whatever you made it to be outside of CFN.
Its not a good practice to manually change your resources outside of CFN. You will have to update your template to match your current ASG state.
I am using cloudformation to deploy/update some stacks, including RDS. At the moment I successfully created my database and I want to update the port since I set the wrong one.
Reading the docs I encounter that by updating it via cloudformation, I must rename the stack in order to be able to update the port, therefore it will just create a new RDS instance with the new port and destroy the old one, this means I must create a backup first and later restore it.
I was reading that updating the port "manually" from the AWS console does not behave that way and no data loss should happen, so far I have not tried.
So, for example:
My deployed cloudformation is the following:
cloudformation template -> port 123
deployed stack -> port 123
I need to update it to port 234
then my cloudformation will still know the latest deploy happen to port 123
deployed stack -> 234 (the updated via aws console)
My question is the following: In the case, I am able to update the port via AWS Console, what will happen with the cloudformation template? should I update the template to have port 234 or I should leave it with port 123 so it wont try to make an update? (that would cause inconsistency on what I have at the template and what is really deployed but I wonder if would work)
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html#update-replacement
I dont have problems with all the vpc, security groups, rules, etc, its basically just the port.
Any idea about this?
Modify resources manually outside in AWS console (or CLI, SDK) which were created by CFN is a bad practice which results in a stack drift. The drift can lead to future issues; from docs:
Regardless, changes made outside of CloudFormation can complicate stack update or deletion operations.
Specifically to answer your question:
In the case, I am able to update the port via AWS Console, what will happen with the cloudformation template?
Nothing will happen per-se. CFN will not be aware by itself of any changes made to your RDS outside of CFN. But your stack will be considered as drifted, which, as AWS docs write, can lead to issues. Thus its better to avoid such situation.
should I update the template to have port 234 or I should leave it with port 123 so it wont try to make an update?
Yes, if you want to avoid drift. But updating port through CFN will lead to replacement of your db instance and data lost. So you must take all precautions on how to do it. Best would be to create some dummy RDS db, and do a test-run on it to understand the procedure and consequences of changing its port.
I'm trying to automate the creation of a self terminating AWS EC2 instance using AWS LAMBDA. Self-Terminating AWS EC2 Instance?
posted sometime back was helpful to do it via console but I was wondering if this is possible with AWS LAMBDA.
To give some additional context, I'm trying to create a workflow like this:
Get user registration through a unique link sent to them (A simple static intranet website)
POST the form details to AWS LAMBDA function and create an instance from a snapshot which will automatically terminate in say 12 hours
Send the user an email with the connection information (Possibly attaching a RDP file)
The issue I face now is with the second point. I also need to cap the registrations to 10 per day.
PS: Any recommendations or advice on this workflow will be helpful too. (I understand the security issues but this is for intranet and will never be visible to the public)
There are many ways to achieve what you want. Two ways I can think of:
When you launch the instance from Lambda, pass userdata to the instance that sets up a cronjob to terminate the instance using AWS CLI after 12 hours. You can get the instance's id from AWS metadata server. For this you need to have AWS CLI installed and the EC2 instance is attached an IAM role with sufficient privileges to terminate the instance. Or if Python is already preinstalled in that instance, you can write a small python script using Boto3 that terminates the instance 12 hours after the instance is launched.
Easiest and the cleanest option - Schedule a lambda event to fire 12 hours later. You need to pass the instance-id to the second lambda which can terminate the previously launched instance.
In both cases, you need to attach sufficient privileges to the lambda function or to the launched instance to terminate the instance.
These are the step I'd follow
Steps thinking on Microservice, however, you can implement this approach in one Lambda function only.
Create a either a DynamoDB table or RDS instance table to store the instance id TTL (Time to live), and the creation time.
Create a Lambda function with a specific purpose: Terminate instances by instance id.
Create a Lambda function to check the instances according their creation date.
Create a Schedule Cloudwatch event rule that will execute a Lambda function (Created Lambda in step 3) every 10, 30, or whatever N minutes (I recommend a good time interval regarding the amount of instances which will need to be removed at a specific time) to check the created date of instances.
If the creation date of a specific instance is greater or equals to TTL (Time to live), invoke a Lambda function (Created Lambda in step 2) for deletion of that instance.
Wait for Lambda invocation and then delete the table's row related to that instance.
Hope it helps to accomplish your scenario
Another way to do this is to create an EC2 instance from your lambda function. When you call the AWS API to create the EC2 instance, provide a userData script. (You'll probably need to anyway to install whatever software you need).
As part of your userData script, run shutdown -H +12, which will schedule a shutdown event of the machine via Old School Unix Commands.
If you want more detail, take a look at the blog post I stole the shutdown idea from
Lets say an AWS stack was created using CloudFormation.
Now one of those resources was modified outside CloudFormation.
1) Is it possible to have CloudFormation specifically create those resources? Based on my understanding, we can't do that because CloudFormation does not identify a difference, and so does not create the modified resources. Is my observation correct?
2) Also, what options do I have to revert a stack to its original state, if modified outside CloudFormation?
This is one possible hack you could use without deleting the entire stack.
From the template remove the specific resource which got deleted accidentally.
Now update the stack which makes your stack and resources in your account in sync.
Revert the template to its state before step1 and update again which will create the resource which got deleted accidentally.
Unfortunately the answer for both your questions is NO.
If you modify the resources in the stack after stack creation status is COMPLETE, there is nothing CF can do since it doesn't keep track of modification to resources
You have no option other than deleting the current stack and create a new one
First, beware that modifying CloudFormation-created resources outside of CloudFormation is explicitly discouraged, according to AWS CloudFormation Best Practices:
Manage All Stack Resources Through AWS CloudFormation
After you launch a stack, use the AWS CloudFormation console, API, or AWS CLI to update resources in your stack. Do not make changes to stack resources outside of AWS CloudFormation. Doing so can create a mismatch between your stack's template and the current state of your stack resources, which can cause errors if you update or delete the stack.
However, if you've modified a CloudFormation-managed resource accidentally and need to recover, you may have some limited options beyond simply deleting and re-creating the stack altogether (which may not be an acceptable option):
It is not possible for CloudFormation to automatically update its internal state based on the current state of an externally-modified resource.
However, depending on the exact resource type, in some cases you can manually update CloudFormation afterwards by applying a stack update that matches the current state of the resource.
Similarly, it is not possible for CloudFormation to automatically revert an externally-modified resource back to its original unmodified CloudFormation state.
However, depending on the exact resource type, in some cases you can either:
Revert a resource by manually updating the resource back to its original state;
Update the resource by applying a stack update, bringing both the CloudFormation stack and the managed resource to an altogether new state that will once again be in sync.
To force the EC2 re-creating, I do use a simple trick, when I'm deploying, I jump between AMI's IDs (I took two similar AMI's ID), that had helped me when I'm testing user data or things that I want to test during the EC2 bootstrap. Again, it just works for EC2.
Unfortunately, the answer is NO
if you made changes in the stack after the creation, Cloudformation can't track those changes.
if you need to revert those changes, you must delete the stack and rebuild.
How can I force delete my AWS cloud formation stack? The tied resources were already deleted previously like the DNS, route53, etc. Whenever I delete the stack, it failed because it couldn't find the resources.
The specified hosted zone does not exist. DELETE_FAILED.
This made the deletion halt and made the stack still active. How can I force clean-up and remove this stack?
Based on this thread, it appears that stack deletion can be problematic if the stack is altered.
https://forums.aws.amazon.com/thread.jspa?threadID=71050&start=0&tstart=0
Looks like you need to go via Support to get it deleted.
That said, I have been able to delete stacks where I have manually altered the components. Sometimes you need to wait a couple of hours after you terminate instances, VPCs, subnets etc before the stack deletion actually succeeds.
AWS Cloud formation stack does not get deleted until all dependency resources either used in other stack or busy with non stack resources. So better see which resource is shared some where else and try to release that.
hope that help.
It seems the Route53 records were either deleted manually before cloudformation or there was some failure within cloudformation after it deleted the record. Its stuck because it can't find the hosted zone because it was already deleted if indeed this is what was described in your post. You can try to update the stack with the original script and then delete the stack via cloudformation without any manual intervention.
Check your IAM role and the policy, review the privileges that you have associated with the role, so that if there is any permission issue, that should resolve the issue.
Refer: https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-stack-delete-failed/
Need to check the reason for failure in stack details
Either delete this manually, or retain those resources while eleteing
eg:
$ aws cloudformation delete-stack --stack-name my-stack --retain-resources myresource1 myresource2