Usecase
I have cloudformation template with resources and lambda functions . Usually the cloudformation will rollback the stack when there is failure in creating it.
But what if i had pushed some resources or lambda functions with wrong logic or with less configurations . In such case i want to rollback the stack to previous working stack.
I Know i have rollback triggers to monitor my stack and trigger alarms. But i want to do it manually because, what if i had missing that test scenario and because of that the alarms not triggered.
Is it possible with cloudformation ?
Thanks
Any help is appreciated
No you can't do that with Cloudformation.
Your CD pipeline should keep track of any code builds and template configuration and you should rollback with old/archived code by pushing out a new stack with that old code.
Related
We have an automated build and deploy pipeline using AWS CodePipeline and CloudFormation. Our stack contains a Dyanmodb resource which has critical data and should never be deleted with the stack. To prevent it from getting deleted we added a DeletionPolicy flag. This separates out the resource from the stack but we have other resources which depend upon this dynamodb resource.
So we updated our template.yml to import this resource back into the stack during deployment. Now during deployment the dynamodb resource gets imported successfully with status IMPORT_COMPLETE but we don't receive any events after that. The pipeline stays in deploy step for hours and no resource updates. There are no error messages also so its very hard to debug. Please help!
I am running a stack on cloudformation that creates some resources like Route53, etc...
I want to be able to recreate only some of the resources with the same value.
for example, One of the stack events creates image on ECR and I want to rebuild it. Run rollback on that event and then create it again with the same parameters.
How can I do that?
It is not possible to specify parts of a stack to 'rebuild'.
For some resources, you can modify an attribute to trigger a redeployment. The documentation will say Update requires: Replacement.
For other resources, you could:
Remove the resource from the template file
Update the stack with the template, which will cause CloudFormation to attempt to remove the resource (if it still exists)
Restore the template to the previous contents
Update the stack again, which will cause CloudFormation to deploy the 'new' resources
I wanted to update my stack. The stack failed with error Function not found: arn:aws:lambda....
And stack in status UPDATE_ROLLBACK_IN_PROGRESS more than 5 hours. How do I stop this process?
If you deleted the function outside of CloudFormation, then you can manually create a new function of the same name. This sometimes helps.
You can also wait till the rollback timeouts. And it usually does after a while, but the time varies.
Another reason why it gets stuck in this state could be due to nested stacks:
Nested Stacks are Stuck in UPDATE_COMPLETE_CLEANUP_IN_PROGRESS, UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS, or UPDATE_ROLLBACK_IN_PROGRESS
In this case a recommended option is indeed to contact support:
To fix the stack, contact AWS customer support.
Recent AWS blog post also describes the issue and possible solutions:
Why is my AWS CloudFormation stack stuck in the state CREATE_IN_PROGRESS, UPDATE_IN_PROGRESS, UPDATE_ROLLBACK_IN_PROGRESS, or DELETE_IN_PROGRESS?
Regarding the time to wait, the timeout varies:
In most situations, you must wait for your AWS CloudFormation stack to time out. The timeout length varies, and is based on the individual resource stabilization requirements that AWS CloudFormation waits for to reach the desired state.
In our case, we have mistakenly passed wrong image name to cloudformation template. After realising the mistake, we tried to stop the stack update, which made the stack stuck for forever in UPDATE_ROLLBACK_IN_PROGRESS status. SO during ECS service creation it got stuck.
Solution:
in Stack event check in which step is in progress. (our case ECS service update)
Go to ECS service.
Click on Update service.
Choose older task definitions.
And Update.
Your Task definition is reset to previous version. And roll back will complete successfully.
Lets say I have a CloudFormation stack running, which creates and deploys an Lambda function. In the AWS Console, if I connect my Lambda function to an API in API Gateway, will my CloudFormation Template be updated immediately if the Lambda function successfully integrates with the API?
It's a one way traffic from Cloudformation to resources.
Meaning if you modify your Cloudformation template and update the stack then the resources that were created by Cloudformation get modified/updated. However the other way is not true. Meaning if you modify your resources the Cloudformation template does not get updated.
Moreover, as a good practice you should avoid modifying the resources directly because you may end up breaking the Cloudformation's update stack functionality for that that stack
I create CloudFormation template for our resources, it includes Lambda functions, API Gateways, Roles, etc. For verifying our template I create CloudFormation stack using it, check some resources which I updated and after that I delete stack. But last time I got such message when I tried to delete stack:
CloudFormation is waiting for NetworkInterfaces associated with the
Lambda Function to be cleaned up.
I tried to stop deletion process and restart it, but I faced the same issue again. What is the problem and how can I fix that?
This is a well known issue. There are couple of things you can do.
1) Wait for the deletion to fail. Then try to delete. It should show you a checkbox to skip NetworkInterface. Select that.
2) Go to EC2-->NetworkInterfaces and detach/delete the NIC that was used by your resources. Then delete your CFT stack.