Brand new to AWS and trying to put together a stack with a CloudFormation template.
The stack will have two EC2 Instances with a windows service running on each. Some of the storage will be on S3 and some will be on AWS Glacier.
I can't find samples or instructions on how to add Glacier as Resource in the CF Template.
Am I missing something and this is not possible through a CF template?
Has anyone done this before, and can someone provide a sample, if it is possible?
Thanks.
As of 2013-02-27, CloudFormation does not support Glacier.
If/when it does, you'll see Glacier show up in the CloudFormation resource types documentation here:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html
Any support for auto-migration from S3 to Glacier should show up here:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html
To register your desire for Amazon to work on this feature, add a +1 comment in this forum thread:
https://forums.aws.amazon.com/thread.jspa?threadID=117947
Workaround would be following:
Instead you can user the S3 rules to attach a Glacier policy OR S3 Lifecycle Rule. This rule will move the object automatically to the Glacier or even delete object after some time.
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-lifecycleconfig-rule.html
AWS CloudFormation has added support for custom resources, where you can use AWS Lambda functions to do the job of creating resources that CloudFormation doesn't natively support. The resource type in the CloudFormation template file would end up being AWS::CloudFormation::CustomResource or Custom::String
For more info, check these official AWS docs:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cfn-customresource.html?shortFooter=true
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html?shortFooter=true
Related
I am creating cloudFormation stack for s3 bucket (with the help of yaml template file). Is there a way by which we can automatically delete the created buckets? Can we configure the yaml templates such that the s3 bucket gets deleted after some time of its creation? If not what is the best way to programmaticaly delete the s3 buckets?
Tried to add
DeletionPolicy: Delete
But it is for retention of deleted files.
The best way to achieve this is to create a cloudwatch event at a specific time that would trigger a lambda. This lambda could delete the file in the bucket and then delete the bucket.
You can build all that using cloudformation template.
As far as I know
DeletePolicy
is used to determine what will happen when a bucket gets deleted.
You should look into lifecycle policy instead if you want some sort of time based automation behind it.
Here are some examples:
AWS DOCS
Is there any way to rollback all the changes made by an AWS Lambda by using the SDK? E.g. the lambda created and launched by a CloudFormation template creates a bucket via the AWS SDK. In case something fails it would be great to have a 'stack rollback' for the same stack that deployed the lambda as well (and all the resources created by the lambda reverted as well).
Or alternatively: how can I 'remember' from my lambda which resources were created so that I can rollback them and delete them when the lambda is called afterwards with a 'Delete' event?
I'm assuming you mean custom resources, as that's the only way you can run scripts in cloudformation.
Custom resources have a property called pysicalReourceId. You can use it after your create event to provide info over the resource you've created. When updating or deleting the resource, the id is provided to the lambda event so you can use it. A guide can also be found here: https://advancedweb.hu/how-to-use-the-physicalresourceid-for-cloudformation-custom-resources/
If for some reason it's not possible to use the resource ID I'd use tagging. When creating, tag your resources and when deleting, fetch the resources based on their tag and delete them.
Fairly new to cloudformation templating but all I am looking to create a template where I create a S3 bucket and import contents into that bucket from another S3 bucket from a different account (that is also mine). I realize CloudFormation does not natively supports importing contents into S3 bucket, and I have to utilize custom resource. I could not find any reference/resources that does such tasks. Hoping someone could point out some examples or maybe even some guidance as to how to tackle this.
Thank you very much!
Can't provide full code, but can provide some guidance. There are few ways of doing this, but I will list one:
Create a bucket policy for the bucket in the second account. The policy should allow the first account (one with cfn) to read it. There are many resources on doing this. One from AWS is here.
Create a standalone lambda function in the first account with execution role allowing it to the read bucket from the second account. This is not a custom resource yet. The purpose of this lambda function is to test the cross-account permissions, and your code which reads objects from it. This is like a test function to sort out all the permissions and polish object copying code from one bucket to other.
Once your lambda function works as intended, you modify it (or create new one) as a custom resource in CFN. As a custom resource, it will need to take your newly created bucket in cfn as one of its arguments. For easier creation of custom resources this aws helper can be used.
Note, that the lambda execution timeout is 15 minutes. Depending on how many objects you have, it may be not enough.
Hope this helps.
If Custom Resources scare you, then a simpler way is to launch an Amazon EC2 instance with a startup script specified via User Data.
The CloudFormation template can 'insert' the name of the new bucket into the script by referencing the bucket resource that was created. The script could then run an AWS CLI command to copy the files across.
Plus, it's not expensive. A t3.micro instance is about 1c/hour and it is charged per second, so it's pretty darn close to free.
I've been working with CloudFormation YAML for awhile and have found it to be comprehensive - until now. I'm struggling in trying to use SAM/CloudFormation to create a Lambda function that is triggered whenever an object is added to an existing S3 bucket.
All of the examples I've seen thus far seem to require that you create the bucket in the same CloudFormation script as you create the Lambda function. This doesn't work for me, because we have a design goal to be able to use CloudFormation redeploy our entire stack to different regions or AWS accounts and quickly stand up our application. S3 bucket names must be globally unique, so if I create the bucket in CloudFormation, the script will break when I try to deploy it to a different region/account. I could probably get around this by creating buckets with the account name/region in the name, but that's just not desirable from a bucket sprawl perspective.
So, does anyone have a solution for creating a Lambda function in CloudFormation that is triggered by objects being written to an existing S3 bucket?
Thanks!
This is impossible, according to the SAM team. This is something which the underlying CloudFormation service can't do.
There is a possible workaround, if you implement a Custom resource which would trigger a separate Lambda function to modify the existing bucket and link it to the Lambda function that you want to deploy.
As "implement a Custom Resource" isn't very specific: Here is an AWS github repo with scaffold code to help write it, and then you declare something like the following in your template (where LambdaToBucket) is the custom function you wrote. I've found that you need to configure two things in that function: one is a bucket notification configuration on the bucket (saying tell Lambda about changes), the other is a Lambda Permission on the function (saying allow invocations from S3).
Resources:
JoinLambdaToBucket:
Type: Custom::JoinLambdaToExistingBucket
Properties:
ServiceToken: !GetAtt LambdaToBucket.Arn
Is it possible to send a SNS notification after the CFT completion in AWS ? Is there any way to get the progress of the launching CFT in AWS.
When create resources using a CF template there is an Advanced section of the Options menu. From there you can set Notification options using SNS and Topics.
When you start the CF process you can also view the status and importantly where the template might have failed.
You cannot specify notification ARNs via a CloudFormation template itself. You can specify them if you use the console to create the stack creation. But you cannot use the console to update the ARNs once the stack has been created. You can, however, use aws-cli to update the stack with notifications ARNs once it has been created, eg:
aws cloudformation update-stack --stack-name stack-name --use-previous-template --notification-arns "arn:aws:sns:us-east-1:${ACCOUNT_ID}:${TOPIC_NAME}"
Replace the variable ${VARIABLE} with the literal values from your account.
There's also knowledge center article from AWS where you can replace ROLLBACK_IN_PROGRESS statement with any other state of CloudFormation to get SNS Notification.
You can trick CloudFormation into sending SNS messages from inside the template:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html
Custom resources enable you to write custom provisioning logic in
templates that AWS CloudFormation runs anytime you create, update (if
you changed the custom resource), or delete stacks. For example, you
might want to include resources that aren't available as AWS
CloudFormation resource types. You can include those resources by
using custom resources. That way you can still manage all your related
resources in a single stack.
Use the AWS::CloudFormation::CustomResource or Custom::String resource
type to define custom resources in your templates. Custom resources
require one property: the service token, which specifies where AWS
CloudFormation sends requests to, such as an Amazon SNS topic.