Say I have a Log Group in cloudWatch, which I want to subscribe to a lambda with a filter (Subscriptions->Stream to AWS Lambda).
I want to achieve it with cloudFormation template, but from cloudFormation doc, it seems that the only two available cloudWatch resources are Alarm/Dashboard.
Questions is:
Is there any way to write cloudWatch Log Group subscription in
cloudFormation?
If not, any alternative way (say from lambda
resource configuration in cloudFormation template)?
Oh, that's a tricky one. I only figured it out by creating one in the console and reverse enginerring - ick. But you're lucky - I have it on hand :P This is the json I was using for subscribing a lambda to a vpc flow log.
Note that the 'VPCFlowLogsGroup' is the logical Id of the log group, the 'FlowLogsCollector' that of the lambda.
"FlowLogsCollectorEventPermission": {
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Principal" : { "Fn::Sub": "logs.${AWS::Region}.amazonaws.com" },
"Action" : "lambda:InvokeFunction",
"FunctionName" : { "Fn::GetAtt": [ "FlowLogsCollector", "Arn" ] },
"SourceAccount": { "Ref": "AWS::AccountId" },
"SourceArn" : { "Fn::GetAtt": [ "VPCFlowLogsGroup", "Arn" ] }
}
},
"FlowLogsCollectorSubscription": {
"Type" : "AWS::Logs::SubscriptionFilter",
"DependsOn": "FlowLogsCollectorEventPermission",
"Properties" : {
"LogGroupName" : { "Ref" : "VPCFlowLogsGroup" },
"FilterPattern" : "",
"DestinationArn" : { "Fn::GetAtt" : [ "FlowLogsCollector", "Arn" ] }
}
},
Related
I created the serverless Lambda application by using an AWS Toolkit for Visual Studio template (I used Tutorial: Build and Test a Serverless Application with AWS Lambda). I had selected 'Empty Serverless Project' and created simple lambda function linked to API Gateway.
The CloudFormation template looks like:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Transform" : "AWS::Serverless-2016-10-31",
"Description" : "An AWS Serverless Application.",
"Resources" : {
"Get" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Handler": "AWSServerless::AWSServerless.Functions::Get",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [ "AWSLambdaBasicExecutionRole" ],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
}
}
}
},
"Outputs" : {
"ApiURL" : {
"Description" : "API endpoint URL for Prod environment",
"Value" : { "Fn::Sub" : "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" }
}
}
}
Now I need to secure my API Gateway with Access and Secret Keys. I have investigated a bit and if I am correct it should look like next:
"security":[{"sigv4":[]}]
But it still isn't clear to me where should I apply it? Possible that I am wrong and it could be done in another way. So my question is:
How to secure API Gateway with Access and Secret Keys in CloudFormation?
You can use API key or Authorizers
The following examples create a custom authorizer that is an AWS Lambda function.
"Authorizer": {
"Type": "AWS::ApiGateway::Authorizer",
"Properties": {
"AuthorizerCredentials": { "Fn::GetAtt": ["LambdaInvocationRole", "Arn"] },
"AuthorizerResultTtlInSeconds": "300",
"AuthorizerUri" : {"Fn::Join" : ["", [
"arn:aws:apigateway:",
{"Ref" : "AWS::Region"},
":lambda:path/2015-03-31/functions/",
{"Fn::GetAtt" : ["LambdaAuthorizer", "Arn"]}, "/invocations"
]]},
"Type": "TOKEN",
"IdentitySource": "method.request.header.Auth",
"Name": "DefaultAuthorizer",
"RestApiId": {
"Ref": "RestApi"
}
}
}
(Update)
SO thread on How to use authorizers in the template
Reference an Authorizer definition in an API Gateway path
I'm trying to learn working of CustomResources in CloudFormation Template. Created simple template to create s3 bucket. But on creating stack, it remains in Create in progress state for long time and no bucket is created.
Is there anything, I'm missing in below validated template:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Building A bucket With customeResources in CloudFormation",
"Parameters" : {
"NewBucket": {
"Default": "",
"Description": "S3 bucket containing customer assets",
"Type": "String"
}
},
"Conditions": {
"NewBucket": {
"Fn::Not": [
{
"Fn::Equals": [
{
"Ref": "NewBucket"
},
""
]
}
]
}
},
"Resources" : {
"CustomResource": {
"Properties": {
"S3Bucket": {
"Ref": "NewBucket"
},
"ServiceToken": "SNS topic ARN"
},
"Type": "AWS::CloudFormation::CustomResource"
}
},
"Outputs": {
"BucketName": {
"Value": {
"Fn::GetAtt": [ "CustomResource", {"Ref": "NewBucket"} ]
}
}
}
}
It would appear that your SNS-backed custom resource is not sending a response back to cloud formation, and it is stuck waiting for that response.
From Amazon Simple Notification Service-backed Custom Resources:
The custom resource provider processes the data sent by the template
developer and determines whether the Create request was successful.
The resource provider then uses the S3 URL sent by AWS CloudFormation
to send a response of either SUCCESS or FAILED.
When the request is made to the SNS service provider, it include the following object:
{
"RequestType": "Create",
"ServiceToken": "arn:aws:sns:us-west-2:2342342342:Critical-Alerts-development",
"ResponseURL": "https:\/\/cloudformation-custom-resource-response-uswest2.s3-us-west-2.amazonaws.com\/arn%3Aaws%3Acloudformation%3Aus-west-2%3A497903502641%3Astack\/custom-resource\/6bf07a80-d44a-11e7-84df-503aca41a029%7CCustomResource%7C5a695f41-61d7-475b-9110-cdbaec04ee55?AWSAccessKeyId=AKIAI4KYMPPRGIACET5Q&Expires=1511887381&Signature=WmHQVqIDCBwQSfcBMpzTfiWHz9I%3D",
"StackId": "arn:aws:cloudformation:us-west-2:asdasdasd:stack\/custom-resource\/6bf07a80-d44a-11e7-84df-503aca41a029",
"RequestId": "5a695f41-61d7-475b-9110-cdbaec04ee55",
"LogicalResourceId": "CustomResource",
"ResourceType": "AWS::CloudFormation::CustomResource",
"ResourceProperties": {
"ServiceToken": "arn:aws:sns:us-west-2:234234234:Critical-Alerts-development",
"S3Bucket": "test-example-com"
}
}
You will need to send a success/fail response to the ResponseURL provided in the event for Cloud Formation to continue processing.
I would also note that the bucket will not be created unless your custom service provider creates it. The Custom Resource function is only sending the request to the provider.
I have a Lambda function deployed to several regions. I would like to publish a message to SNS that will invoke these functions.
Using aws-cli I've created the topics, given Lambda permission to talk to SNS, and create the subscriptions. The subscription appears to be created successfully, and I can see it in the AWS console. But, it doesn't work. The lambda function does not get invoked.
This is CloudFormation based example. You have to add invoke permission for SNS to the Lambda functions:
{
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"FunctionName" : { "Fn::GetAtt" : [ "YourLambda", "Arn" ] },
"Action" : "lambda:InvokeFunction",
"Principal" : "sns.amazonaws.com",
"SourceArn" : { "Ref" : "YourSNSTopicArn" }
}
}
Then you need to subscribe your Lambdas to your SNS topic. This can be done via API call or through CloudFormation.
{
"Type" : "AWS::SNS::Topic",
"Properties" : {
"TopicName" : "YourTopicName",
"Subscription" : [ {
"Endpoint" : { "Fn::GetAtt" : [ "YourLambda", "Arn" ] },
"Protocol": "lambda"
} ]
}
}
If you're missing any of this, your Lambdas won't invoke. Source for the above information is the official blog article Invoking Lambda functions via SNS.
I've got an AWS CloudFormation template that creates an SNS topic and a subscription:
"AcceptedTopic":{
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"TopicName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"Subscription": [{
"Endpoint": {"Fn::GetAtt" : [ "SomeQueue" , "Arn" ]},
"Protocol": "sqs"
}]
}
}
I need to specify the 'Raw Message Delivery' subscription attribute. How can I do that in AWS CloudFormation?
As of this writing, AWS CloudFormation doesn't support that natively. As an alternate, you can create a Lambda-backed custom resource to get around this limitation and set that attribute using set-subscription-attributes instead. Here are some helpful resources to help accomplish that:
Lambda-backed custom resources
SNS' set-subscription-attributes API
Now AWS CloudFormation supports it with AWS::SNS::Subscription. So instead of adding the subscription as a property of the topic, add an Subscription resource linked above.
A caveat though, is that if you already created a topic with that subscription and are now trying to add the attribute, it'd fail miserably with Invalid Parameter error. The cause is it's considering the standalone Subscription added in the template as a new resource and trying to create it. I haven't found a good way around this other than deleting that subscription manually, which is not good practice in production environment.
My solution around this is separating it into 2 steps. First, remove the property subscription from the topic and add a Subscription resource. Then, add new attributes to the subscription resource.
First:
{
"AcceptedTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
},
"TopicName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
}
}
}
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs"
}
}
Then:
{
...
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs",
"RawMessageDelivery": "true"
}
}
I'm very new to Amazon cloudformation technique. I'm trying to launch an ec2 instance along with the IAM roles.
I have cloudformation script for this. But the problem I face is the IAM roles and Ec2 instances are created, but they aren't tied with each other.
I did create the IAM-roles using AWS::IAM::Role and AWS::IAM::InstanceProfile.
Is there any other command that I should use?
Thanks in advance.
Had to dig to get the final result, but here's an example of
Defining an access role (this will allow the EC2 instance to step into / assume to role),
Defining a policy for the role (i.e. when the EC2 assumes the role, what resources does it have access to),
Defining the instance profile (that is referenced by the EC2 instance, and has the access role mapped in)
"S3AccessRole" : {
"Type" : "AWS::IAM::Role",
"Properties" : {
"AssumeRolePolicyDocument" : {
"Statement" : [ {
"Effect" : "Allow",
"Principal" : {
"Service" : [ "ec2.amazonaws.com" ]
},
"Action" : [ "sts:AssumeRole" ]
} ]
},
"Path" : "/"
}
},
"S3RolePolicies" : {
"Type" : "AWS::IAM::Policy",
"Properties" : {
"PolicyName" : "s3access",
"PolicyDocument" : {
"Statement" : [ {
"Effect" : "Allow",
"Action" : "s3:*",
"Resource" : "*"
}]
},
"Roles" : [ { "Ref" : "S3AccessRole" } ]
}
},
"S3InstanceProfile" : {
"Type" : "AWS::IAM::InstanceProfile",
"Properties" : {
"Path" : "/",
"Roles" : [ { "Ref" : "S3AccessRole" } ]
}
}
The policy above allows all access to s3 resources. Adjust according to your needs. The IamInstanceProfile reference in the EC2 instance properties would refer be { "Ref" : "S3InstanceProfile" }
Note that as of May 2015, when you creating a stack that creates IAM roles, you need to check a box acknowledging such creation, otherwise you'll get a "Stack creation error: Requires capabilities : [CAPABILITY_IAM]" error.
The easiest way to solve such problems is to use CloudFormer. CloudFormer is a tool that creates a starting point template from the AWS resources you already have running in your environment.
The CloudFormer tool is packaged as a standalone application that you
can launch inside your AWS environment. The application is started on
a t1.micro Amazon EC2 instance via AWS CloudFormation.
Once you have launched Cloud Former, you will get a web interface (check the URL in the Output section of the launched stack), that will be able to describe all your resources in a specific region. It will lead you through which resources you wish in each category (DNS, Network, Compute...). At the end you can see the template and copy it, or save it in S3.
But if you wish to do it manually, you need to add the AWS::IAM::InstanceProfile you created to the Properties of AWS::EC2::Instance as IamInstanceProfile
{
"Type" : "AWS::EC2::Instance",
"Properties" : {
"AvailabilityZone" : String,
"BlockDeviceMappings" : [ EC2 Block Device Mapping, ... ],
"DisableApiTermination" : Boolean,
"EbsOptimized" : Boolean,
"IamInstanceProfile" : String,
"ImageId" : String,
"InstanceType" : String,
...
"UserData" : String,
"Volumes" : [ EC2 MountPoint, ... ]
}
}
See more details on AWS::EC2::Instance here
Suppose the AWS::IAM::InstanceProfile resource you create is called MyNewRole. To create an instance with that role (in the same CloudFormation template) set the EC2 resource's IamInstanceProfile property to a Ref to that resource. Here's an example (with lots of other details left out):
"Resources": {
"MyNewRole": {
"Type": "AWS::IAM::InstanceProfile",
... more stuff here
},
"MyNewEc2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"IamInstanceProfile": { "Ref": "MyNewRole" },
... more stuff here
}
}
}