Stack is hung using CloudFormation with SNS-backed CustomResources - amazon-web-services

I'm trying to learn working of CustomResources in CloudFormation Template. Created simple template to create s3 bucket. But on creating stack, it remains in Create in progress state for long time and no bucket is created.
Is there anything, I'm missing in below validated template:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Building A bucket With customeResources in CloudFormation",
"Parameters" : {
"NewBucket": {
"Default": "",
"Description": "S3 bucket containing customer assets",
"Type": "String"
}
},
"Conditions": {
"NewBucket": {
"Fn::Not": [
{
"Fn::Equals": [
{
"Ref": "NewBucket"
},
""
]
}
]
}
},
"Resources" : {
"CustomResource": {
"Properties": {
"S3Bucket": {
"Ref": "NewBucket"
},
"ServiceToken": "SNS topic ARN"
},
"Type": "AWS::CloudFormation::CustomResource"
}
},
"Outputs": {
"BucketName": {
"Value": {
"Fn::GetAtt": [ "CustomResource", {"Ref": "NewBucket"} ]
}
}
}
}

It would appear that your SNS-backed custom resource is not sending a response back to cloud formation, and it is stuck waiting for that response.
From Amazon Simple Notification Service-backed Custom Resources:
The custom resource provider processes the data sent by the template
developer and determines whether the Create request was successful.
The resource provider then uses the S3 URL sent by AWS CloudFormation
to send a response of either SUCCESS or FAILED.
When the request is made to the SNS service provider, it include the following object:
{
"RequestType": "Create",
"ServiceToken": "arn:aws:sns:us-west-2:2342342342:Critical-Alerts-development",
"ResponseURL": "https:\/\/cloudformation-custom-resource-response-uswest2.s3-us-west-2.amazonaws.com\/arn%3Aaws%3Acloudformation%3Aus-west-2%3A497903502641%3Astack\/custom-resource\/6bf07a80-d44a-11e7-84df-503aca41a029%7CCustomResource%7C5a695f41-61d7-475b-9110-cdbaec04ee55?AWSAccessKeyId=AKIAI4KYMPPRGIACET5Q&Expires=1511887381&Signature=WmHQVqIDCBwQSfcBMpzTfiWHz9I%3D",
"StackId": "arn:aws:cloudformation:us-west-2:asdasdasd:stack\/custom-resource\/6bf07a80-d44a-11e7-84df-503aca41a029",
"RequestId": "5a695f41-61d7-475b-9110-cdbaec04ee55",
"LogicalResourceId": "CustomResource",
"ResourceType": "AWS::CloudFormation::CustomResource",
"ResourceProperties": {
"ServiceToken": "arn:aws:sns:us-west-2:234234234:Critical-Alerts-development",
"S3Bucket": "test-example-com"
}
}
You will need to send a success/fail response to the ResponseURL provided in the event for Cloud Formation to continue processing.
I would also note that the bucket will not be created unless your custom service provider creates it. The Custom Resource function is only sending the request to the provider.

Related

AWS Lambda Controller route not matching, with {proxy+} - 404 Not Found

I have a .NET core 3.1 project with a GET endpoint. Locally the route works fine - "/api" GET request returns JSON string.
After publishing to my AWS Lambda function, and invoking the lambda via Postman, I get a 404 Not Found response to the "/api" GET request. Full URL: "https://(lambda domain)/default/MyLambda2/api"
In my project's Startup.cs ConfigureServices and Configure methods, I added "LambdaLogger.Log" statements. My log lines show up in CloudWatch (so I am definitely reaching the app). It's just after that, the route fails.
In ConfigureServices method, I have
services.AddControllers();
In Configure method, I have
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
And in the Controller file
namespace LambdaTest.Controllers
{
[Route("api")]
public class ValuesController : ControllerBase
{
[HttpGet]
public IEnumerable<string> Get()
{
return new string[] { "value1", "value2" };
}
}
}
I have an API Gateway trigger defined for the lambda, with
API endpoint: "https://(lambda domain)/default/MyLambda2/{proxy+}"
API type: REST
Authorization: NONE
Method: ANY
In Resources tab, I have...
/MyLambda2
ANY
/{proxy+}
ANY
OPTIONS
... and I have all Resources deployed to "default" stage (via Actions > Deploy API).
Here is the default auto-generated serverless.template file...
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "An AWS Serverless Application that uses the ASP.NET Core framework running in Amazon Lambda.",
"Parameters": {},
"Conditions": {},
"Resources": {
"AspNetCoreFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "LambdaTest::LambdaTest.LambdaEntryPoint::FunctionHandlerAsync",
"Runtime": "dotnetcore3.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambda_FullAccess"
],
"Events": {
"ProxyResource": {
"Type": "Api",
"Properties": {
"Path": "/{proxy+}",
"Method": "ANY"
}
},
"RootResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "ANY"
}
}
}
}
}
},
"Outputs": {
"ApiURL": {
"Description": "API endpoint URL for Prod environment",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}
What could be causing the route to not get picked up in lambda?
Thanks
Found the issue. I needed my route on the controller to be
[Route("MyLambda2/api")]

How to import existing S3 bucket from another stack in CloudFormation?

I'd like to import an existing S3 bucket that is created in another stack. I tried to do it using two CF templates below but when I try to create a consumer stack it leads to a CREATE_FAILED event with an "already exists" error message.
Producer template:
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Main stack that creates S3 bucket.",
"Resources": {
"myS3Bucket" : {
"Type" : "AWS::S3::Bucket",
"DeletionPolicy" : "Retain",
"Properties": {"BucketName": "bucket-name" }
}
},
"Outputs": {
"BucketId": {
"Description": "Bucket ID",
"Value": { "Ref": "myS3Bucket" },
"Export": { "Name": { "Fn::Sub": "${AWS::StackName}-BUCKETID" }}
}
}
}
And this is a consumer stack template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation Sample Template.",
"Resources" : {
"S3ImportedBucket" : {
"Type" : "AWS::S3::Bucket",
"Properties" : {
"BucketName" : { "Fn::ImportValue" : "main-BUCKETID"}
}
}
}
}
What should be changed in the templates to make it work?
tl;dr:
the concept of outputs is right but you are using it in a wrong way
explanation
In cloudformation templates, you are posting
Both create a resource of s3 bucket type with the same name so whatever runs first will create the s3 bucket
you can check here for reference
however, the output can be used as a reference in other templates like if you are uploading something from lambda so you can set that bucket as env var and use it inside the lambda (just an example)

Developing an app to create a stack from a cloudformation template

I am new to AWS and am currently working on simple tasks.
I have created a free tier EC2 instance using a cloudformation template. Now my next task is to write a simple application that uses respective AWS SDK to call CloudFormation API to create a stack from the template.
Here is the cloudformation template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Ec2 Template",
"Metadata": {
"Copyright":[
"Copyright 2017, All rights reserved"
],
"Comments":[
"Create an EC2"
]
},
"Parameters": {
"KeyName": {
"Type": "AWS::EC2::KeyPair::KeyName",
"Description": "Name of an existing EC2 KeyPair to enable access to join ECS instances."
},
"InstanceTypeParameter":{
"Type": "String",
"Default": "t2.micro",
"AllowedValues": [
"t2.micro",
"m1.small",
"m1.large"
],
"Description": "Enter t2.micro, m1.small, or m1.large. Default is t2.micro."
},
"EcsSecurityGroupLb":{
"Type": "AWS::EC2::SecurityGroup::Id",
"Description":"The ECS ELB Security Group."
},
"vpcid":{
"Type": "AWS::EC2::VPC::Id"
},
"mySubnetIDs": {
"Description":"Subnet IDs",
"Type":"AWS::EC2::Subnet::Id"
}
},
"Resources":{
"Ec2Instance":{
"Type":"AWS::EC2::Instance",
"Properties":{
"ImageId": "ami-bf4193c7",
"KeyName": {
"Ref": "KeyName"
},
"InstanceType":{
"Ref": "InstanceTypeParameter"
},
"NetworkInterfaces":[
{
"AssociatePublicIpAddress":"true",
"DeviceIndex":"0",
"SubnetId":{
"Ref":"mySubnetIDs"
},
"GroupSet":[
{
"Ref": "EcsSecurityGroupLb"
}
]
}
],
"BlockDeviceMappings":[
{
"DeviceName": "/dev/sdc",
"VirtualName":"ephemeral0"
}
]
}
}
},
"Outputs":{
"Ec2Instance":{
"Description": "InstanceId of newly created EC2 instance",
"Value": {
"Ref": "Ec2Instance"
}
},
"InstanceIPAddress":{
"Value":{ "Fn::GetAtt": ["Ec2Instance", "PublicIp"]},
"Description": "Public IP address of instance"
}
}
}
I have gone through a lot of documentation but haven't really understood as to how to proceed. I would like to know if there are any good tutorials on this.
Looking for suggestions with respect to the steps as well.
Thanks!
Since you have to (as a task requirement) write the application yourself, you'll need to use one of the AWS SDKs that are available.
The SDK you choose will depend on what programming language you are most comfortable using (or required by your task).
Roughly, your program will need to do the following:
With the AWS SDK, create an AWS session which uses your IAM user's API keys.
Grab the Cloudformation template off of your local system.
With the AWS SDK, invoke Cloudformation to create the resource stack with your template.
(Optionally) Wait until the stack is complete and output a report on the stack status.
Good luck!

How to secure AWS API Gateway with Access and Secret Keys in CloudFormation?

I created the serverless Lambda application by using an AWS Toolkit for Visual Studio template (I used Tutorial: Build and Test a Serverless Application with AWS Lambda). I had selected 'Empty Serverless Project' and created simple lambda function linked to API Gateway.
The CloudFormation template looks like:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Transform" : "AWS::Serverless-2016-10-31",
"Description" : "An AWS Serverless Application.",
"Resources" : {
"Get" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Handler": "AWSServerless::AWSServerless.Functions::Get",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [ "AWSLambdaBasicExecutionRole" ],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
}
}
}
},
"Outputs" : {
"ApiURL" : {
"Description" : "API endpoint URL for Prod environment",
"Value" : { "Fn::Sub" : "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" }
}
}
}
Now I need to secure my API Gateway with Access and Secret Keys. I have investigated a bit and if I am correct it should look like next:
"security":[{"sigv4":[]}]
But it still isn't clear to me where should I apply it? Possible that I am wrong and it could be done in another way. So my question is:
How to secure API Gateway with Access and Secret Keys in CloudFormation?
You can use API key or Authorizers
The following examples create a custom authorizer that is an AWS Lambda function.
"Authorizer": {
"Type": "AWS::ApiGateway::Authorizer",
"Properties": {
"AuthorizerCredentials": { "Fn::GetAtt": ["LambdaInvocationRole", "Arn"] },
"AuthorizerResultTtlInSeconds": "300",
"AuthorizerUri" : {"Fn::Join" : ["", [
"arn:aws:apigateway:",
{"Ref" : "AWS::Region"},
":lambda:path/2015-03-31/functions/",
{"Fn::GetAtt" : ["LambdaAuthorizer", "Arn"]}, "/invocations"
]]},
"Type": "TOKEN",
"IdentitySource": "method.request.header.Auth",
"Name": "DefaultAuthorizer",
"RestApiId": {
"Ref": "RestApi"
}
}
}
(Update)
SO thread on How to use authorizers in the template
Reference an Authorizer definition in an API Gateway path

Howto specify 'Raw Message Delivery' for an SNS subscription using AWS CloudFormation?

I've got an AWS CloudFormation template that creates an SNS topic and a subscription:
"AcceptedTopic":{
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"TopicName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"Subscription": [{
"Endpoint": {"Fn::GetAtt" : [ "SomeQueue" , "Arn" ]},
"Protocol": "sqs"
}]
}
}
I need to specify the 'Raw Message Delivery' subscription attribute. How can I do that in AWS CloudFormation?
As of this writing, AWS CloudFormation doesn't support that natively. As an alternate, you can create a Lambda-backed custom resource to get around this limitation and set that attribute using set-subscription-attributes instead. Here are some helpful resources to help accomplish that:
Lambda-backed custom resources
SNS' set-subscription-attributes API
Now AWS CloudFormation supports it with AWS::SNS::Subscription. So instead of adding the subscription as a property of the topic, add an Subscription resource linked above.
A caveat though, is that if you already created a topic with that subscription and are now trying to add the attribute, it'd fail miserably with Invalid Parameter error. The cause is it's considering the standalone Subscription added in the template as a new resource and trying to create it. I haven't found a good way around this other than deleting that subscription manually, which is not good practice in production environment.
My solution around this is separating it into 2 steps. First, remove the property subscription from the topic and add a Subscription resource. Then, add new attributes to the subscription resource.
First:
{
"AcceptedTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
},
"TopicName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
}
}
}
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs"
}
}
Then:
{
...
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs",
"RawMessageDelivery": "true"
}
}