Retain Output on Stack-Update - amazon-web-services

I wrote a Cloudformation template that creates a stack with a DynamoDB Table:
"FeedStageDynamoTable" : {
"Type" : "AWS::DynamoDB::Table",
"UpdateReplacePolicy" : "Retain",
"Properties" : {
"TableName" : { "Fn::Sub": [ "feed-${Year}-${Environment}-table", { "Year": {"Ref" : "BundleYear" }, "Environment" : {"Ref" : "DeployEnvironment"}} ]},
"AttributeDefinitions" : [
{"AttributeName" : "Guid", "AttributeType" : "S"}
],
"KeySchema" : [
{"AttributeName" : "Guid", "KeyType" : "HASH"}
],
"ProvisionedThroughput" : {
"ReadCapacityUnits" : "2",
"WriteCapacityUnits" : "2"
},
"StreamSpecification": {
"StreamViewType": "NEW_AND_OLD_IMAGES"
}
}
}
and an Output for the Table's Stream:
"Outputs" : {
"FeedStageTableStreamArn": {
"Description" : "The security group ID to use for public web servers",
"Value" : { "Fn::GetAtt" : ["FeedStageDynamoTable", "StreamArn"] },
"Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-FeedStageDynamoTableStreamArn" }}
},
The output is used by a lambda function from another template (for the second stack):
"NotifyWebConsumer" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Environment": {
"Variables" : {
"EnvironmentCodename" : { "Fn::Sub": [ "${Environment}", { "Environment" : {"Ref" : "DeployEnvironment"}} ]}
}
},
"Handler": "AmazonServerlessStageWebUpdate::AmazonServerlessStageWebUpdate.Functions::NotifyWebConsumer",
"FunctionName": "NotifyWebConsumer",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Policies":[{ "Fn::ImportValue" : {"Fn::Sub" : "${DeployEnvironment}-LambdaExecutionPolicy"} }],
"Role":{"Fn::GetAtt": ["NotifyConsumerRole","Arn"]},
"Events": {
}
}
}
...
"EventSourceMapping": {
"Type": "AWS::Lambda::EventSourceMapping",
"Properties": {
"EventSourceArn": { "Fn::ImportValue" : {"Fn::Sub" : "StageEnvironment-FeedStageDynamoTableStreamArn"} },
"FunctionName" : {
"Fn::GetAtt": [
"NotifyWebConsumer", "Arn"
]
},
"StartingPosition" : "LATEST"
}
}
After I published both stacks I can not update the first one anymore, because the output stream is in use by the second stack. Now my question: is there a cloudformation property like "UpdateReplacePolicy": "Retain" for outputs? Or is there any other way to update the first stack without deleting the second one?

Related

Target group health check for ECS service failing upon creation with cloudformation

I've been hosting an api as a service on ecs with an alb. Whenver the service is running it fails the health check and continually kills and restarts the task. The task logs always show it repsonding to the health checks with response 200 so it should be passing. I've checked that my security groups and vpc's are configured correctly. I will post my cloud formation template below in hopes someone can find my mistake. I've also tried increasing the health check graces period greatly and it hasn't helped the issue.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"6514f997-7e0a-48c6-9e5f-95465c35ff00": {
"size": {
"width": 60,
"height": 60
},
"position": {
"x": 280,
"y": 140
},
"z": 0,
"embeds": []
}
}
},
"Parameters": {
"ClusterName": {
"Type" : "String",
"Description" : "Name of the ECS Cluster",
"Default" : "Rest-Api-Explorer"
},
"Subnet": {
"Description": "User Specified Subnet",
"Type": "AWS::EC2::Subnet::Id",
"ConstraintDescription": "Must be a valid Subnet of specified VPC."
}
},
"Mappings" : {
"SubnetToVPCMap" : {
"subnet-58863c01" : {"VPC" : "vpc-6581f100"},
"subnet-ae349ec6" : {"VPC" : "vpc-259a344d"},
"subnet-b9349ed1" : {"VPC" : "vpc-259a344d"},
"subnet-d8349eb0" : {"VPC" : "vpc-259a344d"},
"subnet-e9b7d69e" : {"VPC" : "vpc-6581f100"},
"subnet-ee97d48b" : {"VPC" : "vpc-6581f100"}
},
"VPCToSecurityGroupMap" : {
"vpc-6581f100" : {"SecurityGroup" : "sg-a5c672c1"},
"vpc-259a344d" : {"SecurityGroup" : "sg-1d658072"}
},
"VPCToTag" : {
"vpc-6581f100" : {"Tag" : [{ "Key": "Name", "Value": "Rest API Explorer"}, {"Key": "Type", "Value": "Development"}]},
"vpc-259a344d" : {"Tag" : [{ "Key": "Name", "Value": "Rest API Explorer"}, {"Key": "Type", "Value": "Production"}]}
}
},
"Resources": {
"ECSCluster":{
"Type" : "AWS::ECS::Cluster",
"Properties" : {
"ClusterName" : {"Ref" : "ClusterName"},
"Tags": {
"Fn::FindInMap" : [ "VPCToTag", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "Tag"]
}
}
},
"EC2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"IamInstanceProfile": "ecsInstanceRole",
"ImageId": "ami-00e0090ac21971297",
"InstanceType": "t2.medium",
"KeyName": "community-admin",
"SecurityGroupIds": [
{"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]}
],
"SubnetId": {
"Ref": "Subnet"
},
"Tags": {
"Fn::FindInMap" : [ "VPCToTag", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "Tag"]
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xe\n",
"echo ECS_CLUSTER=", { "Ref" : "ClusterName" },
" >> /etc/ecs/ecs.config\n"
]
]
}
}
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "6514f997-7e0a-48c6-9e5f-95465c35ff00"
}
}
},
"ECSALB":{
"Type":"AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties":{
"Name":"rest-api-explorer-2",
"Scheme":"internal",
"LoadBalancerAttributes": [
{
"Key":"idle_timeout.timeout_seconds",
"Value":"60"
}
],
"SecurityGroups":[
{
"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]
}
],
"Subnets" : [
"subnet-58863c01",
"subnet-e9b7d69e",
"subnet-ee97d48b"
]
}
},
"ALBListener":{
"Type":"AWS::ElasticLoadBalancingV2::Listener",
"DependsOn": "ECSTG",
"Properties":{
"DefaultActions":[
{
"Type":"forward",
"TargetGroupArn":{
"Ref":"ECSTG"
}
}
],
"LoadBalancerArn":{
"Ref":"ECSALB"
},
"Port":"80",
"Protocol":"HTTP"
}
},
"ECSTG":{
"Type":"AWS::ElasticLoadBalancingV2::TargetGroup",
"DependsOn":"ECSALB",
"Properties":{
"HealthCheckIntervalSeconds":15,
"HealthCheckPath":"/api/rest/console",
"HealthCheckPort":"traffic-port",
"HealthCheckProtocol":"HTTP",
"HealthCheckTimeoutSeconds":10,
"HealthyThresholdCount":3,
"Name":"rest-api-target-group-2",
"Port":3000,
"Protocol":"HTTP",
"TargetType":"ip",
"UnhealthyThresholdCount":5,
"VpcId":{
"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]
}
}
},
"service":{
"Type":"AWS::ECS::Service",
"DependsOn": "ALBListener",
"Properties":{
"Cluster":{
"Ref":"ECSCluster"
},
"DeploymentConfiguration": {
"MaximumPercent": 100,
"MinimumHealthyPercent": 0
},
"DesiredCount":"1",
"HealthCheckGracePeriodSeconds" : 90,
"LaunchType": "FARGATE",
"LoadBalancers":[
{
"ContainerName":"api-rest-explorer",
"ContainerPort":"3000",
"TargetGroupArn":{
"Ref":"ECSTG"
}
}
],
"NetworkConfiguration": {
"AwsvpcConfiguration" : {
"AssignPublicIp" : "ENABLED",
"SecurityGroups" : [
{
"Fn::FindInMap" : [ "VPCToSecurityGroupMap", {"Fn::FindInMap" : [ "SubnetToVPCMap", {"Ref" : "Subnet"}, "VPC"]} , "SecurityGroup"]
}
],
"Subnets" : [
{"Ref": "Subnet"}
]
}
},
"SchedulingStrategy": "REPLICA",
"ServiceName" : "rest-api-explorer-fargate",
"TaskDefinition": "rest-api-explorer-fargate:4"
}
}
}
}

x-amazon-apigateway-integration get lambda arn for uri property

The x-amazon-apigateway-intregration extension has a URI property that takes an arn value of the backend. When I try to deploy this with SAM I get the following error
Unable to parse API definition because of a malformed integration at path /auth/signIn
How can I get the ARN value of my lambda function that is specified in my template?
Template:
"api" : {
"Type" : "AWS::Serverless::Api",
"Properties" : {
"StageName" : "prod",
"DefinitionUri" : "src/api/swagger.json"
}
},
"signIn": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "index.signIn",
"Runtime": "nodejs8.10",
"CodeUri": "./src",
"FunctionName": "signIn",
"ReservedConcurrentExecutions": 15,
"Timeout": 5,
"Events": {
"SignIn": {
"Type": "Api",
"Properties": {
"Path": "/signIn",
"Method": "post",
"RestApiId" : {
"Ref": "api"
}
}
}
}
}
},
Swagger Definition:
"paths" : {
"/auth/signIn" : {
"post" : {
"x-amazon-apigateway-integration" : {
"httpMethod" : "POST",
"type" : "aws_proxy",
"uri" : {
"Fn::Sub" : {
"Fn::GetAtt": [
"signIn",
"Arn"
]}
}
},
"parameters" : [
{
"name" : "email",
"in" : "body",
"required" : "true",
"schema" : {
"type" : "string"
}
},
{
"name" : "password",
"in" : "body",
"required" : "true",
"schema" : {
"type" : "string"
}
}
],
"responses" : {
"200" : {
"description" : "Authenticated"
},
"default" : {
"description" : "Unexpected Error"
}
}
}
},

AWS SES Configset - Can't create an event destination to SNS using AWS cloud formation stack

I can't seem to be able to create a new AWS SES configset using AWS Cloud Formation stack. The error says 'YAML not well-formed'
Below is my json template for the CF stack:
"Resources" : {
"ConfigSet": {
"Type": "AWS::SES::ConfigurationSet",
"Properties": {
"Name": "CS_EMAIL_TRACKING"
}
},
"CWEventDestination": {
"Type": "AWS::SES::ConfigurationSetEventDestination",
"Properties": {
"ConfigurationSetName": "CS_EMAIL_TRACKING",
"EventDestination": {
"Name": "CS_EMAIL_TRACKING_CW_DESTINATION",
"Enabled": true,
"MatchingEventTypes": ["bounce", "complaint", "delivery", "open", "reject", "renderingFailure", "send"],
"CloudWatchDestination": {
"DimensionConfigurations": [{
"DimensionName": "AGS",
"DimensionValueSource": "messageTag",
"DefaultDimensionValue": "MY_AGS"
}, {
"DimensionName": "Component",
"DimensionValueSource": "messageTag",
"DefaultDimensionValue": "Mail"
}, {
"DimensionName": "ses:caller-identity",
"DimensionValueSource": "messageTag",
"DefaultDimensionValue": "shouldbeautoset"
}]
}
}
}
},
"SNSEventDestination": {
"Type": "AWS::SES::ConfigurationSetEventDestination",
"Properties": {
"ConfigurationSetName": "CS_EMAIL_TRACKING",
"EventDestination": {
"Name": "CS_EMAIL_TRACKING_SNS_DESTINATION",
"Enabled": true,
"MatchingEventTypes": ["bounce", "complaint", "delivery", "reject", "send"],
"SNSDestination": {
"TopicARN": "arn:aws:sns:us-east-1:99999999:SES-STATUS_TRACKING_TOPIC"
}
}
}
}
}
The above json looks good to me but......
Can anybody help? Am I missing something?
Thanks!
Edit: I got the stack working with parameters. Now, though I face another problem with SNSDestination being the EventDestination.
It says Unsupported property for EventDestination, even though the AWS documentation says that its a valid property:
Below is my final code:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS SES ConfigurationSet ${CRED_AGS} Template",
"Parameters": {
"ConfigSetName": {
"Type" : "String",
"Default" : "${CONFIGSET_NAME}"
},
"EventCWDestinationName" : {
"Type" : "String",
"Default" : "${CW_DESTINATION_NAME}"
},
"EventSNSDestinationName" : {
"Type" : "String",
"Default" : "${SNS_DESTINATION_NAME}"
},
"EventTypeBounce" : {
"Type" : "String",
"Default" : "bounce"
},
"EventTypeComplaint" : {
"Type" : "String",
"Default" : "complaint"
},
"EventTypeDelivery" : {
"Type" : "String",
"Default" : "delivery"
},
"EventTypeOpen" : {
"Type" : "String",
"Default" : "open"
},
"EventTypeReject" : {
"Type" : "String",
"Default" : "reject"
},
"EventTypeRenderingFailure" : {
"Type" : "String",
"Default" : "renderingFailure"
},
"EventTypeSend" : {
"Type" : "String",
"Default" : "send"
},
"DimensionValueSourceMsgTag" : {
"Type" : "String",
"Default" : "messageTag"
},
"DimensionNameAGS" : {
"Type" : "String",
"Default" : "AGS"
},
"DefaultDimensionValueAGS" : {
"Type" : "String",
"Default" : "${CRED_AGS}"
},
"DimensionNameComponent" : {
"Type" : "String",
"Default" : "Component"
},
"DefaultDimensionValueComponent" : {
"Type" : "String",
"Default" : "Mail"
},
"DimensionNameIdentity" : {
"Type" : "String",
"Default" : "ses:caller-identity"
},
"DefaultDimensionValueIdentity" : {
"Type" : "String",
"Default" : "shouldbeautoset"
}
},
"Resources": {
"ConfigSet" : {
"Type" : "AWS::SES::ConfigurationSet",
"Properties" : {
"Name" : {
"Ref" : "ConfigSetName"
}
}
},
"CWEventDestination" : {
"Type" : "AWS::SES::ConfigurationSetEventDestination",
"Properties" : {
"ConfigurationSetName" : {
"Ref": "ConfigSetName"
},
"EventDestination" : {
"Name" : {
"Ref" : "EventCWDestinationName"
},
"Enabled" : true,
"MatchingEventTypes" : [
{
"Ref" : "EventTypeBounce"
},
{
"Ref" : "EventTypeComplaint"
},
{
"Ref" : "EventTypeDelivery"
},
{
"Ref" : "EventTypeOpen"
},
{
"Ref" : "EventTypeReject"
},
{
"Ref" : "EventTypeRenderingFailure"
},
{
"Ref" : "EventTypeSend"
}
],
"CloudWatchDestination" : {
"DimensionConfigurations" : [
{
"DimensionName" : {
"Ref" : "DimensionNameAGS"
},
"DimensionValueSource" : {
"Ref" : "DimensionValueSourceMsgTag"
},
"DefaultDimensionValue" : {
"Ref": "DefaultDimensionValueAGS"
}
},
{
"DimensionName" : {
"Ref" : "DimensionNameComponent"
},
"DimensionValueSource" : {
"Ref" : "DimensionValueSourceMsgTag"
},
"DefaultDimensionValue" : {
"Ref" : "DefaultDimensionValueComponent"
}
},
{
"DimensionName" : {
"Ref" : "DimensionNameIdentity"
},
"DimensionValueSource" : {
"Ref" : "DimensionValueSourceMsgTag"
},
"DefaultDimensionValue" : {
"Ref" : "DefaultDimensionValueIdentity"
}
}
]
}
}
}
},
"SNSEventDestination" : {
"Type" : "AWS::SES::ConfigurationSetEventDestination",
"Properties" : {
"ConfigurationSetName" : {
"Ref": "ConfigSetName"
},
"EventDestination" : {
"Name" : {
"Ref" : "EventSNSDestinationName"
},
"Enabled" : true,
"MatchingEventTypes" : [
{
"Ref" : "EventTypeBounce"
},
{
"Ref" : "EventTypeComplaint"
},
{
"Ref" : "EventTypeDelivery"
},
{
"Ref" : "EventTypeReject"
},
{
"Ref" : "EventTypeSend"
}
],
"SNSDestination" : {
"TopicARN" : "${SNS_DELIVERY_TOPIC}"
}
}
}
}
}
}
Can someone pls help? BTW, I have the lastest AWS cli with me.
i think SNSDestination is not supported now. it probably is by default there when you set the configset
Cloudformation is case-sensitive, so it should be SnsDestination, not SNSDestination

Getting "Template validation error: Invalid template resource property 'VPCID' "

I am trying to create a VPC using AWS CloudFormation.
I built a VPC on its own, then I updated the existing stack by adding more components to the JSON template (components like subnets, internet gateways, NATs, route tables, etc.) - one component at a time.
My VPC created successfully but when I tried to update the stack with an internet gateway and attach to the VPC, I started getting the error Template validation error: Invalid template resource property 'VPCID'.
My JSON template is as follows:
{
"Parameters": {
"CIDRRange": {
"Description": "VPCCIDR Range (will be a /16 block)",
"Type": "String",
"Default": "10.251.0.0",
"AllowedValues": ["10.250.0.0","10.251.0.0"]
}
},
"Resources": {
"VPCBase": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": { "Fn::Join" : ["", [{ "Ref" : "CIDRRange" }, "/16"]] },
"EnableDnsSupport": "True",
"EnableDnsHostnames": "True",
"Tags": [{ "Key": "Name", "Value": { "Fn::Join" : ["", [{ "Ref" : "AWS::StackName" }, "-VPC"]] } }]
}
},
"IGWBase" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
"Tags" : [{ "Key": "Name", "Value": { "Fn::Join" : ["", [{ "Ref" : "AWS::StackName" }, "-IGW"]] } }]
}
},
"VGAIGWBase" : {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : { "Ref" : "IGWBase" },
"VpcId" : { "Ref" : "VPCBase" }}
},
"Outputs": {
"VPCID" : { "Value" : { "Ref" : "VPCBase" } },
"DefaultSG" : { "Value" : { "Fn::GetAtt" : ["VPCBase", "DefaultSecurityGroup"] }}
}
}
}
Your formatting is a bit of a mess - I'd recommend going with yaml over json - but the problem is you're not closing the Resources: section.
You can validate a template with the cli with
aws cloudformation validate-template --template-body file://path.json
"VGAIGWBase" : {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : { "Ref" : "IGWBase" },
"VpcId" : { "Ref" : "VPCBase" }
}
}
}, << ADD THIS
"Outputs": {
"VPCID" : { "Value" : { "Ref" : "VPCBase" } },
"DefaultSG" : { "Value" : { "Fn::GetAtt" : ["VPCBase", "DefaultSecurityGroup"] }}
}
}

HoztedZone error on cloudformation

I´m using cloudformation to create a ecs container and add this new container into route53 hostzone.
But when I run this script I´m having problems with the HostedZone tags
Here The error
A client error (ValidationError) occurred when calling the CreateStack operation: Invalid template parameter property 'Properties'
Here the json
"Parameters" : {
"InstanceType" : {
"Description" : "Container Instance type",
"Type" : "String",
"Default" : "t2.medium",
"AllowedValues" : [ "t2.micro", "t2.small", "t2.medium", "m3.medium", "m3.large", "m3.xlarge", "m3.2xlarge" ],
"ConstraintDescription" : "must be a valid EC2 instance type."
},
"HostedZone" : {
"Type": "AWS::Route53::HostedZone",
"Properties": {
"HostedZoneConfig": {
"Comment": "My hosted zone for example.com"
},
"Name": "***.couchbase.com",
"VPCs": [
{
"VPCId": "*********",
"VPCRegion": "eu-west-1"
}
],
"HostedZoneTags": [
{
"Key": "Name",
"Value": "Couchbase DNS"
}
]
}
}
},
"Resources" : {
"ContainerInstance" : {
"Type": "AWS::EC2::Instance",
"Properties": {
"Tags": [{
"Key" : "Name",
"Value" : "Couchbase-1"
},
{
"Key" : "Type",
"Value" : "ECS-Couchbase"
}],
"IamInstanceProfile" : { "Ref" : "ECSIamInstanceProfile" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
"InstanceType" : { "Ref" : "InstanceType" },
"SecurityGroups" : [ "ssh","default", "couchbase" ],
"KeyName" : { "Ref" : "KeyName" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -xe\n",
"echo ECS_CLUSTER=", { "Ref" : "ClusterName" },
" >> /etc/ecs/ecs.config\n"
]]}}
}
},
"CouchbaseDNSRecord" : {
"Type" : "AWS::Route53::RecordSet",
"Properties" : {
"HostedZoneName" : {
"Fn::Join" : [ "", [
{ "Ref" : "HostedZone" }, "."
] ]
},
"Comment" : "DNS name for my instance.",
"Name" : {
"Fn::Join" : [ "", [
{"Ref" : "ContainerInstance"}, ".",
{"Ref" : "AWS::Region"}, ".",
{"Ref" : "HostedZone"} ,"."
] ]
},
"Type" : "A",
"TTL" : "900",
"ResourceRecords" : [
{ "Fn::GetAtt" : [ "ContainerInstance", "PublicIp" ] }
]
}
},
The HostedZone should be inside the Resources section.
"Parameters" : {
"InstanceType" : {
...
}
},
"Resources" : {
"HostedZone" : {
...
},
"ContainerInstance" : {
...
},
...
}
All the resources you want to create using Cloudformation should be within the resources section. This gives a better anatomy of the template, http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html