Lambda Cloudformation Encountered unsupported property Value Error - amazon-web-services

The cloudformation is failing when trying to create lambda function with the error message "Encountered unsupported property Value"
There is no reference to the unsupported value and I couldn't find any incorrect value. All the values were used from AWS lambda cloud formation template only.
Also for the Dev I get the error indicating security group is string type but for QA doesn't get the error.
Can you please check point out what's causing unsupported value error and how to resolve security group related error for Dev environment.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Create Lambda Function For abc",
"Parameters": {
"ID" : {
"Description" : "OwnerContact Value",
"Type" : "String",
"Default" : "abc#xyz.com"
},
"abcVPCNAME": {
"Description": "abc VPC NAME",
"Type": "String",
"Default": "abc-e-dev",
"AllowedValues": [
"abc-e-dev",
"abc-e-qa",
"abc-e-prod",
"abc-w-qa",
"abc-w-prod",
]
}
},
"Mappings" : {
"params" : {
"abc-e-dev" : {
"S3bukcet" : "abc-dev-east",
"S3Key" : "/lambda/abc_S3.zip",
"TicketSNS" : "arn:aws:sns:us-east-1:212:abc",
"HOSTNAME" : "abc.com",
"ROLENAME" : "arn:aws:iam::454:role/Lambda-role",
"Subnets" : ["subnet-1","subnet-2","subnet-3"],
"SecGrps" : ["sg-1","sg-2"],
"TAG1" : "xyz",
"TAG2" : "123"
},
"abc-e-qa" : {
"S3bukcet" : "abc-qa-east",
"S3Key" : "/lambda/abc_S3.zip",
"TicketSNS" : "arn:aws:sns:us-east-1:212:abc",
"HOSTNAME" : "xyz.com",
"ROLENAME" : "arn:aws:iam::454:role/Lambda-role",
"Subnets" : ["subnet-1","subnet-2","subnet-3"],
"SecGrps" : "sg-123",
"TAG1" : "xyz",
"TAG2" : "123"
},
}
},
"Resources": {
"abcS3Get": {
"Type" : "AWS::Lambda::Function",
"Properties" : {
"Code" : {
"S3Bucket" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "S3bukcet" ]},
"S3Key" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "S3Key" ]}
},
"DeadLetterConfig" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "TicketSNS" ]},
"Description" : "abc Lambda Function For File Pickup",
"Environment" : {
"Key": "abcHOST",
"Value": { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "HOSTNAME" ]}
},
"FunctionName" : "abc-S3-Pickup",
"Handler" : "abc_S3_Get.lambda_handler",
"MemorySize" : 128,
"Role" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "ROLENAME" ]},
"Runtime" : "python2.7",
"Timeout" : 3,
"VpcConfig" : {
"SecurityGroupIds" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "SecGrps" ]},
"SubnetIds" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "Subnets" ]}
},
"Tags" : [{
"Key" : "KEY1",
"Value" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "TAG1" ]}
},
{
"Key" : "KEY2",
"Value" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "TAG2" ]}
},
{
"Key" : "KEY3",
"Value" : {"Ref":"ID"}
}
]
}
}
}
}

Found the resolution. It was issue with Value parameter in Environment which is incorrect.
Corrected to below and resolved the issue.
"Environment" : {
"Variables" : {
"abcHOST": {
"Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "HOSTNAME" ]
}
}
There were couple of other issues as well.
"S3Key" : "/lambda/abc_S3.zip",
should be
"S3Key" : "lambda/abc_S3.zip",
Also Deadletterconfig parameter needs to be altered as well.
Current Value:
"DeadLetterConfig" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "TicketSNS" ]},
Correct Value:
"DeadLetterConfig" : {
"TargetArn" : { "Fn::FindInMap" : [ "params", {"Ref":"abcVPCNAME"}, "TicketSNS" ]}
},
The CFT started working after all of the above changes.

Related

Retain Output on Stack-Update

I wrote a Cloudformation template that creates a stack with a DynamoDB Table:
"FeedStageDynamoTable" : {
"Type" : "AWS::DynamoDB::Table",
"UpdateReplacePolicy" : "Retain",
"Properties" : {
"TableName" : { "Fn::Sub": [ "feed-${Year}-${Environment}-table", { "Year": {"Ref" : "BundleYear" }, "Environment" : {"Ref" : "DeployEnvironment"}} ]},
"AttributeDefinitions" : [
{"AttributeName" : "Guid", "AttributeType" : "S"}
],
"KeySchema" : [
{"AttributeName" : "Guid", "KeyType" : "HASH"}
],
"ProvisionedThroughput" : {
"ReadCapacityUnits" : "2",
"WriteCapacityUnits" : "2"
},
"StreamSpecification": {
"StreamViewType": "NEW_AND_OLD_IMAGES"
}
}
}
and an Output for the Table's Stream:
"Outputs" : {
"FeedStageTableStreamArn": {
"Description" : "The security group ID to use for public web servers",
"Value" : { "Fn::GetAtt" : ["FeedStageDynamoTable", "StreamArn"] },
"Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-FeedStageDynamoTableStreamArn" }}
},
The output is used by a lambda function from another template (for the second stack):
"NotifyWebConsumer" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Environment": {
"Variables" : {
"EnvironmentCodename" : { "Fn::Sub": [ "${Environment}", { "Environment" : {"Ref" : "DeployEnvironment"}} ]}
}
},
"Handler": "AmazonServerlessStageWebUpdate::AmazonServerlessStageWebUpdate.Functions::NotifyWebConsumer",
"FunctionName": "NotifyWebConsumer",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Policies":[{ "Fn::ImportValue" : {"Fn::Sub" : "${DeployEnvironment}-LambdaExecutionPolicy"} }],
"Role":{"Fn::GetAtt": ["NotifyConsumerRole","Arn"]},
"Events": {
}
}
}
...
"EventSourceMapping": {
"Type": "AWS::Lambda::EventSourceMapping",
"Properties": {
"EventSourceArn": { "Fn::ImportValue" : {"Fn::Sub" : "StageEnvironment-FeedStageDynamoTableStreamArn"} },
"FunctionName" : {
"Fn::GetAtt": [
"NotifyWebConsumer", "Arn"
]
},
"StartingPosition" : "LATEST"
}
}
After I published both stacks I can not update the first one anymore, because the output stream is in use by the second stack. Now my question: is there a cloudformation property like "UpdateReplacePolicy": "Retain" for outputs? Or is there any other way to update the first stack without deleting the second one?

cloudformation failing with security group mismatch

I'm trying to setup my cloudformation for my database:
"VPC" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsSupport" : "false",
"EnableDnsHostnames" : "false",
"InstanceTenancy" : "default",
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Subnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"CidrBlock" : "10.0.0.0/16",
"AvailabilityZone" : { "Fn::Select": [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" } }]},
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Allow http to client host",
"VpcId" : {"Ref" : "VPC"},
"SecurityGroupIngress" : [{
"IpProtocol" : "tcp",
"FromPort" : "3306",
"ToPort" : "3306",
"CidrIp" : "10.0.0.0/16"
}],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Database" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"DBName" : { "Fn::Join": ["", { "Fn::Split": [".", { "Ref" : "DomainName" }]}]},
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.t2.micro",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : { "Ref": "DBUsername" },
"MasterUserPassword" : { "Ref": "DBPassword" },
"VPCSecurityGroups" : [ { "Fn::GetAtt": [ "SecurityGroup", "GroupId" ] } ],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
},
"DeletionPolicy" : "Snapshot"
},
Should be setting up a VPC for the database. But when I run the cloudformation template I get the following error:
UPDATE_FAILED AWS::RDS::DBInstance Database Database is in vpc-3081245b, but Ec2 Security Group sg-b122ffca is in vpc-f7173290
How do I get my database in the VPC properly?
As part of your Database definition, you can specify a DBSubnetGroupName.
A DB Subnet Group provides a list of subnets in which the Database is allowed to run. Each subnet in a DB Subnet Group belongs to a VPC.
Therefore, you need to do the following to your Amazon CloudFormation template:
Add a AWS::RDS::DBSubnetGroup, specifying the Subnet already defined in your template
Add a DBSubnetGroupName parameter to your AWS::RDS::DBInstance definition

HoztedZone error on cloudformation

I´m using cloudformation to create a ecs container and add this new container into route53 hostzone.
But when I run this script I´m having problems with the HostedZone tags
Here The error
A client error (ValidationError) occurred when calling the CreateStack operation: Invalid template parameter property 'Properties'
Here the json
"Parameters" : {
"InstanceType" : {
"Description" : "Container Instance type",
"Type" : "String",
"Default" : "t2.medium",
"AllowedValues" : [ "t2.micro", "t2.small", "t2.medium", "m3.medium", "m3.large", "m3.xlarge", "m3.2xlarge" ],
"ConstraintDescription" : "must be a valid EC2 instance type."
},
"HostedZone" : {
"Type": "AWS::Route53::HostedZone",
"Properties": {
"HostedZoneConfig": {
"Comment": "My hosted zone for example.com"
},
"Name": "***.couchbase.com",
"VPCs": [
{
"VPCId": "*********",
"VPCRegion": "eu-west-1"
}
],
"HostedZoneTags": [
{
"Key": "Name",
"Value": "Couchbase DNS"
}
]
}
}
},
"Resources" : {
"ContainerInstance" : {
"Type": "AWS::EC2::Instance",
"Properties": {
"Tags": [{
"Key" : "Name",
"Value" : "Couchbase-1"
},
{
"Key" : "Type",
"Value" : "ECS-Couchbase"
}],
"IamInstanceProfile" : { "Ref" : "ECSIamInstanceProfile" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
"InstanceType" : { "Ref" : "InstanceType" },
"SecurityGroups" : [ "ssh","default", "couchbase" ],
"KeyName" : { "Ref" : "KeyName" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -xe\n",
"echo ECS_CLUSTER=", { "Ref" : "ClusterName" },
" >> /etc/ecs/ecs.config\n"
]]}}
}
},
"CouchbaseDNSRecord" : {
"Type" : "AWS::Route53::RecordSet",
"Properties" : {
"HostedZoneName" : {
"Fn::Join" : [ "", [
{ "Ref" : "HostedZone" }, "."
] ]
},
"Comment" : "DNS name for my instance.",
"Name" : {
"Fn::Join" : [ "", [
{"Ref" : "ContainerInstance"}, ".",
{"Ref" : "AWS::Region"}, ".",
{"Ref" : "HostedZone"} ,"."
] ]
},
"Type" : "A",
"TTL" : "900",
"ResourceRecords" : [
{ "Fn::GetAtt" : [ "ContainerInstance", "PublicIp" ] }
]
}
},
The HostedZone should be inside the Resources section.
"Parameters" : {
"InstanceType" : {
...
}
},
"Resources" : {
"HostedZone" : {
...
},
"ContainerInstance" : {
...
},
...
}
All the resources you want to create using Cloudformation should be within the resources section. This gives a better anatomy of the template, http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html

How are my ENIs being created?

I have the below code but I don't see where an actual ENI is being created "AWS::EC2::NetworkInterface". I assume this is being done within the UserData section of the script after the "UTMLaunchConfiguration" runs. I'm still learning so I'm sure this isn't a terribly difficult config but I'm still learning how to read it.
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Sophos UTM HA template",
"Mappings" : {
"RegionMap" : {
"us-gov-west-1" : { "BYOL" : "ami-09f7972a", "AZ1" : "us-gov-west-1a", "AZ2" : "us-gov-west-1b", "EC2API" : "ec2.us-gov-west-1.amazonaws.com" }
}
},
"Parameters" : {
"01LicenseType": {
"Description": "Hourly or BYOL",
"Type" : "String",
"Default" : "Hourly",
"AllowedValues" : [ "Hourly", "BYOL" ]
},
"02InstanceType" : {
"Description" : "Amazon EC2 instance type",
"Type" : "String",
"Default" : "m3.medium",
"AllowedValues" : [ "t1.micro", "m1.small", "m3.medium", "m3.large", "m3.xlarge", "m3.2xlarge", "c3.large", "c3.xlarge", "c3.2xlarge", "c3.4xlarge", "c3.8xlarge"]
,
"ConstraintDescription" : "must be a valid EC2 instance type."
},
"03Hostname" : {
"Description" : "The hostname field can have a maximum of 64 characters. Hostname can only contain lower and uppercase letters, numbers, dots(.), dashes(-) and needs to begin with a letter or a number.",
"Type" : "String",
"MaxLength" : "64",
"AllowedPattern" : "^[0-9A-Za-z]{1}([A-Za-z0-9.-]*)$",
"ConstraintDescription" : "Invalid hostname. The hostname field can have a maximum of 64 characters. Hostname can only contain lower- and uppercase letters, numbers, dots(.), dashes(-) and needs to begin with a letter or a number."
},
"04City" : {
"Description" : "City",
"Type" : "String"
},
"05Country" : {
"Description" : "Country",
"Type" : "String",
"AllowedValues" : ["Andorra","United Arab Emirates","Afghanistan","Antigua and Barbuda","Anguilla","Albania","Armenia","Netherlands Antilles", "Angola", "Antarctica","Argentina","American Samoa","Austria","Australia","Aland Islands","Aruba","Azerbaidjan","Bosnia and Herzegovina", "Barbados", "Bangladesh", "Belgium","Burkina Faso","Bulgaria","Bahrain","Burundi","Benin","Saint Barthelemey","Bermuda","Brunei Darussalam", "Bolivia", "Brazil", "Bahamas", "Bhutan","Bouvet Island","Botswana","Belarus","Belize","Canada","Cocos Islands","Congo, Democratic Rebpulic of the","Central African Republic", "Congo","Switzerland","Ivory Coast","Cook Islands","Chile","Cameroon","China","Colombia","Costa Rica","Cuba","Cape Verde","Christmas Island", "Cyprus","Czech Republic","Germany","Djibouti","Denmark","Dominica","Dominican Republic", "Algeria", "Ecuador", "Estonia", "Egypt", "Western Sahara", "Eritrea","Spain","Ethiopia","Finland","Fiji","Falkland Islands","Micronesia","Faroe Islands","France","Gabon","Great Britain", "Grenada", "Georgia", "French Guyana", "Guernsey", "Ghana", "Gibraltar", "Greenland"," Gambia", "Guinea", "Guadeloupe","Equatorial Guinea","Greece","S. Georgia & S. Sandwich Isls.", "Guatemala","Guam (USA)","Guinea Bissau","Guyana","Hong Kong","Heard and McDonald Islands", "Honduras", "Croatia", "Haiti", "Hungary", "Indonesia","Ireland","Israel","Isle of Man","India","British Indian Ocean Territory", "Iraq", "Iran", "Iceland", "Italy", "Jersey", "Jamaica", "Jordan","Japan","Kenya","Kyrgyz Republic (Kyrgyzstan)","Cambodia, Kingdom of","Kiribati","Comoros","Saint Kitts & Nevis Anguilla","North Korea", "South Korea","Kuwait","Cayman Islands","Kazakhstan","Laos","Lebanon","Saint Lucia","Liechtenstein","Sri Lanka", "Liberia", "Lesotho", "Lithuania", "Luxembourg","Latvia","Libya","Morocco","Monaco","Moldavia","Montenegro","Saint Martin (French)","Madagascar", "Marshall Islands", "Macedonia", "Mali","Myanmar","Mongolia","Macau","Northern Mariana Islands","Martinique (French)","Mauritania", "Montserrat", "Malta", "Mauritius", "Maldives", "Malawi","Mexico","Malaysia","Mozambique","Namibia","New Caledonia (French)","Niger","Norfolk Island", "Nigeria", "Nicaragua", "Netherlands", "Norway","Nepal","Nauru","Niue","New Zealand","Oman","Panama","Peru","Polynesia (French)","Papua New Guinea", "Philippines", "Pakistan", "Poland","Saint Pierre and Miquelon","Pitcairn Island","Puerto Rico","Palestinian Territory (Occupied)","Portugal", "Palau", "Paraguay", "Qatar", "Reunion (French)","Romania","Serbia","Russian Federation","Rwanda","Saudi Arabia","Solomon Islands", "Seychelles", "Sudan", "Sweden", "Singapore","Saint Helena","Slovenia","Svalbard Jan Mayen Islands","Slovak Republic","Sierra Leone","San Marino", "Senegal", "Somalia", "Suriname", "Saint Tome and Principe","El Salvador","Syria","Swaziland","Turks and Caicos Islands","Chad","French Southern Territories", "Togo", "Thailand", "Tadjikistan", "Tokelau","Timor-Leste","Turkmenistan","Tunisia","Tonga","Turkey","Trinidad and Tobago", "Tuvalu", "Taiwan", "Tanzania", "Ukraine", "Uganda", "United Kingdom","USA Minor Outlying Islands","United States","Uruguay","Uzbekistan","Holy See (Vatican City State)","Saint Vincent & Grenadines", "Venezuela","Virgin Islands (British)","Virgin Islands (USA)","Vietnam","Vanuatu","Wallis and Futuna Islands", "Samoa", "Yemen", "Mayotte","South Africa","Zambia","Zimbabwe"]
},
"06Email": {
"Default": "",
"Description": "UTM admin email",
"Type": "String"
},
"07Organization" : {
"Description" : "Name of your Organization",
"Type" : "String"
},
"08AdminPassword" : {
"Description" : "UTM admin password",
"Type" : "String",
"NoEcho" : "True"
},
"09KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access.",
"Type": "AWS::EC2::KeyPair::KeyName",
"Default" : ""
},
"10TrustedNetwork" : {
"Description" : "Trusted network. Only connections from this network are allowed to ports 22 and 8080. E.g 92.198.130.0/24",
"Type" : "String",
"Default" : "0.0.0.0/0"
},
"VPC" : {
"Description" : "VPC Id to deploy resources into",
"Type" : "String"
},
"Subnet1" : {
"Description" : "Public_Subnet_AZ1_Outside",
"Type" : "String"
},
"Subnet2" : {
"Description" : "Public_Subnet_AZ2_Outside",
"Type" : "String"
},
"ExistingS3Bucket" : {
"Description" : "Optional. The S3 Bucket to store and restore backups. If left empty a new bucket will be created automatically.",
"Type" : "String",
"Default": ""
},
"ExistingElasticIP" : {
"Description" : "Optional. The Elastic IP to assign in the UTM instance. If left empty a new Elastic IP will be allocated automatically.",
"Type" : "String",
"Default": ""
},
"LicensePool" : {
"Description" : "Optional. S3 bucket where the licenses are stored",
"Type" : "String",
"Default": ""
}
},
"Conditions" : {
"CreateS3Bucket" : { "Fn::Equals" : [{"Ref" : "ExistingS3Bucket"}, ""] },
"AllocateElasticIP" : { "Fn::Equals" : [{"Ref" : "ExistingElasticIP"}, ""] }
},
"Resources": {
"PublicNetworkAcl" : {
"Type" : "AWS::EC2::NetworkAcl",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"Tags" : [
{"Key" : "Application", "Value" : { "Ref" : "AWS::StackId"} },
{"Key" : "Network", "Value" : "Public" }
]
}
},
"InboundHTTPPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "100",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "false",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "80", "To" : "80"}
}
},
"InboundDynamicPortsPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "200",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "false",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "1024", "To" : "65535"}
}
},
"InboundSSHPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "300",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "false",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "22", "To" : "22"}
}
},
"InboundRedUDPPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "400",
"Protocol" : "17",
"RuleAction" : "allow",
"Egress" : "false",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "3410", "To" : "3410"}
}
},
"OutboundSSHPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "105",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "22", "To" : "22"}
}
},
"OutboundHTTPPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "205",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "80", "To" : "80"}
}
},
"OutBoundDynamicPortPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "305",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "1024", "To" : "65535"}
}
},
"OutboundHTTPSPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "405",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "443", "To" : "443"}
}
},
"OutboundRedPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "406",
"Protocol" : "6",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "3000", "To" : "3000"}
}
},
"OutboundRedUDPPublicNetworkAclEntry" : {
"Type" : "AWS::EC2::NetworkAclEntry",
"Properties" : {
"NetworkAclId" : {"Ref" : "PublicNetworkAcl"},
"RuleNumber" : "407",
"Protocol" : "17",
"RuleAction" : "allow",
"Egress" : "true",
"CidrBlock" : "0.0.0.0/0",
"PortRange" : {"From" : "3410", "To" : "3410"}
}
},
"IPAddress" : {
"Type" : "AWS::EC2::EIP",
"Condition": "AllocateElasticIP",
"Properties" : {
"Domain": "vpc"
}
},
"S3Bucket" : {
"Type" : "AWS::S3::Bucket",
"Condition" : "CreateS3Bucket",
"DeletionPolicy" : "Retain",
"Properties" : {
"LifecycleConfiguration" : {
"Rules" : [
{
"Prefix" : "confd_backup",
"ExpirationInDays" : "3",
"Status" : "Enabled"
},
{
"Prefix" : "postgres_basebackup",
"ExpirationInDays" : "3",
"Status" : "Enabled"
},
{
"Prefix" : "postgres_wal",
"ExpirationInDays" : "3",
"Status" : "Enabled"
}
]
}
}
},
"UTMRole" : {
"Type" : "AWS::IAM::Role",
"Properties" : {
"AssumeRolePolicyDocument" : {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [ "ec2.amazonaws.com" ]
},
"Action": [ "sts:AssumeRole" ]
}]
},
"Path": "/",
"Policies": [{
"PolicyName" : "UTMPolicy",
"PolicyDocument" : {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*"
},
{
"Effect": "Allow",
"NotAction": "iam:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "cloudformation:UpdateStack",
"Resource": "*"
}]
}
}]
}
},
"UTMInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [{
"Ref": "UTMRole"
}]
}
},
"UTMSecurityGroup": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Identifying security group",
"VpcId" : { "Ref" : "VPC" }
}
},
"TrustedNetworkGroup": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable TCP access from trusted network",
"SecurityGroupIngress" : [ {
"IpProtocol" : "tcp",
"FromPort" : "0",
"ToPort" : "65535",
"CidrIp" : { "Ref" : "10TrustedNetwork"}
} ],
"VpcId" : { "Ref" : "VPC" }
}
},
"UntrustedGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Untrusted network restricted from accessing port 22 and 8080.",
"SecurityGroupIngress" : [
{
"IpProtocol" : "tcp",
"FromPort" : "12",
"ToPort" : "21",
"CidrIp" : "0.0.0.0/0"
},
{
"IpProtocol" : "tcp",
"FromPort" : "23",
"ToPort" : "8079",
"CidrIp" : "0.0.0.0/0"
},
{
"IpProtocol" : "tcp",
"FromPort" : "8081",
"ToPort" : "65535",
"CidrIp" : "0.0.0.0/0"
},
{
"IpProtocol" : "udp",
"FromPort" : "3410",
"ToPort" : "3410",
"CidrIp" : "0.0.0.0/0"
}
],
"VpcId" : { "Ref" : "VPC" }
}
},
"UTMScalingGroup": {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"UpdatePolicy" : {
"AutoScalingRollingUpdate" : {
"MinInstancesInService" : "1",
"MaxBatchSize" : "1",
"PauseTime" : "PT5M"
}
},
"Properties" : {
"DesiredCapacity" : "1",
"MaxSize" : "2",
"MinSize" : "1",
"TerminationPolicies" : [
"NewestInstance"
],
"VPCZoneIdentifier" : [
{ "Ref" : "Subnet1" },
{ "Ref" : "Subnet2" }
],
"LaunchConfigurationName" : {"Ref": "UTMLaunchConfiguration"},
"NotificationConfiguration" : {
"TopicARN" : { "Ref" : "UnhealthyTopic" },
"NotificationTypes" : [
"autoscaling:EC2_INSTANCE_LAUNCH",
"autoscaling:EC2_INSTANCE_LAUNCH_ERROR",
"autoscaling:EC2_INSTANCE_TERMINATE",
"autoscaling:EC2_INSTANCE_TERMINATE_ERROR"
]
},
"Tags" : [{
"Key" : "Name",
"Value" : "HA-UTM",
"PropagateAtLaunch" : "true"
}]
}
},
"UTMLaunchConfiguration": {
"Type" : "AWS::AutoScaling::LaunchConfiguration",
"Properties" : {
"AssociatePublicIpAddress" : true,
"IamInstanceProfile" : { "Ref" : "UTMInstanceProfile" },
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, {"Ref": "01LicenseType"} ]},
"InstanceType" : { "Ref" : "02InstanceType" },
"KeyName" : { "Ref" : "09KeyName" },
"BlockDeviceMappings" : [{
"DeviceName" : "/dev/sda",
"Ebs" : { "VolumeSize" : "100" }
}],
"SecurityGroups" : [
{ "Ref": "UTMSecurityGroup" },
{ "Ref": "TrustedNetworkGroup" },
{ "Ref": "UntrustedGroup" }
],
"UserData" : { "Fn::Base64" : { "Fn::Join" : [ "", [
"#!/bin/bash\n",
"echo AWS_REGION=",{ "Ref" : "AWS::Region" },">>/etc/environment\n",
"export AWS_REGION=",{ "Ref" : "AWS::Region" },"\n",
"echo EC2_URL=",{ "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "EC2API" ]},">>/etc/environment\n",
"export EC2_URL=",{ "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "EC2API" ]},"\n",
"date +'UserData start %c' > /tmp/user_data.log\n",
"echo 'version: \"0.1.0\"' >> /etc/cloud/user_data.yml\n",
"echo 'instance_role: \"ha_standalone\"' >> /etc/cloud/user_data.yml\n",
"echo 'deployment_type: \"ha_warm_standby\"' >> /etc/cloud/user_data.yml\n",
"echo 'license_pool: \"", { "Ref": "LicensePool" }, "\"' >> /etc/cloud/user_data.yml\n",
"cc=`/usr/local/bin/confd-client.plx country_name_to_code \"",{ "Ref": "05Country" },"\"`\n",
"echo \"{ hostname => '", { "Ref": "03Hostname" }, "', organization => '", { "Ref": "07Organization"}, "', city => '", { "Ref" : "04City" },"', country => '$cc', email => '", { "Ref": "06Email" }, "', password => '", { "Ref": "08AdminPassword" }, "' }\" > /var/confd/var/storage/setup.ph\n",
"/etc/init.d/confd restart\n",
"sleep 5\n",
"echo 'elastic_ip: \"",
{ "Fn::If" : [
"AllocateElasticIP",
{ "Ref" : "IPAddress" },
{ "Ref" : "ExistingElasticIP" }
]
},"\"' >> /etc/cloud/user_data.yml\n",
"echo 's3_bucket: \"",
{ "Fn::If" : [
"CreateS3Bucket",
{ "Ref" : "S3Bucket" },
{ "Ref" : "ExistingS3Bucket" }
]
},"\"' >> /etc/cloud/user_data.yml\n",
"echo 'stack_name: \"", { "Ref": "AWS::StackName"} ,"\"' >> /etc/cloud/user_data.yml\n",
"date +'UserData confd config start %c' >> /tmp/user_data.log\n",
"echo '{' > /tmp/user_data.config\n",
"echo '\"cloudwatch\" => { \"status\" => 1 },' >> /tmp/user_data.config\n",
"echo '\"confd\" => { \"backup\" => 1, \"backup_interval\" => 300, \"restore\" => 1, \"restore_done\" => 0 },' >> /tmp/user_data.config\n",
"echo '\"instance_role\" => \"ha_standalone\",' >> /tmp/user_data.config\n",
"echo '\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },' >> /tmp/user_data.config\n",
"echo '\"syslog\" => { \"backup\" => 1, \"restore\" => 1, \"restore_period\" => 8 },' >> /tmp/user_data.config\n",
"echo '\"s3_bucket\" => \"",
{ "Fn::If" : [
"CreateS3Bucket",
{ "Ref" : "S3Bucket" },
{ "Ref" : "ExistingS3Bucket" }
]
},"\",' >> /tmp/user_data.config\n",
"echo '\"stack_name\" => \"", { "Ref": "AWS::StackName"},"\",' >> /tmp/user_data.config\n",
"echo '\"elastic_ip\" => \"",
{ "Fn::If" : [
"AllocateElasticIP",
{ "Ref" : "IPAddress" },
{ "Ref" : "ExistingElasticIP" }
]
}, "\",' >> /tmp/user_data.config\n",
"echo '\"trusted_network\" => \"", { "Ref": "10TrustedNetwork"},"\"' >> /tmp/user_data.config\n",
"echo '}' >> /tmp/user_data.config\n",
"/usr/local/bin/confd-client.plx -noquote -stdin set \\'ha\\' \\'aws\\' < /tmp/user_data.config >> /tmp/user_data.log\n",
"date +'UserData confd config end %c' >> /tmp/user_data.log\n",
"date +'UserData ha aws start %c' >> /tmp/user_data.log\n",
"/etc/init.d/ha_aws start\n",
"date +'UserData awslogs agent setup start %c' >> /tmp/user_data.log\n",
"/usr/local/bin/awslogs-agent-setup-v1.0.py -n -r ", { "Ref" : "AWS::Region" }, " -c /etc/cloud/awslogs.conf >> /tmp/user_data.log\n",
"date +'UserData awslogs agent setup end %c' >> /tmp/user_data.log\n",
"/usr/local/bin/confd-client.plx trigger ha_aws\n",
"exit 0\n"
] ] } }
}
},
"IngressTrafficMinimal": {
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmDescription": "Terminate if the incoming traffic is suspiciously low",
"MetricName": "NetworkIn",
"Namespace": "AWS/EC2",
"Statistic": "Sum",
"Period": "60",
"EvaluationPeriods": "2",
"Threshold": "100",
"AlarmActions": [ { "Ref": "UnhealthyTopic" } ],
"Dimensions": [{
"Name": "AutoScalingGroupName",
"Value": { "Ref": "UTMScalingGroup" }
}],
"ComparisonOperator": "LessThanThreshold"
}
},
"EgressTrafficMinimal": {
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmDescription": "Terminate if the outgoing traffic is suspiciously low",
"MetricName": "NetworkOut",
"Namespace": "AWS/EC2",
"Statistic": "Sum",
"Period": "60",
"EvaluationPeriods": "2",
"Threshold": "100",
"AlarmActions": [ { "Ref": "UnhealthyTopic" } ],
"Dimensions": [ {
"Name": "AutoScalingGroupName",
"Value": { "Ref": "UTMScalingGroup" }
}],
"ComparisonOperator": "LessThanThreshold"
}
},
"UnhealthyTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"Subscription": [{
"Endpoint": { "Ref": "06Email" },
"Protocol": "email"
}]
}
}
},
"Outputs" : {
"PublicIPAddress" : {
"Value" : { "Ref" : "IPAddress" },
"Description" : "Use this IP to connect and to forward traffic to the UTM."
},
"ScalingGroup" : {
"Value" : { "Ref": "UTMScalingGroup" },
"Description" : "The HA Scaling group."
},
"S3Bucket" : {
"Value" : { "Ref": "S3Bucket" },
"Description" : "The S3 Bucket."
}
}
}
https://www.sophos.com/en-us/support/knowledgebase/122202.aspx
from the looks of it confd-client.plx does most of the magic (via the user_data.config that is builds as you've guessed in the user data portion).
The UTMRole has the permissions to basically bind the elastic ip to the instance.
as a side note, you cannot (easily) associate elastic IPs to autoscalling groups (because by their nature they are ephemeral). The autoscalling is basically associated with the LaunchConfig in the template.
also as a side note this is highly specific to the Sophos stuff and it's not how one would typically go about using Cloudformation on AWS.

AWS cloudformation optional line

I'm trying to setup a cloudformation template that will either launch a clean instance or one from snapshots. I'd like to be able to use an if / else type statement so that would look something like
pseudo code:
if InputSnapshotId:
"SnapshotId" : {"Ref" : "InputSnapshotId"},
else:
"Size" : 20,
In cloudformation I have tried a number of things like:
"WebserverInstanceDataVolume" : {
"Type" : "AWS::EC2::Volume",
"Properties" : {
"Fn::If" : [
{"Ref" : "FromSnapshot"},
{"SnapshotId" : { "Ref" : "InputSnapshotId" }},
{"Size" : "20"}
],
"VolumeType" : "standard",
"AvailabilityZone" : { "Fn::GetAtt" : [ "WebserverInstance", "AvailabilityZone" ]},
"Tags" : [
{"Key" : "Role", "Value": "data" },
]
},
"DeletionPolicy" : "Delete"
},
Or wrapping the in Fn::If in {}:
{"Fn::If" : [
{"Ref" : "FromSnapshot"},
{"SnapshotId" : { "Ref" : "InputSnapshotId" }},
{"Size" : "20"}
]}
All of which kicks different types or errors. The first one gives a "Encountered unsupported property Fn::If" in cloudformation, the second, just isn't valid JSON. I could snapshot an empty volume and define a size parameter then always pass a SnapshotId and size but I feel like there must be a way to have an optional line in cloudformation.
Any ideas?
You can do like this:
"Conditions" : {
"NotUseSnapshot" : {"Fn::Equals" : [{"Ref" : "InputSnapshotId"}, ""]}
},
"Resources" : {
"WebserverInstanceDataVolume" : {
"Type" : "AWS::EC2::Volume",
"Properties" : {
"Size" : {
"Fn::If" : [
"NotUseSnapshot",
"20",
{"Ref" : "AWS::NoValue"}
]
},
"SnapshotId" : {
"Fn::If" : [
"NotUseSnapshot",
{"Ref" : "AWS::NoValue"},
{"Ref" : "InputSnapshotId"}
]
},
"VolumeType" : "standard",
"AvailabilityZone" : { "Fn::GetAtt" : [ "WebserverInstance", "AvailabilityZone" ]},
"Tags" : [
{"Key" : "Role", "Value": "data" }
]
},
"DeletionPolicy" : "Delete"
}
}
Here is a link to a functional template: https://github.com/caussourd/public-cloudformation-templates/blob/master/conditional_volume_creation.template