Cloudformation - ElastiCache::SubnetGroup not honouring resource name - amazon-web-services

I have a weird issue with CloudFormation that seems either to be a bug, or more likely - i've missed something pretty basic.
I have the following template (a snippet) defining two subnets and a subnet group as follow:
...
"redissubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.8.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [
{
"Key": "Name",
"Value": "redissubnet1"
}
]
}
},
"redissubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.9.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [
{
"Key": "Name",
"Value": "redissubnet2"
}
]
}
},
"SubnetGroupName": {
"Type": "AWS::ElastiCache::SubnetGroup",
"Properties": {
"Description": "Subnet group for main application redis elastic cache",
"SubnetIds": [
{
"Ref": "redissubnet1"
},
{
"Ref": "redissubnet2"
}
]
}
}
...
All resources are created, yet the SubnetGroup name - "SubnetGroupName" - is not honoured. AWS auto-assigns a name in the format [a-z]-[a-z]-[a-z0-9]
Has anyone encountered this?
What I'm actually trying to do is reference this subnet group by name in the creation of an ElastiCache::Cluster - however because the resource name is not honoured I can't do so.
Anyone have any ideas? All help gratefully received :)

Answer was to reference the subnet group name in the elastic cache resource, as follows:
{
"subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.8.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [{
"Key": "Name",
"Value": "subnet1"
}]
}
},
"subnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.9.0/24",
"AvailabilityZone": "us-east-1c",
"VpcId": {
"Ref": "myVPC"
},
"Tags": [{
"Key": "Name",
"Value": "subnet2"
}]
}
},
"redis1": {
"Type": "AWS::ElastiCache::SubnetGroup",
"Properties": {
"Description": "Subnet group for main application redis elastic cache",
"SubnetIds": [{
"Ref": "subnet1"
}, {
"Ref": "subnet2"
}]
}
},
"mainredis": {
"Type": "AWS::ElastiCache::CacheCluster",
"Properties": {
"AutoMinorVersionUpgrade": "true",
"CacheNodeType": "cache.t2.small",
"CacheSubnetGroupName": {
"Ref": "redis1"
},
"ClusterName": "mainredis",
"Engine": "redis",
"NumCacheNodes": "1",
"Port": "6379",
"Tags": [{
"Key": "Name",
"Value": "mainredis"
}, {
"Key": "Function",
"Value": "Main redis store"
}],
"VpcSecurityGroupIds": [
"redissecuritygroup"
]
}
}
}

Related

Why does my subnet and VPC show side by side on CloudFormation

I've been trying to figure out why my VPC and subnet show side by side instead of the subnet inside of the VPC? (I used Atom to generate this.)
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "vpc",
"Metadata": {
},
"Parameters": {"siggyVpcCidr": {
"Description": "vpc cidr",
"Type": "String",
"Default": "10.0.0.0/16"
},
"siggySubnetCidr": {
"Description": "cidr for the subnet",
"Type": "String",
"Default": "10.0.1.0/2"
},
"Subnet1Az": {
"Description": "AZ for siggySubnetCidr",
"Type": "AWS::EC2::AvailabilityZone::Name"
}
},
"Mappings": {
},
"Conditions": {
},
"Resources": {
"siggyVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": { "Ref": "siggyVpcCidr" },
"Tags": [{ "Key": "Name", "Value": "siggyVpc" }]
}
},
"siggyIgw": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
"Tags": [{ "Key": "Name", "Value": "siggyIgw1" }]
}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"VpcId": { "Ref": "siggyVpc" },
"InternetGatewayId": { "Ref": "siggyIgw" }
}
},
"SubnetSiggy": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": { "Ref": "Subnet1Az" },
"VpcId": { "Ref": "siggyVpc" },
"CidrBlock": { "Ref": "siggySubnetCidr" },
"Tags": [{ "Key": "Name", "Value": "siggySubnetCidr" }]
}
}
},
"Outputs": {
}
}
They are separate resources. CloudFormation templates arrange resources in a flat array. This is pretty much true of most resources. Some resources can be implicitly defined when creating resources, but that probably won't be reflected with an export where you create a template from existing resources.
You would need to inspect the VpcId property to determine the VPC to which the subnet belongs.

AWS CDK: A load balancer cannot be attached to multiple subnets in the same Availability Zone

I'm using AWS CDK and it is failing at App-Load-Balancer level and surprisingly it works for Web-Load-balancer. Looking at generated CloudFormation, it is clear that all "Private Subnets" are getting created in separated AZ and associated Auto-Scaling Group too is creating the instance across multiple-AZs. However, when the CDK is executed, it is failing with the error message - A load balancer cannot be attached to multiple subnets in the same Availability Zone (Service: AmazonElasticLoadBalancing; Status Co
de: 400; Error Code: InvalidConfigurationRequest; Request ID: 62c554cb-34ab-43ef-bac0-be2f0d6fc742; Proxy: null)
APP Server characteristics
AUTOSCALING CF Snippet:
"InstaLendaappASGapp1appsvrASG950CF7C4": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"MaxSize": "3",
"MinSize": "1",
"DesiredCapacity": "2",
"LaunchConfigurationName": {
"Ref": "InstaLendaappASGapp1appsvrLaunchConfig18DAF6BB"
},
"Tags": [
{
"Key": "Name",
"PropagateAtLaunch": true,
"Value": "webapp-dc-3-tier-ha/InstaLend-a-appASG-app-1-appsvr-"
}
],
"TargetGroupARNs": [
{
"Ref": "InstaLendaapplbInstaLendalstnrPrivate80InstaLendatgtprivateGroup8D2C8D01"
}
],
"VPCZoneIdentifier": [
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet1Subnet35AF6769"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet2SubnetD8513C5D"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet3SubnetB7B2D12C"
}
]
}
LOADBALANCER CF Snippet:
"InstaLendaapplbCC4F6682": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"LoadBalancerAttributes": [
{
"Key": "deletion_protection.enabled",
"Value": "false"
}
],
"Name": "InstaLend-a-app-lb",
"Scheme": "internal",
"SecurityGroups": [
{
"Fn::GetAtt": [
"InstaLendasginternal8649CE7C",
"GroupId"
]
}
],
"Subnets": [
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet1Subnet35AF6769"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet2SubnetD8513C5D"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNSubnet3SubnetB7B2D12C"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNDBSubnet1Subnet2DD722D8"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNDBSubnet2Subnet59278CD3"
},
{
"Ref": "InstaLendavpcInstaLendaprivateSNDBSubnet3SubnetCC805230"
}
],
"Type": "application"
},
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete",
"Metadata": {
"aws:cdk:path": "webapp-dc-3-tier-ha/InstaLend-a-app-lb/Resource"
}
}
While LB is selecting 6 subnets, ASG associated is selecting only 3 subnets. The details of 3 ASG Subnets (i.e. Private Subnets) had been pasted below:
1st SUBNET
"InstaLendavpcInstaLendaprivateSNSubnet1Subnet35AF6769": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.2.3.0/24",
"VpcId": {
"Ref": "InstaLendavpcE5C8A638"
},
"AvailabilityZone": "us-east-2a",
"MapPublicIpOnLaunch": false,
"Tags": [
{
"Key": "aws-cdk:subnet-name",
"Value": "InstaLend-a-privateSN"
....
2nd SUBNET
"InstaLendavpcInstaLendaprivateSNSubnet2SubnetD8513C5D": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.2.4.0/24",
"VpcId": {
"Ref": "InstaLendavpcE5C8A638"
},
"AvailabilityZone": "us-east-2b",
"MapPublicIpOnLaunch": false,
"Tags": [
{
"Key": "aws-cdk:subnet-name",
"Value": "InstaLend-a-privateSN"
},
{
"Key": "aws-cdk:subnet-type",
"Value": "Private"
},
....
3rd SUBNET
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.2.5.0/24",
"VpcId": {
"Ref": "InstaLendavpcE5C8A638"
},
"AvailabilityZone": "us-east-2c",
"MapPublicIpOnLaunch": false,
"Tags": [
{
"Key": "aws-cdk:subnet-name",
"Value": "InstaLend-a-privateSN"
},
{
"Key": "aws-cdk:subnet-type",
"Value": "Private"
},

Invalid template property or properties [Vpcname]

I am trying to create a stack with below templates. Same template worked without parameters.
When I Added parameters I got an error which says -Invalid template property or properties
[Vpcname].
I am not sure what more to add as I have checked every aspect from amazon documentation.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Here are some details about the template.",
"Vpcname": {
"Description": "What do you want your VPC to be called as ?",
"Type": "String",
"Default": "fromCf"
},
"Resources": {
"testvpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "192.168.0.0/16",
"EnableDnsHostnames": true,
"EnableDnsSupport": true,
"InstanceTenancy": "default",
"Tags": [
{
"Key": "Name",
"Value": {
"Ref": "Vpcname"
}
}
]
}
},
"WebSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "ap-south-1a",
"VpcId": {
"Ref": "testvpc"
},
"CidrBlock": "192.168.0.0/24",
"Tags": [
{
"Key": "Name",
"Value": "fromCfWebSub"
}
]
}
},
"AppSubnet": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"AvailabilityZone": "ap-south-1b",
"VpcId": {
"Ref": "testvpc"
},
"CidrBlock": "192.168.1.0/24",
"Tags": [
{
"Key": "Name",
"Value": "fromCfAppSub"
}
]
}
}
}
}
I got it fixed Guys. I was not including "Parameters" : { } I should have put it this ways - "Parameters" : {"Vpcname": { "Description": "What do you want your VPC to be called as ?", "Type": "String", "Default": "fromCf" }}

AWS Cloudfromation and autoscaling : The requested configuration is currently not supported. Launching EC2 instance failed

I want to replicate the infrastructure from one region(us-east-1) to another(us-east-2). so,I have generated a cloudfromation template of an existing infrastructure with the help of cloudformer tool.
"asgamazonecsclisetupapijoulebugprodEcsInstanceAsg1EIBNOXSXJ7HD": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": [
"us-east-2b",
"us-east-2c"
],
"Cooldown": "300",
"DesiredCapacity": "3",
"HealthCheckGracePeriod": "300",
"HealthCheckType": "ELB",
"MaxSize": "16",
"MinSize": "3",
"VPCZoneIdentifier": [
{
"Ref": "subnet81c8ebab"
},
{
"Ref": "subnet5df40214"
}
],
"LaunchConfigurationName": {
"Ref": "lcamazonecsclisetupapijoulebugprodAMI2017d"
},
"LoadBalancerNames": [
{
"Ref": "elbJBAPILiveCleanbit2016"
}
],
"Tags": [
{
"Key": "Name",
"Value": "Live - Cleanbit2016 - joulebug-api",
"PropagateAtLaunch": true
}
],
"TerminationPolicies": [
"Default"
]
}
},
"lcamazonecsclisetupapijoulebugprodAMI2017d": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"AssociatePublicIpAddress": true,
"ImageId": "ami-0e6d83442546978bc",
"InstanceType": "c3.large",
"KeyName": "cleanbit2016_vpc",
"IamInstanceProfile": "amazon-ecs-cli-setup-api-joulebug-prod-EcsInstanceProfile-1M4GOHBP3FP5L",
"InstanceMonitoring": "true",
"SecurityGroups": [
{
"Ref": "sgCleanbit2016WebServerSG"
}
],
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"SnapshotId": "snap-0b2477be9c863d014",
"VolumeSize": 8
}
},
{
"DeviceName": "/dev/xvdcz",
"Ebs": {
"VolumeSize": 22
}
}
]
}
},
"elbJBAPILiveCleanbit2016": {
"Type": "AWS::ElasticLoadBalancing::LoadBalancer",
"Properties": {
"Policies": [
{
"PolicyName": "AWSConsole-SSLNegotiationPolicy-JB-API-Live-Cleanbit2016-1467998170471",
"PolicyType": "SSLNegotiationPolicyType",
}
],
}
}
"subnet81c8ebab": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.3.0/24",
"AvailabilityZone": "us-east-2b",
"VpcId": {
"Ref": "vpcdcbd08bb"
},
"Tags": [
{
"Key": "Name",
"Value": "Cleanbit2016 - Public 1b"
}
]
}
},
"sgCleanbit2016WebServerSG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Web server security group for public subnet in vpc.",
"VpcId": {
"Ref": "vpcdcbd08bb"
},
"Tags": [
{
"Key": "Name",
"Value": "Cleanbit2016_ WebServerSG"
}
]
}
},
While launching the template in other region(us-east-2), it is throwing following error:
The requested configuration is currently not supported. Please check the documentation for supported configurations. Launching EC2 instance failed.
You don't have details of the regions you're using, but if you are trying to do this outside of us-east-1 the Availablility Zones won't work. It also looks like you have a number of other parts of the stack hard-coded, which may not work in another region.
And if you are trying to do this in us-east-1, there is the possibility that one of the AZs is unavailable to you - see this question for more details.
You have not provided enough information to be able to diagnose the situation.
I took your template, removed the portions that were incomplete (eg removed the Load Balancer because it was missing Listeners), simplified a few things and it works fine:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"Tags": [
{
"Key": "Name",
"Value": "Lab VPC"
}
]
}
},
"ASG": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": [
"us-east-2b",
"us-east-2c"
],
"Cooldown": "300",
"DesiredCapacity": "1",
"HealthCheckGracePeriod": "300",
"MaxSize": "16",
"MinSize": "1",
"VPCZoneIdentifier": [
{
"Ref": "Subnet1"
},
{
"Ref": "Subnet2"
}
],
"LaunchConfigurationName": {
"Ref": "LaunchConfig"
}
}
},
"LaunchConfig": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"AssociatePublicIpAddress": true,
"ImageId": "ami-0b59bfac6be064b78",
"InstanceType": "t2.micro",
"InstanceMonitoring": "true",
"SecurityGroups": [
{
"Ref": "WebServerSG"
}
]
}
},
"Subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": "us-east-2b",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "Public 1"
}
]
}
},
"Subnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone": "us-east-2c",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "Public 2"
}
]
}
},
"WebServerSG": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Web server security group for public subnet in vpc.",
"VpcId": {
"Ref": "VPC"
},
"Tags": [
{
"Key": "Name",
"Value": "WebServerSG"
}
]
}
}
}
}
Therefore, your problem lies in part of the template you did not provide.
You could start with this version, then progressively add back parts of your template until you discover what is causing the error.
If you wish to create a CloudFormation template that can run in multiple regions, you should not refer to specific Availability Zones (eg us-east-2b).
You can use Fn::GetAZs - AWS CloudFormation to obtain a list of AZs in region.
After a lot of debugging, when I started launching the things manually, I found the same error and I got to know that c3.large is causing the error. When I launch the template with c4.large it successfully launched the template from us-east-1 to us-east-2.

Trying to setup lambda to access my RDS server on AWS but getting timeouts

I know AWS allows this now and has instructions for a manual setup. I'm trying to set this up in a cloudformation though and am running into difficulties. Currently when I try to access my RDS server I am getting connect ETIMEDOUT errors.
I have my VPC, Subnets and security groups setup with the following:
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport": "false",
"EnableDnsHostnames": "false",
"InstanceTenancy": "default",
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SubnetA": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": { "Fn::Select": [ "0", { "Fn::GetAZs": { "Ref": "AWS::Region" } }]},
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SubnetB": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone": { "Fn::Select": [ "1", { "Fn::GetAZs": { "Ref": "AWS::Region" } }]},
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SubnetGroup": {
"Type": "AWS::RDS::DBSubnetGroup",
"Properties": {
"DBSubnetGroupDescription": "Database Access",
"SubnetIds" : [{ "Ref": "SubnetA" }, { "Ref": "SubnetB" }],
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
"SecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Database Access",
"VpcId": {"Ref": "VPC"},
"SecurityGroupIngress" : [{
"IpProtocol": "tcp",
"FromPort": "3306",
"ToPort": "3306",
"CidrIp": "10.0.0.0/16"
}],
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},
I don't actually use SubnetB but to make AWS::RDS::DBSubnetGroup you need subnets in at least two availability zones.
My RDS database is setup with the VPC and Security group.
"Database": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName": { "Fn::Join": ["", { "Fn::Split": [".", { "Ref": "DomainName" }]}]},
"AllocatedStorage": "5",
"DBInstanceClass": "db.t2.micro",
"Engine": "MySQL",
"EngineVersion": "5.5",
"MasterUsername": { "Ref": "DBUsername" },
"MasterUserPassword": { "Ref": "DBPassword" },
"DBSubnetGroupName": { "Ref": "SubnetGroup" },
"VPCSecurityGroups" : [{ "Fn::GetAtt": [ "SecurityGroup", "GroupId" ] }],
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
},
"DeletionPolicy": "Snapshot"
},
So I think the database side is correctly in the VPC and able to talk both subnets.
My lambda is setup in the security group and in SubnetA.
"LambdaFunctionUpdate": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"ZipFile": "exports.handler = function (event, context) { context.succeed(\"Hello, World!\"); };"
},
"Description": "Used to create and or sync database tables to the application models",
"Handler": "index.handler",
"MemorySize": 128,
"Role": { "Fn::GetAtt": ["LambdaExecutionRole", "Arn" ] },
"Runtime": "nodejs4.3",
"Timeout": 30,
"VpcConfig": {
"SecurityGroupIds": [{ "Fn::GetAtt": ["SecurityGroup", "GroupId"] }],
"SubnetIds": [{"Ref": "SubnetA"}]
}
}
},
At the end of everything I'm outputting the database endpoint information.
"Outputs": {
"DatabaseEndpoint": {
"Value": { "Fn::Join" : [":", [{ "Fn::GetAtt": ["Database", "Endpoint.Address" ] }, { "Fn::GetAtt": ["Database", "Endpoint.Port" ] }]]},
"Description": "Database endpoint"
}
}
When I run my lambda and try to connect to the RDS server with the endpoint given I get timeout errors.
{
"errorMessage": "connect ETIMEDOUT",
"errorType": "SequelizeConnectionError",
"stackTrace": [
"Handshake._callback (/var/task/node_modules/sequelize/lib/dialects/mysql/connection-manager.js:95:20)",
"Handshake.Sequence.end (/var/task/node_modules/mysql/lib/protocol/sequences/Sequence.js:86:24)",
"Protocol.handleNetworkError (/var/task/node_modules/mysql/lib/protocol/Protocol.js:364:14)",
"Connection._handleNetworkError (/var/task/node_modules/mysql/lib/Connection.js:428:18)",
"Connection._handleConnectTimeout (/var/task/node_modules/mysql/lib/Connection.js:424:8)",
"Socket.g (events.js:260:16)",
"emitNone (events.js:67:13)",
"Socket.emit (events.js:166:7)",
"Socket._onTimeout (net.js:318:8)",
"_runOnTimeout (timers.js:524:11)",
"_makeTimerTimeout (timers.js:515:3)",
"Timer.unrefTimeout (timers.js:584:5)"
]
}
The VPC configuration was incorrect. DNS services can't be turned off.
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true",
"InstanceTenancy": "default",
"Tags" : [{ "Key": "Name", "Value": { "Ref": "DomainName" } }]
}
},