cloudformation failing with security group mismatch - amazon-web-services

I'm trying to setup my cloudformation for my database:
"VPC" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsSupport" : "false",
"EnableDnsHostnames" : "false",
"InstanceTenancy" : "default",
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Subnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"CidrBlock" : "10.0.0.0/16",
"AvailabilityZone" : { "Fn::Select": [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" } }]},
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Allow http to client host",
"VpcId" : {"Ref" : "VPC"},
"SecurityGroupIngress" : [{
"IpProtocol" : "tcp",
"FromPort" : "3306",
"ToPort" : "3306",
"CidrIp" : "10.0.0.0/16"
}],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Database" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"DBName" : { "Fn::Join": ["", { "Fn::Split": [".", { "Ref" : "DomainName" }]}]},
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.t2.micro",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : { "Ref": "DBUsername" },
"MasterUserPassword" : { "Ref": "DBPassword" },
"VPCSecurityGroups" : [ { "Fn::GetAtt": [ "SecurityGroup", "GroupId" ] } ],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
},
"DeletionPolicy" : "Snapshot"
},
Should be setting up a VPC for the database. But when I run the cloudformation template I get the following error:
UPDATE_FAILED AWS::RDS::DBInstance Database Database is in vpc-3081245b, but Ec2 Security Group sg-b122ffca is in vpc-f7173290
How do I get my database in the VPC properly?

As part of your Database definition, you can specify a DBSubnetGroupName.
A DB Subnet Group provides a list of subnets in which the Database is allowed to run. Each subnet in a DB Subnet Group belongs to a VPC.
Therefore, you need to do the following to your Amazon CloudFormation template:
Add a AWS::RDS::DBSubnetGroup, specifying the Subnet already defined in your template
Add a DBSubnetGroupName parameter to your AWS::RDS::DBInstance definition

Related

Getting "Template validation error: Invalid template resource property 'VPCID' "

I am trying to create a VPC using AWS CloudFormation.
I built a VPC on its own, then I updated the existing stack by adding more components to the JSON template (components like subnets, internet gateways, NATs, route tables, etc.) - one component at a time.
My VPC created successfully but when I tried to update the stack with an internet gateway and attach to the VPC, I started getting the error Template validation error: Invalid template resource property 'VPCID'.
My JSON template is as follows:
{
"Parameters": {
"CIDRRange": {
"Description": "VPCCIDR Range (will be a /16 block)",
"Type": "String",
"Default": "10.251.0.0",
"AllowedValues": ["10.250.0.0","10.251.0.0"]
}
},
"Resources": {
"VPCBase": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": { "Fn::Join" : ["", [{ "Ref" : "CIDRRange" }, "/16"]] },
"EnableDnsSupport": "True",
"EnableDnsHostnames": "True",
"Tags": [{ "Key": "Name", "Value": { "Fn::Join" : ["", [{ "Ref" : "AWS::StackName" }, "-VPC"]] } }]
}
},
"IGWBase" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
"Tags" : [{ "Key": "Name", "Value": { "Fn::Join" : ["", [{ "Ref" : "AWS::StackName" }, "-IGW"]] } }]
}
},
"VGAIGWBase" : {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : { "Ref" : "IGWBase" },
"VpcId" : { "Ref" : "VPCBase" }}
},
"Outputs": {
"VPCID" : { "Value" : { "Ref" : "VPCBase" } },
"DefaultSG" : { "Value" : { "Fn::GetAtt" : ["VPCBase", "DefaultSecurityGroup"] }}
}
}
}
Your formatting is a bit of a mess - I'd recommend going with yaml over json - but the problem is you're not closing the Resources: section.
You can validate a template with the cli with
aws cloudformation validate-template --template-body file://path.json
"VGAIGWBase" : {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : { "Ref" : "IGWBase" },
"VpcId" : { "Ref" : "VPCBase" }
}
}
}, << ADD THIS
"Outputs": {
"VPCID" : { "Value" : { "Ref" : "VPCBase" } },
"DefaultSG" : { "Value" : { "Fn::GetAtt" : ["VPCBase", "DefaultSecurityGroup"] }}
}
}

Is there a way to configure the 'Stack Name' of nested stacks in AWS cloudformation ?

I am trying to create a nested stack using AWS CloudFormation.
I need to specify the 'Stack Name' of the nested stack.
I tried using the 'Tags' property with Key: 'Stack Name'. But that didn't help.
Is there any way to provide the Stack Name as an input while creating nested stack?
In a given template, you have access to the pseudo-parameters stackId and stackName. Those can be marked for export, then referenced in another stack. Joined, they give you the child stack's name.
I found the following video quite useful despite its age: https://youtu.be/6R44BADNJA8
This sample template may also be helpful in understanding how these work: https://s3.amazonaws.com/cloudformation-examples/user-guide/cross-stack/SampleNetworkCrossStack.template
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template VPC_with_PublicIPs_And_DNS: Sample template that creates a VPC with DNS and public IPs enabled. Note that you are billed for the AWS resources that you use when you create a stack from this template.", "Resources" : { "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "EnableDnsSupport" : "true", "EnableDnsHostnames" : "true", "CidrBlock" : "10.0.0.0/16" } }, "PublicSubnet" : { "Type" : "AWS::EC2::Subnet", "Properties" : { "VpcId" : { "Ref" : "VPC" }, "CidrBlock" : "10.0.0.0/24" } }, "InternetGateway" : { "Type" : "AWS::EC2::InternetGateway" }, "VPCGatewayAttachment" : { "Type" : "AWS::EC2::VPCGatewayAttachment", "Properties" : { "VpcId" : { "Ref" : "VPC" }, "InternetGatewayId" : { "Ref" : "InternetGateway" } } }, "PublicRouteTable" : { "Type" : "AWS::EC2::RouteTable", "Properties" : { "VpcId" : { "Ref" : "VPC" } } }, "PublicRoute" : { "Type" : "AWS::EC2::Route", "DependsOn" : "VPCGatewayAttachment", "Properties" : { "RouteTableId" : { "Ref" : "PublicRouteTable" }, "DestinationCidrBlock" : "0.0.0.0/0", "GatewayId" : { "Ref" : "InternetGateway" } } }, "PublicSubnetRouteTableAssociation" : { "Type" : "AWS::EC2::SubnetRouteTableAssociation", "Properties" : { "SubnetId" : { "Ref" : "PublicSubnet" }, "RouteTableId" : { "Ref" : "PublicRouteTable" } } }, "PublicSubnetNetworkAclAssociation" : { "Type" : "AWS::EC2::SubnetNetworkAclAssociation", "Properties" : { "SubnetId" : { "Ref" : "PublicSubnet" }, "NetworkAclId" : { "Fn::GetAtt" : ["VPC", "DefaultNetworkAcl"] } } }, "WebServerSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable HTTP ingress", "VpcId" : { "Ref" : "VPC" }, "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0" } ] } } }, "Outputs" : { "VPCId" : { "Description" : "VPC ID", "Value" : { "Ref" : "VPC" }, "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-VPCID" }} }, "PublicSubnet" : { "Description" : "The subnet ID to use for public web servers", "Value" : { "Ref" : "PublicSubnet" }, "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-SubnetID" }} }, "WebServerSecurityGroup" : { "Description" : "The security group ID to use for public web servers", "Value" : { "Fn::GetAtt" : ["WebServerSecurityGroup", "GroupId"] }, "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-SecurityGroupID" }} } } }
As commented below, the answer is no for now.
Original:Yes, it's the resource name that gets taken as the stack name. Below you'd get a stack called myStackName
YAML
AWSTemplateFormatVersion: "2010-09-09"
Resources:
myStackName:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: "https://s3.amazonaws.com/cloudformation-templates-us-east-1/EC2ChooseAMI.template"
Parameters:
InstanceType: "t1.micro"
KeyName: "mykey"
JSON
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Resources" : {
"myStackName" : {
"Type" : "AWS::CloudFormation::Stack",
"Properties" : {
"TemplateURL" : "https://s3.amazonaws.com/cloudformation-templates-us-east-1/EC2ChooseAMI.template",
"Parameters" : {
"InstanceType" : "t1.micro",
"KeyName" : "mykey"
}
}
}
}
}

HoztedZone error on cloudformation

I´m using cloudformation to create a ecs container and add this new container into route53 hostzone.
But when I run this script I´m having problems with the HostedZone tags
Here The error
A client error (ValidationError) occurred when calling the CreateStack operation: Invalid template parameter property 'Properties'
Here the json
"Parameters" : {
"InstanceType" : {
"Description" : "Container Instance type",
"Type" : "String",
"Default" : "t2.medium",
"AllowedValues" : [ "t2.micro", "t2.small", "t2.medium", "m3.medium", "m3.large", "m3.xlarge", "m3.2xlarge" ],
"ConstraintDescription" : "must be a valid EC2 instance type."
},
"HostedZone" : {
"Type": "AWS::Route53::HostedZone",
"Properties": {
"HostedZoneConfig": {
"Comment": "My hosted zone for example.com"
},
"Name": "***.couchbase.com",
"VPCs": [
{
"VPCId": "*********",
"VPCRegion": "eu-west-1"
}
],
"HostedZoneTags": [
{
"Key": "Name",
"Value": "Couchbase DNS"
}
]
}
}
},
"Resources" : {
"ContainerInstance" : {
"Type": "AWS::EC2::Instance",
"Properties": {
"Tags": [{
"Key" : "Name",
"Value" : "Couchbase-1"
},
{
"Key" : "Type",
"Value" : "ECS-Couchbase"
}],
"IamInstanceProfile" : { "Ref" : "ECSIamInstanceProfile" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
"InstanceType" : { "Ref" : "InstanceType" },
"SecurityGroups" : [ "ssh","default", "couchbase" ],
"KeyName" : { "Ref" : "KeyName" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -xe\n",
"echo ECS_CLUSTER=", { "Ref" : "ClusterName" },
" >> /etc/ecs/ecs.config\n"
]]}}
}
},
"CouchbaseDNSRecord" : {
"Type" : "AWS::Route53::RecordSet",
"Properties" : {
"HostedZoneName" : {
"Fn::Join" : [ "", [
{ "Ref" : "HostedZone" }, "."
] ]
},
"Comment" : "DNS name for my instance.",
"Name" : {
"Fn::Join" : [ "", [
{"Ref" : "ContainerInstance"}, ".",
{"Ref" : "AWS::Region"}, ".",
{"Ref" : "HostedZone"} ,"."
] ]
},
"Type" : "A",
"TTL" : "900",
"ResourceRecords" : [
{ "Fn::GetAtt" : [ "ContainerInstance", "PublicIp" ] }
]
}
},
The HostedZone should be inside the Resources section.
"Parameters" : {
"InstanceType" : {
...
}
},
"Resources" : {
"HostedZone" : {
...
},
"ContainerInstance" : {
...
},
...
}
All the resources you want to create using Cloudformation should be within the resources section. This gives a better anatomy of the template, http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html

How to add a RDS instance to a VPC using aws cloudformation

When I launch a RDS instance manually I'm able to assign what VPC I want it to be part of. I'm trying to create a stack using AWS cloudformation, however I do not see an API to be able to do that. I can create my VPC in the stack and then reference it for security groups both EC2 and DB security groups and they both end up been part of the VPC however the RDS instance itself does not. Is there a way to assign the VPC to the RDS instance?
Below is my template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"30e03bfc-b61a-4d6c-89db-1b62b258a305": {
"size": {
"width": 80,
"height": 80
},
"position": {
"x": 700,
"y": 170
},
"z": 0,
"embeds": []
}
}
},
"Parameters": {
"DBPreferredBkupWindow": {
"Description" : "The daily time range (in UTC) during which automated backups are created, ideally off peak-hours.",
"Type" : "String",
"MinLength" : "1",
"MaxLength" : "11",
"AllowedPattern" : "\\d[0-23]:\\d[0-59]-\\d[0-23]:\\d[0-59]",
"Default" : "01:00-02:00"
}
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock" : "172.16.0.0/16",
"EnableDnsSupport" : true
}
},
"DB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName" : "ems",
"Engine" : "postgres",
"EngineVersion" : "9.4.7",
"DBInstanceClass" : "db.t1.micro",
"DBInstanceIdentifier" : "rltdb",
"MasterUsername" : "pgadmin",
"MasterUserPassword" : "pgadmin1",
"AllocatedStorage" : "100",
"Iops" : "1000",
"BackupRetentionPeriod" : "7",
"PreferredBackupWindow" : { "Ref" : "DBPreferredBkupWindow" },
"MultiAZ" : true,
"PubliclyAccessible" : false,
"AutoMinorVersionUpgrade" : false,
"VPCSecurityGroups" : [{ "Ref" : "SecurityGroup" } ]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "30e03bfc-b61a-4d6c-89db-1b62b258a305"
}
}
},
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VPC" },
"DBSecurityGroupIngress" : { "EC2SecurityGroupName": { "Ref": "SecurityGroup"} },
"GroupDescription" : "Database Access"
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"GroupDescription" : "Enable database access for application",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "5432", "ToPort" : "5432", "CidrIp" : "0.0.0.0/0"}
]
}
}
}
}
You have to create a DBSubnetGroup and at least two subnets in your CloudFormation template.
"subnet-1" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.1.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"subnet-2" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.2.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"DBSubnetGroup" : {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"SubnetIds" : [
{ "Ref" : "subnet-1" },
{ "Ref" : "subnet-2" }
],
}
},
and in last you have to include DBSubnetGroup in your "DB" Object.
"DBSubnetGroupName": { "Ref": "DBSubnetGroup" }
You need to include the DBSubnetGroupName:
A DB subnet group to associate with the DB instance.
If there is no DB subnet group, then it is a non-VPC DB instance.
Create a DBSubnetGroup resource using subnets in your VPC, then tie that to your DBInstance:
"DBSubnetGroupName": { "Ref": "MySubnetGroup" }

AWS CloudFormation identify IP of another instance - circular dependency

I'm provisioning two instances with CloudFormation template. "MASTER" and "SLAVE".
At the userdata script I need to pass slave's private IP to master and master's IP to slave.
Here is my template:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "",
"Parameters" : {
},
"Resources" : {
"InstanceSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Default Ports",
"SecurityGroupIngress" : [
{ "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0"}
]
}
},
"MASTER" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"Tags":[{"Key":"Name", "Value":"MASTER"}],
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"KeyName" : "mykey",
"ImageId" : "ami-a25415cb",
"InstanceType": "m1.large",
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -ex", "\n",
"wget https://s3.amazonaws.com/mybucket/bootstrap.sh","\n",
"ROLE=MASTER SLAVEIP=",?????," sh bootstrap.sh","\n"
] ] } }
}
},
"SLAVE" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"Tags":[{"Key":"Name", "Value":"SLAVE"}],
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"KeyName" : "mykey",
"ImageId" : "ami-a25415cb",
"InstanceType": "m1.large",
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -ex", "\n",
"wget https://s3.amazonaws.com/mybucket/bootstrap.sh","\n",
"ROLE=SLAVE MASTERIP=",?????," sh bootstrap.sh","\n"
] ] } }
}
},
},
"Outputs" : {
}
}
What would be right replacement for ????? if it's possible at all and if not - what alternative can I use?
UPD: found this: {"Fn::GetAtt": ["MASTER","PrivateIp"]}, it works fine on it's own, but fails with "Template validation error: Circular dependency between resources: [SLAVE, MASTER]" if I'm trying to do both master and slave IPs.
If you're using VPCs and Subnets, you can do this by creating an AWS::EC2::NetworkInterface for each instance. Then use { "Fn::GetAtt": [ "MyNetworkInterface", "PrimaryPrivateIpAddress" ] } in user data to reference the internal IP address of the network interface
You associate a network interface with an EC2 instance using the NetworkInterfaces property
...
"MasterNetInt" : {
"Type" : "AWS::EC2::NetworkInterface",
"Properties" : {
"SubnetId": { "Ref" : "MySubnet" }
}
},
"SlaveNetInt" : {
"Type" : "AWS::EC2::NetworkInterface",
"Properties" : {
"SubnetId": { "Ref" : "MySubnet" }
}
},
"Master" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"KeyName" : "mykey",
"ImageId" : "ami-a25415cb",
"InstanceType": "m1.large",
"SubnetId": { "Ref" : "MySubnet" },
"NetworkInterfaces": [ { "Ref" : "MasterNetInt" } ],
"UserData": { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -ex", "\n",
"wget https://s3.amazonaws.com/mybucket/bootstrap.sh","\n",
"ROLE=MASTER SLAVEIP=", { "Fn::GetAtt": [ "SlaveNetInt", "PrimaryPrivateIpAddress" ] }," sh bootstrap.sh","\n"
] ] } }
}
},
"Slave" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"KeyName" : "mykey",
"ImageId" : "ami-a25415cb",
"InstanceType": "m1.large",
"SubnetId": { "Ref" : "MySubnet" },
"NetworkInterfaces": [ { "Ref" : "SlaveNetInt" } ],
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -ex", "\n",
"wget https://s3.amazonaws.com/mybucket/bootstrap.sh","\n",
"ROLE=SLAVE MASTERIP=", { "Fn::GetAtt": [ "MasterNetInt", "PrimaryPrivateIpAddress" ] }," sh bootstrap.sh","\n"
] ] } }
}
}
...
If you're unfamiliar with setting up a VPC and Subnets, read the following documentation: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario1.html
and reference these template examples: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/sample-templates-services-us-west-2.html#d0e113371
The basic requirements are:
AWS::EC2::VPC
AWS::EC2::InternetGateway
AWS::EC2::VPCGatewayAttachment
AWS::EC2::RouteTable
AWS::EC2::Route
AWS::EC2::Subnet
AWS::EC2::SubnetRouteTableAssociation
AWS::EC2::NetworkAcl
AWS::EC2::SubnetNetworkAclAssociation
AWS::EC2::NetworkAclEntry
AWS::EC2::NetworkInterface
AWS::EC2::Instance