How to automatically add an autoscaled EC2 instance to a Security Group? - amazon-web-services

I want to set up my AWS to autoscale EC2 instances (Windows Server 2012). The catch is that they need to have their IP addresses added to a Security Group so they can communicate with another EC2.
Is there a way that AWS can handle this automatically through its autoscaling feature? (The closest I could find was to assign an IAM role to the new instances, but I don't I can add an IAM role to a Security Group, I can only add IP addresses.)
The way I am currently looking into is to use the AWS CLI (command line) as a startup script.
ec2-authorize mySecurityGroup -p 1433 -s xx.xx.xx.xx/32
But how do I get the public IP of the current instance? Is there a AWS CLI command to get this? I'd rather not depend on an external website like "curl echoip.com". I heard about ec2-metadata, but I don't think that works for Windows, and I'd prefer not to use another third party software.

Create a security group called web. For the sake of an example, lets say the id of that group is: sg-7aa91911
Create a security group called db.
Add a new rule to the db security group for port 1433 with the source of sg-7aa91911
Create an Autoscaling launch configuration and set the SecurityGroups to sg-7aa91911 and any other configuration you need.
Create Autoscaling group with that launch configuration.
I wrote up a quick CloudFormation template to do this task. You should be able to just run it and it will create an Autoscaling group with the connected security groups. It'll also create a blank instance where you can store your db.
If you prefer not to use a CloudFormation template, just look at where the security groups are defined. It shows how the 2 security groups are to be connected
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "test tempalte",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-1" : { "AMI" : "ami-951945d0" },
"us-west-2" : { "AMI" : "ami-16fd7026" },
"eu-west-1" : { "AMI" : "ami-24506250" },
"sa-east-1" : { "AMI" : "ami-3e3be423" },
"ap-southeast-1" : { "AMI" : "ami-74dda626" },
"ap-northeast-1" : { "AMI" : "ami-dcfa4edd" }
}
},
"Resources" : {
"WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : { "Fn::GetAZs" : "" },
"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
"MinSize" : "1",
"MaxSize" : "10",
"DesiredCapacity" : "1"
}
},
"LaunchConfig" : {
"Type" : "AWS::AutoScaling::LaunchConfiguration",
"Properties" : {
"InstanceType" : "m1.small",
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ {"Ref" : "websg"} ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}
}
},
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
},
"websg" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable SSH and access, 8080, and 80",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "8080", "ToPort" : "8080", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0"},
{"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0"}
]
}
},
"dbsg" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Port opened only to security group",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "1433", "ToPort" : "1433", "SourceSecurityGroupName" : {"Ref" : "websg"}
}
]
}
}
}
}

Related

cloudformation failing with security group mismatch

I'm trying to setup my cloudformation for my database:
"VPC" : {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.0.0.0/16",
"EnableDnsSupport" : "false",
"EnableDnsHostnames" : "false",
"InstanceTenancy" : "default",
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Subnet" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"CidrBlock" : "10.0.0.0/16",
"AvailabilityZone" : { "Fn::Select": [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" } }]},
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Allow http to client host",
"VpcId" : {"Ref" : "VPC"},
"SecurityGroupIngress" : [{
"IpProtocol" : "tcp",
"FromPort" : "3306",
"ToPort" : "3306",
"CidrIp" : "10.0.0.0/16"
}],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
}
},
"Database" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"DBName" : { "Fn::Join": ["", { "Fn::Split": [".", { "Ref" : "DomainName" }]}]},
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.t2.micro",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : { "Ref": "DBUsername" },
"MasterUserPassword" : { "Ref": "DBPassword" },
"VPCSecurityGroups" : [ { "Fn::GetAtt": [ "SecurityGroup", "GroupId" ] } ],
"Tags" : [ { "Key" : "Name", "Value" : "DomainName" } ]
},
"DeletionPolicy" : "Snapshot"
},
Should be setting up a VPC for the database. But when I run the cloudformation template I get the following error:
UPDATE_FAILED AWS::RDS::DBInstance Database Database is in vpc-3081245b, but Ec2 Security Group sg-b122ffca is in vpc-f7173290
How do I get my database in the VPC properly?
As part of your Database definition, you can specify a DBSubnetGroupName.
A DB Subnet Group provides a list of subnets in which the Database is allowed to run. Each subnet in a DB Subnet Group belongs to a VPC.
Therefore, you need to do the following to your Amazon CloudFormation template:
Add a AWS::RDS::DBSubnetGroup, specifying the Subnet already defined in your template
Add a DBSubnetGroupName parameter to your AWS::RDS::DBInstance definition

How to add a RDS instance to a VPC using aws cloudformation

When I launch a RDS instance manually I'm able to assign what VPC I want it to be part of. I'm trying to create a stack using AWS cloudformation, however I do not see an API to be able to do that. I can create my VPC in the stack and then reference it for security groups both EC2 and DB security groups and they both end up been part of the VPC however the RDS instance itself does not. Is there a way to assign the VPC to the RDS instance?
Below is my template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Metadata": {
"AWS::CloudFormation::Designer": {
"30e03bfc-b61a-4d6c-89db-1b62b258a305": {
"size": {
"width": 80,
"height": 80
},
"position": {
"x": 700,
"y": 170
},
"z": 0,
"embeds": []
}
}
},
"Parameters": {
"DBPreferredBkupWindow": {
"Description" : "The daily time range (in UTC) during which automated backups are created, ideally off peak-hours.",
"Type" : "String",
"MinLength" : "1",
"MaxLength" : "11",
"AllowedPattern" : "\\d[0-23]:\\d[0-59]-\\d[0-23]:\\d[0-59]",
"Default" : "01:00-02:00"
}
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock" : "172.16.0.0/16",
"EnableDnsSupport" : true
}
},
"DB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName" : "ems",
"Engine" : "postgres",
"EngineVersion" : "9.4.7",
"DBInstanceClass" : "db.t1.micro",
"DBInstanceIdentifier" : "rltdb",
"MasterUsername" : "pgadmin",
"MasterUserPassword" : "pgadmin1",
"AllocatedStorage" : "100",
"Iops" : "1000",
"BackupRetentionPeriod" : "7",
"PreferredBackupWindow" : { "Ref" : "DBPreferredBkupWindow" },
"MultiAZ" : true,
"PubliclyAccessible" : false,
"AutoMinorVersionUpgrade" : false,
"VPCSecurityGroups" : [{ "Ref" : "SecurityGroup" } ]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "30e03bfc-b61a-4d6c-89db-1b62b258a305"
}
}
},
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VPC" },
"DBSecurityGroupIngress" : { "EC2SecurityGroupName": { "Ref": "SecurityGroup"} },
"GroupDescription" : "Database Access"
}
},
"SecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"VpcId" : { "Ref" : "VPC" },
"GroupDescription" : "Enable database access for application",
"SecurityGroupIngress" : [
{"IpProtocol" : "tcp", "FromPort" : "5432", "ToPort" : "5432", "CidrIp" : "0.0.0.0/0"}
]
}
}
}
}
You have to create a DBSubnetGroup and at least two subnets in your CloudFormation template.
"subnet-1" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.1.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"subnet-2" : {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"CidrBlock" : "172.16.2.0/24",
"VpcId" : { "Ref" : "VPC" }
}
},
"DBSubnetGroup" : {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"SubnetIds" : [
{ "Ref" : "subnet-1" },
{ "Ref" : "subnet-2" }
],
}
},
and in last you have to include DBSubnetGroup in your "DB" Object.
"DBSubnetGroupName": { "Ref": "DBSubnetGroup" }
You need to include the DBSubnetGroupName:
A DB subnet group to associate with the DB instance.
If there is no DB subnet group, then it is a non-VPC DB instance.
Create a DBSubnetGroup resource using subnets in your VPC, then tie that to your DBInstance:
"DBSubnetGroupName": { "Ref": "MySubnetGroup" }

Can I use public security groups in cloud formation for ec2 launch configs?

Can I use public security groups in cloud formation for ec2 launch configs ?
I am trying to simply direct the cloudformation stack to use an existing set and it fails to create those resources.
I tried to create those security groups as a rolling set (ssh1,ssh2...) for each stack but no such luck either.
It would be really cool if your answer would include a code snippet as to how to do this exactly. thanks.
You can refer the sample template for deep understanding.
Amazon EC2 instance in a security group
Here is the part you are interesting.
"Resources" : {
"EC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"InstanceType" : { "Ref" : "InstanceType" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], # <--Here is the refer
"KeyName" : { "Ref" : "KeyName" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] }
}
},
"InstanceSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable SSH access via port 22",
"SecurityGroupIngress" : [ {
"IpProtocol" : "tcp",
"FromPort" : "22",
"ToPort" : "22",
"CidrIp" : { "Ref" : "SSHLocation"}
} ]
}
}
},
Above sample to create a new security group, and assign it to a new ec2 instance. If you have exist security group, then you needn't the part InstanceSecurityGroup, and assign the real security group name to ec2 instance at:
"SecurityGroups" : "REAL Security Group Name",

How do I connect my AWS::EC2::DBSecurityGroup to my AWS::RDS::DBSecurityGroup in a VPC context?

I have this AWS::EC2::SecurityGroup:
"InstanceSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable HTTP access on the configured port",
"VpcId" : { "Ref" : "VpcId" },
"SecurityGroupIngress" : [ {
"IpProtocol" : "tcp",
"FromPort" : { "Ref" : "WebServerPort" },
"ToPort" : { "Ref" : "WebServerPort" },
"SourceSecurityGroupId" : { "Ref" : "LoadBalancerSecurityGroup" }
} ]
}
}
and I have this AWS::RDS::DBSecurityGroup
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"DBSecurityGroupIngress": { "EC2SecurityGroupName": { "Ref": "InstanceSecurityGroup"} },
"GroupDescription" : "Frontend Access"
}
}
when I try to bring up this stack, I get:
Invalid security group , groupId=, groupName= sg-a381fdc6.
Edit 1: Reading a bit more suggests I need AWS::RDS::DBSecurityGroup to be associated with my VPC, so I change to this:
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VpcId" },
"DBSecurityGroupIngress": { "EC2SecurityGroupName": { "Ref": "InstanceSecurityGroup"} },
"GroupDescription" : "Frontend Access"
}
}
and when I bring up the stack I get
Please see the documentation for authorizing DBSecurityGroup ingress. For VPC, EC2SecurityGroupId is required. To authorize only the source address of this request (and no other address), pass 205.251.233.35/32 as the CIDRIP parameter.
EC2SecurityGroupId is the ID of the security group, not the name of it, and that ID is assigned outside my control, so I don't know what value to put in here.
How do I connect my AWS::EC2::DBSecurityGroup to my AWS::RDS::DBSecurityGroup in a VPC context?
The problem is that your { "Ref": "InstanceSecurityGroup"} doesn't hold the id only the name. To get a hold on the EC2SecurityGroupId use Fn::GetAtt.
Your template for the DBSecurityGroup should look something like this (notice how Ref have been replaced by Fn::GetAtt:
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VpcId" },
"DBSecurityGroupIngress": { "EC2SecurityGroupId": { "Fn::GetAtt" : [ "InstanceSecurityGroup", "GroupId" ] } },
"GroupDescription" : "Frontend Access"
}
When you RDS Security group is defined inside a VPC, you must refer to other security group by group-id, not by group name.
See
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-security-group.html
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-security-group-rule.html
"For VPC DB Security Groups, use EC2SecurityGroupId. For EC2 Security Groups, use EC2SecurityGroupOwnerId and either EC2SecurityGroupName or EC2SecurityGroupId."
You can get the Security group ID by using the "Ref" function as described here
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-security-group.html
So, your modified Security Group should be
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VpcId" },
"DBSecurityGroupIngress": { "EC2SecurityGroupId": { "Ref": "InstanceSecurityGroup"} },
"GroupDescription" : "Frontend Access"
}
}

how to construct a string of physical subnet ids to create db subnet group on the fly in a cloudformation script?

I'm trying to build a CLoudFormation script that launches an instance and a db into a vpc at the same time. the issue is the db requires two AZ's so i create a second subnet and now i just need to reference the two subnet physical ids in a 'MyDBSubnetGroup' var. I can get the logical IDs for the subnets i created but dont know how to ref those physical IDs. ANyone know? THanks!!
Heres my code:
"MyDBSubnetGroup" : {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"DBSubnetGroupDescription" : "Subnets available for the RDS DB Instance",
"SubnetIds" : { "Fn::Join" : [ " ", [{"Ref" : "PublicSubnetAZ1"}, ", ", {"Ref" : "PublicSubnetAZ2"}, " " ]]}
}
},
I run into the same issue, after working with AWS support I understood that List of String does not mean what we initially thought. Also, if you want to place the DB inside a VPC you must not use AWS::RDS::DBSecurityGroup objects.
Here is a full sample, it took me a while to get it working:
"dbSubnetGroup" : {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"DBSubnetGroupDescription" : "Availability Zones for RDS DB",
"SubnetIds" : [ { "Ref" : "subnetPrivate1" },
{ "Ref" : "subnetPrivate2" } ]
}
},
"dbInstance" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"DBInstanceIdentifier" : { "Fn::Join" : [ "",
[ { "Ref" : "AWS::StackName" },
"DB" ] ] },
"DBName" : "dbname",
"DBSubnetGroupName" : { "Ref" : "dbSubnetGroup" },
"MultiAZ" : "true",
"AllocatedStorage" : "8",
"BackupRetentionPeriod" : "0",
"DBInstanceClass" : "db.m1.medium",
"Engine" : "postgres",
"MasterUserPassword" : "masteruserpassword",
"MasterUsername" : "masterusername",
"VPCSecurityGroups" : [ { "Ref" : "sgVpc" }, { "Ref" : "sgDB" } ]
}
},
If you map the subnet ids you can access them with something like this.
"AWSRegionSubnet":{
"us-east-1":{
"RDSSubnets":[
"subnet-aaaaaaaa",
"subnet-bbbbbbbb"
]
},
"us-west-2":{
"RDSSubnets":[
"subnet-cccccccc",
"subnet-dddddddd"
]
}
}
"RDSSubnet":{
"Type":"AWS::RDS::DBSubnetGroup",
"Properties":{
"DBSubnetGroupDescription":"Some cool notes here",
"SubnetIds":{
"Fn::FindInMap":[
"AWSRegionSubnet",
{
"Ref":"AWS::Region"
},
"RDSSubnets"
]
}
}
}