Cloudformation - Using parameters in EC2 server - amazon-web-services

Below is my cloudformation template. Basically I want to use a parameter called "User" in a command in EC2 ubuntu server.
I have tried many solutions, but the mkdir command is not fails, rest of the template is working.
In the below example I have tried to set the "User" value as a tag as well as a property. Both failed. Suggest if you have seen any such examples.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"UserName": {
"Type": "String",
"Description": "Enter the username",
"AllowedPattern": "[a-zA-Z0-9_\\-]+",
"ConstraintDescription": "It should be a combination of characters from [A-Z],[a-z],[0-9],- and _."
},
"KeyName": {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription" : "must be the name of an existing EC2 KeyPair."
}
},
"Resources": {
"EC2Ansiblemaster2743": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"User" : { "Ref": "UserName" },
"AWS::CloudFormation::Init" : {
"configSets" : {
"CreateFile" : [
"create_file"
]
},
"create_file" : {
"files" : {
"/home/ubuntu/healthy.txt": {
"content": "Hello"
}
},
"commands" : {
"mkdir" : {
"command" : "sudo mkdir /home/$User"
}
}
}}
},
"Properties": {
"AvailabilityZone": "us-east-2a",
"ImageId": "ami-05c1fa8df71875112",
"InstanceType": "t2.micro",
"SourceDestCheck": false,
"KeyName" : { "Ref" : "KeyName" },
"Tags" : [
{
"Key" : "User",
"Value" : { "Ref": "UserName" }
}],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xe\n",
"sudo apt-get update\n",
"sudo apt-get -y install python-pip\n",
"sudo pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
"# Install the files and packages from the metadata\n",
"sudo /usr/local/bin/cfn-init ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource EC2Ansiblemaster2743 ",
" --configsets CreateFile ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n",
"# Signal the status from cfn-init\n",
"sudo /usr/local/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource EC2Ansiblemaster2743 ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
},
"SecurityGroupIds": [
{
"Ref": "EC2SG2AnsM0012743"
}
]
}
},
"EC2SG2AnsM0012743": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupName": "SGAnsibleMas2743",
"GroupDescription": "First",
"SecurityGroupIngress": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "0",
"ToPort": "65535",
"IpProtocol": "tcp",
"Description": "From Anywhere"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "22",
"ToPort": "22",
"IpProtocol": "tcp",
"Description": "SSH from anywhere , from and to port will be same - 22"
}
]
},
}
}
}

I used the Fn:Sub to get the parameter to Userdata.
{
"Fn::Sub":
[
"UserName=${variable}\n",
{
"variable": {
"Ref": "UserName"
}
}]
Later I curl 169.254.169.254/latest/user-data to extract the Username string.
Please let me know if any easier solution is there for this issue.

Use the join intrinsic function to properly form the command string by referencing the username in the template.
"command" : "Fn::Join": [ "", "mkdir /home/", { "Ref": "UserName" }]
The issue is that your mkdir command does not resolve as you have it. There is no linkage between the username parameter and the command in the init section. By using the Join function I am forming the command with the correct parameter value.

Related

Why am I getting an error that there are unresolved VPC dependencies?

I'm trying to implement configuration management that meets the following criteria for an assighment:
https://s3.amazonaws.com/seis615/AnsiblePress.json
Take a quick look at the template in a text editor. Notice how the UserData property for the mgmt1 instance is configured. When CloudFormation launches this stack, it will automatically install and configure Ansible software on the management server. It’s very common to use a small amount of scripting code to bootstrap configuration management software onto a new system. Once Ansible is installed it can be used to install and configure other servers in the environment.
The CloudFormation template is missing a couple resources that you will need to add:
An application load balancer with a logical name of webserverlb which distributes HTTP (port 80) requests to the web1 and web2 instances. The health check endpoint for the load balancer should be the root (/) directory.
A db.t2.micro RDS database instance (not a cluster) running a MariaDB 10.2.21 database called wordpress located in a private VPC subnet. Use the logical name wordpressdb for the CloudFormation RDS resource. RDS and EC2 instances actually pre-date the arrival of VPCs in AWS so confusingly there are two different ways to configure these resources. You need to make sure this database instance is designed to run inside a VPC with the proper database subnet group and security group resources defined.
A security group called WebserverLbSecurityGroup which allows incoming http access from the Internet.
A security group called WordpressDbSecurityGroup which allows incoming access on the standard MySQL port from the WebServerSecurityGroup
An input parameter called DBName which will define the database name to create (default to wordpress)
An input parameter called DBUser which will be used for the database server username.
An input parameter called DBPassword which will be used for the database server password.
A stack output called wordpressDbEndpoint which shows the MariaDB instance endpoint address.
A stack output called wordpressLbEndpoint which shows the application load balancer URL.
The JSON I've configured (below) gives me the following template format error and I don't know why:
Template format error: Unresolved resource dependencies [wordpressVPC] in the Resources block of the template
{"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"SSMAccessRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version" : "2012-10-17",
"Statement": [ {
"Effect": "Allow",
"Principal": {
"Service": [ "ec2.amazonaws.com" ]
},
"Action": [ "sts:AssumeRole" ]
} ]
},
"Path": "/"
}
},
"SSMRolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "ssmProperties",
"PolicyDocument": {
"Version" : "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:DescribeParameters",
"ssm:PutParameter",
"ssm:GetParameters",
"ssm:DeleteParameter"
],
"Resource": {
"Fn::Join" : [
"",
[
"arn:aws:ssm:",
{ "Ref" : "AWS::Region" },
":",
{ "Ref" : "AWS::AccountId"},
{
"Fn::Join" : [
"",
[ ":parameter/", { "Ref": "AWS::StackName" }, ".*" ]
]
}
]
]
}
}
]
},
"Roles": [ { "Ref": "SSMAccessRole" } ]
}
},
"SSMInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ { "Ref": "SSMAccessRole" } ]
}
},
"web1pem" : {
"Type" : "AWS::SSM::Parameter",
"Properties" : {
"Name" : {
"Fn::Join" : [
"",
[ { "Ref": "AWS::StackName" }, ".web1pem" ]
]
},
"Type" : "String",
"Value" : "0",
"Description": "web1 instance private key."
}
},
"web2pem" : {
"Type" : "AWS::SSM::Parameter",
"Properties" : {
"Name" : {
"Fn::Join" : [
"",
[ { "Ref": "AWS::StackName" }, ".web2pem" ]
]
},
"Type" : "String",
"Value" : "0",
"Description": "web2 instance private key."
}
},
"wordpressVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true",
"CidrBlock": "10.0.0.0/16",
"Tags": [
{
"Key": "Environment",
"Value": "Test"
}
]
}
},
"publicSubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"publicSubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "1", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"privateSubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.2.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"privateSubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.3.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "1", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"web1": {
"Type": "AWS::EC2::Instance",
"DependsOn": [
"web1pem"
],
"Properties": {
"InstanceType": "t2.micro",
"ImageId": {"Ref": "AMI"},
"IamInstanceProfile": {
"Ref": "SSMInstanceProfile"
},
"KeyName": {
"Ref": "KeyName"
},
"NetworkInterfaces": [
{
"GroupSet": [
{
"Ref": "WebServerSecurityGroup"
}
],
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"DeleteOnTermination": "true",
"SubnetId": {
"Ref": "publicSubnet1"
}
}
],
"Tags": [
{
"Key": "Name",
"Value": "web1"
}
],
"UserData" : {
"Fn::Base64" : {
"Fn::Join" : [
"", [
"#!/bin/bash -xe\n",
"ssh-keygen -f /home/ec2-user/.ssh/web1-key.pem -q -N \"\"\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web1-key.pem\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web1-key.pem.pub\n",
"PEMFILE=`cat /home/ec2-user/.ssh/web1-key.pem`\n",
"aws ssm put-parameter --name ", { "Ref" : "web1pem" }, " --type String --value \"${PEMFILE}\" --overwrite --region ", { "Ref" : "AWS::Region" },"\n",
"cat /home/ec2-user/.ssh/web1-key.pem.pub >> /home/ec2-user/.ssh/authorized_keys\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource web1 ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT5M"
}
}
},
"web2": {
"Type": "AWS::EC2::Instance",
"DependsOn": [
"web1pem"
],
"Properties": {
"InstanceType": "t2.micro",
"ImageId": {"Ref": "AMI"},
"IamInstanceProfile": {
"Ref": "SSMInstanceProfile"
},
"KeyName": {
"Ref": "KeyName"
},
"NetworkInterfaces": [
{
"GroupSet": [
{
"Ref": "WebServerSecurityGroup"
}
],
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"DeleteOnTermination": "true",
"SubnetId": {
"Ref": "publicSubnet2"
}
}
],
"Tags": [
{
"Key": "Name",
"Value": "web2"
}
],
"UserData" : {
"Fn::Base64" : {
"Fn::Join" : [
"", [
"#!/bin/bash -xe\n",
"ssh-keygen -f /home/ec2-user/.ssh/web2-key.pem -q -N \"\"\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web2-key.pem\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web2-key.pem.pub\n",
"PEMFILE=`cat /home/ec2-user/.ssh/web2-key.pem`\n",
"aws ssm put-parameter --name ", { "Ref" : "web2pem" }, " --type String --value \"${PEMFILE}\" --overwrite --region ", { "Ref" : "AWS::Region" },"\n",
"cat /home/ec2-user/.ssh/web2-key.pem.pub >> /home/ec2-user/.ssh/authorized_keys\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource web2 ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT5M"
}
}
},
"WebServerSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"GroupDescription": "Allow access from HTTP and SSH traffic",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": {"Ref": "YourIp"}
}
]
}
},
"WebServerSGIngressTCP22": {
"Type": "AWS::EC2::SecurityGroupIngress",
"Metadata": {
"Comment": "SSH ingress security rule"
},
"Properties" : {
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"SourceSecurityGroupId": { "Ref": "WebServerSecurityGroup" },
"GroupId": { "Fn::GetAtt": ["WebServerSecurityGroup", "GroupId"]}
}
},
"InternetGateway": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"InternetGatewayId": {
"Ref": "InternetGateway"
},
"VpcId": {
"Ref": "wordpressVpc"
}
}
},
"PublicRouteTable": {
"Type": "AWS::EC2::RouteTable",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
}
}
},
"PublicRoute": {
"Type": "AWS::EC2::Route",
"Properties": {
"DestinationCidrBlock": "0.0.0.0/0",
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"GatewayId": {
"Ref": "InternetGateway"
}
},
"DependsOn": [
"InternetGateway", "AttachGateway"
]
},
"Public1RouteTableAssociation": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"SubnetId": {
"Ref": "publicSubnet1"
}
}
},
"Public2RouteTableAssociation": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"SubnetId": {
"Ref": "publicSubnet2"
}
}
},
"webserverlb": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"IpAddressType": "ipv4",
"SecurityGroups": [
{
"Ref": "webserverlbSecurityGroup"
}
],
"Subnets": [
{
"Ref": "publicSubnet1"
},
{
"Ref": "publicSubnet2"
}
],
"Tags": [
{
"Key": "Name",
"Value": "webserverlb"
}
]
},
"DependsOn": [
"webserversSecurityGroup"
]
},
"webserverlbSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"GroupDescription": "Allows incoming requests from port 80 via HTTP.",
"SecurityGroupIngress": [
{
"IpProtocol": "TCP",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0",
"Description": "Allows 80 from Internet"
}
]
}
},
"wordpressdb": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"AvailabilityZone": "us-east-1a",
"DBInstanceClass": "db.t2.micro",
"DBName": "wordpress",
"Engine": "mariadb",
"EngineVersion": "10.2.21",
"MultiAZ": 1,
"Tags": [
{
"Key": "Name",
"Value": "wordpressdb"
}
]
},
"DependsOn": [
"wordpressdbSecurityGroup"
]
},
"wordpressdbSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"GroupDescription": "Enable access to the db via port 3306.",
"Tags": [
{
"Key": "Name",
"Value": "wordpressdbSecurityGroup"
}
],
"SecurityGroupIngress": [
{
"IpProtocol": "TCP",
"FromPort": "3306",
"ToPort": "3306",
"Description": "Enable HTTP access."
}
]
}
}
},
"Parameters": {
"KeyName": {
"Description": "Name of your EC2 KeyPair to enable SSH access to the instances.",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription": "must be the name of an existing EC2 KeyPair."
},
"YourIp": {
"Description": "The current CIDR IP address of your workstation (x.x.x.x/32). http://checkip.amazonaws.com/",
"Type": "String",
"AllowedPattern": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\\/(1[6-9]|2[0-9]|3[0-2]))$",
"ConstraintDescription": "Must be a valid IP CIDR range of the form x.x.x.x/x."
},
"AMI": {
"Description": "The EC2 instance AMI",
"Type": "String",
"Default": "ami-00dc79254d0461090"
},
"DBName": {
"Description": "Name of the database",
"Type" : "String",
"Default": "wordpress"
},
"DBUser": {
"Default": "admin",
"NoEcho": "false",
"Description" : "The WordPress database admin account user name",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*"
},
"DBPassword": {
"NoEcho": "true",
"Description" : "The password of the database.",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*"
}
},
"Outputs": {
"web1PublicIp": {
"Value": {"Fn::GetAtt": ["web1","PublicIp"]},
"Description": "web1 public IP"
},
"we2PublicIp": {
"Value": {"Fn::GetAtt": ["web2","PublicIp"]},
"Description": "web2 public IP"
},
"mgmt1PublicIp": {
"Value": {"Fn::GetAtt": ["mgmt1","PublicIp"]},
"Description": "mgmt1 public IP"
}
}
}
Because CloudFormation is case sensitive. Your vpc resource is called wordpressVpc, but in some places you are using wordpressVPC.
Recommend trying the CloudFormation Linter in VSCode to see some of these errors inline while authoring templates along with autocompletion and documentation links:
E3005 DependsOn should reference other resources at Resources/webserverlb/DependsOn/0
E1012 Ref wordpressVPC not found as a resource or parameter
E1012 Ref wordpressVPC not found as a resource or parameter
E3002 Invalid Property Resources/wordpressdb/Properties/VpcId
E3003 Property DBSecurityGroupIngress missing at Resources/wordpressdbSecurityGroup/Properties
E1012 Ref wordpressVPC not found as a resource or parameter
E3002 Invalid Property Resources/wordpressdbSecurityGroup/Properties/VpcId
E3002 Invalid Property Resources/wordpressdbSecurityGroup/Properties/SecurityGroupIngress
E1010 Invalid GetAtt mgmt1.PublicIp for resource mgmt1PublicIp

ubuntu 16.04 cloud formation template

I am trying to setup a cloud formation template to deploy ubuntu 16.04. The template deploys fine and I can SSH to the server., but the extra packages I defined, like apache and php modules will not install. I can't seem to figure out why that is? I looked at several code sniplets online, but can't seem to get it to work properly. Can some one tell me what I am doing wrong here?
The Code:
{
"Description" : "Staging single instance",
"Outputs": {
"InstanceID": {
"Description": "The WWW instance id",
"Value": { "Ref": "StageInstance" }
}
},
"Parameters": {
"AMI": {
"Description": "The Amazon Ubuntu AMI",
"Type": "String",
"Default": "ami-996372fd"
},
"EBSVolumeSize": {
"Description": "The size of the EBS volume for the transcoder",
"Type": "String",
"Default": "20"
},
"InstanceType": {
"AllowedValues": [
"t2.micro",
"t2.small",
"t2.medium",
"t2.large",
"c4.large",
"c4.xlarge",
"c4.2xlarge",
"c4.4xlarge",
"c4.8xlarge"
],
"ConstraintDescription": "must be a valid EC2 instance type",
"Default": "t2.micro",
"Description": "EC2 instance type",
"Type": "String"
},
"KeyName": {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to NAT instances.",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription" : "Must be the name of an existing EC2 KeyPair." },
"Subnet": {
"Description": "The subnet to place the instance in...",
"Type": "AWS::EC2::Subnet::Id",
"ConstraintDescription" : "Must be a valid Subnet."
},
"VPC": {
"Description": "The VPC to deploy to...",
"Type": "AWS::EC2::VPC::Id",
"ConstraintDescription" : "Must be a valid VPC."
}
},
"Resources": {
"StageSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"VpcId": {"Ref": "VPC"},
"GroupDescription": "Allow SSH, HTTP, and HTTPS access",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "443",
"ToPort": "443",
"CidrIp": "0.0.0.0/0"
}
]
}
},
"StageInstance": {
"Type" : "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"apt": {
"awscli": [ ],
"apache2": [ ],
"php7.0": [ ],
"libapache2-mod-php7.0": [ ],
"php7.0-cli": [ ],
"php7.0-common": [ ],
"php7.0-mbstring": [ ],
"php7.0-gd": [ ],
"php7.0-intl": [ ],
"php7.0-xml": [ ],
"php7.0-mysql": [ ],
"php7.0-mcrypt": [ ],
"php7.0-zip": [ ]
}
}
}
}
},
"Properties": {
"BlockDeviceMappings" : [{
"DeviceName" : "/dev/sda1",
"Ebs" : {"VolumeSize" : {"Ref" : "EBSVolumeSize"}}
}],
"SecurityGroupIds": [{"Ref": "StageSecurityGroup"}],
"KeyName": {"Ref": "KeyName" },
"ImageId": {"Ref": "AMI"},
"SubnetId": {"Ref": "Subnet"},
"InstanceType": {"Ref": "InstanceType"},
"Tags": [
{"Key" : "Name", "Value" : "Ldn_Svr_Stage_Web"}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -x\n",
"apt-get update\n",
"apt-get install -y python-pip\n",
"pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
"cfn-init -v –resource StageInstance",
" –stack ",
{
"Ref": "AWS::StackName"
},
" –region ",
{
"Ref": "AWS::Region"
},
"\n",
"cfn-signal -e $? –resource StageInstance",
" –stack ",
{
"Ref": "AWS::StackName"
},
" –region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
}
}
}
}
Here is the working code for a simple example....
{
"Description": "stack descripton.....",
"Parameters": {
"AMI": {
"Default": "ami-996372fd",
"Description": "The Amazon Ubuntu AMI",
"Type": "String"
},
"EBSVolumeSize": {
"Default": "20",
"Description": "The size of the EBS volume",
"Type": "String"
},
"InstanceName": {
"Default": "My instance name",
"Description": "EC2 Instance Name",
"Type": "String"
},
"InstanceType": {
"AllowedValues": [
"t2.micro",
"t2.small",
"t2.medium",
"t2.large",
"c4.large",
"c4.xlarge",
"c4.2xlarge",
"c4.4xlarge",
"c4.8xlarge"
],
"ConstraintDescription": "must be a valid EC2 instance type",
"Default": "t2.micro",
"Description": "EC2 instance type",
"Type": "String"
},
"KeyName": {
"ConstraintDescription": "Must be the name of an existing EC2 KeyPair.",
"Description": "Name of an existing EC2 KeyPair to enable SSH access to NAT instances.",
"Type": "AWS::EC2::KeyPair::KeyName"
},
"Subnet": {
"ConstraintDescription": "Must be a valid Subnet.",
"Description": "The subnet to place the instance in...",
"Type": "AWS::EC2::Subnet::Id"
},
"VPC": {
"ConstraintDescription": "Must be a valid VPC.",
"Description": "The VPC to deploy to...",
"Type": "AWS::EC2::VPC::Id"
}
},
"Resources": {
"Instance": {
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"apt": {
"awscli": [],
"ruby": [],
"unzip": []
}
},
"files": {
"/tmp/sample.conf": {
"content": {
"Fn::Join": [
"",
[
"write a sample file to OS and give permissions.... \n"
]
]
},
"mode": "000644",
"owner": "root",
"group": "root"
}
},
"commands": {
"01update_pkgs": {
"command": "apt-get update && apt-get upgrade -y"
},
"02next_command": {
"command": "echo the numbering here has to be in order... 01, 02, 03 and so on... last item is not comma'd"
}
}
}
}
},
"Properties": {
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"VolumeSize": {
"Ref": "EBSVolumeSize"
}
}
}
],
"ImageId": {
"Ref": "AMI"
},
"InstanceType": {
"Ref": "InstanceType"
},
"KeyName": {
"Ref": "KeyName"
},
"SecurityGroupIds": [
{
"Ref": "SecurityGroup"
}
],
"SubnetId": {
"Ref": "Subnet"
},
"Tags": [
{
"Key": "Name",
"Value": {
"Ref": "InstanceName"
}
}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xe\n",
"apt-get install -y python-setuptools\n",
"mkdir -p /opt/aws/bin\n",
"wget https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
"easy_install --script-dir /opt/aws/bin aws-cfn-bootstrap-latest.tar.gz\n",
" apt-get update\n",
"/opt/aws/bin/cfn-init --region ",
{
"Ref": "AWS::Region"
},
" -s ",
{
"Ref": "AWS::StackName"
},
" -r Instance\n"
]
]
}
}
},
"Type": "AWS::EC2::Instance"
},
"EIPAddress": {
"Type": "AWS::EC2::EIP"
},
"IPAssoc": {
"Type": "AWS::EC2::EIPAssociation",
"Properties": {
"InstanceId": {
"Ref": "Instance"
},
"EIP": {
"Ref": "EIPAddress"
}
}
},
"SecurityGroup": {
"Properties": {
"GroupDescription": "Allow SSH, HTTP, and HTTPS access",
"SecurityGroupIngress": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "22",
"IpProtocol": "tcp",
"ToPort": "22"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"IpProtocol": "tcp",
"ToPort": "443"
}
],
"VpcId": {
"Ref": "VPC"
}
},
"Type": "AWS::EC2::SecurityGroup"
}
}
}
Create a apache.json file :
{
"Resources" :
{
"WebServer":{
"Type": "AWS::EC2::Instance",
"Properties":{
"ImageId" : "ami-9a7724c8",
"KeyName" : "demo",
"InstanceType":"t1.micro",
"SecurityGroups" : [ "demo" ],
"UserData" : {"Fn::Base64" : {"Fn::Join" : ["", [
"#!/bin/bash\n",
"# Launching Instance",
"apt-get -y install apache2 \n",
"apt-get -y install php php5-mysql \n"
] ] } }
}
}
},
"Outputs":{
"WebsiteURL" : {
"Value" : { "Fn::Join" : ["", ["http://", { "Fn::GetAtt" : [
"WebServer", "PublicDnsName" ]}]] },
"Description" : "URL for newly created apache server "
}
}
}
Then create a cloud formation stack using this json file.
Kindly validate the json file before saving it.
Hope it will help !

cfn-signal in aws cloudformation with launchConfiguration and AutoScalingGroup

I have the following configuration in my AWS Cloudformation template.
The template creates one EC2 instance based on instance1. I am using the reference to same instance in my LaunchConfiguration to create the instances of same type.
The problem I am facing is by including the CreationPolicy element in my AutoScalingGroup template. I get the following error when ASG launches an instance and waits for the cfn-signal.
+ /opt/aws/bin/cfn-signal -e 0 --stack ss07 --resource Instance1 --region us-west-2
ValidationError: Resource Instance1 is in CREATE_COMPLETE state and cannot be signaled
It seems like somehow the reference is made to an already existing instance1 and not to the instance being created by LaunchConfig. I saw examples which had the LaunchConfig embedded within, but I want to keep the instance details at one place instead of two places.
"instance1": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"configSets": {
"install": ["yum_packages","install_cfn"]
},
"yum_packages": {
"packages" : {
"yum" : {
"awslogs" : [],
"ruby" : [],
"wget" : [],
"httpd" : []
}
}
},
"install_cfn": {
"files": {
"/etc/cfn/cfn-hup.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\n",
"stack=",
{
"Ref": "AWS::StackId"
},
"\n",
"region=",
{
"Ref": "AWS::Region"
},
"\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
},
"/etc/cfn/hooks.d/cfn-auto-reloader.conf": {
"content": {
"Fn::Join": [
"",
[
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.WebServer.Metadata.AWS::CloudFormation::Init\n",
"action=/opt/aws/bin/cfn-init -v ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource splitsweetInstance ",
" --configsets install ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
}
},
"services": {
"sysvinit": {
"cfn-hup": { "enabled": "true", "ensureRunning": "true", "files": [
"/etc/cfn/cfn-hup.conf",
"/etc/cfn/hooks.d/cfn-auto-reloader.conf"
]
}
}
}
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT10M"
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": [
"AWSRegionArch2AMI", {
"Ref": "AWS::Region"
}, {
"Fn::FindInMap": [
"AWSInstanceType2Arch", {
"Ref": "instanceType1"
},
"Arch"
]
}
]
},
"InstanceType": {"Ref": "instanceType1"},
"KeyName": {"Ref": "KeyName"},
"Monitoring": "false",
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -xe\n",
"yum install -y aws-cfn-bootstrap\n",
"# Install the files and packages from the metadata\n",
"/opt/aws/bin/cfn-init -v ",
" --stack ", { "Ref" : "AWS::StackName" },
" --resource splitsweetInstance ",
" --configsets install ",
" --region ", { "Ref" : "AWS::Region" }, "\n",
"wget ", { "Fn::FindInMap": [ "Region2CodeDeployAgent", { "Ref": "AWS::Region"}, "url"] }, "\n",
"chmod +x ./install\n",
"./install auto\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ", { "Ref" : "AWS::StackName" },
" --resource splitsweetInstance ",
" --region ", { "Ref" : "AWS::Region" }, "\n"
]]}},
"Tags": [
{
"Key": "Name",
"Value": "inst1"
}
],
"SecurityGroupIds": [
{ "Fn::GetAtt" : [ "instance1Sg", "GroupId" ] }
]
}
}
My Launch Config is as follows -
"LaunchConfig1": {
"Type" : "AWS::AutoScaling::LaunchConfiguration",
"Properties" : {
"ImageId": {
"Fn::FindInMap": [
"AWSRegionArch2AMI", {
"Ref": "AWS::Region"
}, {
"Fn::FindInMap": [
"AWSInstanceType2Arch", {
"Ref": "instanceType1"
},
"Arch"
]
}
]
},
"InstanceId" : { "Ref":"instance1"},
"InstanceMonitoring" : "false",
"InstanceType" : { "Ref": "instanceType1"},
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Fn::GetAtt" : [ "instance1Sg", "GroupId" ] } ]
}
}
This is my AutoScalingGroup template -
"AutoScalingGroup1": {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : { "Fn::GetAZs": { "Ref": "AWS::Region" } },
"Cooldown" : "60",
....
"LaunchConfigurationName" : {"Ref":"LaunchConfig1"},
"MaxSize" : "3",
"MinSize" : "1",
"TargetGroupARNs" : [ {"Ref":"TargetGroup1"} ],
"VPCZoneIdentifier" : [ { "Ref": "subnetCache1" }, { "Ref": "subnetCache2" }, { "Ref": "subnetCache3" } ]
},
"CreationPolicy" : {
"ResourceSignal" : {
"Timeout" : "PT6M",
"Count" : "1"
}
}
}
The problem is that by specifying the InstanceId property in your LaunchConfiguration resource, it is reusing the same UserData that was used to launch the initial EC2 instance, including the hard-coded reference to the Logical Resource signaled by the cfn-signal command. According to the documentation,
When you use an instance to create a launch configuration, all properties are derived from the instance with the exception of BlockDeviceMapping and AssociatePublicIpAddress. You can override any properties from the instance by specifying them in the launch configuration.
To have cfn-signal signal the correct Resource, you will need to override the UserData in your LaunchConfiguration resource to contain a User-Data script that references the Launch Configuration rather than the original EC2 instance. Unfortunately, this will require either duplicating the User-Data script, or rewriting the script to dynamically figure out the Logical Resource associated with the instance the script is currently running on, so the same exact User-Data can be used in both the original EC2 instance and the auto scaling group.
I agree with wjordan that part of the problem you are having is with the InstanceId property in LaunchConfiguration. Also looking at your code sample I could not figure out where splitsweetInstance was referenced.
However my CloudFormation AutoScalingGroup was not set up this way and I still got the same error CREATE_COMPLETE state and cannot be signaled.
The solution for me was setting the DesiredCapacity on the AutoScalingGroup (hopefully this helps others as well) was found here:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html#cfn-as-group-desiredcapacity
As per the docs:
CloudFormation will not mark the Auto Scaling group as successful (by setting its status to CREATE_COMPLETE) until the desired capacity is reached.
This is what that portion of my CloudFormation template looks like:
"WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"VPCZoneIdentifier" : { "Ref" : "Subnets" },
"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
"MinSize" : "2",
"MaxSize" : "4",
"DesiredCapacity" : "2",
"TargetGroupARNs" : [ { "Ref" : "ALBTargetGroup" } ]
}
Below is the "signal" portion of UserData section:
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ", { "Ref" : "AWS::StackName" },
" --resource WebServerGroup ",
" --region ", { "Ref" : "AWS::Region" }, "\n",
To solve this problem I ensured that I added both CreationPolicy & UpdatePolicy in Autoscaling group before using cfn-signal in UserData section of the LaunchTemplate
CreationPolicy:
AutoScalingCreationPolicy:
MinSuccessfulInstancesPercent: somepercent
ResourceSignal:
Count: somenumber
Timeout: someminutes
UpdatePolicy:
AutoScalingScheduledAction:
IgnoreUnmodifiedGroupSizeProperties: 'true'
AutoScalingRollingUpdate:
MinInstancesInService: 'somenumber'
MaxBatchSize: 'somenumber'
PauseTime: someminutes
WaitOnResourceSignals: 'true'

CloudFormation template failing to create "files" under MetaData Section

Could somebody help me point what wrong am I doing here. I am failing to understand how the meta data part uses the authentication resource, In the AWS::CloudFormation::Authentication part I've mentioned the role same as the one attached to the instance, Yet I'm unable to create the file "some.txt"
{
"Parameters": {
"SecurityGroupId": {
"Description": "Security group for instance",
"Type": "AWS::EC2::SecurityGroup::Id"
}
},
"Resources": {
"MyInstance": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"configsets": {
"InstallIt": ["config1"]
},
"config1": {
"files": {
"/home/ec2-user/some.txt": {
"content": "This is my name ",
"encoding": "base64",
"mode": "000644",
"owner": "root",
"group": "root"
}
}
}
},
"AWS::CloudFormation::Authentication": {
"HelpMe": {
"type": "S3",
"buckets": "poc-bucket",
"roleName": "EC2andS3"
}
}
},
"Properties": {
"KeyName": "GoldenImage-NV-Anant",
"DisableApiTermination": "false",
"ImageId": "ami-0b33d91d",
"InstanceType": "t2.micro",
"Monitoring": "false",
"SubnetId": "subnet-73487a59",
"SecurityGroupIds": [{
"Ref": "SecurityGroupId"
}],
"IamInstanceProfile": {
"Ref": "MyInstanceProfile"
},
"Tags": [{
"Key": "Name",
"Value": "GeicoUserDataPocInstance"
}],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"", [
"#!/bin/bash -ex \n",
"echo \"hello dudes\" > /home/ec2-user/hello.txt \n",
"yum update -y aws-cfn-bootstrap\n",
"/opt/aws/bin/cfn-init -v",
" --stack ", {
"Ref": "AWS::StackId"
},
" --resource MyInstance ",
" --configsets InstallIt ",
" --region ", {
"Ref": "AWS::Region"
}, "\n",
"echo \"bye dudes\" > /home/ec2-user/bye.txt", "\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ", {
"Ref": "AWS::StackId"
},
" --resource MyInstance ",
" --region ", {
"Ref": "AWS::Region"
}, "\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT90M",
"Count": "1"
}
}
},
"MyInstanceProfile": {
"Description": "Instance profile for the instance",
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": ["EC2andS3"]
}
}
}
}
configsets should be configSets with capital S:
"configSets": {
"InstallIt": ["config1"]
},
buckets property needs to be a list of strings (this might not be necessary, the documentation is a bit unclear):
"buckets": ["poc-bucket"]
AWS::CloudFormation::Authentication resource shouldn't be necessary unless the source of your file is an S3 bucket. Even then, it still shouldn't be necessary when using an attached instance profile, since it will use the instance profile for authentication by default.

CloudFormation WaitCondition requires public access

I have a CloudFormation script which deploys a single instance in the VPC.
The script only works when the instance has a public IP. Without a public IP address, the template deployment fails at WaitCondition stage. I guess thats because the WaitCondition requires instance has access to the Internet?
Instead of assigning a public IP, will it work if I set the proxy address on the server? So the instance access the Internet via web proxy. If that's the case, what's the web URLs need to be whitelisted on the proxy? and can I add the add proxy address command in the userdata?
Below is my template:
"Mappings": {
"AWSRegion2AMI": {
"ap-southeast-2": {
"Windows2008r2": "ami-27b39a44",
"Windows2012r2": "ami-83b198e0"
}
}
},
"Resources": {
"DomainController": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"files": {
"c:\\cfn\\cfn-hup.conf": {
"content": {
"Fn::Join": ["", [
"[main]\n",
"stack=", {
"Ref": "AWS::StackId"
}, "\n",
"region=", {
"Ref": "AWS::Region"
}, "\n"
]]
}
},
"c:\\cfn\\hooks.d\\cfn-auto-reloader.conf": {
"content": {
"Fn::Join": ["", [
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.DomainController.Metadata.AWS::CloudFormation::Init\n",
"action=cfn-init.exe -v -s ", {
"Ref": "AWS::StackId"
},
" -r DomainController",
" --region ", {
"Ref": "AWS::Region"
}, "\n"
]]
}
},
"C:\\cfn\\RunCommand.bat": {
"content": "%~1\nIF %ERRORLEVEL% GTR 10 ( exit /b 1 ) else ( exit /b 0 )"
}
},
"commands": {
"1-run-dcpromo": {
"command": {
"Fn::Join": ["", [
"C:\\cfn\\RunCommand.bat \"dcpromo /unattend /ReplicaOrNewDomain:Domain /NewDomain:Forest /NewDomainDNSName:", {
"Ref": "DomainDNSName"
},
" /ForestLevel:4 /DomainNetbiosName:", {
"Ref": "DomainNetBIOSName"
},
" /DomainLevel:4 /InstallDNS:Yes /ConfirmGc:Yes /CreateDNSDelegation:No /DatabasePath:\"C:\\Windows\\NTDS\" /LogPath:\"C:\\Windows\\NTDS\" /SYSVOLPath:\"C:\\Windows\\SYSVOL\" /SafeModeAdminPassword=", {
"Ref": "RestoreModePassword"
},
" /RebootOnCompletion:Yes\""
]]
},
"waitAfterCompletion": "forever"
},
"2-signal-success": {
"command": {
"Fn::Join": ["", [
"cfn-signal.exe -e 0 \"", {
"Fn::Base64": {
"Ref": "DomainControllerWaitHandle"
}
}, "\""
]]
}
}
},
"services": {
"windows": {
"cfn-hup": {
"enabled": "true",
"ensureRunning": "true",
"files": ["c:\\cfn\\cfn-hup.conf", "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf"]
}
}
}
}
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": ["AWSRegion2AMI", {
"Ref": "AWS::Region"
}, "Windows2008r2"]
},
"InstanceType": {
"Ref": "InstanceType"
},
"NetworkInterfaces": [{
"AssociatePublicIpAddress": "false",
"DeviceIndex": "0",
"SubnetId": {
"Ref": "SubnetId"
}
}],
"KeyName": {
"Ref": "KeyName"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": ["", [
"<script>\n",
"cfn-init.exe -v -s ", {
"Ref": "AWS::StackId"
},
" -r DomainController ",
" --region ", {
"Ref": "AWS::Region"
}, "\n",
"</script>"
]]
}
}
}
},
"DomainControllerWaitCondition": {
"Type": "AWS::CloudFormation::WaitCondition",
"DependsOn": "DomainController",
"Properties": {
"Handle": {
"Ref": "DomainControllerWaitHandle"
},
"Timeout": "1500"
}
},
"DomainControllerWaitHandle": {
"Type": "AWS::CloudFormation::WaitConditionHandle"
}
Pass in the proxy by adding on of those parameters to cfn-signal.exe:
--http-proxy
An HTTP proxy (non-SSL). Use the following format: http://user:password#host:port
--https-proxy
An HTTPS proxy. Use the following format: https://user:password#host:port
The signal is sent to an S3 Bucket. You can whitelist something like:
https://cloudformation-waitcondition-*.s3.amazonaws.com
Note: You can also use the S3 VPC endpoint feature to allow resources inside private subnets to access S3. http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html