Cloudformation Init config files not writing the files - amazon-web-services

I am using AWS Cloudformation scripts to bring up a Auto-scaling Ec2 Instance - sample code provided below
"GatewayLabAutoScalingGroup": {
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"commands": {
"a_install_pip": {
"command": "pip install requests boto3"
},
"c_restart_cron": {
"command": "service crond restart"
},
"d_restart_cfn_hup": {
"command": "service cfn-hup restart"
}
},
"files": {
"/etc/cfn/cfn-hup.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\nstack=",
{
"Ref": "AWS::StackName"
},
"\nregion=",
{
"Ref": "AWS::Region"
},
"\nverbose=true\ninterval=1\n"
]
]
},
"group": "root",
"mode": "000644",
"owner": "root"
},
"/usr/local/sbin/join_ad_script.sh": {
"content": {
"Fn::Join": [
"",
[
"sudo yum -y update\nsudo yum -y install sssd realmd krb5-workstation\nsudo realm leave\n\nDOMAIN=\"",
{
"Ref": "SimpleADDomain"
},
"\"\n\ncat <<EOF > /etc/resolv.conf\nnameserver ",
{
"Fn::Select": [
0,
{
"Fn::GetAtt": [
"WorkspacesSimplead",
"DnsIpAddresses"
]
}
]
},
"\nnameserver ",
{
"Fn::Select": [
1,
{
"Fn::GetAtt": [
"WorkspacesSimplead",
"DnsIpAddresses"
]
}
]
},
"\nEOF\n\n# empty all current sssd cache\nsss_cache -E\n\necho ",
{
"Ref": "SimpleADPassword"
},
" | sudo realm join -U Administrator#${DOMAIN^^} ${DOMAIN^^} --verbose\nsudo sed -re 's/^(PasswordAuthentication)([[:space:]]+)no/\\1\\2yes/' -i.`date -I` /etc/ssh/sshd_config\necho \"enumerate=true\" >> /etc/sssd/sssd.conf\nsudo service sssd restart\nsudo service sshd restart\n\n# empty all current sssd cache\nsss_cache -E\n"
]
]
},
"group": "root",
"mode": "000755",
"owner": "root"
}
}
}
}
},
"Properties": {
"AvailabilityZones": [
{
"Fn::Select": [
0,
{
"Fn::GetAZs": ""
}
]
}
],
"HealthCheckGracePeriod": 300,
"HealthCheckType": "EC2",
"LaunchConfigurationName": {
"Ref": "GatewayLabLaunchConfiguration"
},
"LoadBalancerNames": [
],
"MaxSize": 2,
"MinSize": 1,
"Tags": [
{
"Key": "Name",
"PropagateAtLaunch": true,
"Value": "hub-autoscaling"
}
}
],
"VPCZoneIdentifier": [
{
"Ref": "EC2SubnetSubnet1"
}
]
},
"Type": "AWS::AutoScaling::AutoScalingGroup",
"UpdatePolicy": {
"AutoScalingRollingUpdate": {
"MaxBatchSize": 1,
"MinInstancesInService": 1,
"PauseTime": "PT60S"
}
}
}
The files are not written to in the instance
The instance is coming up in a Private VPC
We have a proxy configured on port 8080
The works fine when the instance is connected to a NAT Gateway without a proxy
I do have ports 80, 22 & 443 opened up

the userdata statements are run initially
They then call the cfn-init scripts
There were some errors in the scripts and they never completed; one of the problems as mentioned above was my instance behind a proxy
Getting the proxy configuration in as part of the UserData helped

Related

Why am I getting an error that there are unresolved VPC dependencies?

I'm trying to implement configuration management that meets the following criteria for an assighment:
https://s3.amazonaws.com/seis615/AnsiblePress.json
Take a quick look at the template in a text editor. Notice how the UserData property for the mgmt1 instance is configured. When CloudFormation launches this stack, it will automatically install and configure Ansible software on the management server. It’s very common to use a small amount of scripting code to bootstrap configuration management software onto a new system. Once Ansible is installed it can be used to install and configure other servers in the environment.
The CloudFormation template is missing a couple resources that you will need to add:
An application load balancer with a logical name of webserverlb which distributes HTTP (port 80) requests to the web1 and web2 instances. The health check endpoint for the load balancer should be the root (/) directory.
A db.t2.micro RDS database instance (not a cluster) running a MariaDB 10.2.21 database called wordpress located in a private VPC subnet. Use the logical name wordpressdb for the CloudFormation RDS resource. RDS and EC2 instances actually pre-date the arrival of VPCs in AWS so confusingly there are two different ways to configure these resources. You need to make sure this database instance is designed to run inside a VPC with the proper database subnet group and security group resources defined.
A security group called WebserverLbSecurityGroup which allows incoming http access from the Internet.
A security group called WordpressDbSecurityGroup which allows incoming access on the standard MySQL port from the WebServerSecurityGroup
An input parameter called DBName which will define the database name to create (default to wordpress)
An input parameter called DBUser which will be used for the database server username.
An input parameter called DBPassword which will be used for the database server password.
A stack output called wordpressDbEndpoint which shows the MariaDB instance endpoint address.
A stack output called wordpressLbEndpoint which shows the application load balancer URL.
The JSON I've configured (below) gives me the following template format error and I don't know why:
Template format error: Unresolved resource dependencies [wordpressVPC] in the Resources block of the template
{"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"SSMAccessRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version" : "2012-10-17",
"Statement": [ {
"Effect": "Allow",
"Principal": {
"Service": [ "ec2.amazonaws.com" ]
},
"Action": [ "sts:AssumeRole" ]
} ]
},
"Path": "/"
}
},
"SSMRolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "ssmProperties",
"PolicyDocument": {
"Version" : "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:DescribeParameters",
"ssm:PutParameter",
"ssm:GetParameters",
"ssm:DeleteParameter"
],
"Resource": {
"Fn::Join" : [
"",
[
"arn:aws:ssm:",
{ "Ref" : "AWS::Region" },
":",
{ "Ref" : "AWS::AccountId"},
{
"Fn::Join" : [
"",
[ ":parameter/", { "Ref": "AWS::StackName" }, ".*" ]
]
}
]
]
}
}
]
},
"Roles": [ { "Ref": "SSMAccessRole" } ]
}
},
"SSMInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ { "Ref": "SSMAccessRole" } ]
}
},
"web1pem" : {
"Type" : "AWS::SSM::Parameter",
"Properties" : {
"Name" : {
"Fn::Join" : [
"",
[ { "Ref": "AWS::StackName" }, ".web1pem" ]
]
},
"Type" : "String",
"Value" : "0",
"Description": "web1 instance private key."
}
},
"web2pem" : {
"Type" : "AWS::SSM::Parameter",
"Properties" : {
"Name" : {
"Fn::Join" : [
"",
[ { "Ref": "AWS::StackName" }, ".web2pem" ]
]
},
"Type" : "String",
"Value" : "0",
"Description": "web2 instance private key."
}
},
"wordpressVpc": {
"Type": "AWS::EC2::VPC",
"Properties": {
"EnableDnsSupport": "true",
"EnableDnsHostnames": "true",
"CidrBlock": "10.0.0.0/16",
"Tags": [
{
"Key": "Environment",
"Value": "Test"
}
]
}
},
"publicSubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"publicSubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "1", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"privateSubnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.2.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "0", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"privateSubnet2": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"CidrBlock": "10.0.3.0/24",
"AvailabilityZone" : {
"Fn::Select" : [ "1", { "Fn::GetAZs" : { "Ref" : "AWS::Region" }}]
}
}
},
"web1": {
"Type": "AWS::EC2::Instance",
"DependsOn": [
"web1pem"
],
"Properties": {
"InstanceType": "t2.micro",
"ImageId": {"Ref": "AMI"},
"IamInstanceProfile": {
"Ref": "SSMInstanceProfile"
},
"KeyName": {
"Ref": "KeyName"
},
"NetworkInterfaces": [
{
"GroupSet": [
{
"Ref": "WebServerSecurityGroup"
}
],
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"DeleteOnTermination": "true",
"SubnetId": {
"Ref": "publicSubnet1"
}
}
],
"Tags": [
{
"Key": "Name",
"Value": "web1"
}
],
"UserData" : {
"Fn::Base64" : {
"Fn::Join" : [
"", [
"#!/bin/bash -xe\n",
"ssh-keygen -f /home/ec2-user/.ssh/web1-key.pem -q -N \"\"\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web1-key.pem\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web1-key.pem.pub\n",
"PEMFILE=`cat /home/ec2-user/.ssh/web1-key.pem`\n",
"aws ssm put-parameter --name ", { "Ref" : "web1pem" }, " --type String --value \"${PEMFILE}\" --overwrite --region ", { "Ref" : "AWS::Region" },"\n",
"cat /home/ec2-user/.ssh/web1-key.pem.pub >> /home/ec2-user/.ssh/authorized_keys\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource web1 ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT5M"
}
}
},
"web2": {
"Type": "AWS::EC2::Instance",
"DependsOn": [
"web1pem"
],
"Properties": {
"InstanceType": "t2.micro",
"ImageId": {"Ref": "AMI"},
"IamInstanceProfile": {
"Ref": "SSMInstanceProfile"
},
"KeyName": {
"Ref": "KeyName"
},
"NetworkInterfaces": [
{
"GroupSet": [
{
"Ref": "WebServerSecurityGroup"
}
],
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"DeleteOnTermination": "true",
"SubnetId": {
"Ref": "publicSubnet2"
}
}
],
"Tags": [
{
"Key": "Name",
"Value": "web2"
}
],
"UserData" : {
"Fn::Base64" : {
"Fn::Join" : [
"", [
"#!/bin/bash -xe\n",
"ssh-keygen -f /home/ec2-user/.ssh/web2-key.pem -q -N \"\"\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web2-key.pem\n",
"chown ec2-user:ec2-user /home/ec2-user/.ssh/web2-key.pem.pub\n",
"PEMFILE=`cat /home/ec2-user/.ssh/web2-key.pem`\n",
"aws ssm put-parameter --name ", { "Ref" : "web2pem" }, " --type String --value \"${PEMFILE}\" --overwrite --region ", { "Ref" : "AWS::Region" },"\n",
"cat /home/ec2-user/.ssh/web2-key.pem.pub >> /home/ec2-user/.ssh/authorized_keys\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource web2 ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT5M"
}
}
},
"WebServerSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
},
"GroupDescription": "Allow access from HTTP and SSH traffic",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": {"Ref": "YourIp"}
}
]
}
},
"WebServerSGIngressTCP22": {
"Type": "AWS::EC2::SecurityGroupIngress",
"Metadata": {
"Comment": "SSH ingress security rule"
},
"Properties" : {
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"SourceSecurityGroupId": { "Ref": "WebServerSecurityGroup" },
"GroupId": { "Fn::GetAtt": ["WebServerSecurityGroup", "GroupId"]}
}
},
"InternetGateway": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"InternetGatewayId": {
"Ref": "InternetGateway"
},
"VpcId": {
"Ref": "wordpressVpc"
}
}
},
"PublicRouteTable": {
"Type": "AWS::EC2::RouteTable",
"Properties": {
"VpcId": {
"Ref": "wordpressVpc"
}
}
},
"PublicRoute": {
"Type": "AWS::EC2::Route",
"Properties": {
"DestinationCidrBlock": "0.0.0.0/0",
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"GatewayId": {
"Ref": "InternetGateway"
}
},
"DependsOn": [
"InternetGateway", "AttachGateway"
]
},
"Public1RouteTableAssociation": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"SubnetId": {
"Ref": "publicSubnet1"
}
}
},
"Public2RouteTableAssociation": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"RouteTableId": {
"Ref": "PublicRouteTable"
},
"SubnetId": {
"Ref": "publicSubnet2"
}
}
},
"webserverlb": {
"Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
"Properties": {
"IpAddressType": "ipv4",
"SecurityGroups": [
{
"Ref": "webserverlbSecurityGroup"
}
],
"Subnets": [
{
"Ref": "publicSubnet1"
},
{
"Ref": "publicSubnet2"
}
],
"Tags": [
{
"Key": "Name",
"Value": "webserverlb"
}
]
},
"DependsOn": [
"webserversSecurityGroup"
]
},
"webserverlbSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"GroupDescription": "Allows incoming requests from port 80 via HTTP.",
"SecurityGroupIngress": [
{
"IpProtocol": "TCP",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0",
"Description": "Allows 80 from Internet"
}
]
}
},
"wordpressdb": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"AvailabilityZone": "us-east-1a",
"DBInstanceClass": "db.t2.micro",
"DBName": "wordpress",
"Engine": "mariadb",
"EngineVersion": "10.2.21",
"MultiAZ": 1,
"Tags": [
{
"Key": "Name",
"Value": "wordpressdb"
}
]
},
"DependsOn": [
"wordpressdbSecurityGroup"
]
},
"wordpressdbSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"VpcId": {
"Ref": "wordpressVPC"
},
"GroupDescription": "Enable access to the db via port 3306.",
"Tags": [
{
"Key": "Name",
"Value": "wordpressdbSecurityGroup"
}
],
"SecurityGroupIngress": [
{
"IpProtocol": "TCP",
"FromPort": "3306",
"ToPort": "3306",
"Description": "Enable HTTP access."
}
]
}
}
},
"Parameters": {
"KeyName": {
"Description": "Name of your EC2 KeyPair to enable SSH access to the instances.",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription": "must be the name of an existing EC2 KeyPair."
},
"YourIp": {
"Description": "The current CIDR IP address of your workstation (x.x.x.x/32). http://checkip.amazonaws.com/",
"Type": "String",
"AllowedPattern": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\\/(1[6-9]|2[0-9]|3[0-2]))$",
"ConstraintDescription": "Must be a valid IP CIDR range of the form x.x.x.x/x."
},
"AMI": {
"Description": "The EC2 instance AMI",
"Type": "String",
"Default": "ami-00dc79254d0461090"
},
"DBName": {
"Description": "Name of the database",
"Type" : "String",
"Default": "wordpress"
},
"DBUser": {
"Default": "admin",
"NoEcho": "false",
"Description" : "The WordPress database admin account user name",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*"
},
"DBPassword": {
"NoEcho": "true",
"Description" : "The password of the database.",
"Type": "String",
"MinLength": "1",
"MaxLength": "16",
"AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*"
}
},
"Outputs": {
"web1PublicIp": {
"Value": {"Fn::GetAtt": ["web1","PublicIp"]},
"Description": "web1 public IP"
},
"we2PublicIp": {
"Value": {"Fn::GetAtt": ["web2","PublicIp"]},
"Description": "web2 public IP"
},
"mgmt1PublicIp": {
"Value": {"Fn::GetAtt": ["mgmt1","PublicIp"]},
"Description": "mgmt1 public IP"
}
}
}
Because CloudFormation is case sensitive. Your vpc resource is called wordpressVpc, but in some places you are using wordpressVPC.
Recommend trying the CloudFormation Linter in VSCode to see some of these errors inline while authoring templates along with autocompletion and documentation links:
E3005 DependsOn should reference other resources at Resources/webserverlb/DependsOn/0
E1012 Ref wordpressVPC not found as a resource or parameter
E1012 Ref wordpressVPC not found as a resource or parameter
E3002 Invalid Property Resources/wordpressdb/Properties/VpcId
E3003 Property DBSecurityGroupIngress missing at Resources/wordpressdbSecurityGroup/Properties
E1012 Ref wordpressVPC not found as a resource or parameter
E3002 Invalid Property Resources/wordpressdbSecurityGroup/Properties/VpcId
E3002 Invalid Property Resources/wordpressdbSecurityGroup/Properties/SecurityGroupIngress
E1010 Invalid GetAtt mgmt1.PublicIp for resource mgmt1PublicIp

I am not able to copy S3 file to EC2 instance using Userdata in CloudFormation

I am unable to download an S3 file to my EC2 instance using CloudFormation Userdata property. I have assigned an IAM role but still not able to get it resolved.
I assigned the role inside the template.
I tried passing Access Key and Secret Access Key - same result.
"Parameters": {
"VpcId": {
"Type": "AWS::EC2::VPC::Id",
"Description": "Id of an existing VPC to use for "
},
"SubnetId": {
"Type": "AWS::EC2::Subnet::Id",
"Description": "Id of an existing subnet id to use for "
},
"SecurityGroupIds": {
"Description": "Security groups ",
"Type": "List<AWS::EC2::SecurityGroup::Id>",
"ConstraintDescription": "using existing security be list of EC2 security group ids"
},
"instanceType": {
"Type": "String",
"Default": "t2.micro",
"AllowedValues": [
"t2.micro"
],
"Description": "Enter Instance Type "
},
"AWSREGION": {
"Type": "String",
"Default": "us-east-1",
"AllowedValues": [
"us-east-1"
],
"Description": "Enter AWS_REGION."
}
},
"Resources": {
"InstanceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": [ "ec2.amazonaws.com" ] },
"Action": [ "sts:AssumeRole" ]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "S3_Access",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::mybucketlocation/*"]
}
]
}
}
]
}
},
"InstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ { "Ref": "InstanceRole" }
]
}
},
"EdgeNode": {
"Type": "AWS::EC2::Instance",
"Properties": {
"IamInstanceProfile": { "Ref": "InstanceProfile" },
"InstanceType": { "Ref" : "instanceType" },
"ImageId": "ami-0cc96feef8c6bbff3",
"SubnetId": { "Ref" : "SubnetId" },
"KeyName": "my-key",
"SecurityGroupIds": {
"Ref": "SecurityGroupIds"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"\n",
[
"#!/bin/bash",
"echo \"\" > /home/xyz/index.txt",
{
"Fn::Join": [
"",
[
"echo \"AWS_REGION: ",
{
"Ref": "AWSREGION"
},
"\" >> /home/xyz/index.txt"
]
]
},
{
"Fn::Join": ["", [
"<script>\n",
"cfn-init.exe -v -s ", { "Ref" : "AWS::StackId" }, " -r Instance --region ", { "Ref" : "AWS::Region" }, "\n",
"</script>"
] ]
}
]
]
}
}
},
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"commands" : {
"Pullcode" : {
"command" : "aws s3 sync s3://mybucketlocation /home/xyz/ --debug"
}
}
}
},
"AWS::CloudFormation::Designer": {
"id": "e37a9183-9f81c2fbd39"
}
}
}
}
In cloud-init-output.log I got this:
/var/lib/cloud/instance/scripts/part-001: line 7: syntax error near unexpected token newline'
/var/lib/cloud/instance/scripts/part-001: line 7:'
Jun 21 11:45:05 cloud-init[4071]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [2]
Jun 21 11:45:05 cloud-init[4071]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
Jun 21 11:45:05 cloud-init[4071]: util.py[WARNING]: Running module scripts-
These lines seem strange:
"Fn::Join": ["", [
"<script>\n",
"cfn-init.exe -v -s ", { "Ref" : "AWS::StackId" }, " -r Instance --region ", { "Ref" : "AWS::Region" }, "\n",
"</script>"
You are launching an Amazon EC2 instance. However, these lines look like they were taken from User Data for a Windows instance.
Also, you are prompting the user for a Region, but the script is already running in a specific region, so you can use { "Ref" : "AWS::Region" } to access the value.
You probably want your User Data script to look like this:
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"\n",
[
"#!/bin/bash",
{
"Fn::Sub": "echo AWS_REGION: ${AWS::REGION} >>/home/xyz/index.txt"
},
{
"Fn::Sub": "cfn-init -v -s ${AWS::StackId} -r EdgeNode --region ${AWS::Region}"
},
]
]
}
}
I didn't test it, so you might need to tweak some things.
Userdata is always a dreadful property to get right. You can try cloudkast which is an online cloudformation template generator. It makes it very easy for your to use intrinsic functions in cloudformation which I belive has a bit of a learning curve.

AWS cloud formation cannot valid instance id

I am creating a cloudformation template in AWS. I am still getting an error:
WaitCondition received failed message: 'ValidationError:' for uniqueId: i-04885f92b0b4a99ab
I have tried change instance id. and many other way. none of these works.
The error occurred on AWS::CloudFormation::WaitCondition
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"KeyName": {
"Description": "Name of an existing EC2 KeyPair to enable SSH access to the instances",
"Type": "AWS::EC2::KeyPair::KeyName"
},
"InstanceType": {
"Description": "FormEngine EC2 instance type",
"Type": "String",
"Default": "t2.micro"
}
},
"Mappings": {
"AWSInstanceType2Arch": {
"t2.micro": {
"Arch": "64"
}
},
"AWSRegionArch2AMI": {
"us-west-2": {
"64": "ami-f2d3638a"
}
}
},
"Resources": {
"WebServerGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Enable SSH and HTTP access",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
}
]
}
},
"CfnUser": {
"Type": "AWS::IAM::User",
"Properties": {
"Path": "/",
"Policies": [
{
"PolicyName": "Admin",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}
}
]
}
},
"HostKeys": {
"Type": "AWS::IAM::AccessKey",
"Properties": {
"UserName": {
"Ref": "CfnUser"
}
}
},
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"packages": {
"yum": {
"java-1.6.0-openjdk": [],
"tomcat6": [],
"httpd": []
}
},
"files": {
"/var/www/html/index.html": {
"source": "http://a00807428-lab02/index.html",
"mode": "000600",
"owner": "apache",
"group": "apache"
},
"/var/www/html/index.html": {
"source": "http://a00807428-lab02/index.html",
"mode": "000600",
"owner": "apache",
"group": "apache"
}
}
}
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": [
"AWSRegionArch2AMI",
{
"Ref": "AWS::Region"
},
{
"Fn::FindInMap": [
"AWSInstanceType2Arch",
{
"Ref": "InstanceType"
},
"Arch"
]
}
]
},
"InstanceType": {
"Ref": "InstanceType"
},
"SecurityGroups": [
{
"Ref": "WebServerGroup"
}
],
"KeyName": {
"Ref": "KeyName"
},
"Tags": [
{
"Key": "Name",
"Value": "WebServer"
}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -v\n",
"date > /home/ec2-user/starttime\n",
"yum update -y aws-cfn-bootstrap\n",
"## Error reporting helper function\n",
"function error_exit\n",
"{\n",
" /opt/aws/bin/cfn-signal -e 1 -r \"$1\" '",
{
"Ref": "WaitHandle"
},
"'\n",
" exit 1\n",
"}\n",
"## Initialize CloudFormation bits\n",
"/opt/aws/bin/cfn-init -v -s ",
{
"Ref": "AWS::StackId"
},
" -r FormEngine",
" --access-key ",
{
"Ref": "HostKeys"
},
" --secret-key ",
{
"Fn::GetAtt": [
"HostKeys",
"SecretAccessKey"
]
},
" --region ",
{
"Ref": "AWS::Region"
},
" > /tmp/cfn-init.log 2>&1 || error_exit $(</tmp/cfn-init.log)\n",
"# Configure Apache HTTPD\n",
"chkconfig httpd on\n",
"chkconfig --level 345 httpd on\n",
"# Start servers\n",
"/etc/init.d/httpd start\n",
"# Send signal to WaitHandle that the setup is completed\n",
"/opt/aws/bin/cfn-signal",
" -e 0",
" '",
{
"Ref": "WaitHandle"
},
"'",
"\n",
"date > /home/ec2-user/stoptime"
]
]
}
}
}
},
"WaitHandle": {
"Type": "AWS::CloudFormation::WaitConditionHandle"
},
"WaitCondition": {
"Type": "AWS::CloudFormation::WaitCondition",
"DependsOn": "WebServer",
"Properties": {
"Handle": {
"Ref": "WaitHandle"
},
"Timeout": "1200"
}
},
"IPAddress": {
"Type": "AWS::EC2::EIP"
},
"IPAssoc": {
"Type": "AWS::EC2::EIPAssociation",
"Properties": {
"InstanceId": {
"Ref": "WebServer"
},
"EIP": {
"Ref": "IPAddress"
}
}
}
},
"Outputs": {
"InstanceIPAddress": {
"Value": {
"Ref": "IPAddress"
},
"Description": "public IP address of the new WebServer"
},
"InstanceName": {
"Value": {
"Fn::GetAtt": [
"WebServer",
"PublicDnsName"
]
},
"Description": "public DNS name of the new WebServer"
}
}
}
succeed to created cloudformation.

CloudFormation WaitCondition requires public access

I have a CloudFormation script which deploys a single instance in the VPC.
The script only works when the instance has a public IP. Without a public IP address, the template deployment fails at WaitCondition stage. I guess thats because the WaitCondition requires instance has access to the Internet?
Instead of assigning a public IP, will it work if I set the proxy address on the server? So the instance access the Internet via web proxy. If that's the case, what's the web URLs need to be whitelisted on the proxy? and can I add the add proxy address command in the userdata?
Below is my template:
"Mappings": {
"AWSRegion2AMI": {
"ap-southeast-2": {
"Windows2008r2": "ami-27b39a44",
"Windows2012r2": "ami-83b198e0"
}
}
},
"Resources": {
"DomainController": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"files": {
"c:\\cfn\\cfn-hup.conf": {
"content": {
"Fn::Join": ["", [
"[main]\n",
"stack=", {
"Ref": "AWS::StackId"
}, "\n",
"region=", {
"Ref": "AWS::Region"
}, "\n"
]]
}
},
"c:\\cfn\\hooks.d\\cfn-auto-reloader.conf": {
"content": {
"Fn::Join": ["", [
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.DomainController.Metadata.AWS::CloudFormation::Init\n",
"action=cfn-init.exe -v -s ", {
"Ref": "AWS::StackId"
},
" -r DomainController",
" --region ", {
"Ref": "AWS::Region"
}, "\n"
]]
}
},
"C:\\cfn\\RunCommand.bat": {
"content": "%~1\nIF %ERRORLEVEL% GTR 10 ( exit /b 1 ) else ( exit /b 0 )"
}
},
"commands": {
"1-run-dcpromo": {
"command": {
"Fn::Join": ["", [
"C:\\cfn\\RunCommand.bat \"dcpromo /unattend /ReplicaOrNewDomain:Domain /NewDomain:Forest /NewDomainDNSName:", {
"Ref": "DomainDNSName"
},
" /ForestLevel:4 /DomainNetbiosName:", {
"Ref": "DomainNetBIOSName"
},
" /DomainLevel:4 /InstallDNS:Yes /ConfirmGc:Yes /CreateDNSDelegation:No /DatabasePath:\"C:\\Windows\\NTDS\" /LogPath:\"C:\\Windows\\NTDS\" /SYSVOLPath:\"C:\\Windows\\SYSVOL\" /SafeModeAdminPassword=", {
"Ref": "RestoreModePassword"
},
" /RebootOnCompletion:Yes\""
]]
},
"waitAfterCompletion": "forever"
},
"2-signal-success": {
"command": {
"Fn::Join": ["", [
"cfn-signal.exe -e 0 \"", {
"Fn::Base64": {
"Ref": "DomainControllerWaitHandle"
}
}, "\""
]]
}
}
},
"services": {
"windows": {
"cfn-hup": {
"enabled": "true",
"ensureRunning": "true",
"files": ["c:\\cfn\\cfn-hup.conf", "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf"]
}
}
}
}
}
},
"Properties": {
"ImageId": {
"Fn::FindInMap": ["AWSRegion2AMI", {
"Ref": "AWS::Region"
}, "Windows2008r2"]
},
"InstanceType": {
"Ref": "InstanceType"
},
"NetworkInterfaces": [{
"AssociatePublicIpAddress": "false",
"DeviceIndex": "0",
"SubnetId": {
"Ref": "SubnetId"
}
}],
"KeyName": {
"Ref": "KeyName"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": ["", [
"<script>\n",
"cfn-init.exe -v -s ", {
"Ref": "AWS::StackId"
},
" -r DomainController ",
" --region ", {
"Ref": "AWS::Region"
}, "\n",
"</script>"
]]
}
}
}
},
"DomainControllerWaitCondition": {
"Type": "AWS::CloudFormation::WaitCondition",
"DependsOn": "DomainController",
"Properties": {
"Handle": {
"Ref": "DomainControllerWaitHandle"
},
"Timeout": "1500"
}
},
"DomainControllerWaitHandle": {
"Type": "AWS::CloudFormation::WaitConditionHandle"
}
Pass in the proxy by adding on of those parameters to cfn-signal.exe:
--http-proxy
An HTTP proxy (non-SSL). Use the following format: http://user:password#host:port
--https-proxy
An HTTPS proxy. Use the following format: https://user:password#host:port
The signal is sent to an S3 Bucket. You can whitelist something like:
https://cloudformation-waitcondition-*.s3.amazonaws.com
Note: You can also use the S3 VPC endpoint feature to allow resources inside private subnets to access S3. http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html

Run UserData after instance initialize

I´m using cloudformation to create an ec2 instance through an AMI which contains a docker image.
The idea is use userData to run the docker image once that the instance is initialized.
But once that I connect by ssh and check the docker containers, no docker container is running.
This is my cloudformation template
"instancei0755a35a9c753812d": {
"Type": "AWS::EC2::Instance",
"Properties": {
"DisableApiTermination": "false",
"InstanceInitiatedShutdownBehavior": "stop",
"ImageId": "ami-foooo",
"InstanceType": "t2.medium",
"KeyName": "jenkins_price",
"Monitoring": "false",
"Tags": [
{
"Key": "Name",
"Value": "Jenkins"
},
{
"Key": "Type",
"Value": "jenkins"
}
],
"Volumes": [
{
"Device": "/dev/sdf",
"VolumeId": {
"Ref": "volumevolab668c6e"
}
}
],
"NetworkInterfaces": [
{
"AssociatePublicIpAddress": true,
"DeleteOnTermination": "true",
"DeviceIndex": 0,
"SubnetId": {
"Ref": "subnet"
},
"GroupSet": [
{
"Ref": "jenkinsSg"
},
{
"Ref": "usersSshSg"
}
]
}
],
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -v\n",
"# Run Jenkins docker image\n",
"cd /home/ec2-user/price_aws/images/jenkins\n",
"docker-compose up &\n"
]
]
}
}
}
}
}
As far as I know UserData should be executed once the instance is running.
I´m doing something wrong?
Regards.