I'm creating a SpotFleet request using CloudFormation, but whenever I try to deploy it, fails with the message:
Unable to fetch parameters [ami-09bee01cc997a78a6] from parameter store for this account.
I'm using the following code (snippet):
SpotFleet:
Type: AWS::EC2::SpotFleet
Properties:
SpotFleetRequestConfigData:
ExcessCapacityTerminationPolicy: default
InstanceInterruptionBehavior: terminate
IamFleetRole: !GetAtt SpotFleetRole.Arn
TargetCapacity: !Ref ClusterSize
TerminateInstancesWithExpiration: false
LaunchSpecifications:
- IamInstanceProfile:
Arn: !Ref ECSInstanceProfile
ImageId: !Ref LatestAmiId
InstanceType: !Ref SpotFleetInstanceType
KeyName: !Ref KeyName
Monitoring:
Enabled: false
SecurityGroups:
- GroupId: !Ref ECSHostSecurityGroup
SubnetId: !Ref PublicSubnet
...
The ImageId is:
LatestAmiId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: '/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id'
When I manually fetch the image id from the AWS CLI I get:
aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id --region us-east-1 --output json | jq -r ."Parameters[].Value"
Output:
ami-09bee01cc997a78a6
If a manually hardcode this value into the ImageId parameter, it returns the same error (Unable to fetch parameters...)
Why it is failing if I am able to fetch the id value from the CLI?
Thanks to #Marcin's comment and this answer, I found the problem.
The problem is that I was using the wrong parameter type for the Image Id. I was using:
LatestAmiId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: '/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id'
and I should be using this:
LatestAmiId:
Description: Linux AMI
Type: AWS::EC2::Image::Id
Related
Parameters:
ClusterName:
Type: String
ClusterVersion:
Type: Number
AllowedValues: [1.21, 1.20, 1.19, 1.18]
RoleArnValue:
Type: String
ListOfSubnetIDs:
Description: Array of Subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ListOfSecurityGroupIDs:
Description: Array of security group ids
Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
EKSCluster:
Type: AWS::EKS::Cluster
Properties:
Name: !Sub ${ClusterName}
Version: !Sub ${ClusterVersion}
RoleArn: !Sub ${RoleArnValue}
ResourcesVpcConfig:
SecurityGroupIds:
- !Sub ${ListOfSecurityGroupIDs}
SubnetIds:
- !Sub ${ListOfSubnetIDs}
Above is the .yaml clouldformation template I have created so i can spin up eks cluster. Then i am using aws cli to spin up the cluster using the following command.
aws cloudformation deploy --template-file eks.yaml --stack-name cluster-test --parameter-overrides ClusterName=Dev ClusterVersion=1.21 ListOfSubnetIDs=subnet-11111d11b11b011f4,subnet-99999d237f87f11d7,subnet-222222c110c7e4be7,subnet-88888884de8d25176 ListOfSecurityGroupIDs=sg-01111111a21221 RoleArnValue=arn:aws:iam::123456546456:role/cluster-ServiceRole-WMIC72AOWSP0 --capabilities CAPABILITY_NAMED_IAM
I get the following error
An error occurred (ValidationError) when calling the CreateChangeSet operation: Template error: variable ListOfSecurityGroupIDs in Fn::Sub expression does not resolve to a string
I am not sure why. Am i using !sub in correctly? Would really appreciate input on this.
Since you want to reference the parameters you provided the template as they are, you should use the Ref function.
Here's an example of a valid template:
Parameters:
ClusterName:
Type: String
RoleArnValue:
Type: String
ListOfSubnetIDs:
Description: Array of Subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ListOfSecurityGroupIDs:
Description: Array of security group ids
Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
EKSCluster:
Type: AWS::EKS::Cluster
Properties:
Name: !Ref ClusterName
RoleArn: !Ref RoleArnValue
ResourcesVpcConfig:
SecurityGroupIds: !Ref ListOfSecurityGroupIDs
SubnetIds: !Ref ListOfSubnetIDs
and here's how I deployed it:
aws cloudformation deploy --template-file eks.yml --stack-name cluster-test --parameter-overrides ClusterName=Dev ListOfSubnetIDs=subnet-be0a99c4,subnet-c71046ae ListOfSecurityGroupIDs=sg-009690ac6b3bff6df,sg-009a3f1cb63943941 -RoleArnValue=...
Sub should be used when you want to perform string manipulation. Checkout the examples from the documentation.
I have a cloudformation template to create an ec2-instance. That template also starts an httpd along with some content that is served.
I'm using the Parameter section to allow a key to be specified or selected - see snippet below:
Parameters:
paramKeyPair:
Description: KeyPairName
Type: AWS::EC2::KeyPair::KeyName
I'm calling the ec2-instance through the AWS CLI like this :
aws cloudformation create-stack --stack-name stack-ec2instance --template-body file://demo-ec2instance --parameters ParameterKey=paramKeyPair,ParameterValue=peterKeyPair
So the instance can be created and the keypair can be passed through as an argument - BUT - frankly I don't actually care that much if the instance can be access. It's just a web server that can be spun up or down. SSH access is nice but no big deal.
In fact, if I removed the keypair Parameter from the cloudformation template - and removed the associated reference in the AWS CLI call - Cloudformation will happily spin up the instance without a keypair. Great !
What I would really like is for cloudformation to deal with the keypair being present or not.
I thought the best way to do this would be to update the code so that the parameter has a default value of "None" (for example) and then the ec2-instance could be run from the AWS CLI and if the keypair parameter is not specified then AWS would know not to bother with the keypair at all.
The problem is that by specifying the Type as AWS::EC2::KeyPair::KeyName, the AWS CLI expects an actual value.
I'm out of ideas - if anyone else has figured this out - I would really appreciate it.
Thankyou
Peter.
If I understand you correctly you want to be able to keep the parameter in your Cloudformation template, but only "allocate" a key pair to an instance if you specify a value, otherwise don't allocate a key pair to the ec2 instance resource. You can do this with AWS::NoValue pseudo parameter.
Here is a sample template:
Description: My EC2 instance
Parameters:
SSHKeyName:
Type: String
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Resources:
Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: <InstanceImageID>
InstanceType: t2.micro
KeyName: !Ref SSHKeyName
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
<other properties as required
So what this does is the condition checks if a SSHKeyName value is blank, if it's blank then the KeyName property will be ignored, if it isn't blank then it will use the value of SSHKeyName.
Thankyou WarrenG, your solution worked with one small exception which was to change the parameter type from AWS::EC2::KeyPair::KeyName to String.
Without your help I am certain I would have burned many more hours on this.
So in conclusion, the fix was
1: Change the Parameter type to String.
Parameters:
SSHKeyName:
Type: String
2: Add a function that determines if the key is present.
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Use the function within the resources section.
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
Within my question I kept the code snippets to a minimum for readability but now I have marked this as solved I'm adding two blocks of code just for documentation and incase this helps anyone else.
One example of calling the template through the AWS CLI.
aws cloudformation create-stack --stack-name stack-ec2instance --template-body file://demo-ec2instance --parameters ParameterKey=paramSubnetId,ParameterValue=$SubnetId ParameterKey=paramKeyPair,ParameterValue=peterKeyPair ParameterKey=paramSecurityGroupIds,ParameterValue=$SecurityGroupId
The template to create a EC2 instance.
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SSHKeyName:
Description: EC2 KeyPair for SSH access.
Type: String
Conditions:
Has-EC2-Key:
!Not [ !Equals [ !Ref SSHKeyName, '' ] ]
Mappings:
RegionMap:
eu-west-1:
AMI: ami-3bfab942
eu-west-2:
AMI: ami-098828924dc89ea4a
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
httpd: []
php: []
files:
/var/www/html/index.php:
content: !Sub |
<?php print "Hello Peter !"; ?>
services:
sysvinit:
httpd:
enabled: true
ensureRunning: true
Properties:
InstanceType: t2.micro
ImageId:
Fn::FindInMap:
- RegionMap
- !Ref AWS::Region
- AMI
SecurityGroupIds:
- !Ref MySecurityGroup
KeyName:
Fn::If:
- Has-EC2-Key
- Ref: SSHKeyName
- Ref: AWS::NoValue
UserData:
'Fn::Base64':
!Sub |
#!/bin/bash -xe
# Ensure AWS CFN Bootstrap is the latest
yum install -y aws-cfn-bootstrap
# Install the files and packages from the metadata
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
MySecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Open Ports 22 and 80
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
Outputs:
Website:
Description: The Public DNS for the EC2 Instance
Value: !Sub 'http://${EC2Instance.PublicDnsName}'
I want to create a simple rds proxy. I use the attached cloudformation template. However aws cannot create resource "AWS::RDS::DBProxyTargetGroup". The error info is not enough for debugging: "Resource timed out waiting for completion". Please anyone provide me anwser?
Target group was created, but info was not updated in cloudformation event
rds_proxy_console
CF_event
failed event
Resources:
RDSProxy:
Type: "AWS::RDS::DBProxy"
Properties:
Auth:
- AuthScheme: SECRETS
IAMAuth: DISABLED
SecretArn: !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:${SecretsManagerName}"
DBProxyName: !Ref ProxyName
EngineFamily: !Ref ProxyEngineFamily
RoleArn: !GetAtt SecretsManagerRole.Arn
VpcSecurityGroupIds: !Ref ProxyVpcSecurityGroupIds
VpcSubnetIds: !Ref ProxyVpcSubnetIds
RDSProxyTargetGroup:
Type: "AWS::RDS::DBProxyTargetGroup"
Properties:
DBClusterIdentifiers: !Ref ProxyTargetDBClusterIdentifiers
DBProxyName: !Ref RDSProxy
TargetGroupName: default
I'm writing a cloudformation template in YAML format.
Now i'm stuck with appending the id of the ebs volume to ec2 user data.
Type: AWS::EC2::Volume
Properties:
Size: 50
AvailabilityZone: ap-southeast-1b
Tags:
- Key: Name
Value: Logstash Volume
LogStashMountPoint:
Type: AWS::EC2::VolumeAttachment
Properties:
InstanceId:
Ref: LogstashInstance
VolumeId:
Ref: LogstashVolume
Device: "/dev/xvdf"
LogstashInstance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile:
Ref: LogstashInstanceProfile
InstanceType: t2.micro
KeyName: chuongtest
ImageId: ami-0cd31be676780afa7
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash -xe
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
{LogstashVolume: Ref: LogstashVolume}
touch /home/ec2-user/ebscomplete.txt
curl "http://169.254.169.254/latest/meta-data/instance-id" >> /home/ec2-user/ec2id.txt
touch /home/ec2-user/ec2complete.txt
touch /home/ec2-user/complete.txt
- LogstashVolume: !Ref LogstashVolume
SecurityGroupIds:
- Ref: LogstashSecurityGroup
SubnetId: subnet-0d0e0989f57b96389
Tags:
- Key: Name
Value: Logstash Instance
UserData script with Resource Attribute CloudFormation
I'm following this link but it still doesn't work
When the new instance is launched. It has nothing in /home/ec2-user.
I looked everywhere and this is my final but it didn't work
Can anyone help me with it?
There is at least one syntax error in your UserData:
echo ${LogstashVolume}) >> /home/ec2-user/ebsid.txt
should be
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
For further debugging of your UserData, login to the instance and check /var/log/could-init-output.log file.
p.s.
The following is also incorrect:
{LogstashVolume: Ref: LogstashVolume}
Its not a valid bash command.
I am trying to create a condition based on an optional parameter. The option is whether to run some additional installation from my userData script for an EC2 deployment.
The parameters and conditions look like this:
Parameters:
EnvType:
Description: Option to install New Relic Infrastructure.
Default: apm
Type: String
AllowedValues:
- apm
- +infra
Then my EC2 Resource with conditional startup scripts
Resources:
Ec2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
ImageId: ami-9c9443e3 #Amazon Linux AMI in Tokyo
KeyName: tokyocloudformation
IamInstanceProfile: 'S3EC2'
SecurityGroupIds:
- !Ref myNewSecurityGroup
UserData:
Condition: apmOnly
Fn::Base64:
|
#!/bin/bash
installstuff
Condition: addInfrastructureAgent
Fn::Base64:
|
#!/bin/bash
installstuff
installsomeotherstuff
The error message I'm getting is:
Template validation error: Template format error: Unresolved dependencies [EnvTyp]. Cannot reference resources in the Conditions block of the template
I get what the error is saying I believe, but it doesn't seem to fit with the examples given by AWS.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html .
This AWS example clearly uses a !Ref in the Conditions block.
EnvType:
Description: Environment type.
Default: test
Type: String
AllowedValues:
- prod
- test
ConstraintDescription: must specify prod or test.
Conditions:
CreateProdResources: !Equals [ !Ref EnvType, prod ]
Can someone provide some feedback on how to implement this conditional or why this error message is being thrown for this implementation?
According to the docs, Conditions should be used at the top level of the resource you want to conditionally create.
Putting a Condition inside the Instance UserData section isn't supported.
To use Conditions in your situation, you'd want separate Resources conditionally created based on the Parameter.
Resources:
Ec2InstanceAPMOnly:
Type: AWS::EC2::Instance
Condition: apmOnly
Properties:
InstanceType: t2.micro
ImageId: ami-9c9443e3 #Amazon Linux AMI in Tokyo
KeyName: tokyocloudformation
IamInstanceProfile: 'S3EC2'
SecurityGroupIds:
- !Ref myNewSecurityGroup
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
installstuff
Ec2InstanceWithInfrastructureAgent:
Type: AWS::EC2::Instance
Condition: addInfrastructureAgent
Properties:
InstanceType: t2.micro
ImageId: ami-9c9443e3 #Amazon Linux AMI in Tokyo
KeyName: tokyocloudformation
IamInstanceProfile: 'S3EC2'
SecurityGroupIds:
- !Ref myNewSecurityGroup
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
installstuff
installsomeotherstuff
You can use Conditions in userData as well. i followed this post https://www.singlestoneconsulting.com/blog/cloudformation-mapping-and-conditionals-making-your-templates-more-universal/ and it worked perfactlly