Review needed to correct this ElastiCache CloudFormation Template - amazon-web-services

This template is used to create a ElastiCache- Redis cluster.
Its showing me errors like - 1 validation error detected: Value '[AWS::ElastiCache::CacheCluster, AWS::EC2::SecurityGroup::Id]' at 'typeNameList' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 204, Member must have length greater than or equal to 10, Member must satisfy regular expression pattern: [A-Za-z0-9]{2,64}::[A-Za-z0-9]{2,64}::[A-Za-z0-9]{2,64}(::MODULE){0,1}].
Wanted to know if the parameters are declared rightly or not.
AWSTemplateFormatVersion: 2010-09-09
Description: Create ElastiCache and related resources
Parameters:
VPC:
Description: VPC
Type: AWS::EC2::VPC::Id
Subnet:
Description: Subnet
Type: AWS::EC2::Subnet::Id
ClusterName:
Description: Custom name of the cluster. Auto generated if you
don't supply your own.
Type: String
CacheNodeType:
Description: Cache node instance class, e.g. cache.t2.micro.
Type: String
Default: cache.t2.micro
ConstraintDescription: Node instance class not supported
AllowedValues:
- cache.t2.micro
- cache.t2.small
- cache.t2.medium
- cache.m4.large
- cache.m4.xlarge
- cache.m4.2xlarge
- cache.m4.4xlarge
- cache.m4.10xlarge
- cache.r4.large
- cache.r4.xlarge
- cache.r4.2xlarge
- cache.r4.4xlarge
- cache.r4.8xlarge
- cache.r4.16xlarge
CacheEngine:
Description: The underlying cache engine, either Redis or
Memcached
Type: String
Default: redis
ConstraintDescription: Node instance class not supported
AllowedValues:
- redis
- memcached
CacheNodeCount:
Description: Number of nodes in the cluster. Only used with
memcached engine, for redis this value will be set to 1.
Type: Number
MinValue: 1
MaxValue: 15
ConstraintDescription: Node count must be between 1 and 15
Default: 1
AutoMinorVersionUpgrade:
Description: Whether or not minor version upgrades to the cache
engine should be applied automatically during the maintenance
window.
Type: String
Default: true
AllowedValues:
- true
- false
Resources:
SecurityGroup:
Type: AWS::EC2::SecurityGroup::Id
Properties:
GroupDescription: ElastiCache Security Group
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 6379
ToPort: 6379
- IpProtocol: tcp
FromPort: 11211
ToPort: 11211
Tags:
-
Key: Name
Value: "App-SG"
ElastiCacheCluster:
Type: AWS::ElastiCache::CacheCluster
Properties:
AutoMinorVersionUpgrade: !Ref AutoMinorVersionUpgrade
Engine: !Ref CacheEngine
CacheNodeType: !Ref CacheNodeType
ClusterName : !Ref ClusterName
NumCacheNodes: !Ref CacheNodeCount
CacheSubnetGroupName: !Ref Subnet
VpcSecurityGroupIds: !Ref SecurityGroup
Tags:
- Key: Name
Value: ElastiCache-Redis
Also it will be helpful to review the entire template to avoid more errors.Main issues seem to appear from resource section.

This question is old and I'm sure OP found the resolution but for anyone else who may come across this question - I believe the issue is that ElastiCacheCluster.VpcSecurityGroupIds must be a list, whereas !Ref SecurityGroup will simply return the ID of the security group as a string.
In addition, since both resources are created by the same template, we should make sure that the security group will be created first, otherwise referencing it will cause an error when the Elasticache cluster is created.
Here's the ElastiCache::CacheCluster resource with a list as the value of VpcSecurityGroupIds and a dependency on the security group.
ElastiCacheCluster:
Type: AWS::ElastiCache::CacheCluster
Properties:
AutoMinorVersionUpgrade: !Ref AutoMinorVersionUpgrade
Engine: !Ref CacheEngine
CacheNodeType: !Ref CacheNodeType
ClusterName : !Ref ClusterName
NumCacheNodes: !Ref CacheNodeCount
CacheSubnetGroupName: !Ref Subnet
VpcSecurityGroupIds:
- !Ref SecurityGroup
Tags:
- Key: Name
Value: ElastiCache-Redis
DependsOn: SecurityGroup

I had the same error messages and landed here.
Turns out that in my case, the problem was an incorrect type.
I had AWS::IAM:Role instead of AWS::IAM::Role (1 : vs 2)

Related

How to use existing VPC in AWS CloudFormation template for new SecurityGroup

I am trying to EC2 instance (new), Security group (new) and VPC(existing). Here is my cloudformation template.
When I run the template in Stack, I got error as *"Value () for parameter groupId is invalid. The value cannot be empty"*. How to solve this?
Template:
Parameters:
VPCID:
Description: Name of an existing VPC
Type: AWS::EC2::VPC::Id
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.medium
AllowedValues:
- t2.medium
- t2.large
AccessLocation:
Description: The IP address range that can be used to access to the EC2 instances
Type: String
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SecurityGroups:
- !Ref 'InstanceSecurityGroup'
KeyName: !Ref 'KeyName'
ImageId: !Ref 'ImageId'
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPCID
GroupDescription: Enable SSH
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: !Ref 'AccessLocation'
SecurityGroups can only be used for default VPC. Since you are explicitly assigning VPCID to InstanceSecurityGroup, this will be considered as non-default, resulting in failed deployment.
You must use SecurityGroupIds (not SecurityGroups) in your case as your VPC use will be considered as non-default:
SecurityGroupIds:
- !GetAtt 'InstanceSecurityGroup.GroupId'
The error in EC2Instance resource in SecurityGroups attribute. SecurityGroups needs an array of GroupId but when you use !Ref InstanceSecurityGroup this returns ResourceId. So you need to use GetAtt instead to get GroupId.
Parameters:
VPCID:
Description: Name of an existing VPC
Type: AWS::EC2::VPC::Id
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.medium
AllowedValues:
- t2.medium
- t2.large
AccessLocation:
Description: The IP address range that can be used to access to the EC2 instances
Type: String
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SecurityGroups:
- !GetAtt InstanceSecurityGroup.GroupId
KeyName: !Ref 'KeyName'
ImageId: !Ref 'ImageId'
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref VPCID
GroupDescription: Enable SSH
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: !Ref 'AccessLocation'
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-security-group.html

I have a problem with multiple Security Groups in CloudFormation

I am trying to select two Security Groups, one is from parameters and other is a new one.
But getting below errors. How can I solve this?
AWSTemplateFormatVersion : 2010-09-09
Description: "simple web layer"
Parameters:
ImageId:
Description: 'web Layer'
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: ami-244333
Vpc:
Description: enter vpc
Type: List<AWS::EC2::VPC::Id>
Securitycab:
Type: String
Default: 0.0.0.0/0
Description: enter vpc ip range
Subnets:
Description: enter subnet
Type: List<AWS::EC2::Subnet::Id>
Default: subnet-384736383838
Securitygroup:
Description: enter sc
Type: List<AWS::EC2::SecurityGroup::Id>
Default: sg-237377383773873
Resources:
Lpsecurity:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: vpc-1a2b3c4d
GroupDescription: Sample target security group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: !Ref Securitycab
MyEC2Instance1:
Type: 'AWS::EC2::Instance'
Properties:
ImageId: !Ref ImageId
InstanceType: t2.large
VpcId: !Ref Vpc
SubnetId: !Select [ 0, !Ref Subnets ]
SecurityGroupIds:
- !Ref Securitygroup
!Ref Lpsecurity
Error : Unsolved Resource dependence [Securitygroup !Ref Lpsecurity] in the resource block of the template.
A list should have dashes before every element.
Here is the suggested code from your other I have a problem in cloud formation. Error when using Fn::Join with a parameter question:
SecurityGroupIds:
- !GetAtt Securitygroup.GroupId
- !GetAtt Lpsecurity.GroupId
Alternatively, you could possibly use:
SecurityGroupIds:
- !Ref Securitygroup
- !Ref Lpsecurity

Unable to create an aurora RDS DB cluster due to disparate security groups

I have been attempting to set up a cloud formation script to create a VPC hosting fragate containers and a aurora DB. When attempting to deploy my aurora script I receive the following.
The DB instance and EC2 security group are in different VPCs. The DB instance is in vpc-f0ec9d98 and the EC2 security group is in vpc-01c5e9bcdb87dc39c (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 7aa14530-d73c-4b27-a6d6-fcc8aea61d93)
I do not understand why this is the case as I am using the same security group created by my VPC script, my aurora script is as follows
Aurora
Description: Set up a serverles PostgreSQL cluster with a bastion host (using Aurora)
Parameters:
DatabaseName:
Type: String
EngineVersion:
Type: String
Default: 11.4
MasterUsername:
Type: String
Default: root
MasterUserPassword:
Type: String
Default: root
NoEcho: true
VpcId:
Type: AWS::EC2::VPC::Id
VpcSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
BastionImageId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: /aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-ebs
BastionKeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: EC2 key used to connect to the bastion host
DeletionProtection:
Type: String
Default: false
AllowedValues:
- true
- false
Resources:
Cluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora-postgresql
EngineVersion: !Ref EngineVersion
DatabaseName: !Ref DatabaseName
MasterUsername: !Ref MasterUsername
MasterUserPassword: !Ref MasterUserPassword
DBClusterIdentifier: !Ref AWS::StackName
BackupRetentionPeriod: 35
DeletionProtection: !Ref DeletionProtection
VpcSecurityGroupIds:
- !Ref VpcSecurityGroupId
BastionSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: !Sub Bastion for ${AWS::StackName}
SecurityGroupEgress:
- CidrIp: 0.0.0.0/0
FromPort: -1
ToPort: -1
IpProtocol: -1
- DestinationSecurityGroupId: !Ref VpcSecurityGroupId
IpProtocol: tcp
FromPort: 3306
ToPort: 3306
SecurityGroupIngress: []
VpcId: !Ref VpcId
Bastion:
Type: AWS::EC2::Instance
Properties:
DisableApiTermination: true
ImageId: !Ref BastionImageId
InstanceType: t2.nano
KeyName: !Ref BastionKeyName
Monitoring: false
SecurityGroupIds:
- !Ref VpcSecurityGroupId
- !Ref BastionSecurityGroup
UserData: !Base64 'yum install postgresql --assumeyes' # if this script does not work this line broke it
Outputs:
Host:
Value: !GetAtt Cluster.Endpoint.Address
Export:
Name: !Sub ${AWS::StackName}Host
Name:
Value: !Ref DatabaseName
Export:
Name: !Sub ${AWS::StackName}Name
BastionHost:
Value: !GetAtt Bastion.PublicDnsName
Export:
Name: !Sub ${AWS::StackName}BastionHost
BastionIp:
Value: !GetAtt Bastion.PublicIp
Export:
Name: !Sub ${AWS::StackName}BastionIp
BastionSecurityGroupId:
Value: !GetAtt BastionSecurityGroup.GroupId
Export:
Name: !Sub ${AWS::StackName}BastionSecurityGroupId
Without the inclusion of the DBSubnetGroupName property in the AWS::RDS::DBCluster resource, it looks like CloudFormation is attempting to launch the cluster in the default VPC. A DB subnet group allows you to specify a particular VPC when you create DB instances.
Try adding this property and referencing an associated subnet parameter/resource and the issue should be resolved.
Information about creating RDS instances within a VPC can be found in the RDS User Guide.

The DB instance and EC2 security group are in different VPCs, cloudFormation error

I want to automate the process of creating RDS. I would like to create RDS Aurora.
When deploying the application, stack cloudFormation is validated and I have an error:
An error occurred: DatabaseCluster - The DB instance and EC2 security group are in different VPCs.
Can you say what's wrong?
I followed this post Issue with creating a Postgres RDS in Cloudformation Template but this doesn't work.
Here is part of my serverless.yml file
resources:
Resources:
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
ServerlessRDS:
Type: AWS::RDS::DBInstance
Properties:
Engine: aurora
DBClusterIdentifier: !Ref "DatabaseCluster"
DBInstanceIdentifier: db-name-${opt:stage, self:provider.stage}
DBInstanceClass: db.t2.medium
VPCSecurityGroups:
- !Ref VpcSecurityGroup
DBSubnetGroupName: !Ref myDBSubnetGroup
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
ServerlessVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: "10.0.0.0/16"
myDBSubnetGroup:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "description"
SubnetIds:
- !Ref ServerlessSubnetA
- !Ref ServerlessSubnetB
ServerlessSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1b"
CidrBlock: "10.0.0.0/24"
ServerlessSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: ServerlessVPC
AvailabilityZone: "eu-west-1a"
CidrBlock: "10.0.1.0/24"
You need to add DBSubnetGroupName parameter to AWS::RDS::DBCluster Resource.
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: name${opt:stage, self:provider.stage}
Engine: aurora
MasterUsername: ${ssm:MasterUsername-${opt:stage, self:provider.stage}}
MasterUserPassword: ${ssm:MasterUserPassword-${opt:stage, self:provider.stage}}
Port: "3306"
VpcSecurityGroupIds:
- !Ref VpcSecurityGroup
DBSubnetGroupName:
Ref: myDBSubnetGroup
Also you may want to add explicit dependencies on ServerlessSubnetA and ServerlessSubnetB in VpcSecurityGroup Resource to kind of group resource creation by the service and to avoid any race conditions.
VpcSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- ServerlessSubnetA
- ServerlessSubnetB
Properties:
VpcId:
Ref: ServerlessVPC
GroupDescription: "Allow all traffic"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
I encountered this situation and found someone had changed the dBs VPC.
Someone had taken a snapshot and restored it to a different VPC and that obviously didn't have the Security Group needed.
This probably happened because the website talking to the database was running on a different VPC.
To resolve it, you need to make a tough decision, because at this point you may end up having to blow the whole thing away and if its part of a group of CFN templates then that could spell even more problems with domino effects.
It's probably worth you try restore the dB back to the original VPC (or change Subnet Group/VPC) where the CFN template thinks it lives and where the security group was created by the CFN template and then re-run the CFN template.
If this fails, for example the database version has recently been updated you might get stuck with:
Cannot upgrade postgres from 9.6.22 to 9.6.11
At this point, the only recourse I know of is to delete the CFN stack and re-run it.

CloudFormation: simple example

I would like to build EC2 behind ELB(Elastic Load Balancer).
What would be the yaml/json code of doing this?
If you are looking for sample templates in json/yaml with Cloud Formation designer, you can use this sample templates provided by AWS.
Below is a sample CF template for simple 1 EC2/1 ELB stack
AWSTemplateFormatVersion: '2010-09-09'
Description: '1 EC2 Instance and 1 ELB'
Parameters:
AppServer:
Description: Hostname of Server
Type: String
Default: ec2instance01
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[0-9a-zA-Z-]*'
ConstraintDescription: 'Must contain valid DNS characters, AD length limit.'
AMI:
Description: AMI to deploy AWSLinux Instances
Type: String
Default: ami-xxxxxxxx
InstanceType:
Description: Application EC2 instance type
Type: String
Default: t2.micro
AllowedValues:
- t2.micro
- t2.2xlarge
- m4.2xlarge
ConstraintDescription: Must be a valid EC2 instance type.
VPCID:
Description: Name of the VPC
Type: 'AWS::EC2::VPC::Id'
Default: vpc-xxxxxxxx
ConstraintDescription: Must be a valid VPC.
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: 'AWS::EC2::KeyPair::KeyName'
Default: XXXX-key
MinLength: '1'
MaxLength: '255'
AllowedPattern: '[\x20-\x7E]*'
ConstraintDescription: Must contain only ASCII characters.
SubnetIdPrivateEastC:
Description: Private subnet for confidential apps in us-east-1c
Type: 'AWS::EC2::Subnet::Id'
Default: subnet-xxxxxxxx
MinLength: '1'
MaxLength: '255'
ConstraintDescription: Must be a valid Private Subnet.
SubnetIdPrivateEastD:
Description: Private subnet for confidential apps in us-east-1d
Type: 'AWS::EC2::Subnet::Id'
Default: subnet-xxxxxxxx
MinLength: '1'
MaxLength: '255'
ConstraintDescription: Must be a valid Private Subnet.
InstanceProfile:
Description: Instance Profile Name
Type: String
Default: xxxx-role
MinLength: '0'
MaxLength: '255'
AllowedPattern: '[\x20-\x7E]*'
ConstraintDescription: Must contain a vailed instance profile name
RootVolumeSize:
Description: Size (GB) of root EBS volume for application instance
Type: Number
Default: '10'
MinValue: '10'
MaxValue: '1024'
SwapDisk:
Description: Size (GB) of application EBS volume for instance
Type: Number
Default: '2'
MinValue: '2'
MaxValue: '128'
SubnetAvailabilityZone:
Description: Availability Zone for subnet
Type: String
Default: us-east-1d
AllowedValues:
- us-east-1c
- us-east-1d
ConstraintDescription: Must be a valid Availability zone.
PrivateSubnets:
Type: List<AWS::EC2::Subnet::Id>
Description: 'Private subnet for the ELB in us-east-1c and us-east-1d'
Default: "subnet-xxxxxxxx,subnet-xxxxxxxx"
Resources:
ec2instance01:
Type: 'AWS::EC2::Instance'
Properties:
DisableApiTermination: 'true'
AvailabilityZone: us-east-1d
ImageId:
Ref: AMI
InstanceType:
Ref: InstanceType
KeyName:
Ref: KeyName
SecurityGroupIds:
- Ref: WebSG
IamInstanceProfile:
Ref: InstanceProfile
SubnetId:
Ref: SubnetIdPrivateEastD
#EbsOptimized: true
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
VolumeSize:
Ref: RootVolumeSize
VolumeType: gp2
- DeviceName: /dev/sds
Ebs:
VolumeSize:
Ref: SwapDisk
VolumeType: gp2
Tags:
- Key: Name
Value:
Ref: AppServer
UserData:
'Fn::Base64': !Sub |-
#!/bin/bash -v
yum update -y aws-cfn-bootstrap
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
date > /home/ec2-user/starttime
date > /home/ec2-user/stoptime
echo END
WebSG:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Web SG
VpcId:
Ref: VPCID
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 10.0.0.0/8
Tags:
- Key: Name
Value: web_sg
ElbSG:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: ELB SG
VpcId:
Ref: VPCID
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '443'
ToPort: '443'
CidrIp: '0.0.0.0/0'
Tags:
- Key: Name
Value: elb_sg
ElasticLoadBalancer:
Type: AWS::ElasticLoadBalancing::LoadBalancer
DependsOn:
- ec2instance01
Properties:
LoadBalancerName: elb_01
SecurityGroups:
- Ref: ElbSG
Subnets: !Ref PrivateSubnets
Scheme: internal
Instances:
- Ref: ec2instance01
Listeners:
- LoadBalancerPort: '80'
InstancePort: '8080'
InstanceProtocol: HTTP
Protocol: HTTP
AccessLoggingPolicy:
EmitInterval: '60'
Enabled: 'False'
S3BucketName: elb-logs
S3BucketPrefix: ELB
HealthCheck:
Target: TCP:8080
HealthyThreshold: '5'
UnhealthyThreshold: '10'
Interval: '30'
Timeout: '5'
ConnectionDrainingPolicy:
Enabled: true
Timeout: '60'
Tags:
- Key: Name
Value: ELB_Name
Outputs:
ElbDNS:
Description: ELB DNS
Value:
'Fn::GetAtt':
- ElasticLoadBalancer
- DNSName
AppServerPrivateIP:
Description: Private IP address of instance ec2instance01
Value:
'Fn::GetAtt':
- ec2instance01
- PrivateIp
It looks like you are using the CloudFormation template designer. When you use the designer, it generates a CloudFormation template for you. You can see this by selecting the Template tab towards the bottom-left of the screen. You are also given the choice of JSON or YAML.