CloudFormation fails to find RDS Subnet Group because of lowercase - amazon-web-services

I created my RDS Subnet Group via CloudFormation referencing a parameter ProjectName
DB:
Type: AWS::RDS::DBInstance
Properties:
DBSubnetGroupName: !Ref RDSSubnetGroup
Problem now is CloudFormation says it cannot find my subnet group:
DB subnet group 'AbcDef' does not exist because its actually abcdef ... how can I resolve this?
I tried looking for a toLower function but seems like theres none?
The other option appears to be recreate the stack?

Unfortunately everything you do in CloudFormation templates is case-sensitive including property names and parameter values. You may have to recreate the stack.
As you correctly pointed out, there is no Fn::ToLower function. If you really want to achieve what you are trying to, the only way to do it as of now is create Lambda backed custom resource which basically will convert your string to lower case and return it but it is not worth doing it as there are plenty of challenges you will come across when dealing with custom resources.

I have also found that DB Subnet Groups have their name forcibly changed to lowercase when viewed in the RDS console. Very unusual behavior.
However, I have created them in CloudFormation and it has not caused the error you describe. Here are the bits from my CloudFormation template:
###########
# DB Subnet Group
###########
DBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Lab DB Subnet Group
DBSubnetGroupName: Lab DB Subnet Group
SubnetIds:
- !Ref PrivateSubnet1
- !Ref PrivateSubnet2
Tags:
-
Key: Name
Value: DBSubnetGroup
###########
# RDS Database
###########
RDSDatabase:
Type: AWS::RDS::DBInstance
Properties:
DBName: inventory
DBInstanceIdentifier: inventory-db
AllocatedStorage: 5
DBInstanceClass: db.t2.micro
Engine: MySQL
MasterUsername: master
MasterUserPassword: lab-password
MultiAZ: false
DBSubnetGroupName: !Ref DBSubnetGroup
VPCSecurityGroups:
- !Ref DBSecurityGroup
Tags:
-
Key: Name
Value: inventory-db

I would suggest rewriting the function_name and the name of the DBSubnetGroup to dbsubnetgroup
This will fix the issue I suppose.

Had the same issue, tried all possible , the only way to fix , was to create a new DB subnet group with the name lower :
rdssubnetgrouplower:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "Private subnet group to keep the cluster private"
DBSubnetGroupName: rdssubnetgrouplower
SubnetIds:
- !Ref PrivateSubnet1
- !Ref PrivateSubnet2
Tags:
-
Key: Name
Value: rdssubnetgrouplower
and then in RDS definition used it :
MySQLABC:
Type: "AWS::RDS::DBCluster"
Properties:
DBSubnetGroupName: !Ref rdssubnetgrouplower
...
...
this worked and had the cluster up. in TF there is the lower function :)

Related

Review needed to correct this ElastiCache CloudFormation Template

This template is used to create a ElastiCache- Redis cluster.
Its showing me errors like - 1 validation error detected: Value '[AWS::ElastiCache::CacheCluster, AWS::EC2::SecurityGroup::Id]' at 'typeNameList' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 204, Member must have length greater than or equal to 10, Member must satisfy regular expression pattern: [A-Za-z0-9]{2,64}::[A-Za-z0-9]{2,64}::[A-Za-z0-9]{2,64}(::MODULE){0,1}].
Wanted to know if the parameters are declared rightly or not.
AWSTemplateFormatVersion: 2010-09-09
Description: Create ElastiCache and related resources
Parameters:
VPC:
Description: VPC
Type: AWS::EC2::VPC::Id
Subnet:
Description: Subnet
Type: AWS::EC2::Subnet::Id
ClusterName:
Description: Custom name of the cluster. Auto generated if you
don't supply your own.
Type: String
CacheNodeType:
Description: Cache node instance class, e.g. cache.t2.micro.
Type: String
Default: cache.t2.micro
ConstraintDescription: Node instance class not supported
AllowedValues:
- cache.t2.micro
- cache.t2.small
- cache.t2.medium
- cache.m4.large
- cache.m4.xlarge
- cache.m4.2xlarge
- cache.m4.4xlarge
- cache.m4.10xlarge
- cache.r4.large
- cache.r4.xlarge
- cache.r4.2xlarge
- cache.r4.4xlarge
- cache.r4.8xlarge
- cache.r4.16xlarge
CacheEngine:
Description: The underlying cache engine, either Redis or
Memcached
Type: String
Default: redis
ConstraintDescription: Node instance class not supported
AllowedValues:
- redis
- memcached
CacheNodeCount:
Description: Number of nodes in the cluster. Only used with
memcached engine, for redis this value will be set to 1.
Type: Number
MinValue: 1
MaxValue: 15
ConstraintDescription: Node count must be between 1 and 15
Default: 1
AutoMinorVersionUpgrade:
Description: Whether or not minor version upgrades to the cache
engine should be applied automatically during the maintenance
window.
Type: String
Default: true
AllowedValues:
- true
- false
Resources:
SecurityGroup:
Type: AWS::EC2::SecurityGroup::Id
Properties:
GroupDescription: ElastiCache Security Group
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 6379
ToPort: 6379
- IpProtocol: tcp
FromPort: 11211
ToPort: 11211
Tags:
-
Key: Name
Value: "App-SG"
ElastiCacheCluster:
Type: AWS::ElastiCache::CacheCluster
Properties:
AutoMinorVersionUpgrade: !Ref AutoMinorVersionUpgrade
Engine: !Ref CacheEngine
CacheNodeType: !Ref CacheNodeType
ClusterName : !Ref ClusterName
NumCacheNodes: !Ref CacheNodeCount
CacheSubnetGroupName: !Ref Subnet
VpcSecurityGroupIds: !Ref SecurityGroup
Tags:
- Key: Name
Value: ElastiCache-Redis
Also it will be helpful to review the entire template to avoid more errors.Main issues seem to appear from resource section.
This question is old and I'm sure OP found the resolution but for anyone else who may come across this question - I believe the issue is that ElastiCacheCluster.VpcSecurityGroupIds must be a list, whereas !Ref SecurityGroup will simply return the ID of the security group as a string.
In addition, since both resources are created by the same template, we should make sure that the security group will be created first, otherwise referencing it will cause an error when the Elasticache cluster is created.
Here's the ElastiCache::CacheCluster resource with a list as the value of VpcSecurityGroupIds and a dependency on the security group.
ElastiCacheCluster:
Type: AWS::ElastiCache::CacheCluster
Properties:
AutoMinorVersionUpgrade: !Ref AutoMinorVersionUpgrade
Engine: !Ref CacheEngine
CacheNodeType: !Ref CacheNodeType
ClusterName : !Ref ClusterName
NumCacheNodes: !Ref CacheNodeCount
CacheSubnetGroupName: !Ref Subnet
VpcSecurityGroupIds:
- !Ref SecurityGroup
Tags:
- Key: Name
Value: ElastiCache-Redis
DependsOn: SecurityGroup
I had the same error messages and landed here.
Turns out that in my case, the problem was an incorrect type.
I had AWS::IAM:Role instead of AWS::IAM::Role (1 : vs 2)

RDS Proxy Target groups Unavailable

I have just created RDS Proxy by Cloud Formation
In Proxies dashboard, it showed RDS proxy is available, but Target groups are unavailable, I can't debug this and got stuck in Cloud Formation update state
This is my Cloud formation config,
I used all in-out bound traffic security group for both rds proxy and rds instance, but it doesn't seem to work...
So do I have any wrong config? I have stuck at this all day
RDSInstance:
DependsOn: DBSecurityGroup
Type: AWS::RDS::DBInstance
Properties:
AllocatedStorage: '20'
AllowMajorVersionUpgrade: false
AutoMinorVersionUpgrade: true
AvailabilityZone: ${self:provider.region}a
DBInstanceClass: db.t2.micro
DBName: mydb
VPCSecurityGroups:
- "Fn::GetAtt": [ DBSecurityGroup, GroupId ]
Engine: postgres
EngineVersion: '11.9'
MasterUsername: postgres
MasterUserPassword: Fighting001
PubliclyAccessible: true
DBSubnetGroupName:
Ref: DBSubnetGroup
# VPCSecurityGroups:
# Ref: VPC
DBSecretsManager:
Type: AWS::SecretsManager::Secret
Properties:
Description: 'Secret Store for database connection'
Name: postgres
SecretString:
'password'
RDSProxy:
DependsOn: DBSecurityGroup
Type: AWS::RDS::DBProxy
Properties:
Auth:
- AuthScheme: SECRETS
SecretArn:
Ref: DBSecretsManager
IAMAuth: DISABLED
DBProxyName: ${self:provider.stackName}-db-proxy
DebugLogging: true
EngineFamily: 'POSTGRESQL'
RoleArn: 'my role arn'
VpcSecurityGroupIds:
- "Fn::GetAtt": [ DBSecurityGroup, GroupId ]
VpcSubnetIds:
- Ref: PublicSubnetA
- Ref: PublicSubnetB
RDSProxyTargetGroup:
Type: AWS::RDS::DBProxyTargetGroup
Properties:
DBProxyName:
Ref: RDSProxy
DBInstanceIdentifiers: [Ref: RDSInstance]
TargetGroupName: "default"
ConnectionPoolConfigurationInfo:
MaxConnectionsPercent: 45
MaxIdleConnectionsPercent: 40
ConnectionBorrowTimeout: 120
A likely reason why your template fails is that your AWS::SecretsManager::Secret is not used and has incorrect values.
Your DB uses:
MasterUsername: postgres
MasterUserPassword: Fighting001
But your DBSecretsManager is:
SecretString:
'password'
which is incorrect. I would suggest setting up manually everything in the AWS console first. Then you can check what is the correct form of the SecretString for your use-case.
While this isnt the cause of the original issue mentioned above, it may help someone who reaches this post in future.
Make sure your RDS instance and the security group associated with it are using the same port.
I experienced the same outcome because my RDS security group was configured using a different port than the RDS instance.
By default, Aurora Postgres will use port 3306, however my security group was using 5432 (because it was copied from an old Postgres non-Aurora RDS instance). I updated my RDS instance to use port 5432 by specifying the Port property which resolved this issue.

CloudFormation - ReplicationGroup - Specify the name

I'm create a CloudFormation template to create Redis in ElastiCache Cluster, everything works fine.
However I'm not able to modify the name of the cluster.
Here is my code:
Resources:
Redis:
Type: 'AWS::ElastiCache::ReplicationGroup'
Properties:
ReplicationGroupDescription: RedisCluster
NumCacheClusters: '2'
Engine: redis
MultiAZEnabled: yes
CacheNodeType: cache.t2.micro
AutomaticFailoverEnabled: 'true'
CacheSubnetGroupName: !Ref RedisSubnetGroup
EngineVersion: 5.0.6
PreferredMaintenanceWindow: mon:03:00-mon:04:00
Tags:
- Key: Cluster Name
Value: "Redis"
- Key: Name
Value: "Redis"
SecurityGroupIds:
- !Ref RedisSecurityGroup
It's look like the tags doesn't work in this case.
What is the right way to specify the name for the cluster ?
The name is set using ReplicationGroupId:
The replication group identifier. This parameter is stored as a lowercase string.

Unable to create autoscalling group for application load balancer using aws cloud formation

AWS CFN build is failing with status "Value of property TargetGroupARNs must be of type List of String" for the following autoscalling group:
InfyASG:
Type: AWS::AutoScaling::AutoScalingGroup
UpdatePolicy:
AutoScalingRollingUpdate:
MaxBatchSize: "4"
MinInstancesInService:
Ref: InfyASGMin
Properties:
Cooldown: "300"
DesiredCapacity:
Ref: InfyASGDesiredSize
MaxSize:
Ref: InfyASGMaxSize
MinSize:
Ref: InfyASGMin
HealthCheckGracePeriod: "300"
HealthCheckType: ELB
VPCZoneIdentifier:
Ref: PrivateSubnet
LaunchConfigurationName:
Ref: InfyLaunchConfig
TargetGroupARNs:
- !GetAtt "InfyTG.LoadBalancerArns"
Tags:
- Key: Owner
Value:
Ref: BaseOwner
PropagateAtLaunch: true
- Key: Name
Value:
Fn::Sub: ${BaseName}-${Environment}-InfyASG
PropagateAtLaunch: true
- Key: Application
Value:
Ref: Application
PropagateAtLaunch: true
- Key: Environment
Value:
Ref: Environment
PropagateAtLaunch: true
- Key: Role
Value:
Fn::Sub: ${BaseName}-${Environment}-Role
PropagateAtLaunch: true
My target group name is "InfyTG". Only "TargetGroupARNs" was given under autoscaling group for mapping. I also have used the following, but that did not work.
TargetGroupARNs:
Ref: InfyTG
Should be problem mapping targetgroup to autoscaling group.
I try searching how the mapping will be between autoscaling group, target group, application load balancer but it didn't helped me fixing the above problem.
All I need is highly available application server under a target group so that I can route the traffic through application load balancer to the instance. If the instance is down, autoscalling group should generate new one and register it under the same target group.
Kindly help me with this.
Try this:
TargetGroupARNs: [!Ref InfyTG]
Alternatively (and you were close with your second form):
TargetGroupARNs:
- !Ref InfyTG

AWS CloudFormation RDS Instance fails to create - cannot find DBCluster

I am playing around with cloudformation and running into an issue with DBSubnetGroup that I cannot get around. My goal is to build a simple setup:
a VPC with two subnets
an RDS database subnet group on those subnets
an RDS database cluster in that db subnet
a single RDS instance in that cluster
In cloudformation, I keep getting an error:
Could not find DB Cluster: MyRDSClusterId (Service: AmazonRDS; Status Code:
404; Error Code: DBClusterNotFoundFault; Request ID: ...)
Everything looks right to me, and cloudformation says my DBCluster was created correctly. What am I doing wrong here? Any insights into what I've done wrong would be greatly appreciated.
Here is my cloudformation template:
AWSTemplateFormatVersion: "2010-09-09"
Description: Stack with DBSubnetGroup, DBCluster, and one DBInstance
Resources:
MyAppVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 192.168.0.0/16
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
MyAppRDSSubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1a
VpcId: !Ref MyAppVPC
CidrBlock: 192.168.0.0/24
MapPublicIpOnLaunch: true
MyAppRDSSubnetB:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1b
VpcId: !Ref MyAppVPC
CidrBlock: 192.168.1.0/24
MapPublicIpOnLaunch: true
MyDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: My App DBSubnetGroup for RDS
SubnetIds:
- !Ref MyAppRDSSubnetA
- !Ref MyAppRDSSubnetB
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: MyDB
DBClusterIdentifier: MyRDSClusterId
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
MyRDSInstance:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref MyRDSCluster
DBInstanceClass: db.t2.small
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
Some charcter in your DBClusterIdentifier name is in uppercase.
What cloudformation does is, it automatically convert all uppercase character to lowercase.
Now when you try to attach DBInstances under the DBCluster it is not able to find with the DBClusterIdentifier you have mentioned as it contains some uppercase.
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: MyDB
DBClusterIdentifier: MyRDSClusterId <- here it converts all string to lowercase
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
MyRDSInstance:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref MyRDSCluster <- here it does not, so mismatch in name
DBInstanceClass: db.t2.small
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
Solution is : give DBClusterIdentifier all in lowercase.
I hope you got your Answer :)
I removed the "DBClusterIdentifier" property from my cluster definition and suddenly everything worked. Hope this helps someone else one day.
Now it looks like:
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: My
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
Hope this helps someone else one day.
I have to say, I'm not sure I totally understand why this fixed the issue, and an explanation would be helpful (i have been searching for a while with no luck). Why can't I specify my own DBClusterIdentifier in this template?