AWS CloudFormation RDS Instance fails to create - cannot find DBCluster - amazon-web-services

I am playing around with cloudformation and running into an issue with DBSubnetGroup that I cannot get around. My goal is to build a simple setup:
a VPC with two subnets
an RDS database subnet group on those subnets
an RDS database cluster in that db subnet
a single RDS instance in that cluster
In cloudformation, I keep getting an error:
Could not find DB Cluster: MyRDSClusterId (Service: AmazonRDS; Status Code:
404; Error Code: DBClusterNotFoundFault; Request ID: ...)
Everything looks right to me, and cloudformation says my DBCluster was created correctly. What am I doing wrong here? Any insights into what I've done wrong would be greatly appreciated.
Here is my cloudformation template:
AWSTemplateFormatVersion: "2010-09-09"
Description: Stack with DBSubnetGroup, DBCluster, and one DBInstance
Resources:
MyAppVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 192.168.0.0/16
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
MyAppRDSSubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1a
VpcId: !Ref MyAppVPC
CidrBlock: 192.168.0.0/24
MapPublicIpOnLaunch: true
MyAppRDSSubnetB:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1b
VpcId: !Ref MyAppVPC
CidrBlock: 192.168.1.0/24
MapPublicIpOnLaunch: true
MyDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: My App DBSubnetGroup for RDS
SubnetIds:
- !Ref MyAppRDSSubnetA
- !Ref MyAppRDSSubnetB
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: MyDB
DBClusterIdentifier: MyRDSClusterId
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
MyRDSInstance:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref MyRDSCluster
DBInstanceClass: db.t2.small
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora

Some charcter in your DBClusterIdentifier name is in uppercase.
What cloudformation does is, it automatically convert all uppercase character to lowercase.
Now when you try to attach DBInstances under the DBCluster it is not able to find with the DBClusterIdentifier you have mentioned as it contains some uppercase.
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: MyDB
DBClusterIdentifier: MyRDSClusterId <- here it converts all string to lowercase
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
MyRDSInstance:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref MyRDSCluster <- here it does not, so mismatch in name
DBInstanceClass: db.t2.small
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
Solution is : give DBClusterIdentifier all in lowercase.
I hope you got your Answer :)

I removed the "DBClusterIdentifier" property from my cluster definition and suddenly everything worked. Hope this helps someone else one day.
Now it looks like:
MyRDSCluster:
Type: AWS::RDS::DBCluster
Properties:
BackupRetentionPeriod: 1
DatabaseName: My
DBSubnetGroupName: !Ref MyDBSubnetGroup
Engine: aurora
MasterUsername: exampleUsername
MasterUserPassword: examplePassword
Hope this helps someone else one day.
I have to say, I'm not sure I totally understand why this fixed the issue, and an explanation would be helpful (i have been searching for a while with no luck). Why can't I specify my own DBClusterIdentifier in this template?

Related

Serverless error - Value of property DBSubnetGroupName must be of type String

I am attempting to geneate an Aurora serverless database for an application using Serverless to build it.
I copied a CloudFormation template I found online for this portion of my setup. I reference the below resource from my Serverless.yml file (which I've excluded because it's very large).
Serverless fails with the message:
An error occurred: AuroraCluster - Value of property DBSubnetGroupName
must be of type String.
I am confused about this because, to my observation, DBSubnetGroupName is a string (which as you can see is set to the value of 'db-subnet-group').
I am new to both Serverless and CloudFormation, and am doing my best to piece a solution together for a product I am building. Any suggestion as to how to fix this problem would be much appreciated.
Below is my Serverless file. Many thanks in advance for any assistance!
Resources:
ServerlessSecurityGroup:
DependsOn: VPC
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SecurityGroup for Serverless Functions
VpcId:
Ref: VPC
PrivateSubnetA:
DependsOn: VPC
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: VPC
AvailabilityZone: "eu-west-1a"
CidrBlock: "10.0.1.0/24"
PrivateSubnetB:
DependsOn: VPC
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: VPC
AvailabilityZone: "eu-west-1b"
CidrBlock: "10.0.64.0/19"
AuroraSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Subnet group for Aurora Database
DBSubnetGroupName: "db-subnet-group"
SubnetIds:
- Ref: PrivateSubnetA
- Ref: PrivateSubnetB
AuroraCluster:
Type: AWS::RDS::DBCluster
DeletionPolicy: ${self:custom.deletion_policy}
Properties:
DBClusterIdentifier: ${self:custom.aurora_db_name}
MasterUsername: !Sub ${self:custom.aurora_db_name}
MasterUserPassword: asdfasdfasdf223
DatabaseName: "somename"
Engine: aurora
EngineMode: serverless
DBSubnetGroupName:
- Ref: AuroraSubnetGroup
VpcSecurityGroupIds:
- Ref: ServerlessSecurityGroup
EnableHttpEndpoint: true
ScalingConfiguration:
AutoPause: true
MinCapacity: 1
MaxCapacity: 2
SecondsUntilAutoPause: 3600
Outputs:
AuroraCluster:
Value:
Ref: AuroraCluster
In your AWS::RDS::DBCluster it should be:
DBSubnetGroupName: !Ref AuroraSubnetGroup
In your example, you're passing DBSubnetGroupName as an array, not a string:
DBSubnetGroupName:
- Ref: AuroraSubnetGroup
In order to pass it as a string, you should use this notation:
DBSubnetGroupName: Ref: AuroraSubnetGroup

Cloudformation Elasticache: "cannot be specified along with CacheSecurityGroupNames"

ERROR:
Properties [AZMode, CacheSubnetGroupName, SecurityGroupIds, PreferredAvailabilityZones, SnapshotArns, SnapshotRetentionLimit, SnapshotWindow, Tags] cannot be specified along with CacheSecurityGroupNames, please update your template to the latest version
Type: AWS::ElastiCache::CacheCluster
Properties:
Engine: redis
AZMode: single-az
EngineVersion: 5.0.6
CacheParameterGroupName: !Ref CacheParameterGroup
ClusterName: !Sub ${SystemName}-${Env}-elasticache
CacheNodeType: !Ref ElasticacheNodeType
NumCacheNodes: "1"
Port: 6379
PreferredMaintenanceWindow: !Ref MaintenanceWindow
PreferredAvailabilityZone: ap-northeast-1a
SnapshotRetentionLimit: !Ref SnapshotRetentionLimit
SnapshotWindow: !Ref SnapshotWindow
CacheSubnetGroupName: !Ref CacheSubnetGroup
CacheSecurityGroupNames:
- Fn::ImportValue: DevElasticacheSG
Whats the reason for above error in cloudformation?
CacheSecurityGroupNames can only be used when your cluster is not in VPC:
Use this parameter only when you are creating a cluster outside of an Amazon Virtual Private Cloud (Amazon VPC).
My guess is that maybe you wanted to use VpcSecurityGroupIds instead.

CloudFormation Template Error when placing EC2 Instance in particular subnet

Working on a CF example template and am attempting to place an EC2 instance into a subnet created from the same template. When doing things as is, I get this error:
The parameter groupName cannot be used with the parameter subnet
Here's a subset of the template:
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 172.34.0.0/16
EnableDnsHostnames: False
Tags:
Subnet1:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-west-1a
CidrBlock: 172.34.0.0/24
VpcId:
Ref: VPC
MapPublicIpOnLaunch: False
WebServer1:
Type: AWS::EC2::Instance
Properties:
ImageId: !FindInMap [RegionMap, !Ref "AWS::Region", AMI]
InstanceType: t2.micro
SubnetId:
Ref: Subnet1
SecurityGroups:
- Ref: ServerSecurityGroup
Tags:
- Key: "Name"
Value: "Tyler-Cloudformation"
How can I refer to Subnet 1 when creating WebServer1 and place it within that subnet? What am I missing here? Thanks in advance!
The error is probably because you are using SecurityGroups rather then SecurityGroupIds. The former parameter is only used for default-VPCs, while for non-default ones, the latter must be used.
Thus, instead of SecurityGroups you should have:
SecurityGroupIds:
- !GetAtt ServerSecurityGroup.GroupId

Log Retention for Aurora Serverless in CloudFormation

I have a Aurora Cluster (Serverless - PostgreSQL) setup in CloudFormation and I want to configure the Log Retention to be around 7 days but I haven't been able to find where to set this setting.
This is my CloudFormation definition for the DBCluster:
AuroraDBCluster:
Type: AWS::RDS::DBCluster
DeletionPolicy: Delete
UpdateReplacePolicy: Delete
Properties:
Engine: aurora-postgresql
EngineMode: serverless
EngineVersion: 10.7
DatabaseName: test-db
DeletionProtection: false
ScalingConfiguration:
AutoPause: True
MaxCapacity: 8
MinCapacity: 2
SecondsUntilAutoPause: 300
VpcSecurityGroupIds:
- !Ref AuroraClusterSecurityGroup
Port: !Ref DBPort
MasterUsername:
!Join ['', ['{{resolve:secretsmanager:', !Ref AuroraMasterSecret, ':SecretString:username}}' ]]
MasterUserPassword:
!Join ['', ['{{resolve:secretsmanager:', !Ref AuroraMasterSecret, ':SecretString:password}}' ]]
DBSubnetGroupName: !Ref DBSubnetGroup
BackupRetentionPeriod: 35
DBClusterParameterGroupName: !Ref RDSDBClusterParameterGroup
StorageEncrypted: true
KmsKeyId: !Ref AuroraKMSCMK
I have created a different LogGroup like this:
AuroraClusterLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
RetentionInDays: 7
LogGroupName: !Join ["", ["/aws/rds/cluster/", !Ref AuroraDBCluster, /postgresql]]
But when I deploy the stack, it says:
CREATE_FAILED AWS::Logs::LogGroup AuroraClusterLogGroup /aws/rds/cluster/aurora-serverless-db-ufeihfhef74465/postgresql already exists
Because (I think) the AuroraDBCluster resource creates its own LogGroup with the same name.
I have reviewed the AWS::RDS::DBCluster documentation but I have not found any references for the Log Retention.
What can I do in this case?
Thanks!
If Aurora has already created its own log group, you can't change it from CloudFormation. The only way to do this would be use custom resource in your template.
In the custom resource you could use put-retention-policy to modify the retention time of chosen log group.

Unable to create a CoudFormation RDS template which provision new instance from point in time snapshots

I've trying to write a CloudFormation to provision a new RDS instance from the point in time snapshot an existing RDS DB.
However, I came to know that you can't specify db-name when you provide a snapshot in CloudFormation template and thus it will always restore it to the original DB.
I have got this article for the same on aws blogs, though I'm looking if there is any out of the box solution for the same.
Edit 1
RDS snippet from my Cloud Formation
Resources:
MyDB:
Type: AWS::RDS::DBInstance
Properties:
DBName: Fn::If ["UseDbSnapshot", !Ref AWS:NoValue, !Ref MyDBName]
DBSecurityGroups:
- !Ref MyDbSecurityByEC2SecurityGroup
- !Ref MyDbSecurityByCIDRIPGroup
AllocatedStorage: 20
DBInstanceClass: db.m1.small
Engine: MySQL
MasterUsername: root
MasterUserPassword: password
DBSnapshotIdentifier: Fn::If ["UseDbSnapshot", !Ref DBSnapshotIdentifier, !Ref AWS::NoValue]
DeletionPolicy: Snapshot
What can I try to fix this?
You have to keep in mind, that a couple of RDS properties (such as MasterUsername) are not customizable when you want to restore from snapshot. The AWS documentation says:
If you specify the SourceDBInstanceIdentifier or DBSnapshotIdentifier property, don't specify this property. The value is inherited from the source DB instance or snapshot.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html
So just omit those parameters like this:
Parameters:
DBSnapshotIdentifier:
Description: 'Optional identifier for the DB cluster snapshot from which you want to restore'
Type: String
Default: ''
Conditions:
HasDBSnapshotIdentifier: !Not [!Equals [!Ref DBSnapshotIdentifier, '']]
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Properties:
...
DBSnapshotIdentifier: !If [HasDBSnapshotIdentifier, !Ref DBSnapshotIdentifier, !Ref 'AWS::NoValue']
KmsKeyId: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !If [HasEncryption, !Ref KmsKey, !Ref 'AWS::NoValue']]
MasterUsername: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !Ref MasterUsername]
MasterUserPassword: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !Ref MasterPassword]
StorageEncrypted: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !If
...