Unable to create aurora postgresSql database using cloudformat yaml template.
Please help me on this.
From AWS::RDS::DBCluster - AWS CloudFormation:
The following example creates an Amazon Aurora PostgreSQL DB cluster that exports logs to Amazon CloudWatch Logs. For more information about exporting Aurora DB cluster logs to Amazon CloudWatch Logs.
AWSTemplateFormatVersion: 2010-09-09
Description: >-
AWS CloudFormation Sample Template for sending Aurora DB cluster logs to
CloudWatch Logs: Sample template showing how to create an Aurora PostgreSQL DB
cluster that exports logs to CloudWatch Logs. **WARNING** This template
enables log exports to CloudWatch Logs. You will be billed for the AWS
resources used if you create a stack from this template.
Parameters:
DBUsername:
NoEcho: 'true'
Description: Username for PostgreSQL database access
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: must begin with a letter and contain only alphanumeric characters.
DBPassword:
NoEcho: 'true'
Description: Password for PostgreSQL database access
Type: String
MinLength: '8'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]*'
ConstraintDescription: must contain only alphanumeric characters.
Resources:
RDSCluster:
Type: 'AWS::RDS::DBCluster'
Properties:
MasterUsername: !Ref DBUsername
MasterUserPassword: !Ref DBPassword
DBClusterIdentifier: aurora-postgresql-cluster
Engine: aurora-postgresql
EngineVersion: '10.7'
DBClusterParameterGroupName: default.aurora-postgresql10
EnableCloudwatchLogsExports:
- postgresql
RDSDBInstance1:
Type: 'AWS::RDS::DBInstance'
Properties:
DBInstanceIdentifier: aurora-postgresql-instance1
Engine: aurora-postgresql
DBClusterIdentifier: !Ref RDSCluster
PubliclyAccessible: 'true'
DBInstanceClass: db.r4.large
RDSDBInstance2:
Type: 'AWS::RDS::DBInstance'
Properties:
DBInstanceIdentifier: aurora-postgresql-instance2
Engine: aurora-postgresql
DBClusterIdentifier: !Ref RDSCluster
PubliclyAccessible: 'true'
DBInstanceClass: db.r4.large
Related
Hi I am trying to create a cloudformation template to migrate data from postgres in ec2 to rds.I got this template from this link. https://hands-on.cloud/aws-cloudformation-how-to-create-dms-infrastructure-for-relational-db-migration/.
I modified it but i got a error
"Template format error: Every Parameters member must be an object".
In the link he created vpc and security group.i already have vpc and security to configure.
Can u anybod please
Parameters:
ReplicationInstanceAllocatedStorage:
Description: >
T he amount of storage (in gigabytes) to be initially allocated
for the replication instance.
Type: Number
Default: 256
ReplicationInstanceClass:
Description: >
The compute and memory capacity of the replication instance as specified
by the replication instance class.
Valid Values: dms.t2.micro | dms.t2.small | dms.t2.medium | dms.t2.large |
dms.c4.large | dms.c4.xlarge | dms.c4.2xlarge | dms.c4.4xlarge
Type: String
Default: dms.r5.xlarge
SrcDbName: postgres
Type: String
SrcDbEngine: postgres
Type: String
SrcDbServerName: postgres
Type: String
SrcDbPort: 5432
Type: Number
SrcDbUsername: postgres
Type: String
SrcDbPassword: postgres
Type: String
Resources:
ReplicationInstance:
Type: AWS::DMS::ReplicationInstance
Properties:
AllocatedStorage: !Ref ReplicationInstanceAllocatedStorage
AllowMajorVersionUpgrade: false
AutoMinorVersionUpgrade: false
MultiAZ: false
PubliclyAccessible: false
ReplicationInstanceClass: !Sub '${ReplicationInstanceClass}'
ReplicationInstanceIdentifier: !Sub '${AWS::StackName}-replication-instance'
DmsEndpointSource:
Type: AWS::DMS::Endpoint
Properties:
DatabaseName: !Ref SrcDbName
EndpointType: 'source'
EngineName: !Ref SrcDbEngine
ServerName: !Ref SrcDbServerName
Port: !Ref SrcDbPort
Username: !Ref SrcDbUsername
Password: !Ref SrcDbPassword
There's a couple of errors in your template.
SrcDbName: postgres
Type: String
and all others in that form are not correct. They should be
SrcDbName:
Description: Source Database Name
Type: String
Default: postgres
for example.
You'll also need your replication destination endpoint and a replication task (IIRC).
I am using cloudformation to provision RDS aurora to AWS and using AWS::RDS::DBCluster and AWS::RDS::DBInstance resources in the template. I have different environments, e.g. dev, uat and prod. Each environment has different number of db instances under the cluster. How can I set the number of db instances as a variable in the cloudformation template?
Below is my template for AWS::RDS::DBInstance. As you can see there are three instances in the template. It is only for production not dev. How can I use a parameter to indicate the number of instances? For example, deploy 1 instance in dev and 3 for prod.
AuroraDBFirstInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: ${self:provider.postgresqlInstanceClass}
Engine: aurora-postgresql
EngineVersion: ${self:provider.postgresqlEngineVersion}
DBClusterIdentifier: !Ref AuroraDBCluster
PubliclyAccessible: ${self:provider.publiclyAccessible}
AuroraDBSecondInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: ${self:provider.postgresqlInstanceClass}
Engine: aurora-postgresql
EngineVersion: ${self:provider.postgresqlEngineVersion}
DBClusterIdentifier: !Ref AuroraDBCluster
PubliclyAccessible: ${self:provider.publiclyAccessible}
AuroraDBThirdInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: ${self:provider.postgresqlInstanceClass}
Engine: aurora-postgresql
EngineVersion: ${self:provider.postgresqlEngineVersion}
DBClusterIdentifier: !Ref AuroraDBCluster
PubliclyAccessible: ${self:provider.publiclyAccessible}
You can add it as a parameter and pass as you run the stack, you can even make a mappings like this:
Environment:
Type: String
AllowedValues:
- dev
- uat
- prod
Mappings:
EnvironmentToDb
dev:
Cluster: 1
uat:
Cluster: 2
prod:
Cluster: 3
Then you can reference it using:
DBClusterIdentifier: !FindInMap [EnvironmentToDb, !Ref 'Environment', Cluster]
When creating an AWS::RDS::DBCluster(Aurora-Serverless) in AWS CloudFormation, there is a property MasterUserPassword and its input is specified as string.
So, how to put a stack definition yaml so that RDS definition does not use a string but a random password generated by AWS::SecretsManager::Secret? Is it possible to reference Secrets-manager-generated password with !Ref, !GetAtt or any other means?
Resources:
AuroraMysqlAppCredentialSecretStore:
Type: AWS::SecretsManager::Secret
Properties:
Name: AuroraMysqlAppCredentialSecretStore
GenerateSecretString:
SecretStringTemplate: '{"username": "admin"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludeCharacters: '"#/\'
ApprovalDbCluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora
EngineMode: serverless
EngineVersion: '5.6'
DatabaseName: MyDatabaseName
MasterUsername: admin
MasterUserPassword: # HOW TO REFERENCE THE PASSWORD HERE??
DBClusterIdentifier: my-cluster-1
BackupRetentionPeriod: 35
DeletionProtection: false
ScalingConfiguration:
AutoPause: true
MaxCapacity: 8
MinCapacity: 2
SecondsUntilAutoPause: 300
DBSubnetGroupName: my-subnet-name
A final note: Docs state that MasterUserPassword should not be specified if SourceDBInstanceIdentifier or DBSnapshotIdentifier property is given, but my configuration has neither, so apparently I should specify MasterUserPassword.
You can use dynamic references in cloudformation https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html
ApprovalDbCluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora
EngineMode: serverless
EngineVersion: '5.6'
DatabaseName: MyDatabaseName
MasterUsername: admin
MasterUserPassword: '{{resolve:ssm-secure:MasterPassword:10}}' #See link
I am trying to write Cloudformation template to get a aws Global Aurora Database. However I am not able to figure out where and how to add the Global database identifier. Can someone help Cloudformation snippet?
below is my code:
Description: RDS Aurora MySQL cluster.
Parameters:
DatabaseName:
Default: "testglobalaurora"
Description: The database name
Type: String
DatabaseInstanceType:
Default: db.r4.large
AllowedValues:
- db.r4.large
- db.r4.xlarge
- db.r4.2xlarge
- db.r4.4xlarge
- db.r4.8xlarge
- db.r4.16xlarge
Description: "The instance type to use for the database."
Type: String
DatabasePassword:
Default: "testglobalaurora"
AllowedPattern: "[a-zA-Z0-9]+"
ConstraintDescription: must contain only alphanumeric characters. Must have length 8-41.
Description: The database admin account password.
MaxLength: '41'
MinLength: '8'
NoEcho: 'true'
Type: String
DatabaseUsername:
Default: "testglobalaurora"
AllowedPattern: "[a-zA-Z0-9]+"
ConstraintDescription: must contain only alphanumeric characters. Must have length 1-16
Description: The database admin account user name.
MaxLength: '16'
MinLength: '1'
Type: String
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Database Configuration
Parameters:
- DatabaseInstanceType
- DatabaseName
- DatabaseUsername
- DatabasePassword
ParameterLabels:
DatabaseName:
default: Database name
DatabaseInstanceType:
default: Database Instance Type
DatabasePassword:
default: Database Password
DatabaseUsername:
default: Database Username
Resources:
ParameterGroup:
Type: "AWS::RDS::DBParameterGroup"
Properties:
Description: testglobalaurora DB parameter group
Family: aurora5.6
Parameters:
max_connections: 300
DatabaseCluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora
EngineMode: global
MasterUsername:
Ref: DatabaseUsername
MasterUserPassword:
Ref: DatabasePassword
BackupRetentionPeriod: 35
PreferredBackupWindow: 02:00-03:00
PreferredMaintenanceWindow: mon:03:00-mon:04:00
VpcSecurityGroupIds:
- Ref: DatabaseSecurityGroup
DatabaseInstance:
Type: AWS::RDS::DBInstance
Properties:
Engine: aurora
EngineVersion : 5.6.10a
DBClusterIdentifier:
Ref: DatabaseCluster
DBInstanceClass:
Ref: DatabaseInstanceType
DBParameterGroupName: !Ref ParameterGroup
PubliclyAccessible: "true"
DBInstanceIdentifier: !Ref DatabaseName
DatabaseSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: vpc-55378f2f
GroupDescription: Access to database
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: 3306
ToPort: 3306
IpProtocol: tcp
Tags:
- Key: Name
Value: !Sub ${DatabaseName}-security-group
Outputs:
DatabaseEndpoint:
Description: The database endpoint
Value: !GetAtt DatabaseCluster.Endpoint.Address
DatabasePort:
Description: The database port
Value: !GetAtt DatabaseCluster.Endpoint.Port
My output
"
global-database-1-cluster-1 Regional Aurora MySQL 5.6.10a
global-database-1-instance-1 Writer Aurora MySQL 5.6.10a
"
Actual ouput
"
test-it Global Aurora MySQL 5.6.10a
global-database-1-cluster-1 Primary Aurora MySQL 5.6.10a
global-database-1-instance-1 Writer Aurora MySQL 5.6.10a
"
I recently ran across the need to create a global RDS with Cloudformation. Here is a minimal Cloudformation that got me started.
AWSTemplateFormatVersion: "2010-09-09"
Description: Global RDS database stack
Parameters:
DatabaseInstanceType:
Default: db.r4.large
AllowedValues:
- db.r4.large
- db.r4. # add the other r4 instances
Description: "The instance type to use for the database."
Type: String
DatabasePassword:
Default: SomePassword1
AllowedPattern: "[a-zA-Z0-9]+"
ConstraintDescription: must contain only alphanumeric characters. Must have length 8-41.
Description: The database admin account password.
MaxLength: '41'
MinLength: '8'
NoEcho: 'true'
Type: String
DatabaseUsername:
Default: globaladmin
ConstraintDescription: must contain only alphanumeric characters. Must have length 1-16
Description: The database admin account user name.
MaxLength: '16'
MinLength: '1'
Type: String
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Database Configuration
Parameters:
- DatabaseInstanceType
- DatabaseName
- DatabaseUsername
- DatabasePassword
ParameterLabels:
DatabaseName:
default: Database name
DatabaseInstanceType:
default: Database Instance Type
DatabasePassword:
default: Database Password
DatabaseUsername:
default: Database Username
Resources:
GlobalDbCluster:
Type: AWS::RDS::DBCluster
Properties:
Engine: aurora
EngineMode: global
EngineVersion: 5.6.10a
MasterUsername: !Ref DatabaseUsername
MasterUserPassword: !Ref DatabasePassword
DBClusterParameterGroupName: !Ref GlobalDbParamGroup
GlobalDbParamGroup:
Type: AWS::RDS::DBClusterParameterGroup
Properties:
Description: "parameter group for the global database"
Family: aurora5.6
Parameters:
character_set_database: utf32
InstanceOne:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: !Ref DatabaseInstanceType
DBClusterIdentifier: !Ref GlobalDbCluster
Engine: aurora
InstanceTwo:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: !Ref DatabaseInstanceType
DBClusterIdentifier: !Ref GlobalDbCluster
Engine: aurora
You need to create a Global Cluster with an identifier, and use that identifier in your DB Cluster. That portion is missing in your CFN template.
Something like:
GlobalCluster:
Type: AWS::RDS::GlobalCluster
Properties:
Engine: aurora
EngineVersion: 5.6.10a
Region: us-east-1
and then use it in your DatabaseCluster properties using GlobalClusterIdentifier: <id>
However, looking at the official docs for the CFN types [1] for RDS, it does not list GlobalCluster. So either its just not documented or this Resource type has not been registered with Cloudformation. If the latter is the case, then you may want to open a support case and put in a feature request.
[1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_RDS.html
I am trying to create a read replica in west region for an RDS data base in east through cloud formation template.
I am getting an error:
Cannot create a cross region unencrypted read replica from encrypted source.
However, I have tried to provide kms key id and marked CopyTagsToSnapshot as true . Here is how my cloud formation looks like:
Resources:
MyDB:
Type: AWS::RDS::DBInstance
Properties:
SourceDBInstanceIdentifier: !Ref ReadReplicaURL
AllocatedStorage: !Ref DBAllocatedStorage
CopyTagsToSnapshot: true
DBSubnetGroupName: !Ref DBSubnetGroup
VPCSecurityGroups:
- !Ref DBSG1
KmsKeyId: !Ref DBEncryptionKey
StorageEncrypted: true
DBInstanceClass: !Ref DBInstanceClass
DBInstanceIdentifier: !Ref DBInstanceIdentifier
Iops: !Ref DBIops
MonitoringInterval: !Ref DBMonitoringInterval
Engine: !Ref Engine
MonitoringRoleArn: !Ref DBMonitoringRoleARN
Port: !Ref DBPort
PreferredMaintenanceWindow: !Ref DBPreferredMaintenanceWindow
StorageType: io1
Answer I got from AWS rep:
Unfortunately, creation of encrypted RDS cross-region read replicas is not possible through CloudFormation currently. There is an active feature request to implement this functionality to which I have added your voice. Once the feature is implemented, it will be announced on this page:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/ReleaseHistory.html