I am trying to create a Document Db connection in Glue Cloudformation with secret manager dynamic reference
({{resolve:secretsmanager:${DBSecretManagerArn}:SecretString:username}})
During build there is no evaluation failure but before the job finishes it failed - An error occurred (ValidationError) when calling the UpdateStack operation: Given input did not match expected format. I haven't found much documentation online. Can someone please guide if you have faced such issues.
Some-Name:
Type: AWS::Glue::Connection
Properties:
CatalogId: !Ref AWS::AccountId
ConnectionInput:
Name: connection
Description: "Document DB database."
ConnectionType: MONGODB
ConnectionProperties:
CONNECTION_URL: !Ref ConnectionUri
USERNAME: !Sub '{{resolve:secretsmanager:${DBSecretManagerArn}:SecretString:username}}'
PASSWORD: !Ref Password
JDBC_ENFORCE_SSL: false
PhysicalConnectionRequirements:
AvailabilityZone: !Select
- 0
- Fn::GetAZs: !Ref 'AWS::Region'
SecurityGroupIdList:
- sg-some
SubnetId: !ImportValue some-value
Related
I want to create a simple rds proxy. I use the attached cloudformation template. However aws cannot create resource "AWS::RDS::DBProxyTargetGroup". The error info is not enough for debugging: "Resource timed out waiting for completion". Please anyone provide me anwser?
Target group was created, but info was not updated in cloudformation event
rds_proxy_console
CF_event
failed event
Resources:
RDSProxy:
Type: "AWS::RDS::DBProxy"
Properties:
Auth:
- AuthScheme: SECRETS
IAMAuth: DISABLED
SecretArn: !Sub "arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:${SecretsManagerName}"
DBProxyName: !Ref ProxyName
EngineFamily: !Ref ProxyEngineFamily
RoleArn: !GetAtt SecretsManagerRole.Arn
VpcSecurityGroupIds: !Ref ProxyVpcSecurityGroupIds
VpcSubnetIds: !Ref ProxyVpcSubnetIds
RDSProxyTargetGroup:
Type: "AWS::RDS::DBProxyTargetGroup"
Properties:
DBClusterIdentifiers: !Ref ProxyTargetDBClusterIdentifiers
DBProxyName: !Ref RDSProxy
TargetGroupName: default
I am trying to create the glue security configuration using cloudformation script but I am getting the following error:
Property validation failure: [Value of property {/EncryptionConfiguration/S3Encryptions} does not match type {Array}]
What is the right way to give the S3encryption?
AWSTemplateFormatVersion: 2010-09-09
Description: Script creates resources for GlueSecurityConfiguration
Resources:
GlueSecurityConfiguration:
Type: AWS::Glue::SecurityConfiguration
Properties:
EncryptionConfiguration:
S3Encryptions:
KmsKeyArn: !Ref KMSArn
S3EncryptionMode: SSE-KMS
JobBookmarksEncryption:
KmsKeyArn: !Ref KMSArn
CloudWatchEncryption:
KmsKeyArn: !Ref KMSArn
Name: !Sub '${SystemValue}-${SubSystemValue}'
I think it should be
- KmsKeyArn: !Ref KMSArn
S3EncryptionMode: SSE-KMS
since S3Encryptions expects an array.
I have a Aurora Cluster (Serverless - PostgreSQL) setup in CloudFormation and I want to configure the Log Retention to be around 7 days but I haven't been able to find where to set this setting.
This is my CloudFormation definition for the DBCluster:
AuroraDBCluster:
Type: AWS::RDS::DBCluster
DeletionPolicy: Delete
UpdateReplacePolicy: Delete
Properties:
Engine: aurora-postgresql
EngineMode: serverless
EngineVersion: 10.7
DatabaseName: test-db
DeletionProtection: false
ScalingConfiguration:
AutoPause: True
MaxCapacity: 8
MinCapacity: 2
SecondsUntilAutoPause: 300
VpcSecurityGroupIds:
- !Ref AuroraClusterSecurityGroup
Port: !Ref DBPort
MasterUsername:
!Join ['', ['{{resolve:secretsmanager:', !Ref AuroraMasterSecret, ':SecretString:username}}' ]]
MasterUserPassword:
!Join ['', ['{{resolve:secretsmanager:', !Ref AuroraMasterSecret, ':SecretString:password}}' ]]
DBSubnetGroupName: !Ref DBSubnetGroup
BackupRetentionPeriod: 35
DBClusterParameterGroupName: !Ref RDSDBClusterParameterGroup
StorageEncrypted: true
KmsKeyId: !Ref AuroraKMSCMK
I have created a different LogGroup like this:
AuroraClusterLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
RetentionInDays: 7
LogGroupName: !Join ["", ["/aws/rds/cluster/", !Ref AuroraDBCluster, /postgresql]]
But when I deploy the stack, it says:
CREATE_FAILED AWS::Logs::LogGroup AuroraClusterLogGroup /aws/rds/cluster/aurora-serverless-db-ufeihfhef74465/postgresql already exists
Because (I think) the AuroraDBCluster resource creates its own LogGroup with the same name.
I have reviewed the AWS::RDS::DBCluster documentation but I have not found any references for the Log Retention.
What can I do in this case?
Thanks!
If Aurora has already created its own log group, you can't change it from CloudFormation. The only way to do this would be use custom resource in your template.
In the custom resource you could use put-retention-policy to modify the retention time of chosen log group.
I am trying to create a database on glue using cloud formation but it fails with the below error. Am I missing something?
Property validation failure: [The property {/DatabaseInput} is required, The property {/CatalogId} is required]
This is how my template code block looks like
GlueDatabase:
Type: AWS::Glue::Database
Properties:
CatalogId: !Ref AWS::AccountId
DatabaseInput: !Ref TeamName
According to the docs the DatabaseInput should have the following structure:
GlueDatabase:
Type: AWS::Glue::Database
Properties:
CatalogId: !Ref AWS::AccountId
DatabaseInput:
Description: String
LocationUri: String
Name: String
Parameters: Json
Thus the question is, what TeamName is in your tempalte?
I've trying to write a CloudFormation to provision a new RDS instance from the point in time snapshot an existing RDS DB.
However, I came to know that you can't specify db-name when you provide a snapshot in CloudFormation template and thus it will always restore it to the original DB.
I have got this article for the same on aws blogs, though I'm looking if there is any out of the box solution for the same.
Edit 1
RDS snippet from my Cloud Formation
Resources:
MyDB:
Type: AWS::RDS::DBInstance
Properties:
DBName: Fn::If ["UseDbSnapshot", !Ref AWS:NoValue, !Ref MyDBName]
DBSecurityGroups:
- !Ref MyDbSecurityByEC2SecurityGroup
- !Ref MyDbSecurityByCIDRIPGroup
AllocatedStorage: 20
DBInstanceClass: db.m1.small
Engine: MySQL
MasterUsername: root
MasterUserPassword: password
DBSnapshotIdentifier: Fn::If ["UseDbSnapshot", !Ref DBSnapshotIdentifier, !Ref AWS::NoValue]
DeletionPolicy: Snapshot
What can I try to fix this?
You have to keep in mind, that a couple of RDS properties (such as MasterUsername) are not customizable when you want to restore from snapshot. The AWS documentation says:
If you specify the SourceDBInstanceIdentifier or DBSnapshotIdentifier property, don't specify this property. The value is inherited from the source DB instance or snapshot.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html
So just omit those parameters like this:
Parameters:
DBSnapshotIdentifier:
Description: 'Optional identifier for the DB cluster snapshot from which you want to restore'
Type: String
Default: ''
Conditions:
HasDBSnapshotIdentifier: !Not [!Equals [!Ref DBSnapshotIdentifier, '']]
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
Properties:
...
DBSnapshotIdentifier: !If [HasDBSnapshotIdentifier, !Ref DBSnapshotIdentifier, !Ref 'AWS::NoValue']
KmsKeyId: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !If [HasEncryption, !Ref KmsKey, !Ref 'AWS::NoValue']]
MasterUsername: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !Ref MasterUsername]
MasterUserPassword: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !Ref MasterPassword]
StorageEncrypted: !If [HasDBSnapshotIdentifier, !Ref 'AWS::NoValue', !If
...