I want to set the Recrawl Policy while creating Glue Crawler through CloudFormation Stack.
Not able find way to set Recrawl policy through CloudFormation.
Please help on adding this configuration in this templateother wise let me know if it can't be done.
Parameters:
CFNCrawlerName:
Type: String
Default: cfn-crawler-flights-1
CFNDatabaseName:
Type: String
Default: cfn-database-flights-1
CFNTablePrefixName:
Type: String
Default: cfn_sample_1_
Resources:
CFNRoleFlights:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- "glue.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
Policies:
- PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action: "*"
Resource: "*"
CFNDatabaseFlights:
Type: AWS::Glue::Database
Properties:
CatalogId: !Ref AWS::AccountId
DatabaseInput:
Name: !Ref CFNDatabaseName
Description: "AWS Glue container to hold metadata tables for the flights crawler"
CFNCrawlerFlights:
Type: AWS::Glue::Crawler
Properties:
Name: !Ref CFNCrawlerName
Role: !GetAtt CFNRoleFlights.Arn
Description: AWS Glue crawler to crawl flights data
DatabaseName: !Ref CFNDatabaseName
Targets:
S3Targets:
- Path: "s3://crawler-public-us-east-1/flight/2016/csv"
TablePrefix: !Ref CFNTablePrefixName
SchemaChangePolicy:
UpdateBehavior: "UPDATE_IN_DATABASE"
DeleteBehavior: "LOG"
Configuration: "{\"Version\":1.0,\"CrawlerOutput\":{\"Partitions\":{\"AddOrUpdateBehavior\":\"InheritFromTable\"},\"Tables\":{\"AddOrUpdateBehavior\":\"MergeNewColumns\"}}}"
I found this Cloud formation template on here
Related
I have a requirement to select all the rules in AWS Config while deploying the resources in newly created account through Cloudformation. But I don't know how to select all the AWS Managed rules as in Console through Cloudformation. Any help would be very helpful.
AWSTemplateFormatVersion: 2010-09-09
Description: Enable AWS Config
Parameters:
AllSupported:
Type: String
Default: True
Description: Indicates whether to record all supported resource types.
AllowedValues:
- True
- False
IncludeGlobalResourceTypes:
Type: String
Default: True
Description: Indicates whether AWS Config records all supported global resource types.
AllowedValues:
- True
- False
ResourceTypes:
Type: List<String>
Description: A list of valid AWS resource types to include in this recording group, such as AWS::EC2::Instance or AWS::CloudTrail::Trail.
Default: <All>
DeliveryChannelName:
Type: String
Default: <Generated>
Description: The name of the delivery channel.
Frequency:
Type: String
Default: 24hours
Description: The frequency with which AWS Config delivers configuration snapshots.
AllowedValues:
- 1hour
- 3hours
- 6hours
- 12hours
- 24hours
Conditions:
IsAllSupported: !Equals
- !Ref AllSupported
- True
IsGeneratedDeliveryChannelName: !Equals
- !Ref DeliveryChannelName
- <Generated>
Mappings:
Settings:
FrequencyMap:
1hour : One_Hour
3hours : Three_Hours
6hours : Six_Hours
12hours : Twelve_Hours
24hours : TwentyFour_Hours
Resources:
ConfigBucket:
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Type: AWS::S3::Bucket
Properties:
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
ConfigBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ConfigBucket
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: AWSConfigBucketPermissionsCheck
Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action: s3:GetBucketAcl
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}"
- Sid: AWSConfigBucketDelivery
Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action: s3:PutObject
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}/AWSLogs/${AWS::AccountId}/*"
- Sid: AWSConfigBucketSecureTransport
Action:
- s3:*
Effect: Deny
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}"
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}/*"
Principal: "*"
Condition:
Bool:
aws:SecureTransport:
false
ConfigRecorderRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action:
- sts:AssumeRole
Path: /
ManagedPolicyArns:
- !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AWS_ConfigRole"
ConfigRecorder:
Type: AWS::Config::ConfigurationRecorder
DependsOn:
- ConfigBucketPolicy
Properties:
RoleARN: !GetAtt ConfigRecorderRole.Arn
RecordingGroup:
AllSupported: !Ref AllSupported
IncludeGlobalResourceTypes: !Ref IncludeGlobalResourceTypes
ResourceTypes: !If
- IsAllSupported
- !Ref AWS::NoValue
- !Ref ResourceTypes
ConfigDeliveryChannel:
Type: AWS::Config::DeliveryChannel
DependsOn:
- ConfigBucketPolicy
Properties:
Name: !If
- IsGeneratedDeliveryChannelName
- !Ref AWS::NoValue
- !Ref DeliveryChannelName
ConfigSnapshotDeliveryProperties:
DeliveryFrequency: !FindInMap
- Settings
- FrequencyMap
- !Ref Frequency
S3BucketName: !Ref ConfigBucket
ConfigRuleForVolumeTags:
DependsOn: ConfigRecorder
Type: AWS::Config::ConfigRule
Properties:
InputParameters:
tag1Key: CostCenter
Scope:
ComplianceResourceTypes:
- "AWS::EC2::Volume"
Source:
Owner: AWS
SourceIdentifier: "REQUIRED_TAGS"
# Like this I need all the AWS Managed rules
You can't do this. There are no loops in cloudformation. But you could create a macro if you want such a functionality.
I've two stacks called "createIAMRole", "createElasticSearch" and "createdLambda". What I want is I want to use IAM Role ARN from first stack called "createIAMRole" in both "createElasticSearch" and "createdLambda".
createIAMRole
AWSTemplateFormatVersion: '2010-09-09'
Description: >
blah.
Resources:
myIAMRole:
Type: AWS::IAM::Role
Properties:
..
..
Policies:
- PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- dynamodb:*
Resource: "*"
PolicyName: "myIAMRolePolicy"
Outputs:
myIAMRole:
Description: myIAMRole to use Stacks
Value: !Ref myIAMRole
"createElasticSearch"
Resources:
ElasticsearchDomain:
Type: AWS::Elasticsearch::Domain
Properties:
..
..
AccessPolicies:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
AWS:
- >>> THERE IS I WANT TO ADD <<<
Action: "es:*"
Resource: "*"
AdvancedOptions:
rest.action.multi.allow_explicit_index: "true"
Please let me know how to do it, thanks.
For the createIAMRole you need to export the output:
Outputs: myIAMRole:
Description: myIAMRole to use Stacks
Value: !Ref myIAMRole
Export:
Name: myIAMRole
And for the createElasticSearch you need to "ImportValue":
Fn::ImportValue:
!Sub "${myIAMStackName}-myIAMRole"
More Information: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html
I currently have the following cloudformation .yaml file:
Resources:
DynamoTable:
Type: "AWS::DynamoDB::Table"
Properties:
...
...
...
How do I give other resources permission to query this table?
Resources:
Service:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
...
...
TaskPolicyArn: !Ref ThisServicePolicy
DynamoTable:
Type: "AWS::DynamoDB::Table"
Properties:
AttributeDefinitions:
...
...
...
ThisServicePolicy:
Type: "AWS::IAM::ManagedPolicy"
Properties:
ManagedPolicyName: SomePolicyName
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "dynamodb:GetItem"
- "dynamodb:BatchGetItem"
- "dynamodb:Query"
Resource: "*"
I have A template that creates IAM roles In cloud Formation YAML. I need service Anr in next template, But I am getting this error.
Template contains errors.: Invalid template resource property 'Fn::ImportValue'
IAMStack
Resources:
CodeDeployTrustRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Sid: '1'
Effect: Allow
Principal:
Service:
- codedeploy.us-east-1.amazonaws.com
- codedeploy.us-west-2.amazonaws.com
Action: sts:AssumeRole
Path: "/"
CodeDeployRolePolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: CodeDeployPolicy
PolicyDocument:
Statement:
- Effect: Allow
Resource:
- "*"
Action:
- ec2:Describe*
- Effect: Allow
Resource:
- "*"
Action:
- autoscaling:CompleteLifecycleAction
- autoscaling:DeleteLifecycleHook
- autoscaling:DescribeLifecycleHooks
- autoscaling:DescribeAutoScalingGroups
- autoscaling:PutLifecycleHook
- autoscaling:RecordLifecycleActionHeartbeat
Roles:
- Ref: CodeDeployTrustRole
InstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
InstanceRolePolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: InstanceRole
PolicyDocument:
Statement:
- Effect: Allow
Action:
- autoscaling:Describe*
- autoscaling:EnterStandby
- autoscaling:ExitStandby
- cloudformation:Describe*
- cloudformation:GetTemplate
- s3:Get*
Resource: "*"
Roles:
- Ref: InstanceRole
InstanceRoleInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: "/"
Roles:
- Ref: InstanceRole
Outputs:
CodeDeployServiceRoleARN:
Value:
Fn::GetAtt:
- CodeDeployTrustRole
- Arn
==================================================================================
CodeDeploystack
---
AWSTemplateFormatVersion: '2010-09-09'
Description: This template will create an s3bucket
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: PublicRead
BucketName: CodeDeploy
CodeDeployApplication:
Type: 'AWS::CodeDeploy::Application'
Properties:
ComputePlatform: ec2
DeploymentGroup:
Type: AWS::CodeDeploy::DeploymentGroup
Properties:
ApplicationName:
!Ref CodeDeployApplication
Deployment:
Description: First time
IgnoreApplicationStopFailures: true
Revision:
RevisionType: S3
S3Location:
Bucket:
Ref: S3Bucket
ServiceRoleArn:
'Fn::ImportValue': !Sub '${IAMStack}-CodeDeployServiceRoleARN'
Outputs:
S3BucketName:
Value:
Ref: S3Bucket
Description: Name of S3 bucket
I tried rewriting your second template with the import function. Can you try something like this:
AWSTemplateFormatVersion: '2010-09-09'
Description: This template will create an s3bucket
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: PublicRead
BucketName: CodeDeploy
CodeDeployApplication:
Type: "AWS::CodeDeploy::Application"
Properties:
ComputePlatform: ec2
DeploymentGroup:
Type: AWS::CodeDeploy::DeploymentGroup
Properties:
ApplicationName: !Ref CodeDeployApplication
Deployment:
Description: First time
IgnoreApplicationStopFailures: true
Revision:
RevisionType: S3
S3Location: !Ref S3Bucket
ServiceRoleArn:
Fn::ImportValue:
Fn::Sub "${IAMStack}-CodeDeployServiceRoleARN"
Outputs:
S3BucketName:
Value: !Ref S3Bucket
Description: Name of S3 bucket
I think some quotes may be off in your version.
Issue fixed, I just change the region
I have this S3 Bucket and Policy that I am deploying to CloudFormation.
Resources:
ReportsBucket:
Type: AWS::S3::Bucket
BucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ReportsBucket
PolicyDocument:
Id: ReportPolicy
Version: "2012-10-17"
Statement:
- Sid: ReportBucketPolicyDoc
Effect: Allow
Action: "s3:*"
Principal:
AWS: !Join ['', ["arn:aws:iam::", !Ref "AWS::AccountId", ":root"]]
Resource: !Join ['', ['arn:aws:s3:::', !Ref S3Bucket, '/*']]
It fails with,
UPDATE_ROLLBACK_IN_PROGRESS AWS::CloudFormation::Stack {my stack name} The following resource(s) failed to create: [BucketPolicy].
CREATE_FAILED AWS::S3::BucketPolicy BucketPolicy Statement is missing required element
What's wrong with my policy?
It has two problems:
Missing AWSTemplateFormatVersion on the first line (the required element)
Reference to S3Bucket that should be ReportsBucket
Updated version:
AWSTemplateFormatVersion: 2010-09-09
Resources:
ReportsBucket:
Type: AWS::S3::Bucket
BucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ReportsBucket
PolicyDocument:
Id: ReportPolicy
Version: "2012-10-17"
Statement:
- Sid: ReportBucketPolicyDoc
Effect: Allow
Action: "s3:*"
Principal:
AWS: !Join ['', ["arn:aws:iam::", !Ref "AWS::AccountId", ":root"]]
Resource: !Join ['', ['arn:aws:s3:::', !Ref ReportsBucket, '/*']]