I'm trying to create a new role and a policy using cloudformation.
When deploying it I'm getting the following error :
Syntax errors in policy. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: 848a408e-b0f1-11e8-90b6-cf2a19d18ad2)
AWSTemplateFormatVersion: 2010-09-09
Description: >
AWS CloudFormation Template
Parameters:
StackName:
Type: String
Description: stack test
Default: stackTest
DclEnvironment:
Type: String
Description: Env
AllowedValues :
- test
- dev
- stage
- prod
Default: dev
Domain:
Type: String
Description: Private Domain name
Default: int.mydomain.com
VpcId:
Type: AWS::EC2::VPC::Id
Default: xxxx
AppAmiId:
Type: AWS::EC2::Image::Id
Description: Ec2 AMI ID
Default: ami-XXXX
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Key Name
Default: xxxx
SecurityGroupIds:
Type: CommaDelimitedList
Description: Comma-separated list of existing security group IDs in your VPC
Default: sg-xxxx
SubnetA:
Description: Subnet from AZ a
Type: String
Default: subnet-xxxxx
SubnetB:
Description: Subnet from AZ b
Type: String
Default: subnet-xxxx
SubnetC:
Description: Subnet from AZ c
Type: String
Default: subnet-xxxx
DbSubnetGroupA:
Type: String
Description: Subnet from AZ A
Default: subnet-xxxx
DbSubnetGroupB:
Type: String
Description: Subnet from AZ B
Default: subnet-xxxxx
DbSubnetGroupC:
Type: String
Description: Subnet from AZ C
Default: subnet-xxxxx
Resources:
monitoringRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join
- "-"
- - !Ref DclEnvironment
- "iam-01"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- ec2.amazonaws.com
Path: "/"
policyEC2Monitoring:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Join
- "-"
- - !Ref DclEnvironment
- "policy-01"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ec2:Describe*
Ressource: "*"
- Effect: Allow
Action:
- elasticloadbalancing:Describe*
Ressource: "*"
- Effect: Allow
Action:
- cloudwatch:ListMetrics*
- cloudwatch:GetMetricStatistics
- cloudwatch:Describe*
Ressource: "*"
- Effect: Allow
Action:
- autoscaling:Describe*
Ressource: "*"
Roles:
- !Ref monitoringRole
instanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
InstanceProfileName: !Join
- "-"
- - !Ref DclEnvironment
- "inp-01"
Path: "/"
Roles:
- !Ref monitoringRole
Thanks by advance,
Fas3r.
EDIT
resource should be surrounded by [ "*" ] if more than 1 action;
If one action, it's not necessary to add a new line, it can be :
Action : actionName
br.
As the error suggests, your yaml has invalid syntax.
You can use web tools like http://www.yamllint.com/ to troubleshoot the syntax issues.
Here's the correct syntax yaml file:
AWSTemplateFormatVersion: 2010-09-09
Description: >
AWS CloudFormation Template
Parameters:
StackName:
Type: String
Description: stack test
Default: stackTest
DclEnvironment:
Type: String
Description: Env
AllowedValues :
- test
- dev
- stage
- sbox
- prod
Default: dev
DclPod:
Type: String
Description: Pod Name
Default: enel
DclService:
Type: String
Description: Pod Name
Default: monitoring
Domain:
Type: String
Description: Private Domain name
Default: int.mydomain.com
VpcId:
Type: AWS::EC2::VPC::Id
Default: vpc-4ac3bb21
AppAmiId:
Type: AWS::EC2::Image::Id
Description: Ec2 AMI ID
Default: ami-XXXX
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Key Name
Default: c3-kp-01
SecurityGroupIds:
Type: CommaDelimitedList
Description: Comma-separated list of existing security group IDs in your VPC
Default: sg-07f5186b
SubnetA:
Description: Subnet from AZ a
Type: String
Default: subnet-7d576316
SubnetB:
Description: Subnet from AZ b
Type: String
Default: subnet-496a0834
SubnetC:
Description: Subnet from AZ c
Type: String
Default: subnet-7d576316
DbSubnetGroupA:
Type: String
Description: Subnet from AZ A
Default: subnet-1154607a
DbSubnetGroupB:
Type: String
Description: Subnet from AZ B
Default: subnet-3d650740
DbSubnetGroupC:
Type: String
Description: Subnet from AZ C
Default: subnet-4d027e00
Resources:
monitoringRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join
- "-"
- - !Ref DclEnvironment
- !Ref DclPod
- !Ref DclService
- "iam-01"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- ec2.amazonaws.com
Path: "/"
policyEC2Monitoring:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Join
- "-"
- - !Ref DclEnvironment
- !Ref DclPod
- !Ref DclService
- "policy-01"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ec2:Describe*
Ressource: "*"
- Effect: Allow
Action:
- elasticloadbalancing:Describe*
Ressource: "*"
- Effect: Allow
Action:
- cloudwatch:ListMetrics*
- cloudwatch:GetMetricStatistics
- cloudwatch:Describe*
Ressource: "*"
- Effect: Allow
Action:
- autoscaling:Describe*
Ressource: "*"
Roles:
- !Ref monitoringRole
instanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
InstanceProfileName: !Join
- "-"
- - !Ref DclEnvironment
- !Ref DclPod
- !Ref DclService
- "inp-01"
Path: "/"
Roles:
- !Ref monitoringRole
Hope it helps.
Related
I have a requirement to select all the rules in AWS Config while deploying the resources in newly created account through Cloudformation. But I don't know how to select all the AWS Managed rules as in Console through Cloudformation. Any help would be very helpful.
AWSTemplateFormatVersion: 2010-09-09
Description: Enable AWS Config
Parameters:
AllSupported:
Type: String
Default: True
Description: Indicates whether to record all supported resource types.
AllowedValues:
- True
- False
IncludeGlobalResourceTypes:
Type: String
Default: True
Description: Indicates whether AWS Config records all supported global resource types.
AllowedValues:
- True
- False
ResourceTypes:
Type: List<String>
Description: A list of valid AWS resource types to include in this recording group, such as AWS::EC2::Instance or AWS::CloudTrail::Trail.
Default: <All>
DeliveryChannelName:
Type: String
Default: <Generated>
Description: The name of the delivery channel.
Frequency:
Type: String
Default: 24hours
Description: The frequency with which AWS Config delivers configuration snapshots.
AllowedValues:
- 1hour
- 3hours
- 6hours
- 12hours
- 24hours
Conditions:
IsAllSupported: !Equals
- !Ref AllSupported
- True
IsGeneratedDeliveryChannelName: !Equals
- !Ref DeliveryChannelName
- <Generated>
Mappings:
Settings:
FrequencyMap:
1hour : One_Hour
3hours : Three_Hours
6hours : Six_Hours
12hours : Twelve_Hours
24hours : TwentyFour_Hours
Resources:
ConfigBucket:
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Type: AWS::S3::Bucket
Properties:
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
ConfigBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ConfigBucket
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: AWSConfigBucketPermissionsCheck
Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action: s3:GetBucketAcl
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}"
- Sid: AWSConfigBucketDelivery
Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action: s3:PutObject
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}/AWSLogs/${AWS::AccountId}/*"
- Sid: AWSConfigBucketSecureTransport
Action:
- s3:*
Effect: Deny
Resource:
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}"
- !Sub "arn:${AWS::Partition}:s3:::${ConfigBucket}/*"
Principal: "*"
Condition:
Bool:
aws:SecureTransport:
false
ConfigRecorderRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- config.amazonaws.com
Action:
- sts:AssumeRole
Path: /
ManagedPolicyArns:
- !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AWS_ConfigRole"
ConfigRecorder:
Type: AWS::Config::ConfigurationRecorder
DependsOn:
- ConfigBucketPolicy
Properties:
RoleARN: !GetAtt ConfigRecorderRole.Arn
RecordingGroup:
AllSupported: !Ref AllSupported
IncludeGlobalResourceTypes: !Ref IncludeGlobalResourceTypes
ResourceTypes: !If
- IsAllSupported
- !Ref AWS::NoValue
- !Ref ResourceTypes
ConfigDeliveryChannel:
Type: AWS::Config::DeliveryChannel
DependsOn:
- ConfigBucketPolicy
Properties:
Name: !If
- IsGeneratedDeliveryChannelName
- !Ref AWS::NoValue
- !Ref DeliveryChannelName
ConfigSnapshotDeliveryProperties:
DeliveryFrequency: !FindInMap
- Settings
- FrequencyMap
- !Ref Frequency
S3BucketName: !Ref ConfigBucket
ConfigRuleForVolumeTags:
DependsOn: ConfigRecorder
Type: AWS::Config::ConfigRule
Properties:
InputParameters:
tag1Key: CostCenter
Scope:
ComplianceResourceTypes:
- "AWS::EC2::Volume"
Source:
Owner: AWS
SourceIdentifier: "REQUIRED_TAGS"
# Like this I need all the AWS Managed rules
You can't do this. There are no loops in cloudformation. But you could create a macro if you want such a functionality.
I am trying to create a stack using cloudformation to launch an emr by triggering a lambda function. But when the stack is getting created I am having this error for creating ManagedApiPolicy:
Resource a36ef9be-97e1-4949-9b04-c1979eda5955 must be in ARN format or "*"
Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument;
Where am I going wrong?
My code is:
AWSTemplateFormatVersion: 2010-09-09
Description: EMR Cluster for Spark
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Common Parameters
Parameters:
- SystemValue
- SubSystemValue
- RevenueStreamValue
- EnvironmentValue
- KMSArn
- Label:
default: EMR Parameters
default: Lambda Parameters
Parameters:
- EMRClusterName
- KeyName
- MasterInstanceType
- CoreInstanceType
- NumberOfCoreInstances
- VpcId
- VPCSubnetIds
- LogUriFolder
- S3DataUri
- ReleaseLabel
- ApplicationsList
- PackageBucketName
- EbsRootVolumeSize
- FunctionName1Value
- Label:
default: S3 Parameters
Parameters:
- EDWBucketName
ParameterLabels:
SystemValue:
default: System
SubSystemValue:
default: SubSystem
RevenueStreamValue:
default: Revenue Stream
EnvironmentValue:
default: Environment Value
FunctionName1Value:
default: Lambda-1 Name
EMRClusterName:
default: EMR Cluster Name
KeyName:
default: Key Name
MasterInstanceType:
default: Master Instance Type
CoreInstanceType:
default: Core Instance Type
NumberOfCoreInstances:
default: Number Of Core Instances
VpcId:
default: VPC ID
VPCSubnetIds:
default: VPC Subnet ID
LogUriFolder:
default: Log Uri Folder
S3DataUri:
default: S3 Data Uri
ReleaseLabel:
default: Release Label
ApplicationsList:
default: Applications List
KMSArn:
default: KMS Arn
EDWBucketName:
default: EDW Bucket Name
PackageBucketName:
default: Package Bucket Name
EbsRootVolumeSize:
default: Ebs Root Volume Size
#########################################################################
Parameters:
KMSArn:
Type: String
Description: Enter KMS ARN based on your subsystem.
Default: 'a36ef9be-97e1-4949-9b04-c1979eda5955'
SystemValue:
Type: String
Description: Enter System Name
Default: 'Messaging'
SubSystemValue:
Type: String
Description: Enter SubSystem Name
Default: 'edw'
RevenueStreamValue:
Type: String
Description: Enter Revenue Stream Name
Default: 'edw'
FunctionName1Value:
Type: String
Default: 'EMRCluster'
Description: Enter 1st Lambda Function Name(Do not append System & Sub-System Name).
EDWBucketName:
Type: String
Default: 'crx-dev-messaging-edw'
Description: Enter crx-[env]-messaging-edw
PackageBucketName:
Type: String
Default: 'crx-dev-deployments'
Description: Enter crx-[env]-deployments
EnvironmentValue:
Type: String
Default: 'dev'
LambdaRuntime:
Type: String
Default: 'Python-2.7'
TagEnvironmentValue:
Type: String
Default: 'dev'
VpcId:
Type: 'AWS::EC2::VPC::Id'
Default: vpc-7c368507
VpcCIDR:
Type: String
Default: '10.10.16.0/20'
VPCSubnetIds:
Type: AWS::EC2::Subnet::Id
Default: subnet-4d527c62
Env:
Type: String
Default: 'dev'
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Ane<Env>-Messaging-EDW
Default: development
MasterInstanceType:
Type: String
Description: Instance type to be used for the master instance.
Default: 'm5.xlarge'
AllowedValues:
- m5.xlarge
- m1.medium
CoreInstanceType:
Type: String
Description: Instance type to be used for core instances.
Default: 'm5.xlarge'
AllowedValues:
- m5.xlarge
- m1.medium
NumberOfCoreInstances:
Description: Must be a valid number - 2
Type: Number
Default: '2'
LogUriFolder:
Type: String
Description: S3 Folder name for spark logs (spark-logs)
Default: 's3n://crx-dev-messaging-edw/spark-logs/'
S3DataUri:
Type: String
Description: Must be a valid S3 bucket URL
Default: 's3n://crx-messaging-edw'
EbsRootVolumeSize:
Type: String
Description: Specify the volume size
Default: '200'
ReleaseLabel:
Type: String
Description: Must be a valid EMR release version emr-6.0.0
Default: 'emr-6.0.0'
ApplicationsList:
Type: String
Description: Please select which application will be installed on the cluster
this would be either Ganglia,spark,Hadoop etc.,
Default: 'Spark'
AllowedValues:
- Hadoop
- Ganglia
- Spark
#########################################################################
Resources:
BlockPublicAclsucket1:
Type: 'AWS::S3::Bucket'
Properties:
BucketName: !Sub 'crx-test-${EnvironmentValue}-${SubSystemValue}'
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms
KMSMasterKeyID: !Ref KMSArn
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
#EbsRootVolumeSize: !Ref EbsRootVolumeSize
SvcAccessSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupName: !Sub SG-${Env}-${SystemValue}-${SubSystemValue}-ServiceAccess
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-Service Access Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-ServiceAccess'
EmrMasterSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupName: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRMaster'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-EMR Managed Master Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRMaster'
EmrSlaveSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupName: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRSlave'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-EMR Managed Slave Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRSlave'
LambdaSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupName: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-Lambda Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}'
Lambda1:
Type: 'AWS::Lambda::Function'
DependsOn:
- LambdaExecutionRole
- ManagerApiPolicy
Properties:
Description: Script to launch EMR
FunctionName: !Sub ' ${SystemValue}-${SubSystemValue}-${FunctionName1Value}'
Handler: lambda_function.lambda_handler
Code:
S3Bucket: !Ref PackageBucketName
S3Key: emr-launch.zip
Role: !GetAtt
- LambdaExecutionRole
- Arn
Runtime: !Ref LambdaRuntime
Timeout: '900'
MemorySize: 512
Environment:
Variables:
parameterPath: !Sub '/crx/${SystemValue}/${SubSystemValue}/'
region: !Ref 'AWS::Region'
VpcConfig:
SubnetIds: !Ref VPCSubnetIds
SecurityGroupIds:
- !GetAtt
- LambdaSecurityGroup
- GroupId
Tags:
- Value: !Sub '${SystemValue}-${SubSystemValue}-${FunctionName1Value}'
Key: Name
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Value: !Ref RevenueStreamValue
Key: RevenueStream
- Value: !Ref TagEnvironmentValue
Key: Environment
LambdaExecutionRole:
Type: 'AWS::IAM::Role'
DependsOn:
- LambdaSecurityGroup
Properties:
RoleName: !Sub '${SystemValue}-${SubSystemValue}'
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- states.amazonaws.com
- events.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagerApiPolicy:
Type: 'AWS::IAM::ManagedPolicy'
DependsOn:
- LambdaExecutionRole
Properties:
ManagedPolicyName: !Sub '${SystemValue}-${SubSystemValue}'
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: common
Effect: Allow
Action:
- 'ec2:CreateNetworkInterface'
- 'ec2:Describe*'
- 'ec2:Get*'
- 'ec2:DeleteNetworkInterface'
- 'kms:GenerateDataKey'
- 'kms:ListAliases'
- 'kms:ListKeys'
- 'kms:ReEncryptTo'
- 'kms:CreateKey'
- 'kms:GenerateRandom'
- 'kms:ReEncryptFrom'
- 'logs:CreateLogGroup'
- 'logs:PutLogEvents'
- 'logs:CreateLogStream'
- 'ssm:DescribeParameters'
- 'lambda:InvokeFunction'
Resource: '*'
- Sid: ssm
Effect: Allow
Action:
- 'ssm:GetParametersByPath'
- 'ssm:GetParameters'
- 'ssm:GetParameter'
Resource:
- !Join
- ''
- - 'arn:aws:ssm:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':parameter/crx/'
- !Sub '${SystemValue}/${SubSystemValue}*'
- Sid: kms
Effect: Allow
Action: 'kms:*'
Resource:
- !Ref KMSArn
- !Join
- ''
- - 'arn:aws:kms:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':alias/'
- !Sub '${SystemValue}-${SubSystemValue}'
- Sid: s3
Effect: Allow
Action:
- 's3:PutObject'
- 's3:GetObject'
- 's3:List*'
Resource:
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-${PackageBucketName}'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-${PackageBucketName}/*'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-public-assets'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-public-assets/*'
Roles:
- !Ref LambdaExecutionRole
# LambdaExecutionRoleforEMRPolicy:
# DependsOn:
# - LambdaExecutionRole
# Type: 'AWS::IAM::Policy'
# Properties:
# PolicyName: LambdaforEMRRolePolicy
# Roles:
# !Ref LambdaExecutionRole
# AssumeRolePolicyDocument:
# Version: 2012-10-17
# Statement:
# Action:
# - 'elasticmapreduce: *'
# Resource: '*'
# - Effect: Allow
# Action:
# - logs:CreateLogGroup
# - logs:CreateLogStream
# - logs:PutLogEvents
# - Resource: arn:aws:*:*:*
# - Effect: Allow
# Action:
# - elasticmapreduce:'*'
# - Resource: '*'
PermissionForEventsToInvokeLambda:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName:
!Ref LambdaExecutionRole
Action: lambda:InvokeFunction
Principal: apigateway.amazonaws.com
SourceArn: !GetAtt
- Lambda1
- Arn
EMRClusterServiceRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role
Path: /
EMRClusterinstanceProfileRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role
Path: /
EMRClusterinstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !Ref EMRClusterinstanceProfileRole
I think in
- Sid: kms
Effect: Allow
Action: 'kms:*'
Resource:
- !Ref KMSArn
KMSArn defaults to the resource id and not to the resource arn:
Parameters:
KMSArn:
Type: String
Description: Enter KMS ARN based on your subsystem.
Default: 'a36ef9be-97e1-4949-9b04-c1979eda5955'
So you can build an arn of the resource id using join.
I think in the other usage of KMSArn in
Resources:
BlockPublicAclsucket1:
Type: 'AWS::S3::Bucket'
Properties:
BucketName: !Sub 'crx-test-${EnvironmentValue}-${SubSystemValue}'
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms
KMSMasterKeyID: !Ref KMSArn
you can also provide an arn according to the documentation. Alternatively you could change the default of KMSArn from resource id to arn.
I'm trying my hand at Cloudformation nested stacks. The idea is that I create a VPC, S3 bucket, Codebuild project, and Codepipeline pipeline using Cloudformation.
My Problem: Cloudformation is saying that the following parameters (outputted by child stacks) require values:
Vpc
PrivateSubnet1
PrivateSubnet2
PrivateSubnet3
BucketName
These params should have values as the value exists when I look at a completed child stack in the console.
I'll just show the templates for the parent, s3, and codepipeline. With regards to these three templates the problem is that I am unable to use an output BucketName from S3Stack in my CodePipelineStack
My Code:
cfn-main.yaml
AWSTemplateFormatVersion: 2010-09-09
Description: root template for codepipeline poc
Parameters:
BucketName:
Type: String
VpcName:
Description: name of the vpc
Type: String
Default: sandbox
DockerUsername:
Type: String
Description: username for hub.docker
Default: seanturner026
DockerPassword:
Type: String
Description: password for hub.docker
Default: /codebuild/docker/password
Environment:
Type: String
Description: environment
AllowedValues:
- dev
- prod
Default: dev
Vpc:
Type: AWS::EC2::VPC::Id
PrivateSubnet1:
Type: AWS::EC2::Subnet::Id
PrivateSubnet2:
Type: AWS::EC2::Subnet::Id
PrivateSubnet3:
Type: AWS::EC2::Subnet::Id
GithubRepository:
Type: String
Description: github repository
Default: aws-codepipeline-poc
GithubBranch:
Type: String
Description: github branch
Default: master
GithubOwner:
Type: String
Description: github owner
Default: SeanTurner026
GithubToken:
Type: String
Description: github token for codepipeline
NoEcho: true
Resources:
VpcStack:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
VpcName: !Ref VpcName
TemplateURL: resources/vpc.yaml
S3Stack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: resources/s3.yaml
CodeBuildStack:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
Environment: !Ref Environment
DockerUsername: !Ref DockerUsername
DockerPassword: !Ref DockerPassword
Vpc: !GetAtt VpcStack.Outputs.VpcId
PrivateSubnet1: !GetAtt VpcStack.Outputs.PrivateSubnetId1
PrivateSubnet2: !GetAtt VpcStack.Outputs.PrivateSubnetId2
PrivateSubnet3: !GetAtt VpcStack.Outputs.PrivateSubnetId3
TemplateURL: resources/codebuild.yaml
CodePipelineStack:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
Environment: !Ref Environment
GithubRepository: !Ref GithubRepository
GithubBranch: !Ref GithubBranch
GithubOwner: !Ref GithubOwner
GithubToken: !Ref GithubToken
S3: !GetAtt S3Stack.Outputs.BucketName
TemplateURL: resources/codepipeline.yaml
s3.yaml
AWSTemplateFormatVersion: 2010-09-09
Description: s3 bucket for aws codepipeline poc
Resources:
S3:
Type: "AWS::S3::Bucket"
Properties:
BucketName: "aws-sean-codepipeline-poc"
Outputs:
BucketName:
Description: S3 bucket name
Value: !Ref S3
codepipeline.yaml -- Please see ArtifactStore. This is where cloudformation is seeing my parameter BucketName as value-less.
AWSTemplateFormatVersion: 2010-09-09
Description: codepipeline for aws codepipeline poc
Parameters:
BucketName:
Type: String
Environment:
Type: String
Description: environment
AllowedValues:
- dev
- prod
Default: dev
GithubRepository:
Type: String
Description: github repository
Default: aws-codepipeline-poc
GithubBranch:
Type: String
Description: github branch
Default: master
GithubOwner:
Type: String
Description: github owner
Default: SeanTurner026
GithubToken:
Type: String
Description: github token for codepipeline
NoEcho: true
Resources:
CodePipelineRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: !Join
- ""
- - !Ref AWS::StackName
- "-code-pipeline-role-"
- !Ref Environment
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
Effect: "Allow"
Principal:
Service: "codepipeline.amazonaws.com"
Action: "sts:AssumeRole"
CodePipelinePolicy:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: !Join
- ""
- - !Ref AWS::StackName
- "-code-pipeline-policy-"
- !Ref Environment
PolicyDocument:
Version: "2012-10-17"
Statement:
Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- s3:putObject
- s3:getObject
- codebuild:*
Resource:
- "*"
Roles:
- !Ref CodePipelineRole
Pipeline:
Type: "AWS::CodePipeline::Pipeline"
Properties:
Name: !Join
- ""
- - "code-pipeline-poc-"
- !Ref AWS::StackName
ArtifactStore:
Location: !Ref BucketName
Type: S3
RestartExecutionOnUpdate: true
RoleArn: !Join
- ""
- - "arn:aws:iam::"
- !Ref AWS::AccountId
- ":role/"
- !Ref CodePipelineRole
Stages:
- Name: checkout-source-code
Actions:
- Name: SourceAction
RunOrder: 1
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
Configuration:
Owner: !Ref GithubOwner
Repo: !Ref GithubRepository
Branch: !Ref GithubBranch
PollForSourceChanges: true
OAuthToken: !Ref GithubToken
OutputArtifacts:
- Name: source-code
- Name: docker-build-push
Actions:
- Name: build-push-job
RunOrder: 1
InputArtifacts:
- Name: source-code
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref BuildPushJob
OutputArtifacts:
- Name: build-push-job
Sorry if this is too verbose. If missed above, the problem is that ArtifactStore in the codepipeline.yaml is seeing my parameter BucketName as value-less, despite the value being outputted by S3Stack.
You pass the parameter as S3 but the template is expecting it as BucketName.
I've made a parent (nested) stack template that references 4 child templates. When I launch the stack through aws cloudformation create-stack, I get the following error for the parent stack:
Embedded stack AlignmentLambdaFunction was not successfully created: The following resource(s) failed to create: [CloudspanLambdaFunction, HaploLambdaExecutionRole, AlignmentLambdaExecutionRole].
And I get this error within one of the nested stacks that was getting created from the parent: Policy contains a statement with one or more invalid principals (for MasterGCPStorageKey (which is a resource in the Lambda child above)
I don't understand the source of the error. I thought maybe it was because of needing a DependsOn for the ExecutionRoles, but that didn't resolve the error.
Parent Stack:
AWSTemplateFormatVersion: "2010-09-09"
Description: "Master template for wgs-pipeline. Calls to other stack templates."
Parameters:
CloudspanLambdaFuncS3BucketName:
Type: String
CloudspanLambdaFuncS3KeyName:
Default: 'sfn.deployable.zip'
Type: String
CloudspanLambdaFuncModuleName:
Default: 'cloudspan'
Type: String
AlignmentLambdaFuncS3BucketName:
Type: String
AlignmentLambdaFuncS3KeyName:
Type: String
AlignmentLambdaFuncModuleName:
Type: String
HaploLambdaFuncS3BucketName:
Type: String
HaploLambdaFuncS3KeyName:
Type: String
HaploLambdaFuncModuleName:
Type: String
KMSAdminUserARN:
Type: String
KMSEndUserARN:
Type: String
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
InternetGateway:
Type: AWS::EC2::InternetGateway
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: 'VPC'
VPCGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId:
Ref: 'VPC'
InternetGatewayId:
Ref: 'InternetGateway'
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: EC2 Security Group for instances launched in the VPC by Batch
VpcId:
Ref: 'VPC'
StepFunctionsActivitiesInstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Allow http to client host
VpcId:
Ref: VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 128.218.0.0/16
Subnet:
Type: AWS::EC2::Subnet
Properties:
CidrBlock: 10.0.0.0/24
VpcId:
Ref: 'VPC'
AvailabilityZone:
Ref: GPCESubnetAZ1
MapPublicIpOnLaunch: 'True'
DependsOn: VPC
Route:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: 'RouteTable'
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: 'InternetGateway'
DependsOn:
- RouteTable
- InternetGateway
SubnetRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId:
Ref: 'RouteTable'
SubnetId:
Ref: 'Subnet'
DependsOn:
- RouteTable
- Subnet
# Beginning of reference to child stacks
ClouspanLambdaFunction:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
CloudspanLambdaFuncS3BucketName:
Ref: CloudspanLambdaFuncS3BucketName
CloudspanLambdaFuncS3KeyName:
Ref: CloudspanLambdaFuncS3KeyName
CloudspanLambdaFuncModuleName:
Ref: CloudspanLambdaFuncModuleName
KMSAdminUserARN:
Ref: KMSAdminUserARN
KMSEndUserARN:
Ref: KMSEndUserARN
TemplateURL: https://s3.amazonaws.com/CFNTemplate/lambda_resources.stack.yaml
TimeoutInMinutes: 1
AlignmentLambdaFunction:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AlignmentLambdaFuncS3BucketName:
Ref: AlignmentLambdaFuncS3BucketName
AlignmentLambdaFuncS3KeyName:
Ref: AlignmentLambdaFuncS3KeyName
AlignmentLambdaFuncModuleName:
Ref: AlignmentLambdaFuncModuleName
KMSAdminUserARN:
Ref: KMSAdminUserARN
KMSEndUserARN:
Ref: KMSEndUserARN
TemplateURL: https://s3.amazonaws.com/CFNTemplate/lambda_resources.stack.yaml
TimeoutInMinutes: 1
HaploLambdaFunction:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
HaploLambdaFuncS3BucketName:
Ref: HaploLambdaFuncS3BucketName
HaploLambdaFuncS3KeyName:
Ref: HaploLambdaFuncS3KeyName
HaploLambdaFuncModuleName:
Ref: HaploLambdaFuncModuleName
KMSAdminUserARN:
Ref: KMSAdminUserARN
KMSEndUserARN:
Ref: KMSEndUserARN
TemplateURL: https://s3.amazonaws.com/CFNTemplate/lambda_resources.stack.yaml
TimeoutInMinutes: 1
Lambda Child Stack (relevant for error):
AWSTemplateFormatVersion: '2010-09-09'
Description: lambda function and execution role stack.
Parameters:
CloudspanLambdaFuncS3BucketName:
Type: String
Default: 'claudia-test-transfer'
CloudspanLambdaFuncS3KeyName:
Default: 'sfn.deployable.zip'
Type: String
CloudspanLambdaFuncModuleName:
Default: 'cloudspan'
Type: String
AlignmentLambdaFuncS3BucketName:
Type: String
Default: 'claudia-test-transfer'
AlignmentLambdaFuncS3KeyName:
Type: String
Default: 'alignment_processing.deployable.zip'
AlignmentLambdaFuncModuleName:
Type: String
Default: 'alignment_processing'
HaploLambdaFuncS3BucketName:
Type: String
Default: 'claudia-test-transfer'
HaploLambdaFuncS3KeyName:
Type: String
Default: 'sentieon_haplotyper.deployable.zip'
HaploLambdaFuncModuleName:
Type: String
Default: 'sentieon_haplotyper'
KMSAdminUserARN:
Type: String
KMSEndUserARN:
Type: String
Resources:
CloudspanLambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Handler:
Fn::Join: [ ".", [ Ref: CloudspanLambdaFuncModuleName, "handler"] ]
Role:
Fn::GetAtt: [ CloudspanLambdaExecutionRole, Arn ]
Code:
S3Bucket:
Ref: CloudspanLambdaFuncS3BucketName
S3Key:
Ref: CloudspanLambdaFuncS3KeyName
Runtime: "python3.6"
Timeout: "60"
DependsOn: CloudspanLambdaExecutionRole
AlignmentLambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Handler:
Fn::Join: [ ".", [ Ref: AlignmentLambdaFuncModuleName, "handler"] ]
Role:
Fn::GetAtt: [ AlignmentLambdaExecutionRole, Arn ]
Code:
S3Bucket:
Ref: AlignmentLambdaFuncS3BucketName
S3Key:
Ref: AlignmentLambdaFuncS3KeyName
Runtime: "python3.6"
Timeout: "60"
DependsOn: AlignmentLambdaExecutionRole
HaploLambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Handler:
Fn::Join: [ ".", [ Ref: HaploLambdaFuncModuleName, "handler"] ]
Role:
Fn::GetAtt: [ HaploLambdaExecutionRole, Arn ]
Code:
S3Bucket:
Ref: HaploLambdaFuncS3BucketName
S3Key:
Ref: HaploLambdaFuncS3KeyName
Runtime: "python3.6"
Timeout: "60"
DependsOn: HaploLambdaExecutionRole
CloudspanLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: "sts:AssumeRole"
Policies:
- PolicyName: CanListBuckets
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "s3:GetBucketLocation"
- "s3:ListAllMyBuckets"
Resource: "arn:aws:s3:::*"
- PolicyName: CanLog
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
AlignmentLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: "sts:AssumeRole"
Policies:
- PolicyName: CanListBuckets
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "s3:GetBucketLocation"
- "s3:ListAllMyBuckets"
Resource: "arn:aws:s3:::*"
- PolicyName: CanCallBatch
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "batch:*"
Resource: "*"
- PolicyName: CanLog
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
HaploLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: "sts:AssumeRole"
Policies:
- PolicyName: CanListBuckets
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "s3:GetBucketLocation"
- "s3:ListAllMyBuckets"
Resource: "arn:aws:s3:::*"
- PolicyName: CanCallBatch
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "batch:*"
Resource: "*"
- PolicyName: CanLog
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
MasterGCPStorageKey:
Type: "AWS::KMS::Key"
Properties:
Description: Symmetric Master Key for GCP Storage Credentials off-line encryption/on-line decryption protocol
Enabled: True
EnableKeyRotation: True
KeyPolicy:
Version: "2012-10-17"
Statement:
- Sid: "Allow Lambda Excution Role access to GCP Storage decryption key"
Effect: "Allow"
Principal:
# ARN of CloudspanLambdaExecutionRole
AWS:
Fn::GetAtt: [ CloudspanLambdaExecutionRole, Arn ]
Action:
- kms:Decrypt
- kms:DescribeKey
# in this context "*" means "this" CMK
Resource: "*"
- Sid: "Allow Administrator to admin the GCP Storage decryption key"
Effect: "Allow"
Principal:
# ARN of the KMS admin IAM user
AWS:
Ref: KMSAdminUserARN
Action:
- "kms:Create*"
- "kms:Describe*"
- "kms:Enable*"
- "kms:List*"
- "kms:Put*"
- "kms:Update*"
- "kms:Revoke*"
- "kms:Disable*"
- "kms:Get*"
- "kms:Delete*"
- "kms:TagResource"
- "kms:UntagResource"
- "kms:ScheduleKeyDeletion"
- "kms:CancelKeyDeletion"
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
# in this context "*" means "this" CMK
Resource: "*"
- Sid: "Allow End User to encrypt the GCP Storage creds"
Effect: "Allow"
Principal:
# ARN of the KMS IAM end user
AWS:
Ref: KMSEndUserARN
Action:
- "kms:Encrypt"
- "kms:ReEncrypt"
- "kms:DescribeKey"
# in this context "*" means "this" CMK
Resource: "*"
DependsOn: CloudspanLambdaExecutionRole
I also was getting the following error after re-deploying a CloudFormation stack I had removed (via Serverless):
We encountered the following errors while processing your request:
Policy contains a statement with one or more invalid principals.
In my case, the original role which was assigned to my KMS encryption key was removed. KMS still keeps a reference to the removed role, and apparently adding a newly created role of the same type creates this error.
I solved this by simply removing the old reference to the removed role, under IAM > Encryption Keys > YOUR_KEY_NAME > Key Policy > Key Users
I have turned my original cloudformation stack template into multiple child templates which I then call from a master template. One of the child templates possesses the following snippet, for which I'm getting the error:
An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unresolved resource dependencies [GeneralPurposeContainerRole] in the Resources block of the template
BatchResourcesStack (child stack):
---
AWSTemplateFormatVersion: '2010-09-09'
Description: batch resources stack.
Parameters:
GPCEName:
Type: String
GPCEMaxVcpus:
Type: Number
Description: Max number of VCPUs for entire cluster, there are caveats to this
GPCEMinVcpus:
Type: Number
Description: Min number of VCPUs for entire cluster, there are caveats to this
GPCEDesiredVcpus:
Type: Number
Description: Desired number of VCPUs for entire cluster, there are caveats to this
GPCEVpcId:
Type: String
GPCESubnetAZ1:
Type: String
GPCEAmi:
Type: String
GPCEInstanceTypes:
Type: CommaDelimitedList
GPCESSHKeyPair:
Type: String
StackUID:
Type: String
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Subnet:
Type: AWS::EC2::Subnet
Resources:
BatchServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: batch.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSBatchServiceRole
IamInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref 'EcsInstanceRole'
EcsInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2008-10-17'
Statement:
- Sid: ''
Effect: Allow
Principal:
Service: ec2.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
GeneralPurposeContainerRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
Path: '/'
Policies:
- PolicyName: ContainerS3Access
PolicyDocument:
Statement:
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:DeleteObject
- s3:List*
Resource:
# TODO: Make these non psychcore-specific
- arn:aws:s3:::pipeline-validation/*
- arn:aws:s3:::wgs-pipeline-vqsr-test/*
- arn:aws:s3:::test-references/*
- arn:aws:s3:::psychcore-pipelines/output/*
- arn:aws:s3:::psychcore-pipelines/validation/samples/*
- arn:aws:s3:::psychcore-data/reference_indexs/*
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-data
- arn:aws:s3:::*
- Effect: Allow
Action:
- s3:List*
- s3:ListMultipartUploadParts
- s3:AbortMultipartUpload
- s3:ListBucketMultipartUploads
Resource:
- arn:aws:s3:::pipeline-validation
- arn:aws:s3:::wgs-pipeline-vqsr-test
- arn:aws:s3:::test-references
- arn:aws:s3:::psychcore-pipelines/output/
- arn:aws:s3:::psychcore-pipelines/validation/samples/
- arn:aws:s3:::psychcore-data/reference_indexs/
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-data
- arn:aws:s3:::*
GeneralPurposeComputeEnvironment:
Type: "AWS::Batch::ComputeEnvironment"
Properties:
Type: MANAGED
ComputeEnvironmentName: !Join
- '-'
- - !Ref GPCEName
- !Ref StackUID
ComputeResources:
MinvCpus:
Ref: GPCEMinVcpus
MaxvCpus:
Ref: GPCEMaxVcpus
DesiredvCpus:
Ref: GPCEDesiredVcpus
SecurityGroupIds:
- Ref: SecurityGroup
Subnets:
- Ref: Subnet
Type: 'EC2'
ImageId:
Ref: GPCEAmi
InstanceRole:
Ref: IamInstanceProfile
InstanceTypes:
Ref: GPCEInstanceTypes
Ec2KeyPair:
Ref: GPCESSHKeyPair
Tags:
Key: Name
Value: "VariantCallingBatchComputeEnvironment"
ServiceRole:
Ref: BatchServiceRole
State: ENABLED
DependsOn:
- SecurityGroup
- Subnet
- IamInstanceProfile
- BatchServiceRole
GeneralPurposeQueue:
Type: "AWS::Batch::JobQueue"
Properties:
ComputeEnvironmentOrder:
- Order: 1
ComputeEnvironment: !Ref GeneralPurposeComputeEnvironment
Priority: 1
State: ENABLED
JobQueueName: !Join
- '-'
- - "GeneralPurposeQueue"
- !Ref StackUID
DependsOn:
- GeneralPurposeComputeEnvironment
- BatchServiceRole
Parent.yaml (contains parts relevant to above child stack):
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Master template for wgs-pipeline. Calls to other stack templates."
Parameters:
GPCEName:
Default: 'GeneralPurposeVariantCallingCE'
Type: String
GPCEMaxVcpus:
Default: 128
Type: Number
Description: Max number of VCPUs for entire cluster, there are caveats to this
GPCEMinVcpus:
Default: 0
Type: Number
Description: Min number of VCPUs for entire cluster, there are caveats to this
GPCEDesiredVcpus:
Default: 0
Type: Number
Description: Desired number of VCPUs for entire cluster, there are caveats to this
GPCEVpcId:
Type: String
GPCESubnetAZ1:
Default: 'us-east-1a'
Type: String
GPCEAmi:
Default: "ami-ce6cdfb4"
Type: String
GPCEInstanceTypes:
Default: "i3.xlarge, i3.2xlarge, i3.4xlarge, i3.8xlarge, i3.16xlarge"
Type: CommaDelimitedList
GPCESSHKeyPair:
Type: String
StackUID:
Default: "1234"
Type: String
Resources:
Subnet:
Type: AWS::EC2::Subnet
Properties:
CidrBlock: 10.0.0.0/24
VpcId: !Ref 'VPC'
AvailabilityZone: !Ref GPCESubnetAZ1
MapPublicIpOnLaunch: 'True'
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: EC2 Security Group for instances launched in the VPC by Batch
VpcId: !Ref 'VPC'
BatchResourcesStack:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
GPCEName:
Ref: GPCEName
GPCEMaxVcpus:
Ref: GPCEMaxVcpus
GPCEMinVcpus:
Ref: GPCEMinVcpus
GPCEDesiredVcpus:
Ref: GPCEDesiredVcpus
GPCEVpcId:
Ref: GPCEVpcId
GPCESubnetAZ1:
Ref: GPCESubnetAZ1
GPCEAmi:
Ref: GPCEAmi
GPCEInstanceTypes:
Ref: GPCEInstanceTypes
GPCESSHKeyPair:
Ref: GPCESSHKeyPair
StackUID:
Ref: StackUID
SecurityGroup:
Ref: SecurityGroup
Subnet:
Ref: Subnet
TemplateURL: https://s3.amazonaws.com/CFNTemplate/batch_resources.stack.yaml
Timeout: "100"
I don't understand what the error is specifically pointing to. I put the entire Batch-child.yaml file through a YAML validator and it passed, so it shouldn't be from a formatting/indention error per se. Also, the GeneralPurposeContainerRole resource does not get referenced to anywhere else in the template, nor even in the parent stack template.