I have this under parameter section ,
Parameters:
PlatformSelect:
Description: Cockpit platform Select.
Type: String
Default: qa-1
AllowedValues: [qa-1, qa-2, staging, production]
I need to reference this value in my UserData. I’m using Mappings in between.
Mappings:
bootstrap:
ubuntu:
print: echo ${PlatformSelect} >>test.txt
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref ‘InstanceType’
KeyName: !Ref ‘KeyName’
Tags:
- Key: Name
Value: Test
UserData:
Fn::Base64:
Fn::Join:
- ‘’
- - |
#!/bin/bash
- Fn::FindInMap:
- bootstrap
- ubuntu
- print
- |2+
This is not working. Not sure the way I refer it is wrong in first place!!
Should I use something before it like, ‘${AWS::Parameters:PlatformSelect}’ ?
Is there a reason why you are using Mapping in between?
You could easily use !Sub instead
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
Tags:
- Key: Name
Value: Test
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
${PlatformSelect}
What about a combination of Fn::Join and Ref
UserData:
Fn::Base64:
Fn::Join:
- ''
- - '#!/bin/bash\n'
- 'print: echo'
- !Ref 'PlatformSelect'
- '>>test.txt\n'
Related
When I try to launch windows server using this template, I get an error:
Property validation failure: [Value of property {/LaunchTemplateData} does not match type {Object}]
I used this template:
Parameters:
1InstanceType:
Type: String
Default: t2.small
AllowedValues:
- t2.small
- m3.medium
- m3.xlarge
- i3.xlarge
2SecurityGroup:
Type: 'AWS::EC2::SecurityGroup::Id'
3KeyName:
Type: 'AWS::EC2::KeyPair::KeyName'
4LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: /aws/service/ami-windows-latest/Windows_Server-2016-English-Full-Base
Resources:
Ec2LaunchTemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateName: WindowsDesktop
LaunchTemplateData:
- ImageId: !Ref 4LatestAmiId
InstanceType: !Ref 1InstanceType
SecurityGroups:
- GroupId: !Ref 2SecurityGroup
KeyName: !Ref 3KeyName
InstanceMarketOptions:
MarketType: spot
SpotOptions:
SpotInstanceType: persistent
InstanceInterruptionBehavior: stop
Similar code works for linux servers. It seems that Windows spot instance (or template) with persistence request can not be created using cloudformation.
The error means that your LaunchTemplateData is not an object, but it is a list in your case. This is because extra - before ImageId. So it should be:
Resources:
Ec2LaunchTemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateName: WindowsDesktop
LaunchTemplateData:
ImageId: !Ref 4LatestAmiId
InstanceType: !Ref 1InstanceType
SecurityGroups:
- GroupId: !Ref 2SecurityGroup
KeyName: !Ref 3KeyName
InstanceMarketOptions:
MarketType: spot
SpotOptions:
SpotInstanceType: persistent
InstanceInterruptionBehavior: stop
I'm writing a cloudformation template in YAML format.
Now i'm stuck with appending the id of the ebs volume to ec2 user data.
Type: AWS::EC2::Volume
Properties:
Size: 50
AvailabilityZone: ap-southeast-1b
Tags:
- Key: Name
Value: Logstash Volume
LogStashMountPoint:
Type: AWS::EC2::VolumeAttachment
Properties:
InstanceId:
Ref: LogstashInstance
VolumeId:
Ref: LogstashVolume
Device: "/dev/xvdf"
LogstashInstance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile:
Ref: LogstashInstanceProfile
InstanceType: t2.micro
KeyName: chuongtest
ImageId: ami-0cd31be676780afa7
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash -xe
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
{LogstashVolume: Ref: LogstashVolume}
touch /home/ec2-user/ebscomplete.txt
curl "http://169.254.169.254/latest/meta-data/instance-id" >> /home/ec2-user/ec2id.txt
touch /home/ec2-user/ec2complete.txt
touch /home/ec2-user/complete.txt
- LogstashVolume: !Ref LogstashVolume
SecurityGroupIds:
- Ref: LogstashSecurityGroup
SubnetId: subnet-0d0e0989f57b96389
Tags:
- Key: Name
Value: Logstash Instance
UserData script with Resource Attribute CloudFormation
I'm following this link but it still doesn't work
When the new instance is launched. It has nothing in /home/ec2-user.
I looked everywhere and this is my final but it didn't work
Can anyone help me with it?
There is at least one syntax error in your UserData:
echo ${LogstashVolume}) >> /home/ec2-user/ebsid.txt
should be
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
For further debugging of your UserData, login to the instance and check /var/log/could-init-output.log file.
p.s.
The following is also incorrect:
{LogstashVolume: Ref: LogstashVolume}
Its not a valid bash command.
I am new to writing yaml scripts and I keep getting this error 'Invalid template resource property 'Ref'' when I try to create a stack on Cloudformation. Is there something I am missing in my code. Please do let me know. Thanks! I have written this template to bring up an emr cluster using yaml.
AWSTemplateFormatVersion: 2010-09-09
Description: EMR Cluster for Spark
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Common Parameters
Parameters:
- SystemValue
- SubSystemValue
- RevenueStreamValue
- EnvironmentValue
- KMSArn
- Label:
default: EMR Parameters
default: Lambda Parameters
Parameters:
- EMRClusterName
- KeyName
- MasterInstanceType
- CoreInstanceType
- NumberOfCoreInstances
- VpcId
- VPCSubnetIds
- LogUriFolder
- S3DataUri
- ReleaseLabel
- ApplicationsList
- PackageBucket
- EbsRootVolumeSize
- FunctionName1Value
- Label:
default: S3 Parameters
Parameters:
- EDWBucketName
ParameterLabels:
SystemValue:
default: System
SubSystemValue:
default: SubSystem
RevenueStreamValue:
default: Revenue Stream
EnvironmentValue:
default: Environment Value
FunctionName1Value:
default: Lambda-1 Name
EMRClusterName:
default: EMR Cluster Name
KeyName:
default: Key Name
MasterInstanceType:
default: Master Instance Type
CoreInstanceType:
default: Core Instance Type
NumberOfCoreInstances:
default: Number Of Core Instances
VpcId:
default: VPC ID
VPCSubnetIds:
default: VPC Subnet ID
LogUriFolder:
default: Log Uri Folder
S3DataUri:
default: S3 Data Uri
ReleaseLabel:
default: Release Label
ApplicationsList:
default: Applications List
KMSArn:
default: KMS Arn
EDWBucketName:
default: EDW Bucket Name
PackageBucket:
default: Package Bucket Name
EbsRootVolumeSize:
default: Ebs Root Volume Size
#########################################################################
Parameters:
KMSArn:
Type: String
Description: Enter KMS ARN based on your subsystem.
Default: 'a36ef9be-97e1-4949-9b04-c1979eda5955'
SystemValue:
Type: String
Description: Enter System Name
Default: 'Messaging'
SubSystemValue:
Type: String
Description: Enter SubSystem Name
Default: 'EDW'
RevenueStreamValue:
Type: String
Description: Enter Revenue Stream Name
Default: 'edw'
FunctionName1Value:
Type: String
Default: 'EMRCluster'
Description: Enter 1st Lambda Function Name(Do not append System & Sub-System Name).
EDWBucketName:
Type: String
Default: 'crx-dev-messaging-edw'
Description: Enter crx-[env]-messaging-edw
PackageBucket:
Type: String
Default: 'crx-dev-deployments'
Description: Enter crx-[env]-deployments
EnvironmentValue:
Type: String
Default: 'dev'
LambdaRuntime:
Type: String
Default: 'Python-2.7'
TagEnvironmentValue:
Type: String
Default: 'dev'
VpcId:
Type: 'AWS::EC2::VPC::Id'
Default: vpc-7c368507
VpcCIDR:
Type: String
Default: '10.10.16.0/20'
VPCSubnetIds:
Type: AWS::EC2::Subnet::Id
Default: subnet-4d527c62
Env:
Type: String
Default: 'dev'
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Ane<Env>-Messaging-EDW
Default: development
MasterInstanceType:
Type: String
Description: Instance type to be used for the master instance.
Default: 'm5.xlarge'
AllowedValues:
- m5.xlarge
- m1.medium
CoreInstanceType:
Type: String
Description: Instance type to be used for core instances.
Default: 'm5.xlarge'
AllowedValues:
- m5.xlarge
- m1.medium
NumberOfCoreInstances:
Description: Must be a valid number - 2
Type: Number
Default: '2'
LogUriFolder:
Type: String
Description: S3 Folder name for spark logs (spark-logs)
Default: 's3n://crx-dev-messaging-edw/spark-logs/'
S3DataUri:
Type: String
Description: Must be a valid S3 bucket URL
Default: 's3n://crx-messaging-edw'
EbsRootVolumeSize:
Type: String
Description: Specify the volume size
Default: '200'
ReleaseLabel:
Type: String
Description: Must be a valid EMR release version emr-6.0.0
Default: 'emr-6.0.0'
ApplicationsList:
Type: String
Description: Please select which application will be installed on the cluster
this would be either Ganglia,spark,Hadoop etc.,
Default: 'Spark'
AllowedValues:
- Hadoop
- Ganglia
- Spark
#########################################################################
Resources:
Bucket1:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub 'crx-${EnvironmentValue}-${SubSystemValue}'
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: aws:kms
KMSMasterKeyID: !Ref KMSArn
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
EbsRootVolumeSize: !Ref EbsRootVolumeSize
SvcAccessSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub SG-${Env}-${SystemValue}-${SubSystemValue}-ServiceAccess
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-Service Access Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-ServiceAccess'
EmrMasterSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRMaster'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-EMR Managed Master Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRMaster'
EmrSlaveSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRSlave'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-EMR Managed Slave Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${Env}-${SystemValue}-${SubSystemValue}-EMRSlave'
LambdaSecurityGroup:
Type: AWS::EC2::SecurityGroup
Condition : HasFunctionName1
Properties:
GroupName: !Sub 'SG-${SGEnvironmentValue}-${SystemValue}-${SubSystemValue}'
VpcId: !Ref VpcId
GroupDescription: !Sub 'SG-${SystemValue}-${SubSystemValue}-Lambda Security Group'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: !Ref VpcCIDR
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
Tags:
- Key: Environment
Value: !Ref TagEnvironmentValue
- Key: RevenueStream
Value: !Ref RevenueStreamValue
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Key: Name
Value: !Sub 'SG-${SGEnvironmentValue}-${SystemValue}-${SubSystemValue}'
Lambda1:
Type: AWS::Lambda::Function
DependsOn:
- LambdaExecutionRole
- ManagerApiPolicy
Properties:
Description: Script to launch EMR
FunctionName: !Sub ' ${SystemValue}-${SubSystemValue}-${FunctionName1Value}'
Handler: lambda_function.lambda_handler
Code:
S3Bucket: !Ref PackageBucket
S3Key: emr-launch.zip
Role: !GetAtt
- LambdaExecutionRole
- Arn
Runtime: !Ref LambdaRuntime
Timeout: '900'
MemorySize: 512
Environment:
Variables:
parameterPath: !Sub '/crx/${SystemValue}/${SubSystemValue}/'
region: !Ref 'AWS::Region'
VpcConfig:
SubnetIds: !Ref VPCSubnetIds
SecurityGroupIds:
- !GetAtt
- LambdaSecurityGroup
- GroupId
Tags:
- Value: !Sub '${SystemValue}-${SubSystemValue}-${FunctionName1Value}'
Key: Name
- Key: SubSystem
Value: !Ref SubSystemValue
- Key: System
Value: !Ref SystemValue
- Value: !Ref RevenueStreamValue
Key: RevenueStream
- Value: !Ref TagEnvironmentValue
Key: Environment
LambdaExecutionRole:
Type: AWS::IAM::Role
DependsOn:
- LambdaSecurityGroup
Properties:
RoleName: !Sub '${SystemValue}-${SubSystemValue}'
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- states.amazonaws.com
- events.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagerApiPolicy:
Type: 'AWS::IAM::ManagedPolicy'
DependsOn:
- LambdaExecutionRole
Properties:
ManagedPolicyName: !Sub '${SystemValue}-${SubSystemValue}'
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: common
Effect: Allow
Action:
- 'ec2:CreateNetworkInterface'
- 'ec2:Describe*'
- 'ec2:Get*'
- 'ec2:DeleteNetworkInterface'
- 'kms:GenerateDataKey'
- 'kms:ListAliases'
- 'kms:ListKeys'
- 'kms:ReEncryptTo'
- 'kms:CreateKey'
- 'kms:GenerateRandom'
- 'kms:ReEncryptFrom'
- 'logs:CreateLogGroup'
- 'logs:PutLogEvents'
- 'logs:CreateLogStream'
- 'ssm:DescribeParameters'
- 'lambda:InvokeFunction'
Resource: '*'
- Sid: ssm
Effect: Allow
Action:
- 'ssm:GetParametersByPath'
- 'ssm:GetParameters'
- 'ssm:GetParameter'
Resource:
- !Join
- ''
- - 'arn:aws:ssm:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':parameter/crx/'
- !Sub '${SystemValue}/${SubSystemValue}*'
- Sid: kms
Effect: Allow
Action: 'kms:*'
Resource:
- !Ref KMSArn
- !Join
- ''
- - 'arn:aws:kms:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':alias/'
- !Sub '${SystemValue}-${SubSystemValue}'
- Sid: s3
Effect: Allow
Action:
- 's3:PutObject'
- 's3:GetObject'
- 's3:List*'
Resource:
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-${PackageBucketName}'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-${PackageBucketName}/*'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-public-assets'
- !Join
- ''
- - 'arn:aws:s3:'
- '::'
- !Sub 'crx-${EnvironmentValue}-public-assets/*'
Roles:
- !Ref LambdaExecutionRole
PermissionForEventsToInvokeLambda:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName:
!Ref EmrCloudWatchEventLambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn:
Fn::GetAtt:
- EventRuleEMRtest
- Arn
EMRClusterServiceRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role
Path: /
EMRClusterinstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !Ref EMRClusterinstanceProfileRole
Line 192 in the above template
EbsRootVolumeSize: !Ref EbsRootVolumeSize
is not inside any of the resources block.
The CloudFormation Linter and Visual Studio Code extension catch these template errors:
W2030 You must specify a valid Default value for LambdaRuntime (Python-2.7).
Valid values are ['dotnetcore1.0', 'dotnetcore2.0', 'dotnetcore2.1', 'dotnetcore3.1', 'go1.x', 'java8', 'java11', 'nodejs', 'nodejs4.3-edge', 'nodejs4.3', 'nodejs6.10', 'nodejs8.10', 'nodejs10.x', 'nodejs12.x', 'provided', 'python2.7', 'python3.6', 'python3.7', 'python3.8', 'ruby2.5', 'ruby2.7']
template.yaml:111:5
E3001 Invalid resource attribute Ref for resource EbsRootVolumeSize
template.yaml:191:3
E3001 Type not defined for resource EbsRootVolumeSize
template.yaml:191:3
E3004 Circular Dependencies for resource EbsRootVolumeSize. Circular dependency with [EbsRootVolumeSize]
template.yaml:191:3
E8002 Condition HasFunctionName1 is not defined.
template.yaml:270:5
E1019 Parameter SGEnvironmentValue for Fn::Sub not found at Resources/LambdaSecurityGroup/Properties/GroupName/Fn::Sub
template.yaml:272:7
E1019 Parameter SGEnvironmentValue for Fn::Sub not found at Resources/LambdaSecurityGroup/Properties/Tags/4/Value/Fn::Sub
template.yaml:293:11
E3002 Property SubnetIds should be of type List or Parameter should be a list for resource Lambda1
template.yaml:318:9
E1019 Parameter PackageBucketName for Fn::Sub not found at Resources/ManagerApiPolicy/Properties/PolicyDocument/Statement/3/Resource/0/Fn::Join/1/2/Fn::Sub
template.yaml:422:13
E1019 Parameter PackageBucketName for Fn::Sub not found at Resources/ManagerApiPolicy/Properties/PolicyDocument/Statement/3/Resource/1/Fn::Join/1/2/Fn::Sub
template.yaml:422:13
E1012 Ref EmrCloudWatchEventLambda not found as a resource or parameter
template.yaml:451:7
E1010 Invalid GetAtt EventRuleEMRtest.Arn for resource PermissionForEventsToInvokeLambda
template.yaml:456:9
E3001 Invalid resource attribute ManagedPolicyArns for resource EMRClusterServiceRole
template.yaml:472:5
E3001 Invalid resource attribute Path for resource EMRClusterServiceRole
template.yaml:474:5
E1012 Ref EMRClusterinstanceProfileRole not found as a resource or parameter
template.yaml:481:9
It is caused by this:
EbsRootVolumeSize: !Ref EbsRootVolumeSize
Whatever it is, it should not be in the location it is currently present.
Also ManagedPolicyArns and Path in EMRClusterServiceRole resource have incorrect indentations. Could be due to copy-paste to SO though.
Other issue is that condition HasFunctionName1 is not defined. Same goes for SGEnvironmentValue and EventRuleEMRtest.
I'm trying to find a way to use ImportValue inside If function but can't find a proper syntax. Any help is appreciated.
Below the code I'm trying:
SomeTaskdefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: 'FamilyName'
ContainerDefinitions:
- Name: ContainerName
Image: 'imagename:net/v2/'
Environment:
- Name: ENV_VARIABLE_1
Value:
Fn::If:
Fn::Equals:
Fn::ImportValue:
!Sub "${ImportStackname}-ECSCluster"
''
'notpresent'
'present'
I come across here with similar problem.. my idea was to either be able specify DatabaseHost as parameter, if left empty - value should be taken from DatabaseStack export. Here is my sample code - it uses !ImportValue inside !If function. You will get the idea (instead of constructing only Value - construct whole Name Value list object)
Conditions:
DatabaseHostPresent: !Not [ !Equals [ !Ref DatabaseHost, ""]]
Resources:
...
ContainerDefinitions:
- Name: !Sub ${ApplicationName}-web-${EnvironmentName}
Environment:
- !If
- DatabaseHostPresent
- Name: DB_HOST
Value: !Ref DatabaseHost
- Name: DB_HOST
Value: !ImportValue
Fn::Sub: ${DatabaseStack}-EndpointAddress
I don't believe this is possible. You cannot use ImportValue inside an Equals function.
Another way to work around this is to use a nested template:
Add a Parameter to the Child template
In the Parent template, use the Import to populate the Parameter
In the Child template, you can do everything you'd want including using it in the Conditions section
Untested example:
# in Parent
Resources:
ChildStack:
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
Stackname: {'Fn::ImportValue': !Sub "${ImportStackname}-ECSCluster"}
TemplateURL: './child.yaml'
# in Child
Parameters:
Stackname:
Type: String
Default: ''
Conditions:
HasStack: !Not [!Equals [!Ref Stackname, '']]
Resources:
SomeTaskdefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: 'FamilyName'
ContainerDefinitions:
- Name: ContainerName
Image: 'imagename:net/v2/
Environment:
- Name: ENV_VARIABLE_1
Value: !If [HasStack, 'present', 'notpresent']
Using a combination of intrinsic function I've obtained the result:
parameter1: !If
- condition1
- !ImportValue
Fn::Sub: "${name}-dev-parameter"
- AWS::NoValue
Make sure to use the sequence of intrinsic function as I've listed. It seems to be the only way.
Below my cloudformation template:
I have added all the resource code, please excuse the indentation issue(copy paste thing), I assured you the template is running.
---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sets up your AWS Batch Environment for running workflows
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Compute Environment Config
Parameters:
- ComputeEnvironmentName
- VpcId
- SubnetIds
- MinvCpus
- MaxvCpus
- DesiredvCpus
- Label:
default: Job Definition
Parameters:
- JobDefinitionName
- DockerImage
- Vcpus
- Memory
- Command
- RetryNumber
- Label:
default: Job Queue
Parameters:
- JobQueueName
Parameters:
VpcId:
Type: 'AWS::EC2::VPC::Id'
Description: >-
VpcId of where the whole batch should be deployed. The VPC should have
2 private subnets.
SubnetIds:
Type: List<AWS::EC2::Subnet::Id>
Description: Subnets you want your batch compute environment to launch in. Recommend private subnets
MinvCpus:
Type: String
Description: Minimum number of CPUs in the compute environment. Default 0.
Default: 0
AllowedPattern: "[0-9]+"
DesiredvCpus:
Type: String
Description: Desired number of CPUs in the compute environment to launch with. Default 0.
Default: 0
AllowedPattern: "[0-9]+"
MaxvCpus:
Type: String
Description: Maximum number of CPUs in the compute environment. Should be >= than MinCpus
Default: 256
AllowedPattern: "[0-9]+"
RetryNumber:
Type: String
Default: "1"
Description: Number of retries for each AWS Batch job. Integer required.
MaxLength: 1
AllowedPattern: "[1-9]"
ConstraintDescription: Value between 1 and 9
DockerImage:
Type: String
Description: Docker image used to run your jobs
Vcpus:
Type: Number
Description: vCPUs available to Jobs. Default is usually fine
Default: 2
Memory:
Type: Number
Description: Memory (in MB) available to Jobs. Default is usually fine
Default: 2000
JobQueueName:
Type: String
Description: Enter job queue Name
JobDefinitionName:
Type: String
Description: Enter JobDefinition Name for the batch
ComputeEnvironmentName:
Type: String
Description: Enter name of the Compute Environment
VPCCidr:
Type: String
Description: 'Cidr Block of the VPC, allows for ssh access internally.'
Default: '10.0.0.0/8'
MinLength: "9"
MaxLength: "18"
AllowedPattern: "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})"
ConstraintDescription: "Must be valid CIDR notation (i.e. x.x.x.x/x)."
Command:
Type: CommaDelimitedList
Description: The command that is passed to the container
CreateNewRepository:
Default: false
Description: >-
Set this to true if you want to create a new Repository, else
it will not create a new one
Type: String
AllowedValues:
- true
- false
RepositoryName:
Type: String
Description: Enter name of the new Repository.
Conditions:
CreateRepository: !Equals
- !Ref CreateNewRepository
- true
isCommandPresent: !Not [!Equals [!Ref CreateNewRepository, '']]
Resources:
JobDefinition:
Type: AWS::Batch::JobDefinition
Properties:
Type: container
JobDefinitionName: !Ref JobDefinitionName
ContainerProperties:
Image: !Ref DockerImage
Vcpus: !Ref Vcpus
Memory: !Ref Memory
Command: !Ref Command
ReadonlyRootFilesystem: true
Privileged: true
RetryStrategy:
Attempts: !Ref RetryNumber
JobQueue:
Type: AWS::Batch::JobQueue
Properties:
ComputeEnvironmentOrder:
- Order: 1
ComputeEnvironment: !Ref MyComputeEnv
State: ENABLED
Priority: 10
JobQueueName: !Ref JobQueueName
myVPCSecurityGroup:
Type: "AWS::EC2::SecurityGroup"
Properties:
GroupDescription: Security group for batch process.
SecurityGroupEgress:
- CidrIp: 0.0.0.0/0
IpProtocol: '-1'
SecurityGroupIngress:
- CidrIp: !Ref VPCCidr
IpProtocol: tcp
FromPort: '22'
ToPort: '22'
VpcId: !Ref VpcId
MyComputeEnv:
Type: AWS::Batch::ComputeEnvironment
Properties:
Type: MANAGED
ServiceRole: !GetAtt awsBatchServiceRole.Arn
ComputeEnvironmentName: !Ref ComputeEnvironmentName
ComputeResources:
MinvCpus: !Ref MinvCpus
MaxvCpus: !Ref MaxvCpus
DesiredvCpus: !Ref DesiredvCpus
SecurityGroupIds: [!GetAtt myVPCSecurityGroup.GroupId]
Type: EC2
Subnets: !Ref SubnetIds
InstanceRole: !GetAtt InstanceProfile.Arn
InstanceTypes:
- optimal
State: ENABLED
awsBatchServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- "batch.amazonaws.com"
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSBatchServiceRole
ecsInstanceRole:
Type: AWS::IAM::Role
Properties:
RoleName: InstanceRole
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- "ec2.amazonaws.com"
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
- "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role"
InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
InstanceProfileName: InstanceProfile
Roles:
- !Ref ecsInstanceRole
MyRepository:
Type: AWS::ECR::Repository
Condition: CreateRepository
Properties:
RepositoryName: !Ref RepositoryName
RepositoryPolicyText:
Version: "2012-10-17"
Statement:
-
Sid: AllowPushPull
Effect: Allow
Principal: "*"
Action:
- "ecr:*"
I am getting this error:
Operation failed, ComputeEnvironment went INVALID with error: CLIENT_ERROR - The security group 'sg-d9b85d91' does not exist
I don't know what is wrong with the code but strangely, the SecurityGroupIds created by myVPCSecurityGroup is sg-2869f263 but ComputeEnvironment is trying to find sg-d9b85d91.
taking a stab in the dark here just working for my mobile phone but I think it's because you don't have a V PC to your computer environment possibly
Disabling the Compute Environment in the UI and enabling it back again fixed the issue.