I have a aws cloudformation template for my batch definition. I have 2 compute environnement définition, a on demand and a spot.
DragenComputeEnvironmentSpot:
Type: AWS::Batch::ComputeEnvironment
Properties:
ComputeEnvironmentName: dragen-spot
ServiceRole: !Ref BatchServiceRole
Type: MANAGED
State: ENABLED
ComputeResources:
BidPercentage: !Ref BidPercentage
Ec2KeyPair: !Ref Ec2KeyPair
ImageId: !Ref ImageId
InstanceRole: !Ref DragenInstanceRole
InstanceTypes: [!Ref InstanceType]
MinvCpus: !Ref MinvCpus
DesiredvCpus: !Ref DesiredvCpus
MaxvCpus: !Ref MaxvCpus
SecurityGroupIds:
- !Ref BatchSecurityGroup
SpotIamFleetRole: !Ref SpotFleetRole
Subnets: !Ref SubnetIds
Type: SPOT
LaunchTemplate:
LaunchTemplateId: !Ref DragenLaunchTemplate
DragenComputeEnvironmentOnDemand:
Type: AWS::Batch::ComputeEnvironment
Properties:
ComputeEnvironmentName: dragen-ondemand
ServiceRole: !Ref BatchServiceRole
Type: MANAGED
State: ENABLED
ComputeResources:
Ec2KeyPair: !Ref Ec2KeyPair
ImageId: !Ref ImageId
InstanceRole: !Ref DragenInstanceRole
InstanceTypes: [!Ref InstanceType]
MinvCpus: !Ref MinvCpus
DesiredvCpus: !Ref DesiredvCpus
MaxvCpus: !Ref MaxvCpus
SecurityGroupIds:
- !Ref BatchSecurityGroup
Subnets: !Ref SubnetIds
Type: EC2
LaunchTemplate:
LaunchTemplateId: !Ref DragenLaunchTemplate
Then in the jobqueue définition i have :
DragenJobQueue:
Type: AWS::Batch::JobQueue
Properties:
JobQueueName: dragen-queue
Priority: 100
State: ENABLED
ComputeEnvironmentOrder:
- Order: 1
ComputeEnvironment: !Ref DragenComputeEnvironmentOnDemand
- Order: 2
ComputeEnvironment: !Ref DragenComputeEnvironmentSpot
So if i understand well the on demand has the priority (so spot will never be use). But i would like to know if there is a way to override during the job launch ? Some job fit well with on demand but others are better on spot.
Related
I'm experiencing a very annoying problem. I created a CI/CD pipelines using AWS CodePipeline and CloudFormation.
This is the template.yml used by CloudFormation to create a ScheduledTask on ECS.
AWSTemplateFormatVersion: "2010-09-09"
Description: Template for deploying a ECR image on ECS
Resources:
VPC:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: "10.0.0.0/16"
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
Subnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Sub "10.0.0.0/20"
MapPublicIpOnLaunch: true
Subnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs ""]
CidrBlock: !Sub "10.0.32.0/20"
MapPublicIpOnLaunch: true
InternetGateway:
Type: "AWS::EC2::InternetGateway"
VPCGatewayAttachment:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
RouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
RouteTableAssociation1:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref Subnet1
RouteTableId: !Ref RouteTable
RouteTableAssociation2:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref Subnet2
RouteTableId: !Ref RouteTable
InternetRoute:
Type: "AWS::EC2::Route"
DependsOn: VPCGatewayAttachment
Properties:
GatewayId: !Ref InternetGateway
RouteTableId: !Ref RouteTable
DestinationCidrBlock: "0.0.0.0/0"
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: "SLAComputation"
LoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: ecs-services
Subnets:
- !Ref "Subnet1"
- !Ref "Subnet2"
SecurityGroups:
- !Ref LoadBalancerSecurityGroup
LoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref LoadBalancer
Protocol: HTTP
Port: 80
DefaultActions:
- Type: forward
TargetGroupArn: !Ref DefaultTargetGroup
LoadBalancerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for loadbalancer to services on ECS
VpcId: !Ref "VPC"
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
IpProtocol: -1
DefaultTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: default
VpcId: !Ref "VPC"
Protocol: "HTTP"
Port: "80"
CloudWatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: "sla_computation"
RetentionInDays: 1
ContainerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref "VPC"
GroupDescription: for ecs containers
SecurityGroupIngress:
- SourceSecurityGroupId: !Ref "LoadBalancerSecurityGroup"
IpProtocol: -1
Task:
Type: AWS::ECS::TaskDefinition
Properties:
Family: apis
Cpu: 1024
Memory: 2048
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn: !Ref ECSTaskExecutionRole
ContainerDefinitions:
- Name: ass001
Image: !Sub 649905970782.dkr.ecr.eu-west-1.amazonaws.com/ass001:latest
Cpu: 1024
Memory: 2048
HealthCheck:
Command: [ "CMD-SHELL", "exit 0" ]
Interval: 30
Retries: 5
Timeout: 10
StartPeriod: 30
PortMappings:
- ContainerPort: 8080
Protocol: tcp
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: "sla_computation"
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: "ass001"
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: employee-tg
VpcId: !Ref VPC
Port: 80
Protocol: HTTP
Matcher:
HttpCode: 200-299
TargetType: ip
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
ListenerArn: !Ref LoadBalancerListener
Priority: 2
Conditions:
- Field: path-pattern
Values:
- /*
Actions:
- TargetGroupArn: !Ref TargetGroup
Type: forward
ECSTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs-tasks.amazonaws.com]
Action: ["sts:AssumeRole"]
Path: /
Policies:
- PolicyName: AmazonECSTaskExecutionRolePolicy
PolicyDocument:
Statement:
- Effect: Allow
Action:
# ECS Tasks to download images from ECR
- "ecr:GetAuthorizationToken"
- "ecr:BatchCheckLayerAvailability"
- "ecr:GetDownloadUrlForLayer"
- "ecr:BatchGetImage"
# ECS tasks to upload logs to CloudWatch
- "logs:CreateLogStream"
- "logs:PutLogEvents"
Resource: "*"
TaskSchedule:
Type: AWS::Events::Rule
Properties:
Description: SLA rule ass001
Name: ass001
ScheduleExpression: cron(0/5 * * * ? *)
State: ENABLED
Targets:
- Arn:
!GetAtt ECSCluster.Arn
Id: dump-data-ecs-task
RoleArn:
!GetAtt ECSTaskExecutionRole.Arn
EcsParameters:
TaskDefinitionArn:
!Ref Task
TaskCount: 1
LaunchType: FARGATE
PlatformVersion: LATEST
NetworkConfiguration:
AwsVpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- sg-07db5ae6616a8c5fc
Subnets:
- subnet-031d0787ad492c1c4
TaskSchedule:
Type: AWS::Events::Rule
Properties:
Description: SLA rule ass002
Name: ass002
ScheduleExpression: cron(0/5 * * * ? *)
State: ENABLED
Targets:
- Arn:
!GetAtt ECSCluster.Arn
Id: dump-data-ecs-task
RoleArn:
!GetAtt ECSTaskExecutionRole.Arn
EcsParameters:
TaskDefinitionArn:
!Ref Task
TaskCount: 1
LaunchType: FARGATE
PlatformVersion: LATEST
NetworkConfiguration:
AwsVpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- sg-07db5ae6616a8c5fc
Subnets:
- subnet-031d0787ad492c1c4
Outputs:
ApiEndpoint:
Description: Employee API Endpoint
Value: !Join ["", ["http://", !GetAtt LoadBalancer.DNSName, "/employees"]]
Export:
Name: "EmployeeApiEndpoint"
The ScheduledTask is created successfully but it is not running actually. Very strange. But the strangest thing is that the ScheduledTask starts working when I click on "Edit" from the AWS console and (without making any change) I save.
The main issue I see is that you are using wrong role for your scheduled rule. It can't be !GetAtt ECSTaskExecutionRole.Arn. Instead you should create new role (or edit existing one) which has AmazonEC2ContainerServiceEventsRole AWS Managed policy.
It works after you edit in console, because AWS console will probably create the correct role in the background and use it instead of yours.
All that I am trying to do is follow this guide to get Atlassian services (Jira/JSM/Confluence) up and running on AWS https://aws-quickstart.github.io/quickstart-atlassian-jira/
I have followed all the details precisely and am currently at the point where I need to create the stack on cloudformation. When I paste in the .yaml url I get the error in the title, pointing me to an issue with the resources. I have looked through it and I'm not sure what the problem is, help would be appreciated.
Resources:
VPCStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub
- https://${QSS3BucketName}.${QSS3Region}.amazonaws.com/${QSS3KeyPrefix}submodules/quickstart-atlassian-services/templates/quickstart-vpc-for-atlassian-services.yaml
- QSS3Region: !If
- GovCloudCondition
- s3-us-gov-west-1
- s3
Parameters:
AccessCIDR: !Ref 'AccessCIDR'
AvailabilityZones: !Join
- ','
- !Ref 'AvailabilityZones'
ExportPrefix: !Ref 'ExportPrefix'
KeyPairName: !Ref 'KeyPairName'
PrivateSubnet1CIDR: !Ref 'PrivateSubnet1CIDR'
PrivateSubnet2CIDR: !Ref 'PrivateSubnet2CIDR'
PublicSubnet1CIDR: !Ref 'PublicSubnet1CIDR'
PublicSubnet2CIDR: !Ref 'PublicSubnet2CIDR'
VPCCIDR: !Ref 'VPCCIDR'
BastionHostRequired: !Ref 'BastionHostRequired'
JiraDCStack:
DependsOn: VPCStack
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub
- https://${QSS3BucketName}.${QSS3Region}.amazonaws.com/${QSS3KeyPrefix}templates/quickstart-jira-dc.template.yaml
- QSS3Region: !If ["GovCloudCondition", "s3-us-gov-west-1", "s3"]
Parameters:
CatalinaOpts: !Ref 'CatalinaOpts'
CidrBlock: !Ref 'AccessCIDR'
CloudWatchIntegration: !Ref 'CloudWatchIntegration'
ClusterNodeInstanceType: !Ref 'ClusterNodeInstanceType'
ClusterNodeMax: !Ref 'ClusterNodeMax'
ClusterNodeMin: !Ref 'ClusterNodeMin'
ClusterNodeVolumeSize: !Ref 'ClusterNodeVolumeSize'
CustomDnsName: !Ref 'CustomDnsName'
DBEngine: !Ref DBEngine
DBEngineVersion: !Ref DBEngineVersion
DBInstanceClass: !Ref 'DBInstanceClass'
DBIops: !Ref 'DBIops'
DBMasterUserPassword: !Ref 'DBMasterUserPassword'
DBMaxIdle: !Ref 'DBMaxIdle'
DBMaxWaitMillis: !Ref 'DBMaxWaitMillis'
DBMinEvictableIdleTimeMillis: !Ref 'DBMinEvictableIdleTimeMillis'
DBMinIdle: !Ref 'DBMinIdle'
DBMultiAZ: !Ref 'DBMultiAZ'
DBPassword: !Ref 'DBPassword'
DBPoolMaxSize: !Ref 'DBPoolMaxSize'
DBPoolMinSize: !Ref 'DBPoolMinSize'
DBRemoveAbandoned: !Ref 'DBRemoveAbandoned'
DBRemoveAbandonedTimeout: !Ref 'DBRemoveAbandonedTimeout'
DBStorage: !Ref 'DBStorage'
DBStorageEncrypted: !Ref 'DBStorageEncrypted'
DBStorageType: !Ref 'DBStorageType'
DBTestOnBorrow: !Ref 'DBTestOnBorrow'
DBTestWhileIdle: !Ref 'DBTestWhileIdle'
DBTimeBetweenEvictionRunsMillis: !Ref 'DBTimeBetweenEvictionRunsMillis'
DeploymentAutomationRepository: !Ref 'DeploymentAutomationRepository'
DeploymentAutomationBranch: !Ref 'DeploymentAutomationBranch'
DeploymentAutomationKeyName: !Ref 'DeploymentAutomationKeyName'
DeploymentAutomationPlaybook: !Ref 'DeploymentAutomationPlaybook'
DeploymentAutomationCustomParams: !Ref 'DeploymentAutomationCustomParams'
ExportPrefix: !Ref 'ExportPrefix'
HostedZone: !Ref 'HostedZone'
InternetFacingLoadBalancer: !Ref 'InternetFacingLoadBalancer'
JiraProduct: !Ref 'JiraProduct'
JiraVersion: !Ref 'JiraVersion'
JvmHeapOverride: !Ref 'JvmHeapOverride'
KeyPairName: !Ref 'KeyPairName'
MailEnabled: !Ref 'MailEnabled'
QSS3BucketName: !Ref 'jira-bucket-aws'
QSS3KeyPrefix: !Ref 'QSS3KeyPrefix'
SSLCertificateARN: !Ref 'SSLCertificateARN'
TomcatAcceptCount: !Ref 'TomcatAcceptCount'
TomcatContextPath: !Ref 'TomcatContextPath'
TomcatDefaultConnectorPort: !Ref 'TomcatDefaultConnectorPort'
TomcatEnableLookups: !Ref 'TomcatEnableLookups'
TomcatMaxThreads: !Ref 'TomcatMaxThreads'
TomcatMinSpareThreads: !Ref 'TomcatMinSpareThreads'
TomcatProtocol: !Ref 'TomcatProtocol'
TomcatRedirectPort: !Ref 'TomcatRedirectPort'
BastionHostRequired: !Ref 'BastionHostRequired'
Your missing the template parameters section. If you have the AWS CLI installed you can run aws cloudformation validate-template with either --template-body file://jira-cfn.yaml or --template-url https://path.to/your/cfn/template.yaml to check for template errors. You can also run the validation from the CloudFront console if you go to the editor and load/paste in the template.
Running the below command to validate the template produced this output (i saved your template as jira-cfn.yaml in the current working directory):
aws cloudformation validate-template --template-body file://jira-cfn.yaml
An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unresolved resource dependencies
[
TomcatEnableLookups,
DBIops,
SSLCertificateARN,
QSS3KeyPrefix,
JiraProduct,
TomcatMinSpareThreads,
DBMaxWaitMillis,
DBStorageEncrypted,
BastionHostRequired,
DBTestOnBorrow,
DeploymentAutomationPlaybook,
MailEnabled,
DBMultiAZ,
JiraVersion,
QSS3BucketName,
JvmHeapOverride,
DBRemoveAbandoned,
DBInstanceClass,
DBMasterUserPassword,
jira-bucket-aws,
CloudWatchIntegration,
ClusterNodeInstanceType,
DBMinIdle,
DBPassword,
ClusterNodeMin,
DeploymentAutomationCustomParams,
DBStorage,
TomcatMaxThreads,
InternetFacingLoadBalancer,
DeploymentAutomationBranch,
ClusterNodeMax,
DBMaxIdle,
TomcatAcceptCount,
DBStorageType,
DBEngine,
AccessCIDR,
TomcatDefaultConnectorPort,
DBMinEvictableIdleTimeMillis,
DeploymentAutomationKeyName,
TomcatRedirectPort,
DBTestWhileIdle,
HostedZone,
CatalinaOpts,
KeyPairName,
CustomDnsName,
TomcatContextPath,
DBRemoveAbandonedTimeout,
ClusterNodeVolumeSize,
DBPoolMinSize,
DBTimeBetweenEvictionRunsMillis,
DBEngineVersion,
ExportPrefix,
DBPoolMaxSize,
DeploymentAutomationRepository,
TomcatProtocol
] in the Resources block of the template
I am facing issue with Cloudformation on joining a list of strings after getting it from a map.
Error caused is Security Group expects a list of string.
"123456":
us-east-1:
#VPC1
VpcId: vpc-xxxx
VpcCidr: 10.78.160.0/20
SecurityGroups: sg-123
AvailabilityZones: us-east-1a,us-east-1b,us-east-1c
"123457":
us-east-1:
#VPC
VpcId: vpc-xxxx
VpcCidr: 10.78.160.0/20
SecurityGroups: sg-123,sg-124
AvailabilityZones: us-east-1a,us-east-1b,us-east-1c
LaunchConfig:
Type: AWS::AutoScaling::LaunchConfiguration
Metadata:
AWS::CloudFormation::Init:
config:
Properties:
InstanceType:
Ref: InstanceType
SecurityGroups: !Join [ ",", [!FindInMap [!Ref "AWS::AccountId", !Ref "AWS::Region", SecurityGroups], !Ref EC2SG ]]
IamInstanceProfile:
please let me know how to get the list for securitygroups and adding it to another resource
Solved by using this question
How do I use nested lists or append to a list in Cloudformation?
Lets take mappings:
Mappings:
"111222333444":
us-east-1:
SecurityGroups:
- sg-abc
- sg-xyz
us-west-2:
SecurityGroups:
- sg-1234
- sg-3456
Security groups can be referred as
SecurityGroupIds:
"Fn::FindInMap":
[!Ref AWS::AccountId, !Ref AWS::Region, SecurityGroups]
Define as array in mappings and pass the full array, this way we can avoid splitting and joining.
Full Template , tested on a Lambda
AWSTemplateFormatVersion: "2010-09-09"
Description: "Test"
Mappings:
"111222333444":
us-east-1:
SecurityGroups:
- sg-abc
- sg-xyz
us-west-2:
SecurityGroups:
- sg-1234
- sg-3456
Resources:
TestLambda:
Type: "AWS::Lambda::Function"
Properties:
Handler: com.test.MainClass::handleRequest
Runtime: java8
FunctionName: "Test-Lambda"
Code:
S3Bucket: code-bucket
S3Key: !Sub "artifacts/test.jar"
Description: "Test Lambda"
MemorySize: 512
Timeout: 60
Role: !Ref my-role-arn
VpcConfig:
SecurityGroupIds:
"Fn::FindInMap":
[!Ref AWS::AccountId, !Ref AWS::Region, SecurityGroups]
SubnetIds:
- subnet-1
- subnet-2
I'm trying to attach an ephemeral volume to a service that we run on Fargate, so it can generate some files before copying them into S3. When I launch the service without the volume information, the CloudFormation template is created successfully, and the service runs.
However, when putting the volume parameters, it fails with this error:
Model validation failed (#/Volumes: expected type: JSONArray, found: JSONObject #/ContainerDefinitions/0/MountPoints: expected type: JSONArray, found: JSONObject #/ContainerDefinitions/0/PortMappings/0/ContainerPort: expected type: Number, found: String)
And this is the template:
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Join ['-', [!Ref Env, !Ref ShortServiceName, cluster]]
TaskDefinition:
Type: AWS::ECS::TaskDefinition
DependsOn: LogGroup
Properties:
Family: !Join ['-', [!Ref Env, !Ref ShortServiceName, 'taskdefinition']]
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: !Ref Cpu
Memory: !Ref Memory
Volumes:
Name: !Ref VolumeName
DockerVolumeConfiguration:
Autoprovision: True
Scope: Task
ExecutionRoleArn: !Ref ExecutionRole
TaskRoleArn: !Ref TaskRole
ContainerDefinitions:
- Name: !Join ['-', [!Ref Env, !Ref ShortServiceName]]
Image: !Ref Image
RepositoryCredentials:
CredentialsParameter: !Ref RepositoryCredentials
PortMappings:
- ContainerPort: !Ref ContainerPort
MountPoints:
ContainerPath: "/app"
SourceVolume: !Ref VolumeName
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-region: !Ref AWS::Region
awslogs-group: !Ref LogGroup
awslogs-stream-prefix: ecs
ContainerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
Tags:
-
Key: Name
Value: !Join ['-', [!Ref ShortServiceName, 'app-sg']]
GroupDescription: !Join ['-', [!Ref ShortServiceName, ContainerSecurityGroup]]
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: !Ref ContainerPort
ToPort: !Ref ContainerPort
SourceSecurityGroupId: !Ref ManagementSecurityGroup
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Ref ServiceName
Cluster: !Ref Cluster
TaskDefinition: !Ref TaskDefinition
DeploymentConfiguration:
MinimumHealthyPercent: 50
MaximumPercent: 200
DesiredCount: !Ref DesiredCount
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
Subnets:
- !Ref AppSubnetA
- !Ref AppSubnetB
SecurityGroups:
- !Ref ManagementSecurityGroup
- !Ref ContainerSecurityGroup
LogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['/', [/ecs, !Ref Env, !Ref ServiceName]]
Outputs:
ContainerSecurityGroup:
Description: ContainerSecurityGroup
Value: !Ref ContainerSecurityGroup
I have searched high and low for the issue, yet haven't found anything related to the problem. Also, the ContainerPort parameter works fine as a string when not attaching the volume. I have also tried changing the type from String to Number but keep getting the same found JsonObject, whilst it was expecting JsonArray.
Could someone throw me in the right direction, please?
Cheers!
MountPoints should be a list of MountPoint. Thus in you case it should be (note -):
MountPoints:
- ContainerPath: "/app"
SourceVolume: !Ref VolumeName
put it as an entire JSON Object, no quotes!. Example
MyRepo:
Type: AWS::ECR::Repository
Properties:
RepositoryName: my/repo
EncryptionConfiguration: {"encryptionType": "AES256"}
DeletionPolicy: Retain
I'm recently learning ECS from AWS documents from Module Two - Deploy the Monolith | AWS.
While I read the YAML file for the CloudFormation, the file creates two EC2 instances in the cluster and also specified two public subnets in the VPC. I'm new to the VPC, so is it because of the creation of 2 EC2 instances so two public subnets are needed?
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
DesiredCapacity:
Type: Number
Default: '2'
Description: Number of instances to launch in your ECS cluster.
MaxSize:
Type: Number
Default: '2'
Description: Maximum number of instances that can be launched in your ECS cluster.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.micro
AllowedValues: [t2.micro, t2.small, t2.medium, t2.large, m3.medium, m3.large,
m3.xlarge, m3.2xlarge, m4.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m4.10xlarge,
c4.large, c4.xlarge, c4.2xlarge, c4.4xlarge, c4.8xlarge, c3.large, c3.xlarge,
c3.2xlarge, c3.4xlarge, c3.8xlarge, r3.large, r3.xlarge, r3.2xlarge, r3.4xlarge,
r3.8xlarge, i2.xlarge, i2.2xlarge, i2.4xlarge, i2.8xlarge]
ConstraintDescription: Please choose a valid instance type.
Mappings:
AWSRegionToAMI:
us-east-1:
AMIID: ami-eca289fb
us-east-2:
AMIID: ami-446f3521
us-west-1:
AMIID: ami-9fadf8ff
us-west-2:
AMIID: ami-7abc111a
eu-west-1:
AMIID: ami-a1491ad2
eu-central-1:
AMIID: ami-54f5303b
ap-northeast-1:
AMIID: ami-9cd57ffd
ap-southeast-1:
AMIID: ami-a900a3ca
ap-southeast-2:
AMIID: ami-5781be34
SubnetConfig:
VPC:
CIDR: '10.0.0.0/16'
PublicOne:
CIDR: '10.0.0.0/24'
PublicTwo:
CIDR: '10.0.1.0/24'
Resources:
# VPC into which stack instances will be placed
VPC:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: true
EnableDnsHostnames: true
CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
PublicSubnetOne:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
MapPublicIpOnLaunch: true
PublicSubnetTwo:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
MapPublicIpOnLaunch: true
InternetGateway:
Type: AWS::EC2::InternetGateway
GatewayAttachement:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref 'VPC'
InternetGatewayId: !Ref 'InternetGateway'
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref 'VPC'
PublicRoute:
Type: AWS::EC2::Route
DependsOn: GatewayAttachement
Properties:
RouteTableId: !Ref 'PublicRouteTable'
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Ref 'InternetGateway'
PublicSubnetOneRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetOne
RouteTableId: !Ref PublicRouteTable
PublicSubnetTwoRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetTwo
RouteTableId: !Ref PublicRouteTable
# ECS Resources
ECSCluster:
Type: AWS::ECS::Cluster
EcsSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: ECS Security Group
VpcId: !Ref 'VPC'
EcsSecurityGroupHTTPinbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
EcsSecurityGroupSSHinbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
EcsSecurityGroupALBports:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '31000'
ToPort: '61000'
SourceSecurityGroupId: !Ref 'EcsSecurityGroup'
CloudwatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']]
RetentionInDays: 14
ECSALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: demo
Scheme: internet-facing
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: '30'
Subnets:
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
SecurityGroups: [!Ref 'EcsSecurityGroup']
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
LaunchConfigurationName: !Ref 'ContainerInstances'
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
CreationPolicy:
ResourceSignal:
Timeout: PT15M
UpdatePolicy:
AutoScalingReplacingUpdate:
WillReplace: 'true'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !FindInMap [AWSRegionToAMI, !Ref 'AWS::Region', AMIID]
SecurityGroups: [!Ref 'EcsSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
- 'elasticloadbalancing:DeregisterTargets'
- 'elasticloadbalancing:Describe*'
- 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
- 'elasticloadbalancing:RegisterTargets'
- 'ec2:Describe*'
- 'ec2:AuthorizeSecurityGroupIngress'
Resource: '*'
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'ecs:CreateCluster'
- 'ecs:DeregisterContainerInstance'
- 'ecs:DiscoverPollEndpoint'
- 'ecs:Poll'
- 'ecs:RegisterContainerInstance'
- 'ecs:StartTelemetrySession'
- 'ecs:Submit*'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'ecr:GetAuthorizationToken'
- 'ecr:BatchGetImage'
- 'ecr:GetDownloadUrlForLayer'
Resource: '*'
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'application-autoscaling:*'
- 'cloudwatch:DescribeAlarms'
- 'cloudwatch:PutMetricAlarm'
- 'ecs:DescribeServices'
- 'ecs:UpdateService'
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
Outputs:
ClusterName:
Description: The name of the ECS cluster, used by the deploy script
Value: !Ref 'ECSCluster'
Export:
Name: !Join [':', [!Ref "AWS::StackName", "ClusterName" ]]
Url:
Description: The url at which the application is available
Value: !Join ['', [!GetAtt 'ECSALB.DNSName']]
ALBArn:
Description: The ARN of the ALB, exported for later use in creating services
Value: !Ref 'ECSALB'
Export:
Name: !Join [':', [!Ref "AWS::StackName", "ALBArn" ]]
ECSRole:
Description: The ARN of the ECS role, exports for later use in creating services
Value: !GetAtt 'ECSServiceRole.Arn'
Export:
Name: !Join [':', [!Ref "AWS::StackName", "ECSRole" ]]
VPCId:
Description: The ID of the VPC that this stack is deployed in
Value: !Ref 'VPC'
Export:
Name: !Join [':', [!Ref "AWS::StackName", "VPCId" ]]
In your example, two AZs are being used which requires two subnets (one for each AZ). This is not related to the number of EC2 instances.
A typical best practices with AWS and other cloud vendors is to use multiple availability zones (AZ) for fault tolerance. For AWS each AZ needs its own subnet. Auto scaling and load balancing will attempt to keep the number of instances the same in each AZ.
PS. If I was learning AWS, I would not start with this example. This example is very complex but very realistic for a real world deployment. There are lots of cloudformation examples that are much easy to master to start with.