Crcular Dependency in Cloud Formation - amazon-web-services

I'm stuck at circular dependency loop in Cfn (ECS Stack), This can be easily resolve by segregating resources in different stack, but challenge is to resolve it within same/single stack. Spent a night solving it, still not getting any close to resolve it.
After a lot of debugging finally though of seeking some help, let me know if anyone can assist me in this, I'd really appreciate any leads or help.
`
AWSTemplateFormatVersion: '2010-09-09'
Description: This stack will deploy following resources , May
# Metadata:
Parameters:
VPC:
Description: Select One VPC available in your existing account
Type: AWS::EC2::VPC::Id
PubSubnets:
Type: 'List<AWS::EC2::Subnet::Id>'
Description: The list of PubSubnetIds in selected VPC)
PvtSubnets:
Type: 'List<AWS::EC2::Subnet::Id>'
Description: The list of PvtSubnetIds in selected VPC)
ClientName:
Type: String
Default: test
# Mappings:
# Conditions:
Resources:
ELBTargetGroup:
Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
# DependsOn:
# - ElasticLoadBalancer
Properties:
Name: "ELB-TG"
HealthCheckIntervalSeconds: 6
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Port: 80
Protocol: HTTP
UnhealthyThresholdCount: 2
VpcId: !Ref VPC
TargetType: instance
ELBSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "ELBTraffic"
GroupDescription: "Enable HTTP access on the inbound port for ELB"
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Tags:
- Key: Name
Value: ELBSecurityGroup
ElasticLoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
# DependsOn:
# - ELBSecurityGroup
Properties:
Subnets:
- !Ref PubSubnets
SecurityGroups:
- !Ref ELBSecurityGroup
ElbListener:
Type: 'AWS::ElasticLoadBalancingV2::Listener'
# DependsOn:
# - ElasticLoadBalancer
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref ELBTargetGroup
LoadBalancerArn: !Ref ElasticLoadBalancer
Port: 80
Protocol: HTTP
AsgConfig:
Type: AWS::EC2::LaunchTemplate
DependsOn:
- ELBSecurityGroup
Properties:
LaunchTemplateName: !Sub ${ClientName}-launch-template
LaunchTemplateData:
ImageId: ami-0171959e760b38d59
InstanceType: t3.medium
SecurityGroups:
- !Ref ELBSecurityGroup
#ImageId: "ami-0171959e760b38d59"
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
sudo yum install -y python-pip pip
yum install -y aws-cfn-bootstrap
/opt/aws/apitools/cfn-init-2.0-6/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource AsGroup --region ${AWS::Region}
/opt/aws/apitools/cfn-init-2.0-6/bin/cfn-init -v --stack ${AWS::StackName} --resource AsgConfig --region ${AWS::Region} -c default
AsGroup:
Type: AWS::AutoScaling::AutoScalingGroup
DependsOn:
- AsgConfig
Properties:
VPCZoneIdentifier:
- !Ref PvtSubnets
LaunchTemplate:
LaunchTemplateId: !Ref AsgConfig
Version: !GetAtt AsgConfig.LatestVersionNumber
# LaunchConfigurationName: !Ref AsgConfig
MinSize: '1'
DesiredCapacity: '2'
MaxSize: '4'
TargetGroupARNs:
- !Ref ELBTargetGroup
CapacityProvider:
Type: AWS::ECS::CapacityProvider
DependsOn:
- AsGroup
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref AsGroup
CodeCommitRepository1:
Type: AWS::CodeCommit::Repository
Properties:
RepositoryDescription: "HTML code"
RepositoryName: "HTML_code"
CodeCommitRepository2:
Type: AWS::CodeCommit::Repository
Properties:
RepositoryDescription: "Python code"
RepositoryName: "Python_code"
CodeCommitRepository3:
Type: AWS::CodeCommit::Repository
Properties:
RepositoryDescription: "Node code"
RepositoryName: "Node_code"
ECRrepo:
Type: AWS::ECR::Repository
Properties:
RepositoryName: "cfn_repo"
ECSCluster:
Type: AWS::ECS::Cluster
DependsOn:
- CapacityProvider
Properties:
CapacityProviders:
- !Ref CapacityProvider
ClusterName: !Ref ClientName
Configuration:
ExecuteCommandConfiguration:
Logging: DEFAULT
ClusterSettings:
- Name: containerInsights
Value: enabled
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
RoleName: "ecs-service-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: "ecs-execution-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
ContainerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "ContainerSecurityGroup"
GroupDescription: "Security group for container"
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Service:
Type: AWS::ECS::Service
DependsOn:
- ECSCluster
- ElasticLoadBalancer
# - ECSServiceRole
# - TaskDefinition
- ELBTargetGroup
Properties:
Cluster: !Ref ECSCluster
Role: !Ref ECSServiceRole
DesiredCount: 1
TaskDefinition: !Ref TaskDefinition
LaunchType: EC2
LoadBalancers:
- ContainerName: "deployment-container"
ContainerPort: 80
TargetGroupArn: !Ref ELBTargetGroup
AutoScalingRole:
Type: AWS::IAM::Role
Properties:
RoleName: service-auto-scaling-role
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ["sts:AssumeRole"]
Policies:
- PolicyName: service-auto-scaling-policy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ecs:DescribeServices
- ecs:UpdateService
- cloudwatch:PutMetricAlarm
- cloudwatch:DescribeAlarms
- cloudwatch:DeleteAlarms
Resource:
- "*"
ScalableTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
# DependsOn:
# -
Properties:
RoleARN: !GetAtt AutoScalingRole.Arn
ResourceId: !Sub service/${ClientName}/Service
ServiceNamespace: ecs
ScalableDimension: ecs:Service:DesiredCount
MinCapacity: 1
MaxCapacity: 5
ScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
DependsOn:
- ScalableTarget
Properties:
PolicyName: service-auto-scaling-policy
PolicyType: TargetTrackingScaling
ScalingTargetId: !Ref ScalableTarget
TargetTrackingScalingPolicyConfiguration:
PredefinedMetricSpecification:
PredefinedMetricType: ECSServiceAverageCPUUtilization
TargetValue: 80.0
TaskDefinition:
Type: AWS::ECS::TaskDefinition
# DependsOn:
# - ExecutionRole
# - Service
Properties:
Family: deployment-task
Cpu: "256"
Memory: "512"
NetworkMode: bridge
ExecutionRoleArn: !Ref ExecutionRole
ContainerDefinitions:
- Name: deployment-container
Image: cfn_repo/cfnrepo:latest
PortMappings:
- ContainerPort: 80
RequiresCompatibilities:
- EC2
# Outputs:
# outputELBTargetGroup:
# Description: A reference to the created Target Group
# Value: !Ref ELBTargetGroup
# outputELBSecurityGroup:
# Description: A reference to the created Security Group
# Value: !Ref ELBSecurityGroup
# outputElasticLoadBalancer:
# Description: A reference to the created Elastic Load Balancer
# Value: !Ref ElasticLoadBalancer
# outputElasticListener:
# Description: A reference to the created Elastic Load Balancer Listener
# Value: !Ref ElbListener
# outputAsgConfig:
# Description: Id for autoscaling launch configuration
# Value: !Ref AsgConfig
# outputAsgGroup:
# Description: Id for autoscaling group
# Value: !Ref AsgGroup
# outputECSCluster:
# Description: Cluster name
# Value: !Ref ECSCluster
`

You have a circular dependency, as follows:
AsGroup depends on AsgConfig
AsgConfig depends on ECSCluster because of echo ECS_CLUSTER=${ECSCluster} in user data
ECSCluster depends on CapacityProvider
CapacityProvider depends on AsGroup (which is #1 above)
I suggest that instead of configuring the ECSCluster with the CapacityProvider, you simply create the ECSCluster without a capacity provider (and without the DependsOn) and add a later AWS::ECS::ClusterCapacityProviderAssociations to associate the CapacityProvider with the ECSCluster.
For example (note that I have not tested this so some tweaks may be required):
CapacityProvider:
Type: AWS::ECS::CapacityProvider
DependsOn:
- AsGroup
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref AsGroup
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Ref ClientName
Configuration:
ExecuteCommandConfiguration:
Logging: DEFAULT
ClusterSettings:
- Name: containerInsights
Value: enabled
ClusterCapacityProviderAssociation:
Type: AWS::ECS::ClusterCapacityProviderAssociations
Properties:
CapacityProviders:
- !Ref CapacityProvider
Cluster: ECSCluster
DefaultCapacityProviderStrategy:
- Base: 0
Weight: 1
CapacityProvider: !Ref CapacityProvider

Related

Error connecting to EFS mount from AWS ECS fargate container?

I'm trying to setup via CloudFormation an EFS mount for self-hosted Prometheus. Below is the CloudFormation for my setup:
Resources:
ServiceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: 'Prometheus SG'
VpcId:
Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
SecurityGroupIngress:
# Allow access from the Load Balancer only
- SourceSecurityGroupId:
Fn::ImportValue: !Sub '${LBStackName}-SG-LB'
IpProtocol: tcp
FromPort: 9090
ToPort: 9090
Tags:
- Key: Name
Value: !Sub 'SG-Prometheus-LB-${Stage}'
EFSSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: 'Prometheus EFS SG'
VpcId:
Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
SecurityGroupIngress:
# Allow access from the ECS Service only
- SourceSecurityGroupId: !Ref ServiceSecurityGroup
IpProtocol: tcp
FromPort: 2049
ToPort: 2049
Tags:
- Key: Name
Value: !Sub 'SG-Prometheus-EFS-${Stage}'
MountTarget1:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId:
Fn::ImportValue: !Sub '${DataStackName}-EFSID'
SecurityGroups:
- !Ref EFSSecurityGroup
SubnetId:
Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-A1'
MountTarget2:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId:
Fn::ImportValue: !Sub '${DataStackName}-EFSID'
SecurityGroups:
- !Ref EFSSecurityGroup
SubnetId:
Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-B1'
Prometheus:
Type: AWS::ECS::Service
DependsOn: ListenerRule
Properties:
Cluster: !Ref Cluster
LaunchType: FARGATE
DesiredCount: !FindInMap [ ECSTaskDefinition, !Ref Stage, DesiredTaskCount ]
TaskDefinition: !Ref TaskDefinition
HealthCheckGracePeriodSeconds: 300
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 100
LoadBalancers:
- ContainerName: 'Prometheus-Container'
ContainerPort: 9090
TargetGroupArn: !Ref TargetGroup
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref ServiceSecurityGroup
Subnets:
- Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-A1'
- Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-B1'
EnableECSManagedTags: true
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: Prometheus
RequiresCompatibilities:
- FARGATE
Cpu: !FindInMap [ ECSTaskDefinition, !Ref Stage, CPU ]
Memory: !FindInMap [ ECSTaskDefinition, !Ref Stage, Memory ]
ContainerDefinitions:
- Name: 'Prometheus-Container'
Essential: true
Image: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${RepositoryName}:${ImageTag}'
Cpu: !FindInMap [ ECSTaskDefinition, !Ref Stage, CPU ]
Memory: !FindInMap [ ECSTaskDefinition, !Ref Stage, Memory ]
PortMappings:
- ContainerPort: 9090
MountPoints:
- SourceVolume: 'Prometheus-Volume'
ContainerPath: '/prometheus'
ReadOnly: false
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group:
Fn::ImportValue: !Sub '${DataStackName}-CW-LogsGroup'
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: 'PrometheusApp'
Volumes:
- Name: 'Prometheus-Volume'
EFSVolumeConfiguration:
FilesystemId:
Fn::ImportValue: !Sub '${DataStackName}-EFSID'
RootDirectory: "/"
TransitEncryption: ENABLED
NetworkMode: awsvpc
TaskRoleArn:
Fn::ImportValue: !Sub '${LBStackName}-TaskRoleArn'
ExecutionRoleArn: !GetAtt ExecutionRole.Arn
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
VpcId:
Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
TargetType: ip
Port: 9090
Protocol: HTTP
Matcher:
HttpCode: '200'
TargetGroupAttributes:
- Key: 'deregistration_delay.timeout_seconds'
Value: '60'
HealthCheckIntervalSeconds: 10
HealthCheckPath: /status
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 2
HealthyThresholdCount: 2
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
ListenerArn:
Fn::ImportValue: !Sub '${LBStackName}-LB-LISTENER'
Priority: 2
Conditions:
- Field: path-pattern
Values:
- /*
Actions:
- TargetGroupArn: !Ref TargetGroup
Type: forward
TaskPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: 'Prometheus-TaskPolicy'
Roles:
- Fn::ImportValue: !Sub '${LBStackName}-TaskRole'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: EnablePutMetricData
Effect: Allow
Resource: '*'
Action:
- cloudwatch:PutMetricData
ExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: ["ecs-tasks.amazonaws.com"]
Action:
- sts:AssumeRole
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Sub "Prometheus-${Stage}"
Everything gets deployed successfully but the containers die with the error message: ResourceInitializationError: failed to invoke EFS utils commands to set up EFS volumes: stderr: Failed to resolve "fs-xxxxxxxxxxx.efs.us-east-1.amazonaws.com. I've checked https://aws.amazon.com/premiumsupport/knowledge-center/fargate-unable-to-mount-efs/ and don't think we have an issue with zones.
Any ideas welcome.

Unable to register ec2 instances with ECS cluster

Error:
service was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster
I see the list of resources are properly created:
VPC, subnets, route tables, internet gateways, NatGW, EC2 instance, security groups, load balancer.
Ec2 instance is up and running but still the deployment is stuck in progress and times out with rollback state.
I added the signaling script as well:
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
Don't know what else is missing.
Cloud formation template:
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
VPCEnv:
Type: String
MinLength: 1
Description: 'The id for references to test Services created items.'
Environment:
Type: String
Description: 'Environment to create backend infra for'
KeyName:
Type: String
Description: 'Name of an existing EC2 KeyPair to enable SSH access to the ECS instances.'
DesiredCapacity:
Type: String
Default: '1'
Description: 'Number of instances to launch in your ECS cluster.'
MaxSize:
Type: String
Default: '1'
Description: Maximum number of instances that can be launched in your ECS cluster.
InstanceType:
Description: 'EC2 instance type'
Type: String
Default: 't2.medium'
BackendContainerImage:
Type: String
MinLength: 1
Version:
Type: String
MinLength: 1
AMIID:
Type: String
MinLength: 1
Resources:
ExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service: ['ecs-tasks.amazonaws.com']
Action: ['sts:AssumeRole']
Policies:
- PolicyName: !Sub test-${Environment}-execution-user-role
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action: ['ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession',
'ecs:UpdateContainerInstancesState', 'ecs:Submit*', 'ecr:GetAuthorizationToken',
'ecr:BatchCheckLayerAvailability', 'ecr:GetDownloadUrlForLayer', 'ecr:BatchGetImage',
'logs:CreateLogStream', 'logs:PutLogEvents', 'ssm:GetParameter', 'kms:Decrypt', 'ssm:GetParameters']
Resource: '*'
ECSCluster:
Type: AWS::ECS::Cluster
EcsSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: ECS Security Group
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
SecurityGroupIngress:
-
IpProtocol: tcp
FromPort: '22'
ToPort: '22'
SourceSecurityGroupId:
Fn::ImportValue: !Sub "${VPCEnv}-BastionSecurityGroup"
-
IpProtocol: tcp
FromPort: '31000'
ToPort: '61000'
SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
LoadBalancerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub "test-${Environment}-LBSecurityGroup"
GroupDescription: test service Load Balancer Security Group
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
SecurityGroupIngress:
-
IpProtocol: tcp
FromPort: '80'
ToPort: '80'
SourceSecurityGroupId:
Fn::ImportValue: !Sub "${VPCEnv}-APILoadBalancerSecurityGroup"
testServiceTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub 'test-${Environment}'
ExecutionRoleArn: !Ref ExecutionRole
ContainerDefinitions:
- Name: !Sub 'test-${Environment}-container'
Cpu: 600
Essential: 'true'
Image: !Ref BackendContainerImage
Memory: 1800
PortMappings:
- ContainerPort: 3000
ECSALBDNS:
Type: "AWS::Route53::RecordSet"
Properties:
AliasTarget:
DNSName: !GetAtt [ ECSALB, DNSName ]
HostedZoneId: !GetAtt [ ECSALB, CanonicalHostedZoneID ]
Comment: Internal DNS entry for audit service load balancer.
HostedZoneId: Z03303053NOQR6YO05FA7
Name: !Sub "api.internal.audit.service.${Environment}.altusplatform.com."
Type: A
ECSALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: !Sub "test-${Environment}-lb"
Scheme: internal
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: '20'
Subnets:
Fn::Split:
- ','
- Fn::ImportValue: !Sub "${VPCEnv}-PrivateSubnets2"
SecurityGroups:
- !Ref LoadBalancerSecurityGroup
- Fn::ImportValue : !Sub "${VPCEnv}-APILoadBalancerSecurityGroup"
ALBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ECSServiceRole
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
LoadBalancerArn: !Ref 'ECSALB'
Port: '80'
Protocol: HTTP
ECSALBListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
DependsOn: ALBListener
Properties:
Actions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
Conditions:
- Field: path-pattern
Values: [/]
ListenerArn: !Ref 'ALBListener'
Priority: 1
ECSTG:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: ECSALB
Properties:
HealthCheckIntervalSeconds: 10
HealthCheckPath: /health
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Name: !Sub "test-${Environment}-tg"
Port: 80
Protocol: HTTP
UnhealthyThresholdCount: 2
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
ECSCapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref 'ECSAutoScalingGroup'
ManagedScaling:
MaximumScalingStepSize: 10
MinimumScalingStepSize: 1
Status: ENABLED
TargetCapacity: 100
Tags:
- Key: environment
Value: !Sub '${Environment}'
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
Fn::Split:
- ','
- Fn::ImportValue: !Sub "${VPCEnv}-PrivateSubnets2"
LaunchConfigurationName: !Ref 'ContainerInstances'
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !Sub '${AMIID}'
SecurityGroups: [!Ref 'EcsSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
yum install -y awslogs jq
region=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
sed -i -e "s/region = us-east-1/region = $region/g" /etc/awslogs/awscli.conf
yum install -y https://amazon-ssm-$region.s3.amazonaws.com/latest/linux_amd64/amazon-ssm-agent.rpm
service:
Type: AWS::ECS::Service
DependsOn: ALBListener
Properties:
Cluster: !Ref 'ECSCluster'
DesiredCount: '2'
LoadBalancers:
- ContainerName: !Sub 'test-${Environment}-container'
ContainerPort: 3000
TargetGroupArn: !Ref 'ECSTG'
Role: !Ref 'ECSServiceRole'
TaskDefinition: !Ref 'testServiceTaskDefinition'
DeploymentConfiguration:
MaximumPercent: 150
MinimumHealthyPercent: 50
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 'elasticloadbalancing:DeregisterTargets',
'elasticloadbalancing:Describe*', 'elasticloadbalancing:RegisterInstancesWithLoadBalancer',
'elasticloadbalancing:RegisterTargets', 'ec2:Describe*', 'ec2:AuthorizeSecurityGroupIngress']
Resource: '*'
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
DependsOn: service
Properties:
MaxCapacity: 1
MinCapacity: 1
ResourceId: !Join ['', [service/, !Ref 'ECSCluster', /, !GetAtt [service, Name]]]
RoleARN: !GetAtt [AutoscalingRole, Arn]
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
ServiceScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: AStepPolicy
PolicyType: StepScaling
ScalingTargetId: !Ref 'ServiceScalingTarget'
StepScalingPolicyConfiguration:
AdjustmentType: PercentChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- MetricIntervalLowerBound: 0
ScalingAdjustment: 200
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession', 'ecs:UpdateContainerInstancesState',
'ecs:Submit*', 'ecr:GetAuthorizationToken', 'ecr:BatchCheckLayerAvailability', 'ecr:GetDownloadUrlForLayer', 'ecr:BatchGetImage',
'logs:CreateLogStream', 'logs:PutLogEvents']
Resource: '*'
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action: ['application-autoscaling:*', 'cloudwatch:DescribeAlarms', 'cloudwatch:PutMetricAlarm',
'ecs:DescribeServices', 'ecs:UpdateService']
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
Outputs:
ecsservice:
Value: !Ref 'service'
ecscluster:
Value: !Ref 'ECSCluster'
ECSALB:
Description: Your ALB DNS URL
Value: !Join ['', [!GetAtt [ECSALB, DNSName]]]
taskdef:
Value: !Ref 'testServiceTaskDefinition'
Exported values:
Update: Added the ECSCapacityProvider with no luck
I have encountered the above issue when configuring autoscaling with an ECS Cluster. In order for autoscaling to correctly report the instances it creates back to the ECS Cluster an autoscaling group capacity provider needs to be created:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupProvider
Name: String
Tags:
- Tag
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/asg-capacity-providers.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-capacityprovider.html

Application Load Balancer's DNS is timing out eventhough the Target group is in healthy state

I changed the status code for the health check from 200 to 302. After which the Target group turned to a healthy state. I wasn't able to get the healthy state with the 200 code. But when I try to access the DNS of the ALB. It times out and haven't been able to figure out why?
The ecs drupal instances logs provide these outputs "GET / HTTP/1.1" 302 573 "-" "ELB-HealthChecker/2.0" drupal
Any help would be much appreciated
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Name of an existing EC2 KeyPair to enable SSH access to the ECS instances.
VpcId:
Type: AWS::EC2::VPC::Id
Description: Select a VPC that allows instances access to the Internet.
SubnetId:
Type: List<AWS::EC2::Subnet::Id>
Description: Select at two subnets in your selected VPC.
DesiredCapacity:
Type: Number
Default: '1'
Description: Number of instances to launch in your ECS cluster.
MaxSize:
Type: Number
Default: '1'
Description: Maximum number of instances that can be launched in your ECS cluster.
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.medium
AllowedValues:
- t2.micro
- t2.small
- t2.medium
- t2.large
- m3.medium
- m3.large
- m3.xlarge
- m3.2xlarge
- m4.large
- m4.xlarge
- m4.2xlarge
- m4.4xlarge
- m4.10xlarge
- c4.large
- c4.xlarge
- c4.2xlarge
- c4.4xlarge
- c4.8xlarge
- c3.large
- c3.xlarge
- c3.2xlarge
- c3.4xlarge
- c3.8xlarge
- r3.large
- r3.xlarge
- r3.2xlarge
- r3.4xlarge
- r3.8xlarge
- i2.xlarge
- i2.2xlarge
- i2.4xlarge
- i2.8xlarge
ConstraintDescription: Please choose a valid instance type.
Mappings:
AWSRegionToAMI:
us-east-1:
AMIID: ami-0be13a99cd970f6a9
us-east-2:
AMIID: ami-0a9e12068cb98a01d
us-west-1:
AMIID: ami-0fa6c8d131a220017
us-west-2:
AMIID: ami-078c97cf1cefd1b38
eu-west-1:
AMIID: ami-0c9ef930279337028
eu-central-1:
AMIID: ami-065c1e34da68f2b02
ap-northeast-1:
AMIID: ami-02265963d1614d04d
ap-southeast-1:
AMIID: ami-0b68661b29b9e058c
ap-southeast-2:
AMIID: ami-00e4b147599c13588
ap-south-1:
AMIID: ami-036eaa870decb368d
Resources:
ECSCluster:
Type: AWS::ECS::Cluster
#DependsOn: ECSAutoScalingGroup
#Properties:
#CapacityProviders:
#- Nse
ECSCapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref ECSAutoScalingGroup
Name: Nse
EcsSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: ECS Security Group
VpcId: !Ref 'VpcId'
EcsSecurityGroupHTTPinbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '80'
ToPort: '80'
CidrIp: 0.0.0.0/0
EcsSecurityGroupDrupalinbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '8080'
ToPort: '8080'
CidrIp: 0.0.0.0/0
EcsSecurityGroupSQLInbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '3306'
ToPort: '3306'
CidrIp: 0.0.0.0/0
EcsSecurityGroupSSHinbound:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: 0.0.0.0/0
EcsSecurityGroupALBports:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref 'EcsSecurityGroup'
IpProtocol: tcp
FromPort: '31000'
ToPort: '61000'
SourceSecurityGroupId: !Ref 'EcsSecurityGroup'
CloudwatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [ECSLogGroup, !Ref 'AWS::StackName']]
RetentionInDays: 14
taskdefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Join ['', [!Ref 'AWS::StackName', -ecs-demo-app]]
ExecutionRoleArn: arn:aws:iam::268500393272:role/ecsTaskExecutionRole
#TaskRoleArn: arn:aws:iam::aws:policy/AmazonS3FullAccess
NetworkMode: bridge
#RequiredCompatibilities:
#- "EC2"
ContainerDefinitions:
- Name: drupal
Cpu: 256
#Essential: false
Image: drupal:latest
Memory: 512
Hostname: drupal
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref 'CloudwatchLogsGroup'
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: ecs-demo-app
MountPoints:
- ContainerPath: /var/www/html
SourceVolume: drupal-data
PortMappings:
- ContainerPort: 80
HostPort: 8080
Volumes:
- Name: drupal-data
ECSALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
#DependsOn: service
Properties:
Name: ECSALB
Scheme: internet-facing
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: '60'
Subnets: !Ref 'SubnetId'
SecurityGroups: [!Ref 'EcsSecurityGroup']
ALBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ECSServiceRole
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
#RedirectConfig:
#Protocol: "HTTP"
#Host: "#{host}"
#Path: "/#{path}"
#Query: "#{query}"
#Port: 80
#StatusCode: "HTTP_302"
LoadBalancerArn: !Ref 'ECSALB'
Port: '80'
Protocol: HTTP
ECSALBListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
DependsOn: ALBListener
Properties:
Actions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
#RedirectConfig:
#Protocol: "HTTP"
#Host: "#{host}"
#Path: "/#{path}"
#Query: "#{query}"
#Port: 80
#StatusCode: "HTTP_302"
Conditions:
- Field: path-pattern
Values: [/]
ListenerArn: !Ref 'ALBListener'
Priority: 1
ECSTG:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: ECSALB
Properties:
HealthCheckIntervalSeconds: 80
HealthCheckPath: /
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 60
HealthyThresholdCount: 2
Matcher:
HttpCode: 302
Name: ECSTG
Port: 8080
Protocol: HTTP
TargetType: instance
UnhealthyThresholdCount: 2
VpcId: !Ref 'VpcId'
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier: !Ref 'SubnetId'
#VPCZoneIdentifier:
#- subnet-0c228c2e5e42708aa
#- subnet-0bf3fcea01d2dd0a4
#- subnet-0c6a01197480771b3
LaunchConfigurationName: !Ref 'ContainerInstances'
#LoadBalancerNames:
#- ECSALB
TargetGroupARNs: [!Ref 'ECSTG']
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
CreationPolicy:
ResourceSignal:
Timeout: PT15M
UpdatePolicy:
AutoScalingReplacingUpdate:
WillReplace: 'true'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !FindInMap [AWSRegionToAMI, !Ref 'AWS::Region', AMIID]
SecurityGroups: [!Ref 'EcsSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
KeyName: !Ref 'KeyName'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum update -y
yum install tmux -y
yum install -y aws-cfn-bootstrap
#/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
#sudo cat /var/log/cloud-init-output.log
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
service:
Type: AWS::ECS::Service
DependsOn:
- ALBListener
#- ECSAutoScalingGroup
Properties:
Cluster: !Ref 'ECSCluster'
DesiredCount: '1'
HealthCheckGracePeriodSeconds: 2147483647
LoadBalancers:
- ContainerName: drupal
ContainerPort: '80'
TargetGroupArn: !Ref 'ECSTG'
#LoadBalancerName: !GetAtt ECSALB.LoadBalancerName
#LoadBalancerName: ECSALB
#NetworkConfiguration:
# AwsvpcConfiguration:
# AssignPublicIp: ENABLED
# SecurityGroups:
# - !Ref 'EcsSecurityGroup'
# Subnets: !Ref 'SubnetId'
#ServiceName: Ecs
Role: !Ref 'ECSServiceRole'
TaskDefinition: !Ref 'taskdefinition'
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 'elasticloadbalancing:DeregisterTargets',
'elasticloadbalancing:Describe*', 'elasticloadbalancing:RegisterInstancesWithLoadBalancer',
'elasticloadbalancing:RegisterTargets', 'ec2:Describe*', 'ec2:AuthorizeSecurityGroupIngress']
Resource: '*'
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
#DependsOn: service
Properties:
MaxCapacity: 3
MinCapacity: 1
ResourceId: !Join ['', [service/, !Ref 'ECSCluster', /, !GetAtt [service, Name]]]
RoleARN: !GetAtt [AutoscalingRole, Arn]
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
ServiceScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: AStepPolicy
PolicyType: StepScaling
ScalingTargetId: !Ref 'ServiceScalingTarget'
StepScalingPolicyConfiguration:
AdjustmentType: PercentChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- MetricIntervalLowerBound: 0
ScalingAdjustment: 200
ALB500sAlarmScaleUp:
Type: AWS::CloudWatch::Alarm
Properties:
EvaluationPeriods: '1'
Statistic: Average
Threshold: '10'
AlarmDescription: Alarm if our ALB generates too many HTTP 500s.
Period: '60'
AlarmActions: [!Ref 'ServiceScalingPolicy']
Namespace: AWS/ApplicationELB
Dimensions:
- Name: LoadBalancer
Value: !GetAtt
- ECSALB
- LoadBalancerFullName
ComparisonOperator: GreaterThanThreshold
MetricName: HTTPCode_ELB_5XX_Count
EC2Role:
Type: AWS::IAM::Role
Properties:
#ManagedPolicyArns:
#- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession',
'ecs:Submit*', 'logs:CreateLogStream', 'logs:PutLogEvents']
Resource: '*'
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action: ['application-autoscaling:*', 'cloudwatch:DescribeAlarms', 'cloudwatch:PutMetricAlarm',
'ecs:DescribeServices', 'ecs:UpdateService']
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
Outputs:
ecsservice:
Value: !Ref 'service'
ecscluster:
Value: !Ref 'ECSCluster'
ECSALB:
Description: Your ALB DNS URL
Value: !Join ['', [!GetAtt [ECSALB, DNSName]]]
taskdef:
Value: !Ref 'taskdefinition'
Based on the comments.
The template is fine. The ALB does not work because it is placed in private subnets along with ECS service. Assuming that private subnets are correctly setup to work with NAT gateway and access the internet, the following should be made:
Place ALB in public subnets - it must be there, as otherwise it will no be accessible from the internet.
Also double check all the route tables for NAT, public subnets, internet gateway.

Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions

I am trying to create a cluster, service and task. The error occurs in Myservice as it says Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions. What am I doing wrong? I haven't attached all associated files, I have just provided the yml file where I think the error occurs.
role.yml
---
AWSTemplateFormatVersion: 2010-09-09
Resources:
ExRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Policies:
- PolicyName: AccessECR
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ecr:BatchGetImage
- ecr:GetAuthorizationToken
- ecr:GetDownloadUrlForLayer
Resource: '*'
ContainerInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
Path: '/'
InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref ContainerInstanceRole
Outputs:
ExRole:
Description: Task excution role
Value: !Ref ExRole
Export:
Name: "ExRole"
InstanceProfile:
Description: profile for container instances
Value: !Ref InstanceProfile
Export:
Name: "InstanceProfile"
Clusterandservice.yml
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Default: wahaj(webserver)
DesiredCapacity:
Type: Number
Default: 2
MinSize:
Type: Number
Default: 1
MaxSize:
Type: Number
Default: 4
InstanceProfile:
Type: String
DefaultTargetGroup:
Type: String
Task:
Type: String
Albsg:
Type: String
VpcID:
Type: String
SubnetA:
Type: String
SubnetB:
Type: String
Resources:
MyCluster:
Type: AWS::ECS::Cluster
Properties: {}
wahajwebserver:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: wahaj-webserver
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 0
ToPort: 65535
SourceSecurityGroupId: !Ref Albsg
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId: !Ref VpcID
Myservice:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref MyCluster
DeploymentController:
Type: ECS
DesiredCount: 2
LaunchType: EC2
LoadBalancers:
- ContainerName: python
ContainerPort: 8080
TargetGroupArn: !Ref DefaultTargetGroup
Role: !Ref InstanceProfile
SchedulingStrategy: REPLICA
ServiceName: Python-service
TaskDefinition: !Ref Task
ec2instance:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y && yum install -y aws-cfn-bootstrap
echo ECS_CLUSTER=${MyCluster} >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST= >> /etc/ecs/ecs.config
/opt/aws/bin/cfn-signal -e $? \
--stack ${AWS::StackName} \
--resource myASG
--region ${AWS::Region}
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: 30
VolumeType: gp2
ImageId: ami-06e05a843071324d1
InstanceType: t2.small
IamInstanceProfile: !Ref InstanceProfile
KeyName: !Ref KeyName
SecurityGroups:
- Ref: wahajwebserver
myASG:
Type: AWS::AutoScaling::AutoScalingGroup
CreationPolicy:
ResourceSignal:
Timeout: PT5M
Count: !Ref DesiredCapacity
Properties:
#AutoScalingGroupName: myASG
MinSize: !Ref MinSize
MaxSize: !Ref MaxSize
DesiredCapacity: !Ref DesiredCapacity
HealthCheckGracePeriod: 300
LaunchConfigurationName:
Ref: ec2instance
VPCZoneIdentifier:
- !Ref SubnetA
- !Ref SubnetB
TargetGroupARNs:
- !Ref DefaultTargetGroup
The following in Myservice
Role: !Ref InstanceProfile
is incorrect. The InstanceProfile is only for ec2instance.
Try your service without the role:
Myservice:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref MyCluster
DeploymentController:
Type: ECS
DesiredCount: 2
LaunchType: EC2
LoadBalancers:
- ContainerName: python
ContainerPort: 8080
TargetGroupArn: !Ref DefaultTargetGroup
# Role: !Ref InstanceProfile # commented out
SchedulingStrategy: REPLICA
ServiceName: Python-service
TaskDefinition: !Ref Task
The ECS service role in Myservice shouldn't be required:
Prior to the introduction of a service-linked role for Amazon ECS, you were required to create an IAM role for your Amazon ECS services which granted Amazon ECS the permission it needed. This role is no longer required, however it is available if needed. For more information, see Legacy IAM Roles for Amazon ECS.
Update:
Missing \ in UserData. it should be:
/opt/aws/bin/cfn-signal -e $? \
--stack ${AWS::StackName} \
--resource myASG \
--region ${AWS::Region}

Unable to create Ec2 instance with autoscaling and register them to ECS cluster

Hi I am trying to create a cluster with service and task. I have a python app in Ecr that connects to redis container. The issue is I am unable to create ec2 instance with autoscaling. Myservice in Cluster and Autoscaling doesn't builds.
Cluster:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Alb:
Type: String
DefaultTargetGroup:
Type: String
ECSServiceRole:
Type: String
Task:
Type: String
Resources:
MyCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: Flask-redis
Myservice:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref MyCluster
DeploymentController:
Type: ECS
DesiredCount: 2
LaunchType: EC2
LoadBalancers:
- ContainerName: python
ContainerPort: 5000
TargetGroupArn: !Ref DefaultTargetGroup
Role: !Ref ECSServiceRole
SchedulingStrategy: REPLICA
ServiceName: Python-service
TaskDefinition: !Ref Task
Autoscaling:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SubnetA:
Type: String
SubnetB:
Type: String
VpcID:
Type: String
Albsg:
Type: String
Alb:
Type: String
ECSServiceRole:
Type: String
Resources:
wahajwebserver:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: wahaj-webserver
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
SourceSecurityGroupId: !Ref Albsg
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId: !Ref VpcID
ec2instance:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
# UserData:
# Fn::Base64: !Sub |
# #!/bin/bash -xe
# echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
# yum install -y aws-cfn-bootstrap
# /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-0bdcc6c05dec346bf
InstanceType: t2.small
IamInstanceProfile: !Ref ECSServiceRole
KeyName: wahaj(webserver)
SecurityGroups:
- Ref: wahajwebserver
myASG:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AvailabilityZones:
- "us-east-2a"
- "us-east-2b"
AutoScalingGroupName: myASG
LoadBalancerNames:
- Ref: Alb
MinSize: "2"
MaxSize: "2"
DesiredCapacity: "2"
HealthCheckGracePeriod: 300
LaunchConfigurationName:
Ref: ec2instance
VPCZoneIdentifier:
- Ref: "SubnetA"
- Ref: "SubnetB"
Task:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
ExRole:
Type: String
Resources:
Task:
Type: AWS::ECS::TaskDefinition
Properties:
Family: redis-python
Cpu: 1 vCPU
ExecutionRoleArn: !Ref ExRole
Memory: 1 GB
NetworkMode: bridge
RequiresCompatibilities:
- EC2
TaskRoleArn: !Ref ExRole
ContainerDefinitions:
- Essential: true
Image: redis:latest
Name: redis
- Essential: true
Image: 021146125185.dkr.ecr.us-east-2.amazonaws.com/python-app:latest
Name: python
PortMappings:
- ContainerPort: 5000
HostPort: 0
Protocol: tcp
DependsOn:
- Condition: START
ContainerName: redis
Links:
- redis
Outputs:
Task:
Description: Contains all the task specifications
Value: !Ref Task
Export:
Name: "Task"
Application Load balancer:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SubnetA:
Type: String
SubnetB:
Type: String
VpcID:
Type: String
Resources:
Albsg:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: alb-sg
VpcId: !Ref VpcID
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
GroupDescription: Security Group for demo server
Alb:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
IpAddressType: ipv4
Name: Alb
Scheme: internet-facing
SecurityGroups:
- !Ref Albsg
Subnets:
- Ref: "SubnetA"
- Ref: "SubnetB"
Type: application
DefaultTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: Alb
Properties:
Name: alb-tg
VpcId: !Ref VpcID
Port: 80
Protocol: HTTP
LoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref Alb
Port: 80
Protocol: HTTP
DefaultActions:
- Type: forward
TargetGroupArn: !Ref DefaultTargetGroup
Outputs:
Albsg:
Description: security group for application load balancer
Value: !Ref Albsg
Export:
Name: "Albsg"
Alb:
Description: application load balancer
Value: !Ref Alb
Export:
Name: "Alb"
DefaultTargetGroup:
Description: Default Target Group
Value: !Ref DefaultTargetGroup
Export:
Name: "DefaultTargetGroup"
role:
---
AWSTemplateFormatVersion: 2010-09-09
Resources:
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 'elasticloadbalancing:DeregisterTargets',
'elasticloadbalancing:Describe*', 'elasticloadbalancing:RegisterInstancesWithLoadBalancer',
'elasticloadbalancing:RegisterTargets', 'ec2:Describe*', 'ec2:AuthorizeSecurityGroupIngress']
Resource: '*'
ExRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Policies:
- PolicyName: AccessECR
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ecr:BatchGetImage
- ecr:GetAuthorizationToken
- ecr:GetDownloadUrlForLayer
Resource: '*'
Outputs:
ExRole:
Description: Task excution role
Value: !Ref ExRole
Export:
Name: "ExRole"
ECSServiceRole:
Description: application load balancer
Value: !Ref ECSServiceRole
Export:
Name: "ECSServiceRole"
I am unable to create Ec2 instance with autoscaling and register them to ECS cluster. Any help would be appreciated.
Since you haven't provided any error messages, the initial look at Autoscaling template shows several issues.
LoadBalancerNames
LoadBalancerNames:
- Ref: Alb
This is only for CLB, but you are using ALB. For ALB you should be using TargetGroupARNs parameter.
UserData
You commented out the UserData. Not sure why? It should be there as you should register the instance with the cluster. I guess due to mistakes? see below.
cfn-signal
In the UserData, you are using --resource ECSAutoScalingGroup. But it should be --resource myASG
ASG and creation policy
cfn-signal has no effect as your ASG is missing CreationPolicy.
ECSCluster
Again in the UserData: reference to ECSCluster is not defined.
ImageId
Just by looking at ami-0bdcc6c05dec346bf I'm not sure if this is ECS optimized AMI. This have to be checked.
ECSServiceRole
Its form is not shown. Could be incorrect or even fully missing.
Update and fixed template
Upon further examination many more issues were found. Just to name few:
missing instance profile
missing/wrong container instance role
incorrect security groups
Not using ImportValue
Missing some output exports
The fixed and modified templates was tested in us-east-1 and default VPC. It will not work in any other region unless AMI is adjusted. Also due to lack of your ECR image, I had to create my own with my own ports, so I also had to modify this as well to even attempt a verification.
roles
---
AWSTemplateFormatVersion: 2010-09-09
Resources:
ExRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Policies:
- PolicyName: AccessECR
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ecr:BatchGetImage
- ecr:GetAuthorizationToken
- ecr:GetDownloadUrlForLayer
Resource: '*'
ContainerInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal: {'Service': ['ec2.amazonaws.com']}
Action: ['sts:AssumeRole']
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
Path: '/'
InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref ContainerInstanceRole
Outputs:
ExRole:
Description: Task excution role
Value: !Ref ExRole
Export:
Name: "ExRole"
InstanceProfile:
Description: profile for container instances
Value: !Ref InstanceProfile
Export:
Name: "InstanceProfile"
alb
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SubnetA:
Type: AWS::EC2::Subnet::Id
SubnetB:
Type: AWS::EC2::Subnet::Id
VpcID:
Type: AWS::EC2::VPC::Id
Resources:
Albsg:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: alb-sg
VpcId: !Ref VpcID
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
Description: For traffic from Internet
GroupDescription: Security Group for demo server
Alb:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
IpAddressType: ipv4
Name: Alb
Scheme: internet-facing
SecurityGroups:
- !Ref Albsg
Subnets:
- Ref: "SubnetA"
- Ref: "SubnetB"
Type: application
DefaultTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: Alb
Properties:
Name: alb-tg
VpcId: !Ref VpcID
Port: 8080
Protocol: HTTP
LoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref Alb
Port: 80
Protocol: HTTP
DefaultActions:
- Type: forward
TargetGroupArn: !Ref DefaultTargetGroup
Outputs:
Albsg:
Description: security group for application load balancer
Value: !Ref Albsg
Export:
Name: "Albsg"
Alb:
Description: application load balancer
Value: !Ref Alb
Export:
Name: "Alb"
DefaultTargetGroup:
Description: Default Target Group
Value: !Ref DefaultTargetGroup
Export:
Name: "DefaultTargetGroup"
SubnetA:
Value: !Ref SubnetA
Export:
Name: "SubnetA"
SubnetB:
Value: !Ref SubnetB
Export:
Name: "SubnetB"
VpcID:
Value: !Ref VpcID
Export:
Name: "VpcID"
task
---
AWSTemplateFormatVersion: 2010-09-09
Resources:
Task:
Type: AWS::ECS::TaskDefinition
Properties:
Family: redis-python
Cpu: 1 vCPU
ExecutionRoleArn: !ImportValue ExRole
Memory: 1 GB
NetworkMode: bridge
RequiresCompatibilities:
- EC2
TaskRoleArn: !ImportValue ExRole
ContainerDefinitions:
- Essential: true
Image: redis:latest
Name: redis
- Essential: true
Image: xxxxxx.dkr.ecr.us-east-1.amazonaws.com/xxxxxx:latest
Name: python
PortMappings:
- ContainerPort: 8080
#HostPort: 0
Protocol: tcp
DependsOn:
- Condition: START
ContainerName: redis
Links:
- redis
Outputs:
Task:
Description: Contains all the task specifications
Value: !Ref Task
Export:
Name: "Task"
service_and_asg
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Default: wahajwebserver
DesiredCapacity:
Type: Number
Default: 2
MinSize:
Type: Number
Default: 1
MaxSize:
Type: Number
Default: 4
Resources:
MyCluster:
Type: AWS::ECS::Cluster
Properties: {}
wahajwebserver:
Type: AWS::EC2::SecurityGroup
Properties:
#GroupName: wahaj-webserver
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 0
ToPort: 65535
SourceSecurityGroupId: !ImportValue Albsg
Description: For traffic from Internet
GroupDescription: Security Group for demo server
VpcId: !ImportValue VpcID
Myservice:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref MyCluster
DeploymentController:
Type: ECS
DesiredCount: 2
LaunchType: EC2
LoadBalancers:
- ContainerName: python
ContainerPort: 8080
TargetGroupArn: !ImportValue DefaultTargetGroup
#Role: !Ref ECSServiceRole
SchedulingStrategy: REPLICA
#ServiceName: Python-service
TaskDefinition: !ImportValue Task
ec2instance:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y && yum install -y aws-cfn-bootstrap
echo ECS_CLUSTER=${MyCluster} >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST= >> /etc/ecs/ecs.config
/opt/aws/bin/cfn-signal -e $? \
--stack ${AWS::StackName} \
--resource myASG \
--region ${AWS::Region}
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: 30
VolumeType: gp2
ImageId: ami-07a63940735aebd38 # before ami-0bdcc6c05dec346bf
InstanceType: t2.small
IamInstanceProfile: !ImportValue InstanceProfile
KeyName: !Ref KeyName
SecurityGroups:
- Ref: wahajwebserver
myASG:
Type: AWS::AutoScaling::AutoScalingGroup
CreationPolicy:
ResourceSignal:
Timeout: PT5M
Count: !Ref DesiredCapacity
Properties:
#AutoScalingGroupName: myASG
MinSize: !Ref MinSize
MaxSize: !Ref MaxSize
DesiredCapacity: !Ref DesiredCapacity
HealthCheckGracePeriod: 300
LaunchConfigurationName:
Ref: ec2instance
VPCZoneIdentifier:
- !ImportValue SubnetA
- !ImportValue SubnetB
TargetGroupARNs:
- !ImportValue DefaultTargetGroup
Screenshots showing an ECS cluster with two tasks and 2 tasks (1 task on each instance)