AWS ECS Scheduled task not running when released by CI/CD - amazon-web-services

Question

I'm experiencing a very annoying problem. I created a CI/CD pipelines using AWS CodePipeline and CloudFormation.
This is the template.yml used by CloudFormation to create a ScheduledTask on ECS.
AWSTemplateFormatVersion: "2010-09-09"
Description: Template for deploying a ECR image on ECS
Resources:
VPC:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: "10.0.0.0/16"
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
Subnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Sub "10.0.0.0/20"
MapPublicIpOnLaunch: true
Subnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs ""]
CidrBlock: !Sub "10.0.32.0/20"
MapPublicIpOnLaunch: true
InternetGateway:
Type: "AWS::EC2::InternetGateway"
VPCGatewayAttachment:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
RouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
RouteTableAssociation1:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref Subnet1
RouteTableId: !Ref RouteTable
RouteTableAssociation2:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
SubnetId: !Ref Subnet2
RouteTableId: !Ref RouteTable
InternetRoute:
Type: "AWS::EC2::Route"
DependsOn: VPCGatewayAttachment
Properties:
GatewayId: !Ref InternetGateway
RouteTableId: !Ref RouteTable
DestinationCidrBlock: "0.0.0.0/0"
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: "SLAComputation"
LoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: ecs-services
Subnets:
- !Ref "Subnet1"
- !Ref "Subnet2"
SecurityGroups:
- !Ref LoadBalancerSecurityGroup
LoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref LoadBalancer
Protocol: HTTP
Port: 80
DefaultActions:
- Type: forward
TargetGroupArn: !Ref DefaultTargetGroup
LoadBalancerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for loadbalancer to services on ECS
VpcId: !Ref "VPC"
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
IpProtocol: -1
DefaultTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: default
VpcId: !Ref "VPC"
Protocol: "HTTP"
Port: "80"
CloudWatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: "sla_computation"
RetentionInDays: 1
ContainerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref "VPC"
GroupDescription: for ecs containers
SecurityGroupIngress:
- SourceSecurityGroupId: !Ref "LoadBalancerSecurityGroup"
IpProtocol: -1
Task:
Type: AWS::ECS::TaskDefinition
Properties:
Family: apis
Cpu: 1024
Memory: 2048
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn: !Ref ECSTaskExecutionRole
ContainerDefinitions:
- Name: ass001
Image: !Sub 649905970782.dkr.ecr.eu-west-1.amazonaws.com/ass001:latest
Cpu: 1024
Memory: 2048
HealthCheck:
Command: [ "CMD-SHELL", "exit 0" ]
Interval: 30
Retries: 5
Timeout: 10
StartPeriod: 30
PortMappings:
- ContainerPort: 8080
Protocol: tcp
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: "sla_computation"
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: "ass001"
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: employee-tg
VpcId: !Ref VPC
Port: 80
Protocol: HTTP
Matcher:
HttpCode: 200-299
TargetType: ip
ListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
ListenerArn: !Ref LoadBalancerListener
Priority: 2
Conditions:
- Field: path-pattern
Values:
- /*
Actions:
- TargetGroupArn: !Ref TargetGroup
Type: forward
ECSTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs-tasks.amazonaws.com]
Action: ["sts:AssumeRole"]
Path: /
Policies:
- PolicyName: AmazonECSTaskExecutionRolePolicy
PolicyDocument:
Statement:
- Effect: Allow
Action:
# ECS Tasks to download images from ECR
- "ecr:GetAuthorizationToken"
- "ecr:BatchCheckLayerAvailability"
- "ecr:GetDownloadUrlForLayer"
- "ecr:BatchGetImage"
# ECS tasks to upload logs to CloudWatch
- "logs:CreateLogStream"
- "logs:PutLogEvents"
Resource: "*"
TaskSchedule:
Type: AWS::Events::Rule
Properties:
Description: SLA rule ass001
Name: ass001
ScheduleExpression: cron(0/5 * * * ? *)
State: ENABLED
Targets:
- Arn:
!GetAtt ECSCluster.Arn
Id: dump-data-ecs-task
RoleArn:
!GetAtt ECSTaskExecutionRole.Arn
EcsParameters:
TaskDefinitionArn:
!Ref Task
TaskCount: 1
LaunchType: FARGATE
PlatformVersion: LATEST
NetworkConfiguration:
AwsVpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- sg-07db5ae6616a8c5fc
Subnets:
- subnet-031d0787ad492c1c4
TaskSchedule:
Type: AWS::Events::Rule
Properties:
Description: SLA rule ass002
Name: ass002
ScheduleExpression: cron(0/5 * * * ? *)
State: ENABLED
Targets:
- Arn:
!GetAtt ECSCluster.Arn
Id: dump-data-ecs-task
RoleArn:
!GetAtt ECSTaskExecutionRole.Arn
EcsParameters:
TaskDefinitionArn:
!Ref Task
TaskCount: 1
LaunchType: FARGATE
PlatformVersion: LATEST
NetworkConfiguration:
AwsVpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- sg-07db5ae6616a8c5fc
Subnets:
- subnet-031d0787ad492c1c4
Outputs:
ApiEndpoint:
Description: Employee API Endpoint
Value: !Join ["", ["http://", !GetAtt LoadBalancer.DNSName, "/employees"]]
Export:
Name: "EmployeeApiEndpoint"
The ScheduledTask is created successfully but it is not running actually. Very strange. But the strangest thing is that the ScheduledTask starts working when I click on "Edit" from the AWS console and (without making any change) I save.

Answer 1

The main issue I see is that you are using wrong role for your scheduled rule. It can't be !GetAtt ECSTaskExecutionRole.Arn. Instead you should create new role (or edit existing one) which has AmazonEC2ContainerServiceEventsRole AWS Managed policy.
It works after you edit in console, because AWS console will probably create the correct role in the background and use it instead of yours.