service cc-ui-service was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster.
I too got this error and from the beginning there was not even a cluster. So maybe you can use my effort (done with Fargate but some of it could also work for ec2).
Parameters:
Image:
Description: The docker image
Type: String
Default: f00b4r
ImageVersion:
Description: The docker image version
Type: String
Default: "1.0.42"
LogGroupName:
Description: The CloudWatch log group
Type: String
Default: f00b4r-logs
Resources:
AvTestCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: NewCluster
ServiceTaskRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${AWS::StackName}-TaskRole"
Path: "/"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub "${AWS::StackName}-TaskRolePolicy"
PolicyDocument:
Statement:
- Effect: Allow
Action:
- s3:*
- dynamodb:*
Resource: "*" # TODO: Set to least privilege
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: ExecutionRoleFargate
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
ServiceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: vpc-02d42422a429042
GroupDescription: Access to the ECS Service
SecurityGroupIngress:
- CidrIp: 42.42.42.42/16
IpProtocol: -1
EcsService:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref 'TestCluster'
DesiredCount: '1'
LaunchType: FARGATE
DeploymentConfiguration:
MaximumPercent: 100
MinimumHealthyPercent: 0
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
SecurityGroups:
- !Ref ServiceSecurityGroup
Subnets:
- subnet-0345b4296042a84
- subnet-02f3452b9c142de
TaskDefinition: !Ref 'TaskDefinition'
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: new-latest
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: 256
Memory: 0.5GB
ExecutionRoleArn: !Ref ExecutionRole
TaskRoleArn: !GetAtt ServiceTaskRole.Arn
ContainerDefinitions:
- Name: new-latest
Essential: true
Image:
Fn::Join:
- ""
- - "84272424226"
- ".dkr.ecr.eu-west-1.amazonaws.com/"
- !Ref Image
- ":"
- !Ref ImageVersion
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group:
!Ref LogGroupName
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: new-latest
Memory: 128
Command: ["sh", "-c", !Join [ "", [ "echo HelloFromFargate" ] ] ]
RestoreUptimeQuotationDDBECSRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: NewPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: 'ecs:RunTask'
Resource: !Ref TaskDefinition
LogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
RetentionInDays: 30
LogGroupName:
!Ref LogGroupName
Related
I am trying to deploy a CloudFormation template that defines a ECS cluster, service, and task definition. When the service tries to start a task, it gives the following error:
service ECSService failed to launch a task with (error ECS was unable to assume the role 'arn:aws:iam:::role/ExecutionRole' that was provided for this task. Please verify that the role being passed has the proper trust relationship and permissions and that your IAM user has permissions to pass this role.).
I define the role as:
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: ExecutionRole
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service: ecs-tasks.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
Path: /myroles
And the service:
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: Cluster
Service:
Type: AWS::ECS::Service
DependsOn:
- ExecutionRole
Properties:
Cluster: !Ref Cluster
DesiredCount: 1
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- !Ref SecurityGroup
Subnets:
- !Ref PublicSubnet
ServiceName: ECSService
TaskDefinition: !Ref TaskDefinition
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- Name: Container
Image: !Ref TaskImage
Cpu: .25 vCPU
ExecutionRoleArn: !Ref ExecutionRole
Family: GoCapture
Memory: 0.5 GB
NetworkMode: awsvpc
RuntimePlatform:
CpuArchitecture: X86_64
OperatingSystemFamily: LINUX
This all seems to match the documentation. But clearly I have something wrong. What am I missing?
For reference, the full template is here. I only changed some names in the copied version here. But otherwise it should be the same at the time I originally wrote this question. Any other differences are due to pushing changes to my branch to try them out.
In the github version you have Path: /gocapture/, while in SO you don't have it. Thus as I wrote earlier, your code in question is different then your actual code.
I modify your code to remove Path: /gocapture/.
Parameters:
TaskImage:
Type: String
Resources:
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: GoCaptureExecutionRole
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service: ecs-tasks.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
#Path: /gocapture/
TaskRole:
Type: AWS::IAM::Role
Properties:
RoleName: GoCaptureTaskRole
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service: ecs-tasks.amazonaws.com
#Path: /gocapture/
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: GoCaptureSecurityGroup
GroupDescription: Security Group for Go Capture ECS Service
VpcId: !Ref Vpc
PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
CidrBlock: 10.0.0.0/16
VpcId: !Ref Vpc
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: GoCaptureCluster
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 1
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- !Ref SecurityGroup
Subnets:
- !Ref PublicSubnet
ServiceName: GoCaptureECSService
TaskDefinition: !Ref TaskDefinition
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- Name: GoCaptureContainer
Image: !Ref TaskImage
Cpu: .25 vCPU
ExecutionRoleArn: !Ref ExecutionRole
Family: GoCapture
Memory: 0.5 GB
NetworkMode: awsvpc
RuntimePlatform:
CpuArchitecture: X86_64
OperatingSystemFamily: LINUX
TaskRoleArn: !Ref TaskRole
I'm trying to write a template that configures the whole ecs fargate server and its code pipeline.
There is no problem in all other configurations, but the image is empty because it is right after creating the ecr in cloudformation, and the create ecs service refers to the empty image and the process does not end.
So I want to push the server image to ecr with code build and then ecs service create to work, but I don't know how.
Could it be possible to trigger code build or code pipeline inside cloudformation?
If not, is there any way to do docker build & push?
Yes it can be done, I used it before to perform jobs like a database restore as part of stack creation (don't ask). What you need:
A custom resource lambda which kicks off the codebuild job. It will receive the codebuild project to start via a property passed to it in the cloudformation resource definition (assuming that the codebuild is determined at deploy time; else feel free to make the lambda know about which codebuild to run in whatever way makes the most sense to you).
The endpoint to call when the custom resource is complete. A custom resource will stay in CREATE_IN_PROGRESS state until the endpoint is called. It's been a while since I've used custom resources so don't remember where it comes from but I think it's found in the event that the custom resource lambda is invoked with.
Your codebuild job needs that endpoint and needs to be able to send a (GET or POST?) request to it, on both success and failure cases (you pass different params signifying success or failure).
So the overall sequence of steps is:
Define/reference the custom resource in your template, passing in whatever properties the lambda needs.
Deploy stack, custom resource lambda is invoked.
The custom resource status goes into CREATE_IN_PROGRESS
Lambda kicks off codebuild, passing in custom resource endpoint as a param or env var, and returns.
Codebuild starts doing its work.
Until the endpoint is invoked, the custom resource will remain as CREATE_IN_PROGRESS, and the stack create/update process will wait for it, even if it takes hours.
When codebuild has finished its work, it uses curl or similar to invoke that endpoint to signal it's complete.
Custom resource status goes to CREATE_COMPLETE (assuming you've invoked the endpoint with params saying it was successful).
Stack creation completes (or moves on to any resources that were dependent on the custom resource).
Yes, it is possible to trigger the Fargate deployment after pushing the image rather than when the CloudFormation template is run.
The trick is to set the DesiredCount property of AWS::ECS::Service to zero:
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref SecG
Subnets: !Ref Subs
ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
TaskDefinition: !Ref TaskDefinition
That said, you can also choose to create a repo with an initial commit that will trigger the build as soon as the template is done executing. This requires you to upload the zipped source code to an S3 bucket and configure the CodeCommit repository like so:
Repo:
Type: AWS::CodeCommit::Repository
Properties:
Code:
BranchName: main
S3:
Bucket: some-bucket
Key: code.zip
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryDescription: Repository
Triggers:
- Name: Trigger
CustomData: The Code Repository
DestinationArn: !Ref Topic
Branches:
- main
Events: [all]
Note that the some-bucket S3 bucket needs to contain the zipped .Dockerfile and any source code without any .git directory included.
You can see my implementation of this system and the rest of the stack below:
AWSTemplateFormatVersion: '2010-09-09'
Description: CloudFormation Stack to Trigger CodeBuild via CodePipeline
Parameters:
SecG:
Description: Single security group
Type: AWS::EC2::SecurityGroup::Id
Subs:
Description: Comma separated subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ImagesFile:
Type: String
Default: images.json
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
Properties:
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
Tags:
- Key: UseWithCodeDeploy
Value: true
CodeBuildServiceRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: !Sub 'ssm-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- ssm:GetParameters
- secretsmanager:GetSecretValue
Resource: '*'
- PolicyName: !Sub 'logs-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
- PolicyName: !Sub 'ecr-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- ecr:BatchCheckLayerAvailability
- ecr:CompleteLayerUpload
- ecr:GetAuthorizationToken
- ecr:InitiateLayerUpload
- ecr:PutImage
- ecr:UploadLayerPart
- lightsail:*
Resource: '*'
- PolicyName: !Sub bkt-${ArtifactBucket}-${AWS::Region}
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- s3:ListBucket
- s3:GetBucketLocation
- s3:ListBucketVersions
- s3:GetBucketVersioning
Resource:
- !Sub arn:aws:s3:::${ArtifactBucket}
- arn:aws:s3:::some-bucket
- PolicyName: !Sub obj-${ArtifactBucket}-${AWS::Region}
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- s3:GetObject
- s3:PutObject
- s3:GetObjectAcl
- s3:PutObjectAcl
- s3:GetObjectTagging
- s3:PutObjectTagging
- s3:GetObjectVersion
- s3:GetObjectVersionAcl
- s3:PutObjectVersionAcl
Resource:
- !Sub arn:aws:s3:::${ArtifactBucket}/*
- arn:aws:s3:::some-bucket/*
CodeDeployServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Sid: '1'
Effect: Allow
Principal:
Service:
- codedeploy.us-east-1.amazonaws.com
- codedeploy.eu-west-1.amazonaws.com
Action: sts:AssumeRole
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeDeployRoleForECS
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRoleForLambda
CodeDeployRolePolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Sub 'CDPolicy-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Statement:
- Effect: Allow
Resource:
- '*'
Action:
- ec2:Describe*
- Effect: Allow
Resource:
- '*'
Action:
- autoscaling:CompleteLifecycleAction
- autoscaling:DeleteLifecycleHook
- autoscaling:DescribeLifecycleHooks
- autoscaling:DescribeAutoScalingGroups
- autoscaling:PutLifecycleHook
- autoscaling:RecordLifecycleActionHeartbeat
Roles:
- !Ref CodeDeployServiceRole
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: codepipeline.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: !Sub 'root-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Resource:
- !Sub 'arn:aws:s3:::${ArtifactBucket}/*'
- !Sub 'arn:aws:s3:::${ArtifactBucket}'
Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
- Resource: "*"
Effect: Allow
Action:
- ecs:*
- Resource: "*"
Effect: Allow
Action:
- iam:PassRole
Condition:
StringLike:
iam:PassedToService:
- ecs-tasks.amazonaws.com
- Resource: !GetAtt Build.Arn
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
- codebuild:BatchGetBuildBatches
- codebuild:StartBuildBatch
- Resource: !GetAtt Repo.Arn
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetRepository
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
AmazonCloudWatchEventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: cwe-pipeline-execution
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- !GetAtt Repo.Arn
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- main
Targets:
-
Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn
Id: codepipeline-Pipeline
Topic:
Type: AWS::SNS::Topic
Properties:
Subscription:
- Endpoint: user#example.com
Protocol: email
TopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Sid: AllowPublish
Effect: Allow
Principal:
Service:
- 'codestar-notifications.amazonaws.com'
Action:
- 'SNS:Publish'
Resource:
- !Ref Topic
Topics:
- !Ref Topic
Repo:
Type: AWS::CodeCommit::Repository
Properties:
Code:
BranchName: main
S3:
Bucket: some-bucket
Key: code.zip
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryDescription: Repository
Triggers:
- Name: Trigger
CustomData: The Code Repository
DestinationArn: !Ref Topic
Branches:
- main
Events: [all]
RepoUser:
Type: AWS::IAM::User
Properties:
Path: '/'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeCommitPowerUser
RepoUserKey:
Type: AWS::IAM::AccessKey
Properties:
UserName:
!Ref RepoUser
Registry:
Type: AWS::ECR::Repository
Properties:
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryPolicyText:
Version: '2012-10-17'
Statement:
- Sid: AllowPushPull
Effect: Allow
Principal:
AWS:
- !GetAtt CodeDeployServiceRole.Arn
Action:
- ecr:GetDownloadUrlForLayer
- ecr:BatchGetImage
- ecr:BatchCheckLayerAvailability
- ecr:PutImage
- ecr:InitiateLayerUpload
- ecr:UploadLayerPart
- ecr:CompleteLayerUpload
Build:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
phases:
pre_build:
commands:
- echo "[`date`] PRE_BUILD"
- echo "Logging in to Amazon ECR..."
- aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.$REGION.amazonaws.com
- IMAGE_URI="$ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:$TAG"
build:
commands:
- echo "[`date`] BUILD"
- echo "Building Docker Image..."
- docker build -t $REPO:$TAG .
- docker tag $REPO:$TAG $IMAGE_URI
post_build:
commands:
- echo "[`date`] POST_BUILD"
- echo "Pushing Docker Image..."
- docker push $IMAGE_URI
- echo Writing image definitions file...
- printf '[{"name":"svc","imageUri":"%s"}]' $IMAGE_URI > $FILE
artifacts:
files: $FILE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/standard:6.0
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: REGION
Type: PLAINTEXT
Value: !Ref AWS::Region
- Name: ACCOUNT
Type: PLAINTEXT
Value: !Ref AWS::AccountId
- Name: TAG
Type: PLAINTEXT
Value: latest
- Name: REPO
Type: PLAINTEXT
Value: !Ref Registry
- Name: FILE
Type: PLAINTEXT
Value: !Ref ImagesFile
PrivilegedMode: true
Name: !Ref AWS::StackName
ServiceRole: !GetAtt CodeBuildServiceRole.Arn
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
ArtifactStore:
Type: S3
Location: !Ref ArtifactBucket
Stages:
- Name: Source
Actions:
- Name: Site
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
Configuration:
RepositoryName: !GetAtt Repo.Name
BranchName: main
PollForSourceChanges: 'false'
InputArtifacts: []
OutputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: Build
Actions:
- Name: Docker
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
Configuration:
ProjectName: !Ref Build
InputArtifacts:
- Name: SourceArtifact
OutputArtifacts:
- Name: BuildArtifact
RunOrder: 1
- Name: Deploy
Actions:
- Name: Fargate
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: ECS
Configuration:
ClusterName: !Ref Cluster
FileName: !Ref ImagesFile
ServiceName: !GetAtt Service.Name
InputArtifacts:
- Name: BuildArtifact
RunOrder: 1
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
FargateTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
TaskRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
-
Name: svc
Image: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${Registry}:latest
PortMappings:
- ContainerPort: 8080
Cpu: 256
ExecutionRoleArn: !Ref FargateTaskExecutionRole
Memory: 512
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
RuntimePlatform:
CpuArchitecture: ARM64
OperatingSystemFamily: LINUX
TaskRoleArn: !Ref TaskRole
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref SecG
Subnets: !Ref Subs
ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
TaskDefinition: !Ref TaskDefinition
Outputs:
ArtifactBucketName:
Description: ArtifactBucket S3 Bucket Name
Value: !Ref ArtifactBucket
ArtifactBucketSecureUrl:
Description: ArtifactBucket S3 Bucket Domain Name
Value: !Sub 'https://${ArtifactBucket.DomainName}'
ClusterName:
Value: !Ref Cluster
ServiceName:
Value: !GetAtt Service.Name
RepoUserAccessKey:
Description: S3 User Access Key
Value: !Ref RepoUserKey
RepoUserSecretKey:
Description: S3 User Secret Key
Value: !GetAtt RepoUserKey.SecretAccessKey
BuildArn:
Description: CodeBuild URL
Value: !GetAtt Build.Arn
RepoArn:
Description: CodeCommit Repository ARN
Value: !GetAtt Repo.Arn
RepoName:
Description: CodeCommit Repository NAme
Value: !GetAtt Repo.Name
RepoCloneUrlHttp:
Description: CodeCommit HTTP Clone URL
Value: !GetAtt Repo.CloneUrlHttp
RepoCloneUrlSsh:
Description: CodeCommit SSH Clone URL
Value: !GetAtt Repo.CloneUrlSsh
PipelineUrl:
Description: CodePipeline URL
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
RegistryUri:
Description: ECR Repository URI
Value: !GetAtt Registry.RepositoryUri
TopicArn:
Description: CodeCommit Notification SNS Topic ARN
Value: !Ref Topic
Hope this helps!
Error:
service was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster
I see the list of resources are properly created:
VPC, subnets, route tables, internet gateways, NatGW, EC2 instance, security groups, load balancer.
Ec2 instance is up and running but still the deployment is stuck in progress and times out with rollback state.
I added the signaling script as well:
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
Don't know what else is missing.
Cloud formation template:
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
VPCEnv:
Type: String
MinLength: 1
Description: 'The id for references to test Services created items.'
Environment:
Type: String
Description: 'Environment to create backend infra for'
KeyName:
Type: String
Description: 'Name of an existing EC2 KeyPair to enable SSH access to the ECS instances.'
DesiredCapacity:
Type: String
Default: '1'
Description: 'Number of instances to launch in your ECS cluster.'
MaxSize:
Type: String
Default: '1'
Description: Maximum number of instances that can be launched in your ECS cluster.
InstanceType:
Description: 'EC2 instance type'
Type: String
Default: 't2.medium'
BackendContainerImage:
Type: String
MinLength: 1
Version:
Type: String
MinLength: 1
AMIID:
Type: String
MinLength: 1
Resources:
ExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service: ['ecs-tasks.amazonaws.com']
Action: ['sts:AssumeRole']
Policies:
- PolicyName: !Sub test-${Environment}-execution-user-role
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action: ['ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession',
'ecs:UpdateContainerInstancesState', 'ecs:Submit*', 'ecr:GetAuthorizationToken',
'ecr:BatchCheckLayerAvailability', 'ecr:GetDownloadUrlForLayer', 'ecr:BatchGetImage',
'logs:CreateLogStream', 'logs:PutLogEvents', 'ssm:GetParameter', 'kms:Decrypt', 'ssm:GetParameters']
Resource: '*'
ECSCluster:
Type: AWS::ECS::Cluster
EcsSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: ECS Security Group
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
SecurityGroupIngress:
-
IpProtocol: tcp
FromPort: '22'
ToPort: '22'
SourceSecurityGroupId:
Fn::ImportValue: !Sub "${VPCEnv}-BastionSecurityGroup"
-
IpProtocol: tcp
FromPort: '31000'
ToPort: '61000'
SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
LoadBalancerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub "test-${Environment}-LBSecurityGroup"
GroupDescription: test service Load Balancer Security Group
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
SecurityGroupIngress:
-
IpProtocol: tcp
FromPort: '80'
ToPort: '80'
SourceSecurityGroupId:
Fn::ImportValue: !Sub "${VPCEnv}-APILoadBalancerSecurityGroup"
testServiceTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub 'test-${Environment}'
ExecutionRoleArn: !Ref ExecutionRole
ContainerDefinitions:
- Name: !Sub 'test-${Environment}-container'
Cpu: 600
Essential: 'true'
Image: !Ref BackendContainerImage
Memory: 1800
PortMappings:
- ContainerPort: 3000
ECSALBDNS:
Type: "AWS::Route53::RecordSet"
Properties:
AliasTarget:
DNSName: !GetAtt [ ECSALB, DNSName ]
HostedZoneId: !GetAtt [ ECSALB, CanonicalHostedZoneID ]
Comment: Internal DNS entry for audit service load balancer.
HostedZoneId: Z03303053NOQR6YO05FA7
Name: !Sub "api.internal.audit.service.${Environment}.altusplatform.com."
Type: A
ECSALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: !Sub "test-${Environment}-lb"
Scheme: internal
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: '20'
Subnets:
Fn::Split:
- ','
- Fn::ImportValue: !Sub "${VPCEnv}-PrivateSubnets2"
SecurityGroups:
- !Ref LoadBalancerSecurityGroup
- Fn::ImportValue : !Sub "${VPCEnv}-APILoadBalancerSecurityGroup"
ALBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ECSServiceRole
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
LoadBalancerArn: !Ref 'ECSALB'
Port: '80'
Protocol: HTTP
ECSALBListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
DependsOn: ALBListener
Properties:
Actions:
- Type: forward
TargetGroupArn: !Ref 'ECSTG'
Conditions:
- Field: path-pattern
Values: [/]
ListenerArn: !Ref 'ALBListener'
Priority: 1
ECSTG:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: ECSALB
Properties:
HealthCheckIntervalSeconds: 10
HealthCheckPath: /health
HealthCheckProtocol: HTTP
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Name: !Sub "test-${Environment}-tg"
Port: 80
Protocol: HTTP
UnhealthyThresholdCount: 2
VpcId:
Fn::ImportValue: !Sub "${VPCEnv}-VPC"
ECSCapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref 'ECSAutoScalingGroup'
ManagedScaling:
MaximumScalingStepSize: 10
MinimumScalingStepSize: 1
Status: ENABLED
TargetCapacity: 100
Tags:
- Key: environment
Value: !Sub '${Environment}'
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
Fn::Split:
- ','
- Fn::ImportValue: !Sub "${VPCEnv}-PrivateSubnets2"
LaunchConfigurationName: !Ref 'ContainerInstances'
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !Sub '${AMIID}'
SecurityGroups: [!Ref 'EcsSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
yum install -y awslogs jq
region=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
sed -i -e "s/region = us-east-1/region = $region/g" /etc/awslogs/awscli.conf
yum install -y https://amazon-ssm-$region.s3.amazonaws.com/latest/linux_amd64/amazon-ssm-agent.rpm
service:
Type: AWS::ECS::Service
DependsOn: ALBListener
Properties:
Cluster: !Ref 'ECSCluster'
DesiredCount: '2'
LoadBalancers:
- ContainerName: !Sub 'test-${Environment}-container'
ContainerPort: 3000
TargetGroupArn: !Ref 'ECSTG'
Role: !Ref 'ECSServiceRole'
TaskDefinition: !Ref 'testServiceTaskDefinition'
DeploymentConfiguration:
MaximumPercent: 150
MinimumHealthyPercent: 50
ECSServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['elasticloadbalancing:DeregisterInstancesFromLoadBalancer', 'elasticloadbalancing:DeregisterTargets',
'elasticloadbalancing:Describe*', 'elasticloadbalancing:RegisterInstancesWithLoadBalancer',
'elasticloadbalancing:RegisterTargets', 'ec2:Describe*', 'ec2:AuthorizeSecurityGroupIngress']
Resource: '*'
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
DependsOn: service
Properties:
MaxCapacity: 1
MinCapacity: 1
ResourceId: !Join ['', [service/, !Ref 'ECSCluster', /, !GetAtt [service, Name]]]
RoleARN: !GetAtt [AutoscalingRole, Arn]
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
ServiceScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: AStepPolicy
PolicyType: StepScaling
ScalingTargetId: !Ref 'ServiceScalingTarget'
StepScalingPolicyConfiguration:
AdjustmentType: PercentChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- MetricIntervalLowerBound: 0
ScalingAdjustment: 200
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action: ['ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession', 'ecs:UpdateContainerInstancesState',
'ecs:Submit*', 'ecr:GetAuthorizationToken', 'ecr:BatchCheckLayerAvailability', 'ecr:GetDownloadUrlForLayer', 'ecr:BatchGetImage',
'logs:CreateLogStream', 'logs:PutLogEvents']
Resource: '*'
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action: ['application-autoscaling:*', 'cloudwatch:DescribeAlarms', 'cloudwatch:PutMetricAlarm',
'ecs:DescribeServices', 'ecs:UpdateService']
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
Outputs:
ecsservice:
Value: !Ref 'service'
ecscluster:
Value: !Ref 'ECSCluster'
ECSALB:
Description: Your ALB DNS URL
Value: !Join ['', [!GetAtt [ECSALB, DNSName]]]
taskdef:
Value: !Ref 'testServiceTaskDefinition'
Exported values:
Update: Added the ECSCapacityProvider with no luck
I have encountered the above issue when configuring autoscaling with an ECS Cluster. In order for autoscaling to correctly report the instances it creates back to the ECS Cluster an autoscaling group capacity provider needs to be created:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupProvider
Name: String
Tags:
- Tag
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/asg-capacity-providers.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-capacityprovider.html
I'm trying to bring up a CloudFormation stack to run an ECS service on EC2. My stack creation fails upon creation of the Auto Scaling group and the error in the console Activity tab shows:
Status: Failed
Description:
Launching a new EC2 instance. Status Reason: The requested configuration is currently not supported. Please check the documentation for supported configurations. Launching EC2 instance failed.
Cause:
At 2020-10-26T23:47:46Z a user request update of AutoScalingGroup constraints to min: 1, max: 1, desired: 1 changing the desired capacity from 0 to 1. At 2020-10-26T23:47:48Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 0 to 1.
I have tried to play around with my CFT but with no luck so far.
AWSTemplateFormatVersion: '2010-09-09'
Description: Hhhhhhhhh Feed Services Containers
Parameters:
VpcId:
Type: String
SubnetId:
Type: String
ECSCluster:
Type: String
Default: dev-ecs
EcsSecurityGroup:
Type: String
Default: sg-74cb7b0c
FeedServicesSecurityGroup:
Type: String
Default: sg-0a695957eec3371bc
DesiredCount:
Type: Number
Default: '1'
EC2InstanceAMI:
Type: String
Default: 'ami-0dba2cb6798deb6d8'
InstanceType:
Type: String
Default: c6g.4xlarge
KeyName:
Type: String
Default: devops
Color:
Type: String
AllowedValues: ['blue', 'green']
Description: The deployment color
Default: 'blue'
XxxRouteTableId:
Type: String
Default: rtb-03eeb623aac1c1ccf
Resources:
YyyXxxLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [/ecs/feed-services-Yyy-Xxx, !Ref Color]]
YyyStableLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [/ecs/feed-services-Yyy-stable, !Ref Color]]
ZzzXxxLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [/ecs/feed-services-Zzz-Xxx, !Ref Color]]
ZzzStableLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [/ecs/feed-services-Zzz-stable, !Ref Color]]
WwwXxxLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Join ['-', [/ecs/feed-services-Www-Xxx, !Ref Color]]
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: feed-services
ExecutionRoleArn: arn:aws:iam::xxxxxxxxx:role/ecs-task-execution-role
TaskRoleArn: !Ref FeedServicesRole
ContainerDefinitions:
- Name: feed-services-Yyy-Xxx
Image: xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/feed-services/feed-services-Yyy-Xxx
Essential: True
Memory: 512
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref YyyXxxLogsGroup
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
- Name: feed-services-Yyy-stable
Image: xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/feed-services/feed-services-Yyy-stable
Essential: True
Memory: 512
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref YyyStableLogsGroup
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
- Name: feed-services-Zzz-Xxx
Image: xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/feed-services/feed-services-Zzz-Xxx
Essential: True
Memory: 8192
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref ZzzXxxLogsGroup
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
- Name: feed-services-Zzz-stable
Image: xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/feed-services/feed-services-Zzz-stable
Essential: True
Memory: 512
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref ZzzStableLogsGroup
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
- Name: feed-services-Www-Xxx
Image: xxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/feed-services/feed-services-Www-Xxx
Essential: True
Memory: 512
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref WwwXxxLogsGroup
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
NetworkMode: awsvpc
FeedServicesRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ['ec2.amazonaws.com']
Action: ['sts:AssumeRole']
Policies:
- PolicyName: !Join ['-', [feed-services, !Ref Color, read-secrets]]
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'secretsmanager:ListSecrets'
- 'secretsmanager:DescribeSecret'
- 'secretsmanager:GetRandomPassword'
- 'secretsmanager:GetResourcePolicy'
- 'secretsmanager:GetSecretValue'
- 'secretsmanager:ListSecretVersionIds'
Resource: ['arn:aws:secretsmanager:us-east-1:xxxxxxxxx:secret:prod/feed-services']
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier: [!Ref SubnetId]
LaunchConfigurationName: !Ref ContainerInstances
MinSize: '1'
MaxSize: '1'
DesiredCapacity: '1'
CreationPolicy:
ResourceSignal:
Timeout: PT15M
UpdatePolicy:
AutoScalingReplacingUpdate:
WillReplace: 'true'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
LaunchConfigurationName: !Join ['-', [feed-services, !Ref Color, launch-configuration]]
AssociatePublicIpAddress: True
ImageId: !Ref EC2InstanceAMI
SecurityGroups: [!Ref FeedServicesSecurityGroup]
InstanceType: !Ref InstanceType
IamInstanceProfile: !Ref EC2InstanceProfile
PlacementTenancy: default
KeyName: !Ref KeyName
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
FeedServices:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref ECSCluster
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 100
DesiredCount: !Ref DesiredCount
LaunchType: EC2
NetworkConfiguration:
AwsVpcConfiguration:
AssignPublicIp: DISABLED
SecurityGroups: [!Ref FeedServicesSecurityGroup]
Subnets: [!Ref SubnetId]
ServiceName: !Join ['-', [feed-services, !Ref Color]]
TaskDefinition: !Ref TaskDefinition
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
DependsOn: FeedServices
Properties:
MaxCapacity: 1
MinCapacity: 1
ResourceId: !Join [ '', [ feed-services/, !Ref 'ECSCluster', /, !GetAtt [ FeedServices, Name ] ] ]
RoleARN: !GetAtt [ AutoscalingRole, Arn ]
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ ec2.amazonaws.com ]
Action: [ 'sts:AssumeRole' ]
Path: /
Policies:
- PolicyName: !Join ['-', [feed-services, !Ref Color, ecs-role]]
PolicyDocument:
Statement:
- Effect: Allow
Action: [ 'ecs:CreateCluster', 'ecs:DeregisterContainerInstance', 'ecs:DiscoverPollEndpoint',
'ecs:Poll', 'ecs:RegisterContainerInstance', 'ecs:StartTelemetrySession',
'ecs:Submit*', 'logs:CreateLogStream', 'logs:PutLogEvents' ]
Resource: '*'
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ application-autoscaling.amazonaws.com ]
Action: [ 'sts:AssumeRole' ]
Path: /
Policies:
- PolicyName: !Join ['-', [feed-services, !Ref Color, autoscaling-role]]
PolicyDocument:
Statement:
- Effect: Allow
Action: [ 'application-autoscaling:*', 'cloudwatch:DescribeAlarms', 'cloudwatch:PutMetricAlarm',
'ecs:DescribeServices', 'ecs:UpdateService' ]
Resource: '*'
SubnetRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref XxxRouteTableId
SubnetId: !Ref SubnetId
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [ !Ref 'EC2Role' ]
Outputs:
feedservices:
Value: !Ref FeedServices
taskdefinition:
Value: !Ref TaskDefinition
Based on your parameter defaults, you're trying to launch an Ubuntu Server 20.04 x86 AMI (ami-0dba2cb6798deb6d8) on an instance type (c6g.4xlarge) that requires an ARM based AMI.
Try switching the AMI to ami-0ea142bd244023692, which (at the time of this writing) is the ARM based AMI for Ubuntu Server 20.04
I have turned my original cloudformation stack template into multiple child templates which I then call from a master template. One of the child templates possesses the following snippet, for which I'm getting the error:
An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unresolved resource dependencies [GeneralPurposeContainerRole] in the Resources block of the template
BatchResourcesStack (child stack):
---
AWSTemplateFormatVersion: '2010-09-09'
Description: batch resources stack.
Parameters:
GPCEName:
Type: String
GPCEMaxVcpus:
Type: Number
Description: Max number of VCPUs for entire cluster, there are caveats to this
GPCEMinVcpus:
Type: Number
Description: Min number of VCPUs for entire cluster, there are caveats to this
GPCEDesiredVcpus:
Type: Number
Description: Desired number of VCPUs for entire cluster, there are caveats to this
GPCEVpcId:
Type: String
GPCESubnetAZ1:
Type: String
GPCEAmi:
Type: String
GPCEInstanceTypes:
Type: CommaDelimitedList
GPCESSHKeyPair:
Type: String
StackUID:
Type: String
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Subnet:
Type: AWS::EC2::Subnet
Resources:
BatchServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: batch.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSBatchServiceRole
IamInstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Roles:
- !Ref 'EcsInstanceRole'
EcsInstanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2008-10-17'
Statement:
- Sid: ''
Effect: Allow
Principal:
Service: ec2.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
GeneralPurposeContainerRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
Path: '/'
Policies:
- PolicyName: ContainerS3Access
PolicyDocument:
Statement:
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:DeleteObject
- s3:List*
Resource:
# TODO: Make these non psychcore-specific
- arn:aws:s3:::pipeline-validation/*
- arn:aws:s3:::wgs-pipeline-vqsr-test/*
- arn:aws:s3:::test-references/*
- arn:aws:s3:::psychcore-pipelines/output/*
- arn:aws:s3:::psychcore-pipelines/validation/samples/*
- arn:aws:s3:::psychcore-data/reference_indexs/*
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-data
- arn:aws:s3:::*
- Effect: Allow
Action:
- s3:List*
- s3:ListMultipartUploadParts
- s3:AbortMultipartUpload
- s3:ListBucketMultipartUploads
Resource:
- arn:aws:s3:::pipeline-validation
- arn:aws:s3:::wgs-pipeline-vqsr-test
- arn:aws:s3:::test-references
- arn:aws:s3:::psychcore-pipelines/output/
- arn:aws:s3:::psychcore-pipelines/validation/samples/
- arn:aws:s3:::psychcore-data/reference_indexs/
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-pipelines
- arn:aws:s3:::psychcore-data
- arn:aws:s3:::*
GeneralPurposeComputeEnvironment:
Type: "AWS::Batch::ComputeEnvironment"
Properties:
Type: MANAGED
ComputeEnvironmentName: !Join
- '-'
- - !Ref GPCEName
- !Ref StackUID
ComputeResources:
MinvCpus:
Ref: GPCEMinVcpus
MaxvCpus:
Ref: GPCEMaxVcpus
DesiredvCpus:
Ref: GPCEDesiredVcpus
SecurityGroupIds:
- Ref: SecurityGroup
Subnets:
- Ref: Subnet
Type: 'EC2'
ImageId:
Ref: GPCEAmi
InstanceRole:
Ref: IamInstanceProfile
InstanceTypes:
Ref: GPCEInstanceTypes
Ec2KeyPair:
Ref: GPCESSHKeyPair
Tags:
Key: Name
Value: "VariantCallingBatchComputeEnvironment"
ServiceRole:
Ref: BatchServiceRole
State: ENABLED
DependsOn:
- SecurityGroup
- Subnet
- IamInstanceProfile
- BatchServiceRole
GeneralPurposeQueue:
Type: "AWS::Batch::JobQueue"
Properties:
ComputeEnvironmentOrder:
- Order: 1
ComputeEnvironment: !Ref GeneralPurposeComputeEnvironment
Priority: 1
State: ENABLED
JobQueueName: !Join
- '-'
- - "GeneralPurposeQueue"
- !Ref StackUID
DependsOn:
- GeneralPurposeComputeEnvironment
- BatchServiceRole
Parent.yaml (contains parts relevant to above child stack):
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Master template for wgs-pipeline. Calls to other stack templates."
Parameters:
GPCEName:
Default: 'GeneralPurposeVariantCallingCE'
Type: String
GPCEMaxVcpus:
Default: 128
Type: Number
Description: Max number of VCPUs for entire cluster, there are caveats to this
GPCEMinVcpus:
Default: 0
Type: Number
Description: Min number of VCPUs for entire cluster, there are caveats to this
GPCEDesiredVcpus:
Default: 0
Type: Number
Description: Desired number of VCPUs for entire cluster, there are caveats to this
GPCEVpcId:
Type: String
GPCESubnetAZ1:
Default: 'us-east-1a'
Type: String
GPCEAmi:
Default: "ami-ce6cdfb4"
Type: String
GPCEInstanceTypes:
Default: "i3.xlarge, i3.2xlarge, i3.4xlarge, i3.8xlarge, i3.16xlarge"
Type: CommaDelimitedList
GPCESSHKeyPair:
Type: String
StackUID:
Default: "1234"
Type: String
Resources:
Subnet:
Type: AWS::EC2::Subnet
Properties:
CidrBlock: 10.0.0.0/24
VpcId: !Ref 'VPC'
AvailabilityZone: !Ref GPCESubnetAZ1
MapPublicIpOnLaunch: 'True'
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: EC2 Security Group for instances launched in the VPC by Batch
VpcId: !Ref 'VPC'
BatchResourcesStack:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
GPCEName:
Ref: GPCEName
GPCEMaxVcpus:
Ref: GPCEMaxVcpus
GPCEMinVcpus:
Ref: GPCEMinVcpus
GPCEDesiredVcpus:
Ref: GPCEDesiredVcpus
GPCEVpcId:
Ref: GPCEVpcId
GPCESubnetAZ1:
Ref: GPCESubnetAZ1
GPCEAmi:
Ref: GPCEAmi
GPCEInstanceTypes:
Ref: GPCEInstanceTypes
GPCESSHKeyPair:
Ref: GPCESSHKeyPair
StackUID:
Ref: StackUID
SecurityGroup:
Ref: SecurityGroup
Subnet:
Ref: Subnet
TemplateURL: https://s3.amazonaws.com/CFNTemplate/batch_resources.stack.yaml
Timeout: "100"
I don't understand what the error is specifically pointing to. I put the entire Batch-child.yaml file through a YAML validator and it passed, so it shouldn't be from a formatting/indention error per se. Also, the GeneralPurposeContainerRole resource does not get referenced to anywhere else in the template, nor even in the parent stack template.