Why can't I run multiple tasks in AWS ECS? - amazon-web-services

Question

I'm working on a file processing system where files can be uploaded to S3 and then processed in a container. I have been using triggering ECS to run tasks from lambda and passing a few environment variables.
S3 -> Lambda -> ECS
I'm running into a problem where I can't seem to run more than 1 task at once. If a task is already running then any subsequent tasks that get run are stuck in "PROVISIONING" and eventually disappear altogether.
Here is my lambda function that runs the ECS task:
const params: RunTaskRequest = {
launchType: "FARGATE",
cluster: "arn:aws:ecs:us-east-1:XXXXXXX:cluster/FileProcessingCluster",
taskDefinition: "XXX",
networkConfiguration: {
awsvpcConfiguration: {
subnets: [
"subnet-XXX",
"subnet-XXX"
],
securityGroups: [
"..."
],
assignPublicIp: "DISABLED"
}
},
overrides: {
containerOverrides: [
{
name: "FileProcessingContainer",
environment: [
...
]
},
]
},
};
try {
await ecs.runTask(params).promise();
}catch (e) {
console.error(e, e.stack)
}
I'm using AWS-CDK to create the ECS infrastructure:
const cluster = new ecs.Cluster(this, 'FileProcessingCluster', {
clusterName: "FileProcessingCluster"
});
const taskDefinition = new ecs.FargateTaskDefinition(this, "FileProcessingTask", {
memoryLimitMiB: 8192,
cpu: 4096,
});
taskDefinition.addContainer("FileProcessingContainer", {
image: ecs.ContainerImage.fromAsset("../local-image"),
logging: new ecs.AwsLogDriver({
streamPrefix: `${id}`
}),
memoryLimitMiB: 8192,
cpu: 4096,
});
Is there some something I'm missing here? Perhaps a setting related to concurrent tasks?

Answer 1

It turns out that I misconfigured the subnets in the task definition, this was preventing the image pull from ECR.
You can read more about it here:
ECS task not starting - STOPPED (CannotPullContainerError: “Error response from daemon request canceled while waiting for connection”
And:
https://aws.amazon.com/premiumsupport/knowledge-center/ecs-pull-container-error/