deploy ecs-cli fargate cluster with load balancer - amazon-web-services

I am trying to create a Fargate cluster with ecs-cli using a load balancer
I came up so far with a script to deploy it without, so far my script is
building image
pushing it to ECR
echo ""
echo "creating task execution role"
aws iam wait role-exists --role-name $task_execution_role 2>/dev/null || \ aws iam --region $REGION create-role --role-name $task_execution_role \
--assume-role-policy-document file://task-execution-assume-role.json || return 1
echo ""
echo "adding AmazonECSTaskExecutionRole Policy"
aws iam --region $REGION attach-role-policy --role-name $task_execution_role \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy || return 1
echo ""
echo "creating task role"
aws iam wait role-exists --role-name $task_role 2>/dev/null || \
aws iam --region $REGION create-role --role-name $task_role \
--assume-role-policy-document file://task-role.json
echo ""
echo "adding AmazonS3ReadOnlyAccess Policy"
aws iam --region $REGION attach-role-policy --role-name $task_role \
--policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess || return 1
echo ""
echo "configuring cluster"
ecs-cli configure --cluster $CLUSTER --default-launch-type FARGATE --config-name $CLUSTER --region $REGION || return 1
ecs-cli down --force --cluster-config $CLUSTER --ecs-profile $profile_name || return 1
ecs-cli up --force --cluster-config $CLUSTER --ecs-profile $profile_name || return 1
echo ""
echo "adding ingress rules to security groups"
aws ec2 authorize-security-group-ingress --group-id $SGid --protocol tcp \
--port 80 --cidr 0.0.0.0/0 --region $REGION || return
ecs-cli compose --project-name $SERVICE_NAME service up --create-log-groups \
--cluster-config $CLUSTER --ecs-profile $profile_name
ecs-cli compose --project-name $SERVICE_NAME service ps \
--cluster-config $CLUSTER --ecs-profile $profile_name
aws ec2 describe-instances --query 'Reservations[].Instances[].[InstanceId,InstanceType,PublicIpAddress,Tags[?Key==`Name`]| [0].Value]' --output table
this works. service is up and I can access it from the public ip.
I now would like to add a load balancer
so I can expose a DNS with route53
Following a few other questions’ advice (this one in particular)
I came up with this
echo ""
echo "configuring cluster"
ecs-cli compose --project-name $CLUSTER create
ecs-cli configure --cluster $CLUSTER --default-launch-type FARGATE --config-name $CLUSTER --region $REGION
echo ""
echo "creating a new AWS CloudFormation stack called amazon-ecs-cli-setup-"$CLUSTER
ecs-cli up --force --cluster-config $CLUSTER --ecs-profile $profile_name
echo "create elb & add a dns CNAME for the elb dns"
aws elb create-load-balancer --load-balancer-name $SERVICE_NAME --listeners Protocol="TCP,LoadBalancerPort=8080,InstanceProtocol=TCP,InstancePort=80" --subnets $subnet1 $subnet2 --security-groups $SGid --scheme internal
echo "create service with above created task definition & elb"
aws ecs create-service \
--cluster $CLUSTER \
--service-name ecs-simple-service-elb \
--cli-input-json file://ecs-simple-service-elb.json
ecs-cli compose --project-name $SERVICE_NAME service up --create-log-groups \
--cluster-config $CLUSTER --ecs-profile $profile_name
echo ""
echo "here are the containers that are running in the service"
ecs-cli compose --project-name $SERVICE_NAME service ps --cluster-config $CLUSTER --ecs-profile $profile_name
and I get the following error messages:
create elb & add a dns CNAME for the elb dns
An error occurred (InvalidParameterException) when calling the CreateService operation: Unable to assume role and validate the listeners configured on your load balancer. Please verify that the ECS service role being passed has the proper permissions.
INFO[0002] Using ECS task definition TaskDefinition="dashboard:4"
WARN[0003] Failed to create log group dashboard-ecs in us-east-1: The specified log group already exists
INFO[0003] Auto-enabling ECS Managed Tags
ERRO[0003] Error creating service error="InvalidParameterException: subnet cannot be blank." service=dashboard
INFO[0003] Created an ECS service service=dashboard taskDefinition="dashboard:4"
FATA[0003] InvalidParameterException: subnet cannot be blank.
here are the containers that are running in the service
Name State Ports TaskDefinition Health
dashboard/4d0ebb65b20e4010b93cb99fb5b9e21d/web STOPPED ExitCode: 137 80->80/tcp dashboard:4 UNKNOWN
My task execution role and task role have this policy attached
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
and the JSON I pass to create service is (copied from the documentation):
{
"serviceName": "dashboard",
"taskDefinition": "dashboard",
"loadBalancers": [
{
"loadBalancerName": "dashboard",
"containerName": "dashboard",
"containerPort": 80
}
],
"desiredCount": 10,
"role": "ecsTaskExecutionRole"
}
what permissions am I missing and what should I change?

IIRC, your ECS service role should have AmazonEC2ContainerServiceRole role permissions to access your ELB and validate the listeners.
See here - https://aws.amazon.com/premiumsupport/knowledge-center/assume-role-validate-listeners/
and here - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_managed_policies.html#AmazonEC2ContainerServiceRole

Related

ECS Cluster with Launch Type EC2 does not create ARM64 Instances

I am using ECS-CLI to provision a cluster.
This is my cluster config:
ecs-cli configure --cluster production --region ap-southeast-1 --default-launch-type EC2 --config-name production-config
This is my command:
ecs-cli up --capability-iam \
--keypair keyname \
--region ap-southeast-1 \
--launch-type EC2 \
--size 2 \
--instance-type t2.small \
--ecs-profile production \
--cluster-config production-config
--force --verbose
I can check my cluster status. Everything works fine.
aws ecs describe-clusters --clusters production --region ap-southeast-1
{
"clusters": [
{
"clusterArn": "arn:aws:ecs:ap-southeast-1:665186350589:cluster/production",
"clusterName": "production",
"status": "ACTIVE",
"registeredContainerInstancesCount": 2,
"runningTasksCount": 1,
"pendingTasksCount": 0,
"activeServicesCount": 1,
"statistics": [],
"tags": [],
"settings": [],
"capacityProviders": [],
"defaultCapacityProviderStrategy": []
}
],
"failures": []
}
However, if I were to switch to an ARM64 based instance type such as t4g.small or m6g.medium, I get:
"registeredContainerInstancesCount": 0,
I know ap-southeast-1 supports ARM64 architecture. I have scanned the documentation but cannot find a reason this is happening.
Thanks in advance.

aws_ecs_cluster local-exec aws not found

resource "aws_ecs_cluster" "demo" {
name = var.ecs_cluster_name
capacity_providers = [local.cluster_name]
default_capacity_provider_strategy {
capacity_provider = local.cluster_name
}
# We need to terminate all instances before the cluster can be destroyed.
# (Terraform would handle this automatically if the autoscaling group depended
# on the cluster, but we need to have the dependency in the reverse
# direction due to the capacity_providers field above).
provisioner "local-exec" {
when = destroy
interpreter = ["bash", "-c"]
command = <<EOT
set -e
CAP_PROVS="$(aws ecs describe-clusters --clusters "${self.arn}" \
--query 'clusters[*].capacityProviders[*]' --output text)"
ASG_ARNS="$(aws ecs describe-capacity-providers \
--capacity-providers "$CAP_PROVS" \
--query 'capacityProviders[*].autoScalingGroupProvider.autoScalingGroupArn' \
--output text)"
if [ -n "$ASG_ARNS" ] && [ "$ASG_ARNS" != "None" ]
then
for ASG_ARN in $ASG_ARNS
do
ASG_NAME=$(echo $ASG_ARN | cut -d/ -f2-)
aws autoscaling update-auto-scaling-group \
--auto-scaling-group-name "$ASG_NAME" \
--min-size 0 --max-size 0 --desired-capacity 0
INSTANCES="$(aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names "$ASG_NAME" \
--query 'AutoScalingGroups[*].Instances[*].InstanceId' \
--output text)"
aws autoscaling set-instance-protection --instance-ids $INSTANCES \
--auto-scaling-group-name "$ASG_NAME" \
--no-protected-from-scale-in
done
fi
EOT
}
}
aws_ecs_service local-exec not working and the error say aws command not found
https://github.com/hashicorp/terraform-provider-aws/issues/11409
Error: Error deleting ECS cluster: ClusterContainsContainerInstancesException: The Cluster cannot be deleted while Container InstancesainsContainerInstancesException: The Cluster cantainer Instances are actnot be deleted while Container Instances are active or draining
Error waiting for internet gateway (igw 'detached' (last state:-0e21e59722a46f970) to detach: timeout while waiting for state to become 'detached' (last state: 'detaching', timeout: 15m0s)
aws_ecs_cluster.demo (local-exec): /bin/bash: aws: command not found

Jenkins pipeline - check if an ECS service exist

I'm using AWS ECS CLI within Jenkins pipeline to automate my CICD. What I'm trying to do is I want to create a service based upon the task definition if the service does not exist yet, if the service already exists I just want to update it instead. Here is the create-service cli command:
aws ecs create-service \
--cluster $stg_cluster \
--task-definition $task_def \
--service-name $ecs_service \
--desired-count 1 \
--launch-type EC2 \
--scheduling-strategy REPLICA \
--load-balancers \"targetGroupArn=$target_group_arn,containerName=$container_name,containerPort=80\" \
--deployment-configuration \"maximumPercent=200,minimumHealthyPercent=100\"
It works fine for the first time but will fail at subsequent deployments because of this error:
An error occurred (InvalidParameterException) when calling the CreateService operation: Creation of service was not idempotent.
I believe I have to use the command update-service instead but not sure how to write ECS CLI command to check if an ECS service has already existed. One way I can think of is I can check the returned code from the create-service cli command see if it equals to 0 but again not sure how to retrieve it from the pipeline. Thanks for your help.
First you can run aws ecs describe-services to check if that service is already there and the state is ACTIVE. If it's active then you can run aws ecs update-service to update existing service. If it's not ACTIVE then you can run aws ecs create-service to create the service.
This is a sample code you can use to check if a service is already created and active:
aws ecs describe-services --cluster CLUSTER_NAME --services SERVIVE_NAME | jq --raw-output 'select(.services[].status != null ) | .services[].status'
and then use an if condition to run update-service or create-service.
This is my code in case anyone is interested.
#Jenkinsfile pipeline
stage('Deploy')
{
steps {
....
script {
int count = sh(script: """
aws ecs describe-services --cluster $ecs_cluster --services $ecs_service | jq '.services | length'
""",
returnStdout: true).trim()
echo "service count: $count"
if (count > 0) {
//ECS service exists: update
echo "Updating ECS service $ecs_service..."
sh(script: """
aws ecs update-service \
--cluster $ecs_cluster \
--service $ecs_service \
--task-definition $task_def \
""")
}
else {
//ECS service does not exist: create new
echo "Creating new ECS service $ecs_service..."
sh(script: """
aws ecs create-service \
--cluster $ecs_cluster \
--task-definition $task_def \
--service-name $ecs_service \
--desired-count 1 \
--launch-type EC2 \
--scheduling-strategy REPLICA \
--load-balancers \"targetGroupArn=${target_group_arn},containerName=$app_name,containerPort=80\" \
--deployment-configuration \"maximumPercent=200,minimumHealthyPercent=100\"
""")
}
}
}
}
Use query parameter in aws cli :
#!/bin/bash
CLUSTER="test-cluster-name"
SERVICE="test-service-name"
echo "check ECS service exists"
status=$(aws ecs describe-services --cluster ${CLUSTER} --services ${SERVICE} --query 'failures[0].reason' --output text)
if [[ "${status}" == "MISSING" ]]; then
echo "ecs service ${SERVICE} missing in ${CLUSTER}"
exit 1
fi
same in jenkins pipline:
stage('Deploy')
{
steps {
....
script {
string status = sh(script: """
aws ecs describe-services --cluster $ecs_cluster --services $ecs_service --query 'failures[0].reason' --output text
""",
returnStdout: true).trim()
echo "service status: $status"
if (status == 'MISSING') {
echo "ecs service: $ecs_service does not exists in $ecs_cluster"
// put create service logic
}
}}}

How to save output of AWS CLI in a variable?

I want to save output of an AWS CLI in a variable and use that variable in another AWS CLI, what I did is as follows:
taskarn= aws ecs list-tasks --cluster mycluster --service-name "myService" --region "eu-west-1" --output text | grep "arn" | tr -d '"'
echo $taskarn; //empty
aws ecs stop-task --cluster mycluster --task $taskarn --region "eu-west-1"
when I echo $taskarn, it is empty.
Any help would be appreciated.
I used the following command and it works fine:
taskarn=$(aws ecs list-tasks --cluster mycluster --service-name "myservice" --region "eu-west-1" | grep "arn" | tr -d '"')
echo $taskarn;
aws ecs stop-task --cluster mycluster --task $taskarn --region "eu-west-1"
Use backquote to execute the command and assign the result to the variable.
taskarn=`aws ecs list-tasks --cluster mycluster --service-name "myService" --region "eu-west-1" --output text | grep "arn" | tr -d '"'`
But the correct way is to use the --query option of the CLI to extract what you want.

How to list only the instance id and related tags

I using the CLI tools and I want to list only the instance ID and related tags of an instance. The command that I am using is ec2-describe instances. I have tried certain filters but nothing is working out.
Can anybody help ??
error :
./Instance_Audit.sh: line 24: $: command not found
./Instance_Audit.sh: line 25: syntax error near unexpected token `do'
./Instance_Audit.sh: line 25: ` do echo $ID ; echo $(aws ec2 describe-instances --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done'
Using the aws cli, I can produce a few different alternatives of the tag output:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[]]'
Result: i-xxxxx [{u'Value': 'key1value', u'Key': 'key1name'}, {u'Value': 'key2value', u'Key': 'key2name'}]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].*]'
Result: i-xxxxx [['key1value', 'key1name'], ['key2value', 'key2name']]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].Key,Tags[].Value]'
Result: i-xxxx ['key1name', 'key2name'] ['key1value' 'key2value']
Is any of those formats what you're looking for?
The AWS CLI command
aws ec2 describe-instances --region us-west-2 --query 'Reservations[].Instances[].[InstanceId,Tags[]]' --output text
produces these results:
i-789c55c3
Name Instance1
aws:cloudformation:logical-id Instance1
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
i-77c108cc
Name Instance2
aws:cloudformation:logical-id Instance2
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
If that is not in the format you are looking for, can you provide an example of they format you are expecting?
If you want to only display the instance id and the tags associated to that instance, you can use something like :
$ for ID in $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[].InstanceId" --output text); do echo $ID ; echo $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done
i-60****2a
[ [ { "Value": "SB", "Key": "Name" } ] ]
i-1a****56
[ [ { "Value": "Course and Schedule on Apache 2.4", "Key": "Name" } ] ]
i-88eff3cb
[ [ { "Value": "secret.com", "Key": "Name" } ] ]
i-e7****a5
[ [ { "Value": "2014.03 + Docker", "Key": "Name" } ] ]
I could not find a way to do that with only one AWS CLI call. Should someone come up with a shorter solution, I would love to hear from you.
If you want to filter to certain tag key/value only, you can edit the aws ec2 describe-instances to add a --filter option.
For example :
aws ec2 describe-instances --region eu-west-1 --filter "Name=tag:Name,Values=SB"
--Seb
simple answer:
aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].[InstanceId,Tags[*]]'
not so simple answer .. the same principle could be applied to both instances and vpc's
# how-to get all the tags per instance-id with aws ec2 for prd env
while read -r i ; do \
aws ec2 describe-tags --filters "Name=resource-id,Values=$i" \
--profile prd; done < <(aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].InstanceId' --profile prd)
# how-to get all the tags per virtual-private-cloud-id
while read -r v; do aws ec2 describe-tags \
--filters "Name=resource-id,Values=$v" ; \
done < <(aws ec2 describe-vpcs \
--query 'Vpcs[*].VpcId'|perl -ne 's/\t/\n/g;print')