I want to save output of an AWS CLI in a variable and use that variable in another AWS CLI, what I did is as follows:
taskarn= aws ecs list-tasks --cluster mycluster --service-name "myService" --region "eu-west-1" --output text | grep "arn" | tr -d '"'
echo $taskarn; //empty
aws ecs stop-task --cluster mycluster --task $taskarn --region "eu-west-1"
when I echo $taskarn, it is empty.
Any help would be appreciated.
I used the following command and it works fine:
taskarn=$(aws ecs list-tasks --cluster mycluster --service-name "myservice" --region "eu-west-1" | grep "arn" | tr -d '"')
echo $taskarn;
aws ecs stop-task --cluster mycluster --task $taskarn --region "eu-west-1"
Use backquote to execute the command and assign the result to the variable.
taskarn=`aws ecs list-tasks --cluster mycluster --service-name "myService" --region "eu-west-1" --output text | grep "arn" | tr -d '"'`
But the correct way is to use the --query option of the CLI to extract what you want.
Related
My GitLab-ci.yml file consists of the below stage-
.deploy_cloudformation_template:
extends: .deployment_job_template
when: on_success
script:
# Deploy the template.
aws cloudformation deploy --profile $AWS_PROFILE --region us-east-1
--stack-name "pa-pa-${ENVIRONMENT_NAME}"
--template-file ./cfn/stack-template.yml
--capabilities CAPABILITY_IAM
--no-fail-on-empty-changeset
--parameter-overrides $(cat ./cfn/params-${ENVIRONMENT_NAME}.ini | tr "\n" " ")
# Delete any failed changesets in the stack, which are created when there's no change in the template.
aws cloudformation list-change-sets --profile $AWS_PROFILE --region us-east-1
--stack-name "pa-pa-${ENVIRONMENT_NAME}"
--query Summaries[][ChangeSetName,Status]
--output text | grep FAILED | awk '{print $1}' | xargs -I {} aws cloudformation delete-change-set
--profile $AWS_PROFILE --region us-east-1
--change-set-name {} --stack-name "pa-zappa-${ENVIRONMENT_NAME}"
If there are no failed changeset then the xargs should not run anything. I tried -r, --no-run-if-empty but that is not supported.
Anything else that i can use to achieve this?
I am trying to create a Fargate cluster with ecs-cli using a load balancer
I came up so far with a script to deploy it without, so far my script is
building image
pushing it to ECR
echo ""
echo "creating task execution role"
aws iam wait role-exists --role-name $task_execution_role 2>/dev/null || \ aws iam --region $REGION create-role --role-name $task_execution_role \
--assume-role-policy-document file://task-execution-assume-role.json || return 1
echo ""
echo "adding AmazonECSTaskExecutionRole Policy"
aws iam --region $REGION attach-role-policy --role-name $task_execution_role \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy || return 1
echo ""
echo "creating task role"
aws iam wait role-exists --role-name $task_role 2>/dev/null || \
aws iam --region $REGION create-role --role-name $task_role \
--assume-role-policy-document file://task-role.json
echo ""
echo "adding AmazonS3ReadOnlyAccess Policy"
aws iam --region $REGION attach-role-policy --role-name $task_role \
--policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess || return 1
echo ""
echo "configuring cluster"
ecs-cli configure --cluster $CLUSTER --default-launch-type FARGATE --config-name $CLUSTER --region $REGION || return 1
ecs-cli down --force --cluster-config $CLUSTER --ecs-profile $profile_name || return 1
ecs-cli up --force --cluster-config $CLUSTER --ecs-profile $profile_name || return 1
echo ""
echo "adding ingress rules to security groups"
aws ec2 authorize-security-group-ingress --group-id $SGid --protocol tcp \
--port 80 --cidr 0.0.0.0/0 --region $REGION || return
ecs-cli compose --project-name $SERVICE_NAME service up --create-log-groups \
--cluster-config $CLUSTER --ecs-profile $profile_name
ecs-cli compose --project-name $SERVICE_NAME service ps \
--cluster-config $CLUSTER --ecs-profile $profile_name
aws ec2 describe-instances --query 'Reservations[].Instances[].[InstanceId,InstanceType,PublicIpAddress,Tags[?Key==`Name`]| [0].Value]' --output table
this works. service is up and I can access it from the public ip.
I now would like to add a load balancer
so I can expose a DNS with route53
Following a few other questions’ advice (this one in particular)
I came up with this
echo ""
echo "configuring cluster"
ecs-cli compose --project-name $CLUSTER create
ecs-cli configure --cluster $CLUSTER --default-launch-type FARGATE --config-name $CLUSTER --region $REGION
echo ""
echo "creating a new AWS CloudFormation stack called amazon-ecs-cli-setup-"$CLUSTER
ecs-cli up --force --cluster-config $CLUSTER --ecs-profile $profile_name
echo "create elb & add a dns CNAME for the elb dns"
aws elb create-load-balancer --load-balancer-name $SERVICE_NAME --listeners Protocol="TCP,LoadBalancerPort=8080,InstanceProtocol=TCP,InstancePort=80" --subnets $subnet1 $subnet2 --security-groups $SGid --scheme internal
echo "create service with above created task definition & elb"
aws ecs create-service \
--cluster $CLUSTER \
--service-name ecs-simple-service-elb \
--cli-input-json file://ecs-simple-service-elb.json
ecs-cli compose --project-name $SERVICE_NAME service up --create-log-groups \
--cluster-config $CLUSTER --ecs-profile $profile_name
echo ""
echo "here are the containers that are running in the service"
ecs-cli compose --project-name $SERVICE_NAME service ps --cluster-config $CLUSTER --ecs-profile $profile_name
and I get the following error messages:
create elb & add a dns CNAME for the elb dns
An error occurred (InvalidParameterException) when calling the CreateService operation: Unable to assume role and validate the listeners configured on your load balancer. Please verify that the ECS service role being passed has the proper permissions.
INFO[0002] Using ECS task definition TaskDefinition="dashboard:4"
WARN[0003] Failed to create log group dashboard-ecs in us-east-1: The specified log group already exists
INFO[0003] Auto-enabling ECS Managed Tags
ERRO[0003] Error creating service error="InvalidParameterException: subnet cannot be blank." service=dashboard
INFO[0003] Created an ECS service service=dashboard taskDefinition="dashboard:4"
FATA[0003] InvalidParameterException: subnet cannot be blank.
here are the containers that are running in the service
Name State Ports TaskDefinition Health
dashboard/4d0ebb65b20e4010b93cb99fb5b9e21d/web STOPPED ExitCode: 137 80->80/tcp dashboard:4 UNKNOWN
My task execution role and task role have this policy attached
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
and the JSON I pass to create service is (copied from the documentation):
{
"serviceName": "dashboard",
"taskDefinition": "dashboard",
"loadBalancers": [
{
"loadBalancerName": "dashboard",
"containerName": "dashboard",
"containerPort": 80
}
],
"desiredCount": 10,
"role": "ecsTaskExecutionRole"
}
what permissions am I missing and what should I change?
IIRC, your ECS service role should have AmazonEC2ContainerServiceRole role permissions to access your ELB and validate the listeners.
See here - https://aws.amazon.com/premiumsupport/knowledge-center/assume-role-validate-listeners/
and here - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_managed_policies.html#AmazonEC2ContainerServiceRole
I'm using AWS ECS CLI within Jenkins pipeline to automate my CICD. What I'm trying to do is I want to create a service based upon the task definition if the service does not exist yet, if the service already exists I just want to update it instead. Here is the create-service cli command:
aws ecs create-service \
--cluster $stg_cluster \
--task-definition $task_def \
--service-name $ecs_service \
--desired-count 1 \
--launch-type EC2 \
--scheduling-strategy REPLICA \
--load-balancers \"targetGroupArn=$target_group_arn,containerName=$container_name,containerPort=80\" \
--deployment-configuration \"maximumPercent=200,minimumHealthyPercent=100\"
It works fine for the first time but will fail at subsequent deployments because of this error:
An error occurred (InvalidParameterException) when calling the CreateService operation: Creation of service was not idempotent.
I believe I have to use the command update-service instead but not sure how to write ECS CLI command to check if an ECS service has already existed. One way I can think of is I can check the returned code from the create-service cli command see if it equals to 0 but again not sure how to retrieve it from the pipeline. Thanks for your help.
First you can run aws ecs describe-services to check if that service is already there and the state is ACTIVE. If it's active then you can run aws ecs update-service to update existing service. If it's not ACTIVE then you can run aws ecs create-service to create the service.
This is a sample code you can use to check if a service is already created and active:
aws ecs describe-services --cluster CLUSTER_NAME --services SERVIVE_NAME | jq --raw-output 'select(.services[].status != null ) | .services[].status'
and then use an if condition to run update-service or create-service.
This is my code in case anyone is interested.
#Jenkinsfile pipeline
stage('Deploy')
{
steps {
....
script {
int count = sh(script: """
aws ecs describe-services --cluster $ecs_cluster --services $ecs_service | jq '.services | length'
""",
returnStdout: true).trim()
echo "service count: $count"
if (count > 0) {
//ECS service exists: update
echo "Updating ECS service $ecs_service..."
sh(script: """
aws ecs update-service \
--cluster $ecs_cluster \
--service $ecs_service \
--task-definition $task_def \
""")
}
else {
//ECS service does not exist: create new
echo "Creating new ECS service $ecs_service..."
sh(script: """
aws ecs create-service \
--cluster $ecs_cluster \
--task-definition $task_def \
--service-name $ecs_service \
--desired-count 1 \
--launch-type EC2 \
--scheduling-strategy REPLICA \
--load-balancers \"targetGroupArn=${target_group_arn},containerName=$app_name,containerPort=80\" \
--deployment-configuration \"maximumPercent=200,minimumHealthyPercent=100\"
""")
}
}
}
}
Use query parameter in aws cli :
#!/bin/bash
CLUSTER="test-cluster-name"
SERVICE="test-service-name"
echo "check ECS service exists"
status=$(aws ecs describe-services --cluster ${CLUSTER} --services ${SERVICE} --query 'failures[0].reason' --output text)
if [[ "${status}" == "MISSING" ]]; then
echo "ecs service ${SERVICE} missing in ${CLUSTER}"
exit 1
fi
same in jenkins pipline:
stage('Deploy')
{
steps {
....
script {
string status = sh(script: """
aws ecs describe-services --cluster $ecs_cluster --services $ecs_service --query 'failures[0].reason' --output text
""",
returnStdout: true).trim()
echo "service status: $status"
if (status == 'MISSING') {
echo "ecs service: $ecs_service does not exists in $ecs_cluster"
// put create service logic
}
}}}
I have few tasks running i want to get the latest(task which started last, with last created-at) task arn.
I am using this command
aws ecs list-tasks --cluster {cluster} --family {family} --desired-status 'STOPPED'
after exploring a little i got to know about sort_by. I tried
aws ecs list-tasks --cluster {cluster} --family {family} --desired-status 'STOPPED' --query "sort_by(taskArns, &CreatedAt)"
But this gives error
In function sort_by(), invalid type for value: <some_task_arn>, expected one of: ['string', 'number'], received: "null"
list-tasks will not give you that information. Have to use both list-tasks and describe-tasks.
I will provide example on my cluster, with RUNNING state tasks. You will have to adjust it to your need.
1. Get the list of tasks
task_arns=$(aws ecs list-tasks --cluster ${cluster[Name]} \
--desired-status 'RUNNING' \
--query 'taskArns' --output text)
echo ${task_arns}
Should give a list of Arns of your tasks, e.g.:
arn:aws:ecs:us-east-1:275795381673:task/0053c603-a6c9-4044-89f5-b0edc8f6de3f arn:aws:ecs:us-east-1:275795381673:task/0b4626ea-0f2b-4c99-9e90-010e8a0c8ad3 arn:aws:ecs:us-east-1:275795381673:task/0d4aa5f2-f547-45ad-b1f8-ed84ef1d678c arn:aws:ecs:us-east-1:275795381673:task/190a320f-4b68-497a-921e-439460447d45 arn:aws:ecs:us-east-1:275795381673:task/c979f4a2-3665-4c56-93c6-e9b88f6b3519
2. Get sorted tasks Arns
aws ecs describe-tasks --cluster ${cluster[Name]} \
--tasks ${task_arns} \
--query "tasks[] | reverse(sort_by(#, &createdAt)) | [].[createdAt,taskArn]" \
--output table
Should give, eg:
----------------------------------------------------------------------------------------------------
| DescribeTasks |
+----------------+---------------------------------------------------------------------------------+
| 1589888493.15 | arn:aws:ecs:us-east-1:275795381673:task/c979f4a2-3665-4c56-93c6-e9b88f6b3519 |
| 1589888501.348| arn:aws:ecs:us-east-1:275795381673:task/190a320f-4b68-497a-921e-439460447d45 |
| 1589888499.438| arn:aws:ecs:us-east-1:275795381673:task/0d4aa5f2-f547-45ad-b1f8-ed84ef1d678c |
| 1589888500.312| arn:aws:ecs:us-east-1:275795381673:task/0b4626ea-0f2b-4c99-9e90-010e8a0c8ad3 |
| 1589888497.701| arn:aws:ecs:us-east-1:275795381673:task/0053c603-a6c9-4044-89f5-b0edc8f6de3f |
+----------------+---------------------------------------------------------------------------------+
3. Get last element sorted tasks Arns
aws ecs describe-tasks --cluster ${cluster[Name]} \
--tasks ${task_arns} \
--query "tasks[] | reverse(sort_by(#, &createdAt)) | [-1].[taskArn]" \
--output text
Should give:
arn:aws:ecs:us-east-1:275795381673:task/0053c603-a6c9-4044-89f5-b0edc8f6de3f
Assuming you work from a Linux/Unix-Shell I would suggest sending the output to the sort command. This works by adding
| sort
to your command at the end of the line:
aws ecs list-tasks --cluster {cluster} --family {family} --desired-status 'STOPPED' | sort
I hope this helps.
Can anyone please provide CLI command to get the stack name and region that created a particular IAM role?
you will have to write a small looping script that goes through all the region and all the stack (with let's say 'CREATE_COMPLETE' status) and use the describe-stack-resources CLI command.
https://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stack-resources.html
Here is a small example:
#!/bin/bash
for region in us-east-2 us-east-1 us-west-1 us-west-2 ap-east-1 ap-south-1 ap-northeast-3 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1 ca-central-1 cn-north-1 cn-northwest-1 eu-central-1 eu-west-1 eu-west-2 eu-west-3 eu-north-1 me-south-1 sa-east-1
do
echo "Processing region $region ..."
for stack in $(aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE --output json --region $region | jq '.StackSummaries[] | .StackId' | sed -e 's/^"//' -e 's/"$//')
do
echo "Processing stack $stack ..."
aws cloudformation describe-stack-resources --stack-name $stack --output json --region $region | jq '.StackResources[] | select(.ResourceType=="AWS::IAM::Role") | select(.PhysicalResourceId=="PUT_YOUR_ROLE_NAME_HERE")'
done
done
Don't forget that if you have your role ARN you can easily get your account number and you role name. The format being
arn:aws:iam::account-id:role/role-name
I hope that helps, sorry about the oneliner, it's less readable.
aws cloudformation describe-stacks --stack-name myteststack