Unable to find Minimum/Maximum task value for ECS - amazon-web-services

I am working with aws boto3 lib and trying to retrieve certain values.
I first retrieve all cluster list, then fetch specific services, then call describe-service for them.
But I am unable to retrieve two fields Minimum tasks and Maximum tasks for services which get displayed on AWS ECS console page under Auto Scaling tab.
Anybody has any idea how to get these values from?

The ECS console hides this fact, but those are actually in the Application AutoScaling configuration, not the ECS configuration. I believe you would need to call describe_scalable_targets in ApplicationAutoScaling to get those values.

Thanks Mark B for help.
You are right and I understand that aws ecs service has to register with autoscaling service which is a separate service. I am providing sample cli and python code to retrieve these values for other now.
aws ecs describe-services --cluster MAGIC-Bonus-Wrappers --services service-name
aws application-autoscaling describe-scalable-targets --service-namespace ecs --resource-ids service/cluster-name/service-name
Python Code:
client = session.client('application-autoscaling')
response = client.describe_scalable_targets(
ServiceNamespace='ecs',
ResourceIds=[serviceId])
def_val = -1, -1
if "ScalableTargets" in response and len(response['ScalableTargets']) > 0 :
target = response['ScalableTargets'][0]
if 'MinCapacity' in target and 'MaxCapacity' in target:
return target['MinCapacity'], target['MaxCapacity']
else:
return def_val

Related

aws ecs with capacityProviderStrategy | can deploymentController be updated once ecs service is created?

i created the ecs service with deployment controller type as ECS with capacityProviderStrategy and then wanted to modify it to CODE_DEPLOY. Is this restricted for some reason ?
i do not see an option in the UI with modify service
Tried the command \n aws ecs update-service --cluster=my-cluster --service=my-service --region=us-east-2 --deploymentController=CODE_DEPLOY
i get the error below
To see help text, you can run:
aws help
aws help
aws help
Unknown options: --deployment_controller={type=CODE_DEPLOY}
I tried my luck with a variation of --deployment_controller=CODE_DEPLOY however options itself is unknown to cli
Update:
During create service I noticed that CODE_DEPLOY controller is disabled when i choose capacityProviderStrategy. It looks like a limitation by design. Is it true ?
ref: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_definition_parameters.html
Also from aws console, it looks like deployment controller is not editable (service recreate needed) https://i.stack.imgur.com/mRMv3.png

AWS Elastic Search service too_many_buckets_exception

Cloud: AWS
ES Version: 7.4
Error message while searching:
{"error":{"message":"[too_many_buckets_exception] Trying to create too many buckets. Must be less than or equal to: [110000] but was [110001]. This limit can be set by changing the [search.max_buckets] cluster level setting., with { max_buckets=110000 }"}}
The question is where/how can I set this property from AWS console OR AWS cli.
The AWS console
Regards
Amit Meena

How to run an AWS ECS task inside a service of AWS ECS cluster and not ouside the service from Circle CI's "aws-ecs/run-task"

I am using Circle CI to build and push the image to AWS ECR, then use this image to deploy a container(with FARGATE as instance) in a service inside a cluster in AWS ECS. The problem is, the tasks are being run outside of this service, but in the same cluster.
Here's task sitting along the task that was started automatically by AWS:
The one that has group called service:adp-ecs-service is the one running inside service and the one with group adp-ecs-service is the one that is running outside of service. The one that has group called service:adp-ecs-service will be restarted automatically with the image in ECR tagged 'latest_ci', if i stop it, but other one won't start. And this service can only have one service at a time.
By looking at group in this image, I tried to specify the name of the service in group tag in 'config.yml' file in multiple ways, but to no avail. Here's all I have tried(you can see these in my commits here):
'service:adp-ecs-service'
service:adp-ecs-service
Both of them(with and without quotes) genereted following error:
An error occurred (InvalidParameterException) when calling the RunTask operation: Invalid namespace for group.
adp-ecs-service
service
Both of them ran the task outside the service(you can see the output of 3 in the image shown above. I have even tried environment variables for 1 and 2 too.
Here is the code for "aws-ecs/run-task" in "/.circleci/config.yml" (complete file can be found here):
- aws-ecs/run-task:
requires:
- aws-ecr/build-and-push-image
task-definition: 'adp-ecs-family'
group: adp-ecs-service
cluster: 'adp-cluster'
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
aws-region: 'ap-south-1'
awsvpc: true
launch-type: FARGATE
subnet-ids: $AWS_SUBNETS
security-group-ids: $AWS_ADP_SG
started-by: 'circle-ci'
assign-public-ip: ENABLED
And here is the service:
As you can see in the above picture, running task count and the desired task count is 1, but there are 2 tasks running in this cluster, one outside of the service.
What I want here is, if I run a new task inside the service, it should start a container by pulling the latest image from ECR by stopping the previous one. So, How do I properly specify the name of the service and accomplish this?
Simply put aws-ecs/run-task is not what you want. You need to deploy the new task definition to the service, not run it.
You are looking for aws-ecs/deploy-service-update and update-task-definition if you haven't already.

AWS ECS Fargate ALB cannot validate targetGroupArn

NOTE: This is about ELBv2 ALBs, not legacy load balancers, ELBv1, but my humble rep won't let me improve the tagging.
I'm attempting to create an AWS ECS Fargate service. I have created Application Load Balancers one for each container in the Task, and since create-service now supports multiple loadbalancers according to the docs - we should be all good, right? Since it's an ALB, I specify targetGroupArn rather than loadBalancerName, and since ECS has a service linked role created by default, AWSServiceRoleForECS I ought to be able to map target groups and go ahead and create the service, right? Not like it used to be when you had to create ecsServiceRole manually from what the internet tells me.
my command is
aws ecs create-service --region $REGION --cluster $CLUSTER --service-name production-svc --task-definition $TASK_ARN --desired-count 2 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[${SUBNET_1}, ${SUBNET_2}],securityGroups=[${SECURITYGROUP_ID}]}" --load-balancers=tgt-a,containerName=a,containerPort=5000,targetGroupArn=tgt-b,containerName=b,containerPort=6000
And the error I get is
An error occurred (InvalidParameterException) when calling the CreateService operation: Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions.
Now, looking though the internet it could either be that my load balancers don't exist (you'll have to trust me, they do - I supply target group ARNs, and those target groups exist as confirmed with aws elbv2 cli and are mapped to valid active application load balancers that all live in the same region as the cluster), or that the service mapped role (the one AWS created for me earlier) won't have enough rights to assume to verify the targetGroupArn.
Do I really have to add rights to the automatic ECS service mapped role? If so - which rights?
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:Describe*",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
My Automatic role (that has AmazonECSServiceRolePolicy) already has all of those? Which ones are missing?
I was having the same issue, and can confirm what Ali says:
Turns out that if you are creating the service via CLI with aws ecs create-service you need the full arn for the loadbalancer.
The documentation confirms that:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ecs/create-service.html
It's a bit confusing because in the CloudFormation template you can just use the "short" name. Also the error message is a bit misleading.
Hope this helps.

AWS SSM describe-instance-information doesn't find my instances

I am using boto3 to control my EC2 instances on AWS from a python environment, using ec2 and ssm services. I have created an IAM account, that has access to AmazonSSMFullAccess and AmazonEC2FullAccess policies.
ec2 = boto3.client(
'ec2',
region_name='eu-west-1',
aws_access_key_id='…',
aws_secret_access_key='…/…+…'
)
ssm = boto3.client(
'ssm',
region_name='eu-west-1',
aws_access_key_id='…',
aws_secret_access_key='…/…+…'
)
I ran:
ec2.describe_instances()['Reservations']
Witch returned a list of all my instances.
But when I run:
ssm.describe_instance_information()
I get an empty list, though I have at least one instance running on AWS Linux AMI (ami-ca0135b3), and six others on recent Ubuntu AMIs. They are all in eu-west-1 (Ireland).
They should have SSM Agent preinstalled : (https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-install-ssm-agent.html)
I sshed into the AWS Linux instance, and tried to get the logs for ssm using:
sudo tail -f /var/log/amazon/ssm/amazon-ssm-agent.log
But nothing happens there when I run my python code. A sequence of messages gets displayed from time to time :
HealthCheck reporting agent health.
error when calling AWS APIs. error details - NoCredentialProviders: no valid providers in chain. Deprecated.
I also tried running a command through the web interface, selected ' AWS-RunRemoteScript' but no instance is shown below.
My goal is to run:
ssm.send_command(
DocumentName="AWS-RunShellScript",
Parameters={'commands': [command]},
InstanceIds=[instance_id],
)
But it gives me the following error, probably due to the previous problem.
botocore.errorfactory.InvalidInstanceId: An error occurred (InvalidInstanceId) when calling the SendCommand operation
The agent is pre-installed, but the instance (not just your IAM user) still needs the proper role to communicate with the systems manager. Particularly this step of Configuring Access to Systems Manager.
By default, Systems Manager doesn't have permission to perform actions
on your instances. You must grant access by using an IAM instance
profile. An instance profile is a container that passes IAM role
information to an Amazon EC2 instance at launch.
You should review the whole configuration guide and make sure you have configured all required roles appropriately.