Cloud: AWS
ES Version: 7.4
Error message while searching:
{"error":{"message":"[too_many_buckets_exception] Trying to create too many buckets. Must be less than or equal to: [110000] but was [110001]. This limit can be set by changing the [search.max_buckets] cluster level setting., with { max_buckets=110000 }"}}
The question is where/how can I set this property from AWS console OR AWS cli.
The AWS console
Regards
Amit Meena
Related
I am working with aws boto3 lib and trying to retrieve certain values.
I first retrieve all cluster list, then fetch specific services, then call describe-service for them.
But I am unable to retrieve two fields Minimum tasks and Maximum tasks for services which get displayed on AWS ECS console page under Auto Scaling tab.
Anybody has any idea how to get these values from?
The ECS console hides this fact, but those are actually in the Application AutoScaling configuration, not the ECS configuration. I believe you would need to call describe_scalable_targets in ApplicationAutoScaling to get those values.
Thanks Mark B for help.
You are right and I understand that aws ecs service has to register with autoscaling service which is a separate service. I am providing sample cli and python code to retrieve these values for other now.
aws ecs describe-services --cluster MAGIC-Bonus-Wrappers --services service-name
aws application-autoscaling describe-scalable-targets --service-namespace ecs --resource-ids service/cluster-name/service-name
Python Code:
client = session.client('application-autoscaling')
response = client.describe_scalable_targets(
ServiceNamespace='ecs',
ResourceIds=[serviceId])
def_val = -1, -1
if "ScalableTargets" in response and len(response['ScalableTargets']) > 0 :
target = response['ScalableTargets'][0]
if 'MinCapacity' in target and 'MaxCapacity' in target:
return target['MinCapacity'], target['MaxCapacity']
else:
return def_val
I have a working AWS Amplify application which I deployed manually. To specify this application using the CLI, I need the so-called appId. But where can I find this? In the console, I can find the name and the appArn but not the appId.
If you have access to the AWS console and can see the App ARN field, you can find the AppId:
arn:aws:amplify:eu-west-1:xxxxxxxxxx/APPID
It has this pattern: d[a-z0-9]+
I have a quick question about usage of AWS SNS.
I have deployed an EC2 (t2.micro, Linux) instance in us-west-1 (N.California). I have written a python script using boto3 to send a simple text message to my phone. Later I discovered, there is no SNS service for instances deployed out of us-east-1 (N.Virginia). Till this point it made sense, because I see this below error when i execute my python script, as the region is defined as "us-west-1" in aws configure (AWS cli) and also in my python script.
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameter) when calling the Publish operation: Invalid parameter: PhoneNumber Reason:
But to test, when I changed the "region" in aws conifgure and in my python script to "us-east-1", my script pushed a text message to my phone. Isn't it weird? Can anyone please explain why this is working just by changing region in AWS cli and in my python script, though my instance is still in us-west-1 and I dont see "Publish text message" option on SNS dashboard on N.california region?
Is redefining the aws cli with us-east-1 similar to deploying a new instance altogether in us-east-1? I dont think so. Correct me if I am wrong. Or is it like having an instance in us-west-1, but just using SNS service from us-east-1? Please shed some light.
Here is my python script, if anyone need to look at it (Its a simple snippet).
import boto3
def send_message():
# Create an SNS client
client = boto3.client("sns", aws_access_key_id="XXXX", aws_secret_access_key="XXXX", region_name="us-east-1")
# Send your sms message.
client.publish(PhoneNumber="XXXX",Message="Hello World!")
if __name__ == '__main__':
send_message()
Is redefining the aws cli with us-east-1 similar to deploying a new
instance altogether in us-east-1?
No, it isn't like that at all.
Or is it like having an instance in us-west-1, but just using SNS
service from us-east-1?
Yes, that's all you are doing. You can connect to any AWS regions' API from anywhere on the Internet. It doesn't matter that it is running on an EC2 instance in a specific region, it only matters what region you tell the SDK/CLI to use.
You could run the same code on your local computer. Obviously your local computer is not running on AWS so you would have to tell the code which AWS region to send the API calls to. What you are doing is the same thing.
Code running on an EC2 server is not limited into using the AWS API in the same region that the EC2 server is in.
Did you try creating a topic before publishing to it? You should try create a topic and then publish to that topic.
I believe I might be missing a piece here,
I've added the aws account.
hal config provider aws account add spinnakermaster \
--account-id XXXXXXXXXXXX --asume-role role/spinnakerManaged
I've added the credentials for the AWS User.
hal config provider aws edit --access-key-id XXXXXXXXXXXXXXXXXXXX --secret-access-key
And prompted to its corresponding secret-access-key.
I've edited in the .hal directory the config file:
aws:
enabled: false
accounts:
- name: spinnakermaster
requiredGroupMembership: []
accountId: 'ZZZZZZZZZZZZZZZZZZ'
regions: []
assumeRole: role/spinnakerManaged
primaryAccount: spinnakermaster
accessKeyId: XXXXXXXXXXXXXXXXXXXX
secretAccessKey: YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
defaultKeyPairTemplate: '{{name}}-keypair'
defaultRegions:
- name: Canada
defaults:
iamRole: BaseIAMRole
And I am deploying Spinnaker with AWS support which execute with one hiccup:
Problems in default.provider.aws.spinnakermaster:
- WARNING No validation for the AWS provider has been
implemented.
Which step/info/config am I missing.
Regards
EN
updated. This warning is OK and will not affect your executions.
My suggestions after installing Spinnaker in EC2 local debian, Azure AKS and Minnaker on EC2.
Please dont install a microservice architecture in a monolith environment such as local Debian. It doesnt work
At All Cost Focus on the correct AWS Managed and Managing IAM structure. Please Follow Armory Spinnaker instructions on how to achieve this Armory IAM structure
Previous misleading answer: As of Now Spinnaker version 1.16.4 and based on the official documentation. There are 2 ways to manage the AWS infrastructure:
with aWS key and secret
with IAM role attached to the AWS EC2 instance running the spinnaker.
This error usually comes up when halyard cannot recognize the Key and secret for the corresponding account. Check halyard Code Documentation
One way to resolve it depending on your deployment type is adding an AWS account with the corresponding Key and Secret values. Check Halyard add-account
Documentation AWS Cloud Provider
So I have a springboot application that I 'd like to deploy to AWS using boxfuse.
Deployement onto the AWS Services yields the following error:
ERROR: Your AWS account does not have a default VPC in the eu-central-1 region
=> change the AWS region in the Boxfuse console
Going to the boxfuse Console:
Your AWS account has no default VPC in eu-central-1. Please ask AWS support to recreate it or select a different region
Where exactly can I fix this in AWS? How can I change the region for boxfuse / fix this by maybe creating a new VPC ?
Well, apparently the only way to do this is through the AWS Customer Support. Just wanted to mention in Case someone had the same issue.