I understand that I get 5 EIP per region and if I need more I need to request service limit increase.
My question is how many can I get after service limit increase? Is it possible to get 50 EIP?
Please note that the DNS IP that AWS provides is not an option since that cannot be assigned to after instance is created and DNS IP cannot be attached to the second interface of my device (it can only be assigned to eth0 of a given EC2 instance)
All I need is a public IP that I can assign to eth1 or eth2
And I need to do that for about 50 EC2 instances.
Yes you can get more EIP on request. You can then attach/reattach them for your EC2 instances.
Contact AWS Support - Indicating the Service Limit Increase.
EIP Service Limit Increase - Link
General EC2 Limits Info - Link
you can increase your EIP service limit increase by
Service Quota console https://console.aws.amazon.com/servicequotas/home/services/ec2/quotas/L-0263D0A3
By AWS Support Console (Service Limit Increase)
By API calls like boto3 https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/service-quotas.html#ServiceQuotas.Client.request_service_quota_increase or using cli like https://docs.aws.amazon.com/cli/latest/reference/service-quotas/request-service-quota-increase.html
we can automate this process in aws
create cloudtrail for multiregion enable log stream
Using Log Group we will add Log Subscription filter where we have to define filter pattern "AllocateAddress" and add lambda function with it
In our lambda function we will create Python Script in which
we will check all region total number of Elastic Ip present in all region and fetch applied quota value from Service Quota then make math expression ((current EIP value/Applied Quota Value)*100) if the
value is grater then 80% then sent request to increase request value
Related
It seems ECS API hangs when calling ssm.ap-southeast-2.amazonaws.com. Below is the debug results where it hangs
2020-06-11 22:47:10,831 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (2): ssm.ap-southeast-2.amazonaws.com:443
This works fine on EC2 instance. Just inside ECS task container doesn't work and connection gets timed out.
What could be the reason behind this?
Works fine on EC2 instance
Hmm... I think your container is a victim of IMDSv2. Please allow me to explain.
Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories, for example, host name, events, and security groups. You can query instance metadata by calling the following URL:
http://169.254.169.254/latest/meta-data/
On Nov 19, 2019, v2 of the Instance Metadata Service was released. One of the features introduced with EC2 Instance Metadata Service version 2 (IMDSv2) is "Protecting against open layer 3 firewalls and NATs" 1 which sets a TTL (or hop limit 2) of 1 on low level IP packets containing the secret token so the packet can only cross one host. The TTL of 1 means that the instance is not able to forward the packet to a Docker container running on an ECS Container instance as that would be counted as another hop.
From 1:
With IMDSv2, setting the TTL value to “1” means that requests from the EC2 instance itself will work because they’re returned to the caller (on the instance) before the subtraction occurs. But if the EC2 instance has been misconfigured as an open router, layer 3 firewall, VPN, tunnel, or NAT device, the response containing the token will have its TTL reduced to zero before leaving the instance, and the packet containing the response will be discarded on its way out of the instance, preventing transport to the attacker. The information simply won’t make it further than the EC2 instance itself, which means that an attacker won’t get the response back with the token, and with it the ability to access instance metadata, even if they’ve been successful at getting past all other defenses.
A consequence of this change is Docker containers running on ECS instances in Bridge or AWSVPC mode can no longer query the metadata endpoint. The following request will timeout:
$ curl -X PUT -H "x-aws-ec2-metadata-token-ttl-seconds: 120" "http://169.254.169.254/latest/api/token"
If using AWS CLI, it has a fallback mechanism to IMDSv1 but after a long delay (5 seconds) which makes it rather unusable.
From: https://github.com/aws/aws-sdk-js/issues/3024#issuecomment-589135606 :
From v2.575.0, the SDK is configured to default to the IMDSv2 workflow and, by default, will try three times (with a timeout of one second between attempts) to obtain the required token. If all three attempts fail, the SDK will then fall back to the IDMSv1 workflow.
Option 1 (Use with caution)
It is possible to use the 'modify-instance-metadata-options' 3 AWS CLI call on the Container Instance to change the TTL to a higher value by specifying a value for the --http-put-response-hop-limit flag.
The following AWS CLI command modifies the value to '2' when run on the EC2 instance:
$ aws ec2 modify-instance-metadata-options --instance-id $(curl 169.254.169.254/latest/meta-data/instance-id) --http-put-response-hop-limit 2 --http-endpoint enabled
... after which the curl command against token endpoint was successful from the Docker container.
A Lambda function can be invoked from Autoscaling lifecycle hook to configure the value '2' on any launching instance with ModifyInstanceMetadataOptions api call. Another option is to place this command in EC2 Instance's UserData so every instance can 'self-configure' itself with the updated hop limit. Please note that in this case, Instance profile should have associated policy with 'ec2:ModifyInstanceMetadataOptions' permission for this call to be successful.
Option 2 (Recommended)
With regards to ECS, accessing the instance credentials from a container is not considered a best practice, instead the recommendation is to set a task role and use the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable to retrieve container specific credentials from the ECS agent, by for example using the "curl 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" command, up to date versions of the AWS CLI use this by default.
You can read more about the task role credentials here 4. A similar endpoint for task metadata is also available 5.
More discussion can be found here:
https://github.com/aws/aws-sdk-ruby/issues/2177
https://github.com/aws/containers-roadmap/issues/670
https://github.com/aws/aws-sdk-js/issues/3024
While updating EC2 instance of Elastic Beanstalk the following error message appeared. But for my other account the following error, not observed:
Updating Auto Scaling group named: awseb-e-tiknsbmh4d-stack-AWSEBAutoScalingGroup-R3UXFI8KMCSN failed Reason: You have requested more instances (1) than your current instance limit of 0 allows for the specified instance type
Is it account base specific? And what is the best way to fix it?
Yes, there are limits which are account-specific.
You can check your EC2 quotas using the AWS Quota Service. [1]
You can view the EC2 quotas which apply for the account you are currently signed in by visiting the following URL: https://eu-central-1.console.aws.amazon.com/servicequotas/home?region=REGION#!/services/ec2/quotas by replacing REGION with your region identifier, e.g. us-east-1.
If the quota is marked as adjustable, you can request an increase via this service.
References
[1] https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html
This message is related to Amazon EC2. Even though the instance was launched by Elastic Beanstalk, the limit is related to the number of concurrent Amazon E2 instances permitted.
There are two types of limits:
Total number of concurrent instances (default: 20)
Number of concurrent instances per instance type (default varies)
You can view these limits in the Limits section of the Amazon EC2 management console. You can also request an increase in the limits.
The limits are "per account, per region".
I would like to investigate that whether it is possible or not that if someone tries to hit the application url then instance should be re-enabled and remain active as long as there is active use. If the resources are inactive for 10 to 20 mins they should automatically disable themselves i.e. instance should get disabled.
Here there are Multi host Application deployed on ec2 instance and configured record set dns in Route53.
Pls suggest
Create a ec2 start lambda function that gets called when you hit the url hosted on APIGateway backed by this lambda, once the instance is up redirect to the actual ec2 instance url(So the lambda will have to keep checking the status of the ec2 instance once running redirect to this url).
On starting the ec2 instance trigger another lambda on the event pattern on the state as running basis which will attach a cloudwatch alarm to the instance.
The cloudwatch alarm will check on the cpu usage and if it goes below 10% for 3 consecutive times it will stop the instance.
The lambda should have the role with policy having full access to the EC2 instance(later on change it to the required method privileges).
blog on stop/start ec2 instance
aws knowledge centre
aws Instance Scheduler
Create an AWS Alarm to monitor usage or activity.
Use an SNS topic to trigger a Lambda function based on the Alarm
Turn off the ec2 instance based using Python in Lambda
This should help with the code:
https://medium.com/geekculture/terraform-setup-for-automatically-turning-off-ec2-instances-upon-inactivity-d7f414390800
I am getting this error Master Instance Group: Exceeded EC2 Instance Quota, when I create a new cluster on Amazon EMR with 1 Master node only or 1 Master and 2 Core nodes. However, there are no EC2 instances running on my account.
What should I do? I raised a ticket, asking if I can get a quicker solution here.
In your case I think you are trying to access a new region or new instance type, AWS sometimes does that when you are in free tier, they allow access to 2-3 regions or free instance types only. Then you have to request access from AWS by raising a case.
But in Normal scenario, this is what happens:
You may face an error like Exceeded EC2 Instance Quota while you are trying to spin up new instances either standalone or in cluster.
This error is caused because you have hit the limit on number of instances allowed in your AWS account.
This limit is region and instance size specific. To get rid of this error you will have to request Amazon to increase the EC2 instance limit.
Requesting a limit increase is simple. Below are the steps:
Most service limit increases can be requested through the AWS Support Center by choosing Create Case and then choosing Service Limit Increase.
Most service limits are specific to a particular AWS Region, so be sure to submit a request for each Region you plan to use. Many services support requesting multiple limit increases for the same service through one support case. After creating your first request, choose Add another request and then choose a new limit type or Region.
In amazon aws how many m1.small spot request can i create in one region by default
when i see in Instances limit it shows me some thing like
Spot Instance requests default Request limit increase
but currently i am running 36 m1.small
and it gives me an error while creating new spot request
MaxSpotInstanceCountExceeded
so my question is how many spot request can i make of one particular type (eg.m1.small or m1.large)
or how does amazon aws calculate MaxSpotInstanceCountExceeded
MaxSpotInstanceCount is the hard limit of on number of spot instances that you can launch irrespective of their type. the default is 5. However, in your case it seems to be 36.
You can request AWS to increase the limit by filling out this form: https://aws.amazon.com/support/createCase?type=service_limit_increase&serviceLimitIncreaseType=ec2-instances
For any other doubts, please reach out to their support: https://aws.amazon.com/support