We have a custom VPC with public and private subnets, in several Availbility Zone (AZ). Via, AWS CLI, we would like to launch/run ec2 instances in that VPC, private subnets, spread out to different AZ, thus spread out among the private subnet (as a subnet can only be in a single AZ)
From thhis doc, when launching without subnet: the cli will choose the default VPC and a subnet for you. But I cannot see how to specify a VPC rather than using the default one.
Is there a convenient solution that is not manually specifying subnet (and managing ourself the spread out) ?
This is actually a bit similar to this question: Boto3 run_instances: How to specify more than one Subnet?
"Convenient Solution"? No.
Since your goal is to "spread out" instances among the private subnets, you could write a script that:
Retrieves a list of subnets
Retrieves a list of currently-running Amazon EC2 instances
Determines which subnet has the least number of running instances
Launches an instance in that subnet
Alternatively, you could launch the instances in an EC2 Auto Scaling group because Auto Scaling always tries to balance instances across Availability Zones. If you have one subnet per AZ, then Auto Scaling will always keep the instances balanced across AZs.
However, if you use an Auto Scaling group, then each instance would need to be identical (as defined in the EC2 Launch Template) and you would add/remove instances by asking Auto Scaling to change the Desired Capacity rather than directly launching an instance.
See: What is Amazon EC2 Auto Scaling?
Related
The definition of the vpc_zone_identifier parameter is a list of subnet IDs to launch resources in. Subnets automatically determine which availability zones the group will reside.
So suppose I list eu-west-1a and eu-west-1c for that parameter and a desired capacity of 3.
Is my ASG going to deploy my desired capacity randomly across the AZs (e.g. 2 + 1) or it will deploy 3 per AZ?
There will be only 3 instances distributed across the two AZs if the selected AZs have enough capacity. AWS tries to prioritize high-availability, so it will try to place the instances evenly across the AZs (2+1 in your case). Exact details are:
Amazon EC2 Auto Scaling attempts to distribute instances evenly between the Availability Zones that are enabled for your Auto Scaling group. Amazon EC2 Auto Scaling does this by attempting to launch new instances in the Availability Zone with the fewest instances. If the attempt fails, however, Amazon EC2 Auto Scaling attempts to launch the instances in another Availability Zone until it succeeds. For Auto Scaling groups in a VPC, if there are multiple subnets in an Availability Zone, Amazon EC2 Auto Scaling selects a subnet from the Availability Zone at random.
If ec2 instance is spinned up in an subnet,subnet's NACl rules imply to the instances of that subnet,But where in case of rds, "subnet group" is attached to the rds instance. if I have 2 subnet's in subnet's group. In this scenario, which subnet's NACL rules are applied to the rds ?
When you launch an RDS instance each instance will only be launched in a single subnet, the cluster on the overhand will spread instances across the subnets i.e. read replicas and Multi-AZ.
Each instance if you look at its properties will have availability zone, by using this you can limit down to the availability zone of the host. Assuming you only have a single subnet per AZ in your subnet group you can then identify the subnet.
If you have multiple subnets per AZ you would need to DIG (or ping) the RDS instances hostname to get the IP address. Then you would need to filter to determine which range it is in.
I'm trying to use CDK (Typescript) to create an RDS (Postgres) instance within an ISOLATED subnet. As far as I understood, the default VPC created automatically in each of my account regions only has Public Subnets.
If I then want to launch a Fargate instance on top of one of those Public Subnets, do I need to create a VPC Peering between the ISOLATED subnet and the default VPC?
Any idea how to create that using CDK?
"Any idea how to create that using CDK?" -> yes:
Use AWS CDK to provision a new VPC similar to this. As you can see, there are subnets for public, applicational and database layers;
[OPTIONAL] For a better AWS CDK project organization, you can use multiple stacks like network (VPC resources), compute (Fargate resources), and database (RDS). You can pass the vpc on network stack to compute & database stacks;
Be sure that you set vpc on the ECS cluster and at load-balanced level (if applicable) and put RDS database in the isolated/database by using vpcSubnets attribute
To sum up, one VPC. No applicational or database exposed to the public. I have a git repo in case you want to look for sample code/template
VPC peering is when you have two VPC with different CIDR ranges. You have one VPC hence no need to use VPC peering.
I created two subnets without any preference of the availability zone(AZ). So, now I have two subnets in one AZ. I know that I can extend the VPC and create new subnets but I simply want to move a subnet into a different zone. Is there any way I can move a subnet to a different AZ on AWS?
This is not possible.
An Availability Zone is nominated when the Subnet is created. It is not possible to change the AZ on a Subnet.
You should either create another subnet, or delete the subnet and recreate it in a different AZ.
I have a pretty standard stack, RDS, 2 EC2 instances using ELB. Because I wanted the ELB to be restricted to a particular IP range I've launched the stack in VPC, for DR reasons across 2 subnets.
I use several ephemeral ec2 machines, which when not in VPC I allowed to startup in any availability zone. Which (as far as I understand) would mean that if an AZ were unavailable then machines would be started in other zones.
Is there a way to emulate this in VPC? Is there a way of saying launch a machine in any subnet in a VPC?
If not its fairly easy to workaround by picking a subnet at random, and if it fails trying another. Just wondered if there was a supported method that's cleaner?
I'm using python and boto.
thanks
Which (as far as I understand) would mean that if an AZ were unavailable then machines would be started in other zones.
That's correct and it would indeed be nice to have this option available within an Amazon VPC as well when running instances directly via the available Amazon EC2 API actions.
Unfortunately both the RunInstances and the RequestSpotInstances API actions only allow to specify the optional parameters SubnetId or LaunchSpecification.SubnetId respectively (The ID of the subnet in which to launch the [Spot] Instance), thus won't have any information into which VPC you would want to launch the instance if no subnet is specified.
Workaround
You can achieve the desired behavior indirectly via Auto Scaling by means of its CreateAutoScalingGroup API action, see parameter VPCZoneIdentifier:
A comma-separated list of subnet identifiers of Amazon Virtual Private Clouds (Amazon VPCs).
This feature is also available via the AutoScalingGroup resource type within AWS CloudFormation.