Does anyone have any guidance on what the most efficient way to launch an EMR cluster using EC2 spot instances? I'm using a 10-node r5d.8xlarge (1 master 9 core) and usually I will clone a previous instance. What's been happening more often than not is that I'll get an "out of capacity" error. When that happens I change the subnet and try again. It's really a guessing game at this point and I'm sure there's a much better way to do this, unfortunately I haven't found one.
There's no need for it to be a guessing game. You can use Instance Fleets - Amazon EMR to specify multiple:
Instance Types
For each fleet, you specify up to five EC2 instance types. Amazon EMR
chooses any combination of these EC2 instance types to fulfill your
target capacities.
Availability Zones
When you use instance fleets, you can specify multiple EC2 subnets
within a VPC, each corresponding to a different Availability Zone.
Instances are always provisioned in only one Availability Zone.
See also: New – Amazon EMR Instance Fleets | AWS News Blog
Related
When I manually create an AWS ECS cluster in the UI, by default it shows the default VPC with the default subnets selected, one in each availability region, which I infer to be best practice.
I'm creating a new ECS Fargate deployment using CloudFormation, and I believe the best practice is to create a new VPC with its own subnets, similarly each in a different availability zone.
Rather than hard-coding them in CloudFormation, how can I enumerate the availability zones for the current region and create subnets for each of them?
I'm new to CloudFormation. I've seen that there is a built-in function to enumerate availability zones, so with some thought I'm sure I can figure this out on my own. And I'll be sure to add that as an answer here. But being what surely is a common use case, perhaps there is an existing best-practices template somewhere that already shows how to do that so I don't have to reinvent it. (I haven't been able to find it immediately by searching.)
My AWS solution spans over 3 availability zones. In my backend the user is able to trigger a heavy compute job with beefy px instances. Therefore I wrote a CFN template, which provision all resorucess to execute the compute job (secret store, IAM Role, EC2 instance, log group). However when I try to create the template, it returns with a 500 and states that no capacity for my instance type is available for the availability zone i choose. My template provides a subnet for the EC2 instance and an availability zone for the attached volume. In the end I don't care in which availability zone the ec2 is provisioned as long it is in one of my subnets. Does someone know a way to provision an EC2 instance and it's volume (with cloudforamtion) by not specifically choosing one availability zone, but rather provide a range of subnets/availability zones ?
TLDR:
Does someone know a way to provision an EC2 instance and it's volume (with cloudforamtion) by not specifically choosing one availability zone, but rather provide a range of subnets/availability zones ?
We have an AWS account with multiple EC2 instances and an RDS instance. These have been running since before VPCs were introduced. Now I need to upgrade the RDS instance class and the new class can only be established in a VPC. So if our RDS instance is modified (converted) into the new class and the new RDS instance is in a VPC will the EC2 instances that use this RDS need to be moved into the VPC at the same time or can this be a separate process?
RDS and EC2 security groups are not cross-compatible between classic and vpc EC2. If you have an RDS with an EC2 sec group reference you will not be able to add a classic ec2 sec group to your vpc RDS sec group.
Differences Between Security Groups for EC2-Classic and EC2-VPC
Your best bet is to migrate your EC2 infrastructure to VPC along with your RDS. You can of course grant global access to your RDS if you want to separate the process and then lock it back down when you've migrated your EC2 infrastructure but this is a big no no (huge security risk).
This is a non trivial process and I feel your pain. The least complex thing is to incur downtime during migration so that you don't have divergence between RDS instances. But that's not ideal especially when you have a large RDS storage-wise. BTW, you will want to test how long the process takes to snapshot and restore the RDS since it might take a while.
An alternative if you can manage it is to force your app to use read-only replicas while you migrate the RDS. That way your app can still function (kind of) and you won't have to worry about reconciliation between RDS instances post-migration. But obviously your app needs to be aware that it's in read-only mode or you'll end up with erratic behavior.
Forgot to mention there is something called classic-link which might help here but I've personally never used classic-link. Check it out in an AWS whitepaper write up here:
Move Amazon RDS MySQL
Databases to Amazon VPC
using Amazon EC2
ClassicLink and Read
Replicas
Thanks in advance....
we have RDS replication with multi-AZ option. Is there any way to replicate our AWS EC2 instances without using Auto-scaling option?
On the EC2 console listing running instances, select the instance you want to replicate and under Actions click Launch More Like This.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/launch-more-like-this.html
Yes, configure your EC2 instance exactly the way you want it to be. Stop the instance and create an AMI of this instance. Then launch as many EC2 instances as you want using the AMI. Each EC2 instance will be almost identical. There will be differences due to licensing, IP addressing, instance ID, etc.
This is the same technique that you would use with Auto Scaling Groups (e.g. new instances are launched from a specific AMI).
Another option is to use Launch More Like This from the EC2 Console. However, this feature does not clone your instance, only replicates some configuration details.
my aws auto-scaled instances are not picked up by load-balancer and the auto-scaled instances are recreated frequently,
also is there any problem in using auto-scaled instances and static instances at the same time in aws ELB ?
what are the precautions to take when doing so if it is possible
is there any disadvantages doing so ?
Need to make sure that your autoscaling group is registered with the load balancer appropriately, and that you have the appropriate policies. Really need more details to answer this though.
Don't do it. If you need an instance to be running all the time, configure your group to have a minimum of the number of "static" instances. If you need to run a "static" instance, and a scaling group - you're probably thinking about the problem the wrong way.
One reason could be: If you have configured your autoscaling group for multiple availability zones, but those zones are not added to the associated load balancer. In Management Console, go to Load Balancers -> Instances and verify Availability Zones.
I would go with #Peter H. Modify your design so you don't depend on any particular instance for persistent data. Store persistent data externally in a database or on S3.