my aws auto-scaled instances are not picked up by load-balancer and the auto-scaled instances are recreated frequently,
also is there any problem in using auto-scaled instances and static instances at the same time in aws ELB ?
what are the precautions to take when doing so if it is possible
is there any disadvantages doing so ?
Need to make sure that your autoscaling group is registered with the load balancer appropriately, and that you have the appropriate policies. Really need more details to answer this though.
Don't do it. If you need an instance to be running all the time, configure your group to have a minimum of the number of "static" instances. If you need to run a "static" instance, and a scaling group - you're probably thinking about the problem the wrong way.
One reason could be: If you have configured your autoscaling group for multiple availability zones, but those zones are not added to the associated load balancer. In Management Console, go to Load Balancers -> Instances and verify Availability Zones.
I would go with #Peter H. Modify your design so you don't depend on any particular instance for persistent data. Store persistent data externally in a database or on S3.
Related
Does anyone have any guidance on what the most efficient way to launch an EMR cluster using EC2 spot instances? I'm using a 10-node r5d.8xlarge (1 master 9 core) and usually I will clone a previous instance. What's been happening more often than not is that I'll get an "out of capacity" error. When that happens I change the subnet and try again. It's really a guessing game at this point and I'm sure there's a much better way to do this, unfortunately I haven't found one.
There's no need for it to be a guessing game. You can use Instance Fleets - Amazon EMR to specify multiple:
Instance Types
For each fleet, you specify up to five EC2 instance types. Amazon EMR
chooses any combination of these EC2 instance types to fulfill your
target capacities.
Availability Zones
When you use instance fleets, you can specify multiple EC2 subnets
within a VPC, each corresponding to a different Availability Zone.
Instances are always provisioned in only one Availability Zone.
See also: New – Amazon EMR Instance Fleets | AWS News Blog
Is there a way like assigning a specific tag for the EC2 instances to automatically attached to the load balancer on AWS?
I believe I had done that in the past but unable to find that option now.
Since you say you've done it in the past, I believe you're thinking of a feature offered by EC2 auto-scaling groups (ASGs). ASG is a capability of the EC2 infrastructure that scales machine counts up and down based on workload or maintains a set number of healthy instances always running (destroying and replacing failed instances). When an ASG is attached to a load balancer, the instances controlled by the ASG are automatically registered and deregistered from the balancer.
Amazon EC2 Auto Scaling integrates with Elastic Load Balancing to enable you to attach one or more load balancers to an existing Auto Scaling group. After you attach the load balancer, it automatically registers the instances in the group and distributes incoming traffic across the instances.
https://docs.aws.amazon.com/autoscaling/ec2/userguide/attach-load-balancer-asg.html
Is this possible to reuse ELBs so that I can save money.
I have 12 running instances , and 20 ELBs ( some are classic ELB and others application ELB)
I noticed that in my $900 bill per month, $350 is taken only for ELB. Why so much?
Is there any way to find out unused ELBs? so I can delete them
In this scenario, it's better to bring in Application Load Balancers. You can assign multiple target groups to a single application load balancer. Different instances go in different target groups.
AWS Documentation have all the details required for you to get started:
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/index.html
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html
I'm thinking of using cloud formation as a means of blue-green deployment.
Part of it is putting instances of the Autoscaling group instances on stand by.
Is that possible?
Amazon EC2 Auto Scaling groups are responsible for launching and terminating Amazon EC2 instances. Note that instances are launched as new instances -- they are not kept on standby.
You can certainly do blue-green deployments by using two separate CloudFormation stacks, each with their own Auto Scaling group and, presumably, Elastic Load Balancer.
Both Auto Scaling groups would be 'operating', but not necessarily receiving traffic. You would then need some mechanism to 'switch' between the blue/green groups, such as changing a DNS entry in Route 53 to point to the different Load Balancer.
Adding to John's Answer, On route 53 you can use weighted load balancing to route traffic. It allows you to route percentage traffic to the setups.
I am looking for how to specify the zone I want to deploy to in a single instance deployment, with autoscaling, while also having automatic failover to another zone -- Do any options exist to achieve this?
More context
Due to how reserved instances are linked to a single availability zone (AZ), we find it to be a good strategy (from an "ease of management"/simplicity perspective), when buying reserved instances for our dev environment, to buy them all in a single zone and then launch all dev instances in that single zone. (In production, we buy across zones and run with autoscale groups that specify to deploy across all zones).
I am looking for how to:
Specify the AZ that I want an instance to be deployed to, so that I can leverage the reserved instances that are tied to a single (and consistent) AZ.
while also having
The ability to failover to an alternate zone if the primary zone fails (yes, you will pay more money until you move the reserved instances, but presumably the failover is temporary e.g. 8 hours, and you can fail back once the zone is back online).
The issue is that I can see how you can achieve 1 or 2, but not 1 and 2 at the same time.
To achieve 1, I would specify a single subnet (and therefore AZ) to deploy to, as part of the autoscale group config.
To achieve 2, I would specify more than one subnet in different AZs, while keeping the min/max/capacity setting at 1. If the AZ that the instance non-deterministically got deployed to fails, the autoscale group will spin up an instance in the other AZ.
One cannot do 1 and 2 together to achieve a preference for which zone an autoscale group of min/max/capacity of 1 gets deployed to while also having automatic failover if the zone the server is in fails; they are competing solutions.
This solution uses all AWS mechanisms to achieve the desired effect:
Launch the instance into the preferred zone by specifying that zone's subnet in the 1st autoscale group's config; this group's min/max/capacity is set to 1/1/1.
Create a second autoscale group with the same launch config as the 1st, but this other autoscale group is set to a min/max/desired of 0/1/0; this group should be configured with the subnets in every available zone in the region except the one specified in the 1st autoscale group.
Associate the 2nd autoscale group with the same ELB that is associated with the 1st autoscale group.
Set up a CloudWatch alarm that triggers on the unhealthy host alarm for #1's autoscale group; have the alarm change the #2 autoscale group's to a min/max/desired of 1/1/1. (As well as send out a notification so that you know this happened).
If you don't expect to get unhealthy host alarms except in the cases where there is an actual host failure or if the AZ goes down -- which is true in our case -- this is a workable solution.
As you have already figured out, (as of mid-2015) that's not possible. Auto-scaling doesn't have the concept of failover, strictly speaking. It expects you to provide more than one AZ and machines enough in each one if you want to have high availability. If you don't, then you aren't going to get it.
The only possible workaround I can imagine for this is setting up a watchdog yourself which changes the auto-scaling group's subnet once an AZ becomes unavailable. Not so hard to do, but no so reliable as well.