I’m having a personal website hosted at AWS EC2 with ELB. Today I have started my AWS EC2 instances (I had turned off due to non usage and Of course, I can save some cost) and tried to load my website via AWS Elastic loadbalancer public dns url but it was not coming up in my browser, instead of webpage I got a blank white page. So I checked my AWS EC2 instances and ELB services.
In the Elastic Load Balancer section, I can see that the status message is showing the registered AWS EC2 instances are “Out of Service”! I tried to change the health check parameter values, nothing happened! So I deregister the EC2 instances from the loadbalancer and register the same again. After few minutes the instances are coming up to “In Service”. It took sometime because the EC2 instances should register into the loadbalancer and health check. Finally I brought my website up.
Solutions tried --
If you have launched your instance in EC2-VPC, by default, the IP address associated with your instance does not change when you stop and then start the instance. However, when you stop and then start your EC2-VPC instance, your load balancer might take sometime to recognize that the stopped instance has started. During this time your load balancer is not connected to the restarted instance. I recommend that you reregister your restarted instance with the load balancer.
My instance is in EC2-VPC and I tried the baove and when I re-register the instance falls back in the load balancer but otherwise I am just waiting to no avail. Any reason?
This is very common issue in for aws elb. What you can do is add following lines at
end of your /etc/rc.local (assuming you are running linux box)
elb-deregister-instances-from-lb <load_balancer_name> --instances <instance-id>
elb-register-instances-with-lb <load_balancer_name> --instances <instance-id>
It first deregisters your instance from elb and then registers back the instance.
Regards
Rajarshi Haldar
Related
My main issue is trying to work out why my health checks are failing on ECS.
My setup
I have successfully set up an ECS cluster using an EC2 auto-scaling group. All the EC2 are in private subnets with NAT gateways.
I have a load-balancer all connected up to the target group which is linked to ECS.
When I try and get an HTTP response from the load balancer from my local machine, it times out. So I am obviously not getting responses back from the containers.
I have been able to ssh into the EC2 instances and confirmed the following:
ECS is deploying containers onto the EC2 instances, then after some time killing them and then firing them up again
I can curl the healthcheck endpoint from the EC2 instance (localhost) and it runs successfully
I can reach the internet from the EC2 instance, eg curl google.com returns an html response
My question is there seems to be two different types of health-check going on, and I can't figure out which is which.
ELB health-checks
The ELB seems, as far as I can tell, to use the health-checks defined in the target group.
The target group is defined as a list of EC2 instances. So does that mean the ELB is sending requests to the instances to see if they are running?
This would of course fail because we cannot guarantee that ECS will have deployed a container to each instance.
ECS health-checks
ECS however is responsible for deploying containers into these instances, in what could turn out to be a many-to-many relationship.
So surely ECS would be querying the actual running containers to find out if they are healthy and then killing them if required.
My confusion / question
I don't really understand what role the ELB has in managing the EC2 instances in this context.
It doesn't seem like the EC2 instances are being stopped and started. However from reading the docs it seems to indicate that the ASG / ELB will manage the EC2 instances and restart them if they fail the healthcheck.
Does ECS somehow override this default behaviour and take responsibility for running the healthchecks instead of the ELB?
And if not, won't the health check just fail on any EC2 instance that happens not to have a container running on it?
I recently took over architecture from a 3rd party to help a client. I'm new to AWS, so this is probably simple, and I just couldn't find it in the docs/stack overflow. They had an existing EC2 instance that had both a node app and a react app deployed, from different repos. Each were deployed using their own pipeline. The source, build, and deploy steps were working for both, and I verified the artifacts were being generated and stored in S3. The load balancer had a target group that hit a single machine in one subnet. The app was running just fine until this morning, and I'm trying to figure out if it's something I did.
My goal this morning was to spin up a new EC2 instance (for which I have the keys, so I can connect directly), a new load balancer that pointed to my machine, and space in S3 for new pipelines I created to store artifacts. I created an AMI from their EC2 instance with the running app and used it to provision my own on the same subnet as their instance. I used the existing security group for my machine. I created a target group to target my machine for use with my load balancer. I created a load balancer to route traffic to this new machine. I then created two pipelines, similar to theirs, but with different artifact locations in S3, and a source of my own repo where I have a copy of the code. I got deployments through the pipeline to work. Everything was great until I was about to test my system, when I was informed their app was down.
I tried hitting it and got a 502, bad gateway. I checked the load balancer and it sees traffic coming in, but gave a 502 for all responses. I checked the target group and it's now showing their EC2 instance as unhealthy. I tried rebooting the machine, but it's still unhealthy, then I tried creating another version of their machine in another subnet, and ensured it was targeted by the target group, but the new instance showed up as unhealthy as well. I can't SSH into the machine because I don't have the key used to create the EC2 instance. If anyone knows where I should look to bring it back online, I'd be forever in your debt.
I undid everything I created this morning, stopping my EC2 instance, and deleting my load balancer, but their app is still returning a 502, showing the instance as unhealthy in their target group.
These are some things to help you debug:
You first need to access the EC2 directly and not through the Load Balancer. Check that the application is running. If the EC2 is in private VPC, you can start an EC2 instance with a public IP and use it as a bastion host.
You will need to have SSH access to the EC2 machine at some point, so that you can look at the logs. This question has answers on how to replace the key pair.
I have created a classic load balancer and auto scaling policy which launch 2 instance successfully; now when I logged in through ssh to one of the load balancer.
ssh -i "mykeypair.pem" ec2-user#my-load-balancer-1222.us-east-1.elb.amazonaws.com
we looged in with the teminla
[ec2-user#ip-10-0-1-86 ~ ] << here this Ip is one of the instance which was created by auto scaling
now I want to check the security-group of the elb from curl http://169.254.169.254/latest/meta-data/security-groups command but it display the instnace security group name not the elb security group.
My question is how can we check the elb security group?
It seems you have SSH'ed into one of the 2 instances behind the load balancer (I doubt you can SSH into the ELB itself), so that's why you're seeing the security group of that instance.
I believe the way to check the ELB's security group is by using the AWS CLI (or one of their SDKs), using the
aws elb describe-load-balancers --load-balancer-name my-load-balancer
You can find more details in the docs
Note: of course, if you wanted to run this command from within the EC2 instance you SSH'ed into, you would need to make sure you have access to make that invocation. See here for more info, on getting set up.
You should not SSH into an instance via a Load Balancer.
An SSH session is persistent -- you wish to continue talking to the same server. This clashes with the concept of a Load Balancer, which distributes traffic across multiple servers.
My question is simple. Does it make sense to have an Amazon Elastic Load Balancer (ELB) with just one EC2 instance?
If I understood right, ELB will switch traffic between EC2 instances. However, I have just one EC2 instance. So, does it make sense?
On the other hand, I´m using Route 53 to route my domain requests example.com, and www.example.com to my ELB, and I don´t see how to redirect directly to my EC2 instance. So, do I need an ELB for routing purposes?
Using an Elastic Load Balancer with a single instance can be useful. It can provide your instance with a front-end to cover for a disaster situation.
For example, if you use an auto-scaling group with min=max=1 instance, with an Elastic Load Balancer, then if your instance is terminated or otherwise fails:
auto-scaling will launch a new replacement instance
the new instance will appear behind the load balancer
your user's traffic will flow to the new instance
This will happen automatically: no need to change DNS, no need to manually re-assign an Elastic IP address.
Later on, if you need to add more horsepower to your application, you can simply increase your min/max values in your autoscaling group without needing to change your DNS structure.
It's much easier to configure your SSL on an ELB than an EC2, just a few clicks in the AWS console. You can even hand pick the SSL protocols and ciphers.
It's also useful that you can associate different security groups to the actual EC2 and the forefront ELB. You can leave the ELB in the DMZ and protect your EC2 from being accessible by public and potentially vulnerable to attacks.
There is no need to use a Load Balancer if you are only running an single Amazon EC2 instance.
To point your domain name to an EC2 instance:
In the EC2 Management Console, select Elastic IP
Allocate New Address
Associate the address with your EC2 instance
Copy the Elastic IP address and use it in your Route 53 sub-domain
The Elastic IP address can be re-associated with a different EC2 instance later if desired.
Later, if you wish to balance between multiple EC2 instances:
Create an Elastic Load Balancer
Add your instance(s) to the Load Balancer
Point your Route 53 sub-domain to the Load Balancer
With NO ELB :-
Less Secure (DOS Attacks possible as HTTP 80 will be open to all, instead of being open only to ELB)
You won't have the freedom of terminating an instance to save EC2 hrs without worrying about remapping your elastic IP(not a big deal tho)
If you don't use ELB and your ec2 instance becomes unhealthy/terminates/goesDown
Your site will remain down (It will remain up if you use ELB+Scaling Policies)
You will have to remap your elastic IP
You pay for the time your elastic IP is not pointing to an instance around $0.005/hr
You get 750 hours of Elastic Load Balancing plus 15 GB data processing with the free tier so why not use it along with a min=1,max=1 scaling policy
On top of the answer about making SSL support easier by putting a load balancer in front of your EC2 instance, another potential benefit is HTTP/2. An Application Load Balancer (ALB) will automatically handle HTTP/2 traffic and convert up to 128 parallel requests to individual HTTP/1.1 requests across all healthy targets.
For more information, see: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#listener-configuration
It really depends on what are you running in the EC2 instance.
While with only one EC2 instance it's not necessary to use ELB (all your traffic will go to that instance anyways), if your EC2 service has to scale in the near future, is not a bad idea to invest some time now and get familiar with ELB.
This way, when you need to scale, it's just a matter of firing up additional instances, because you have the ELB part done.
If your EC2 service won't scale in the near future, don't worry too much!
About the second part, you definitely can route directly to your EC2 instance, you just need the EC2 instance IP. Take a look at the amazon route53 docs. Mind that if your IP is not static (you don't setup an Amazon Elastic IP), you'd need to change the IP mapping everytime the EC2 ip changes.
You can also use an ELB in front of EC2 if for example you want it to be publically reachable, without having to use up an Elastic IP address. As said previously they work well too with ASG's
When I added EC2 instance to 'Elastic Load Balancer' by clicking edit instances button in AWS management console, it gives me status message 'Out Of Service'. Is there is any dependencies to attach instance to load balancer ?
Check instance security group inbound rules, should be added ELB security group
I found out the solution, that is
Health check was not configured properly, it gives 302(redirect error).
I reconfigured health check again with proper ping path...
It's working fine..