AWS Load balancer OutOfService, Instance though InService - amazon-web-services

Followed the entire tutorial at http://docs.aws.amazon.com/gettingstarted/latest/wah-linux/getting-started-create-as.html and the problem I am having is with the load balancer.
According to the Instances page and the Auto Scaling Groups page, the two instances I am running are InService and Healthy.
On the Load Balancers page under the Instances tab for the load balancer, it lists each instance as OutOfService and each Availability Zone's Health status as "No (Availability Zone contains no healthy instances)".
The part that is most confusing to me (and I believe to be the main issue) is that when I visit one of the instance's at its public dns ec2-server-name.us-west-2.compute.amazonaws.com, it displays the default "Amazon Linux AMI Test Page" but the headers of the page return 403.
So I imagine the issue is that the load balancer is running its health check and seeing that the instance's returned status code is 403. Why is it returning a 403 error code though? When I load the instance in browser I can see the page and everything seems to work, but when I visit the load balancer all I get is a blank white page.

Discovered the issue. Amazon's default "Amazon Linux AMI Test Page" returns a 403 error by default. Once new files are uploaded to /var/www/html and replace the default page, the problem goes away.

Related

I get 404 error when doing a synchronous request to AWS Gateway with Load Balancer

I created a REST AWS API Gateway and it worked perfectly when it was targeting a single ec2 instance. I then went on to set it up with an EC2 Load Balancer for a Target Group with 2 EC2 instances. Now when I make a request that I synchronously get the status of, I get a 404 error. My guess is that the initial job was posted on one machine and then I try to access it on the other machine yielding a 404 error. I tried to enable stickiness to the target group, but that did nothing. Any suggestions?
Stickiness config
I would suggest you to check the logs on your EC2 instances and see which is the exact request routed from the LB to the EC2 machine. My experience is that LB calls the EC2 instances using their internal IP address and the URL might be modified, based on configuration.
Checking the logs will help you debug this error. With stickness you're doing good.

Why is my EC2 instance is returning 400 error

I have deployed a Django website on AWS Elastic Beanstalk - the environment is green and website available via the url provided. However, I have set an application load balancer with the EC2 instance of the ELB application as the target group, and this is returning Bad Request (400). When trying to access the instance via the public DNS I get the same Bad Request (400).
I have checked the security groups and all required ports (80 for http, 443 for https and 22 for ssh) are open on the instances security group and the target groups security group.
I am unsure what else could be causing this as the django app is available and the instance has status running and I can ssh into it. Have checked the AWS docs and other similar questions on SO but can't find any possible solutions.
So, the answer to this was 'check the logs' facepalm. Logs clearly showed there was a disallowed host error which was easily fixed. Credit to Iain Shelvington who reminded me to check the logs.

AWS - ELB - Magento2 How do I perform health checks

I have an ELB setup for my Magento2 application. The application is running on EC2 instances. In Magento 2 I need to specify a base url for the site. I am setting that as my load balancer public dns.
When the ELB performs Health Checks on the individual EC2 Instances they are returning a 302 as magento is trying to redirect the call to the public dns record for the ELB.
How do I deal with this?
I created a file health.html and placed this in the root magento folder on the EC2 instances.
I updated the health check to load /health.html.
This works fine and the Load Balancers are able to direct traffic to these instances as they are healthy.
This is not really ideal and mainly served to verify configurations between M2 and the ELB and EC2 instances.
I would like the health check to make sure Magento2 is actually healthy.
You could assign the health endpoint to a magento action directly.
I updated the health check to load /health.html.
Set that to an HTTP request declared in your Application routes, and add your checks there. /health/action for example.
I found the answer, There is a setting in Stores->Configuration->General->Web->Url-Options that allows you to turn off the auto redirect. I disabled this and the checks are now working

http 502 errors when new instance is being created in a group

We are using cross region load balancing. When we get heavy traffic all at once, within 1 region, it begins to spin up new instances. While it is starting new instances, we get random HTTP 502 errors. Screenshots of configurations below. Is there any way to avoid the 502 errors while it is scaling up?
Image links of configuration below.
Instance Group Configuration (same setting on all regions)
Load Balancer
Thanks in advance for the help!
HTTP load balancer and the instances will have different external IPs.
1) Try accessing through one instance's external IP first to make sure the backend works. If it doesn't work, usually it's firewall settings problem.
2) HTTP 502 from load balancer usually indicates the health check of the load balancer thought the backend is unhealthy, check your health check config then.
See another similar question Google Load-balancer randomly failing requests to backend

AWS load balancer and maintenance page

I'm using AWS Load Balancer with 3 EC2 servers, and I'm trying to serve a Maintenance page when site is under maintenance.
This page need to return 503 HTTP code, because it is a proper code for a maintenance mode and will prevent possible problems with SEO.
When I return 503 code from any of my servers, Load Balancer makes it "Not In Service", and when all servers return 503, website returns a blank page (because all servers are disconnected).
My questions are:
1) Is there any way to serve a custom static page with a message for visitors from Load balancer if there is no healthy servers?
2) Or how to configure Load Balancer's Health Check that it will not consider 503 as a reason to mark server as "unhealthy"?
Thanks!
I've been searching for a quick way to do this. We need to return a 503 error to the world during DB upgrade, but white list a few IPs of developers so they can test it before opening back up to public.
Found a one spot solution::
Go to the Loader Balancer in EC2 and select the load balancer you would like to target. Below, you should see Listeners. Click on a listener, and edit the rule. Create a rule like this:
Now everyone gets a pretty maintenance page returned with a 503 error code, and only two IP addresses in the first rule will be able to browse to the site. Order is important, where the two IP exceptions are on top, then it goes down the list. The last item is always there by default.
Listener Rules for Your Application Load Balancer:
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-update-rules.html
You could implement an additional route in your app server, let's say /hcm (for health check maintenance), that always responds 200 OK. When it's time for maintenance, you programmatically modify the ELB health check to use the /hcm target which returns 200 OK rather than / or /index.html, which both return 503 Service Unavailable. Revert these changes when exiting maintenance.
Might not meet your 503 requirement but a good option for this is using s3 and dns failover: https://aws.amazon.com/blogs/aws/create-a-backup-website-using-route-53-dns-failover-and-s3-website-hosting/
The load balancer will serve a 503 for you when you no longer have any healthy server behind it so you should not do anything special.
If you return anything but a 200 on the health check, ELB will take the machine out of the load balancer after it fails the configured number of health checks.
So to recap, you can potentially serve 503 from your app when in maintenance, but you have to return 200 for health checks all the time. If you don't care about the content of the page, you can simply remove the machines from the load balancer (or fail health checks) and the LB will do the right thing for you.