I have a load balancer (LB) and an EC2 instance on AWS. My LB has my domain name associated and supports HTTP and HTTPS connections. It has a health-check configured to an endpoint on my EC2 instance (it's running node).
When trying to hit an endpoint via my domain name, the LB doesn't route traffic to my EC2 because it doesn't see it as a healthy instance. I can hit the endpoint directly with the IP address instead. What sort of response do I need to configure so that my EC2 can be recognized as healthy?
Edit: Using an application load balancer.
Edit 2: Health check configuration.
Protocol: HTTPS
Path : /callback
Port : 443
Healthy threshold : 5
Unhealthy threshold : 2
Timeout : 5
Interval : 30
Success codes : 200
You need to provide a path on the EC2 instance - you do NOT need to provide anything in DNS. It should look something like:
Protocol:HTTP
Port: 80
Path: / (or any valid URL on your host that's
a good example of your page working)
No DNS names need to be in there, remember - the ELB already knows which server(s) it's checking against, it just needs to know what to check on that server. Also make sure your security groups allow the ELB to talk to the server on the required ports.
Solved: with the application LB, all that is needed is a 200-level status code from a designated url. This means that you cannot return a simple text response like "Hello World" when they send their health check request.
Related
Basically,
there's an API running on port 7000 on ec2 instance with endpoint "/api/users/data".
So my url becomes:
"http://ec2-public-ip-address:7000/api/users/data"
I configured Target group health-check to use above path with 7000 port and it returned my ec2 as healthy.
Now I want request coming from ELB to be forwarded to same path. Though in ELB logs I checked that port 7000 is being used, but path is still "ec2-private-ip-address:7000".
When I send requests using the ALB's DNS host, the listener's path, and the web services endpoint path, I don't get a response within the expected timeframe, which I've determined by successfully sending
requests directly to each of the tasks using their public ip addresses, they return successful responses.
For example:
The ALB's DNS entry: http://myapp-alb-11111111.us-west-1.elb.amazonaws.com
The web app, "abc", listens on port 80 for requests on "/api/health".
The web app is using "abc-svc/*" as the path in the listener.
The web app was assigned a public ip address of 10.88.77.66.
Sending a GET request to 'http://10.88.77.66/api/health' is successful.
Sending a GET request to 'http://myapp-alb-11111111.us-west-1.elb.amazonaws.com/abc-svc/api/health' does not return within several minutes, which is not expected behavior.
I've looked through the logs, but cannot find anything that is amiss. I'd appreciate any ideas or suggestions...
AWS CONFIGURATION
I have three docker images that are running in ECS. Each image is assigned to a separate service. Each service has a single task. Port 80 is open in the security group from the Internet to the ALB. Port 80 is open from the ALB to each task. The ALB's listener for port 80 is using path-based routing. There is a separate, unique path for each service. Each task contains a docker linux, spring boot 2, web service. Each web service's router has a "/api/health" route that expects a GET request with no parameters and returns a simple string. We are not using HTTP or SSL at this time.
Thank you for your time and interest.
Mike
There is a different reason for that but some of the common issues that you can debug
Check health check for each target group under LB target group, if its unhealthy LB will never route the traffic
Verify the target port is correct
Verify Target group associated properly with LB and is not showing unused.
Verify LB security group
Check the response from LB is it gateway timeout or service unavailbe if gateway timeout its not reachable if service unavailable probably restarting
Services Event logs, check that service is in steady-state or not, if not its mean restarting again and again
Check deployment logs of service, if you see unhealthy target group message then update the target group health path with status code
I created a load balancer and assigned it one of the running EC2 instance. After creation, I navigated to Target Group section in the AWS Console under Load Balancing and when I selected the target group that was assigned to the load balancer, it shows registered instance status as "Unhealthy" and there was a message above registered instance pane that says "None of these Availability Zones contains a healthy target. Requests are being routed to all targets". While creating the load balancer, I selected all the subnets (availability zones).
settings I used for health check are mentioned below,
Protocol: HTTP
Path: /healthcheck.html
Port: traffic port
Healthy threshold: 3
Unhealthy threshold: 2
Timeout: 5
Interval: 10
Success codes: 200
So why does my registered instance status as "Unhealthy" and how can I rectify/resolve that to change the status to "In-service"?
Unhealthy indicates that the health check is failing for the instance.
Things to check:
Check that the instance is running a web server
Check that the web page at healthcheck.html responds with a valid 200 response
Check that instance has a security group that permits access on Port 80 (HTTP)
In my case health check configuration on ALB is / with https.
I resolved with below steps.
Check the security groups - whether we have opened the required ports from ALB SG to EC2 SG.
Login to server and check does IIS server's default site has 443 port opened if your health-check is on 443. (whatever port you are using for health checks).
Use the curl command to troubleshoot the issue.
If you would like to check on HTTPS use the below command to check the response. Use -k or --insecure to ignore the SSL issue.
curl https://[serverIP] -k
For HTTP test use the below command.
curl http://[serverIP]
If you are sharing the load balancer among several EC2 instances that run similar services, make sure each of your services run in a different port otherwise your service won't be reachable and therefore your health check won't pass
I have an EC2 instance with a few applications successfully deployed onto it, listening for connections on ports 3000/3001/3002. I can correctly load a web page from it by connecting to its public DNS or public IP on the given port. I.e. curl http://<ec2-ip-address>:3000 works. So I know that the apps are running, and I know that the port bindings/firewall rules/EC2 security groups are all set up correctly to receive connections from the outside world.
I also have an Application Load Balancer, which is supposed to route traffic to the 3 apps depending on the host name, but it always gives me "504 Gateway Time-out". I've checked all the settings but I can't see what's wrong and I'm not really sure how to troubleshoot it from here.
The ALB has a single HTTPS/443 listener, with a cert that's valid for mydomain.com, app1.mydomain.com, app2.mydomain.com, app2.mydomain.com.
The listener has 3 rules, plus the default rule:
Host == app1.mydomain.com => app1-target-group
Host == app2.mydomain.com => app2-target-group
Host == app3.mydomain.com => app3-target-group
Default action (last resort) => default-target-group
Each target group contains only the single EC2 instance, over HTTP, with the following ports:
app1-target-group: 3000
app2-target-group: 3001
app3-target-group: 3002
default-target-group: 3000
Given that I can access the app directly, I'm sure it must be a problem with the way I've configured the ALB/listener/target groups. But the 504 doesn't give me much to go on.
I've tried to turn on access logs to an S3 bucket, but it doesn't seem to be writing anything there. There's a single object called ELBAccessLogTestFile, and no actual logs in the bucket.
EDIT: Some more information... I actually have nginx installed on the EC2 instance, which is where I was previously doing the SSL termination and hostname-to-port mapping/routing. If I change the default-target-group above to point to port 443 over HTTPS, then it works!
So for some reason, routing traffic
- from the ALB to the EC2 instance over HTTPS on port 443 -> OK!
- from the ALB to the EC2 instance over HTTP on port 3000 -> Broken!
But again, I can hit the instance directly on HTTP/3000 from my laptop.
Communication between resources in the same security group is not open by default. Security group membership alone does not provide special access. You still need to open the ports in the security group to allow other resources in the security group to access those ports. You can specify the security group ID in the rule's source field if you don't want to open it up beyond the resources in the security group.
I have an domain that needs to be routed to both an Application Load Balancer and an EC2-instance depending on the URL path. The Application Load Balancer has a limit of 10 rules per ALB, and I need more.
So to workaround this limit of 10 URLs I would like to setup a request pipeline as follows:
ALB for domain.com -> Docker container with HAProxy with routing rules/reverse proxy -> routes to another ALB or EC2-instance
The setup is fine, I'm having problems with setting up the HAProxy and it's health check. I would like the ALB to health check on a different port rather than the traffic port. In HAProxy I can simply setup multiple frontends, one for the routing (port 80) and one for health check (port 60000). But if I enter port 60000 in the ALBs target group I can't deploy another service due to the dynamic mapping.
Any ideas how to solve this? I rather not expose the health check on port 80 due to it being available for the public net but if that's the only solution it's fine (but how to do it?).
I ended up with using monitor-uri as the healthcheck, not ideal since it's exposed to port 80 but no secret info is showing there anyway.