I setup a load balancer in east region and then tried to access it by ip in the central region. Accessing the LB from the central region hangs like the packets are dropped. I was successful in accessing the VM that is in the LB's instance group directly so it does appear that it is LB specific.
I tried updating the routes and firewall, but that appeared to already be correct and didn't have an affect.
setting up new virtual machines just to route traffic to the instance group seems very error prone.
It is not possible.
the load balancing they use doesn't require a server and only works within a single region.
Related
I have an Elastic Beanstalk server behind an Application Load Balancer, all inside a VPC. The first call to the server after leaving it along for a while takes a very long time. It's almost as if the instance is being booted up right then! Instead of being already on...
This issue does not present locally, nor outside of a VPC, it only happens in the VPC on AWS so something in my configuration must be off.
The VPC has 3 public and 3 private subnets, in the same availability zones, and the public subnets all have auto-assign public IP on
I've assigned these to the network settings on my Elastic Beanstalk environment, assigning the public subnets to the public load balancer, and then the private subnets to the private instance.
I've set the auto scaling load balanced group with minimum of 3 instances, and confirmed they're running
Despite this, after leaving the site alone for a while... the first new call to the server consistently takes over one minute, and then works great. I assume I'm just missing something small but cannot figure out what it is...
Thanks in advance!
I am convinced this is not an application issue because, on first load the call takes over one minute, but on subsequent loads it's near instant, and this behavior is constant across days. Locally, I never have this issue. Outside a VPC, I never have this issue.
first/slow load (after leaving the app alone overnight)
second/fast load (refreshing right after the above)
UPDATE
AWS support suggested I deassociate the subnets from my route tables. I did that and now all subnets public and private are showing current routing table Main. Now though, instead of taking a long time all calls to my server are failing!
I tried attaching the internet gateway in that VPC to the routing table via edge association but I'm getting the error that
Route table contains unsupported route destination. The unsupported route destination is less specific than or non-overlapping with VPC local CIDR
There is one public subnet with overlapping CIDRs with the internet gateway (10.1.0.0/24 on the subnet and 10.1.0.0/24 on the gateway). I tried manually associating that to the Main routing table but still get the same error
I this this what a similar question. Assuming you are using Auto-Scaling most likely you will want to add a LifeCycle hook. You can do that in ElasticBeanstalk as shown here, by editing the .ebextensions/as-hook.config file. You will use the EC2_INSTANCE_LAUNCHING transition. In your hook make a request similar to what your application does which can ensure that the instance is ready to start serving traffic.
If you still have issues you could use more techniques for debugging such as X-Ray but you would need to then run the Agent and configure the sdk for your app which would be a bit of work so it may not be worth the effort. If it is simply the first request which takes a long time hooks will be the way to go.
Is it possible in GCP to create an internal load balancer that balances the traffic between two Compute Instances in different regions?
Two instances (written NAT on them) are in different regions (e.g one in us-central1 and other in asia-south1) serving something on the same ports and the internal load balancer (e.g with IP: 170.0.0.4) is accepting requests from the clients and forwarding them to these VMs.
This would help in creating a highly available service (NAT in this case) that will work even when one VM or the service or region is down.
EDIT:
Adding some more details here:
Both VMs and the Load Balancer have internal IPs.
Both VMs and the Load Balancer are in the same VPC network
I need a layer 7 (HTTP(S)) internal lLoad balancer.
Internal Load balancing is only regional and since you want to have back-end in different regions it will still work but you will have to set up one by yourself.
It's not possible "out of the box".
You can have a look at the Internal Load Balacing documentation which explains how this works and why. Here's also a table with available Load balancing options.
If you want to configure your own LB then maybe try Envoy Proxy (or Nginx, or any solution you want).
In essence - unless you set up your own load balancer GCP doesn't have the functionality.
You could also use external load balancer (which is risky) and use it to load balance traffic and restrict external traffic to your instances (just a bunch of known IP's or ranges).
With only two endpoints, it is impossible to use a load-balancer, because there is nothing to balance. You could only put both of them behind a load balancer, then it could balance their traffic. Moving both into the same region might be the only option to use the internal load balancer, but also here, the servers need to be put behind it.
We have an application (lets call it AppA) that in running in an AWS VPC in a private subnet and there is another application (AppB) which is an on-prem app that talks to AppA via direct connect. AppA is a cluster over multiple AZ’s and we want to put a load balancer in front of it. Putting a load balancer we run into the risk of creating a single point of failure which we want to eliminate. App B is designed only to recognise AppA’s IP address only (no DNS). My question is how we can avoid single point of failure in this situation
The load balancer availability is stated to be 99.99% by AWS, which meets the Highly Available requirements.
https://aws.amazon.com/elasticloadbalancing/
You can see this by performing a dig command on your ELB and seeing that it returns multiple addresses.
By default an ELB will only send traffic to instances in the region the ELB is in. If you want cross region failover you would need to look here - https://aws.amazon.com/blogs/aws/amazon-route-53-elb-integration-dns-failover/
If I have the following VPC in AWS:
10.0.0.0/16 and I provision an application load balancer (internal) and AWS selects the following ip addresses for me 10.0.0.9 and 10.0.0.12 inside the subnets I choose.
Question: Do the internal addresses (10.0.0.9 and 10.0.0.12) that are picked ever change for the life of the load balancer?
I understand if I delete the load balancer, it will pick new ones.
I also understand that an internet application load balancer IP changes regularly (and thats why people use Network load balancers for static ips) but not much is said about the private internal ips associated with the load balancers.
Any information would be great.
Yes, they could change for application load balancers.
As the application load balancer scales with traffic it will "launch" more instances behind the scenes and use more IPs in your subnets (ENI creation). You don't see those instances in the console but you can have a look at the elastic network interfaces in the console. When it scales down, it's not guaranteed that you get the same IPs. This is why you always need some free space in the subnets used by your application load balancer.
Network load balancers have static private IPs (one ENI per availability zone). Those can be used in the security group of your instances, see Target Security Groups.
Yes it may change with thin the range provided... when more instanaces are launched if ASG is configured or without it due to increased load on the application, there can be more IPs used instead of old ones and it is in the range of your Subnet parameters.
I'm looking for a way to allow private traffic over the public ELB for inter-region communication between the workers. One way to do this is by checking the private IPs from the ELB frequently and update the DNS record.
Getting the private IPs is possible with the CLI:
aws ec2 describe-network-interfaces --region "us-west-1" --filters Name=requester-id,Values='amazon-elb' | jq -r '.NetworkInterfaces[].PrivateIpAddresses[].PrivateIpAddress'
Will give you back a list of used IPs (number depends on the amount of availability zones selected when creating the loadbalancer).
Next step would be to update this in Route53 when changed.
Lambda might be an idea to do this but I noticed that getting these ips takes some time and it sometimes even hit the 3 seconds timeout of lambda. So looking for a better way to do this.
I am looking for a way to access services/applications in a remote k8s cluster(C2) hosted in a different region(R2) from a client application in my current cluster(C1 in region R1).
Server application needs to load-balanced(fqdn preferred over IP)
Communication is through private network, no internet
I tried using an internal-LB for C2 which doesn't work and later realized it to be a regional product.
Moreover, it seems, the same constraint is true for vpc peering also.
Please suggest how to achieve this.
You can't use any internal GCP LB on a regional level. However, you may be able to use an Nginx internal ingress as it may not be limited to the same region.
Otherwise you can use Creating VPC-native clusters using Alias IPs which can allow you to call on pods directly. It will not offer built in load balancing but it is an alternative.
Finally, if you need to use the internal load balancers, you can create a VPN tunnel between the two regions and create a route that forces traffic through the gateway. Traffic coming through the tunnel will be regional to the ILB, but this config is more expensive and with more moving parts, there's a higher chance of failure