I have a Kubernetes cluster created through the Kops tool. And I have a requirement to expose my service using a network load balancer. And the target groups should be based on IP based. I have found the answer using the annotation mentioned in the site https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/service/nlb_ip_mode/.
This seems to work only when we have the cluster created through EKS. Since I'm using a kops tool could you please help me in installing alb load balancer controller which is one of the requirements to create a nlb-IP loadbalancers?
If you want to use IP targets, not instance targets, you need to use a CNI that provisions VPC IPs per pod. Those are:
Cilium with ipam
Lyft VPC
AWS VPC
Then you need to install AWS LB controller, which supports this mode both for NLB and ALB. I would wait until kOps 1.20, which will support installing this controller out of the box, including the various permissions that needs setting.
Related
We are designing a new cluster for our application. We are required to use AWS EKS and Consul. We have the following questions:
1) Is it possible to set an AWS ALB ingress (Application load balancing on Amazon EKS - Amazon EKS) as downstream from consul so I can manage it in the rules?
In our local tests we used an nginx ingress and it worked perfectly, but in EKS, nginx ingress uses classic load balancers and these will be deprecated on August 15, 2022 (Elastic Load Balancing migrate-classic-load-balancer.html).
Obviously we can’t create a new project with something that is going to be deprecated so soon.
2) Is ingress-gateway a replacement? Is it possible to create ingress-gateway using ALB ingress-controller from EKS? In the same case, ingress-gateway uses in AWS Classic load balancer and we have the same problem when deprecation.
3) Following this guide: Deploy Consul on Amazon Elastic Kubernetes Service (EKS) | Consul - HashiCorp Learn I see that no type of ingress controller is taken into account, so does it make sense to control external access to services from Consul? Or would income control suffice?
Thank you very much!
Any advice or documentation will be appreciated.
Cheers!
In Kubernetes configuration, for external service component we use:
type: LoadBalancer
If we have k8s cluster running inside a cloud provider like AWS, which provides it own loadbalancer, how does all this work then? Do we need to configure so that one of these loadbalancers is not active?
AWS now takes over the open source project: https://kubernetes-sigs.github.io/aws-load-balancer-controller
It works with EKS(easiest) clusters as well as non-EKS clusters(need to install aws vpc cni etc to make IP target mode work, which is required if you have a peered VPC environment.)
This is the official/native solution of managing AWS LB(aka ELBv2) resources(App ELB, Network ELB) using K8s. Kubernetes in-tree controller always reconciles Service object with type: LoadBalancer
Once configured correctly, AWS LB controller will manage the following 2 types of LBs:
Application LB, via Kubernetes Ingress object. It operates on L7 and provides features related to HTTP
Network LB, via Kubernetes Service object with correct annotations. It operates on L4 and provides less features but claimed MUCH higher throughput.
To my knowledge, this works best when used with external-dns together -- it automatically updates your Route53 record with your LB A records thus makes the whole service discovery solution k8s-y.
Also in general, should prevent usage of classic ELB, as it's marked as deprecated by AWS.
I was using AWS ECS fargate for running my application. I am migrating to AWS EKS. When I use ECS, I deployed a ALB to route request to my service in ECS cluster.
In kubernete, I read this doc https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer, it seems that Kubernete itself has a loadbalance service. And it seems that it creates an external hostname and IP address.
so my question is do I need to deploy AWS ALB? If no, how can I pub this auto-generated hostname in route53? Does it change if I redeploy the service?
Yes you need it to create Kubernetes Ingress using AWS ALB Ingress Controller, the following link explain how use ALB as Ingress controller in EKS: This
You don't strictly need an AWS ALB for apps in your EKS cluster, but you probably want it.
When adopting Kubernetes, it is handy to manage some infrastructure parts from the Kubernetes cluster in a similar way to how you mange apps and in some cases there are a tight coupling between the app and configuration of your load balancer, therefore it makes sense to manage the infrastructure the same way.
A Kubernetes Service of type LoadBalancer corresponds to a network load balancer (also known as L4 load balancer). There is also Kubernetes Ingress that corresponds to an application load balancer (also known as L7 load balancer).
To use an ALB or Ingress in Kubernetes, you also need to install an Ingress Controller. For AWS you should install AWS Load Balancer Controller, this controller now also provides features in case you want to use a network load balancer, e.g. by using IP-mode or expose services using an Elastic IP. Using a pre-configured IP should help with using Route53.
See the EKS docs about EKS network load balancing and EKS application load balancing
As already mentioned from the other guys, yes it is NOT required but it is very helpful to use an ALB.
There are a couple of different solutions to that.. my favorite solution is
Use an Ingress Controller like the ingress-nginx (there are multiple different Ingress Controllers available for Kubernetes, a very good comparison is provided here
Configure the IngressController Service to use NodePort and use a port like 30080
Create an own AWS ALB with Terraform for an example and add the NodePort 30080 to the TargetGroup
Create a Ingress resource to configure the IngressController
If you still have some questions, just ask them here :)
No you don't need ALB and yes, you can use Route53 in an automated manner. Here's a nice article that describes the latter:
https://www.padok.fr/en/blog/external-dns-route53-eks
Can EKS Fargate be used in a private EKS cluster which has no outbound internet access?
According to the AWS documentation, the aws-alb-ingress controller is not supported for private EKS clusters with no outbound internet access:
https://docs.aws.amazon.com/eks/latest/userguide/private-clusters.html
AWS Fargate is supported with private clusters. You must include the
STS VPC endpoint. For more information, see VPC endpoints for private
clusters. You must use a third-party ingress controller with AWS
Fargate because the ALB Ingress Controller on Amazon EKS does not work
in private clusters and because Classic Load Balancers and Network
Load Balancers are not supported on pods running on Fargate.
Unfortunately AWS provides no suggestions here on what the third-party options would be. I have not been able to find any information specific to EKS Fargate Private Clusters.
Questions:
1.) Is there an open source ingress controller that uses ALB that would work for Fargate?
2.) Is there a specific reason why the aws-alb-ingress controller will not work in a private cluster? I might be able to request outbound access for specific ports, if that is the issue, but AWS does not provide any detail on this.
That paragraph in the docs has changed as-of mid/end October, and now says
AWS Fargate is supported with private clusters. You must include the STS VPC endpoint. For more information, see VPC endpoints for private clusters. You can use the AWS load balancer controller to deploy AWS Application Load Balancers and Network Load Balancers with. The controller supports network load balancers with IP targets, which are required for use with Fargate. For more information, see Application load balancing on Amazon EKS and Load balancer – IP targets.
I emphasised the changed part.
So you now can use ALB-based Ingress with private clusters, and the newly-introduced IP-target mode for LoadBalancer Service supports private clusters too.
Note that this requires AWS Load Balancer Controller, which is the new version of aws-alb-ingress-controller.
I ended up doing this with app mesh ingress:
https://aws.amazon.com/blogs/containers/introducing-ingress-support-in-aws-app-mesh/
After trying kubernetes on a few KVMs with kubeadm, I'd like to setup a proper auto-scalable cluster on AWS with kops and serve a few websites with it.
The mind-blowing magic of kops create cluster ... gives me a bunch of ec2 instances, makes the k8s API available at test-cluster.example.com and even configures my local ~/.kube/config so that I can kubectl apply -f any-stuff.yaml right away. This is just great!
I'm at the point when I can send my deployments to the cluster and configure the ingress rules – all this stuff is visible in the dashboard. However, at the moment it's not very clear how I can associate the nodes in my cluster with the domain names I've got.
In my small KVM k8s I simply install traefik and expose it on ports :80 and :443. Then I go to my DNS settings and add a few A records, which point to the public IP(s) of my cluster node(s). In AWS, there is a dynamic set of VMs, some of which may go down when the cluster is not under heavy load. So It feels like I need to use an external load balancer given that my traefik helm chart service exposes two random ports instead of fixed :80 and :443, but I'm not sure.
What are the options? What is their cost? What should go to DNS records in case if the domains are not controlled by AWS?
Configuring your service as a LoadBalancer service is not sufficient for your cluster to to setup the actual loadbalancer, you need an ingress controller running like the one above.
You should add the kops nginx ingress addon: https://github.com/kubernetes/kops/tree/master/addons/ingress-nginx
In this case the nginx ingress controller on AWS will find the ingress and create an AWS ELB for it. I am not sure of the cost, but its worth it.
You can also consider Node Ports which you can access against the node's public ips and node port (be sure to add a rule to your security group)
You can also consider the new AWS ELB v2 or ALB which supports Http/2 and websockets. You can use the alb-ingress-controller https://github.com/coreos/alb-ingress-controller for this.
Finally if you want SSL (which you should) consider the kube-lego project which will automate getting SSL certs for you. https://github.com/jetstack/kube-lego
In my case I used nginx-ingress-controller. I think that setup with traefik will be the same.
1) Set traefik service type as loadBalancer.
Kubernetes will add an ELB rule.
2) Set CNAME or ALIAS in Route53 to ELB hostname.
You can use https://github.com/kubernetes-incubator/external-dns for synchronize exposed services and ingresses with Route53.