We have an EKS cluster in AWS and i am using istio as service mesh in my cluster. We are using istio only for injecting the sidecar into applications and to trace the application traffic through zipkin. To access the application from outside we are not using istio-ingressgateway instead we are using ALB & ELBs
So my problem is I am not getting any traces to zipkin / kiali when i am accessing my application through AWS LBs. Do i have to use istio-ingressgateway to record the traces in zipkin and view in kiali or is there a way to get traces using ALB/ELB as a loadbalancer?
Related
I'm trying to deploy an SMTP service into Kubernetes (EKS), and I'm having trouble with ingress. I'd like not to have to deploy SMTP, but I don't have that option at the moment. Our Kubernetes cluster is using ingress nginx controller, and the docs point to a way to expose TCP connection. I have TCP exposed on the controller via a configmap like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: ingress-nginx-tcp
namespace: ingress-nginx
data:
'25': some-namespace/smtp:25
The receiving service is listening on port 25. I can verify that the k8s part is working. I've used port forwarding to forward it locally and verified with telnet that it's working. I can also access the SMTP service with telnet from a host in the VPC. I just can not access it from the NLB. I've tried 2 different setups:
the ingress-nginx controller nlb.
provisioning a separate nlb that points to the endpoint IP of the service. The TGs are healthy, and I can access the service from a host in the same vpc, that's not in the cluster.
I've verified a least a few dozen times that the security groups are open to all traffic on port 25.
Does anyone have any insights on how to access to expose the service through the NLB?
We're serving our product on AWS EKS where the service is created of type LoadBalancer. The ELB IP is assigned by AWS and this is what is being shared to the client.
However, when we re-deploy the service when we're making some changes/improvements, the ELB IP changes. Since this is causing us to frequently send mails to all the clients, we would need a dedicated IP which needs to be mapped to LB and thus will not change with re-deployment of the service.
Any existing AWS solution or a nice pointer to solve this situation would be helpful.
You can use elastic ip as is described here How to provide elastic ip to aws eks for external service with type loadbalancer?, and here https://docs.aws.amazon.com/es_es/eks/latest/userguide/network-load-balancing.html, just adding an anotation service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eipalloc-xxxxxxxxxxxxxxxxx,eipalloc-yyyyyyyyyyyyyyyyy to the nlb:
service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eipalloc-05666791973f6a240
Another way is to use a domain name (my way). Then use https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/aws.md annotations to link your Service or Ingress with a dns name and configure external-dns to use your dns provider like Route53.
For example:
---
apiVersion: v1
kind: Service
metadata:
name: ambassador
namespace: ambassador
annotations:
external-dns.alpha.kubernetes.io/hostname: 'myserver.mydomain.com'
Every time your LoadBalancer changes the ip the dns server will be updated by the correct ip.
In order to have better control over exposed resources, you can use Ingress Controller such as AWS Load Balancer Controller https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/
With it, you'll be able to re-use the same ALBs for multiple Kubernetes services using alb.ingress.kubernetes.io/group.name annotation. It will create multiple listener rules based on Ingress configuration.
(Applicable if you're not restricted by hardcoded FW rules or similar configurations, that will require you to have static IPs, which is not recommended today)
So, I am very new to using EKS with NLB ingress and managing my own worker nodes using nodegroup (ASG).
If I create a NLB ingress for the cluster and deploy multiple services inside the node group, how does NLB know that it has to load balance across service separately?
Generally, when I have not used EKS and created by own k8s cluster, I have spun one NLB per service. Not sure how would it work in case of EKS with one NLB ingress for the whole cluster with multiple service inside.
Or, do I need to create multiple NLBs somehow?
Any help would be highly appreciated
when I have not used EKS and created by own k8s cluster, I have spun one NLB per service
AWS EKS is no different on this point. For a Network Load Balancer, NLB, e.g. on TCP/UDP level, you use a Kubernetes Service of type: LoadBalancer. But there are options, configured by the annotations on the Service. The most recent feature is IP mode. See EKS Network Load Balancing doc for more configuration alternatives.
Example:
kind: Service
apiVersion: v1
metadata:
name: nlb-ip-svc
annotations:
# route traffic directly to pod IPs
service.beta.kubernetes.io/aws-load-balancer-type: "nlb-ip"
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
type: LoadBalancer
selector:
app: nginx
If I create a NLB ingress for the cluster and deploy multiple services inside the node group, how does NLB know that it has to load balance across service separately?
The load balancer uses the target pods that is matched by the selector: in your Service.
The alternative is to use an Application Load Balancer, ALB that is working on the HTTP/HTTPS level using the Kubernetes Ingress resources. The ALB requires an Ingress controller installed in the cluster and the controller for the ALB is recently updated, see AWS Load Balancer Controller
I am new to Kubernetes and AWS and exploring different AWS technologies for a project. One thing I am doing as part of that is to see how we can have routes in API Gateway connect to an EKS cluster (in a VPC).
This is what I have working:
An EKS Cluster
In the EKS Cluster I have nginx ingress-controller running
I have an EC2 inside the VPC and verified that I can reach a service running in the cluster through EC2 by using the ingress-controller url
This is what I am trying:
I tried to create an API Gateway route to access the same service using the ingress-controller url -> To achieve that, I am trying the steps here (because my cluster is in a VPC): https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-nlb-for-vpclink-using-console.html
One thing that is not clear to me is that, how do I specify the ingress-controller url as a target for the NLB? The only targets that I can specify are EC2 instances, but I want to direct the traffic through the ingress-controller (which is a service of type loadbalancer in K8s).
If I am doing this wrong way, please advice the right way of exposing EKS cluster in API Gateway through the nginx ingress controller. Thanks!
I have found the problem. When using nginx-ingress-controller, I just had to specify the annotation that it is of type "nlb"
service.beta.kubernetes.io/aws-load-balancer-type: nlb
Once I deploy the ingress controller with this annotation, it automatically creates an nlb in aws and sets the target according to the ingress defined! I was creating a new nlb myself and then trying to point to the ingress-controller which is not needed (nor the right way).
I have a k8 cluster deployed in AWS using kube-aws. When I deploy a service, a new ELB is added for exposing the service to internet. Can I use ingress-controller to replace ELB or is there any other way to expose services other than ELB?
First, replace type: LoadBalancer with type: ClusterIP in your service definition. Then you have to configure the ingress and deploy a controller, like Nginx
If you are looking for a full example, I have one here: nginx-ingress-controller.
The ingress will expose you services using some of your workers public IPs, usually 2 of them. Just check your ingress kubectl get ing -o wide and create the DNS records.