I'm using EKS and latest Istio installed via Helm. I'm trying to implement TLS based on a wildcard cert we have for our domain in AWS certificate manager. I'm running into a problem where the connection between the client and the NLB works, with TLS being terminated there, but the NLB can't talk to the istio LB over the secure port. In the AWS console I can rewrite the forwarding rules to forward traffic from port 443 to the standard istio http target, but I can't find a way to do this via code. I'm trying to avoid all click-ops. Here is my Helm overrides for the gateway:
gateways:
istio-ingressgateway:
serviceAnnotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:XXXXXXXXXXXXXXXXXX:certificate/XXXXXXXXXXXXXXXXXXXX"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
So what I'm expecting to occur here is:
Client:443 --> NLB:443 --> Istio Gateway:80
but what I end up with is
Client:443 --> NLB:443 --> Istio Gateway:443
Does anyone have any thoughts on how to get this to work via code? Alternately if someone can point me to a resource to get tls communication between the NLB and Istio working I'm good with that too.
Probably, what is happening is that if you terminate TLS on the load balancer it won't carry SNI to the target group. I had the exact same issue and I ended up solving it by setting the host as '*' on the ingress Gateway and then specifying the hosts on the different VirtualServices (as recommended here and also on istio's official docs).
Your service annotation already correct what is mising is to change istio gateway port 443 to HTTP
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: http-gateway-external
namespace: istio-ingress
spec:
selector:
istio: gateway-external
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
- hosts:
- '*'
port:
name: https
number: 443
protocol: HTTP # Change from HTTPS to HTTP
Related
I have installed nginx ingress controller of type NLB inside EKS cluster and it is of type internal.
The ingress controller created a network load balancer, with listeners 80 and 443,
with port 443 we can't attach an ssl cert for nlb type, only when I use listener type tls it is able to allow us to add ssl cert from AWS ACM.
Now the issue is, I am trying to expose a frontend application through this NLB nginx ingress controller,
when the NLB lister port is 443, it is able to access the application but complains with ssl cert (fake Kubernetes cert), when I change the listener from 443 to tls in NLB, it throws error "400 "The plain HTTP request was sent to HTTPS port" error"
Like many solutions out there mentioning changing the targetPort from https: https to https: http , I tried but with that too same error "The page isn't working,ERR_TOOMANY_REQUESTS"
Could anyone help me how to resolve this issue?
Any ideas or suggestions would be highly appreciated
To resolve the issue with the SSL certificate and the "400 "The plain HTTP request was sent to HTTPS port" error", you may need to modify your ingress configuration to specify that the ingress should listen for HTTPS traffic on port 443. This can be done by adding the following annotations to your ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
name: example
namespace: example
spec:
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example
port:
name: https
tls:
- hosts:
- example.com
secretName: example-tls
In the example above, nginx.ingress.kubernetes.io/ssl-redirect tells the ingress to redirect HTTP traffic to HTTPS. nginx.ingress.kubernetes.io/secure-backends tells the ingress to encrypt the traffic between the ingress and the backend services. `secret
We are trying to bring up a gRPC microservice on AWS EKS. We've gotten to the point where we have an ALB up, however it's giving us this error: A certificate must be specified for HTTPS listeners
Here is our service YAML:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: dev
name: some-service-name
annotations:
alb.ingress.kubernetes.io/backend-protocol-version: GRPC
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 80}, {"HTTPS": 50051}]'
alb.ingress.kubernetes.io/target-type: ip
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internal
service.beta.kubernetes.io/aws-load-balancer-internal: "true"
spec:
rules:
- http:
paths:
- backend:
serviceName: some-service-name
servicePort: 50051
path: /*
We don't want to expose this service externally, and only want internal services to hit it. I feel like we don't even need HTTPS for this, and can use HTTP, however it looks like gRPC requires HTTPS.
What's the correct way to get this working? The examples I've seen seem to be for external-facing services mostly. Do we need to create a private certificate authority, create a certificate from it, and then attribute it to the HTTPS listener in the load balancer settings?
Thanks!
ALB controller expects certificate ARN when we mention listen ports as HTTPS.
There are two options to get it working -
Make listen ports as HTTP
Or add the associated certificate
Since you want to use GRPC, I do not think there is no hard rule to use HTTPS for GTPC
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/annotations/#backend-protocol
And it is still recommended to do HTTPS for internal communication
I deployed kong ingress controller on aws eks cluster with fargate option.
I am unable to access out application over the internet over http port.
I am keep getting -ERR_CONNECTION_TIMED_OUT in browser.
I did follow the Kong deployment as per steps given at -
https://github.com/Kong/kubernetes-ingress-controller/blob/master/docs/deployment/eks.md
Kong-proxy service is created wihtout issue.
kong-proxy service is created yet its “EXTERNAL-IP” is still showing pending.
We are able to access our local application in internal network (by logging on to running pod) via Kong-proxy CLUSTER-IP without any problem using curl.
A nlb load balancer is also created automatically in aws console when we created kong-proxy service. Its DNS name we are using to try to connect from internet.
Kindly help me understand what could be the problem.
My kong-proxy yaml is-
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-type: nlb
name: kong-proxy
namespace: kong
spec:
externalTrafficPolicy: Local
ports:
- name: proxy
port: 80
protocol: TCP
targetPort: 80
- name: proxy-ssl
port: 443
protocol: TCP
targetPort: 443
selector:
app: ingress-kong
type: LoadBalancer
I don't think it's supported now as per https://github.com/aws/containers-roadmap/issues/617
I’m trying to configure SSL for an AWS Load Balancer for my AWS EKS cluster. The load balancer is proxying to a Traefik instance running on my cluster. This works fine over HTTP.
Then I created my AWS Certificate in the Cert Manager, copied the ARN and followed this part of the documentation: Services - Kubernetes
But the certificate is not linked to the Listeners in the AWS Load Balancer. I can’t find further documentations or a working example on the web. Can anyone point me out to one?
The LoadBalancer configuration looks like this:
apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"traefik-ingress-service","namespace":"kube-system"},"spec":{"ports":[{"name":"web","port":80,"targetPort":80},{"name":"admin","port":8080,"targetPort":8080},{"name":"secure","port":443,"targetPort":443}],"selector":{"k8s-app":"traefik-ingress-lb"},"type":"LoadBalancer"}}
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-north-1:000000000:certificate/e386a77d-26d9-4608-826b-b2b3a5d1ec47
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
creationTimestamp: 2019-01-14T14:33:17Z
name: traefik-ingress-service
namespace: kube-system
resourceVersion: "10172130"
selfLink: /api/v1/namespaces/kube-system/services/traefik-ingress-service
uid: e386a77d-26d9-4608-826b-b2b3a5d1ec47
spec:
clusterIP: 10.100.115.166
externalTrafficPolicy: Cluster
ports:
- name: web
port: 80
protocol: TCP
targetPort: 80
- name: admin
port: 8080
protocol: TCP
targetPort: 8080
- name: secure
port: 443
protocol: TCP
targetPort: 80
selector:
k8s-app: traefik-ingress-lb
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- hostname: e386a77d-26d9-4608-826b-b2b3a5d1ec47.eu-north-1.elb.amazonaws.com
Kind Regards and looking forward to your answers.
I had a similar issue since I'm using EKS v1.14 (and nginx-ingress-controller) and a Network Load Balancer, and according to Kubernetes, it's possible since Kubernetes v1.15 - GitHub Issue. And since 10-March-2020 - Amazon EKS now supports Kubernetes version 1.15
So if it's still relevant, read more about it here - How do I terminate HTTPS traffic on Amazon EKS workloads with ACM?.
I ran into the same problem and discovered that the issue was that the certificate type that I chose (ECDSA 384-bit) wasn't compatible with the Classic Load Balancer (but was supported by the new Application Load Balancer). When I switched to an RSA certificate it worked correctly.
I had created a service with the type load balancer and I also configured SSL certificate to it, everything working fine but it's not redirecting my HTTP calls to https until I give https manually before my domain.
Here is my svc.yml
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
annotations:
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "True"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
dns.alpha.kubernetes.io/external: test.example.com
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxxxxxxxxxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
labels:
app: nginx
spec:
type: LoadBalancer
loadBalancerIP:
ports:
- port: 80
name: http
targetPort: 80
- port: 443
name: https
targetPort: 80
selector:
app: nginx
I believe, k8s service object does not have redirection functionality, it is designed to provide a static IP (clusterIP) to the pods who has ephemeral IP. It enables pods to have service discovery functionality in the cluster
A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them - sometimes called a micro-service. The set of Pods targeted by a Service is (usually) determined by a Label Selector (see below for why you might want a Service without a selector).
As an example, consider an image-processing backend which is running with 3 replicas. Those replicas are fungible - frontends do not care which backend they use. While the actual Pods that compose the backend set may change, the frontend clients should not need to be aware of that or keep track of the list of backends themselves. The Service abstraction enables this decoupling.
k8s service
Redirection should happen at the Ingress Level(L7) or at the load balancer(L4) of the cloud provider.