as far as I get the ingress controller documentation, a simple creation of a Service and an Ingress without special annotations should create internet-facing load balancers, weirdly it is creating internal load balancers. So I added the annotation service.beta.kubernetes.io/aws-load-balancer-internal: "false" which is not working either. By the way, I am using NGINX as ingress controller, in the test cluster currently in version 0.8.21. Probably I should update it some time.
Here's my simple spec-file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/tls-acme: "true"
kubernetes.io/ingress.class: nginx
service.beta.kubernetes.io/aws-load-balancer-internal: "false"
labels:
external: "true"
comp: ingress-nginx
env: develop
name: develop-api-external-ing
namespace: develop
spec:
rules:
- host: api.example.com
http:
paths:
- backend:
serviceName: api-external
servicePort: 3000
path: /
tls:
- hosts:
- api.example.com
secretName: api-tls
---
apiVersion: v1
kind: Service
metadata:
labels:
app: api
env: develop
name: api-external
namespace: develop
spec:
ports:
- name: http
port: 3000
protocol: TCP
targetPort: 3000
selector:
app: api
env: develop
sessionAffinity: None
type: ClusterIP
You are not wrong, a service and a ingress should create a load balancer... but you should look at the documentation a bit more...
An ingress needs a NodePort service, yours is ClusterIP. So even if it created something it wouldn't work.
In your ingress you are using kubernetes.io/ingress.class: nginx meaning you want to override the default usage of the ingress and force it to register to the ingress-nginx.
So to make it work, change the type of your service, remove the ingress-class annotation.
You can setup NLB ( Network load balancer) and provide the URL on ingress rule host values. You don't need to expose the underneath backend service either as NodePort or as another load balancer.
Related
Our ingress controller configuration for nginx and aws load balancer
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-type: nlb
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
example Ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test
annotations:
kubernetes.io/ingress.class: nginx
namespace: dev
spec:
rules:
- host: test.domain.com
http:
paths:
- path: /main
pathType: Prefix
backend:
service:
name: main
port:
number: 8000
We are experiencing an issue when changing ACM SSL certificate, after updating LoadBalancer configuration by updating kubernetes deployments, we observe that new certificate is associated with load balancer but the https connection is not working. The only way to make it work was by removing all ingress related resources: controller, jobs, load balancer and create it again, after all works.
I tried to restart nginx controller, I am thinking if ACM cert is saved somewhere in kubernetes during initializing of ingress nginx controller, and after the value of SSL cert the Load Balancer is updated but nginx ingress controller still keep reference of SSL certificate ?
How You guys are updating SSL ACM certs for nginx ingress controller which use aws network load balancer ? What is the correct way of updating nginx ingress controller after updating SSL cert from ACM ?
Thanks,
Igor
I want to deploy Application Load Balancer for Traefik running in Kubernetes. So, I tried the following:
kind: Service
apiVersion: v1
metadata:
name: traefik-application-elb
namespace: kube-system
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-type: "elb"
service.beta.kubernetes.io/aws-load-balancer-name: "eks-traefik-application-elb"
service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "App=traefik,Env=dev"
spec:
type: LoadBalancer
ports:
- protocol: TCP
name: web
port: 80
- protocol: TCP
name: websecure
port: 443
selector:
app: traefik
But internet-facing Classic load balancer was created.
Also tried service.beta.kubernetes.io/aws-load-balancer-type: "alb" but nothing changed.
What is wrong here ?
From the docs: https://docs.aws.amazon.com/eks/latest/userguide/network-load-balancing.html
When you create a Kubernetes Service of type LoadBalancer, the AWS cloud provider load balancer controller creates AWS Classic Load Balancers by default, but can also create AWS Network Load Balancers.
ALB is not supported as a k8s load balancer. You can specify an NLB if desired.
However you can use ALB as an ingress controller with a deployment like this - https://aws.amazon.com/blogs/containers/introducing-aws-load-balancer-controller/
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: SearchApp
annotations:
# share a single ALB with all ingress rules with search-app-ingress
alb.ingress.kubernetes.io/group.name: search-app-ingress
spec:
defaultBackend:
service:
name: search-svc # route traffic to the service search-svc
port:
number: 80
I am new at kubernetes so apologies in advance for any silly questions and mistakes. I am trying to setup external access through ingress for ArgoCD. My setup is an aws eks cluster. I have setup alb following the guide here. I have also setup external dns service as described here. I also followed the verification steps in that guide and was able to confirm that the dns record got created as well and i was able to access the foo service.
For argoCD I installed the manifests via
kubectl create namespace argocd
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml -n argocd
The argoCD docs mention adding a service to split up http and grpc and an ingress setup here. I followed that and installed those as well
apiVersion: v1
kind: Service
metadata:
annotations:
alb.ingress.kubernetes.io/backend-protocol-version: HTTP2
external-dns.alpha.kubernetes.io/hostname: argocd.<mydomain.com>
labels:
app: argogrpc
name: argogrpc
namespace: argocd
spec:
ports:
- name: "443"
port: 443
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/name: argocd-server
sessionAffinity: None
type: ClusterIP
apiVersion: networking.k8s.io/v1 # Use extensions/v1beta1 for Kubernetes 1.18 and older
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/backend-protocol: HTTPS
alb.ingress.kubernetes.io/conditions.argogrpc: |
[{"field":"http-header","httpHeaderConfig":{"httpHeaderName": "Content-Type", "values":["application/grpc"]}}]
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
name: argocd
namespace: argocd
spec:
rules:
- host: argocd.<mydomain.com>
http:
paths:
- backend:
service:
name: argogrpc
port:
number: 443
pathType: ImplementationSpecific
- backend:
service:
name: argocd-server
port:
number: 443
pathType: ImplementationSpecific
tls:
- hosts:
- argocd.<mydomain.com>
The definitions are applied successfully but I don't see the dns record created neither any external IP listed. Am I missing any steps or is there any misconfiguration here? Thanks in advance!
Service type needs to be NodePort.
Im have k8s app (Web api) which first exposed via NodePort (I've used port forwarding to run it and it works as expected)
run it like localhost:8080/api/v1/users
Than I've created a service with type LoadBalancer to expose it outside, which works as expected.
e.g. http://myhost:8080/api/v1/users
apiVersion: v1
kind: Service
metadata:
name: fzr
labels:
app: fzr
tier: service
spec:
type: LoadBalancer
ports:
- port: 8080
selector:
app: fzr
Now we need to make it secure and after reading about this topic we have decided to use ingress for it.
This is what I did
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ctr-ingress
selector:
app: fzr
spec:
ports:
- name: https
port: 443
targetPort: https
now I want to run it like
https://myhost:443/api/v1/users
This is not working, im not able to run the application with port 443 as https, please advice?
It looks to me like you are using a yaml template for a type service to deploy your ingress but not correctly. targetPort should be a numeric port, and anyway, I don't think "https" is a correct value (I might be wrong though).
Something like this:
apiVersion: v1
kind: Service
type: NodePort
metadata:
name: fzr-ingress
spec:
type: NodePort
selector:
app: fzr
ports:
- protocol: TCP
port: 443
targetPort: 8080
Now you have a nodeport service listening on 443 and forwarding the traffic to your fzr pods listening on port 8080.
However, the fact you are listening on port 443 does nothing to secure your app by itself. To encrypt the traffic you need a TLS certificate that you have to make available to the ingress as a secret.
If this seems somewhat complicated (because it is) you could look into deploying an Nginx ingress from a helm chart
In any case your ingress yaml would look something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: gcs-ingress
namespace: default
spec:
rules:
- host: myhost
http:
paths:
- backend:
serviceName: fzr
servicePort: 443
path: /api/v1/users
tls:
- hosts:
- myhost
secretName: myhosts-tls
More info on how to configure this here
We have a cluster of several nodes so I can't do a NodePort and just go to my node-ip (which it's what I've done for testing prometheus).
I did a helm install stable/prometheus and stable/grafana at "monitoring" namespace.
Everything looks okay so far.
Then, I'm trying to create an LB service to access Grafana, which gets created, I can see the CNAME pointing to the A record for the ELB at AWS, but when accessing the URL of Grafana, nothing happens, no HTTP error, no problem page, nothing.
Here's the service-elb.yaml:
apiVersion: v1
kind: Service
metadata:
name: grafana-lb
namespace: monitoring
labels:
app: grafana
annotations:
dns.alpha.kubernetes.io/external: grafana-testing.country.ourdomain
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xxxxxx
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: '443'
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
spec:
selector:
app: grafana
tier: frontend
type: LoadBalancer
ports:
- name: https
port: 443
targetPort: 80
- name: http
port: 80
targetPort: 3000
loadBalancerSourceRanges:
- somerange
- someotherrange
- etc etc
BTW, Got an error of permissions regarding serviceaccount if I don't create the chart with --set rbac.create=false
I recently use a nginx-proxy-pass for Kibana and also use a LB service similar to this with no issue. But I'm missing something here and can't find out what it is yet.
Any help will be much appreciated. I'll update if I make it work.
Solved, had to remove the "tier" selector and just use a spec like this:
spec:
selector:
app: grafana
type: LoadBalancer
ports:
- name: http
port: 3000