Kubernetes Ingress Controller can't reach the site
Hi, this is the first time I am trying to deploy an application with kubernetes. The problem I am facing is I want to be able link subdomains with my svc, but when I try to navigate to the links I get
This site can’t be reached
I will explain the steps I made for these, probably I something is wrong or missing
I installed ingress-controller on google cloud platform
In GCP -> Networking Services -> Cloud DNS
a. I pointed testcompany.com with google dns
b. I created an A record pointing the public IP from the previous step "ingress-nginx-controller"
my svc manifest
apiVersion: v1
kind: Service
metadata:
namespace: staging
name: testcompany-svc
labels:
app: testcompany-svc
spec:
type: NodePort
ports:
- name: test-http
port: 80
protocol: TCP
targetPort: 3001
selector:
app: testcompany
my ingress manifest
apiVersion: networking.k8s.io/v1beta1
- host: api.testcompany.com
http:
paths:
- backend:
serviceName: testcompany-svc
servicePort: test-http
Everything is green and it seems to be working, but when I try to reach the url I get the This site can’t be reached
Update 1
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: staging
name: ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: front.stagingtestcompany.com
http:
paths:
- backend:
serviceName: testcompanyfront-svc
servicePort: testcompanyfront-http
- host: api.stagingtestcompanysrl.com
http:
paths:
- backend:
serviceName: testcompanynodeapi-svc
servicePort: testcompanyapi-http
You should check this, in order:
your Service, Pod, Ingress are in the same namespace: kubectl get all -n staging
your Pod is listening on port 3001: run it locally if you can, or use kubectl port-forward pods/[pod-name] -n staging 3001:3001 and try it locally with http://localhost:3001/...
your Service is reaching your Pod correctly: use kubectl port-forward service/testcompany-svc -n staging 3001:3001 and try it locally with http://localhost:3001/...
check any other Ingress spec rules before the one you posted
check for firewall rules in your VPC network, they should allow traffic from Google LBs
Related
I'm trying to make sense of an example Kubernetes YAML config file that I am trying to customize:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-web-server
namespace: myapp
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/security-groups: my-sec-group
app.kubernetes.io/name: my-alb-ingress-web-server
app.kubernetes.io/component: my-alb-ingress
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: my-web-server
servicePort: 8080
The documentation for this example claims its for creating an "Ingress", or a K8s object that manages inbound traffic to a service or pod.
This particular Ingress resource appears to use AWS ALB (Application Load Balancers) and I need to adapt it to create and Ingress resource in GCP/GKE.
I'm Googling the Kubernetes documentation high and low and although I found the kubernetes.io/ingress.class docs I don't see where they define "alb" as a valid value for this property. I'm asking because I obviously need to find the correct kubernetes.io/ingress.class value for GCP/GKE and I assume if I can find the K8s/AWS Ingress documentation I should be able to find the K8s/GCP Ingress documentation.
I'm assuming K8s has AWS, GCP, Azure, etc. built-in client to kubectl for connecting to these clouds/providers?
So I ask: how does the above configuration tell K8s that we are creating an AWS Ingress (as opposed to an Azure Ingress, GCP Ingress, etc.) and where is the documentation for this?
The documentation you're looking for is :
https://cloud.google.com/kubernetes-engine/docs/concepts/ingress
https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-multi-ssl
An example of an ingress resource :
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-front-api
namespace: example
annotations:
networking.gke.io/managed-certificates: "front.example.com, api.example.com"
kubernetes.io/ingress.global-static-ip-name: "prod-ingress-static-ip"
spec:
rules:
- host: front.example.com
http:
paths:
- backend:
service:
name: front
port:
number: 80
path: /*
pathType: ImplementationSpecific
- host: api.example.com
http:
paths:
- backend:
service:
name: api
port:
number: 80
path: /*
pathType: ImplementationSpecific
I have been trying to create a kubernetes cluster on Google kubernetes Engine. My pods are sucessfully running but the problem is with the ingress controller. The ingress conroller is not showing the external IP to access the application.
And the YAML file for nginx ingress controller looks like this :
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: http-ingress
labels:
app: ingress
annotations:
kubernetes.io/ingress.class: addon-http-application-routing
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: nodeapp1svc
servicePort: 80
- path: /app1
backend:
serviceName: nodeapp2svc
servicePort: 80
- path: /app2
backend:
serviceName: nodeapp2svc
servicePort: 80
What can I do next?
It looks like the problem is related with your annotations, specifically with this one:
kubernetes.io/ingress.class: addon-http-application-routing
The ingress.class you're trying to use is something specific to Azure AKS so definitely you cannot use it on your GKE Cluster.
Note that you can omit kubernetes.io/ingress.class annotation at all if you want your default GKE Ingress controller - ingress-gce to be used.
I tested it on my GKE cluster and without the mentioned above annotation it works just fine.
As to your specific setup, I noticed one more problem, namely your nodeapp[1-3]svc Services are of a type ClusterIP and they need to be either NodePort or LoadBalancer.
If you run:
kubectl describe ingress http-ingress
and take a look at the events section, you may encounter the error message like the one below:
loadbalancer-controller error while evaluating the ingress spec: service "default/nodeapp1svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"
Summary:
use the correct ingress.class i.e. omit this annotation at all and the default ingress controller will be used.
make sure your backends are exposed via NodePort rather than ClusterIP.
I am trying to implement SonarQube in a Kubernetes cluster. The deployment is running properly and is also exposed via a Virtual Service. I am able to open the UI via the localhost:port/sonar but I am not able to access it through my external ip. I understand that sonar binds to localhost and does not allow access from outside the remote server. I am running this on GKE with a MYSQL database. Here is my YAML file:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sonarqube
namespace: sonar
labels:
service: sonarqube
version: v1
spec:
replicas: 1
template:
metadata:
name: sonarqube
labels:
name: sonarqube
spec:
terminationGracePeriodSeconds: 15
initContainers:
- name: volume-permission
image: busybox
command:
- sh
- -c
- sysctl -w vm.max_map_count=262144
securityContext:
privileged: true
containers:
- name: sonarqube
image: sonarqube:6.7
resources:
limits:
memory: 4Gi
cpu: 2
requests:
memory: 2Gi
cpu: 1
args:
- -Dsonar.web.context=/sonar
- -Dsonar.web.host=0.0.0.0
env:
- name: SONARQUBE_JDBC_USERNAME
valueFrom:
secretKeyRef:
name: cloudsql-db-credentials
key: username
- name: SONARQUBE_JDBC_PASSWORD
valueFrom:
secretKeyRef:
name: cloudsql-db-credentials
key: password
- name: SONARQUBE_JDBC_URL
value: jdbc:mysql://***.***.**.*:3306/sonar?useUnicode=true&characterEncoding=utf8
ports:
- containerPort: 9000
name: sonarqube-port
---
apiVersion: v1
kind: Service
metadata:
labels:
service: sonarqube
version: v1
name: sonarqube
namespace: sonar
spec:
selector:
name: sonarqube
ports:
- name: http
port: 80
targetPort: sonarqube-port
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sonarqube-internal
namespace: sonar
spec:
hosts:
- sonarqube.staging.jeet11.internal
- sonarqube
gateways:
- default/ilb-gateway
- mesh
http:
- route:
- destination:
host: sonarqube
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sonarqube-external
namespace: sonar
spec:
hosts:
- sonarqube.staging.jeet11.com
gateways:
- default/elb-gateway
http:
- route:
- destination:
host: sonarqube
---
The deployment completes successfully. My exposed services gives a public ip that has been mapped to the host url but I am unable to access the service at the host url.
I need to change the mapping such that sonar binds with the server ip but I am unable to understand how to do that. I cannot bind it to my cluster ip, neither to my internal or external service ip.
What should I do? Please help!
I had the same issue recently and I managed to get this resolved today.
I hope the following solution will work for anyone facing the same issue!.
Environment
Cloud Provider: Azure - AKS
This should work regardless of whatever provider you use.
Istio Version: 1.7.3
K8 Version: 1.16.10
Tools - Debugging
kubectl logs -n istio-system -l app=istiod
logs from Istiod and events happening in the control plane.
istioctl analyze -n <namespace>
This generally gives you any warnings and errors for a given namespace.
Lets you know if things are misconfigured.
Kiali - istioctl dashboard kiali
See if you are getting inbound traffic.
Also, shows you any misconfigurations.
Prometheus - istioctl dashboard prometheus
query metric - istio_requests_total. This shows you the traffic going into the service.
If there's any misconfiguration you will see the destination_app as unknown.
Issue
Unable to access sonarqube UI via external IP, but accessible via localhost (port-forward).
Unable to route traffic via Istio Ingressgateway.
Solution
Sonarqube Service Manifest
apiVersion: v1
kind: Service
metadata:
name: sonarqube
namespace: sonarqube
labels:
name: sonarqube
spec:
type: ClusterIP
ports:
- name: http
port: 9000
targetPort: 9000
selector:
app: sonarqube
status:
loadBalancer: {}
Your targetport is the container port. To avoid any confusion just assign the service port number as same as the service targetport.
The port name is very important here. “Istio required the service ports to follow the naming form of ‘protocol-suffix’ where the ‘-suffix’ part is optional” - KIA0601 - Port name must follow [-suffix] form
Istio Gateway and VirtualService manifest for sonarqube
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sonarqube-gateway
namespace: sonarqube
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 9000
name: http
protocol: HTTP
hosts:
- "XXXX.XXXX.com.au"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sonarqube
namespace: sonarqube
spec:
hosts:
- "XXXX.XXXX.com.au"
gateways:
- sonarqube-gateway
http:
- route:
- destination:
host: sonarqube
port:
number: 9000
Gateway protocol must be set to HTTP.
Gateway Server Port and VirtualService Destination Port is the same. If you have different app Service Port, then your VirtualService Destination Port number should match the app Service Port. The Gateway Server Port should match the app Service Targetport.
Now comes to the fun bit! The hosts. If you want to access the service outside of the cluster, then you need to have your host-name (whatever host-name that you want to map the sonarqube server) as an DNS A record mapped to the External Public IP address of the istio-ingressgateway.
To get the EXTERNAL-IP address of the ingressgateway, run kubectl -n istio-system get service istio-ingressgateway.
If you do a simple nslookup (run - nslookup <hostname>), The IP address you get must match with the IP address that is assigned to the istio-ingressgateway service.
Expose a new port in the ingressgateway
Note that your sonarqube gateway port is a new port that you are introducing to Kubernetes and you’re telling the cluster to listen on that port. But your load balancer doesn’t know about this port. Therefore, you need to open the specified gateway port on your kubernetes external load balancer. Ref - Info
You don’t need to manually change your load balancer service. You just need to update the ingress gateway to include the new port, which will update the load balancer automatically.
You can identify if the port is causing issues by running istioctl analyze -n sonarqube. You should get the following warning;
[33mWarn[0m [IST0104] (Gateway sonarqube-gateway.sonarqube) The gateway refers to a port that is not exposed on the workload (pod selector istio=ingressgateway; port 9000) Error: Analyzers found issues when analyzing namespace: sonarqube. See https://istio.io/docs/reference/config/analysis for more information about causes and resolutions.
You should get the corresponding error in the control plane. Run kubectl logs -n istio-system -l app=istiod.
At this point you need to update the Istio ingressgateway service to expose the new port. Run kubectl edit svc istio-ingressgateway -n istio-system and add the following section to the ports.
Bypass creating a new port
In the previous section you saw how to expose a new port. This is optional and depending on your use case.
In this section you will see how to use a port that is already exposed.
If you look at the service of the istio-ingressgateway. You can see that there are default ports exposed. Here we are going to use port 80.
Your setup will look like the following;
To void specifying the port with your host name just add match uri prefix, as shown in the virtualservice manifest.
Time for testing
If everything works up to this point as expected, then you are good to go.
During testing I made one mistake by not specifying the port. If you get 404 status, Which is still a good thing, in this way you can verify what server it is using. If you setup things correctly, it should use the istio-envoy server, not the nginx.
Without specifiying the port. This will only work if you add the match uri prefix.
Donot pass argument just try running without it once working for me.
This is how my deployment file hope helpful
apiVersion: v1
kind: Service
metadata:
name: sonarqube-service
spec:
selector:
app: sonarqube
ports:
- protocol: TCP
port: 9000
targetPort: 9000
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: sonarqube
name: sonarqube
spec:
replicas: 1
template:
metadata:
labels:
app: sonarqube
spec:
containers:
- name: sonarqube
image: sonarqube:7.1
resources:
requests:
memory: "1200Mi"
cpu: .10
limits:
memory: "2500Mi"
cpu: .50
volumeMounts:
- mountPath: "/opt/sonarqube/data/"
name: sonar-data
- mountPath: "/opt/sonarqube/extensions/"
name: sonar-extensions
env:
- name: "SONARQUBE_JDBC_USERNAME"
value: "root" #Put your db username
- name: "SONARQUBE_JDBC_URL"
value: "jdbc:mysql://192.168.112.4:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true" #DB URL
- name: "SONARQUBE_JDBC_PASSWORD"
value : password
ports:
- containerPort: 9000
protocol: TCP
volumes:
- name: sonar-data
persistentVolumeClaim:
claimName: sonar-data
- name: sonar-extensions
persistentVolumeClaim:
claimName: sonar-extensions
I deployed a "monolithic" app into kubernetes on AWS. This app works fine through the ALB.
Next I want to deploy a small service at the same cluster and map traffic to it through the same ALB ingress.
Here is how the Ingress manifest looks like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: scala-backend-ingress
namespace: prod
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
labels:
app: akka-backend
spec:
rules:
- http:
paths:
- path: /proxy/service/*
backend:
serviceName: proxy-service-np
servicePort: 80
- path: /*
backend:
serviceName: akka-main-np
servicePort: 80
Unfortunately when I call:
GET www.aliace.example.com/proxy/service/traffic/data
I receive back 502 Bad Gateway response with header Server → awselb/2.0.
All traffic to /* is handled properly.
The problem was not in kubernetes.
The application in the container was bounded to localhost instead of 0.0.0.0
can you try as below
- path: /proxy/service/*/*
backend:
serviceName: proxy-service-np
servicePort: 80
recently I am trying to set up CI/CD flow with Kubernetes v1.7.3 and jenkins v2.73.2 on AWS in China (GFW blocking dockerhub).
Right now I can expose services with traefik but it seems I cannot expose the same service with the same URL with two different ports.
Ideally I would want expose http://jenkins.mydomain.com as jenkins-ui on port 80, as well as the jenkin-slave (jenkins-discovery) on port 50000.
For example, I'd want this to work:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jenkins
namespace: default
spec:
rules:
- host: jenkins.mydomain.com
http:
paths:
- path: /
backend:
serviceName: jenkins-svc
servicePort: 80
- host: jenkins.mydomain.com
http:
paths:
- path: /
backend:
serviceName: jenkins-svc
servicePort: 50000
and my jenkins-svc is defined as
apiVersion: v1
kind: Service
metadata:
name: jenkins-svc
labels:
run: jenkins
spec:
selector:
run: jenkins
ports:
- port: 80
targetPort: 8080
name: http
- port: 50000
targetPort: 50000
name: slave
In reality the latter rule overwrites the former rule.
Furthermore, There are two plugins I have tried: kubernetes-cloud and kubernetes.
With the former option I cannot configure jenkins-tunnel URL, so the slave fails to connect with the master; with the latter option I cannot pull from a private docker registry such as AWS ECR (no place to provice credential), therefore not able to create the slave (imagePullError).
Lastly, really I am just trying to get jenkins to work (create slaves with my custom image, build with slaves and delete slaves after jobs' finished ), any other solution is welcomed.
If you want your jenkins to be reachable from outside of your cluster then you need to change your ingress configuration.
Default type of ingress type is ClusterIP
Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. This is the default ServiceType
You want it type to be NodePort
Exposes the service on each Node’s IP at a static port (the NodePort). A ClusterIP service, to which the NodePort service will route, is automatically created. You’ll be able to contact the NodePort service, from outside the cluster, by requesting :
So your service should look like:
apiVersion: v1
kind: Service
metadata:
name: jenkins-svc
labels:
run: jenkins
spec:
selector:
run: jenkins
type: NodePort
ports:
- port: 80
targetPort: 8080
name: http
- port: 50000
targetPort: 50000
name: slave