ISTIO setup for egress gateway to route database connections - istio

I have installed istio with istioctl in my k8s with this command :
istioctl install -s "components.egressGateways[0].name=istio-egressgateway" -s "components.egressGateways[0].enabled=true" --set "values.gateways.istio-egressgateway.ports[0].port=80" --set "values.gateways.istio-egressgateway.ports[0].name=http" --set "values.gateways.istio-egressgateway.ports[1].port=443" --set "values.gateways.istio-egressgateway.ports[1].name=https" --set "values.gateways.istio-egressgateway.ports[2].port=27017" --set "values.gateways.istio-egressgateway.ports[2].name=mongo" --set "values.gateways.istio-egressgateway.ports[3].port=3306" --set "values.gateways.istio-egressgateway.ports[3].name=mysql" --set "values.gateways.istio-egressgateway.ports[4].port=3307" --set "values.gateways.istio-egressgateway.ports[4].name=mysql1"
then i have create serviceentry with this file:
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"ServiceEntry","metadata":{"annotations":{},"name":"mysql","namespace":"istio-system"},"spec":{"addresses":["170.187.156.142/32"],"endpoints":[{"address":"170.187.156.142"}],"hosts":["my-mysql.tcp.svc"],"location":"MESH_EXTERNAL","ports":[{"name":"tcp","number":3306,"protocol":"TCP"}],"resolution":"STATIC"}}
creationTimestamp: "2022-04-11T07:37:54Z"
generation: 5
name: mysql
namespace: istio-system
resourceVersion: "1506047"
uid: 4b872f99-97b6-4e4e-a9fc-8f9763422224
spec:
addresses:
- 170.187.XX.XX/32
endpoints:
- address: XX.XX.XX.XX
hosts:
- my-mysql.tcp.svc
location: MESH_EXTERNAL
ports:
- name: tcp
number: 3306
protocol: TCP
resolution: STATIC
then i have created gateway with this file:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"Gateway","metadata":{"annotations":{},"name":"istio-egressgateway","namespace":"istio-system"},"spec":{"selector":{"istio":"egressgateway"},"servers":[{"hosts":["my-mysql.tcp.svc"],"port":{"name":"tcp","number":3306,"protocol":"TCP"}}]}}
creationTimestamp: "2022-04-11T07:13:09Z"
generation: 7
name: istio-egressgateway
namespace: istio-system
resourceVersion: "1506384"
uid: af4f8b04-b0ce-41e2-9c71-008221ae3c64
spec:
selector:
istio: egressgateway
servers:
- hosts:
- '*'
port:
name: tcp
number: 3306
protocol: TCP
then i have created destination rule with this file:
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"DestinationRule","metadata":{"annotations":{},"name":"egressgateway-for-mysql","namespace":"istio-system"},"spec":{"host":"istio-egressgateway.istio-system.svc.cluster.local","subsets":[{"name":"mysql"}]}}
creationTimestamp: "2022-04-11T07:37:59Z"
generation: 1
name: egressgateway-for-mysql
namespace: istio-system
resourceVersion: "895373"
uid: a861360a-5bfc-4d6e-adc8-d784e2f8f543
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mysql
and finally created VirtualService with this file:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"direct-mysql-through-egress-gateway","namespace":"istio-system"},"spec":{"gateways":["mesh","istio-egressgateway"],"hosts":["my-mysql.tcp.svc"],"tcp":[{"match":[{"destinationSubnets":["170.187.156.142/32"],"gateways":["mesh"],"port":3306}],"route":[{"destination":{"host":"istio-egressgateway.istio-system.svc.cluster.local","port":{"number":3306},"subset":"mysql"}}]},{"match":[{"gateways":["istio-egressgateway"],"port":3306}],"route":[{"destination":{"host":"my-mysql.tcp.svc","port":{"number":3306}},"weight":100}]}]}}
creationTimestamp: "2022-04-11T07:38:02Z"
generation: 8
name: direct-mysql-through-egress-gateway
namespace: istio-system
resourceVersion: "1505867"
uid: eb04cc8c-6183-4b7c-a97f-d01c2755c05b
spec:
gateways:
- mesh
- istio-egressgateway
hosts:
- my-mysql.tcp.svc
tcp:
- match:
- destinationSubnets:
- 170.187.156.142/32
gateways:
- mesh
port: 3306
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
port:
number: 3306
subset: mysql
- match:
- gateways:
- istio-egressgateway
port: 3306
route:
- destination:
host: my-mysql.tcp.svc
port:
number: 3306
weight: 100
It is working fine i am checking the ip via TCPdump in mysql server and i am getting the istio-egressgateway node ip but i dont know the approach if i want to add second mysql server ip with diffrent port and want to route that too via egressGW .

Related

Istio gateway ingress is not working in a local microk8s cluster

I am trying to make an Istio gateway (with certificates from for public access to a deployed application. Here are the configurations:
Cert manager installed in cluster via helm:
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true
Certificate issuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: kube-system
spec:
acme:
email: xxx#gmail.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Secret resource that will be used to store the account's private key.
name: letsencrypt-staging
# Add a single challenge solver, HTTP01 using istio
solvers:
- http01:
ingress:
class: istio
Certificate file:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: url-certs
namespace: istio-system
annotations:
cert-manager.io/issue-temporary-certificate: "true"
spec:
secretName: url-certs
issuerRef:
name: letsencrypt-staging
kind: ClusterIssuer
commonName: bot.demo.live
dnsNames:
- bot.demo.live
- "*.demo.live"
Gateway file:
# gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: public-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
tls:
httpsRedirect: true
- port:
number: 443
name: https-url-1
protocol: HTTPS
hosts:
- "*"
tls:
mode: SIMPLE
credentialName: "url-certs" # This should match the Certificate secretName
Application Deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: microbot
name: microbot
namespace: bot-demo
spec:
replicas: 1
selector:
matchLabels:
app: microbot
template:
metadata:
labels:
app: microbot
spec:
containers:
- name: microbot
image: dontrebootme/microbot:v1
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
Virtual service and application service:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: microbot-virtual-svc
namespace: bot-demo
spec:
hosts:
- bot.demo.live
gateways:
- istio-system/public-gateway
http:
- match:
- uri:
prefix: "/"
route:
- destination:
host: microbot-service
port:
number: 9100
---
apiVersion: v1
kind: Service
metadata:
name: microbot-service
namespace: bot-demo
spec:
selector:
app: microbot
ports:
- port: 9100
targetPort: 80
Whenever I try to curl https://bot.demo.live, I get a certificate error. The certificate issuer is working. I just can't figure out how to expose the deployed application via the istio gateway for external access. bot.demo.live is already in my /etc/hosts/ file and and I can ping it just fine.
What am I doing wrong?

AWS route53 external dns point only one A records to the (E)LB of the (nginx) ingress controller

I have created two ingresses, one for Grafana and one for my App. When the external dns write them into the route53 hosted zone as a A record, only one of them (the Myapp dns) get the (E)LB alias (dns), though the second A record get the internal IP as an ip address in the route53 A record.
the big question:
is there a way to set them all to the same alias/or at list to the same elb?
why it doesn't success doing so by default?
using:
terraform:
helm_release:
nginx-controller-bitnami
external-dns-bitnami
prometheus community
grafana-ingress:
apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: kube-prometheus-stack
meta.helm.sh/release-namespace: prometheus
generation: 1
labels:
app.kubernetes.io/instance: kube-prometheus-stack
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: grafana
app.kubernetes.io/version: 8.3.3
helm.sh/chart: grafana-6.20.5
name: kube-prometheus-stack-grafana
namespace: prometheus
resourceVersion: "2419"
spec:
rules:
- host: grafana.dns.io
http:
paths:
- backend:
service:
name: kube-prometheus-stack-grafana
port:
number: 80
path: /
pathType: Prefix
status:
loadBalancer:
ingress:
- ip: 10.0.1.19
kind: List
metadata:
resourceVersion: ""
selfLink: ""
my-app-ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
ingress.kubernetes.io/ssl-redirect: /
nginx.ingress.kubernetes.io/rewrite-target: /$1
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: "some.website.io"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: "{{ .Release.Name }}-app"
port:
number: 80
status:
loadBalancer:
ingress:
- ip: 10.0.1.19
nginx-controller-service
kind: Service
metadata:
annotations:
external-dns.alpha.kubernetes.io/hostname: website.my-app.io, some.grafana.io
meta.helm.sh/release-name: nginx-ingress-controller
meta.helm.sh/release-namespace: default
finalizers:
- service.kubernetes.io/load-balancer-cleanup
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: nginx-ingress-controller
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: nginx-ingress-controller
helm.sh/chart: nginx-ingress-controller-9.1.4
name: nginx-ingress-controller
namespace: default
resourceVersion: "997"
spec:
clusterIP: 172.30.0.140
clusterIPs:
- 172.30.0.140
externalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http
nodePort: 30337
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 31512
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: nginx-ingress-controller
app.kubernetes.io/name: nginx-ingress-controller
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- hostname: someElbDns.eu-west-1.elb.amazonaws.com
Since your ingress controller service is of type LoadBalancer, creating this service will provision an NLB for you which will have your nginx controller pods as target groups.
A request will pass from the NLB to the ingress pods and then to grafana/your app subsequently.
I would try to use the same host (website.io) and the same nginx ingress class so both ingresses get assigned the same NLB address:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: default
name: grafana-ingress
annotations:
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- grafana.website.io
secretName: tls-secret-website-io
rules:
- host: website.io
- http:
paths:
- path: /
backend:
serviceName: grafana-service
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: default
name: awebsite-ingress
annotations:
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- subdomain.website.io
secretName: tls-secret-website-io
rules:
- host: website.io
- http:
paths:
- path: /(/|$)(.*)
backend:
serviceName: website-service
servicePort: 80
kind: Service
apiVersion: v1
metadata:
name: nginx-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: nginx-controller
app.kubernetes.io/part-of: nginx-controller
annotations:
# by default the type is elb (classic load balancer).
service.beta.kubernetes.io/aws-load-balancer-type: nlb
spec:
# this setting is to make sure the source IP address is preserved.
externalTrafficPolicy: Local
type: LoadBalancer
selector:
app.kubernetes.io/name: nginx-controller
app.kubernetes.io/part-of: nginx-controller
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
Last step is to create a CNAME entry in Rote53 from your domain to the loadbalancer DNS.

How to assign an external IP to the Istio Ingress Gateway with IstioOperator? [GKE]

I want to assign an external IP to the Ingress Gateway of Istio.
I want to use the Istio Operator Spec.
So far I got this:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
name: istiocontrolplane
spec:
profile: demo
components:
ingressGateways:
- name: istio-ingressgateway
enabled: true
loadBalancerIP: 1.2.3.4
addonComponents:
grafana:
enabled: false
prometheus:
enabled: true
It's assigning an automatically IP to the Service:
kubectl get svc -n istio-system
Is not showing 1.2.3.4. for the EXTERNAL-IP
Is it only possible if I really own this IP over GCP?
First you have to create an IP resource in GCP, and then you can give that IP here in the yaml below.
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
ingressGateways:
- enabled: true
k8s:
overlays:
- api_version: autoscaling/v1
kind: HorizontalPodAutoscaler
name: istio-ingressgateway
patches:
- path: spec.minReplicas
value: 3
- path: spec.maxReplicas
value: 5
- path: spec.metrics[0].resource.targetAverageUtilization
value: 60
service:
loadBalancerIP: XXX.XXX.XXX.XXX
loadBalancerSourceRanges: []
ports:
- name: status-port
port: 15020
targetPort: 15020
- name: http2
port: 80
targetPort: 80
- name: https
port: 443
- name: tcp
port: 31400
targetPort: 31400
- name: tls
port: 15443
targetPort: 15443
label:
app: istio-ingressgateway
istio: ingressgateway
name: istio-ingressgateway

How to configure NGINX in AWS

I have set up k8s cluster in AWS ec2 instances with 1 parent and 2 child nodes using kops.
Deployed 2 services and running with LoadBalancer service type in browser.
Now I installed NGINX but through LB ip not able to hit my service. It is giving 504 GATEWAY_TIME_OUT exception. Googled it but no success, Where am I going wrong? Here is my sample code...[AWS FREE ACCOUNT]
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: $APP_NAME
name: $APP_NAME
namespace: $NAMESPACE
spec:
replicas: 1
selector:
matchLabels:
app: $APP_NAME
template:
metadata:
labels:
app: $APP_NAME
spec:
imagePullSecrets:
- name: $IMG_PULL_SECRET
containers:
- image: $IMAGE_REG/$APP_NAME:$IMAGE_TAG
name: $APP_NAME
imagePullPolicy: Always
ports:
- containerPort: ${CONTAINER_PORT}
protocol: TCP
env:
- name: spring.cloud.config.uri
value: 'http://config-server-service'
service.yml
apiVersion: v1
kind: Service
metadata:
labels:
app: $APP_NAME
name: $APP_NAME
namespace: $NAMESPACE
spec:
type: $SERVICE_TYPE
#type: $SERVICE_TYPE
ports:
- port: 80
targetPort: ${CONTAINER_PORT}
protocol: TCP
selector:
app: $APP_NAME
ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ${APP_NAME}
namespace: $NAMESPACE
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/session-cookie-name: JSESSIONID
nginx.ingress.kubernetes.io/ssl-passthrough: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx"
kubernetes.io/ingress.allow-http: "true"
# kubernetes.io/ingress.global-static-ip-name: "my-gateway"
spec:
rules:
- http:
paths:
- path: /${APP_NAME}
backend:
serviceName: ${APP_NAME}
servicePort: 80

Istio Gateway not being applied to istio-ingressgateway

I'm trying to make istio work with my mssql service. The istio-ingressgateway LoadBalancer doesn't seems to be updated with the correct port value.
I'm running on GKE on 1.10+
apiVersion: v1
kind: Service
metadata:
name: mssql
labels:
app: mssql
service: mssql
spec:
selector:
app: mssql
ports:
- protocol: TCP
port: 1433
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: public-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 1433
protocol: TCP
name: tcp-1433
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vservice-mssql
spec:
hosts:
- "*"
gateways:
- public-gateway
tcp:
- match:
- port: 1433
route:
- destination:
host: mssql
port:
number: 1433
After applying the configuration on running I expected to have the port open on the istio-ingressgateway but I'm having this as a result:
istio-ingressgateway LoadBalancer 10.8.1.100 **REDACTED** 80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30160/TCP,8060:32736/TCP,853:30641/TCP,15030:31124/TCP,15031:30849/TCP 90d
The port I opened on the gateway is not listed.
Use following resources def
apiVersion: v1
kind: Service
metadata:
name: mssql
labels:
app: mssql
service: mssql
spec:
selector:
app: mssql
ports:
- protocol: TCP
port: 1433
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: public-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 15011
protocol: TCP
name: tcp-15011
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vservice-mssql
spec:
hosts:
- "*"
gateways:
- public-gateway
tcp:
- match:
- port: 15011
route:
- destination:
host: mssql
port:
number: 1433
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: mssql
namespace: istio-system
spec:
host: mssql`enter code here`
trafficPolicy:
tls:
mode: DISABLE
Use Node Port 30160 to access mssql
15011:30160/TCP
istio-ingressgateway LoadBalancer 10.8.1.100 **REDACTED** 80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30160/TCP,8060:32736/TCP,853:30641/TCP,15030:31124/TCP,15031:30849/TCP 90d
If you installed istio using the operator, you have to manually add ports to the operator spec:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
name: istio-control-plane
spec:
values:
ingressGateways:
- enabled: true
name: istio-ingressgateway
k8s:
service:
ports:
- name: status-port
port: 15021
targetPort: 15021
- name: http2
port: 80
targetPort: 8080
- name: https
port: 443
targetPort: 8443
- name: tcp
port: 31400
targetPort: 31400
- name: tls
port: 15443
targetPort: 15443
- # ADD PORTS HERE
Those ports listed here are the default ports. I'm not sure if you remove them, if istio-operator will "merge" the default ports with the ones you will add. But you can try that out (let us know!)