Scenario:
I have 2 clusters: A and B both with istio installed. I want to expose service-1 in cluster A as service-1.suffix, and let service-2 in cluster B access service-1 by: service-1.suffix. The folloing picture illustrates my idea.
In cluster A, I define a virtualService and Gateway to route the requests to service-1.
Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: service-1
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "service-1.suffix"
VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: service-1
spec:
hosts:
- service-1.default.svc.cluster.local
- "service-1.suffix"
gateways:
- service-1
- mesh
http:
- route:
- destination:
host: service-1.default.svc.cluster.local
port:
number: 8080
This is working fine as I can use curl to access it successfully.
curl -I -HHost:service-1.suffix http://cluster_A_proxy:31380
The next step is creating Egress and VirtualService in Cluster B. Here are my definition files:
ServiceEntry:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: service-1
spec:
hosts:
- "service-1.suffix" #the global suffix mcm.com could be defined in mcm.
#addresses:
#- xxx/32
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
location: MESH_EXTERNAL
endpoints:
- address: 1.1.1.1 #The cluster A proxy ip
ports:
http: 31380
VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: service-1
spec:
hosts:
- "service-1.suffix"
http:
- route:
- destination:
host: "service-1.suffix"
port:
number: 80
In Cluster B, when I try to use curl to resolve service-1.suffix, I got a DNS error saying this cannot be resolved.
curl: (6) Could not resolve host: service-1.suffix
How can I fix this?
#The command I am using in an istio app in Cluster B:
kubectl exec -it pod_name -c container_name bash
curl -I -HHost:service-1.suffix http://service-1.suffix
Edit:
When I use another resolvable hostname like www.google.com in serviceentry I can get it through, the requests to www.google.com will be redirected to service-1 in cluster A. Just the same, if I use nip.io as my suffix, it works well. However, the made up name service-1.suffix could not be resolved.
Define a Kubernetes ExternalName service with a random IP:
kind: Service
apiVersion: v1
metadata:
name: service1
spec:
type: ExternalName
externalName: 1.1.1.1
Related
Is there a way to remove x-envoy-peer-metadata or restrict the data that goes into this header? Looks like this is default behavior at egress level and it has sensitive information related to k8s and other backend components.
(Updated, needed two files)
This is how it possible to do it via istio
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
labels:
app.kubernetes.io/part-of: my-namespace
name: my-namespace-google-remove-header
namespace: my-namespace
spec:
hosts:
- www.google.com
http:
- route:
- destination:
host: www.google.com
headers:
request:
remove:
- x-forwarded-proto
- x-envoy-decorator-operation
- x-envoy-peer-metadata-id
- x-envoy-peer-metadata
Then a ServiceEntry for the mesh is also needed
Below you can change port to 443 and protocol HTTPS if you need that instead
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: mesh-external-www-google-com
spec:
hosts:
- www.google.com
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
resolution: NONE
I have deployed an application in my minikube cluster, the application access is ok via ingress gateway service at http://172.18.97.73:31566/.
minikube ip
172.18.97.73
kubectl get svc -n istio-system
istio-ingressgateway NodePort 10.99.4.153 <none> 15021:31123/TCP,80:31566/TCP,443:32094/TCP,15012:30004/TCP,15443:30369/TCP 3h48m
but kiali graph shows as unknown the origin of the traffic.
kiali graph
i can't understand what is wrong, could you help me?
the manifest is here:
apiVersion: v1
kind: Service
metadata:
name: application
spec:
ports:
- port: 8082
selector:
app: app
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: http-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: application-vs
spec:
hosts:
- "*"
gateways:
- "default/http-gateway"
http:
- route:
- destination:
host: application.default.svc.cluster.local
port:
number: 8082
We are using istio as a service mesh to secure our cluster. We have several web applications exposed through the ingress gateway as follows ingress-gateway-id:80/app1/, ingress-gateway-id:80/app2/ and ingress-gateway-id:80/app3/.
We have a gateway that routes traffic of the ingress gateway on port 80.
For each application, we create a virtual service that routes the traffic from (for example) ingress-gateway-id:80/app1/app1-api-uri/ to app1-service/app1-api-uri/
The main issue we are currently facing is that some applications work by only / (for example) app2-service/ which forces us to allow / through the virtual service and restrict the ingress gateway to allow only one application through the ingress gateway (without specifying hosts in headers as all our applications are web apps therefore accessible through a browser in our use case).
My question is how to allow multiple applications to access / through my ingress gateway (on the same port 80 for example) without the need to deal with setting host headers from the client (in our case the browser)?
If you don't want to use your domains as a virtual service hosts I would say the only options here would be to
use rewrite in your virtual service.
use custom headers
There is an example about rewrite from istio documentation.
HTTPRewrite
HTTPRewrite can be used to rewrite specific parts of a HTTP request before forwarding the request to the destination. Rewrite primitive can be used only with HTTPRouteDestination. The following example demonstrates how to rewrite the URL prefix for api call (/ratings) to ratings service before making the actual API call.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ratings-route
spec:
hosts:
- ratings.prod.svc.cluster.local
http:
- match:
- uri:
prefix: /ratings
rewrite:
uri: /v1/bookRatings
route:
- destination:
host: ratings.prod.svc.cluster.local
subset: v1
There is an example for 2 nginx deployments, both serving on /.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx1
spec:
selector:
matchLabels:
run: nginx1
replicas: 1
template:
metadata:
labels:
run: nginx1
app: frontend
spec:
containers:
- name: nginx1
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx1 > /usr/share/nginx/html/index.html"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx2
spec:
selector:
matchLabels:
run: nginx2
replicas: 1
template:
metadata:
labels:
run: nginx2
app: frontend
spec:
containers:
- name: nginx2
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx2 > /usr/share/nginx/html/index.html"]
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: frontend
spec:
ports:
- port: 80
protocol: TCP
selector:
app: frontend
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: comp-ingress-gateway
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nginxvirt
spec:
gateways:
- comp-ingress-gateway
hosts:
- '*'
http:
- name: match
match:
- uri:
prefix: /a
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v1
port:
number: 80
- name: default
match:
- uri:
prefix: /b
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v2
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: nginxdest
spec:
host: nginx.default.svc.cluster.local
subsets:
- name: v1
labels:
run: nginx1
- name: v2
labels:
run: nginx2
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
And a test with curl.
curl -v xx.xxx.xxx.x/a
HTTP/1.1 200 OK
Hello nginx1
curl -v xx.xxx.xxx.x/b
HTTP/1.1 200 OK
Hello nginx2
There is an example about custom headers in istio documentation.
Goal
I'm trying to setup a
Cloud LB -> GKE [istio-gateway -> my-service]
This was working before, however, I have to recreate the cluster 2 days ago and run into this problem. Maybe some version change?
This is my ingress manifest file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: "my-dev-ingress"
namespace: "istio-system"
annotations:
kubernetes.io/ingress.global-static-ip-name: "my-dev-gclb-ip"
ingress.gcp.kubernetes.io/pre-shared-cert: "my-dev-cluster-cert-05"
kubernetes.io/ingress.allow-http: "false"
spec:
backend:
serviceName: "istio-ingressgateway"
servicePort: 80
Problem
The health check issue by the Cloud LB failed. The backend service created by the Ingress create a /:80 default health check.
What I have tried
1) I tried to set the health check generated by the gke ingress to point to the istio-gateway StatusPort port 15020 in the Backend config console. Then the health check passed for a bit until the backend config revert itself to use the original /:80 healthcheck that it created. I even tried to delete the healthcheck that it created and it just create another one.
2) I also tried using the istio-virtual service to route the healthcheck to 15020 port as shown here with out much success.
3) I also tried just route everything in the virtual-service the healthcheck port
hosts:
- "*"
gateways:
- my-web-gateway
http:
- match:
- method:
exact: GET
uri:
exact: /
route:
- destination:
host: istio-ingress.gke-system.svc.cluster.local
port:
number: 15020
4) Most of the search result I found say that setting readinessProbe in the deployment should tell the ingress to set the proper health check. However, all of my service are under the istio-gateway and I can't really do the same.
I'm very lost right now and will really appreciate it if anyone could point me to the right direction. Thanks
i got it working with gke 1.20.4-gke.2200 and istio 1.9.2, the documentation for this is non existent or i have not found anything, you have to add an annotation to istio-ingressgateway service to use a backend-config when using "istioctl install -f values.yaml" command
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
ingressGateways:
- name: istio-ingressgateway
enabled: true
k8s:
serviceAnnotations:
cloud.google.com/backend-config: '{"default": "istio-ingressgateway-config"}'
then you have to create the backend-config with the correct healthcheck port
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: istio-ingressgateway-config
namespace: istio-system
spec:
healthCheck:
checkIntervalSec: 30
port: 15021
type: HTTP
requestPath: /healthz/ready
with this the ingress should automatically change the configuration for the load balancer health check pointing to istio port 80
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web
namespace: istio-system
annotations:
kubernetes.io/ingress.global-static-ip-name: web
networking.gke.io/managed-certificates: "web"
spec:
rules:
- host: test.example.com
http:
paths:
- path: "/*"
pathType: Prefix
backend:
service:
name: istio-ingressgateway
port:
number: 80
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-web
namespace: istio-system
spec:
hosts:
- test.example.com
gateways:
- web
http:
- match:
- uri:
prefix: "/"
route:
- destination:
port:
number: 8080 #internal service port
host: "internal-service.service-namespace.svc.cluster.local"
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: web
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- test.example.com
you could also set hosts to "*" in the virtualservice and gateway
I am trying to expose kiali on my default gateway. I have other services working for apps in the default namespace but have not been able to route traffic to anything in the istio namespace
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- '*'
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- '*'
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: default
spec:
hosts:
- kiali.dev.example.com
gateways:
- gateway
http:
- route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
The problem was I had mTLS enabled and kiali does not have a sidecar thus can not be validated by mTLS. the solution was to add a destination rule disabling mTLS for it.
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
You should define an ingress gateway and make sure that the hosts in the gateway match the hosts in the virtual service. Also specify the port of the destination. See the Control Ingress Traffic task.
For me this worked!
I ran
istioctl proxy-config routes istio-ingressgateway-866d7949c6-68tt4 -n istio-system -o json > ./routes.json
to get the dump of all the routes. The kiali route got corrupted for some reason. I deleted the virtual service and created it again, that fixed it.
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: istio-system
spec:
gateways:
- istio-system/my-gateway
hosts:
- 'mydomain.com'
http:
- match:
- uri:
prefix: /kiali/
route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
weight: 100
---
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: SIMPLE
---
Note: hosts needed to be set, '*' didnt work for some reason.