Istio Routing - How to route default traffic to a specific service from istio gateway - istio

I am trying to implement following routing strategy in my K8S cluster.
route foo/bar/* requests to bar-service
route foo/* requests (except foo/bar/*) requests to foo-service
I am new to istio and this strategy was already implemented with nginx ingress controller using regex but could not find a way in istio gateway and virtual service.

Try with edit the virtulaservice
name: "/foo/bar/"
match:
uri:
prefix: /foo/bar/*
rewrite:
uri: /
route:
destination:
host: bar-service
port:
number: 80
name: "foo/*"
match:
uri:
prefix: /foo/*
rewrite:
uri: /
route:
destination:
host: foo-service
port:
number: 80

This can be accomplished with a VirtualService and HTTP route rules. Let's say you have App1:
apiVersion: apps/v1
kind: Deployment
metadata:
name: app1
spec:
replicas: 1
selector:
matchLabels:
app: app1
template:
metadata:
labels:
app: app1
spec:
containers:
- name: app1
image: ghcr.io/trstringer/httpbin2:0.1.0
command: ["/httpbin2"]
args:
- "--message-hostname"
- "--port"
- "8080"
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app1
spec:
selector:
app: app1
ports:
- name: http
port: 80
targetPort: 8080
And a second app, App2:
apiVersion: apps/v1
kind: Deployment
metadata:
name: app2
spec:
replicas: 1
selector:
matchLabels:
app: app2
template:
metadata:
labels:
app: app2
spec:
containers:
- name: app2
image: ghcr.io/trstringer/httpbin2:0.1.0
command: ["/httpbin2"]
args:
- "--message-hostname"
- "--port"
- "8080"
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app2
spec:
selector:
app: app2
ports:
- name: http
port: 80
targetPort: 8080
In the event that you want app2 to serve all requests to /specific-route, and all other requests to app1, you'd first have to create a Gateway:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: app-routing
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "httpbin2.com"
Now wire up a VirtualService to this Gateway, which will handle the routing:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: appvs
spec:
hosts:
- "httpbin2.com"
gateways:
- app-routing
http:
- match:
- uri:
prefix: /specific-route
route:
- destination:
host: app2
- match:
- uri:
prefix: /
route:
- destination:
host: app1
If you look at VirtualService.spec.http you'll find the list of HTTPRoutes.
Here is where you define the ordered list of routes. The first one that is matched will be used, so you have to make sure the sub route is listed first.
Now if you do the following:
$ curl -H "host: httpbin2.com" http://20.102.18.97/specific-route/more
(app2-75548c4dd9-cjdzd)
You'll see the request goes to app2. But if you do a different route that doesn't match that, it'll fall back to app1:
curl -H "host: httpbin2.com" http://20.102.18.97/more/things/here
(app1-549c548585-8cqld)

Below virtual service can route requests to different foo, bar service respectively. if further change is required on request path change the regex pattern accordingly
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: fooapplication
namespace: default
spec:
gateways:
- foo-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /foo
route:
- destination:
host: foo-service
port:
number: 8080
- match:
- uri:
regex: /foo.*
rewrite:
uri: /bar-service
route:
- destination:
host: bar-service
port:
number: 8080

Related

Service can't be registered in Target groups

I'm new to using Kubernetes and AWS so there are a lot of concepts I may not understand. I hope you can help me with this problem I am having.
I have 3 services, frontend, backend and auth each with their corresponding nodeport and an ingress that maps the one host to each service, everything is running on EKS and for the ingress deployment I am using AWS ingress controller. Once everything is deployed I try to register the node-group in the targets the frontend and auth services work correctly but backend stays in unhealthy state. I thought it could be a port problem but if you look at auth and backend they have almost the same deployment defined and both are api created with dotnet core. One thing to note is that I can do kubectl port-forward <backend-pod> 80:80 and it is running without problems. And when I run the kubectl describe ingresses command I get this:
Name: ingress
Labels: app.kubernetes.io/managed-by=Helm
Namespace: default
Address: xxxxxxxxxxxxxxxxxxxxxxxxxxx.xxxxx.elb.amazonaws.com
Ingress Class: \<none\>
Default backend: \<default\>
Rules:
Host Path Backends
----------------
domain.com
/ front-service:default-port (10.0.1.183:80,10.0.2.98:80)
back.domain.com
/ backend-service:default-port (\<none\>)
auth.domain.com
/ auth-service:default-port (10.0.1.30:80,10.0.1.33:80)
alb.ingress.kubernetes.io/listen-ports: \[{"HTTPS":443}, {"HTTP":80}\]
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/ssl-redirect: 443
kubernetes.io/ingress.class: alb
Events:
Type Reason Age From Message
-------------------------
Normal SuccessfullyReconciled 8m20s (x15 over 41h) ingress Successfully reconciled
Frontend
apiVersion: apps/v1
kind: Deployment
metadata:
name: front
labels:
name: front
spec:
replicas: 2
selector:
matchLabels:
name: front
template:
metadata:
labels:
name: front
spec:
containers:
- name: frontend
image: {{ .Values.image }}
imagePullPolicy: Always
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: wrfront-{{ .Values.namespace }}-service
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
name: default-port
protocol: TCP
selector:
name: front
---
Auth
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-wrauth-keys
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 200Gi
---
apiVersion: "v1"
kind: "ConfigMap"
metadata:
name: "auth-config-ocpm"
labels:
app: "auth"
data:
ASPNETCORE_URL: "http://+:80"
ASPNETCORE_ENVIRONMENT: "Development"
ASPNETCORE_LOGGINGCONSOLEDISABLECOLORS: "true"
---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
name: "auth"
labels:
app: "auth"
spec:
replicas: 2
strategy:
type: Recreate
selector:
matchLabels:
app: "auth"
template:
metadata:
labels:
app: "auth"
spec:
volumes:
- name: auth-keys-storage
persistentVolumeClaim:
claimName: pvc-wrauth-keys
containers:
- name: "api-auth"
image: {{ .Values.image }}
imagePullPolicy: Always
ports:
- containerPort: 80
volumeMounts:
- name: auth-keys-storage
mountPath: "/app/auth-keys"
env:
- name: "ASPNETCORE_URL"
valueFrom:
configMapKeyRef:
key: "ASPNETCORE_URL"
name: "auth-config-ocpm"
- name: "ASPNETCORE_ENVIRONMENT"
valueFrom:
configMapKeyRef:
key: "ASPNETCORE_ENVIRONMENT"
name: "auth-config-ocpm"
- name: "ASPNETCORE_LOGGINGCONSOLEDISABLECOLORS"
valueFrom:
configMapKeyRef:
key: "ASPNETCORE_LOGGINGCONSOLEDISABLECOLORS"
name: "auth-config-ocpm"
---
apiVersion: v1
kind: Service
metadata:
name: auth-service
spec:
type: NodePort
selector:
app: auth
ports:
- name: default-port
protocol: TCP
port: 80
targetPort: 80
Backend (Service with problem)
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
labels:
app: backend
spec:
replicas: 2
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: {{ .Values.image }}
imagePullPolicy: Always
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
type: NodePort
selector:
name: backend
ports:
- name: default-port
protocol: TCP
port: 80
targetPort: 80
Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
kubernetes.io/ingress.class: alb
# SSL Settings
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/certificate-arn: {{ .Values.certificate }}
spec:
rules:
- host: {{ .Values.host }}
http:
paths:
- path: /
backend:
service:
name: front-service
port:
name: default-port
pathType: Prefix
- host: back.{{ .Values.host }}
http:
paths:
- path: /
backend:
service:
name: backend-service
port:
name: default-port
pathType: Prefix
- host: auth.{{ .Values.host }}
http:
paths:
- path: /
backend:
service:
name: auth-service
port:
name: default-port
pathType: Prefix
I've tried to deploy other services and they work correctly, also running only backend or only another service, but always the same thing happens and always with the backend.
What could be happening? Is it a configuration problem? Some error in Ingress or Deployment? Or is it just the backend service?
I would be very grateful for any help.
domain.com
/ front-service:default-port (10.0.1.183:80,10.0.2.98:80)
back.domain.com
/ backend-service:default-port (\<none\>)
auth.domain.com
/ auth-service:default-port (10.0.1.30:80,10.0.1.33:80)
This one is saying that your backend service is not registered to the Ingress.
One thing to remember is that Ingress registers Services by pods' ClusterIP, like your Ingress output "10.0.1.30:80", not NodePort. And according to docs , I don't know why you can have multiple NodePort services with the same port. But when you do port-forward, you actually open that port on all your instances, I assume you have 2 instances, and then your ALB health check that port and return healthy.
But I think your issue is from your Ingress that can not locate your backend service.
My suggestions are:
Trying with only backend-service with port changed, and maybe without auth and frontend services. Default range of NodePort is 30000 - 32767
Going inside that pod or create a new pod, make a request to that service using its URL to check what it returns. By default, ALB only accept status 200 from its homepage.

AWS Ingress Controller seems to be ignoring host name rules

I am trying to deploy a frontend application to Amazon EKS. The concept is that there will be two deployments as well as two services (frontend-service and stg-frontend-service), one for production and one for staging.
On top of that, there will be an ingress ALB which will redirect traffic based on the hostname. i.e. if the hostname is www.project.io, traffic will be routed to frontend-service and if the hostname is stg-project.io, traffic will be routed to stg-frontend-service.
Here are my deployment and ingress configs
stg-frontend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: stg-frontend-deployment
namespace: project
spec:
replicas: 3
selector:
matchLabels:
app: stg-frontend
template:
metadata:
labels:
app: stg-frontend
spec:
containers:
- name: stg-frontend
image: STAGING_IMAGE
imagePullPolicy: Always
ports:
- name: web
containerPort: 3000
imagePullSecrets:
- name: project-ecr
---
apiVersion: v1
kind: Service
metadata:
name: stg-frontend-service
namespace: project
spec:
selector:
app: stg-frontend
ports:
- protocol: TCP
port: 80
targetPort: 3000
stg-prod-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend-deployment
namespace: project
spec:
replicas: 3
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
spec:
containers:
- name: frontend
image: PRODUCTION_IMAGE
imagePullPolicy: Always
ports:
- name: web
containerPort: 3000
imagePullSecrets:
- name: project-ecr
---
apiVersion: v1
kind: Service
metadata:
name: frontend-service
namespace: project
spec:
selector:
app: frontend
ports:
- protocol: TCP
port: 80
targetPort: 3000
ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: project-ingress
namespace: project
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
rules:
- host: www.project.io
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend-service
port:
number: 80
- host: stg.project.io
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: stg-frontend-service
port:
number: 80
Later, I have used Route 53 to route traffic from both domain to the ALB.
+----------------+------+---------+-----------------------------------------------------+
| Record Name | Type | Routing | Value/Route traffic to |
+----------------+------+---------+-----------------------------------------------------+
| www.project.io | A | Simple | dualstack.k8s-********.us-west-1.elb.amazonaws.com. |
| stg.project.io | A | Simple | dualstack.k8s-********.us-west-1.elb.amazonaws.com. |
+----------------+------+---------+-----------------------------------------------------+
The problem is, ALB ingress is always routing traffic to the first spec rules. In the config above, the first rule is host www.project.io which refers to frontend-service. Whenever I am trying to access www.project.io or stg.project.io, it's showing me a response from frontend-service.
Later, I have switched the rules and put staging rules first, and then it was showing staging service on both domains.
I even created a dummy record like junk.project.io and pointed to the load balancer, it still worked and showed me the same response, even though junk.project.io is not included in my ingress config.
It seems to me that Ingress Config is totally ignoring what the host name is and always returning response from the first rule.
Your host and http values are defined as separate items in the list, try removing the - (hyphen) in front of the http node:
- host: www.project.io
http: # I removed the hyphen here
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend-service
port:
number: 80
- host: stg.project.io
http: # I removed the hyphen here
paths:
- path: /
pathType: Prefix
backend:
service:
name: stg-frontend-service
port:
number: 80

ISTIO External Auth : '503 upstream connect error or disconnect/reset before headers. reset reason: connection terminationroot' when access over HTTPS

ISTIO version: 1.9.4
EKS Cluster version: 1.14
We have deployed ISTIO APP mesh in our project. We have deployed External Authorization using istio's documentation i.e. https://istio.io/latest/docs/tasks/security/authorization/authz-custom/.
External authorizer used (as mentioned in above documentation) : https://raw.githubusercontent.com/istio/istio/release-1.9/samples/extauthz/ext-authz.yaml
When we access any API from going into pod of another API (i.e. over http), using curl command, all works fine. External auth service gets call and all the headers are passed into external authorizer's v3 check method. Below information is passed
source, principal, destination, headers: authority, method, path, accept, content-length, user-agent, x-b3-sampled, x-b3-spanid, x-b3-traceid, x-envoy-attempt-count, x-ext-authz, x-forwarded-client-certx-forwarded-proto, x-request-id.
But when we try to access the same service over https using postman, browser or from going into pod of another API and using curl with https endpoint, we get denied response from external authorizer's v3 check method. Also when we check the logs of external authorizer's v3 check method no headers are passed to it in this case.
Below is setup
Name spaces with ISTIO ejection enable : foo
1. ISTIO Config map changes
data:
mesh: |-
# Add the following content to define the external authorizers.
extensionProviders:
- name: "sample-ext-authz-grpc"
envoyExtAuthzGrpc:
service: "ext-authz.foo.svc.cluster.local"
port: "9000"
- name: "sample-ext-authz-http"
envoyExtAuthzHttp:
service: "ext-authz.foo.svc.cluster.local"
port: "8000"
includeHeadersInCheck: ["x-ext-authz"]
2. External Authorizer
apiVersion: v1
kind: Service
metadata:
name: ext-authz
namespace: foo
labels:
app: ext-authz
spec:
ports:
- name: http
port: 8000
targetPort: 8000
- name: grpc
port: 9000
targetPort: 9000
selector:
app: ext-authz
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ext-authz
namespace: foo
spec:
replicas: 1
selector:
matchLabels:
app: ext-authz
template:
metadata:
labels:
app: ext-authz
spec:
containers:
- image: docker.io/istio/ext-authz:0.6
imagePullPolicy: IfNotPresent
name: ext-authz
ports:
- containerPort: 8000
- containerPort: 9000
3. Enable the external authorization Config
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
namespace: foo
spec:
selector:
matchLabels:
app: user-api
action: CUSTOM
provider:
name: sample-ext-authz-grpc
rules:
- to:
- operation:
paths: ["/user/api/*"]
4. PeerAuth Chagnes
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: mtlsauth
namespace: foo
spec:
mtls:
mode: STRICT
5. Destination Rule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: default
namespace: foo
spec:
host: "*.samplehost.svc.cluster.local"
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
6. Virtual Service File
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sample-gateway
namespace: foo
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "sample.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: user-api
namespace: foo
spec:
hosts:
- "sample.com"
gateways:
- sample-gateway
http:
- match:
- uri:
prefix: /user/api/
route:
- destination:
host: user-api
port:
number: 9500
Logs from ingress gateway:
2021-07-08T11:13:33.554104Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp 172.20.0.51:15012: connect: connection refused"
2021-07-08T11:13:35.420052Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2021-07-08T11:43:24.012961Z warning envoy config StreamAggregatedResources gRPC config stream closed: 0
I am not sure if you are facing the issue but if seems like you have enforced mtls . Thats why in the following config for gateway. You might need to open HTTPS also
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sample-gateway
namespace: foo
spec:
selector:
istio: ingressgateway
servers:
port:
number: 80
name: http
protocol: HTTP
hosts:
"sample.com"
port:
number: 443
name: https
protocol: HTTPS
hosts:
"sample.com"

hosting multiple web apps using the istio ingress gateway

We are using istio as a service mesh to secure our cluster. We have several web applications exposed through the ingress gateway as follows ingress-gateway-id:80/app1/, ingress-gateway-id:80/app2/ and ingress-gateway-id:80/app3/.
We have a gateway that routes traffic of the ingress gateway on port 80.
For each application, we create a virtual service that routes the traffic from (for example) ingress-gateway-id:80/app1/app1-api-uri/ to app1-service/app1-api-uri/
The main issue we are currently facing is that some applications work by only / (for example) app2-service/ which forces us to allow / through the virtual service and restrict the ingress gateway to allow only one application through the ingress gateway (without specifying hosts in headers as all our applications are web apps therefore accessible through a browser in our use case).
My question is how to allow multiple applications to access / through my ingress gateway (on the same port 80 for example) without the need to deal with setting host headers from the client (in our case the browser)?
If you don't want to use your domains as a virtual service hosts I would say the only options here would be to
use rewrite in your virtual service.
use custom headers
There is an example about rewrite from istio documentation.
HTTPRewrite
HTTPRewrite can be used to rewrite specific parts of a HTTP request before forwarding the request to the destination. Rewrite primitive can be used only with HTTPRouteDestination. The following example demonstrates how to rewrite the URL prefix for api call (/ratings) to ratings service before making the actual API call.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ratings-route
spec:
hosts:
- ratings.prod.svc.cluster.local
http:
- match:
- uri:
prefix: /ratings
rewrite:
uri: /v1/bookRatings
route:
- destination:
host: ratings.prod.svc.cluster.local
subset: v1
There is an example for 2 nginx deployments, both serving on /.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx1
spec:
selector:
matchLabels:
run: nginx1
replicas: 1
template:
metadata:
labels:
run: nginx1
app: frontend
spec:
containers:
- name: nginx1
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx1 > /usr/share/nginx/html/index.html"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx2
spec:
selector:
matchLabels:
run: nginx2
replicas: 1
template:
metadata:
labels:
run: nginx2
app: frontend
spec:
containers:
- name: nginx2
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx2 > /usr/share/nginx/html/index.html"]
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: frontend
spec:
ports:
- port: 80
protocol: TCP
selector:
app: frontend
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: comp-ingress-gateway
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nginxvirt
spec:
gateways:
- comp-ingress-gateway
hosts:
- '*'
http:
- name: match
match:
- uri:
prefix: /a
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v1
port:
number: 80
- name: default
match:
- uri:
prefix: /b
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v2
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: nginxdest
spec:
host: nginx.default.svc.cluster.local
subsets:
- name: v1
labels:
run: nginx1
- name: v2
labels:
run: nginx2
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
And a test with curl.
curl -v xx.xxx.xxx.x/a
HTTP/1.1 200 OK
Hello nginx1
curl -v xx.xxx.xxx.x/b
HTTP/1.1 200 OK
Hello nginx2
There is an example about custom headers in istio documentation.

istio: ingress with grpc and http

I have a service listening on two ports; one is http, the other is grpc.
I would like to set up an ingress that can route to both these port, with the same host.
The load balancer would redirect to the http port if http/1.1 is used, and to the grpc port if h2 is used.
Is there a way to do that with istio ?
I made a hello world demonstrating what I am trying to achieve :
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-world
namespace: dev
spec:
replicas: 1
template:
metadata:
annotations:
alpha.istio.io/sidecar: injected
pod.beta.kubernetes.io/init-containers: '[{"args":["-p","15001","-u","1337","-i","172.20.0.0/16"],"image":"docker.io/istio/init:0.1","imagePullPolicy":"Always","name":"init","securityContext":{"capabilities":{"add":["NET_ADMIN"]}}}]'
labels:
app: hello-world
spec:
containers:
- name: grpc-server
image: aguilbau/hello-world-grpc:latest
ports:
- name: grpc
containerPort: 50051
- name: http-server
image: nginx:1.7.9
ports:
- name: http
containerPort: 80
- name: istio-proxy
args:
- proxy
- sidecar
- -v
- "2"
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: docker.io/istio/proxy:0.1
imagePullPolicy: Always
resources: {}
securityContext:
runAsUser: 1337
---
apiVersion: v1
kind: Service
metadata:
name: hello-world
namespace: dev
spec:
ports:
- name: grpc
port: 50051
- name: http
port: 80
selector:
app: hello-world
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-world-http
namespace: dev
annotations:
kubernetes.io/ingress.class: "istio"
spec:
rules:
- host: hello-world
http:
paths:
- backend:
serviceName: hello-world
servicePort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-world-grpc
namespace: dev
annotations:
kubernetes.io/ingress.class: "istio"
spec:
rules:
- host: hello-world
http:
paths:
- backend:
serviceName: hello-world
servicePort: 50051
---
I'm a bit late to the party, but for those of you stumbling on this post, I think you can do this with very little difficulty. I'm going to assume you have istio installed on a kubernetes cluster and are happy using the default istio-ingressgateway:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-world
namespace: dev
spec:
replicas: 1
template:
metadata:
annotations:
alpha.istio.io/sidecar: injected
pod.beta.kubernetes.io/init-containers: '[{"args":["-p","15001","-u","1337","-i","172.20.0.0/16"],"image":"docker.io/istio/init:0.1","imagePullPolicy":"Always","name":"init","securityContext":{"capabilities":{"add":["NET_ADMIN"]}}}]'
labels:
app: hello-world
spec:
containers:
- name: grpc-server
image: aguilbau/hello-world-grpc:latest
ports:
- name: grpc
containerPort: 50051
- name: http-server
image: nginx:1.7.9
ports:
- name: http
containerPort: 80
- name: istio-proxy
args:
- proxy
- sidecar
- -v
- "2"
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: docker.io/istio/proxy:0.1
imagePullPolicy: Always
resources: {}
securityContext:
runAsUser: 1337
---
apiVersion: v1
kind: Service
metadata:
name: hello-world
namespace: dev
spec:
ports:
- name: grpc
port: 50051
- name: http
port: 80
selector:
app: hello-world
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: hello-world-istio-gate
namespace: dev
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
- port:
number: 50051
name: grpc
protocol: GRPC
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello-world-istio-vsvc
namespace: dev
spec:
hosts:
- "*"
gateways:
- hello-world-istio-gate
http:
- match:
- port: 80
route:
- destination:
host: hello-world
port:
number: 80
tcp:
- match:
- port: 50051
route:
- destination:
host: hello-world
port:
number: 50051
The above configuration omits your two Ingresses, and instead includes:
Your deployment
Your service
An istio gateway
An istio virtualservice
There is an important extra piece not shown, and I alluded to it earlier when talking about using the default ingressgateway. The following line found in "hello-world-istio-gateway" gives a clue:
istio: ingressgateway
This refers to a pod in the 'istio-system' namespace that is usually installed by default called 'istio-ingressgateway' - and this pod is exposed by a service also called 'istio-ingressgateway.' You will need to open up ports on the 'istio-ingressgateway' service.
As an example, I edited my (default) ingressgateway and added a port opening for HTTP and GRPC. The result is the following (edited for length) yaml code:
dampersand#kubetest1:~/k8s$ kubectl get service istio-ingressgateway -n istio-system -o yaml
apiVersion: v1
kind: Service
metadata:
<omitted for length>
labels:
app: istio-ingressgateway
chart: gateways-1.0.3
heritage: Tiller
istio: ingressgateway
release: istio
name: istio-ingressgateway
namespace: istio-system
<omitted for length>
ports:
- name: http2
nodePort: 31380
port: 80
protocol: TCP
targetPort: 80
<omitted for length>
- name: grpc
nodePort: 30000
port: 50051
protocol: TCP
targetPort: 50051
selector:
app: istio-ingressgateway
istio: ingressgateway
type: NodePort
The easiest way to make the above change (for testing purposes) is to use:
kubectl edit svc -n istio-system istio-ingressgateway
For production purposes, it's probably better to edit your helm chart or your istio.yaml file or whatever you initially used to set up the ingressgateway.
As a quick aside, note that my test cluster has istio-ingressgateway set up as a NodePort, so what the above yaml file says is that my cluster is port forwarding 31380 -> 80 and 30000 -> 50051. You may (probably) have istio-ingressgateway set up as a LoadBalancer, which will be different... so plan accordingly.
Finally, the following blog post is some REALLY excellent background reading for the tools I've outlined in this post! https://blog.jayway.com/2018/10/22/understanding-istio-ingress-gateway-in-kubernetes/
You may be able to do something like that if you move the grpc-server and http-server containers into different pods with unique labels (i.e., different versions of the service, so to speak) and then using Istio route rules, behind the Ingress, split the traffic. A rule with a match for header Upgrade: h2 could send traffic to the grpc version and a default rule would send the rest of the traffic to http one.