If the virtual service is used with the mesh gateway then retryRemoteLocalities and attempts are not working. Is it by design or I am missing something?
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: myapp
spec:
hosts:
- myapp.default.svc.cluster.local
gateways:
- mesh
http:
- name: myapp
route:
- destination:
host: myapp
port:
number: 8000
retries:
attempts: 5
retryOn: 500,502,retriable-status-codes,connect-failure,reset,gateway-error,refused-stream
retryRemoteLocalities: true
Istio 1.14.1
Kubernetes: 1.24.3
Related
I have an Istio gateway, Istio VirtualService, and app deployed running as a service. The virtual service is deployed like so:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-app-route
spec:
hosts:
- "*"
gateways:
- my-gateway
http:
- match:
- uri:
prefix: "/my-app"
route:
- destination:
host: my-app-service
port:
number: 8000
When I navigate to http://myurl.com/my-app I would expect to see a JSON response from the service it is pointing to, but I just get a 404. However if I remove the match and redeploy like so:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-app-route
spec:
hosts:
- "*"
http:
- route:
- destination:
host: my-app-service
port:
number: 800
And then go to http://myurl.com/my-app I can see the expected JSON response. Any help?
ISTIO version: 1.9.4
EKS Cluster version: 1.14
We have deployed ISTIO APP mesh in our project. We have deployed External Authorization using istio's documentation i.e. https://istio.io/latest/docs/tasks/security/authorization/authz-custom/.
External authorizer used (as mentioned in above documentation) : https://raw.githubusercontent.com/istio/istio/release-1.9/samples/extauthz/ext-authz.yaml
When we access any API from going into pod of another API (i.e. over http), using curl command, all works fine. External auth service gets call and all the headers are passed into external authorizer's v3 check method. Below information is passed
source, principal, destination, headers: authority, method, path, accept, content-length, user-agent, x-b3-sampled, x-b3-spanid, x-b3-traceid, x-envoy-attempt-count, x-ext-authz, x-forwarded-client-certx-forwarded-proto, x-request-id.
But when we try to access the same service over https using postman, browser or from going into pod of another API and using curl with https endpoint, we get denied response from external authorizer's v3 check method. Also when we check the logs of external authorizer's v3 check method no headers are passed to it in this case.
Below is setup
Name spaces with ISTIO ejection enable : foo
1. ISTIO Config map changes
data:
mesh: |-
# Add the following content to define the external authorizers.
extensionProviders:
- name: "sample-ext-authz-grpc"
envoyExtAuthzGrpc:
service: "ext-authz.foo.svc.cluster.local"
port: "9000"
- name: "sample-ext-authz-http"
envoyExtAuthzHttp:
service: "ext-authz.foo.svc.cluster.local"
port: "8000"
includeHeadersInCheck: ["x-ext-authz"]
2. External Authorizer
apiVersion: v1
kind: Service
metadata:
name: ext-authz
namespace: foo
labels:
app: ext-authz
spec:
ports:
- name: http
port: 8000
targetPort: 8000
- name: grpc
port: 9000
targetPort: 9000
selector:
app: ext-authz
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ext-authz
namespace: foo
spec:
replicas: 1
selector:
matchLabels:
app: ext-authz
template:
metadata:
labels:
app: ext-authz
spec:
containers:
- image: docker.io/istio/ext-authz:0.6
imagePullPolicy: IfNotPresent
name: ext-authz
ports:
- containerPort: 8000
- containerPort: 9000
3. Enable the external authorization Config
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
namespace: foo
spec:
selector:
matchLabels:
app: user-api
action: CUSTOM
provider:
name: sample-ext-authz-grpc
rules:
- to:
- operation:
paths: ["/user/api/*"]
4. PeerAuth Chagnes
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: mtlsauth
namespace: foo
spec:
mtls:
mode: STRICT
5. Destination Rule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: default
namespace: foo
spec:
host: "*.samplehost.svc.cluster.local"
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
6. Virtual Service File
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sample-gateway
namespace: foo
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "sample.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: user-api
namespace: foo
spec:
hosts:
- "sample.com"
gateways:
- sample-gateway
http:
- match:
- uri:
prefix: /user/api/
route:
- destination:
host: user-api
port:
number: 9500
Logs from ingress gateway:
2021-07-08T11:13:33.554104Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp 172.20.0.51:15012: connect: connection refused"
2021-07-08T11:13:35.420052Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
2021-07-08T11:43:24.012961Z warning envoy config StreamAggregatedResources gRPC config stream closed: 0
I am not sure if you are facing the issue but if seems like you have enforced mtls . Thats why in the following config for gateway. You might need to open HTTPS also
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sample-gateway
namespace: foo
spec:
selector:
istio: ingressgateway
servers:
port:
number: 80
name: http
protocol: HTTP
hosts:
"sample.com"
port:
number: 443
name: https
protocol: HTTPS
hosts:
"sample.com"
We are using istio as a service mesh to secure our cluster. We have several web applications exposed through the ingress gateway as follows ingress-gateway-id:80/app1/, ingress-gateway-id:80/app2/ and ingress-gateway-id:80/app3/.
We have a gateway that routes traffic of the ingress gateway on port 80.
For each application, we create a virtual service that routes the traffic from (for example) ingress-gateway-id:80/app1/app1-api-uri/ to app1-service/app1-api-uri/
The main issue we are currently facing is that some applications work by only / (for example) app2-service/ which forces us to allow / through the virtual service and restrict the ingress gateway to allow only one application through the ingress gateway (without specifying hosts in headers as all our applications are web apps therefore accessible through a browser in our use case).
My question is how to allow multiple applications to access / through my ingress gateway (on the same port 80 for example) without the need to deal with setting host headers from the client (in our case the browser)?
If you don't want to use your domains as a virtual service hosts I would say the only options here would be to
use rewrite in your virtual service.
use custom headers
There is an example about rewrite from istio documentation.
HTTPRewrite
HTTPRewrite can be used to rewrite specific parts of a HTTP request before forwarding the request to the destination. Rewrite primitive can be used only with HTTPRouteDestination. The following example demonstrates how to rewrite the URL prefix for api call (/ratings) to ratings service before making the actual API call.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ratings-route
spec:
hosts:
- ratings.prod.svc.cluster.local
http:
- match:
- uri:
prefix: /ratings
rewrite:
uri: /v1/bookRatings
route:
- destination:
host: ratings.prod.svc.cluster.local
subset: v1
There is an example for 2 nginx deployments, both serving on /.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx1
spec:
selector:
matchLabels:
run: nginx1
replicas: 1
template:
metadata:
labels:
run: nginx1
app: frontend
spec:
containers:
- name: nginx1
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx1 > /usr/share/nginx/html/index.html"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx2
spec:
selector:
matchLabels:
run: nginx2
replicas: 1
template:
metadata:
labels:
run: nginx2
app: frontend
spec:
containers:
- name: nginx2
image: nginx
ports:
- containerPort: 80
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello nginx2 > /usr/share/nginx/html/index.html"]
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: frontend
spec:
ports:
- port: 80
protocol: TCP
selector:
app: frontend
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: comp-ingress-gateway
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nginxvirt
spec:
gateways:
- comp-ingress-gateway
hosts:
- '*'
http:
- name: match
match:
- uri:
prefix: /a
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v1
port:
number: 80
- name: default
match:
- uri:
prefix: /b
rewrite:
uri: /
route:
- destination:
host: nginx.default.svc.cluster.local
subset: v2
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: nginxdest
spec:
host: nginx.default.svc.cluster.local
subsets:
- name: v1
labels:
run: nginx1
- name: v2
labels:
run: nginx2
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
And a test with curl.
curl -v xx.xxx.xxx.x/a
HTTP/1.1 200 OK
Hello nginx1
curl -v xx.xxx.xxx.x/b
HTTP/1.1 200 OK
Hello nginx2
There is an example about custom headers in istio documentation.
I am trying to expose kiali on my default gateway. I have other services working for apps in the default namespace but have not been able to route traffic to anything in the istio namespace
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- '*'
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- '*'
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: default
spec:
hosts:
- kiali.dev.example.com
gateways:
- gateway
http:
- route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
The problem was I had mTLS enabled and kiali does not have a sidecar thus can not be validated by mTLS. the solution was to add a destination rule disabling mTLS for it.
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
You should define an ingress gateway and make sure that the hosts in the gateway match the hosts in the virtual service. Also specify the port of the destination. See the Control Ingress Traffic task.
For me this worked!
I ran
istioctl proxy-config routes istio-ingressgateway-866d7949c6-68tt4 -n istio-system -o json > ./routes.json
to get the dump of all the routes. The kiali route got corrupted for some reason. I deleted the virtual service and created it again, that fixed it.
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: istio-system
spec:
gateways:
- istio-system/my-gateway
hosts:
- 'mydomain.com'
http:
- match:
- uri:
prefix: /kiali/
route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
weight: 100
---
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: SIMPLE
---
Note: hosts needed to be set, '*' didnt work for some reason.
Scenario:
I have 2 clusters: A and B both with istio installed. I want to expose service-1 in cluster A as service-1.suffix, and let service-2 in cluster B access service-1 by: service-1.suffix. The folloing picture illustrates my idea.
In cluster A, I define a virtualService and Gateway to route the requests to service-1.
Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: service-1
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "service-1.suffix"
VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: service-1
spec:
hosts:
- service-1.default.svc.cluster.local
- "service-1.suffix"
gateways:
- service-1
- mesh
http:
- route:
- destination:
host: service-1.default.svc.cluster.local
port:
number: 8080
This is working fine as I can use curl to access it successfully.
curl -I -HHost:service-1.suffix http://cluster_A_proxy:31380
The next step is creating Egress and VirtualService in Cluster B. Here are my definition files:
ServiceEntry:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: service-1
spec:
hosts:
- "service-1.suffix" #the global suffix mcm.com could be defined in mcm.
#addresses:
#- xxx/32
ports:
- number: 80
name: http
protocol: HTTP
resolution: STATIC
location: MESH_EXTERNAL
endpoints:
- address: 1.1.1.1 #The cluster A proxy ip
ports:
http: 31380
VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: service-1
spec:
hosts:
- "service-1.suffix"
http:
- route:
- destination:
host: "service-1.suffix"
port:
number: 80
In Cluster B, when I try to use curl to resolve service-1.suffix, I got a DNS error saying this cannot be resolved.
curl: (6) Could not resolve host: service-1.suffix
How can I fix this?
#The command I am using in an istio app in Cluster B:
kubectl exec -it pod_name -c container_name bash
curl -I -HHost:service-1.suffix http://service-1.suffix
Edit:
When I use another resolvable hostname like www.google.com in serviceentry I can get it through, the requests to www.google.com will be redirected to service-1 in cluster A. Just the same, if I use nip.io as my suffix, it works well. However, the made up name service-1.suffix could not be resolved.
Define a Kubernetes ExternalName service with a random IP:
kind: Service
apiVersion: v1
metadata:
name: service1
spec:
type: ExternalName
externalName: 1.1.1.1