I am using istio in knative. Since the client ip did not come, I activated the reverse-prox. I added envoyfilter. but I get "upstream connect error or disconnect/reset before headers. reset reason: connection termination" error in services on knative.
if it's not knative service I can access it.
virtualservice
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: test-entry-route
namespace: default
spec:
gateways:
- knative-serving/knative-ingress-gateway
hosts:
- "test.net"
http:
- match:
- uri:
prefix: /test/ #working
rewrite:
uri: /
route:
- destination:
host: user-api.test.svc.cluster.local
port:
number: 80
- match:
- uri:
prefix: /user/ #not working
route:
- destination:
host: istio-ingressgateway.istio-system.svc.cluster.local
port:
number: 80
weight: 100
rewrite:
authority: user-api.poker-test.k8s.test.net
uri: /
EnvoyFilter
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: proxy-protocol
namespace: istio-system
spec:
configPatches:
- applyTo: LISTENER
patch:
operation: MERGE
value:
listener_filters:
- name: envoy.listener.proxy_protocol
- name: envoy.listener.tls_inspector
workloadSelector:
labels:
istio: ingressgateway
service load balancer
annotations:
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: 'true'
externalTrafficPolicy: Local
Related
I am trying to implement following routing strategy in my K8S cluster.
route foo/bar/* requests to bar-service
route foo/* requests (except foo/bar/*) requests to foo-service
I am new to istio and this strategy was already implemented with nginx ingress controller using regex but could not find a way in istio gateway and virtual service.
Try with edit the virtulaservice
name: "/foo/bar/"
match:
uri:
prefix: /foo/bar/*
rewrite:
uri: /
route:
destination:
host: bar-service
port:
number: 80
name: "foo/*"
match:
uri:
prefix: /foo/*
rewrite:
uri: /
route:
destination:
host: foo-service
port:
number: 80
This can be accomplished with a VirtualService and HTTP route rules. Let's say you have App1:
apiVersion: apps/v1
kind: Deployment
metadata:
name: app1
spec:
replicas: 1
selector:
matchLabels:
app: app1
template:
metadata:
labels:
app: app1
spec:
containers:
- name: app1
image: ghcr.io/trstringer/httpbin2:0.1.0
command: ["/httpbin2"]
args:
- "--message-hostname"
- "--port"
- "8080"
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app1
spec:
selector:
app: app1
ports:
- name: http
port: 80
targetPort: 8080
And a second app, App2:
apiVersion: apps/v1
kind: Deployment
metadata:
name: app2
spec:
replicas: 1
selector:
matchLabels:
app: app2
template:
metadata:
labels:
app: app2
spec:
containers:
- name: app2
image: ghcr.io/trstringer/httpbin2:0.1.0
command: ["/httpbin2"]
args:
- "--message-hostname"
- "--port"
- "8080"
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app2
spec:
selector:
app: app2
ports:
- name: http
port: 80
targetPort: 8080
In the event that you want app2 to serve all requests to /specific-route, and all other requests to app1, you'd first have to create a Gateway:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: app-routing
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "httpbin2.com"
Now wire up a VirtualService to this Gateway, which will handle the routing:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: appvs
spec:
hosts:
- "httpbin2.com"
gateways:
- app-routing
http:
- match:
- uri:
prefix: /specific-route
route:
- destination:
host: app2
- match:
- uri:
prefix: /
route:
- destination:
host: app1
If you look at VirtualService.spec.http you'll find the list of HTTPRoutes.
Here is where you define the ordered list of routes. The first one that is matched will be used, so you have to make sure the sub route is listed first.
Now if you do the following:
$ curl -H "host: httpbin2.com" http://20.102.18.97/specific-route/more
(app2-75548c4dd9-cjdzd)
You'll see the request goes to app2. But if you do a different route that doesn't match that, it'll fall back to app1:
curl -H "host: httpbin2.com" http://20.102.18.97/more/things/here
(app1-549c548585-8cqld)
Below virtual service can route requests to different foo, bar service respectively. if further change is required on request path change the regex pattern accordingly
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: fooapplication
namespace: default
spec:
gateways:
- foo-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /foo
route:
- destination:
host: foo-service
port:
number: 8080
- match:
- uri:
regex: /foo.*
rewrite:
uri: /bar-service
route:
- destination:
host: bar-service
port:
number: 8080
I wanted to setup and use istio egress gateway.
I followed this link https://preliminary.istio.io/latest/blog/2018/egress-tcp/ and made this manifest:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-oracle
spec:
hosts:
- my.oracle.instance.com
addresses:
- 192.168.100.50/32
ports:
- name: tcp
number: 1521
protocol: tcp
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- hosts:
- my.oracle.instance.com
port:
name: tcp
number: 1521
protocol: TCP
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: egressgateway-destination-rule-for-oracle
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: external-oracle
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-external-oracle-through-egress-gateway
spec:
gateways:
- mesh
- istio-egressgateway
hosts:
- my.oracle.instance.com
tcp:
- match:
- destinationSubnets:
- 192.168.100.50/32
gateways:
- mesh
port: 1521
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
port:
number: 1521
subset: external-oracle
- match:
- gateways:
- istio-egressgateway
port: 1521
route:
- destination:
host: my.oracle.instance.com
port:
number: 1521
weight: 100
And then my application not able to start because a JDBC error.
I started to watch the egress-gateway pod's logs but I not see any sign of traffic.
So I googled and found this link: https://istio.io/latest/blog/2018/egress-monitoring-access-control/ to boost my egress-gateway pod logging but this looking a bit deprecated for me.
cat <<EOF | kubectl apply -f -
# Log entry for egress access
apiVersion: "config.istio.io/v1alpha2"
kind: logentry
metadata:
name: egress-access
namespace: istio-system
spec:
severity: '"info"'
timestamp: request.time
variables:
destination: request.host | "unknown"
path: request.path | "unknown"
responseCode: response.code | 0
responseSize: response.size | 0
reporterUID: context.reporter.uid | "unknown"
sourcePrincipal: source.principal | "unknown"
monitored_resource_type: '"UNSPECIFIED"'
---
# Handler for error egress access entries
apiVersion: "config.istio.io/v1alpha2"
kind: stdio
metadata:
name: egress-error-logger
namespace: istio-system
spec:
severity_levels:
info: 2 # output log level as error
outputAsJson: true
---
# Rule to handle access to *.cnn.com/politics
apiVersion: "config.istio.io/v1alpha2"
kind: rule
metadata:
name: handle-politics
namespace: istio-system
spec:
match: request.host.endsWith("cnn.com") && request.path.startsWith("/politics") && context.reporter.uid.startsWith("kubernetes://istio-egressgateway")
actions:
- handler: egress-error-logger.stdio
instances:
- egress-access.logentry
---
# Handler for info egress access entries
apiVersion: "config.istio.io/v1alpha2"
kind: stdio
metadata:
name: egress-access-logger
namespace: istio-system
spec:
severity_levels:
info: 0 # output log level as info
outputAsJson: true
---
# Rule to handle access to *.com
apiVersion: "config.istio.io/v1alpha2"
kind: rule
metadata:
name: handle-cnn-access
namespace: istio-system
spec:
match: request.host.endsWith(".com") && context.reporter.uid.startsWith("kubernetes://istio-egressgateway")
actions:
- handler: egress-access-logger.stdio
instances:
- egress-access.logentry
EOF
But when I want to apply to this I have this error:
no matches for kind "logentry" in version "config.istio.io/v1alpha2"
no matches for kind "stdio" in version "config.istio.io/v1alpha2"
no matches for kind "rule" in version "config.istio.io/v1alpha2"
no matches for kind "stdio" in version "config.istio.io/v1alpha2"
no matches for kind "rule" in version "config.istio.io/v1alpha2"
There is a new api version of there kind's?
istioctl version
client version: 1.12.0
control plane version: 1.12.0
data plane version: 1.12.0 (28 proxies)
There is a way to make a working istio egress-gateway with a logging (as the istio ingress gateway logging works).
Is there a way to remove x-envoy-peer-metadata or restrict the data that goes into this header? Looks like this is default behavior at egress level and it has sensitive information related to k8s and other backend components.
(Updated, needed two files)
This is how it possible to do it via istio
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
labels:
app.kubernetes.io/part-of: my-namespace
name: my-namespace-google-remove-header
namespace: my-namespace
spec:
hosts:
- www.google.com
http:
- route:
- destination:
host: www.google.com
headers:
request:
remove:
- x-forwarded-proto
- x-envoy-decorator-operation
- x-envoy-peer-metadata-id
- x-envoy-peer-metadata
Then a ServiceEntry for the mesh is also needed
Below you can change port to 443 and protocol HTTPS if you need that instead
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: mesh-external-www-google-com
spec:
hosts:
- www.google.com
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
resolution: NONE
Following this doc I got istio-ingressgateway running but using curl to test the URL I am facing this problem:
curl: (7) Failed to connect to httpbin.example.com port 31390: Connection refused
This is the Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: mygateway
spec:
selector:
istio: ingressgateway # use istio default ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: httpbin-credential # must be the same as secret
hosts:
- httpbin.example.com
and the virtual service:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- "httpbin.example.com"
gateways:
- mygateway
http:
- match:
- uri:
prefix: /status
- uri:
prefix: /delay
route:
- destination:
port:
number: 8000
host: httpbin
I am using istio 1.5.4.
This is the command that result in timeout:
curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
I am trying to expose kiali on my default gateway. I have other services working for apps in the default namespace but have not been able to route traffic to anything in the istio namespace
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- '*'
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- '*'
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: default
spec:
hosts:
- kiali.dev.example.com
gateways:
- gateway
http:
- route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
The problem was I had mTLS enabled and kiali does not have a sidecar thus can not be validated by mTLS. the solution was to add a destination rule disabling mTLS for it.
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
You should define an ingress gateway and make sure that the hosts in the gateway match the hosts in the virtual service. Also specify the port of the destination. See the Control Ingress Traffic task.
For me this worked!
I ran
istioctl proxy-config routes istio-ingressgateway-866d7949c6-68tt4 -n istio-system -o json > ./routes.json
to get the dump of all the routes. The kiali route got corrupted for some reason. I deleted the virtual service and created it again, that fixed it.
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: istio-system
spec:
gateways:
- istio-system/my-gateway
hosts:
- 'mydomain.com'
http:
- match:
- uri:
prefix: /kiali/
route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
weight: 100
---
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: SIMPLE
---
Note: hosts needed to be set, '*' didnt work for some reason.