I have a scenario where I am terminating the TLS traffic at my Istio Ingress Gateway and forwarding it to respective services. Also have mTls and ISTIO_MUTUAL set. I need to validate the client certificate that is being sent when calling our Ingress-Gateway in the cluster and I am planning to do it in code for which I need the entire cert forwarded to my service code.
I tried configuring the EnvoyFilter as below :
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: xfcc-forward
namespace: xfcc
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
sni: "*"
patch:
operation: MERGE
value:
set_current_client_cert_details:
cert: true
This fails while applying and I guess it might be because I am not configuring it right. Could someone help me with the right configuration of the EF for my scenario.
I think I got the Configuration working with this :
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: xfcc-forward
namespace: xfcc
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.http_connection_manager"
patch:
operation: MERGE
value:
typed_config:
"#type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager"
forward_client_cert_details: ALWAYS_FORWARD_ONLY
set_current_client_cert_details:
subject: true
cert: true
chain: true
I can see the XFCC header with the cert values, but the certs are the ones passed on by Citadel for mTls and not the one that the client has sent in the header, which I am still looking at.
Related
I am trying to override Istio's default retry back off interval with the help of an EnvoyFilter.
I have three services, each calling it's successor. Service-2 has retries enabled with a VirtualService.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
labels:
app: service-2
name: service-2-vs
namespace: retry-test
spec:
hosts:
- service-2
http:
- route:
- destination:
host: service-2
retries:
attempts: 5
retryOn: 5xx
The retries are working, but when I apply an EnvoyFilter to override the default retry back off interval I see no effects.
I used the following EnvoyFilter for overriding the back off intervals.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: service-2-envoy-config
namespace: retry-test
spec:
workloadSelector:
labels:
app: service-2
configPatches:
- applyTo: HTTP_ROUTE
match:
context: SIDECAR_OUTBOUND
routeConfiguration:
vhost:
name: "service-2.retry-test.svc.cluster.local:5002"
patch:
operation: MERGE
value:
route:
retry_policy:
retry_back_off:
base_interval: "1s"
max_interval: "5s"
I also tried configuring the EnvoyFilter for Service-1 since this would be the service sending requests to Service-2, but this didn't work either.
When checking with Kiali I can see that the EnvoyFilter gets applied to the correct service and when looking at Envoy configs of the workload I can see the following got applied.
"route": {
"cluster": "outbound|5002||service-2.retry-test.svc.cluster.local",
"max_grpc_timeout": "0s",
"retry_policy": {
"host_selection_retry_max_attempts": "5",
"num_retries": 5,
"retry_back_off": {
"base_interval": "1s",
"max_interval": "5s"
},
...
}
}
Can someone help me to figure out how to apply the right EnvoyFilter to override the default back off interval?
Posting this if anyone runs into the same problem and finds this question.
When using an EnvoyFilter you need to apply the filter for the service that will send the requests with the vhost being the service the requests are sent to.
In this case this would mean the Envoyfilter gets applied to service-1 with vhost of the routeConfiguration being service-2.
The according yaml file looks like this:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: service-1-envoy-config
namespace: retry-test
spec:
workloadSelector:
labels:
app: service-1
configPatches:
- applyTo: HTTP_ROUTE
match:
context: SIDECAR_OUTBOUND
routeConfiguration:
vhost:
name: "service-2.retry-test.svc.cluster.local:5002"
patch:
operation: MERGE
value:
route:
retry_policy:
retry_back_off:
base_interval: "1s"
max_interval: "5s"
I was trying to access OpenFaaS through istio in which I have included gateway and virtual service.
I need to create a separate endpoint for the OpenFaaS eg.: "http://istio_ingress_Loadbalancer/openfaas" - This should give me OpenFaaS UI.
Can anyone please help me, regarding I have hard time accessing this?
Below is the code I have written for gateway and virtual service.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: openfaas-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: openfaas-vs
spec:
hosts:
- "*"
gateways:
- openfaas-gateway.openfaas.svc.cluster.local
http:
- match:
- uri:
prefix: /openfaas
route:
- destination:
host: gateway.openfaas.svc.cluster.local
port:
number: 8080
Add the namespace property in your Gateway yaml file.
Reference the gateway in your VirtualService yaml file with the following format : <gateway-namespace>/<gateway-name>
https://istio.io/latest/docs/reference/config/networking/virtual-service/#VirtualService
LetsEncrypt not verifying via Kubernetes ingress and loadbalancer in AWS EKS
ClientIssuer
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: cert-manager
spec:
acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: my#email.com
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
Ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: echo-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
tls:
- hosts:
- echo0.site.com
secretName: echo-tls
rules:
- host: echo0.site.com
http:
paths:
- backend:
serviceName: echo0
servicePort: 80
Events
12m Normal IssuerNotReady certificaterequest/echo-tls-3171246787 Referenced issuer does not have a Ready status condition
12m Normal GeneratedKey certificate/echo-tls Generated a new private key
12m Normal Requested certificate/echo-tls Created new CertificateRequest resource "echo-tls-3171246787"
4m29s Warning ErrVerifyACMEAccount clusterissuer/letsencrypt-staging Failed to verify ACME account: context deadline exceeded
4m29s Warning ErrInitIssuer clusterissuer/letsencrypt-staging Error initializing issuer: context deadline exceeded
kubectl describe certificate
Name: echo-tls
Namespace: default
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1alpha3
Kind: Certificate
Metadata:
Creation Timestamp: 2020-04-04T23:57:22Z
Generation: 1
Owner References:
API Version: extensions/v1beta1
Block Owner Deletion: true
Controller: true
Kind: Ingress
Name: echo-ingress
UID: 1018290f-d7bc-4f7c-9590-b8924b61c111
Resource Version: 425968
Self Link: /apis/cert-manager.io/v1alpha3/namespaces/default/certificates/echo-tls
UID: 0775f965-22dc-4053-a6c2-a87b46b3967c
Spec:
Dns Names:
echo0.site.com
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-staging
Secret Name: echo-tls
Status:
Conditions:
Last Transition Time: 2020-04-04T23:57:22Z
Message: Waiting for CertificateRequest "echo-tls-3171246787" to complete
Reason: InProgress
Status: False
Type: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal GeneratedKey 18m cert-manager Generated a new private key
Normal Requested 18m cert-manager Created new CertificateRequest resource "echo-tls-3171246787"
Been going at this for a few days now. I have tried with different domains, but end up with same results. Am I missing anything here/steps. It is based off of this tutorial here
Any help would be appreciated.
Usually with golang applications the error context deadline exceeded means the connection timed out. That sounds like the cert-manager pod was not able to reach the ACME API, which can happen if your cluster has an outbound firewalls, and/or does not have a NAT or Internet Gateway attached to the subnets
This might be worthwhile to look at. I was facing similar issue.
Change LoadBalancer in ingress-nginx service.
Add/Change externalTrafficPolicy: Cluster.
Reason being, pod with the certificate-issuer wound up on a different node than the load balancer did, so it couldn’t talk to itself through the ingress.
Below is complete block taken from https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/provider/cloud-generic.yaml
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
#CHANGE/ADD THIS
externalTrafficPolicy: Cluster
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---
I am trying to add ext_authz filter to istio ingress-gateway for requests authentication. But when I'm adding this filter to cluster it seems like it is not added to envoy configuration, i.e. it is not working.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: authn-filter
namespace: istio-system
spec:
filters:
- insertPosition:
index: FIRST
listenerMatch:
portNumber: 433
listenerType: GATEWAY
listenerProtocol: HTTP
filterType: HTTP
filterName: "envoy.ext_authz"
filterConfig:
http_service:
server-uri:
uri: http://auth.default.svc.cluster.local:8080
cluster: outbound|8080||auth.default.svc.cluster.local
timeout: 2s
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: auth-virtualservice
spec:
hosts:
- "*"
gateways:
- gateway.default.svc.cluster.local
http:
- match:
- uri:
prefix: "/auth"
route:
- destination:
host: auth.default.svc.cluster.local
I figured that out, the problem was caused by the old version of istio deployed on the cluster
I am trying to expose kiali on my default gateway. I have other services working for apps in the default namespace but have not been able to route traffic to anything in the istio namespace
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- '*'
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- '*'
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: default
spec:
hosts:
- kiali.dev.example.com
gateways:
- gateway
http:
- route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
The problem was I had mTLS enabled and kiali does not have a sidecar thus can not be validated by mTLS. the solution was to add a destination rule disabling mTLS for it.
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
You should define an ingress gateway and make sure that the hosts in the gateway match the hosts in the virtual service. Also specify the port of the destination. See the Control Ingress Traffic task.
For me this worked!
I ran
istioctl proxy-config routes istio-ingressgateway-866d7949c6-68tt4 -n istio-system -o json > ./routes.json
to get the dump of all the routes. The kiali route got corrupted for some reason. I deleted the virtual service and created it again, that fixed it.
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: kiali
namespace: istio-system
spec:
gateways:
- istio-system/my-gateway
hosts:
- 'mydomain.com'
http:
- match:
- uri:
prefix: /kiali/
route:
- destination:
host: kiali.istio-system.svc.cluster.local
port:
number: 20001
weight: 100
---
apiVersion: 'networking.istio.io/v1alpha3'
kind: DestinationRule
metadata:
name: kiali
namespace: istio-system
spec:
host: kiali.istio-system.svc.cluster.local
trafficPolicy:
tls:
mode: SIMPLE
---
Note: hosts needed to be set, '*' didnt work for some reason.