I am trying to do a canary deployment in GKE. I need to enable IAP for all the deployments in this.
I can build the canary using both Istio and nginx-ingress for my usecase. But I cant figure how to enable IAP for the either of them. I provisioned a GLB (Global HTTP load balancer) and tried to add the ingresses as backends in both cases. That failed as I expected because health checks and things didnt work.
You need to have an HTTPS load balancer to be able to enable IAP. You can click on this link which provides step by step instructions on how to enable IAP within GKE. I would also highly suggest reading the section “Before you begin” as you will need the prerequisites mentioned to enable IAP.
Related
I'm facing an issue on the Service annotation that enables ALPN policy in an AWS load balancer.
I'm testing an application in production, managed by EKS. I need to enable a Network Load Balancer (NLB) on AWS to manage some ingress rules (tls cert and so on...).
Among annotations is available:
service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Preferred
I think I need this to enable ALPN in the TLS handshake.
The issue is that it does not apply to my load balancer (other annotations works), I can confirm it by accessing the AWS dashboard or by executing curl -s -vv https://my.example.com. To enable this ALPN policy I must apply this patch manually, e.g. through the dashboard.
What am I missing? I wonder if that annotation could only be available for the load balancer controller and not for the base Service for NLBs.
EDIT: I found some github issues that requested for this feature in the legacy mode without using a third party controller, here is a comment that resumes all. Since it seems to be an unavailable feature (for now), how can I achieve the configuration result using terraform for example? Do I need to create the NLB first and then attach to my Service?
I am running a cluster in EKS, with k8s 1.21.5
I know by default k8s has a Cloud Controller Manager which can be used to create Load balancers and by default it will create a classic LB in AWS.
I realize CLB are going away and I should use NLB or ALB and rather install the AWS Load Balancer controller instead but I want to work out why my annotations dont work.
What I am trying to do is setup a TLS listen using an ACM certificate because by default its all setup as TCP
Here are my annotations
# service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:<region>:<account>:certificate/<id>
# service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl
# service.beta.kubernetes.io/aws-load-balancer-ssl-ports: <port>
I have followed the k8s docs here which specify which annotations to use https://kubernetes.io/docs/concepts/services-networking/service/#ssl-support-on-aws
And I have checked in the k8s code that these annotations are present
https://github.com/kubernetes/kubernetes/blob/v1.21.5/staging/src/k8s.io/legacy-cloud-providers/aws/aws.go#L162
https://github.com/kubernetes/kubernetes/blob/v1.21.5/staging/src/k8s.io/legacy-cloud-providers/aws/aws.go#L167
https://github.com/kubernetes/kubernetes/blob/v1.21.5/staging/src/k8s.io/legacy-cloud-providers/aws/aws.go#L181
When I create my service with these annotations, the service in k8s says pending
Can anyone tell me why it wont work or give me any insight?
What I have been doing is manually configuring the LB after its created, but I want to get away from doing that
#congbaoguier
Thanks for your advice to look at the logs, I was being a complete dummy. After enabling my logging on control plane I was able to see that there was an issue with my ACM ARN and weirdly I have no idea where I got that ARN from, what I check it in ACM it was WRONG DOH
Updating my ARN it now works, so thanks for the push to use my brain again :P
I am running Kubernetes on AWS, and exposing services using a Service with type: LoadBalancer, which provisions an ELB. Is there any way to control the ELB cipher configuration with annotations on this service? I need to disable TLS 1.0 and 1.1.
I am aware that I can do this by hand, but I would like for Kubernetes to do this for me, otherwise I'll have to remember to do it again the next time a new ELB is provisioned (Kubernetes upgrade, config change, etc).
If I understood you right, you would like to adjust security policies directly from Service.yml file.
From what I see, here you can find a list of all the annotations that are supported at the moment.
There is one called "aws-load-balancer-ssl-negotiation-policy". For me it looks exactly as the one you are looking for.
// ServiceAnnotationLoadBalancerSSLNegotiationPolicy is the annotation used on
// the service to specify a SSL negotiation settings for the HTTPS/SSL listeners
// of your load balancer. Defaults to AWS's default
const ServiceAnnotationLoadBalancerSSLNegotiationPolicy = "service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy"
The link to that file is listed under official documentation on K8s.
Additionally, there is a predefined policy ELBSecurityPolicy-TLS-1-2-2017-01 that uses only TLS v1.2 ( with 1.0 and 1.1 disabled).
Hope that helps.
you can use for example annotations like:
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-1-2017-01
ALB Ingress Controller SSL Policy Annotations
Edit the Security policy on the HTTPS listener on the Load Balancer.
I am currently implementing a blue-green setup in GCE. Essentially what I'm doing is putting an nginx loadbalancer up as a backend to a GCP one, as switching the rules over to hit a different pool in nginx can be done quickly.
If there are no nginx backends available I want to error handle to a custom page held in a bucket but I can't see a clear way to do this with the standard health checks. Has anyone encountered this type of thing before?
If there are no nginx backends available I want to error handle to a
custom page held in a bucket
Google Load Balancers do not provide a feature for this requirement.
Not able to add custom path rules lo Google CDN Loadbalancer
Despite some minor issues like address flapping between custom ingress controller IP and reserved CDN IP, we are implementing CDN for our GKE hosted app following this tutorial (https://cloud.google.com/kubernetes-engine/docs/how-to/cdn-backendconfig)
Almost everything is fine, but when trying to add some path rules, via k8s manifest or Google loadbalancer UI, they take no effect at all, in fact, in the UI case, the rules disappear after 2 minutes...
Any thoughts?
Try using "kubectl replace" when dealing with ingress manifest. Google Cloud does not allow updates to ingress after it is created. So in Kubernetes it might look like you make changes but they will not get applied in Google Cloud.
Using kubectl describe, in the Events section, I found this warning:
Warning Translate 114s (x32 over 48m) loadbalancer-controller error while evaluating the ingress spec: service "xxx-staging/statics-bucket" is type "ClusterIP", expected "NodePort" or "LoadBalancer"
So, this is the problem, I will try to change this and post here the resolution.