nginx ingress redirect to https based on header - amazon-web-services

I have a EKS setup where traffic is sent in the following way.
Users -> Cloudfront -> ALB -> EKS. EKS has an NginX ingress controller.
Currently "force-ssl-redirect" is enabled and hence, NginX ingress controller redirects all HTTP traffic to HTTPS.
I want Cloudfront to connect with ALB using HTTP. Hence, I am looking to have a conditional HTTPS redirect in NginX controller.
Hence,
I will set a custom header to requests in from Cloudfront
If this new header is found, I want NginX controller to return the correct response in HTTP. If not, I want to redirect to HTTPS.
How can this be done?

I assume that you would like to add a custom header on Cloudfront, and then do a redirect on the Nginx side.
You can add custom headers to the request via Cloudfront Functions - https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html
Then I think you need to use the nginx.ingress.kubernetes.io/configuration-snippet annotation in the Ingress Kubernetes resource to add custom configuration to the Nginx location. Maybe something like this can work:
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: redirect
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($http_x_custom_header) {
return 301 http://$host$request_uri;
}
return 301 https://$host$request_uri;
spec:
rules:
- host: ...

You can use the NginX configuration map for conditional HTTPS redirection.
You need to create Nginx Configuration with a custom snippet for checking the custom header set by CloudFront.
apiVersion: v1
Kind: ConfigMap
Metadata:
Name: nginx-config
Data:
Ssl-redirect-snippet: |
If ($http_cf_custom_header) {
Return 301 http://$server_name$request_uri;
}
Return 301 https://$server_name$request_uri;
If header is present, NginX returns a 301 redirect with HTTP; if it is not present it will redirect to same URL using HTTPS
Now you need to add configuration map to your NginX ingress controller
apiVersion: networking.k8s.io/v1
Kind: Ingress
Metadata:
Name: my-ingress
Annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
Include /etc/nginx/ssl-redirect-snippet;
Spec:
Rules:
-host: my.domain.com
Http:
Paths:
-Path: /
pathType: Prefix
Backend:
Service:
Name: my-service
Port:
Name: http
You need to set CLoudFront to include ‘http_cf_custom_header’ when you are forwarding to ALB. Check this official page and for further information check AWS official documentation.

Related

Nginx ingress controller of type internal nlb giving 400 "The plain HTTP request was sent to HTTPS port" error

I have installed nginx ingress controller of type NLB inside EKS cluster and it is of type internal.
The ingress controller created a network load balancer, with listeners 80 and 443,
with port 443 we can't attach an ssl cert for nlb type, only when I use listener type tls it is able to allow us to add ssl cert from AWS ACM.
Now the issue is, I am trying to expose a frontend application through this NLB nginx ingress controller,
when the NLB lister port is 443, it is able to access the application but complains with ssl cert (fake Kubernetes cert), when I change the listener from 443 to tls in NLB, it throws error "400 "The plain HTTP request was sent to HTTPS port" error"
Like many solutions out there mentioning changing the targetPort from https: https to https: http , I tried but with that too same error "The page isn't working,ERR_TOOMANY_REQUESTS"
Could anyone help me how to resolve this issue?
Any ideas or suggestions would be highly appreciated
To resolve the issue with the SSL certificate and the "400 "The plain HTTP request was sent to HTTPS port" error", you may need to modify your ingress configuration to specify that the ingress should listen for HTTPS traffic on port 443. This can be done by adding the following annotations to your ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
name: example
namespace: example
spec:
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example
port:
name: https
tls:
- hosts:
- example.com
secretName: example-tls
In the example above, nginx.ingress.kubernetes.io/ssl-redirect tells the ingress to redirect HTTP traffic to HTTPS. nginx.ingress.kubernetes.io/secure-backends tells the ingress to encrypt the traffic between the ingress and the backend services. `secret

AWS Ingress for socket application

I have multiple microservices deployed on aws eks. one the microservices has external http access configured with my ingress file as below:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cc-ingress
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: microservice-name
port:
number: 3001
I want to deploy another microservice which interacts with socket protocol and in my local environment I call the service with postman like below:
ws://localhost:3001/some-route
so I need to deploy this microservice in aws and provide external access for it. I would be appreciated if anyone could help me.
Thanks for any comments or guides.
web socket works on top of https protocol, by adding an upgrade:connection header to the request. So, you don't have to rewrite the ingress specifically for web sockets. Just make sure you have https ports opened and the app is listening to it.

Setting HTTPS with Traefik2 on AWS EKS with ACM

I have an AWS EKS cluster with Traefik2 deployed via helm with the following config. The ACM cert set is a wildcard cert *.example.com
service:
enabled: true
type: LoadBalancer
annotations: {
service.beta.kubernetes.io/aws-load-balancer-internal: "true",
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443",
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http",
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-east-1:xxxxxxxxx:certificate/"
With the following IngressRoute set for the dashboard.
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: dashboard
namespace: traefik
spec:
entryPoints:
- web
- websecure
routes:
- match: Host(`traefik.example.com`)
kind: Rule
services:
- name: api#internal
kind: TraefikService
The issue is after pointing the domain in R53 to the Traefik CLB I can only hit the dashboard with HTTP access and not HTTPS. When trying to access via HTTPS I am receiving the "404 page not found" error. The goal is to eventually just have HTTP redirect to HTTPS but unable to hit HTTPS in the first place.
Could there me something in the configuration that I am missing?

How to remove or modify header from istio ingress gateway

Chrome browser redirects all my domain and subdomain requests to HTTPS, this is unwanted behavior in my case.
according to https://www.chromium.org/hsts, this is HSTS policy that been added to chrome browser to the domain and all subdomains.
I am using Istio version 1.7.4 and noticed that the Istio ingress gateway add the header strict-transport-security that causes this issue.
strict-transport-security: max-age=15552000; includeSubDomains
how can I remove this header from the ingress gateway?
You can use VirtualService to add or remove certain headers.
The example from the official Istio documentation shows the way how you can remove it:
Headers
Message headers can be manipulated when Envoy forwards requests to, or responses from, a destination service. Header manipulation rules can be specified for a specific route destination or for all destinations. The following VirtualService adds a test header with the value true to requests that are routed to any reviews service destination. It also removes the foo response header, but only from responses coming from the v1 subset (version) of the reviews service.
v1alpha3
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews-route
spec:
hosts:
- reviews.prod.svc.cluster.local
http:
- headers:
request:
set:
test: true
route:
- destination:
host: reviews.prod.svc.cluster.local
subset: v2
weight: 25
- destination:
host: reviews.prod.svc.cluster.local
subset: v1
headers:
response:
remove:
- foo # <-- HERE!
weight: 75
Istio.io: Latest: Docs: Reference: Config: Networking: Virtual service: Headers
Additional resources:
Istio.io: Latest: Docs

Exposing a service on EKS using NGINX ingress and issues with load balancer

I am trying to set up a service and expose it externally on EKS. I have already done it on GKE pretty easily but now AWS is giving me a hard time.
My NGINX yaml looks something like that:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- app.mydomain.com
secretName: myapp-tls
rules:
- host: app.mydomain.com
http:
paths:
- path: /
backend:
serviceName: myapp-service
servicePort: 80
And then I have my domain app.mydomain.com on Google Domains pointing at the ingress external address. There is also a cert-manager service running in order to support HTTPS.
However, while basically the same setup worked completely out of the box on GKE, EKS gives me a hard time.
From what I understand it has something to do with EKS default LoadBalancer being layer 4 in comparison to Google's layer 7 (Which explains HTTPS not working) but there is also issues with redirections of the domain as it just resolves as the ingress address instead of my desired address and thus my app doesn't show up.
The domain is registered over Google Domains and I'm creating Synthetic Records (for my subdomain) that points to my ingress external address on EKS. The same scheme works perfectly fine on GKE but here it resolves the address as the ingress address instead of my domain which results in 404 on the ingress side.
I was wondering if someone could please point me to how to properly set it up? Should I give up on nginx ingress on EKS and move onto ALB? and how to properly associate the domain?
Thank you very much in advance!
Edit:
output of kubectl describe ingress myapp-ingress:
Name: myapp-ingress
Namespace: default
Address: ********************************-****************.elb.eu-west-1.amazonaws.com
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
myapp-tls terminates app.mydomain.com
Rules:
Host Path Backends
---- ---- --------
app.mydomain.com
/ myapp-service:80 (172.31.2.238:8000)
Annotations: cert-manager.io/cluster-issuer: myapp-letsencrypt-prod
kubernetes.io/ingress.class: nginx
Events: <none>
Should I give up on nginx ingress on EKS and move onto ALB
No. NGinX ingress controllers work perfectly well on EKS. It is possible to configure them as either layer 4 or layer 7; we use it in layer 7 mode.
Can you update your question with the output of
kubectl get ingress myapp-ingress
I think your ingress path is also incorrect. Unless I'm mistaken that's just routing the root of your app, not all uris. We use the scheme
spec:
rules:
- host: service.d.tld
http:
paths:
- path: /?(.*) # <---
backend:
serviceName: my-service
servicePort: http
Are you seeing errors in the nginx ingress controller's logs? That + kubectl events are both useful for debugging purposes.
I'd disable TLS everywhere and get your service working on http, then work stepwise on getting TLS enabled on the ingress controller.
Edit: Based on your response above,
curl -H "Host: app.mydomain.com" http://<elb-address>:80
SHOULD call through to your service behind the ingress.
How is app.mydomain.com defined? Is it a CNAME to the dns entry?