I have a Kubernetes service with externalIP which points to an RDS instance
The service name is rds
We are using pods with hostname as rds.default.svc.cluster.local to connect to the database using a DNS redirect using the above service. It works fine without SSL but for SSL to work it also needs the Host Name, which in this case is not visible and the certificate check fails.
Is there a way to get the ExternalName from the endpoint rds.default.svc.cluster.local. we know that it works if we directly provide the hostname = but this makes it a lot difficult to manage for each service running in Kubernetes.
Service calling this host is using sequelize to establish a TLS connection
YAML for ExternalIP service
apiVersion: v1
kind: Service
metadata:
name: rds
namespace: default
spec:
externalName: <endpoint>.rds.amazonaws.com
ports:
- name: rds-aurora
port: 5432
protocol: TCP
targetPort: 5432
type: ExternalName
Error
[queryReportMessage]: Error occured: SequelizeConnectionError: Hostname/IP does not match certificate's altnames: Host: rds.default.svc.cluster.local. is not in the cert's altnames: DNS:*.<URL>.rds.amazonaws.com
Related
I have installed nginx ingress controller of type NLB inside EKS cluster and it is of type internal.
The ingress controller created a network load balancer, with listeners 80 and 443,
with port 443 we can't attach an ssl cert for nlb type, only when I use listener type tls it is able to allow us to add ssl cert from AWS ACM.
Now the issue is, I am trying to expose a frontend application through this NLB nginx ingress controller,
when the NLB lister port is 443, it is able to access the application but complains with ssl cert (fake Kubernetes cert), when I change the listener from 443 to tls in NLB, it throws error "400 "The plain HTTP request was sent to HTTPS port" error"
Like many solutions out there mentioning changing the targetPort from https: https to https: http , I tried but with that too same error "The page isn't working,ERR_TOOMANY_REQUESTS"
Could anyone help me how to resolve this issue?
Any ideas or suggestions would be highly appreciated
To resolve the issue with the SSL certificate and the "400 "The plain HTTP request was sent to HTTPS port" error", you may need to modify your ingress configuration to specify that the ingress should listen for HTTPS traffic on port 443. This can be done by adding the following annotations to your ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
name: example
namespace: example
spec:
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example
port:
name: https
tls:
- hosts:
- example.com
secretName: example-tls
In the example above, nginx.ingress.kubernetes.io/ssl-redirect tells the ingress to redirect HTTP traffic to HTTPS. nginx.ingress.kubernetes.io/secure-backends tells the ingress to encrypt the traffic between the ingress and the backend services. `secret
I'm using EKS and latest Istio installed via Helm. I'm trying to implement TLS based on a wildcard cert we have for our domain in AWS certificate manager. I'm running into a problem where the connection between the client and the NLB works, with TLS being terminated there, but the NLB can't talk to the istio LB over the secure port. In the AWS console I can rewrite the forwarding rules to forward traffic from port 443 to the standard istio http target, but I can't find a way to do this via code. I'm trying to avoid all click-ops. Here is my Helm overrides for the gateway:
gateways:
istio-ingressgateway:
serviceAnnotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:XXXXXXXXXXXXXXXXXX:certificate/XXXXXXXXXXXXXXXXXXXX"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
So what I'm expecting to occur here is:
Client:443 --> NLB:443 --> Istio Gateway:80
but what I end up with is
Client:443 --> NLB:443 --> Istio Gateway:443
Does anyone have any thoughts on how to get this to work via code? Alternately if someone can point me to a resource to get tls communication between the NLB and Istio working I'm good with that too.
Probably, what is happening is that if you terminate TLS on the load balancer it won't carry SNI to the target group. I had the exact same issue and I ended up solving it by setting the host as '*' on the ingress Gateway and then specifying the hosts on the different VirtualServices (as recommended here and also on istio's official docs).
Your service annotation already correct what is mising is to change istio gateway port 443 to HTTP
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: http-gateway-external
namespace: istio-ingress
spec:
selector:
istio: gateway-external
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
- hosts:
- '*'
port:
name: https
number: 443
protocol: HTTP # Change from HTTPS to HTTP
I have a domain (example.com) already configured in Cloud DNS. With this domain I can access microservices that are in a GKE cluster. I use istio-ingressgateway IP in CloudDNS to make the association between the cluster
Now I have another domain (newexample.com) with a custom certificate for https connections. Is there a way to redirect all the requests to newexample.com to example.com? I do not want to change anything in gke/istio configuration if possible.
Each method will require some of the reconfiguration in either GKE/Istio side.
One of the solutions to this is to have a CNAME record in a Cloud DNS and a SSL certificate with Alternative Names.
With above solution you will be able to send requests to your GKE/Istio cluster with both domain names assuming correct Istio configuration.
What is CNAME?
CNAME is a Canonical Name Record or Alias Record.
A type of resource record in the Domain Name System (DNS), that specifies that one domain name is an alias of another canonical domain name.
Example of a CNAME record:
DNS name Type TTL Data
old.domain. A 60 1.2.3.4
new.domain. CNAME 60 old.domain.
Alternative Names:
A SAN or subject alternative name is a structured way to indicate all of the domain names and IP addresses that are secured by the certificate.
Enstrustdatacard.com: What is a san and how is it used
You can create SSL cerificate create to support both:
old.domain
new.domain
There are plenty options to do that for example Let's Encrypt or Cert Manager.
Example
I've created an example to show you how to do it:
Configure DNS zone in Cloud DNS
Create a basic app with a service
Create a certificate for example app
Create Istio resources to allow connections to example app
Test
Configure DNS zone in Cloud DNS
You will need to have 2 records:
A record with IP of your Ingress Gateway and name: old.domain
CNAME record pointing to old.domain with name: new.domain
Please take a look on official documentation: Cloud.google.com: DNS: Records
Create a basic app with a service
Below is example app with a service which will respond with a basic hello:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-dp
spec:
selector:
matchLabels:
app: hello-dp
replicas: 1
template:
metadata:
labels:
app: hello-dp
spec:
containers:
- name: hello
image: "gcr.io/google-samples/hello-app:2.0"
env:
- name: "PORT"
value: "50001"
---
apiVersion: v1
kind: Service
metadata:
name: hello-sv
spec:
selector:
app: hello-dp
ports:
- name: hello-port
protocol: TCP
port: 50001
targetPort: 50001
type: ClusterIP
Create a certificate for example app
As said previously, certificate with Alternative Names can be created with Let's Encrypt. I created it with:
GCE VM with Ubuntu 16.04
Open port 80
Domain name old.domain pointing to public ip address of a VM
Guide: Linode.com: Docs: Install let's encrypt to create a SSL certificate
Command to create certificate:
$ ./letsencrypt-auto certonly --standalone -dold.domain-dnew.domain
Certificate created in /etc/letsencrypt/archive/ used in creating tls secret for GKE with command:
$ kubectl create secret tlsssl-certificate--cert cert1.pem --key privkey1.pem
Please have in mind that this certificate was created only for testing purposes and I strongly advise using dedicated solution like: Cert-manager
PS: If you used this method please revert back changes in the Cloud DNS to point the Istio gateway.
Create Istio resources to allow connections to example app
Below are example Istio resources allowing connections to example app with support for HTTPS:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: hello-gw
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: ssl-certificate
hosts:
- "old.domain"
- "new.domain"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello-vs
spec:
hosts:
- "old.domain"
- "new.domain"
gateways:
- hello-gw
http:
- route:
- destination:
host: hello-sv
port:
number: 50001
Please take a specific look on:
tls:
mode: SIMPLE
credentialName: ssl-certificate
This part will ensure that connection to the cluster will use HTTPS
Additionally:
hosts:
- "old.domain"
- "new.domain"
Above definition in both resources will allow only connections with specified domains.
Test
When applied all of the above resources you should be able to enter in your browser:
https://old.domain
https://new.domain
and be greeted with below message and valid SSL certificate:
Hello, world!
Version: 2.0.0
Hostname: hello-dp-5dd8b85b56-bk7zr
I deployed kong ingress controller on aws eks cluster with fargate option.
I am unable to access out application over the internet over http port.
I am keep getting -ERR_CONNECTION_TIMED_OUT in browser.
I did follow the Kong deployment as per steps given at -
https://github.com/Kong/kubernetes-ingress-controller/blob/master/docs/deployment/eks.md
Kong-proxy service is created wihtout issue.
kong-proxy service is created yet its “EXTERNAL-IP” is still showing pending.
We are able to access our local application in internal network (by logging on to running pod) via Kong-proxy CLUSTER-IP without any problem using curl.
A nlb load balancer is also created automatically in aws console when we created kong-proxy service. Its DNS name we are using to try to connect from internet.
Kindly help me understand what could be the problem.
My kong-proxy yaml is-
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-type: nlb
name: kong-proxy
namespace: kong
spec:
externalTrafficPolicy: Local
ports:
- name: proxy
port: 80
protocol: TCP
targetPort: 80
- name: proxy-ssl
port: 443
protocol: TCP
targetPort: 443
selector:
app: ingress-kong
type: LoadBalancer
I don't think it's supported now as per https://github.com/aws/containers-roadmap/issues/617
I have a cluster created using eksctl and also valid certificates created under ACM, I have used DNS method to verify the domain ownership and its succesfully completed.
below are the details i see when executing kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
eks-learning-ingress my-host.com b29c58-production-91306872.us-east-1.elb.amazonaws.com 80 18h
when i access the application using https://b29c58-production-91306872.us-east-1.elb.amazonaws.com, i see it load the application with a security warning because that not the domain name with which the certifcates are created. When i try to execute https://my-host.com i am getting a timeout.
I have 2 questions
1) I am using CNAMES to point my domain to AWS ELB, the values i added for CNAME are
name: my-host.com, points to: b29c58-production-91306872.us-east-1.elb.amazonaws.com. Is this correct?
2) below is my ingress resource defination, may be i am missing something as requests are not coming in to the application
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: eks-learning-ingress
namespace: production
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/certificate-arn: arn:dseast-1:255982529496:sda7-a148-2d878ef678df
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}, {"HTTP": 8080}, {"HTTPS": 8443}]'
labels:
app: eks-learning-ingress
spec:
rules:
- host: my-host.com
http:
paths:
- path: /*
backend:
serviceName: eks-learning-service
servicePort: 80
Any help would be really great. Thanks.
My go-to solution is using an A-record in Route 53. Instead of adding an IP, you select the "alias" option and select your load balancer. Amazon will handle the rest.
I think you have a port mismatch. https:// will use port 443, not port 80, but your ingress appears to be accepting requests on port 80 rather than 443.
If 443 was configured I'd expect to see it listed under ports as 80, 443
Can you verify with telnet or nc or curl -vvvv that your ingress is actually accepting requests on port 443? If it is, check the response body reported by curl - it should give you some indication as to why the request is not propagating downwards to your service.
We use nginx-ingress so unfortunately I can't look at local ingress config and compare it to yours.