How to set TLS to a service in EKS with PCA on AWS? - amazon-web-services

Question

I created a TLS-enabled service with AWS PCA and cert-manager by this post:
https://aws.amazon.com/blogs/security/tls-enabled-kubernetes-clusters-with-acm-private-ca-and-amazon-eks-2/
After I deployed a demo application with ingress, I tested access on control node
$ curl https://demo.my-org.com --cacert cacert.pem
Got message
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
The cacert.pem was downloaded from AWS PCA's Certificate body. Things look good in K8s for AWSPCAClusterIssuer and Certificate. The certificate description got these events:
$ kubectl describe certificate rsa-cert-2048 -n acm-pca-lab-demo
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Issuing 47m cert-manager Existing issued Secret is not up to date for spec: [spec.commonName spec.dnsNames]
Normal Reused 47m cert-manager Reusing private key stored in existing Secret resource "rsa-example-cert-2048"
Normal Requested 47m cert-manager Created new CertificateRequest resource "rsa-cert-2048-pp4c4"
Normal Issuing 47m cert-manager The certificate has been successfully issued
If I access from browser got 502 error. The certificate page shown a fake certificate and an alt DNS name.
I'm sure the private CA in AWS was actived successfully. Its arn and region were been set to the EKS node policy and AWSPCAClusterIssuer. What's wrong about the settings? How to diagnose the issue?
deployed resources
I checked the deployed resources in acm-pca-lab-demo namespace.
$ kubectl get secret -n acm-pca-lab-demo
NAME TYPE DATA AGE
default-token-jmxt7 kubernetes.io/service-account-token 3 10h
rsa-example-cert-2048 kubernetes.io/tls 3 10h
$ kubectl get all -n acm-pca-lab-demo
NAME READY STATUS RESTARTS AGE
pod/hello-world-57df4c69f9-nnjrl 1/1 Running 0 10h
pod/hello-world-57df4c69f9-r8f4p 1/1 Running 0 10h
pod/hello-world-57df4c69f9-xgm6w 1/1 Running 0 10h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-world ClusterIP 102.30.45.163 <none> 80/TCP 10h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/hello-world 3/3 3 3 10h
NAME DESIRED CURRENT READY AGE
replicaset.apps/hello-world-57df4c69f9 3 3 3 10h
$ kubectl get ingress -n acm-pca-lab-demo
NAME CLASS HOSTS ADDRESS PORTS AGE
acm-pca-demo-ingress <none> demo.my-org.com 11111111111111111111111111111111-2222222222222222.elb.us-east-1.amazonaws.com 80, 443 10h
On the browser, I also got these messages:
The certificate is not trusted because it is self-signed.
HTTP Strict Transport Security: false
HTTP Public Key Pinning: false
certificate file
I downloaded the PCA .pem file from AWS console here. Is it correct?
It's -----BEGIN CERTIFICATE----- started file.

Answer 1

Check your ingress configuration, share the YAML config-if possible which you have used with application deployment.
there could be chances there is not secret attached to ingress, due to that K8s Nginx ingress controller by default attaching the default FAKE cert instead of your generated cert.
For example :
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: acm-pca-demo-ingress
namespace: acm-pca-lab-demo
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- www.rsa-2048.example.com
secretName: rsa-example-cert-2048
rules:
- host: www.rsa-2048.example.com
http:
paths:
- path: /
pathType: Exact
backend:
service:
name: hello-world
port:
number: 80
as shown above rsa-example-cert-2048, make sure your secret exists in the namespace in which ingress there.