Kubernetes forbidden: User "system:anonymous" cannot get path "/" - amazon-web-services

Question

I'm struggling to expose my app over the Internet when deployed to AWS EKS.
I have created a deployment and a service, I can see both of these running when using kubectl. I can see that the app has successfully connected to an external database as it runs a script on startup that initialises said database.
My issue is arising when trying to access the app over the internet. I have tried accessing the cluster endpoint and I am getting this error:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User "system:anonymous" cannot get path "/"",
"reason": "Forbidden",
"details": {
},
"code": 403
}
However, if I access the "/readyz" path I get "ok" returned.
"/version" returns the following:
{
"major": "1",
"minor": "16+",
"gitVersion": "v1.16.8-eks-e16311",
"gitCommit": "e163110a04dcb2f39c3325af96d019b4925419eb",
"gitTreeState": "clean",
"buildDate": "2020-03-27T22:37:12Z",
"goVersion": "go1.13.8",
"compiler": "gc",
"platform": "linux/amd64"
}
My deployment.yml file contains the following:
apiVersion: apps/v1
kind: Deployment
metadata:
name: client
labels:
app: client
spec:
replicas: 1
selector:
matchLabels:
app: client
template:
metadata:
labels:
app: client
spec:
containers:
- name: client
image: image/repo
ports:
- containerPort: 80
imagePullPolicy: Always
My service.yml:
apiVersion: v1
kind: Service
metadata:
name: client
labels:
run: client
spec:
type: LoadBalancer
ports:
- name: "80"
port: 80
targetPort: 80
protocol: TCP
selector:
run: client
I can see the Load Balancer has been created in the AWS console and I have tried updating the security group of the LB to be able to talk to the cluster endpoint. The LB dashboard is showing the one attached instance is 'OutOfService' and also under the monitoring tab, I can see one Unhealthy Host.
I've tried accessing the Load Balancer endpoint as provided in the EC2 area of the console (this matches what is returned from kubectl get services as the EXTERNAL-IP of the LB service) and I'm getting an empty response from there.
curl XXXXXXX.eu-west-2.elb.amazonaws.com:80
curl: (52) Empty reply from server
This is the same when accessing in a web browser.
I seem to be going round in circles with this one any help at all would be greatly appreciated.

Answer 1

I've tried accessing the Load Balancer endpoint
You are accessing the EKS URL, which is the kubernetes apiserver endpoint, and not the LoadBalancer that was (hopefully) created for your client Service
You will want to kubectl get -o wide svc client and if it was successful in provisioning a LoadBalancer for you, then its URL will appear in the output. You can get more details about that situation by kubectl describe svc client, which will include any events that affected it during provisioning

Answer 2

Because your eks instance is OutOfService in the LoadBalancer section, you should check which port the LoadBalancer is doing Health Check on.
You can do that by executing kubectl get svc client -oyaml and seeing the nodePort section.
After that, check that your LoadBalancer is doing the Health Check to this exact port, if not than change it to the correct one.
If you have the correct port and but the instance is still OutOfService then i suggest you go to the security group of your eks instance and give the specific port access from the ELB.

Answer 3

I never got to the bottom of the issue here. I started again and used A pre-made Helm chart for the software I was trying to deploy and it worked.