I am trying to expose my applications running in my kubernetes cluster through AWS load balancer.
I followed the document https://cloudyuga.guru/blog/cloud-controller-manager and got till the point where i added --cloud-provider=external in kubeadm.conf file.
But this document is based on Digitial Ocean cloud and i'm working on AWS, i'm confused if i have to run any deployment.yaml file to get the pods running which are in pending status if so please provide me the link, i'm stuck at this point.
NAME READY STATUS RESTARTS AGE
coredns-66bff467f8-dlx76 0/1 Pending 0 3m32s
coredns-66bff467f8-svb6z 0/1 Pending 0 3m32s
etcd-ip-172-31-74-144.ec2.internal 1/1 Running 0 3m38s
kube-apiserver-ip-172-31-74-144.ec2.internal 1/1 Running 0 3m38s
kube-controller-manager-ip-172-31-74-144.ec2.internal 1/1 Running 0 3m37s
kube-proxy-rh8g4 1/1 Running 0 3m32s
kube-proxy-vsvlt 1/1 Running 0 3m28s
kube-scheduler-ip-172-31-74-144.ec2.internal 1/1 Running 0 3m37s
The coredns pods are pending because you have not installed a Pod Network add-on yet. From the docs here you can choose any supported Pod Network add-on. For example to use calico
kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml
After the Pod Network add-on is installed the coredns pods should come up.
Related
I want to deploy JupyterHub on AWS' EKS service, and I am following the Zero to JupyterHub with Kubernetes guide to achieve this.
I am using the eksctl tool to deploy one cluster with a node group with one node that is represented by a t3.medium EC2 instance. After I deployed JupyterHub according to the instructions given in the guide, I get the following output when running kubectl get pods:
NAME READY STATUS RESTARTS AGE
continuous-image-puller-kl67x 1/1 Running 0 56s
hub-84b6467ff8-spjws 0/1 Pending 0 56s
proxy-79d75ddf8d-76rqm 1/1 Running 0 56s
user-scheduler-795f7d845f-7b8bn 1/1 Running 0 56s
user-scheduler-795f7d845f-mgks9 1/1 Running 0 56s
One pod, hub-84b6467ff8-spjws, remains in peding mode. kubectl describe pods outputs the following at the end:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m9s default-scheduler Successfully assigned jh/user-scheduler-795f7d845f-mgks9 to ip-192-168-20-191.eu-west-1.compute.internal
Normal Pulling 3m8s kubelet Pulling image "k8s.gcr.io/kube-scheduler:v1.23.10"
Normal Pulled 3m6s kubelet Successfully pulled image "k8s.gcr.io/kube-scheduler:v1.23.10" in 2.371033007s
Normal Created 3m3s kubelet Created container kube-scheduler
Normal Started 3m3s kubelet Started container kube-scheduler
Warning Unhealthy 3m3s kubelet Readiness probe failed: Get "https://192.168.8.94:10259/healthz": dial tcp 192.168.8.94:10259: connect: connection refused
I am having troubles understanding what "Readiness probe failed: Get "https://192.168.8.94:10259/healthz": dial tcp 192.168.8.94:10259: connect: connection refused" really means. I know there are similar questions relating to this, but so far their answers didn't work for me. I tried to have multiple nodes in the node group with nodes that have more storage, and I made sure the role have the right permission (according to the guide).
I am clearly missing something here, and I am more happy if someone could shed some light on this situation for me.
I am learning AWS EKS now and I want to know how to access etcd, kube-apiserver and other control plane components?
For example, when we run command as below in minikube, we can find etcd-minikube,kube-apiserver-minikube
[vagrant#localhost ~]$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-6955765f44-lrt6z 1/1 Running 0 176d
kube-system coredns-6955765f44-xbtc2 1/1 Running 1 176d
kube-system etcd-minikube 1/1 Running 1 176d
kube-system kube-addon-manager-minikube 1/1 Running 1 176d
kube-system kube-apiserver-minikube 1/1 Running 1 176d
kube-system kube-controller-manager-minikube 1/1 Running 1 176d
kube-system kube-proxy-69mqp 1/1 Running 1 176d
kube-system kube-scheduler-minikube 1/1 Running 1 176d
kube-system storage-provisioner 1/1 Running 2 176d
And then, we can access them by below command:
[vagrant#localhost ~]$ kubectl exec -it -n kube-system kube-apiserver-minikube -- /bin/sh
# kube-apiserver
W0715 13:56:17.176154 21 services.go:37] No CIDR for service cluster IPs specified.
...
My question: I want to do something like the above example in AWS EKS, but I cannot find kube-apiserver
xiaojie#ubuntu:~/environment/calico_resources$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system aws-node-flv95 1/1 Running 0 23h
kube-system aws-node-kpkv9 1/1 Running 0 23h
kube-system aws-node-rxztq 1/1 Running 0 23h
kube-system coredns-cdd78ff87-bjnmg 1/1 Running 0 23h
kube-system coredns-cdd78ff87-f7rl4 1/1 Running 0 23h
kube-system kube-proxy-5wv5m 1/1 Running 0 23h
kube-system kube-proxy-6846w 1/1 Running 0 23h
kube-system kube-proxy-9rbk4 1/1 Running 0 23h
AWS EKS is a managed kubernetes offering. Kubernetes control plane components such as API Server, ETCD are installed, managed and upgraded by AWS. Hence you can neither see these components nor can exec into these components.
In AWS EKS you can only play with the worker nodes
You are at the left ... AWS is at the right
EKS is not a managed service for the whole kubernetes cluster.
EKS is a managed service only for Kubernetes Master nodes.
That's why, it's worth to operate EKS with tools (.e.g; terraform) that helps provisioning the whole cluster in no time .. as explained here.
As what Arghya Sadhu and Abdennour TOUMI said, EKS Encapsulates most Control Plane Components but kube-proxy, See here.
Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to stand up or maintain your own Kubernetes control plane.
So, I have tried to find the way to configure these Components instead of accessing these container and input command, but finally I give up. See this Github issue.
As i went through few resources on the internet i managed to create classic load balancer by setting the flags in my kube.apiserver, kubelet.service, kubecontroller manager and created a cluster and deployed a sample nginx file and it has exposed the application but i see that it has created classic load balancer and what i wanted is an application load balancer to be created am i supposed to do anymore changes? and also when i deploy a kibana helm chart i do get an load balancer external ip but when i access it i don't see any page.
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-client-5df74c974d-dp6xw 1/1 Running 0 5h52m
pod/elasticsearch-data-0 1/1 Running 0 5h52m
pod/elasticsearch-master-0 1/1 Running 0 5h52m
pod/fluent-bit-h9kgm 1/1 Running 0 5h52m
pod/kibana-b9d8dc6d5-cbj8j 1/1 Running 0 7s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-client ClusterIP 10.100.13.46 <none> 9200/TCP 5h52m
service/elasticsearch-discovery ClusterIP None <none> 9300/TCP 5h52m
service/kibana LoadBalancer 10.100.14.245 adaec083b81644ecbb87d4d2ba0dc070-693460825.us-east-1.elb.amazonaws.com 443:32734/TCP 7s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/fluent-bit 1 1 1 1 1 <none> 5h52m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/elasticsearch-client 1/1 1 1 5h52m
deployment.apps/kibana 1/1 1 1 7s
NAME DESIRED CURRENT READY AGE
replicaset.apps/elasticsearch-client-5df74c974d 1 1 1 5h52m
replicaset.apps/kibana-b9d8dc6d5 1 1 1 7s
NAME READY AGE
statefulset.apps/elasticsearch-data 1/1 5h52m
statefulset.apps/elasticsearch-master 1/1 5h52m
As you can see above i'm able to get a LoadBalancer <externalIP> but i don't see anything when i open that link.
And also my requirement was to deploy an Application Load Balancer and after i would deploy an Ingress helm chart and in the ingress resources i would specify the paths and ports.
From the docs as of now only ELB and NLB is supported loadbalancer type for AWS.
Edit:
Using LoadBalancer type service you can have single NLB/ELB for the nginx ingress controller and use it for as many ingress resource as you want to route traffic to backend cluster IP type service.
But if you wan to use ALB you have to manually create it (following AWS docs) and configure it to forward traffic to your Kubernetes nodes NodePort where nginx ingress controller is running. Creating a LoadBalancer type service will not work in this case. You will have to create a NodePort service for the nginx ingress controller.
I have configured a cluster in EKS but I am not able to deploy a web service into EKS. I tried the steps mentioned in https://github.com/spjenk/HelloSpringEKS but I failed to create tiller-deploy.
C:\WINDOWS\system32>kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system tiller-deploy-5d6cc99fc-hn6dm 0/1 ContainerCreating 0 6h45m
The tiller-deploy just stuck in ContainerCreating status.
Any help would be appreciated.
Thanks.
I am looking for these logs:
/var/log/kube-apiserver.log
/var/log/kube-scheduler.log
/var/log/kube-controller-manager.log
In EKS user does not have access to the control plane and can't see these files directly.
I am aware of CloudTrail integration announced by AWS. But it shows events not from k8s API, but AWS EKS API like CreateCluster event. Also the open question how to get scheduler and controller manager logs.
There is no pods for api and controller in pods list.
$ kubectl get po --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system aws-node-9f4lm 1/1 Running 0 2h
kube-system aws-node-wj2cg 1/1 Running 0 2h
kube-system kube-dns-64b69465b4-4gw6n 3/3 Running 0 2h
kube-system kube-proxy-7mt7l 1/1 Running 0 2h
kube-system kube-proxy-vflzv 1/1 Running 0 2h
There is no master nodes in the node list
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-0-92.ec2.internal Ready <none> 9m v1.10.3
ip-10-0-1-63.ec2.internal Ready <none> 9m v1.10.3
Logs can be send to CloudWatch (not free of charge). The following logs can be individually selected to be send to CloudWatch:
API server
Audit
Authenticator
Controller Manager
Scheduler
Logs can be enabled via UI or AWS CLI. See Amazon EKS Control Plane Logging
Things like kube-api server logs, the kube-scheduler logs, the kube-controller manager logs, etc. will be available in CloudWatch logs. While (as you have stated) EKS API calls will be logged to cloudtrail.
**I take that back, I guess AWS EKS has not gotten around to that yet. You will need to use an EFK stack to get the logs.
Someone has already put together a quick how-to:
https://github.com/aws-samples/aws-workshop-for-kubernetes/tree/master/02-path-working-with-clusters/204-cluster-logging-with-EFK