While I'm trying to get the pods or node states, from Google Cloud Platform Cloud Shell, I'm facing this error? Can someone please help me? I can see the output of the "kubectl config view".
Posting this answer as community wiki for better visibility and the fact that the possible solution was posted in the comments:
Does this answer your question? Unable to connect to the server: dial tcp i/o time out
Adding to that:
Below command:
$ kubectl config view
is used to show the configuration stored in your ./kube/config file. The fact that you can see the output of this command doesn't mean you have correct cluster configured to use with kubectl.
From the perspective of Google Cloud Platform and Cloud Shell
There is an official documentation regarding troubleshooting issues with GKE:
Cloud.google.com: Kubernetes Engine: Docs: Troubleshooting
There could be several reasons why you are getting following error:
You are referencing wrong cluster in your ~/.kube/config file.
$ gcloud container clusters get-credentials CLUSTER_NAME --zone=ZONE - you will need to run this command to fetch the correct configuration
You can also get above command from the Kubernetes Engine page (Connect button)
You are referencing a cluster in your ~/.kube/config file that was deleted
You created Private GKE cluster
For more information you can look in the Cloud Console -> Kubernetes Engine -> CLUSTER_NAME
You can also run:
$ gcloud container clusters list - this command will show clusters and their state (status) they are in
$ gcloud container clusters describe CLUSTER_NAME --zone=ZONE - this command will show you the configuration of the cluster
Related
I want to execute a script which is in my compute engine using cloudbuild but somehow cloudbuild is not able to ssh into my vm , in my vm "OS LOGIN" is enabled and also have only internal ip.
here is my cloudbuild.yaml file
steps:
name: 'gcr.io/cloud-builders/gcloud' id: Update staging server entrypoint: /bin/sh args:
'-c'
|
set -x &&
gcloud compute ssh vm_name --zone=us-central1-c --command='/bin/sh /pullscripts/pull.sh'
I am attaching my error pics
cloudbuild error page 1
cloudbuild error page 2
Also my question is , is it possible connect a vm using cloud sdk if "os login" is enabled.
You'll probably have to add the roles/iap.tunnelResourceAccessor role to the cloudbuild service account. Please read this Google documentation, which shows you what to do with a certain error code.
Error code 4033
Either you don't have permission to access the instance, the instance doesn't exist, or the instance is stopped.
in fact, you can use gcloudbuild to connect in any vm, just need a docker configuration and upload the files (private_key, scripts, etc). I've this repo to solve this problem: https://github.com/jmbl1685/gcloudbuild-vm-ssh-connect
I hope that the above help you
Try adding --internal-ip which looks like as follows:
gcloud compute ssh vm_name --zone=us-central1-c --internal-ip
I've updated gcloud to the latest version (159.0.0)
I created a Google Container Engine node, and then followed the instructions in the prompt.
gcloud container clusters get-credentials prod --zone us-west1-b --project myproject
Fetching cluster endpoint and auth data.
kubeconfig entry generated for prod
kubectl proxy
Unable to connect to the server: error executing access token command
"/Users/me/Code/google-cloud-sdk/bin/gcloud ": exit status
Any idea why is it not able to connect?
You can try to run to see if the config was generated correctly:
kubectl config view
I had a similar issue when trying to run kubectl commands on a new Kubernetes cluster just created on Google Cloud Platform.
The solution for my case was to activate Google Application Default Credentials.
You can find a link below on how to activate it.
Basically, you need to set an environmental variable to the path of the .json with the credentials from GCP
GOOGLE_APPLICATION_CREDENTIALS -> c:\...\..\..Credentials.json exported from Google Cloud
https://developers.google.com/identity/protocols/application-default-credentials
I found this solution on a kuberenetes github issue: https://github.com/kubernetes/kubernetes/issues/30617
PS: make sure you have also set the environmental variables for:
%HOME% to %USERPROFILE%
%KUBECONFIG% to %USERPROFILE%
It looks like the default auth plugin for GKE might be buggy on windows. kubectl is trying to run gcloud to get a token to authenticate to your cluster. If you run kubectl config view you can see the command it tried to run, and run it yourself to see if/why it fails.
As Alexandru said, a workaround is to use Google Application Default Credentials. Actually, gcloud container has built in support for doing this, which you can toggle by setting a property:
gcloud config set container/use_application_default_credentials true
or set environment variable
%CLOUDSDK_CONTAINER_USE_APPLICATION_DEFAULT_CREDENTIALS% to true.
Using GKE, update the credentials from the "Kubernetes Engine/Cluster" management worked for me.
The cluster line provides "Connect" button that copy the credentials commands into console. And this refresh the used token. And then kubectl works again.
Why my token expired? well, i suppose GCP token are not eternal.
So, the button plays the same command automatically that :
gcloud container clusters get-credentials your-cluster ...
Bruno
I've been struggling with configuring Kubernetes for many hours and I don't know how to move it forward.
What I did :
I created few services using spring cloud
I created docker images for each service
I pushed those images to docker hub
I launched AWS by running
export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash
Command kubectl cluster-info shows that it actually works.
I created Kubernetes pods for each service. Command kubectl get pods
shows that all pods have status running.
The problem is that when I log to my AWS account I don't see any running instance, although I can see kubernetes-staging created in my S3 bucket.
My goal is to actually access my service , not on localhost. How can I do it ?
You should be able to see instances of course - as #kichik mentioned check whether your AWS console is using the same region as the deployment scripts.
To use your services/applications the next step is to expose them to the public with Kubernetes services as described here and here
What I am trying to do:
I have setup kubernete cluster using documentation available on Kubernetes website (http_kubernetes.io/v1.1/docs/getting-started-guides/aws.html). Using kube-up.sh, i was able to bring kubernete cluster up with 1 master and 3 minions (as highlighted in blue rectangle in the diagram below). From the documentation as far as i know we can add minions as and when required, So from my point of view k8s master instance is single point of failure when it comes to high availability.
Kubernetes Master HA on AWS
So I am trying to setup HA k8s master layer with the three master nodes as shown above in the diagram. For accomplishing this I am following kubernetes high availability cluster guide, http_kubernetes.io/v1.1/docs/admin/high-availability.html#establishing-a-redundant-reliable-data-storage-layer
What I have done:
Setup k8s cluster using kube-up.sh and provider aws (master1 and minion1, minion2, and minion3)
Setup two fresh master instance’s (master2 and master3)
I then started configuring etcd cluster on master1, master 2 and master 3 by following below mentioned link:
http_kubernetes.io/v1.1/docs/admin/high-availability.html#establishing-a-redundant-reliable-data-storage-layer
So in short i have copied etcd.yaml from the kubernetes website (http_kubernetes.io/v1.1/docs/admin/high-availability/etcd.yaml) and updated Node_IP, Node_Name and Discovery Token on all the three nodes as shown below.
NODE_NAME NODE_IP DISCOVERY_TOKEN
Master1
172.20.3.150 https_discovery.etcd.io/5d84f4e97f6e47b07bf81be243805bed
Master2
172.20.3.200 https_discovery.etcd.io/5d84f4e97f6e47b07bf81be243805bed
Master3
172.20.3.250 https_discovery.etcd.io/5d84f4e97f6e47b07bf81be243805bed
And on running etcdctl member list on all the three nodes, I am getting:
$ docker exec <container-id> etcdctl member list
ce2a822cea30bfca: name=default peerURLs=http_localhost:2380,http_localhost:7001 clientURLs=http_127.0.0.1:4001
As per documentation we need to keep etcd.yaml in /etc/kubernete/manifest, this directory already contains etcd.manifest and etcd-event.manifest files. For testing I modified etcd.manifest file with etcd parameters.
After making above changes I forcefully terminated docker container, container was existing after few seconds and I was getting below mentioned error on running kubectl get nodes:
error: couldn't read version from server: Get httplocalhost:8080/api: dial tcp 127.0.0.1:8080: connection refused
So please kindly suggest how can I setup k8s master highly available setup on AWS.
To configure an HA master, you should follow the High Availability Kubernetes Cluster document, in particular making sure you have replicated storage across failure domains and a load balancer in front of your replicated apiservers.
Setting up HA controllers for kubernetes is not trivial and I can't provide all the details here but I'll outline what was successful for me.
Use kube-aws to set up a single-controller cluster: https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html. This will create CloudFormation stack templates and cloud-config templates that you can use as a starting point.
Go the AWS CloudFormation Management Console, click the "Template" tab and copy out the complete stack configuration. Alternatively, use $ kube-aws up --export to generate the cloudformation stack file.
User the userdata cloud-config templates generated by kube-aws and replace the variables with actual values. This guide will help you determine what those values should be: https://coreos.com/kubernetes/docs/latest/getting-started.html. In my case I ended up with four cloud-configs:
cloud-config-controller-0
cloud-config-controller-1
cloud-config-controller-2
cloud-config-worker
Validate your new cloud-configs here: https://coreos.com/validate/
Insert your cloud-configs into the CloudFormation stack config. First compress and encode your cloud config:
$ gzip -k cloud-config-controller-0
$ cat cloud-config-controller-0.gz | base64 > cloud-config-controller-0.enc
Now copy the content into your encoded cloud-config into the CloudFormation config. Look for the UserData key for the appropriate InstanceController. (I added additional InstanceController objects for the additional controllers.)
Update the stack at the AWS CloudFormation Management Console using your newly created CloudFormation config.
You will also need to generate TLS asssets: https://coreos.com/kubernetes/docs/latest/openssl.html. These assets will have to be compressed and encoded (same gzip and base64 as above), then inserted into your userdata cloud-configs.
When debugging on the server, journalctl is your friend:
$ journalctl -u oem-cloudinit # to debug problems with your cloud-config
$ journalctl -u etcd2
$ journalctl -u kubelet
Hope that helps.
There is also kops project
From the project README:
Operate HA Kubernetes the Kubernetes Way
also:
We like to think of it as kubectl for clusters
Download the latest release, e.g.:
cd ~/opt
wget https://github.com/kubernetes/kops/releases/download/v1.4.1/kops-linux-amd64
mv kops-linux-amd64 kops
chmod +x kops
ln -s ~/opt/kops ~/bin/kops
See kops usage, especially:
kops create cluster
kops update cluster
Assuming you already have s3://my-kops bucket and kops.example.com hosted zone.
Create configuration:
kops create cluster --state=s3://my-kops --cloud=aws \
--name=kops.example.com \
--dns-zone=kops.example.com \
--ssh-public-key=~/.ssh/my_rsa.pub \
--master-size=t2.medium \
--master-zones=eu-west-1a,eu-west-1b,eu-west-1c \
--network-cidr=10.0.0.0/22 \
--node-count=3 \
--node-size=t2.micro \
--zones=eu-west-1a,eu-west-1b,eu-west-1c
Edit configuration:
kops edit cluster --state=s3://my-kops
Export terraform scripts:
kops update cluster --state=s3://my-kops --name=kops.example.com --target=terraform
Apply changes directly:
kops update cluster --state=s3://my-kops --name=kops.example.com --yes
List cluster:
kops get cluster --state s3://my-kops
Delete cluster:
kops delete cluster --state s3://my-kops --name=kops.identityservice.co.uk --yes
I'm trying to setup example from Running Wordpress with a Single Pod.
I've done Before You Begin section:
$ gcloud config list
[compute]
zone = europe-west1-c
[core]
account = user#email.com
disable_usage_reporting = False
project = com-project-default
I've done the steps from the tutorial:
"Step 1: Create your cluster" logs here
"Step 2: Create your pod" logs here
"Step 3: Allow external traffic" logs here
More logs:
$kubectl get pods - log (toggle text wrapping)
$gcloud compute firewall-rules list - log
So, when I try to connect to http://104.155.7.213/ I'm receiving "This web page is not available: ERR_CONNECTION_REFUSED".
I tried to add "Allow HTTP traffic" explicitly to the node in Compute Engine VMs dashboard and also I tried to use "kubectl run" instead of deprecated "kubectl run-container", but it doesn't help. Also sometimes I'm receiving "last termination: exit code 2" (1 or 2) in "message" column when run "kubectl get pods" (but not this time)`
Info:
GKE from June 10, 2015
$ kubectl version
Client Version: version.Info{Major:"0", Minor:"18", GitVersion:"v0.18.1", GitCommit:"befd1385e5af5f7516f75a27a2628272bb9e9f36", GitTreeState:"clean"}
Server Version: version.Info{Major:"0", Minor:"18", GitVersion:"v0.18.2", GitCommit:"1f12b893876ad6c41396222693e37061f6e80fe1", GitTreeState:"clean"}
$ gcloud version
Google Cloud SDK 0.9.64
alpha 2015.06.02
bq 2.0.18
bq-nix 2.0.18
compute 2015.06.09
core 2015.06.09
core-nix 2015.06.02
dns 2015.06.02
gcloud 2015.06.09
gcutil-msg 2015.06.09
gsutil 4.13
gsutil-nix 4.12
kubectl
kubectl-linux-x86_64 0.18.1
preview 2015.06.09
sql 2015.06.09
Thank you for your help in advance!
If you want to access the container directly using the node VM's IP address, you need to specify a host port in addition to a container port, e.g.
kubectl run-container wordpress --image=tutum/wordpress --port=80 --hostport=80
Alternatively, you can access wordpress via the proxy running on the master by running kubectl proxy and then pointing your web browser at http://localhost:8001/api/v1beta3/proxy/namespaces/default/pods/wordpress-3gaq6.