Stumped on this issue and hoping someone who knows more can help me.
Trying to follow a guide with a proof of concept for cluster autoscaling on AWS for Kubernetes (https://renzedevries.wordpress.com/2017/01/10/autoscaling-your-kubernetes-cluster-on-aws/). I built my cluster on AWS using kops following this guide (https://medium.com/containermind/how-to-create-a-kubernetes-cluster-on-aws-in-few-minutes-89dda10354f4).
The issue appears to be with the cluster autoscaling deployment. When I run:
kubectl logs cluster-autoscaler-
I get the following output:
I0706 13:26:36.338072 1 leaderelection.go:210] failed to renew
lease kube-system/cluster-autoscaler
I0706 13:26:38.776977 1 leaderelection.go:210] failed to renew
lease kube-system/cluster-autoscaler
I0706 13:26:43.119763 1 leaderelection.go:210] failed to renew
lease kube-system/cluster-autoscaler
I0706 13:26:47.116439 1 leaderelection.go:210] failed to renew
I've been looking into the error and it was to do with the namespace. When I run the pod in a different namespace or the namespace recommended (kube-system - https://github.com/kubernetes/contrib/issues/2402) I still get the same error. Not sure what it causing it.
Thanks in advance for the help!
Follow this guide here to configure Cluster Autoscaler for Kubernetes running in AWS.
Configure Cluster Autoscaler in Kubernetes
It should do the work without any errors.
Related
I set up the pipeline three months ago and everything has been running fine with the same appspec.yml. But now suddenly AWS CodeDeploy gives the error that it can't find the appspec.yml although it is already there.It failed at the very first ApplicationStop event itself. The error is as follows:
The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems. (Error code: HEALTH_CONSTRAINTS)
Then when I looked into the details, this is what it said:
My appspec.yml is as follows:
version: 0.0
os: windows
files:
- source: \
destination: c:\home\afb
file_exists_behavior: OVERWRITE
I also have the folder (c:\home\afb) created already on my EC2 instance. The health of the EC2 instance is fine as I can see on the Dashboard and also access it via RDP. The CodeDeploy agent is also running fine on EC2.
Please help. Thanks in advance for any advice!
I have single ec2 instance running on ubuntu server and I am trying to implement CI/CD flow using codedeploy and source is bit-bucket.I jave also installed codedeploy-agent on ec2 instance and it is installed and running successfully but whenever I am deploying code on ec2 deployment is failing with an error shown below:
The overall deployment failed because too many individual instances failed deployment, too few
healthy instances are available for deployment, or some instances in your deployment group are
experiencing problems.
In the CodeDeploy agent log file that I am accessing using less /var/log/aws/codedeploy-agent/codedeploy-agent.log showing below error:
ERROR [codedeploy-agent(31598)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller:
Missing credentials - please check if this instance was started with an IAM instance profile
I am unable to understand how can I overcome this error someone let me know.
CodeDeploy agent requires IAM permissions provided by IAM role/profile of your instance. The exact permissions needed are given in AWS docs:
Step 4: Create an IAM instance profile for your Amazon EC2 instances
I have an issue with k8s cluster on aws.
I tried to create cluster on aws.
The first step was creation:
kops create cluster --name=kubernetes.xarva.stream --state=s3://kops-bucket-pnz-se-kube1 --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=kubernetes.xarva.stream --zones=eu-central-1a
and it was successfully completed. But when I tried to update the cluster with the command:
kops update cluster kubernetes.xarva.stream --state=s3://kops-b ucket-pnz-se-kube1 --yes
I got this error:
error writing completed cluster spec: error writing configuration file s3://kops -bucket-pnz-se-kube1/kubernetes.xarva.stream/cluster.spec: error writing s3://ko ps-bucket-pnz-se-kube1/kubernetes.xarva.stream/cluster.spec: RequestTimeout: You r socket connection to the server was not read from or written to within the tim eout period. Idle connections will be closed.
status code: 400, request id: 2***********3, host id: *********************************************6
On the s3 bucket I found the configuration, so seems that it's read issue.
Does anybody face with this problem? Any ideas how to solve it?
Thanks
RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
This error says you have a problem with access to AWS S3 from a place where you have installed kops.
Make sure you have access to S3 and try again or try to do this from another place.
I am actually trying to deploy my application using Kubernetes in the AWS Kops. For this i followed the steps given in the AWS workshop tutorial.
https://github.com/aws-samples/aws-workshop-for-kubernetes/tree/master/01-path-basics/101-start-here
I created a AWS Cloud9 environment by logging in as a IAM user and installed kops and other required software's as well. When i try to create the cluster using the following command
kops create cluster --name cs.cluster.k8s.local --zones $AWS_AVAILABILITY_ZONES
--yes
i get an error like below in the cloud9 IDE
error running tasks: deadline exceeded executing task IAMRole/nodes.cs.cluster.k8s.local. Example error: error creating IAMRole: InvalidClientTokenId: The security token included in the request is invalid
status code: 403, request id: 30fe2a97-0fc4-11e8-8c48-0f8441e73bc3
I am not able to find a way to solve this issue. Any help on this would be appreciable.
I found the issue and fixed it. Actually
I did not export the following 2 environment variables in the terminal where I am running create cluster. These 2 below variables are required while creating a cluster using kops
export AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id)
export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key)
I went through the firstRun steps here (AWS login required).
I have uploaded a docker image to the ECR and the cluster launches successfully, all steps succeed (ECS status - 4 of 4 complete and EC2 instance status - 14 of 14 complete).
There is no instance attached to the cluster although it is running (see screenshots). What am I doing wrong?
The permissions were missing, or in other words, EMR_EC2_DefaultRule wasn't a AmazonEC2ContainerServiceforEC2Role.
It's explained here.
Would've expected the setup to fail if the role does not grant enough permissions for the EC2 instance to actually connect to the cluster.