I am using GithHub Actions with CodeBuild,when I run GitHub Actions,I am getting error message CodeBuild project name can not be found.The issue is that my codebuild project is in my assumed role(sandbox_role) but github action is looking for the project in the root account that i configured as environment variable in github secret.How can I configure my GitHub Action workflows to first connect to the root then from there assume sandbox_role to get my codebuild project?below is my code sample..I am using terragrunt/terraform code to provision my environment
name:'GitHub Actions For CodeBuild'
on:
pull_request:
branches:
- staging
jobs:
CodeBuild:
name:'Build'
runs-on:ubuntu-latest
steps:
-name:'checkout'
uses:actions/checkout#v2
-name:configure AWS credentials
uses:aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{secrets.AWS_ACCESS_KEY_ID}}
aws-secret-access-key: ${{secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
-name:Run CodeBuild
uses: aws-actions/aws-codebuild-run-build#v1
with:
project-name: CodeBuild
buildspec-override: staging/buildspec.yml
env-vars-for-codebuild: |
TF_INPUT,
AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY,
AWS_REGION,
env:
TF_INPUT: false
AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY_ID}}
AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: us-east-1
Not pretty sure if this works, but whenever I use roles I also pass the role ARN "to tell" AWS which is the role you are using and which permissions it should have.
This role ARN can be added in the configuration and credentials file:
Configuration:
region = us-east-1
output = json
role_arn = arn:aws:iam::account_id:role/role-name
source_profile=default
Credentials:
[default]
aws_access_key_id="your_key_id"
aws_secret_access_key="your_access_key"
aws_session_token="your_session_token"
source_profile=default
I am making CICD pipeline, using AWS CodeBuild to build and deploy application(service) to aws eks cluster. I have installed kubectl and aws-iam-authenticator properly,
getting aws instead of aws-iam-authenticator in command
kind: Config
preferences: {}
users:
- name: arn:aws:eks:ap-south-1:*******:cluster/DevCluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- eks
- get-token
- --cluster-name
- DevCluster
command: aws
env: null
[Container] 2019/05/14 04:32:09 Running command kubectl get svc
error: the server doesn't have a resource type "svc"
I donot want to edit configmap manually because it comes through pipeline.
As #Priya Rani said in the comments, he found the solution.
There is no issue with configmap file. Its all right.
1) I need to make Cloudformation (cluster+nodeinstance)trusted role to communicate with Codebuild by editing trusted role.
2) Need to add usedata section to communicate node instance with clusters.
Why you don't just load a proper/dedicated kube config file, by setting KUBECONFIG env variable inside your CICD pipeline, like this:
export KUBECONFIG=$KUBECONFIG:~/.kube/config-devel
which would include a right command to use with aws-iam-authenticator:
#
#config-devel
#
...
kind: Config
preferences: {}
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "<cluster-name>"
I have been trying to follow the getting started guide to EKS.
When I tried to call kubectl get service I got the message: error: You must be logged in to the server (Unauthorized)
Here is what I did:
1. Created the EKS cluster.
2. Created the config file as follows:
apiVersion: v1
clusters:
- cluster:
server: https://*********.yl4.us-west-2.eks.amazonaws.com
certificate-authority-data: *********
name: *********
contexts:
- context:
cluster: *********
user: aws
name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: heptio-authenticator-aws
args:
- "token"
- "-i"
- "*********"
- "-r"
- "arn:aws:iam::*****:role/******"
Downloaded and installed latest aws cli
Ran aws configure and set the credentials for my IAM user and the region as us-west-2
Added a policy to the IAM user for sts:AssumeRole for the EKS role and set it up as a trusted relationship
Setup kubectl to use the config file
I can get a token when I run heptio-authenticator-aws token -r arn:aws:iam::**********:role/********* -i my-cluster-ame
However when I try to access the cluster I keep receiving error: You must be logged in to the server (Unauthorized)
Any idea how to fix this issue?
When an Amazon EKS cluster is created, the IAM entity (user or role) that creates the cluster is added to the Kubernetes RBAC authorization table as the administrator. Initially, only that IAM user can make calls to the Kubernetes API server using kubectl.
eks-docs
So to add access to other aws users, first
you must edit ConfigMap to add an IAM user or role to an Amazon EKS cluster.
You can edit the ConfigMap file by executing:
kubectl edit -n kube-system configmap/aws-auth, after which you will be granted with editor with which you map new users.
apiVersion: v1
data:
mapRoles: |
- rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-NodeInstanceRole-74RF4UBDUKL6
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
mapUsers: |
- userarn: arn:aws:iam::111122223333:user/ops-user
username: ops-user
groups:
- system:masters
mapAccounts: |
- "111122223333"
Pay close attention to the mapUsers where you're adding ops-user together with mapAccounts label which maps the AWS user account with a username on Kubernetes cluster.
However, no permissions are provided in RBAC by this action alone; you must still create role bindings in your cluster to provide these entities permissions.
As the amazon documentation(iam-docs) states you need to create a role binding on the kubernetes cluster for the user specified in the ConfigMap. You can do that by executing following command (kub-docs):
kubectl create clusterrolebinding ops-user-cluster-admin-binding --clusterrole=cluster-admin --user=ops-user
which grants the cluster-admin ClusterRole to a user named ops-user across the entire cluster.
I am sure issue is resolved but I will be putting more information here so if any other people are still facing the issue related to any of the below setup then they can use the steps below.
When we create the EKS cluster by any method via CloudFormation/CLI/EKSCTL the IAM role/user who created the cluster will automatically binded to the default kubernetes RBAC API group system:masters (https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) and in this way creator of the cluster will get the admin access to the cluster. Although we can always give the access to other IAM user/role using the aws-auth file but for that we must have to use the IAM user/role who created the cluster.
To verify the role/user for the EKS cluster we can search for the CreateCluster" Api call on cloudtrail and it will tell us the creator of the cluster in the sessionIssuer section for field arn (https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html).
When we create the cluster using the IAM role or IAM user, setting up the access for the EKS cluster will become little tricky when we created the cluster using the role compare to user.
I will put the steps we can follow for each different method while setting up the access to EKS cluster.
Scenario-1: Cluster was Created using the IAM user (For example "eks-user")
Confirm that IAM user credentials are set properly on AWS cli who has created the cluster via running the command aws sts get-caller-identity
$ aws sts get-caller-identity
{
"Account": "xxxxxxxxxxxx",
"UserId": "xxxxxxxxxxxxxxxxxxxxx",
"Arn": "arn:aws:iam::xxxxxxxxxxx:user/eks-user"
}
After that update the kubeconfig file using the below command
aws eks --region region-code update-kubeconfig --name cluster_name
Attaching the config file how it looks like once updated via above command. Please do not directly edit this file until and unless necessary.
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: CERT
server: https://xxxxxxx.sk1.us-east-1.eks.amazonaws.com
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
contexts:
- context:
cluster: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
current-context: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- eks-cluster
command: aws
Once above setup is done you should be able to run the kubectl command.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP xxx.xx.x.x <none> 443/TCP 12d
Scenario-2: Cluster was Created using the IAM Role (For example "eks-role")
Mainly there are four different way to setup the access via cli when cluster was created via IAM role.
1. Setting up the role directly in kubeconfig file.
In this case we do not have to make any assume role api call via cli manually, before running kubectl command because that will be automatically done by aws/aws-iam-authenticator set in the kube config file.
Lets say now we are trying to setup the access for the user eks-user the first make sure that user does have permission to assume the role eks-role
Add the assume role permission to the eks-user
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::xxxxxxxxxxx:role/eks-role"
}
]
}
Edit the trust relationship on the role so that it will allow the eks-user to assume the role.
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxxxx:user/eks-user"
},
"Action": "sts:AssumeRole"
}
]
}
Confirm that IAM user credentials are set properly on AWS cli who has created the cluster via running the command aws sts get-caller-identity. Important thing to remember it should show us the IAM user ARN not the IAM assumed ROLE ARN.
$ aws sts get-caller-identity
{
"Account": "xxxxxxxxxxxx",
"UserId": "xxxxxxxxxxxxxxxxxxxxx",
"Arn": "arn:aws:iam::xxxxxxxxxxx:user/eks-user"
}
After that update the kubeconfig file using the below command
aws eks --region region-code update-kubeconfig --name cluster_name --role-arn arn:aws:iam::xxxxxxxxxxx:user/eks-role
Attaching the config file how it looks like once updated via above command. Please do not directly edit this file until and unless necessary.
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: CERT
server: https://xxxxxxx.sk1.us-east-1.eks.amazonaws.com
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
contexts:
- context:
cluster: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
current-context: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- eks-cluster
- --role
- arn:aws:iam::xxxxxxx:role/eks-role
command: aws
Once above setup is done you should be able to run the kubectl command.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP xxx.xx.x.x <none> 443/TCP 12d
2. If you have setup the AWS profile (https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) on CLI and if you want to use that with the kube config.
Confirm that profile is set properly so that it can use the credentials for the eks-user
$ cat ~/.aws/config
[default]
output = json
region = us-east-1
[eks]
output = json
region = us-east-1
[profile adminrole]
role_arn = arn:aws:iam::############:role/eks-role
source_profile = eks
$ cat ~/.aws/credentials
[default]
aws_access_key_id = xxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[eks]
aws_access_key_id = xxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Once this profile configuration is done please confirm that profile configuration is fine by running the command aws sts get-caller-identity --profile eks
$ aws sts get-caller-identity --profile eks
{
"Account": "xxxxxxxxxxxx",
"UserId": "xxxxxxxxxxxxxxxxxxxxx",
"Arn": "arn:aws:iam::xxxxxxxxxxx:user/eks-user"
}
After that update the kubeconfig file using the below command with the profile and please make sure we are not using the role here.
aws eks update-kubeconfig --name devel --profile eks
Attaching the config file how it looks like once updated via above command. Please do not directly edit this file until and unless necessary.
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: CERT
server: https://xxxxx.sk1.us-east-1.eks.amazonaws.com
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
contexts:
- context:
cluster: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
current-context: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- eks-cluster
command: aws
env:
- name: AWS_PROFILE
value: eks
Once above setup is done you should be able to run the kubectl command.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP xxx.xx.x.x <none> 443/TCP 12d
3. Assume the role by any other way, For example we can attach the IAM role to the instance directly.
If role is directly attached to the instance profile then we can follow the similar steps as we followed while setting up the access for IAM user in Scenario-1
Verify that we have attached the correct role to EC2 instance and as this instance profile will come into least precedence, this step will also verify that there are no any other credentials setup on the instnace.
[ec2-user#ip-xx-xxx-xx-252 ~]$ aws sts get-caller-identity
{
"Account": "xxxxxxxxxxxx",
"UserId": "xxxxxxxxxxxxxxxxxxxxx:i-xxxxxxxxxxx",
"Arn": "arn:aws:sts::xxxxxxxxxxxx:assumed-role/eks-role/i-xxxxxxxxxxx"
}
After that update the kubeconfig file using the below command
aws eks --region region-code update-kubeconfig --name cluster_name
Attaching the config file how it looks like once updated via above command. Please do not directly edit this file until and unless necessary.
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: CERT
server: https://xxxxxxx.sk1.us-east-1.eks.amazonaws.com
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
contexts:
- context:
cluster: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
current-context: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- eks-cluster
command: aws
Once above setup is done you should be able to run the kubectl command.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP xxx.xx.x.x <none> 443/TCP 12d
4. Manually assuming the IAM role via aws sts assume-role command.
Assume the role eks-role manually by running the cli command.
aws sts assume-role --role-arn arn:aws:iam::xxxxxxxxxxx:role/eks-role --role-session-name test
{
"AssumedRoleUser": {
"AssumedRoleId": "xxxxxxxxxxxxxxxxxxxx:test",
"Arn": "arn:aws:sts::xxxxxxxxxxx:assumed-role/eks-role/test"
},
"Credentials": {
"SecretAccessKey": "xxxxxxxxxx",
"SessionToken": xxxxxxxxxxx",
"Expiration": "xxxxxxxxx",
"AccessKeyId": "xxxxxxxxxx"
}
}
After that set the required environment variable using the value from above output so that we can use the correct credentials generated from the session.
export AWS_ACCESS_KEY_ID=xxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxx
export AWS_SESSION_TOKEN=xxxxxxxxxx
After that verify that we assumed the IAM role by running the command aws sts get-caller-identity.
aws sts get-caller-identity
{
"Account": "xxxxxxxxxx",
"UserId": "xxxxxxxxxx:test",
"Arn": "arn:aws:sts::xxxxxxxxxx:assumed-role/eks-role/test"
}
After that update the kubeconfig file using the below command
aws eks --region region-code update-kubeconfig --name cluster_name
Attaching the config file how it looks like once updated via above command. Please do not directly edit this file until and unless necessary.
$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: CERT
server: https://xxxxxxx.sk1.us-east-1.eks.amazonaws.com
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
contexts:
- context:
cluster: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
current-context: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-east-1:xxxxxxx:cluster/eks-cluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- eks-cluster
command: aws
Once above setup is done you should be able to run the kubectl command.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP xxx.xx.x.x <none> 443/TCP 12d
NOTE:
I have try to cover major use case here but there might be other use case too where we need to setup the access to the cluster.
Also the above tests are mainly aiming at the first time setup of the EKS cluster and none of the above method is touching the aws-auth configmap yet.
But once you have given access to other IAM user/role to EKS cluster via aws-auth (https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) file you can use the same set of commands for those users too as mentioned in above answer.
Update: If you are using the SSO then setup will be preety much same but one thing we have to consider is either in case of SSO or while using the role directly if we are trying to update path based role in ConfigMap then we have to get rid of the paths in role for example instead arn:aws:iam::xxxxxxxxxxx:role/path-1/subpath-1/eks-role of this use arn:aws:iam::xxxxxxxxxxx:role/eks-role so basically we are getting rid of the /path-1/subpath-1 becuase when we run kubectl command it will first make AssumeRole api call and if we see assumed role ARN then it will not contians the path so if we include the path then it EKS will deny those requests.
If you are using eksctl to manage your aws eks deployments you can add the user to the config map with one command:
eksctl create iamidentitymapping --cluster <cluster-name> --arn arn:aws:iam::<id>:user/<user-name> --group system:masters --username ops-user
I commented out the last two lines of the config file
# - "-r"
# - "arn:aws:iam::**********:role/**********"
and it worked though I have no idea why
You need to create the cluster under the same IAM profile that you are accessing it from via AWS cli.
Said in another way, inside ~/.aws/credentials, the profile that is accessing kubectl must match exactly the same IAM that was used to create the cluster.
My recommendation is to use AWS cli to create your clusters as creating from the GUI may be more confusing than helpful. The Getting Started guide is your best bet to get up and running.
Also, make sure your users are in the aws-auth k8s ConfigMap:
https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html
I had the same problem . It's likely that you are using a root account. It appears root accounts are blocked from assuming the required roles. This error can sometimes be cloaked if you are using expired keys.
I had the same problem, my AWS credentials for CLI change frequently. These steps fixed the problem:
export AWS_ACCESS_KEY_ID="***************"
export AWS_SECRET_ACCESS_KEY="*************"
export AWS_SESSION_TOKEN="************************"
If you have created the EKS cluster with kops, Then all you need to do is update your kubecfig file with following kops command
kops export kubecfg --admin
If you have exhausted all of the above solutions and are still getting the same error. Make sure kubectl is actually using the AWS credentials you think you are.
I made a very stupid mistake. I have environment variables setup for different accounts and kubectl seems to always pick up the one from AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY. So precedence of options on this page may help https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
This happens also to me with local environment on minikube, independently of EKS. My problem is related to this issue: https://github.com/kubernetes/kubernetes/issues/76774
The solution i adopted is to remove the cache directories of kubectl: rm -rf ~/.kube/{cache,http-cache}. I guess is the only workaround at the time of writing.
In my case it is the AWS profile issue, be sure to use aws sts get-caller-identity to verify the IAM user.
I just debugged this issue. I have a question. Are you running this on a corporate wifi network? If yes, could you create an EC2 instance and then test if you are able to do kubectl get svc?
Also, try if this command works
kubectl get svc --insecure-skip-tls-verify
The issue for me was that I had set up environment variables with different, invalid, AWS credentials (maybe a long time ago, and forgot). I realised this after running aws configure list and seeing that the credentials are different from what I expected with aws configure list --profile default. Finding and deleting those invalid environment variables fixed the issue, now I can run kubectl get svc.
The issue is with the policy added for the roles created. We must have AWSEKSCNI policy.
Better to create eks using the command:
eksctl create cluster --name ekscluster --version 1.19 --with-oidc
--vpc-public-subnets=subnet-08c6b0b0166abc1d1,subnet-02822a142bb5a802a
--vpc-private-subnets=subnet-09bbf4871902ee64c,subnet-0926c224909b5a811
This will create and assign the policy automatically using cloudformation.
In my case the scenario was, that I replaced a person who worked as a devops and created all the infrastructure on aws. He had his own IAM user (so yes, no root access case, however could be also applicable) with all AWS permission and I had my own IAM user also with all available AWS permissions.
So, he left the company and that was the problem, since I didn't get any access to cluster which by defauly is shared to the creator only ... and in fact all of my approaches to get the access to cluster were failed, even despite a fact that I had all permissions.
The good thing was, guy whom I replaced was still had his IAM user available (not removed). And what I did, I simply generated new AWS access pair under his account and set them as my default aws credentials on my ubuntu host (from which I've tried to access the cluster). Important part was to make sure that after running aws sts get-caller-identity command it meant to be HIS account to appear on the output. In that case I've been able to run all the kubectl commands that I wanted without error You must be logged in to the server (Unauthorized) message.
So the solution is - be lucky to find cluster creator credentials and use them! (sounds like a crime, however ...)
I had the same issue.
Refer the answers:
Cannot create namespaces in AWS Elastic Kubernetes Service - Forbidden
User cannot log into EKS Cluster using kubectl
The correct way is to:
Create an IAM Group with the following permissions.
1. AmazonEKSClusterPolicy
2. AmazonEKSWorkerNodePolicy
3. AmazonEC2ContainerRegistryReadOnly
4. AmazonEKS_CNI_Policy
5. AmazonElasticContainerRegistryPublicReadOnly
6. EC2InstanceProfileForImageBuilderECRContainerBuilds
7. AmazonElasticContainerRegistryPublicFullAccess
8. AWSAppRunnerServicePolicyForECRAccess
9. AmazonElasticContainerRegistryPublicPowerUser
10. SecretsManagerReadWrite
Create an IAM user (AWS_ACCESS_KEY_ID and AWS_SECRET_KEY_ID will be provided) and add the user to the IAM Group created above.
Next, login to AWS Console as the IAM user and create the EKS Cluster.
Next, use the AWS_ACCESS_KEY_ID and AWS_SECRET_KEY_ID to setup the AWS CLI in local machine.
Next, run the following commands in the local machine:
1. aws sts get-caller-identity
2. aws eks describe-cluster --name [cluster-name] --region [aws-region] --query cluster.status (To check the status of the Cluster)
3. aws eks update-kubeconfig --name [cluster-name] --region [aws-region]
After this, you will be able to run the kubectl commands.
If you need to add additional users to the EKS Cluster, create the additional IAM user, add the user to the IAM Group in AWS. Next, log into the EKS cluster as the original IAM user and run: kubectl edit -n kube-system configmap/aws-auth.
Add the following block of code to the existing configuration:
(Make sure to add changes to the ARN and username)
mapUsers: |
- userarn: arn:aws:iam::111122223333:user/admin
username: admin
groups:
- system:masters
Refer the following link to understand this better: https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html
Hope this become useful.
Happy learning!
In addition to the great answers that have already been given, I would like to add a good way to troubleshoot issues. In my case, I was trying to run kubectl in an ECS task as part of an AWS pipeline, and kubectl version was failing with the "You must be logged in to the server" message.
What was happening was that the ECS task was assuming a service role, and then aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $AWS_DEFAULT_REGION was being run. It turns out the service role, CodeBuildServiceRole, was not mapped to an RBAC user via a clusterrolebinding in the EKS cluster, and the aws-iam-authenticator EKS service was denying access to the AWS service account (or something like that).
What helped me figure out what was going on in this case was to go to CloudWatch → Log Insights and run the following query against the /aws/eks/<cluster-name>/cluster log group:
fields #timestamp, #message
| sort #timestamp desc
| filter #logStream like /authenticator/
| filter #message like /access denied/
| limit 50
It was clear from the log output that the authenticator service was expecting an ARN in configmap/aws-auth that was lowercase: arn:aws:iam::123456789:role/codebuildservicerole. Once I fixed the case of the ARN in the configmap, kubectl version started working in the ECS task.
$ kubectl edit -n kube-system configmap/aws-auth
I had the same problem and the fix was to remove an alias I had added when setting up minikube. Their docs say to add this alias:
alias kubectl="minikube kubectl --"
Which breaks some things like Azure integrations, as you would expect.
I got this error when I created the eks cluster using the root from the eks console. I recreated the eks cluster using an IAM user and use the access keys to update the aws configure. It worked. Now you can add additional IAM users to issue kubectl commands.
I was trying to create an EKS cluster with a private endpoint. Read this thread several times and the thing that worked for me:
Created a user: admin (with access to the console)
Logged into the console using that user
Created the cluster using the admin user
Created my kube/config file using aws eks (aws eks --region update-kubeconfig --name )
Done (kubectl get ns)
The last 2 commands were executed from an ec2 instance part of the same VPC.
I faced the same issue. I tried to configure AWS CLI directly with access key and secret key and it worked.
It should be bug to not able to assume role. Try setup cli and test.
I followed these docs. It took some time to understand things.
But finally implemented the things easily with full understanding.
https://kubernetes.io/docs/reference/access-authn-authz/rbac/
https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html
When invoking eksctl with an assumed role through sso, the following steps got me access to the cluster.
eksctl.exe create cluster --name test --without-nodegroup --profile <your_profile>.
Update the kubeconfig :
eksctl utils write-kubeconfig --cluster=test --profile<your_profile>.
Enable Cloudwatch to get the iam role arn:
eksctl utils update-cluster-logging --enable-types=all --cluster=test --approve --profile=<your_profile>.
run "kubectl get pods -A" . server responds with "error: You must be logged in to the server (Unauthorized)" message.
Switch to the console and get the role arn from the cloudwatch group audit log.
Update the aws-auth config map by mapping the derived arn to system:masters group. The username is users:name field in your current kubeconfig context as extracted it below
user_id=` k config view --minify -o json | jq -r '.users[0].name'`
eksctl create iamidentitymapping --cluster test --group system:masters --username ${user_id} --arn arn:aws:iam::<account_id>:role/<assumed_role> --profile <your_profile>
For me adding the user in a single line like below worked
kubectl edit -n kube-system configmap/aws-auth
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::******:role/eksctl-atoa-microservices-nodegro-NodeInstanceRole-346C48Q1W7OB
username: system:node:{{EC2PrivateDNSName}}
mapUsers: "- groups:\n - system:masters\n userarn: arn:aws:iam::*****:user/<username>
\ \n username: <username>\n"
The AWS CLI command tasks in Ansible playbooks work fine form command line if AWS credentials are specified as environment variables as per boto requirements. More info can be found here Environment Variables.
But they fail to run in Tower because it exports another set of env. vars:
AWS_ACCESS_KEY
AWS_SECRET_KEY
In order to make them work in Tower just add the below in task definition:
environment:
AWS_ACCESS_KEY_ID: "{{ lookup('env','AWS_ACCESS_KEY') }}"
AWS_SECRET_ACCESS_KEY: "{{ lookup('env','AWS_SECRET_KEY') }}"
e.g. this task:
- name: Describe instances
command: aws ec2 describe-instances --region us-east-1
will transform to:
- name: Describe instances
command: aws ec2 describe-instances --region us-east-1
environment:
AWS_ACCESS_KEY_ID: "{{ lookup('env','AWS_ACCESS_KEY') }}"
AWS_SECRET_ACCESS_KEY: "{{ lookup('env','AWS_SECRET_KEY') }}"
NOTE: This only injects env.var. for the specific task - not the whole playbook!
So you have to amend this way every AWS CLI task.
Put your environment variable in a file:
export AWS_ACCESS_KEY=
export AWS_SECRET_KEY=
save the file in ~/.vars in the remote host and then in your playbook.
- name: Describe instances
command: source ~/.vars && aws ec2 describe-instances --region us-east-2
for security you could delete the file after run and copy again in the next play.
While this may not be applicable to tower we use the opensource version. Setup your .aws and/or .boto files.