I am running a K8S cluster in AWS with EKS and external-DNS with an ingress-Nginx-ingress-controller. Application is working fine and accessible on load-balancer. Now I am trying to point my DNS to this cluster i.e. e1.exapmle.com and for this, I am following below documentation.
https://github.com/kubernetes-sigs/external-dns
The issue I am facing is that I have two AWS accounts (QA, Staging) and hosted zones are created in staging account and the cluster is running on QA account. I have tried cross-account IAM roles for this to work but I can only see hosted zones via the web console, cli or external-DNS pods are not able to able to communicate with Route53.
This is what I have done so far:
Create an IAM policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::hostedzone/*"
]
},
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones",
"route53:ListResourceRecordSets"
],
"Resource": [
"*"
]
}
]
}
Create a cross-account IAM role trusting account staging and trusted account QA.
Assum policy in the trusted account by specifying IAM role ARN for the above policy.
By doing above I only managed to see hosted zones via web console when I switch roles.
I assume you provided a role arn of a role in the EKS account to the extern-dns pods which is allowed to assume to role inside you HostedZone account!?
This will not work because then (as you do inside the console) you need to explicitly assume the role with existing permissions.
You need to pass the Cross-Account role with your attached route53 policy directly to the external-dns pods.
Pod -> assumes role in different account -> receives temp. credentials
pod -> assumed role in eks account (-> allowd to assume cross-accout role)
last step won’t happen i guess
Related
{
"Sid": "ElasticBeanstalkHealthAccess",
"Action": [
"elasticbeanstalk:PutInstanceStatistics"
],
"Effect": "Allow",
"Resource": [
"arn:aws:elasticbeanstalk:*:*:application/*",
"arn:aws:elasticbeanstalk:*:*:environment/*"
]
}
That's a part of the IAM profile for the elastic beanstalk instance.
If you choose AWSElasticBeanstalkWebTier or AWSElasticBeanstalkWorkerTier as IAM Instance profile, the ElasticBeanstalkHealthAccess permissions will be added already.
See https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/iam-instanceprofile.html
There are two IAM roles associated with an Elastic Beanstalk Environment:
Service role: used to manage the environment
Instance role: role assumed by the running application. It is used to provide access to other AWS services.
You need to find your instance role in IAM console and attach the permission that you see in the documentation. This will allow your application to send statistics.
I have an EKS cluster and EC2. I would like to create an instance profile and attach it to the EC2 - this profile should allow ONLY READ access to the EKS cluster.
Will the following policy be apt for this requirement?:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"eks:ListNodegroups",
"eks:DescribeFargateProfile",
"eks:ListTagsForResource",
"eks:ListAddons",
"eks:DescribeAddon",
"eks:ListFargateProfiles",
"eks:DescribeNodegroup",
"eks:ListUpdates",
"eks:DescribeUpdate",
"eks:AccessKubernetesApi",
"eks:DescribeCluster",
"eks:ListClusters",
"eks:DescribeAddonVersions"
],
"Resource": "*"
}
]
}
Depends on what you mean by "read" access.
If you mean "read" from AWS perspective as in being able to use the AWS CLI to tell you about EKS, yes that would be sufficient to get you started. This will not include any kubectl commands.
But if you mean read as in being able to execute kubectl commands against the cluster, then you will not be able to achieve that with this.
To implement read access to the cluster itself using kubectl commands, your EC2 instance will need a minimum of the following IAM permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:DescribeCluster",
"eks:ListClusters"
],
"Resource": "*"
}
]
}
With this, your EC2 will be able to execute eksctl utils write-kubeconfig --cluster=cluster-name to configure the kubeconfig. This also assumes you have the required components installed on your EC2 to run kubectl.
You also need to set up permissions within your cluster because the IAM permissions alone don't actually grant any visibility within the cluster itself.
The role you assign to your EC2 would need to be added to the aws-auth configmap in the kube-system namespace. See Managing users or IAM roles for your cluster from AWS docs for examples.
Unfortunately I don't believe there's a simple out-of-the-box RBAC role you can use that gives you read-only access to the entire cluster. Kubernetes provides four user-facing roles and of them, only the system:masters group has cluster-wide access.
Have a look at Kubernetes Using RBAC Authorization documentation - specifically on user-facing roles.
You will need to design a permission strategy to fit your needs, but you do have the default role view that you can start from. The default view user-facing role is tied to a ClusterRoleBinding and was designed / intended to be used in a namespace specific capacity.
Permissions and RBAC for Kubernetes is a very deep rabbit-hole.
I am setting up an AWS ECS Service using cloudformation and yaml syntax.
At some point, in the relevant documentation there is a property called Role whose definitions is the following:
Role
The name or ARN of an AWS Identity and Access Management (IAM) role that allows your Amazon ECS container agent to make calls to your
load balancer.
Note In some cases, you might need to add a dependency on the service role's policy. For more information, see IAM role policy in
DependsOn Attribute.
Required: No
Type: String
Update requires: Replacement
Since I intend to place the specific service behind an Application Load Balancer, is this property needed?
If so, do I need to create a new policy or are there any pre-defined policies that can serve this purpose?
Would it be enough if I just added the role/policy on the EC2 container instance level (e.g. append it in the relevant template that creates the ECS cluster offered by amazon?)
I would really appreciate any examples or use cases if any because the documentation is vague and incomplete on the topic.
You need the Role attribute if you want to use the Application Load balancer with your ECS service. As per the description the role allows your ECS service agent to connect to load balancer. If you are not using load balancer then, the field is optional.
Also setting the role on EC2 instance level is not needed. Since there are some policies which are related to ECS services, adding it at instance level role is not valid.
Please find below AWS managed polices to create the role:
{
"AttachedPolicies": [
{
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole",
"PolicyName": "AmazonEC2ContainerServiceAutoscaleRole"
},
{
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role",
"PolicyName": "AmazonEC2ContainerServiceforEC2Role"
},
{
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole",
"PolicyName": "AmazonEC2ContainerServiceRole"
}
]
}
Trust relationship:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecs.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
To register your service with application load balancer you need the role for the service itself, but some permission needs for ECS container to register with cluster.
Amazon Elastic Container Service uses AWS Identity and Access
Management (IAM) service-linked roles. A service-linked role is a
unique type of IAM role that is linked directly to Amazon ECS.
Service-linked roles are predefined by Amazon ECS and include all the
permissions that the service requires to call other AWS services on
your behalf.
A service-linked role makes setting up Amazon ECS easier because you
don’t have to manually add the necessary permissions. Amazon ECS
defines the permissions of its service-linked roles, and unless
defined otherwise, only Amazon ECS can assume its roles. The defined
permissions include the trust policy and the permissions policy, and
that permissions policy cannot be attached to any other IAM entity.
So if you check this role this contain property
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
To Debug remove the role to service , it will not register.
If you create service using console by default it attach the role.
Amazon ECS needs permissions to register and deregister container
instances with your load balancer when tasks are created and stopped.
In most cases, the Amazon ECS service role is automatically created
for you in the Amazon ECS console first run experience. You can use
the following procedure to check and see if your account already has
an Amazon ECS service role.
This managed polici by AWS having ARN
arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole
Policy ARN
arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole
Description Default policy for Amazon ECS service role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:Describe*",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:Describe*",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets"
],
"Resource": "*"
}
]
}
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/check-service-role.html
this is how its look like
If you do not assign a role to ECS container instance it will never show in you ECS cluster and it will throw an error
2018-09-06T15:26:22Z [ERROR] Could not register: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
under /var/logs/ecs
tail -f ecs-agent.log.2018-09-06
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using-service-linked-roles.html
I have followed the documentation for spinning up an EKS cluster that says to make a service role with certain policies.
https://docs.aws.amazon.com/eks/latest/userguide/eks-ug.pdf
To create your Amazon EKS service role
1. Open the IAM console at https://console.aws.amazon.com/iam/.
2. Choose Roles, then Create role.
3. Choose EKS from the list of services, then Allows Amazon EKS to manage your clusters on your behalf for your use case, then Next: Permissions.
4. Choose Next: Review.
5. For Role name, enter a unique name for your role, such as eksServiceRole, then choose Create role.
When I create a basic hello world app, it throws an AccessDenied error.
Error creating load balancer (will retry): failed to ensure load balancer for service default/nginx:
AccessDenied: User: arn:aws:sts::*************:assumed-role/eks-service-role/************* is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::*************:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing
The two Policies that were added (AmazonEKSClusterPolicy, AmazonEKSServicePolicy) do not have the iam:CreateServiceLinkedRole action allowed. Are we supposed to add this outside of the policies defined in the guide? Or is this something that should be included in the EKS policies?
It seems that the EKS userguide assumes you have created load balancers in your AWS account prior to creating the EKS cluster, and thus have an existing AWSServiceRoleForElasticLoadBalancing service role in AWS IAM.
As described in https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/elb-service-linked-roles.html#create-service-linked-role
You don't need to manually create the AWSServiceRoleForElasticLoadBalancing role. Elastic Load Balancing creates this role for you when you create a load balancer.
EKS is attempting to do this for you, resulting in the access denied exception using the default policies.
Other options to explicitly create the service-linked role prior to EKS cluster creation include:
AWS CLI
aws iam create-service-linked-role --aws-service-name "elasticloadbalancing.amazonaws.com"
Terraform
resource "aws_iam_service_linked_role" "elasticloadbalancing" {
aws_service_name = "elasticloadbalancing.amazonaws.com"
}
Or, manually create a load balancer from the UI Console.
Regardless of provisioning options, you should know things will work when you see the following role in AWS IAM
arn:aws:iam::<ACCOUNT_ID>:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing
I got it worked by adding this policy to the EKS Role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/*"
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeAccountAttributes"
],
"Resource": "*"
}
]
}
I want to add permissions to my instance's IAM roles to allow for the use of Cloudwatch Log Service. Upon looking at the Configuration Guide for the service, I see the following passage:
The CloudWatch Logs agent supports IAM roles and users.
If your instance already has an IAM role associated with it, make sure that you include the IAM policy below.
If you don't already have an IAM role assigned to your instance,
you'll need to use your IAM credentials for the next steps because you cannot assign an IAM role to an existing instance;
you can only specify a role when you launch a new instance.
I'm having a hard time figuring exactly what this means. When I created a new instance, I made an IAM role with the CloudWatchLogsFullAccess policy and an inline policy the configuration guide tells you to add:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogStreams"
],
"Resource": [
"arn:aws:logs:*:*:*"
]
}
]
}
This was for a new instance, however the above passage confuses me about adding permissions to existing instances. Can I just add the CloudWatchLogsFullAccess policy to the existing instance's role and that inline policy as well?
While it was true an IAM role can be assigned to an instance only during instance launch, AWS announced in February 2017 that it is now possible to replace or attach an IAM role to an existing instance.
Using AWS Console: Easily Replace or Attach an IAM Role to an Existing EC2 Instance by Using the EC2 Console
Using AWS CLI: Attach an AWS IAM Role to an Existing Amazon EC2 Instance by Using the AWS CLI