Related
I have a CDK project that creates a CodePipeline which deploys an application on ECS. I had it all previously working, but the VPC was using a NAT gateway, which ended up being too expensive. So now I am trying to recreate the project without requiring a NAT gateway. I am almost there, but I have now run into issues when the ECS service is trying to start tasks. All tasks fail to start with the following error:
ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 5 time(s): failed to fetch secret
At this point I've kind of lost track of the different things I have tried, but I will post the relevant bits here as well as some of my attempts.
const repository = ECR.Repository.fromRepositoryAttributes(
this,
"ecr-repository",
{
repositoryArn: props.repository.arn,
repositoryName: props.repository.name,
}
);
// vpc
const vpc = new EC2.Vpc(this, this.resourceName(props, "vpc"), {
maxAzs: 2,
natGateways: 0,
enableDnsSupport: true,
});
const vpcSecurityGroup = new SecurityGroup(this, "vpc-security-group", {
vpc: vpc,
allowAllOutbound: true,
});
// tried this to allow the task to access secrets manager
const vpcEndpoint = new EC2.InterfaceVpcEndpoint(this, "secrets-manager-task-vpc-endpoint", {
vpc: vpc,
service: EC2.InterfaceVpcEndpointAwsService.SSM,
});
const secrets = SecretsManager.Secret.fromSecretCompleteArn(
this,
"secrets",
props.secrets.arn
);
const cluster = new ECS.Cluster(this, this.resourceName(props, "cluster"), {
vpc: vpc,
clusterName: `api-cluster`,
});
const ecsService = new EcsPatterns.ApplicationLoadBalancedFargateService(
this,
"ecs-service",
{
taskSubnets: {
subnetType: SubnetType.PUBLIC,
},
securityGroups: [vpcSecurityGroup],
serviceName: "api-service",
cluster: cluster,
cpu: 256,
desiredCount: props.scaling.desiredCount,
taskImageOptions: {
image: ECS.ContainerImage.fromEcrRepository(
repository,
this.ecrTagNameParameter.stringValue
),
secrets: getApplicationSecrets(secrets), // returns
logDriver: LogDriver.awsLogs({
streamPrefix: "api",
logGroup: new LogGroup(this, "ecs-task-log-group", {
logGroupName: `${props.environment}-api`,
}),
logRetention: RetentionDays.TWO_MONTHS,
}),
},
memoryLimitMiB: 512,
publicLoadBalancer: true,
domainZone: this.hostedZone,
certificate: this.certificate,
redirectHTTP: true,
}
);
const scalableTarget = ecsService.service.autoScaleTaskCount({
minCapacity: props.scaling.desiredCount,
maxCapacity: props.scaling.maxCount,
});
scalableTarget.scaleOnCpuUtilization("cpu-scaling", {
targetUtilizationPercent: props.scaling.cpuPercentage,
});
scalableTarget.scaleOnMemoryUtilization("memory-scaling", {
targetUtilizationPercent: props.scaling.memoryPercentage,
});
secrets.grantRead(ecsService.taskDefinition.taskRole);
repository.grantPull(ecsService.taskDefinition.taskRole);
I read somewhere that it probably has something to do with Fargate version 1.4.0 vs 1.3.0, but I'm not sure what I need to change to allow the tasks to access what they need to run.
You need to create an interface endpoints for Secrets Manager, ECR (two types of endpoints), CloudWatch, as well as a gateway endpoint for S3.
Refer to the documentation on the topic.
Here's an example in Python, it'd work the same in TS:
vpc.add_interface_endpoint(
"secretsmanager_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
)
vpc.add_interface_endpoint(
"ecr_docker_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR_DOCKER,
)
vpc.add_interface_endpoint(
"ecr_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.ECR,
)
vpc.add_interface_endpoint(
"cloudwatch_logs_endpoint",
service=ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
)
vpc.add_gateway_endpoint(
"s3_endpoint",
service=ec2.GatewayVpcEndpointAwsService.S3
)
Keep in mind that interface endpoints cost money as well, and may not be cheaper than a NAT.
I have defined a CDK app stack using TypeScript (sensitive information rendomized in the code below):
import * as cdk from "#aws-cdk/core";
import * as ec2 from "#aws-cdk/aws-ec2";
import * as ecs from "#aws-cdk/aws-ecs";
import * as ecr from "#aws-cdk/aws-ecr";
import * as ecr_assets from "#aws-cdk/aws-ecr-assets";
import * as ecs_patterns from "#aws-cdk/aws-ecs-patterns";
import * as sm from "#aws-cdk/aws-secretsmanager";
export class CdkAppStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Create a Docker image and upload it to the Amazon Elastic Container Registry (ECR)
const dockerImage = new ecr_assets.DockerImageAsset(this, "ApiDockerImage", {
directory: "/home/ec2-user/environment/node-test"
});
// Create a new VPC and NAT Gateway
const vpc = new ec2.Vpc(this, "ApiVpc", {
maxAzs: 3 // Default is all AZs in region
});
// Create a new Amazon Elastic Container Service (ECS) cluster
const cluster = new ecs.Cluster(this, "ApiCluster", {
vpc: vpc
});
// Create a load-balanced Fargate service and make it public
new ecs_patterns.ApplicationLoadBalancedFargateService(this, "ApiFargateService", {
cluster: cluster, // Required
cpu: 512, // Default is 256
desiredCount: 2, // Default is 1
taskImageOptions: {
image: ecs.ContainerImage.fromDockerImageAsset(dockerImage),
containerPort: 8080,
enableLogging: true,
secrets: sm.Secret.fromSecretCompleteArn(this, "ImportedSecret", "arn:aws:secretsmanager:ap-south-1:762589711820:secret:/api/production/FrOibp")
},
memoryLimitMiB: 2048, // Default is 512
publicLoadBalancer: true // Default is false
});
}
}
Deployment with cdk deploy is successful if I remove the secrets key from taskImageOptions but with the secrets there, I get this error when trying to deploy:
ec2-user:~/environment/cdk-app (master) $ cdk deploy
⨯ Unable to compile TypeScript:
lib/cdk-app-stack.ts:42:9 - error TS2322: Type 'ISecret' is not assignable to type '{ [key: string]: Secret; }'.
Index signature is missing in type 'ISecret'.
42 secrets: secret
~~~~~~~
Subprocess exited with error 1
I'm doing something wrong here in trying to use secrets from Secrets Manager. What is the right way to reference secrets in a ApplicationLoadBalancedFargateService?
There are two issues here:
secrets is of type index signature. you should therefore name your secret (this is the environment variable that will be exposed in your container)
an ecs.Secret is expected (you can create it from an sm.Secret)
here is a working version:
new ecs_patterns.ApplicationLoadBalancedFargateService(this, "ApiFargateService", {
cluster: cluster, // Required
cpu: 512, // Default is 256
desiredCount: 2, // Default is 1
taskImageOptions: {
image: ecs.ContainerImage.fromDockerImageAsset(dockerImage),
containerPort: 8080,
enableLogging: true,
secrets: {
"MY_SECRET": ecs.Secret.fromSecretsManager( sm.Secret.fromSecretCompleteArn(this, "ImportedSecret", "arn:aws:secretsmanager:ap-south-1:762589711820:secret:/api/production/FrOibp"))
}
},
memoryLimitMiB: 2048, // Default is 512
publicLoadBalancer: true // Default is false
});
AWS supports IAM Roles for Service Accounts (IRSA) that allows cluster operators to map AWS IAM Roles to Kubernetes Service Accounts.
To do so, one has to create an iamserviceaccount in an EKS cluster:
eksctl create iamserviceaccount \
--name <AUTOSCALER_NAME> \
--namespace kube-system \
--cluster <CLUSTER_NAME> \
--attach-policy-arn <POLICY_ARN> \
--approve \
--override-existing-serviceaccounts
The problem is that I don't want to use the above eksctl command because I want to declare my infrastructure using terraform.
Does eksctl command do anything other than creating a service account? If it only creates a service account, what is the YAML representation of it?
I am adding my answer here because I stumble upon the same issue, and accepted answer (and other answers above), do not provide full resolution to the issue - no code examples. They are just guidelines which I had to use to research much deeper. There are some issues which is really easy to miss - and without code examples its quite hard to conclude what is happening (especially part related with Conditions/StringEquals while creating IAM role)
The whole purpose of creating a service account which is going to be tied with the role - is possibility of creating aws resources from within cluster (most common case is load balancer, or roles for pushing logs to the cloudwatch).
So, question is how we can do this, using terraform, instead of using eks commands.
What we need to do, is:
create eks oidc (which can be done with terraform)
create AWS IAM role (which can be done with terraform), create and use proper policies
Create k8s service account (needs to be done with kubectl commands - or with terraform using kubernetes resources
Annotate k8s service account with IAM role we created (meaning that we are linking k8s service account with IAM role)
After this setup, our k8s service account will have k8s cluster role and k8s cluster role binding (which will allow that service account to perform actions within the k8s) and, our k8s service account will have IAM role attached to it, which will allow to perform actions outside of the cluster (like creating aws resources)
So lets start with it. Assumption bellow is that your eks cluster is already created with terraform, and we are focusing on creating resources areound that eks cluster necessary for working service account.
Create eks_oidc
### First we need to create tls certificate
data "tls_certificate" "eks-cluster-tls-certificate" {
url = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
}
# After that create oidc
resource "aws_iam_openid_connect_provider" "eks-cluster-oidc" {
client_id_list = ["sts.amazonaws.com"]
thumbprint_list = [data.tls_certificate.eks-cluster-tls-certificate.certificates[0].sha1_fingerprint]
url = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
}
Now, lets create AWS IAM role with all necessary policies.
Terraform declarative code bellow will:
create ALBIngressControllerIAMPolicy policy
create alb-ingress-controller-role role
attach ALBIngressControllerIAMPolicyr policy to alb-ingress-controller-role role
attach already existing AmazonEKS_CNI_Policy policy to the role
Make a note that i used suffixes as alb ingress controller here, because that is primary use of my role from within the cluster. You can change the name of policy of the role or you can change permission access for the policy as well in dependency of what you are planing to do with it.
data "aws_caller_identity" "current" {}
locals {
account_id = data.aws_caller_identity.current.account_id
eks_oidc = replace(replace(aws_eks_cluster.eks-cluster.endpoint, "https://", ""), "/\\..*$/", "")
}
# Policy which will allow us to create application load balancer from inside of cluster
resource "aws_iam_policy" "ALBIngressControllerIAMPolicy" {
name = "ALBIngressControllerIAMPolicy"
description = "Policy which will be used by role for service - for creating alb from within cluster by issuing declarative kube commands"
policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Effect = "Allow",
Action = [
"elasticloadbalancing:ModifyListener",
"wafv2:AssociateWebACL",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"wafv2:GetWebACLForResource",
"elasticloadbalancing:RegisterTargets",
"iam:ListServerCertificates",
"wafv2:GetWebACL",
"elasticloadbalancing:SetIpAddressType",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:SetWebAcl",
"ec2:DescribeInternetGateways",
"elasticloadbalancing:DescribeLoadBalancers",
"waf-regional:GetWebACLForResource",
"acm:GetCertificate",
"shield:DescribeSubscription",
"waf-regional:GetWebACL",
"elasticloadbalancing:CreateRule",
"ec2:DescribeAccountAttributes",
"elasticloadbalancing:AddListenerCertificates",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"waf:GetWebACL",
"iam:GetServerCertificate",
"wafv2:DisassociateWebACL",
"shield:GetSubscriptionState",
"ec2:CreateTags",
"elasticloadbalancing:CreateTargetGroup",
"ec2:ModifyNetworkInterfaceAttribute",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"ec2:RevokeSecurityGroupIngress",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"shield:CreateProtection",
"acm:DescribeCertificate",
"elasticloadbalancing:ModifyRule",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:DescribeRules",
"ec2:DescribeSubnets",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"waf-regional:AssociateWebACL",
"tag:GetResources",
"ec2:DescribeAddresses",
"ec2:DeleteTags",
"shield:DescribeProtection",
"shield:DeleteProtection",
"elasticloadbalancing:RemoveListenerCertificates",
"tag:TagResources",
"elasticloadbalancing:RemoveTags",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:DescribeListeners",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateSecurityGroup",
"acm:ListCertificates",
"elasticloadbalancing:DescribeListenerCertificates",
"ec2:ModifyInstanceAttribute",
"elasticloadbalancing:DeleteRule",
"cognito-idp:DescribeUserPoolClient",
"ec2:DescribeInstanceStatus",
"elasticloadbalancing:DescribeSSLPolicies",
"elasticloadbalancing:CreateLoadBalancer",
"waf-regional:DisassociateWebACL",
"elasticloadbalancing:DescribeTags",
"ec2:DescribeTags",
"elasticloadbalancing:*",
"elasticloadbalancing:SetSubnets",
"elasticloadbalancing:DeleteTargetGroup",
"ec2:DescribeSecurityGroups",
"iam:CreateServiceLinkedRole",
"ec2:DescribeVpcs",
"ec2:DeleteSecurityGroup",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:SetSecurityGroups",
"elasticloadbalancing:DescribeTargetGroups",
"shield:ListProtections",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:DeleteListener"
],
Resource = "*"
}
]
})
}
# Create IAM role
resource "aws_iam_role" "alb-ingress-controller-role" {
name = "alb-ingress-controller"
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Federated": "${aws_iam_openid_connect_provider.eks-cluster-oidc.arn}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${replace(aws_iam_openid_connect_provider.eks-cluster-oidc.url, "https://", "")}:sub": "system:serviceaccount:kube-system:alb-ingress-controller",
"${replace(aws_iam_openid_connect_provider.eks-cluster-oidc.url, "https://", "")}:aud": "sts.amazonaws.com"
}
}
}
]
}
POLICY
depends_on = [aws_iam_openid_connect_provider.eks-cluster-oidc]
tags = {
"ServiceAccountName" = "alb-ingress-controller"
"ServiceAccountNameSpace" = "kube-system"
}
}
# Attach policies to IAM role
resource "aws_iam_role_policy_attachment" "alb-ingress-controller-role-ALBIngressControllerIAMPolicy" {
policy_arn = aws_iam_policy.ALBIngressControllerIAMPolicy.arn
role = aws_iam_role.alb-ingress-controller-role.name
depends_on = [aws_iam_role.alb-ingress-controller-role]
}
resource "aws_iam_role_policy_attachment" "alb-ingress-controller-role-AmazonEKS_CNI_Policy" {
role = aws_iam_role.alb-ingress-controller-role.name
policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
depends_on = [aws_iam_role.alb-ingress-controller-role]
}
After executing terraform above, you have successfully created terraform part of the resources. Now we need to create a k8s service account and bind IAM role with that service account.
Creating cluster role, cluster role binding and service account
You can use
https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/master/docs/examples/rbac-role.yaml
directly (from the master branch), but having in mind that we need to annotate the iam arn, i have tendency to download this file, update it and store it as updated within my kubectl config files.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
rules:
- apiGroups:
- ""
- extensions
resources:
- configmaps
- endpoints
- events
- ingresses
- ingresses/status
- services
- pods/status
verbs:
- create
- get
- list
- update
- watch
- patch
- apiGroups:
- ""
- extensions
resources:
- nodes
- pods
- secrets
- services
- namespaces
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: alb-ingress-controller
subjects:
- kind: ServiceAccount
name: alb-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/name: alb-ingress-controller
name: alb-ingress-controller
namespace: kube-system
annotations:
eks.amazonaws.com/role-arn: <ARN OF YOUR ROLE HERE>
...
At the bottom of this file, you will notice annotation where you will need to place your ANR role.
Double check
And that would be it. After that you have a k8s service account which is connected with iam role.
Check with:
kubectl get sa -n kube-system
kubectl describe sa alb-ingress-controller -n kube-system
And you should get output similar to this (annotations is the most important part, because it confirms the attachment of iam role):
Name: alb-ingress-controller
Namespace: kube-system
Labels: app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=alb-ingress-controller
Annotations: eks.amazonaws.com/role-arn: <YOUR ANR WILL BE HERE>
meta.helm.sh/release-name: testrelease
meta.helm.sh/release-namespace: default
Image pull secrets: <none>
Mountable secrets: alb-ingress-controller-token-l4pd8
Tokens: alb-ingress-controller-token-l4pd8
Events: <none>
From now on, you can use this service to manage internal k8s resources and external which are allowed by the policies you attached.
In my case, as mentioned before, I used it (beside other things) for creation of alb ingress controller and load balancer, hence all of the prefixes with "alb-ingress"
First, you should define IAM role in Terraform.
Second, you should configure aws-auth configmap in Kubernetes to map the IAM role to Kubernetes user or serviceaccount. You can do that in Terraform using Kubernetes provider.
There is already a Terraform module terraform-aws-eks which manages all aspects of EKS cluster. You may take some ideas from it.
After Vasili Angapov's helps, now I can answer the question:
Yes It does more than just creating a service account. It does three things:
It Creates an IAM role.
It attaches the desired iam-policy (--attach-policy-arn
<POLICY_ARN>) to the created IAM role.
It creates a new kubernetes service account annotated with the arn of the created IAM role.
Now It's easy to declare the above steps using kubernetes and aws providers in terraform.
A role created for that purpose looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<account-id>:oidc-provider/oidc.eks.<region>.amazonaws.com/id/<oidc-id>"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.<region>.amazonaws.com/id/<oidc-id>": "system:serviceaccount:<kube-serviceaccount-namespace>:<kube-serviceaccount-name>"
}
}
}
]
}
I highly recommend you to use iam_assumable_role_admin terraform module for creating this IAM Role for you.
Docs
Example
I am setting up a database cluster (Aurora MySQL 5.7) using the DatabaseCluster Construct from #aws-cdk/aws-rds.
My question, where in the setup can I change the Certificate authority? I want to programmatically setup the database to use rds-ca-2019 instead of rds-ca-2015. Note, I want to change this using CDK, not by "clicking in the AWS GUI".
The image below shows which setting I am referring to.
I have been browsing the docs for RDS CDK, and tried to Google this without success.
This guide describes the manual steps on how to do this.
AWS CDK RDS module
DatabaseCluster Construct
Low-level Cluster (CfnCluster)
BTW, my current current config looks a bit like this:
const cluster = new rds.DatabaseCluster(this, 'aurora-cluster', {
clusterIdentifier: 'aurora-cluster',
engine: rds.DatabaseClusterEngine.AURORA_MYSQL,
masterUser: {
username: 'someuser',
password: 'somepassword'
},
defaultDatabaseName: 'db',
instances: 2,
instanceIdentifierBase: 'aurora-',
instanceProps: {
instanceType: ...,
vpcSubnets: {
subnetType: ec2.SubnetType.PUBLIC,
},
vpc: myVpc
},
removalPolicy: cdk.RemovalPolicy.DESTROY,
parameterGroup: {
parameterGroupName: 'default.aurora-mysql5.7'
},
port: 3306,
storageEncrypted: true
});
Apparently Cloudformation doesn't support the certificate authority field, and therefore CDK can't either.
https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/211
I upvoted the issue; feel free to join me!
I am trying to create a VPC controlled Elastic Search Service on AWS. The problem is I keep getting the error when I run the following code: 'ValidationException: Before you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC'.
const AWS = require('aws-sdk');
AWS.config.update({region:'<aws-datacenter>'});
const accessPolicies = {
Statement: [{
Effect: "Allow",
Principal: {
AWS: "*"
},
Action: "es:*",
Resource: "arn:aws:es:<dc>:<accountid>:domain/<domain-name/*"
}]
};
const params = {
DomainName: '<domain>',
/* required */
AccessPolicies: JSON.stringify(accessPolicies),
AdvancedOptions: {
EBSEnabled: "true",
VolumeType: "io1",
VolumeSize: "100",
Iops: "1000"
},
EBSOptions: {
EBSEnabled: true,
Iops: 1000,
VolumeSize: 100,
VolumeType: "io1"
},
ElasticsearchClusterConfig: {
DedicatedMasterCount: 3,
DedicatedMasterEnabled: true,
DedicatedMasterType: "m4.large.elasticsearch",
InstanceCount: 2,
InstanceType: 'm4.xlarge.elasticsearch',
ZoneAwarenessEnabled: true
},
ElasticsearchVersion: '5.5',
SnapshotOptions: {
AutomatedSnapshotStartHour: 3
},
VPCOptions: {
SubnetIds: [
'<redacted>',
'<redacted>'
],
SecurityGroupIds: [
'<redacted>'
]
}
};
const es = new AWS.ES();
es.createElasticsearchDomain(params, function (err, data) {
if (err) {
console.log(err, err.stack); // an error occurred
} else {
console.log(JSON.stringify(data, null, 4)); // successful response
}
});
The problem is I get this error: ValidationException: Before you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC. I cannot seem to figure out how to create this service linked role for the elastic search service. In the aws.amazon.com IAM console I cannot select that service for a role. I believe it is supposed to be created automatically.
Has anybody ran into this or know the way to fix it?
The service-linked role can be created using the AWS CLI.
aws iam create-service-linked-role --aws-service-name opensearchservice.amazonaws.com
Previous answer: before the service was renamed, you would do the following:
aws iam create-service-linked-role --aws-service-name es.amazonaws.com
You can now create a service-linked role in a CloudFormation template, similar to the Terraform answer by #htaccess. See the documentation for the CloudFormation syntax for Service-Linked Roles for more details
YourRoleNameHere:
Type: 'AWS::IAM::ServiceLinkedRole'
Properties:
AWSServiceName: es.amazonaws.com
Description: 'Role for ES to access resources in my VPC'
For terraform users who hit this error, you can use the aws_iam_service_linked_role resource to create a service linked role for the ES service:
resource "aws_iam_service_linked_role" "es" {
aws_service_name = "es.amazonaws.com"
description = "Allows Amazon ES to manage AWS resources for a domain on your behalf."
}
This resource was added in Release 1.15.0 (April 18, 2018) of the AWS Provider.
Creating a elasticsearch domain with VPC and using aws-sdk/cloudformation is currently not supported. The elasticsearch service requires a special service linked role to create the network interfaces in the specified VPC. This currently possible using console / cli(#Oscar Barrett's answer below).
However, there is a workaround to get this working and it is described as follows:
Create a test elasticsearch domain with VPC access using console.
This will create a service linked role named AWSServiceRoleForAmazonElasticsearchService [Note: You can not create the role with specified name manually or through thr console]
Once this role is created, use aws-sdk or cloudformation to create elasticsearch domain with VPC.
You can delete the test elasticsearch domain later
Update: The more correct way to create the service role is described in #Oscar Barrett's answer. I was thinking to delete my answer; but the other facts about the actual issue is still more relevant, thus keeping my answer here.
Do it yourself in CDK:
const serviceLinkedRole = new cdk.CfnResource(this, "es-service-linked-role", {
type: "AWS::IAM::ServiceLinkedRole",
properties: {
AWSServiceName: "es.amazonaws.com",
Description: "Role for ES to access resources in my VPC"
}
});
const esDomain = new es.CfnDomain(this, "es", { ... });
esDomain.node.addDependency(serviceLinkedRole);