I'm trying to create elasticsearch cluster using terraform.
Using terraform 0.11.13
Please can someone point out why I'm not able to create log groups? What is the Resource Access Policy? is it the same as the data "aws_iam_policy_document" I'm creating?
Note: I'm using elasticsearch_version = "7.9"
code:
resource "aws_cloudwatch_log_group" "search_test_log_group" {
name = "/aws/aes/domains/test-es7/index-logs"
}
resource "aws_elasticsearch_domain" "amp_search_test_es7" {
domain_name = "es7"
elasticsearch_version = "7.9"
.....
log_publishing_options {
cloudwatch_log_group_arn = "${aws_cloudwatch_log_group.search_test_log_group.arn}"
log_type = "INDEX_SLOW_LOGS"
enabled = true
}
access_policies = "${data.aws_iam_policy_document.elasticsearch_policy.json}"
}
data "aws_iam_policy_document" "elasticsearch_policy" {
version = "2012-10-17"
statement {
effect = "Allow"
principals {
identifiers = ["*"]
type = "AWS"
}
actions = ["es:*"]
resources = ["arn:aws:es:us-east-1:xxx:domain/test_es7/*"]
}
statement {
effect = "Allow"
principals {
identifiers = ["es.amazonaws.com"]
type = "Service"
}
actions = [
"logs:PutLogEvents",
"logs:PutLogEventsBatch",
"logs:CreateLogStream",
]
resources = ["arn:aws:logs:*"]
}
}
I'm getting this error
aws_elasticsearch_domain.test_es7: Error creating ElasticSearch domain: ValidationException: The Resource Access Policy specified for the CloudWatch Logs log group /aws/aes/domains/test-es7/index-logs does not grant sufficient permissions for Amazon Elasticsearch Service to create a log stream. Please check the Resource Access Policy.
For ElasticSearch (ES) to be able to write to CloudWatch (CW) Logs, you have to provide a resource-based policy on your CW logs.
This is achieved using aws_cloudwatch_log_resource_policy which is missing from your code.
In fact, TF docs have a ready to use example of how to do it for ES, thus you should be able to just copy and paste it.
ES access policies are different from CW log policies, as they determine who can do what on your ES domain. Thus, you would have to adjust that part of your code to meet your requirements.
Related
I am very new to Terraform, so still finding my way around at the moment.
I am needing to add SES permissions to a Lambda function, for sending emails.
I thought it would be as simple as adding the DynamoDB permissions, but there seems to be a different format aws_ses_identity_policy instead of aws_iam_policy_attachment, and as a result, in the todo problem line, I can’t seem to just use .arn to link the policy to the Role.
Is there a different way of doing this? Am I looking at older versions of the library? Any help would be appreciated. Thanks.
### DynamoDB
…
resource "aws_iam_policy" "DynamoDBCrudPolicy" {
name = "DynamoDBCrudPolicy"
policy = data.aws_iam_policy_document.dynamodbPolicyDocument.json
}
### SES
data "aws_iam_policy_document" "sesPolicyDocument" {
statement {
actions = ["SES:SendEmail", "SES:SendRawEmail"]
resources = [aws_ses_domain_identity.SESPolicy.arn]
principals {
identifiers = ["*"]
type = "AWS"
}
}
}
resource "aws_ses_domain_identity" "SESPolicyDomainID" {
domain = "example.com"
}
resource "aws_ses_identity_policy" "SESPolicy" {
identity = aws_ses_domain_identity.SESPolicyDomainID.arn
name = "SESPolicy"
policy = data.aws_iam_policy_document.sesPolicyDocument.json
}
## Attach Policies to Role
resource "aws_iam_policy_attachment" "DynamoDBCrudPolicy_iam_policy_attachment" {
name = "DynamoDBCrudPolicy_iam_policy_attachment"
roles = [ aws_iam_role.DomainRole.name ]
policy_arn = aws_iam_policy.DynamoDBCrudPolicy.arn
}
resource "aws_iam_policy_attachment" "SES_iam_policy_attachment" {
name = "SESPolicy_iam_policy_attachment"
roles = [ aws_iam_role.DomainRole.name ]
# Todo problem here
policy_arn = aws_ses_identity_policy.SESPolicy.arn
}
I have a sagemaker model that will be deployed in different aws regions. This model will download models from an s3 bucket that is in region x. As long as the model is deployed in region x, the endpoint works.
However, when I deploy the sagemaker model from region y, it fails with the message.
Error: error creating SageMaker model: ValidationException: Could not
access model data at s3://mmmm/. Please ensure that the role
"arn:aws:iam::xxxx:role/dev-xxx-iam-role" exists and that its trust
relationship policy allows the action "sts:AssumeRole" for the service
principal "sagemaker.amazonaws.com". Also ensure that the role has
"s3:GetObject" permissions and that the object is located in region x.
My iam role permissions are as follows:
resource "aws_iam_policy_attachment" "sm_full_access_attach" {
name = "sm-full-access-attachment"
roles = [aws_iam_role.sagemaker_inferencer_iam_role.name]
policy_arn = "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
}
resource "aws_iam_policy_attachment" "s3_full_access_attach" {
name = "s3-full-access-attachment"
roles = [aws_iam_role.sagemaker_inferencer_iam_role.name]
policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
with the assume role policy of:
data "aws_iam_policy_document" "sm_assume_role_policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["sagemaker.amazonaws.com"]
}
}
}
The iam role is as follows:
resource "aws_iam_role" "sagemaker_inferencer_iam_role" {
name = "${var.app_environment}-inferencer-sm-${var.aws_region}-iam-role"
assume_role_policy = data.aws_iam_policy_document.sm_assume_role_policy.json
}
And the above works for same region buckets.
I can of course create have buckets in each region with replication rules from the original bucket. However, this is costly as the model files are huge. So I created a multi-region accesspoint for my original bucket with an alias zzz.mrap
However, when I specify the accesspoint alias in the aws_sagemaker_model resource as follows:
resource "aws_sagemaker_model" "sagemaker_multimodel" {
name = "${var.app_environment}-inferencer-sm-${var.aws_region}-model"
execution_role_arn = aws_iam_role.sagemaker_inferencer_iam_role.arn
primary_container {
image = local.multi_model_inferencer_container_name
mode = "MultiModel"
model_data_url = "s3://zzz.mrap/"
}
}
I get the following error:
Error: error creating SageMaker model: ValidationException: Could not
access model data at s3://zzzz.mrap/. Please ensure that the role
"arn:aws:iam::878435376106:role/dev-xxx-iam-role" exists and that its
trust relationship policy allows the action "sts:AssumeRole" for the
service principal "sagemaker.amazonaws.com". Also ensure that the role
has "s3:GetObject" permissions and that the object is located in
region x.
AWS says that I only have to replace the bucket name by the alias and that they support sagemaker, however this does not seem to be the case.
What am I doing wrong?
I want AWS instance that is allowed to read its own tags, but not of any other resources? Normally, idea of instance being allowed to do something is expressed by iam_role and aws_profile_instance, but when writing policy for the role, I can't refer to ARN of instance, since it creates loop.
It makes sense: normally, Terraform creates resources in order, and once created it never revisits them. What I want requires creating instance without iam role, and attach role to instance after instance is created.
Is it possible with Terraform?
EDIT: (minimal example):
+; cat problem.tf
resource "aws_instance" "problem" {
instance_type = "t2.medium"
ami = "ami-08d489468314a58df"
iam_instance_profile = aws_iam_instance_profile.problem.name
}
resource "aws_iam_policy" "problem" {
name = "problem"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{ Effect = "Allow"
Action = ["ssm:GetParameters"]
Resource = [aws_instance.problem.arn]
}
]
})
}
resource "aws_iam_role" "problem" {
name = "problem"
managed_policy_arns = [aws_iam_policy.problem.id]
# Copy-pasted from aws provider documentation. AWS is overcomplicated.
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "ec2.amazonaws.com"
}
},
]
})
}
resource "aws_iam_instance_profile" "problem" {
name = "problem"
role = aws_iam_role.problem.name
}
+; terraform apply -refresh=false
Acquiring state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
╷
│ Error: Cycle: aws_iam_instance_profile.problem, aws_instance.problem, aws_iam_policy.problem, aws_iam_role.problem
│
│
╵
The problem here arises because you've used the managed_policy_arns shorthand to attach the policy to the role in the same resource that declares the role. That shorthand can be convenient in simple cases, but it can also create cycle problems as you've seen here because it causes the role to refer to the policy, rather than the policy to refer to the role.
The good news is that you can avoid a cycle here by declaring that relationship in the opposite direction, either by using the separate aws_iam_policy_attachment resource type -- which only declares the connection between the role and the policy -- or by using aws_iam_role_policy to declare a policy that's directly attached to the role. You only really need the separate attachment if you intend to attach the same policy to multiple principals, so I'm going to show the simpler approach with aws_iam_role_policy here:
resource "aws_instance" "example" {
instance_type = "t2.medium"
ami = "ami-08d489468314a58df"
iam_instance_profile = aws_iam_instance_profile.example.name
}
resource "aws_iam_instance_profile" "example" {
name = "example"
role = aws_iam_role.example.name
}
resource "aws_iam_role" "example" {
name = "example"
# Allow the EC2 service to assume this role, so
# that the EC2 instance can act as it through its
# profile.
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "ec2.amazonaws.com"
}
},
]
})
}
resource "aws_iam_role_policy" "example" {
name = "example"
role = aws_iam_role.example.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = ["ssm:GetParameters"]
Resource = [aws_instance.example.arn]
},
]
})
}
Now all of the dependency edges go in the correct order to avoid a cycle.
The policy won't be connected to the role until both the role and the instance are both created, so it's important to consider here that the software running in the instance might start up before the role's policy is assigned, and so it should be prepared to encounter access violation errors for some time after boot and keep trying periodically until it succeeds, rather than aborting at the first error.
If this is part of a shared module that's using the functionality of the EC2 instance as part of the abstraction it's creating, it can help the caller of the module to be explicit about that hidden dependency on the aws_iam_role_policy by including it in any output values that refer to behavior of the EC2 instance that won't work until the role policy is ready. For example, if the EC2 instance is providing an HTTPS service on port 443 that won't work until the policy is active:
output "service_url" {
value = "https://${aws_instance.example.private_ip}/"
# Anything which refers to this output value
# should also wait until the role policy is
# created before taking any of its actions,
# even though Terraform can't "see" that
# dependency automatically.
depends_on = [aws_iam_role_policy.example]
}
My request seems simple and yet I can't realize it with Terraform.
I want to create a new AWS Policy based on SecurityAudit managed policy to which I want to add a condition
"Condition": {
"StringNotEquals": {
"aws:TagKeys/MyTag": "disabled"
}
}
I tried to use aws_iam_policy_document feature and then attached my policy to my role
data "aws_iam_policy_document" "security-audit-policy-override" {
statement {
principals {
type = "Federated"
identifiers = ["arn:aws:iam::aws:policy/SecurityAudit"]
}
condition {
test = "StringNotEquals"
values = ["aws:TagKeys/MyTag"]
variable = "disabled"
}
}
}
resource "aws_iam_role_policy" "security-audit-override" {
policy = data.aws_iam_policy_document.security-audit-policy-override.json
role = aws_iam_role.my_role.name
}
But I have the following mistake when I do a terraform apply command :
Error: Error putting IAM role policy terraform-XXXXXXXXX: MalformedPolicyDocument: Policy document should not specify a principal.
So, do you know how to override an existing managed AWS IAM Policy ?
Theoretically, you can do this, but not with SecurityAudit. This is because this policy has over 12000 characters. But user managed policies are limited to 6,144.
So you have to split SecurityAudit into two or three user managed policies yourself. The best way is to construct these policies manually, or trim SecurityAudit down significantly.
as i'm new with terraform, i'd like to ask your help once i got stuck for almost a day.
When trying to apply a IAC to deploy a Nginx service into a ECS(EC2 launch type) on aws i'm facing the following problem:
Error: Error creating IAM Role nginx-iam_role: MalformedPolicyDocument: Has prohibited field Resource status code: 400, request id: 0f1696f4-d86b-4ad1-ba3b-9453f3beff2b
I have already checked the documentation and the syntax is fine. What else could be wrong?
Following the snippet code creating the IAM infra:
provider "aws" {
region = "us-east-2"
}
data "aws_iam_policy_document" "nginx-doc-policy" {
statement {
sid = "1"
actions = [
"ec2:*"
]
resources = ["*"]
}
}
resource "aws_iam_role" "nginx-iam_role" {
name = "nginx-iam_role"
path = "/"
assume_role_policy = "${data.aws_iam_policy_document.nginx-doc-policy.json}"
}
resource "aws_iam_group_policy" "nginx-group-policy" {
name = "my_developer_policy"
group = "${aws_iam_group.nginx-iam-group.name}"
policy = "${data.aws_iam_policy_document.nginx-doc-policy.json}"
}
resource "aws_iam_group" "nginx-iam-group" {
name = "nginx-iam-group"
path = "/"
}
resource "aws_iam_user" "nginx-user" {
name = "nginx-user"
path = "/"
}
resource "aws_iam_user_group_membership" "nginx-membership" {
user = "${aws_iam_user.nginx-user.name}"
groups = ["${aws_iam_group.nginx-iam-group.name}"]
}
If you guys need the remaining code: https://github.com/atilasantos/iac-terraform-nginx.git
You are trying to use the aws_iam_policy_document.nginx-doc-policy policy as an assume_role_policy which does not work as an assume role policy needs to define a principal that you trust and want to grant access to assume the role you are creating.
An assume role policy could look like this is you want to grant access to the role to EC2 instances via instance profiles. At the end you can attach your initial role via a new resource as an inline policy to the role:
data "aws_iam_policy_document" "instance-assume-role-policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["ec2.amazonaws.com"]
}
}
}
resource "aws_iam_role" "nginx-iam_role" {
name = "nginx-iam_role"
path = "/"
assume_role_policy = data.aws_iam_policy_document.instance-assume-role-policy.json
}
resource "aws_iam_role_policy" "role_policy" {
name = "role policy"
role = aws_iam_role.nginx-iam_role.id
policy = data.aws_iam_policy_document.nginx-doc-policy.json
}
Instead of attaching the policy as an inline policies you can also create an IAM Policy and attach it to the various iam resources. (e.g.: aws_iam_policy and aws_iam_role_policy_attachment for roles.)
We created a bunch of open-source IAM modules (and others) to make IAM handling easier: Find them here on github. But there are more modules out there that you can try.