i want attaches a policy to an S3 bucket resource.
my terraform infra,
resource "aws_s3_bucket" "storage" {
bucket = "${var.service}-${local.stage}-storage"
acl = "public-read"
tags = {
Service = var.service
Stage = local.stage
}
cors_rule {
allowed_headers = [
"*"
]
allowed_methods = [
"GET",
"HEAD"
]
allowed_origins = [
"*"
]
max_age_seconds = 3000
}
}
this bucket for web static file hosting.i need bucket policy to public.
my policy in terraform,
resource "aws_s3_bucket_policy" "storage-policy" {
bucket = aws_s3_bucket.storage.id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "????????",
"Statement": [
{
"Sid": "????????",
"Effect": "Allow",
"Principal": "*",
"Action": "*",
"Resource": "arn:aws:s3:::BUCKET-NAME/*"
}
]
}
POLICY
}
in this code, i need get Id, Sid field value.
how can i get this?
thanks.
The Id and Sid are whatever you want. For example:
resource "aws_s3_bucket_policy" "storage-policy" {
bucket = aws_s3_bucket.storage.id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "my-bucket-polict",
"Statement": [
{
"Sid": "allow-all-access",
"Effect": "Allow",
"Principal": "*",
"Action": "*",
"Resource": "arn:aws:s3:::BUCKET-NAME/*"
}
]
}
POLICY
}
Related
I am trying to create an IAM role/policy to enable my EC2 instance to be able to list and attach EBS volumes (via scripts that call the aws cli). I want this policy to allow only listing/attaching of EBS volumes that have a specific tag.
I've noticed that the script is able to list/attach volumes when I set Resources: "*" and no Conditions in my policy below.
But as soon as I introduce the policy I have below, the AWS cli throws the following error:
./aws ec2 describe-volumes
An error occurred (UnauthorizedOperation) when calling the DescribeVolumes operation: You are not authorized to perform this operation.
Here is the IAM policy I have so far defined in terraform:
resource "aws_iam_role" "web_role" {
name = "web_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_instance_profile" "web_profile" {
name = "web_profile"
role = aws_iam_role.web_role.name
}
resource "aws_iam_role_policy" "web_disk_policy" {
name = "web_disk_policy"
role = aws_iam_role.web_role.id
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:DetachVolume",
"ec2:DescribeVolumes"
],
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*"
],
"Condition": {
"StringEquals": {
"ec2:ResourceTag/app": "web"
}
}
}
]
}
EOF
}
And my EC2 instances is created with the following:
resource "aws_instance" "web_vm" {
...
iam_instance_profile = aws_iam_instance_profile.web_profile.name
...
tags = {
app = "web"
}
}
And disk created with:
resource "aws_ebs_volume" "ebs-volume-1" {
availability_zone = "us-west-2a"
size = 10
tags = {
app = "web"
}
}
DescribeVolumes does not support aws:ResourceTag/${TagKey} condition, nor any other.
Most Describe/List type operations that target many resources aren't compatible with the conditional logic. DescribeVolumes does not work with conditions so split that off into a different statement.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:DetachVolume"
],
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*"
],
"Condition": {
"StringEquals": {
"ec2:ResourceTag/app": "web"
}
}
},
{
"Effect": "Allow",
"Action": "ec2:DescribeVolumes",
"Resource": "*"
}
]
}
Following on the back of this answer, I'm trying to create an aws_iam_role that allows for access to ECR. However, when I define the following:
resource "aws_iam_role" "jenkins_ecr_role" {
name = "JenkinsECRRole"
assume_role_policy = <<END_OF_POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecr.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
END_OF_POLICY
}
I get the error:
Error: Error creating IAM Role JenkinsECRRole:
MalformedPolicyDocument: Invalid principal in policy: "SERVICE":"*"
It looks like ecr.amazonaws.com is a valid principal according to the AWS documentation. What am I doing wrong?
It seems that this question has not been fully answered so I'll try to give an example on how I added ecr for my gitlab-runner ec2 instance. First off the ec2 instance needs a iam_instance_profile which it seems that you already have. Second: You need the assume-role so that the instance can assume the role of the service.
Notice am given access to all the resources not just an specific one(you can adjusted as needed)
The role
resource "aws_iam_role" "role" {
name = "${local.env}-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": ["ec2.amazonaws.com"]
},
"Action": "sts:AssumeRole"
}
]
}
EOF
}
The policy to allow access to ECR
resource "aws_iam_policy" "policy" {
name = "${local.env}-ecr-access-policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = [
"ecr:*",
]
Effect = "Allow"
Resource = "*"
},
]
})
}
The attachment of the policy to the role
resource "aws_iam_policy_attachment" "attach" {
name = "${local.env}-attach"
roles = ["${aws_iam_role.role.name}"]
policy_arn = "${aws_iam_policy.policy.arn}"
}
And finally the profile the EC2 will need
resource "aws_iam_instance_profile" "profile" {
name = "${local.env}-gitlab-runner-instance-profile"
role = aws_iam_role.role.name
}
The EC2 instance
resource "aws_instance" "ec2" {
ami = "ami-06c94f9acb4ba21b2"
instance_type = "t2.small"
associate_public_ip_address = true
key_name = "<key_name>"
subnet_id = <subnet>
iam_instance_profile = aws_iam_instance_profile.profile.name
vpc_security_group_ids = ["<security_group>"]
tags = {
Environment = "dev"
}
}
I think you are trying to give EC2 permission to access the ECR. For this, you need to create IAM policy for ECR and create IAM role for EC2 and attach the role with the policy.
Please refer to this code:
resource "aws_iam_role" "role" {
name = "test-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_policy" "policy" {
name = "test-policy"
description = "A test policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ecr:*",
"cloudtrail:LookupEvents"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "test-attach" {
role = "${aws_iam_role.role.name}"
policy_arn = "${aws_iam_policy.policy.arn}"
}
Hope, it helps.
You need to attach a below mentioned manual policy to your IAM role.
resource "aws_iam_role_policy" "role_policy" {
name = "${aws_iam_role.role.name}"
role = "${aws_iam_role.role.id}"
policy = <<EOF
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "new statement",
"Effect": "Allow",
"Principal": {
"AWS": "<IAM-ROLE-ARN>"
},
"Action": [
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:CompleteLayerUpload",
"ecr:GetDownloadUrlForLayer",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:UploadLayerPart"
]
}
]
}
EOF
}
Your role is created for a Service that will need the policy to use other resources, I'm not sure if this helps.
You need to create a role that something will be able to assume, ec2, sagemaker, s3, something. ECR does not assume a role as it's just a registry.
For example, I have a Sagemaker instance:
resource "aws_iam_role" "sagemaker_model" {
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "sagemaker.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
Then that instance needs permission to use other Resource (ECR):
resource "aws_iam_policy" "ecr" {
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ecr:*"
],
"Resource": [
"*"
]
}
]
}
EOF
}
Then I attach that policy to the previous role:
resource "aws_iam_role_policy_attachment" "model_attach_ecr" {
role = aws_iam_role.sagemaker_model.name
policy_arn = aws_iam_policy.ecr.arn
}
Although ECR has a specific property that it has its own access policy, you will need to allow that the previously created role can access the specific container registry by creating one aws_ecr_repository_policy:
resource "aws_ecr_repository_policy" "policy" {
repository = <aws_ecr_repository.repo.name>
policy = <<EOF
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "new statement",
"Effect": "Allow",
"Principal": {
"AWS": "${aws_iam_role.sagemaker_model.arn}"
},
"Action": [
"ecr:*"
]
}
]
}
EOF
}
In this one, you will need to replace <aws_ecr_repository.repo.name> by the actual repository name.
I hope this helps.
I'm confused as to how I should use terraform to connect Athena to my Glue Catalog database.
I use
resource "aws_glue_catalog_database" "catalog_database" {
name = "${var.glue_db_name}"
}
resource "aws_glue_crawler" "datalake_crawler" {
database_name = "${var.glue_db_name}"
name = "${var.crawler_name}"
role = "${aws_iam_role.crawler_iam_role.name}"
description = "${var.crawler_description}"
table_prefix = "${var.table_prefix}"
schedule = "${var.schedule}"
s3_target {
path = "s3://${var.data_bucket_name[0]}"
}
s3_target {
path = "s3://${var.data_bucket_name[1]}"
}
}
to create a Glue DB and the crawler to crawl an s3 bucket (here only two), but I don't know how I link the Athena query service to the Glue DB. In the terraform documentation for Athena, there doesn't appear to be a way to connect Athena to a Glue catalog but only to an S3 Bucket. Clearly, however, Athena can be integrated with Glue.
How can I terraform an Athena database to use my Glue catalog as its data source rather than an S3 bucket?
Our current basic setup for having Glue crawl one S3 bucket and create/update a table in a Glue DB, which can then be queried in Athena, looks like this:
Crawler role and role policy:
The assume_role_policy of the IAM role needs only Glue as principal
The IAM role policy allows actions for Glue, S3, and logs
The Glue actions and resources can probably be narrowed down to the ones really needed
The S3 actions are limited to those needed by the crawler
resource "aws_iam_role" "glue_crawler_role" {
name = "analytics_glue_crawler_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "glue.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_role_policy" "glue_crawler_role_policy" {
name = "analytics_glue_crawler_role_policy"
role = "${aws_iam_role.glue_crawler_role.id}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"glue:*",
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:GetBucketAcl",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::analytics-product-data",
"arn:aws:s3:::analytics-product-data/*",
]
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:*:*:/aws-glue/*"
]
}
]
}
EOF
}
S3 Bucket, Glue Database and Crawler:
resource "aws_s3_bucket" "product_bucket" {
bucket = "analytics-product-data"
acl = "private"
}
resource "aws_glue_catalog_database" "analytics_db" {
name = "inventory-analytics-db"
}
resource "aws_glue_crawler" "product_crawler" {
database_name = "${aws_glue_catalog_database.analytics_db.name}"
name = "analytics-product-crawler"
role = "${aws_iam_role.glue_crawler_role.arn}"
schedule = "cron(0 0 * * ? *)"
configuration = "{\"Version\": 1.0, \"CrawlerOutput\": { \"Partitions\": { \"AddOrUpdateBehavior\": \"InheritFromTable\" }, \"Tables\": {\"AddOrUpdateBehavior\": \"MergeNewColumns\" } } }"
schema_change_policy {
delete_behavior = "DELETE_FROM_DATABASE"
}
s3_target {
path = "s3://${aws_s3_bucket.product_bucket.bucket}/products"
}
}
I had many things wrong in my Terraform code. To start with:
The S3 bucket argument in the aws_athena_database code refers to the bucket for query output not the data the table should be built from.
I had set up my aws_glue_crawler to write to a Glue database rather than an Athena db. Indeed, as Martin suggested above, once correctly set up, Athena was able to see the tables in the Glue db.
I did not have the correct policies attached to my crawler. Initially, the only policy attached to the crawler role was
resource "aws_iam_role_policy_attachment" "crawler_attach" {
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
role = "${aws_iam_role.crawler_iam_role.name}"
}
after setting a second policy that explicitly allowed all S3 access to all of the buckets I wanted to crawl and attaching that policy to the same crawler role, the crawler ran and updated tables successfully.
The second policy:
resource "aws_iam_policy" "crawler_bucket_policy" {
name = "crawler_bucket_policy"
path = "/"
description = "Gives crawler access to buckets"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1553807998309",
"Action": "*",
"Effect": "Allow",
"Resource": "*"
},
{
"Sid": "Stmt1553808056033",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket0"
},
{
"Sid": "Stmt1553808078743",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket1"
},
{
"Sid": "Stmt1553808099644",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket2"
},
{
"Sid": "Stmt1553808114975",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket3"
},
{
"Sid": "Stmt1553808128211",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket4"
}
]
}
EOF
}
I'm confident that I can get away from hardcoding the bucket names in this policy but I don't yet know how to do that.
I would like to create a aws_iam_role with terraform but after running terraform applyI get the following error message:
aws_iam_role.role: Error Updating IAM Role (edb_eb_role) Assume Role Policy: MalformedPolicyDocument: Has prohibited field Resource
That is my policy:
resource "aws_iam_role" "role" {
name = "edb_eb_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
},
{
"Action": [
"logs:*"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"*"
]
}
]
}
EOF
}
What did I wrong? I also tried to do it only with Principals but then I get the message that "Principals" is also not prohibited?
Assume_role_policy don't accept the aws policy json files. So the above code is not working. For detailed explanation of assume_role_policy in aws_iam_role, see this thread.
Update the code as shown below and execute.
variable policy_arn{
default = "arn:aws:iam::aws:policy/service-role/AWSLambdaRole"
}
resource "aws_iam_role" "edb_role" {
name = "edb_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": ["ec2.amazonaws.com" ]
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "test-attach" {
role = "${aws_iam_role.edb_role.name}"
policy_arn = "${var.policy_arn}"
}
output "role" {
value = "${aws_iam_role.edb_role.name}"
}
Here, we are using the AWSLambdaRole Policy present in Policies section of IAM.
Add multiple policies to a role using aws_iam_role_policy_attach
Use the default policies provided by aws as show above. Else to create a new policy, see the docs here
Im trying to setup a redshift cluster and an IAM role that will have access to the cluster. I am using terraform for this. According the documentation I need to create a Service role and attach the AmazonS3ReadOnlyAccess policy to it. I have the following config in my terraform script:
resource "aws_iam_role" "my_admin_role" {
name = "my-role"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*",
"redshift:*"
],
"Resource": "*"
}
]
}
EOF
}
However this gives me an error:
Errors:
* aws_iam_role.my_admin_role: "assume_role_policy": required field is not set
* aws_iam_role.my_admin_role: : invalid or unknown key: policy
How do I setup a service role for redshift ?
You mix the resources aws_iam_role and aws_iam_role_policy
Sample usage of resource aws_iam_role
resource "aws_iam_role" "test_role" {
name = "test_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
Sample usage of resource aws_iam_role_policy
resource "aws_iam_role_policy" "test_policy" {
name = "test_policy"
role = "${aws_iam_role.test_role.id}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:Describe*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
EOF
}