Related
I am trying to create a Glue job in an account that has full access to all resources
The script gets generated but when I try and save it I get the following error
[gluestudio-service.us-east-1.amazonaws.com] createJob: AccessDeniedException: Account XXXXX is denied access.
The role I am using has nothing fancy in it and has full access to glue and all S3 resources.
Its not an organisational account
Any insight would be greatly appreciated
I am using the managed AWSGlueServiceRole as well as the AmazonS3FullAccess roles
I have added the follwoing
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"glue:UseGlueStudio",
"iam:ListRoles",
"iam:ListUsers",
"iam:ListGroups",
"iam:ListRolePolicies",
"iam:GetRole",
"iam:GetRolePolicy",
"glue:SearchTables",
"glue:GetConnections",
"glue:GetJobs",
"glue:GetTables",
"glue:BatchStopJobRun",
"glue:GetSecurityConfigurations",
"glue:DeleteJob",
"glue:GetDatabases",
"glue:CreateConnection",
"glue:GetSchema",
"glue:GetTable",
"glue:GetMapping",
"glue:CreateJob",
"glue:DeleteConnection",
"glue:CreateScript",
"glue:UpdateConnection",
"glue:GetConnection",
"glue:StartJobRun",
"glue:GetJobRun",
"glue:UpdateJob",
"glue:GetPlan",
"glue:GetJobRuns",
"glue:GetTags",
"glue:GetJob"
],
"Resource": "*"
},
{
"Action": [
"iam:PassRole"
],
"Effect": "Allow",
"Resource": "arn:aws:iam::*:role/AWSGlueServiceRole*",
"Condition": {
"StringLike": {
"iam:PassedToService": [
"glue.amazonaws.com"
]
}
}
}
]
}```
Add Trusted Policy like the following in the Role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "glue.amazonaws.com",
"AWS": "arn:aws:iam::123456789:root"
},
"Action": "sts:AssumeRole"
}
]
}
I have two S3 buckets in two different regions on two different accounts. I want to use a S3 replication rule to replicate all files (including existing ones) from bucket-a to bucket-b.
bucket-a is an existing bucket with objects in it already, bucket-b is a new, empty bucket.
I created a replication rule and ran the batch operation job to replicate existing objects. After the job finished, 63% of objects failed to replicate, with the errors DstPutObjectNotPermitted or DstMultipartUploadNotPermitted and no further information. Nothing comes up on Google for these errors. (these are coming from the csv file that gets generated after job completion). The remaining objects got replicated as expected.
Here's my configuration:
bucket-a has versioning enabled and it is encrypted with a default aws-managed KMS key. ACL's are enabled, and this is the bucket policy:
{
"Version": "2008-10-17",
"Id": "NoBucketDelete",
"Statement": [
{
"Sid": "NoBucketDeleteStatement",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:DeleteBucket",
"Resource": "arn:aws:s3:::bucket-a"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-a/*",
"arn:aws:s3:::bucket-a"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
bucket-b also has versioning and ACL's enabled, and is encrypted with a customer-managed key.
The bucket policy is:
{
"Version": "2012-10-17",
"Id": "Policy1644945280205",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-b/*",
"arn:aws:s3:::bucket-b"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
},
{
"Sid": "Stmt1644945277847",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:role/bucket-replication-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateTags",
"s3:ObjectOwnerOverrideToBucketOwner",
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::bucket-b/*"
}
]
}
...and the KMS key policy is
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Allow access through S3 for all principals in the account that are authorized to use S3",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"kms:CallerAccount": "12345",
"kms:ViaService": "s3.us-west-2.amazonaws.com"
}
}
},
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::12345:user/root",
"arn:aws:iam::12345:user/user"
]
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:user/user"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:user/user"
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
]
}
I have a role in account-a, bucket-replication-role, with a trust relationship allowing S3 assume role and an attached policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
and an attached policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::bucket-b/*"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:us-east-1:12345:key/[account-a-kms-key-id]"
]
},
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Encrypt"
],
"Resource": [
"arn:aws:kms:us-west-2:12345:key/[account-b-kms-key-id]"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Resource": "arn:aws:s3:::bucket-b/*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetReplicationConfiguration",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket-a"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObjectVersionForReplication",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersionTagging"
],
"Resource": [
"arn:aws:s3:::bucket-a/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ReplicateObject",
"s3:ReplicateTags"
],
"Resource": "arn:aws:s3:::bucket-b/*"
}
]
}
Here is my replication rule, on bucket-a
The above role is attached as well, during creation.
and the batch operation is the default one that gets prompted on the replication rule creation.
The files are just small png's, jsons, html files, etc- nothing weird in there. You can see the replication status FAILED in the object information
Most of my policy rules came from this AWS support page: https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-replication/
Update
I added the following policy to account-b KMS key:
{
"Sid": "AllowS3ReplicationSourceRoleToUseTheKey",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:role/bucket-replication-role"
},
"Action": ["kms:GenerateDataKey", "kms:Encrypt"],
"Resource": "*"
}
and the DstPutObjectNotPermitted errors have gone away, now its just the DstMultipartUploadNotPermitted errors I'm seeing.
Update 2
I tried to recreate the issue with two new buckets, and can not reproduce the issue, so I assume it's something to do with how some of the older files in bucket-a are stored.
This required some help from AWS Support, this was the relevant points of their response:
"DstMultipartUploadNotPermitted" status code indicates that the source objects are multipart uploads and the permissions required for their replication haven't been granted in the resource policies. Note that if a source object is uploaded using multipart upload to the source bucket, then the IAM replication role will also upload the replica object to destination bucket using multipart upload.
I would like to inform you that some extra permissions are to be granted for allowing multipart uploads in an S3 bucket. The list of permissions required for the IAM replication role to perform multipart uploads when KMS encryption is involved are listed below.
s3:PutObject on resource "arn:aws:s3:::DESTINATION-BUCKET/*"
kms:Decrypt and kms:GenerateDataKey on resource "arn:aws:kms:REGION:DESTINATION-ACCOUNT-ID:key/KEY-ID"
...as well as
ensure that the destination bucket policy is granting the "s3:PutObject" permission on resource "arn:aws:s3:::bucket-b/*" to the IAM replication role "arn:aws:iam::12345:role/bucket-replication-role".
...and finally
I would also request you to please grant "kms:Decrypt", and "kms:GenerateDataKey" permissions on the destination KMS key to the IAM replication role "arn:aws:iam::12345:role/bucket-replication-role" in the destination KMS key policy.
After adding all these additional permissions, everything worked as expected.
I'll like my Iam policy used for a cross account to access just a single S3 bucket as in the example below but it fails with permission denied. The failure occurs when i switch to a cross account role on the console in AccountA and attempt to access the S3 bucket in accountB. However, I am able to view the S3 bucket in accountB when I change the "Resource" on the Iam policy to allow everything.
xx.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "mysid",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mybucket/*",
"arn:aws:s3:::mybucket/"
]
}
]
}
However, I am able to view the S3 bucket in accountB when I change the "Resource" on the Iam policy to allow everything. eg
xxx.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "mysid",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "*"
]
}
]
}
but this is not what i want.
other files used include:
xx.tpl
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "${Sid}",
"Effect": "${Effect}",
"Action": "${Action}",
"Resource": "${Resource}"
}
]
}
xx.tf
data "aws_iam_policy_document" "s3_write" {
count = length(var.s3_bucket_names)
statement {
actions = ["s3:PutObject", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:PutObjectAcl", "s3:List*", "s3:Get*", "s3:*"]
resources = ["arn:aws:s3:::${aws_s3_bucket.mybucket[count.index].id}/*", "arn:aws:s3:::${aws_s3_bucket.mybucket[count.index].id}"]
principals {
identifiers = var.principals
type = "AWS"
}
}
resource "aws_s3_bucket_policy" "s3_lb" {
count = length(var.s3_bucket_names)
bucket = aws_s3_bucket.mybucket[count.index].id
policy = data.aws_iam_policy_document.s3_write[count.index].json
}
s3 bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::xxxx:role/test-role1",
"arn:aws:iam::xxxx:role/test-role2",
"arn:aws:iam::xxxx:role/test-role3",
"arn:aws:iam::xxxx-other:role/s3-list-role"
]
},
"Action": [
"s3:PutObjectAcl",
"s3:PutObject",
"s3:List*",
"s3:Get*",
"s3:DeleteObjectVersion",
"s3:DeleteObject",
"s3:*"
],
"Resource": [
"arn:aws:s3:::mybucket/*",
"arn:aws:s3:::mybucket"
]
}
]
}
I changed this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "mysid",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mybucket/*",
"arn:aws:s3:::mybucket/"
]
}
]
}
To this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::mybucket"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::mybucket/*"
]
}
]
}
Giving my IAM user cross account access to both the console and CLI. The first allow statement is required for console cross account access.
If arn:aws:iam::xxxx:role/test-role1 has this policy attached, then a session with that role will get access to the bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "mysid",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mybucket/*",
"arn:aws:s3:::mybucket"
]
}
]
}
The S3 bucket policy grants this principal access. The problem is the trailing slash on the bucket ARN (second resource listed in the policy above.
Here is my policy which grants read/write access still not able to write into S3 bucket
Problem
Still getting below error:
Failed to upload /tmp/test.txt to bucketname/Automation_Result_2019-07-09 04:20:32_.csv: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ConsoleAccess",
"Effect": "Allow",
"Action": [
"s3:GetAccountPublicAccessBlock",
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicyStatus",
"s3:GetBucketPublicAccessBlock",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": [
"arn:aws:s3:::bucketname"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Action": "s3:*Object",
"Resource": [
"arn:aws:s3:::bucketname/*"
]
}
]
}
Bucket policy
{
"Version": "2012-10-17",
"Id": "MYBUCKETPOLICY",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
}
}
}
]
}
Python code (within Lambda function)
Relevant part of code
s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
target_bucket = 'bucket-name'
target_file = "Output/Automation_Result_"+EST+"_.txt"
s3.meta.client.upload_file('/tmp/test.txt', target_bucket, target_file, ExtraArgs={"ServerSideEncryption": "aws:kms", "SSEKMSKeyId":"XXXXXXX-XXXX-XXXX" })
This is how my bucket public access looks like!
It works fine for me!
I took your policy, renamed the bucket and attached it to a user as their only policy.
I was then able to successfully copy an object to and from the bucket.
If it is not working for you, then either you are not using the credentials that are associated with this policy, or there is another policy that is preventing the access, such as a Deny policy or a scope-limiting policy.
Referring to this Doc
I have created IAM policy which allows accessing only one EC2 Instance.And I have created an IAM user with the policy with that policy. But when I logged in with that user into my AWS account I got the Error "An error occurred fetching instance data: You are not authorized to perform this operation."
Policy document:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:*"
],
"Condition": {
"StringEquals": {
"ec2:ResourceTag/test": "test"
}
},
"Resource": [
"arn:aws:ec2:us-east-1:AccountNumber:instance/*
],
"Effect": "Allow"
}
]
}
You must add EC2 describe to describe all EC2 resources, then base on other statement to filter resource by tag.
But with this policy, other IAM account still viewable other EC2 instances without any permission.
Here is what you need.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1507182214000",
"Effect": "Allow",
"Action": [
"ec2:*"
],
"Condition": {
"StringEquals": {
"ec2:ResourceTag/TAG_NAME": "TAG_VALUE"
}
},
"Resource": [
"arn:aws:ec2:AWS_REGION:AWS_ACCOUNT:instance/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeTags"
],
"Resource": "*"
},
{
"Effect": "Deny",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": "*"
}
]
}