Here is my policy which grants read/write access still not able to write into S3 bucket
Problem
Still getting below error:
Failed to upload /tmp/test.txt to bucketname/Automation_Result_2019-07-09 04:20:32_.csv: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ConsoleAccess",
"Effect": "Allow",
"Action": [
"s3:GetAccountPublicAccessBlock",
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicyStatus",
"s3:GetBucketPublicAccessBlock",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": [
"arn:aws:s3:::bucketname"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Action": "s3:*Object",
"Resource": [
"arn:aws:s3:::bucketname/*"
]
}
]
}
Bucket policy
{
"Version": "2012-10-17",
"Id": "MYBUCKETPOLICY",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
}
}
}
]
}
Python code (within Lambda function)
Relevant part of code
s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
target_bucket = 'bucket-name'
target_file = "Output/Automation_Result_"+EST+"_.txt"
s3.meta.client.upload_file('/tmp/test.txt', target_bucket, target_file, ExtraArgs={"ServerSideEncryption": "aws:kms", "SSEKMSKeyId":"XXXXXXX-XXXX-XXXX" })
This is how my bucket public access looks like!
It works fine for me!
I took your policy, renamed the bucket and attached it to a user as their only policy.
I was then able to successfully copy an object to and from the bucket.
If it is not working for you, then either you are not using the credentials that are associated with this policy, or there is another policy that is preventing the access, such as a Deny policy or a scope-limiting policy.
Related
I have two S3 buckets in two different regions on two different accounts. I want to use a S3 replication rule to replicate all files (including existing ones) from bucket-a to bucket-b.
bucket-a is an existing bucket with objects in it already, bucket-b is a new, empty bucket.
I created a replication rule and ran the batch operation job to replicate existing objects. After the job finished, 63% of objects failed to replicate, with the errors DstPutObjectNotPermitted or DstMultipartUploadNotPermitted and no further information. Nothing comes up on Google for these errors. (these are coming from the csv file that gets generated after job completion). The remaining objects got replicated as expected.
Here's my configuration:
bucket-a has versioning enabled and it is encrypted with a default aws-managed KMS key. ACL's are enabled, and this is the bucket policy:
{
"Version": "2008-10-17",
"Id": "NoBucketDelete",
"Statement": [
{
"Sid": "NoBucketDeleteStatement",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:DeleteBucket",
"Resource": "arn:aws:s3:::bucket-a"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-a/*",
"arn:aws:s3:::bucket-a"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
bucket-b also has versioning and ACL's enabled, and is encrypted with a customer-managed key.
The bucket policy is:
{
"Version": "2012-10-17",
"Id": "Policy1644945280205",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-b/*",
"arn:aws:s3:::bucket-b"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
},
{
"Sid": "Stmt1644945277847",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:role/bucket-replication-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateTags",
"s3:ObjectOwnerOverrideToBucketOwner",
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::bucket-b/*"
}
]
}
...and the KMS key policy is
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Allow access through S3 for all principals in the account that are authorized to use S3",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"kms:CallerAccount": "12345",
"kms:ViaService": "s3.us-west-2.amazonaws.com"
}
}
},
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::12345:user/root",
"arn:aws:iam::12345:user/user"
]
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:user/user"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:user/user"
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
]
}
I have a role in account-a, bucket-replication-role, with a trust relationship allowing S3 assume role and an attached policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
and an attached policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::bucket-b/*"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:us-east-1:12345:key/[account-a-kms-key-id]"
]
},
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Encrypt"
],
"Resource": [
"arn:aws:kms:us-west-2:12345:key/[account-b-kms-key-id]"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Resource": "arn:aws:s3:::bucket-b/*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetReplicationConfiguration",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket-a"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObjectVersionForReplication",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersionTagging"
],
"Resource": [
"arn:aws:s3:::bucket-a/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ReplicateObject",
"s3:ReplicateTags"
],
"Resource": "arn:aws:s3:::bucket-b/*"
}
]
}
Here is my replication rule, on bucket-a
The above role is attached as well, during creation.
and the batch operation is the default one that gets prompted on the replication rule creation.
The files are just small png's, jsons, html files, etc- nothing weird in there. You can see the replication status FAILED in the object information
Most of my policy rules came from this AWS support page: https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-replication/
Update
I added the following policy to account-b KMS key:
{
"Sid": "AllowS3ReplicationSourceRoleToUseTheKey",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::12345:role/bucket-replication-role"
},
"Action": ["kms:GenerateDataKey", "kms:Encrypt"],
"Resource": "*"
}
and the DstPutObjectNotPermitted errors have gone away, now its just the DstMultipartUploadNotPermitted errors I'm seeing.
Update 2
I tried to recreate the issue with two new buckets, and can not reproduce the issue, so I assume it's something to do with how some of the older files in bucket-a are stored.
This required some help from AWS Support, this was the relevant points of their response:
"DstMultipartUploadNotPermitted" status code indicates that the source objects are multipart uploads and the permissions required for their replication haven't been granted in the resource policies. Note that if a source object is uploaded using multipart upload to the source bucket, then the IAM replication role will also upload the replica object to destination bucket using multipart upload.
I would like to inform you that some extra permissions are to be granted for allowing multipart uploads in an S3 bucket. The list of permissions required for the IAM replication role to perform multipart uploads when KMS encryption is involved are listed below.
s3:PutObject on resource "arn:aws:s3:::DESTINATION-BUCKET/*"
kms:Decrypt and kms:GenerateDataKey on resource "arn:aws:kms:REGION:DESTINATION-ACCOUNT-ID:key/KEY-ID"
...as well as
ensure that the destination bucket policy is granting the "s3:PutObject" permission on resource "arn:aws:s3:::bucket-b/*" to the IAM replication role "arn:aws:iam::12345:role/bucket-replication-role".
...and finally
I would also request you to please grant "kms:Decrypt", and "kms:GenerateDataKey" permissions on the destination KMS key to the IAM replication role "arn:aws:iam::12345:role/bucket-replication-role" in the destination KMS key policy.
After adding all these additional permissions, everything worked as expected.
I would like to create an AWS policy to limit the s3:PutObject access on a path in a bucket.
Easy would you say, but:
I need to set the path with a regex MyBucket/*/Folder1/Folder1-1/Object
It's a cross-account access
I try to do this but it's not working.
On Source Account User policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::MyBucket",
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:GetObjectVersion",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::MyBucket/*",
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::MyBucket/*",
],
"Condition": {
"StringLike": {
"s3:prefix": "/*/Folder1/Folder1-1/*"
}
}
}
]
}
On Destination Account bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::MyAccountID:user/MyUser"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::MyBucket",
"Condition": {
"StringLike": {
"s3:prefix": "*/Folder1/Folder1-1/*"
}
}
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::MyAccountID:user/MyUser"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::MyBucket/*/Folder1/Folder1-1/*"
}
]
}
To clarify my Bucket structure :
|MyBucket
|-Client1
|-|-Folder1
|-|-|-Folder1-1
|-|-|-|-Object
|-Client1
|-|-Folder1
|-|-|-Folder1-1
|-|-|-|-Object
|-ClientXX
|-|-Folder1
|-|-|-Folder1-1
|-|-|-|-Object
I would like my user get PutObject access only not the path Client*/Folder1/Folder1-1/ could you please help me?
Wildcards are not supported in the middle of a string. However, you could use an IAM policy variable:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": ["s3:ListBucket"],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::mybucket"],
"Condition": {"StringLike": {"s3:prefix": ["${aws:username}/Folder1/Folder1-1/*"]}}
},
{
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::mybucket/${aws:username}/Folder1/Folder1-1/*"]
}
]
}
The ${aws:username} variable will insert the username of the user. This way, the wildcard is at the end of the string, which is valid.
This is a common way to allow multiple IAM Users to access the same bucket, but each only receives access to their folder within the bucket. This policy could be created on an IAM Group, and the IAM Group could then be assigned to each IAM User without the need to modify it for their particular folder.
I wanted to allow all s3 actions on a particular bucket "test-bucket" for a specific role "test-role". Deny the bucket for all others. The s3 policy I have written :
{
"Version": "2012-10-17",
"Id": "Policy1601973417173",
"Statement": [
{
"Sid": "Allow role test-role",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxx:role/test-role"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::test-bucket/*"
},
{
"Sid": "Deny rest",
"Effect": "Deny",
"NotPrincipal": {
"AWS": "arn:aws:iam::xxxxxxxx:role/test-role"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::test-bucket/*"
}
]
}
Even after applying the above policy, the sso users which are mapped to the role "test-role" is getting Access denied on the bucket.
Note : The AWS console shows logged in user as "Federated Login: test-role/sam#abc.com".
I have also tried the "assumed-role" options are still failing. Any help appreciated.
Try this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::test-bucket",
"arn:aws:s3:::test-bucket/*"
],
"Condition": {
"StringNotLike": {
"aws:userId": [
"AIDA<udserid-1-suppressed>:*",
"AIDA<udserid-1-suppressed>",
"AIDA<udserid-2-suppressed>:*",
"AIDA<udserid-2-suppressed>",
"AIDA<udserid-n-suppressed>:*",
"AIDA<udserid-n-suppressed>",
"111111111111"
]
}
}
}
]
}
I have a lambda function using a role with the following policy excerpt
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::ipwl-lambda-config/*",
"arn:aws:s3:::ipwl-lambda-config"
]
}
My bucket policy looks like the following
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::ipwl-lambda-config/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "AllowLambda",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountid:role/iam_for_lambda"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::ipwl-lambda-config/*",
"arn:aws:s3:::ipwl-lambda-config"
]
}
]
}
I've allowed GetObject and ListBucket on both the role and the bucket policy. However when my function runs
s3_obj = s3_res.Object(s3_bucket, s3_object)
I get
[ERROR] ClientError: An error occurred (AccessDenied) when calling the
GetObject operation: Access Denied
What more permissions do I have to add? The object is there, I can get it when I run the code locally using an admin role.
Update
I've checked to make sure the bucket and object names are correct dozens of times. The exception is actually coming from the second line here according to the stacktrace
s3_res = boto3.resource('s3')
s3_obj = s3_res.Object(s3_bucket, s3_object)
data = s3_obj.get()['Body'].read()
KMS should only be a factor for PutObject. We have a support account so I may check with them and update with their findings.
To download a KMS-encrypted object from S3, you not only need to be able to get the object. You also need to be able to decrypt the AWS KMS key.
Here's an example of an IAM policy that your Lambda function should have:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "s3get",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::ipwl-lambda-config/*"
},
{
"Sid": "kmsdecrypt",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:example-region-1:123456789012:key/example-key-id"
}
]
}
The key policy also needs to allow the IAM role to decrypt the key, something like this:
{
"Sid": "kmsdecrypt",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/xyz"
},
"Action": "kms:Decrypt",
"Resource": "*"
}
I want to restrict access to a S3 bucket to all roles except select few roles using S3 Bucket policy.but here while i am switching into my writer and reader role its access denied.
Bucket Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::581262627839:role/Rk-S3-Reader-I-Role"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::rkimpdocs"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::581262627839:role/Rk-S3-Writer-I-Role"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::rkimpdocs/*"
},
{
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::rkimpdocs",
"arn:aws:s3:::rkimpdocs/*"
],
"Condition": {
"ForAllValues:StringEquals": {
"aws:TagKeys": [
"JD",
"devops"
]
}
}
}
]
}
IAM Role Permission for writer role (Rk-S3-Writer-I-Role)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::rkimpdocs"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::rkimpdocs",
"arn:aws:s3:::rkimpdocs/*"
]
}
]
}
Output :
Access denied on both bucket policy and switching into mention role. Any help / suggestion would be helpful.
An explicit Deny will override any Allow. In your policies, the Deny in the bucket policy is causing the access denied. To give access to specific IAM roles and denying others, you should use "NotPrincipal" element. Please refer this blog which explains your exact use case.
Also, "aws:Tagkeys" condition is not supported in S3, so you have to omit that as well.