Currently I have group with a policy that looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1449507915000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::artmakeit",
"arn:aws:s3:::artmakeit/*"
]
},
{
"Sid": "ListAllBuckets",
"Action": "s3:ListAllMyBuckets",
"Effect": "Allow",
"Resource": "*"
}
]
}
So, the IAM user, my client, can access his bucket correctly. But I am wondering if I can only let him see his bucket and not the complete list of my buckets. Is there a way to achieve this? I guess I should change the ListAllBuckets permission, but I don't know what to write instead, any ideas?
If users intend to access their buckets via the AWS Management Console, then they require the ListAllBuckets permission.
It is not possible to 'hide' the complete bucket list when using the console -- either they see them all or none at all.
Access via other methods (eg the AWS Command-Line Interface (CLI) or an API call) does not require that this permission be assigned.
Related
My objective is userone buckets shoud not show to other users:
s3:ListAllMyBucket
Returns a list of all buckets owned by the authenticated sender of the request. To use this operation, you must have the s3:ListAllMyBuckets permission.
This is my policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:ListAllMyBuckets"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
s3.ListAllMyBuckets is not working i don't know why?
If i misunderstand something please let me know
This Solution works but i need to know why s3:ListAllMyBuckets not working or if misunderstand something please let me know
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::prefix*"
]
}
]
}
So there's no concept of a 'bucket owner' in MinIO as there is in AWS S3. The s3:ListAllMyBuckets operation effectively grants access to the ListBuckets API operation.
For what you want, there are a few patterns you can explore:
Using prefixes in a bucket per user and configuring the resource as "arn:aws:s3:::${aws:username}"
Creating a bucket per-user and creating a corresponding policy for that user only granting access to that bucket
MinIO adopts S3's deny-by-default attitude, so as long as you explicitly state which resources a user has access to, the others will fall off on their own.
Be default our users have full S3 access via IAM, I have one bucket however that I need to limit access to one specific user, and block all other users.
I followed this guide here https://aws.amazon.com/premiumsupport/knowledge-center/explicit-deny-principal-elements-s3/
and made this bucket policy -
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::XXXXXXXXXXXX:user/USERWHONEEDSACCESS"
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::NAMEOFBUCKET/*"
},
{
"Sid": "",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::NAMEOFBUCKET/*",
"Condition": {
"StringNotLike": {
"aws:userid": "USERWHONEEDSACCESS:*"
}
}
}
]
}
However it no worky. Any suggestions?
You can try the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Principal": {
"AWS": [
"arn:aws:iam::XXXXXXXXXXXX:user/USERWHONEEDSACCESS"
]
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::nameofbucket/*",
"arn:aws:s3:::nameofbucket"
],
"Effect": "Allow"
},
{
"NotPrincipal": {
"AWS": [
"arn:aws:iam::XXXXXXXXXXXX:user/USERWHONEEDSACCESS"
]
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::nameofbucket/*",
"arn:aws:s3:::nameofbucket"
],
"Effect": "Deny"
}
]
}
In the How to Restrict Amazon S3 Bucket Access to a Specific IAM Role blog post you can read more about using NotPrincipal and restricting access to a single IAM User, specifically:
You can use the NotPrincipal element of an IAM or S3 bucket policy to limit resource access to a specific set of users. This element allows you to block all users who are not defined in its value array, even if they have an Allow in their own IAM user policies.
To generate this policy code snippet, I used this: https://asecure.cloud/a/s3_restrict_iam_user/ and I pre-filled the iamPrincipal and bucketName parameters with your example values.
While #Rigerta 's answer will work, I think it's worthy to explain why and how you can make your policy work
If you notice, in your policy you're specifying that only that user will be able to access all objects in your bucket
"Resource": "arn:aws:s3:::NAMEOFBUCKET/*"
However, the way IAM permissions work for S3 buckets is a bit tricky. Yes, that user has access to all objects and if he/she tries to push/pull an object via cli the operation will probably succeed, although via AWS console the bucket is unreachable. It's because the user has only access to the objects in the bucket, not the bucket itself
Therefore, you need to add the bucket to your resources. Changing
"Resource": "arn:aws:s3:::NAMEOFBUCKET/*"
by
"Resource": ["arn:aws:s3:::NAMEOFBUCKET/*", "arn:aws:s3:::NAMEOFBUCKET"]
should make it work.
You can check this blogpost for an example of an IAM policy for accessing a bucket. Notice how different actions are granted to different resources
Make sure that you are using an IAM unique identifier in your condition (it should start with the letters AIDA for IAM users).
"StringNotLike": {
"aws:userid": "AIDAXXXXXXXXXXXXX:*"
}
I suspect that you have written the username in your condition because you use the same placeholder as in the Principal. The IAM User Id is distinct from the username and the arn and cannot be found through the Console, but you can for example retrieve it with the aws cli get-user command.
In JupyterHub, installed in an EC2 instance with an IAM role which allows access to a specific S3 bucket when I try to access a file in that bucket with this code:
s3nRdd = spark.sparkContext.textFile("s3n://bucket/file")
I get this error:
IllegalArgumentException: u'AWS Access Key ID and Secret Access Key
must be specified as the username or password (respectively) of a s3n
URL, or by setting the fs.s3n.awsAccessKeyId or
fs.s3n.awsSecretAccessKey properties (respectively).'
However, when I export the AWS access key id and secret access key in the kernel configuration having the same policy as that role, the read for that file succeeds.
As the best practice is to use IAM roles, why doesn't the EC2 role work in this situation?
--update--
The EC2 IAM role has these 2 policies:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1488892557621",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/*"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "ec2:*",
"Effect": "Allow",
"Resource": "*"
},
{
"Sid": "Stmt1480684159000",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"*"
]
}
]
}
Also, I am using hadoop version 2.4.0 which doesn't support s3a protocol and updating is not an option.
S3n doesn't support IAM roles, and 2.4 is a very outdated version anyway. Not as buggy as 2.5 when it comes to s3n, but still less than perfect.
If you want to use IAM roles, you are going to have to switch to S3a, and yes, for you, that does mean upgrading Hadoop. sorry.
You must create a bucket policy to allow access from particular IAM roles. Since S3 doesn't trust the roles, the API just fallback and ask for access key.
Just add soemthing like this in your bucket policy, replace all the custom <> parameter with your own values.
{
"Version": "2012-10-17",
"Id": "EC2IAMaccesss",
"Statement": [{
"Sid": "MyAppIAMRolesAccess",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<acc_id>:role/<yourIAMroleName>"
]
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<yourbucket>/*",
"arn:aws:s3:::<yourbucket>"
]
}
]
}
(updates)
Make sure you give proper policy to the EC2 IAM Roles, because IAM roles is very powerful, no Policy is attach to it out of the box. You must assign a policy, e.g. for minimal S3 access, add AWSS3ReadOnly policy to the roles.
You may encounter issues of spark problematic interaction with IAM roles. Please check the documentation on spark access through s3n:// schema. Otherwise, use s3a://
I have configured my S3 bucket with Bucket Policy that looks like this
{
"Version": "2012-10-17",
"Id": "Policy100000000000",
"Statement": [
{
"Sid": "Stmt1463490591045",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucketname/*"
},
{
"Sid": "Stmt1463490591012",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::012345678900:user/user1",
"arn:aws:iam::012345678900:user/user2"
]
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucketname"
},
{
"Sid": "Stmt1463490660089",
"Effect": "Deny",
"NotPrincipal": {
"AWS": [
"arn:aws:iam::012345678900:user/user1",
"arn:aws:iam::012345678900:user/user2"
]
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucketname/*.xml"
}
]
}
The goal is to allow access to xml files in the bucket root to the selected users only. The rule doesn't seem to be working, since I get access denied
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>DE3DB1FF18B53997</RequestId><HostId>Iy+RnfkFKygJWkSTI0dXjssFsGFP2MydZZi/R5KBw5M8mZnfClt6HMOKJvAwy7sJgSx9BJQ3DbN=</HostId></Error>
I've tried fetching the xml files with AWS Node.js and Python SDKs and with aws-cli. I keep getting the same access denied message.
The AWS documentation regarding Bucket Policies is quite scattered around and has not provided me with a solution to the problem. There's very little documentation at all about using notPrincipal in the policy.
The ListBucket permission works all right, which means that the problem is specific to the rule, not the aim users.
The goal is to allow access to xml files in the bucket root to the selected users only
As per current documentation, s3 do not support file listing resource per postfix/filetype. It only support with prefix, so you would need to put a star without .xml at the end (which allow to access all objects at the folder layer), then you could implement logic to your app if you would allow to access the file or not.
For the bucket policy, by default, s3 policy would give access to user from the account (where the bucket created), as long as the IAM policy have the permission to do so. This is defined from ACL (Access Control List), go to S3 > Permission > Access Control List to check it out.[ AWS S3 ACL docs ]. So the first 2 statement might not be necessary in the statements. For the last statement, this might work but need an additional assumed-role ARN which will vary depending on what is defined for the role session name.
It is recommended to not use the NotPrincipal, and instead use the Condition key at the statement. Put the roleId as the userId at the StringNotLike statement to ignore the deny statement for the particular roleId. Also include the account number at the userId. Example as follows.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::MyExampleBucket",
"arn:aws:s3:::MyExampleBucket/*"
],
"Condition": {
"StringNotLike": {
"aws:userId": [
"ROLE_ID_HERE:*",
"ACCOUNT_NUMBER_HERE"
]
}
}
}
]
}
Check out on this AWS blog for more info:
https://aws.amazon.com/blogs/security/how-to-restrict-amazon-s3-bucket-access-to-a-specific-iam-role/
Your last deny policy simply doesn't talk about what should happen (allow or deny) to the requests with principal user1 or user2. When you send an s3 request as user1 or user2, the bucket policy won't have any effect (since it doesn't have any rule matching the principal user1 or user2 w.r.t the given action and the given resource).
The goal is to allow access to xml files in the bucket root to the selected users only
In this situation, you can mention a rule for explicitly allowing those users the access to your xml files.
{
"Sid": "Stmt1463490660089",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::012345678900:user/user1",
"arn:aws:iam::012345678900:user/user2"
]
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucketname/*.xml"
}
I have an S3 bucket that has in its policy permission for my CloudFront origin access identity:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <mine>"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<my-bucket>/*"
}
]
}
Additionally I've created a group and attached the AmazonS3FullAccess managed policy to it and added an IAM user to that group. The managed policy looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
}
]
}
However when that user tries to add anything to the bucket, I get a 403 (access denied). I'm unsure if any other operations work, I haven't written code to try them. It's only when I specifically allow that user's ARN access to the bucket directly in the bucket policy that they're allowed to add objects. What am I missing? It seems like the above group policy should allow members of that group access to all operations in all buckets, but it doesn't do that.
EDIT: After a whole bunch more poring over documentation, I think I've figured out that Amazon doesn't intend for groups to be used this way. They want you to use roles instead, which you can assign to an EC2 instance for automagic credential management. That's fine, but then I can't figure out how to test my code (using the aws-sdk Ruby gem) locally. Amazon says to define environment variables for your access ID and key - but what access ID and key? There's no such thing for a role...
Try replacing (in your policy):
"Resource": "arn:aws:s3:::<my-bucket>/*"
with:
"Resource": ["arn:aws:s3:::<my-bucket>", "arn:aws:s3:::<my-bucket>/*",]
and:
"Principal": {"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <mine>"},
with:
"Principal": {"AWS": "arn:aws:iam::ACCOUNT_ID:user/USERNAME"},