There's a specific bucket (not from my account) which I want to have access. The authors of the bucket have a github site here, and they made the data accessible. The problem comes when I try to aws s3 ls which I get the following error:
aws s3 \
--region eu-west-1 \
ls s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/
A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied
This is somewhat strange because the bucket has a Requester Pays policy meaning that access is limited to authenticated requests only, and I have a user with an access key, secret access key and my account/s3 buckets/etc ... is in same region (eu-west) than the bucket I'm trying to access.
So, I'm asking for help as I do not understand why I don't have access.
Thanks in advance for the help!
Your error says: A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied
This means that you have not been granted permission to list that bucket (or at least that path within the bucket).
Since the bucket is in a different AWS account, the only way these permissions can be granted to you is via an Amazon S3 bucket policy. Only the owner of the bucket can configure the bucket policy (or, more accurately, any user within that account who has the necessary permissions to edit the Bucket Policy).
(If it was in the same account, it would be possible to also use an IAM User policy to grant access.)
Alternatively, the bucket owners could create an IAM Role that has the necessary permissions and they could configure a Trust relationship to your normal IAM User. You would assume the role and then access the bucket. However, this would defeat the Requester Pays capabilities because they would be charged for such access.
This happens when weather bucket has requester pays policy and you don't include requester pays header explicitly in your request
Related
After creating an S3 bucket in AWS - I want to find out how permissions for Read and Write access to the S3 bucket should be provided?
I am thinking that IAM user permissions and S3 bucket permissions are different. What I am mean is - an IAM user with no permissions on AWS can be granted access to the S3 bucket by just adding him to the S3 buckets permissions for read or read+write accordingly. And S3 permissions have got nothing to do with IAM permissions>
Is the above understanding correct? Please correct and elaborate if the above is wrong?
And S3 permissions have got nothing to do with IAM permissions
This is not the case. The effective permission for the bucket and its objects are union of the IAM identity based permissions (i.e. those attached to IAM user,role, group) plus bucket policies plus other permissions (SCPs, IAM boundaries), which I don't mention for simplicity.
By default bucket and objects are private. You can Allow access to them using either IAM policies, bucket policies or both. Effective permissions will be the union of all these individual permissions. This means that you can use bucket policy to grant access to object1 and IAM policy to grant access to object2 to the same IAM user. The said user will be able to access both objects.
This behavior changes when there is Deny, as deny always wins. So an IAM user can have Allow in IAM policies to a given bucket, but the bucket can have Deny for that user. The result will be Deny always.
Subsequently, all these mean that bucket polices and IAM permissions should be considered together and are inter-connected with each other. Thus, you can't say that "S3 permissions have got nothing to do with IAM permissions".
An IAM User has no permissions by default. If they wish to do anything via an authenticated API call, they must be granted permission to do so.
Even if an Amazon S3 bucket policy grants access to an IAM User, they cannot access content in Amazon S3 unless their IAM permissions allow them to make the associated API call (eg ListBuckets or GetObject).
Please note that I am referring to an authenticated API call, not an Anonymous HTTP requests via a web browser URL.
A Bucket Policy can grant "public access" by referring to "Principal": "*" (which means that even unauthenticated users can retrieve content via a URL), or it can grant access to specific IAM Users and IAM Roles. However, if that user/role doesn't have permission to call any S3 Actions, then they cannot access the bucket.
A Deny policy from either IAM or a Bucket Policy takes precedence over Allow. So, an IAM User with permission to access a bucket can be denied by the Bucket Policy, and vice versa.
In AWS IAM is the place to generate all policies and to assign them to users and/or roles.
As a general rule, AWS recommends using S3 bucket policies or IAM policies for access control.
"S3 ACLs are a legacy access control mechanism that predates IAM. However, if you already use S3 ACLs and you find them sufficient,
there is no need to change."
By default, a user/role is assuming the policies provided by an SCP which stands for "service control policy". It is very similar to the policies you probably faced but this is provided by the account level/or organization level.
Among all options to generate and assume policies, there is a logical "and" which mean that in case there is at least 1 "deny" among the policies it will restrict access of a user/role to a specific bucket for instance in your case
I found an issue with a S3 bucket.
The bucket don't have any ACL associated, and the user that create the bucket was deleted.
How it's possible add some ACL in the bucket to get the control back?
For any command using AWS CLI, the result are the same always: An error occurred (AccessDenied) when calling the operation: Access Denied
Also in AWS console the access is denied.
First things first , AccessDenied error in AWS indicates that your AWS user does not have access to S3 service , Get S3 permission to your IAM user account , if in case you had access to AWS S3 service.
The thing is since you are using cli make sure AWS client KEY and secret are still correctly in local.
Now the interesting use case :
You have access to S3 service but cannot access the bucket since the bucket had some policies set
In this case if user who set the policies left and no user was able to access this bucket, the best way is to ask AWS root account holder to change the bucket permissions
An IAM user with the managed policy named AdministratorAccess should be able to access all S3 buckets within the same AWS account. Unless you have applied some unusual S3 bucket policy or ACL, in which case you might need to log in as the account's root user and modify that bucket policy or ACL.
See Why am I getting an "Access Denied" error from the S3 when I try to modify a bucket policy?
I just posted this on a related thread...
https://stackoverflow.com/a/73977525/999943
https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-owner-full-control-acl/
Basically when putting objects from the non-bucket owner, you need to set the acl at the same time.
--acl bucket-owner-full-control
I want to sync data between two s3 buckets.
The problem is that each one is owned by different AWS accounts (i.e. access key id and secret access key).
I tried to make the destination bucket publicly writable, but I still get
fatal error: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied
How to solve this?
I solved by giving permissions to write the destination bucket to the source bucket's AWS account.
I went to bucket "Permissions" tab of the destination bucket, "Access for other AWS accounts" and I gave permissions to the source bucket's AWS account by using the account email.
Then I copied the files by using AWS CLI (don't forget to grant full access to the recipient account!):
aws s3 cp s3://<source_bucket>/<folder_path>/ s3://<destination_bucket> --recursive --profile <source_AWSaccount_profile> --grants full=emailaddress=<destination_account_emailaddress>
I was trying few things with aws s3 bucket policy and the documentation for put-bucket-policy says that the user should have PutBucketPolicy on the bucket and should be the owner.
I do not understand the use of PutBucketPolicy permission then.
Also is the bucket owner given a default PutBucketPolicy permission on his bucket?
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTpolicy.html
The confusion here, I suspect, is related to the fact that users don't own buckets. The "owner" of a bucket is an individual AWS account.
You can't successfully grant PutBucketPolicy to any user in a different AWS account -- only your own account's user(s).
There's an illusion of circular logic here: How can I set a bucket policy... allowing myself to set the bucket policy... unless I am already able to set the bucket policy... which would make it unnecessary to set a bucket policy allowing me to set the bucket policy?
This is not as it seems: the problem is resolved by the fact that IAM user policies can grant a user permission to set the bucket policy, and the root account can do this by default -- which is why you should not use your root account credentials routinely: they are too privileged, if they fall into the wrong hands.
When a user has Resource-based permissions to a ressource but does not have User-based permissions for that service. Can he use that service than?
example : user Jack has Resource based permission to use the S3 bucket 'jamm'. But Jack has no permission to use S3. Can Jack use the S3 bucket?
If you don't have permissions to access the S3 service, then you cannot use it at all.
In order to access any S3 bucket, you must have permissions to execute the S3 commands such as s3:GetObject. These permissions tells AWS which commands the user is allowed to execute. Anything not explicitly allowed is automatically denied.
The S3 bucket policy (your resource-level permissions) instruct the S3 service which users are allowed to access the bucket. But that only happens after the user has been given the needed permissions to execute S3 commands with which to access the bucket.
So you need:
Give the user permissions to execute the S3 commands to access the bucket (default is none), and
Give the bucket a policy to restrict the users that can access the bucket (default is anyone in the AWS account)
It is possible to restrict some S3 commands to your bucket, so the user has permission to execute s3:GetObject (for example), but only on your bucket.
But some commands, such as s3:ListAllMyBuckets cannot be restricted this way.