I have an app in play store that needs to download extra resources after installation. I thought that using Amazon s3 is a good idea to achieve it.(tell me if it is a bad idea)
Now, I would like to allow any logged in user through OpenID to access resources in the bucket.
Problem is;
I can't allow federated users to access items in bucket. What ever i have tried so far is the following access policy;
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReadObjects",
"Effect": "Allow",
"Principal": {"Federated": "accounts.google.com"},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mysuperbbucket/*"
}
]
}
but somehow AWS bucket policy editor raises an exception on it and saying that "Federated Principal Not Supported: The policy type does not support a federated provider in the principal element. Use a supported principal."
Alright, what to do now?
Related
I wish to have a bucket that only one IAM user could access using the AWS Console, list its content and access object files inside it.
So, I have created the IAM user, the bucket itself, and later:
bucket policy as follow:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0000000:user/dave"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::testbucket1234"
},
{
"Sid": "statement2",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0000000:user/dave"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::testbucket1234/*"
}
]
}
And also a inline policy attached to my user's group, as follow:
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:*Object",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::testbucket1234/*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "*"
}
]
}
Now: I can list my buckets, access the desired bucket, list its content (so far so good). The problem is when I try to open one file object inside the bucket and I get "access denied" error. If I turn the object public, I can access it, but I can also access it using other IAM accounts, and that is not the intention. I want to access the bucket, list its contents and access objects only by usage of this specific IAM account. What am I doing wrong? How can I reach this goal? Thanks in advance.
By default, no IAM User can access any bucket. It is only by granting permissions to users that they can access resources.
However, many people tend to grant Amazon S3 permissions for all buckets, at least for Administrators. This then makes it difficult to remove permissions so that a bucket can only be accessed by one user. While it can be done with Deny policies, such policies are difficult to craft correctly.
For situations where specific data should only be accessed by one user, or a specific group of users (eg HR staff), I would recommend that you create a separate AWS Account and only grant permission to specific IAM Users or IAM Groups via a Bucket Policy (which works fine cross-account). This way, any generic policies that grant access to "all buckets" will not apply to buckets in this separate account.
Update: Accessing private objects
Expanding on what is mentioned in the comments below, a private object in Amazon S3 can be accessed by an authorized user. However, when accessing the object, it is necessary to identify who is accessing the object and their identity must be proved. This can be done in one of several ways:
In the Amazon S3 management console, use the Open command (in the Actions menu). This will open the object using a pre-signed URL that authorizes the access based upon the user who logged into the console. The same method is used for the Download option.
Using the AWS Command-Line Interface (CLI), you can download objects. The AWS CLI needs to be pre-configured with your IAM security credentials to prove your identity.
Programs using an AWS SDK can access S3 objects using their IAM security credentials. In fact, the AWS CLI is simply a Python program that uses the AWS SDK.
If you want to access the object via a URL, an application can generate an Amazon S3 pre-signed URLs. This URL includes the user's identity and a security signature that grants access to a private object for a limited period (eg 5 minutes). This method is commonly used when web applications want to grant access to a private object, such as a document or photo. The S3 management console actually uses this method when a user selects Actions/Open, so that the user can view a private object in their browser.
How do I set the S3 read permission to only my IAM User can access? I have my IAM User config in the backend for my hybrid app but I still can't get access to S3 list.
Here is my bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxx:user/xxx#xxx.com"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-s3-bucket/*"
}
]
}
I tried was s3 ls s3://my-s3-bucket on terminal it is showing the list of items in my s3 bucket but not the backend. If I change the Principal to "*" I can access without any problem. (I want to limit to only my app can access it with the IAM User credential I have).
The error I have
{"code":"InternalError","message":"Access Denied"}
Thank you.
The aws s3 ls command is used to list the contents of a bucket but your policy is only granting permission to GetObject (which means to read the contents of an object).
If you wish to allow listing of the bucket, you would also need to grant s3:ListBucket permissions.
Bucket Policies vs IAM Policies
Typically, Bucket Policies are used to grant public or cross-account access.
If you wish to grant access to a specific IAM User, then it is better to add a policy on the IAM User themselves:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-s3-bucket/*"
}
]
}
However, you say "only one user". This becomes more difficult if any other users have been granted access to ALL S3 buckets with an Admin-like policy. In this situation, it you would need to add a Deny to the Bucket Policy to prevent access by anyone who has been granted access to all Buckets. This starts to get a little tricky because Deny policies have a habit of denying more than you expect.
If the bucket contains sensitive information, another option is to put the bucket in a different AWS Account and then only grant cross-account access to the specific IAM Users who need access. This prevents people gaining Admin-like access and avoids the need to use a Deny policy.
For s3 ls s3://my-s3-bucket to work you need s3:ListBucket permissions along with bucket resource:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxx:user/xxx#xxx.com"
},
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": ["arn:aws:s3:::my-s3-bucket/*",
"arn:aws:s3:::my-s3-bucket"]
}
]
}
we have the following scenario:
AWS Account A (application) writes data from an application to an S3 bucket owned by account B (data lake). The analysts in account C (reporting) want to proccess the data and build reports and dashboards on top of it.
Account A can write data to the data lake with --acl bucket-owner-full-control to allow Account B the access. But Account C still cannot see and process the data.
One (in our eyes bad) solution is to copy the data to the same location (overwrite) as account B, effectively taking ownership for the data in the process and eliminating the issue. We don't want it, because ... ugly
We tried assuming roles in the different accounts, but it does not work for all our infrastructure. E.g. S3 access via CLI or console is OK, but using it from EMR in account C does not. Also we have on-premise infrastructure (local taskrunners), where this mechanism is not an option.
Maintaining IAM roles for all accounts and users is too much effort. We aim for an automatic solution, not one that we have to take action every time a new user or account is added.
Do you have any suggestions?
One nice and clean way is to use a bucket policy granting read access to the external account (account C) by supplying the account ARN as the principal.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Grant read access to reporting account",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::insertReportingAccountIdHere:root"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::yourdatalakebucket",
"arn:aws:s3:::yourdatalakebucket/*"
]
}
]
}
This lets the reporting account manage the (ListBucket, gGtObject) permissions on the bucket for its own users, meaning you can now create an IAM policy on Account C with the permission to fetch data from the specified data lake bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow reading files from the data lake",
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::yourdatalakebucket",
"arn:aws:s3:::yourdatalakebucket/*"
]
}
]
}
This policy can then be attached to any Account C IAM role or user group you want. For example, you could attach it to your standard Developer or Analyst roles to give access to large groups of users, or you could attach it to a service role to give a particular service access to the bucket.
There is a guide on the Amazon S3 documentation site on how to do this.
You can do via the following documentation,
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_enable-console-saml.html
Steps:
Create SAML provider
Create Role for the SAML provider, example below
Assign the users role based on saml conditions
E.g., You can create S3 Readers, S3 Writers and assign permissions based on that.
Example Assume Role with SAML:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Federated": "arn:aws:iam::ACCOUNT-ID-WITHOUT-HYPHENS:saml-provider/ExampleOrgSSOProvider"},
"Action": "sts:AssumeRoleWithSAML",
"Condition": {"StringEquals": {
"saml:edupersonorgdn": "ExampleOrg",
"saml:aud": "https://signin.aws.amazon.com/saml"
}}
}]
}
Hope it helps.
In our case, we solved it using roles in the DataLake account (B), both for write (WriterRole) and read (ReaderRole) access. When writing to the DataLake from Account A, your writer assumes the "WriterRole" in Account B, that has the required permission. When reading from Account C, you assume the "ReaderRole".
The issues with EMR reading, we solved with EMRFS using IAM roles for reading (https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-emrfs-iam-roles.html)
In our environment, all IAM user accounts are assigned a customer-managed policy that grants read-only access to a lot of AWS services. Here's what I want to do:
Migrate a sql server 2012 express database from on-prem to a RDS instance
Limit access to the S3 bucket containing the database files
Here's the requirements according to AWS:
A S3 bucket to store the .bak database file
A role with access to the bucket
SQLSERVER_BACKUP_RESTORE option attached to RDS instance
So far, I've done the following:
Created a bucket under the name "test-bucket" (and uploaded the .bak file here)
Created a role under the name "rds-s3-role"
Created a policy under the name "rds-s3-policy" with these settings:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::test-bucket/"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObjectMetaData",
"s3:GetObject",
"s3:PutObject",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::test-bucket/*"
}
]
}
Assigned the policy to the role
Gave the AssumeRole permissions to the RDS service to assume the role created above
Created a new option group in RDS with the SQLSERVER_BACKUP_RESTORE option and linked it to my RDS instance
With no restrictions on my S3 bucket, I can perform the restore just fine; however, I can't find a solid way of restricting access to the bucket without hindering the RDS service from doing the restore.
In terms of my attempts to restrict access to the S3 bucket, I found a few posts online recommending using an explicit Deny statement to deny access to all types of principals and grant access based on some conditional statements.
Here's the contents of my bucket policy:
{
"Version": "2012-10-17",
"Id": "Policy1486769843194",
"Statement": [
{
"Sid": "Stmt1486769841856",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::test-bucket",
"arn:aws:s3:::test-bucket/*"
],
"Condition": {
"StringNotLike": {
"aws:userid": [
"<root_id>",
"<user1_userid>",
"<user2_userid>",
"<user3_userid>",
"<role_roleid>:*"
]
}
}
}
]
}
I can confirm the bucket policy does restrict access to only the IAM users that I specified, but I am not sure how it treats IAM roles. I used the :* syntax above per a document I found on the aws forums where the author stated the ":*" is a catch-all for every principal that assumes the specified role.
The only thing I'm having a problem with is, with this bucket policy in place, when I attempt to do the database restore, I get an access denied error. Has anyone ever done something like this? I've been going at it all day and haven't been able to find a working solution.
The following, admittedly, is guesswork... but reading between the lines of the somewhat difficult to navigate IAM documentation and elsewhere, and taking into account the way I originally interpreted it (incorrectly), I suspect that you are using the role's name rather than the role's ID in the policy.
Role IDs look similar to AWSAccessKeyIds except that they begin with AROA....
For the given role, find RoleId in the output from this:
$ aws iam get-role --role-name ROLE-NAME
https://aws.amazon.com/blogs/security/how-to-restrict-amazon-s3-bucket-access-to-a-specific-iam-role/
Use caution when creating a broad Deny policy. You can end up denying s3:PutBucketPolicy to yourself, which leaves you in a situation where your policy prevents you from changing the policy... in which case, your only recourse is presumably to persuade AWS support to remove the bucket policy. A safer configuration would be to use this to deny only the object-level permissions.
This is the policy I have:
{
"Version": "2012-10-17",
"Id": "Policy1477084949492",
"Statement": [
{
"Sid": "Stmt1477084932198",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::__redacted__"
},
{
"Sid": "Stmt1477084947291",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::__redacted__/*"
}
]
}
I am able to view the files in the bucket via aws s3 ls but am not able to download.
My understanding is that these permissions should give full access to any AWS identity.
Question- Is there some reason that is not the case here?
Your policy works for me when I test it in my account.
In IAM, a deny overwrites an allow, and I suspect that you have a conflicting policy somewhere. Check all user policies, and groups that the user is a member of for conflicting policies.
You don't explicitly say you are doing this, but just to cover all bases. If you are running the s3 get on an instance with an IAM Role associated with it, check to make sure that the IAM Roles permissions are appropriate.
Depending on what you are actually doing this could explain your situation. If you are using an EC2 instance with an IAM Role, it will be using that IAM Role for permissions by default not your IAM User permissions. If you run aws configure and explicitly configure it with IAM User issued key and secret then it will use the IAM User policies.
Best practices say that if you are performing work on an EC2 instance, where possible and where your use case allows for it; you should not be using keys and secrets on the host but using an EC2 IAM Role.
Additional Reading:
IAM Policy Evaluation Logic
http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html