I am planning to use S3 for my application. I want to grant permission for users of my application to access data stored in S3 for sometime and they will use GET on the objects stored.
Can somebody explain which one is preferred and why for my requirement ??
Thanks in advance.
UPDATE
Both IAM and temporary credentials can generate the signature for URL. Is there any advantage in using one than other ?? Is my understanding correct w.r.t temporary credentials ??
question is a bit misleading.
you have 3 things here:
1) IAM credentials
2) Temporary credentials via IAM STS
3) Presigned URL
If you do the presigned URL you can just provide the URL and the user can consume it before it expires.
It really does not matter if you are using permananent or temporary IAM credentials to generate the presigned URL as far as the URL is concerned.
If you are on an EC2 instance I would avocate for using an instance role and giving that the permissions needed to do the signing and let the sdk/cli handle it for you (it will under the hood use temporary credentials via IAM STS)
In my opinion, if you need to access to a S3 bucket from S3 Client for AWS SDK, I will use Temoporary credentials.
If you need to access to S3 bucket from a link of the webview, I will use Signed URLs.
Related
I have provided AmazonS3FullAccess policy for both the IAM user and group. Also the buket that I am trying to access says "Objects can be public". I have explicitly made the folder inside the bucket public. Despite all this I am getting access denied error when I tried to access it through its url. Any idea on this?
Objects in Amazon S3 are private by default. This means that objects are not accessible by anonymous users.
You have granted permission for your IAM User to be able to access S3. Therefore, you have access to the objects but you must identify yourself to S3 so that it can verify your identity.
You should be able to access S3 content:
Via the Amazon S3 management console
Using the AWS CLI (eg aws s3 ls s3://bucketname)
Via authenticated requests in a web browser
I suspect that you have been accessing your bucket via an unauthenticated request (eg bucketname.s3.amazonaws.com/foo.txt. Unfortunately, this does not tell Amazon S3 who you are, so it will deny the request.
To access content with this type of URL, you can generate an Amazon S3 pre-signed URLs, which appends some authentication information to the URL to prove your identity. An easy way to generate the URL is with the AWS CLI:
aws s3 presign s3://bucketname/foo.txt
It will return a URL that looks like this:
https://bucketname.s3.amazonaws.com/foo.txt?AWSAccessKeyId=AKIAxxx&Signature=xxx&Expires=1608175109
The URL will be valid for one hour by default, up to 7 days.
There are two ways I will recommend.
go to s3 dashboard, and download the object you need, one by one manually, the bucket can be kept private at the same time.
build a gateway/a small service, to handle authentication for you, set a policy and give the permission to the service container/lambda to visit the private bucket, and restrict only specific users to download the objects.
References
download from aws s3
aws policy, permission and roles
I need to validate client and then generate temporary credentials (valid for few seconds) using which client can upload a file on my S3 bucket. I cannot create a user for the client. First I validate the client using OAuth and if the client is valid, I need to enable it to upload the file to S3. I know about presigned URL way, but am wondering is there another way.
using which client can upload a file on my S3 bucket.
..
know about presigned URL way, but am wondering is there another way
As already answered, I see two ways. The presigned url or assumed IAM role (e. g. though cognito or own/custom identity broker)
There is a significant difference.
While using the presign url, it allows the client to upload/update a very specific object defined by the url in S3. I like this approach because of its simplicity, more control over expiration and I imho more secure (less work around managing permissions)
When using the assumed credentials, you may give the user more privileges (e. g. upload any object with specific prefix, tag the object, ,..). However you may have more work to manage the permission and control the expiration (default lifetime of the assumed role credentials is 15min and can be prolonged to 12h).
First I validate the client using OAuth
Still you may create a presigned url using the assumed (temporary) identity
You could use Amazon Cognito with OAuth. This will assume an IAM role and generate temporary credentials for you.
I'm not sure if this is the appropriate use case, so please tell me what to look for if I'm incorrect in my assumption of how to do this.
What I'm trying to do:
I have an s3 bucket with different 'packs' that users can download. Upon their purchase, they are given a user role in Wordpress. I have an S3 browser set up via php that makes requests to the bucket for info.
Based on their 'role', it will only show files that match prefix (whole pack users see all, single product people only see single product prefix).
In that way, the server will be sending the files on behalf of the user, and changing IAM roles based on the user's permission level. Do I have to have it set that way? Can I just analyze the WP role and specify and endpoint or query that notes the prefixes allowed?
Pack users see /
Individual users see /--prefix/
If that makes sense
Thanks in advance! I've never used AWS, so this is all new to me. :)
This sounds too complex. It's possible to do with AWS STS but it would be extremely fragile.
I presume you're hiding the actual S3 bucket from end users and are streaming through your php application? If so, it makes more sense to do any role-based filtering in the php application as you have far more logic available to you there - IAM is granular, but restrictions to resources in S3 is going to be funky and there's always a chance you'll get something wrong and expose the incorrect downloads.
Rather do this inside your app:
establish the role you've granted
issue the S3 ls command filtered by the role - i.e. if the role permits only --prefix, issue the ls command so that it only lists files matching --prefix
don't expose files in the bucket globally - only your app should have access to the S3 bucket - that way people also can't share links once they've downloaded a pack.
this has the added benefit of not encoding your S3 bucket structure in IAM, and keeps your decision logic isolated to code.
There are basically three ways you can grant access to private content in Amazon S3.
Option 1: IAM credentials
You can add a policy to an IAM User, so that they can access private content. However, such credentials should only be used by staff in your own organization. it should not be used to grant access to application users.
Option 2: Temporary credentials via STS
Your application can generate temporary credentials via the AWS Security Token Service. These credentials can be given specific permissions and are valid for a limited time period. This is ideal for granting mobile apps access to Amazon S3 because they can communicate directly with S3 without having to go via the back-end app. The credentials would only be granted access to resources they are permitted to use.
These types of credentials can also be used by web applications, where the web apps make calls directly to AWS services (eg from Node/JavaScript in the browser). However, this doesn't seem suitable for your WordPress situation.
Option 3: Pre-Signed URLs
Imagine a photo-sharing application where users can access their private photos, and users can also share photos with other users. When a user requests access to a particular photo (or when the back-end app is creating an HTML page that uses a photo), the app can generate a pre-signed URL that grants temporary access to an Amazon S3 object.
Each pre-signed URL gives access only to a single S3 object and only for a selected time period (eg 5 minutes). This means that all the permission logic for whether a user is entitled to access a file can be performed in the back-end application. When the back-end application provides a pre-signed URL to the user's browser, the user can access the content directly from Amazon S3 without going via the back-end.
See: Amazon S3 pre-signed URLs
Your situation sounds suitable for Option #3. Once you have determined that a user is permitted to access a particular file in S3, it can generate the pre-signed URL and include it as a link (or even in <img src=...> tags). The user can then download the file. There is no need to use IAM Roles in this process.
So I'm hosting an web app on EC2 which let users upload/download photos from S3. EC2 and S3 are in the same region.
Credentials: Following the official docs, credentials should be automatically supplied by the IAM role(with s3fullAccess permission) I created and associated to EC2.
Problem: User (from browser) can't download photos, but can successfully upload them. Download link was generated by aws-sdk's getSignedUrl() method.
Error Message:
InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
Firstly, access key seems correct as users can successfully upload stuffs.
My guess is that, since the user clicks link in the browser to download, there's no interaction between EC2 and S3, thus IAM role is not used when downloading. But isn't the signedDownloadUrl generated with EC2 credentials? And there's access key pairs supplied altogether in the query string. So my guess might be wrong..
Any suggestion/idea is appreciated.
Edit
I didn't create an IAM user, only have an IAM role. Don't know if this matters.
ASIA... is a temporary access key id from a set of temporary credentials for an IAM role. (Role credentials are always temporary.) AKIA... is an IAM user.
The problem here is that the x-amz-security-token accompanying the credentials was not used in the signing process that generated the S3 signed URL, so it doesn't appear in the signed URL... but it needs to be there for this to work.
If you are signing your request using temporary security credentials [...] you must include the corresponding security token in your request by adding the x-amz-security-token header.
http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#UsingTemporarySecurityCredentials
If you don't include x-amz-security-token, the system assumes it should look up the access-key-id in the IAM ussr database, and it won't be found there, which causes the InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records error.
Note that you can't simply add it to the already-generated URL. It needs to be incorporated into the signing process.
I'm kinda new to AWS S3 - using EC2 (hosting web app) and S3 (storing resources) in the same AWS region, and assigned EC2 with an IAM role s3access, so EC2 can download from S3 easily.
The question is, how should a client download from S3? Apparently the client doesn't have IAM role or Access Key like EC2 does. Seems the client only have a signedDownloadUrl generated by aws-sdk, but it also requires access key.
So, should I make the bucket public then any client can download, or should I find some approach to supply the client with credentials?
All objects by default are private. Only the object owner has permission to access these objects.
So if you want to share an object with someone you can
You can make it public or
You, the object owner can share objects with others by creating a pre-signed URL, using their own security credentials, to grant time-limited permission to download the objects.
For more details on pre-signed URLs refer S3 Share Objects with PreSignedURL