I'm trying to make a private bucket on AWS s3 and read and write to it from an Android App.
I found many of resources and questions but nothing of it gave me a complete answer.
I made a private one but don't know what should I write in Bucket Privacy, and what I need to access it from an Android app.
How can I do it?
More details:
My app is an e-commerce app has users, they upload photos to the app and view it from a website and from the app too.
All data on my server except photos, I want to upload the photos to a private bucket on s3 and view those by URLs saved on my server. Can I do it?
Restrict access to IAM role for S3 bucket. Use this role in Android App.
The normal architecture for this is:
Keep the Amazon S3 bucket private (no Bucket Policy)
Users of the Android app provide their login information to the app, which authenticates against your back-end service
Once authenticated, your back-end service can generate temporary credentials using AWS Security Token Service (STS) — permissions are assigned to these credentials that grant access to Amazon S3 (eg only for a certain path within a particular bucket)
The mobile app can then use these temporary credentials to directly communicate with Amazon S3 to upload/download objects
You could use Amazon Cognito for authentication and provisioning of credentials, or you could code your own authentication process (eg checking against your own database).
References:
Using Amazon Cognito for Mobile Apps - AWS Identity and Access Management
AWS IAM Now Supports Amazon, Facebook, and Google Identity Federation | AWS News Blog
About SAML 2.0-based Federation - AWS Identity and Access Management
If you only wish to view photos, then a simpler method would be for the back-end to generate Amazon S3 pre-signed URLs, which permit time-limited access to private objects in Amazon S3.
See: Amazon S3 pre-signed URLs
You can do this both by:
creating a bucket policy for specific IAM user
creating an ACL policy for an account and then delegating permissions to a specific user
Related
How to access, upload and delete objects of the S3 bucket from the web URL securely?
We are accessing the objects in S3 from our Application. But that bucket is public which is not secure.
I have tried CloudFront with OAI on the s3 bucket and putting bucket private but access is denied from the application when trying to upload an object to the s3 bucket.
We want to upload and delete objects in s3 bucket. We want that bucket to private only. And we want to do this from web applications only not from CLI, not from any tool. How could we achieve this?
Any suggestion will be appreciated.
Thanks!
Your application can use an AWS SDK to communicate directly with AWS services.
Your application will require a set of credentials to gain access to resources in your AWS Account. This can be done in one of two ways:
If your application is running on an Amazon EC2 instance, assign an IAM Role to the instance
Otherwise, create an IAM User and store the AWS credentials on the application server by using the AWS CLI aws configure command
You can control the exact permissions and access given to the IAM Role / IAM User. This can include granting the application permission to access your Amazon S3 buckets. This way, the buckets can be kept private, but the application will be able to upload/download/delete objects in the bucket.
To add more to the previous answer, you can find many S3 SDK examples in the AWS Github located here:
https://github.com/awsdocs/aws-doc-sdk-examples
If you look under each programming language, you will find Amazon S3 examples. You can use the AWS SDK to perform actions on a bucket when its private. You can take security a step further too and use encryption as shown here:
https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/java/example_code/s3/src/main/java/aws/example/s3/S3EncryptV2.java
Also, you can interact with Amazon S3 bucket from a web app as well by using the AWS SDK. Here is an example of building a web app using Spring Boot that interacts with an Amazon S3 bucket by reading all of the objects in the bucket.
Creating an example AWS photo analyzer application using the AWS SDK for Java
It's a bad practice to use long term credentials. AWS recommends to use short term credentials along with STS. Here is an article using Python/Flask to upload a file into private S3 bucket using STS/short term credentials.
Connect on-premise Python application with AWS services using Roles
I could have listed down all the steps in this post. But, it's a bit too long and so the reference to the above link.
I have provided AmazonS3FullAccess policy for both the IAM user and group. Also the buket that I am trying to access says "Objects can be public". I have explicitly made the folder inside the bucket public. Despite all this I am getting access denied error when I tried to access it through its url. Any idea on this?
Objects in Amazon S3 are private by default. This means that objects are not accessible by anonymous users.
You have granted permission for your IAM User to be able to access S3. Therefore, you have access to the objects but you must identify yourself to S3 so that it can verify your identity.
You should be able to access S3 content:
Via the Amazon S3 management console
Using the AWS CLI (eg aws s3 ls s3://bucketname)
Via authenticated requests in a web browser
I suspect that you have been accessing your bucket via an unauthenticated request (eg bucketname.s3.amazonaws.com/foo.txt. Unfortunately, this does not tell Amazon S3 who you are, so it will deny the request.
To access content with this type of URL, you can generate an Amazon S3 pre-signed URLs, which appends some authentication information to the URL to prove your identity. An easy way to generate the URL is with the AWS CLI:
aws s3 presign s3://bucketname/foo.txt
It will return a URL that looks like this:
https://bucketname.s3.amazonaws.com/foo.txt?AWSAccessKeyId=AKIAxxx&Signature=xxx&Expires=1608175109
The URL will be valid for one hour by default, up to 7 days.
There are two ways I will recommend.
go to s3 dashboard, and download the object you need, one by one manually, the bucket can be kept private at the same time.
build a gateway/a small service, to handle authentication for you, set a policy and give the permission to the service container/lambda to visit the private bucket, and restrict only specific users to download the objects.
References
download from aws s3
aws policy, permission and roles
Let's say I want to store some files for each user which is using my website on s3. Later I want authenticate each request to s3 to make sure that user has access to the files she is requesting. I guess this can't be done using presigned URLs or signed cookies(using cloud front). So which Amazon service should I use for that? What is the simplest way to achieve this?
Lets saying I'm authenticating users using jwt and its possible to recognize wheather a user has access to a file or not by the filename and content of the jwt.
I'm sorry that I don't have enough reputation to comment so I'll post an answer here.
One solution is:
AWS Cognito (Federated Identities)
S3 (one bucket)
S3 bucket policies allow you to restrict access to "user folders" equivalent here to "identity" by the prefix like yourbucket/<cognito_identity_id>/* Each user on your webpage will have its own federated identity.
When you create and configure the identity pool in AWS define a custom authentication provider and authenticate users "by the developer" in your backend.
Also, associate the authenticated identities to one IAM Role with access to the S3 bucket where you will keep the data. The bucket policy will take care of only allowing each user to their files and not to others. (See referenced links for policy example and more)
Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket
Access to User level folders using Amazon S3 and Cognito
Developer Authenticated Identities (Identity Pools)
I am developing a web application with two mobile(Android & iOs) based applications of the same. Currently the files uploaded are open to all, which in terms means that anyone with the direct image link can open it using a web browser.
How can I protect or limit the file access to the users of my mobile applications or web application ?
NB: As a beginner, I am not sure about the configuration details to be provided along with question, If I need to give more details on my s3 config. please specify it, I can add it to the s question to make the question more meaningful, so sorry for the inconvenience.
I think an easier approach than pre-signed urls would be to use Amazon Cognito to provide access to AWS resources to your trusted applications, even to unauthenticated users.
To do this you would create an Identity Pool for your application (just need one pool for all 3 of your clients) and then configure it so that when a client provides a valid Identity Pool Id they can assume an IAM role with permissions to access AWS resources.
Then you control what S3 bucket permissions the IAM role they assume would have - you could allow unauthenticated users access to read the S3 objects, or force them to create accounts to be able to read/write to S3 buckets (this is very easy with Cognito - users can sign up with facebook, gmail, their own email, etc.)
There's a step-by-step guide here for setting up an identity pool with Cognito, and then allowing unauthenticated users to assume an IAM role that can access the contents of an S3 bucket
The above causes the same set of permissions for all guest user accounts - that have assumed an IAM role through Amazon Cognito by identifying themselves as part of an identity pool.
edit: I should point out that if you authenticate via Cognito, you'll need to access the S3 bucket through the S3 Transfer Manager from the AWS SDK
AWS provides SDKs on iOS and Android to upload to their servers yet you don't want your key exposed if it was decompiled. How do you prevent that? Is there a way to sign a URL on the backend and give it to the client so they can upload directly onto S3 Without exposing the key?
You can use Amazon STS (Security Token Service) to generate unique sign-in credentials that also expire automatically.
You can also control permissions on these temporary credentials obtained via STS (for example to allow only upload permissions in a certain S3 bucket).