S3 website endpoint access - amazon-web-services

The s3 website endpoint docs say:
"In order for your customers to access content at the website
endpoint, you must make all your content publicly readable."
Does this mean:
Configuring a bucket for website hosting automatically makes all of
that bucket's content readable via the website endpoint (regardless
of per object permissioning)
OR
Per object permissioning can prevent specific content from being accessible via the website endpoint while other content is accessible via the website endpoint.
or some other explanation I'm not thinking of.

Simply enabling the web site hosting feature does not implicitly make the entire bucket public.
The web site endpoint for the bucket does not support accessing private objects using pre-signed URLs or the Authorization header... so if you want to make objects accessible, you have to do it explicitly at either the object or bucket level, using ACLs or policy statements.
If you don't make them accessible, they remain inaccessible.

Two aspects here because there are two different ways you can give permissions to bucket,
Bucket level permission (Permission for all files in the bucket)
Object permission (permission for each file inside the bucket)
If you want bucket level permission then you have to create a bucket policy and in that bucket policy if you gave permission to all (Principal: "*") then everybody can access all files in the bucket. Ideally, if you are hosting a static website in an S3 bucket then you should give full read-only permission to the public for all files. In this scenario, you can set a bucket level policy to give read-only permission to the public.
Object level permission you can set by access control list and you can give it for each object in the bucket. For example, if you have 10 files in the bucket and you have given only two files public access then only that file can be accessed by public not the remaining ones

Related

Should my bucket be public for my usecase and how should I avoid bad practice?

I'm new to AWS tools and although I have tried to search thoroughly for an answer I wasn't able to fixate on a solution.
My usecase is this:
I have a bucket where I need to store images, upload them via my server however I need to display them on my website.
Should my bucket be public?
If not, what should I do to allow everyone to read those images but not be able to mass upload on it from origins who are not my server?
If you want the images to be publicly accessible for your website, then the objects need to be public.
This can be done by creating a Bucket Policy that makes the whole bucket, or part of the bucket, publicly accessible.
Alternatively, when uploading the images, you can use ACL='public-read', which makes the individual objects public even if the bucket isn't public. This way, you can have more fine-grained control over what content in the bucket is public.
Both of these options require you to turn off portions of S3 Block Public Access to allow the Bucket Policy or ACLs.
When your server uploads to S3, it should be using Amazon S3 API calls using a set of AWS credentials (Access Key, Secret Key) from an IAM User. Grant the IAM User permission to put objects in the bucket. This way, that software can upload to the bucket totally independently to whether the bucket is public. (Never make a bucket publicly writable/uploadable, otherwise people can store anything in there without your control.)
upload them via my server however I need to display them on my website.
In that case only your server can upload the images. So if you are hosting your web app on EC2 or ECS, then you use instance role and task role to provide S3 write access.
Should my bucket be public?
It does not have to. Often CloudFront is used to host images or files from S3 using OAI. This way your bucket remains fully private.

How to access objects in S3 bucket, without making the object's folder public

I have provided AmazonS3FullAccess policy for both the IAM user and group. Also the buket that I am trying to access says "Objects can be public". I have explicitly made the folder inside the bucket public. Despite all this I am getting access denied error when I tried to access it through its url. Any idea on this?
Objects in Amazon S3 are private by default. This means that objects are not accessible by anonymous users.
You have granted permission for your IAM User to be able to access S3. Therefore, you have access to the objects but you must identify yourself to S3 so that it can verify your identity.
You should be able to access S3 content:
Via the Amazon S3 management console
Using the AWS CLI (eg aws s3 ls s3://bucketname)
Via authenticated requests in a web browser
I suspect that you have been accessing your bucket via an unauthenticated request (eg bucketname.s3.amazonaws.com/foo.txt. Unfortunately, this does not tell Amazon S3 who you are, so it will deny the request.
To access content with this type of URL, you can generate an Amazon S3 pre-signed URLs, which appends some authentication information to the URL to prove your identity. An easy way to generate the URL is with the AWS CLI:
aws s3 presign s3://bucketname/foo.txt
It will return a URL that looks like this:
https://bucketname.s3.amazonaws.com/foo.txt?AWSAccessKeyId=AKIAxxx&Signature=xxx&Expires=1608175109
The URL will be valid for one hour by default, up to 7 days.
There are two ways I will recommend.
go to s3 dashboard, and download the object you need, one by one manually, the bucket can be kept private at the same time.
build a gateway/a small service, to handle authentication for you, set a policy and give the permission to the service container/lambda to visit the private bucket, and restrict only specific users to download the objects.
References
download from aws s3
aws policy, permission and roles

Amazon S3 download authentication

I have created a bucket in Amazon S3 and have uploaded 2 files in it and made them public. I have the links through which I can access them from anywhere on the Internet. I now want to put some restriction on who can download the files. Can someone please help me with that. I did try the documentation, but got confused.
I want that at the time of download using the public link it should ask for some credentials or something to authenticate the user at that time. Is this possible?
By default, all objects in Amazon S3 are private. You can then add permissions so that people can access your objects. This can be done via:
Access Control List permissions on individual objects
A Bucket Policy
IAM Users and Groups
A Pre-Signed URL
As long as at least one of these methods is granting access, your users will be able to access the objects from Amazon S3.
1. Access Control List on individual objects
The Make Public option in the Amazon S3 management console will grant Open/Download permissions to all Internet users. This can be used to grant public access to specific objects.
2. Bucket Policy
A Bucket Policy can be used to grant access to a whole bucket or a portion of a bucket. It can also be used to specify limits to access. For example, a policy could make a specific directory within a bucket public to users from a specific range of IP addresses, during particular times of the day, and only when accessing the bucket via SSL.
A bucket policy is a good way to grant public access to many objects (eg a particular directory) without having to specify permissions on each individual object. This is commonly used for static websites served out of an S3 bucket.
3. IAM Users and Groups
This is similar to defining a Bucket Policy, but permissions are assigned to specific Users or Groups of users. Thus, only those users have permission to access the objects. Users must authenticate themselves when accessing the objects, so this is most commonly used when accessing objects via the AWS API, such as using the aws s3 commands from the AWS Command-Line Interface (CLI).
Rather than being prompted to authenticate, users must provide the authentication when making the API call. A simple way of doing this is to store user credentials in a local configuration file, which the CLI will automatically use when calling the S3 API.
4. Pre-Signed URL
A Pre-Signed URL can be used to grant access to S3 objects as a way of "overriding" access controls. A normally private object can be accessed via a URL by appending an expiry time and signature. This is a great way to serve private content without requiring a web server.
Typically, an application constructs a Pre-Signed URL when it wishes to grant access to an object. For example, let's say you have a photo-sharing website and a user has authenticated to your website. You now wish to display their pictures in a web page. The pictures are normally private, but your application can generate Pre-Signed URLs that grant them temporary access to the pictures. The Pre-Signed URL will expire after a particular date/time.
Regarding the pre-signed URL, the signature is in the request headers, hence it should be within HTTPS/TLS encryption. But do check for yourself.

Amazon S3 see private files

I am using Amazon S3 to upload files into different folders. All folders and files are public and can be seen by anyone. I created a private folder, where i want to put private images so that only i can see them. I already created a bucket policy rule that will deny the access to that folder. But how can i see the files ? Is there a special link like this https://s3.amazonaws.com/bucket/private_folder/file.jpg?secret_key=123 that will let me and someone who know`s that secret key to see the files ?
Is there any way of uploading private files that can be seen by using a secret_key, url or something like that ?
By default, all objects in Amazon S3 are private. Objects can then be made "public" by adding permissions, via one of:
Object Access Control List (ACL): Setting the permission directly on the object
Bucket Policy: Relates to the bucket, can define rules relating to sub-directories, key name (filenames), time-of-day, IP address, etc
IAM Policy: Relates to specific Users or Groups
As long as one of these methods grants access, the person will be able to access the object. It is also possible to assign Deny permissions that override Allow permissions.
When an object is being accessed via an un-authenticated URL (eg s3.amazonaws.com/bucket-name/object-key), the above rules determine access. However, even "private" files can be accessed if you authenticate against the service, such as calling an S3 API with your user credentials or using a pre-signed URL.
To see how this works, click a private file in the Amazon S3 Management Console, then choose Open from the Actions menu. The object will be opened. This is done by providing the browser with a pre-signed URL that includes a cryptographically-sized URL and a period of validity. The URL will work to Get the private file only until a defined time.
So, to answer your question, you can still access private files via:
The Open command in the console
Pre-Signed URLs in a web browser
Authenticated API calls
Just be careful that you don't define DENY rules that override even your ability to access files. It's easier to simply ALLOW the directories you'd like to be public.
See: Query String Request Authentication Alternative
You may access s3 private file by creating a temporary Url using temporaryUrl method.
$url = Storage::temporaryUrl(
'file.jpg', now()->addMinutes(5)
);
The params of temporaryUrl method are $path, $expiration and $option = []. The first two params are required and the default value of $option is an empty array.

Restrict Access to S3 bucket on AWS

I am storing files in a S3 bucket. I want the access to the files be restricted.
Currently, anyone with the URL to the file is able to access the file.
I want a behavior where file is accessed only when it is accessed through my application. The application is hosted on EC2.
Following are 2 possible ways I could find.
Use "referer" key in bucket policy.
Change "allowed origin" in CORS configuration
Which of the above 2 should be used, given the fact that 'referer' could be spoofed in the request header.
Also can cloudfront play a role over here?
I would recommend using a Pre-Signed URL that permits access to private objects stored on Amazon S3. It is a means of keeping objects secure, yet grant temporary access to a specific object.
It is created via a hash calculation based on the object path, expiry time and a shared Secret Access Key belonging to an account that has permission to access the Amazon S3 object. The result is a time-limited URL that grants access to the object. Once the expiry time passes, the URL does not return the object.
Start by removing existing permissions that grant access to these objects. Then generate Pre-Signed URLs to grant access to private content on a per-object basis, calculated every time you reference an S3 object. (Don't worry, it's fast to do!)
See documentation: Sample code in Java
When dealing with a private S3 bucket, you'll want to use an AWS SDK appropriate for your use case.
Here lies SDKs for many different languages: http://aws.amazon.com/tools/
Within each SDK, you can find sample calls to S3.
If you are trying to make private calls via browser-side JavaScript, you can use CORS.