I've got an S3 bucket with a few files in. Public access disabled
I've also got an EC2 instance which I want to be able to access all files in the bucket.
I created a role with permissions like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::mybucketname"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::mybucketname/*"
]
}
]
}
I assigned the role to my EC2 instance, but I still get 403 forbidden if I try and access a file in the bucket from my EC2 instance.
Not sure what i've done wrong.
Thanks
When accessing private objects in an Amazon S3 bucket, it is necessary to provide authentication to prove that you are permitted to access the object.
It would appear that you are attempting to access the file without any authentication information, by simply accessing the URL: mybucket.s3.eu-west-2.amazonaws.com/myfile
If you wish to access an object this way, you can create an Amazon S3 pre-signed URL, which provides time-limited access to a private object in Amazon S3. It appends a 'signature' to the URL to prove that you are authorised to access it.
Alternatively, you could access the object via the AWS Command-Line Interface (CLI), or via the AWS SDK in your preferred programming language. This way, API requests will be authenticated against the S3 service.
Related
What is s3 bucket policy permission to provide an IAM user to access object url which is basically an HTTPs url for the object that i have uploaded to S3 bucket.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::bucket"
},
{
"Sid": "GetObject",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::bucket/*"
}
]
}
With above policy i can download the object into my local , but i cant access it with object url which includes Https link. If i keep the s3 bucket full public , only then i can have the https access to the object url.
I dont want to provide full public access and how to provide access to this with bucket policy?
You can get https url by generating s3 pre-signed urls for the objects. This will allow for temporary access using the urls generated.
Other than that, a common choice is to share your s3 objects with an outside world without making your bucket public using CloudFront as explained in:
Amazon S3 + Amazon CloudFront: A Match Made in the Cloud
Objects in Amazon S3 are private by default. They are not accessible via an anonymous URL.
If you want a specific IAM User to be able to access the bucket, then you can add permissions to the IAM User themselves. Then, when accessing the bucket, they will need to identify themselves and prove their identity. This is best done by making API calls to Amazon S3, which include authentication.
If you must access the private object via a URL, then you can create an Amazon S3 pre-signed URL, which is a time-limited URL that provides temporary access to a private object. This proves your ownership and will let S3 serve the content to you. A pre-signed URL can be generated with a couple of lines of code.
I wish to have a bucket that only one IAM user could access using the AWS Console, list its content and access object files inside it.
So, I have created the IAM user, the bucket itself, and later:
bucket policy as follow:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0000000:user/dave"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::testbucket1234"
},
{
"Sid": "statement2",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0000000:user/dave"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::testbucket1234/*"
}
]
}
And also a inline policy attached to my user's group, as follow:
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:*Object",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::testbucket1234/*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "*"
}
]
}
Now: I can list my buckets, access the desired bucket, list its content (so far so good). The problem is when I try to open one file object inside the bucket and I get "access denied" error. If I turn the object public, I can access it, but I can also access it using other IAM accounts, and that is not the intention. I want to access the bucket, list its contents and access objects only by usage of this specific IAM account. What am I doing wrong? How can I reach this goal? Thanks in advance.
By default, no IAM User can access any bucket. It is only by granting permissions to users that they can access resources.
However, many people tend to grant Amazon S3 permissions for all buckets, at least for Administrators. This then makes it difficult to remove permissions so that a bucket can only be accessed by one user. While it can be done with Deny policies, such policies are difficult to craft correctly.
For situations where specific data should only be accessed by one user, or a specific group of users (eg HR staff), I would recommend that you create a separate AWS Account and only grant permission to specific IAM Users or IAM Groups via a Bucket Policy (which works fine cross-account). This way, any generic policies that grant access to "all buckets" will not apply to buckets in this separate account.
Update: Accessing private objects
Expanding on what is mentioned in the comments below, a private object in Amazon S3 can be accessed by an authorized user. However, when accessing the object, it is necessary to identify who is accessing the object and their identity must be proved. This can be done in one of several ways:
In the Amazon S3 management console, use the Open command (in the Actions menu). This will open the object using a pre-signed URL that authorizes the access based upon the user who logged into the console. The same method is used for the Download option.
Using the AWS Command-Line Interface (CLI), you can download objects. The AWS CLI needs to be pre-configured with your IAM security credentials to prove your identity.
Programs using an AWS SDK can access S3 objects using their IAM security credentials. In fact, the AWS CLI is simply a Python program that uses the AWS SDK.
If you want to access the object via a URL, an application can generate an Amazon S3 pre-signed URLs. This URL includes the user's identity and a security signature that grants access to a private object for a limited period (eg 5 minutes). This method is commonly used when web applications want to grant access to a private object, such as a document or photo. The S3 management console actually uses this method when a user selects Actions/Open, so that the user can view a private object in their browser.
I have an s3 bucket that is used to host a static site that is accessed through cloudfront.
I wish to use the s3 <RoutingRules> to redirect any 404 to the root of the request hostname. To do this I need to set the cloudfront origin to use the s3 "website endpoint".
However, it appears that to allow Cloudfront to access the s3 bucket via the "website endpoint" and not the "s3 REST API endpoint", I need to explicitly make the bucket public, namely, with a policy rule like:
{
"Sid": "AllowPublicGetObject",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::dev.ts3.online-test/*"
},
{
"Sid": "AllowPublicListBucket",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::dev.ts3.online-test"
}
That's all well and good. It works. However AWS gives me a nice big shiny warning saying:
This bucket has public access. You have provided public access to this bucket. We highly recommend that you never grant any kind of public access to your S3 bucket.
So I have two questions I suppose:
Surely this warning should be caveated, and is just laziness on AWS' part? Everything in the bucket is static files that can be freely requested. There is no protected or secret content in the bucket. I don't see why giving public read is a security hole at all...
For peace of mind, is there any way to specify a principalId in the bucket policy that will only grant this to cloudfront? (I know if I use the REST endpoint I can set it to the OAI, but I can't use the rest endpoint)
The first thing about the warning.
The list buckets view shows whether your bucket is publicly accessible. Amazon S3 labels the permissions for a bucket as follows:
Public –
Everyone has access to one or more of the following: List objects, Write objects, Read and write permissions.
Objects can be public –::
The bucket is not public, but anyone with the appropriate permissions can grant public access to objects.
Buckets and objects not public –:
- The bucket and objects do not have any public access.
Only authorized users of this account –:
Access is isolated to IAM users and roles in this account and AWS service principals because there is a policy that grants public access.
So the warning due to first one. Recomended policy by AWS for s3 static website is below.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::example-bucket/*"
]
}
]
}
Add a bucket policy to the website bucket that grants everyone access
to the objects in the bucket. When you configure a bucket as a
website, you must make the objects that you want to serve publicly
readable. To do so, you write a bucket policy that grants everyone
s3:GetObject permission. The following example bucket policy grants
everyone access to the objects in the example-bucket bucket.
BTW public access should be only GET, not anything else, Its totally fine to allow GET request for your static website on S3.
static-website-hosting
I have a static website hosted on S3, I have set all files to be public.
Also, I have an EC2 instance with nginx that acts as a reverse proxy and can access the static website, so S3 plays the role of the origin.
What I would like to do now is set all files on S3 to be private, so that the website can only be accessed by traffic coming from the nginx (EC2).
So far I have tried the following. I have created and attached a new policy role to the EC2 instance with
Policies Granting Permission: AmazonS3ReadOnlyAccess
And have rebooted the EC2 instance.
I then created a policy in my S3 bucket console > Permissions > Bucket Policy
{
"Version": "xxxxx",
"Id": "xxxxxxx",
"Statement": [
{
"Sid": "xxxxxxx",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::XXXXXXXXXX:role/MyROLE"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::XXX-bucket/*"
}
]
}
As principal I have set the ARN I got when I created the role for the EC2 instance.
"Principal": {
"AWS": "arn:aws:iam::XXXXXXXXXX:role/MyROLE"
},
However, this does not work, any help is appreciated.
If the Amazon EC2 instance with nginx is merely making generic web requests to Amazon S3, then the question becomes how to identify requests coming from nginx as 'permitted', while rejecting all other requests.
One method is to use a VPC Endpoint for S3, which allows direct communication from a VPC to Amazon S3 (rather than going out an Internet Gateway).
A bucket policy can then restrict access to the bucket such that it can only be accessed via that endpoint.
Here is a bucket policy from Example Bucket Policies for VPC Endpoints for Amazon S3:
The following is an example of an S3 bucket policy that allows access to a specific bucket, examplebucket, only from the VPC endpoint with the ID vpce-1a2b3c4d. The policy uses the aws:sourceVpce condition key to restrict access to the specified VPC endpoint.
{
"Version": "2012-10-17",
"Id": "Policy",
"Statement": [
{
"Sid": "Access-to-specific-VPCE-only",
"Action": "s3:*",
"Effect": "Allow",
"Resource": ["arn:aws:s3:::examplebucket",
"arn:aws:s3:::examplebucket/*"],
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpce-1a2b3c4d"
}
},
"Principal": "*"
}
]
}
So, the complete design would be:
Object ACL: Private only (remove any current public permissions)
Bucket Policy: As above
IAM Role: Not needed
Route Table configured for VPC Endpoint
Permissions in Amazon S3 can be granted in several ways:
Directly on an object (known as an Access Control List or ACL)
Via a Bucket Policy (which applies to the whole bucket, or a directory)
To an IAM User/Group/Role
If any of the above grant access, then the object can be accessed publicly.
Your scenario requires the following configuration:
The ACL on each object should not permit public access
There should be no Bucket Policy
You should assign permissions in the Policy attached to the IAM Role
Whenever you have permissions relating to a User/Group/Role, it is better to assign the permission in IAM rather than on the Bucket. Use Bucket Policies for general access to all users.
The policy on the Role would be:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowBucketAccess",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::my-bucket/*"
]
}
]
}
This policy is directly applied to the IAM Role, so there is no need for a principal field.
Please note that this policy only allows GetObject -- it does not permit listing of buckets, uploading objects, etc.
You also mention that "I have set all files to be public". If you did this by making each individual object publicly readable, then anyone will still be able to access the objects. There are two ways to prevent this -- either remove the permissions from each object, or create a Bucket Policy with a Deny statement that stops access, but still permits the Role to get access.
That's starting to get a bit tricky and hard to maintain, so I'd recommend removing the permissions from each object. This can be done via the management console by editing the permissions on each object, or by using the AWS Command-Line Interface (CLI) with a command like:
aws s3 cp s3://my-bucket s3://my-bucket --recursive --acl private
This copies the files in-place but changes the access settings.
(I'm not 100% sure whether to use --acl private or --acl bucket-owner-full-control, so play around a bit.)
I have an EC2 instance attached with an IAM Role. That role has full s3 access. The aws cli work perfectly, and so does the meta-data curl check to get the temporary Access and Secret keys.
I have also read that when the Access and Secret keys are missing from the settings module, boto will automatically get the temporary keys from the meta-data url.
I however cannot access the css/js files stored on the bucket via the browser. When I add a bucket policy allowing a principal of *, everything works.
I tried the following policy:
{
"Version": "2012-10-17",
"Id": "PolicyNUM",
"Statement": [
{
"Sid": "StmtNUM",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::account-id:role/my-role"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
But all css/js are still getting 403's. What can I change to make it work?
Requests from your browser don't have the ability to send the required authz headers, which boto is handling for you elsewhere. The bucket policy cannot determine the principal and is correctly denying the request.
Add another sid to Allow principle * access to everything under /public, for instance.
The reason is that AWS is setting your files to binary/octet-stream.
check this solution to handle it.