I want to be able to publish test reports to S3 and have it accessible to the URL sent at the end of the Drone build.
Is it possible to have the S3 static site not view-able by anyone? So its only accessible by people who can already access resources in the VPC using a VPN.
I read that the content must have public read access, so checking if that is avoidable.
Yes:
Set up the static website as normal,
Add a VPC endpoint for S3,
Use a bucket policy to deny all but traffic from your VPC.
Here is a good article describing it in more detail: https://blog.monsterxx03.com/2017/08/19/build-private-staticwebsite-on-s3/
The other option is to write an S3 bucket policy like below, where x.x.x.x/x is the CIDR of the VPC:
{
"Id": "Policy1564215115240",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1564215036691",
"Action": "s3:*",
"Effect": "Deny",
"Resource": "arn:aws:s3:::<s3 bucket name>",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": "x.x.x.x/x"
}
},
"Principal": "*"
}
]
}
Related
I uploaded a static html site to s3 following this guideline: https://support.cloudflare.com/hc/en-us/articles/360037983412-Configuring-an-Amazon-Web-Services-static-site-to-use-Cloudflare
On s3 I created 2 bucket:
Root domain bucket: test1014.xyz (just a redirect to subdomain)
Subdomain bucket: www.test1014.xyz (contains the html file)
For the subdomain bucket, I blocked all public access and added a permission for cloudflare:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::www.test1014.xyz/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"173.245.48.0/20",
"103.21.244.0/22",
"103.22.200.0/22",
"103.31.4.0/22",
"141.101.64.0/18",
"108.162.192.0/18",
"190.93.240.0/20",
"188.114.96.0/20",
"197.234.240.0/22",
"198.41.128.0/17",
"162.158.0.0/15",
"104.16.0.0/13",
"104.24.0.0/14",
"172.64.0.0/13",
"131.0.72.0/22",
"2400:cb00::/32",
"2606:4700::/32",
"2803:f800::/32",
"2405:b500::/32",
"2405:8100::/32",
"2a06:98c0::/29",
"2c0f:f248::/32"
]
}
}
}
]
}
On cloudflare I added 2 domains:
CNAME | test1014.xyz | test1014.xyz.s3-website-ap-southeast-1.amazonaws.com
CNAME | www | www.test1014.xyz.s3-website-ap-southeast-1.amazonaws.com
Basically I just followed the guideline and still keep getting "This site can’t be reached ".
I already updated my domain nameserver to cloudflare.
Amazon S3 content is private by default.
The policy you show is a Deny policy. It is normally used in addition to an Allow policy to override the settings.
Therefore, you should either add an "Allow All" policy as well, or modify the Deny policy to be an Allow policy that grants access only if access via those IP addresses.
By the way, Deny policies are very difficult to get right. For example, it would also mean that YOU cannot access an object in S3 (eg to download a file) unless you do it via CloudFlare. They are best avoided if possible.
Update: Here's a bucket policy that only permits access (theoretically) to CloudFlare. It avoids using a Deny policy.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::www.test1014.xyz/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"173.245.48.0/20",
"103.21.244.0/22",
"103.22.200.0/22",
"103.31.4.0/22",
"141.101.64.0/18",
"108.162.192.0/18",
"190.93.240.0/20",
"188.114.96.0/20",
"197.234.240.0/22",
"198.41.128.0/17",
"162.158.0.0/15",
"104.16.0.0/13",
"104.24.0.0/14",
"172.64.0.0/13",
"131.0.72.0/22",
"2400:cb00::/32",
"2606:4700::/32",
"2803:f800::/32",
"2405:b500::/32",
"2405:8100::/32",
"2a06:98c0::/29",
"2c0f:f248::/32"
]
}
}
}
]
}
I have a simple website hosted on aws s3. And currently it is being accessible by anyone.
But I need to restrict the access.
So, I have an aws client vpn endpoint setup with the CIDR block of 10.3.0.0/22
So is it possible to give access only to anyone who's ONLY connected to VPN and restrict the access anything else.
We can restrict S3 access to certain ip range. Here are some examples.
Requests to S3 is allowed only when sourceIp falls under 10.3.0.0/22
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadForGetBucketObjects",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-s3-static-assets-bucket/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"10.3.0.0/22"
]
}
}
}
]
}
I will be using Cloudflare as a proxy for my S3 website bucket to make sure users can't directly access the website with the bucket URL.
I have an S3 bucket set up for static website hosting with my custom domain: www.mydomain.com and have uploaded my index.html file.
I have a CNAME record with www.mydomain.com -> www.mydomain.com.s3-website-us-west-1.amazonaws.com and Cloudflare Proxy enabled.
Issue: I am trying to apply a bucket policy to Deny access to my website bucket unless the request originates from a range of Cloudflare IP addresses. I am following the official AWS docs to do this, but every time I try to access my website, I get a Forbidden 403 AccessDenied error.
This is my bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CloudflareGetObject",
"Effect": "Deny",
"NotPrincipal": {
"AWS": [
"arn:aws:iam::ACCOUNT_ID:user/Administrator",
"arn:aws:iam::ACCOUNT_ID:root"
]
},
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::www.mydomain.com/*",
"arn:aws:s3:::www.mydomain.com"
],
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"2c0f:f248::/32",
"2a06:98c0::/29",
"2803:f800::/32",
"2606:4700::/32",
"2405:b500::/32",
"2405:8100::/32",
"2400:cb00::/32",
"198.41.128.0/17",
"197.234.240.0/22",
"190.93.240.0/20",
"188.114.96.0/20",
"173.245.48.0/20",
"172.64.0.0/13",
"162.158.0.0/15",
"141.101.64.0/18",
"131.0.72.0/22",
"108.162.192.0/18",
"104.16.0.0/12",
"103.31.4.0/22",
"103.22.200.0/22",
"103.21.244.0/22"
]
}
}
}
]
}
By default, AWS Deny all the request. Source
Your policy itself does not grant access to the Administrator [or any other user], it only omits him from the list of principals that are explicitly denied. To allow him access to the resource, another policy statement must explicitly allow access using "Effect": "Allow". Source
Now, we have to create Two Policy Statment's:- First with Allow and Second with Deny. Then, It is better to have only One Policy With "allow" only to Specific IP.
It is better not to complicate simple things like using Deny with Not Principal and NotIPAddress. Even AWS says :
Very few scenarios require the use of NotPrincipal, and we recommend that you explore other authorization options before you decide to use NotPrincipal. Source
Now, the questions come on how to whitelist Cloudflare IP's???.
Let's go with a simple approach. Below is the Policy. Replace your bucket name and your Cloudflare Ip's. I have tested it and it is running.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowCloudFlareIP",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:getObject",
"Resource": [
"arn:aws:s3:::my-poc-bucket",
"arn:aws:s3:::my-poc-bucket/*"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"IP1/32",
"IP2/32"
]
}
}
}
]
}
Trying to host submit form index.hmtl file in s3 bucket with out public access
{
"Version": "2008-10-17",
"Id": "permissionTohttps",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxx:root"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::test-2018/test/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "true"
}
}
}
]
}
Trying to host submit form index.hmtl file in s3 bucket with out public access
If you enable website hosting on an S3 bucket, it's going to be public.
It's not clear exactly how you want to restrict access to the web-form, but you could accomplish this by disabling website hosting on the bucket, then setting up an AWS CloudFront distribution that uses your S3 bucket as the origin (the source files for the distribution). You can then require signed URLs to access the CloudFront distribution (which would require you to have some other website that can issue signed URLs). You could alternatively use WAF rules (Web ACLs) that only allow requests from an IP range that you desire.
I am trying to apply the following policy in order to restrict my_bucket's access to a particular VPC.
When I try to apply this as a bucket policy, I get an Policy has an invalid condition key - ec2:Vpc.
How do I correct this?
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "*",
"Resource": "arn:aws:s3:::my_bucket/*",
"Condition":{
"StringNotEquals": {
"ec2:Vpc": "arn:aws:ec2:region:account:vpc/vpc-ccccccc"
}
}
}
]
}
I just got this to work. I had to do two things. 1) Create the bucket policy on the S3 bucket, 2) create a "VPC Endpoint"
My S3 bucket policy looks like this (of course put in your bucket name and VPC identifier):
{
"Version": "2012-10-17",
"Id": "Policy1234567890123",
"Statement": [
{
"Sid": "Stmt1234567890123",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::my_bucket/*",
"Condition": {
"StringEquals": {
"aws:sourceVpc": "vpc-12345678"
}
}
}
]
}
The S3 bucket also has some permissions outside the bucket policy to allow access from the AWS Console. Doing the above did not give access. To get access, I also had to go to AWS Console -> VPC -> Endpoints, and then create an endpoint. I attached the newly created endpoint to the only routing policy the account has at the moment (that has all subnets attached to it) and I used the default policy of
{
"Statement": [
{
"Action": "*",
"Effect": "Allow",
"Resource": "*",
"Principal": "*"
}
]
}
Once I created the endpoint, I was able to read from the S3 bucket from any EC2 instance in my VPC simply using wget with the right URL. I am still able to access the bucket from the AWS Console. But if I try to access the URL from outside the VPC, I get 403 forbidden. Thus, access to the S3 bucket is restricted to a single VPC, just like what you are looking for.
This is apparently a new feature. See this AWS blog entry for more information.
Two things that bit me and which might be helpful to add to Eddie's nice answer are:
First, you won't be able to view your bucket (or even modify its policy once you set the policy above) in the S3 AWS console unless you also give your AWS users permissions to manipulate the bucket. To do that, find your AWS account number (displayed in upper-right here), and add this statement to the bucket policy statements list:
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::YOUR_AWS_ACCOUNT_NUMBER:root"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::my_bucket",
"arn:aws:s3:::my_bucket/*"
]
},
Second, if you have more than one VPC, say vpc-XXXXXX and vpc-YYYYYY to give access to, the statement in Eddie's answer needs to be tweaked to something like the following (note the "Allow" "StringEquals" and list of sourceVpc values:
...
"Effect": "Allow",
...
"Condition": {
"StringEquals": {
"aws:sourceVpc": [
"vpc-XXXXXXXX",
"vpc-YYYYYYYY"
]
}
No, you can't do that.
Here's another person asking the same: https://forums.aws.amazon.com/thread.jspa?threadID=102387
Some have gotten overly creative with the problem trying to solve it with networking: https://pete.wtf/2012/05/01/how-to-setup-aws-s3-access-from-specific-ips/
I prefer a more simple route, S3 allows you to sign urls to solve this very problem, but inside of your VPC you may wish to not have to think about signing - or you just couldn't sign, for example you might be using wget, etc. So I wrote this little micro-service for that very reason: https://github.com/rmmeans/S3-Private-Downloader
Hope that helps!
UPDATED:
AWS now has a feature for VPC endpoints: https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/, you should use that and not what I previously suggested.