I configured S3 Gateway VPC endpoint in my account and created a S3 bucket (Bucket-A) with no public access. I configured the below Bucket Policy in Bucket-A and tried to access a png file in that bucket using CURL and WGET from an EC2 instance and ended up with "HTTP/1.1 403 Forbidden" error. But call via AWS CLI works fine which proves that traffic via VPCEndpoint works fine. eg: aws s3 ls s3://Bucket-A.
Can you people help me to understand what configuration should be done additionally to make the CURL and WGET GET operation work.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::1234567890:root"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::Bucket-A/*",
"Condition": {
"StringEquals": {
"aws:sourceVpc": "vpc-xyzxyzxyz"
}
}
}
]
}
VPC Endpoint Policy:
{
"Statement": [
{
"Action": "*",
"Effect": "Allow",
"Resource": "*",
"Principal": "*"
}
]
}
Related
So I want my S3 bucket to be publicly accessible but only if the request is sent through the VPC endpoint. I allowed public access on both my bucket level and account level and also added the following statements to my bucket policy:
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"${media_storage_bucket_arn}",
"${media_storage_bucket_arn}/*"
]
},
{
"Sid": "Access-to-specific-VPCE-only",
"Effect": "Deny",
"Principal": {
"AWS": "arn:aws:iam::${current_account}:root"
},
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"${media_storage_bucket_arn}",
"${media_storage_bucket_arn}/*"
],
"Condition": {
"StringEquals": {
"aws:sourceVpce": "${vpc_endpoint}"
}
}
}
I have an EC2 server in a private subnet that needs to read images from an S3 bucket using a curl command and the object URL, so far, the easiest way to accomplish this would be to lift all the public access blocks, but this compromises the safety of the files in the bucket, so that is why I implemented the vpc endpoint statement to restrict access if the request is not sent through the endpoint, This works fine, but it still allows me to read any object in the bucket through its URL even if the request is not sent through the vpc endpoint. I'm sure there has to be an easier/better approach.
This question already exists:
Is there a way to curl an S3 bucket from an EC2 instance
Closed 5 months ago.
I am trying to send a curl request to an S3 bucket from my EC2 to retrieve a specific object within the bucket:
I want to create a transparent proxy with caching implemented by nginx so the aws cli wont work for this.
The EC2 instance (Linux machine) works as a proxy server with NGINX to send HTTP requests to the bucket for caching purposes, I do not have an SSL cert on this instance.
The bucket only contains images.
The curl request looks like this:
curl my-bucket.s3.eu-west-1.amazonaws.com/1450/1349/5467_1012.jpg
But I get an Access Denied error
I have attached a full read access policy to my EC2 instance role.
Here is my bucket policy:
{
"Version": "2012-10-17",
"Id": "MediaStorageBucketPolicy",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${current_account}:root"
},
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"${media_storage_bucket_arn}",
"${media_storage_bucket_arn}/*"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${current_account}:root"
},
"Action": [
"s3:PutObject"
],
"Resource": [
"${media_storage_bucket_arn}",
"${media_storage_bucket_arn}/*"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:I am::<account number>:role/ssm-ec2-service-role"
]
},
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"${media_storage_bucket_arn}",
"${media_storage_bucket_arn}/*"
]
}
]
}
Can this be achieved without making the bucket public?
The solution I found was creating an IAM user with S3 read access that can then be used within the EC2 (credentials of the user) to pull data from the bucket, see the following doc:
http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
I am using S3 VPC endpoint (type: gateway). I've wanted to restrict access to S3 resources from my private subnet only to my resources that belong to my account. I've used below IAM:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:PrincipalAccount": ["myAccountId"]
}
}
}
]
}
this works fine for S3, however it breaks the access to ECR from the node itself, so it no longer can pull the docker images.
This is what happens when I use docker pull:
error pulling image configuration: error parsing HTTP 403 response body: invalid character '<' looking for beginning of value: "<Error>...Access Denied...</Error>"
This is the message from the Kubernetes node that tries to fetch the image:
Failed to pull image "<accountId>.dkr.ecr.eu-west-1.amazonaws.com/<repoName>:<tag>": failed to copy: httpReaderSeeker: failed open: unexpected status code https://<accountId>.dkr.ecr.eu-west-1.amazonaws.com/v2/<repoName/blobs/sha256:<hash>: 403 Forbidden
If I remove the principal account condition:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": "*"
}
]
}
things are working OK.
It seem to me that ECR endpoint creates pre-signed URL to S3, which then gets blocked by my above policy.
Is there any way to allow ECR pre-signed URL, whilst still restricting access to only AWS resources that belong to my account?
After displaying Docker daemon logs I've noticed that it tries to fetch the resource from: prod-eu-west1-starport-layer-bucket.s3.eu-west-1.amazonaws.com.
I've then found AWS specific documentation which describes minimal permissions required:
https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html#ecr-setting-up-s3-gateway
I've then added this block to my S3 VPC endpoint IAM:
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::prod-eu-west-1-starport-layer-bucket/*"
}
and things started to work fine.
I'm running a NodeJS script and using the aws-sdk package to write files to an S3 bucket. This works fine when I run the script locally, but not from a ECS Fargate service, that's when I get Error: AccessDenied: Access Denied.
The service has the allowed VPC vpc-05dd973c0e64f7dbc. I've tried adding an Internet Gateway to this VPC, and also an endpoint (as seen in the attached image) - but nothing resolves the Access Denied error. Any ideas what I'm missing here?
SOLVED: the problem was me misunderstanding aws:sourceVpce. It requires the VPC endpoint id and not the VPC id. **
Endpoint
Internet Gateway
Bucket policy:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E3MKW5OAU5CHLI"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::mywebsite.com/*"
},
{
"Sid": "Stmt1582486025157",
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::mywebsite.com/*",
"Principal": "*",
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpc-05dd973c0e64f7dbc"
}
}
}
]
}
Please add an bucket policy that allows access from the VPC endpoint.
Update your bucket policy with a condition, that allows users to access the S3 bucket when the request is from the VPC endpoint that you created. To white list those users to download objects, you can use a bucket policy that's similar to the following:
Note: For the value of aws:sourceVpce, enter the VPC endpoint ID of the endpoint that you created.
{
"Version": "2012-10-17",
"Id": "Policy1314555909999",
"Statement": [
{
"Sid": "<<Access-to-specific-VPConly>>",
"Principal": "*",
"Action": "s3:GetObject",
"Effect": "Allow",
"Resource": ["arn:aws:s3:::awsexamplebucket/*"],
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpce-1c2g3t4e"
}
}
}
]
}
Do we need to make the S3 bucket public? if we want to use a VPC endpoint to access the bucket. The bucket is private and I have a bucket policy as follows.
{
"Version": "2012-10-17",
"Id": "Policy1415115909152",
"Statement": [
{
"Sid": "Access-to-specific-VPCE-only",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::bucket-name",
"arn:aws:s3:::bucket-name/*"
],
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpce-XXXXXXXXX"
}
}
}
]
}
I am getting the following error while accessing it
403 Forbidden
Code: AccessDenied
Message: Access Denied
RequestId: 3B5263AFE5F08F7D
HostId: M2+BaRG/GqiasUSkPo9rC46aC84pmZHNcbSnA2UcWcHxWntFRWjcli7VdN0wLpnsSZgK659008Y=
I have enabled static website hosting on the bucket, Idea was to access it privately in the VPC.