My CI pipeline deposits intermediate artifacts in an S3 bucket that I don't want to be accessible to the public, but that does need to be accessible to myself and certain IP addresses.
Currently I have the following bucket permission policy in place:
{
"Version": "2012-10-17",
"Id": "CIBucketPolicy",
"Statement": [
{
"Sid": "IpAllowList",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::ci-bucket-name",
"arn:aws:s3:::ci-bucket-name/*"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"ip-address1",
"ip-address2",
"ip-address3"
]
}
}
}
]
}
One of the artifacts that my CI deposits here is a raw disk image, which is used in later stages to build an Amazon Machine Image using aws ec2 import-snapshot. The problem here is that the aws ec2 import-snapshot stage keeps failing with the error message:
ClientError: Disk validation failed [We do not have access to the given resource. Reason 403 Forbidden]
I'm fairly confident that something about my bucket permission policy is blocking ec2 from being able to access the bucket even though they're in the same availability zone, but I haven't figured out how to overcome this without simply removing the IP address condition.
Do I need to add something special to this policy for EC2 to have access? Perhaps I'm missing an S3 Action that EC2 needs to be able to perform in order to access the image file? Any advice would be greatly appreciated!
Related
So my bucket was and is still functioning correctly, I'm able to upload images through the API with no issues. However, I was messing around with the user policy and I made a change to the Resource for my User Policy and this caused some settings to change.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1420751757000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": CHANGE MADE HERE
}
]
}
When I try to upload an image through my AWS account (not using the API), then the ACL public access is private by default. I tried changing my Policy version back to what I had, but no change. I am pretty inexperienced with S3, so if I'm missing crucial info regarding this issue I can provide it.
If you want all objects to be public, then you should use a Bucket Policy.
This should typically be limited to only allowing people to download (Get) an object if they know the name of the object. You can use this Bucket Policy (which goes on the bucket itself):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::YOUR-BUCKET-NAME/*"
]
}
]
}
This policy is saying: "Allow anyone to get an object from this bucket, without knowing who they are"
It does not allow listing of the bucket, upload to the bucket or deleting from the bucket. If you wish to do any of these operations, you would need to use your own credentials via an API call or using the AWS CLI.
For examples of bucket policies, see: Bucket policy examples - Amazon Simple Storage Service
Your IAM User should probably have a policy like this:
{
"Version":"2012-10-17",
"Statement":[
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
"Resource": [
"arn:aws:s3:::YOUR-BUCKET-NAME",
"arn:aws:s3:::YOUR-BUCKET-NAME/*"
]
}
]
}
This is saying: "Allow this IAM User to do anything in Amazon S3 to this bucket and the contents of this bucket"
That will grant you permission to do anything with the bucket (including uploading, downloading and deleting objects, and deleting the bucket).
For examples of IAM Policies, see: User policy examples - Amazon Simple Storage Service
I'm using WorkSpaces Web (not WorkSpaces!) with an S3 VPC endpoint. I would like to be able to restrict S3 access via the S3 endpoint policy to only the buckets required by WorkSpaces Web. I cannot find any documentation with the answers, and AWS support does not seem to know what these buckets are. How can I find out what buckets the service is talking to? I see the requests in VPC flow logs, but that obviously doesn't show what URL or bucket it is trying to talk to. I have tried the same policy used for WorkSpaces (below), but it was not correct (or possibly not enough). I have confirmed that s3:GetObject is the only action needed.
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "Access-to-specific-bucket-only",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::aws-windows-downloads-us-east-1/*",
"arn:aws:s3:::amazon-ssm-us-east-1/*",
"arn:aws:s3:::amazon-ssm-packages-us-east-1/*",
"arn:aws:s3:::us-east-1-birdwatcher-prod/*",
"arn:aws:s3:::aws-ssm-distributor-file-us-east-1/*",
"arn:aws:s3:::aws-ssm-document-attachments-us-east-1/*",
"arn:aws:s3:::patch-baseline-snapshot-us-east-1/*",
"arn:aws:s3:::amazonlinux.*.amazonaws.com/*",
"arn:aws:s3:::repo.*.amazonaws.com/*",
"arn:aws:s3:::packages.*.amazonaws.com/*"
]
}
]
}
I have launched an EC2 instance which is needed to connect to s3 bucket.
i created IAM role and linked it to EC2 instance. and from EC2 awscli i can list the files, however i deployed a container in that EC2 and when trying to list the file, I am getting the error -
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
IAM-Role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::my-bucket/*",
]
}
]
}
Can somebody please suggest. why i can access the s3 from an ec2 instance but not from the container running on the same EC2 instance.
The ListBucket call is applied at the bucket level, so you need to add the bucket as a resource in your IAM policy (as written, you were just allowing access to the bucket's files):
"Resource": [
"arn:aws:s3:::my-bucket",
"arn:aws:s3:::my-bucket/*"
]
See this for more information about the resource description needed for each permission.
The fact that you were able to get the bucket listing from a shell running on the EC2 instance indicates to me that you have another user configured. Look for files in $HOME/.aws and environment variables that start with AWS.
I am trying to download AWS Codedeploy Agent file in my Amazon Linux. I followed instructions as mentioned in http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html, for Amazon Linux, have created appropriate instance profile, service role etc. Everything is latest (Amazon Linux, CLI Packages, it is a brand new instance and I have tried this with at least 3 more brand new instances with same result). All instances have full outbound internet access.
But following statement for downloading install from S3 always fails,
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1
With Error,
A client error (403) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining
Can anyone help me with this error?
I figured out the problem, According to Codedeploy documentation for IAM Instance profile
http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-create-iam-instance-profile.html
following permissions needs to be given to your IAM instance profile.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:Get*",
"s3:List*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
But I limited the resource to my code bucket since I don't want my instances to access other buckets directly. But turns out I also need to give additional permission for aws-codedeploy-us-east-1/* s3 resource for being able to download the agent. This is not very clear in the document for setting up IAM instance profile for Codedeploy.
More restrictive policy that works:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::aws-codedeploy-us-east-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-2/*",
"arn:aws:s3:::aws-codedeploy-ap-south-1/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-1/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-1/*",
"arn:aws:s3:::aws-codedeploy-eu-central-1/*",
"arn:aws:s3:::aws-codedeploy-eu-west-1/*",
"arn:aws:s3:::aws-codedeploy-sa-east-1/*"
]
}
]
}
I am looking to lock down an S3 bucket for security purposes - i'm storing deployment images in the bucket.
What I want to do is create a bucket policy that supports anonymous downloads over http only from EC2 instances in my account.
Is there a way to do this?
An example of a policy that I'm trying to use (it won't allow itself to be applied):
{
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[my bucket name]",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:ec2:us-east-1:[my account id]:instance/*"
}
}
}
]
}
Just to clarify how this is normally done. You create a IAM policy, attach it to a new or existing role, and decorate the ec2 instance with the role. You can also provide access through bucket policies, but that is less precise.
Details below:
S3 buckets are default deny except for my the owner. So you create your bucket and upload the data. You can verify with a browser that the files are not accessible by trying https://s3.amazonaws.com/MyBucketName/file.ext. Should come back with error code "Access Denied" in the xml. If you get an error code of "NoSuchBucket", you have the url wrong.
Create an IAM policy based on arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess. Starts out looking like the snip below. Take a look at the "Resource" key, and note that it is set to a wild card. You just modify this to be the arn of your bucket. You have to do one for the bucket and its contents so it becomes: "Resource": ["arn:aws:s3:::MyBucketName", "arn:aws:s3:::MyBucketName/*"]
Now that you have a policy, what you want to do is to decorate your instances with a IAM Role that automatically grants it this policy. All without any authentication keys having to be in the instance. So go to Role, create new role, make an Amazon EC2 role, find the policy you just created, and your Role is ready.
Finally you create your instance, and add the IAM role you just created. If the machine already has its own role, you just have to merge the two roles into a new one for the machine. If the machine is already running, it wont get the new role until you restart.
Now you should be good to go. The machine has the rights to access the s3 share. Now you can use the following command to copy files to your instance. Note you have to specify the region
aws s3 cp --region us-east-1 s3://MyBucketName/MyFileName.tgz /home/ubuntu
Please Note, the term "Security through obscurity" is only a thing in the movies. Either something is provably secure, or it is insecure.
I used something like
{
"Version": "2012-10-17",
"Id": "Allow only My VPC",
"Statement": [
{
"Sid": "Allow only My VPC",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject", "s3:ListBucket",
"Resource": [
"arn::s3:::{BUCKET_NAME}",
"arn::s3:::{BUCKET_NAME}/*"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "{VPC_ID}" OR "aws:sourceVpce": "{VPCe_ENDPOINT}"
}
}
}
]
}