I have launched an EC2 instance which is needed to connect to s3 bucket.
i created IAM role and linked it to EC2 instance. and from EC2 awscli i can list the files, however i deployed a container in that EC2 and when trying to list the file, I am getting the error -
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
IAM-Role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::my-bucket/*",
]
}
]
}
Can somebody please suggest. why i can access the s3 from an ec2 instance but not from the container running on the same EC2 instance.
The ListBucket call is applied at the bucket level, so you need to add the bucket as a resource in your IAM policy (as written, you were just allowing access to the bucket's files):
"Resource": [
"arn:aws:s3:::my-bucket",
"arn:aws:s3:::my-bucket/*"
]
See this for more information about the resource description needed for each permission.
The fact that you were able to get the bucket listing from a shell running on the EC2 instance indicates to me that you have another user configured. Look for files in $HOME/.aws and environment variables that start with AWS.
Related
My CI pipeline deposits intermediate artifacts in an S3 bucket that I don't want to be accessible to the public, but that does need to be accessible to myself and certain IP addresses.
Currently I have the following bucket permission policy in place:
{
"Version": "2012-10-17",
"Id": "CIBucketPolicy",
"Statement": [
{
"Sid": "IpAllowList",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::ci-bucket-name",
"arn:aws:s3:::ci-bucket-name/*"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"ip-address1",
"ip-address2",
"ip-address3"
]
}
}
}
]
}
One of the artifacts that my CI deposits here is a raw disk image, which is used in later stages to build an Amazon Machine Image using aws ec2 import-snapshot. The problem here is that the aws ec2 import-snapshot stage keeps failing with the error message:
ClientError: Disk validation failed [We do not have access to the given resource. Reason 403 Forbidden]
I'm fairly confident that something about my bucket permission policy is blocking ec2 from being able to access the bucket even though they're in the same availability zone, but I haven't figured out how to overcome this without simply removing the IP address condition.
Do I need to add something special to this policy for EC2 to have access? Perhaps I'm missing an S3 Action that EC2 needs to be able to perform in order to access the image file? Any advice would be greatly appreciated!
I have a compute environment with 'ecsInstanceRole'. It contains the policies 'AmazonS3FullAccess' and 'AmazonEC2ContainerServiceforEC2Role'
Since I am using the AmazonS3FullAccess policy, I assume the batch job has permission to list, copy, put etc.
-The image I am using is a custom docker image that has a startup script which uses "aws s3 ls <S3_bucket_URL>"
When I start this image on an EC2 instance, it runs fine and lists the contents of the bucket
when I do the same as a batch job, I get the access denied error seen above.
I dont understand how this is happening.
Things I have tried so far:
having the bucket policy as
.
{
"Version": "2012-10-17",
"Id": "Policy1546414123454",
"Statement": [
{
"Sid": "Stmt1546414471931",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<Account Id>:root"
},
"Action": [
"s3:ListBucket",
"s3:ListBucketVersions"
],
"Resource": [
"arn:aws:s3:::"bucketname",
"arn:aws:s3:::bucketname/*"
]
}
]
}
Granted public access to the bucket
Quoting the reply from #JohnRotenstein because I cannot mark it as an answer.
"If you are using IAM Roles, there is no need for a Bucket Policy. (Also, there is a small typo in that policy, before bucketname but I presume that was due to a Copy & Paste error.) It would appear that a role has not been assigned to your ECS task: IAM Roles for Tasks - Amazon Elastic Container Service"
Solution: I had toattach an S3 access policy to my current Job Role.
Scenario: I have an EC2 instance and a S3 bucket under the same account, and my web app on that EC2 wants access to resources in that bucket.
Following official docs, I created an IAM role with s3access and assigned it to the EC2 instance. To my understanding, now my web app should be able to access the bucket. However, after trials, seems I have to add a allowPublicRead bucket policy like this:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mybucket/*"
}
]
}
Otherwise I got access forbidden.
But why should I use this allowPublicRead bucket policy, since I already granted s3access IAM role to the EC2 instance?
S3 s3:GetObject will only allow access to objects from your ec2 instance and what you want is to access these objects from your web-app which means from your browser, in this case these images/objects will be rendered to user browser and if its a public facing application then you need to assign AllowPublicRead permission as well.
I am trying to download AWS Codedeploy Agent file in my Amazon Linux. I followed instructions as mentioned in http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html, for Amazon Linux, have created appropriate instance profile, service role etc. Everything is latest (Amazon Linux, CLI Packages, it is a brand new instance and I have tried this with at least 3 more brand new instances with same result). All instances have full outbound internet access.
But following statement for downloading install from S3 always fails,
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1
With Error,
A client error (403) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining
Can anyone help me with this error?
I figured out the problem, According to Codedeploy documentation for IAM Instance profile
http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-create-iam-instance-profile.html
following permissions needs to be given to your IAM instance profile.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:Get*",
"s3:List*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
But I limited the resource to my code bucket since I don't want my instances to access other buckets directly. But turns out I also need to give additional permission for aws-codedeploy-us-east-1/* s3 resource for being able to download the agent. This is not very clear in the document for setting up IAM instance profile for Codedeploy.
More restrictive policy that works:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::aws-codedeploy-us-east-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-2/*",
"arn:aws:s3:::aws-codedeploy-ap-south-1/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-1/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-1/*",
"arn:aws:s3:::aws-codedeploy-eu-central-1/*",
"arn:aws:s3:::aws-codedeploy-eu-west-1/*",
"arn:aws:s3:::aws-codedeploy-sa-east-1/*"
]
}
]
}
I am looking to lock down an S3 bucket for security purposes - i'm storing deployment images in the bucket.
What I want to do is create a bucket policy that supports anonymous downloads over http only from EC2 instances in my account.
Is there a way to do this?
An example of a policy that I'm trying to use (it won't allow itself to be applied):
{
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[my bucket name]",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:ec2:us-east-1:[my account id]:instance/*"
}
}
}
]
}
Just to clarify how this is normally done. You create a IAM policy, attach it to a new or existing role, and decorate the ec2 instance with the role. You can also provide access through bucket policies, but that is less precise.
Details below:
S3 buckets are default deny except for my the owner. So you create your bucket and upload the data. You can verify with a browser that the files are not accessible by trying https://s3.amazonaws.com/MyBucketName/file.ext. Should come back with error code "Access Denied" in the xml. If you get an error code of "NoSuchBucket", you have the url wrong.
Create an IAM policy based on arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess. Starts out looking like the snip below. Take a look at the "Resource" key, and note that it is set to a wild card. You just modify this to be the arn of your bucket. You have to do one for the bucket and its contents so it becomes: "Resource": ["arn:aws:s3:::MyBucketName", "arn:aws:s3:::MyBucketName/*"]
Now that you have a policy, what you want to do is to decorate your instances with a IAM Role that automatically grants it this policy. All without any authentication keys having to be in the instance. So go to Role, create new role, make an Amazon EC2 role, find the policy you just created, and your Role is ready.
Finally you create your instance, and add the IAM role you just created. If the machine already has its own role, you just have to merge the two roles into a new one for the machine. If the machine is already running, it wont get the new role until you restart.
Now you should be good to go. The machine has the rights to access the s3 share. Now you can use the following command to copy files to your instance. Note you have to specify the region
aws s3 cp --region us-east-1 s3://MyBucketName/MyFileName.tgz /home/ubuntu
Please Note, the term "Security through obscurity" is only a thing in the movies. Either something is provably secure, or it is insecure.
I used something like
{
"Version": "2012-10-17",
"Id": "Allow only My VPC",
"Statement": [
{
"Sid": "Allow only My VPC",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject", "s3:ListBucket",
"Resource": [
"arn::s3:::{BUCKET_NAME}",
"arn::s3:::{BUCKET_NAME}/*"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "{VPC_ID}" OR "aws:sourceVpce": "{VPCe_ENDPOINT}"
}
}
}
]
}