I would like to be able to change the policies on s3 buckets using cloudformation. However when I attempt to do this I encounter the error:
2017-12-21 18:49:10 UTC TestBucketpolicyAwsS3Bucketpolicy CREATE_FAILED API: s3:PutBucketPolicy Access Denied
Here is an example of a cloudformation template that fails due to this issue:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "",
"Resources": {
"TestBucketpolicyAwsS3Bucketpolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": "alex-test-bucket-123",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Resource": "arn:aws:s3:::alex-test-bucket-123/*",
"Action": [
"s3:GetObject*",
"s3:DeleteObject*",
"s3:PutObject*"
]
}
]
}
}
}
}
}
I have tried changing policies on both my IAM user and the actual bucket I want to manage with cloudformation, but neither solution has resolved the issue. How can I get remove this "s3:PutBucketPolicy" restriction?
Edit: I think the issue may be that only IAM roles can access the "s3:PutBucketPolicy" operation. I may need to create a role with s3 access then establish a trust relationship with the user that runs this cloudformation template.
https://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html
IAM users cannot directly run s3:PutBucketPolicy operations. You need to create a separate IAM role and attach it to your user with a trust relationship to assume that IAM role.
Your role will need s3 and cloudformation access. The policy document below will work.
{
"Version": "2012-10-17",
"Statement": {
"Action": [
"s3:*",
"cloudformation:*"
],
"Resource": "*",
"Effect": "Allow"
}
}
The arn of your IAM role will then need to be set in your config or the AWS_STS_ROLE_ARN environmental variable along with your aws access keys.
Once you assume the role you will then be able to change s3 bucket policies.
Note that this will override any permissions your user has when you set your AWS_STS_ROLE_ARN in your config or environmental variables.
Related
Can I use string interpolation for the Resource key of IAM statements? I am trying to grant access to a bucket based on the team a user is tagged as.
This statement does not work
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::${aws:PrincipalTag/team}.example.com"
}
]
}
If I replace ${aws:PrincipalTag/team} with a hardcoded team name it works as expected.
You can accomplish this by using conditional keys in either the iam role/user's policy or the s3 bucket policy.
IAM Policy
For using conditional keys in the iam policy, you will need to add a statement that limits the users s3 actions to resources that have been tagged with a particular resource tag. This will prevent the user from access s3 objects that do not have a particular tag. One problem that you may discover is that you will then need to make sure all objects you wish to interact with are also tagged.
Example of a iam policy:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws: aws:ResourceTag/owningTeam": "team1"
}
}
}]
}
s3 Bucket Policy
For using conditional keys in the s3 bucket policy, you will need to add a statement that limits access to the bucket to users/roles with a particular principal tag. The user will still need to have permissions to perform s3 actions, but the resource policy will restrict the access. In addition to condition keys, you can also limit based on the user/role arn.
Example of a bucket policy:
{
"Version": "2012-10-17",
"Id": "S3PolicyId1",
"Statement": [
{
"Sid": "statement1",
"Effect": "Allow",
"Principal": "*",
"Action":"s3:GetObject",
"Resource": "arn:aws:s3:::awsexamplebucket1/*",
"Condition" : {
"StringEquals" : {
"aws:PrincipalTag/team": "team1"
}
}
}
]
}
Since you ultimately want to leverage the tags on the principal, I'd recommend the bucket policy approach.
Resources:
IAM Policy vs Bucket Policy
aws conditional keys
I am the root user of my account and i created one new user and trying to give access to s3 via s3 bucket policy:
Here is my policy details :-
{ "Id": "Policy1542998309644", "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1542998308012", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": "arn:aws:s3:::aws-bucket-demo-1", "Principal": { "AWS": [ "arn:aws:iam::213171387512:user/Dave" ] } } ]}
in IAM i have not given any access to the new user. I want to provide him access to s3 via s3 bucket policy. Actually i would like to achieve this : https://aws.amazon.com/premiumsupport/knowledge-center/s3-console-access-certain-bucket/ But not from IAM , I want to use only s3 bucket policy.
Based on the following AWS blog post (the blog shows IAM policy, but it can be adapted to a bucket policy):
How can I grant a user Amazon S3 console access to only a certain bucket?
you can make the following bucket policy:
{
"Id": "Policy1589632516440",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1589632482887",
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::aws-bucket-demo-1",
"Principal": {
"AWS": [
"arn:aws:iam::213171387512:user/Dave"
]
}
},
{
"Sid": "Stmt1589632515136",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::aws-bucket-demo-1/*",
"Principal": {
"AWS": [
"arn:aws:iam::213171387512:user/Dave"
]
}
}
]
}
This will require user to url directly to the bucket:
https://s3.console.aws.amazon.com/s3/buckets/aws-bucket-demo-1/
The reason is that the user does not have permissions to list all buckets available. Thus he/she has to go directly to the one you specify.
Obviously the IAM user needs to have AWS Management Console access enabled when you create him/her in the IAM service. With Programmatic access only, IAM users can't use console and no bucket policy can change that.
You will need to use ListBuckets.
It seems like you want this user to only be able to see your bucket but not access anything in it.
Anyone knows how I can track if IAM pass role action (inside a role) is being used by any services or not? As per AWS document the "iam:PassRole" action is not tracked under IAM access advisor: Refining Permissions Using Service Last Accessed Data - AWS Identity and Access Management
I have this policy inside a IAM role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "*"
}
]
}
I have an IAM role(MyIAMrole) which has already been created. I want to attach a policy to this role using a Cloudformation template.
"Mypolicy":{
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "assume-role-policy",
"PolicyDocument": {
"Version" : "2012-10-17",
"Statement": [
{ "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "*" }
]
},
"Roles": [ { "Ref": "arn:aws:iam::*:role/MyIAMrole" } ]
}
}
When I try to validate this I am getting an error saying "Unreolved reference options".
How to attach this policy to an already existing role?
I managed to get your code snippet to work by referring to the Name of a role rather than the ARN.
As per the AWS::IAM::Policy documentation:
Roles: The names of AWS::IAM::Roles to which this policy will be attached.
However, while the stack went to CREATE_COMPLETE, I couldn't see the policy listed in the Policies section of IAM, nor could I see the policy attached to the referenced role.
It might be that you cannot use CloudFormation to attach a policy to an existing role. You might need to create the Role as part of the CloudFormation template to be able to attach a role.
In JupyterHub, installed in an EC2 instance with an IAM role which allows access to a specific S3 bucket when I try to access a file in that bucket with this code:
s3nRdd = spark.sparkContext.textFile("s3n://bucket/file")
I get this error:
IllegalArgumentException: u'AWS Access Key ID and Secret Access Key
must be specified as the username or password (respectively) of a s3n
URL, or by setting the fs.s3n.awsAccessKeyId or
fs.s3n.awsSecretAccessKey properties (respectively).'
However, when I export the AWS access key id and secret access key in the kernel configuration having the same policy as that role, the read for that file succeeds.
As the best practice is to use IAM roles, why doesn't the EC2 role work in this situation?
--update--
The EC2 IAM role has these 2 policies:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1488892557621",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/*"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "ec2:*",
"Effect": "Allow",
"Resource": "*"
},
{
"Sid": "Stmt1480684159000",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"*"
]
}
]
}
Also, I am using hadoop version 2.4.0 which doesn't support s3a protocol and updating is not an option.
S3n doesn't support IAM roles, and 2.4 is a very outdated version anyway. Not as buggy as 2.5 when it comes to s3n, but still less than perfect.
If you want to use IAM roles, you are going to have to switch to S3a, and yes, for you, that does mean upgrading Hadoop. sorry.
You must create a bucket policy to allow access from particular IAM roles. Since S3 doesn't trust the roles, the API just fallback and ask for access key.
Just add soemthing like this in your bucket policy, replace all the custom <> parameter with your own values.
{
"Version": "2012-10-17",
"Id": "EC2IAMaccesss",
"Statement": [{
"Sid": "MyAppIAMRolesAccess",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<acc_id>:role/<yourIAMroleName>"
]
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<yourbucket>/*",
"arn:aws:s3:::<yourbucket>"
]
}
]
}
(updates)
Make sure you give proper policy to the EC2 IAM Roles, because IAM roles is very powerful, no Policy is attach to it out of the box. You must assign a policy, e.g. for minimal S3 access, add AWSS3ReadOnly policy to the roles.
You may encounter issues of spark problematic interaction with IAM roles. Please check the documentation on spark access through s3n:// schema. Otherwise, use s3a://