I have a lambda that I need to write that requires the bucket location for a given S3 bucket. I've set the permission to allow GetBucketLocation on all S3 buckets but I still get the following error:
An error occurred (AccessDenied) when calling the GetBucketLocation operation: Access Denied
I have even attached admin role rights to the function and I still get the above message. Is there any way to have GetBucketLocation to work?
For completeness the function looks as following:
# Libraries
# Standard Libaries
import json
import time
import urllib
# Third Party Libraries
import boto3
CLOUDFRONT_CLIENT = boto3.client('cloudfront')
S3_CLIENT = boto3.client('s3')
def get_cloudfront_distribution_id(bucket):
bucket_location = S3_CLIENT.get_bucket_location(Bucket=bucket) # <<<--- THIS PART FAILS
bucket_origin = f'{bucket}.s3.{bucket_location}.amazonaws.com'
cf_distro_id = None
# Create a reusable Paginator
paginator = CLOUDFRONT_CLIENT.get_paginator('list_distributions')
page_iterator = paginator.paginate()
...
return cf_distro_id
def lambda_handler(event, context):
'''
Creates a cloudfront invalidation for content added to a S3 bucket
'''
# Get the object from the event.
bucket = event['Records'][0]['s3']['bucket']['name']
key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])
if not key.startswith('/'):
key = f'/{key}'
cf_distro_id = get_cloudfront_distribution_id(bucket)
...
The IAM role
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
# Roles #
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
LambdaRole:
Description: IAM role to allow Lambda to invalidates content in a cloudfront distro when S3 content changes
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Sid: 1
ManagedPolicyArns:
- !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
RoleName: !Join
- '-'
- - !Ref AWS::StackName
- Role
The IAM Policy
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
# Policies #
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
LambdaPolicy:
Description: Setting IAM policy for service role for Lambda to invalidate cloudfront S3 cache
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Statement:
- Action:
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketVersioning
- s3:PutObject
Effect: Allow
Resource:
- !Sub 'arn:${AWS::Partition}:s3:::${ArtifactsBucket}'
- !Sub 'arn:${AWS::Partition}:s3:::${ArtifactsBucket}/*'
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:DeleteLogGroup
- logs:DeleteLogStream
- logs:PutLogEvents
Effect: Allow
Resource:
- !Sub 'arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:*'
- Action:
- cloudfront:CreateInvalidation
- cloudfront:GetDistribution
- cloudfront:GetStreamingDistribution
- cloudfront:GetDistributionConfig
- cloudfront:GetInvalidation
- cloudfront:ListInvalidations
- cloudfront:ListStreamingDistributions
- cloudfront:ListDistributions
Effect: Allow
Resource: '*'
- Action:
- s3:GetBucketLocation
- s3:ListBucket
Effect: Allow
Resource:
- !Sub 'arn:${AWS::Partition}:s3:::*'
PolicyName: !Join
- '-'
- - !Ref AWS::StackName
- LambdaPolicy
Roles:
- !Ref LambdaRole
The generated policy found in IAM
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cloudfront:GetDistribution",
"cloudfront:GetStreamingDistribution",
"cloudfront:ListInvalidations",
"cloudfront:ListDistributions",
"cloudfront:GetInvalidation",
"cloudfront:ListStreamingDistributions",
"cloudfront:GetDistributionConfig",
"cloudfront:CreateInvalidation"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"logs:CreateLogStream",
"logs:DeleteLogGroup",
"s3:GetBucketVersioning",
"logs:CreateLogGroup",
"logs:DeleteLogStream",
"logs:PutLogEvents",
"s3:GetObjectVersion"
],
"Resource": [
"arn:aws:logs:us-east-1:123456789000:*",
"arn:aws:s3:::123456789000-util-invalidate-cloudfront-artifactsbucket/*",
"arn:aws:s3:::123456789000-util-invalidate-cloudfront-artifactsbucket"
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::*"
}
]
}
Related
I have the following serverless.yml, file which deploys an application into AWS by creating a S3 bucket and a lambda function. Yet, the IAM role created for the lambda function is the standard one that allows to log into CloudFront (see below), not access the S3. The authorizations defined in the IAM role are not granted to the lambda. Am i missing anything? Do I have to reference the IAM role in the lambda function definition in serverless.yml?
service: webanalysistool
custom:
stage: ${opt:stage, 'dev'}
# plugins:
# - serverless-offline
provider:
name: aws
runtime: nodejs14.x
memorySize: 1024
stage: ${self:custom.stage}
# todo change it to your aws config
profile: cl_dev
versionFunctions: false
environment:
bucketName: "webanalysistool-${self:custom.stage}"
architecture: arm64
iam:
role:
statements:
# Allow functions to list all buckets
- Effect: Allow
Action: "s3:ListBucket"
Resource: "*"
# Allow functions to read/write objects in a bucket
- Effect: Allow
Action:
- "s3:GetObject"
- "s3:PutObject"
Resource:
- "arn:aws:s3:::${self:provider.environment.bucketName}/*"
package:
exclude:
- "node_modules/aws-sdk/**"
functions:
analyse:
handler: src/handler.start
timeout: 150
events:
- s3:
bucket: ${self:provider.environment.bucketName}
event: s3:ObjectCreated:*
rules:
- prefix: input/
IAM role created for the lambda function during deployment (I masked the AWS ID):
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup"
],
"Resource": [
"arn:aws:logs:us-east-1:999999999999:log-group:/aws/lambda/webanalysistool-dev*:*"
],
"Effect": "Allow"
},
{
"Action": [
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:us-east-1:999999999999:log-group:/aws/lambda/webanalysistool-dev*:*:*"
],
"Effect": "Allow"
}
]
}
Working for me
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:PutObject"
- "s3:GetObject"
- "s3:PutBucketAcl"
- "s3:PutObjectAcl"
- "s3:DeleteObject"
Resource:
- "arn:aws:s3:::${YOU_BUCKET}/*"
events:
- s3:
existing: true
bucket:
!Ref YOU_BUCKET
event: s3:ObjectCreated:*
rules:
- prefix: input/
I am trying to dynamically change the s3 resource name based on current cloud formation stack region. Cloudformation stack updates without any error.
Am I doing something wrong? I am expecting to have a policy with {AWS::Region} resolved to us-east-1.
Version: 2012-10-17
Statement:
- Sid: RestrictS3Access
Effect: Allow
Action:
- 's3:GetObject'
Resource:
- !Sub "arn:aws:s3:::dnsa-${AWS::Region}test/${cognito-identity.amazonaws.com:sub}"
- !Sub "arn:aws:s3:::dnsa-${AWS::Region}test/${cognito-identity.amazonaws.com:sub}/*"
I am expecting to see the policy as follows. I am checking results from aws console.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::dnsa-us-east-1/${cognito-identity.amazonaws.com:sub}",
"arn:aws:s3:::dnsa-us-east-1/${cognito-identity.amazonaws.com:sub}/*"
],
"Effect": "Allow",
"Sid": "RestrictS3Access"
}
If you want ${cognito-identity.amazonaws.com:sub} to remain unchanged, you need to escape it with ${!}.
Version: 2012-10-17
Statement:
- Sid: RestrictS3Access
Effect: Allow
Action:
- 's3:GetObject'
Resource:
- !Sub "arn:aws:s3:::dnsa-${AWS::Region}test/${!cognito-identity.amazonaws.com:sub}"
- !Sub "arn:aws:s3:::dnsa-${AWS::Region}test/${!cognito-identity.amazonaws.com:sub}/*"
I have a bucket policy which is restricting other users to access. But I want, For aws services it should be accessible like EMR etc.
I found same question is asked here:
S3 Bucket Policy to Allow access to specific users and restrict all . But I want to add services also. Like aws services can access that bucket but not users.
This is my bucket policy:
AUser:
Description: Name of the AUser
Type: String
BUser:
Description: Name of the BUser
Type: String
MetadataBucket:
Description: Name of the Metadata Bucket
Type: String
Resources:
MetadataBucketSecurity:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: MetadataBucket
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Deny
NotPrincipal:
AWS:
- !Sub 'arn:aws:iam::${AWS::AccountId}:user/${AUser}'
- !Sub 'arn:aws:iam::${AWS::AccountId}:user/${BUser}'
Action:
- 's3:ListBucket'
- 's3:RestoreObject'
- 's3:ReplicateObject'
- 's3:PutObject'
- 's3:PutBucketNotification'
- 's3:PutBucketLogging'
- 's3:PutObjectTagging'
- 's3:DeleteObject'
- 's3:GetObjectAcl'
- 's3:GetObject'
- 's3:GetBucketLogging'
- 's3:GetBucketAcl'
- 's3:ListBucketByTags'
- 's3:GetObjectVersionAcl'
- 's3:GetBucketPolicy'
Resource:
- !Sub 'arn:aws:s3:::${Bucket}'
- !Sub 'arn:aws:s3:::${Bucket}/*'
I tried to add this services directly, but it did not worked.
AUser:
Description: Name of the AUser
Type: String
BUser:
Description: Name of the BUser
Type: String
MetadataBucket:
Description: Name of the Metadata Bucket
Type: String
Resources:
MetadataBucketSecurity:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: MetadataBucket
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Deny
NotPrincipal:
AWS:
- !Sub 'arn:aws:iam::${AWS::AccountId}:user/${AUser}'
- !Sub 'arn:aws:iam::${AWS::AccountId}:user/${BUser}'
Service:
- 'elasticmapreduce.amazonaws.com'
- 'ec2.amazonaws.com'
Action:
- 's3:ListBucket'
- 's3:RestoreObject'
- 's3:ReplicateObject'
- 's3:PutObject'
- 's3:PutBucketNotification'
- 's3:PutBucketLogging'
- 's3:PutObjectTagging'
- 's3:DeleteObject'
- 's3:GetObjectAcl'
- 's3:GetObject'
- 's3:GetBucketLogging'
- 's3:GetBucketAcl'
- 's3:ListBucketByTags'
- 's3:GetObjectVersionAcl'
- 's3:GetBucketPolicy'
Resource:
- !Sub 'arn:aws:s3:::${Bucket}'
- !Sub 'arn:aws:s3:::${Bucket}/*'
After that I tried this also:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"NotPrincipal": {
"AWS": [
"arn:aws:iam::ano:root",
"arn:aws:iam::ano:user/AUser",
]
},
"Action": [
"s3:ListBucket",
"s3:RestoreObject",
"s3:ReplicateObject",
"s3:PutBucketNotification",
"s3:PutBucketLogging",
"s3:PutObjectTagging",
"s3:DeleteObject",
"s3:GetObjectAcl",
"s3:GetBucketLogging",
"s3:GetBucketAcl",
"s3:ListBucketByTags",
"s3:GetObjectVersionAcl",
"s3:GetBucketPolicy"
],
"Resource": [
"arn:aws:s3:::Bucket1",
"arn:aws:s3:::Bucket1/*"
]
},
{
"Sid": "InventoryAndAnalyticsExamplePolicy",
"Effect": "Allow",
"Principal": {
"Service": [
"elasticmapreduce.amazonaws.com",
"ec2.amazonaws.com",
"s3.amazonaws.com"
]
},
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::Bucket1",
"arn:aws:s3:::Bucket1/*"
]
}
]
}
But still it is not working
Is there any way where I can give access of S3 buckets to particular users and restrict others and with that AWS services should be access to that bucket?
I continue to get the error:
software.amazon.awssdk.services.kms.model.KmsException: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
When attempting to decrypt.
I've created a Task execution role with the permission:
"AssumeRolePolicyDocument": {
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
],
"Policies": [
{
"PolicyName": "AllowKmsDecrypt",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
{"Ref": "PrincipalSourceKeyArn"}
]
}
]
}
}
]
And the task definition is associated with the role:
"ExecutionRoleArn": {"Ref": "TaskExecutionRoleArn"},
Hm. What else could i be missing?
From these docs it turns out an IAM policy is not sufficient:
IAM policies by themselves are not sufficient to allow access to a
CMK. However, you can use them in combination with a CMK's key policy
if the key policy enables it. Giving the AWS account full access to
the CMK does this; it enables you to use IAM policies to give IAM
users and roles in the account access to the CMK
I needed to update the KMS KeyPolicy to include:
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": { "Fn::Join" : ["" , ["arn:aws:iam::", {"Ref" : "AWS::AccountId"} ,":root" ]] }
},
"Action": "kms:*",
"Resource": "*"
}
In case of this line
{ "Fn::Join" : ["" , ["arn:aws:iam::", {"Ref" : "AWS::AccountId"} ,":root" ]] }
you are allowing to use this key by root account only.
Usually, this key policy has to provide manage possibilities for users and only a few specific actions for services or other users that use this key. So as for me the whole set up has to look something like this:
KMSKeyEncryption:
Type: AWS::KMS::Key
Properties:
Enabled: true
EnableKeyRotation: false
KeyPolicy:
Version: 2012-10-17
Statement:
- Principal:
AWS:arn of the users/roles who are allowed to manage this key
Effect: Allow
Action:
- kms:Create*
- kms:Describe*
- kms:Enable*
- kms:List*
- kms:Put*
- kms:Update*
- kms:Revoke*
- kms:Disable*
- kms:Get*
- kms:Delete*
- kms:ScheduleKeyDeletion
- kms:CancelKeyDeletion
- kms:Encrypt*
- kms:Decrypt*
Resource: "*"
- Principal: "*" # this is not specific enough, should be strict
Effect: Allow
Action:
- kms:Decrypt*
Resource: "*"
PolicyDecryptKms:
Type: AWS::IAM::ManagedPolicy
Properties:
ManagedPolicyName: DecryptKmsPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: AllowDecryptValues
Effect: Allow
Action:
- kms:Decrypt*
Resource: !GetAtt KMSKeyEncryption.Arn
RoleECSTaskContainer:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2008-10-17
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
RoleName: ECSTaskContainerRole
ManagedPolicyArns:
- !Ref PolicyDecryptKms
I have created a bucket with cloudformation:
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
BucketName:
Type: String
Description: "Choose a name for the S3 Bucket"
Default: "myrandomnameforbucket"
S3Bucket:
Type: "AWS::S3::Bucket"
Properties:
AccessControl: "Private"
BucketName: !Ref BucketName
Now I'm writing a bucketPolicy but I'm facing some issues. What I want to achieve:
USER A (UserA) can upload to S3
USER A (UserA) can NOT DELETE from S3
ALL users (in my environment, not public) can read from S3
ALL users (in my environment, nob public) can delete from S3
How can I achieve this?
At the moment I denied a delete from userA and allowed an upload from userA.
- Effect: Deny
Principal:
AWS:
!GetAtt UserA.Arn
Action: "s3:DeleteObject"
Resource:
Fn::Join: ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
- Effect: Allow
Principal:
AWS:
!GetAtt UserA.Arn
Action: "s3:PutObject"
Resource:
Fn::Join: ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
- Effect: Allow
Principal: "?" # * is public?
Action: s3:GetObject
Resource:
Fn::Join: ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
I understand there are two questions:
how to grant access to all IAM users when excluding anonymous users
how to restrict one user more than the others, that is: removing rights you just granted to others
The first question sounds easy at first, as the documentation states:
In resource-based policies, use the Principal element to specify the accounts or users who are allowed to access the resource
So this would mean you could do something like:
Principal:
AWS: !Ref "AWS::AccountId"
But when I tried it just didn't work. When setting the arn of a specific user it worked for me. This seems like a bug to me. Or an unclarity in the documentation. There is this other report I found.
Anyway, what you can do is to use Principal: AWS: "*" and then use a Condition to restrict to IAM users only.
The second question is much easier: policies are evaluated such that explicit denys have priority over general allows, see documentation.
The resulting policy can be e.g. written like this:
S3Policy:
Type: "AWS::S3::BucketPolicy"
Properties:
Bucket: !Ref S3Bucket
PolicyDocument:
Statement:
- Effect: Deny
Action: "s3:DeleteObject"
Resource: !Join ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
Principal:
AWS: !GetAtt UserA.Arn
- Effect: Allow
Action: "s3:PutObject"
Resource: !Join ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
Principal:
AWS: !GetAtt UserA.Arn
- Effect: Allow
Action: ["s3:GetObject", "s3:DeleteObject"]
Resource: !Join ["", ["arn:aws:s3:::", Ref: "S3Bucket", "/*"]]
Principal:
AWS: "*"
Condition:
StringEquals:
"aws:PrincipalType": ["User"]
{
"Version": "2012-10-17",
"Id": "S3PolicyId1",
"Statement": [
{
"Sid": "AllowGet",
"Effect": "Allow",
"Principal":{"AWS":"arn:aws:iam::account-number-without-hyphens:user/user1"},
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::s3_bucket_name",
"arn:aws:s3:::s3_bucket_name/*"
]
},
{
"Sid": "DenyDeleteObject",
"Effect": "Deny",
"Principal": {"AWS":"arn:aws:iam::account-number-without-hyphens:user/user1"},
"Action": "s3:Delete*",
"Resource": [
"arn:aws:s3:::s3_bucket_name",
"arn:aws:s3:::s3_bucket_name/*"
]
},
{
"Sid": "Allow anyone in your account to access bucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::account-number-without-hyphens:root"
},
"Action": [
"s3:Get*",
"s3:List*",
"s3:Put*",
"s3:Delete*"
],
"Resource": [
"arn:aws:s3:::s3_bucket_name",
"arn:aws:s3:::s3_bucket_name/*"
]
}
]
}
Here is a template in JSON format I quickly put together. My assumption here is that your group "ALL users (in my environment, nob public)" is everybody in the account. Hence, we define it in the third block. You can always manipulate the principal with whatever you want.
If you have any question please ask, I am happy to help.