I've been developing an aws lambda function with python and serverless. It reads files from a s3 bucket and stores data from it in another bucket as a csv. The code works perfectly with invoke local but after deploying I get:
An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
This is the function:
s3_client.put_object(Body=data, Bucket=bucket_name, Key=key_name)
and part of the serverless yml
iamRoleStatements:
- Effect: Allow
Action:
- KMS:Decrypt
Resource: '*'
- Effect: Allow
Action:
- 's3:*'
Resource:
- 'arn:aws:s3:::output_bucket'
- 'arn:aws:s3:::output_bucket/*'
- Effect: Allow
Action:
- 's3:ListBucket'
Resource:
- 'arn:aws:s3:::input_bucket'
- 'arn:aws:s3:::input_bucket/*'
- Effect: Allow
Action:
- 's3:GetObject'
Resource:
- 'arn:aws:s3:::input_bucket'
- 'arn:aws:s3:::input_bucket/*'
Is there any other configuration I need? Am I missing something obvious?
For me the indent looks wrong. Please make sure to use it right.
iamRoleStatements:
- Effect: Allow
Action:
- 'kms:Decrypt'
Resource: '*'
- Effect: Allow
Action:
- 's3:*'
Resource:
- 'arn:aws:s3:::output_bucket'
- 'arn:aws:s3:::output_bucket/*'
- Effect: Allow
Action:
- 's3:ListBucket'
- 's3:GetObject'
Resource:
- 'arn:aws:s3:::input_bucket'
- 'arn:aws:s3:::input_bucket/*'
FYI: I merged s3:ListBucket and s3:GetObject into one statement.
Related
I have a some Dynamo DB tables, and a DAX cluster sitting above this to cache queries and results. It works great except for when I use BatchGetItem.
[ERROR] DaxServiceError: An error occurred (Unknown) when calling the BatchGetItem operation: Client does not have permission to invoke BatchGetItem
I have granted both Lambda and the Cluster an IAM role with a wildcard, ie:
dynamodb:*
dax:*
Previously I tried setting these to the specific permission:
dax:BatchGetItem
But neither make a difference.
Googling returns ZERO results for "Client does not have permission to invoke BatchGetItem"
https://www.google.com/search?q=%22Client+does+not+have+permission+to+invoke+BatchGetItem%22&rlz=1C1YTUH_en-GBGB1008GB1008&oq=%22Client+does+not+have+permission+to+invoke+BatchGetItem%22&aqs=chrome..69i57.2944j0j7&sourceid=chrome&ie=UTF-8
In general there seems to be very little around about DAX. I haven't seen anything yet to imply this feature is not supported by AWS. In fact - if there is a permission for it, then it must be implemented in some capacity.
I'm quite stumped and would be every grateful to anyone who has any ideas.
[EDIT] Here is the IAM definition:
DynamoDAXCacheClusterIamRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- dax.amazonaws.com
Version: '2012-10-17'
RoleName: ${file(env.yml):${opt:stage}.DAX_CLUSTER_ROLE}
Policies:
-
PolicyName: ${file(env.yml):${opt:stage}.DAX_POLICY_NAME}-dax
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Resource: '*'
Action:
- 'dax:*'
-
PolicyName: ${file(env.yml):${opt:stage}.DAX_POLICY_NAME}-dynamo
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Resource: '*'
Action:
- 'dynamodb:*'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
I know it's probably too open but this is just whilst I work out the issue
[EDIT2] Here is the policy that the Lambda's have:
- Effect: Allow
Action:
- dynamodb:DescribeTable
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
- dynamodb:BatchGetItem
Resource: "arn:aws:dynamodb:us-east-1:*:*"
- Effect: Allow
Action:
- sqs:SendMessage
- sqs:PurgeQueue
- sqs:ListQueues
Resource: "arn:aws:sqs:us-east-1:*:*"
- Effect: "Allow"
Action:
- "execute-api:Invoke"
Resource: ["arn:aws:execute-api:*:*:**/#connections/*"]
- Effect: "Allow"
Resource: "arn:aws:sns:us-east-1:*:*"
Action:
- "sns:*"
- Effect: Allow
Resource: "arn:aws:ses:us-east-1:*:*/*"
Action:
- SES:SendEmail
- SES:SendRawEmail
- SES:SendTemplatedEmail
- SES:SendBulkTemplatedEmail
- SES:UpdateEmailTemplate
- SES:UpdateTemplate
- Effect: Allow
Resource: "arn:aws:dax:us-east-1:*:*/*"
Action:
- 'dax:*'
I am a complete duff.
There was two stacks sharing the same DAX and one was missing the permission
I have added the following policy to my cloud formation template:
ASGNotificationPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
ManagedPolicyName: !Sub '${EnvironmentName}-asg-notification-policy'
Description: ASG notificatoin policy for node termination handler IAM Policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: 'EC2InstanceManagement'
- Effect: Allow
Action:
- 'ec2:AttachClassicLinkVpc'
- 'ec2:CancelSpotInstanceRequests'
- 'ec2:CreateFleet'
- 'ec2:CreateTags'
- 'ec2:DeleteTags'
- 'ec2:Describe*'
- 'ec2:DetachClassicLinkVpc'
- 'ec2:ModifyInstanceAttribute'
- 'ec2:RequestSpotInstances'
- 'ec2:RunInstances'
- 'ec2:StartInstances'
- 'ec2:StopInstances'
- 'ec2:TerminateInstances'
Resource:
- '*'
- Sid: 'EC2InstanceProfileManagement'
- Effect: Allow
Action:
- 'iam:PassRole'
Resource:
- '*'
Condition:
StringLike:
'iam:PassedToService': 'ec2.amazonaws.com*'
- Sid: 'EC2SpotManagement'
- Effect: Allow
Action:
- 'iam:CreateServiceLinkedRole'
Resource:
- '*'
Condition:
StringLike:
'iam:AWSServiceName': 'spot.amazonaws.com'
- Sid: 'ELBManagement'
- Effect: Allow
Action:
- 'elasticloadbalancing:Register*'
- 'elasticloadbalancing:Deregister*'
- 'elasticloadbalancing:Describe*'
Resource:
- '*'
- Sid: 'CWManagement'
- Effect: Allow
Action:
- 'cloudwatch:DeleteAlarms'
- 'cloudwatch:DescribeAlarms'
- 'cloudwatch:GetMetricData'
- 'cloudwatch:PutMetricAlarm'
Resource:
- '*'
- Sid: 'SNSManagement'
- Effect: Allow
Action:
- 'sns:Publish'
Resource:
- '*'
- Sid: 'EventBridgeRuleManagement'
- Effect: Allow
Action:
- 'events:PutRule'
- 'events:PutTargets'
- 'events:RemoveTargets'
- 'events:DeleteRule'
- 'events:DescribeRule'
Resource:
- '*'
Condition:
StringEquals:
'events:ManagedBy': 'autoscaling.amazonaws.com'
- Sid: 'SystemsManagerParameterManagement'
- Effect: Allow
Action:
- 'ssm:GetParameters'
Resource:
- '*'
When I'm running the stack, I get a syntax error:
What is the issue with this policy syntax? I searched for possible reasons for the syntax issue, but I didn't find a solution for it.
I have added Sid and also tried to fix indents but nothing helped.
I have been successfully putting IAM role statements in my serverless.yml for S3 access for some time and assumed everything would work similarly when I added SQS. I took the very simple arn:aws:s3::: and replaced the s3 with SQS, as you can see below:
iam:
role:
statements:
- Effect: 'Allow'
Action:
- 's3:ListBucket'
Resource:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- mahtestbucket
- Effect: 'Allow'
Action:
- 's3:GetObject'
Resource:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- mahtestbucket
- '/*'
- Effect: 'Allow'
Action:
- 's3:ListBucket'
Resource:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- mahfailbucket
- Effect: 'Allow'
Action:
- 's3:PutObject'
Resource:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- mahfailbucket
- '/*'
- Effect: 'Allow'
Action:
- 'sqs:SendMessage'
- 'sqs:GetQueueUrl'
- 'sqs:GetQueueAttributes'
Resource:
Fn::Join:
- ''
- - 'arn:aws:sqs:::'
- mahqueue
This deploys with no errors. I can see the policy statement but it just doesn't work. The lambda function fails when it attempts to access the SQS resource.
I decided to post this with my own answer since it took me a few hours of wasted time to verify the policy was being created and determine what is going on. I believe since S3 does not usually include a region definition, and buckets are global, the simpler specification works. SQS and probably most other resources, however, need these things. Once I determined this might be the issue, it took a little research to get the correct syntax working:
- Effect: 'Allow'
Action:
- 'sqs:SendMessage'
- 'sqs:GetQueueUrl'
- 'sqs:GetQueueAttributes'
- 'sqs:ListQueues'
Resource:
Fn::Join:
- ''
- - !Sub 'arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:'
- mahqueue
I am trying to create an AWS KMS Key Policy and have been plagued trying to get Cloudformation to accept the key policy. Everything I have been able to find and read says this policy should be valid and the syntax is correct as it runs, but returns MalformedPolicyDocumentExceptionnull (Service: AWSKMS; Status Code: 400;
Has anyone else run into this, if so, any thoughts or suggestions on how I can resolve the errors? I've been stuck and banging my head on this one and can't see what I'm missing and my google-fu is failing me.
Code Snippet:
SnowflakeProdKMS:
Type: AWS::KMS::Key
Properties:
Description: KMS key used by Snowflake to encrypt/decrypt data stored in s3
Enabled: True
EnableKeyRotation: False
KeyPolicy:
Version: 2012-10-17
Id: key-default-1
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:root
Action:
- kms:*
Resource: '*'
- Sid: Enable AWSAdminRole to have full permissions to KMS key
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:/role/AWSAdminRole
Action: kms:*
Resource: '*'
- Sid: Allow use of the key by other roles
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:role/AWSAdminRole
# - !Sub arn:aws:iam::${AWS::AccountId}:role/SnowflakeAccessRole
Action:
- kms:Encrypt
- kms:Decrypt
- kms:ReEncrypt
- kms:GenerateDataKey
- kms:DescribeKey
Resource: '*'
- Sid: Allow attachment of persistent resources
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:role/AWSAdminRole
# - !Sub arn:aws:iam::${AWS::AccountId}:role/SnowflakeAccessRole
- !Sub arn:aws:iam::${AWS::AccountId}:root
Action:
- kms:CreateGrant
- kms:ListGrants
- kms:RevokeGrant
Resource: '*'
Condition:
Bool:
- kms:GrantIsForAWSResource: 'true'
After much trial and error and reaching out to other partners I found the solution for the above issue.
The Condition on snippet above was incorrect and should have been formatted as follows:
Condition:
Bool:
"kms:GrantIsForAWSResource": true
Once changed to this the policy went in without issue.
I am trying to access S3 bucket from a Camel Router running in docker container deployed in ECS.
Host EC2 instance has IAM policies which provide access to S3 bucket and I have verified this by running a standalone router directly in EC2. But when I tried the program inside a container it is throwing an exception stating that "Access Denied" to S3 bucket.
After this, I tried providing a Task IAM Role by adding TaskRoleArn in my Cloudformation but that didn't help, still the same exception.
Not sure if I am missing anything here, any suggestions?
[Edit, adding the IAM policy here]
This is my IAM policy for host, similar one I created for Task Role as well with the trust relationship to ‘ecs-tasks.amazonaws.com’
This is my IAM policy for host, similar one I created for Task Role as well with the trust relationship to ‘ecs-tasks.amazonaws.com’
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
- ecs.amazonaws.com
Action:
- sts:AssumeRole
Path: /Developer/
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:CreateLogGroup
- logs:DescribeLogGroups
- logs:DescribeLogStreams
- logs:PutLogEvents
- logs:GetLogEvents
- logs:FilterLogEvents
Resource: '*'
- PolicyName: ep-bucket-policy
PolicyDocument:
Statement:
- Sid: bucket1
Effect: Allow
Action:
- s3:GetObject
- s3:GetObjectAcl
- s3:AbortMultipartUpload
- s3:GetObjectTagging
- s3:DeleteObject
- s3:ListMultipartUploadParts
Resource: *
- Sid: bucket2
Effect: Allow
Action: s3:ListObjects
Resource: "*"
- PolicyName: ep-ecr-policy
PolicyDocument:
Statement:
- Sid: ecr1
Effect: Allow
Action:
- ecr:BatchCheckLayerAvailability
- ecr:GetDownloadUrlForLayer
- ecr:GetRepositoryPolicy
- ecr:DescribeRepositories
- ecr:ListImages
- ecr:BatchGetImage
Resource:
- !Sub 'arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/*’
- Sid: ecr2
Effect: Allow
Action:
- ecr:GetAuthorizationToken
Resource: '*'
- PolicyName: ep-ecs-policy
PolicyDocument:
Statement:
- Sid: ecs1
Effect: Allow
Action:
- ecs:SubmitTaskStateChange
- ecs:RegisterContainerInstance
- ecs:SubmitContainerStateChange
- ecs:DeregisterContainerInstance
Resource: !Sub 'arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/Custom-*’
- Sid: ecs2
Effect: Allow
Action:
- ecs:Poll
- ecs:DiscoverPollEndpoint
- ecs:StartTelemetrySession
- ecs:CreateCluster
Resource: "*"