After deploying a Lambda through SAM, I was getting a 403 inside the Lambda when attempting to download from S3. I checked the Lambda's Role in IAM Management Console, and I saw that the Role only had AWSLambdaBasicExecutionRole. However, it should also have the policies from the SAM template for S3 read/write.
Here is a snippet from my SAM template (with some things renamed):
MyFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
Metadata:
DockerTag: my-tag
DockerContext: ./src/stuff
Dockerfile: Dockerfile
Policies:
- Version: '2012-10-17'
- S3ReadPolicy:
BucketName: !Ref MyBucket
- S3WritePolicy:
BucketName: !Ref MyBucket
Shouldn't the S3ReadPolicy and S3WritePolicy be a part of the Lambda's Role?
Is there something I'm missing?
I know I could manually add the policies needed, but obviously I want as much as possible to be happening automatically via SAM.
Other details: I'm not sure if it matters, but for the sake of additional context, the Lambda is part of a Step Function state machine. I'm using the boto3 library for making the request to download from S3. I get a {'Code': '403', 'Message': 'Forbidden'} error from boto3.
(answering my own question). The issue was the Policies section needed to be under Properties.
MyFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
Policies:
- Version: '2012-10-17'
- S3ReadPolicy:
BucketName: !Ref MyBucket
- S3WritePolicy:
BucketName: !Ref MyBucket
Metadata:
DockerTag: my-tag
DockerContext: ./src/stuff
Dockerfile: Dockerfile
Related
I have the following CloudFormation template:
AWSTemplateFormatVersion: '2010-09-09'
Resources:
HostingBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: "my-hosting-bucket"
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
MyFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Runtime: nodejs12.x
Role: !GetAtt 'LambdaExecutionRole.Arn'
Timeout: 300
Code:
S3Bucket: !Ref 'HostingBucket'
S3Key: !Sub 'lambda/backend.zip'
But while deploying this I get the following error:
Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: Lambda, Status Code: 400, Request ID: SOME-REQUEST-ID)"
Now I understand why the error occurs: my lambda is trying to get the deployed code, but it's not there yet because the S3 is created in the same stack. Probably I will need the following steps:
Create S3 bucket (via CloudFormation)
Deploy lambda code to created S3 bucket
Create lambda (via CloudFormation)
How can I deploy this CloudFormation template before deploying my Lambda handler to S3? I'd like to use the following flow:
Create S3 bucket and lambda (via CloudFormation)
Deploy lambda code to created S3 bucket
Have lambda pick up the newest code (which happens automatically iirc)
You would typically deploy your artifact into S3 during build time in a pipeline.
After reading S3WritePolicy documentation, it's not clear if it allows multiple buckets.
I'm currently doing this:
SampleLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- S3WritePolicy:
BucketName: bucket-1
but if I wanted to include multiple buckets, i.e.:
SampleLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- S3WritePolicy:
BucketName:
- bucket-1
- bucket-2
would this be allowed?
Does S3WritePolicy allow multiple buckets in AWS SAM template?
Yes.
would this be allowed?
No, but the below would be allowed.
This is because it's a SAM policy template & is essentially generating a policy for a single bucket. You can however use it as many times as needed.
SampleLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- S3WritePolicy:
BucketName:
- bucket-1
- S3WritePolicy:
BucketName:
- bucket-2
I'm trying to provide my Lambda function with the S3FullAccessPolicy policy. Note the target bucket is not configured within the template.yaml - it already exists. Considering the syntax examples from this documentation I have three options:
1.AWS managed policy named:
Policies:
- S3FullAccessPolicy
2.AWS SAM policy template (SQSPollerPolicy) defined:
Policies:
- S3FullAccessPolicy:
BucketName: abc-bucket-name
3.Or an inline policy document:
Policies:
- Statement:
...
In trying #1 I get an error that says it seems to suggest I need to provide an arn. If this is the case where would I provide it? The error:
1 validation error detected: Value 'S3FullAccessPolicy' at 'policyArn' failed to satisfy constraint:
Member must have length greater than or equal to 20
For #2 I provide the bucket name but it says that the policy is 'invalid'. I've tried adding quotes and replacing the name with an arn - but no luck.
And #3 - I can find the code for the policy here but that's in yaml so I wonder if that's even what I'm supposed to be using.
What am I missing here? I'm open to using any one of these options but right now I'm 0/3.
The full Lambda function:
testFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: lambda/testFunction/
Handler: app.lambda_handler
Runtime: python3.8
Timeout: 900
Policies:
- S3FullAccessPolicy
I used below template without any issues.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./hello_world/
Handler: app.lambda_handler
Runtime: python3.8
Tracing: Active
Policies:
- S3FullAccessPolicy:
BucketName: existingbucketname # bucket name without arn
Ran it using below command and it deployed successfully.
sam deploy --stack-name sample-stack --s3-bucket bucket-to-deploy --capabilities CAPABILITY_IAM
I've followed the AWS SAM documentation for hands-on learning experience and have reached this particular section: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-s3.html
Above page in the documentation explains how to map the S3 events to any resource and I've done something similar on my local machine. My local template below.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
CreateThumbnail:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.6
Timeout: 60
Policies: AWSLambdaExecute
Events:
CreateThumbnailEvent:
Type: S3
Properties:
Bucket: !Ref testBucket
Events: s3:ObjectCreated:*
testBucket:
Type: AWS::S3::Bucket
Now, when I build and deploy above SAM template, while a new S3 bucket is getting created, the 'object creation' event is not mapped to the Lambda function created. PSB screenshot.
I haven't found any articles or blogs on this error (probably, I'm doing a silly mistake here)
Please help.
Thanks.
I realized that for some reason, the AWS Lambda console is not showing the trigger event but the mapping is successful. I've also validated by uploading sample image to the bucket which triggered the lambda.
The event mapping can be seen in the properties section of the S3 bucket created.
S3 test bucket created -> Properties -> Events
You can add a AWS::Lambda::Permission to your Resources as described in this AWS SAM Github issue
LambdaInvokePermission:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: !GetAtt MyFunction.Arn
Action: 'lambda:InvokeFunction'
Principal: 's3.amazonaws.com'
SourceAccount: !Sub ${AWS::AccountId}
SourceArn: !GetAtt MyBucket.Arn
First create bucket where you plan to save packaged code
aws s3 mb s3://youtbucketname
After sam build
go for
sam package --template-file template.yaml --s3-bucket <yourbucketname>
I followed this tutorial to setup an AWS Lambda function that is invoked upon an upload to S3 and populates DynamoDB.
I'm trying to achieve the same with AWS SAM for which I need to define a template.yaml file with the configuration information. I keep getting this error when deploying with Cloudformation -
Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Circular dependency between resources: [LambdaPerm]
I could not find a lot of information on this so I'm struggling to debug. What's causing this error and how can I resolve this? Here's my template configuration -
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
Gradebook:
Type: AWS::Serverless::Function
Properties:
FunctionName: LambdaGradebookServerless
Handler: serverless.LambdaGradebook
Runtime: java8
CodeUri: ./target/serverless-0.0.1-SNAPSHOT.jar
Role: arn:aws:iam::xxxxxxxxxxxx:role/lambda-s3-execution-role
LambdaPerm:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName:
Ref: Gradebook
Principal: s3.amazonaws.com
SourceAccount:
Ref: AWS::xxxxxxxxxxxx
SourceArn:
Fn::Join:
- ':'
- - arn
- aws
- s3
- ''
- ''
- Ref: gradebookBucket
gradebookBucket:
Type: AWS::S3::Bucket
Properties:
Bucket: gradebook-lambda
NotificationConfiguration:
LambdaConfigurations:
- Event: s3:ObjectCreated:*
Function:
Ref: Gradebook
To avoid this circular dependency, create the S3 bucket and the Lambda function independently, then update the stack with an S3 notification configuration.
If anyone comes searching for this, I wrote a blog to address this scenario.
https://aws.amazon.com/blogs/mt/resolving-circular-dependency-in-provisioning-of-amazon-s3-buckets-with-aws-lambda-event-notifications/
The implementation approach is similar to what jarmod describes in the accepted answer. The event notification is setup later using a CloudFormation custom resource.
I got the circular dependency error and it turns out it was a missing parameter that I was referencing from the resource on the CloudFormation template!.