Event trigger using serverless framwork on an existing S3 bucket - amazon-web-services

I'm trying to set a lambda to trigger when an object is created in an S3 bucket.
My serverless.yml includes this:
handleNewRawObjectInS3:
handler: lambdas/handleNewRawObjectInS3/handleNewRawObjectInS3.handleS3Event
events:
- s3:
bucket: ${file(../evn.${opt:stage, 'dev'}.json):RAW_IMAGE_BUCKET}
event: s3:ObjectCreated:*
This results in an error:
Error:
CREATE_FAILED: S3Bucketxxxxxxxxxrawimages (AWS::S3::Bucket)
xxxxxxxxx-raw-images already exists in stack arn:aws:cloudformation:us-east-1:xxxx:stack/s3-xxxxxxxxx-raw-images/310e7010-xxx-xxx-xxxx-12f066874c93
I already have the bucket created -- created via uploading a cloudformation template directly (our corporate version of AWS doesn't allow us to use serverless framework to create a bucket). How to add S3 trigger event on AWS Lambda function using Serverless framework? indicates this was not possible with older versions of serverless, but following the rabbit hole, you can see a feature request ... and later an answer that shows you need to add 'existing: true'.
So I add that to my serverless framework setup:
service: my-service-events
provider:
name: aws
runtime: nodejs14.x
region: ${file(../evn.${opt:stage, 'dev'}.json):REGION}
stage: ${opt:stage, 'dev'}
deploymentBucket: #must name manually-created bucket for deployment because enterprise doesn't allow automated bucket creation
name: ${file(../evn.${opt:stage, 'dev'}.json):DEPLOYMENT_BUCKET}
iam: #must name a role because enterprise doesn't allow automated role creation
role: myServerlessRole # This is a reference to the resource name from the role created in the resources -> iam roles section
deploymentRole: ${file(../evn.${opt:stage, 'dev'}.json):DEPLOYMENT_ROLE}
resources:
- ${file(../iam-roles.${opt:stage, 'dev'}.yml)}
functions:
handleNewRawObjectInS3:
handler: lambdas/handleNewRawObjectInS3/handleNewRawObjectInS3.handleS3Event
events:
- s3:
bucket: ${file(../evn.${opt:stage, 'dev'}.json):RAW_IMAGE_BUCKET}
event: s3:ObjectCreated:*
existing: true
The IAM file/role referenced above looks like this:
Resources:
myServerlessRole:
Type: AWS::IAM::Role
Properties:
PermissionsBoundary: arn:aws:iam::xxx:policy/csr-Developer-Permissions-Boundary
Path: /my/default/path/
RoleName: myServerlessRole-${self:service}
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Policies:
- PolicyName: myPolicyName
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- 'Fn::Join':
- ':'
-
- 'arn:aws:logs'
- Ref: 'AWS::Region'
- Ref: 'AWS::AccountId'
- 'log-group:/aws/lambda/*:*:*'
- Effect: "Allow"
Action:
- "s3:*"
Resource: "arn:aws:s3:::*" #
- Effect: "Allow"
Action:
- "lambda:*"
Resource: "*"
- Effect: "Allow"
Action:
- cloudfront:CreateDistribution
- cloudfront:GetDistribution
- cloudfront:UpdateDistribution
- cloudfront:DeleteDistribution
- cloudfront:TagResource
Resource: "arn:aws:cloudfront:::*"
Trying to deploy this gets me the error:
Error:
CREATE_FAILED: CustomDashresourceDashexistingDashs3LambdaFunction (AWS::Lambda::Function)
Resource handler returned message: "The role defined for the function cannot be assumed by Lambda. (Service: Lambda, Status Code: 400, Request ID: 812c5384-1c26-42c9-bdef-1ce4a59f2be4)" (RequestToken: 9cdbb5af-3bc7-d6bf-384b-5126d1048ccd, HandlerErrorCode: InvalidRequest)
How can I deploy this lambda triggered by an s3 event?

The issue comes from the fact that CustomDashresourceDashexistingDashs3LambdaFunction lambda runs as the deployementRole and not under the defined role (which is the default role that lambdas run under). Given the deployment role doesn't normally need to assumeRole my deployment role did not have the assumeRole permission.
The fix for this is to ensure that the sts:assumeRole trust relationship has been applied to the deploymentRole like so:
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}

Related

AWS event bridge rule role not attaching to event bridge when creating via CloudFormation

I have a Cloudformation template to create an event bridge rule with target to a central event bus running in another account. When i run the below code , both IAM role and event bridge is getting created but the IAM role is not getting attached to eventbridge rule. Below is the yaml template i am using.
Please see the attached screenshot also.
AWSTemplateFormatVersion: 2010-09-09
Resources:
EventRuleRegion1:
Type: AWS::Events::Rule
Properties:
Description: Event rule to send events to monitoring account event bus
EventBusName: default
EventPattern:
source:
- aws.ec2
Name: ec2-lifecycle-events2
RoleArn: !GetAtt
- EventBridgeIAMrole
- Arn
State: ENABLED
Targets:
- Arn: >-
arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
Id: 'central-eventbus-sydney'
EventBridgeIAMrole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: !Sub events.amazonaws.com
Action: 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: PutEventsDestinationBus
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'events:PutEvents'
Resource:
- >-
arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
Manually Created Event rule which shows the role created and attached.
Policy in the role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"events:PutEvents"
],
"Resource": [
"arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney"
]
}
]
}
Cross-account permissions are not set using role, but EventBus resource permissions. From [docs][1]:
The permissions for an event bus are granted or denied using a resource-based policy attached to the event bus.
To do this in CloudFormation, you have to develop your own custom resource.
Update
You haven't specified RoleArn for your target. This is different RoleArn that you have now.

Kinesis Firehose Delivery Stream is unable to assume role while using cloudformation

I am writing a cloudformation template that creates a Kinesis Firehose Delivery Stream and sends the data to S3 bucket. The source stream is a Kinesis Steam. It creates the s3 bucket, Policies, and roles but when it tries to create the Kinesis Firehose Delivery Stream, it fails saying unable to assume role
After some research i found that Delivery should not be created using the root account. I tried creating a new user but it still gave me the same error.
# creates the Kinesis Stream
KinesisStream:
Type: AWS::Kinesis::Stream
Properties:
Name: HealthApp
RetentionPeriodHours: 24
ShardCount: 8
# creates the firehose delivery stream
KinesisFirehoseDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: HealthAppFirehose
DeliveryStreamType: KinesisStreamAsSource
KinesisStreamSourceConfiguration:
KinesisStreamARN:
Fn::GetAtt:
- KinesisStream
- Arn
RoleARN:
Fn::GetAtt:
- FirehoseDeliveryIAMRole
- Arn
S3DestinationConfiguration:
BucketARN: !GetAtt MyS3Bucket.Arn
Prefix: cloudformation-test/kinesis-fh
BufferingHints:
IntervalInSeconds: 60
SizeInMBs: 100
CloudWatchLoggingOptions:
Enabled: 'false'
CompressionFormat: UNCOMPRESSED
RoleARN:
Fn::GetAtt:
- FirehoseDeliveryIAMRole
- Arn
DependsOn:
- FirehoseDeliveryIAMPolicy
FirehoseDeliveryIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
Effect: Allow
Principal:
Service: firehose.amazonaws.com
Action: sts:AssumeRole
Condition:
StringEquals:
sts:ExternalId: ACCOUNT_NUMBER
FirehoseDeliveryIAMPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: HealthAppPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:AbortMultipartUpload
- s3:GetBucketLocation
- s3:GetObject
- s3:ListBucket
- s3:ListBucketMultipartUploads
- s3:PutObject
Resource:
- arn:aws:s3:::health-app-bucket/cloudformation-test/kinesis-fh*
- Effect: Allow
Action:
- kinesis:DescribeStream
- kinesis:GetShardIterator
- kinesis:GetRecords
Resource:
Fn::GetAtt:
- KinesisStream
- Arn
Roles:
- Ref: FirehoseDeliveryIAMRole
DependsOn:
- KinesisStream
​
Outputs:
kinesisStreamArn:
Description: Kinesis Stream ARN
Value:
Fn::GetAtt:
- KinesisStream
- Arn
firehoseDeliveryStreamArn:
Description: Firehose Delivery Stream ARN
Value:
Fn::GetAtt:
- KinesisFirehoseDeliveryStream
- Arn
firehoseDeliveryRoleArn:
Description: Firehose Delivery Role ARN
Value:
Fn::GetAtt:
- FirehoseDeliveryIAMRole
- Arn
I want the delivery stream to succesfully be created. Any help would be appreciated.
Thank you
Two things to check for:
I wonder if ACCOUNT_NUMBER is being set and interpreted properly. You can check this by removing the entire Condition statement as a test. As a test (not for production) remove the following and see if it works
Condition:
StringEquals:
sts:ExternalId: ACCOUNT_NUMBER
Does the user you created have access to create IAM policies and roles? You said
I tried creating a new user but it still gave me the same error
Again, for testing/debugging only you can give this user the following policy and see if it works
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "iam:*",
"Resource": "*"
}
}
If that is the problem this use this to determine the actual polices needed for your IAM User that is executing the CloudFormation

AWS Lambda and IAM error on deploy: The role defined for the function cannot be assumed by Lambda

In my AWS project, I use the serverless framework to deploy lambda function and IAM roles.
So I created 6 lambda functions, all using the same IAM Role below:
functions:
auto-delete-identity:
handler: src/auto-delete-identity.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: auto-delete-identity
auto-move-to-user-group:
handler: src/auto-move-to-user-group.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: auto-move-to-user-group
auto-validate-user-creation:
handler: src/auto-validate-user-creation.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: auto-validate-user-creation
auto-validation-user-email-modification:
handler: src/auto-validation-user-email-modification.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: auto-validation-user-email-modification
hello-demo:
handler: src/hello-demo.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: hello-demo
reset-user-password:
handler: src/reset-user-password.handler
role: arn:aws:iam::123456789012:role/lambdaIAMRole
name: reset-user-password
resources:
Resources:
lambdaIAMRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: lambdaIAMRole
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "sts:AssumeRole"
Effect: "Allow"
Principal:
Service:
- "lambda.amazonaws.com"
Policies:
- PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
Effect: "Allow"
Resource:
- !Sub "arn:aws:logs:eu-central-1:123456789012:log-group:/aws/lambda/*:*"
PolicyName: "myLambdaPolicy"
When I deploy using the serverless deploy command, I sometimes got the following error:
An error occurred: HelloDashdemoLambdaFunction - The role defined for the function cannot be assumed by Lambda. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 4099072a-809d-4f1c-b83e-7f4f5dd5170b).
It looks like a random bug, since it doesn’t occurs everytime. Also, when it occurs, it doesn’t always occurs on the same function.
Did I do something wrong? How can I fix that?
Thanks for your help.
I think that the problem is that in your Lambda function declarations, you're referring to the IAM role as role: arn:aws:iam::123456789012:role/lambdaIAMRole. This is an absolute ARN and is how you would indicate an IAM role (or other resource) that was created and managed outside of your serverless.yml template.
In your case, the quickest fix is to simply replace role: arn:aws:iam::123456789012:role/lambdaIAMRole with role: lambdaIAMRole. The latter refers to an AWS resource declared inside the template.
An even better fix, assuming that all of your Lambda functions will have the same role, is to remove your lambdaIAMRole declaration entirely and then remove all role: arn:aws:iam::123456789012:role/lambdaIAMRole properties from the Lambda functions. The role declaration adds nothing over the default IAM role that the Serverless Framework will implicitly generate for you and assign to the Lambda functions. This is one of the things that makes the framework valuable - it provides good defaults to save you the time and effort. Examples here.

Use External Id in cross-account role of AWS using Serverless Framework

Here is the problem: I need to use the Lambda function in AWS Account A (In root AWS Account A) to write some data to the DynamoDB tables in AWS Account B (in root AWS Account B). All the project in written with Serverless Framework in Node.js.
I know I need to use Cross-Account Role in the Lambda function to do that. Since Serverless use the template that shares a lot of things similar to the cloudformation template. Then I did some research on how to use the cross account role, here is the AWS Documentation https://docs.aws.amazon.com/zh_cn/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Principal": {"AWS": "Example Corp's AWS Account ID"},
"Condition": {"StringEquals": {"sts:ExternalId": "12345"}}
}
}
this is the sample iam specification for the task like this. The yaml template in the serverless looks like this in my project:
- Effect: Allow
Principal:
AWS: 'AWS Account B External ID'
Action:
- sts:AssumeRole
Resource:
- '*'
But When I tried to deploy the serverless template:
I got the error like this:
An error occurred: IamRoleLambdaExecution - Policy document should not specify a principal. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: XXXX-XXXX-XXXX-XXXX-XXXX).
I am wondering what is the correct way to specify the template according to the demand.
Appreciate to the help of any kinds
You can allow for assumed roles using an external ID by creating an AWS::IAM:Role resource. The example below gives access to an S3 bucket, but the approach is the same.
ExternalS3AccessRole:
Type: AWS::IAM::Role
Properties:
RoleName: SomeRoleName
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
AWS: "Example Corp's AWS Account ID"
Condition:
StringEquals:
'sts:ExternalId': "12345"
Policies:
- PolicyName: ExternalS3AccessPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:Get*
- s3:List*
Resource:
- "arn:aws:s3:::<BUCKET_NAME>/*"
- Effect: Allow
Action:
- s3:Get*
- s3:List*
Resource:
- "arn:aws:s3:::<BUCKET_NAME>"

AWS CodePipeline error: Cross-account pass role is not allowed

I am trying to create an AWS CodePipeline that deploys the production code to a separate account. The code consists of a lambda function which is setup using a sam template and cloudformation. I have it currently deploying to the same account without error. I added another stage that has a manual approval action and after approval it should deploy to the other account. It fails with the following error:
Cross-account pass role is not allowed (Service: AmazonCloudFormation; Status Code: 403; Error Code: AccessDenied; Request ID: d880bdd7-fe3f-11e7-8a8c-7dcffeae19ae)
I have a role in the production account that has a trust relationship back to the dev account that has the pipeline. I gave the pipeline role and the production role administrator policies just to make sure it was not a policy issue. I edited the pipeline using the technique in this walkthrough. I am following the walkthrough loosely since they are setting their scenario up just slightly different from what I am doing.
The deploy section in my pipeline looks like:
{
"name": "my-stack",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "CloudFormation",
"version": "1"
},
"runOrder": 2,
"configuration": {
"ActionMode": "CHANGE_SET_REPLACE",
"Capabilities": "CAPABILITY_IAM",
"ChangeSetName": "ProductionChangeSet",
"RoleArn": "arn:aws:iam::000000000000:role/role-to-assume",
"StackName": "MyProductionStack",
"TemplatePath": "BuildArtifact::NewSamTemplate.yaml"
},
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "BuildArtifact"
}
]
}
I am able to assume into the role in the production account using the console. I am not sure how passrole is different but from everything I have read it requires the same assume role trust relationship.
How can I configure IAM for cross account pipelines?
Generally, if you want to do anything across multiple accounts you have to allow this on the both sides. This is done via role-assuming.
The pipeline distributed parts communicate via pipeline artifacts which are saved in a S3 bucket and de/encrypted with a KMS encryption key. This key must be accesible from all the accounts where the pipeline is distributed in.
key in the CI account
KMSKey:
Type: AWS::KMS::Key
Properties:
EnableKeyRotation: true
KeyPolicy:
Version: "2012-10-17"
Id: pipeline-kms-key
Statement:
- Sid: Allows admin of the key
Effect: Allow
Principal:
AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
Action: ["kms:*"]
Resource: "*"
- Sid: Allow use of the key from the other accounts
Effect: Allow
Principal:
AWS:
- !Sub "arn:aws:iam::${DevAccountId}:root"
- !GetAtt CodePipelineRole.Arn
Action:
- kms:Encrypt
- kms:Decrypt
- kms:ReEncrypt*
- kms:GenerateDataKey*
- kms:DescribeKey
Resource: "*"
KMSAlias:
Type: AWS::KMS::Alias
Properties:
AliasName: !Sub alias/codepipeline-crossaccounts
TargetKeyId: !Ref KMSKey
The S3 bucket must allow the access from different accounts via a policy:
pipeline stack in the CI account
S3ArtifactBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref S3ArtifactBucket
PolicyDocument:
Statement:
- Action: ["s3:*"]
Effect: Allow
Resource:
- !Sub "arn:aws:s3:::${S3ArtifactBucket}"
- !Sub "arn:aws:s3:::${S3ArtifactBucket}/*"
Principal:
AWS:
- !GetAtt CodePipelineRole.Arn
- !Sub "arn:aws:iam::${DevAccountId}:role/cross-account-role"
- !Sub "arn:aws:iam::${DevAccountId}:role/cloudformation-role"
CodePipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Type: S3
Location: !Ref S3ArtifactBucket
EncryptionKey:
Id: !Ref KMSKey
Type: KMS
...
The pipeline (CI account) has to have a permission to assume a role in the other (DEV) account:
pipeline stack in the CI account
CodePipelinePolicy:
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Statement:
- Action: ["sts:AssumeRole"]
Resource: !Sub "arn:aws:iam::${DevAccountId}:role/cross-account-role
Effect: Allow
...
And that role has to allow to be assumed to the pipeline:
pipeline stack in the DEV account
CrossAccountRole:
Type: AWS::IAM::Role
Properties:
RoleName: cross-account-role
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS: !Sub "arn:aws:iam::${CIAccountId}:root"
Action: sts:AssumeRole
CrossAccountPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: CrossAccountPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- cloudformation:*
- codebuild:*
- s3:*
- iam:PassRole
Resource: "*"
- Effect: Allow
Action: ["kms:Decrypt", "kms:Encrypt"]
Resource: !Ref KMSKey
Roles: [!Ref CrossAccountRole]
The pipeline (managed and executed from the CI account) must assume a role from the other account to execute the action from within the account:
pipeline stack in the CI account
CodePipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: pipeline
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
...
- Name: StagingDev
Actions:
- Name: create-changeset
InputArtifacts:
- Name: BuildArtifact
OutputArtifacts: []
ActionTypeId:
Category: Deploy
Owner: AWS
Version: "1"
Provider: CloudFormation
Configuration:
StackName: app-stack-dev
ActionMode: CHANGE_SET_REPLACE
ChangeSetName: app-changeset-dev
Capabilities: CAPABILITY_NAMED_IAM
TemplatePath: "BuildArtifact::template.yml"
RoleArn: !Sub "arn:aws:iam::${DevAccountId}:role/cloudformation-role" # the action will be executed with this role
RoleArn: !Sub "arn:aws:iam::${DevAccountId}:role/cross-account-role" # the pipeline assume this role to execute this action
...
The code above shows how to execute a CloudFormation action in a different account, the approach is the same for different actions like CodeBuild or CodeDeploy.
There is a nice sample https://github.com/awslabs/aws-refarch-cross-account-pipeline from AWS team.
Another example is here https://github.com/adcreare/cloudformation/tree/master/code-pipeline-cross-account
Or you can take a look at my whole working code here https://github.com/ttulka/aws-samples/tree/master/cross-account-pipeline
I think the issue is that your CloudFormation role is in the other account but your action role is not. Only the pipeline role is allowed to assume an action role in a different account.
The action role is the one located directly under the ActionDeclaration.
Basically your roles should be configured as follows:
Pipeline role: Account A
Action role: Account B
CloudFormation role: Account B
There's some information on setting up cross-account actions here: https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-create-cross-account.html
Here's where the action role is defined: https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ActionDeclaration.html