I want to run a lambda in Account B when any object comes into Account A S3 bucket.
But I heard that we can access Lambda from the same account S3 only, for cross-account S3 Lambda access I must run Lambda within same account and make another trigger which runs another account Lambda:
S3(Account A)--> Lambda(Account B)- not possible
S3(Account A)--> Lambda(Account A)-->Lambda(Account B)- Possible
Can someone help me which option is possible? If so how?
#John's Solution works but there are certain steps I would like to add to his answer.
The S3 bucket and the Lambda need to be in the same region. For example, both should be created in us-east-1 region. Different regions would throw an error as below:
The notification destination service region is not valid for the bucket location constraint
Below is the Steps I followed to create the trigger:
Account-A.S3-bucket -> Account-B.Lambda-function
From Terminal, switch to Account-B's AWS profile where the Lambda would reside
Run the below command, change the parameters for your case:
aws lambda add-permission \
--region {Account-B.Lambda region Eg. us-east-1} \
--function-name {Account-B.Lambda name} \
--statement-id 1 \
--principal s3.amazonaws.com \
--action lambda:InvokeFunction \
--source-arn arn:aws:s3:::{Account-A.S3 name} \
--source-account {Account-A.account-id} \
--profile {Account-B.profile-name}
You might get statement-id exists error, increment statement-id and re-run command again in this case.
Go to Account-A's S3 bucket and under Properties's tab > under Events
Select Add Notification
Add the following fields:
Name: ObjectCreation
Events: ObjectCreate (All)
Send to: Lambda function
Lambda: Add Lambda function ARN
Lambda function ARN:
your-lambda-arn
Note: The Lambda function might still show an error but new objects added in the S3 bucket trigger the lambda and print(event) logs appear in Cloudwatch logs.
I managed to successfully trigger an AWS Lambda function in Account B from an upload to an Amazon S3 bucket in Account A.
Account-A.S3-bucket -> Account-B.Lambda-function
Here's what I did:
Created the Amazon S3 bucket in Account A
Created the Lambda function in Account B
Added a Resource-Based Policy for AWS Lambda to the Lambda function via the AWS Command-Line Interface (CLI) that allowed the S3 bucket to call lambda:InvokeFunction on the Lambda function
Added a Bucket Policy to the S3 bucket to permit GetObject access from anywhere (this should be locked-down further, but was sufficient for the experiment)
Configured an Event for ObjectCreate (All) on the S3 bucket, referencing the Lambda function via its ARN
Uploaded a file to the Account-A.S3-bucket
The Account-B.Lambda-function was successfully triggered
I then repeated the experiment with the bucket in a different region and it failed, saying:
The notification destination service region is not valid for the bucket location constraint
Here is how you do this in clear steps:
I defined (Customer Account) as the account that contains the S3 resource, "Service Account" as the account that contains the Lambda function, that will access the S3 resource.
Create assumed role on Customer Account with full S3 access,
Create trust policy in assumed role pointing at Lambda ARN
Attach IAM policy to Lambda execution role on Service Account - pointing at Customer account / assumed role
(Reference: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-assume-iam-role/)
Create object notification event on target S3 bucket on customer account, to notify Lambda ARN on service account.
(Reference: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)
In the new S3 console, go to S3 console and open your bucket. Click on the Properties tab -> Events. You need to give S3 permission to invoke the Lambda function. Refer: configure Amazon s3 bucket to run Lambda function created in another account
Both options should be possible. So you can go with the first option, which is minimalistic.
Use the Cross Account access feature in IAM to grant access to S3(Account A) from Lambda(Account B).
This is achieved by creating a IAM Role in Account B which is granted to acceses to the bucket in Account A and allowed to assume by the Lambda (In Account B).
For further details refer the following documentation from AWS.
Using Resource-Based Policies for AWS Lambda [Example 2: Bucket
Owner Granting Cross-Account Bucket Permissions
Related
I am trying to follow the aws lambda tutorial, currenty at:
https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
I am on the step to
"add permissions to the function policy"
It says to:
"
Run the following Lambda CLI add-permission command to grant Amazon S3 service principal (s3.amazonaws.com) permissions to perform the lambda:InvokeFunction action. Note that permission is granted to Amazon S3 to invoke the function only if the following conditions are met:
An object-created event is detected on a specific bucket.
The bucket is owned by your account. If you delete a bucket, it is possible for another account to create a bucket with the same ARN.
"
and then it gives a command line command to enter:
aws lambda add-permission --function-name CreateThumbnail --principal s3.amazonaws.com --statement-id s3invoke --action "lambda:InvokeFunction" --source-arn arn:aws:s3:::sourcebucket --source-account account-id
I do not own the bucket I want to trigger on. It is an already established bucket set up by my company and I've been tasked with learning lambda and triggering when a file is uploaded to it.
How do I get the account id to use and can I use an account id other than my own in that command?
Also, is the sourcebucket arn always "arn:aws:s3:::?" Is it diplayed somewhere?
Christopher,
You'll need to enter the account id in which bucket is already created by your company. Moreover, you'll definitely need to enter the arn (amazon resource name) of that bucket for which you're setting up the trigger.
This is how ARN looks like:
arn:aws:s3:::bucket-name
You'll find the account id in account settings on AWS console.
For any bucket's ARN, click on the bucket's row anywhere, it'll open a slide window on right and you can see the button "Copy bucket ARN", you can see the format is always 'arn:aws:s3:::l'
QUESTION: Could you see any trigger for S3 automatically generated inside Lambda function after this command? I cannot and it should be there. Like I see Last modified column updated but cannot see any trigger generated automatically.
My use-case is to push data from one AWS account S3 bucket to another AWS account S3 bucket continuously. A cross account push.
Iām using lambda to do this job.
Assume in AWS account A, data is frequently landed from some source into S3 bucket. I need to create an S3 trigger which will invoke Lambda function in AWS account A and push account A S3 bucket data to another S3 bucket in AWS account B.
Is this possible?
Yes!
Firstly, if the buckets are in different regions, you could use Cross-Region Replication and Amazon S3 will do it all for you automatically.
If not, then your proposed plan looks fine. It would involve:
An Amazon S3 Event to trigger the Lambda function whenever a new object is created
The Lambda function receives the Bucket Name and Key of the new object
The Lambda function should then call CopyObject() to copy the object to the other bucket (in the other account)
The most important element is to give permissions to the Lambda function running in Account-A to write to the bucket in Account-B. This can be done by:
Creating an IAM Role (Role-A) in Account-A that is used by the Lambda function
Adding a Bucket Policy to the bucket in Account-B that permits PutObject from Role-A (by specifying the ARN of Role-A)
I have a Lambda .jar that I build from a Jenkins box in an AWS account ("Account_Bld"). Once built, I copy the .jar over to an S3 bucket in a different AWS account ("Account_Dst"), and I attempt to update the Lambda in Account_Dst based on the newly copied .jar in S3.
I'm using this command as part of my deploy script, which is a slight modification of another version that works when everything is located in the same account:
aws lambda update-function-code --function-name arn:aws:lambda:us-east-1:{Account_Dst_Id}:function:{lambda_function_name} --zip-file fileb://{jar_file_relative_path} --region us-east-1
Not surprisingly, I get this error:
An error occurred (AccessDeniedException) when calling the UpdateFunctionCode operation: User: arn:aws:sts::{Account_Bld_Id}:assumed-role/{jenkins_ec2_role}/{jenkins_ec2_instance_id} is not authorized to perform: lambda:UpdateFunctionCode on resource: arn:aws:lambda:us-east-1:{Account_Dst_Id}:function:{lambda_function_name}
I have given jenkins_ec2_role rights to update the Lambda in the other account, but it makes sense that I would need to reciprocate those rights somewhere in Account_Dst -- assuming there is a simple solution to this problem.
Now, possible resolutions. I could assume a role in Account_Dst that has the correct rights and update the Lambda, but that's more setup hassle than it is worth to me right now. I've seen some Google suggestions that I could use CodePipeline, but obviously I'm using Jenkins, so that doesn't seem like a good solution, either.
So, the question is, is there an easy solution here that I am missing?
This is now possible. A Lambda resource based policy can be configured to allow a principal from another account to perform actions e.g. lambda:UpdateFunctionCode or lambda:Invoke.
In case of UpdateFunctionCode, the documentation states:
FunctionName
The name of the Lambda function.
Name formats
Function name ā my-function.
Function ARN ā arn:aws:lambda:us-west-2:123456789012:function:my-function.
Partial ARN ā 123456789012:function:my-function.
...
Source: https://docs.aws.amazon.com/lambda/latest/dg/API_UpdateFunctionCode.html
The Lambda Function permission in account 222222222222 must be configured to allow the principal from account 111111111111 to update the function code:
aws lambda add-permission --function-name my-function --statement-id xaccount --action lambda:UpdateFunctionCode --principal 111111111111 --output out.txt
Source:
https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html#permissions-resource-xaccountinvoke
Then the function code in account 222222222222 can be updated from account 111111111111:
aws lambda update-function-code --function-name arn:aws:lambda:us-west-2:222222222222:function:my-function --zip-file fileb://soure.zip
Granting permissions in Account_Bld to access Account_Dst is not sufficient to gain access to another account. This is good, because you wouldn't want people granting themselves access to other people's accounts.
The destination account needs to accept the incoming request. The method varies by service. For example, Amazon S3 can create a Bucket Policy to permit access from other accounts, as can Amazon SQS.
However, there is no such concept in Lambda to configure incoming requests from other accounts. There is simply nowhere that can be configured to allow update-function-code from another account.
Therefore, you will need to do as you suggested:
Create an IAM User or IAM Role in Account_Dst
Use the credentials from the Account_Dst IAM User (simplest) or use the existing Account_Bld credentials to assume the Role in Account_Dst (a few more lines of code)
Call update-function-code using those credentials
I want to run a lambda in Account B when any object comes into Account A S3 bucket.
But I heard that we can access Lambda from the same account S3 only, for cross-account S3 Lambda access I must run Lambda within same account and make another trigger which runs another account Lambda:
S3(Account A)--> Lambda(Account B)- not possible
S3(Account A)--> Lambda(Account A)-->Lambda(Account B)- Possible
Can someone help me which option is possible? If so how?
#John's Solution works but there are certain steps I would like to add to his answer.
The S3 bucket and the Lambda need to be in the same region. For example, both should be created in us-east-1 region. Different regions would throw an error as below:
The notification destination service region is not valid for the bucket location constraint
Below is the Steps I followed to create the trigger:
Account-A.S3-bucket -> Account-B.Lambda-function
From Terminal, switch to Account-B's AWS profile where the Lambda would reside
Run the below command, change the parameters for your case:
aws lambda add-permission \
--region {Account-B.Lambda region Eg. us-east-1} \
--function-name {Account-B.Lambda name} \
--statement-id 1 \
--principal s3.amazonaws.com \
--action lambda:InvokeFunction \
--source-arn arn:aws:s3:::{Account-A.S3 name} \
--source-account {Account-A.account-id} \
--profile {Account-B.profile-name}
You might get statement-id exists error, increment statement-id and re-run command again in this case.
Go to Account-A's S3 bucket and under Properties's tab > under Events
Select Add Notification
Add the following fields:
Name: ObjectCreation
Events: ObjectCreate (All)
Send to: Lambda function
Lambda: Add Lambda function ARN
Lambda function ARN:
your-lambda-arn
Note: The Lambda function might still show an error but new objects added in the S3 bucket trigger the lambda and print(event) logs appear in Cloudwatch logs.
I managed to successfully trigger an AWS Lambda function in Account B from an upload to an Amazon S3 bucket in Account A.
Account-A.S3-bucket -> Account-B.Lambda-function
Here's what I did:
Created the Amazon S3 bucket in Account A
Created the Lambda function in Account B
Added a Resource-Based Policy for AWS Lambda to the Lambda function via the AWS Command-Line Interface (CLI) that allowed the S3 bucket to call lambda:InvokeFunction on the Lambda function
Added a Bucket Policy to the S3 bucket to permit GetObject access from anywhere (this should be locked-down further, but was sufficient for the experiment)
Configured an Event for ObjectCreate (All) on the S3 bucket, referencing the Lambda function via its ARN
Uploaded a file to the Account-A.S3-bucket
The Account-B.Lambda-function was successfully triggered
I then repeated the experiment with the bucket in a different region and it failed, saying:
The notification destination service region is not valid for the bucket location constraint
Here is how you do this in clear steps:
I defined (Customer Account) as the account that contains the S3 resource, "Service Account" as the account that contains the Lambda function, that will access the S3 resource.
Create assumed role on Customer Account with full S3 access,
Create trust policy in assumed role pointing at Lambda ARN
Attach IAM policy to Lambda execution role on Service Account - pointing at Customer account / assumed role
(Reference: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-assume-iam-role/)
Create object notification event on target S3 bucket on customer account, to notify Lambda ARN on service account.
(Reference: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)
In the new S3 console, go to S3 console and open your bucket. Click on the Properties tab -> Events. You need to give S3 permission to invoke the Lambda function. Refer: configure Amazon s3 bucket to run Lambda function created in another account
Both options should be possible. So you can go with the first option, which is minimalistic.
Use the Cross Account access feature in IAM to grant access to S3(Account A) from Lambda(Account B).
This is achieved by creating a IAM Role in Account B which is granted to acceses to the bucket in Account A and allowed to assume by the Lambda (In Account B).
For further details refer the following documentation from AWS.
Using Resource-Based Policies for AWS Lambda [Example 2: Bucket
Owner Granting Cross-Account Bucket Permissions
I have a Lambda function written in Java I and I want it to access S3 (putObject).
I do not want to use or store credentials in my Lambda function in order to access S3. Instead, I would like to use IAM roles.
How can I code an AWS S3 client inside my java code (that would be ran by Lambda) that won't need any credentials and assume that the Lambda has the appropriate Role?
You don't need to store credentials in your lambda functions. All funtions run with a role - the role you set when you created the function. Since the lambda function has a role, you can add or remove permissions from this role as needed, without changing the function itself
Manage Permissions: Using an IAM Role (Execution Role)
Each Lambda function has an IAM role (execution role) associated with
it. You specify the IAM role when you create your Lambda function.
Permissions you grant to this role determine what AWS Lambda can do
when it assumes the role. There are two types of permissions that you
grant to the IAM role:
If your Lambda function code accesses other AWS resources, such as to
read an object from an S3 bucket or write logs to CloudWatch Logs, you
need to grant permissions for relevant Amazon S3 and CloudWatch
actions to the role. If the event source is stream-based (Amazon
Kinesis Streams and DynamoDB streams), AWS Lambda polls these streams
on your behalf. AWS Lambda needs permissions to poll the stream and
read new records on the stream so you need to grant the relevant
permissions to this role.
http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html