I am trying to follow the aws lambda tutorial, currenty at:
https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
I am on the step to
"add permissions to the function policy"
It says to:
"
Run the following Lambda CLI add-permission command to grant Amazon S3 service principal (s3.amazonaws.com) permissions to perform the lambda:InvokeFunction action. Note that permission is granted to Amazon S3 to invoke the function only if the following conditions are met:
An object-created event is detected on a specific bucket.
The bucket is owned by your account. If you delete a bucket, it is possible for another account to create a bucket with the same ARN.
"
and then it gives a command line command to enter:
aws lambda add-permission --function-name CreateThumbnail --principal s3.amazonaws.com --statement-id s3invoke --action "lambda:InvokeFunction" --source-arn arn:aws:s3:::sourcebucket --source-account account-id
I do not own the bucket I want to trigger on. It is an already established bucket set up by my company and I've been tasked with learning lambda and triggering when a file is uploaded to it.
How do I get the account id to use and can I use an account id other than my own in that command?
Also, is the sourcebucket arn always "arn:aws:s3:::?" Is it diplayed somewhere?
Christopher,
You'll need to enter the account id in which bucket is already created by your company. Moreover, you'll definitely need to enter the arn (amazon resource name) of that bucket for which you're setting up the trigger.
This is how ARN looks like:
arn:aws:s3:::bucket-name
You'll find the account id in account settings on AWS console.
For any bucket's ARN, click on the bucket's row anywhere, it'll open a slide window on right and you can see the button "Copy bucket ARN", you can see the format is always 'arn:aws:s3:::l'
QUESTION: Could you see any trigger for S3 automatically generated inside Lambda function after this command? I cannot and it should be there. Like I see Last modified column updated but cannot see any trigger generated automatically.
Related
This AWS CLI command:
aws lambda add-permission --function-name my_test_Lambda_fn --statement-id test_id --principal iotanalytics.amazonaws.com --action lambda:InvokeFunction
Gives the following output:
{
"Statement": "{\"Sid\":\"test_id \",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"iotanalytics.amazonaws.com\"}...
}
I wanted to see the above in IAM console, so I tried looking at the roles used by IoTAnalytics and Lambda; and in their respective policies; but the above output is not part of any policy. Where in the IAM can I see the policy configured by the above AWS CLI command? I think that it configured at principal level, but where do I see it in the IAM console?
This isn't actually an IAM policy (although it might resemble one). In fact it is a type of resource policy (in this case named Function policy).
Certain services such as S3, SNS, SQS and in this case Lambda have the ability to have a policy attached which dictates how other entities can interact with them such as other AWS accounts or services that do not support an attached IAM role.
From within the console on the Lambda function itself access the Permissions tab, then at the bottom of the page is a sub-item named Resource-based policy. This will contain the policy that you have added.
You have to go into the Lambda console, select your Function and then you can click on Permissions to see the permissions attached to your lambda.
These are resource-based permission for the lambda function. You can view them in AWS console -> Permissions -> Resource-based policy:
I have a Lambda .jar that I build from a Jenkins box in an AWS account ("Account_Bld"). Once built, I copy the .jar over to an S3 bucket in a different AWS account ("Account_Dst"), and I attempt to update the Lambda in Account_Dst based on the newly copied .jar in S3.
I'm using this command as part of my deploy script, which is a slight modification of another version that works when everything is located in the same account:
aws lambda update-function-code --function-name arn:aws:lambda:us-east-1:{Account_Dst_Id}:function:{lambda_function_name} --zip-file fileb://{jar_file_relative_path} --region us-east-1
Not surprisingly, I get this error:
An error occurred (AccessDeniedException) when calling the UpdateFunctionCode operation: User: arn:aws:sts::{Account_Bld_Id}:assumed-role/{jenkins_ec2_role}/{jenkins_ec2_instance_id} is not authorized to perform: lambda:UpdateFunctionCode on resource: arn:aws:lambda:us-east-1:{Account_Dst_Id}:function:{lambda_function_name}
I have given jenkins_ec2_role rights to update the Lambda in the other account, but it makes sense that I would need to reciprocate those rights somewhere in Account_Dst -- assuming there is a simple solution to this problem.
Now, possible resolutions. I could assume a role in Account_Dst that has the correct rights and update the Lambda, but that's more setup hassle than it is worth to me right now. I've seen some Google suggestions that I could use CodePipeline, but obviously I'm using Jenkins, so that doesn't seem like a good solution, either.
So, the question is, is there an easy solution here that I am missing?
This is now possible. A Lambda resource based policy can be configured to allow a principal from another account to perform actions e.g. lambda:UpdateFunctionCode or lambda:Invoke.
In case of UpdateFunctionCode, the documentation states:
FunctionName
The name of the Lambda function.
Name formats
Function name – my-function.
Function ARN – arn:aws:lambda:us-west-2:123456789012:function:my-function.
Partial ARN – 123456789012:function:my-function.
...
Source: https://docs.aws.amazon.com/lambda/latest/dg/API_UpdateFunctionCode.html
The Lambda Function permission in account 222222222222 must be configured to allow the principal from account 111111111111 to update the function code:
aws lambda add-permission --function-name my-function --statement-id xaccount --action lambda:UpdateFunctionCode --principal 111111111111 --output out.txt
Source:
https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html#permissions-resource-xaccountinvoke
Then the function code in account 222222222222 can be updated from account 111111111111:
aws lambda update-function-code --function-name arn:aws:lambda:us-west-2:222222222222:function:my-function --zip-file fileb://soure.zip
Granting permissions in Account_Bld to access Account_Dst is not sufficient to gain access to another account. This is good, because you wouldn't want people granting themselves access to other people's accounts.
The destination account needs to accept the incoming request. The method varies by service. For example, Amazon S3 can create a Bucket Policy to permit access from other accounts, as can Amazon SQS.
However, there is no such concept in Lambda to configure incoming requests from other accounts. There is simply nowhere that can be configured to allow update-function-code from another account.
Therefore, you will need to do as you suggested:
Create an IAM User or IAM Role in Account_Dst
Use the credentials from the Account_Dst IAM User (simplest) or use the existing Account_Bld credentials to assume the Role in Account_Dst (a few more lines of code)
Call update-function-code using those credentials
I want to run a lambda in Account B when any object comes into Account A S3 bucket.
But I heard that we can access Lambda from the same account S3 only, for cross-account S3 Lambda access I must run Lambda within same account and make another trigger which runs another account Lambda:
S3(Account A)--> Lambda(Account B)- not possible
S3(Account A)--> Lambda(Account A)-->Lambda(Account B)- Possible
Can someone help me which option is possible? If so how?
#John's Solution works but there are certain steps I would like to add to his answer.
The S3 bucket and the Lambda need to be in the same region. For example, both should be created in us-east-1 region. Different regions would throw an error as below:
The notification destination service region is not valid for the bucket location constraint
Below is the Steps I followed to create the trigger:
Account-A.S3-bucket -> Account-B.Lambda-function
From Terminal, switch to Account-B's AWS profile where the Lambda would reside
Run the below command, change the parameters for your case:
aws lambda add-permission \
--region {Account-B.Lambda region Eg. us-east-1} \
--function-name {Account-B.Lambda name} \
--statement-id 1 \
--principal s3.amazonaws.com \
--action lambda:InvokeFunction \
--source-arn arn:aws:s3:::{Account-A.S3 name} \
--source-account {Account-A.account-id} \
--profile {Account-B.profile-name}
You might get statement-id exists error, increment statement-id and re-run command again in this case.
Go to Account-A's S3 bucket and under Properties's tab > under Events
Select Add Notification
Add the following fields:
Name: ObjectCreation
Events: ObjectCreate (All)
Send to: Lambda function
Lambda: Add Lambda function ARN
Lambda function ARN:
your-lambda-arn
Note: The Lambda function might still show an error but new objects added in the S3 bucket trigger the lambda and print(event) logs appear in Cloudwatch logs.
I managed to successfully trigger an AWS Lambda function in Account B from an upload to an Amazon S3 bucket in Account A.
Account-A.S3-bucket -> Account-B.Lambda-function
Here's what I did:
Created the Amazon S3 bucket in Account A
Created the Lambda function in Account B
Added a Resource-Based Policy for AWS Lambda to the Lambda function via the AWS Command-Line Interface (CLI) that allowed the S3 bucket to call lambda:InvokeFunction on the Lambda function
Added a Bucket Policy to the S3 bucket to permit GetObject access from anywhere (this should be locked-down further, but was sufficient for the experiment)
Configured an Event for ObjectCreate (All) on the S3 bucket, referencing the Lambda function via its ARN
Uploaded a file to the Account-A.S3-bucket
The Account-B.Lambda-function was successfully triggered
I then repeated the experiment with the bucket in a different region and it failed, saying:
The notification destination service region is not valid for the bucket location constraint
Here is how you do this in clear steps:
I defined (Customer Account) as the account that contains the S3 resource, "Service Account" as the account that contains the Lambda function, that will access the S3 resource.
Create assumed role on Customer Account with full S3 access,
Create trust policy in assumed role pointing at Lambda ARN
Attach IAM policy to Lambda execution role on Service Account - pointing at Customer account / assumed role
(Reference: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-assume-iam-role/)
Create object notification event on target S3 bucket on customer account, to notify Lambda ARN on service account.
(Reference: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)
In the new S3 console, go to S3 console and open your bucket. Click on the Properties tab -> Events. You need to give S3 permission to invoke the Lambda function. Refer: configure Amazon s3 bucket to run Lambda function created in another account
Both options should be possible. So you can go with the first option, which is minimalistic.
Use the Cross Account access feature in IAM to grant access to S3(Account A) from Lambda(Account B).
This is achieved by creating a IAM Role in Account B which is granted to acceses to the bucket in Account A and allowed to assume by the Lambda (In Account B).
For further details refer the following documentation from AWS.
Using Resource-Based Policies for AWS Lambda [Example 2: Bucket
Owner Granting Cross-Account Bucket Permissions
I want to run a lambda in Account B when any object comes into Account A S3 bucket.
But I heard that we can access Lambda from the same account S3 only, for cross-account S3 Lambda access I must run Lambda within same account and make another trigger which runs another account Lambda:
S3(Account A)--> Lambda(Account B)- not possible
S3(Account A)--> Lambda(Account A)-->Lambda(Account B)- Possible
Can someone help me which option is possible? If so how?
#John's Solution works but there are certain steps I would like to add to his answer.
The S3 bucket and the Lambda need to be in the same region. For example, both should be created in us-east-1 region. Different regions would throw an error as below:
The notification destination service region is not valid for the bucket location constraint
Below is the Steps I followed to create the trigger:
Account-A.S3-bucket -> Account-B.Lambda-function
From Terminal, switch to Account-B's AWS profile where the Lambda would reside
Run the below command, change the parameters for your case:
aws lambda add-permission \
--region {Account-B.Lambda region Eg. us-east-1} \
--function-name {Account-B.Lambda name} \
--statement-id 1 \
--principal s3.amazonaws.com \
--action lambda:InvokeFunction \
--source-arn arn:aws:s3:::{Account-A.S3 name} \
--source-account {Account-A.account-id} \
--profile {Account-B.profile-name}
You might get statement-id exists error, increment statement-id and re-run command again in this case.
Go to Account-A's S3 bucket and under Properties's tab > under Events
Select Add Notification
Add the following fields:
Name: ObjectCreation
Events: ObjectCreate (All)
Send to: Lambda function
Lambda: Add Lambda function ARN
Lambda function ARN:
your-lambda-arn
Note: The Lambda function might still show an error but new objects added in the S3 bucket trigger the lambda and print(event) logs appear in Cloudwatch logs.
I managed to successfully trigger an AWS Lambda function in Account B from an upload to an Amazon S3 bucket in Account A.
Account-A.S3-bucket -> Account-B.Lambda-function
Here's what I did:
Created the Amazon S3 bucket in Account A
Created the Lambda function in Account B
Added a Resource-Based Policy for AWS Lambda to the Lambda function via the AWS Command-Line Interface (CLI) that allowed the S3 bucket to call lambda:InvokeFunction on the Lambda function
Added a Bucket Policy to the S3 bucket to permit GetObject access from anywhere (this should be locked-down further, but was sufficient for the experiment)
Configured an Event for ObjectCreate (All) on the S3 bucket, referencing the Lambda function via its ARN
Uploaded a file to the Account-A.S3-bucket
The Account-B.Lambda-function was successfully triggered
I then repeated the experiment with the bucket in a different region and it failed, saying:
The notification destination service region is not valid for the bucket location constraint
Here is how you do this in clear steps:
I defined (Customer Account) as the account that contains the S3 resource, "Service Account" as the account that contains the Lambda function, that will access the S3 resource.
Create assumed role on Customer Account with full S3 access,
Create trust policy in assumed role pointing at Lambda ARN
Attach IAM policy to Lambda execution role on Service Account - pointing at Customer account / assumed role
(Reference: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-assume-iam-role/)
Create object notification event on target S3 bucket on customer account, to notify Lambda ARN on service account.
(Reference: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)
In the new S3 console, go to S3 console and open your bucket. Click on the Properties tab -> Events. You need to give S3 permission to invoke the Lambda function. Refer: configure Amazon s3 bucket to run Lambda function created in another account
Both options should be possible. So you can go with the first option, which is minimalistic.
Use the Cross Account access feature in IAM to grant access to S3(Account A) from Lambda(Account B).
This is achieved by creating a IAM Role in Account B which is granted to acceses to the bucket in Account A and allowed to assume by the Lambda (In Account B).
For further details refer the following documentation from AWS.
Using Resource-Based Policies for AWS Lambda [Example 2: Bucket
Owner Granting Cross-Account Bucket Permissions
Is it possible to get the ARN of an S3 bucket via the AWS command line?
I have looked through the documentation for aws s3api ... and aws s3 ... and have not found a way to do this.
It's always arn:PARTITION:s3:::NAME-OF-YOUR-BUCKET. If you know the name of the bucket and in which partition it's located, you know the ARN. No need to 'get' it from anywhere.
The PARTITION will be aws, aws-us-gov, or aws-cndepending on whether you're in general AWS, GovCloud, or China resepectively.
You can also select your S3 bucket ARN by selecting it using the tick mark at the s3 management console, which will pop up a Side bar. where there is a provision to copy your S3 bucket ARN.S3 management console with bucket ARN
aws articles spell out the arn format but never say go here to see it. Highlighting my s3 bucket and seeing that Copy Bucket ARN kept me sane for a few more hours.