I'm stuck on a missing permissions issue trying to create a Lambda function.
The execution role I've configured has the following permissions:
$ aws --output=text iam get-role-policy --policy-name=MyRolePolicy --role-name=my-role
<snip>
POLICYDOCUMENT 2012-10-17
STATEMENT Allow
ACTION s3:Get*
ACTION s3:List*
ACTION logs:CreateLogGroup
ACTION logs:CreateLogStream
ACTION logs:PutLogEvents
ACTION ec2:DescribeNetworkInterfaces
ACTION ec2:CreateNetworkInterface
ACTION ec2:DeleteNetworkInterface
And when I create a Lambda function with that role, creation succeeds:
$ aws lambda create-function \
--function-name=my-test --runtime=java8 \
--role='arn:aws:iam::1234567890:role/my-role' \
--handler=MyHandler \
--code=S3Bucket=my-bucket,S3Key=app.zip
<result successful>
However, when I create the function using the same arguments (esp. the same execution role) I get the following error:
Boto3 Usage
client.create_function(
FunctionName=function_name,
Runtime='java8',
Role=getenv('execution_role_arn'),
Handler='MyHandler',
Code={
"S3Bucket": bucket,
"S3Key": artifact_name
},
Publish=True,
VpcConfig={
'SubnetIds': getenv('vpc_subnet_ids').split(','),
'SecurityGroupIds': getenv('vpc_security_group_ids').split(',')
}
)
Boto3 Result
{
'Error':{
'Message':'The provided execution role does not have permissions to call CreateNetworkInterface on EC2',
'Code':'InvalidParameterValueException'
},
'ResponseMetadata':{
'RequestId':'47b6640a-f3fe-4550-8ac3-38cfb2842461',
'HTTPStatusCode':400,
'HTTPHeaders':{
'date':'Wed, 24 Jul 2019 10:55:44 GMT',
'content-type':'application/json',
'content-length':'119',
'connection':'keep-alive',
'x-amzn-requestid':'47b6640a-f3fe-4550-8ac3-38cfb2842461',
'x-amzn-errortype':'InvalidParameterValueException'
},
'RetryAttempts':0
}
}
Creating a function via the console with this execution role works as well, so I must be missing something in how I'm using Boto3, but I'm at a loss to explain.
Hopefully someone can catch a misapplication of Boto3 here, cause I'm at a loss!
Your boto3 code is specifying a VPC:
VpcConfig={
'SubnetIds': getenv('vpc_subnet_ids').split(','),
'SecurityGroupIds': getenv('vpc_security_group_ids').split(',')
However, the CLI version is not specifying a VPC.
Therefore, the two requests are not identical. That's why one works and the other does not work.
From Configuring a Lambda Function to Access Resources in an Amazon VPC - AWS Lambda:
To connect to a VPC, your function's execution role must have the following permissions.
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface
These permissions are included in the AWSLambdaVPCAccessExecutionRole managed policy.
The lambda has a role that allows ec2:CreateNetworkInterface and not the account executing script.
The current role assigned to lambda function allows for the lambda to create VpcConfig.
Check that the account running the script to provision the lambda is allowed the ec2:CreateNetworkInterface action.
Related
I am following the aws tutorial on how to create thumbnails using lambda. I run this command to create a function:
aws lambda create-function --function-name CreateThumbnailDev --zip-file fileb://lambda-dev.zip --handler index.handler --runtime nodejs12.x --timeout 10 --memory-size 1024 --role arn:aws:iam::XXXXXXXXXXXXX:role/<MY_ROLE>
MY_ROLE is a role a created for executing lambda. The above command returns this error:
An error occurred (AccessDeniedException) when calling the CreateFunction operation: User: arn:aws:iam::XXXXXXXXXXXXX:user/MY_USER is not authorized to perform: lambda:CreateFunction .....
the problem is that XXXXXXXXXXXXX is my account id (as the tutorial says to use if) but user/MY_USER is the user I configured my AWS CLI locally, it exists but it doesn't have the necessary policy attached.
How should I configure the client to execute this command?
Here we have two different principals:
Role associated to the lambda function: permissions that the lambda function itself will have
Policies associated to your IAM user: what you can do
Looks like your IAM user named MY_USER, the one you're using for creating the lambda, is missing permissions for that. You'll need to grant them, either attaching a policy directly to the user or adding the user to a group that has the policies
You can check the official documentation for that
This stack was working at one point... I'm not sure what's going on. This permission is no longer doing what it did before, or has become invalid.
I have a Lambda function that rotates a Secret, so naturally it must be triggered by Secrets Manager. So I built up the Permission as follows
import * as aws from '#pulumi/aws'
export const accessTokenSecret = new aws.secretsmanager.Secret('accessTokenSecret', {});
export const smPermission = new aws.lambda.Permission(`${lambdaName}SecretsManagerPermission`, {
action: 'lambda:InvokeFunction',
function: rotateKnacklyAccessTokenLambda.name,
principal: 'secretsmanager.amazonaws.com',
sourceArn: accessTokenSecret.arn,
})
And the Policy,
{
Action: [
'secretsmanager:GetResourcePolicy',
'secretsmanager:GetSecretValue',
'secretsmanager:DescribeSecret',
'secretsmanager:ListSecrets',
'secretsmanager:RotateSecret',
],
Resource: 'arn:aws:secretsmanager:*:*:*',
Effect: 'Allow',
},
Running pulumi up -y yields
aws:secretsmanager:SecretRotation (knacklyAccessTokenRotation):
error: 1 error occurred:
* error enabling Secrets Manager Secret "" rotation: AccessDeniedException: Secrets Manager cannot invoke the specified Lambda function. Ensure that the function policy grants access to the principal secretsmanager.amazonaws.com.
This error confuses me, because the Policy created for the Lambda will not accept the Principal param (which makes sense, the same behaviour happens in the AWS Console), so I'm sure they mean Permission instead of Policy.
Based on the log I can tell that the Permission is being created way after the Lambda/Secrets Manager is, I'm not sure if this is a Pulumi issue similar to how it destroys stacks in the incorrect order (Roles and Policies for example).
I can see the Permission in the AWS Lambda configuration section, so maybe it's ok?
As explained in the Docs , I set up Lambda#edge for cloudfront trigger of Viewer Response.
The lambda function code :
'use strict';
exports.handler = (event, context, callback) => {
console.log('----EXECUTED------');
const response = event.Records[0].cf.response;
console.log(event.Records[0].cf_response);
callback(null, response);
};
I have set up trigger appropriately for the Viewer Response event.
Now when I make a request through cloudfront, it must be logged in cloudwatch, but it doesn't.
If I do a simple Test Lambda Function (using Button), it is logged properly.
What might be the issue here ?
When you deploy Lambda#Edge function, It is deployed to all edge cache regions across the world with their version Replica of the Lambda Edge function. Regional edge caches are a subset of the main AWS regions and edge locations.
When a user requests to the nearest pop/edge, the lambda associated with the edge cache region will get called. All logs of Lambda associated with those regions will in their edge cache region CloudWatch logs.
For example:
If a user is hitting us-east-1 region then its associated logs will be in us-east-1.
To know exactly where (on which region) your function is logging, you can run this AWS CLI script:
FUNCTION_NAME=function_name_without_qualifiers
for region in $(aws --output text ec2 describe-regions | cut -f 3)
do
for loggroup in $(aws --output text logs describe-log-groups --log-group-name "/aws/lambda/us-east-1.$FUNCTION_NAME" --region $region --query 'logGroups[].logGroupName')
do
echo $region $loggroup
done
done
on which you have to replace "function_name_without_qualifiers" with the name of your lambda#edge. Link
Hope it helps.
For those who have also searched for logs and couldn't find them with the script provided by #Kannaiyan.
TL;DR
Use this IAM Role for your Lambda function
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:*:*:log-group:*:*"
]
}
]
}
====
Make sure you have correct IAM role. If you created a Lambda first and then deployed it to Lambda#Edge, automatically generated IAM Role will only have permissions enough to log data in a single region into the log group named after the Lambda function, whilst using Lambda#Edge means it'll try to log data in different regions into the "/aws/lambda/." log group. Therefore it is necessary to change the IAM Role to allow creation of log group and write access there in different regions. In the TL;DR section, I provided the sample IAM Role, but make sure to narrow down the access to the specific list of log groups in production
According to AWS Documentation for Lambda#Edge Functions:
When you check for the log files, be aware that log files are stored in the Region closest to the location where the function is executed. So if you visit a website from, for example, London, you must change the Region to view the CloudWatch Logs for the London Region.
The Lambda#Edge logs and what region a request was executed in are available in the AWS CloudFront console, although the path is convoluted and AWS did a really lousy job of documenting the steps.
Here are the steps that work as of this posting:
Navigate to the AWS CloudFront console.
Click the "Monitoring" link under the "Telemetry" section (not "Logs", that takes to you CloudFront logs).
Click on the "Lambda#Edge" tab.
Choose your Lambda function and then click the "View metrics" button.
You can then use the "Invocations" graph to determine in which region a specific invocation of the Lambda function was executed. Once you have the region, you can, at long last, use the "View function logs" drop-down menu to view the Lambda function's logs for a specific region.
I figured this out by digging around in the console for a long time. The "documentation" for this logging is here, but doesn't seem to explain how to actually find Lambda#Edge logs for a specific region.
If anyone happens to find proper documentation about this, please update the post.
Following on from #yevhenii-hordashnyk's answer, if you're using "Serverless" framework, by default it creates a IAM user with logging permissions for the execution region only, and it is locked to the application name (which does not work for Edge functions because they are prefixed by the region of the installed function, thus requiring different permissions).
You have to specify a custom role, and apply that role to your function as per https://www.serverless.com/framework/docs/providers/aws/guide/iam
Note the following snippet uses * instead of - Ref: 'AWS::Region', as well as additional edgelambda.amazonaws.com service in Trust Relationships.
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- edgelambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: myPolicyName
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow # note that these rights are given in the default policy and are required if you want logs out of your lambda(s)
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- 'Fn::Join':
- ':'
-
- 'arn:aws:logs'
- '*'
- Ref: 'AWS::AccountId'
- 'log-group:/aws/lambda/*:*:*'
By default it does add the `AWSLambdaVPCAccessExecutionRole` policy to the lambda role, but I do not know why it does not create the Log Stream. Maybe I've missed something, but after doing the above, it works now.
I want to grant vpc access for my lambda function. I use the following aws cli command.
aws lambda update-function-configuration \
--function-name SampleFunction \
--vpc-config SubnetIds=subnet-xxxx,SecurityGroupIds=sg-xxxx
But I receive the following error:
An error occurred (AccessDeniedException) when calling the
UpdateFunctionConfiguration operation: Your access has been denied by
EC2, please make sure your request credentials have permission to
DescribeSecurityGroups for sg-xxxx. EC2 Error Code:
UnauthorizedOperation. EC2 Error Message: You are not authorized to
perform this operation.
I have granted the following permission to both my lambda role and the user who execute the aws command.
- "ec2:CreateNetworkInterface"
- "ec2:DescribeNetworkInterfaces"
- "ec2:DeleteNetworkInterface"
- "ec2:DescribeSecurityGroups"
I further tried to grant full access to both the lambda role and the user. But still received the same error
Can anyone suggest what else I can try?
The trick is to add the pipeline / worker role / user which is deploying the lambda function) have access to network related policies. The lambda function should itself suffice with managed policy - AWSLambdaVPCAccessExecutionRole
arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Action:
ec2:DescribeSecurityGroups
ec2:DescribeSubnets
ec2:DescribeVpcs
Effect: Allow
Resource: '*'
Your users IAM policy needs further permissions.
For example ec2:CreateSecurityGroup & etc. Have a look at this documentation to add requred permissions.
I experienced the same issue. Despite the IAM policy for the user having the required permissions, I could not use the aws cli to crate a lambda function with a VPC config (aws lambda create-function) or modify an existing function to add a VPC config (aws lambda update-function-configuration).
The only way I could get this to work was to create the lambda function without a VPC config. I then modified the function to add the VPC config information (vpc, subnet and security groups) via the AWS console (in Lambda > Fucntions > My Function > Network). I was only able to use the console to do this, introducing a manual step in an otherwise fully automated process.
To answer some of the questions above about which user needs the ec2:DescribeSecurityGroups and related permissions. It is the user running the cli command or logged in to the console. The function does not need a policy providing these permissions. The only special permissions needed for a function with a VPC config are:
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface
These allow the function to create ENIs within your VPC using the subnet and security group you provide as described here.
Both the Lambda funtion's role and the user role (either cloudformation or cmline user) must have:
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
- ec2:DescribeSecurityGroups
- ec2:DescribeSubnets
or ec2:* if ok for your use case'security
I had the same issue deploying a lambda with a VPC config using SAM/cloudformation and resolved it by adding this above.
on github issue some people say it is because of cloudformation order creation it is not (or maybe not anymore because I tested adding 20 dummy resource and still the same issue only resolved by adding the permissions above)
cheers,
I am trying to create a lifecycle event for an autoscaling group in AWS Amazon cloudformation, however I keep getting a really ambiguous error back when deploying my stack:
Unable to publish test message to notification target
arn:aws:sns:us-east-1:000000000000:example-topic using IAM role arn:aws:iam::000000000000:role/SNSExample. Please check your target and role configuration and try to put lifecycle hook again.
I have tested the SNS topic and it can send emails fine and my code appears to be in line with what Amazon suggest:
"ASGLifecycleEvent": {
"Type": "AWS::AutoScaling::LifecycleHook",
"Properties": {
"AutoScalingGroupName": "ASG-179ZOVNY8SEFT",
"LifecycleTransition": "autoscaling:EC2_INSTANCE_LAUNCHING",
"NotificationTargetARN": "arn:aws:sns:us-east-1:000000000000:example-topic",
"RoleARN": "arn:aws:iam::000000000000:role/SNSExample"
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "83129091-8efc-477d-86ef-9a08de4d6fac"
}
}
}
And I have granted full access to everything in that IAM role, however I still get this error message. Does anyone have any other ideas what could really be causing this error?
Your SNSExample role needs to delegate permissions from the AutoScalingNotificationAccessRole managed policy to the autoscaling.amazonaws.com service via an associated Trust Policy (the AssumeRolePolicyDocument Property in the CloudFormation Resource):
SNSExample:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AutoScalingNotificationAccessRole
(You can also delegate access to sns:Publish action instead of using the managed policy, but I recommend the managed policy because it will stay up to date if additional permissions are required for this service in the future.)
See the Receive Notification Using Amazon SNS part of the Auto Scaling Lifecycle Hooks section of the Auto Scaling User Guide for more information.