In Azure, if I were to configure a function app to post to a storage account queue, I would used managed identity for the authentication. This would remove any need to store credentials for the queue.
Does AWS have something equivalent to managed identity that could be used for a Lambda posting to SQS?
With Serverless Framework
provider:
iamRoleStatements:
- Effect: Allow
Action:
- sqs:SendMessage
- sqs:DeleteMessage
Resource:
- !GetAtt MyQueue.Arn
resources:
Resources:
MyQueue:
Type: "AWS::SQS::Queue"
Properties:
QueueName: ${self:custom.sqs.name}
Related
I've defined an SNS topic, an SQS queue, and an SNS subscription resource in a Cloudformation stack. All three are in the same stack, same region, and same AWS account.
Resources:
SqsQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: 'some-queue'
SnsTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: 'some-topic'
SnsSubscription:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !GetAtt [SqsQueue, Arn]
Protocol: sqs
TopicArn: !Ref SnsTopic
When I run the stack, all three resources are created successfully, but when I publish a message from SNS, it's never received by the SQS queue.
I've been following this link (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/) and to my knowledge I've done everything I've needed to. What else am I missing?
Thanks!
Additional info
If I delete the subscription that Cloudformation created via the console and then create a new one via the console, messages are published fine. So it must be something incorrect about the subscription.
I used the AWS CLI to compare the properties of the subscription created by the Cloudformation template to the one created by the console. They are the exact same.
You need to add a policy to allow the SNS topic to publish to your queue. Something like this:
SnsToQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref SqsQueue
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: allow-sns-messages
Effect: Allow
Principal: '*'
Resource: !GetAtt SqsQueue.Arn
Action: SQS:SendMessage,
Condition:
ArnEquals:
aws:SourceArn: !Ref SnsTopic
Using the serverless.yml configuration file, I'm trying to extend the permissions of the default lambda IAM role to allow my lambda to send SQS messages to a SQS that was previously created, but I can't manage to make it work.
In my serverless.yml file, I have the following provider settings:
provider:
name: aws
runtime: python3.8
region: eu-central-1
iam:
role:
statements:
- Effect: Allow
Action:
- sqs:SendMessage
- sqs:GetQueueUrl
- sqs:GetQueueAttributes
Resource:
Fn::Join:
- ':'
- - 'arn:aws:sqs'
- Ref: AWS::Region
- Ref: AWS::AccountId
- test-output-sqs
According to the documentation, this additional statement should be merged into the generated policy.
However, it is not. I am missing something ?
I am trying to create an AWS SQS queue using the serverless framwework,
But I am getting the following error on deploying the severless.yaml
The provided execution role does not have permissions to call SendMessage on SQS
The issue is the IAM role is created by serverless framework and I have no control of what permissions the framework adds to the role,
Ideally, if the function trigger is an SQS, or needs a DLQ configured,
I was hoping the framework would add Send and Receive message permissions to the role, but I guess it did not
Serveless.yaml -
service: dlq
provider:
name: aws
runtime: nodejs12.x
profile: csStage
region: ap-southeast-1
plugins:
- serverless-plugin-lambda-dead-letter
functions:
dlqFunction:
handler: handler.hello
deadLetter:
sqs: dl-queue
You have complete control over the permissions added to that role. You can add an iamRoleStatements section to your serverless.yml file under provider that describes the permissions you wish to apply to the role applied to functions. It would look something like:
provider:
iamRoleStatements:
- Effect: Allow
Action:
- "sqs:SendMessage"
Resource:
- arn:aws:sqs:region:accountid:queueid
You can find out more in the official documentation here: https://www.serverless.com/framework/docs/providers/aws/guide/iam/#iam/
You can use iamRoleStatements to give that permission to your Lambda function. The following template worked for me:
service: dlq
provider:
name: aws
runtime: nodejs12.x
profile: csStage
region: ap-southeast-1
iamRoleStatements:
- Effect: Allow
Action:
- sqs:SendMessage
Resource: !GetAtt DLQ.Arn
plugins:
- serverless-plugin-lambda-dead-letter
functions:
dlqFunction:
handler: handler.hello
deadLetter:
targetArn:
GetResourceArn: DLQ
resources:
Resources:
DLQ:
Type: AWS::SQS::Queue
Properties:
QueueName: dl-queue
Using the serverless framework, I am trying to build a Lambda function that periodically rotates a secret stored in AWS Secrets Manager.
I am having trouble configuring the roles needed for the Secret Manager to execute the Lambda. In my serverless.yml I have defined the following resources:
resources:
Resources:
RotateKeysRole:
Type: AWS::IAM::Role
Properties:
RoleName: rotate-keys-role
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- secretsmanager.amazonaws.com
Action: sts:AssumeRole
and attach this role to the rotation Lambda like this:
functions:
rotateKeys:
handler: lambdas.rotate_keys.handler
role: RotateKeysRole
Yet, when I try to set up Secrets Manager to use this Lambda for rotating secrets I will get the following error message:
Secrets Manager cannot invoke the specified Lambda function. Ensure
that the function policy grants access to the principal
secretsmanager.amazonaws.com
which puzzles me as this principal is specified. Inspecting the role in the IAM console did not reveal anything that seemed wrong to me.
How do I correctly configure the role setup in this scenario?
The procedure of setting up permissions for a lambda function which rotates AWS Secrets Manager secrets is explained in the docs. [1]
To put it in a nutshell, you need two steps:
Add a trust policy to the lambda function. This can be achieved using the CloudFormation resource AWS::Lambda::Permission in the serverless.yml file. However, it is a little bit tricky to set this up, because you need to depend on the function being created. That is why the DependsOn is necessary and its value must be structured as follows: <function-name-with-first-letter-uppercase>LambdaFunction.
Add statements for the lambda function to call the AWS Secrets Manager API to update the secret. In the following example, I added these statements (for the Single user rotation case - see docs [1]) to the customer managed policy called rotateKeysPolicy.
Note: The function name is referenced in the DependsOn attribute. It is also referenced in the condition StringEquals and the attribute FunctionName as: arn:aws:lambda:${self:custom.region}:${self:custom.accountId}:function:${self:service}-${self:provider.stage}-rotateKeys. Keep in mind to change them if you change your function name.
Here is how the serverless.yml file should look like:
service:
name: <your-service-name>
provider:
name: aws
region: '<your-region>'
custom:
region: ${self:provider.region}
accountId: <your-account-id>
resources:
Resources:
FunctionRole:
Type: AWS::IAM::Role
Properties:
RoleName: basic-function-role
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: rotateKeysPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- secretsmanager:DescribeSecret
- secretsmanager:GetSecretValue
- secretsmanager:PutSecretValue
- secretsmanager:UpdateSecretVersionStage
Resource: '*'
Condition:
StringEquals:
'secretsmanager:resource/AllowRotationLambdaArn': "arn:aws:lambda:${self:custom.region}:${self:custom.accountId}:function:${self:service}-${self:provider.stage}-rotateKeys"
- Effect: Allow
Action:
- secretsmanager:GetRandomPassword
Resource: '*'
- Effect: Allow
Action:
- ec2:CreateNetworkInterface
- ec2:DeleteNetworkInterface
- ec2:DescribeNetworkInterfaces
Resource: '*'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
LambdaInvokePermission:
Type: AWS::Lambda::Permission
DependsOn: RotateKeysLambdaFunction
Properties:
FunctionName: "arn:aws:lambda:${self:custom.region}:${self:custom.accountId}:function:${self:service}-${self:provider.stage}-rotateKeys"
Action: lambda:InvokeFunction
Principal: 'secretsmanager.amazonaws.com'
functions:
rotateKeys:
handler: lambdas.rotate_keys.handler
role: FunctionRole
You have to replace <your-service-name>, <your-region>, <your-account-id> and upload your rotation code using e.g. the package -> include attributes.
Note: There are templates for the lambda function which update the secrets. [2][3]
Please also keep in mind to configure your VPC correctly for the lambda function being able to access the AWS Secrets Manager service over the network. [4]
References
[1] https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-required-permissions.html
[2] https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-create-generic-template.html
[3] https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas
[4] https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotation-network-rqmts.html
I had the same issue today. I ran this and it worked for me:
aws lambda add-permission \
--function-name ARN_of_lambda_function \
--principal secretsmanager.amazonaws.com \
--action lambda:InvokeFunction \
--statement-id SecretsManagerAccess
https://docs.aws.amazon.com/secretsmanager/latest/userguide/troubleshoot_rotation.html
Your policy is incorrect.
The service is secretsmanager but the action you defined is sts:AssumeRole which is from AWS Security Token Service.
A full access policy would be:
Effect: "Allow"
Action: "secretsmanager:*"
Resource: "*"
But you should limit the actions and the Resource the lambda can use.
For this you can use the policy builder which can be found in IAM->Policies.
After creating a policy in the editor you can click on the JSON Tab and see the format. Then you need to adapt it to your serverless yaml format.
I hope I can help you!
Dominik
Does anyone know of a definitive guide for converting cloudformation to yaml in Serverless v1.0?
I can make quite a few things work but I'm completely stuck on how to setup Firehose and allow a lambda to write to it.
I think the resources section would look something like this:
resources:
Resources:
FirehoseBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: my-firehose-bucket
FirehoseDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: "FirehoseDeliveryStream"
S3DestinationConfiguration:
BucketARN:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- Ref: FirehoseBucket
BufferingHints:
IntervalInSeconds: 60
SizeInMBs: 5
CompressionFormat: GZIP
Prefix: ${prefix}
RoleARN: "arn:aws:iam::${account-number}:role/${project}-${env}-IamRoleLambda"
But I have no idea how to convert the Cloudformation for the IAM section, which is described here: http://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html.
Any pointers, examples or guides on CF->Yaml conversion much appreciated!
The IAM statements for the Lambda Execution Role (ie the role that the Lambda function assumes when executing) is not specified in the resources.
So if you need permissions to do something from inside your Lambda function you need to give the assumed role permission.
This is specified in the provider section. So in your case (I just copied something from your link, you will have to change it to what you need) it will be something like this (assuming nodejs runtime):
provider:
name: aws
runtime: nodejs4.3
iamRoleStatements:
- Effect: Allow
Action:
- firehose:DeleteDeliveryStream
- firehose:PutRecord
- firehose:PutRecordBatch
- firehose:UpdateDestination
Resource:
- arn:aws:firehose:region:account-id:deliverystream/delivery-stream-name
- Effect: Allow
Action:
- logs:PutLogEvents
Resource:
- arn:aws:logs:region:account-id:log-group:log-group-name:log-stream:log-stream-name
Bonus: For general resources specified in cloud formation json format use an online converter from json to yaml. Much easier to get the started that way.