Lambda trigger doesn't replicate to SQS source across accounts - amazon-web-services

I'm trying to add an SQS as a source/trigger to a lambda. I can do this just fine if both components reside within the same account. When I add the trigger to the lambda, the lambda trigger configuration replicates over to the SQS queue to pair the two.
When I try this same thing on my lambda when the SQS is remote in a different account the Lambda trigger is established, but when viewing the remote SQS it doesn't show a trigger configured. This seems to result in the trigger not working on the lambda when a message is added to the queue. The SQS policy on the remote queue is also giving permissions explicitly to the other account as well.
Any thoughts?

Scenario:
Amazon SQS queue in Account-A
AWS Lambda function in Account-B
Goal: SQS triggers Lambda function
Since this involves cross-account access, you will need to grant permissions for the IAM Role used by the Lambda function to access the SQS queue. (Lambda pulls from the queue, rather than SQS pushing to Lambda.)
The steps are:
In the SQS queue, edit the Access Policy to include permission for the IAM Role used by the Lambda function:
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNT-1:root"
},
"Action": "SQS:*",
"Resource": "arn:aws:sqs:ap-southeast-2:ACCOUNT-1:queue-name"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNT-2:role/lambda-role-name"
},
"Action": [
"SQS:ChangeMessageVisibility",
"SQS:DeleteMessage",
"SQS:ReceiveMessage",
"SQS:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:ap-southeast-2:ACCOUNT-1:queue-name"
}
]
}
The first part of this policy is automatically created by SQS and allows the owning account to use the queue. The second part allows the IAM Role from Account-2 to access the queue in Account-1. The policy was created automatically by SQS when I created the queue and provided the ARN of the IAM Role. However, I had to add SQS:GetQueueAttributes because the Lambda function calls it too.
In the AWS Lambda function in Account-B, click + Trigger, select SQS and enter the ARN of the SQS queue from Account-A
I tried all this and was successfully able to put a message in SQS in Account-B, and then saw Lambda process it in Account-B.

Related

AWS CloudWatch - Log group does not exist

I'm getting this error message when trying to see the log file in AWS CloudWatch for my AWS Lambda function.
An error occurred while describing log streams.
The specified log group does not exist.
Log group does not exist
The specific log group: /aws/lambda/xxxxx does not exist in this account or region.
By the way, I'm using the Singapore region.
Make sure that your Lambda function's execution role has sufficient permissions to write logs to CloudWatch, and that the log group resource in the IAM policy includes your function's name.
In the IAM console, review and edit the IAM policy for the execution role to make sure that:
The write actions CreateLogGroup and CreateLogStream are allowed. You should attach these policies in the IAM roles of the Lambda function
Note: If you don't need custom permissions for your function, you can attach the managed policy AWSLambdaBasicExecutionRole, which allows Lambda to write logs to CloudWatch.
The AWS Region specified in the Amazon Resource Name (ARN) is the
same as your Lambda function's Region.
The log-group resource includes your Lambda function name. For
example, if your function is named myLambdaFunction, the log-group is
/aws/lambda/myLambdaFunction.
Here is an example of the permissions in the JSON format
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:region:accountId:*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
" arn:aws:logs:region:accountId:log-group:/aws/lambda/functionName:*"
]
}
]
}

SQS Encryption using CMK

I am trying to read message from an encrypted SQS. Objects are landed on an S3 Bucket -> Trigger S3 Event -> Message sent to SQS -> SQS triggers Lambda to Process.
I have got this working using an AWS managed CMK. However, I can't get this working using AWS owned CMK e.g. alias/aws/sqs.
The message just goes into messages in flight and does not invoke the Lambda functions.
As per the the AWS documentation here https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-server-side-encryption.html#sqs-encryption-what-does-sse-encrypt If you don't specify a custom CMK, Amazon SQS uses the AWS managed CMK for Amazon SQS. But we can't attach any policies against AWS owned CMK e.g.
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "<<service>>.amazonaws.com"
},
"Action": [
"kms:GenerateDataKey*",
"kms:Decrypt"
],
"Resource": "*"
}]
}
My question is: is it possible to use the AWS owned CMK on an SQS and have Lambda functions be able to read from that queue?
There is a section in the above URL called Enable Compatibility between AWS Services Such as Amazon CloudWatch Events, Amazon S3, and Amazon SNS and Encrypted Queues.
It mentions attaching a policy to the CMK. However, there is an option to use the alias/aws/sqs. I was wondering if I was missing something here.
I spoke with AWS and KMS AWS Managed Key would not work in this scenario. We can't change the key policy for KMS AWS Managed Keys, so wouldn't be possible for scenario: S3 Bucket -> Trigger S3 Event -> Message sent to SQS -> SQS triggers Lambda to Process
I used KMS AWS Customer Managed Key and it worked fine.

AccessDenied when calling the ListQueues operation

This is quite strange when call list queues command using AWS-CLI with the attached policy not working unless I set value for Resource to arn:aws:sqs:*:*:*.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "sqs:*",
"Resource": "arn:aws:sqs:*:45*****65:local-*"
}
]
}
I expect the below command should return all the Queues that start with local-* but I got AccessDenied.
aws sqs list-queues --queue-name-prefix local-*
But for the same Queue, I am able to get its attributes.
aws sqs get-queue-attributes --queue-url https://us-west-2.queue.amazonaws.com/0****5/local-myqueue --attribute-names All
Do I need to changes something in policy or is the behavior of list queue is different?
Thanks in Advance.
According to Actions, Resources, and Condition Keys for Amazon SQS - AWS Identity and Access Management, the SQS commands do not accept conditions. Therefore, the ListQueues() command either works completely (showing all matching queues) or does not return anything (due to Access Denied). The only way to limit which queues are returned is to use the queue-name-prefix.
The Resource attribute in the policy will determine upon which queue various commands can run, which is why you are able to get attributes for a queue matching local-*. If you tried to get the details of a queue named public-xxx, it would be denied. (But ListQueues operates on the service, not a particular queue.)

Lambda service throws error execution role does not have permissions to call receiveMessage on SQS

I have a SQS queue and I want to trigger a lambda function when a message arrives in the queue. I have written the lambda function and that works successfully when I click the "Test" button. When I go to SQS and try to configure it as a lambda trigger I see the error message below.
I have created the SQS queue and lambda function using the same user and role and the lambda function has execute permissions against the same role.
I also have also added SQS receiveMessage permission but it doesn't seem to make a difference unless I'm doing something wrong when I set it.
What could be causing the problem?
Thanks for any help
Hi as far as i can understand your lambda needs the following permission on it aws docs
Hope its not in a VPC.
Or may be give it a god mode on sqs:* just for testing it.
If that works maybe later on you can then go for specific methods only. Attached a policy for a lambda role you might have to change account_number to your account no if you need to invoke another lambda form this lambda
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:eu-west-2:account_number:function:*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"logs:PutLogEvents",
"logs:CreateLogStream",
"logs:CreateLogGroup"
],
"Resource": "*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"sqs:*"
],
"Resource": "*"
}
]
}
Although solution for this may have been achieved by now.. but since this thread was suggested to me at the top.. i will post the answer for other users:
I faced same issue even after giving SQS full access to user. The problem is with the lambda execution role. When lambda is created, it needs to be assigned a lambda execution role. Most users assign the auto-generated execution role while creating lambda. That execution role does not have permissions for SQS.
So open lambda >> Click Permissions tab >> edit execution role at the top >> assign SQS permissions >> boom.
[edit]This is now under Configuration >> Permissions
You need following permissions attached to the role, your lambda assumes
sqs:ReceiveMessage
sqs:DeleteMessage
sqs:GetQueueAttributes
In case you are using Terraform:
data "aws_iam_policy_document" "YOUR_DOCUMENT" {
statement {
sid = "some_id"
actions = [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
]
resources = [
aws_sqs_queue.YOUR_QUEUE.arn
]
}
}
resource "aws_iam_policy" "YOUR_POLICY" {
name = "your_policy"
policy = data.aws_iam_policy_document.YOUR_DOCUMENT.json
}
resource "aws_iam_role_policy_attachment" "POLICY_ATTACHMENT" {
role = aws_iam_role.YOUR_LAMBDA_ROLE.name
policy_arn = aws_iam_policy.YOUR_POLICY.arn
}
resource "aws_lambda_function" "YOUR_LAMBDA" {
....
role = aws_iam_role.YOUR_LAMBDA_ROLE.arn
....
}
I experienced a similar issue when trying to add an SQS trigger to my Lambda function.
An error occurred when creating the trigger: The provided execution role does not have permissions to call ReceiveMessage on SQS
The way I solved it was to simply add permissions to call ReceiveMessage on SQS in the execution role of the Lambda function.
To do this simply:
Go to IAM in the AWS Console
Click on roles
Select your Lambda function execution role or create one if you don't already have
Add the AWS managed LambdaSQSQueueExecutionRole policy to the role. The policy contains all the necessary permissions to call the necessary actions on SQS from Lambda. The ARN of the policy is arn:aws:iam::aws:policy/service-role/AWSLambdaSQSQueueExecutionRole.
Save the role, and the try again to add the trigger. This time it will work fone.

AWS security group rules deployment (lambda->SQS)

On AWS we've implemented functionality that AWS lambda pushes message to AWS queue;
However during this implementation I had to manuall grant permissions to AWS lambda to add message to particular queue. And this apporach with manual clicks not so good for prod deployment.
Any suggestions how to automate process of adding permissions between AWS services (mainly lambda and SQS) and cretate "good" deployment package for prod env ?
Each Lambda function has an attached role, which you can specify permissions for in the IAM dashboard. If you give the Lambda functions' role the permission to push to an SQS queue, you're good to go. For example, attach this JSON as a custom role (see http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSExamples.html):
{
"Version": "2012-10-17",
"Id": "Queue1_Policy_UUID",
"Statement":
{
"Sid":"Queue1_SendMessage",
"Effect": "Allow",
"Principal": {
"AWS": "111122223333"
},
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:us-east-1:444455556666:queue1"
}
}
You can use asterisks to give permission to multiple queues, like:
"Resource": "arn:aws:sqs:us-east-1:444455556666:production-*"
To give sendMessage permission to all queues that start with production-.