Disable/Enable Lambda SNS Trigger Programmatically - amazon-web-services

I need to programmatically disable a lambda's SNS trigger, however, I seem to be unable to do so. I want this to show "Disabled" in the AWS Lambda console for the function:
Here's the code I've tried:
function updateEndpoints(endpoints, enable) {
const promises = [];
endpoints.forEach((endpoint) => {
console.log(`${enable ? 'Enabling' : 'Disabling'} Endpoint: ${endpoint}`);
promises.push(
SNS.setEndpointAttributes({
EndpointArn: endpoint,
Attributes: {
Enabled: enable ? 'True' : 'False',
},
}).promise()
.catch((e) => {
console.error(`Error ${enable ? 'Enabling' : 'Disabling'} Endpoint: ${endpoint}`);
console.error(e);
}));
});
return Promise.all(promises);
}
The endpoint ARN is passed in correctly with a string like (with correct values in place of the <> below):
-
arn:aws:lambda:<region>:<accountId>:function:<functionName>
-
This produces an error from AWS for each endpoint I try to enable or disable:
-
InvalidParameter: Invalid parameter: EndpointArn Reason: Vendor lambda is not of SNS
-
Is it not possible to disable the trigger/endpoint for a lambda via SNS? How would one go about doing this? I would prefer not to have to unsubscribe/subscribe as this would take the subscription objects out of CloudFormation's scope (correct?). I looked at updateEventSourceMappings, however, per the documentation, that only works with DynamoDB streams, Kinesis Streams, and SQS -- not SNS.

I found the (100%) correct way to do this. While the answer from #John Rotenstein could be used, it's not quite right, but should still work.
I found when you click the toggle, the lambda's policy is actually updated:
Enabled:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "my-lambda-1552674933742",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-west-2:1234567890:function:my-lambda",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:sns:us-west-2:1234567890:my-lambda"
}
}
}
]
}
Disabled:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "my-lambda-1552674933742",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "lambda:DisableInvokeFunction",
"Resource": "arn:aws:lambda:us-west-2:1234567890:function:my-lambda",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:sns:us-west-2:1234567890:my-lambda"
}
}
}
]
}
Notice Action is lambda:InvokeFunction vs. lambda:DisableInvokeFunction.
My process to do this is as follows:
- Lambda.listFunctions
- for each function, Lambda.removePermission
- for each function, Lambda.addPermission
Notes:
the Lambda api has a default safety throttle of 100 concurrent executions per account per region.
You can only update resource-based policies for Lambda resources within the scope of the AddPermission and AddLayerVersionPermission API actions. You can't author policies for your Lambda resources in JSON, or use conditions that don't map to parameters for those actions. See docs here
Also, you can use Lambda.getPolicy to see the policy of the lambda to ensure it is updated.

It appears that there is no capability to "disable" a Lambda subscription to an SNS topic.
I base my reasoning on the follow steps I took:
Created an AWS Lambda function
Created an Amazon SNS topic
Subscribed the Lambda function to the SNS topic (done via the SNS console)
Confirmed in the Lambda console that the function subscription to SNS is "enabled"
Ran aws sns list-subscriptions-by-topic --topic-arn arn:aws:sns:ap-southeast-2:123456789012:my-topic
Saw that the Lambda function was subscribed
The response was:
{
"Subscriptions": [
{
"SubscriptionArn": "arn:aws:sns:ap-southeast-2:123456789012:stack:...",
"Owner": "123456789012",
"Protocol": "lambda",
"Endpoint": "arn:aws:lambda:ap-southeast-2:743112987576:function:my-function",
"TopicArn": "arn:aws:sns:ap-southeast-2:123456789012:stack"
}
]
}
I then disabled the trigger in the Lambda console and saved the Lambda function. When I re-ran the above command, the results were empty:
{
"Subscriptions": []
}
When I enabled it again, the subscription returned.
So, my assumption is that, since the "disable/enable" button actually adds and removes a subscription, there does not appear to be any capability to 'disable' a subscription.

Related

Error while doing AWS Lambda Cross Account integration with AWS SNS

I want to send notification from SNS (Account A) to Lambda (Account B). Followed this tutorial but still getting below error:
https://docs.aws.amazon.com/lambda/latest/dg/with-sns-example.html
Error code: AccessDeniedException - Error message: User: arn:aws:sts::AccountA:assumed-role/AdministratorAccessRole/A12345 is not authorized to perform: lambda:AddPermission on resource: arn:aws:lambda:us-east-1:AccountB:function:TestLambda
Below what I did:
1. In Account A, added below policy in Access Policy of SNS:
{
"Version": "2008-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "_abc_",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:root"
},
"Action": [
"SNS:Subscribe",
"SNS:Receive"
],
"Resource": "arn:aws:sns:us-east-1:AccountA:TriggerLambdaB-SNS"
}
]
}
2. In Account B, added below policy in Resource-Based Policy of Lambda:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "_abc_",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:AccountB:function:TestLambda",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:sns:us-east-1:AccountA:TriggerLambdaB-SNS"
}
}
}
]
}
I am able to see the SNS Name under Trigger Lambda section of my Lambda in Account B. But when I am trying to Subscribe the Lambda under SNS, then getting this error. Please guide what am I missing here.
Is it because I am having different types of Role in these accounts like AdminAccessRole in Account A and FederatedRoleAccess in Account B?
You need to run the aws sns subscribe in Account-B (with the Lambda function), not Account-A (with the SNS function).
Otherwise, your setup seems correct.
When I tried running the subscribe command from Account-A, it said:
An error occurred (AuthorizationError) when calling the Subscribe operation: The account ACCOUNT-A is not the owner of the lambda function arn:aws:lambda:ap-southeast-2:ACCOUNT-B:function:foo
While this error is different to yours, your command appears to have been run from Account-A (with SNS) rather than Account-B (with Lambda).
Side-note: There appears to be a small error in the Tutorial: Using AWS Lambda with Amazon Simple Notification Service documentation, where the Resource-Based policy for Lambda (the second one in your Question) is showing a SourceArn that refers to Account-B-Lambda, whereas it should be Account-A-SNS. However, you appear to have gotten this correct in your policy above.

Allow all cloudwatch event rules to have access to lambda function

I have been encountering a hard limit on lambda function policy when trying to provision access for a cloudwatch event rule to trigger the lambda function on a scheduled basis.
An error occurred (PolicyLengthExceededException) when calling the AddPermission operation: The final policy size (20670) is bigger than the limit (20480).
It works for a new lambda function, but eventually its policy will bloat and will hit a hard limit on the number on cloudwatch event rule that can access it.
Some said to re-create the function (delete/create), but this won't be an option in a production environment where cloudwatch events are already configured in it, resulting to the existing ones to lose access to the lambda function.
Using the aws cli, i was able to extract the policy of my lambda function, it loooks like this:
"Statement": [{
"Sid": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:*",
"Resource": "arn:aws:lambda:xxxxx:xxxxxxxxxxx:function:xxxxxxxxxxxxx",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:events:xxxxxxx:xxxxxx:rule/xxxxxxxxx"
}
}
}]
So i was looking onto something like for the AWS:SourceArn
arn:aws:events:xxxxxxx:xxxxxx:rule/*
To avoid hitting a hard limit, but i cannot seem to do it. Even in the lambda function itself on the console, you won't be able to create such a rule that will allow all cloudwatch event of a specified account to have access to the lambda function using a wildcard '*'.
Suggestions are much welcome. Thank you guys
This was accepted without an error:
$ aws lambda add-permission --function-name function_name\
--action 'lambda:InvokeFunction' --principal events.amazonaws.com \
--statement-id '1' \
--source-arn arn:aws:events:ap-southeast-2:123456789012:rule/*
This will accept all CloudWatch Events rules.
You could instead name your rules such that the ones you want to allow can all have the same prefix in their name, eg:
--source-arn arn:aws:events:ap-southeast-2:123456789012:rule/Event-*
I used the above CLI command but got an error in the console, please find the attached screenshot of the errorenter image description here. Please find below function policy of lambda:
{ "Version": "2012-10-17", "Id": "default", "Statement": [
{
"Sid": "events-access",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:096280016729:function:leto_debug_log",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:events:us-east-1:096280016729:rule/*"
}
}
}
] }

Unable to trigger AWS Lambda by upload to AWS S3

I am trying to build a Kibana dashboard fed with twitter data collected via AWS Kinesis firehose where data passes into an S3 bucket which triggers a Lambda function which passes the data to AWS Elastic Search and then to Kibana. I am following this blog https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/
The data is loading into the S3 bucket correctly but it never arrives in Kibana, I believe this is because the Lambda function is not being triggered by events in S3 as I would have hoped (there are no invocations or logs). I think this is because I have not set permissions properly. The Lambda function can be invoked manually by the test event.
On the Lambda function page I chose an existing role which I called lambda_s3_exec_role which has the AWSLambdaExecute policy attached to it but I feel I'm missing something else more specific to S3. I have been unable to follow this line in the blog in the create lambda function section because I do not recognise those options:
"10. Choose lambda_s3_exec_role (if this value does not exist, choose Create new role S3 execution role)."
Can anyone help me create the appropriate role/policy for the Lambda function, or spot what the problem may be?
From view permissions on the Lambda function I currently have:
FUNCTION POLICY
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "****",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "****",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:s3:::****"
}
}
}
]
}
EXECUTION ROLE
{
"roleName": "lambda_s3_exec_role",
"policies": [
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::*"
}
]
},
"name": "AWSLambdaExecute",
"id": "****",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/AWSLambdaExecute"
}
]
}
The permissions you have listed look OK so I am going to try provide some steps that might help find the issue as it is difficult to understand specifically where your issue might be.
Does the execution role have the trust relationship with a trusted entity of lambda.amazonaws.com
Does your event prefix match the prefix in firehose. In the tutorial they are both twitter/raw-data/. If firehose is writing to a path that isn't the event prefix then the event won't be invoked.
Does the lambda trigger any errors when you manually invoke it
Does the lambda write to the logs when you manually invoke it
Test the lambda using dummy data (example data below)
CLI
aws lambda invoke \
--invocation-type RequestResponse \
--function-name helloworld \
--region region \
--log-type Tail \
--payload file://dummy_event.json \
--profile adminuser \
outputfile.txt
Example data
source
dummy_event.json
{
"Records":[
{
"eventVersion":"2.0",
"eventSource":"aws:s3",
"awsRegion":"us-west-2",
"eventTime":"1970-01-01T00:00:00.000Z",
"eventName":"ObjectCreated:Put",
"userIdentity":{
"principalId":"AIDAJDPLRKLG7UEXAMPLE"
},
"requestParameters":{
"sourceIPAddress":"127.0.0.1"
},
"responseElements":{
"x-amz-request-id":"C3D13FE58DE4C810",
"x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
},
"s3":{
"s3SchemaVersion":"1.0",
"configurationId":"testConfigRule",
"bucket":{
"name":"sourcebucket",
"ownerIdentity":{
"principalId":"A3NL1KOZZKExample"
},
"arn":"arn:aws:s3:::sourcebucket"
},
"object":{
"key":"HappyFace.jpg",
"size":1024,
"eTag":"d41d8cd98f00b204e9800998ecf8427e",
"versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
}
}
}
]
}
Struggled with this for a long time and eventually realized that your rule that triggers the lambda cannot have exactly the same name as the lambda itself or it won't work.

How do I confirm a subscription created using CloudFormation?

I have created a CF script that creates an EC2 instance that contains a web service. It also creates an SNS Topic and a Subscription that uses this web service as it's http endpoint.
The script successfully creates the stack; the Topic and the Subscription exist. However, the Subscription remains in the PendingConfirmation state.
What must I do to get my script to confirm this Subscription upon creation?
I had a similar issue and my problem ended up being a misconfigured CloudFomation template. An AWS::SQS::QueuePolicy is required to give your SNS topic permission to send messages to the queue.
"SQSQueuePolicy": {
"Properties": {
"PolicyDocument": {
"Id": "usecase1",
"Statement": [
{
"Action": "SQS:SendMessage",
"Condition": {
"ArnEquals": {
"aws:SourceArn": {
"Ref": "SnsTopic"
}
}
},
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Resource": {
"Fn::GetAtt": [
"SqsQueue",
"Arn"
]
},
"Sid": "1"
}
],
"Version": "2012-10-17"
},
"Queues": [
{
"Ref": "SqsQueue"
}
]
},
"Type": "AWS::SQS::QueuePolicy"
}
You need to Subscribe to endpoint for this to work.
Read the value for SubscribeURL and visit that URL. To confirm the subscription and start receiving notifications at the endpoint, you must visit the SubscribeURLURL (for example, by sending an HTTP GET request to the URL)
When you visit the URL, you will get back a response that looks like the following XML document. The document returns the subscription ARN for the endpoint within the ConfirmSubscriptionResult element.
<ConfirmSubscriptionResponse xmlns="http://sns.amazonaws.com/doc/2010-03-31/">
<ConfirmSubscriptionResult>
<SubscriptionArn>arn:aws:sns:us-west-2:123456789012:MyTopic:2bcfbf39-05c3-41de-beaa-fcfcc21c8f55</SubscriptionArn>
</ConfirmSubscriptionResult>
<ResponseMetadata>
<RequestId>075ecce8-8dac-11e1-bf80-f781d96e9307</RequestId>
</ResponseMetadata>
</ConfirmSubscriptionResponse>
As an alternative to visiting the SubscribeURL, you can confirm the subscription using the ConfirmSubscription action with the Token set to its corresponding value in the SubscriptionConfirmation message. If you want to allow only the topic owner and subscription owner to be able to unsubscribe the endpoint, you call the ConfirmSubscription action with an AWS signature.
You can Refer to this AWS Documentation
Hope this Helps!

SNS topic not publishing to SQS

I am trying to prototype a distributed application using SNS and SQS.I have this topic:
arn:aws:sns:us-east-1:574008783416:us-east-1-live-auction
and this queue:
arn:aws:sqs:us-east-1:574008783416:queue4
I created the queue using the JS Scratchpad. I added the subscription using the Console. I AddPermission to the queue using the scratchpad. The queue policy is now:
{
"Version":"2008-10-17",
"Id":"arn:aws:sqs:us-east-1:574008783416:queue4/SQSDefaultPolicy",
"Statement":[
{
"Sid":"RootPerms",
"Effect":"Allow",
"Principal":{
"AWS":"574008783416"
},
"Action":"SQS:*",
"Resource":"arn:aws:sqs:us-east-1:574008783416:queue4"
}
]
}
I have an email subscription on the same topic and the emails arrive fine but the messages never arrive on the queue. I've tried SendMessage directly to the queue - rather than via SNS - using Scratchpad and it works fine. Any ideas why it won't send to the queue?
This was posted a while back on the AWS forums: https://forums.aws.amazon.com/thread.jspa?messageID=202798
Then I gave the SNS topic the permission to send messages to the SQS queue. The trick here is to allow all principals. SNS doesn't send from your account ID -- it has its own account ID that it sends from.
Adding to Skyler's answer, if like me you cringe at the idea of allowing any principal (Principal: '*'), you can restrict the principal to SNS:
Principal:
Service: sns.amazonaws.com
Although this behavior is undocumented, it works.
Most of the answers (beside #spg answer) propose usage of principal: * - this is very dangerous practice and it will expose your SQS to whole world.
From AWS docs
For resource-based policies, such as Amazon S3 bucket policies, a wildcard (*) in the principal element specifies all users or public access.
We strongly recommend that you do not use a wildcard in the Principal element in a role's trust policy unless you otherwise restrict access through a Condition element in the policy. Otherwise, any IAM user in any account in your partition can access the role.
Therefore it is strongly not recommended to use this principal.
Instead you need to specify sns service as your principal:
"Principal": {
"Service": "sns.amazonaws.com"
},
Example policy:
{
"Version": "2012-10-17",
"Id": "Policy1596186813341",
"Statement": [
{
"Sid": "Stmt1596186812579",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": [
"sqs:SendMessage",
"sqs:SendMessageBatch"
],
"Resource": "Your-SQS-Arn"
}
]
}
With this policy sns will be able to send messages to your SQSs.
There are more permissions for SQS but from what I see SendMessage and SendMessageBatch should be enough for SNS->SQS subscribtion.
Here's a full CloudFormation example of Skyler's answer
{
"Resources": {
"MyTopic": {
"Type": "AWS::SNS::Topic"
},
"MyQueue": {
"Type": "AWS::SQS::Queue"
},
"Subscription": {
"Type" : "AWS::SNS::Subscription",
"Properties" : {
"Protocol" : "sqs",
"TopicArn" : {"Ref": "MyTopic"},
"Endpoint": {"Fn::GetAtt": ["MyQueue", "Arn"]}
}
},
"QueuePolicy": {
"Type": "AWS::SQS::QueuePolicy",
"Properties": {
"Queues": [
{"Ref": "MyQueue"}
],
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "allow-sns-messages",
"Effect": "Allow",
"Principal": {"Service": "sns.amazonaws.com"},
"Action": "sqs:SendMessage",
"Resource": {"Fn::GetAtt": ["MyQueue", "Arn"]},
"Condition": {
"ArnEquals": {
"aws:SourceArn": {"Ref": "MyTopic"}
}
}
}
]
}
}
}
}
}
Amazon has more options in their Sending Amazon SNS Messages to Amazon SQS Queues document.
I just experienced this and took me a while to figure out why:
If I create a SQS subscription from the SNS console, it does not add necessary permissions to the SQS access policy.
If I create the subscription in the SQS console to the same SNS, it does.
Old question but using an AWS SDK version > 1.10
Check out the docs SQS-SNS sendMessage Permission
private static void updateQueuePolicy(AmazonSQS sqs, String queueURL, String topicARN) {
Map<String, String> attributes = new HashMap<String, String>(1);
Action actions = new Action() {
#Override
public String getActionName() {
return "sqs:SendMessage"; // Action name
}
};
Statement mainQueueStatements = new Statement(Statement.Effect.Allow)
.withActions(actions)
.withPrincipals(new Principal("Service", "sns.amazonaws.com"))
.withConditions(
new Condition()
.withType("ArnEquals")
.withConditionKey("aws:SourceArn")
.withValues(topicARN)
);
final Policy mainQueuePolicy = new Policy()
.withId("MainQueuePolicy")
.withStatements(mainQueueStatements);
attributes.put("Policy", mainQueuePolicy.toJson());
updateQueueAttributes(sqs, queueURL, attributes);
}
Which outputs a policy similar to
{
Policy={
"Version":"2012-10-17",
"Id":"MainQueuePolicy",
"Statement":
[
{
"Sid":"1",
"Effect":"Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action":["sqs:SendMessage"],
"Condition":
{"ArnEquals":
{"aws:SourceArn":["arn:aws:sns:us-east-1:3232:testSubscription"]}
}
}
]
}
}
Like the other answers mentioned, you must opt in and grant permission to this SNS topic to publish to your SQS queue.
If you use terraform, you can use the sqs_queue_policy resource.
Here is an example:
resource "aws_sqs_queue_policy" "your_queue_policy" {
queue_url = "${aws_sqs_queue.your_queue.id}"
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "First",
"Effect": "Allow",
"Principal": "*",
"Action": "sqs:SendMessage",
"Resource": "${aws_sqs_queue.your_queue.arn}",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "${aws_sns_topic.your_topic.arn}"
}
}
}
]
}
POLICY
}
If you have enabled encryption on your queue, can also be a reason for SNS not being able to put message on subscriber queue. You need to give access to SNS to that KMS key.
This article explains how to solve this problem:
only you need subscribe the queue to the topic from the queue console.
step one: select the queue
step two: queue Actions
step three: Subscribe Queue to SNS Topic
step choose: the topic
end.
It's by AWS::SQS::QueuePolicy.
You need define a this kind of policy to allow specific SNS perform actions to specific SQS.
QueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- Ref: MyQueue1
- Fn::Sub: arn:aws:sqs:us-east-1:${AWS::AccountId}:my-queue-*
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: allow-sns-messages
Effect: Allow
Principal:
Service: sns.amazonaws.com
Action: sqs:SendMessage
Resource:
- Fn::Sub: arn:aws:sqs:us-east-1:${AWS::AccountId}:my-queue-*
Condition:
ArnEquals:
aws:SourceArn:
- Fn::Sub: arn:aws:sns:us-east-1:${AWS::AccountId}:source-sns-*
With Lambda this kind of policies dont apply. Please if anyone know why please share it.