I am using AWS to chain lambdas together by having a second lambda subscribe to the SNS feed of the first lambda. The second lambda was not receiving anything from the first lambda and I also get no notification in CloudWatch (or my personal email that I subscribed with) that the SNS event has been triggered. So, I am wondering the question: How do I make sure my SNS receives an event from my lambda firing?
When I use the SNS web UI I get logs like these and my lambda logs show they have been touched, but only when I manually send a message.
{
"notification": {
"messageMD5Sum": "e9a72884afc5d9568f2748e34a4e50a4",
"messageId": "04f4a199-cd25-5a45-a877-f054df9b2adf",
"topicArn": "arn:aws:sns:us-east-1:xxxxxxxxxxxx:first-lambda",
"timestamp": "2017-06-28 02:12:14.098"
},
"delivery": {
"deliveryId": "173dca7a-7b31-55a2-8ee9-9bb7698f35f6",
"destination": "arn:aws:lambda:us-east-1:xxxxxxxxxx:function:second-labmda",
"providerResponse": "{\"lambdaRequestId\":\"33972992-5ba7-11e7-b363-09e0f6dc8594\"}",
"dwellTimeMs": 134,
"attempts": 1,
"statusCode": 202
},
"status": "SUCCESS"
}
I've given my lambda publish permissions in it's role, but that only shows up in IAM and when I view its policy inside of the lambda itself on the GUI I just see lambda firing permissions for the lambda itself. I also am sure that my SNS has permissions to write to the lambda because when you set up the lambda (as I did more than once yesterday) it asks for you to grant privileges to the SNS topic. However, even when testing the SNS topic the lambda is never reached, but I do get an email if I subscribe with my email account.
tl;dr
First lambda should be triggering an SNS event that a second lambda subscribes to. The SNS event is not receiving any data from my first lambda except for two logs in the middle of the night that never went through to my second lambda. How do I make sure my SNS receives an event from my lambda firing?
Solved this by adding a service to my lambda to send the message.
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.sns.AmazonSNSClient;
import com.amazonaws.services.sns.model.PublishRequest;
public class SNSPublishService {
private static final String ACCESS_KEY = "credential1";
private static final String SECRET_KEY = "credential2";
private static final String ARN = "arn:aws:sns:<region>:<id>:<topicname>";
public static void publish(String body) throws InterruptedException {
AmazonSNSClient service = new AmazonSNSClient(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
PublishRequest publishRequest = new PublishRequest()
.withTargetArn(ARN)
.withMessage(body);
service.publish(publishRequest);
}}
Related
I want to use a SNS subscription to trigger a Lambda function.
First I have created a topic and subscription via the AWS SDK:
import {SubscribeCommand } from "#aws-sdk/client-sns";
import {snsClient } from "./libs/snsClient.js";
// Set the parameters
const params = {
Protocol: "lambda" /* required */,
TopicArn: "TOPIC_ARN", //TOPIC_ARN
Endpoint: "LAMBDA_FUNCTION_ARN", //LAMBDA_FUNCTION_ARN
};
const run = async () => {
try {
const data = await snsClient.send(new SubscribeCommand(params));
console.log("Success.", data);
return data; // For unit tests.
} catch (err) {
console.log("Error", err.stack);
}
};
run();
It is creating the subscription, but when I check the Lambda, this sub is not appearing as a Lambda trigger.
When I create the subscription manually without using the SDK it's working.
I would like to know why my subscription that created with the SDK is not showing in the Lambda's trigger.
Looks like you are missing the IAM permissions. SNS needs explicit permission to invoke your Lambda.
See How do I subscribe a Lambda function to an Amazon SNS topic in the same account? in the docs. The lambda add-permission CLI step from the example is the same as the AddPermissionCommand in the SDK Lambda client. Provide your SNS topic's ARN.
S3 can be configured to invoke lambda when an object arrives in it.
Is it possible to invoke a REST API (endpoint of a microservice running in EKS) when an object arrives in S3?
From November 2021 is possible to integrate S3 with Amazon EventBridge.
So you can create an EventBridge rule which is triggered on bucket object creation and has API destination as a target.
For this, the option Bucket Properties -> Amazon EventBridge -> "Send notifications to Amazon EventBridge for all events in this bucket" should be enabled.
Then, on EventBridge create the rule with an event pattern like this:
{
"source": ["aws.s3"],
"detail-type": ["Object Created"],
"detail": {
"bucket": {
"name": ["<my-bucket-name>"]
}
}
}
And configure the target as API destination endpoint (configure http method, endpoint, authorization).
You can set up an SNS topic as a target for the event from S3. In the SNS topic, you can add an HTTP/s subscriber, which can be your API endpoint.
Have the Lambda hit the REST API for you.
I want to run a script to publish message to my sns topic. I followed the documentation and my code is this :
import boto3
client = boto3.client('sns')
response = client.publish(
TopicArn='my topic arn',
Message='ptt message '
)
print("Response: {}".format(response))
It prints the response json, but I don't see any message in my AWS SNS console. I have a lambda function which is pretty basic as a subscriber to this sns.
The lambda code :
import json
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
How can I be sure that my message "ptt message" has been published successfully? How can I check it?
For testing of integration you can configure Delivery status logging for your SNS topic. There is a simple setting for this in GUI of SNS. This will export to CloudWatch the logs of your subscription triggers.
Secondly, your Lambda is not doing anything with the event, so your "ptt message stays in the "event" dictionary. You can for example log your event and see the structure where the actual message body comes.
import logging
def lambda_handler(event, context):
logging.info(event)
...
And simply to see that the Lambda was triggered by the subscription, you can open the CloudWatch Logs, find the LogGroup automatically created by AWS for your Lambda (has the name of the function in it) and search the Log Group. There are comfortable filters by "last X minutes".
In order to test this integration you can also publish test messages to SNS Topic from the web interface if this feels more convenient than running a script.
I set up my Lambda function according to the AWS guides by setting a trigger in the setup stage. (the guide except that the guide is using IoT button and I'm using a rule)
It sets up the trigger rule in the AWS IoT console for me. The thing is setup with a certificate and an "iot:*" policy which gives it full IoT access.
The thing is continuously sending messages to the cloud under a certain topic. The messages can be received if I subscribe to it in the AWS IoT Test console.
My lambda function gets triggered if I publish something under that topic from the AWS IoT Test console.
But the function doesn't trigger from the continuous messages sent by the thing. It only triggers from the IoT Test console.
I didn't add any other policy under certificates for the thing in relation to this trigger. Do I have to do so? What should it be?
I tried changing my topic SQL to SELECT * FROM '*'
Try to change your SQL to SELECT * FROM '#'. With # you get every published topic. When you use *, then you don't get topics e.g. sample/newTopic.
With this SQL statement the Lambdas Function gets invoked for every incoming message. When the AWS IoT Console shows the message and your Lambda Function doesn't do anything, try to look if Lambda did a log in CloudWatch.
If your AWS IoT Thing can't trigger AWS Lambda function, you may have a JSON mapping issue and also to improve your SQL query. In my case, I used the following code to provide Lambda a clean input:
SELECT message.reported.* from "#"
With JSON mapping:
{
"desired": {
"light": "green",
"Temperature": "55",
"timestamp": 1526323886
},
"reported": {
"light": "blue",
"Temperature": "55",
"timestamp": 1526323886
},
"delta": {
"light": "green"
}
}
Then you analyze CloudWatch logs:
Then, check your AWS IoT Console for shadow updates (green below - "Atualizações de sombra") and also Publications (orange)
So, your solution will look like this:
For full details of an end-to-end implementation of AWS IoT using Lambda, please access:
IoT Project - CPU Temperature from Ubuntu to AWS IoT
I have an Amazon Lambda instance and an Amazon SNS instance. The Lambda code watches for changes in our database and I would like it to make a call to Amazon SNS to send pushes to our users. For example:
When a user on one of our forums gets a new message, the Lambda code recognizes this change every time it is run (every 10 minutes) and should send a push to the user's smartphone via SNS.
I'm running into a wall when it comes to the documentation; Amazon's docs only talk about how to trigger Lambda code via SNS, but not the reverse. Does anyone have an example of how I can accomplish this?
There is nothing special about pushing SNS notifications in the context of Lambda. I would think of it as just another external service that you interact with.
What you could do is pull in the AWS SDK in your lambda code and after that use the code to make the SNS calls. You will need to inject the right credentials to be able to call the Amazon SNS API (but you probably do something similar for getting the database endpoint and credentials if you are talking to the database)
Yes, you can use AWS Lambda to achieve what you want. You also need to give proper IAM Permissions allowing your Lambda IAM Role to publish messages to you SNS Topic.
Example SNS Publish IAM Policy:
{
"Statement":[ {
"Effect":"Allow",
"Action":"sns:Publish",
"Resource":"arn:aws:sns:*:<your account id>:<your topic id>"
} ]
}
You can use the lambda below to push an SNS message to a user, but you must know what the endpoint ARN is for that user. For example, if in an Android app, when the user logs in you will have the app send a GCM (Google Cloud Messaging) token to your backend (via an API call that triggers a lambda, for example). Your backend, which is connected to GCM, can then use this token to lookup which endpoint ARN corresponds to such user and put that in the lambda below. Alternatively, you can have the app send the endpoint ARN directly to your backend, though I think it might be a bit less secure. Make sure you give IAM permissions to publish to your app via SNS. You can use the lambda below to push the message:
var AWS = require('aws-sdk');
var sns = new AWS.SNS({apiVersion: '2010-03-31'});
exports.handler = (event, context, callback) => {
console.log(JSON.stringify(event))
var payload = {
"default": "The message string.",
"GCM":"{"+
"\"notification\":{"+
"\"body\":\"PUT NOTIFICATION BODY HERE\","+
"\"title\":\"PUT NOTIFICATION TITLE HERE\""+
"}"+
"}"
};
payload = JSON.stringify(payload);
var params = {
TargetArn: 'PUT THE ENDPOINT ARN HERE',
Subject: 'foo2',
MessageStructure: 'json',
Message: payload
}
sns.publish(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
};