I know how to do this in java, but I just can't figure it out in Go at all.
All I want to do, is have a way to detect that an item got created in an S3 bucket, then have that trigger an SNS topic, which then notifies me of the file location in S3.
Has anybody got a working example of how I can do the go side of this for subscribing to the SNS topic or the SNS queue if I need one? Because all I seem to be able to find is Java and Node. I can find publish examples for go, but they are of little use to my use case.
To use SNS you will need a simple HTTP/HTTPS endpoint to receive SNS notifications. Which is divided into two parts (Confirm the subscription and Processing messages from HTTP/HTTPS endpoint)
1. Confirm the subscription
Do something as simple as this:
func confirmSubscription(subcribeURL string) {
response, err := http.Get(subcribeURL)
if err != nil {
fmt.Printf("Unbale to confirm subscriptions")
} else {
fmt.Printf("Subscription Confirmed sucessfully. %d", response.StatusCode)
}
}
2. Processing messages from HTTP/HTTPS endpoint
Parse the request's body, the documentations mentions how the body should be structured.
Sources:
https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html
https://github.com/viveksyngh/aws-sns-subscriber/blob/master/subscriber/subscriber.go
Related
I'm looking to send a message to an SQS queue from a Lambda function written in Go, which in the AWS console I have configured so that the SQS queue is connected as a Destination.
The code sample I've seen (here) suggests I need to load a config, create a client and get a queueURL before issuing the message via a call to SendMsg. This approach works fine when running the code locally, but seems unnecessary within the Lambda given I'm connecting the SQS queue in the console. Do I need to tell my Lambda to find the queueURL when I've connected to it as a destination already?
As an aside, I tried this approach but was receiving the following error when running the code from within the Lambda. Invoking the code outside of a Lambda locally would find the queueURL without issue.
{
"errorMessage": "operation error SQS: GetQueueUrl, https response error StatusCode: 400, RequestID: 2f725471-ec4e-5380-99ed-0f708c1fcfa4, AWS.SimpleQueueService.NonExistentQueue: ",
"errorType": "OperationError"
}
This led me to look at the flip side, where I have a Lambda which is consuming from the queue and thereby has the SQS queue configured as a trigger. I simply need to iterate over the sqsEvent.Records without having to specify config, client or queueURL as I presume this is handled in the console behind the scenes, i.e.
package main
import (
"context"
"fmt"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
)
func handler(ctx context.Context, sqsEvent events.SQSEvent) error {
for _, message := range sqsEvent.Records {
fmt.Printf("The message %s for event source %s = %s \n", message.MessageId, message.EventSource, message.Body)
}
return nil
}
func main() {
lambda.Start(handler)
}
Is there a way of sending a message to a queue from within the Lambda without having to find the queueURL first?
I have configured so that the SQS queue is connected as a Destination.
Sending a message doesn't need you to define a destination per se - destinations are for passing asynchronous Lambda function invocation results. Basically - if you don't explicitly need them (and you know when you do), they won't be what you need.
Give your Lambda permission to send messages to the queue, & use SendMessage using the AWS SDK to send messages. No destinations needed.
AWS.SimpleQueueService.NonExistentQueue
The error message suggests it's not finding the queue, maybe double-check if you're looking in the right region.
I simply need to iterate over the sqsEvent.Records without having to specify config, client or queueURL as I presume this is handled in the console behind the scenes, i.e.
Yes, this is abstracted away - ultimately, your Lambda function is invoked by events and setting Amazon SQS as a trigger allows it to pass events to your function.
Is there a way of sending a message to a queue from within the Lambda without having to find the queueURL first?
Let's rephrase it as - is there a way of sending a letter to someone without having to find their address? Of course not, as we need to know where the letter needs to go.
The same concept applies to SQS queues.
In AWS IOT, I'm able to publish an MQTT message from a device to the topic "topic/messages" :
{
"id":"messageID",
"value":"messageValue"
}
And I want to "augment" it in the server by adding a timestamp on it and let the message to continue on the SAME TOPIC.
So I'd like that subscribers on "topic/messages" receive this:
{
"id":"messageID",
"value":"messageValue",
"serverTimestamp":"1637867431920" <--- Here
}
However, I don't find the way how to process this message and let it flow in the same topic:
I can add a rule
SELECT * , timestamp() as serverTimestamp FROM 'topic/#'
But the rule does not augment the original message but creates an augmented copy and redirects it to some other service (Lambda, DynamoDB, Republish, etc..)
Those services work with the copy of the given value, but not with the original message, so the subscribers still receives the original sent message.
{
"id":"messageID",
"value":"messageValue"
}
I can republish the message using the same topic BUT as the topic has an attached rule, after the republishing, the rule's action is triggered again and again in a recursive loop)...
All the AWS examples I've read are meant to take the message, transform it , and do some other different thing with it (save in DynamoDB, save in a Bucket, end to Salesforce....) but none of them modify the message to be sent.
So what I'm looking for is a way to receive the message, add a field (or more) to it and let it flow in the same topic .
What is the simplest way to do this?
I have setup an SQS, DeadLetterQue (DLQ) and a lambda. I am having some questions regarding the body format that the SQS receives.
After some tries, I can see some messages are ending up in a DLQ.
I have read many documentations, but really I still can't figure some basic things.
My task is fairly simple. I will use Axios to send some payload to external API. That API might give me an error, or not available at that time. This code is very simple, but at the end it will include Axios implementation:
exports.handler = async (event, context) => {
event.Records.forEach(record => {
const { body } = record;
const { messageAttributes } = record;
console.log(body)
console.log(messageAttributes);
});
let somethingWentWrongWithApi = new Error();
throw somethingWentWrongWithApi;
};
After this is run, after some time I see message in DLQ, and original SQS is empty, as I expected.
In real code, I will have catch block. And inside it I will throw the error, exactly as in example. Lambda will try to execute it three times (I am not sure where I got this information). Then, it will return it to SQS, where in turn the SQS will push it to DLQ.
I wonder, should I implement in Lambda retries, and after three times throw the error... or just throw it and rely on existing process (three times retrial)?
I am still reading about different setting on both SQS and Lambda.
To extend execution tries you have to go to:
SQS service;
Right click on your SQS;
Choose a configure Queue option;
Under the Dead Letter Queue Settings line you can see a
Maximum Receives property (which you can set between 1 and 1000).
About errors. If something going wrong on lambda side then lambda throwing an error, you can check what kind of errors throwed with cloudwatch.
Go to aws CloudWatch;
On the right panel find Logs;
Chose Log Group.
If you have any additional questions just summon me. I'll try to answer them and I'll extend that answer.
Thanks to this community I've learned that is possible to send AWS SNS Push notifications via Lambda with node.js (as a result of Parse migration). I am still struggling with the following:
Can this be done client to client x likes y's z. Where x is user 1, y is user 2 and z is the object being liked? If so, it seems like Cognito is not required that it can read directly from the database but is that accurate?
Does anyone have an example of how this was implemented?
Again, we don't want to broadcast to all users on a schedule but rather when a client performs an action.
Thanks so much in advance!
Let's say you have Device1 which creates a piece of content. That is distributed to a number of users. Device2 receives this content and "likes" it.
Assumption:
you have registered for push notifications on the device, and created a SNS endpoint on AWS. You have stored that endpoint ARN in your database, and associated it with either the Cognito Id, or the content Id. If your data is normalized, then you'd typically have the SNS endpoint associated with the device.
Your Lambda will need to have access to that data source and look up that SNS endpoint to send push notifications to. This will depend on what sort of data store you are using (RDS, DynamoDB, something else). What sort of access that is, and how you secure it is a whole other topic.
From your Lambda, you fetch the ARN to send the push notification to. If you pass in the content Id from the Like, and have the Cognito Id from the device that Liked it, you can then look up the information you need. You then construct a SNS payload (I'm assuming APNS in this example), then send it off to SNS.
var message = {
"default": "New Like!",
"APNS": {
"aps": {
"alert": "New Like!"
}
}
};
var deviceParams = {
Message: JSON.stringify(message),
Subject: "New Like",
TargetArn: targetArn,
MessageStructure: "json"
};
self.sns.publish(deviceParams, function (err) {
if (err) {
console.error("Error sending SNS: ", err);
}
});
It's not all done for you like it might be with Parse. You need to work a lot harder on AWS, but you have near unlimited power to do what you want.
If this is a bit too much, you may want to consider Google's newly updated Firebase platform. It's very Parse-like: https://firebase.google.com/
Hope that helps you move forward a bit.
Further reading:
http://docs.aws.amazon.com/sns/latest/dg/mobile-push-apns.html
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/TheNotificationPayload.html
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SNS.html
I am publishing a String message as message payload using SNS notification from Raspberry Pi using Python program and I want to pass that message payload to a Lambda function.
I have configured the requirement in the SNS console on AWS i.e., I have created a topic and added the lambda function to its subscribers.
Now, I want to get that message payload in the lambda function. But I can't find any method that can help me do that. For example, something like getMessage or something similar to that.
So my questions are: Since I have configured the publishing and subscription on AWS, can I assume that the clients are connected and if I publish a message I should be getting that at the subscriber's end which is my lambda function here?
Also, what's the technique in which I can get the message payload in my lambda function?
I am adding the below as per cjwfuller's suggestion.
Below I have written down the method for publishing in Python
client_boto = boto3.client('sns', aws_access_key_id='###',
aws_secret_access_key='###', region_name='us-west-2')
REGION = 'us-west-2'
TOPIC = 'arn:aws:sns:us-west-2:***:topic_name'
MSG = ntpath.basename(f_string)
SUBJECT_boto = 'File Name'
pub =client_boto.publish(TopicArn = TOPIC, Message = MSG,
Subject=SUBJECT_boto)
I am writing the subscribing code in Java.
Since my lambda func is already subscribed to it on AWS console,
should my Java program include the subscription again or is there a
way to get the msg payload directly.
Which language are you writing the function in? JavaScript as an example:
exports.myHandler = function(event, context, callback) {
console.log("value1 = " + event.key1);
console.log("value2 = " + event.key2);
// ...
}
Source: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
It's useful testing the sort of stuff in the AWS Lambda console before writing all the code.
Since I have configured the publishing and subscription on AWS, can I assume that the clients are connected
Clients aren't really "connected", they're simply subscribed to a topic.
publish a message I should be getting that at the subscriber's end which is my lambda function here?
Sounds like you're doing the right sort of thing - posting example code will help us come up with more precise answers.
On searching, I have found the class for SNSEvent which is, https://github.com/aws/aws-lambda-java-libs/blob/master/aws-lambda-java-events/src/main/java/com/amazonaws/services/lambda/runtime/events/SNSEvent.java
This Class contains all the methods related to and needed to get the message payload.
The Lambda function handler in Java goes something like this,
example;
public void handleRequest(SNSEvent input, Context context){
String this_takes_message=input.getRecords().get(0).getSNS().getMessage();
}