I'm trying to follow along the amazon sns publish example using the Amazon documentation site but it's vague on Message, MessageAttributes and MessageStructure.
First of all, is the Message property going to be a string even if you set MessageStructure to json? e.g. If I want to send an object instead of just a string message. If it's string - do I need to JSON.stringify my object before passing it as a Message property?
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SNS.html#publish-property
Should I be doing this using MessageAttributes instead? What is that property - the amazon documentation merely states it "Message attributes for Publish action" which seems like a tautology.
http://docs.aws.amazon.com/sns/latest/APIReference/API_Publish.html
Setting MessageStructure to json is only used if you are going to send a json-formatted message structure to SNS in the specific jormat SNS understands. This is only used when you are publishing to multiple endpoint types and want to vary the message body by endpoint type. This isn't the same as "I want to send a message where the body has been serialized as JSON."
If you are sending "a JSON object," you need to stringify it, and send it just as you would any other (non-JSON) messages, because SNS messages are, fundamentally, strings.
MessageAttributes are something else entirely. They allow you to send pseudo-out-of-band key/value pairs along with your message, which can be useful for example if your message has been gzipped and base64 encoded (again, for example) you could attach an "external_id" attribute that the recipient could evaluate to decide whether it needed to unpack the whole message or could just discard it.
Related
In AWS IOT, I'm able to publish an MQTT message from a device to the topic "topic/messages" :
{
"id":"messageID",
"value":"messageValue"
}
And I want to "augment" it in the server by adding a timestamp on it and let the message to continue on the SAME TOPIC.
So I'd like that subscribers on "topic/messages" receive this:
{
"id":"messageID",
"value":"messageValue",
"serverTimestamp":"1637867431920" <--- Here
}
However, I don't find the way how to process this message and let it flow in the same topic:
I can add a rule
SELECT * , timestamp() as serverTimestamp FROM 'topic/#'
But the rule does not augment the original message but creates an augmented copy and redirects it to some other service (Lambda, DynamoDB, Republish, etc..)
Those services work with the copy of the given value, but not with the original message, so the subscribers still receives the original sent message.
{
"id":"messageID",
"value":"messageValue"
}
I can republish the message using the same topic BUT as the topic has an attached rule, after the republishing, the rule's action is triggered again and again in a recursive loop)...
All the AWS examples I've read are meant to take the message, transform it , and do some other different thing with it (save in DynamoDB, save in a Bucket, end to Salesforce....) but none of them modify the message to be sent.
So what I'm looking for is a way to receive the message, add a field (or more) to it and let it flow in the same topic .
What is the simplest way to do this?
I'm building an application that enables clients to book appointments with service providers. I'm using SNS -> SQS -> Lambda to process various emails that need to be sent when booking an appointment. IE I currently send an SNS message like so (in node.js):
await sns.publish({
Message: 'booking-request',
TopicArn: process.env.AWS_BOOKING_REQUEST_TOPIC_ARN,
MessageAttributes: {
artistEmail: SNSMF.string(artist.email),
artistName: SNSMF.string(artist.name),
clientEmail: SNSMF.string(req.body.email),
clientName: SNSMF.string(`${req.body.firstName} ${req.body.lastName}`),
date: SNSMF.string(moment(req.body.date).tz(studio.timeZone).format())
}
}).promise();
This all works fine, but I'm using MessageAttributes to pass the pertinent appointment details so my notifications layer can send the proper emails.
My main questions is, am I using MessageAttributes in the proper way, or is there a better way to pass all of this data? Should the data be the message itself? I ask because I believe that you can only have 10 MessageAttributes and I'm going to run into a limit with the appointment details (currently collecting about 10-12 data points about the appointment that I want to include in the emails). Any ideas? Thank you!
Normally, the 'main' information you wish to pass would be in the Body of the message. It is quite common the use JSON to pass various types of information.
MessageAttributes are normally something about the message itself rather than the content of the message, such as timestamps, priority and user information.
Given your requirements, I putting your data in the Body (eg in JSON) would avoid hitting limits and would also be more extensible.
I've created a lex sms chatbot and published it to a Twilio SMS Channel. Within my java lambda fulfillment handleRequest function I receive 2 parameters: an Input object and a Context. Input has some type of system generated userId, but I need the phone number of the incoming sms message from Twilio.
I've configured Twilio to call Lex via webhook per these instructions:
http://docs.aws.amazon.com/lex/latest/dg/twilio-bot-association.html#twilio-bot-assoc-create-assoc
I'm using the system created webhook callback url between Lex and Twilio and would like to avoid writing my own Lex-Twilio go between with Gateway API and lambda if possible.
I think I'm missing a configuration in Twilio to send this to Lex maybe? I haven't setup any TwilML app or anything, just linked the SMS number to my webhook callback url for lex. Everything works fine except getting the incoming sms number.
UPDATE:
I exhausted all possibilities with my input object and context in my java lambda function. I guess there's no way to get at the payload from twilio in Java. I tried switching to an input stream handler but still only had the input format defined in the documentation here: http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
I had to bite the bullet and build my own gateway API and Node handler. It's just a layer that sits before lex and translates twilio into lex input format and vis versa on the response. It updates the userId with the incoming phone number.
This tutorial was very useful in doing this:
https://aws.amazon.com/blogs/ai/integrate-your-amazon-lex-bot-with-any-messaging-service/
It was a lot of work just to get one additional field, but It does provide me the flexibility to switch SMS providers in the future easier.
I am not sure as what exactly stopped you from getting the user phone number in your lambda function. You can always get sender phone number from 'userId' field from the input event in your lambda function. You can refer to the documentation to get more information.
What is the purpose of using message body in SQS while you're already able to add message attributes?
Let's take an example, we want to push a message to new-user queue when a new user registered, I imagine the message will have an attribute userId, I don't see the use of body here.
Message attributes are supposed to be used as message metadata (like timestamp or possibly some category) and not the message itself.
Ideally, message payload should be given in the message body
So, for example if you are supporting JSON and XML payloads then possibly you can put payload type as message attribute and then when you fetch the message, based on this payload type attribute you decide between the JSON message processor or XML message processor. This is just a superficial example to explain the usage of attributes and body
Following is the extract from AWS Doc
Amazon SQS provides support for message attributes. Message attributes allow you to provide structured metadata items (such as timestamps, geospatial data, signatures, and identifiers) about the message. Message attributes are optional and separate from, but sent along with, the message body. This information can be used by the receiver of the message to help decide how to handle the message without having to first process the message body. Each message can have up to 10 attributes. To specify message attributes, you can use the AWS Management Console, AWS software development kits (SDKs), or query API.
To map with the traditional queue provider such as rabbitMQ or Kafka world.
We can understand as below:
message_body=message_payload
message_attributes=message_headers ( can be used to apply different routing and filtering message using their headers information)
In fact, I prefer the term payload and headers more than what terms used in aws sqs, its abit confusing.
Message attributes sound more like the attributes of message payload
I set the MX records for my domain to the ones AWS provides for receiving emails with SES.
Now I want to process incoming emails via Lambda function.
To be more specific, I would like to scan the message body for certain keywords and perform a task based on this.
Does the lambda function get the email body? When looking through the event variable, I could only find a subject line.
All examples I found went to S3 to get the message body. Does the body really not get sent to Lambda and it is necessary to fetch it via S3?
Message bodies can get very large. That's why you need to download them from S3.