Wrong SQS AWS message when I'm subscribed from a SNS Topic - amazon-web-services

Question

I'm having problems with the next design:
When I'm receiving the message in my SQS Subscriber, the model of message it's wrong, example:
{
"Type" : "Notification",
"MessageId" : "7a6789f0-02f0-5ed3-8a11-deebcd08f145",
"TopicArn" : "arn:aws:sns:us-east-2:167186109795:name_sns_topic",
"Message" : "My JSON message",
"Timestamp" : "1987-04-23T17:17:44.897Z",
"SignatureVersion" : "1",
"Signature" : "string",
"SigningCertURL" : "url",
"UnsubscribeURL" : "url",
"MessageAttributes" : {
"X-Header1" : {"Type":"String","Value":"value1"},
"X-Header2" : {"Type":"String","Value":"value2"},
"X-Header3" : {"Type":"String","Value":"value3"},
"X-HeaderN" : {"Type":"String","Value":"value4"}
}
}
The common model when recieve message from SQS should be:
{
"Records": [
{
"messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
"receiptHandle": "MessageReceiptHandle",
"body": "Hello from SQS!",
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1523232000000",
"SenderId": "123456789012",
"ApproximateFirstReceiveTimestamp": "1523232000001"
},
"messageAttributes": {},
"md5OfBody": "7b270e59b47ff90a553787216d55d91d",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:{partition}:sqs:{region}:123456789012:MyQueue",
"awsRegion": "{region}"
}
]
}
In my handler Java Lambda (example code) is throwing an exception because the estructure of de message received is not SQS Event:
public class MyHandler implements RequestHandler<SQSEvent, String> {
#Override
public String handleRequest(SQSEvent event, Context context) {
LambdaLogger logger = context.getLogger();
for (SQSEvent.SQSMessage msg : event.getRecords()) {
logger.log("SQS message body: " + msg.getBody());
logger.log("Get attributes: " + msg.getMessageAttributes().toString());
msg.getMessageAttributes()
.forEach(
(k, v) -> {
logger.log("key: " + k + "value: " + v.getStringValue());
});
}
return "Successful";
}
}
How can I do for handle the message thats its receiving ?

Answer 1

In my opinion this isn't documented too well but it's not bad once you figure it out.
The first thing is that I don't use the predefined Lambda objects. I read everything into a String and take it from there. So the base of my Lamda function is:
public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
// copy InputStream to String, avoiding 3rd party libraries
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
String jsonString = result.toString();
}
When you "go direct" from SNS to Lambda the message looks something like (some fields removed for sake of length):
{
"Records": [
{
"EventSource": "aws:sns",
"EventVersion": "1.0",
"Sns": {
"Type": "Notification",
"Subject": "the message subject",
"Message": "{\"message\": \"this is the message\", \"value\": 100}",
"Timestamp": "2020-04-24T21:44:28.220Z",
"SignatureVersion": "1"
}
}
]
}
I had sent in a test message in JSON with two simple fields. Using JsonPath the "message" field inside of everything is read with:
String snsMessage = JsonPath.read(jsonString, "$.Records[0].Sns.Message");
String realMessage = JsonPath.read(snsMessage, "$.message");
But when it goes SNS -> SQS -> Lambda (or, indeed any SNS -> SQS path) the SNS message is now mostly wrapped and escaped in an SQS message:
{
"Records": [
{
"messageId": "ca8c53e5-8417-4479-a720-d4ecf970ca68",
"body": "{\n \"Type\" : \"Notification\",\n \"Subject\" : \"the message subject\",\n \"Message\" : \"{\\\"message\\\": \\\"this is the message\\\", \\\"value\\\": 100}\"\n}",
"attributes": {
"ApproximateReceiveCount": "1"
},
"md5OfBody": "6a4840230aca6a7bf7934bf191a529b8",
"eventSource": "aws:sqs"
}
]
}
So in this case, the value is in Records[0].body but that contains another JSON object. I'll admit that there is likely an easier way but from what I found I had to parse 3 times:
String sqsBody = <as read in lambda>;
String recordBody = JsonPath.read(sqsBody, "$.Records[0].body");
String internalMessage = JsonPath.read(recordBody, "$.Message");
// now read out of the sns message
String theSnsMessage = JsonPath.read(message, "$.message");