When a message is sent using sendMessage() function with a MessageDeduplicationId to prevent duplicates.
However, the sendMessage function always returns true. In other words, the sendMessage function always returns a value that says message successfully sent to SQS, however the message actually is not available.
I change a field 'sent_to_sqs' in the DB based on the sendMessage() return value.
Can anybody help me?
The design assumption is that if you send a message with the same deduplication-id, then that is the exact same message you already sent (otherwise, it should not have the same deduplication-id), so SQS pretends to accept it again, on the assumption that something must have gone wrong on your side, preventing you from realizing that the message was already submitted the first time.
“the sendMessage function always returns a value that says message successfully sent to SQS”
Yes, because it already was, earlier.
SQS doesn't verify that the message is actually a duplicate -- it assumes that it is, because you've claimed it must be by submitting the same deduplication-id within the 5 minute window. If the message has already been handled, then it will not be available on the queue, by design.
Message Deduplication ID
The token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval.
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html#FIFO-queues-understanding-logic
Related
I have a pretty standard setup of feeding SQS to Lambda. The lambda reads the message and makes a web request to a defined endpoint.
If I encounter an exception during processing of the SQS message that is due to the form of the message then I put the message on a dead letter queue.
If I encounter an error with the web request, I put the message back on the feeding queue to make the HTTP request at a later time.
This seems to work fine, but we just ran into an issue where an HTTP endpoint was down for 4 days and the feeding queue dropped the message. I imagine this has something to do with the retention period setting of the queue.
Questions
Is there a way to know, in the lambda, how many times a message has been replayed?
How did the feeder queue know that the message that was re-enqueued was the same as the one that was originally put on the queue?
I'm currently not explicitly deleting a message off the queue. Not having that, hasn't seemed to cause any issues, no re-processing of messages or anything. Should I be explicitly deleting them?
The normal process would be:
The AWS Lambda function is triggered, with the message(s) passed via the event parameter
If the Lambda function successfully processes the message(s), it should return a 'success' code (200) and the message is automatically removed from the queue
If the Lambda function is unable to process the message, it should return a 'failure' code (eg 400) and Amazon SQS will automatically attempt to re-process the message (unless it has exceeded the retry count)
If the Lambda function fails (eg due to a timeout), Amazon SQS will automatically attempt to re-process the message (unless it has exceeded the retry count)
If a message has exceeded its retry count, Amazon SQS will move the message to the Dead Letter Queue
To answer your questions:
If you wish to take responsibility for these activities yourself, you can use the ApproximateReceiveCount attribute on the message. In the request, it appears that you should add AttributeNames=['ApproximateReceiveCount'], but the documentation is a bit contradictory. You might need to use All instead.
Since you are sending a new message to the queue, Amazon SQS is not aware that it is the same message. The message is not 're-enqueued' since it is a new message.
When your Lambda function returns 'success' (200), the message is being deleted off the queue for you.
You might consider using the standard functionality for retries and Dead Letter Queues rather than implementing that logic yourself.
I've created an AWS SQS, and I'm just testing through the console by sending some messages. I was expecting to see the messages that I've sent listed under the "Received messages" title. But, to my surprise, approximately only one of every ten messages that I send seems to be received.
I test this, as shown in the screenshot, by clicking on "Poll for Messages" and then on "Send a message".
I'm bewildered by the fact that sometimes a message is actually received and displayed.
Wouldn't the expected behavior be to receive every single message I send?
Any idea on what's going on, please?
Have you checked message deduplication is active? There might be some deduplication by message data itself.
Or another possible reason might be some other consumer gets the message before your poll.
So I am building a small application that uses SQS. I have a simple handler process that determines if a given message is considered processed, marked for retry (to be re-queued) or is not able to be processed (should be sent to dead letter).
However based on the docs it would appear the only way to truly send a message to DL is by using a redrive policy which operates over # of receives a message has racked up. Because of the nature of my application, I could have several valid retries if my process isn't ready to handle a given message, but there are also times I may want to DL a message I have just received. Does AWS/Boto3 not provide a way to mark a specific message for DL?
I know I can just send the message myself to another queue I consider my own DL, I would just rather use AWS' built in tools for this.
I don't believe there is any limitation that would prevent you from sending the message to the deal-letter-queue by yourself.
So just read the message from the Q, if you know it needs to go to the DLQ directly, send it to the DLQ and remove it from the regular Q.
While using AWS SQS, does "SentTimestamp" attribute of a message change after it is received from the queue, but not deleted and returned back to the queue after the visibility timeout expired?
No, and that behavior would be undesirable because SentTimestamp specifically describes when the message was first sent to the queue.
This documentation that describes the visibility timeout offers some insight:
Immediately after the component receives the message, the message is
still in the queue. However, you don't want other components in the
system receiving and processing the message again. Therefore, Amazon
SQS blocks them with a visibility timeout, which is a period of time
during which Amazon SQS prevents other consuming components from
receiving and processing that message.
The important takeaway here is that the message never really leaves the queue, it is simply hidden from other clients that are receiving messages. So message contents like the MessageID and SentTimeout won't change. On the other hand, things related to receiving the message like RecieptHandle and Receive Count do change each time the message is received.
You can self-verify this from the AWS web console by:
Creating a message in a queue.
Viewing the message.
Waiting for the visibility timeout to expire. Once it has, open the SQS console again in a new tab.
Viewing the message again, in the new tab. Compare the contents of both received messages.
Given an Amazon SQS message, is there a way to tell if it is still in flight via the API? Or, would I need to note the timestamp when I receive the message, subtract that from the current time, and check if that is less than the visibility timeout?
The normal flow for using Amazon Simple Queueing Service (SQS) is:
A message is pushed onto a queue using SendMessage (it can remain in the queue for up to 14 days)
An application uses ReceiveMessage to retrieve a message from the queue (no guarantee of first-in-first-out)
When the application has finished processing the message, it calls DeleteMessage (it can also call ChangeMessageVisibility to extend the time until it times-out)
If the application does not delete the message within a pre-configured time period, SQS makes the message reappear on the queue
If a message is retrieved from the queue more than a pre-configured number of times, the message can be moved to a Dead Letter queue
It is not possible to obtain information about a specific message. Rather, the application asks for a message (or a batch of messages), upon which the message becomes invisible (or 'in flight'). This also gives access to a ReceiptHandle that can be used with DeleteMessage or ChangeMessageVisibility.
The closest option is to call GetQueueAttributes. The value for ApproximateNumberOfMessagesNotVisible will indicate the number of in-flight messages but it will not give insight into a particular message.