I have a topic that has multiple subscriptions. When one of the subscription fails the message is moved to the DLQ. Is there a way to find out which subscription failed? and if the same message is processed again then how to make sure that the subscriptions that have already processed the message do not process it again?
In topic subscription if there are n numbers of subscription and when message is sent to the topic then each message is sent to the individual subscription. Depending on the filter condition the subscription will either accept or discard the message. Each subscription will have their own dead letter so you can read the messages from the individual subscription dead letter.
Related
I have two way SMS setup with Pinpoint where the incoming SMS gets sent to an SNS Topic. I'd like to re-use this SNS topic for other phone numbers that are using two way. However, the subscribers to that topic will need to only see messages for the phone number that that particular subscriber is concerned with.
I know about subscription filter policies. However, the incoming SMS doesn't have a "Message Attributes" key and I haven't seen a way I can promote the "to" phone number or "keyword" to the message attribute so I can filter on it.
Is there a way to parse the body of the message to create the filter for a subscription?
There are no properties of a two way message that are promoted to be filtered on. I ended up sending the message to a lambda that promotes the properties I needed and re-published the message to another topic. I then had subscription filters subscribed to that topic.
I am working on implementing event driven using GCP Pub/Sub.
I have a topic called orders, and that topic will have a subscription named orderPlacedSubscription I have two services that want to listen to all messages for this subscription and perform different actions,
so I have paymentService and notificationService, the paymentService will listen to each message filtered by orderPlacedSubscription and process the payment as well as the notification service will listen to the same message and send a notification.
My question
does pub-sub support having two subscribers related to one subscription and both receive messages and acknowledge them separately?
In case if each subscriber can acknowledge the message separately without affecting the other subscriber, does google cloud pub-sub support retry for different subscribers in case of failure from one subscriber?
Yes, a subscription can have multiple subscriber clients.
In a subscription workflow, if a message is not acknowledged by a subscriber, Pub/Sub will attempt to redeliver the outstanding message. In the process of redelivering the outstanding message, Pub/Sub holds back and tries not to deliver the outstanding message to any other subscriber on the same subscription. Once the outstanding message is acknowledged it can be delivered to other subscribers.
You can refer subscription workflow and this documentation for more information.
I have several services, and each service creates an SNS topic which sends an email to me. e.g., whenever code deploy fails, I receive an email from an SNS topic, whenever codepipeline fails I receive an email from another SNS topic, whenever a specific lambda fails, I receive another SNS topic. None of the above failures is urgent, so it does not need to be taken care immediately.
How can I receive only one email report once a day for all the above failures? Maybe I should let the corresponding service trigger SQS instead of SNS and then once every day I pull all messages from SQS?
The option you mentioned with an SQS queue subscribed to the SNS topic is probably the best way to go. You can then use a scheduled Lambda function that runs once a day to poll the messages in the queue and send you an email with the aggregated information.
When I publish to a topic it hits ALL subscribers ALWAYS?
I have a topic with several subscriptions, sometimes I want to publish a message to just one of those subscriptions.
Is there a way to do this or do I need to create another topic and have the subscription in 2 topics? In that case I'm bugging the user (assuming this use case is to message users) twice right?
Yes -- when a message is sent to an Amazon SNS topic, all subscribers receive the message.
If you wish to contact a specific subscriber, your code will have to contact them directly (eg via email).
Amazon SNS also has the ability to send an SMS message to one or more recipients without using an SNS Topic. So, if your desired recipients are on SMS, this is a simple task.
I am sending many messages but the subscribers change very often (per message), how can can i choose my specific subscribers on every message? the number of possibilities is too much to create topic to each combination.
You can't. A message is delivered to all subscribers of the SNS topic.
Once you subscribe an endpoint to a topic and the subscription is
confirmed, the endpoint will receive all messages published to that
topic. Source
You can manage the endpoints yourself on the back end. You then publish to an individual endpoint. For large groups you will have to build in flood protection.