Can ZeroMQ Publisher Subscriber sockets be configured so that a newly-connected client always receive last published message (if any)?
What am I trying to do: My message is a kind of system state so that new one deprecates previous one. All clients has to have current states. It works for already connected clients (subscribers) but when the new subscriber appears it has to wait for a new state update that triggers a new message. Can I configure PubSub model to send the state to the client immediately after connection or do I have to use a different model?
There is an example in
the ZMQ guide called Last Value Caching. The idea is to put a proxy in between that caches the last messages for each topic and forwards it to new subscribes. It uses an XPUB instead of a PUB socket to react on new connections.
Related
In a question and response here:
ActiveMQ Producer Multiple Queues One Session
The topic of a single producer sending messages to more them one destination is covered with a solution in Java.
Can the same thing in CPP/CMS be done?
I've tried to replicate that code using cms/activemq API but when I try to send a message to a different queue(destination), I receive error messages stating the producer can only send to the old destination.
Without writing the exact code here is the flow...
Create new Factory
Set broker URI
Create Connection
Connection start
Create Session
Create MessageProducer with a temporary queue
Create a new queue
Use session to create message
MessageProducer send using new queue and message
It is unclear what you code has done since you didn't include it but given the minimal input my guess is that you are creating a fixed destination producer by calling session->createProducer with some destination (sounds like a temp queue). This creates a producer married to that destination for life and calling the send methods that take a destination are required to throw. If you want to pool a producer and send to many different addresses then you need to create it with a NULL destination.
Is it possible to ensure that a message was successfully delivered to an Event Hub when sending it with the log-to-eventhub policy in API Management?
Edit: In our solution we cannot allow any request to proceed if a message was not delivered to the Event Hub. As far as I can tell the log-to-eventhub policy doesn't check for this.
Welcome to Stackoveflow!
Note: Once the data has been passed to an Event Hub, it is persisted and will wait for Event Hub consumers to process it. The Event Hub does not care how it is processed; it just cares about making sure the message will be successfully delivered.
For more details, refer “Why send to an Azure Event Hub?”.
Hope this helps.
Event Hubs is built on top of Service Bus. According to the Service Bus documentation,
Using any of the supported Service Bus API clients, send operations into Service Bus are always explicitly settled, meaning that the API operation waits for an acceptance result from Service Bus to arrive, and then completes the send operation.
If the message is rejected by Service Bus, the rejection contains an error indicator and text with a "tracking-id" inside of it. The rejection also includes information about whether the operation can be retried with any expectation of success. In the client, this information is turned into an exception and raised to the caller of the send operation. If the message has been accepted, the operation silently completes.
When using the AMQP protocol, which is the exclusive protocol for the .NET Standard client and the Java client and which is an option for the .NET Framework client, message transfers and settlements are pipelined and completely asynchronous, and it is recommended that you use the asynchronous programming model API variants.
A sender can put several messages on the wire in rapid succession without having to wait for each message to be acknowledged, as would otherwise be the case with the SBMP protocol or with HTTP 1.1. Those asynchronous send operations complete as the respective messages are accepted and stored, on partitioned entities or when send operation to different entities overlap. The completions might also occur out of the original send order.
I think this means the SDK is getting a receipt for each message.
This theory is further aided by the RetryPolicy Class used in the ClientEntity.RetryPolicy Property of the EventHubSender Class.
In the API Management section on logging-to-eventhub, there is also a section on retry intervals. Below that are sections on modifying the return response or taking action on certain status codes.
Once the status codes of a failed logging attempt are known, you can modify the policies to take action on failed logging attempts.
I have implemented a model using google pubsub where the producer sends in the message and the subscriber processes the message and sends the response to the subscription. But how do I map the response to the publisher which sent the request?
Are there any filters that can be put on the subscription so that the response can be tracked? or is there another way of implementing this?
There is no way in Cloud Pub/Sub for the publisher to know that the subscriber processed the message. One of the main goals with the pub/sub paradigm is to separate the publisher from the subscriber and having this kind of dependency tends to break that separation. Once the publish succeeds, then it knows that interested subscribers will receive the message.
If the publisher needs to know that the subscriber completed the processing of the message, then one way to accomplish this is to use a second Pub/Sub topic that sends those messages. The subscriber on the original topic becomes the publisher and the original publisher becomes the subscriber.
I have enabled the "notify-keyspace-events" for redis node, and getting the event published on key change on subscription.
But, I want to understand, what redis does with the events to be published if there are no subscribers to any key.
Any information or links, which could help me understand will be appreciated.
It is a fire and forget model. If there are no subscribers available, it will drop those events. It will even drop even if the subscriber is not available or will not be able to take those events.
Documentation from Redis:
https://redis.io/topics/notifications
Snippet from documentation,
Because Redis Pub/Sub is fire and forget currently there is no way to
use this feature if your application demands reliable notification of
events, that is, if your Pub/Sub client disconnects, and reconnects
later, all the events delivered during the time the client was
disconnected are lost.
i have wrote a Text Message Sender Program via JMS with C++ following.
tibems_status status = TIBEMS_OK;
status = tibemsMsgProducer_SendToDestination(
m_tProducer,
m_tDestination,
m_tMsg );
Suppose status == 0, this means only that Function has worked succesfull. It doesn't mean that my Text Message was sent succesfull
How can I ensure that my Message was sent succesfull? Should I get a ID or Acknowledge from JMS Queue back?
It depends on the Message Delivery Mode.
When a PERSISTENT message is sent, the tibemsMsgProducer_SendToDestination call will wait for the EMS server to reply with a confirmation.
When a NON_PERSISTENT message is sent, the tibemsMsgProducer_SendToDestination call may or may not wait for a confirmation depending on if authorization is enabled and the npsend_check_mode setting. See the EMS docs (linked above) for specific details.
Lastly, when a RELIABLE_DELIVERY message is sent, the tibemsMsgProducer_SendToDestination call does not wait for a confirmation and will only fail if the connection to the EMS server is lost.
However, even in the situations where a confirmation is sent, this is only confirmation that the EMS server has received the message. It does not confirm that the message was received and processed by the message consumer. EMS Monitoring Messages can be used to determine if the message was acknowledged by the consumer.
The message monitoring topics are in the form $sys.monitor.<D>.<E>.<destination>, where <D> matches Q|q|T|t, <E> matches s|r|a|p|\* and <destination> is the name of the destination. For instance to monitor for message acknowledgment for the queue named beterman, your program would subscribe to $sys.monitor.q.a.beterman (or $sys.monitor.Q.a.beterman if you want a copy of the message that was acknowledged).
The monitoring messages contain many properties, including the msg_id, source_name and target_name. You can use that information to correlate it back to the message you sent.
Otherwise, the simpler option is to use a tibemsMsgRequestor instead of a tibemsMsgProducer. tibemsMsgRequestor_Request will send the message and wait for a reply from the recipient. In this case you are best to use RELIABLE_DELIVERY and NO_ACKNOWLEDGE to remove all the confirmation and acknowledgement messages between the producer and the EMS server and the EMS server and the consumer.
However, if you do go down the tibemsMsgRequestor route, then you may also want to consider simply using a HTTP request instead, with a load balancer in place of the EMS server. Architecturally there isn't much difference between the two options (EMS uses persistent TCP connections, HTTP doesn't)
Producer -> EMS Server -> ConsumerA
-> ConsumerB
Client -> Load Balancer -> ServerA
-> ServerB
But with HTTP you have clear semantics for each of the methods. GET is safe (does not change state), PUT and DELETE are idempotent (multiple identical requests should have the same effect as a single request), and POST is non-idempotent (it causes a change in server state each time it is performed), etc. You also have well defined status codes. If you're using tibemsMsgRequestor you'll need to create bespoke semantics and response status, which will require extra effort to create, maintain and to train the other developers in your team on.
Also, it far easier to find developers with HTTP skills than EMS skills and it's far easier to find information HTTP that EMS, so the tibemsMsgRequestor option will make recruiting more difficult and problem solving issues more difficult.
Because of this HTTP is a better option IMO, for request-reply or for when you want to ensure that that the message sent was processed successfully.