Here is what I need to accomplish:
Get some messages in System 1 using a Pull Subscription.
Send each message along with acknowledgement id to System 2
Send acknowledgement to the subscription from that System 2.
So, basically I will create a new Pubsub client and send acknowledgement. How can I make this request?
To complement your answer, I just realized that your client is a Pubsub java class and the method execute() belongs to an instance of Acknowledge.
I found a full example that pulls messages from Pubsub. In fact, the pullMessages() method in such example has the sentence you mentioned. Digging into the java framework, this link mentions how the method exetute() is used.
I added the execute() method to the acknowledgment operation:
client.projects().subscriptions()
.acknowledge(getFullSubscriptionsName(config), acknowledgeRequest)
.execute();
The method execute() was not being invoked.
Related
Problem: My use case is I want to publish thousends of messages to Google Cloud Pub/Sub with a 5min retention period but only retrieve specific messages by their ID - So a cloud function will retrieve one message by ID using the Nodejs SDK and all the untreated messages will be deleted by the retention policy. All the current examples mention are to handle random messages from the subscriber.
Is it possible to just pull 1 message by id or any other metadata and close the connection.
There is no way to retrieve individual messages by ID, no. It doesn't really fit into the expected use cases for Cloud Pub/Sub where the publishers and subscribers are meant to be decoupled, meaning the subscriber inherently doesn't know the message IDs prior to receiving the messages.
You may instead want to transmit the messages via whatever mechanism you are using to making the subscribers aware of the message IDs. Or, if you know at publish time which messages will ultimately need to be retrieved, you could add an attribute to the message to indicate this and use filtering.
I have a Lambda function that's triggered by Amazon MQ running ActiveMQ.
I would like my Lambda function to acknowledge receipt of the message once it has been processed so that ActiveMQ can remove it from the queue or topic.
I can't find any relevant documentation on AWS that tells me how I'm supposed to send that acknowledgement back, or how ActiveMQ will know when a consumer has successfully processed a message.
Any ideas?
If Lambda is using the OpenWire JMS client as noted in their documentation then it must be using one of the acknowledgement modes provided by JMS (i.e. AUTO_ACKNOWLEDGE, CLIENT_ACKNOWLEDGE, or DUPS_OK_ACKNOWLEDGE). However, I don't see any indication in the documentation which mode they're using and any way it can be changed. Therefore my guess is that they're using AUTO_ACKNOWLEDGE since that's usually what integration frameworks, etc. use by default. If that's true then there's nothing your Lambda function needs to do. The message will be acknowledged automatically by the OpenWire JMS client.
Is it possible to ensure that a message was successfully delivered to an Event Hub when sending it with the log-to-eventhub policy in API Management?
Edit: In our solution we cannot allow any request to proceed if a message was not delivered to the Event Hub. As far as I can tell the log-to-eventhub policy doesn't check for this.
Welcome to Stackoveflow!
Note: Once the data has been passed to an Event Hub, it is persisted and will wait for Event Hub consumers to process it. The Event Hub does not care how it is processed; it just cares about making sure the message will be successfully delivered.
For more details, refer “Why send to an Azure Event Hub?”.
Hope this helps.
Event Hubs is built on top of Service Bus. According to the Service Bus documentation,
Using any of the supported Service Bus API clients, send operations into Service Bus are always explicitly settled, meaning that the API operation waits for an acceptance result from Service Bus to arrive, and then completes the send operation.
If the message is rejected by Service Bus, the rejection contains an error indicator and text with a "tracking-id" inside of it. The rejection also includes information about whether the operation can be retried with any expectation of success. In the client, this information is turned into an exception and raised to the caller of the send operation. If the message has been accepted, the operation silently completes.
When using the AMQP protocol, which is the exclusive protocol for the .NET Standard client and the Java client and which is an option for the .NET Framework client, message transfers and settlements are pipelined and completely asynchronous, and it is recommended that you use the asynchronous programming model API variants.
A sender can put several messages on the wire in rapid succession without having to wait for each message to be acknowledged, as would otherwise be the case with the SBMP protocol or with HTTP 1.1. Those asynchronous send operations complete as the respective messages are accepted and stored, on partitioned entities or when send operation to different entities overlap. The completions might also occur out of the original send order.
I think this means the SDK is getting a receipt for each message.
This theory is further aided by the RetryPolicy Class used in the ClientEntity.RetryPolicy Property of the EventHubSender Class.
In the API Management section on logging-to-eventhub, there is also a section on retry intervals. Below that are sections on modifying the return response or taking action on certain status codes.
Once the status codes of a failed logging attempt are known, you can modify the policies to take action on failed logging attempts.
I have implemented a model using google pubsub where the producer sends in the message and the subscriber processes the message and sends the response to the subscription. But how do I map the response to the publisher which sent the request?
Are there any filters that can be put on the subscription so that the response can be tracked? or is there another way of implementing this?
There is no way in Cloud Pub/Sub for the publisher to know that the subscriber processed the message. One of the main goals with the pub/sub paradigm is to separate the publisher from the subscriber and having this kind of dependency tends to break that separation. Once the publish succeeds, then it knows that interested subscribers will receive the message.
If the publisher needs to know that the subscriber completed the processing of the message, then one way to accomplish this is to use a second Pub/Sub topic that sends those messages. The subscriber on the original topic becomes the publisher and the original publisher becomes the subscriber.
I've created a lex sms chatbot and published it to a Twilio SMS Channel. Within my java lambda fulfillment handleRequest function I receive 2 parameters: an Input object and a Context. Input has some type of system generated userId, but I need the phone number of the incoming sms message from Twilio.
I've configured Twilio to call Lex via webhook per these instructions:
http://docs.aws.amazon.com/lex/latest/dg/twilio-bot-association.html#twilio-bot-assoc-create-assoc
I'm using the system created webhook callback url between Lex and Twilio and would like to avoid writing my own Lex-Twilio go between with Gateway API and lambda if possible.
I think I'm missing a configuration in Twilio to send this to Lex maybe? I haven't setup any TwilML app or anything, just linked the SMS number to my webhook callback url for lex. Everything works fine except getting the incoming sms number.
UPDATE:
I exhausted all possibilities with my input object and context in my java lambda function. I guess there's no way to get at the payload from twilio in Java. I tried switching to an input stream handler but still only had the input format defined in the documentation here: http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
I had to bite the bullet and build my own gateway API and Node handler. It's just a layer that sits before lex and translates twilio into lex input format and vis versa on the response. It updates the userId with the incoming phone number.
This tutorial was very useful in doing this:
https://aws.amazon.com/blogs/ai/integrate-your-amazon-lex-bot-with-any-messaging-service/
It was a lot of work just to get one additional field, but It does provide me the flexibility to switch SMS providers in the future easier.
I am not sure as what exactly stopped you from getting the user phone number in your lambda function. You can always get sender phone number from 'userId' field from the input event in your lambda function. You can refer to the documentation to get more information.