I have a state in my step function where I need to make an API call and then wait for the callback. The API call will call a lambda function outside of the step function as the API has a registered callback link for the response and cannot be changed. The lambda function will receive the response and progress the step function by sending task success/failure.
I want to make the request in one state, move to another state and wait for the response. I can save the taskToken in the DB for the lambda function to send success/failure. Is there a way to retrieve the taskToken for the next state so I can do one write in the DB while sending the request. The lambda can retrieve the task token and send the response with success/failure directly to the state to progress the state machine.
The other options I can see is to poll the DB in the next state, move to the next state and write the taskToken in the DB. Any other solution?
Which is the best way to handle this scenario?
Thanks.
Related
I have a scenario where I am sending a message to a request queue which in turns calls a lambda.
That lambda performs some operation and write the data to a response queue(AWS SQS).
How do I retrieve the specific message from my response queue for which request was sent and pass on that to the next step.
Steps:
I have a Step function
At step 1, I am sending the input data to a request queue ( say ABCQueue)
This queue is trigger to a lambda
The lambda processes, and writes data to response queue( say XYZQueue)
In my step function next step is to receive the message from response queue
How do I maintain the sequence here i.e the data I sent that only should I receive from response queue?
Note: Since there would be multiple requests coming in and each request would be getting completed at different time.
Is there any unique id by which I can check the id which I sent is only received back from response queue?
NOTE: I am integrating this with java
I have tried to use task token while using wait for callback.
But that task token expires once I send success from lambda
And when it goes to next state i.e. receive message the task token is not there and I can't get corresponding message
I have a simple problem. I want to create some kind of pipeline. First we have a function that does a request to some external resource (that answer takes a long time - sometimes is more than 60 seconds!). So first function just does a request and go down. And second function handles response of this function.
I currently try to run one lambda which do request, and second which handle response of this request.
I found a repo that has implementation of lambda messaging which is close to my idea https://github.com/pj0616/serverless-async-lambda-api. In this approach we dont wait for the results.
I was trying to run code from here too: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html, but it doesnt wait asynchronously for response.
Is it possible to create this flow? Maybe with lambda or step functions?
Below I put a diagram that shows what I'm trying to do.
I'm trying to build a basic AWS Lambda API and function setup to do the following:
Part 1: Client calls function with api and runs both a background 1 min function to process data and a quick messesge to client in browser.
Part 2: When background function is complete it returns 302 redirect to the client with a generated link.
I'm stuck on Part 2. How can I go from the background function to the API back to the client?
I'm using python boto3 for my Lambda scripts.
This is AWS Lambda so your client doesn't have a persistent connection to the server-side code.
Here is an idea of one way to build this:
your client makes an API request that triggers a Lambda function
on invocation, your Lambda function generates a new, unique id (a UUID), writes that to DynamoDB so that this UUID can later be associated with the result of the background processing
the Lambda kicks off the background processing, passing the UUID to it
the Lambda returns the generated UUID to the client
the background processing happens asynchronously, ultimately writing any results to the DynamoDB item associated with the UUID that triggered it
the client polls another API periodically, say every 10s, sending in the UUID it was given
the polled Lambda takes the presented UUID, does a lookup in DynamoDB and returns a 302 redirect to a URL result, or an indication that the results aren't ready yet (e.g. HTTP 404)
some process that you create removes the item from DynamoDB later (or not)
I have a requirement to record the details of calls initiated by RingOut upon completion of the call. I'm successfully polling the Ring Out Status until the call is finished, but I can't find a way to then query for the User Call Record for the completed RingOut. Is there a way to get the Session Id for a RingOut call which can be used to retrieve the User Call Record and/or query for a User Call Record using a RingOut Id? Better yet, can I create a subscription which will notify my service when a RingOut call is completed?
Instead of using the Polling, try using the push notifications to subscribe for the detailed extension presence event. Within the notification payload, you will be able to detect detect the status of the call. More information can be found here https://developers.ringcentral.com/api-reference/Detailed-Extension-Presence-Event
Hope this help!
ringcentral
I have a function to give recommendations to users. This function need to make a lot of calcs to start, but after start it use the already calculed matrix on memory. After this, any other calc that is made, "fills" the object in memory to continuous learning.
My intention is to use this function to website users, but the response need to come from the same "object" in memory and need to be sequential by request because it is not thread safe.
How is the best way to get this working? My first idea was use signalr so the user dont need to wait to response and a queue to send the requests to objects. But how the signalr can receive the response for this specific request?
The entire flow is:
User enter on a page.
A javascript will call a service with the user ID and actual page.
The server will queue the ID an page.
The service will be calculating the results for each request on queue and sending responses.
The server will "receive" the response and send back to client.
The main problem is that I dont see a way to the service receive the response to send back to client until it is complete, without need to be looping in queues.
Thanks!
If you are going to use SignalR, I would suggest using a hub method to accept these potentially long running requests from the client. By doing so it should be obvious "how the signalr can receive the response for this specific request".
You should be able to queue your calculations from inside your hub method where you will have access to the caller's connection id (via the Context.ConnectionId property).
If you can await the results of your queued operation inside of the hub method you queue from, you can then simply return the result from your hub method and SignalR will flow the result back to the calling JavaScript. You can also use Clients.Caller.... to send the result back.
If you go this route I suggest you use async/await instead of blocking request threads waiting for your long-running calculations to complete.
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server
If you can't process your calculation results from the same method you queued the calculation from, you still have options. Just be sure to queue the caller's connection id and a request id along with the calculation to be processed.
Then, you can process the results of all your calculations from outside of your hub using GlobalHost.ConnectionManager.GetHubContext:
private IHubContext _context = GlobalHost.ConnectionManager.GetHubContext<MyHub>()
// Call ProcessResults whenever results are ready to send back to the client
public void ProcessResults(string connectionId, uint requestId, MyResult result)
{
// Presumably there's JS code mapping request id's to results
// if you can have multiple ongoing requests per client
_context.Clients.Client(connectionId).receiveResult(requestId, result);
}
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server#callfromoutsidehub