Does anyone know the current and correct way to invoke Amazon AWS Lambda functions asynchronously instead of synchronously?
The InvokeAsync API in the AWS Java SDK is still available but marked as deprecated and they suggest you use use the Invoke API. I can't figure out why they would be forcing us to using sync. I have a web frontend that dispatches some batch jobs. I can't expect the frontend to keep a connection open for several minutes while it waits for the response (which is actually e-mailed to them after about 4-5 minutes of processing).
Ideally I'm trying to figure out how to do this with their API Endpoints rather than the Java SDK because the environment (GAE) that I'm running my backend in doesn't support AWS's use of HttpClient.
I'm looking at the latest API docs here, and it looks like only AWSLambdaAsyncClient.invokeAsyncAsync() is deprecated. The AWSLambdaAsyncClient.invokeAsync() method is not marked as deprecated. It looks like they are just doing some code cleanup by removing the need for the InvokeAsyncRequest and InvokeAsyncResult classes and the extra invokeAsyncAsync() methods.
You should be able to use the AWSLambdaAsyncClient.invokeAsync() method which uses InvokeRequest and returns InvokeResult. You might have to set the InvocationType on the InvokeRequest to InvocationType.Event. It's not clear if that's needed if you are using the Async client.
Regarding your second question about calling Lambda functions asynchronously without using the SDK, I would look into using API Gateway as a service proxy. This is the recommended way to expose Lambda functions for asynchronous calls.
The below code can be used to invoke the Lambda asynchronously from another Lambda
AWSLambdaAsyncClient client = new AWSLambdaAsyncClient();
client.withRegion(Regions.fromName(region));
InvokeRequest request = new InvokeRequest();
request.setInvocationType("Event");
request.withFunctionName(functionName).withPayload(payload);
InvokeResult invoke = client.invoke(request);
Approach given in the accepted answer is now deprecated. Answer given by the user #dassum is the approach to be followed, but the answer lacks a bit of explanation.
When creating the InvokeRequest, set the InvocationType as "Event" for asynchronous invocation and "RequestResponse" for synchronous invocation.
AWSLambda lambda = //create your lambda client here
lambda.invoke(new InvokeRequest()
.withFunctionName(LAMBDA_FUNCTION_NAME)
.withInvocationType(InvocationType.Event) //asynchronous
.withPayload(payload))
Reference to docs:
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/lambda/AWSLambda.html#invoke-com.amazonaws.services.lambda.model.InvokeRequest-
You can try the following:
YourCustomRequestBean request = new YourCustomRequestBean();
request.setData1(data1);
request.setData2(data2);
request.setData3(data3);
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setSocketTimeout(xxxx);
clientConfiguration.setRequestTimeout(xxxx);
ObjectMapper objectMapper = new ObjectMapper();
String jsonStr = objectMapper.writeValueAsString(request);
AWSLambdaAsync awsLambdaAsync =
AWSLambdaAsyncClientBuilder.standard().withRegion(<mention your region here>).withClientConfiguration(clientConfiguration).build();
InvokeRequest invokeRequest = new InvokeRequest()
.withFunctionName(lambda function urn)
.withPayload(jsonStr);
awsLambdaAsync.invokeAsync(invokeRequest);
Related
I am trying to invoke a lambda function by the new feature - function URL"S https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html using which I don't require any trigger events.
I am able to get the output for a simple java application using the above approach written in documentation by using Java as runtimeEnvironment and passing Jar file from my local machine.
Now, I want to test lambda functionality by same concept but for Springboot-Application which has restful endpoints for GET,POST,PUT,DELETE for different crud operations connecting with MySql db. The problem I am facing is how to tell handler class that to route to these endpoints and trigger lambda functions.
Any help/suggestions appreciated here. Thanks in Advance !
It probably won't work with this URL's feature but it allows handling Lambda events similar to standard Spring web requests.
You can check this one: https://github.com/MelonProjectCom/lambda-http-router
Example usage:
#PathPostMapping("/example/test")
public String example(#Body String body) {
return "Hello World! - body: " + body;
}
I have a Lambda that requires messages to be sent to another Lambda to perform some action. In my particular case it is passing a message to a Lambda in order for it to perform HTTP requests and refresh cache entries.
Currently I am relying on the AWS SDK to send an SQS message. The mechanics of this are working fine. The concern that I have is that the SQS send method call takes around 50ms on average to complete. Considering I'm in a Lambda, I am unable to perform this in the background and expect for it to complete before the Lambda returns and is frozen.
This is further compounded if I need to make multiple SQS send calls, which is particularly bad as the Lambda is responsible for responding to low-latency HTTP requests.
Are there any alternatives in AWS for communicating between Lambdas that does not require a synchronous API call, and that exhibits more of a fire and forget and asynchronous behavior?
Though there are several approaches to trigger one lambda from another, (in my experience) one of the fastest methods would be to directly trigger the ultimate lambda's ARN.
Did you try invoking one Lambda from the other using AWS SDKs?
(for e.g. in Python using Boto3, I achieved it like this).
See below, the parameter InvocationType = 'Event' helps in invoking target Lambda asynchronously.
Below code takes 2 parameters (name, which can be either your target Lambda function's name or its ARN, params is a JSON object with input parameters you would want to pass as input). Try it out!
import boto3, json
def invoke_lambda(name, params):
lambda_client = boto3.client('lambda')
params_bytes = json.dumps(params).encode()
try:
response = lambda_client.invoke(FunctionName = name,
InvocationType = 'Event',
LogType = 'Tail',
Payload = params_bytes)
except ClientError as e:
print(e)
return None
return response
Hope it helps!
For more, refer to Lambda's Invoke Event on Boto3 docs.
Alternatively, you can use Lambda's Async Invoke as well.
It's difficult to give exact answers without knowing what language are you writing the Lambda function in. To at least make "warm" function invocations faster I would make sure you are creating the SQS client outside of the Lambda event handler so it can reuse the connection. The AWS SDK should use an HTTP connection pool so it doesn't have to re-establish a connection and go through the SSL handshake and all that every time you make an SQS request, as long as you reuse the SQS client.
If that's still not fast enough, I would have the Lambda function handling the HTTP request pass off the "background" work to another Lambda function, via an asynchronous call. Then the first Lambda function can return an HTTP response, while the second Lambda function continues to do work.
You might also try to use Lambda Destinations depending on you use case. With this you don't need to put things in a queue manually.
https://aws.amazon.com/blogs/compute/introducing-aws-lambda-destinations/
But it limits your flexibility. From my point of view chaining lambdas directly is an antipattern and if you would need that, go for step functions
My Lex bot has four intents. Suppose a user asks a question at the very beginning of the conversation and this question is not allotted to any of the four intents. Hence no intent will be established. When this happens, I want to call lambda to run an "intent suggestion model" (built using topic modeling) to suggest the user about what the intent of the question might be. Also, lambda will have to store such queries in a database (s3 or RDB) so that if such queries are repetitive, then that intent can eventually be added to the bot and for other analytical solutions.
What you need is a fallback intent but Lex does not support fallback intents as of now.
You can still achieve this if you use a bridge between your chat client and Lex.
Setup an API Gateway and Lambda function in between your chat-client and the Lex.
Your chat-client will send a request to API Gateway, API Gateway will forward this to Lambda function which will be used to call Lex and get response from it. Lex will have one more lambda function as a webhook.
In the Lambda function you used to call Lex, we can check if any intent was matched or we got an error message, if it's an error message and trigger some action like intent suggestion model.
You need to use boto library to call Lex and use post_text() method.
Hope it helps.
I followed tutorial on creating and invoking step functions
I'm getting output in my GET request of api as
{
"executionArn": "arn:aws:states:ap-northeast-1:123456789012:execution:HelloWorld:MyExecution",
"startDate": 1.486772644911E9
}
But, instead of above response I want my step functions output, which is given by end state as below.
{
"name":"Hellow World"
}
How to achieve this?
Update: You can now use Express Step Functions for synchronous requests.
AWS Step Functions are asynchronous and do not immediately return their results. API Gateway methods are synchronous and have a maximum timeout of 29 seconds.
To get the function output from a Step Function, you have to add a second method in API Gateway which will call the Step Function with the DescribeExecution action. The API Gateway client will have to call this periodically (poll) until the returned status is no longer "RUNNING".
Here's the DescribeExecution documentation
Use Express Step Functions instead. This type of Step Functions can be called synchronously. Go to your API Gateway and in the Integration Request section make sure you have the StartSyncExecution action:
After that, go a bit lower in the same page to the Mapping Templates:
and include the following template for the application/json Content-Type:
#set($input = $input.json('$'))
{
"input": "$util.escapeJavaScript($input)",
"stateMachineArn": "arn:aws:states:us-east-1:your_aws_account_id:stateMachine:your_step_machine_name"
}
After that, go back to the Method Execution and go to the Integration Response and then to the Mapping Templates section:
And use the following template to have a custom response from your lambda:
#set ($parsedPayload = $util.parseJson($input.json('$.output')))
$parsedPayload
My testing Step Function is like this:
And my Lambda Function code is:
Deploy your API Gateway stage.
Now, if you go to Postman and send a POST request with any json body, now you have a response like this:
New Synchronous Express Workflows for AWS Step Functions is the answer:
https://aws.amazon.com/blogs/compute/new-synchronous-express-workflows-for-aws-step-functions/
Amazon API Gateway now supports integration with Step Functions StartSyncExecution for HTTP APIs:
https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-api-gateway-supports-integration-with-step-functions-startsyncexecution-http-apis/
First of all the step functions executes asynchronously and API Gateway is only capable of invoking the step function (Starting a flow) only.
If you are waiting for the results of a step function invocation from a web application, you can use AWS IOT WebSockets for this. The steps are as follows.
Setup AWS IOT topic with WebSockets.
Configure the API Gateway and Step functions invocation.
From the Web Frontend subscribe to the IOT Topic as a WebSocket listener.
At the last step (And in error steps) in the Step Functions workflow use AWS SDK to trigger the IOT Topic which will broadcast the results to the Web App running in the browser using WebSockets.
For more details on WebSockets with AWS IOT refer the medium article Receiving AWS IoT messages in your browser using websockets.
Expanding on what #MikeD at AWS says, if you're certain that the Step Function won't exceed the 30 second timeout, you could create a lambda that executes the step function and then blocks as it polls for the result. Once it has the result, it can return it.
It is a better idea to have the first call return immediately with the execution id, and then pass that id into a second call to retrieve the result, once it's finished.
My Amazon Lambda function (in Python) is called when an object 123456 is created in S3's input_bucket, do a transformation in the object and saves it in output_bucket.
I would like to notify my main application if the request was successful or unsuccessful. For example, a POST http://myapp.com/successful/123456 if the processing is successful and http://myapp.com/unsuccessful/123456 if its not.
One solution I thought is to create a second Amazon Lambda function that is triggered by a put event in output_bucket, and it to do the successful POST request. This solves half of the problem because but I can't trigger the unsuccessful POST request.
Maybe AWS has a more elegant solution using a parameter in Lambda or a service that deals with these types of notifications. Any advice or point in the right direction will be greatly appreciated.
Few possible solutions which I see as elegant
Using SNS Topic: From your transformation lambda, trigger a SNS topic, with success/unsuccess message, where SNS will call a HTTP/HTTPS endpoint with message payload. The advantage here is, your transformation lambda is loosely coupled with endpoint trigger and only connected through messaging.
Using Lambda Step Functions:
You could arrange to run a Lambda function every time a new object is uploaded to an S3 bucket. This function can then kick off a state machine execution by calling StartExecution. The advantage in using step functions is that you can coordinate the components of your application as series of steps in a visual workflow.
I don't think there is any elegant AWS solution, unless you re-architect, something like your lambda sends message to SQS or some intermediatery messaging service with STATUS and then interemdeiatery invokes POST to your application.
If you still want to go with your way of solving, you might need to configure "DeadLetter queue" to do error handling in failure cases (note that use cases described here are not comprehensive, so need to make sure it covers your case) like described here.