I am executing a Java lambda in a step function. I am throwing any exceptions in lambda code as RuntimeExceptions. I am hoping to retry the lambda execution via below code on getting any Runtime exception(since https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html says any unhandled lambda errors come up as Lambda.Unknown). However, this does not retry lambda execution on failure.:
"STATE_NAME": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"OutputPath": "$.Payload",
"Parameters": {
"FunctionName": "arn:aws:lambda:*:$LATEST",
"Payload": {
...
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.Unknown"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
What does work though is if i replace Error condition with States.ALL. However this also would include invalid permissions, state timeouts etc. on which i do Not want to Retry the lambda execution. Is there something i am missing here?
Based on AWS doc (https://docs.aws.amazon.com/step-functions/latest/dg/bp-lambda-serviceexception.html)
Unhandled errors in Lambda are reported as Lambda.Unknown in the error output. These include out-of-memory errors and function timeouts. You can match on Lambda.Unknown, States.ALL, or States.TaskFailed to handle these errors. When Lambda hits the maximum number of invocations, the error is Lambda.TooManyRequestsException. For more information about Lambda Handled and Unhandled errors, see FunctionError in the AWS Lambda Developer Guide.
If you are throwing exception in lambda code, then it will not be classified as Lambda Unhandled errors based on that clasifications.
Related
We have step function code, and when I trigger an API gateway for the first time, I get a 3 second timeout error. As a result, I used "lambda.Timeout" in retry logic, but now it's not giving an error, but when I trigger it for the first time, I get a blank screen, and when I run it again, I get the response. Please suggest me some idea what should i do?
"Timeoutseconds": 3,
"Retry": [
{
"ErrorEquals":[
"Lambda.Timeout"
],
"Intervalseconds": 3,
"MaxAttempts":2,
"BackofRate":2
}
],
Im trying to do the following at AWS Step Functions:
IF ExampleState fails, do "Next":"Anotherlambda"
IF ExampleState completes successfull, end the execution.
How can i do that? The CHOICE state doesn't support ErrorEquals: States.TaskFailed.
In my case, when ExampleState fails, State Machine STOPS and gives me error, but i want to continue and catch some info from the error and save it with another lambda
Thanks!
All i wanted AWS Step Functions to do is, if a State succeeded, finish the execution, if fails, run another lambda. Like an IF / ELSE on programming.
Step Functions gives this easy to you as a CATCH block that only activates if catches an error and does what you want. Here the solution:
"StartAt": "ExampleLambda",
"States": {
"ExampleLambda": {
"Type": "Task",
"Resource": "xxx:function:ExampleLambda",
"Catch": [
{
"ErrorEquals":["States.TaskFailed"],
"Next": "SendToErrorQueue"
}
],
"End": true
}
We are creating a workflow composed of multiple SQL Operations(Aggregations, Transposes etc.) via AWS Step functions. Every operation is modelled as a separate Lambda which houses the SQL query.
Now, every query accepts its input parameters from the state machine, so every lambda task is as below:
"SQLQueryTask": {
"Type": "Task",
"Parameters": {
"param1.$": "$$.Execution.Input.param1",
"param2.$": "$$.Execution.Input.param2"
},
"Resource": "LambdaArn",
"End": true
}
The Parameters block thus repeats for every SQLQuery node.
Added to this since Lambdas can fail intermittently and we would like to retry for them ; we also need to have below retry block in every State:
"Retry": [ {
"ErrorEquals": [ "Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException"],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
} ]
This is making the state definition very complex. Is there No way to extract out the common part of state definition to a reusable piece?
One solution could be using AWS CDK (https://aws.amazon.com/cdk/)
This allows developers to define higher-level abstractions of resources, which can easily be reused.
There are some example here that could be helpful: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-stepfunctions-readme.html
I am using boto3 in my activity workers and I came upon a TaskTimedOut when calling the SendTaskFailure:
botocore.errorfactory.TaskTimedOut: An error occurred (TaskTimedOut) when calling the SendTaskFailure operation: Task Timed Out: 'arn:aws:states:eu-west-2:statemachinearn:activityname'
I think this happens because the connection pool gets full sometimes which makes the request not being fulfilled (even though a new connection is created).
I know it is possible to set a timeout value for Tasks and Parallel States but that does not have anything to do with calling the send_task_failure/send_task_success methods.
Does anyone have any idea on how to solve this?
I can explain one scenario which I encountered in step function.
I was using nested state machines and in my main state machine, I used to get exception like 'timeout when invoking state machine', one in 1000s. As it was transient error, i had to handle it as per specs provided by AWS so I explicitly added retry for framework exceptions like below.
"Retry": [
{
"ErrorEquals": [
"StepFunctions.SdkClientException"
],
"IntervalSeconds": 10,
"MaxAttempts": 4,
"BackoffRate": 2
}
]
I have created a simple AWS state machine with lambda functions. Like below
{
"Comment":"Validates data",
"StartAt": "ChooseDocumentType",
"States": {
"ChooseDocumentType": {
"Type": "Choice",
"Choices":[
{
"Variable":"$.documentType",
"StringEquals":"RETURN",
"Next":"ValidateReturn"
},
{
"Variable":"$.documentType",
"StringEquals":"ASSESSMENT",
"Next":"ValidateAssessment"
}
],
"Default":"DefaultState"
},
"ValidateReturn":{
"Type":"Task",
"Resource":"arn:aws:lambda:us-west-2:111111111:function:ValidateReturn",
"Next":"DefaultState"
},
"ValidateAssessment":{
"Type":"Task",
"Resource":"arn:aws:lambda:us-west-2:111111111:function:ValidateAssessment",
"Next":"DefaultState"
},
"DefaultState":{
"Type":"Pass",
"End":true
}
}
}
Questions
1> How do i create stages for this state machine. (like production, development etc)?
2>Each lambda function has alias pointing to different versions. So development alias always point to $latest version and production alias point to, lets say, version 2. How do i dynamically associate state machine's stages with these lambda alias? So state machine in development stage should use lambda function with alias development and so on.
I am using AWS console to manage state machines and lambdas, and i don't see any action to create stages for state machine
You can declare the alias and the version in the Lambda ARN:
# default, $LATEST
arn:aws:lambda:us-west-2:111111111:function:ValidateAssessment
# using alias
arn:aws:lambda:us-west-2:111111111:function:ValidateAssessment:development
# using version
arn:aws:lambda:us-west-2:111111111:function:ValidateAssessment:2
Use these in the Step Function definition according to your needs.
Re: # 2, if your main concern is controlling which Lambda alias gets invoked, there is a way you can do that via a single step function.
Your step function state definition would be something like:
{
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"InvocationType": "RequestResponse",
"FunctionName": "someFunction",
"Qualifier.$": "$.lambdaAlias",
"Payload": {}
},
}
So where you execute the step function and would specify the stage if there was such a thing, you'd pass a lambdaAlias parameter. (There's nothing magical about that name, you can pull it from whatever step function input parameter you want.)
The request payload to your Lambda would go in Parameters.Payload.
https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html