In an AWS Step Function, in a Choice step, we want to compare a result from an AWS Lambda function to a threshold given as a parameter using "NumericGreaterThan".
In our example, we compare a calculated from a lambda with a threshold given by the event.
I tried defining my step function in the following way:
{
"StartAt": "Check Enough Data",
"States": {
"Check Enough Data": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ID:function:FUNCTION:$LATEST",
"Next": "Validate Count",
"ResultPath": "$.count"
},
"Validate Count": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.count",
"NumericGreaterThan": "$.threshold",
"Next": "Succeed State"
}
],
"Default": "Wait 24 Hours"
},
"Wait 24 Hours": {
"Type": "Wait",
"Seconds": 86400,
"Next": "Check Enough Data"
},
"Succeed State": {
"Type": "Succeed"
}
}
}
but got an error Expected value of type: Integer, Float insted of String.
If I replace "$.threshold" with a hard-coded value (like 20), it works, but the value is not dynamic as I want.
The following input should cause the lambda to get to the Succeed State:
{
"country": "japan",
"threshold": 40
}
I know we can replace the Choice step with another Lambda function, but we do not want to do that from cost-effective issues.
Does anyone have an idea on how to solve the problem?
you can use 'NumericGreaterThanPath' operator as per the docs https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-choice-state.html
Within a Choice Rule, the value of Variable can be compared with another value from the state input by appending 'Path' to name of supported comparison operators.
NumericEqualsPath, NumericGreaterThanPath, NumericGreaterThanEqualsPath, etc.
The comparison operators need to have an integer after ":". It can't be an string.
A workaround is that "Variable": "$.count" change to "Variable": "$.count/$.threshold" so that you can have "NumericGreaterThan": 1.
In that case, you have count and threshold that define the Choice action.
Let me know if that fixes your problem
Precision: "Variable": "$.count" becomes "Variable": "$.ratio"
where ratio = count/threshold
Related
I am trying to pass output of 1st trigger as input to my lambda function. After code run i am not able to see my lambda function is called. Although I seen that Input of 2nd step function is taken as string as it is. It is not passing actual value.
please see below image. Input & Output for 2nd trigger
{
"Comment": "A description of my state machine",
"StartAt": "Pass",
"States": {
"Pass": {
"Type": "Pass",
"Result": {
"method": "GET",
"body": "SELECT * from TestTable"
},
"ResultPath": "$.latest",
"OutputPath": "$.latest",
"Next": "NextPass"
},
"NextPass": {
"Type": "Task",
"Resource": "arn:aws:lambda:<location>:<key>:function:<function-Name>",
"Parameters": {
"method": "$.latest.method",
"body": "$.latest.body"
},
"End": true
}
}
}
To access the data from the input, you need to add ".$" to the end of the Parameter names. This tells Step Functions that you're reading from a JSON path.
So your code should look like the following snippet below:
"Parameters": {
"method.$": "$.latest.method",
"body.$": "$.latest.body"
}
Hope that helps!
I am writing an AWS step function, and for one of the steps, I wish to call a lambda that accepts an array as one of the inputs. However, if I try to pass in a JsonPath into the array, I get
The value for the field 'arrayField.$' must be a STRING that contains a JSONPath but was an ARRAY
My step function definition:
{
"StartAt": "First",
"States": {
"First": {
"Type": "Pass",
"Parameters": {
"type": "person"
},
"ResultPath": "$.output",
"Next": "Second"
},
"Second": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
"Parameters": {
"regularParameter": "some string",
"arrayParameter.$": ["$.output.type"]
},
"Next": "Succeed"
},
"Succeed": {
"Type": "Succeed"
}
}
}
How can I use jsonPath inside the array?
Since a new release you could use the intrinsic function States.Array:
"arrayParameter.$": "States.Array($.output.type)"
https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
As #Seth Miller mentioned JsonPath resolution within arrays doesn't work unfortunately. If the amount of values to replace in the array is small and known there's a simple workaround (in my case I needed an array of size 1).
The steps are:
Initialise the array with the number of values you need;
Replace each value using "ResultPath": "$.path.to.array[n]";
Use "$.path.to.array" in your task.
Simple, working example:
{
"StartAt": "First",
"States": {
"First": {
"Type": "Pass",
"Parameters": {
"type": "person"
},
"ResultPath": "$.output",
"Next": "Initialise Array"
},
"Initialise Array": {
"Comment": "Add an entry for each value you intend to have in the final array, the values here don't matter.",
"Type": "Pass",
"Parameters": [
0
],
"ResultPath": "$.arrayParameter",
"Next": "Fill Array"
},
"Fill Array": {
"Comment": "Replace the first entry of array with parameter",
"Type": "Pass",
"InputPath": "$.output.type",
"ResultPath": "$.arrayParameter[0]",
"End": true
}
}
}
And to use the resulting array in your task example:
"Second": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
"Parameters": {
"regularParameter": "some string",
"arrayParameter.$": "$.arrayParameter"
},
"Next": "Succeed"
},
Another way to approach this is by using Parallel state that outputs an array of objects and then use jsonPath to convert it to a simple array:
{
"StartAt": "Parallel",
"States": {
"Parallel": {
"Type": "Parallel",
"Next": "Use Array",
"ResultPath": "$.items",
"Branches": [
{
"StartAt": "CreateArray",
"States": {
"CreateArray": {
"Type": "Pass",
"Parameters": {
"value": "your value"
},
"End": true
}
}
}
]
},
"Use Array": {
"Type": "Pass",
"Parameters": {
"items.$": "$.items[*].value"
},
"End": true
}
}
}
In this example, Parallel state outputs the following json:
{
"items": [
{
"value": "your value"
}
]
}
And "Use Array" state produces:
{
"items": [
"your value"
]
}
JSONPath inside parameters field need to be a string. So if you want to pass to lambda function a parameter called arrayParameter, you´ll need to make a jsonPath query that extract that array.
For example, if inside the key output is a key called outputArray with the array as its value.
Input JSON:
{
"pre": "sdigf",
"output": {
"result": 1,
"outputArray": ["test1","test2","test.."]
}
}
The parameter sintax:
"arrayParameter.$": "$.output.outputArray"
Reasonable advice
I ran into a use case for JsonPath resolution within arrays today and found (like you have) that the functionality does not exist today. I ended up deciding that doing the data massaging in code was simpler and cleaner. For example, you could create a small Lambda that takes in the object emitted by First and massages it to a format acceptable to Second and adds it to the output (WaterKnight mentions this solution in a comment to another question).
This assumes that you are, for some reason, unable to change the format of the input to that Lambda in Second (which would be the absolute shortest path here).
Unreasonable advice
That said, if you want a way to do this completely within Step Functions that is fairly gross, you can use the result of a Map state that executes Pass states. The output of the Map state is an array that aggregate the output of each constituent Pass state. These Pass states simply emit the value(s) you want in the final array using the Parameters attribute. An example Step Function definition follows. I did warn that it is gross and that I went a different way to solve the problem.
{
"StartAt": "First",
"Comment": "Please don't actually do this",
"States": {
"First": {
"Type": "Pass",
"Parameters": {
"type": "person"
},
"ResultPath": "$.output",
"Next": "Add Array"
},
"Add Array": {
"Comment": "A Map state needs some array to loop over in order to work. We will give it a dummy array. Add an entry for each value you intend to have in the final array. The values here don't matter.",
"Type": "Pass",
"Result": [
0
],
"ResultPath": "$.dummy",
"Next": "Mapper"
},
"Mapper": {
"Comment": "Add a Pass state with the appropriate Parameters for each field you want to map into the output array",
"Type": "Map",
"InputPath": "$",
"ItemsPath": "$.dummy",
"Parameters": {
"output.$": "$.output"
},
"Iterator": {
"StartAt": "Massage",
"States": {
"Massage": {
"Type": "Pass",
"Parameters": {
"type.$": "$.output.type"
},
"OutputPath": "$.type",
"End": true
}
}
},
"ResultPath": "$.output.typeArray",
"Next": "Second"
},
"Second": {
"Comment": "The Lambda in your example is replaced with Pass so that I could test this",
"Type": "Pass",
"Parameters": {
"regularParameter": "some string",
"arrayParameter.$": "$.output.typeArray"
},
"Next": "Succeed"
},
"Succeed": {
"Type": "Succeed"
}
}
}
As many answers correctly pointed out, it's not possible to do it exactly the way you need. But I would suggest another solution: an array of dictionaries. It's not exactly what you need, but is native and not hacky.
"Second": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
"Parameters": {
"regularParameter": "some string",
"arrayParameter": [{"type.$": "$.output.type"}]
},
"Next": "Succeed"
},
The result would be
{
"regularParameter": "some string",
"arrayParameter": [{"type": "SingleItemWrappedToAnArray"}]
}
I want to branch the flow of an AWS State Machine based on how many items are in an array. If the array has 0 items, I want to end the flow. If it has more than 0 items, I want to do some stuff.
For example, I want to do something like the following:
{
"StartAt": "IsBig",
"States": {
"IsBig": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.things.length",
"NumericGreaterThan": 0,
"Next": "Big"
}
],
"Default": "Small"
},
"Big": {
"Type": "Pass",
"Result": "1",
"End": true
},
"Small": {
"Type": "Pass",
"Result": "0",
"End": true
}
}
}
I'd then pass in the following on execution:
{ "things": [1, 2, 3] }
I'd want IsBig to then call Big and end.
Is there a way to do that in AWS states language?
If I can't, I'll just create a Lambda that gets the array's length. I am just curious.
The answer is "no". You can't run a function from the "Variable": "$.things.length" property.
The value in the Variable field is a.... variable. It's not an expression.
The docs don't show any expression evaluation syntax. So, long story short, you can't do what I was looking to do.
Let say part of my Step Function looks like next:
"ChoiceStateX": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.value",
"NumericEquals": 0,
"Next": "ValueIsZero"
}
],
"Default": "DefaultState"
},
"ValueIsZero": {
"Type" : "Task",
"Resource": "arn:aws:lambda:******:function:Zero",
"Next": "NextState"
},
"DefaultState": {
"Type" : "Task",
"Resource": "arn:aws:lambda:******:function:NotZero",
"Next": "NextState"
}
Let assume that input to this state is:
{
"value": 0,
"output1": object1,
"output2": object2,
}
My issue is that I have to pass output1 to ValueIsZero state and output2 to DefaultState. I know that it is possible to change InputPath in ValueIsZero and DefaultState states. But this way isn't acceptable for me because I am calling these states from some other states also.
I tried to modify ChoiceStateX state like next:
"ChoiceStateX": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.value",
"NumericEquals": 0,
"OutputPath": "$.output1",
"Next": "ValueIsZero"
}
],
"Default": "DefaultState"
}
I got next error in this case: Field OutputPath is not supported.
How is it possible to implement this functionality?
PS: In the current moment I am using 'proxy' states between ChoiceStateX and ValueIsZero/DefaultState where modifying the output.
I have checked:
Input and Output Processing
Choice
but haven't found a solution yet.
It looks like it isn't possible to specify different OutputPath for one state.
The solution with proxy states doesn't look graceful.
I have solved this issue in another way in the state before ChoiceStateX. I am setting instances of different types in output property and only route it on ChoiceStateX state.
My input of ChoiceStateX state looks like:
{
"value": value,
"output": value==0 ? object1 : object2
}
End final version of ChoiceStateX state:
"ChoiceStateX": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.value",
"NumericEquals": 0,
"Next": "ValueIsZero"
}
],
"OutputPath": "$.output",
"Default": "DefaultState"
}
It is still isn't perfect, because I implement the same logic in two places.
I need to develop a 'State Machine' using 'AWS Step Functions' that accomplishes the following:
Call a Lambda function that will connect to DynamoDb & retrieve a list of rows. (I know how to do this.)
For each row from previous step, I need to call another Lambda function until all rows are read.
How do I do step #2 above in AWS Step Functions? In other words, how do I iterate over the results from the previous step.
It's not pretty, but this can be accomplished out of the box / without an iterator lambda by using JSONPath's slice operator and a Sentinel value.
Here's an example state machine:
{
"Comment": "Example of how to iterate over an arrray of items in Step Functions",
"StartAt": "PrepareSentinel",
"States": {
"PrepareSentinel": {
"Comment": "First, prepare a temporary array-of-arrays, where the last value has a special SENTINEL value.",
"Type": "Pass",
"Result": [
[
],
[
"SENTINEL"
]
],
"ResultPath": "$.workList",
"Next": "GetRealWork"
},
"GetRealWork": {
"Comment": "Next, we'll populate the first array in the temporary array-of-arrays with our actual work. Change this from a Pass state to a Task/Activity that returns your real work.",
"Type": "Pass",
"Result": [
"this",
"stage",
"should",
"return",
"your",
"actual",
"work",
"array"
],
"ResultPath": "$.workList[0]",
"Next": "FlattenArrayOfArrays"
},
"FlattenArrayOfArrays": {
"Comment": "Now, flatten the temporary array-of-arrays into our real work list. The SENTINEL value will be at the end.",
"Type": "Pass",
"InputPath": "$.workList[*][*]",
"ResultPath": "$.workList",
"Next": "GetNextWorkItem"
},
"GetNextWorkItem": {
"Comment": "Extract the first work item from the workList into currentWorkItem.",
"Type": "Pass",
"InputPath": "$.workList[0]",
"ResultPath": "$.currentWorkItem",
"Next": "HasSentinelBeenReached"
},
"HasSentinelBeenReached": {
"Comment": "Check if the currentWorkItem is the SENTINEL. If so, we're done. Otherwise, do something.",
"Type": "Choice",
"Choices": [
{
"Variable": "$.currentWorkItem",
"StringEquals": "SENTINEL",
"Next": "Done"
}
],
"Default": "DoWork"
},
"DoWork": {
"Comment": "Do real work using the currentWorkItem. Change this to be an activity/task.",
"Type": "Pass",
"Next": "RemoveFirstWorkItem"
},
"RemoveFirstWorkItem": {
"Comment": "Use the slice operator to remove the first item from the list.",
"Type": "Pass",
"InputPath": "$.workList[1:]",
"ResultPath": "$.workList",
"Next": "GetNextWorkItem"
},
"Done": {
"Type": "Succeed"
}
}
}