I have a event with an Arraylist :
I have an Arraylist :
"TelephoneDetails": {
"Telephone": [
{
"Number": "<Number>",
"Type": "<Type>",
"Primary": "<Primary>",
"TextEnabled": "<TextEnabled>"
},{
"Number": "<Number>",
"Type": "<Type>",
"Primary": "<Primary>",
"TextEnabled": "<TextEnabled>"
}
]
}
how to write the InputTransformer for the InputPath for this ? ,
i can get the Telephone[0]using this
{
"Type": "$.detail.payload.TelephoneDetails.Telephone[0].Type",
"Number": "$.detail.payload.TelephoneDetails.Telephone[0].Number",
"Primary": "$.detail.payload.TelephoneDetails.Telephone[0].Primary",
"TextEnabled": "$.detail.payload.TelephoneDetails.Telephone[0].TextEnabled"
}
not understanding how to write, if I have ArrayList of N?
Is simple as that:
"TextEnabled": "$.detail.payload.TelephoneDetails.Telephone[*].TextEnabled"
please note the [*] instead of [0] to let the template engine iterates over the list of Telephones
I think you can't do this with plain EB syntax. Probably the best way would be to put a lambda function as target of your EB rule, make the transformations and then forward it to your target.
Related
I have the following iteration state defined in a Map State:
"WriteRteToDB": {
"Comment": "Write Rte to DB. Also records the risk calculations in the same table.",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"End": true,
"Parameters": {
"FunctionName": "logger-lambda",
"RtInfo.$": "States.Array($)",
"ExecutionId.$": "$$.Execution.Id",
"InitTime.$": "$$.Execution.StartTime"
}
The parameters defined produce the following input:
{
"FunctionName": "logger-lambda",
"RtInfo": {
"status": 200,
"rte": {
"date": "2022-06-05 00:00:00",
"rt_value": 778129128.6631782,
"lower_80": 0,
"upper_80": 0.5,
"location_id": "WeWork Office Space & Coworking, Town Square, Alpharetta, GA, USA",
"syndrome": "Gastrointestinal"
}
},
"InitTime": "2022-06-05T15:04:57.297Z",
"ExecutionId": "arn:aws:states:us-east-1:1xxxxxxxxxx1:execution:RadaRx-rteForecast:0dbf2743-abb5-e0b6-56d0-2cc82a24e3b4"
}
But the following Error is produced:
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'WriteRteToDB' (entered at the event id #28). The Parameters '{\"FunctionName\":\"logger-lambda\",\"RtInfo\":[{\"status\":200,\"rte\":{\"date\":\"2022-12-10 00:00:00\",\"rt_value\":1.3579795204795204,\"lower_80\":0,\"upper_80\":0.5,\"location_id\":\"Atlanta Tech Park, Technology Parkway, Peachtree Corners, GA, USA\",\"syndrome\":\"Influenza Like Illnesses\"}}],\"InitTime\":\"2022-06-05T16:06:10.132Z\",\"ExecutionId\":\"arn:aws:states:us-east-1:1xxxxxxxxxx1:execution:RadaRx-rteForecast:016a37f2-d01c-9bfd-dc3f-1288fb7c1af6\"}' could not be used to start the Task: [The field \"RtInfo\" is not supported by Step Functions]"
}
I have already tried wrapping the RtInfo inside an array of length 1 as you can observe from above, considering that it is a state within the Map State. I have also checked Input size to make sure that it does not cross the Max Input/Output quota of 256KB.
Your task's Parameters has incorrect syntax. Pass RtInfo and the other user-defined inputs under the Payload key:
"Parameters": {
"FunctionName": "logger-lambda",
"Payload": {
"RtInfo.$": "States.Array($)",
"ExecutionId.$": "$$.Execution.Id",
"InitTime.$": "$$.Execution.StartTime"
}
}
Within my EMR module I have a template that is deployed for the cluster configuration, within this template are all the cluster configuration requirements for the given classification type as specified in the variable emr_cluster_applications e.g. Spark, Hadoop, Hive.
Visual:
emr_cluster_applications = ["Spark", "Hadoop", "Hive"]
emr_cluster_configurations = file("./filepath/to/template.json")
This set up works fine however moving forward I'm wondering if the template can be populated based on the values within the emr_cluster_applications variable.
For example in a seperate deployment, if ["Spark", "Hadoop"] were specified as opposed to all three, then the template file would only use the corresponding Spark and Hadoop configuration with Hive being ignored although still present in the file - is this possible?
Update:
Template file:
[
{
"Classification": "spark",
"Properties":{
"maximizeResourceAllocation": "false",
"spark.executor.memoryOverhead": "4G"
}
},
{
"Classification": "hive",
"Properties":{
"javax.jdo.option.ConnectionURL": XXXX
"javax.jdo.option.ConnectionDriverName": XXXX
"javax.jdo.option.ConnectionUserName": XXXX
"javax.jdo.option.ConnectionPassword": XXXX
}
},
{
"Classification": "hbase-site",
"Properties": {
"hbase.rootdir": "XXXXXXXXXX"
}
},
{
"Classification": "hbase",
"Properties":{
"hbase.emr.storageMode": "s3"
"hbase.emr.readreplica.emnabled": "true"
}
}
]
This is the best I could come up with and there might be better solutions, so take this with a grain of salt. I have problems with mapping the Hadoop to two different elements from the JSON, so I had to do some modifications to the variables in order to make it work. I strongly suggest doing any variable manipulation within a locals block in order to avoid clutter in the resources. The locals.tf example:
locals {
emr_template = [
{
"Classification" : "spark",
"Properties" : {
"maximizeResourceAllocation" : "false",
"spark.executor.memoryOverhead" : "4G"
}
},
{
"Classification" : "hive",
"Properties" : {
"javax.jdo.option.ConnectionURL" : "XXXX",
"javax.jdo.option.ConnectionDriverName" : "XXXX",
"javax.jdo.option.ConnectionUserName" : "XXXX",
"javax.jdo.option.ConnectionPassword" : "XXXX"
}
},
{
"Classification" : "hbase-site",
"Properties" : {
"hbase.rootdir" : "XXXXXXXXXX"
}
},
{
"Classification" : "hbase",
"Properties" : {
"hbase.emr.storageMode" : "s3",
"hbase.emr.readreplica.emnabled" : "true"
}
}
]
emr_template_mapping = { for template in local.emr_template : template.Classification => template }
hadoop_enabled = false
hadoop = local.hadoop_enabled ? ["hbase", "hbase-site"] : []
apps_enabled = ["spark", "hive"]
emr_cluster_applications = concat(local.apps_enabled, local.hadoop)
}
You can manipulate which apps will be added with two options:
If the Hadoop is enabled, that means hbase and hbase-site need to be added to the list of the allowed apps. If it is not enabled, then the value of the hadoop variable will be an empty list.
In the apps_enabled local variable you decide which ones you want to enable, i.e., spark, hive, none, or both.
Finally, in the emr_cluster_applications local variable you would use concat to concatenate the two lists into one.
Then, to create a JSON file locally, you could use the local_file option:
resource "local_file" "emr_template_file" {
content = jsonencode([for app in local.emr_cluster_applications :
local.emr_template_mapping["${app}"] if contains(keys(local.emr_template_mapping), "${app}")
]
)
filename = "${path.root}/template.json"
}
The local_file will output a JSON encoded file which can be used where you need it. I am pretty sure there are better ways to do it, so maybe someone else will see this and give a better answer.
I've stuck the problem with defining a rule for matching my events.
Googled, tested.
Let's say, we've the following event which contains the object user in the array events:
{
"version": "0",
"...": "...",
"detail": {
"events": [
{
"user": {
"id": "5efdee60b48e7c1836078290"
}
}
]
}
}
Is there any way to match the user.id in an EventBus rule?
I've already tried to use the following rule which is not valid:
{
"detail": {
"events": [
{
"user": {
"id": [
"5efdee60b48e7c1836078290"
]
}
}
]
}
}
then,
{
"detail": {
"events[0]": {
"user": {
"id": [
"5efdee60b48e7c1836078290"
]
}
}
}
}
also no effect.
I don't want to give up, but I'm tired with it ;)
This pattern should work to match this event:
{
"detail": {
"events": {
"user": {
"id": [
"5efdee60b48e7c1836078290"
]
}
}
}
}
Today, EventBridge only supports matching simple values (string, integer, boolean, null) with an array. You can read more in the service documentation.
I did some playing around with your example but I can't make it work. Based on reading the Arrays in EventBridge Event Patterns I have to conclude that matching inside arrays with complex values is not possible.
The quote that seems to confirm this is "If the value in the event is an array, then the pattern matches if the intersection of the pattern array and the event array is non-empty."
And from the Event Patterns page "Match values are always in arrays." So if your pattern is an array and the value in the event is also an array (this is the example you gave), a "set" based intersection test is performed. Your pattern would have to match the entire array entry, not just a single field like you have in the example.
I'm hustling with regex, and trying to get the id's from this body.
But only the id´s in the members list, and not the id in the verified key. :)
To clarify, I'm using Regular Expression Extractor in JMeter
{
"id": "9c40ffca-0f1a-4f93-b068-1f6332707d02", //<--not this
"me": {
"id": "38a2b866-c8a9-424f-a5d4-93b379f080ce", //<--not this
"isMe": true,
"user": {
"verified": {
"id": "257e30f4-d001-47b3-9e7f-5772e591970b" //<--not this
}
}
},
"members": [
{
"id": "88a2b866-c8a9-424f-a5d4-93b379f780ce", //<--this
"isMe": true,
"user": {
"verified": {
"id": "223e30f4-d001-47b3-9e7f-5772e781970b" //<--not this
}
}
},
{
"id": "53cdc218-4784-4e55-a784-72e6a3ffa9bc", //<--this
"isMe": false,
"user": {
"unverified": {
"verification": "XYZ"
}
}
}
]
}
at the moment I have this regex :
("id": )("[\w-]+")
But as you can see here it returns every guid
Any ideas on how to go on?
Thanks in advance.
Since the input data type is JSON, it is recommended to use the JMeter's JSON Path Extractor Plugin.
Once you add it, use the
$.members[*].id
JSON path expression to match all id values of each members in the document that are the top nodes.
If you may have nested memebers, you may get them all using
$..members[*].id
You may test these expressions at https://jsonpath.com/, see a test:
I need to pass 2 arrays and 2 vars. with API to LAmbda function
I get everytime this:
{"message": "Could not parse request body into json: Unexpected character (\',\' (code 44)): expected a valid value (number, String, array, object, \'true\', \'false\' or \'null\')\n at [Source: [B#5a648099; line: 5, column: 11]"}
My template mapping:
{
"items":
[
#foreach($elem in $input.params('items').split(','))
{
"ids": $elem.ids,
"contents": $elem.contents
}#if($foreach.hasNext),#end
#end
],
"QueryID": $input.params('QueryID'),
"nR": $input.params('nR')
}
Try quoting your values:
{
"items": [
#foreach($elem in $input.params('items').split(','))
{
"ids": "$elem.ids",
"contents": "$elem.contents"
}#if($foreach.hasNext),#end
#end
],
"QueryID": "$input.params('QueryID')",
"nR": "$input.params('nR')"
}
This looks like you are trying to pass items in the "params" field. If you are passing in items, QueryID, and nR every time, just put $input.json('$') (only that, remove everything else, even the surrounding {}). If that doesn't work, refer to #dave-maple's answer