Whats the best way to add a field to the last occurrence object in an array with Ramda? - list

Let's say I have a simple array of objects, that all have a type field :
let arr = [
{
"name": "First",
"type": "test"
},
{
"name": "Second",
"type": "test"
},
{
"name": "Third",
"type": "test2"
},
{
"name": "Fourth",
"type": "test2"
},
{
"name": "Fifth",
"type": "test3"
},
{
"name": "Sixth",
"type": "test3"
}
]
Using Ramda what is the best way to add a field to only the last occurrence of each types?
to get :
let newArr = [
{
"name": "First",
"type": "test"
},
{
"name": "Second",
"type": "test",
"last": true
},
{
"name": "Third",
"type": "test2"
},
{
"name": "Fourth",
"type": "test2",
"last": true
},
{
"name": "Fifth",
"type": "test3"
},
{
"name": "Sixth",
"type": "test3",
"last": true
}
]
I can't really wrap my head around it! Thanks in advance! :)

Here's one possible solution:
// f :: [{ type :: a }] -> [{ type :: a, last :: Boolean }]
const f = R.addIndex(R.map)((x, idx, xs) =>
R.assoc('last',
R.none(R.propEq('type', x.type), R.drop(idx + 1, xs)),
x));
For each value in the list we look ahead to see whether there is a subsequent value with the same type property.

I'm making the guess that your data is grouped as displayed and that elements of different types are not interspersed. If that guess is wrong, there would need to be a different solution.
My version involves two helper functions, one which groups a list according to a predicate which reports whether two (consecutive) values belong together:
const breakWhen = R.curry(
(pred, list) => R.addIndex(R.reduce)((acc, el, idx, els) => {
if (idx === 0 || !pred(els[idx - 1], el)) {
acc.push([el])
} else {
acc[acc.length - 1].push(el);
}
return acc;
}, [], list)
);
And the second a Lens which focuses on the last element of a list:
const lastLens = R.lens(R.last, (a, s) => R.update(s.length - 1, a, s));
With these two, you can build a function such as this:
const checkLasts = R.pipe(
breakWhen(R.eqProps('type')),
R.map(R.over(lastLens, R.assoc('last', true))),
R.flatten
);
checkLasts(arr);
The implementation of breakWhen is pretty awful. I'm sure there's something better. The function combines ideas from Ramda's splitEvery and splitWhen
This is slightly different from David Chambers' solution as it doesn't add a last: false property to the remaining elements. But obviously it's more complex. And either of them would fail if the data is not grouped as expected.

Related

Access an Array Item by index in AWS Dynamodb Query Results "Items" in Step Function

I have this dynamodb:Query in my step function:
{
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
"Next": "If nothing returned by query Or Study not yet Zipped",
"Parameters": {
"TableName": "TEST-StudyProcessingTable",
"ScanIndexForward": false,
"Limit": 1,
"KeyConditionExpression": "OrderID = :OrderID",
"FilterExpression": "StudyID = :StudyID",
"ExpressionAttributeValues": {
":OrderID": {
"S.$": "$.body.order_id"
},
":StudyID": {
"S.$": "$.body.study_id"
}
}
},
"ResultPath": "$.processed_files"
}
The results comes in as an array called Items which is nested under my ResultPath
processed_files.Items:
{
"body": {
"order_id": "1001",
"study_id": "1"
},
"processed_files": {
"Count": 1,
"Items": [
{
"Status": {
"S": "unzipped"
},
"StudyID": {
"S": "1"
},
"ZipFileS3Key": {
"S": "path/to/the/file"
},
"UploadSet": {
"S": "4"
},
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
}
],
"LastEvaluatedKey": {
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
},
"ScannedCount": 1
}
}
My question is how do i access the items inside this array from a choice state in a step function?
I need to query then decide something based on the results by checking the item in a condition in a choice state.
The problem is that since this is an array I can't access it using regular JsonPath (like with Items.item), and in my next step the choice condition does NOT accept an index like processed_files.Items['0'].Status
Ok so the answer was so simple all you need to do is use a number instead of string for the array index like this.
processed_files.Items[0].Status
I was originally mislead by an error I received which said that it expected a ' or '[' after the first '['. I mistakenly thought this meant it only accepts strings.
I was wrong, it works like any other array.
I hope this helps somebody one day.

Passing Input Parameters from one Step Function to another

I do have an Step Function A - which executes a lambda and pull some results.
Same Function has a Map Stage which is iterating over results and should call another Step Function from Map State.
While calling another Step Function B from the map state i am not able to pass the parameter or that one record as Input to Step Function B.
Please suggest how can i use Input for second step function.
Below is the example i am using , orderServiceResponse has a List of orders which I need to iterate and pass that one order to next step function.
"Validate-All" : {
"Type" : "Map",
"InputPath" : "$.orderServiceResponse",
"ItemsPath" : "$.orders",
"MaxConcurrency" : 5,
"ResultPath" : "$.orders",
"Iterator" : {
"StartAt" : "Validate" ,
"States" :{
"Validate" : {
"Type" : "Task"
"Resource" : "arn:aws:states:::states:startExecution.sync:2",
"Parameters" {
"Input" : "$.orders",
"StateMachineArn" : "{arn of Step Function B }
},
"End" : true
}
}
TL;DR Use Parameters with Map Context to add the full input object to each Map element iteration.
You have an array of data you want to process elementwise in a Map State. By default, Map only passes
the array element's data to the map iterator. But we can add additional context to each iteration.
Here is an example - the important bits are commented:
{
"StartAt": "MapState",
"States": {
"MapState": {
"Type": "Map",
"ResultPath": "$.MapResult",
"Next": "Success",
// the map's elements of each get the following:
"Parameters": {
"Index.$": "$$.Map.Item.Index", // the array element's data (we only get this by default)
"Order.$": "$$.Map.Item.Value", // the array element's index 0,1,2...
"FullInput.$": "$" // a copy of the the full input object <-- this is what you were looking for
},
"Catch": [{ "ErrorEquals": ["States.ALL"], "Next": "Fail" }],
// substitute your iterator:
"Iterator": {
"StartAt": "MockTask",
"States": {
"MockTask": {
"End": true,
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:xxxxxxxxxxxx",
"Parameters": {
"expression": "`Order ${$.Order.OrderID} was ordered by ${$.FullInput.CustomerName}`",
"expressionAttributeValues": {
"$.Order.OrderID.$": "$.Order.OrderID",
"$.FullInput.CustomerName.$": "$.FullInput.CustomerName"
}
}
}
}
},
"ItemsPath": "$.Orders"
},
"Success": { "Type": "Succeed" },
"Fail": { "Type": "Fail" }
}
}
Execution Input, 3 Orders:
{
"CustomerID": 1,
"CustomerName": "Morgan Freeman",
"OtherInfo": { "Foo": "Bar" },
"Orders": [{ "OrderID": "A", "Status": "Fulfilled" }, { "OrderID": "B", "Status": "Pending" }, { "OrderID": "C", "Status": "Cancelled" }]
}
Map Iteration 0 Input:
{
"Order": { "OrderID": "A", "Status": "Fulfilled" },
"Index": 0,
"FullInput": { "CustomerID": 1, "CustomerName": "Morgan Freeman", "OtherInfo": { "Foo": "Bar" }, "Orders": [{...
Execution Output MapResult key
{
"MapResult": [
"Order A was ordered by Morgan Freeman",
"Order B was ordered by Morgan Freeman",
"Order C was ordered by Morgan Freeman"
]
...
}

Sorting a list into smaller lists according to a characteristic in dart

Sorry for my poor explanation, I just started learning dart.
With a mock service and a json file I created a set amount of items
Example:
{
"items": [
{
"id": "01",
"type": "a"
},
{
"id": "02",
"type": "b"
},
{
"id": "03",
"type": "c"
}
]
}
when creating the list on the service it creates a single list like this:
if (json['items'] != null) {
final itemList = <ItemList>[];
json['items'].forEach((v) {
itemlistList.add(ItemList.fromJson(v));
});
return ItemList;
} else {
return [];
}
is there a way to, form the create list step to already separate them into 3 different lists for the type a, b, and c items? and if now, where and how would I divide this itemlist into 3 based on the type characteristic of each item?
Using groupBy, as suggested in this extremely similar question: Flutter/Dart how to groupBy list of maps?
import "package:collection/collection.dart";
main(List<String> args) {
var data = [
{"id": "01", "type": "a"},
{"id": "02", "type": "b"},
{"id": "03", "type": "c"},
{"id": "04", "type": "a"},
{"id": "05", "type": "a"},
{"id": "06", "type": "b"},
];
var newMap = groupBy(data, (Map obj) => obj["type"]);
print(newMap);
}

How do I extract data from "List" field

I'm getting JSON data from webservice and trying to make a table . Datadisk is presented as List and clicking into each item will navigate further down the hiearchy like denoted in screenshots below. I need to concatate storageAccountType for each item with | sign, so if there were 2 list items for Greg-VM and it had Standard_LRS for first one and Premium_LRS for second one then new column will list Standard_LRS | Premium_LRS for that row.
Input returned by function is below
[
{
"name": "rhazuremspdemo",
"disk": {
"id": "/subscriptions/24ba3e4c-45e3-4d55-8132-6731cf25547f/resourceGroups/AzureMSPDemo/providers/Microsoft.Compute/disks/rhazuremspdemo_OsDisk_1_346353b875794dd4a7a5c5938abfb7df",
"storageAccountType": "StandardSSD_LRS"
},
"datadisk": []
},
{
"name": "w12azuremspdemo",
"disk": {
"id": "/subscriptions/24ba3e4c-45e3-4d55-8132-6731cf25547f/resourceGroups/AzureMSPDemo/providers/Microsoft.Compute/disks/w12azuremspdemo_OsDisk_1_09788205f8eb429faa082866ffee0f18",
"storageAccountType": "Premium_LRS"
},
"datadisk": []
},
{
"name": "Greg-VM",
"disk": {
"id": "/subscriptions/24ba3e4c-45e3-4d55-8132-6731cf25547f/resourceGroups/GREG/providers/Microsoft.Compute/disks/Greg-VM_OsDisk_1_63ed471fef3e4f568314dfa56ebac5d2",
"storageAccountType": "Premium_LRS"
},
"datadisk": [
{
"name": "Data",
"createOption": "Attach",
"diskSizeGB": 10,
"managedDisk": {
"id": "/subscriptions/24ba3e4c-45e3-4d55-8132-6731cf25547f/resourceGroups/GREG/providers/Microsoft.Compute/disks/Data",
"storageAccountType": "Standard_LRS"
},
"caching": "None",
"toBeDetached": false,
"lun": 0
},
{
"name": "Disk2",
"createOption": "Attach",
"diskSizeGB": 10,
"managedDisk": {
"id": "/subscriptions/24ba3e4c-45e3-4d55-8132-6731cf25547f/resourceGroups/GREG/providers/Microsoft.Compute/disks/Disk2",
"storageAccountType": "Standard_LRS"
},
"caching": "None",
"toBeDetached": false,
"lun": 1
}
]
}
]
How do I do that?
Thanks,
G
This should help you. It steps through the process.
If you have a scenario like this
you can use Add custom Column and type the follwing expression:
=Table.Columns([TableName], "ColumnName")
to get it as list:
Now you can left click on the Custom column and chose Extract Values....
Choose Custom and your delimiter | and hit OK
This way the data who was in your list will now be in the same row with the delimiter

How do I get only the element values that match in the list in the Elastic Search?

[Hi, there]
I want to create an ES query that only retrieves certain elements that match in the list.
Here is my ES index schema.
"test-es-2018":{
"aliases": {},
"mappings": {
"test-1": {
"properties": {
"categoryName": {
"type": "keyword",
"index": false
},
"genDate": {
"type": "date"
},
"docList": {
"properties": {
"rank": {
"type": "integer",
"index": false
},
"doc-info": {
"properties": {
"docId": {
"type": "keyword"
},
"docName": {
"type": "keyword",
"index": false
},
}
}
}
},
"categoryId": {
"type": "keyword"
},
}
}
}
}
There are documents listed in the category. Documents in the list have their own information.
*search query in Kibana.
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 2,
"doc-info": {...}
},
{
"rank": 1,
"doc-info": {...}
},
{
"rank": 5,
"doc-info": {...}
},
],
"categoryId": "201"
}
First, I want to get only the element value that match in the list.
I would like to see only documents with rank 1 in the list. However, if I query using match as below, the result is the same as *search query in kibana.
*match query in Kibana.
GET test-es-2018/_search
{
"query": {
"bool": {
"must": [
{ "match": { "docList.rank": 1 } },
]
}
}
}
In my opinion, it seems to print the entire list because it contains a document with rank one.
What I want is:
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 1,
"doc-info": {...}
},
],
"categoryId": "201"
}
Is this possible?
Second, I want to sort the docList by rank. I tried sorting by creating a query like the following, but it was not sorted.
*sort query in Kibana.
GET test-es-2018/_search?
{
"query" : {
"bool" : {...}
},
"sort" : [
{
"docList.rank" : {
"order" : "asc"
}
}
]
}
What I want is:
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 1,
"doc-info": {...}
},
{
"rank": 2,
"doc-info": {...}
},
{
"rank": 5,
"doc-info": {...}
},
],
"categoryId": "201"
}
I do not know how to access the list. Is there a good idea for both of these issues?
In general you could use source filter to retrieve only part of the document but this way it's not possible to exclude some fields based on their values.
As far as I know Elasticsearch doesn't support changing order of field values in the _source. Partly the desired result can be achieved by using nested fields along with inner_hits -> sort query expression. This way sorted subhits will be returned in the inner_hits section of the response.
P.S. Typically working with Elasticsearch you should consider indexed document as the smallest indivisible search unit.