Get all instances of text in curly braces between brackets - regex

Let's say I have some text like this:
{
"source": "Analytics 13 {Employee_Info.acl} {Employee_Data}",
"lastRecNo": "3",
"columns": {
"ID": "numeric",
"NAME": "character",
"EFFECTIVE_DATE": "date",
"ROLE": "character"
},
"data": [{
"ID": 1,
"NAME": "Bill Smith",
"EFFECTIVE_DATE": "2018-10-01",
"ROLE": "Director {Regional},{Call Center}"
},
{
"ID": 2,
"NAME": "Ellen Jones",
"EFFECTIVE_DATE": "2018-07-01",
"ROLE": "Manager"
},
{
"ID": 3,
"NAME": "Sam Edwards",
"EFFECTIVE_DATE": "2018-09-01",
"ROLE": "Supervisor"
}]
}
And I want to RegEx match every object inside the "data" array (including the curly braces).
So the first match would be:
{
"ID": 1,
"NAME": "Bill Smith",
"EFFECTIVE_DATE": "2018-10-01",
"ROLE": "Director {Regional},{Call Center}"
}
the second would be:
{
"ID": 2,
"NAME": "Ellen Jones",
"EFFECTIVE_DATE": "2018-07-01",
"ROLE": "Manager"
}
and the third would be
{
"ID": 3,
"NAME": "Sam Edwards",
"EFFECTIVE_DATE": "2018-09-01",
"ROLE": "Supervisor"
}
What regex pattern would I use to do that in PowerShell?
Notice the first match actually has some extra curly braces in the text of the "ROLE" field, which shouldn't interfere with the match.
I've tried this so far '(?<={).*?(?=})', but the first match is:
"source": "Analytics 13 {Employee_Info.acl
This result isn't a part of the "data" array and it doesn't include the curly braces in the match. I know I'm missing something that says "make sure we are inside the brackets/"data" array and I'm probably not taking into account the extra curly braces in the "ROLE" field in the first object of the "data" array that I want to ignore.

Your task can be easily done using ConvertFrom-Json and ConvertTo-Json cmdlets.
Here is a brief example:
First you get text file content to variable.
$JSON = #"
[
{
"source": "Analytics 13 {Employee_Info.acl} {Employee_Data}",
"lastRecNo": "3",
"columns": {
"ID": "numeric",
"NAME": "character",
"EFFECTIVE_DATE": "date",
"ROLE": "character"
},
"data": [{
"ID": 1,
"NAME": "Bill Smith",
"EFFECTIVE_DATE": "2018-10-01",
"ROLE": "Director {Regional},{Call Center}"
},
{
"ID": 2,
"NAME": "Ellen Jones",
"EFFECTIVE_DATE": "2018-07-01",
"ROLE": "Manager"
},
{
"ID": 3,
"NAME": "Sam Edwards",
"EFFECTIVE_DATE": "2018-09-01",
"ROLE": "Supervisor"
}]
}
]
"#
Then you just perform converting from JSON using ConvertFrom-Json cmdlet.
ConvertFrom-Json -InputObject $JSON
Output:
source lastRecNo columns data
------ --------- ------- ----
Analytics 13 {Employee_Info.acl} {Employee_Data} 3 #{ID=numeric; NAME=character; EFFECTIVE_DATE=date; ROLE=character} {#{ID=1; NAME=Bill Smith; EFFECTIVE_DATE=2018-10-01; ROLE=Director {Regional},{Call Center}}, #...
You then can return items from DATA to JSON format using ConvertTo-Json cmdlet. All together.
$PSObject = ConvertFrom-Json -InputObject $JSON
foreach ($item in $PSObject.data){
ConvertTo-Json $item
}
Output:
{
"ID": 1,
"NAME": "Bill Smith",
"EFFECTIVE_DATE": "2018-10-01",
"ROLE": "Director {Regional},{Call Center}"
}
{
"ID": 2,
"NAME": "Ellen Jones",
"EFFECTIVE_DATE": "2018-07-01",
"ROLE": "Manager"
}
{
"ID": 3,
"NAME": "Sam Edwards",
"EFFECTIVE_DATE": "2018-09-01",
"ROLE": "Supervisor"
}
You can now add filter conditions for DATA items in foreach loop.

Related

What it is the Grep to remove all lines BUT, name and country

My file has this following pattern.
[
{
"id": 8050879,
"coord": { "lon": -1.65825, "lat": 42.808472 },
"country": "ES",
"geoname": { "cl": "P", "code": "PPLL", "parent": 6359749 },
"name": "Iturrama",
"stat": { "level": 1.0, "population": 24846 },
"stations": [
{ "id": 5493, "dist": 4, "kf": 1 },
{ "id": 28697, "dist": 32, "kf": 1 }
],
"zoom": 14
},
{
"id": 5406990,
"coord": { "lon": -122.064957, "lat": 37.906311 },
"country": "US",
"geoname": { "cl": "P", "code": "PPL", "parent": 5339268 },
"langs": [
{ "bg": "Уолнът Крийк" },
{ "de": "Walnut Creek" },
{ "en": "Walnut Creek" },
{ "eo": "Walnut Creek" },
{ "link": "http://en.wikipedia.org/wiki/Walnut_Creek%2C_California" },
{ "post": "94595" }
],
"name": "Walnut Creek",
"stat": { "level": 1.0, "population": 64173 },
"stations": [
{ "id": 374, "dist": 9, "kf": 1 },
{ "id": 10103, "dist": 9, "kf": 1 },
],
"zoom": 11
},
...
]
I would like to get
[
{
"country": "ES",
"name": "Iturrama"
},
{
"country": "US",
"name": "Walnut Creek"
},
...
]
I have been using
grep -v id filename > result
then
grep -v coord result > result
grep -v geoname result > result
...
until I get my pattern, but I noticed I am deleting anything that has id on it,
So If I have a name: "cIDadel" it will delete too.
Can any one help me with that?
Don't use non-syntax aware tools like grep to parse structured data like JSON. It can't possibly differentiate the underlying types i.e. object/array or any other. Use a proper parser like jq using which you can simply do
jq 'map({country, name})' json_file
See it work in jq-playground. Downloading instructions and setting up is pretty easy - Download jq
If you need to use shell tools for some reason instead of JSON parsing, use AWK.
file.awk
/^\[$/ {print($0)}
/^\{$/ {print($0)}
/"country"/ {print($0)}
/"name"/ {print($0)}
/^ *\},$/ {print($1)}
/^\]$/ {print($0)}
Call:
awk -f file.awk yourdata.txt

How to fetch API data as this. props?

I am trying to fetch data from API in react component as
{this.props.buyer && this.props.buyer[0].phone_number[0].number} - it's throwing error
Cannot read property 'number' of undefined
{this.props.buyer && this.props.buyer[0].name} - it's working fine
This is the API data
Orders: {
buyer:
},
}
[
{
"id": 2,
"name": "Qi Xiang",
"type": "Consignee",
"address": {
"id": 2,
"type": "shipping",
"street": "China China",
"city": "Beijing",
"postal_code": "34343",
"province": "23232",
"country": "CN"
},
"email": null,
"phone_number": {
"number": "323232",
"type": "Phone"
},
"id_image_url": "/api/files/24e49645-df42-4984-a
}
]
},
}
Your phonenumber is not array. You must use this:
this.props.buyer[0].phone_number.number

Elastic Search Sort

I have a table for some activities like
[
{
"id": 123,
"name": "Ram",
"status": 1,
"activity": "Poster Design"
},
{
"id": 123,
"name": "Ram",
"status": 1,
"activity": "Poster Design"
},
{
"id": 124,
"name": "Leo",
"categories": [
"A",
"B",
"C"
],
"status": 1,
"activity": "Brochure"
},
{
"id": 134,
"name": "Levin",
"categories": [
"A",
"B",
"C"
],
"status": 1,
"activity": "3D Printing"
}
]
I want to get this data from elastic search 5.5 by sorting on field activity, but I need all the data corresponding to name = "Ram" first and then remaining in a single query.
You can use function score query to boost the result based on match for the filter(this case ram in name).
Following query should work for you
POST sort_index/_search
{
"query": {
"function_score": {
"query": {
"match_all": {}
},
"boost": "5",
"functions": [{
"filter": {
"match": {
"name": "ram"
}
},
"random_score": {},
"weight": 1000
}],
"score_mode": "max"
}
},
"sort": [{
"activity.keyword": {
"order": "desc"
}
}]
}
I would suggest using a bool query combined with the should clause.
U will also need to use the sort clause on your field.

loopback.io multiple includes

I have a loopback.io API with a MongoDB database.
I have a model called Project with 2 hasMany relationships
Project has many ProjectArticle objects
Project has many ProjectWorkHours objects
When I run I do a request: http://0.0.0.0:3000/api/Projects?filter[include][projectArticles]
I get the following response, which is good
{
"title": "Project 1",
"description": "My project number one",
"dateCreated": "2015-10-31T00:00:00.000Z",
"id": "5634b3af340faf570c7e70a8",
"projectArticles": [
{
"articleName": "brick",
"quantity": 5,
"unitPrice": 2,
"id": "5634b9ea5ab833960c8fbf6d",
"projectId": "5634b3af340faf570c7e70a8"
}
]
}
and when I do: http://0.0.0.0:3000/api/Projects?filter[include][workHours] I get the right answer:
{
"title": "Project 1",
"description": "My project number one",
"dateCreated": "2015-10-31T00:00:00.000Z",
"id": "5634b3af340faf570c7e70a8",
"workHours": [
{
"description": "blabla",
"startDate": "2015-10-31T00:00:00.000Z",
"endDate": "2015-10-31T00:00:00.000Z",
"id": "5634e11d5f6471f10d2e0dd7",
"workHourId": "5634b3af340faf570c7e70a8"
},
{
"description": "blabla 2",
"startDate": "2015-10-31T00:00:00.000Z",
"endDate": "2015-10-31T00:00:00.000Z",
"id": "5634e1265f6471f10d2e0dd8",
"workHourId": "5634b3af340faf570c7e70a8"
}
]
}
but how do i combine the two includes in one request? so I have the following result:
{
"title": "Project 1",
"description": "My project number one",
"dateCreated": "2015-10-31T00:00:00.000Z",
"id": "5634b3af340faf570c7e70a8",
"workHours": [
{
"description": "blabla",
"startDate": "2015-10-31T00:00:00.000Z",
"endDate": "2015-10-31T00:00:00.000Z",
"id": "5634e11d5f6471f10d2e0dd7",
"workHourId": "5634b3af340faf570c7e70a8"
},
{
"description": "blabla 2",
"startDate": "2015-10-31T00:00:00.000Z",
"endDate": "2015-10-31T00:00:00.000Z",
"id": "5634e1265f6471f10d2e0dd8",
"workHourId": "5634b3af340faf570c7e70a8"
}
],
"projectArticles": [
{
"articleName": "brick",
"quantity": 5,
"unitPrice": 2,
"id": "5634b9ea5ab833960c8fbf6d",
"projectId": "5634b3af340faf570c7e70a8"
}
]
}
I tried this: http://0.0.0.0:3000/api/Projects?filter[include][workHours]&[include][projectArticles] but it only applies the first one and the second filter is kind of ignored
any ideas?
I've found the syntax for doing it
http://0.0.0.0:3000/api/Projects?filter={"include":["workHours", "projectArticles"]}

CouchDB-Why my rerduce is always coming as false ? I am not able to reduce anything properly

I am new to CouchDB. I have a 9 gb dataset loaded into my couchdb. I am able to map everything correctly. But I cannot reduce any of the results using the code written in the reduce column. When i tried log, log shows that rereduce values as false. Do i need to do anything special while doing the Map() or how to set the rereduce value is TRUE??
A sample of my data is as follows:
{
"_id": "33d4d945613344f13a3ee92933b160bf",
"_rev": "1-0425ca93e3aa939dff46dd51c3ab86f2",
"release": {
"genres": {
"genre": "Electronic"
},
"status": "Accepted",
"videos": {
"video": [
{
"title": "[1995] bola - krak jakomo",
"duration": 349,
"description": "[1995] bola - krak jakomo",
"src": "http://www.youtube.com/watch?v=KrELXoYThpI",
"embed": true
},
{
"title": "Bola - Forcasa 3",
"duration": 325,
"description": "Bola - Forcasa 3",
"src": "http://www.youtube.com/watch?v=Lz9itUo5xtc",
"embed": true
},
{
"title": "Bola (Darrell Fitton) - Metalurg (MV)",
"duration": 439,
"description": "Bola (Darrell Fitton) - Metalurg (MV)",
"src": "http://www.youtube.com/watch?v=_MYpOOMRAeQ",
"embed": true
}
]
},
"labels": {
"label": {
"catno": "SKA005",
"name": "Skam"
}
},
"companies": "",
"styles": {
"style": [
"Downtempo",
"Experimental",
"Ambient"
]
},
"formats": {
"format": {
"text": "",
"name": "Vinyl",
"qty": 1,
"descriptions": {
"description": [
"12\"",
"Limited Edition",
"33 ⅓ RPM"
]
}
}
},
"country": "UK",
"id": 1928,
"released": "1995-00-00",
"artists": {
"artist": {
"id": 390,
"anv": "",
"name": "Bola",
"role": "",
"tracks": "",
"join": ""
}
},
"title": 1,
"master_id": 13562,
"tracklist": {
"track": [
{
"position": "A1",
"duration": "4:33",
"title": "Forcasa 3"
},
{
"position": "A2",
"duration": "5:48",
"title": "Krak Jakomo"
},
{
"position": "B1",
"duration": "7:50",
"title": "Metalurg 2"
},
{
"position": "B2",
"duration": "6:40",
"title": "Balloom"
}
]
},
"data_quality": "Correct",
"extraartists": {
"artist": {
"id": 388200,
"anv": "",
"name": "Paul Solomons",
"role": "Mastered By",
"tracks": "",
"join": ""
}
},
"notes": "Limited to 480 copies.\nA1 is a shorter version than that found on the 'Soup' LP.\nA2 ends in a lock groove."
}
}
My intention is to count the mapped values. My mapping function is as follows:
function(doc){
if(doc.release)
emit(doc.release.title,1)
}
Map results shows around 5800 results
I want to use the following functions in the reduce tab to count:
Reduce:
_count or _sum
It does not give single rounded value. Even i cannot get the simple _count operations right !!! :(
for screenshot,
Please help me !!!
What you got was the sum of values per title. What you wanted, was the sum of values in general.
Change the grouping drop-down list to none.
Check CouchdDB's wiki for more details on grouping.