awk (or sed/grep) to get occurrences of substring - regex

Question

I have a json string in a bash variable, which is something like this:
{
"items": [
{
"foo": null,
"timestamp": 1553703000,
"bar": 123
},
{
"foo": null,
"timestamp": 1553703200,
"bar": 456
},
{
"foo": null,
"timestamp": 1553703400,
"bar": 789
}
]
}
I want to know how many of those timestamps are after a given datetime, so if I have 1553703100 it'll return 2.
(Bonus imaginary points if you can get me just that number!)
As a step towards that, I want to get just the matches of "timestamp": \d+, in the string so that I can loop through them in a bash script.
I've used sed and grep a bit, but never used awk, and from my reading it seems like that might be the better match for the task.
Other info:
- The json is already pretty-printed, as above, so the timestamps would always be on separate lines.
- This is to run in Cygwin, so I have awk/gawk, sed, and grep/egrep, but probably not others.
- Could be any number of timestamps in the json.

Answer 1

You didn't provide the expected output so it's a guess but is this what you're trying to do?
$ echo "$var" | jq '.items[].timestamp'
1553703000
1553703200
1553703400
or maybe:
$ echo "$var" | jq '.items[].timestamp | select(. > 1553703100)'
1553703200
1553703400
or:
$ echo "$var" | jq '[.items[].timestamp | select(. > 1553703100)] | length'
2
WARNING: I'm just learning jq so there may be better ways to do the above!

Answer 2

edit: The second approach listed below has serious problems that were very helpfully outlined by #EdMorton. I've elected to keep the old code for educational purposes.
Avoided substr() and caught null string i:
$ awk -v dt=1553703100 '
/timestamp/ && $2+0>dt {i++}
END {print i+0}
' <<< "$var"
2
WARNING: PROBLEMATIC CODE
Here I used substr(string, index, [characters]) to trim the comma off your second field. The /timestamp/ regex is not complex; it could be improved if your json became more intricate.
$ awk -v dt=1553703100 '
/timestamp/ && substr($2, 0, length($2)) > dt {i++}
END {print i}
' <<< "$var"
2

Answer 3

You can also implement quickly a python solution:
input:
$ cat data.json
{
"items": [
{
"foo": null,
"timestamp": 1553703000,
"bar": 123
},
{
"foo": null,
"timestamp": 1553703200,
"bar": 456
},
{
"foo": null,
"timestamp": 1553703400,
"bar": 789
}
]
}
code:
$ cat extract_value2.py
import json
tLimit = 1553703100
with open('data.json') as f:
data = json.load(f)
print([t['timestamp'] for t in data["items"] if t['timestamp'] > tLimit])
output:
$ python extract_value2.py
[1553703200, 1553703400]
count code:
$ cat extract_value2.py
import json
tLimit = 1553703100
with open('data.json') as f:
data = json.load(f)
print(len([t['timestamp'] for t in data["items"] if t['timestamp'] > tLimit]))
output:
$ python extract_value2.py
2