grep through a file conditionally in both directions - regex

Question

I have a log file written to by several instances of a cgi script. I need to extract certain information, with the following typical workflow:
search for the first occurrence of RequestString
extract PID from that log line
search backwards for the first occurrence of PID<separator>ConnectionString, to identify the client that initiated the request
do something with ConnectionString and repeat the search from after 'RequestString'
What is the best way to do this? I was thinking of writing a perl script to do this with caching the last N lines, and then match through those lines to perform 3.
Is there any better way to do this? Like extended regex that would do exactly this?
Sample with line numbers for reference -- not part of the file:
1 date pid1 ConnectionString1
2 date pid2 ConnectionString2
3 date pid3 ConnectionString3
4 date pid2 SomeOutput2
5 date pid2 SomeOutput2
6 date pid4 ConnectionString4
7 date pid3 SomeOutput3
8 date pid4 RequestString4
9 date pid1 SomeOutput1
10 date pid1 ConnectionString1
11 date pid1 RequestString1
12 date pid5 RequestString5
When I grep through this sample file, I wish for the following to match:
line 8, paired with line 6
line 11, paired with line 10 (and not with line 1)
Specifically, the following shouldn't be matched:
line 12, because no matching ConnectionString with that pid is found (pid5)
line 1, because there is a new ConnectionString for that pid before the next RequestString for that pid (line 10). Imagine that the first connection attempt failed before logging the RequestString)
any of the lines from pid2/pid3, because hey dont have a RequestString logged.
I could imagine writing a regex with the option for . to match \n:((pid\d)\s*(ConnectionString\d))(?!\1).*\2\s*RequestString\d and then use \3 to identify the client.
However, there are disproportionately more (perhaps between 1000 and 10000 times more) ConnectionStrings than RequestStrings, so my intuition was to first go for the RequestString and then backtrack.
I guess I could play with (?<) for lookbehind, but the lengths between ConnectionStrings and RequestStrings are essentially arbitrary -- will that work well?

Answer 1

Something along these lines:
#!/bin/bash
# Find and number all RequestStrings, then loop through them
grep -n RequestString file | while IFS=":" read n string; do
echo $n,$string # Debug
head -n $n file | tail -r | grep -m1 Connection
done
Output
4,RequestString 1
6189:Connection
7,RequestString 2
7230:Connection
9,RequestString 3
8280:Connection
with this input file
6189:Connection
RequestString 1
7229:Connection
7230:Connection
RequestString 2
8280:Connection
RequestString 3
Note: I used tail -r because OSX lacks tac which I would have preferred.