regular expression to extract tokens from block of text - regex

Question

I have following block of text retrieved from a log file
SELECT statement with ID: AE12400 SELECT /*+ ALL_ROWS */
T1.CONFLICT_ID, T1.LAST_UPD, T1.CREATED,
T1.LAST_UPD_BY, T1.CREATED_BY, T1.MODIFICATION_NUM,
T1.ROW_ID, T1.DFLT_LIC_FLG, T1.NAME, T1.VAL,
:1 FROM SIEBEL.S_LST_OF_VAL T1 WHERE
(T1.ACTIVE_FLG = :2 OR T1.ACTIVE_FLG IS NULL) AND (T1.TYPE = :3
AND T1.BU_ID IS NULL) ORDER BY T1.TYPE, T1.ORDER_BY, T1.VAL
Bind variable 1: ,,,SADMIN,00000002579c129c:0,,List Of Values
(Internal), Bind variable 2: Y Bind variable 3: ZERO_DTIME_MODE
***** SQL Statement Execute Time: 0.028 seconds ***** 3 row(s) retrieved by ID: AE0EF18
I need to get following tokens out of this block
Statement Id : AE12400
SQL_Query: SELECT /*+ ALL_ROWS */
T1.CONFLICT_ID, T1.LAST_UPD, T1.CREATED,
T1.LAST_UPD_BY, T1.CREATED_BY, T1.MODIFICATION_NUM,
T1.ROW_ID, T1.DFLT_LIC_FLG, T1.NAME, T1.VAL,
:1 FROM SIEBEL.S_LST_OF_VAL T1 WHERE
(T1.ACTIVE_FLG = :2 OR T1.ACTIVE_FLG IS NULL) AND (T1.TYPE = :3 AND T1.BU_ID IS NULL) ORDER BY T1.TYPE, T1.ORDER_BY, T1.VAL
Bind Variable : [",,,SADMIN,00000002579c129c:0,,List Of Values (Internal)","Y","ZERO_DTIME_MODE"]
SQL Time: 0.028
SQL Rows: 3
I have come up with following regular experssion so far to extract the statement, time & rows
SQL Rows : \s\d{1,4}\s
SQL Time: \d{1,3}\.\d{1,4}
Statement Id: (ID:)(\s\w+)
But I am not sure how to extract the SQL along with Bind Variables from the text.

Answer 1

Your current patterns are not precise as they may match another sub-strings that are not expected ones. Hence, I'll go to provide all needed expressions:
SQL rows:
\d{1,4}(?=\s*row)
Query running time:
(\d+(?:\.\d+)?)(?=\s*second)
Statement ID:
ID:\s*(\w+)
SQL statement (m: dot matches newlines):
(?m)ID:\s\w+\s(.*?)(?=Bind variable)
Bind variables:
(?m)Bind variable\s*\d+:\s*(.*?)(?=Bind variable|$)
For bind variables you should work with a matchAll() or findAll() similar method in your programming language. [Live demo, look at Match groups block]

Answer 2

These could be cleaned up; they're not very efficient as they are. But this should head you in the right direction.
SQL_Query: SELECT(?! statement with ID)[\W\w]*?(?=Bind variable \d)
If you're regexing a whole log with more than one of those text blocks, you'll have to get all the bind variables first, and then each one taken from that. Otherwise, you can skip that step.
Find bind vars: Bind variable \d+:[\W\w]*?(?=\s+\*\*\*\*\*)
Extract vars: Bind variable \d+:\s*([\W\w]*?)(?=Bind variable)
Also there could be problems if, for example, there is the text "Bind variable" within your SQL Query... But it would be difficult to get 100% on that, and it's unlikely there would be such things mixed into the other parts of the log, I'm guessing.