Splitting of frame in different parts in bigquery - regex

Question

i've got a string frame looking like this in google bigquery:
S,0,2B3,8, C2 B3 00 00 00 00 03 DE
S,0,3FA,6, 00 E0 04 A5 00 0B
S,0,440,8, 80 40 4E A5 00 47 00 64
S,0,450,8, 89 50 01 12 01 19 01 B3
S,0,4B0,8, 80 B0 4E A5 00 43 00 64
my aim is to extract the 8 bytes at the end (eg 80 40 4E A5 00 47 00 64
). possible only the ones beginning with 83 and 84.
i didnt get it to work with neither split, trim, contains nor regexp_extract.
i'd be quite happy if anyone could help me.
regards
/edit
Thank you both very much for your solutions! this helped quite a lot.
#standardSQL
SELECT
*
FROM (
SELECT
timestamp,
REGEXP_EXTRACT(CAN_Frame, r', ([^,]+)$') AS bytes_string,
FROM_HEX(REPLACE(REGEXP_EXTRACT(CAN_Frame, r', ([^,]+)$'), ' ', '')) AS bytes
FROM `data.source`
)
WHERE SUBSTR(bytes, 1, 1) IN (b'\x83', b'\x84')
ORDER BY timestamp DESC
LIMIT 8000
gives me
Row timestamp bytes_string bytes
1 2017-09-29 14:31:02 UTC 84 10 00 25 00 21 00 4F hBAAJQAhAE8=
2 2017-09-29 14:30:42 UTC 83 80 00 01 00 03 00 0D g4AAAQADAA0=
3 2017-09-29 14:30:40 UTC 84 B2 00 27 00 08 00 03 hLIAJwAIAAM=
#standardSQL
SELECT
timestamp,
TRIM(SPLIT(CAN_Frame)[OFFSET(4)]) AS bytes
FROM
`data.source`
WHERE
LENGTH(CAN_Frame) > 1 and
SUBSTR(TRIM(SPLIT(CAN_Frame)[OFFSET(4)]),1,2) IN ('83', '84')
ORDER BY
timestamp DESC
LIMIT
8000
gives me
Row timestamp bytes
1 2017-09-29 14:31:02 UTC 84 10 00 25 00 21 00 4F
2 2017-09-29 14:30:42 UTC 83 80 00 01 00 03 00 0D
3 2017-09-29 14:30:40 UTC 84 B2 00 27 00 08 00 03
is there a possibility to get only the sixth and seventh byte from the bytes_string beginning with 83, to get 4th and 5th byte from the bytes_string beginning with 84 and to geht the 8th byte from string 83 and the 3rd byte from string 84 for further calculations?
best regards

Answer 1

Below is for BigQuery StandardSQL
#standardSQL
WITH `yourTable` AS (
SELECT 'S,0,2B3,8, C2 B3 00 00 00 00 03 DE' AS frame UNION ALL
SELECT 'S,0,3FA,6, 00 E0 04 A5 00 0B' UNION ALL
SELECT 'S,0,440,8, 80 40 4E A5 00 47 00 64' UNION ALL
SELECT 'S,0,450,8, 89 50 01 12 01 19 01 B3' UNION ALL
SELECT 'S,0,4B0,8, 80 B0 4E A5 00 43 00 64'
)
SELECT frame, TRIM(SPLIT(frame)[OFFSET(4)]) AS bytes
FROM `yourTable`
WHERE SUBSTR(TRIM(SPLIT(frame)[OFFSET(4)]), 1, 2) IN ('80', 'C2')

Answer 2

Here is an example that should help. It produces two columns with different interpretations of the bytes: one (bytes_string) is just the end of the strings that you showed, whereas the other (bytes) is the bytes string converted to an actual BYTES type. In the BigQuery UI, make sure to deselect "Use Legacy SQL" under "Show Options" or include the #standardSQL directive:
#standardSQL
WITH Frames AS (
SELECT 'S,0,2B3,8, C2 B3 00 00 00 00 03 DE' AS frame UNION ALL
SELECT 'S,0,3FA,6, 00 E0 04 A5 00 0B' UNION ALL
SELECT 'S,0,440,8, 80 40 4E A5 00 47 00 64' UNION ALL
SELECT 'S,0,450,8, 89 50 01 12 01 19 01 B3' UNION ALL
SELECT 'S,0,4B0,8, 80 B0 4E A5 00 43 00 64'
)
SELECT
frame,
REGEXP_EXTRACT(frame, r', ([^,]+)$') AS bytes_string,
FROM_HEX(REPLACE(REGEXP_EXTRACT(frame, r', ([^,]+)$'), ' ', '')) AS bytes
FROM Frames;
Here is another example that demonstrates filtering on the bytes column to include only values starting with \x83 or \x84 (this will return an empty result set for the sample data you provided):
#standardSQL
WITH Frames AS (
SELECT 'S,0,2B3,8, C2 B3 00 00 00 00 03 DE' AS frame UNION ALL
SELECT 'S,0,3FA,6, 00 E0 04 A5 00 0B' UNION ALL
SELECT 'S,0,440,8, 80 40 4E A5 00 47 00 64' UNION ALL
SELECT 'S,0,450,8, 89 50 01 12 01 19 01 B3' UNION ALL
SELECT 'S,0,4B0,8, 80 B0 4E A5 00 43 00 64'
)
SELECT
*
FROM (
SELECT
frame,
REGEXP_EXTRACT(frame, r', ([^,]+)$') AS bytes_string,
FROM_HEX(REPLACE(REGEXP_EXTRACT(frame, r', ([^,]+)$'), ' ', '')) AS bytes
FROM Frames
)
WHERE SUBSTR(bytes, 1, 1) IN (b'\x83', b'\x84');