How can I parse R syntax from R? - regex

Question

I am trying to put together a little script which would track back the history of an object, by analyzing the R history. I am stuck at efficiently parsing R commands and splitting them. Consider the following R command:
for( i in 1:10 ) { a[i] <- myfunc() ; print( sprintf( "done step %d; proceeding", i ) ) }
Splitting at semicolons or curly brackets is not a problem, but what about the semicolons (or other special things) which are in quotes? I end up going char by char and keeping track of whether I am in a quote of not (and also of the backslashes...). This is footwork, and I'm sure there is something simpler, probably invented around the time Unix was born. A clever regex, maybe?

Answer 1

Use parse. It returns an expression which can be subsetted into the various components of the parse tree:
x <- parse(text='for( i in 1:10 ) { a[i] <- myfunc() ; print( sprintf( "done step %d; proceeding", i ) ) }',n=1)
x[[1]]
for (i in 1:10) {
a[i] <- myfunc()
print(sprintf("done step %d; proceeding", i))
}
x[[1]][[1]]
`for`
x[[1]][[4]]
{
a[i] <- myfunc()
print(sprintf("done step %d; proceeding", i))
}
x[[1]][[4]][[2]]
a[i] <- myfunc()

Answer 2

It depends on how much details you need, but beside the base parse function you might also check out the awesome parser package that is really fast to do its job and also returns much more details compared to parse. Quick demo:
> library(parser)
> parser(text='for( i in 1:10 ) { a[i] <- myfunc() ; print( sprintf( "done step %d; proceeding", i ) ) }')
expression(for (i in 1:10) {
a[i] <- myfunc()
print(sprintf("done step %d; proceeding", i))
})
attr(,"data")
line1 col1 byte1 line2 col2 byte2 token id parent top_level token.desc terminal text
1 1 0 0 1 3 3 270 1 77 0 FOR TRUE for
2 1 3 3 1 4 4 40 2 77 0 '(' TRUE (
3 1 5 5 1 6 6 263 4 77 0 SYMBOL TRUE i
4 1 7 7 1 9 9 271 6 77 0 IN TRUE in
5 1 10 10 1 11 11 261 8 9 0 NUM_CONST TRUE 1
6 1 10 10 1 11 11 78 9 15 0 expr FALSE
7 1 11 11 1 12 12 58 10 15 0 ':' TRUE :
8 1 12 12 1 14 14 261 11 12 0 NUM_CONST TRUE 10
9 1 12 12 1 14 14 78 12 15 0 expr FALSE
10 1 15 15 1 16 16 41 14 77 0 ')' TRUE )
11 1 10 10 1 14 14 78 15 77 0 expr FALSE
12 1 17 17 1 18 18 123 18 74 0 '{' TRUE {
13 1 19 19 1 20 20 263 20 22 0 SYMBOL TRUE a
14 1 20 20 1 21 21 91 21 28 0 '[' TRUE [
15 1 19 19 1 20 20 78 22 28 0 expr FALSE
16 1 21 21 1 22 22 263 23 25 0 SYMBOL TRUE i
17 1 22 22 1 23 23 93 24 28 0 ']' TRUE ]
18 1 21 21 1 22 22 78 25 28 0 expr FALSE
19 1 19 19 1 23 23 78 28 40 0 expr FALSE
20 1 24 24 1 26 26 265 30 40 0 LEFT_ASSIGN TRUE <-
21 1 27 27 1 33 33 297 32 34 0 SYMBOL_FUNCTION_CALL TRUE myfunc
22 1 33 33 1 34 34 40 33 37 0 '(' TRUE (
23 1 27 27 1 33 33 78 34 37 0 expr FALSE
24 1 34 34 1 35 35 41 35 37 0 ')' TRUE )
25 1 27 27 1 35 35 78 37 40 0 expr FALSE
26 1 36 36 1 37 37 59 39 74 0 ';' TRUE ;
27 1 19 19 1 35 35 78 40 74 0 expr FALSE
28 1 38 38 1 43 43 297 44 46 0 SYMBOL_FUNCTION_CALL TRUE print
29 1 43 43 1 44 44 40 45 69 0 '(' TRUE (
30 1 38 38 1 43 43 78 46 69 0 expr FALSE
31 1 45 45 1 52 52 297 48 50 0 SYMBOL_FUNCTION_CALL TRUE sprintf
32 1 52 52 1 53 53 40 49 64 0 '(' TRUE (
33 1 45 45 1 52 52 78 50 64 0 expr FALSE
34 1 54 54 1 80 80 260 52 54 0 STR_CONST TRUE "done step %d; proceeding"
35 1 80 80 1 81 81 44 53 64 0 ',' TRUE ,
36 1 54 54 1 80 80 78 54 64 0 expr FALSE
37 1 82 82 1 83 83 263 58 61 0 SYMBOL TRUE i
38 1 84 84 1 85 85 41 60 64 0 ')' TRUE )
39 1 82 82 1 83 83 78 61 64 0 expr FALSE
40 1 45 45 1 85 85 78 64 69 0 expr FALSE
41 1 86 86 1 87 87 41 66 69 0 ')' TRUE )
42 1 38 38 1 87 87 78 69 74 0 expr FALSE
43 1 89 89 1 90 90 125 71 74 0 '}' TRUE }
44 1 17 17 1 90 90 78 74 77 0 expr FALSE
45 1 0 0 1 90 90 78 77 0 0 expr FALSE