R : how to differentiate between inner and innermost brackets using regex - regex

Question

What I need from the string ((VBD)(((JJ))(CC)((RB)(JJ)))((IN)((DT)(JJ)(NNP)(NNPS)))) is this:
"JJ", "RBJJ", "DTJJNNPNNPS", "JJCCRBJJ", "INDTJJNNPNNPS" "VBDJJCCRBJJINDTJJNNPNNPS"
that is, to find the text between innermost brackets, delete the immediately surrounding brackets so that the text can be combined and extracted. But this comprises of different levels. The uncovering of brackets can't be done all at once because the no, of brackets go out of balance:
str1<-c()
str2<-c()
library(gsubfn)
strr<-c("((VBD)(((JJ))(CC)((RB)(JJ)))((IN)((DT)(JJ)(NNP)(NNPS))))")
repeat {
str1<-unlist(strapply(strr, "((\\(([A-Z])+\\))+)"))
str2<-append(str1, str2)
strr<-gsub("(\\(\\w+\\))", "~\\1~", strr)
strr<-gsub("~\\(|\\)~", "", strr)
if (strr == "") {break}
}
strr
[1] "(VBD(JJCCRBJJINDTJJNNPNNPS"
There are brackets left blocking combining of text which makes it escape the regex. The solution to this I think is, to differentiate between innermost brackets (JJ, RB, JJ, DT, JJ, NNP, NNPS, (2, 4, 5, 7 , 8 , 9 , 10 on the fresh string)) and inner brackets. So that when all the inner most brackets are uncovered step by step and the text combined and extracted, we will reach the whole string. Is there a regular expression to do this? Or is there any other way? Please help.

Answer 1

This doesn't use regexp. In fact, I'm not sure that regexp are powerful enough to solve the problem and that a parser is necessary. Rather than create/define a parser in R, I leverage the existing R code parser. Doing so uses some rather potentially dangerous tricks.
The basic idea is to turn the string into parsable code which generates a tree structure using lists. Then this structure is effectively reverse pruned (keeping only the leaf node inward), and the various strings at each level are created.
Some helper packages
library("plotrix")
library("plyr")
The original string that you gave
strr<-c("((VBD)(((JJ))(CC)((RB)(JJ)))((IN)((DT)(JJ)(NNP)(NNPS))))")
Turn this string into parsable code, quoting what is inside the parentheses, and then making each set of parentheses a call to list. Commas have to be inserted between list items, but the innermost parts are always lists of length 1, so that isn't a problem. Then parse the code.
tmp <- gsub("\\(([^\\(\\)]*)\\)", '("\\1")', strr)
tmp <- gsub("\\(", "list(", tmp)
tmp <- gsub("\\)list", "),list", tmp)
tmp <- eval(parse(text=tmp))
At this point, tmp looks like
> str(tmp)
List of 3
$ :List of 1
..$ : chr "VBD"
$ :List of 3
..$ :List of 1
.. ..$ :List of 1
.. .. ..$ : chr "JJ"
..$ :List of 1
.. ..$ : chr "CC"
..$ :List of 2
.. ..$ :List of 1
.. .. ..$ : chr "RB"
.. ..$ :List of 1
.. .. ..$ : chr "JJ"
$ :List of 2
..$ :List of 1
.. ..$ : chr "IN"
..$ :List of 4
.. ..$ :List of 1
.. .. ..$ : chr "DT"
.. ..$ :List of 1
.. .. ..$ : chr "JJ"
.. ..$ :List of 1
.. .. ..$ : chr "NNP"
.. ..$ :List of 1
.. .. ..$ : chr "NNPS"
The nesting of parentheses is now nesting of lists. A few more helper functions are needed. The first collapses everything below a certain depth and throws away any node above that depth. The second is just a wrapper for paste to work one the elements of a list collectively.
atdepth <- function(l, d) {
if (d > 0 & !is.list(l)) {
return(NULL)
}
if (d == 0) {
return(unlist(l))
}
if (is.list(l)) {
llply(l, atdepth, d-1)
}
}
pastelist <- function(l) {paste(unlist(l), collapse="", sep="")}
Create a list where each element is the tree structure collapsed to a particular depth.
down <- llply(1:listDepth(tmp), atdepth, l=tmp)
Iterating backwards over this list, paste the leaf sets together. Work backwards "up" the (collapsed) trees. Doing this produces some blank strings (where there was a leaf higher up), so these are trimmed out.
out <- if (length(down) > 2) {
c(unlist(llply(length(down):3, function(i) {
unlist(do.call(llply, c(list(down[[i]]), replicate(i-3, llply), pastelist)))
})), unlist(pastelist(down[[2]])))
} else {
unlist(pastelist(down[[2]]))
}
out <- out[out != ""]
The result is what I think you asked for:
> out
[1] "JJ" "RBJJ"
[3] "DTJJNNPNNPS" "JJCCRBJJ"
[5] "INDTJJNNPNNPS" "VBDJJCCRBJJINDTJJNNPNNPS"
> dput(out)
c("JJ", "RBJJ", "DTJJNNPNNPS", "JJCCRBJJ", "INDTJJNNPNNPS", "VBDJJCCRBJJINDTJJNNPNNPS"
)
EDIT:
In response to a comment with a subsequent question: How to adapt this to process over a set of these strings.
The general approach to solving the do-it-multiple-times-for-different-inputs is to create a function which takes a single item as input and returns the associated single output. Then loop over the function with one of the apply family of functions.
Pulling together all the code from earlier into a single function:
parsestrr <- function(strr) {
atdepth <- function(l, d) {
if (d > 0 & !is.list(l)) {
return(NULL)
}
if (d == 0) {
return(unlist(l))
}
if (is.list(l)) {
llply(l, atdepth, d-1)
}
}
pastelist <- function(l) {paste(unlist(l), collapse="", sep="")}
tmp <- gsub("\\(([^\\(\\)]*)\\)", '("\\1")', strr)
tmp <- gsub("\\(", "list(", tmp)
tmp <- gsub("\\)list", "),list", tmp)
tmp <- eval(parse(text=tmp))
down <- llply(1:listDepth(tmp), atdepth, l=tmp)
out <- if (length(down) > 2) {
c(unlist(llply(length(down):3, function(i) {
unlist(do.call(llply, c(list(down[[i]]), replicate(i-3, llply), pastelist)))
})), unlist(pastelist(down[[2]])))
} else {
unlist(pastelist(down[[2]]))
}
out[out != ""]
}
Now given a vector of strings to process, say:
strrs<-c("((VBD)(((JJ))(CC)((RB)(JJ)))((IN)((DT)(JJ)(NNP)(NNPS))))",
"((VBD)(((JJ))(CC)((RB)(XX)(JJ)))((IN)(BB)((DT)(JJ)(NNP)(NNPS))))",
"((VBD)(((JJ)(QQ))(CC)((RB)(JJ)))((IN)((TQR)(JJ)(NNPS))))")
You can process all of them with
llply(strr, parsestrr)
which returns
[[1]]
[1] "JJ" "RBJJ"
[3] "DTJJNNPNNPS" "JJCCRBJJ"
[5] "INDTJJNNPNNPS" "VBDJJCCRBJJINDTJJNNPNNPS"
[[2]]
[1] "JJ" "RBXXJJ"
[3] "DTJJNNPNNPS" "JJCCRBXXJJ"
[5] "INBBDTJJNNPNNPS" "VBDJJCCRBXXJJINBBDTJJNNPNNPS"
[[3]]
[1] "JJQQ" "RBJJ"
[3] "TQRJJNNPS" "JJQQCCRBJJ"
[5] "INTQRJJNNPS" "VBDJJQQCCRBJJINTQRJJNNPS"

Answer 2

I'm not sure if you just want to build a tree structure of balanced text or not.
Or, why you want to strip the containing parenthesis on the inner most level.
Using your example, if it is to be done in stages, the inner most level has to be initially determined. Then parenthesis stripped off in subsequent levels in recursive passes.
This of course requires a way to do balanced text. Some regex engines can do this.
If the engine you are using doesn't support this, it would have to be done manually via text processing.
I happen to have a regex analysis program. I pumped your initial string into it and it visually formatted it via group levels. Each pass, I just stripped the inner parenth's which simulates a recursion.
Maybe this can help you to visualize what needs to be done.
## Pass 0
## ---------
(
( VBD )
(
(
( JJ )
)
( CC )
(
( RB )
( JJ )
)
)
(
( IN )
(
( DT )
( JJ )
( NNP )
( NNPS )
)
)
)
## Pass 1
## ---------
(
( VBD )
(
( JJ )
( CC )
( RB JJ )
)
(
( IN )
( DT JJ NNP NNPS )
)
)
## Pass 2
## ---------
(
( VBD )
( JJ CC RB JJ )
( IN DT JJ NNP NNPS )
)
## Pass 3
## ---------
( VBD JJ CC RB JJ IN DT JJ NNP NNPS )
## Pass 4
## ---------
VBD JJ CC RB JJ IN DT JJ NNP NNPS

Answer 3

You don't really need to think of matching brackets here... Sounds like you just want to recursively match the pattern [()]([^()]*)[()].
That is, "match something containing no ( ) and delimited by ( or )"