Regex group capture in R with multiple capture-groups - regex

In R, is it possible to extract group capture from a regular expression match? As far as I can tell, none of grep, grepl, regexpr, gregexpr, sub, or gsub return the group captures.
I need to extract key-value pairs from strings that are encoded thus:
\((.*?) :: (0\.[0-9]+)\)
I can always just do multiple full-match greps, or do some outside (non-R) processing, but I was hoping I can do it all within R. Is there's a function or a package that provides such a function to do this?

str_match(), from the stringr package, will do this. It returns a character matrix with one column for each group in the match (and one for the whole match):
> s = c("(sometext :: 0.1231313213)", "(moretext :: 0.111222)")
> str_match(s, "\\((.*?) :: (0\\.[0-9]+)\\)")
[,1] [,2] [,3]
[1,] "(sometext :: 0.1231313213)" "sometext" "0.1231313213"
[2,] "(moretext :: 0.111222)" "moretext" "0.111222"

gsub does this, from your example:
gsub("\\((.*?) :: (0\\.[0-9]+)\\)","\\1 \\2", "(sometext :: 0.1231313213)")
[1] "sometext 0.1231313213"
you need to double escape the \s in the quotes then they work for the regex.
Hope this helps.

Try regmatches() and regexec():
regmatches("(sometext :: 0.1231313213)",regexec("\\((.*?) :: (0\\.[0-9]+)\\)","(sometext :: 0.1231313213)"))
[[1]]
[1] "(sometext :: 0.1231313213)" "sometext" "0.1231313213"

gsub() can do this and return only the capture group:
However, in order for this to work, you must explicitly select elements outside your capture group as mentioned in the gsub() help.
(...) elements of character vectors 'x' which are not substituted will be returned unchanged.
So if your text to be selected lies in the middle of some string, adding .* before and after the capture group should allow you to only return it.
gsub(".*\\((.*?) :: (0\\.[0-9]+)\\).*","\\1 \\2", "(sometext :: 0.1231313213)")
[1] "sometext 0.1231313213"

Solution with strcapture from the utils:
x <- c("key1 :: 0.01",
"key2 :: 0.02")
strcapture(pattern = "(.*) :: (0\\.[0-9]+)",
x = x,
proto = list(key = character(), value = double()))
#> key value
#> 1 key1 0.01
#> 2 key2 0.02

This is how I ended up working around this problem. I used two separate regexes to match the first and second capture groups and run two gregexpr calls, then pull out the matched substrings:
regex.string <- "(?<=\\().*?(?= :: )"
regex.number <- "(?<= :: )\\d\\.\\d+"
match.string <- gregexpr(regex.string, str, perl=T)[[1]]
match.number <- gregexpr(regex.number, str, perl=T)[[1]]
strings <- mapply(function (start, len) substr(str, start, start+len-1),
match.string,
attr(match.string, "match.length"))
numbers <- mapply(function (start, len) as.numeric(substr(str, start, start+len-1)),
match.number,
attr(match.number, "match.length"))

I like perl compatible regular expressions. Probably someone else does too...
Here is a function that does perl compatible regular expressions and matches the functionality of functions in other languages that I am used to:
regexpr_perl <- function(expr, str) {
match <- regexpr(expr, str, perl=T)
matches <- character(0)
if (attr(match, 'match.length') >= 0) {
capture_start <- attr(match, 'capture.start')
capture_length <- attr(match, 'capture.length')
total_matches <- 1 + length(capture_start)
matches <- character(total_matches)
matches[1] <- substr(str, match, match + attr(match, 'match.length') - 1)
if (length(capture_start) > 1) {
for (i in 1:length(capture_start)) {
matches[i + 1] <- substr(str, capture_start[[i]], capture_start[[i]] + capture_length[[i]] - 1)
}
}
}
matches
}

As suggested in the stringr package, this can be achieved using either str_match() or str_extract().
Adapted from the manual:
library(stringr)
strings <- c(" 219 733 8965", "329-293-8753 ", "banana",
"239 923 8115 and 842 566 4692",
"Work: 579-499-7527", "$1000",
"Home: 543.355.3679")
phone <- "([2-9][0-9]{2})[- .]([0-9]{3})[- .]([0-9]{4})"
Extracting and combining our groups:
str_extract_all(strings, phone, simplify=T)
# [,1] [,2]
# [1,] "219 733 8965" ""
# [2,] "329-293-8753" ""
# [3,] "" ""
# [4,] "239 923 8115" "842 566 4692"
# [5,] "579-499-7527" ""
# [6,] "" ""
# [7,] "543.355.3679" ""
Indicating groups with an output matrix (we're interested in columns 2+):
str_match_all(strings, phone)
# [[1]]
# [,1] [,2] [,3] [,4]
# [1,] "219 733 8965" "219" "733" "8965"
#
# [[2]]
# [,1] [,2] [,3] [,4]
# [1,] "329-293-8753" "329" "293" "8753"
#
# [[3]]
# [,1] [,2] [,3] [,4]
#
# [[4]]
# [,1] [,2] [,3] [,4]
# [1,] "239 923 8115" "239" "923" "8115"
# [2,] "842 566 4692" "842" "566" "4692"
#
# [[5]]
# [,1] [,2] [,3] [,4]
# [1,] "579-499-7527" "579" "499" "7527"
#
# [[6]]
# [,1] [,2] [,3] [,4]
#
# [[7]]
# [,1] [,2] [,3] [,4]
# [1,] "543.355.3679" "543" "355" "3679"

This can be done using the package unglue, taking the example from the selected answer:
# install.packages("unglue")
library(unglue)
s <- c("(sometext :: 0.1231313213)", "(moretext :: 0.111222)")
unglue_data(s, "({x} :: {y})")
#> x y
#> 1 sometext 0.1231313213
#> 2 moretext 0.111222
Or starting from a data frame
df <- data.frame(col = s)
unglue_unnest(df, col, "({x} :: {y})",remove = FALSE)
#> col x y
#> 1 (sometext :: 0.1231313213) sometext 0.1231313213
#> 2 (moretext :: 0.111222) moretext 0.111222
you can get the raw regex from the unglue pattern, optionally with named capture :
unglue_regex("({x} :: {y})")
#> ({x} :: {y})
#> "^\\((.*?) :: (.*?)\\)$"
unglue_regex("({x} :: {y})",named_capture = TRUE)
#> ({x} :: {y})
#> "^\\((?<x>.*?) :: (?<y>.*?)\\)$"
More info : https://github.com/moodymudskipper/unglue/blob/master/README.md

Related

Extract value from character array in R

I have the following charecter array
head(rest, n=20)
[,1] [,2] [,3]
[1,] "" "" ""
[2,] "" "" ""
[3,] "B" "-1" "-tv"
[4,] "" "" ""
[5,] "" "" ""
[6,] "A" "" ""
[7,] "" "" ""
...
[2893,] "" "" ""
[2894,] "" "" ""
[2895,] "" "" ""
[2896,] "st" "" ""
[2897,] "2" "-th" ""
[2898,] "1" "" ""
I would like to extract the all capital letters, all numbers and all lower case letter while keeping the index values.
I can find all the capital positions letters with this
grep("[A-Z]", rest, perl=TRUE)
and the values with
grep("[A-Z]", rest, perl=TRUE, value=TRUE)
But I can't figure out how to return the value while keeping the index.
I think this might be what you're looking for (using your example data):
rest <- matrix(c('','','','','','','B','-1','-tv','','','','','','','A','','','','','','','','','','','','','','','st','','','2','-th','','1','',''),13,byrow=T);
pat <- c('[A-Z]','[0-9]','[a-z]');
name <- c('house','floor','side');
res <- setNames(as.data.frame(lapply(pat,function(x) { i <- grep(x,rest); x <- rep('',nrow(rest)); x[(i-1)%%nrow(rest)+1] <- rest[i]; x; }),stringsAsFactors=F),name);
res;
## house floor side
## 1
## 2
## 3 B -1 -tv
## 4
## 5
## 6 A
## 7
## 8
## 9
## 10
## 11 st
## 12 2 -th
## 13 1
Actually that's not a great demo because of a dearth of populated cells, here's some randomized data for another demo:
set.seed(9);
R <- 12;
C <- 3;
N <- 5;
rest <- matrix(sample(c(rstr(N,charset=letters,lmin=1,lmax=3),rstr(N,charset=LETTERS,lmin=1,lmax=3),rstr(N,charset=0:9,lmin=1,lmax=3),rep('',R*C-N*3))),R);
rest;
## [,1] [,2] [,3]
## [1,] "AN" "" ""
## [2,] "895" "" ""
## [3,] "698" "" ""
## [4,] "zd" "" "32"
## [5,] "" "" ""
## [6,] "CK" "" ""
## [7,] "" "" ""
## [8,] "JWZ" "" "r"
## [9,] "1" "j" "IX"
## [10,] "" "" "ZFM"
## [11,] "k" "d" ""
## [12,] "" "" "252"
pat <- c('[A-Z]','[0-9]','[a-z]');
name <- c('house','floor','side');
res <- setNames(as.data.frame(lapply(pat,function(x) { i <- grep(x,rest); x <- rep('',R); x[(i-1)%%R+1] <- rest[i]; x; }),stringsAsFactors=F),name);
res;
## house floor side
## 1 AN
## 2 895
## 3 698
## 4 32 zd
## 5
## 6 CK
## 7
## 8 JWZ r
## 9 IX 1 j
## 10 ZFM
## 11 d
## 12 252
Note that I used a little function I wrote called rstr() to produce the random string values. It's not relevant to this question so I haven't posted it, but if you want I can provide it as well in this answer.
By chance in row 11 there's a collision between two side values. You specified in the comments that this can't happen in your actual data, but you can see from the output that the code handles that case gracefully; it ends up keeping the rightmost value that was in the row.
The new requirement of moving single-letter lowercase strings from the third column to the first, concatenating with any existing value in the first column, can be satisfied thusly (continuing with my second demo):
res$house <- ifelse(nchar(res$side)==1,paste0(res$house,res$side),res$house);
res$side <- ifelse(nchar(res$side)==1,'',res$side);
res;
## house floor side
## 1 AN
## 2 895
## 3 698
## 4 32 zd
## 5
## 6 CK
## 7
## 8 JWZr
## 9 IXj 1
## 10 ZFM
## 11 d
## 12 252

Faster way to capture regex

I want to use regex to capture substrings - I already have a working solution, but I wonder if there is a faster solution. I am applying applyCaptureRegex on a vector with about 400.000 entries.
exampleData <- as.data.frame(c("[hg19:21:34809787-34809808:+]","[hg19:11:105851118-105851139:+]","[hg19:17:7482245-7482266:+]","[hg19:6:19839915-19839936:+]"))
captureRegex <- function(captRegEx,str){
sapply(regmatches(str,gregexpr(captRegEx,str))[[1]], function(m) regmatches(m,regexec(captRegEx,m)))
}
applyCaptureRegex <- function(mir,r){
mir <- unlist(apply(mir, 1, function(x) captureRegex(r,x[1])))
mir <- matrix(mir ,ncol=5, byrow = TRUE)
mir
}
Usage and results:
> captureRegex("\\[[a-z0-9]+:([0-9]+):([0-9]+)-([0-9]+):([-+])\\]","[hg19:12:125627828-125627847:-]")
$`[hg19:12:125627828-125627847:-]`
[1] "[hg19:12:125627828-125627847:-]" "12" "125627828" "125627847" "-"
> applyCaptureRegex(exampleData,"\\[[a-z0-9]+:([0-9]+):([0-9]+)-([0-9]+):([-+])\\]")
[,1] [,2] [,3] [,4] [,5]
[1,] "[hg19:21:34809787-34809808:+]" "21" "34809787" "34809808" "+"
[2,] "[hg19:11:105851118-105851139:+]" "11" "105851118" "105851139" "+"
[3,] "[hg19:17:7482245-7482266:+]" "17" "7482245" "7482266" "+"
[4,] "[hg19:6:19839915-19839936:+]" "6" "19839915" "19839936" "+"
Thank you!
Why reinvent the wheel? You have several library packages to choose from with functions that return a character matrix with one column for each capturing group in your pattern.
stri_match_all_regex — stringi
x <- c('[hg19:21:34809787-34809808:+]', '[hg19:11:105851118-105851139:+]', '[hg19:17:7482245-7482266:+]', '[hg19:6:19839915-19839936:+]')
do.call(rbind, stri_match_all_regex(x, '\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])]'))
# [,1] [,2] [,3] [,4] [,5]
# [1,] "[hg19:21:34809787-34809808:+]" "21" "34809787" "34809808" "+"
# [2,] "[hg19:11:105851118-105851139:+]" "11" "105851118" "105851139" "+"
# [3,] "[hg19:17:7482245-7482266:+]" "17" "7482245" "7482266" "+"
# [4,] "[hg19:6:19839915-19839936:+]" "6" "19839915" "19839936" "+"
str_match — stringr
str_match(x, '\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])]')
strapplyc — gsubfn
strapplyc(x, "(\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])])", simplify = rbind)
Below is a benchmark comparison of all combined solutions.
x <- rep(c('[hg19:21:34809787-34809808:+]',
'[hg19:11:105851118-105851139:+]',
'[hg19:17:7482245-7482266:+]',
'[hg19:6:19839915-19839936:+]'), 1000)
applyCaptureRegex <- function(mir, r) {
do.call(rbind, lapply(mir, function(x) regmatches(x, regexec(r, x))[[1]]))
}
gsubfn <- function(x1) strapplyc(x1, '(\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])])', simplify = rbind)
regmtch <- function(x1) applyCaptureRegex(x1, '\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])]')
stringr <- function(x1) str_match(x1, '\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])]')
stringi <- function(x1) do.call(rbind, stri_match_all_regex(x1, '\\[[^:]+:(\\d+):(\\d+)-(\\d+):([-+])]'))
require(microbenchmark)
microbenchmark(gsubfn(x), regmtch(x), stringr(x), stringi(x))
Result
Unit: milliseconds
expr min lq mean median uq max neval
gsubfn(x) 372.27072 382.82179 391.21837 388.32396 396.27361 449.03091 100
regmtch(x) 394.03164 409.87523 419.42936 417.76770 427.08208 456.92460 100
stringr(x) 65.81644 70.28327 76.02298 75.43162 78.92567 116.18026 100
stringi(x) 15.88171 16.53047 17.52434 16.96127 17.76007 23.94449 100

Regular expression matching and replacement R

I am attempting to write a regular expression that replaces each element in this matrix with only the two numbers after the first colon before and after the comma. There is also "./.:.:.:.:." which I would like to change to "0,0".
head(data)
Offspring-95_CAATCG Offspring-96_AAACGG Offspring-97_ACTCTT
[1,] "./.:1,7:8:18:262,0,18" "0/1:18,4:21:56:56,0,591" "0/0:27,0:27:78:0,78,723"
[2,] "0/0:49,0:49:99:0,147,1891" "0/0:107,0:107:99:0,319,4185" "1/1:0,22:22:66:902,66,0"
[3,] "0/0:42,0:42:99:0,126,1324" "./.:.:.:.:." "0/1:35,88:117:99:3152,0,718"
I have tried:
try <- gsub("\\:[0-9]*\\,[0-9]*\\:", \\1, data)
The desired output is:
Offspring-95_CAATCG Offspring-96_AAACGG Offspring-97_ACTCTT
[1,] "1,7" "18,4" "27,0"
[2,] "49,0" "107,0" "0,22"
[3,] "42,0" "0,0" "35,88"
Thanks,
This could be done by
sub('[^:]+:([^:]+).*', '\\1', data)
# Offspring.95_CAATCG Offspring.96_AAACGG Offspring.97_ACTCTT
#[1,] "1,7" "18,4" "27,0"
#[2,] "49,0" "107,0" "0,22"
#[3,] "9,4" "33,13" "13,0"
Visualization
[^:]+:([^:]+).*
Debuggex Demo
Or using regmatches from base R
data[] <- regmatches(data, regexpr('(?<=:)[0-9]+,[0-9]+', data, perl=TRUE))
Visualization
(?<=:)[0-9]+,[0-9]+
Debuggex Demo
The above regex can be used with stringr or stringi (for big dataset)
library(stringr)
`dim<-`(str_extract(data, perl('(?<=:)[0-9]+,[0-9]+')), dim(data))
# [,1] [,2] [,3]
#[1,] "1,7" "18,4" "27,0"
#[2,] "49,0" "107,0" "0,22"
#[3,] "9,4" "33,13" "13,0"
Or
library(stringi)
`dim<-`(stri_extract(data, regex='(?<=:)[0-9]+,[0-9]+'), dim(data))
# [,1] [,2] [,3]
#[1,] "1,7" "18,4" "27,0"
#[2,] "49,0" "107,0" "0,22"
#[3,] "9,4" "33,13" "13,0"
Update
data1[] <- sub('[^:]+:([^:]+).*', '\\1', data1)
data1[!grepl(',', data1)] <- '0,0'
data1
# Offspring.95_CAATCG Offspring.96_AAACGG Offspring.97_ACTCTT
#[1,] "1,7" "18,4" "27,0"
#[2,] "49,0" "107,0" "0,22"
#[3,] "42,0" "0,0" "35,88"
data
data <- structure(c("./.:1,7:8:18:262,0,18", "0/0:49,0:49:99:0,147,1891",
"0/1:9,4:13:99:129,0,334", "0/1:18,4:21:56:56,0,591",
"0/0:107,0:107:99:0,319,4185",
"0/1:33,13:44:99:317,0,1150", "0/0:27,0:27:78:0,78,723",
"1/1:0,22:22:66:902,66,0", "0/0:13,0:13:39:0,39,528"), .Dim = c(3L, 3L),
.Dimnames = list(NULL, c("Offspring.95_CAATCG", "Offspring.96_AAACGG",
"Offspring.97_ACTCTT")))
data1 <- structure(c("./.:1,7:8:18:262,0,18", "0/0:49,0:49:99:0,147,1891",
"0/0:42,0:42:99:0,126,1324", "0/1:18,4:21:56:56,0,591",
"0/0:107,0:107:99:0,319,4185",
"./.:.:.:.:.", "0/0:27,0:27:78:0,78,723", "1/1:0,22:22:66:902,66,0",
"0/1:35,88:117:99:3152,0,718"), .Dim = c(3L, 3L), .Dimnames = list(
NULL, c("Offspring.95_CAATCG", "Offspring.96_AAACGG", "Offspring.97_ACTCTT"
)))
Not regex subbing but probably pretty darn quick.
apply(data, 2, function(x) sapply(strsplit(x, ":"), "[[", 2))
## Offspring.95_CAATCG Offspring.96_AAACGG Offspring.97_ACTCTT
## [1,] "1,7" "18,4" "27,0"
## [2,] "49,0" "107,0" "0,22"
## [3,] "9,4" "33,13" "13,0"
Try this:
out<-list()
for(i in seq(ncol(data)))
out[[i]]<-gsub('[^:]*:([0-9]+,[0-9]+).*','\\1',data[,i])
out<-as.data.frame(out)
dimnames(out)<-dimnames(data)
out

Pasting character vectors, removing NA's and separators between NAs

I've got several character vectors that I want to paste together. The problem is that some of the character vectors are pretty sparse. So, when I paste them, I get NA's and extra separators. How can I efficiently remove the NA's and extra separators while still joining the vectors?
I've got something like:
n1 = c("goats", "goats", "spatula", NA, "rectitude", "boink")
n2 = c("forever", NA, "...yes", NA, NA, NA)
cbind(paste(n1,n2, sep=", "))
which gives me:
[1,] "goats, forever"
[2,] "goats, NA"
[3,] "spatula, ...yes"
[4,] "NA, NA"
[5,] "rectitude, NA"
[6,] "boink, NA"
but I want:
[1,] "goats, forever"
[2,] "goats"
[3,] "spatula, ...yes"
[4,] <NA>
[5,] "rectitude"
[6,] "boink"
There are clearly inefficient and tedious ways of doing this with a lot of regular expressions and string splitting. But anything quick/simple?
Not a lot of regex, just 1 line and 1 more to replace NA
n1 <- c("goats", "goats", "spatula", NA, "rectitude", "boink")
n2 <- c("forever", NA, "...yes", NA, NA, NA)
n3 <- cbind(paste(n1,n2, sep=", "))
n3 <- gsub("(, )?NA", "", n3)
n3[n3==""] <- NA
Code (no regex or string splitting):
vec <- apply(cbind(n1,n2),1,function(x)
ifelse(all(is.na(x)), NA, paste(na.omit(x),collapse=", ")) )
Result:
> vec # as a vector
[1] "goats, forever" "goats" "spatula, ...yes" NA "rectitude" "boink"
> cbind(vec) # as a matrix
vec
[1,] "goats, forever"
[2,] "goats"
[3,] "spatula, ...yes"
[4,] NA
[5,] "rectitude"
[6,] "boink"
Here's an option using the qdap package (though the other options seem better to me as they use base R):
library(qdap)
gsub(" ", ", ", blank2NA(Trim(gsub("NA", "", paste(n1, n2)))))
## [1] "goats, forever" "goats" "spatula, ...yes" NA
## [5] "rectitude" "boink"
Or...
## gsub(" ", ", ", blank2NA(gsub("NA| NA", "", paste(n1, n2))))

Combine rownames from different lists in a dataframe

I have a question about lists in R. I have a list within 16 list containing a list with variables like this:
x
[[1]]
A 1 3
B 4 2
[[2]]
C 23 4
D 9 22
E 4 54
The A,B,C and D are rownames in the lists. Now I want to create a file that paste only the rownames in a dataframe. Each row in the dataframe contains 1 list in the total list.
A B
C D E
Can anyone help me with this? I thought maybe someting like do.call(rbind, rownames(x))
EDIT! 05-08-2011
Is there a way to save the rownames list by list? So in the end there are no NA's in the data and the data is unequal?
Thank you all!
Making an assumption about the nature of x, if we use:
x <- list(matrix(c(1,4,3,2), ncol = 2,
dimnames = list(c("A","B"), NULL)),
matrix(c(23,9,4,4,22,54), ncol = 2,
dimnames = list(c("C","D","E"), NULL)))
which gives:
> x
[[1]]
[,1] [,2]
A 1 3
B 4 2
[[2]]
[,1] [,2]
C 23 4
D 9 22
E 4 54
Then
> lapply(x, rownames)
[[1]]
[1] "A" "B"
[[2]]
[1] "C" "D" "E"
seems the only plausible answer. Unless we pad the ("A","B") vector with something, we can't use a matrix or a data frame because the component lengths do not match. Hence one of the reasons the do.call() idea fails:
> do.call(rbind, rownames(x))
Error in do.call(rbind, rownames(x)) : second argument must be a list
> do.call(rbind, lapply(x, rownames))
[,1] [,2] [,3]
[1,] "A" "B" "A"
[2,] "C" "D" "E"
Warning message:
In function (..., deparse.level = 1) :
number of columns of result is not a multiple of vector length (arg 1)
To pad the result with NA and get a data frame, we could do:
out <- lapply(x, rownames)
foo <- function(x, max, repl = NA) {
if(length(x) == max)
out <- x
else {
out <- rep(repl, max)
out[seq_along(x)] <- x
}
out
}
out <- lapply(out, foo, max = max(sapply(out, length)))
(out <- do.call(rbind, out))
The last line gives:
> (out <- do.call(rbind, out))
[,1] [,2] [,3]
[1,] "A" "B" NA
[2,] "C" "D" "E"
If you want that nicely printed, then
> print(format(out), quote = FALSE)
[,1] [,2] [,3]
[1,] A B NA
[2,] C D E
is an option inside R.
This should do it:
lapply(x, function(curdfr){paste(rownames(curdfr))})
This results in a vector with each element the space-separated rownames of the elements of the list.
Your sample data:
x <- list(
matrix(c(1,4,3,2), nrow = 2, dimnames = list(LETTERS[1:2])),
matrix(c(23,9,4,4,22,54), nrow = 3, dimnames = list(LETTERS[3:5]))
)
What you want:
unlist(lapply(x, rownames))
Or, if you are keen on do.call, then this is equivalent:
do.call(c, lapply(x, rownames))