R: regex from string to two dimensional data frame in one command? - regex

Question

I have a string s containing such key-value pairs, and I would like to construct from it data frame,
s="{'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}"
r1<-sapply(strsplit(s, "[^0-9_]+",as.numeric),as.numeric)
r2<-sapply(strsplit(s, "[^A-Z]+",as.numeric),as.character)
d<-data.frame(id=r2,value=r1)
what gives:
r1
[,1]
[1,] NA
[2,] 121
[3,] 938
[4,] 184
[5,] 338
[6,] 52
r2
[,1]
[1,] ""
[2,] "JJ"
[3,] "NN"
[4,] "DT"
[5,] "VB"
[6,] "RB"
d
id value
1 NA
2 JJ 121
3 NN 938
4 DT 184
5 VB 338
6 RB 52
First I would like don't have NA and "" after using regular expression. I think it should be something like {2,} meaning match all from second occurence, but I can not do that in R.
Another think I would like to do will be: having a data frame with column like below:
m
1 {'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}
2 {'#NN': 168, '#DT': 59, '#VB': 71, '#RB': 5, '#JJ': 35}
3 {'#JJ': 18, '#NN': 100, '#DT': 23, '#VB': 52, '#RB': 11}
4 {'#NN': 156, '#JJ': 39, '#DT': 46, '#VB': 67, '#RB': 21}
5 {'#NN': 112, '#DT': 39, '#VB': 57, '#RB': 8, '#JJ': 32}
6 {'#DT': 236, '#NN': 897, '#VB': 420, '#RB': 122, '#JJ': 240}
7 {'#NN': 316, '#RB': 25, '#DT': 66, '#VB': 112, '#JJ': 81}
8 {'#NN': 198, '#DT': 29, '#VB': 85, '#RB': 37, '#JJ': 44}
9 {'#RB': 30}
10 {'#NN': 373, '#DT': 48, '#VB': 71, '#RB': 21, '#JJ': 36}
11 {'#NN': 49, '#DT': 17, '#VB': 23, '#RB': 11, '#JJ': 8}
12 {'#NN': 807, '#JJ': 135, '#DT': 177, '#VB': 315, '#RB': 69}
I would like to iterate over each row and split it numerical values into the columns named by the key.
Example of few rows showing, how I would like it will looks like:

Answer 1

I would use something that parses JSON, what your data seems to be:
s <- "{'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}"
parse.one <- function(s) {
require(rjson)
v <- fromJSON(gsub("'", '"', s))
data.frame(id = gsub("#", "", names(v)),
value = unlist(v, use.names = FALSE))
}
parse.one(s)
# id value
# 1 JJ 121
# 2 NN 938
# 3 DT 184
# 4 VB 338
# 5 RB 52
For the second part of the question, I would pass a slightly modified version of the parse.one function through lapply, then let plyr's rbind.fill function align the pieces together while filling missing values with NA:
df <- data.frame(m = c(
"{'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}",
"{'#NN': 168, '#DT': 59, '#VB': 71, '#RB': 5, '#JJ': 35}",
"{'#JJ': 18, '#NN': 100, '#DT': 23, '#VB': 52, '#RB': 11}",
"{'#JJ': 12, '#VB': 5}"
))
parse.one <- function(s) {
require(rjson)
y <- fromJSON(gsub("'", '"', s))
names(y) <- gsub("#", "", names(y))
as.data.frame(y)
}
library(plyr)
rbind.fill(lapply(df$m, parse.one))
# JJ NN DT VB RB
# 1 121 938 184 338 52
# 2 35 168 59 71 5
# 3 18 100 23 52 11
# 4 12 NA NA 5 NA

Answer 2

For now, I'll offer a solution to the first part of your question. Clean up your string and use read.table:
s="{'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}"
read.table(text = gsub(",", "\n", gsub("[{|}|#]", "", s)),
header = FALSE, sep = ":", strip.white=TRUE)
# V1 V2
# 1 JJ 121
# 2 NN 938
# 3 DT 184
# 4 VB 338
# 5 RB 52
For the second part, here's another alternative using concat.split from a package I wrote called "splitstackshape":
Sample data:
df <- data.frame(m = c(
"{'#JJ': 121, '#NN': 938, '#DT': 184, '#VB': 338, '#RB': 52}",
"{'#NN': 168, '#DT': 59, '#VB': 71, '#RB': 5, '#JJ': 35}",
"{'#JJ': 18, '#NN': 100, '#DT': 23, '#VB': 52, '#RB': 11}"
))
Similar cleanup as above, plus add an "id" column.
df$m <- gsub("[{|}|#]", "", df$m)
df$id <- 1:nrow(df)
Load the "splitstackshape" package:
# install.packages("splitstackshape")
library(splitstackshape)
df2 <- concat.split(concat.split.multiple(df, "m", ",", "long"),
"m", ":", drop = TRUE)
## df2 <- df2[complete.cases(df2), ] ##
## ^^ might be necessary if there are NAs in the resulting data.frame
The data are now in a "long" format that is easy to manipulate:
df2
# id time m_1 m_2
# 1 1 1 JJ 121
# 2 2 1 NN 168
# 3 3 1 JJ 18
# 4 1 2 NN 938
# 5 2 2 DT 59
# 6 3 2 NN 100
# 7 1 3 DT 184
# 8 2 3 VB 71
# 9 3 3 DT 23
# 10 1 4 VB 338
# 11 2 4 RB 5
# 12 3 4 VB 52
# 13 1 5 RB 52
# 14 2 5 JJ 35
# 15 3 5 RB 11
Here's an example of manipulating the data, using dcast from the "reshape2" package:
library(reshape2)
dcast(df2, id ~ m_1, value.var="m_2")
# id DT JJ NN RB VB
# 1 1 184 121 938 52 338
# 2 2 59 35 168 5 71
# 3 3 23 18 100 11 52