How to add column to a data.table in R that is based on a string in another column? - regex

Question

I would like to add columns to a data.table based on a string in another column. This is my data and the approach that I have tried:
Params
1: { clientID : 459; time : 1386868908703; version : 6}
2: { clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}
3: { clientID : 988; time : 1388939739771}
4: { clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}
5: { clientID : 459; time : 1388090530634}
Code to create this table:
DT = data.table(Params=c("{ clientID : 459; time : 1386868908703; version : 6}","{ clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}","{ clientID : 988; time : 1388939739771}","{ clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}","{ clientID : 459; time : 1388090530634}"))
I would like to parse the text in the "Params"-column and create new columns based on the text in it. For example I would like to have a new column named "user" that only holds the number after "user:" in the Params string. The added column should look like this:
Params user
1: { clientID : 459; time : 1386868908703; version : 6} NA
2: { clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001} 459001
3: { clientID : 988; time : 1388939739771} NA
4: { clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001} 459001
5: { clientID : 459; time : 1388090530634} 459001
I created the following function to parse (in this case for the "user"):
myparse <- function(searchterm, s) {
s <-gsub("{","",s, fixed = TRUE)
s <-gsub(" ","",s, fixed = TRUE)
s <-gsub("}","",s, fixed = TRUE)
s <-strsplit(s, '[;:]')
s <-unlist(s)
if (length(s[which(s==searchterm)])>0) {s[which(s==searchterm)+1]} else {NA}
}
Then I use the following function to add a column:
DT <- transform(DT, user = myparse("user", Params))
This works in the case of "time" which is included in all the rows but does not work in the case of "user" which is only included in two of the rows. The following error is returned:
Error in data.table(list(Params = c("{ clientID : 459; time : 1386868908703; version : 6}", :
argument 2 (nrow 2) cannot be recycled without remainder to match longest nrow (5)
How can I address this? Thanks!

Answer 1

Here's a way to use regular expressions for this task:
myparse <- function(searchterm, s) {
res <- rep(NA_character_, length(s)) # NA vector
idx <- grepl(searchterm, s) # index for strings including the search term
pattern <- paste0(".*", searchterm, " : ([^;}]+)[;}].*") # regex pattern
res[idx] <- sub(pattern, "\\1", s[idx]) # extract target string
return(res)
}
You can use this function to add new columns, e.g., for user:
DT[, user := myparse("user", Params)]
The new column contains NA for the rows with no user field:
DT[, user]
# [1] NA "459001" NA "459001" NA

Answer 2

I would use some external parser, for example:
library(yaml)
DT = data.frame(
Params=c("{ clientID : 459; time : 1386868908703; version : 6}","{ clientID : 459; id : 52a9ea8b534b2b0b5000575f; time : 1386868824339; user : 459001}","{ clientID : 988; time : 1388939739771}","{ clientID : 459; id : 52a9ec00b73cbf0b210057e9; time : 1386868810519; user : 459001}","{ clientID : 459; time : 1388090530634}"),
stringsAsFactors=F
)
conv.to.yaml <- function(x){
gsub('; ','\n',substr(x, 3, nchar(x)-1))
}
tmp <- lapply( DT$Params, function(x) yaml.load(conv.to.yaml(x)) )
then combine the parsed lists into data frame:
unames <- unique( unlist(sapply( tmp, names) ) )
res <- as.data.frame( do.call(rbind, lapply(tmp, function(x)x[unames]) ) )
colnames( res ) <- unames
res
the result is pretty much close to what you have in mind, but you need to think about better handling for the time values:
> res
clientID time version id user
1 459 -405527905 6 NULL NULL
2 459 -405612269 NULL 52a9ea8b534b2b0b5000575f 459001
3 988 1665303163 NULL NULL NULL
4 459 -405626089 NULL 52a9ec00b73cbf0b210057e9 459001
5 459 816094026 NULL NULL NULL