Matching and pasting with data frames in R - regex

Question

I'm having trouble with a match and paste problem. I have a data frame like
df
# X1 X2 X3 X4 X5 X6
#t1 <NA> <NA> AU 78 <NA> <NA>
#t2 dA AK <NA> <NA> 5 <NA>
#t3 ip <NA> <NA> <NA> <NA> <NA>
#t4 <NA> <NA> <NA> <NA> <NA> BA
I want it to look like this after operations,
newdf
# X1 X2 X3 X4 X5 X6
#v1 <NA> <NA> <NA> <NA> <NA> <NA>
#v2 AU78 <NA> <NA> <NA> <NA> <NA>
#v3 AK5 <NA> <NA> <NA> <NA> <NA>
#v4 <NA> <NA> <NA> <NA> <NA> BA
The process should first search for values that start with 'A'. df[1,3], df[2,2] in this case. Then paste that value to any other numbers further to the right of it (there will always be one number to the right of it). Also, to help, there will never be stray characters in between a target element (like 'AK') and the number to the right of it; only NAs will seperate them.
Those combined new values need to be brought to the first column, and one row down from where it was. It does not matter if values existing in the first row are overwritten.
My pattern locator is,
pat.locate <- lapply(df, function(x) grep('^A', x))
un.pat <- unlist(pat.locate)
#X2 X3
# 2 1
This looked like a good start. From there,
df[un.pat, names(un.pat)]
# X2 X3
#t2 AK <NA>
#t1 <NA> AU
So the target values are found with their column and row indexes. But I need the values to the right of those indexes. To subset the entire rows,
full.row <- df[un.pat, ]
# X1 X2 X3 X4 X5 X6
#t2 dA AK <NA> <NA> 5 <NA>
#t1 <NA> <NA> AU 78 <NA> <NA>
I paste the non-NA values, but you can tell what's going to happen,
paste(full.row[!is.na(full.row)], collapse='')
#[1] "dAAKAU785"
To divide it up, an apply over the rows was used:
pasty <- function(x) paste(x[!is.na(x)], collapse='')
pasted.rows <- apply(full.row, 1, pasty)
# t2 t1
#"dAAK5" "AU78"
That still leaves the stray string at the beginning. If I found a good regex to tell it to cast that off I'd have,
good.regex
# t2 t1
# "AK5" "AU78"
I could then subset the whole data frame based on those indices with,
df[names(good.regex), 1] <- good.regex
df
# X1 X2 X3 X4 X5 X6
#t1 AU78 <NA> AU 78 <NA> <NA>
#t2 AK5 AK <NA> <NA> 5 <NA>
#t3 ip <NA> <NA> <NA> <NA> <NA>
#t4 <NA> <NA> <NA> <NA> <NA> BA
But I'm still left with having to move the pasted values down by one.
df[names(good.regex)+1, 1] <- good.regex
#Error in names(good.regex) + 1 : non-numeric argument to binary operator
We obviously can't add a numeric to a named-style subset. I feel like I'm missing some element early on that's leading me down a difficult path to a solution. A regex would have to be a sub out that uses the pattern match and a look-behind that I can't crack. I think I'm working myself into a corner that is unnecessary. Any help is appreciated.
Data
df <- structure(list(X1 = c(NA, "dA", "ip", NA), X2 = c(NA, "AK", NA,
NA), X3 = c("AU", NA, NA, NA), X4 = c("78", NA, NA, NA), X5 = c(NA,
"5", NA, NA), X6 = c(NA, NA, NA, "BA")), .Names = c("X1", "X2",
"X3", "X4", "X5", "X6"), row.names = c("t1", "t2", "t3", "t4"
), class = "data.frame")
newdf <- structure(list(X1 = structure(c(NA, 2L, 1L, NA), .Names = c("v1",
"v2", "v3", "v4"), .Label = c("AK5", "AU78"), class = "factor"),
X2 = structure(c(NA_integer_, NA_integer_, NA_integer_, NA_integer_
), .Names = c("v1", "v2", "v3", "v4"), .Label = character(0), class = "factor"),
X3 = structure(c(NA_integer_, NA_integer_, NA_integer_, NA_integer_
), .Names = c("v1", "v2", "v3", "v4"), .Label = character(0), class = "factor"),
X4 = structure(c(NA_integer_, NA_integer_, NA_integer_, NA_integer_
), .Names = c("v1", "v2", "v3", "v4"), .Label = character(0), class = "factor"),
X5 = structure(c(NA_integer_, NA_integer_, NA_integer_, NA_integer_
), .Names = c("v1", "v2", "v3", "v4"), .Label = character(0), class = "factor"),
X6 = structure(c(NA, NA, NA, 1L), .Names = c("v1", "v2",
"v3", "v4"), .Label = "BA", class = "factor")), .Names = c("X1",
"X2", "X3", "X4", "X5", "X6"), row.names = c("v1", "v2", "v3",
"v4"), class = "data.frame")

Answer 1

For what I understand according to your output example, the point is to collapse a A* character and its following number in the same row, then move this new entity down to the first column one row below. While "erasing" the original line (row 1 of newdf filled with NA) but keeping lines with no-match intact if they're not affected by the previous movement (row 4).
Your main problem was to collapse on the full row, instead of collapsing only its end.
## original data
df <- structure(list(X1 = c(NA, "dA", "ip", NA),
X2 = c(NA, "AK", NA, NA),
X3 = c("AU", NA, NA, NA),
X4 = c("78", NA, NA, NA),
X5 = c(NA, "5", NA, NA),
X6 = c(NA, NA, NA, "BA")),
.Names = c("X1", "X2", "X3", "X4", "X5", "X6"),
row.names = c("t1", "t2", "t3", "t4"), class = "data.frame")
df
X1 X2 X3 X4 X5 X6
t1 <NA> <NA> AU 78 <NA> <NA>
t2 dA AK <NA> <NA> 5 <NA>
t3 ip <NA> <NA> <NA> <NA> <NA>
t4 <NA> <NA> <NA> <NA> <NA> BA
This following function grab the rows with the matching pattern but collapse only from this pattern to the end of the row, while forgetting its beginning. Thus avoiding the problem encountered with non matching stray character (the dA of your example) :
locateAndPaste <- function(x){
if(TRUE %in% grepl('^A', df[x,])){
endRow <- df[x, grep('^A', df[x,]):length(df)]
pasted.rows <- paste(endRow[!is.na(endRow)], collapse='')
}
else{NA}
}
The else element prevents throwing out errors if no match is found.
newEntity <- sapply(1:nrow(df), locateAndPaste)
# [1] "AU78" "AK5" NA NA
Two matching pattern have been found in row 1 and 2 and none in row 3 and 4.
As you can see the collapsing part worked perfectly.
Your second problem was to move one row down, and the impossibility of adding number to a character string. As I'm not subsetting on the names but on the indexes, the problem is easily avoided:
(in order to be complete, I've added a line at the end of this post regarding the conversion to numeric of those names)
## the newEntity element is already ordered according to the original row numbers
originalRowNumbers <- grep("^A", newEntity)
# [1] 1 2
From then, it's pretty straight forward :
newdf <- df ## all operations can be done on the original df,
## this copy is made only for the sake of the example.
## as per your example, "erase" the original lines where a matching pattern was found
## that will also prevent orphan lines if a no match have been found in the above line
newdf[originalRowNumbers, ] <- rep(NA, length(df))
## place the new entity in the first column one row below
newdf[originalRowNumbers+1, 1] <- newEntity[originalRowNumbers]
## fill the rest of this row with NA as per your example
newdf[originalRowNumbers+1, 2:length(df)] <- NA
newdf
X1 X2 X3 X4 X5 X6
t1 <NA> <NA> <NA> <NA> <NA> <NA>
t2 AU78 <NA> <NA> <NA> <NA> <NA>
t3 AK5 <NA> <NA> <NA> <NA> <NA>
t4 <NA> <NA> <NA> <NA> <NA> BA
However, if a matching pattern were to be found in the last row, an extra row will be added to newdf. In order to avoid that, it's possible to shorten the initial selection :
newEntity <- sapply(1:(nrow(df)-1), locateAndPaste)
To be complete : in your example it's possible to grab only the number in the names of good.regex and then feed them to your subset :
idx.goood.regex <- as.numeric(gsub("t","", names(good.regex)))
# [1] 2 1
df[idx.good.regex+1, 1] <- good.regex
Note that only works because good.regex is of class character. An error would occur if good.regex was a data.frame.