How to Write Regular Expression Syntax in R - regex

Question

I have a file of log entries that I want to parse through. All the lines look like this:
F 20160602 14:25:11.321 F7982D50 GET 156.145.15.85:37525 xqixh8sl AES "/pcgc/public/Other/exome/fastq/PCGC0077248_HS_EX__1-06808__v3_FCC49HJACXX_L7_p1of1_P1.fastq.gz" "" 3322771022 (0,0) "1499.61 seconds (17.7 megabits/sec)
Each part has a specific designation which I'll put below.
F -- identifier of the line
20160525 -- date (yyyymmdd)
17:52:38.791 -- timestamp (HH:MM:SS.sss)
F798259D -- transfer identifier
156.145.15.85:46634 -- IP address and related port
xqixh8sl -- username
AES -- encryption level (could be - (dash))
"/pcgc...fastq.gz" -- transferred file (in ")
"" -- additional string (should be empty "")
2951144113 -- transferred bytes
(0,0) -- error
"2289.47 seconds (10.3 megabits/sec)" -- data about the transfer
I have imported the data file and am using the read.pattern() function to parse and separate it into it's fields. I only want the pieces of information that correlate with 2,3,4,5,6,7,8,10, and 12. However, I cannot get the pattern right. Before, I managed to get two of the fields that I needed by using this pattern:
pattern <- "^F ([0-9]+) [^ ]* .* \\(0,0\\) (.*)$"
This gave me a data frame that looked like this:
date speed of data transfer
1 20160525 "1.62 seconds (1.30 kilobits/sec)"
2 20160525 "0.29 seconds (1.93 kilobits/sec)"
3 20160525 "0.01 seconds (34.0 kilobits/sec)"
4 20160525 "0.01 seconds (102 kilobits/sec)"
5 20160525 "38.05 seconds (214 megabits/sec)"
These are only two of the fields I need, but whenever I try to add more that's where I mess up the syntax. For example:
pattern <- "^F\\s([0-9]+)\\s[0-9:.]+\\s([:alnum:])\\s[A-Z]\\s([0-9.:]+)\\s([:alnum:])\\s([•])\\s[:punct:][A-z][:punct:]\\s[:punct:]\\s.* \\(0,0\\) (.*)$"
This did not work. Could someone please help on writing this? It's been driving me crazy. Thanks!

Answer 1

Here's my solution:
library(stringer)
con <- readLines("dataSet.txt")
pattern <- "^F (\\d+) ([:graph:]+) ([:graph:]+) [A-Z]+ ([:graph:]+) ([:graph:]+) ([:graph:]+) ([:graph:]+) [:graph:]+ (\\d+) [:graph:]+ (.+)$"
matches <- str_match(con,pattern)
df <- data.frame(na.omit(matches[,-1]))
colnames(df) <- c("date", "timestamp", "transfer ID", "IP address", "username", "encryption level", "transferred file", "transferred bytes", "speed of data transfer")
This was the result:
1 20160525 08:22:06.838 F798256B 10.199.194.38:57708 wei2dt - "" 264 "1.62 seconds (1.30 kilobits/sec)"
2 20160525 08:28:26.920 F798256C 10.19.105.15:57708 wei2dt - "isi_audit_log.dmp-sv.tmp" 69 "0.29 seconds (1.93 kilobits/sec)"

Answer 2

If all of your lines follow a similar structure, you may be able to get away with simply splitting each row on the spaces.
x <- "F 20160602 14:25:11.321 F7982D50 GET 156.145.15.85:37525 xqixh8sl AES \"/pcgc/public/Other/exome/fastq/PCGC0077248_HS_EX__1-06808__v3_FCC49HJACXX_L7_p1of1_P1.fastq.gz\" \"\" 3322771022 (0,0) \"1499.61 seconds (17.7 megabits/sec)"
library(dplyr)
library(magrittr)
strsplit(x, " ") %>%
unlist() %>%
t() %>%
as.data.frame(stringsAsFactors = FALSE) %>%
setNames(c("id", "date", "timestamp", "transfer_id",
"curl_method", "ip_address", "username", "encryption",
"tranferred_file", "additional_string",
"transferred_bytes", "error",
"rate1", "rate2", "rate3", "rate4")) %>%
mutate(rate = paste(rate1, rate2, rate3, rate4)) %>%
select(-rate1:-rate4)