How to Parse and Compare Files? - clojure

Question

I'd appreciate suggestions/insights on how I can leverage Clojure to efficiently parse and compare two files. There are two (log) files that contain employee attendance; from these files I need to determine all the days that two employees worked the same times, in the same department. Below are examples of the log files.
Note: each file has differing number of entries.
First File:
Employee Id Name Time In Time Out Dept.
mce0518 Jon 2011-01-01 06:00 2011-01-01 14:00 ER
mce0518 Jon 2011-01-02 06:00 2011-01-01 14:00 ER
mce0518 Jon 2011-01-04 06:00 2011-01-01 13:00 ICU
mce0518 Jon 2011-01-05 06:00 2011-01-01 13:00 ICU
mce0518 Jon 2011-01-05 17:00 2011-01-01 23:00 ER
Second File:
Employee Id Name Time In Time Out Dept.
pdm1705 Jane 2011-01-01 06:00 2011-01-01 14:00 ER
pdm1705 Jane 2011-01-02 06:00 2011-01-01 14:00 ER
pdm1705 Jane 2011-01-05 06:00 2011-01-01 13:00 ER
pdm1705 Jane 2011-01-05 17:00 2011-01-01 23:00 ER

Answer 1

if you are not going to do it periodically,
(defn data-seq [f]
(with-open [rdr (java.io.BufferedReader.
(java.io.FileReader. f))]
(let [s (rest (line-seq rdr))]
(doall (map seq (map #(.split % "\\s+") s))))))
(defn same-time? [a b]
(let [a (drop 2 a)
b (drop 2 b)]
(= a b)))
(let [f1 (data-seq "f1.txt")
f2 (data-seq "f2.txt")]
(reduce (fn[h v]
(let [f2 (filter #(same-time? v %) f2)]
(if (empty? f2)
h
(conj h [(first v) (map first f2)])))) [] f1)
)
will get you,
[["mce0518" ("pdm1705")] ["mce0518" ("pdm1705")] ["mce0518" ("pdm1705")]]

Answer 2

I came to somewhat shorter and (IMHO) more readable version
(use ; moar toolz - moar fun
'[clojure.contrib.duck-streams :only (reader)]
'[clojure.string :only (split)]
'[clojure.contrib.str-utils :only (str-join)]
'[clojure.set :only (intersection)])
(defn read-presence [filename]
(with-open [rdr (reader filename)] ; file will be securely (always) closed after use
(apply hash-set ; make employee's hash-set
(map #(str-join "--" (drop 2 (split % #" [ ]+"))) ; right-to-left: split row by spaces then forget two first columns then join using "--"
(drop 1 ; ommit first line
(line-seq rdr)))))) ; read file content line-by-line
(intersection (read-presence "a.in") (read-presence "b.in")) ; now it's simple!
;result: #{"2011-01-01 06:00--2011-01-01 14:00--ER" "2011-01-02 06:00--2011-01-01 14:00--ER" "2011-01-05 17:00--2011-01-01 23:00--ER"}
Assuming a.in and b.in are your files. I also assumed you'll have one hash-set for each employee -- (naive) generalization to N employees would need next six lines:
(def employees ["greg.txt" "allison.txt" "robert.txt" "eric.txt" "james.txt" "lisa.txt"])
(for [a employees b employees :when (and
(= a (first (sort [a b]))) ; thou shall compare greg with james ONCE
(not (= a b)))] ; thou shall not compare greg with greg
(str-join " -- " ; well, it's not pretty... nor pink at least
[a b (intersection (read-presence a) (read-presence b))]))
;result: ("a.in -- b.in -- #{\"2011-01-01 06:00--2011-01-01 14:00--ER\" \"2011-01-02 06:00--2011-01-01 14:00--ER\" \"2011-01-05 17:00--2011-01-01 23:00--ER\"}")
Actually this loop is sooo ugly and it doesn't memorize intermediate results... To be improved.
--edit--
I knew there must be something elegant in core or contrib!
(use '[clojure.contrib.combinatorics :only (combinations)])
(def employees ["greg.txt" "allison.txt" "robert.txt" "eric.txt" "james.txt" "lisa.txt"])
(def employee-map (apply conj (for [e employees] {e (read-presence e)})))
(map (fn [[a b]] [a b (intersection (employee-map a) (employee-map b))])
(combinations employees 2))
;result: (["a.in" "b.in" #{"2011-01-01 06:00--2011-01-01 14:00--ER" "2011-01-02 06:00--2011-01-01 14:00--ER" "2011-01-05 17:00--2011-01-01 23:00--ER"}])
Now it's memorized (parsed data in employee-map), general and... lazy :D