Missed elements in list subtraction proc - list

Question

I have a list of instances from a database, and I am accounting for some number of instances. I am writing a proc which will give me the number of instances which have not yet been accounted for; doing so by 'subtracting' the lists from each other. I am seeing some instances show up in the full db list, the accounted for list and still showing up in the missed list, which implies to me that by subtraction portion of the code is wrong.
Moreover the database has around 31000 elements and almost all are accounted for except 300, there are only around 8 which are double counted which seems odd. Here is my subtraction proc which reads from 2 files and writes to another.
Here is the code:
proc checkDif {} {
set fp [open "accounted_db.tcl" r]
set file_data_accounted [read $fp]
close $fp
set fp [open "innovus_db.tcl" r]
set file_data_db [read $fp]
close $fp
foreach elem $file_data_db {dict set y $elem 1}
foreach elem $file_data_accounted {dict unset y $elem}
set res [dict keys $y]
set fileName "missed_db.tcl"
set fileId [open $fileName "w"]
foreach inst $res {
puts $fileId $inst
}
close $fileId
return [llength $res]
}

Answer 1

If your data is one key per line (as it is in your output) and could contain spaces but not newlines (in the keys) then you should change your input code to do:
set fp [open "accounted_db.tcl" r]
set file_data_accounted [split [read $fp] "\n"]
close $fp
set fp [open "innovus_db.tcl" r]
set file_data_db [split [read $fp] "\n"]
close $fp
In fact, it's probably best to factor out that little code sequence:
proc readLines {filename} {
set fp [open $filename]
set lines [split [read $fp] "\n"]
close $fp
return $lines
}
# Might as well factor out an efficient reverse operation at the same time
proc writeLines {filename lines} {
set fp [open $filename "w"]
puts $fp [join $lines "\n"]
close $fp
}
proc checkDif {} {
set file_data_accounted [readLines "accounted_db.tcl"]
set file_data_db [readLines "innovus_db.tcl"]
set y {}; # In case file_data_db is empty; unlikely, but paranoia is good…
foreach elem $file_data_db {
dict set y $elem 1
}
foreach elem $file_data_accounted {
dict unset y $elem
}
set res [dict keys $y]
writeLines "missed_db.tcl" $res
return [llength $res]
}
Without doing the split, you're relying on the data being a true Tcl list from the outset. It might be… but only if you are generating it as such and just hoping everything works is not a recommended approach (in large scale code). By doing explicit splitting (actually very fast) you avoid problems from things like spaces, backslashes and braces in your data; it's my experience that they don't usually crop up in test data, but do more commonly occur in production data…