Sed : removing lines where pattern result also appears elsewhere - regex

Question

Let's suppose I have this sample
foo/bar/123-465.txt
foo/bar/456-781.txt
foo/bar/102-445.txt
foo/bar/123-721.txt
I want to remove every line where the regex /[0-9]*- result also appears on another line. In other terms : I want to remove every line where the file prefix is present more than once in my file.
Therefore only keeping :
foo/bar/456-781.txt
foo/bar/102-445.txt
I bet sed can do this, but how ?

Answer 1

Ok I misunderstood your problem, here's how to do it:
grep -vf <(grep -o '/[0-9]*-' file | sort | uniq -d) file
In action:
cat file
foo/bar/123-465.txt
foo/bar/456-781.txt
foo/bar/102-445.txt
foo/bar/123-721.txt
grep -vf <(grep -o '/[0-9]*-' file | sort | uniq -d) file
foo/bar/456-781.txt
foo/bar/102-445.txt

Answer 2

awk '
match($0, "[0-9]*-") {
id=substr($0, RSTART, RLENGTH)
if (store[id])
dup[id] = 1
store[id] = $0
}
END {
for(id in store) {
if(! dup[id]) {
print store[id]
}
}
}
'

Answer 3

You can use the following awk script:
example.awk:
{
# Get value of interest (before the -)
prefix=substr($3,0,match($3,/\-/)-1)
# Increment counter for this value (starting at 0)
counter[prefix]++
# Buffer the current line
buffer[prefix]=$0
}
# At the end print every line which's value of interest appeared just once
END {
for(index in counter)
if(counter[index]==1)
print buffer[index]
}
Execute it like this:
awk -F\ -f example.awk input.file