ERE - adding quantifier to group with inner group and back-reference - regex

Question

Was trying to get words with consecutive repeated letters occurring twice or thrice. Not able find a way to use quantifier and capture group using ERE
$ grep --version | head -n1
grep (GNU grep) 2.25
$ # consecutive repeated letters occurring twice
$ grep -m5 -xiE '[a-z]*([a-z])\1[a-z]*[a-z]*([a-z])\2[a-z]*' /usr/share/dict/words
Abbott
Annabelle
Annette
Appaloosa
Appleseed
$ # no output for this, why?
$ grep -m5 -xiE '([a-z]*([a-z])\2[a-z]*){2}' /usr/share/dict/words
Works with -P though
$ grep -m5 -xiP '([a-z]*([a-z])\2[a-z]*){2}' /usr/share/dict/words
Abbott
Annabelle
Annette
Appaloosa
Appleseed
$ grep -m5 -xiP '([a-z]*([a-z])\2[a-z]*){3}' /usr/share/dict/words
Chattahoochee
McConnell
Mississippi
Mississippian
Mississippians
Thanks Casimir et Hippolyte for coming up with simpler input and regex to test this behavior
$ echo 'aazbb' | grep -E '(([a-z])\2[a-z]*){2}' || echo 'No match'
aazbb
$ echo 'aazbbycc' | grep -E '(([a-z])\2[a-z]*){2}([a-z])\3[a-z]*' || echo 'No match'
aazbbycc
$ echo 'aazbbycc' | grep -P '(([a-z])\2[a-z]*){3}' || echo 'No match'
aazbbycc
$ # failing case
$ echo 'aazbbycc' | grep -E '(([a-z])\2[a-z]*){3}' || echo 'No match'
No match
Same behavior seen with sed as well
$ sed --version | head -n1
sed (GNU sed) 4.2.2
$ echo 'aazbb' | sed -E '/(([a-z])\2[a-z]*){2}/! s/.*/No match/'
aazbb
$ echo 'aazbbycc' | sed -E '/(([a-z])\2[a-z]*){2}([a-z])\3[a-z]*/! s/.*/No match/'
aazbbycc
$ # failing case
$ echo 'aazbbycc' | sed -E '/(([a-z])\2[a-z]*){3}/! s/.*/No match/'
No match
Related search links, I checked some of them, but didn't get anything close to this question
https://savannah.gnu.org/bugs/?group=grep
http://lists.gnu.org/archive/html/bug-sed/
If this is solved in newer version of grep or sed, let me know. Also, if the issue is seen in non-GNU implementations

Answer 1

I suppose -E doesn't allow Quantifiers, that's why it works only with -P
to match 2 or more consecutive groups of repeated letters:
grep -P '(?:([a-z])\1*([a-z])\2){1}' /usr/share/dict/words
to match 3 or more consecutive groups of repeated letters:
grep -P '(?:([a-z])\1*([a-z])\2){2}' /usr/share/dict/words
Options:
-P, --perl-regexp PATTERN is a Perl regular expression

Answer 2

Update
After searching around, I installed gnugrep32 on my windows box, then ran
some tests:
I read this from an old SO post:
Non-greedy matching is not part of the Extended Regular Expression syntax supported by grep
So, we use [a-z]{0,20} as a test instead of [a-z]* or [a-z]*? where the ? is ignored (wtf?)
Below are incremental tests useing the overal (){n} to see how far it will go before it STOPS BACKTRACKING
into frames.
Min to work
(([a-z])\2[a-z]{0,20}){1} len = 2 rr
(([a-z])\2[a-z]{0,20}){2} len = 4 rrrr
(([a-z])\2[a-z]{0,20}){3} len = 25 rrrrrrrrrrrrrrrrrrrrrrrrr
(([a-z])\2[a-z]{0,20}){4} len = 47 rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
(([a-z])\2[a-z]{0,20}){5} len = 69 rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
(([a-z])\2[a-z]{0,20}){6} len = 91 rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
From {3} to {6} the delta lengths are equal to 22.
This happens to be the full length of the capture frame expression ([a-z])\2[a-z]{0,20}
when it does not backtrack into previous frames.
Conclusion is that it automatically stops backtracking after 2 frames.
It makes sence given that for example, out of 20 frames, it gets to 16, and finds it cannot match.
Shoud it go back to frame 1 and adjust there and try it all over agaqin.
Why yes it should.
However, it has now consumed so much memory, the bloated pig has to unwind it all.
This could take forever with this old archaic utility.
Hey, better cap it to 2 frames.
Of course, there is no test case for (([a-z])\2[a-z]*){3} since the greedy quantifier *
will consume the entire line on the second frame if they are all [a-z] and never even
start a third frame.

Answer 3

$ # no output for this, why?
$ grep -m5 -xiE '([a-z]*([a-z])\2[a-z]*){2}' /usr/share/dict/words
Because you search for a double group (twice the same) that have a (at least) double letter inside. Something like abbcabbc [(...) = "abbc" 2 times] and not 2 (eventually similar) group that have each a double letter inside likeabbcdeef.
with 2 back ref:
$ grep -iE '[a-z]*([a-z])\1{1,}[a-z]*([a-z])\2{1,}[a-z]*`

Answer 4

I filed an issue https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26864 and the manual is now updated to reflect such issues.
From https://www.gnu.org/software/grep/manual/grep.html#Known-Bugs:
Back-references can greatly slow down matching, as they can generate exponentially many matching possibilities that can consume both time and memory to explore. Also, the POSIX specification for back-references is at times unclear. Furthermore, many regular expression implementations have back-reference bugs that can cause programs to return incorrect answers or even crash, and fixing these bugs has often been low-priority: for example, as of 2020 the GNU C library bug database contained back-reference bugs 52, 10844, 11053, 24269 and 25322, with little sign of forthcoming fixes. Luckily, back-references are rarely useful and it should be little trouble to avoid them in practical applications.