Telling if regular expression contains a single invariable segment - regex

Question

I write a search tool which is optimized to first look for fixed phrases of characters in sentences - consider it a simple "does the sentence contain a particular sequence of characters". The result will be a set of found sentences which can be searched further in a second stage.
For this second stage I like to apply regex search for convenience. But I need to pre-select the items first at the first stage, and I cannot simply get all sentences - the API I need to use for the first stage requires me to search for a phrase of at least one matching char. So there's no way around this.
Now, the user will only enter one regex, and my software needs to first determine if it can perform the first stage search on this. If the user enters something ambiguous, I will then tell the user to change his regex.
I need the algorithm that determines all substrings I can use for the first stage search.
Here are some examples of expected results:
a.b – Yes (searches for "a" or "b" first)
a|b – No (there'd be two distinct first level searches necessary)
[ab] – No (same problem: Not a clear target for the first stage search)
[ab]c – Yes (searches for "c" first)
These are simple examples. But since regex can get quite complicated I wonder if I can construct a regex or other test that will tell me if I have a usable outcome.
I could also live with limiting the regex syntax to the more common cases if that makes the test simpler, e.g. no recursion or whatever could help.

Answer 1

Here is an example, using Haskell. The algorithm should be easily transferred to another language.
Just some boilerplate imports;
import Data.Maybe
import Data.List
import Data.Function
Here is a datatype to represent a Regex. You'll have to parse it yourself or use a library:
data Regex
= Concat Regex Regex -- e.g. /ab/
| Alt Regex Regex -- e.g. /a|b/
| Single Char -- e.g. /a/
| Star Regex -- e.g. /a*/
| CharClass [(Char,Char)] -- a list of ranges. for non-range (e.g. [a]) just use the same char twice
Here is the algorithm:
regexMustMatch (Single x) = [Just x] -- has to match the character
regexMustMatch (Alt _ _) = [Nothing] -- doesn't need to match one thing (you could actually check for equality here, so something like /a|a/ would work)
regexMustMatch (Star _) = [Nothing] -- doesn't need to match one thing
regexMustMatch (CharClass ((a,b):[])) | a == b = [Just a] -- char class must match if it only has one character
regexMustMatch (CharClass _) = [Nothing] -- otherwise doesn't need to match one thing
regexMustMatch (Concat x y) = (regexMustMatch x) ++ (regexMustMatch y) -- must match both parts in sequence
Some methods to make the results usable:
selectAll = map (concatMap (return . fromJust)) .
filter (isJust . head) .
groupBy ((==) `on` isJust)
selectLongest x = case selectAll x of
[] -> ""
xs -> maximumBy (compare `on` length) xs
And some examples:
main = do
-- your tests
-- /ab/
print . selectAll . regexMustMatch $ (Single 'a' `Concat` Single 'b')
-- /a|b/
print . selectAll . regexMustMatch $ (Single 'a' `Alt` Single 'b')
-- /[ab]/
print . selectAll . regexMustMatch $ (CharClass [('a','a'),('b','b')])
-- /[ab]c/
print . selectAll . regexMustMatch $ ((Single 'a' `Alt` Single 'b') `Concat` Single 'c')
-- a few more
-- /[a]/
print . selectAll . regexMustMatch $ (CharClass [('a','a')])
-- /ab*c/
print . selectAll . regexMustMatch $ (Single 'a' `Concat` Star (Single 'b') `Concat` Single 'c')
-- /s(ab*)(cd)/ - these aren't capturing parens, just grouping to test associativity
print . selectAll . regexMustMatch $ (Single 's' `Concat` (Single 'a' `Concat` Star (Single 'b')) `Concat` (Single 'c' `Concat` Single 'd'))
Output:
["ab"] -- /ab/
[] -- /a|b/
[] -- /[ab]/
["c"] -- /[ab]c/
["a"] -- /[a]/
["a","c"] -- /ab*c/
["sa","cd"] -- /s(ab*)(cd)/
The main area where this could be improved is in the algorithm for alternation.
If we have the regex /a*bc*|d*be*/ then b needs to be matched, but this won't pick that up.
Edit: here's an improved algorithm for alternation:
regexMustMatch (Alt x y)
| x' == y' = x'
| otherwise = start ++ [Nothing] ++ common ++ [Nothing] ++ end
where
x' = regexMustMatch x
y' = regexMustMatch y
start = map fst $ takeWhile (uncurry (==)) (zip x' y')
end = map fst $ reverse $ takeWhile (uncurry (==)) (zip (reverse (drop (length start) x')) (reverse (drop (length start) y')))
dropEnds = drop (length start) . reverse . drop (length end) . reverse
common = intercalate [Nothing] $ map (map Just) (selectAll (dropEnds x') `intersect` selectAll (dropEnds y'))
Some more tests with the improved alternation:
/a*bc*|d*be*/ == b
/s(abc*|abe*)/ == sab
/s(a*bc*|d*be*)/ == s, b
/sa*b|b*/ == s
/(abc*|abe*)s/ == ab, s
/(a*bc*|d*be*)s/ == b, s
/(a*b|b*)s/ == s
/s(ab|b)e/ == s, be
/s(ba|b)e/ == sb, e
/s(b|b)e/ == sbe
/s(ac*b|ac*b)e/ == sa, be