I have been asked to help sort out a system that validates some returned MS Excel forms.
A Python program already exists that takes a returned Excel form, and another MS Excel workbook with looks much like the form, but the fields are filled with Regexes.
The Python code then validates the form by testing the returned form, against the Regexes in turn from the Regex form.
This all works as expected, and pumps out a workbook with a list of problems where a match doesn't occur.
However the validation wasn't always returning results as expected and I was asked to try sorting it out.
I have been reviewing the regexes against a document describes what the valid responses in the form should be. I can cope with most of them, but a couple of them got me to thinking. These are ones where the valid entry in the form is a list of items. E.g. a list of words seperated by commas or newlines.
Everytime I come across one of these I have been using the following approach:
^[A-Z]{0,10}((, *|\n)[A-Z]{0,10})*$
So this will match the first uppercase word up to 10 letters, then the rest of the list with each entry preceeded by a , or <CR>. It works, but I wonder if there is a better way? The reason I am thinking this is because the matching pattern for each list entry has to be in the regex twice. So if a problem is spotted, it has to be corrected in two places.
Is there a better way?
Use
^\b((^|, *|\n)[A-Z]{1,10})+$
It makes sure the text starts with a word character (i.e. a-zA-Z0-9_) by checking for a word boundary. Thus preventing the text to start with , or an empty line.
Then, repeating at least once, there should be a line beginning with one of
Start of text
A comma followed by any number of spaces
a newline
then, at least one, at the most ten, uppercase characters.
The first start alternative is obviously only possible for the start of the line, the second for intermediate words and the final for the last word on a line.
See it here at regex101.
The general problem:
I've got lot of data I'm trying to clean up then parse. Each line is really long, but they all have the same structure. It starts with one unique substring, followed by a second unique substring, followed by a substring that repeats about 20 times.
So it's: String A, String B, String C, String C, String C, etc. Every line is in that format.
At the start of String A is an ID, just a unique six digit number. I'm trying to insert that ID at the beginning of String B and all of the String C's.
String C is the problem. I can write regex's for each of the ID, B, and C, but trying to insert the captured ID into all the C's fails. It only works on the last one. That's actually the correct behavior here, but I'm pretty sure there is a way to to treat String C so that it will act like each instance of the substring is separate. And the regex runs over it again and again.
I tried using '\G' syntax but I can't seem to make it work.
So here's a specific example using some massively abridged sample data:
['sample_id':121084,[122,'southwest',7.23,[[['station_01',[1]],['station_02',[1]], ['station_03',[22]],['station_04',[49]],['station_05',[1]],['station_06',[4]],['station_07',[101]],['station_08',[22]]]],[[['run':133225,'marker':'SAM',[[['substation_01',[1]],['substation_02',[3]],['substation_03',[16]],['substation_04',[15]],['substation_05',[14]],['substation_06',[6]],['substation_07',[41]],['substation_08',[19]],['substation_09',[13]],['substation_10',[1]],['substation_11',[13]],['substation_12',[1]]]],'TK',22,34,127],['run':608049,'marker':'TIM',[[['substation_01',[12]],['substation_02',[6]],['substation_03',[17]],['substation_04',[11]],['substation_05',[1]],['substation_06',[6]],['substation_07',[5]],['substation_08',[19]]]],'TM',21,21,966],['run':445801,'marker':'RON',[[['substation_01',[5]],['substation_02',[5]],['substation_03',[6]],['substation_04',[11]],['substation_05',[1]],['substation_06',[15]],['substation_07',[11]],['substation_08',[16]],['substation_09',[1]],['substation_10',[13]],['substation_11',[3]]]],'TR',12,33,521],['run':142278, etc...
Just a note: The only difference between String B and all the String Cs is the number of brackets, but that's actually useful once I start parsing this out (ultimately it'll all be JSON).
What I'm trying to get is:
['sample_id':121084,[122,'southwest',7.23,[[['station_01',[1]],['station_02',[1]],['station_03',[22]],['station_04',[49]],['station_05',[1]],['station_06',[4]],['station_07',[101]],['station_08',[22]]]],[[['sample_id':121084,'run':133225,'marker':'SAM',[[['substation_01',[1]],['substation_02',[3]],['substation_03',[16]],['substation_04',[15]],['substation_05',[14]],['substation_06',[6]],['substation_07',[41]],['substation_08',[19]],['substation_09',[13]],['substation_10',[1]],['substation_11',[13]],['substation_12',[1]]]],'TK',22,34,127],['sample_id':121084,'run':608049,'marker':'TIM',[[['substation_01',[12]],['substation_02',[6]],['substation_03',[17]],['substation_04',[11]],['substation_05',[1]],['substation_06',[6]],['substation_07',[5]],['substation_08',[19]]],'TM',21,21,966],['sample_id':121084,'run':445801,'marker':'RON',[[['substation_01',[5]],['substation_02',[5]],['substation_03',[6]],['substation_04',[11]],['substation_05',[1]],['substation_06',[15]],['substation_07',[11]],['substation_08',[16]],['substation_09',[1]],['substation_10',[13]],['substation_11',[3]]],'TR',12,33,521],['sample_id':121084, etc...
In the latter text block each substring now begins with the ID 'sample_id':121084 (I bolded it to make it slightly easier to see what's going on).
Here's the Regex that gets me up through String C.
\[('sample_id':\d{6},)(?:.+\]\]\],\[\[)\[(.+?\d\],)\[(.+?\d\],)
So I'm trying to insert that first capture group ($1) in front of the second group, then the third group over and over and over (about 20x). If I repeat the last bit, I end up killing all but one of the C Strings, which again, I believe to be the 'proper' behavior. I'm trying to figure out how to get around that.
It's a mess I know. But each of those is just one line, and I've got doc after doc that'll have 100 or so lines like that. So a regex that doesn't break up the lines seems best.
I went over this page a few times trying to engineer a solution, but again, I couldn't make the \G syntax work here.
Collapse and Capture a Repeating Pattern in a Single Regex Expression
Should mention I'm trying to do this in Sublime Text 2. Thanks for any help.