XSLT - Regular Expression Parsing - regex

Question

My current project revolves around translating a number of test cases in a document into a form of XML compatible with a test case management system. In many of these cases, the title is prefixed by a number of ticket identifiers, document location numbers and so on, which need to be removed before they can be uploaded to the system.
Given that many of these ticket identifiers could exist elsewhere in the title and be completely valid, I've written the translation in its current form so that only the start of the string is checked for the regular expression. I have written two approaches, with varying results.
Sample Input
1.
<case-name>3.1.6 (C0) TID#EIIY CHM-2213 BZ-7043 Client side Java Upgrade R8</case-name>
2.
<case-name>4.2.7 (C1) TID#F1DR – AIP - EHD-319087 - BZ6862 - Datalink builder res...</case-name>
Desired Output
1.
<tr:summary>Client side Java Upgrade R8</tr:summary>
2.
<tr:summary>Datalink builder res...</tr:summary>
First Approach
<xsl:template match="case-name">
<tr:summary>
<xsl:variable name="start">
<xsl:apply-templates/>
</xsl:variable>
<xsl:variable name="start" select="normalize-space($start)"/>
<xsl:variable name="noFloat" select="normalize-space(fn:remFirstRegEx($start, '^[0-9]+([.][0-9]+)*' ))"/>
<xsl:variable name="noFloatDash" select="normalize-space(fn:remFirstRegEx($noFloat, '^[\p{Pd}]' ))"/>
<xsl:variable name="noC" select="normalize-space(fn:remFirstRegEx($noFloatDash, '^\(C[0-2]\)' ))"/>
<xsl:variable name="noCDash" select="normalize-space(fn:remFirstRegEx($noC, '^[\p{Pd}]' ))"/>
<xsl:variable name="noTID" select="normalize-space(fn:remFirstRegEx($noCDash, '^(TID)(#|\p{Pd})(\w+)' ))"/>
<xsl:variable name="noTIDDash" select="normalize-space(fn:remFirstRegEx($noTID, '^[\p{Pd}]' ))"/>
<xsl:variable name="noAIP" select="normalize-space(fn:remFirstRegEx($noTIDDash, '^AIP' ))"/>
<xsl:variable name="noAIPDash" select="normalize-space(fn:remFirstRegEx($noAIP, '^[\p{Pd}]' ))"/>
<xsl:variable name="noCHM" select="normalize-space(fn:remFirstRegEx($noAIPDash, '^(CHM)[\p{Pd}]([0-9]+)' ))"/>
<xsl:variable name="noCHMDash" select="normalize-space(fn:remFirstRegEx($noCHM, '^[\p{Pd}]' ))"/>
<xsl:variable name="noEHD" select="normalize-space(fn:remFirstRegEx($noCHMDash, '^(EHD)[\p{Pd}]([0-9]+)' ))"/>
<xsl:variable name="noEHDDash" select="normalize-space(fn:remFirstRegEx($noEHD, '^[\p{Pd}]' ))"/>
<xsl:variable name="noBZ" select="normalize-space(fn:remFirstRegEx($noEHDDash, '^(BZ)(((#|\p{Pd})[0-9]+)|[0-9]+)' ))"/>
<xsl:variable name="noBZDash" select="normalize-space(fn:remFirstRegEx($noBZ, '^[\p{Pd}]' ))"/>
<xsl:variable name="noTT" select="normalize-space(fn:remFirstRegEx($noBZDash, '^(TT)[#](\w)+' ))"/>
<xsl:variable name="noTTDash" select="normalize-space(fn:remFirstRegEx($noTT, '^[\p{Pd}]' ))"/>
<xsl:variable name="nobrack" select="normalize-space(fn:remFirstRegEx($noTTDash, '^\[(.*?)\]' ))"/>
<xsl:variable name="noBrackDash" select="normalize-space(fn:remFirstRegEx($nobrack, '^[\p{Pd}]' ))"/>
<xsl:value-of select="normalize-space($noBrackDash)"/>
</tr:summary>
</xsl:template>
<xsl:function name="fn:remFirstRegEx">
<xsl:param name="inString"/>
<xsl:param name="regex"/>
<xsl:variable name="words" select="tokenize($inString, '\p{Z}')"/>
<xsl:variable name="outString">
<xsl:for-each select="$words">
<xsl:if test="not(matches(., $regex)) or index-of($words, .) > 1">
<xsl:value-of select="."/><xsl:text> </xsl:text>
</xsl:if>
</xsl:for-each>
</xsl:variable>
<xsl:value-of select="string-join($outString, '')">
</xsl:function>
Note: The namespace fn, for the purpose of this translation, is just "function/namespace", used to write my own functions.
First Results
1. Success
<tr:summary>Client side Java Upgrade R8</tr:summary>
2. Failure
<tr:summary>- EHD-319087 - BZ6862 - Datalink builder resolution selector may drop leading zeros on coordinate seconds</tr:summary>
Second Approach
<xsl:function name="fn:remFirstRegEx">
<xsl:param name="inString"/>
<xsl:param name="regex"/>
<xsl:analyze-string select="$inString" regex="$regex">
<xsl:non-matching-substring>
<xsl:value-of select="."/>
</xsl:non-matching-substring>
</xsl:analyze-string>
</xsl:function>
This approach fails completely, I'm including it here because it's the more obvious solution and did not work at all.
It should be noted that there are a large number of regular expressions in the above solution, this is to account for all the possible IDs that might come through. Mercifully, the IDs seem to come in a consistent order.
The problem, as I have concluded, is with the dashes. I have noted that in every case in the documents where translation has failed, the failing ID has been both preceded and followed by a dash. If it only precedes, it'll go through fine. If it only follows, no issues. Both is where it falls down, and curiously, the dash still shows up, even though it has already been seemingly eliminated from the string.
There are two kinds of dashes at play here, a normal dash (–) and a minus sign (-).
Paradoxically: sorry for the long question, and let me know if I've missed anything out.
EDIT: Forgot to say, all regular expressions with the exception of the dashes have been tested elsewhere and are known to work on all input stuff.
EDIT II: Following #acheong87's solution, I tried to run the following:
<xsl:template match="case-name">
<tr:summary>
<xsl:variable name="regEx" select=
"'^[\s\p{Pd}]*(\d+([.]\d+)*)?[\s\p{Pd}]*(\(C[0-2]\))?([\s\p{Pd}]*(TID|AIP|CHM|EHD|BZ|TT)((#|\p{Pd}|)\w+|))*[\s\p{Pd}]*(\[.*?\])?'"/>
<xsl:analyze-string select="string(.)" regex="{$regEx}">
<xsl:non-matching-substring>
<xsl:value-of select="."/>
</xsl:non-matching-substring>
</xsl:analyze-string>
</tr:summary>
</xsl:template>
And Saxon gives me the following error:
Error at xsl:analyze-string at line (for our purposes, 5):
XTDE1150: The regular expression must not be one that matches a zero-length string
I can get why that would come up, given that everything is optional. Is there another way of running it that won't give me this error?
Thanks again.

Answer 1

Here are the main components that would go into a single regex. I've rewritten some of your expressions.
\d+([.]\d+)*
\(C[0-2]\)
TID(#|\p{Pd})\w+
AIP
CHM[\p{Pd}]\d+
EHD[\p{Pd}]\d+
BZ(#|\p{Pd}|)\d+
TT#\w+
\[.*?\]
Each component should be wrapped in (...)? to make it optional, and all components should be joined by the separator, [\s\p{Pd}]*. This produces:
^[\s\p{Pd}]*(\d+([.]\d+)*)?[\s\p{Pd}]*(\(C[0-2]\))?[\s\p{Pd}]*(TID(#|\p{Pd})\w+)?[\s\p{Pd}]*(AIP)?[\s\p{Pd}]*(CHM[\p{Pd}]\d+)?[\s\p{Pd}]*(EHD[\p{Pd}]\d+)?[\s\p{Pd}]*(BZ(#|\p{Pd}|)\d+)?[\s\p{Pd}]*(TT#\w+)?[\s\p{Pd}]*(\[.*?\])?
You can see in this Rubular demo that the above expression indeed matches your two examples.
There may be an elegant simplification you may be interested in.
\d+([.]\d+)*
\(C[0-2]\)
(TID|AIP|CHM|EHD|BZ|TT)((#|\p{Pd}|)\w+|)
\[.*?\]
Maybe some codes like AIP should be separate, but you can see the spirit of this version. That is, it's unlikely that valid titles would begin with such codes; in fact probably more likely that your examples could be missing a possible combination such as EHD#, which may appear in the future but your past-based formulation would miss. (Of course, my point is irrelevant if there is no future—and the data you have is the only data you'll need to process.) If there is a future though, IMO, it's better in this case to loosen the rigor of the expression to capture potential related combinations.
The above would become:
^[\s\p{Pd}]*(\d+([.]\d+)*)?[\s\p{Pd}]*(\(C[0-2]\))?([\s\p{Pd}]*(TID|AIP|CHM|EHD|BZ|TT)((#|\p{Pd}|)\w+|))*[\s\p{Pd}]*(\[.*?\])?
Here is the Rubular demo.

Answer 2

One regex to rule them all looks like
^ # start of string
([0-9]\.[0-9.]+).*? # digits and dots
\((C[0-2])\).*? # C0, C1, C2
((TID#\S+).*?)? # TID...
((AIP).*?)? # AIP...
((CHM\S+).*?)? # CHM...
((EHD\S+).*?)? # EHD...
((BZ\S+).*?)? # BZ...
(\w.*)? # free text
$ # end of string
^([0-9]\.[0-9.]+).*?\((C[0-2])\).*?((TID#\S+).*?)?((AIP).*?)?((CHM\S+).*?)?((EHD\S+).*?)?((BZ\S+).*?)?(\w.*)?$
http://rubular.com/r/pPxKBVwJaE
The .*? eat any delimiter until the next match begins. Most of the matches are optional, possibly even more than you need to be optional. Remove the enclosing (...)? for any match that you want to make mandatory. Optional groups are counted but can be empty.
Putting it all together
<xsl:variable name="linePattern"> <!-- group|contents -->
<xsl:text>^</xsl:text> <!-- start of string -->
<xsl:text>([0-9]\.[0-9.]+).*?</xsl:text> <!-- 1 digits and dots -->
<xsl:text>\((C[0-2])\).*?</xsl:text> <!-- 2 C0, C1, C2 -->
<xsl:text>((TID#\S+).*?)?</xsl:text> <!-- 3, 4 TID... -->
<xsl:text>((AIP).*?)?</xsl:text> <!-- 5, 6 AIP... -->
<xsl:text>((CHM\S+).*?)?</xsl:text> <!-- 7, 8 CHM... -->
<xsl:text>((EHD\S+).*?)?</xsl:text> <!-- 9, 10 EHD... -->
<xsl:text>((BZ\S+).*?)?</xsl:text> <!-- 11, 12 BZ... -->
<xsl:text>(\w.*)?</xsl:text> <!-- 13 free text -->
<xsl:text>$</xsl:text> <!-- end of string -->
</xsl:variable>
<xsl:template match="case-name">
<xsl:analyze-string select="string(.)" regex="{$linePattern}">
<xsl:matching-substring>
<tr:summary>
<part><xsl:value-of select="regex-group(1)" /></part>
<part><xsl:value-of select="regex-group(2)" /></part>
<part><xsl:value-of select="regex-group(4)" /></part>
<part><xsl:value-of select="regex-group(6)" /></part>
<part><xsl:value-of select="regex-group(8)" /></part>
<part><xsl:value-of select="regex-group(10)" /></part>
<part><xsl:value-of select="regex-group(12)" /></part>
<part><xsl:value-of select="regex-group(13)" /></part>
</tr:summary>
</xsl:matching-substring>
<!--
possibly include <xsl:non-matching-substring>, <xsl:fallback>
-->
</xsl:analyze-string>
</xsl:template>
Of course you can deal with the individual match groups any way you like.