convert character if codepoint within given range - xslt

I have a couple of XML files that contain unicode characters with codepoint values between 57600 and 58607. Currently these are shown in my content as square blocks and I'd like to convert these to elements.
So what I'd like to achieve is something like :
<!-- current input -->
<p> Follow the on-screen instructions.</p>
<!-- desired output-->
<p><unichar value="58208"/> Follow the on-screen instructions.</p>
<!-- Where 58208 is the actual codepoint of the unicode character in question -->
I've fooled around a bit with tokenizer but as you need to have reference to split upon this turned out to be over complicated.
Any advice on how to tackle this best ? I've been trying some things like below but got struck (don't mind the syntax, I know it doesn't make any sense)
<xsl:template match="text()">
-> for every character in my string
-> if string-to-codepoints(current character) greater then 57600 return <unichar value="codepoint value"/>
else return character
</xsl:template>

It sounds like a job for analyze-string e.g.
<xsl:template match="text()">
<xsl:analyze-string select="." regex="[-]">
<xsl:matching-substring>
<unichar value="{string-to-codepoints(.)}"/>
</xsl:matching-substring>
<xsl:non-matching-substring>
<xsl:value-of select="."/>
</xsl:non-matching-substring>
</xsl:analyze-string>
</xsl:template>
Untested.

This transformation:
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes"/>
<xsl:template match="/*">
<p>
<xsl:for-each select="string-to-codepoints(.)">
<xsl:choose>
<xsl:when test=". > 57600">
<unichar value="{.}"/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="codepoints-to-string(.)"/>
</xsl:otherwise>
</xsl:choose>
</xsl:for-each>
</p>
</xsl:template>
</xsl:stylesheet>
when applied on the provided XML document:
<p> Follow the on-screen instructions.</p>
produces the wanted, correct result:
<p><unichar value="58498"/> Follow the on-screen instructions.</p>
Explanation: Proper use of the standard XPath 2.0 functions string-to-codepoints() and codepoints-to-string().

Related

xslt remove white space around specific elements (xml to xml)

I have a lot of XML files where an export from a database has added certain whitespace via indentation that now I wish to remove in an XSLT 3.0 transformation to a new xml output. I want to remove the whitespace introduced by the export around <lb> and <pb> (in the original files, before export, these two elements abutted other elements without whitespace - a hidden bug in the export indented them).
This is an example of the problem file to transform:
<p xml:id="MS609-0783-LA" xml:lang="LA">
<seg type="dep_event" xml:id="MS609-0783-1">
<pb n="58r"/>
<lb n="1"/>Item.
<date type="deposition_date" when="1245-07-07">Anno Domini M°CC°XL°V° Nonas Iulii</date><persName
nymRef="#ber_r_baz-hg" role="dep">Ber. R.</persName> testis juratus dixit quod vidit
<persName nymRef="#heretics_in_public" ref="her">hereticos</persName>.</seg>
</p>
Here, an example of the desired XML output:
<p xmlns="http://www.tei-c.org/ns/1.0" xml:id="MS609-0783-LA" xml:lang="LA">
<seg type="dep_event" xml:id="MS609-0783-1"><pb n="58r"/><lb n="1"/>Item.
<date type="deposition_date" when="1245-07-07">Anno Domini M°CC°XL°V° Nonas Iulii</date> <persName
nymRef="#ber_r_baz-hg" role="dep">Ber. R.</persName> testis juratus dixit quod vidit
<persName nymRef="#heretics_in_public" ref="her">hereticos</persName>.</seg>
</p>
I thought, naively, that I could target it so:
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:tei="http://www.tei-c.org/ns/1.0"
version="3.0">
<xsl:mode on-no-match="shallow-copy"/>
<xsl:output method="xml"/>
<xsl:template match="/">
<xsl:apply-templates/>
</xsl:template>
<xsl:template match="text()[normalize-space(.) = '']">
<xsl:choose>
<xsl:when test="./following-sibling::tei:pb">
<xsl:text/>
</xsl:when>
<xsl:when test="./following-sibling::tei:lb">
<xsl:text/>
</xsl:when>
<xsl:otherwise>
<xsl:text> </xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>
But it does not produce the desired result: https://xsltfiddle.liberty-development.net/93nwMpi
Ideally, I am working towards a solution that strips out any blank white space out before or after <pb> and/or <lb> (when they are abutted by other elements), anywhere inside <seg> or its descendants.
Many thanks in advance for pointers.
I don't have a clear understanding on which text nodes you want to strip but
<xsl:template match="tei:seg//text()[not(normalize-space())][following-sibling::node()[1][self::tei:pb | self::tei:lb]]"/>
would strip white-space only ones followed by pb or lb inside of a seg.
Of course you can extend that to the ones preceded by those elements with e.g.
<xsl:template match="tei:seg//text()[not(normalize-space())][preceding-sibling::node()[1][self::tei:pb | self::tei:lb] or following-sibling::node()[1][self::tei:pb | self::tei:lb]]"/>
If the simple blocking based on match patterns doesn't suffice you might want to try whether your definition of "abutted" can be somehow expressed with group-adjacent and xsl:for-each-group and then drop any white space nodes in a group e.g.
<xsl:template match="tei:seg[tei:pb | tei:lb] | tei:seg//*[tei:pb | tei:lb]">
<xsl:copy>
<xsl:for-each-group select="node()" group-adjacent="boolean(self::text()[not(normalize-space())]|self::tei:pb|self::tei:lb)">
<xsl:choose>
<xsl:when test="current-grouping-key() and current-group()[self::tei:pb|self::tei:lb]">
<xsl:sequence select="current-group()[not(self::text())]"/>
</xsl:when>
<xsl:otherwise>
<xsl:apply-templates select="current-group()"/>
</xsl:otherwise>
</xsl:choose>
</xsl:for-each-group>
</xsl:copy>
</xsl:template>

Extract parts of inputs strings

I am trying to extract equipment names from strings and would like if someone could help me find a good way to do this.
My input string can either contain 1 or 2 equipment names, consisting of EQ followed 1 to 3 digits, for example :
LocationEQ3Suffix
LocationEQ5EQ8Suffix
So in the first instance I would need 'EQ3' and in the second instance I would need 'EQ5' and 'EQ8'.
I need the output to be in a text format, for example :
SomeText.EQ3
SomeText.EQ5
SomeText.EQ8
I was thinking there might be a way to do this with xsl:analyze-string and a regex like EQ[0-9]{1,3}.
Any help is appreciated.
I started something like this, but I don't think it's the right approach.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="2.0">
<xsl:output method="text"/>
<xsl:template match="/">
<xsl:variable name="input" select="'LocationEQ3EQ4Funct'"/>
<xsl:choose>
<!-- Case with 2 EQ -->
<xsl:when test="matches($input, 'EQ[0-9]{1,3}EQ[0-9]{1,3}')">
<xsl:value-of select="$input"/>
</xsl:when>
<!-- Case with 1 EQ -->
<xsl:otherwise>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>
You say you want to use xsl:analyze-string but you're not.
An implementation using it would look something like:
<xsl:analyze-string select="input-string" regex="EQ\d{{1,3}}">
<xsl:matching-substring>
<xsl:text>SomeText.</xsl:text>
<xsl:value-of select="." />
<xsl:text>
</xsl:text>
</xsl:matching-substring>
</xsl:analyze-string>
Demo: https://xsltfiddle.liberty-development.net/a9Hk1a

XSLT - replace specific content of the text() node with a new node

I have a xml like this,
<doc>
<p>Biological<sub>89</sub> bases<sub>4456</sub> for<sub>8910</sub> sexual<sub>4456</sub>
differences<sub>8910</sub> in<sub>4456</sub> the brain exist in a wide range of
vertebrate species, including chickens<sub>8910</sub> Recently<sub>8910</sub> the
dogma<sub>8910</sub> of<sub>4456</sub> hormonal dependence for the sexual
differentiation of the brain has been challenged.</p>
</doc>
As you can see there are <sub> nodes and text() node contains inside the <p> node. and every <sub> node end, there is a text node, starting with a space. (eg: <sub>89</sub> bases : here before 'bases' text appear there is a space exists.) I need to replace those specific spaces with nodes.
SO the expected output should look like this,
<doc>
<p>Biological<sub>89</sub><s/>bases<sub>4456</sub><s/>for<sub>8910</sub><s/>sexual<sub>4456</sub>
<s/>differences<sub>8910</sub><s/>in<sub>4456</sub><s/>the brain exist in a wide range of
vertebrate species, including chickens<sub>8910</sub><s/>Recently<sub>8910</sub><s/>the
dogma<sub>8910</sub><s/>of<sub>4456</sub><s/>hormonal dependence for the sexual
differentiation of the brain has been challenged.</p>
</doc>
to do this I can use regular expression like this,
<xsl:template match="p/text()">
<xsl:analyze-string select="." regex="( )">
<xsl:matching-substring>
<xsl:choose>
<xsl:when test="regex-group(1)">
<s/>
</xsl:when>
</xsl:choose>
</xsl:matching-substring>
<xsl:non-matching-substring>
<xsl:value-of select="."/>
</xsl:non-matching-substring>
</xsl:analyze-string>
</xsl:template>
But this adds <s/> nodes to every spaces in the text() node. But I only need thi add nodes to that specific spaces.
Can anyone suggest me a method how can I do this..
If you only want to match text nodes that start with a space and are preceded by a sub element, you can put the condition in your template match
<xsl:template match="p/text()[substring(., 1, 1) = ' '][preceding-sibling::node()[1][self::sub]]">
And if you just want to remove the space at the start of the string, a simple replace will do.
<xsl:value-of select="replace(., '^\s+', '')" />
Try this XSLT
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:output method="xml" indent="no" />
<xsl:template match="p/text()[substring(., 1, 1) = ' '][preceding-sibling::node()[1][self::sub]]">
<s />
<xsl:value-of select="replace(., '^\s+', '')" />
</xsl:template>
<xsl:template match="#*|node()">
<xsl:copy>
<xsl:apply-templates select="#*|node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Just change the regex like so ^( ): it will match only the spaces at the beginning of the text part.
With this XSL snipped:
<xsl:analyze-string select="." regex="^( )">
Here is the result I obtain:
<p>Biological<sub>89</sub><s></s>bases<sub>4456</sub><s></s>for<sub>8910</sub><s></s>sexual<sub>4456</sub>
differences<sub>8910</sub><s></s>in<sub>4456</sub><s></s>the brain exist in a wide range of
vertebrate species, including chickens<sub>8910</sub><s></s>Recently<sub>8910</sub><s></s>the
dogma<sub>8910</sub><s></s>of<sub>4456</sub><s></s>hormonal dependence for the sexual
differentiation of the brain has been challenged.
</p>

parsing string in xslt

I have following xml
<xml>
<xref>
is determined “in prescribed manner”
</xref>
</xml>
I want to see if we can process xslt 2 and return the following result
<xml>
<xref>
is
</xref>
<xref>
determined
</xref>
<xref>
“in prescribed manner”
</xref>
</xml>
I tried few options like replace the space and entities and then using for-each loop but not able to work it out. May be we can use tokenize function of xslt 2.0 but don't know how to use it. Any hint will be helpful.
# JimGarrison: Sorry, I couldn't resist. :-) This XSLT is definitely not elegant but it does (I assume) most of the job:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet
version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes" />
<xsl:variable name="left_quote" select="'<'"/>
<xsl:variable name="right_quote" select="'>'"/>
<xsl:template name="protected_tokenize">
<xsl:param name="string"/>
<xsl:variable name="pattern" select="concat('^([^', $left_quote, ']+)(', $left_quote, '[^', $right_quote, ']*', $right_quote,')?(.*)')"/>
<xsl:analyze-string select="$string" regex="{$pattern}">
<xsl:matching-substring>
<!-- Handle the prefix of the string up to the first opening quote by "normal" tokenizing. -->
<xsl:variable name="prefix" select="concat(' ', normalize-space(regex-group(1)))"/>
<xsl:for-each select="tokenize(normalize-space($prefix), ' ')">
<xref>
<xsl:value-of select="."/>
</xref>
</xsl:for-each>
<!-- Handle the text between the quotes by simply passing it through. -->
<xsl:variable name="protected_token" select="normalize-space(regex-group(2))"/>
<xsl:if test="$protected_token != ''">
<xref>
<xsl:value-of select="$protected_token"/>
</xref>
</xsl:if>
<!-- Handle the suffix of the string. This part may contained protected tokens again. So we do it recursively. -->
<xsl:variable name="suffix" select="normalize-space(regex-group(3))"/>
<xsl:if test="$suffix != ''">
<xsl:call-template name="protected_tokenize">
<xsl:with-param name="string" select="$suffix"/>
</xsl:call-template>
</xsl:if>
</xsl:matching-substring>
</xsl:analyze-string>
</xsl:template>
<xsl:template match="*|#*">
<xsl:copy>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template match="xref">
<xsl:call-template name="protected_tokenize">
<xsl:with-param name="string" select="text()"/>
</xsl:call-template>
</xsl:template>
</xsl:stylesheet>
Notes:
There is the general assumption that white space only serves as a token delimiter and need not be preserved.
“ and rdquo; seem to be invalid in XML although they are valid in HTML. In the XSLT there are variables defined holding the quote characters. They will have to be adapted once you find the right XML representation. You can also eliminate the variables and put the characters right into the regular expression pattern. It will be significantly simplified by this.
<xsl:analyze-string> does not allow a regular expression which may evaluate into an empty string. This comes as a little problem since either the prefix and/or the proteced token and/or the suffix may be empty. I take care of this by artificially adding a space at the beginning of the pattern which allows me to search for the prefix using + (at least one occurence) instead of * (zero or more occurences).

Producing a new line in XSLT

I want to produce a newline for text output in XSLT. Any ideas?
The following XSL code will produce a newline (line feed) character:
<xsl:text>
</xsl:text>
For a carriage return, use:
<xsl:text>
</xsl:text>
My favoured method for doing this looks something like:
<xsl:stylesheet>
<xsl:output method='text'/>
<xsl:variable name='newline'><xsl:text>
</xsl:text></xsl:variable>
<!-- note that the layout there is deliberate -->
...
</xsl:stylesheet>
Then, whenever you want to output a newline (perhaps in csv) you can output something like the following:
<xsl:value-of select="concat(elem1,elem2,elem3,$newline)" />
I've used this technique when outputting sql from xml input. In fact, I tend to create variables for commas, quotes and newlines.
Include the attribute Method="text" on the xsl:output tag and include newlines in your literal content in the XSL at the appropriate points. If you prefer to keep the source code of your XSL tidy use the entity
where you want a new line.
You can use: <xsl:text>
</xsl:text>
see the example
<xsl:variable name="module-info">
<xsl:value-of select="#name" /> = <xsl:value-of select="#rev" />
<xsl:text>
</xsl:text>
</xsl:variable>
if you write this in file e.g.
<redirect:write file="temp.prop" append="true">
<xsl:value-of select="$module-info" />
</redirect:write>
this variable will produce a new line infile as:
commons-dbcp_commons-dbcp = 1.2.2
junit_junit = 4.4
org.easymock_easymock = 2.4
IMHO no more info than #Florjon gave is needed. Maybe some small details are left to understand why it might not work for us sometimes.
First of all, the &#xa (hex) or &#10 (dec) inside a <xsl:text/> will always work, but you may not see it.
There is no newline in a HTML markup. Using a simple <br/> will do fine. Otherwise you'll see a white space. Viewing the source from the browser will tell you what really happened. However, there are cases you expect this behaviour, especially if the consumer is not directly a browser. For instance, you want to create an HTML page and view its structure formatted nicely with empty lines and idents before serving it to the browser.
Remember where you need to use disable-output-escaping and where you don't. Take the following example where I had to create an xml from another and declare its DTD from a stylesheet.
The first version does escape the characters (default for xsl:text)
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" indent="yes" encoding="utf-8"/>
<xsl:template match="/">
<xsl:text><!DOCTYPE Subscriptions SYSTEM "Subscriptions.dtd">
</xsl:text>
<xsl:copy>
<xsl:apply-templates select="*" mode="copy"/>
</xsl:copy>
</xsl:template>
<xsl:template match="#*|node()" mode="copy">
<xsl:copy>
<xsl:apply-templates select="#*|node()" mode="copy"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
and here is the result:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Subscriptions SYSTEM "Subscriptions.dtd">
<Subscriptions>
<User id="1"/>
</Subscriptions>
Ok, it does what we expect, escaping is done so that the characters we used are displayed properly. The XML part formatting inside the root node is handled by ident="yes". But with a closer look we see that the newline character &#xa was not escaped and translated as is, performing a double linefeed! I don't have an explanation on this, will be good to know. Anyone?
The second version does not escape the characters so they're producing what they're meant for. The change made was:
<xsl:text disable-output-escaping="yes"><!DOCTYPE Subscriptions SYSTEM "Subscriptions.dtd">
</xsl:text>
and here is the result:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Subscriptions SYSTEM "Subscriptions.dtd">
<Subscriptions>
<User id="1"/>
</Subscriptions>
and that will be ok. Both cr and lf are properly rendered.
Don't forget we're talking about nl, not crlf (nl=lf). My first attempt was to use only cr:&#xd and while the output xml was validated by DOM properly.
I was viewing a corrupted xml:
<?xml version="1.0" encoding="utf-8"?>
<Subscriptions>riptions SYSTEM "Subscriptions.dtd">
<User id="1"/>
</Subscriptions>
DOM parser disregarded control characters but the rendered didn't. I spent quite some time bumping my head before I realised how silly I was not seeing this!
For the record, I do use a variable inside the body with both CRLF just to be 100% sure it will work everywhere.
You can try,
<xsl:text>
</xsl:text>
It will work.
I added the DOCTYPE directive you see here:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xsl:stylesheet [
<!ENTITY nl "
">
]>
<xsl:stylesheet xmlns:x="http://www.w3.org/2005/02/query-test-XQTSCatalog"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="2.0">
This allows me to use &nl; instead of
to produce a newline in the output. Like other solutions, this is typically placed inside a <xsl:text> tag.
I second Nic Gibson's method, this was
always my favorite:
<xsl:variable name='nl'><xsl:text>
</xsl:text></xsl:variable>
However I have been using the Ant task <echoxml> to
create stylesheets and run them against files. The
task will do attribute value templates, e.g. ${DSTAMP} ,
but is also will reformat your xml, so in some
cases, the entity reference is preferable.
<xsl:variable name='nl'><xsl:text>
</xsl:text></xsl:variable>
I have found a difference between literal newlines in <xsl:text> and literal newlines using
.
While literal newlines worked fine in my environment (using both Saxon and the default Java XSLT processor) my code failed when it was executed by another group running in a .NET environment.
Changing to entities (
) got my file generation code running consistently on both Java and .NET.
Also, literal newlines are vulnerable to being reformatted by IDEs and can inadvertently get lost when the file is maintained by someone 'not in the know'.
I've noticed from my experience that producing a new line INSIDE a <xsl:variable> clause doesn't work.
I was trying to do something like:
<xsl:variable name="myVar">
<xsl:choose>
<xsl:when test="#myValue != ''">
<xsl:text>My value: </xsl:text>
<xsl:value-of select="#myValue" />
<xsl:text></xsl:text> <!--NEW LINE-->
<xsl:text>My other value: </xsl:text>
<xsl:value-of select="#myOtherValue" />
</xsl:when>
</xsl:choose>
<xsl:variable>
<div>
<xsl:value-of select="$myVar"/>
</div>
Anything I tried to put in that "new line" (the empty <xsl:text> node) just didn't work (including most of the simpler suggestions in this page), not to mention the fact that HTML just won't work there, so eventually I had to split it to 2 variables, call them outside the <xsl:variable> scope and put a simple <br/> between them, i.e:
<xsl:variable name="myVar1">
<xsl:choose>
<xsl:when test="#myValue != ''">
<xsl:text>My value: </xsl:text>
<xsl:value-of select="#myValue" />
</xsl:when>
</xsl:choose>
<xsl:variable>
<xsl:variable name="myVar2">
<xsl:choose>
<xsl:when test="#myValue != ''">
<xsl:text>My other value: </xsl:text>
<xsl:value-of select="#myOtherValue" />
</xsl:when>
</xsl:choose>
<xsl:variable>
<div>
<xsl:value-of select="$myVar1"/>
<br/>
<xsl:value-of select="$myVar2"/>
</div>
Yeah, I know, it's not the most sophisticated solution but it works, just sharing my frustration experience with XSLs ;)
I couldn't just use the <xsl:text>
</xsl:text> approach because if I format the XML file using XSLT the entity will disappear. So I had to use a slightly more round about approach using variables
<xsl:variable name="nl" select="'
'"/>
<xsl:template match="/">
<xsl:value-of select="$nl" disable-output-escaping="no"/>
<xsl:apply-templates select="*"/>
</xsl:template>
<xsl:text xml:space="preserve">
</xsl:text>
just add this tag:
<br/>
it works for me ;) .