XSLT Streaming Chained Transform - xslt

I am using Saxon EE to transform a very large document, using streaming transform. I now need to chain multiple XSLTs. How does one go about doing that ?
When not streaming, I have used the XSLTTransformer class as destination, to do chained transforms. If I am not mistaken, I guess I cannot do that, as that would create a result tree as against result stream.
Thanks,
Ani

Pipe the SAX output events of the 1st transform into the SAX input events of the 2nd transform.
I've attached some example Scala code which shows this.
Basically it kicks off the 2nd XSLT first, which behind the scenes invokes the 1st XSLT with the initial input document capturing the intermediate output which is then fed as the input to the 2nd XSLT in real time. Nice.
It uses JAXP interfaces, so no S9 API.
I've successfully tested it with a 1.2GB input XML file.
Hope this helps.
XSLT 3.0 rocks by the way! Good choice.
import javax.xml.transform.sax.{SAXResult, SAXSource}
import javax.xml.transform.stream.{StreamResult, StreamSource}
import javax.xml.transform.{Source, Transformer}
import com.saxonica.config.StreamingTransformerFactory
import org.xml.sax._
import org.xml.sax.helpers.XMLFilterImpl
object Main extends App
{
val transformer1 = transformer("transform-1.xsl")
val transformer2 = transformer("transform-2.xsl")
val inputXML = "big.xml"
transformer2.transform(
new SAXSource(
new Transformer1OutputReader(transformer1, new StreamSource(inputXML)),
null
),
new StreamResult("out.xml")
)
def transformer(xslt : String) =
new StreamingTransformerFactory().newTransformer(new StreamSource(xslt))
}
class Transformer1OutputReader(
transformer1 : Transformer,
source1 : Source) extends XMLFilterImpl
{
def parseImpl() =
{
println("parseImpl()")
val inputToSecondXslt : ContentHandler = getContentHandler
transformer1.transform(
source1,
new SAXResult(inputToSecondXslt)
)
}
override def parse(input : InputSource) = parseImpl
override def parse(systemId : String) = parseImpl
override def setFeature(name: String, value: Boolean) : Unit = {}
}

Related

JenkinsPipelineUnit helper can intercept text param call but not string param call

How can I get JenkinsPipelineUnit to intercept both text() and string() param calls? I have code that triggers a build with a few params. I want to write a nice unit test to check that it does what it should. However, the string() calls are not intercepted so I cannot test them. I can test the text() call.
Yes, it is possible to write unit tests for Jenkins pipeline code vs testing via jenkins production jobs. This project + shared libraries makes jenkins into a much better tool.
Any ideas on how I could do this? I looked in the JenkinsPipelineUnit project but didn't find an example that fit and couldn't figure out where to look in the runtime objects.
I do see that the project's BasePipelineTest.groovy links string to its stringInterceptor which seems to just eat it the string. Maybe, I can unregister theirs...
Example
def triggeringParams = [:]
....
for (def param in ['text', 'string']) {
helper.registerAllowedMethod(param, [LinkedHashMap],
{ LinkedHashMap data ->
triggeringParams << data
}
)
}
thisScript = helper.loadScript('''
package resources
return this''')
def params = []
params << thisScript.text(name: 'MULTILINE_PARAM', value: '\nline1\nline2')
params << thisScript.string(name: 'STRING_PARAM', value: 'a string')
thisScript.build(job: 'myJob', parameters: params)
println triggeringParams
Results
[
[name:JOB_PROPERTIES, value:
line1
line2]
]
Wrong type was the problem. The project's BasePipelineTest.groovy links string to its stringInterceptor which seems to just eat it the string and uses register Map not LinkedHashMap. So, the first is found before mine and boom, the string doesn't show in my collector.
If I modify my code to use the more generic map it works
void addParameterHelpers() {
for (def param in ['text', 'string']) {
helper.registerAllowedMethod(param, [Map],
{ LinkedHashMap data ->
triggeringParams << data
}
)
}
}

NativeScript-vue Barcode creation

Hi i would like to ask if there is anyway to create a barcode in nativescript-vue, there is a lot of vue js example of creating barcode but im not equipped to translate or convert the codes. is there anyone can give me an example of creating barcode on nativescript-vue? thank you.
You can use nativescript-qr-generator & nativescript-base-64-manager.
The base-64 only if you want to convert the QR code as a base-64.
Import it in your template:
import { QrGenerator } from "nativescript-qr-generator";
import { Base64Manager } from "nativescript-base-64-manager";
You would need a data property generatedQR: null and then you would need a method to generate QR. So the QR code is generated and then then your code value is converted to the base64 value. You can also append any data to the result like an url for example: new QrGenerator().generate("url/" + _Base64Manager.btoa(this.code)
generateQR() {
const _Base64Manager = new Base64Manager();
const result = new QrGenerator().generate( _Base64Manager.btoa(this.code),
{
color: "#your_color",
backgroundColor: "#your_color"
}
);
this.generatedQR = new ImageSource(result);
}
and in template you will have the QR code like an image:
<Image :src="generatedQR" row="4" imageFit="aspectFill"/>

Stream records from DataBase using Akka Stream

I have a system using Akka which currently handles incoming streaming data over message queues. When a record arrives then it is processed, mq is acked and record is passed on for further handling within the system.
Now I would like to add support for using DBs as input.
What would be a way to go for the input source to be able to handle DB (should stream in > 100M records at the pace that the receiver can handle - so I presume reactive/akka-streams?)?
Slick Library
Slick streaming is how this is usually done.
Extending the slick documentation a bit to include akka streams:
//SELECT Name from Coffees
val q = for (c <- coffees) yield c.name
val action = q.result
type Name = String
val databasePublisher : DatabasePublisher[Name] = db stream action
import akka.stream.scaladsl.Source
val akkaSourceFromSlick : Source[Name, _] = Source fromPublisher databasePublisher
Now akkaSourceFromSlick is like any other akka stream Source.
"Old School" ResultSet
It is also possible to use a plain ResultSet, without slick, as the "engine" for an akka stream. We will utilize the fact that a stream Source can be instantiated from an Iterator.
First create the ResultSet using standard jdbc techniques:
import java.sql._
val resultSetGenerator : () => Try[ResultSet] = Try {
val statement : Statement = ???
statement executeQuery "SELECT Name from Coffees"
}
Of course all ResultSet instances have to move the cursor before the first row:
val adjustResultSetBeforeFirst : (ResultSet) => Try[ResultSet] =
(resultSet) => Try(resultSet.beforeFirst()) map (_ => resultSet)
Once we start iterating through rows we'll have to pull the value from the correct column:
val getNameFromResultSet : ResultSet => Name = _ getString "Name"
And now we can implement the Iterator Interface to create a Iterator[Name] from a ResultSet:
val convertResultSetToNameIterator : ResultSet => Iterator[Name] =
(resultSet) => new Iterator[Try[Name]] {
override def hasNext : Boolean = resultSet.next
override def next() : Try[Name] = Try(getNameFromResultSet(resultSet))
} flatMap (_.toOption)
And finally, glue all the pieces together to create the function we'll need to pass to Source.fromIterator:
val resultSetGenToNameIterator : (() => Try[ResultSet]) => () => Iterator[Name] =
(_ : () => Try[ResultSet])
.andThen(_ flatMap adjustResultSetBeforeFirst)
.andThen(_ map convertResultSetToNameIterator)
.andThen(_ getOrElse Iterator.empty)
This Iterator can now feed a Source:
val akkaSourceFromResultSet : Source[Name, _] =
Source fromIterator resultSetGenToNameIterator(resultSetGenerator)
This implementation is reactive all the way down to the database. Since the ResultSet pre-fetches a limited number of rows at a time, data will only come off the hard drive through the database as the stream Sink signals demand.
I find Alpakka documentation to be excellent and a much easier way to work with reactive-streams than than the Java Publisher interface.
The Alpakka project is an open source initiative to implement stream-aware, reactive, integration pipelines for Java and Scala. It is built on top of Akka Streams, and has been designed from the ground up to understand streaming natively and provide a DSL for reactive and stream-oriented programming, with built-in support for backpressure
Document for Alpakka with Slick: https://doc.akka.io/docs/alpakka/current/slick.html
Alpakka Github: https://github.com/akka/alpakka

Play WS standalone for 2.5.x

I want to create a Play web service client outside a Play application. For Play WS version 2.4.x it is easy to find that it is done like this:
val config = new NingAsyncHttpClientConfigBuilder().build()
val builder = new AsyncHttpClientConfig.Builder(config)
val client = new NingWSClient(builder.build)
However in 2.5.x the NingWSClient is now deprecated - instead the AhcWSClient should be used.
Unfortunately, I didn't find a complete example that explains the creation and usage of a AhcWsClient outside of Play. Currently I go with this:
import play.api.libs.ws.ahc.AhcWSClient
import akka.stream.ActorMaterializer
import akka.actor.ActorSystem
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
val ws = AhcWSClient()
val req = ws.url("http://example.com").get().map{
resp => resp.body
}(system.dispatcher)
Is this the correct way of creating a AhcWsClient? And is there a way of creating a AhcWSClient without an ActorSystem?
You are probably using compile time dependency injection, otherwise you would just use #Inject() (ws: WSClient), right?.
There is one example in the docs: https://www.playframework.com/documentation/2.5.x/ScalaWS#using-wsclient
So you could write something like this in your application loader:
lazy val ws = {
import com.typesafe.config.ConfigFactory
import play.api._
import play.api.libs.ws._
import play.api.libs.ws.ahc.{AhcWSClient, AhcWSClientConfig}
import play.api.libs.ws.ahc.AhcConfigBuilder
import org.asynchttpclient.AsyncHttpClientConfig
val configuration = Configuration.reference ++ Configuration(ConfigFactory.parseString(
"""
|ws.followRedirects = true
""".stripMargin))
val parser = new WSConfigParser(configuration, environment)
val config = new AhcWSClientConfig(wsClientConfig = parser.parse())
val builder = new AhcConfigBuilder(config)
val logging = new AsyncHttpClientConfig.AdditionalChannelInitializer() {
override def initChannel(channel: io.netty.channel.Channel): Unit = {
channel.pipeline.addFirst("log", new io.netty.handler.logging.LoggingHandler("debug"))
}
}
val ahcBuilder = builder.configure()
ahcBuilder.setHttpAdditionalChannelInitializer(logging)
val ahcConfig = ahcBuilder.build()
new AhcWSClient(ahcConfig)
}
applicationLifecycle.addStopHook(() => Future.successful(ws.close))
And then inject ws to your controllers. I'm not 100% sure with this approach, I would be happy if some Play guru could validate this.
Regarding an ActorSystem, you need it only to get a thread pool for resolving that Future. You can also just import or inject the default execution context:
play.api.libs.concurrent.Execution.Implicits.defaultContext.
Or you can use your own:
implicit val wsContext: ExecutionContext = actorSystem.dispatchers.lookup("contexts.your-special-ws-config").
AFAIK this is the proper way to create the AhcWSClient - at least in 2.5.0 and 2.5.1 - as seen in the Scala API
You can, of course, always take another HTTP client - there are many available for Scala - like Newman, Spray client, etc. (although Spray is also based on Akka so you would have to create an actor system as well)

Saxon XSLT .Net Transformation: What to give in BaseURI when xml and xsl both are passed as strings

This is the code I have for Saxon Transformation of XSLT files which accepts xml and xslt and returns a transformed string. I can have either xsl 1.0 or 2.0 get processed through this function.
DocumentBuilder requires a BaseURI, even if I don't have any file format. I have provided "c:\\" as the BaseURI, inspite I have nothing to do with this directory.
Is there any better way to achieve this thing or write this function?
public static string SaxonTransform(string xmlContent, string xsltContent)
{
// Create a Processor instance.
Processor processor = new Processor();
// Load the source document into a DocumentBuilder
DocumentBuilder builder = processor.NewDocumentBuilder();
Uri sUri = new Uri("c:\\");
// Now set the baseUri for the builder we created.
builder.BaseUri = sUri;
// Instantiating the Build method of the DocumentBuilder class will then
// provide the proper XdmNode type for processing.
XdmNode input = builder.Build(new StringReader(xmlContent));
// Create a transformer for the stylesheet.
XsltTransformer transformer = processor.NewXsltCompiler().Compile(new StringReader(xsltContent)).Load();
// Set the root node of the source document to be the initial context node.
transformer.InitialContextNode = input;
StringWriter results = new StringWriter();
// Create a serializer.
Serializer serializer = new Serializer();
serializer.SetOutputWriter(results);
transformer.Run(serializer);
return results.ToString();
}
If you think that the base URI will never be used (because you never do anything that depends on the base URI) then the best strategy is to set a base URI that will be instantly recognizable if your assumption turns out to be wrong, for example "file:///dummy/base/uri".
Choose something that is a legal URI (C:\ is not).