wso2esb iterate throught wso2dss result - wso2

I want to write next target sequence of actions:
1. call DSS to receive a list of customers
2. enrich each customer's by separate call another DSS service.
So, I thought I should call callout mediator and then iterate it's result using iterator. But I can't understand what should I write in the iterator.
And another question - am I right, that the result of each iteration will be attached under the 'customer' tag?
Details:
XML which returned from DSS is next:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<customers xmlns="crm.crm.crm">
<customer>
<customerId>1</customerId>
<name>Customer #1</name>
<birthdate>2017-01-15T14:54:12.000+03:00</birthdate>
</customer>
</customers>
</soapenv:Body>
</soapenv:Envelope>
Sequence:
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="BatchSequence" statistics="enable" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
<log description="">
<property name="text" value="Start batch seq"/>
</log>
<payloadFactory description="create dss request" media-type="xml">
<format>
<soapenv:Envelope xmlns:crm="crm.crm.crm" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<crm:getCustomers>
<crm:batchSize>3</crm:batchSize>
</crm:getCustomers>
</soapenv:Body>
</soapenv:Envelope>
</format>
<args/>
</payloadFactory>
<callout action="urn:getCustomers" description="dss: main object" initAxis2ClientOptions="false" serviceURL="http://192.168.3.32:9765/services/CrmDataService?wsdl">
<source type="envelope"/>
<target key="customers"/>
</callout>
<log description="">
<property expression="get-property('customers')" name="text"/>
</log>
<iterate description="Enrich customers" expression="/soapenv:Envelope/soapenv:Body/customers/customer" id="iterateId" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<target>
<sequence>
<property description="customerId" expression="/customerId" name="customerID" scope="default" type="STRING"/>
<log description="">
<property expression="get-property('customerID')" name="text"/>
</log>
</sequence>
</target>
</iterate>
<log description="">
<property name="text" value="End batch seq"/>
</log>
</sequence>
output:
[2017-01-27 10:17:17,371] INFO - LogMediator To: , MessageID: urn:uuid:d628e361-beb8-4c26-b06d-3901227ad76a, Direction: request, text = Start batch seq
[2017-01-27 10:17:18,558] INFO - LogMediator To: , MessageID: urn:uuid:d628e361-beb8-4c26-b06d-3901227ad76a, Direction: request, text = 1Customer #12017-01-15T14:54:12.000+03:002Customer #22016-12-16T14:54:20.000+03:003Customer #32016-10-27T14:54:21.000+03:00
[2017-01-27 10:17:18,559] WARN - RuntimeStatisticCollector Events occur after event collection is finished, event - urn_uuid_d628e361-beb8-4c26-b06d-3901227ad7
6a231160071781262
Update 1
Some working code. Not sure that this is correct, because I'm a little bit confusing for a PayloadFactory here..
<payloadFactory description="" media-type="xml">
<format>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>$1</soapenv:Body>
</soapenv:Envelope>
</format>
<args>
<arg evaluator="xml" expression="get-property('customers')"/>
</args>
</payloadFactory>
<iterate continueParent="true" description="" expression="$body/crm:customers/crm:customer" sequential="true" xmlns:crm="crm.crm.crm">
<target>
<sequence>
<property expression="//crm:customerId" name="customerID" scope="default" type="STRING"/>
<log>
<property expression="get-property('customerID')" name="text"/>
</log>
</sequence>
</target>
</iterate>
Update 2
I figured a main problem - callout mediator doesnt' put response to the envelope context(if I understand right). So, we can't use just property to link it with iterator, so, in this case we should link them using smth like Payload factory. Not very usifull
If smbdy knows how to do it more simple(dirrect passing property to the iterator) - pls write to me.
Solution - to use Call medator. Works fine.
thanks to all!

xml nodes in the dss response belongs to a namespace "crm.crm.crm" and you must refer it in your xpath
with iterate mediatior, if you want to preserve source payload, you must use an attribute named preservePayload="true" and tell where the xml framents must be attached with the attribute attachPath ortherwise, inside interate's sequence, you will only have your xml fragment in the soap body
This is a sample that works without preserving source payload :
<iterate xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:crm="crm.crm.crm" expression="$body/crm:customers/crm:customer" id="iterateId">
<target>
<sequence>
<property expression="$body/crm:customer/crm:customerId" name="customerID" scope="default" type="STRING"/>
<log>
<property expression="get-property('customerID')" name="text"/>
</log>
</sequence>
</target>
</iterate>
By default, mediators after iterate will not be executed. If you want to continue the mediation, use attribute continueParent="true"

Related

wso2esb - vfs-Proxy read/write binary with payloadFactory and mtom-optimization

Scenario:
I have a vfs-Proxy monitoring a folder for files with the extention .pdf.
The Proxy parses the file-content in base64binary into
<axis2ns#:binary xmlns:axis2ns#="http://ws.apache.org/commons/ns/payload">JVBERi0xLjMKJfbk/N8K...</axis2ns#:binary>
# is a incremental number and as far as i know, i don't have an influence on this mater. With a payloadFactory i reformat the payload to be surrounded by
<datatype:pdf xmlns:datatype="http://mynamespace.org/payload"> instead.
Problem:
The resulting file looks like this:
--MIMEBoundary_e1f5b2321e28e0a638b52a178d5c7ee40c2f3ae08cd43818
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.f1f5b2321e28e0a638b52a178d5c7ee40c2f3ae08cd43818#apache.org>
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<datatype:pdf xmlns:datatype="http://mynamespace.org/payload">
JVBERi0xLjMKJfbk/N8K...
</datatype:pdf>
</soapenv:Body>
</soapenv:Envelope>
--MIMEBoundary_e1f5b2321e28e0a638b52a178d5c7ee40c2f3ae08cd43818--
instead of an actual pdf-document. What's necessary to change that? I'm pretty sure it has so be something with the content-types or the payloadFactory. Is there a way to specify which tags are treated as payload? I have already tried to a few different types, settings and searched but coudn't find a solution. If i interpret the resulting message correct mtom isn't working as it is supposed to. Shouldn't there be a
<xop:Include href="...">
inside the resulting message instead of the inline base64binary?
Simple Code for reproduction:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="VFSVFS"
transports="vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="OUT_ONLY" value="true"/>
<property name="enableMTOM" value="true" scope="axis2" type="STRING"/>
<property name="enableSWA" value="false" scope="axis2" type="STRING"/>
<property name="transportNonBlocking"
value="true"
scope="axis2"
action="remove"/>
<payloadFactory media-type="xml">
<format>
<datatype:pdf xmlns:datatype="http://mynamespace.org/payload">
$1
</datatype:pdf>
</format>
<args>
<arg evaluator="xml" expression="$body/*[1]"/>
</args>
</payloadFactory>
<property name="messageType" value="application/octet-stream" scope="axis2"/>
<property name="transport.vfs.ReplyFileName" expression="fn:concat(fn:substring-after(get-property('MessageID'), 'urn:uuid:'), 'abc.pdf')" scope="transport"/>
<property name="transport.vfs.Streaming" value="true" scope="transport" type="STRING"/>
<send>
<endpoint xmlns="http://ws.apache.org/ns/synapse" name="FileOut_VFS">
<address uri="vfs:file:///home/user/Development/data/testfiles/init/out" optimize="mtom" />
</endpoint>
</send>
<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">DELETE</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.FileURI">file:///home/user/Development/data/testfiles/init/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///home/user/Development/data/failure</parameter>
<parameter name="transport.vfs.Locking">false</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.pdf</parameter>
<parameter name="transport.vfs.ContentType">application/octet-stream</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.FailedRecordsFileDestination">file:///home/user/Development/data/failure</parameter>
<description/>
</proxy>
The code above is just for the purpose of simplification. The actual project performs those key operations:
read-in a pdf
aggregate the soap-message with additional information from various sources
manipulate the pdf-content with a custom mediator
write-out the new pdf
For the purpose of aggregation and manipulation i want to reformat the initial body with the payloadFactory. But as soon as I change a thing, the message no longer arrives in a fitting format.
This applies even in the simple code above.
Here a few examples of tried out combination's and their results:
<ns:binary xmlns:ns="http://ws.apache.org/commons/ns/payload">$1</ns:binary>
WITH OR WITHOUT
<property name="ContentType" value="application/octet-stream" scope="axis2"/>
RESULTS IN
INFO - AxisEngine [MessageContext: logID=6143bc348d4852f3ffa02dba72391ab0860fe7c27625f167] ContentID is null
[2015-10-16 08:19:40,923] ERROR - AsyncCallback ContentID is null
java.lang.RuntimeException: ContentID is null
EVEN WITH <property name="enableMTOM" value="false" scope="axis2" type="STRING"/>
<ns:text xmlns:ns="http://ws.apache.org/commons/ns/payload">$1</ns:text>
WITH OR WITHOUT
<property name="ContentType" value="application/octet-stream" scope="axis2"/> RESULTS IN
File with base64binary as plain/text in it -> no functioning pdf.
I guess, you should change the format to the following so it will be treated as text.
<format>
<ns:text xmlns:ns="http://ws.apache.org/commons/ns/payload">$1</ns:text>
</format>
Hope that helps. I guess you won't be able to open the file because it will have the wrong encoding.
Regards
Martin
Not sure if I get it right. If I look at the proxy it seems that you'd like to read a pdf and write it to some other location. So you just could use the following to write the file.
<send>
<endpoint>
<address uri="vfs:file:///home/user/Development/data/testfiles/init/out"/>
</endpoint>
</send>
Inside the ESB every message/file that arrives will be converted to a soap message.
Regards
Martin
From this post on the ws-commons-dev mailing list finally send me in the right direction.
The problem is that to make this work the OMElement object has be have a Content ID OR have the flag isBinary set to true.
This is a attribute on the OMTextImpl and from what I can tell this has to be done on the object and can not be set from the XML payload. So in the end I had to add a script mediator to do this.
<payloadFactory media-type="xml">
<format>
<ns:binary xmlns:ns="http://ws.apache.org/commons/ns/payload">$1</ns:binary>
</format>
<args>
<arg evaluator="xml" expression="$body/attachments/content"/>
</args>
</payloadFactory>
<script language="js">
<![CDATA[
var binaryNode =
mc.getEnvelope().getBody().getFirstElement().getFirstOMChild();
binaryNode.setBinary(true);
]]>
</script>
In the example above the base64 data comes in as part of the incoming message.

How to invoke DSS service from ESB - I feel that I'm close, I just need a nudge

I think i'm getting close with this. I'm trying to invoke an insert call on a DSS service from and ESB in WSO2.
I have the DSS service setup and I am able to insert data into the table from the 'try it' link. I copied the WSDL to the ESB and referenced the endpoint. I can see the insert operation from the ESB try it service. I put in my data and click send. I see a 'success' response come back but nothing is being added to the table.
Is anyone willing to nudge me in the right direction with this?
Thank you!
Response from try it service
<success details="in-only operation"/>
proxy
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="MyProxy"
transports="https,http,local,vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="targetfilename" value="TITLES"/>
<log level="full"/>
<clone/>
</inSequence>
</target>
<publishWSDL key="InsertServiceWSDL"/>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileURI">file:///var/process
/rrin</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///var/process
/rroriginal</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///var/process
/rrfault</parameter>
<parameter name="transport.vfs.FileNamePattern">TITLES.xml</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<description/>
</proxy>
You'll need to add an address endpoint pointing to the DSS service. Refer to this sample which is on how to define a proxy service for an axis2 web service. Your scenario is very similar to this.
Here is how I did it. The call will now hit the DSS and insert the data into the table. I am seeing some errors in the log for each XML row it processes and send to the DSS. I'm not sure why yet. I'm still researching that.
Edit: I changed the call mediator to a send mediator and that fixed this issue. I am no longer getting this error message.
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
Message: Premature end of file.
Here is my sequence.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="MySequence">
<log level="custom">
<property name="sequence" value="MySequence"></property>
</log>
<property xmlns:ns="http://org.apache.synapse/xsd" name="filename" expression="get-property('transport', 'FILE_NAME')"></property>
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="show-name" expression="get-property('filename')"></property>
<property xmlns:ns="http://org.apache.synapse/xsd" name="file-name" expression="get-property('targetfilename')"></property>
</log>
<iterate xmlns:ns2="http://org.apache.synapse/xsd" xmlns:ns="http://org.apache.synapse/xsd" xmlns:z="RowsetSchema" expression="//z:row" id="It1">
<target>
<sequence>
<property name="Id" expression="//z:row/#ID"></property>
<property name="vch" expression="//z:row/#vch"></property>
<log level="custom">
<property name="showids" expression="get-property('Id')"></property>
<property name="showvch" expression="get-property('vch')"></property>
</log>
<filter xpath="//z:row[starts-with(#vch, '978')]">
<then>
<payloadFactory media-type="xml">
<format>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:p="http://ws.wso2.org/dataservice">
<soapenv:Body>
<p:insert_AR_operation>
<p:ID xmlns:xs="http://ws.wso2.org/dataservice">$1</p:ID>
<p:vch xmlns:xs="http://ws.wso2.org/dataservice">$2</p:vch>
</p:insert_AR_operation>
</soapenv:Body>
</soapenv:Envelope>
</format>
<args>
<arg expression="get-property('Id')" evaluator="xml"></arg>
<arg expression="get-property('vch')" evaluator="xml"></arg>
</args>
</payloadFactory>
<log level="custom">
<property name="sequence" value="Calling LevelsAR_ISBNService"></property>
</log>
<property name="HTTP_METHOD" value="POST" scope="axis2"></property>
<property name="SOAPAction" value="insert_AR_operation" scope="transport"></property>
<send>
<endpoint>
<address uri="http://*.*.*.*:****/services/AR_Service.HTTPEndpoint/"></address>
</endpoint>
</send>
</then>
<else>
<log level="custom">
<property name="sequence" value="Condition Drop"></property>
</log>
<drop></drop>
</else>
</filter>
</sequence>
</target>
</iterate>
<property xmlns:ns2="http://org.apache.synapse/xsd" xmlns:ns="http://org.apache.synapse/xsd" name="transport.vfs.ReplyFileName" expression="fn:concat(get-property('SYSTEM_DATE', 'yyMMddHHmmss'), '-', get-property('filename'))" scope="transport"></property>
<property name="OUT_ONLY" value="true"></property>
</sequence>

WSO2 - Iterate mediator. Iteration should continue after error

In WSO2 iterate mediator, if an error happens in one iteration, I want the subsequent iterations to continue. How do we ensure that?
I have done the following, but the execution does not continue after the 1st error.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="IterateErrprTest" onError="onErr2Cric">
<payloadFactory media-type="xml">
<format>
<format xmlns="">
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<LIST>
<DETAIL></DETAIL>
<DETAIL></DETAIL>
</LIST>
</soapenv:Body>
</soapenv:Envelope>
</format>
</format>
</payloadFactory>
<iterate xmlns:ns="http://org.apache.synapse/xsd" continueParent="true" expression="//LIST/DETAIL" sequential="true">
<target sequence="2Cric"></target>
</iterate>
</sequence>
<!--- In my target sequence 2Cric, I have introduced an error by calling a non-existent website -->
<sequence xmlns="http://ws.apache.org/ns/synapse" name="2Cric" onError="onErr2Cric">
<log>
<property name="KKLK" value="KKLK KKLK"></property>
</log>
<callout serviceURL="http://www.cricinfo22.com">
<source type="envelope"></source>
<target key="Output"></target>
</callout>
</sequence>
<!--- The error sequence -->
<sequence xmlns="http://ws.apache.org/ns/synapse" name="onErr2Cric">
<log>
<property name="onErr2CricError" value="2Cric error has happened"></property>
</log>
</sequence>
The errors in WSO2 and similar to errors in Java and they are not the same as exceptions in java. Basically, you are not supposed to handle errors.
This means that when there is an error in the iterate mediator, it will exit from the loop.
We handled this by designing around it. We called the same sequence as our error handling sequence with the use of onError attribute. But we do not process the failed record. The code is some what like this.
<sequence xmlns="http://ws.apache.org/ns/synapse"
name="MySequence"
onError="MySequence"
trace="enable"
statistics="enable">
<!-- Error logging is done -->
<!-- Get the list of records to process from the database excluding the ones failed -->
<!-- Our code here does not get the failed -->

Invalid UTF-8 start byte 0x8b error calling a SOAP service after receiving result of a REST call - WSO2 ESB 4.8.1

Implementing a service chain in WSO2 ESB 4.8.1 we have a sequence that calls a REST service and in the receiving sequence we call a SOAP service. In doing so, we get the following error,
<?xml version='1.0' encoding='utf-8'?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body><ns2:Fault xmlns:ns3="http://www.w3.org/2003/05/soap-envelope" xmlns:ns2="http://schemas.xmlsoap.org/soap/envelope/"><faultcode>ns2:Client</faultcode>
<faultstring>Couldn't create SOAP message due to exception: XML reader error: com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 start byte 0x8b (at char #2, byte #-1)</faultstring>
</ns2:Fault>
</S:Body>
</S:Envelope>[\r][\n]" {org.apache.synapse.transport.http.wire}
This happens irrespective of whether we use the result of the previous call as input or even when actual values are hard-coded in the PayloadFactory. Moreover the SOAP service never receives the request.
When the SOAP service is called on it's own through the ESB or through soapUI, it works fine.
The sequence that calls the SOAP service is as follows,
<template xmlns="http://ws.apache.org/ns/synapse" name="tmpl_get_customer_details">
<sequence>
<payloadFactory media-type="xml">
<format>
<getCustomer xmlns="http://customer.frontend.ws.utilibill.com.au/">
<customer xmlns="">
<custNo>12965</custNo>
</customer>
<login xmlns="">
<password>password</password>
<userName>username</userName>
</login>
</getCustomer>
</format>
<args>
<arg xmlns:ns2="http://org.apache.synapse/xsd" xmlns:ns="http://org.apache.synapse/xsd" expression="$ctx:utilibill_customer_number" evaluator="xml"></arg>
</args>
</payloadFactory>
<log level="full">
<property name="POSITION" value="tmpl_get_customer_details"></property>
</log>
<header name="Action" scope="default" value="http://customer.frontend.ws.utilibill.com.au/UtbCustomer/getCustomerRequest"></header>
<header name="To" scope="transport" value="http://customer.frontend.ws.utilibill.com.au/UtbCustomer/getCustomerRequest"></header>
<property name="POST_TO_URI" value="true" scope="axis2" type="STRING"></property>
<log level="full">
<property name="POSITION" value="CALLING_CUSTOMER_SERVICE"></property>
</log>
<send receive="seq_get_services">
<endpoint key="UtbCustomer"></endpoint>
</send>
</sequence>
</template>
Can anyone please point out what maybe causing this?

Converting a SOAP request to a HTTP 200 return on wsO2 ESB

I have a instance where I need to implement a proxy service that takes a SOAP message, forwards it to an internal system (SOAP) and returns a HTTP 200 response to the original server.
Basically the response should be completely void of any soap detail (there's technically no output message in the WSDL that I have to implement).
Here's what I have so far (which simply takes the request & echos it back as the response):
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="ExampleHttp200Return"
transports="http"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<log level="full" separator=", - inSequence: received - "/>
<header name="To" action="remove"/>
<property name="RESPONSE" value="true" scope="default" type="STRING"/>
<property name="SC_ACCEPTED" value="false" scope="axis2"/>
<property name="HTTP_SC" value="200" scope="axis2"/>
<send/>
</inSequence>
</target>
<description/>
</proxy>
If you want to send back an empty response, you can add that before send mediator :
<property name="messageType" scope="axis2" value="text/plain"/>
<enrich>
<source type="inline">
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
<soapenv:Header/>
<soapenv:Body>
<text xmlns="http://ws.apache.org/commons/ns/payload"/>
</soapenv:Body>
</soapenv:Envelope>
</source>
<target type="envelope"/>
</enrich>