What is the best way to iterate inside WSO2 EI? - wso2

After reading of WSO2 EI References, I still confuse about how use iterators inside a EI sequence.
In my case I have a payload like this....
{
...
"array": [
{"cpf": "12345678911"},
{"cnpj":"12345678912346"}
]
}
So I have to iterate to check if those guys exist using another web services. in order to achieve that flow, I am using the iterate mediator to split the message and then building the logic to make those validations as follows..
The code that implements this image is following:
<iterate description="" expression="//jsonObject/array" id="myid">
<target>
<sequence>
<property expression="json-eval($.array.cpf)" name="tipoCPF" scope="default" type="STRING"/>
<filter description="" xpath="boolean(get-property('tipoCPF'))">
<then>
<property expression="json-eval($.array.cpf)" name="uri.var.cpf" scope="default" type="STRING"/>
<call>
<endpoint>
<http method="get" uri-template="http://endpoint/service/{uri.var.cpf}"/>
</endpoint>
</call>
<filter regex="200" source="get-property('axis2','HTTP_SC')">
<then/>
<else>
<payloadFactory description="" media-type="json">
<format>{
"code":"400",
"error":"CPF inexistente"
}</format>
<args/>
</payloadFactory>
<property name="HTTP_SC" scope="axis2" type="STRING" value="400"/>
<respond/>
</else>
</filter>
</then>
<else>
<property expression="json-eval($.array.cnpj)" name="tipoCNPJ" scope="default" type="STRING"/>
<filter xpath="boolean(get-property('tipoCNPJ'))">
<then>
<property expression="json-eval($.array.cnpj)" name="uri.var.cnpj" scope="default" type="STRING"/>
<header name="Authorization" scope="transport" value="Basic Y29yZS5jb25zdWx0aW5nOm8xNXRyZWI="/>
<call>
<endpoint>
<http method="get" uri-template="http://endpoint/service/cnpj/{uri.var.cnpj}"/>
</endpoint>
</call>
<filter regex="200" source="get-property('axis2','HTTP_SC')">
<then/>
<else>
<payloadFactory media-type="json">
<format>{
"code":"400",
"error":"CNPJ inexistente"
}</format>
<args/>
</payloadFactory>
<property name="HTTP_SC" scope="axis2" type="STRING" value="400"/>
<respond/>
</else>
</filter>
</then>
<else>
<call>
<endpoint>
<http method="get" uri-template="http://endpoint/service/info"/>
</endpoint>
</call>
</else>
</filter>
</else>
</filter>
</sequence>
</target>
</iterate>
This iterator work as part inside of the an 'insequence'. The 'Insequence' is deigned to allow to insert new contracts information inside the Database.
Problem: after add this iterator, the service starts to make duplicated insertions inside Database. It´s looks like the iteration don´t finish in the edge of tags 'iterator'. It´s like the iteration continues to the rest of insequence.
Try: In order to solve this problem i try to add an aggregator mediator after the iterator. But or doesn't have any effect end the duplicated insert persist, or I receive an error message.
So What is the correct whey to make this iterations inside WSO2 EI?

As you have mentioned, Iteration will occur even outside the iterate tag, until the Aggregate mediator is used. To resolve this issue, you need to add the aggregate mediator inside the iterate mediator. This will stop the iteration within the iterator tag itself.
For your use case, you may need to set continueParent="true" in the IterateMediator, so that the mediation will continue after the iterate mediator for the insertion Operation in database.

Thanks for the helping Arunan!
After your answer I try to add the Aggregator as follows
The config of aggregator is following:
...
<aggregate id="NIRO">
<completeCondition>
<messageCount max="-1" min="-1"/>
</completeCondition>
<onComplete expression="//jsonObject">
<log description="" level="full" separator=";">
<property expression="json-eval($.)" name="jsonObject"/>
</log>
</onComplete>
</aggregate>
</sequence>
</target>
</iterate>
As you sad I change the iterator property 'Continue Parent' to 'true'. But the problem persists....

As proposed in the other answer, you need to
use an Aggregate mediator to close the iteration
If and only if the Iterator is set to continueParent="true"
However, I am not sure that putting it inside the <iterate> works. Here is a working solution using an Aggregate mediator right after your Iterator.
<sequence>
<iterate continueParent="true" description="" expression="//jsonObject/array" id="myid">
<target>
<your_sequence />
</target>
</iterate>
<aggregate>
<completeCondition>
<messageCount max="-1" min="-1" />
</completeCondition>
<onComplete enclosingElementProperty="//jsonObject/array" expression="/whatever/you/want"/>
</aggregate>
</sequence>
Notice expression="//jsonObject/array" you used in your Iteration, you'll need to use it in the Aggregator's enclosingElementProperty. This is how your EI will know from which iterator it should aggregate from (not 100% sure about this point, more of an empirical consideration).

Related

WSO2 EI/ESB - Calling multiple endpoints in sequence to generate a custom object

What I'm trying to achieve:
Using an Enterprise Integrator API, I want to combine the data from two data service endpoints (call it ORDERS and METADATAS) into a custom object like so:
[
{
Order: {},
Metadata: {}
}, ...
]
The API takes in a single parameter at the moment, USER_ID.
When I run the <call> mediator in blocking mode, I get the following problem:
Current Params: {USER_ID={1,1}, ID=test}
or
Current Params: {USER_ID={test,1}}
When in non-blocking mode, the first call works successfully and the second call can't proceed. It seems it can't access the USER_ID parameter from the URL, nor from the stored property. I thought properties were supposed to persist values?
It should be noted that the values in the logs are 100% accurate (USER_ID property is showing correct value).
INFO {org.apache.synapse.mediators.builtin.LogMediator} - _TEST = 1, _TEST2 = 1, _TEST3 = 1
Here's what I've got thus far for the inSequence:
<resource methods="GET" uri-template="/test?USER_ID={USER_ID}">
<inSequence>
<property expression="get-property('query.param.USER_ID')" name="USER_ID" scope="default" type="STRING"/>
<property expression="$url:USER_ID" name="uri.var.USER_ID" scope="default" type="STRING"/>
<call blocking="true">
<endpoint>
<http method="get" statistics="enable" trace="enable" uri-template="https://localhost:8243/services/ORDERS_DataService/user/{query.param.USER_ID}"/>
</endpoint>
</call>
<log level="custom">
<property expression="get-property('USER_ID')" name="_TEST"/>
<property expression="get-property('query.param.USER_ID')" name="_TEST2"/>
<property expression="get-property('uri.var.USER_ID')" name="_TEST3"/>
</log>
<enrich>
<source clone="false" type="body"/>
<target property="_ORDERS" type="property"/>
</enrich>
<filter regex="200" source="get-property('axis2', 'HTTP_SC')">
<then>
<log level="custom">
<property name="switchlog" value="Case: first call successful"/>
</log>
<call blocking="true">
<endpoint>
<http method="get" statistics="enable" trace="enable" uri-template="https://localhost:8243/services/ORDERS_DataService/metadata/user/{query.param.USER_ID}"/>
</endpoint>
</call>
<enrich>
<source clone="false" type="body"/>
<target property="_METADATAS" type="property"/>
</enrich>
<log level="custom">
<property expression="get-property('_METADATAS')" name="_METADATAS"/>
</log>
</then>
<else/>
</filter>
</inSequence>
</resource>
What have I tried?
Using {USER_ID}, {uri.var.USER_ID}, {$url:USER_ID}, and {query.param.USER_ID} in both of the uri-templates for the <call>s.
Tried non-blocking and blocking mode, getting different results.
My problem is just (for now) with the calling of both sequences and assigning to properties.
I planned to bring them all together with an iterate, and something like the below to map the properties to each of the items in the array I'm looking for:
<!-- Assuming iterate provides _ORDER and _METADATA whilst looping -->
<format>
{
"Order":"$1",
"Metadata":"$2"
}
</format>
<args>
<arg expression="get-property('_ORDER')"/>
<arg expression="get-property('_METADATA')"/>
</args>
And if that doesn't work I was going to evaluate an XSLT transform on the two items, but I haven't thought that far ahead yet, stuck on the two calls not working properly.
I've figured out the answer to the issue. Found this curious line buried in the documentation [1]:
<property name="REST_URL_POSTFIX" scope="axis2" action="remove"/>
Adding it just after grabbing the query parameter works! It does exactly what it says, apparently a Postfix is applied by default. Now to look for a global way to turn that off automatically!
References:
[1] https://docs.wso2.com/display/EI650/Configuring+Specific+Use+Cases

Implement Iterate in parrelel execution with aggregate in a single sequence WSO2 ESB5

What I want to do is implement Iterate in parrelel execution with aggregate in a single sequence (without using call/send medior within it).
When I implement the Aggregate in the out sequence this works fine as mentioned below.
<inSequence>
<property name="it_count" scope="operation" type="STRING" value="0"/>
<iterate expression="//symbols/symbol">
<target>
<sequence>
<log level="custom">
<property name="ITERATING..." expression="$body"/>
</log>
<enrich>
<source type="inline">
<out xmlns="">TEST</out>
</source>
<target xpath="//symbol"/>
</enrich>
<log level="custom">
<property name="ITERATING..." expression="$body"/>
</log>
<loopback/>
</sequence>
</target>
</iterate>
</inSequence>
<outSequence>
<property name="response" scope="default">
<response xmlns=""/>
</property>
<aggregate>
<completeCondition>
<messageCount max="-1" min="-1"/>
</completeCondition>
<onComplete enclosingElementProperty="response" expression="//out">
<log level="custom">
<property name="AGGREGATING..." expression="$body"/>
</log>
</onComplete>
</aggregate>
<send/>
</outSequence>
But I'm facing a difficulty of doing it in the same seqence as like below. It doesn't come even to the Aggegate log. I tried various ways but still faced.
<inSequence>
<property name="it_count" scope="operation" type="STRING" value="0"/>
<iterate expression="//symbols/symbol">
<target>
<sequence>
<log level="custom">
<property name="ITERATING..." expression="$body"/>
</log>
<enrich>
<source type="inline">
<out xmlns="">TEST</out>
</source>
<target xpath="//symbol"/>
</enrich>
<log level="custom">
<property name="ITERATING..." expression="$body"/>
</log>
<loopback/>
</sequence>
</target>
</iterate>
<property name="response" scope="default">
<response xmlns=""/>
</property>
<aggregate>
<completeCondition>
<messageCount max="-1" min="-1"/>
</completeCondition>
<onComplete enclosingElementProperty="response" expression="//symbol">
<log level="custom">
<property name="AGGREGATING..." expression="$body"/>
</log>
</onComplete>
</aggregate>
<respond/>
</inSequence>
<outSequence/>
I know if I use Call/Send mediator within the Iterate I can do this in one sequence. But in my case I don't use one in there.
Can anyone give a clue for this.
In the 2nd case, add attribute continueParent="true" on iterate mediator (you don't want this mediation to stop after iterate mediator) and remove loopback mediator (you don't want to send anything to an out sequence)

local entry as filter criteria in wso2 esb mediation

I have a sequence as part of proxy service which filters based on "Source and Regular Expression". I have defined source as element value coming as part of SOAP request and regular expression as "local entry defined in ESB". However, result is not what I am expecting.
Local Entry is defined as Inline Text (myFields) - FIELD1|FIELD2|FIELD3
Mediation sequence is defined as -
<sequence xmlns="http://ws.apache.org/ns/synapse" name="007">
<property xmlns:ns="http://org.apache.synapse/xsd" name="fieldName" expression="$body/fieldName/text()" scope="default" type="STRING"/>
<filter xmlns:ns="http://org.apache.synapse/xsd" source="get-property('fieldName')" regex="get-property('myFields')">
<then>
<log level="full" separator="*****YES*********">
<property name="myFields" expression="get-property('myFields')"/>
</log>
</then>
<else>
<log level="full" separator="*********NO**************">
<property name="myFields" expression="get-property('myFields')"/>
</log>
</else>
</filter>
</sequence>
When I am sending SOAP request as -
<body>
<fieldName>FIELD1</fieldName>
</body>
execution is always going to else part. Any suggestion ?
With filter mediator, regex attribute must be a string, not an expression.
You can use XPATH2 "matches"
Sample :
<inSequence>
<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
<property name="fieldName" expression="$body/fieldName/text()"/>
<property xmlns:fn="http://www.w3.org/2005/xpath-functions" name="match" expression="fn:matches(syn:get-property('fieldName'),syn:get-property('myFields'))"/>
<filter source="get-property('match')" regex="true">
<then>
<log level="full" separator="*****YES*********">
<property name="myFields" expression="get-property('myFields')"/>
</log>
</then>
<else>
<log level="full" separator="*********NO**************">
<property name="myFields" expression="get-property('myFields')"/>
</log>
</else>
</filter>
<log level="full"/>
</inSequence>

Getting error in Aggregate Mediator?

I am using iterate mediater and aggregate mediator.
My request is:
<p:GetPersonDataOperation xmlns:p="http://tempuri.org">
<!--1 or more occurrences-->
<xs:ID xmlns:xs="http://tempuri.org">1</xs:ID>
</p:GetPersonDataOperation>
and response is :
<GetPersonDataCollection xmlns="http://tempuri.org">
<GetPersonData>
<AppInstanceID>1</AppInstanceID>
<RecordID>349</RecordID>
<ID>1</ID>
<Name>name</Name>
<LastName>lastname</LastName>
<Descr>description</Descr>
<Address>Park Street</Address>
</GetPersonData>
</GetPersonDataCollection>
If i don't use Aggregate mediator then i get the above response, But if i use Aggregate mediator i get request timeOut Exception
My in Sequence is :
<sequence xmlns="http://ws.apache.org/ns/synapse" name="GetPersonDataOperationSeq">
<iterate xmlns:xs="http://tempuri.org" xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://tempuri.org" preservePayload="true" attachPath="//p:GetPersonData" expression="//p:GetPersonData/xs:ID" id="Iterator1">
<target>
<sequence>
<property name="ID" expression="//xs:ID" scope="default" type="STRING"/>
<payloadFactory>
<format>
<p:GetPersonData>
<xs:ID>$1</xs:ID>
</p:GetPersonData>
</format>
<args>
<arg expression="get-property('ID')"/>
</args>
</payloadFactory>
<send receive="AggregatorSeq">
<endpoint key="GetPersonDataEP"/>
</send>
</sequence>
</target>
</iterate>
</sequence>
And From The above in sequence i am redirecting to another Sequence called AggregatorSeq and my AggregatorSeq is:
<sequence xmlns="http://ws.apache.org/ns/synapse" name="AggregatorSeq">
<log level="custom">
<property name="CamHereProp" value="*******************Yes??????????????**********************************************"/>
</log>
<aggregate>
<completeCondition>
<messageCount min="-1" max="-1"/>
</completeCondition>
<onComplete xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://tempuri.org" expression="//p:GetPersonDataCollection">
<send/>
</onComplete>
</aggregate>
</sequence>
What am i doing wrong.Looking forward to your answers.Thanks in advance
Try adding the ID of the iterator to your aggregator. In your case it should be like,
<aggregate id="Iterator1">
Also, if each of your response body starts with <GetPersonData> then you need to add that to the expression in onComplete.
<onComplete xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://tempuri.org" expression="//p:GetPersonData">
Can you specify the issue with the code you have given. You can use <log level="full"/> to debug till what level is your configuration gets executed.

WSO2 ESB If Then Else, authenticate or Error

This may be a basic question, I am just getting used to the WSO2 lingo. I have two services that I can deploy independently with WSDLs and pass the proper SOAP request, and return information accordingly. Now I want to combine them into an 'If then, else' statement sort of deal. This would be set up in a sequence of some sort I believe, just not sure how with the filters.
Send in request with authentication request and info request
Do authentication request – continue if passes, 401 on failure
Do info request – get info
Return the info
If you have a sample I could follow or point me to one of the hundreds WSO2 has, I just haven't been able to pull much from them. XML source example for the config could work also. Thanks for the help, and for my ignorance of WSO2 lingo, and workflow.
You can have a look at filter mediator to filter messages based on conditions Entitlement Mediator. You can find samples here as a reference which will be helpful for your use case.
So I ended up with something very similar to this. If someone down the road comes across this and looking for the wso2 configurations.
<proxy name="name"
transports="https http"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<property xmlns:ns1="ns1"
xmlns:ns="ns"
name="userID"
expression="//ns:AuthenticateRequest/ns:Credentials/ns1:userID"
scope="default"
type="STRING"/>
<property xmlns:ns1="ns1"
xmlns:ns="ns1"
name="password"
expression="//ns:AuthenticateRequest/ns:Credentials/ns1:password"
scope="default"
type="STRING"/>
<log>
<property name="userID" expression="get-property('userID')"/>
<property name="password" expression="get-property('password')"/>
</log>
<header name="Action"
value="http://services.com:port/AuthenticateSecureCredential"/>
<send receive="AuthRecvSequence">
<endpoint>
<address uri="http://server.com:port/DefaultAuthenticationService"/>
</endpoint>
</send>
</inSequence>
</target>
</proxy>
<sequence name="AuthRecvSequence">
<filter xmlns:ns="ns"
source="//ns:AuthenticateSecureCredentialResponse/ns:isAuthenticated"
regex="false">
<then>
<makefault version="soap11">
<code xmlns:soap11Env="http://schemas.xmlsoap.org/soap/envelope/"
value="soap11Env:VersionMismatch"/>
<reason value="Not Authenticated"/>
<role/>
</makefault>
<header name="To" action="remove"/>
<property name="RESPONSE" value="true" scope="default" type="STRING"/>
<send/>
<drop/>
</then>
<else>
<payloadFactory>
<format>
<ns:INFO xmlns:ns="ns"
xmlns:ns1="ns1">
<ns:secureCredentials>
<ns1:userID>$1</ns1:userID>
<ns1:password>$2</ns1:password>
</ns:secureCredentials>
</ns:INFORequest>
</format>
<args>
<arg expression="get-property('userID')"/>
<arg expression="get-property('password')"/>
</args>
</payloadFactory>
<header name="Action"
value="http://services.com/GetINFO"/>
<send receive="INFOrRecvSeq">
<endpoint>
<address uri="http://server:port/INFOService"/>
</endpoint>
</send>
</else>
</filter>
</sequence>
<sequence name="INFORecvSeq">
<send/>
</sequence>
<sequence name="main">
<description>The main sequence for the message mediation</description>
</sequence>