WSO2 ESB "Send" mediator not sending to Out Sequence of Proxy - wso2

Question

I am having a slight issue with 2 sequence's I have created in a WSO2 Esb proxy service.
Here is my configuration:
My Proxy (Pepkor_Product_Search_Proxy) needs to search a web service and data service for product information and bring back the response(aggregated if need be).
My Proxy service calls 2 services ; a data service hosted on the DSS and a Web service hosted on the AS. The proxy request is sent to 2 "request sequences" that format the request to match the expected messages of the respective service calls on the AS and DSS. Once the responses are received , I send them to the 2 "response sequences" in order to format them according(in line with the WSDL) to the proxy service response. However once reaching the send mediators in these sequences , execution stops and they are not received by the Out Sequence of the proxy.
The send mediators are using the default behavior( which should be to send to out sequence of proxy) such as :
All 4 sequences( 2 Rq + 2Rs ) are dynamic and saved in system_/conf: registry
I recieve the message back in soap from only the one service ( it seems the rs sequences are responding and not the out seq of proxy!)
I want to know how I can get these "mediated" responses back into my proxies out sequence...
I have tried the sequence mediator instead of clone but to no avail.
Please see below : the Rq mssg , the proxy and "response" sequences synapse configurations as well as the proxy carbon log.
Thanks in advance and is there any other info you need let me know.
SOAP Rq:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:prod="http://za.co.pepkor/product_service/">
<soapenv:Header/>
<soapenv:Body>
<prod:productSearchReq>
<!--Optional:-->
<productName>NIKE_SHIRTS</productName>
<!--Optional:-->
<productSize>7</productSize>
<!--Optional:-->
<productColour>RED</productColour>
</prod:productSearchReq>
</soapenv:Body>
</soapenv:Envelope>
Proxy:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="Pepkor_Product_Search_Proxy"
transports="https http" startOnLoad="true" trace="disable">
<target>
<inSequence>
<log level="custom">
<property name="Sending request to :"
value="ackServiceSearchSeqRq and shcServiceSearchSeqRq sequences..." />
</log>
<clone sequential="true">
<target sequence="conf:/ackServiceSearchSeqRq" />
<target sequence="conf:/shcServiceSearchSeqRq" />
</clone>
</inSequence>
<outSequence>
<log level="custom">
<property name="Out Sequence reached"
value="++++++++++++++=========Aggregating now========+++++++++++++++++==" />
</log>
<aggregate>
<completeCondition>
<messageCount min="-1" max="-1" />
</completeCondition>
<onComplete expression="//productDetails">
<log level="custom" separator=",">
<property name="::::"
value="======================= Sending Back the Aggregated Responses. ===============" />
</log>
</onComplete>
</aggregate>
<respond />
</outSequence>
<faultSequence />
</target>
<publishWSDL key="gov:service_integration/wsdls/PepKorProductSearch.wsdl" />
</proxy>
Ack Service Rs Sequence
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="ackServiceSearchSeqRs">
<log level="custom">
<property name="Below Response recieved from Ack-WS" value="==================" />
</log>
<log level="full" />
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="product_Colour"
expression="//ns1:productSearchResponse/ns1:return/ax23:product_Colour/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:Colour : "
expression="$ctx:product_Colour" />
</log>
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="product_ID"
expression="//ns1:productSearchResponse/ns1:return/ax23:product_ID/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:ID : "
expression="$ctx:product_ID" />
</log>
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="product_Name"
expression="//ns1:productSearchResponse/ns1:return/ax23:product_Name/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:Name : "
expression="$ctx:product_Name" />
</log>
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="product_Size"
expression="//ns1:productSearchResponse/ns1:return/ax23:product_Size/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:Size : "
expression="$ctx:product_Size" />
</log>
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="source"
expression="//ns1:productSearchResponse/ns1:return/ax23:source/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:Source : "
expression="$ctx:source" />
</log>
<property xmlns:ns1="http://ack.pepkor.org" xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ax23="http://ack.pepkor.org/xsd" name="type"
expression="//ns1:productSearchResponse/ns1:return/ax23:type/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="ackRs:Type : "
expression="$ctx:type" />
</log>
<payloadFactory media-type="xml">
<format>
<productDetails xmlns:pep="http://za.co.pepkor/product_service/">
<productID>$1</productID>
<productName>$2</productName>
<productSize>$3</productSize>
<productColour>$4</productColour>
<productType>$5</productType>
<sourceID>$6</sourceID>
</productDetails>
</format>
<args>
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:product_ID" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:product_Name" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:product_Size" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:product_Colour" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:type" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:source" />
</args>
</payloadFactory>
<send />
</sequence>
Shc Service Rs Sequence
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="shcServiceSearchSeqRs">
<log level="custom">
<property name="Below response recieved from Shc-DS" value="==================" />
</log>
<log level="full" />
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Product_ID" expression="//ns1:productDetails/ns1:Product_ID/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Product_ID : "
expression="$ctx:Product_ID" />
</log>
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Product_Name" expression="//ns1:productDetails/ns1:Product_Name/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Product_Name : "
expression="$ctx:Product_Name" />
</log>
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Product_Size" expression="//ns1:productDetails/ns1:Product_Size/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Product_Size : "
expression="$ctx:Product_Size" />
</log>
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Product_Colour" expression="//ns1:productDetails/ns1:Product_Colour/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Product_Colour : "
expression="$ctx:Product_Colour" />
</log>
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Product_Type" expression="//ns1:productDetails/ns1:Product_Type/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Product_Type : "
expression="$ctx:Product_Type" />
</log>
<property xmlns:ns1="SHC" xmlns:ns="http://org.apache.synapse/xsd"
name="Source" expression="//ns1:productDetails/ns1:Source/text()"
scope="default" type="STRING" />
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" name="shcRs: Source : "
expression="$ctx:Source" />
</log>
<payloadFactory media-type="xml">
<format>
<productDetails xmlns="">
<productID>$1</productID>
<productName>$2</productName>
<productSize>$3</productSize>
<productColour>$4</productColour>
<productType>$5</productType>
<sourceID>$6</sourceID>
</productDetails>
</format>
<args>
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Product_ID" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Product_Name" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Product_Size" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Product_Colour" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Product_Type" />
<arg xmlns:ns="http://org.apache.synapse/xsd" evaluator="xml"
expression="$ctx:Source" />
</args>
</payloadFactory>
<send />
</sequence>
SOAP Rs
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<productDetails>
<productID>214562</productID>
<productName>NIKE_SHIRTS</productName>
<productSize>7</productSize>
<productColour>RED</productColour>
<productType>SHIRT</productType>
<sourceID>SHC</sourceID>
</productDetails>
</soapenv:Body>
</soapenv:Envelope>
WSO2 Carbon Log
https : // docs. google. com/document/d/12At0zir99H9cOo9hzNrgG33r52PHF0W32XLyBT6dzUQ/edit? usp=sharing
Edit
My AggHandlerSequence :
<sequence xmlns="http://ws.apache.org/ns/synapse">
<log level="custom">
<property name="pocSearchRsHandlerSeq reached" value="++++++++++++++=========Aggregating now========+++++++++++++++++=="></property>
</log>
<aggregate>
<completeCondition>
<messageCount min="-1" max="-1"></messageCount>
</completeCondition>
<onComplete xmlns:ns="http://org.apache.synapse/xsd" xmlns:prod="http://za.co.pepkor/product_service/" expression="//prod:productSearchResp/productDetails">
<log level="custom" separator=",">
<property name="::::" value="======================= Sending Back the Aggregated Responses. ==============="></property>
</log>
<respond></respond>
</onComplete>
</aggregate>
</sequence>
And one of my Rs Seq
<sequence xmlns="http://ws.apache.org/ns/synapse">
:
:
<payloadFactory media-type="xml">
<format>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:prod="http://za.co.pepkor/product_service/">
<soapenv:Body>
<prod:productSearchResp>
<productDetails xmlns="">
<productID>$1</productID>
<productName>$2</productName>
<productSize>$3</productSize>
<productColour>$4</productColour>
<productType>$5</productType>
<sourceID>$6</sourceID>
</productDetails>
</prod:productSearchResp>
</soapenv:Body>
</soapenv:Envelope>
:
:
<sequence key="conf:/pocSearchRsHandlerSeq"></sequence>
:
:
But am getting this Rs in soap
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<productDetails>
<productID>2145627</productID>
<productName>NIKE_SHIRTS</productName>
<productSize>7</productSize>
<productColour>RED</productColour>
<productType>SHIRT</productType>
<sourceID>ACK</sourceID>
</productDetails>
<productDetails>
<productID>214562</productID>
<productName>NIKE_SHIRTS</productName>
<productSize>7</productSize>
<productColour>RED</productColour>
<productType>SHIRT</productType>
<sourceID>SHC</sourceID>
</productDetails>
</soapenv:Body>
</soapenv:Envelope>
I however want a response similar to the Payload factory in the Rs seq , have tried playing with the corrolateOn expression. Can I store these aggregated fields as properties and use them in a payload factory or how do I map the Agg responses to ?
Final edit
Got it perfect here's a Rs seq payload:(this will be recieved by agg handler)
<productDetails xmlns="">
<productID>$1</productID>
<productName>$2</productName>
<productSize>$3</productSize>
<productColour>$4</productColour>
<productType>$5</productType>
<sourceID>$6</sourceID>
</productDetails>
And my full agg handler :
<sequence xmlns="http://ws.apache.org/ns/synapse">
<log level="custom">
<property name="pocSearchRsHandlerSeq reached" value="++++++++++++++=========Aggregating now========+++++++++++++++++=="></property>
</log>
<aggregate>
<completeCondition>
<messageCount min="-1" max="-1"></messageCount>
</completeCondition>
<onComplete xmlns:ns="http://org.apache.synapse/xsd" expression="//productDetails">
<log level="custom" separator=",">
<property name="::::" value="======================= Formatting the Aggregated Responses. ==============="></property>
</log>
<log level="full"></log>
<payloadFactory media-type="xml">
<format>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:prod="http://za.co.pepkor/product_service/">
<soapenv:Body>
<prod:productSearchResp> $1 </prod:productSearchResp>
</soapenv:Body>
</soapenv:Envelope>
</format>
<args>
<arg expression="//productDetails" evaluator="xml"></arg>
</args>
</payloadFactory>
<respond></respond>
</onComplete>
</aggregate>
</sequence>

Answer 1

If you want to use aggregate mediator, the 2 responses must be go to the same sequence in which you use this mediator
In your proxy inSequence, you use clone and call "ackServiceSearchSeqRq" and "shcServiceSearchSeqRq" as you already do
In each of theses request seq, you format your request according to the service you want to call and then use send mediator from one of these ways :
1st Option : <send> <endpoint key="xxx"> </send> : the 2 responses from your 2 services will go to your proxy's outSequence
2nd Option : <send receive="MyAggregateSequence"> <endpoint key="xxx"> </send> : the 2 responses from your 2 services will go to "MyAggregateSequence"
In your proxy's outSequence or in "MyAggregateSequence", depending on the option you have chosen, you use filter or switch mediator to determine if it is a response from ackService or from shcService and decide which transformation to apply (pay attention to the productDetails node you create with payloadFactory, if there is no namespace then specify xmlns="" each time : this is not the case in your samples) before calling aggregate mediator. Inside aggregate/onComplete, you can use send mediator with no endpoint to send back your aggregated message to the proxy's caller.
In your case, there is 2 <send/>, one in each of your 2 Rs sequence : the 1st one is sent to the proxy's caller and the 2nd one does nothing (the ESB must says than you try to respond 2 times).

Answer 2

When using the clone mediator and you want to collect the response of the two messages, you must set and id on the clone mediator and using the same id in the aggregator mediator.
< clone id="XXX">
< aggregate id="XXX">
This way the aggregator knows to which calls it needs to wait for.