DataSink step to return each response with all the children - web-services

Question

I learnt very recently how to use data-driven testing in Ready API and loop calls based on the data. My goal is to run the steps in loop and at the end create an auto-export facility with DataSink so that the results get auto exported.
Now when I try go to DataSink, as I understood I need to create column headers as below
to store the corresponding child values
It would work well, if the soap response for each of the siteId has the same XML structure. But in my case each of the 2000+ response that I get has different number of children within
<retrun> </return>
For e.g. please take a look at the response 1 and response 2. Both these responses have different number of children.
Response 1
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<ns2:getSiteInfoResponse xmlns:ns2="http://billing.xyz.cc/">
<return>
<address1>A</address1>
<city>B</city>
<closeDate>2018-10-15T00:00:00-05:00</closeDate>
<contact1/>
<contact2>TBD</contact2>
<country>X1</country>
<customerNbr>288</customerNbr>
<emailAddr1/>
<emailAddr2/>
<fax1>0</fax1>
<fax2>0</fax2>
<gps>C</gps>
<grouping2>Leased</grouping2>
<grouping4>D</grouping4>
<jobTitle1/>
<jobTitle2/>
<phone1>0</phone1>
<phone2>0</phone2>
<siteId>862578</siteId>
<siteName>D</siteName>
<squareFoot>0.0</squareFoot>
<state>E</state>
<weatherStation>D</weatherStation>
<zip4>4</zip4>
<zip5>F</zip5>
</return>
</ns2:getSiteInfoResponse>
</soap:Body>
</soap:Envelope>
Response 2
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<ns2:getSiteInfoResponse xmlns:ns2="http://billing.xyz.cc/">
<return>
<address1>1202</address1>
<city>QA</city>
<contact1/>
<contact2>BL</contact2>
<country>A</country>
<customerNbr>288</customerNbr>
<emailAddr1/>
<emailAddr2/>
<fax1>0</fax1>
<fax2>0</fax2>
<gps>LTE</gps>
<grouping1>1345</grouping1>
<grouping2>Leased</grouping2>
<grouping3>ZX</grouping3>
<grouping4>AA</grouping4>
<grouping5>2000</grouping5>
<jobTitle1/>
<jobTitle2/>
<phone1>0</phone1>
<phone2>0</phone2>
<services>
<accountNbr>11099942</accountNbr>
<liveDt>2013-07-01T00:00:00-05:00</liveDt>
<service>2</service>
<serviceType>gas</serviceType>
<vendorAddr1/>
<vendorAddr2>M</vendorAddr2>
<vendorCity>N</vendorCity>
<vendorName>O</vendorName>
<vendorNbr>P</vendorNbr>
<vendorPhone>Q</vendorPhone>
<vendorState>R</vendorState>
<vendorZip>S</vendorZip>
</services>
<services>
<accountNbr>13064944</accountNbr>
<liveDt>2018-05-20T00:00:00-05:00</liveDt>
<service>2</service>
<serviceType>gas</serviceType>
<vendorAddr1/>
<vendorAddr2>A</vendorAddr2>
<vendorCity>B</vendorCity>
<vendorName>C</vendorName>
<vendorNbr>677</vendorNbr>
<vendorPhone>D</vendorPhone>
<vendorState>E</vendorState>
<vendorZip>F</vendorZip>
</services>
<siteId>101567</siteId>
<siteName>X</siteName>
<squareFoot>4226.0</squareFoot>
<state>Y</state>
<weatherStation>Z</weatherStation>
<zip4>0</zip4>
<zip5>L</zip5>
</return>
</ns2:getSiteInfoResponse>
</soap:Body>
</soap:Envelope>
Now, I need to further create a table using the whole response to be utilized in business intelligence. If I have to create matching headers in DataSink I need to go through each and every responses to ensure that I have created a corresponding property in datasink. It is not humanly possible without compromising the accuracy.
Is there any way to program Ready API to store individual XML response by each looping call in a file specified by me (2000+ XML responses) or to store all the values by children of the response node without needing me to specify all the header names in the DataSink window. If it happens, i will be fine in both cases to utilize a BI tool to create a corresponding table from there.
Thank you in advance.

Answer 1

As you point out, the differing number of children makes the linear data sink problematic.
That said, you can still use datasink to dump out all values in one go. In the datasink, create a single header and use 'get data' to select the root node of your response.
This will obviously generate a massive file, so you have two choices here. Either dump everything into a single file, or you could create a new file per response
If you're wondering about naming of lots of little files, you can generate a filename on the fly for the data sink to use. To do this, create a groovy script inside the loop. In this script, make it return a path and the file name. You could use some timestamp value, e.g. c:\temp\myResults\2020120218150102.txt, which is year, month, data, hour, min seconds and ms. Then, in the Data sink step where you browse for the file name, use get data to 'grab' the result of the groovy script.

Answer 2

#Chris Adams thanks for your awesome idea. Even though I could not completely put this into practice. But because of your idea (Get Data) I took a different route and I got what I wanted.
So this is what I did. Instead of using DataSink I used create file. The idea is whenever I schedule this task the Ready API can run the whole thing in loop and throw the result in a static folder
with file name
containing
site Id obtained from Get Data Raw request agr3
${getSiteInfo#RawRequest#declare namespace bil='http://billing.xyz.cc/'; //bil:getSiteInfo[1]/arg3[1]}.xml
and
file content
with whole response from root node Response
${getSiteInfo#Response#declare namespace soap='http://schemas.xmlsoap.org/soap/envelope/'; //soap:Envelope[1]}
The end result is this
However, I am still interested in this and I could get this part to work.
That said, you can still use datasink to dump out all values in one go. In the datasink, create a single header and use 'get data' to select the root node of your response.