How to handle multiple csv insertion in wso2esb 4.8.0 - wso2

I am using wso2esb4.8.0 .I wish insert the data into tables i am getting the data from client in .CSV format i have changed the VFS transport details in AXIS2 file and my configuration working fine for single CSV file
But my scenario is i have multiple tables like client,company,branch,divsion,party
these are dependent each other it means if i want insert into company i need clientid same as all tables
client give me sample data my clientid is auto genrated function in database it means remaining fields i need to insert and i will get clientid for that i used this query in dblookup mediator
insert into client(clientcode,clientname,clientshortname,createdbyid,modifiedbyid) values(?,?,?,?,?) RETURNING clientid;
and i need to insert company using this id how would i do this i tried for client insertion and unable get any idea for next insertion my configuration is
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="Smook"
transports="https,http,vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<smooks config-key="Smook">
<input type="text"/>
<output type="xml"/>
</smooks>
<dblookup>
<connection>
<pool>
<password>Youtility11</password>
<user>youtilitydba</user>
<url>jdbc:postgresql://localhost:5432/USCProduction</url>
<driver>org.postgresql.Driver</driver>
</pool>
</connection>
<statement>
<sql>insert into mclient(clientcode,clientname,clientshortname,createdbyid,modifiedbyid) values(?,?,?,?,?) RETURNING clientid;</sql>
<parameter expression="//clientcode/text()" type="CHAR"/>
<parameter expression="//clientname/text()" type="CHAR"/>
<parameter expression="//clientshortname/text()" type="CHAR"/>
<parameter expression="//createdbyid/text()" type="BIGINT"/>
<parameter expression="//modifiedbyid/text()" type="BIGINT"/>
<result name="clientid" column="clientid"/>
</statement>
</dblookup>
<log level="full">
<property name="message" expression="get-property('clientid')"/>
</log>
</inSequence>
<outSequence/>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/youtility2/Desktop/smooks/out</parameter>
<parameter name="transport.vfs.FileURI">file:///home/youtility2/Desktop/smooks/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///home/youtility2/Desktop/smooks/failure</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<description/>
</proxy>
my smooke mediator is like this
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd">
<resource-config selector="org.xml.sax.driver">
<resource>org.milyn.csv.CSVReader</resource>
<param name="fields">clientcode,clientname,clientshortname,createdbyid,modifiedbyid</param>
<param name="rootElementName">people</param>
<param name="recordElementName">person</param>
</resource-config>
</smooks-resource-list>
i have clientid but my company csv is same as like above if i insert into company i will get companyid how would i do this any have any idea.

Related

How to save file data present in transport vfs to a parameter in WSO2?

I want to pass the csv file data present in local drive as a string parameter to the DB query. How do I save the file data from vfs to a parameter property?
Also, this is a polling process and hence want to make use of transport vfs if it is possible. Not file connector.
<proxy name="CSVFileProcessorPostDemo" startOnLoad="true" transports="http https vfs" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<dblookup>
<connection>
<pool>
<driver>org.postgresql.Driver</driver>
<url>jdbc:postgresql://**************/**********</url>
<user>*********</user>
<password>*********</password>
</pool>
</connection>
<statement>
<sql><![CDATA[SELECT * FROM ba_ior_integration.fn_transform_ior_msg_to_idoc(?)]]></sql>
<parameter type="VARCHAR" value="{How to read the transport file data?}"/>
<result column="1" name="output_db_idoc"/>
</statement>
</dblookup>
<property expression="get-property('output_db_idoc')" name="dbOutput" scope="default" type="STRING" xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns3="http://org.apache.synapse/xsd"/>
.
.
.
.
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.FileURI">file://D:\ESB\files\input_IOR_files</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file://D:\ESB\files\failure_IOR_files</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.Append">true</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file://D:\ESB\files\output_IOR_files</parameter>
</proxy>
Have searched and found two ways of handling it.
1) It is simply present under $ctx.body tag.
<property expression="$body//m0:text" name="dbInput" scope="default"
type="STRING" xmlns:m0="http://ws.apache.org/commons/ns/payload"/>
And then in dbLookup:
<sql><![CDATA[SELECT * FROM abcde.fn_transform_xyz_msg_to_idoc(?)]]></sql>
<parameter expression="synapse:get-property('dbInput')" type="VARCHAR"/>
<result column="1" name="output_db_idoc"/>
2) Using Java class mediator. The message string is present in MessageContext object.
context.getMessageString() provides the SoapEnvelope required, using which the required inner text can be extracted.
or
context.getSOAPEnvelop(): This did not work for me, shall look into it.
The takeaway is that the data of the files considered under vfs are all present sequentially under MessageContext.

WSO2 ESB MailTransportListener not receiving

I just installed my brand new WSO2 ESB 4.9.0. This is the latest version which I downloaded yesterday.
I am trying to get my ESB to receive e-mail from a server. I have tried POP3 and IMAP. I have been using several references to help me(Since I am a new member, my reputation does not yet allow me to post proper links to all my references):
stackoverflow.com/questions/27376833/wso2-mailto-transport-emptied-my-inbox
javamail.java.net/nonav/docs/api/com/sun/mail/pop3/package-summary.html
javamail.java.net/nonav/docs/api/com/sun/mail/imap/package-summary.html
sparkletechthoughts.blogspot.fr/2013/10/how-to-receive-emails-to-wso2-esb.html
charithaka.blogspot.com/2012/11/how-to-enable-mail-transport-for-wso2.html
irhamiqbal.wordpress.com/how-to-receive-email-using-wso2-esb/
jayalalk.blogspot.com/2014/06/wso2-esb-mail-sending-with-html-complex.html
And of course the official documentation:
https://docs.wso2.com/display/ESB481/MailTo+Transport
https://docs.wso2.com/display/AS501/Using+Mail+transport+to+Invoke+Web+Services
All of my reading leads me to believe that I have everything set up properly (or if not properly, good enough to get some error messages to troubleshoot).
I did restart the WSO2 ESB and the following message shows up in the console log:
TID[-1234] [ESB] [2016-08-02 21:45:37,237] INFO {org.apache.axis2.transport.mail.MailTransportListener} - MAILTO listener started
I get no other indication in the logs one way or another.
I edited (and double checked) my $CARBON_HOME/repository/conf/axis2/axis2.xml, and I'm pretty sure I have the mail transport enabled properly:
<transportReceiver name="mailto" class="org.apache.axis2.transport.mail.MailTransportListener">
<!-- configure any optional POP3/IMAP properties
check com.sun.mail.pop3 and com.sun.mail.imap package documentation for more details-->
</transportReceiver>
<transportSender name="mailto" class="org.apache.axis2.transport.mail.MailTransportSender">
<parameter name="mail.smtp.host">my.smtp.server</parameter>
<parameter name="mail.smtp.port">587</parameter>
<parameter name="mail.smtp.starttls.enable">false</parameter>
<parameter name="mail.smtp.auth">true</parameter>
<parameter name="mail.smtp.user">my#user.id</parameter>
<parameter name="mail.smtp.password">password</parameter>
<parameter name="mail.smtp.from">my#email.com</parameter>
</transportSender>
And, finally, I have my Proxy Service setup (in its current iteration, anyway):
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">my.pop3.email</parameter>
<parameter name="mail.pop3.user">my#email.com</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">jasper#eryjus.com</parameter>
<parameter name="mail.pop3.password">password</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<description/>
</proxy>
With all of this, my question is: is there something else I am missing to get the ESB to at least attempt to retrieve e-mail? What bothers me is I am not getting any error messages to troubleshoot (such as my mail not being in plain test).
I tried the same scenario with a Gmail account and it works fine for me, seems like some mis-configuration in your POP server. You need to add following property as well.
<parameter name="transport.mail.ContentType">text/plain</parameter>
See the full proxy configs below.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">pop.gmail.com</parameter>
<parameter name="mail.pop3.user">ycwso2test</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">ycwso2test#gmail.com</parameter>
<parameter name="mail.pop3.password">wso2qa123</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<parameter name="transport.mail.ContentType">text/plain</parameter>
<description/>
</proxy>
Also if you need to further debug this you can enable DEBUG logs. To do this add the following line to repository/conf/log4j.properties and restart.
log4j.logger.org.apache.axis2.transport.mail.MailTransportListener=DEBUG

WSO2 ESB VFS: Pick up one File at a Time

I have created a simple vfs proxy which consumes a file and places it on the JMS queue. I want to be able to pick up one file at a time. I have my pollinterval set for every 10 minutes.
Are there any properties or settings I am missing to control this functionality?
Sample Proxy Below
<proxy xmlns="http://ws.apache.org/ns/synapse" name="VFSImportProxy" transports="https,http,vfs" statistics="disable" trace="disable" startOnLoad="true">
<target>
<inSequence>
<log separator="," />
<sequence key="smooks_sequence" />
<property name="OUT_ONLY" value="true" scope="default" type="STRING" />
<send>
<endpoint key="MessageStore_EP" />
</send>
</inSequence>
<outSequence />
<faultSequence />
</target>
<parameter name="transport.PollInterval">600000</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileURI">vfs:sftp://locationpickup/local2</parameter>
<parameter name="transport.vfs.MoveAfterProcess">smb://location/archive</parameter>
<parameter name="transport.vfs.MoveAfterFailure">smb://location/failed</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<enableSec />
</proxy>
Yes, the parameter name is transport.vfs.FileProcessCount and it's available in version 4.8.0 and up.
https://docs.wso2.com/display/ESB480/VFS+Transport

How to call other sequence from proxy service in WSO2

I want to call dbreport sequence from my proxy, but now I am facing a problem. I write the sequence and also the vfs proxy. But I can't figure out how to call that sequence from my proxy. I tried this "clone" and it works, but It only work when the proxy only called 1 sequence. My proxy :
<proxy name="SDL4"
transports="vfs"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<class name="insert.data.proxy.TestInsertData4"/>
<log level="custom">
<property name="start" value="start"/>
</log>
<clone>
<target sequence="cobaInsert"/>
</clone>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">xxx</parameter>
<parameter name="transport.PollInterval">1</parameter>
<parameter name="transport.vfs.FileURI">xxx</parameter>
<parameter name="transport.vfs.MoveAfterProcess">xxx</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
and my dbreport sequence
<sequence name="cobaUpdate">
<dbreport>
<connection>
<pool>
<password>postgres</password>
<user>postgres</user>
<url>jdbc:postgresql://xxx.xxx.xxx.xxx/xxxdb</url>
<driver>org.postgresql.Driver</driver>
</pool>
</connection>
<statement>
<sql>update status set status = ? where id = ?</sql>
<parameter xmlns:ns3="http://org.apache.synapse/xsd"
expression="get-property('status')"
type="VARCHAR"/>
<parameter xmlns:ns3="http://org.apache.synapse/xsd"
expression="get-property('id')"
type="INTEGER"/>
</statement>
</dbreport>
Is it the right way to call dbreport sequence from vfs proxy?
Use sequence mediator instead of clone mediator;
Eg:
<sequence key="cobaUpdate"/>

XPath from file in VFS Transport WSO2 ESB

I want to read some values from a file that being processed by the VFS transport.
My file is :
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
So I use a property mediator to get the value I want using XPath. But when I print that property in Log Mediator it prints nothing. My proxy look like this :
<proxy name="Try1"
transports="vfs"
startOnLoad="true"
trace="disable">
<target>
<inSequence>
<property xmlns:ns="http://org.apache.synapse/xsd"
name="source"
expression="//header/source/text()"
scope="default"
type="STRING"/>
<log level="full">
<property xmlns:ns="http://org.apache.synapse/xsd"
name="Value_source"
expression="get-property('source')"/>
</log>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///Users/Target</parameter>
<parameter name="transport.vfs.FileURI">file:///Users/Source</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///Users/Failure</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
The log mediator result is
[2013-06-28 16:59:49,939] INFO - LogMediator Value_source=
So how can I get the values in file using XPath?
The issue is due to the ContentType set for the transport parameter:
Switch
<parameter name="transport.vfs.ContentType">text/plain</parameter>
with
<parameter name="transport.vfs.ContentType">application/xml</parameter>
since you're doing XPath operations on the message.
The problem is with the content of your file. Since it is not well-formed, the xpath expression fails. Isn't it possible for you to have the well-formed xml in the input content?
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
</sales>