XPath from file in VFS Transport WSO2 ESB - wso2

I want to read some values from a file that being processed by the VFS transport.
My file is :
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
So I use a property mediator to get the value I want using XPath. But when I print that property in Log Mediator it prints nothing. My proxy look like this :
<proxy name="Try1"
transports="vfs"
startOnLoad="true"
trace="disable">
<target>
<inSequence>
<property xmlns:ns="http://org.apache.synapse/xsd"
name="source"
expression="//header/source/text()"
scope="default"
type="STRING"/>
<log level="full">
<property xmlns:ns="http://org.apache.synapse/xsd"
name="Value_source"
expression="get-property('source')"/>
</log>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///Users/Target</parameter>
<parameter name="transport.vfs.FileURI">file:///Users/Source</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///Users/Failure</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
The log mediator result is
[2013-06-28 16:59:49,939] INFO - LogMediator Value_source=
So how can I get the values in file using XPath?

The issue is due to the ContentType set for the transport parameter:
Switch
<parameter name="transport.vfs.ContentType">text/plain</parameter>
with
<parameter name="transport.vfs.ContentType">application/xml</parameter>
since you're doing XPath operations on the message.

The problem is with the content of your file. Since it is not well-formed, the xpath expression fails. Isn't it possible for you to have the well-formed xml in the input content?
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
</sales>

Related

How to save file data present in transport vfs to a parameter in WSO2?

I want to pass the csv file data present in local drive as a string parameter to the DB query. How do I save the file data from vfs to a parameter property?
Also, this is a polling process and hence want to make use of transport vfs if it is possible. Not file connector.
<proxy name="CSVFileProcessorPostDemo" startOnLoad="true" transports="http https vfs" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<dblookup>
<connection>
<pool>
<driver>org.postgresql.Driver</driver>
<url>jdbc:postgresql://**************/**********</url>
<user>*********</user>
<password>*********</password>
</pool>
</connection>
<statement>
<sql><![CDATA[SELECT * FROM ba_ior_integration.fn_transform_ior_msg_to_idoc(?)]]></sql>
<parameter type="VARCHAR" value="{How to read the transport file data?}"/>
<result column="1" name="output_db_idoc"/>
</statement>
</dblookup>
<property expression="get-property('output_db_idoc')" name="dbOutput" scope="default" type="STRING" xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns3="http://org.apache.synapse/xsd"/>
.
.
.
.
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.FileURI">file://D:\ESB\files\input_IOR_files</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file://D:\ESB\files\failure_IOR_files</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.Append">true</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file://D:\ESB\files\output_IOR_files</parameter>
</proxy>
Have searched and found two ways of handling it.
1) It is simply present under $ctx.body tag.
<property expression="$body//m0:text" name="dbInput" scope="default"
type="STRING" xmlns:m0="http://ws.apache.org/commons/ns/payload"/>
And then in dbLookup:
<sql><![CDATA[SELECT * FROM abcde.fn_transform_xyz_msg_to_idoc(?)]]></sql>
<parameter expression="synapse:get-property('dbInput')" type="VARCHAR"/>
<result column="1" name="output_db_idoc"/>
2) Using Java class mediator. The message string is present in MessageContext object.
context.getMessageString() provides the SoapEnvelope required, using which the required inner text can be extracted.
or
context.getSOAPEnvelop(): This did not work for me, shall look into it.
The takeaway is that the data of the files considered under vfs are all present sequentially under MessageContext.

Email receiver proxy in wso2 esb 5.0.0 is not working

I have created email receiver proxy in WSO2-ESB5.0, proxy is connecting to email box and and I am seeing text in the logs "Processing message # : 1 in same thread".
But nothing is getting printed in the body of the proxy where I am expecting proxy should read the email body and pass to insequence of the proxy.
And also emails are not MOVED OR DELETED when I set the properties ActionAfterProcess to MOVE/DELETE.
I see some issues here
you have in one hand
<parameter name="transport.mail.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.mail.ActionAfterProcess">DELETE</parameter>
and on another hand you try to move to a folder after process or failure
<parameter name="transport.mail.MoveAfterFailure">Inbox/Failed</parameter>
<parameter name="transport.mail.MoveAfterProcess">Inbox/Processed</parameter>
So either you should remove all those four lines (default behaviour is "REMOVE") or change to
<parameter name="transport.mail.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.mail.ActionAfterProcess">MOVE</parameter>
In order to move processed files
Fetch size sounds really small to me, could you try a higher value like
<parameter name="mail.imap.fetchsize">1024</parameter>
Then I assume Inbox/TestInbox is existing and you successfully connect to the mailbox. Are you sure that the message content type is application/xml (could you simply use plain/text for instance).
Could you also try
<property name="senderAddress" expression="get-property('transport', 'From')"/>
<log level="custom">
<property name="Sender Address" expression="get-property('senderAddress')"/>
</log>
To see if you can get the sender address.
Here is sample of my proxy:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="MailTransportProtocolIMAP"
transports="mailto"
startOnLoad="true"
statistics="enable"
trace="enable">
<description/>
<target>
<inSequence>
<log level="full">
</log>
<log category="ERROR">
<property name="emailBody" expression="$body/*"/>
</log>
<drop/>
</inSequence>
<outSequence>
<log category="ERROR">
<property name="emailBody" expression="$body/*"/>
</log>
<send/>
</outSequence>
</target>
<parameter name="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="mail.imap.starttls.enable">false</parameter>
<parameter name="transport.mail.ContentType">application/xml</parameter>
<parameter name="mail.imap.host">outlook.74.net</parameter>
<parameter name="transport.mail.Address">abc#abc.com</parameter>
<parameter name="mail.imap.user">abc123</parameter>
<parameter name="mail.imap.socketFactory.port">993</parameter>
<parameter name="transport.mail.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.mail.ActionAfterProcess">DELETE</parameter>
<parameter name="transport.PollInterval">2</parameter>
<parameter name="mail.imap.port">995</parameter>
<parameter name="mail.imap.ssl.enable">false</parameter>
<parameter name="transport.mail.MoveAfterFailure">Inbox/Failed</parameter>
<parameter name="transport.mail.Folder">Inbox/TestInbox</parameter>
<parameter name="transport.mail.MoveAfterProcess">Inbox/Processed</parameter>
<parameter name="mail.imap.fetchsize">2</parameter>
<parameter name="mail.imap.socketFactory.fallback">false</parameter>
<parameter name="transport.mail.Protocol">imap</parameter>
<parameter name="mail.imap.password">abc123!</parameter>
</proxy>

WSO2 ESB MailTransportListener not receiving

I just installed my brand new WSO2 ESB 4.9.0. This is the latest version which I downloaded yesterday.
I am trying to get my ESB to receive e-mail from a server. I have tried POP3 and IMAP. I have been using several references to help me(Since I am a new member, my reputation does not yet allow me to post proper links to all my references):
stackoverflow.com/questions/27376833/wso2-mailto-transport-emptied-my-inbox
javamail.java.net/nonav/docs/api/com/sun/mail/pop3/package-summary.html
javamail.java.net/nonav/docs/api/com/sun/mail/imap/package-summary.html
sparkletechthoughts.blogspot.fr/2013/10/how-to-receive-emails-to-wso2-esb.html
charithaka.blogspot.com/2012/11/how-to-enable-mail-transport-for-wso2.html
irhamiqbal.wordpress.com/how-to-receive-email-using-wso2-esb/
jayalalk.blogspot.com/2014/06/wso2-esb-mail-sending-with-html-complex.html
And of course the official documentation:
https://docs.wso2.com/display/ESB481/MailTo+Transport
https://docs.wso2.com/display/AS501/Using+Mail+transport+to+Invoke+Web+Services
All of my reading leads me to believe that I have everything set up properly (or if not properly, good enough to get some error messages to troubleshoot).
I did restart the WSO2 ESB and the following message shows up in the console log:
TID[-1234] [ESB] [2016-08-02 21:45:37,237] INFO {org.apache.axis2.transport.mail.MailTransportListener} - MAILTO listener started
I get no other indication in the logs one way or another.
I edited (and double checked) my $CARBON_HOME/repository/conf/axis2/axis2.xml, and I'm pretty sure I have the mail transport enabled properly:
<transportReceiver name="mailto" class="org.apache.axis2.transport.mail.MailTransportListener">
<!-- configure any optional POP3/IMAP properties
check com.sun.mail.pop3 and com.sun.mail.imap package documentation for more details-->
</transportReceiver>
<transportSender name="mailto" class="org.apache.axis2.transport.mail.MailTransportSender">
<parameter name="mail.smtp.host">my.smtp.server</parameter>
<parameter name="mail.smtp.port">587</parameter>
<parameter name="mail.smtp.starttls.enable">false</parameter>
<parameter name="mail.smtp.auth">true</parameter>
<parameter name="mail.smtp.user">my#user.id</parameter>
<parameter name="mail.smtp.password">password</parameter>
<parameter name="mail.smtp.from">my#email.com</parameter>
</transportSender>
And, finally, I have my Proxy Service setup (in its current iteration, anyway):
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">my.pop3.email</parameter>
<parameter name="mail.pop3.user">my#email.com</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">jasper#eryjus.com</parameter>
<parameter name="mail.pop3.password">password</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<description/>
</proxy>
With all of this, my question is: is there something else I am missing to get the ESB to at least attempt to retrieve e-mail? What bothers me is I am not getting any error messages to troubleshoot (such as my mail not being in plain test).
I tried the same scenario with a Gmail account and it works fine for me, seems like some mis-configuration in your POP server. You need to add following property as well.
<parameter name="transport.mail.ContentType">text/plain</parameter>
See the full proxy configs below.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">pop.gmail.com</parameter>
<parameter name="mail.pop3.user">ycwso2test</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">ycwso2test#gmail.com</parameter>
<parameter name="mail.pop3.password">wso2qa123</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<parameter name="transport.mail.ContentType">text/plain</parameter>
<description/>
</proxy>
Also if you need to further debug this you can enable DEBUG logs. To do this add the following line to repository/conf/log4j.properties and restart.
log4j.logger.org.apache.axis2.transport.mail.MailTransportListener=DEBUG

How to handle multiple csv insertion in wso2esb 4.8.0

I am using wso2esb4.8.0 .I wish insert the data into tables i am getting the data from client in .CSV format i have changed the VFS transport details in AXIS2 file and my configuration working fine for single CSV file
But my scenario is i have multiple tables like client,company,branch,divsion,party
these are dependent each other it means if i want insert into company i need clientid same as all tables
client give me sample data my clientid is auto genrated function in database it means remaining fields i need to insert and i will get clientid for that i used this query in dblookup mediator
insert into client(clientcode,clientname,clientshortname,createdbyid,modifiedbyid) values(?,?,?,?,?) RETURNING clientid;
and i need to insert company using this id how would i do this i tried for client insertion and unable get any idea for next insertion my configuration is
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="Smook"
transports="https,http,vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<smooks config-key="Smook">
<input type="text"/>
<output type="xml"/>
</smooks>
<dblookup>
<connection>
<pool>
<password>Youtility11</password>
<user>youtilitydba</user>
<url>jdbc:postgresql://localhost:5432/USCProduction</url>
<driver>org.postgresql.Driver</driver>
</pool>
</connection>
<statement>
<sql>insert into mclient(clientcode,clientname,clientshortname,createdbyid,modifiedbyid) values(?,?,?,?,?) RETURNING clientid;</sql>
<parameter expression="//clientcode/text()" type="CHAR"/>
<parameter expression="//clientname/text()" type="CHAR"/>
<parameter expression="//clientshortname/text()" type="CHAR"/>
<parameter expression="//createdbyid/text()" type="BIGINT"/>
<parameter expression="//modifiedbyid/text()" type="BIGINT"/>
<result name="clientid" column="clientid"/>
</statement>
</dblookup>
<log level="full">
<property name="message" expression="get-property('clientid')"/>
</log>
</inSequence>
<outSequence/>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/youtility2/Desktop/smooks/out</parameter>
<parameter name="transport.vfs.FileURI">file:///home/youtility2/Desktop/smooks/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///home/youtility2/Desktop/smooks/failure</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<description/>
</proxy>
my smooke mediator is like this
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd">
<resource-config selector="org.xml.sax.driver">
<resource>org.milyn.csv.CSVReader</resource>
<param name="fields">clientcode,clientname,clientshortname,createdbyid,modifiedbyid</param>
<param name="rootElementName">people</param>
<param name="recordElementName">person</param>
</resource-config>
</smooks-resource-list>
i have clientid but my company csv is same as like above if i insert into company i will get companyid how would i do this any have any idea.

How to keep file after being processed in source folder vfs transport WSO2

I use VFS transport in WSO2 to route file from one folder to another. But now I'm facing a problem, I do not want to move the processed file to another folder or delete it. The problem is VFS transport has a default action DELETE or move processed file to another folder. Is it possible to keep the processed file in the source folder? Or do I have to change the code or modify something else? Is there any other way to do this?
FYI, my proxy looks like this,
<proxy name="XXX" transports="vfs" startOnLoad="true" trace="disable">
<description/>
<target>
<inSequence>
<class name="com.test.read"/>
<log level="custom">
<property name="status" value="file read"/>
</log>
</inSequence>
</target>
<parameter name="transport.PollInterval">1</parameter>
<parameter name="transport.vfs.FileURI">file:///source</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///target</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
After the file is processed, the file will be moved to folder target. So how can I keep the file without moving or delete it?
Thanks :)
Just change the transport.vfs.MoveAfterProcess to original location, which will keep the file there even after being processed.
<proxy name="XXX" transports="vfs" startOnLoad="true" trace="disable">
<description/>
<target>
<inSequence>
<class name="com.test.read"/>
<log level="custom">
<property name="status" value="file read"/>
</log>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">1</parameter>
<parameter name="transport.vfs.FileURI">file:///source</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///source</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
</proxy>