Hi i am working on wso2 ESB 4.7.0,
I wish to process the any particular file like .txt,.xls,.xml , my client provide the data in above format files in system folder,i need to pick from there and process that file , i wish to store that data into data base.
Sample .txt file is
ename intime outtime eid
-------------------------
john 9.10 6.10 y001
scott 10.00 7.00 yoo2
tiger 9.00 6.00 y003
above data i need to insert in empdetails table.
I tried with VFS transport in WSO2 ESB, it is able to write the data into text file but how to read from data into a text file.
Help me to solve this.
I understand that you want to read data from a file
To do that, you just need to declare a VFS proxy :
<proxy xmlns="http://ws.apache.org/ns/synapse" name="IncomingFile" transports="vfs" statistics="disable" trace="disable" startOnLoad="true">
<target inSequence="YourSequence" />
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileURI">file:///Your_directory</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///Your_directory_OK</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///Your_directory_KOKO</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain; charset=ISO-8859-1</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
Don't forget to enable VFS transport receiver in you repository/conf/axis2/axis2.xml :
<transportReceiver name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportListener"/>
The message builder associated with text/plain in your axis2 conf will be used to build the message (org.apache.axis2.format.PlainTextBuilder by default : text content will be encapsulated into an xml node)
You may want to develop and use your own message builder, in order to transform the particular file format into a specific xml tree in order to use XPath inside your mediation.
An alternative would be to use smooks.
Related
I am very new in WSO2 ESB and I have the following situation to deal with.
I have a Message Processor which extracts some XML messages from a queue and that send it to an external web service, something like this:
<?xml version="1.0" encoding="UTF-8"?>
<messageProcessor class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor" messageStore="glisMessageStore" name="glisMessageProcessor" targetEndpoint="glisEndpoint" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="client.retry.interval">100</parameter>
<parameter name="max.delivery.attempts">4</parameter>
<parameter name="member.count">1</parameter>
<parameter name="message.processor.reply.sequence">glisResponseSequence</parameter>
<parameter name="max.delivery.drop">Disabled</parameter>
<parameter name="interval">10</parameter>
<parameter name="is.active">true</parameter>
</messageProcessor>
Where the glisResponseSequence.xml contains a pretty complex behavior ending with a callout mediator that send the message to my external web service.
It works fine but now I have also to deal with the rate limitation concept.
When the message processor call the glisResponseSequence.xml sequence it ends with a callout mediator that send the message to my external web service that return a response containing headers like:
X-Rate-Limit-Limit
X-Rate-Limit-Remaining
X-Rate-Limit-Reset
Can I use these headers to instruct the message processor to wait to call the glisResponseSequence.xml sequence or something like this (to avoid to send too much request to the external web service)
I am pretty new in WSO2 ESB and I have the following problem.
Into my ESB flow I have a Store mediator that put an XML document ino a WSO2 MB queue, this is the configuration of my message store:
<?xml version="1.0" encoding="UTF-8"?>
<messageStore class="org.apache.synapse.message.store.impl.jms.JmsStore" name="glisMessageStore" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="store.jms.destination">glisQueue</parameter>
<parameter name="store.producer.guaranteed.delivery.enable">false</parameter>
<parameter name="store.jms.cache.connection">true</parameter>
<parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>
<parameter name="connectionfactory.QueueConnectionFactory">amqp://admin:admin#clientID/carbon?brokerlist='tcp://localhost:5675'</parameter>
<parameter name="store.jms.JMSSpecVersion">1.1</parameter>
</messageStore>
As you can see my documents are putted into the glisQueue queue.
Then I deployed (with this project) a Message Processor named glisMessageProcessor that check if the queue is not empty and if it contains some XML documents it send to and external web service performing an Http call, this is the configuration of my message processor:
<?xml version="1.0" encoding="UTF-8"?>
<messageProcessor class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor" messageStore="glisMessageStore" name="glisMessageProcessor" targetEndpoint="glisEndpoint" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="client.retry.interval">1000</parameter>
<parameter name="max.delivery.attempts">4</parameter>
<parameter name="member.count">1</parameter>
<parameter name="message.processor.reply.sequence">glisResponseSequence</parameter>
<parameter name="max.delivery.drop">Disabled</parameter>
<parameter name="interval">1000</parameter>
<parameter name="is.active">true</parameter>
</messageProcessor>
As you can see it is using the glisMessageStore as data source (so it is using the glisQueue queue defiend in it to obtain the data to send) and send these data to the glisEndpoint that is the endpoint of my web service (it is defined in my project).
It works fine, the data are correctly sended to the external web service represented by the glisEndpoint.
My only problem is: my web service returns a message when is called (containing some information that I need to store in some way).
Can I configure in some way my glissMessageProcessor to retrieve these response message and put it in a log file?
You just have to use log mediator inside the sequence named "glisResponseSequence" : this is what you've defined with "message.processor.reply.sequence" parameter ant this is where goes the response.
I am currently trying to set up a vfs proxy which works fine on a local ftp server. The problem is that our company internet access is blocked by a proxy so I am searching for how to specify this proxy information. As far as I know JSCH is used underneath but I haven't found a way to pass the proxy parameter.
Here is my current proxy xml:
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="ftpCollection" statistics="disable" trace="disable" transports="vfs">
<target>
<inSequence>
<clone>
<target sequence="processSeq"/>
</clone>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="transport.vfs.ReplyFileURI">vfs:sftp://user:pw#192.168.2.99:22/reply</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileProcessInterval">100</parameter>
<parameter name="transport.vfs.FileURI">vfs:sftp://user:pw#192.168.2.99:22/process/</parameter>
<parameter name="transport.vfs.MoveAfterProcess">vfs:sftp://user:pw#192.168.2.99:22/success/</parameter>
<parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://user:pw#192.168.2.99:22/failure/</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.xml</parameter>
<parameter name="transport.vfs.Locking">disable</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.MoveTimestampFormat">yyyy-MM-dd'T'HH:mm:ss.SSS</parameter>
</proxy>
Thanks
SFTP (SSH) through a proxy does work over a HTTP tunnel. You can do that with JSch described on this page.
Now already the VFS transport in WSO2 (which finally uses JSch) does not have an implementation of the HTTP tunneling. So you cannot use a simple VFS WSO2 ProxyService to do that.
You can solve it by implementing your own wso2esb transport listener. Have a look at the code of VFSTransportListener.java (or just copy/paste) and code the JSch access over http yourself. You can then add the transport in the axis2.xml configuration of your wso2esb.
We have a legacy application that is only able to upload files through ftp protocol. Our current application has SOAP interface for similar file submissions. Is there any way to configure the WSO2 ESB to transform the uploaded file through ftp to SOAP request for consuming by our current application?
Yes that is a very common approach. You can create a VFS proxy that is reading from an FTP. You may then make a transformation (depends on the input format) (i.e. XSLT) to fit your SOAP Endpoint and then simply make a <send> to your WS.
Here an example (only pseudo-code - not tested), but will give you a good starting point:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="ftp_proxy_example" transports="vfs" startOnLoad="true" trace="disable">
<parameter name="transport.PollInterval">10</parameter>
<parameter name="transport.vfs.FileURI">vfs:ftp://user:password#server/path</parameter>
<parameter name="transport.vfs.FileNamePattern">.*[.]xml</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<target faultSequence="errorSequence">
<inSequence>
<!-- maybe a transformation -->
<!-- send to your WS endpoint -->
</inSequence>
</target>
</proxy>
i am testing some sample use cases with WSO2 where the ESB must scan an SFTP folder and perform some actions. Here is what i did:
1) I enabled the VFS transport listener in axis2.xml:
<transportReceiver name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportListener"/>
<transportSender name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportSender"/>
2) I inserted the following proxy (i have changed user:pass for obvious reasons) :
<proxy xmlns="http://ws.apache.org/ns/synapse" name="SFTPVFSProxy" transports="vfs" statistics="disable" trace="enable" startOnLoad="true">
<target>
<inSequence>
<log level="full" />
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">3</parameter>
<parameter name="transport.vfs.MoveAfterProcess">vfs:sftp://user:pass#10.254.241.69:22/home/user/sftp/out</parameter>
<parameter name="transport.vfs.FileURI">vfs:sftp://user:pass#10.254.241.69:22/home/user/sftp/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://user:pass#10.254.241.69:22/home/user/sftp/out</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
In both versions 4.5.0 & 4.5.1 nothing is triggered. No output in the log files other than some info that the proxy was successfully deployed. The exact same script works in version 3.0.1.
Furthermore, if i change "sftp" to "ftp" or to local file the listener is triggered normally in v4.X as well which makes me believe it's probably something specific to sftp handling.
Am i doing something wrong or is there a bug in the software?
Thank you in advance,
Thodoris
I think the problem is with the way you have defined the transport.vfs.FileURI property. Assuming your user's username is "user", then when you log in to sftp you go to the /home/user folder. Therefore, try defining the above property as follows.
<parameter name="transport.vfs.FileURI">vfs:sftp://user:pass#10.254.241.69:22/sftp/in</parameter>
or
<parameter name="transport.vfs.FileURI">vfs:sftp://user:pass#10.254.241.69:22/in</parameter>
I cannot say the exact config because I dont know your folder structure.
I have tried this in WSO2 ESB 4.0.3 with SFTP. My username was amilaftp. So I defined the property as follows.
<parameter name="transport.vfs.FileURI">vfs:sftp://amilaftp:password#localhost/in?vfs.passive=true</parameter>
When you try with FTP, you need to give the full path as you have done in the configuration you have provided. Thats why it is working for you with FTP in WSO2 ESB 4.X. If this works for you in 4.0.3, this should work in 4.5.x too. I suggest you to try with 4.0.3 first because it has worked for me.