Messages are vanishing after restarting WSO2 Server - wso2

I am using WSO2 ESB 5.0.0 and Apache activeMQ 5.14 for my project.I am using jms proxy which is listening to a queue and the proxy is using jms message store to store the messages.And there is a message processor which will pick up those messages from that message store for further processing.
But after restarting WSO2 server the messages which were stuck at the queue are getting dequeued and vanished.
Is there any way to store those messages by WSO2?
Here is my proxy service:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="abcMQ"
startOnLoad="true"
statistics="disable"
trace="disable"
transports="jms">
<target>
<inSequence>
<log level="full" separator="**Consumed from abc IN Seq**"/>
<property name="DISABLE_CHUNKING"
scope="axis2"
type="STRING"
value="true"/>
<store messageStore="TEST"/>
</inSequence>
<outSequence>
<send/>
</outSequence>
<faultSequence/>
</target>
<parameter name="transport.jms.Destination">req.Q</parameter>
<parameter name="transport.jms.ContentType">application/XML</parameter>
<description/>
</proxy>
The message store TEST is below:
<messageStore name="TEST" class="org.apache.synapse.message.store.impl.jms.JmsStore" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="java.naming.factory.initial">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>
<parameter name="java.naming.provider.url">tcp://localhost:61616</parameter>
<parameter name="store.jms.destination">TEST.Q</parameter>
<parameter name="store.jms.username">admin</parameter>
<parameter name="store.jms.password">admin</parameter>
<parameter name="store.jms.JMSSpecVersion">1.1</parameter>
<parameter name="store.producer.guaranteed.delivery.enable">true</parameter>
<parameter name="store.failover.message.store.name">fail</parameter>
</messageStore>
The fail message store is:
<messageStore name="fail" class="org.apache.synapse.message.store.impl.jms.JmsStore" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="java.naming.factory.initial">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>
<parameter name="java.naming.provider.url">tcp://localhost:61616</parameter>
<parameter name="store.jms.destination">fail.Q</parameter>
<parameter name="store.jms.username">admin</parameter>
<parameter name="store.jms.password">admin</parameter>
<parameter name="store.jms.JMSSpecVersion">1.1</parameter>
<parameter name="store.producer.guaranteed.delivery.enable">false</parameter>
<parameter name="store.failover.message.store.name">fail</parameter>
</messageStore>
Message processor is:
<messageProcessor name="TestProcessor" class="org.apache.synapse.message.processor.impl.sampler.SamplingProcessor" messageStore="TEST" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="interval">1000</parameter>
<parameter name="concurrency">1</parameter>
<parameter name="sequence">DemoSequence</parameter>
<parameter name="is.active">true</parameter>
</messageProcessor>

Since the in-memory message stores uses an in-memory queue to store the messages, the messages will be lost in case of a restart or shutdown.
That's why its use is not recommended for Production systems or large scale message processing systems. You can view the recommendation here.
Consider using other forms of message stores for better reliability and guaranteed delivery. Refer the following options:
https://docs.wso2.com/display/ESB500/Adding+a+Message+Store
Also consider using a fail-over message store for better reliability.
https://docs.wso2.com/display/ESB500/Guaranteed+Delivery+with+Failover+Message+Store+and+Scheduled+Failover+Message+Forwarding+Processor

Related

WSO2 VFS FTP Proxy create dynamically

At the moment, I am creating VFS Proxy services in WSO2 manually.
But I need to create services like this based on user input. There is no problem in generating a required XML. But I yet to find a way to dynamically import those XML into WSO2.
Does anyone know a way to make this work without manual service creation?
An example service:
<?xml version="1.0" encoding="UTF-8"?><proxy xmlns="http://ws.apache.org/ns/synapse" name="FileProxy" startOnLoad="true" statistics="disable" trace="disable" transports="vfs">
<target>
<inSequence>
<log level="full"/>
<clone>
<target sequence="fileWriteSequence"/>
</clone>
</inSequence>
</target>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.FileURI">file:///in</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///out/out</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///out/out</parameter>
<description/>
</proxy>
There are several ways to achive this.
1.) Use the admin services to create a proxy, that´s what we are doing for a couple of years now without problems
https://docs.wso2.com/display/EI630/Working+with+Admin+Services
2.) Just generate a XML file (like above) in the directory ESB_HOME/repository/deployment/server/synapse-configs/default/proxy-services
Regards
Martin

WSO2 - EI 6.2.0 - Configuring Oracle AQ JMS database with EI to publish/consume messages to/from Oracle AQ JMS

This is Radhakrishnan and new to WSO2 world. Would like to configure the Oracle AQ JMS with EI so that EI can consume/publish messages from Oracle AQ JMS.
Configuration for axis2.xml has been done and written a Proxy to consume message from Oracle AQ JMS.
Now when we are publishing to Oracle AQ JMS we are getting below error and anybody has suggestion/valuable inputs?
Caused by: javax.naming.NameNotFoundException: Name [dynamicQueues//RIB_AQ_USER.ETEXTPAYTERM] is not bound in this Context. Unable to find [dynamicQueues].
at org.apache.naming.NamingContext.lookup(NamingContext.java:825)
at org.apache.naming.NamingContext.lookup(NamingContext.java:173)
at org.wso2.carbon.context.internal.CarbonContextDataHolder$CarbonInitialJNDIContext.lookup(CarbonContextDataHolder.java:709)
Here is the code snippet:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="RMSPayTermOutToRMSProxy"
startOnLoad="true"
statistics="disable"
trace="disable"
transports="http,https">
<target>
<inSequence>
<log level="full"/>
<property name="OUT_ONLY" scope="default" type="STRING" value="true"/>
<property name="FORCE_SC_ACCEPTED" scope="axis2" value="true"/>
<send>
<endpoint>
<address format="pox"
uri="jms2:/RIB_AQ_USER.ETEXTPAYTERM?transport.jms.ConnectionFactoryJNDIName=JMSSenderTopicConnectionFactory"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="serviceType">proxy</parameter>
<description/>
</proxy>
I have faced a similar problem a year ago, and we fixed using the below configurations.
We have configured the Connection Factory(AqQueueConnectionFactory) in axis2.xml
<!-- Oracle AQ Listener Start-->
<transportReceiver name="oracleaq" class="org.apache.axis2.transport.jms.JMSListener">
<parameter name="AqQueueConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">oracle.jms.AQjmsInitialContextFactory</parameter>
<parameter name="db_url" locked="false">jdbc:oracle:thin:#hostname:port:schema</parameter>
<parameter name="java.naming.security.principal" locked="false">username</parameter>
<parameter name="java.naming.security.credentials" locked="false">password</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
</parameter>
<parameter name="AqTopicConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">oracle.jms.AQjmsInitialContextFactory</parameter>
<parameter name="db_url" locked="false">jdbc:oracle:thin:#hostname:port:schema</parameter>
<parameter name="java.naming.security.principal" locked="false">username</parameter>
<parameter name="java.naming.security.credentials" locked="false">password</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>
</parameter>
</transportReceiver>
<!-- Oracle AQ Listener End -->
We have to restart the server after changing the axis2.xml and adding the following jar file (aqapi-10.1.3.1.jar and its dependencies) in the lib folder.
In the Proxy, Parameters section, we defined the Queue configuration to which the Proxy listens to.
<parameter name="transport.jms.DestinationType">queue</parameter>
<parameter name="transport.jms.Destination">Queues/QueueName</parameter>
<parameter name="transport.jms.ContentType">
<rules xmlns="">
<jmsProperty>contentType</jmsProperty>
<default>application/xml</default>
</rules>
</parameter>
<parameter name="transport.jms.ConnectionFactory">AqQueueConnectionFactory</parameter>
AS stated above in the axis2.xml, we enabled the oracleaq as one if the transport of the proxy.
Once the above setup is done, we can listen to the queues/topics from Oracle AQ without any issues.
Thanks.

Email receiver proxy in wso2 esb 5.0.0 is not working

I have created email receiver proxy in WSO2-ESB5.0, proxy is connecting to email box and and I am seeing text in the logs "Processing message # : 1 in same thread".
But nothing is getting printed in the body of the proxy where I am expecting proxy should read the email body and pass to insequence of the proxy.
And also emails are not MOVED OR DELETED when I set the properties ActionAfterProcess to MOVE/DELETE.
I see some issues here
you have in one hand
<parameter name="transport.mail.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.mail.ActionAfterProcess">DELETE</parameter>
and on another hand you try to move to a folder after process or failure
<parameter name="transport.mail.MoveAfterFailure">Inbox/Failed</parameter>
<parameter name="transport.mail.MoveAfterProcess">Inbox/Processed</parameter>
So either you should remove all those four lines (default behaviour is "REMOVE") or change to
<parameter name="transport.mail.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.mail.ActionAfterProcess">MOVE</parameter>
In order to move processed files
Fetch size sounds really small to me, could you try a higher value like
<parameter name="mail.imap.fetchsize">1024</parameter>
Then I assume Inbox/TestInbox is existing and you successfully connect to the mailbox. Are you sure that the message content type is application/xml (could you simply use plain/text for instance).
Could you also try
<property name="senderAddress" expression="get-property('transport', 'From')"/>
<log level="custom">
<property name="Sender Address" expression="get-property('senderAddress')"/>
</log>
To see if you can get the sender address.
Here is sample of my proxy:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="MailTransportProtocolIMAP"
transports="mailto"
startOnLoad="true"
statistics="enable"
trace="enable">
<description/>
<target>
<inSequence>
<log level="full">
</log>
<log category="ERROR">
<property name="emailBody" expression="$body/*"/>
</log>
<drop/>
</inSequence>
<outSequence>
<log category="ERROR">
<property name="emailBody" expression="$body/*"/>
</log>
<send/>
</outSequence>
</target>
<parameter name="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="mail.imap.starttls.enable">false</parameter>
<parameter name="transport.mail.ContentType">application/xml</parameter>
<parameter name="mail.imap.host">outlook.74.net</parameter>
<parameter name="transport.mail.Address">abc#abc.com</parameter>
<parameter name="mail.imap.user">abc123</parameter>
<parameter name="mail.imap.socketFactory.port">993</parameter>
<parameter name="transport.mail.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.mail.ActionAfterProcess">DELETE</parameter>
<parameter name="transport.PollInterval">2</parameter>
<parameter name="mail.imap.port">995</parameter>
<parameter name="mail.imap.ssl.enable">false</parameter>
<parameter name="transport.mail.MoveAfterFailure">Inbox/Failed</parameter>
<parameter name="transport.mail.Folder">Inbox/TestInbox</parameter>
<parameter name="transport.mail.MoveAfterProcess">Inbox/Processed</parameter>
<parameter name="mail.imap.fetchsize">2</parameter>
<parameter name="mail.imap.socketFactory.fallback">false</parameter>
<parameter name="transport.mail.Protocol">imap</parameter>
<parameter name="mail.imap.password">abc123!</parameter>
</proxy>

WSO2 ESB MailTransportListener not receiving

I just installed my brand new WSO2 ESB 4.9.0. This is the latest version which I downloaded yesterday.
I am trying to get my ESB to receive e-mail from a server. I have tried POP3 and IMAP. I have been using several references to help me(Since I am a new member, my reputation does not yet allow me to post proper links to all my references):
stackoverflow.com/questions/27376833/wso2-mailto-transport-emptied-my-inbox
javamail.java.net/nonav/docs/api/com/sun/mail/pop3/package-summary.html
javamail.java.net/nonav/docs/api/com/sun/mail/imap/package-summary.html
sparkletechthoughts.blogspot.fr/2013/10/how-to-receive-emails-to-wso2-esb.html
charithaka.blogspot.com/2012/11/how-to-enable-mail-transport-for-wso2.html
irhamiqbal.wordpress.com/how-to-receive-email-using-wso2-esb/
jayalalk.blogspot.com/2014/06/wso2-esb-mail-sending-with-html-complex.html
And of course the official documentation:
https://docs.wso2.com/display/ESB481/MailTo+Transport
https://docs.wso2.com/display/AS501/Using+Mail+transport+to+Invoke+Web+Services
All of my reading leads me to believe that I have everything set up properly (or if not properly, good enough to get some error messages to troubleshoot).
I did restart the WSO2 ESB and the following message shows up in the console log:
TID[-1234] [ESB] [2016-08-02 21:45:37,237] INFO {org.apache.axis2.transport.mail.MailTransportListener} - MAILTO listener started
I get no other indication in the logs one way or another.
I edited (and double checked) my $CARBON_HOME/repository/conf/axis2/axis2.xml, and I'm pretty sure I have the mail transport enabled properly:
<transportReceiver name="mailto" class="org.apache.axis2.transport.mail.MailTransportListener">
<!-- configure any optional POP3/IMAP properties
check com.sun.mail.pop3 and com.sun.mail.imap package documentation for more details-->
</transportReceiver>
<transportSender name="mailto" class="org.apache.axis2.transport.mail.MailTransportSender">
<parameter name="mail.smtp.host">my.smtp.server</parameter>
<parameter name="mail.smtp.port">587</parameter>
<parameter name="mail.smtp.starttls.enable">false</parameter>
<parameter name="mail.smtp.auth">true</parameter>
<parameter name="mail.smtp.user">my#user.id</parameter>
<parameter name="mail.smtp.password">password</parameter>
<parameter name="mail.smtp.from">my#email.com</parameter>
</transportSender>
And, finally, I have my Proxy Service setup (in its current iteration, anyway):
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">my.pop3.email</parameter>
<parameter name="mail.pop3.user">my#email.com</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">jasper#eryjus.com</parameter>
<parameter name="mail.pop3.password">password</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<description/>
</proxy>
With all of this, my question is: is there something else I am missing to get the ESB to at least attempt to retrieve e-mail? What bothers me is I am not getting any error messages to troubleshoot (such as my mail not being in plain test).
I tried the same scenario with a Gmail account and it works fine for me, seems like some mis-configuration in your POP server. You need to add following property as well.
<parameter name="transport.mail.ContentType">text/plain</parameter>
See the full proxy configs below.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="JasperEmailPOP3"
transports="mailto"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="senderAddress"
expression="get-property('transport', 'From')"
scope="default"
type="STRING"/>
<log level="custom">
<property name="Date" expression="$trp:Date"/>
<property name="Subject" expression="$trp:Subject"/>
<property name="Content-Type" expression="$trp:Content-Type"/>
<property name="From" expression="$trp:From"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="mail.pop3.host">pop.gmail.com</parameter>
<parameter name="mail.pop3.user">ycwso2test</parameter>
<parameter name="transport.mail.Protocol">pop3</parameter>
<parameter name="mail.pop3.socketFactory.port">995</parameter>
<parameter name="transport.mail.Address">ycwso2test#gmail.com</parameter>
<parameter name="mail.pop3.password">wso2qa123</parameter>
<parameter name="mail.pop3.port">995</parameter>
<parameter name="mail.pop3.socketFactory.fallback">false</parameter>
<parameter name="transport.mail.ContentType">text/plain</parameter>
<description/>
</proxy>
Also if you need to further debug this you can enable DEBUG logs. To do this add the following line to repository/conf/log4j.properties and restart.
log4j.logger.org.apache.axis2.transport.mail.MailTransportListener=DEBUG

XPath from file in VFS Transport WSO2 ESB

I want to read some values from a file that being processed by the VFS transport.
My file is :
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
So I use a property mediator to get the value I want using XPath. But when I print that property in Log Mediator it prints nothing. My proxy look like this :
<proxy name="Try1"
transports="vfs"
startOnLoad="true"
trace="disable">
<target>
<inSequence>
<property xmlns:ns="http://org.apache.synapse/xsd"
name="source"
expression="//header/source/text()"
scope="default"
type="STRING"/>
<log level="full">
<property xmlns:ns="http://org.apache.synapse/xsd"
name="Value_source"
expression="get-property('source')"/>
</log>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///Users/Target</parameter>
<parameter name="transport.vfs.FileURI">file:///Users/Source</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///Users/Failure</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
The log mediator result is
[2013-06-28 16:59:49,939] INFO - LogMediator Value_source=
So how can I get the values in file using XPath?
The issue is due to the ContentType set for the transport parameter:
Switch
<parameter name="transport.vfs.ContentType">text/plain</parameter>
with
<parameter name="transport.vfs.ContentType">application/xml</parameter>
since you're doing XPath operations on the message.
The problem is with the content of your file. Since it is not well-formed, the xpath expression fails. Isn't it possible for you to have the well-formed xml in the input content?
<sales>
<header>
<source>STORE1</source>
<target>HEADOFFICE</target>
</header>
</sales>