We are using WSO2 ESB and as transport we have WebSphere MQ which is accessed using JMS.
Problem is that each proxy service works in one thread with WebSphere MQ and because of that we have performance issues.
How can we start multiple instances of proxy service without deploying multiple copies of it? Maybe there are some hidden configuration parameters?
In addition to already mentioned parameter
<parameter name="transport.jms.ConcurrentConsumers">2</parameter>
you might need to add another one:
<parameter name="transport.jms.IdleTaskLimit">2147483647</parameter>
This parameter represents number of idle message read attempts per thread. When counter of such idle read attempts becomes equal to this parameter, the thread stops reading messages (if it is not the only reader).
Given that, setting this parameter to the upper limit of java int provides you with virtually infinite reading by all the threads.
You should define this parameter in your proxy conf :
<parameter name="transport.jms.ConcurrentConsumers">2</parameter>
WSO2 use one consumer by default
Have a look there : http://mmalithh.blogspot.fr/2013_05_01_archive.html
Related
I've just successfully implemented a JMS-message-processor in my WSO2-process. However, when logged in as an admin on the ActiveMQ console, I can view the stats of the queue but I cannot access the contents of the pending message. Instead, I see this error:
Cannot display ObjectMessage body. Reason: Failed to build body from content. Serializable class not available to broker. Reason: java.lang.ClassNotFoundException: org.apache.synapse.message.store.impl.commons.StorableMessage
Accessing the contents of a pending message is possible when using the RabbitMQ, so I figured it should also be possible with this message broker.
I've found solutions online, but they don't seem to be working, or perhaps they're out of date. Can somebody point me to the correct answer to solve this? I'm on WSO2 Integration Studio 8.1.0 and am using ActiveMQ 5.17.1.
It appears that WSO2 is using javax.jms.ObjectMessage despite the fact that there's a long list of reasons why doing so is a bad idea, including the issue you're currently facing, but I digress.
The problem here is that the implementation of the javax.jms.ObjectMessage (i.e. org.apache.synapse.message.store.impl.commons.StorableMessage in this case) must be on the classpath of any application which wants to deserialize that message. This class is not, in fact, on the classpath of ActiveMQ therefore the ActiveMQ web console cannot deserialize the message and display its contents (assuming those contents are human-readable in the first place). That's why a ClassNotFoundException is thrown.
You may be able to resolve the issue by putting org.apache.synapse.message.store.impl.commons.StorableMessage on ActiveMQ's classpath. Aside from that there's really nothing to be done.
I assume this is different for RabbitMQ because in that case WSO2 doesn't use the JMS API and uses the AMQP protocol which is not Java-centric like JMS.
Justin has explained the cause of the issue and as suggested you can try adding the class to ActiveMQ runtime and see whether it resolves the issue. This class is located in the Synapse-Core(synapse-core_2.x.x.wso2vXXX.jar). But remember although this may resolve the ActiveMQ UI issue, this message is not consumable by other systems unless they know how to deserialize it.
Let me add more details on why the message is serialized. Message serialization happens when you use the Store mediator, and when you store a message with Store Mediator it is intended to be only read by a Message Processor. Simply the serialized message can only be consumed by WSO2 Message Processors. In other words, Store Mediator and the Message processors are tightly coupled.
If message serialization is an issue for you. For example, if other systems are consuming the messages that WSO2 publishes you can try using the JMS transport to produce and to consume messages from ActiveMQ. Other than that you can also consider using JMS Inbound Endpoint to consume messages, which all use standard media types when storing the message.
I'm quite a newbie in WSO2 so sorry for the mistakes (and for my english too ... )
I need to implement a proxy with delivery-garantee pattern and here you are my solution (I'm started from this post http://charith.wickramaarachchi.org/2012/05/another-message-redelivery-pattern-with.html):
a proxy invoke an external service giving, as input, the initial
client message
if the external service is running all works fine and
the reply is given to the client
if the external service is down or generate a SOAP fault, I'll
put the message in a store (retry store), and then, using a sampling
processor (after a time "t"), I'll try again for "n" max attempts:
at any attempt, if the external service is down or generate a SOAP
fault, I'll put the message again in the retry store, and the
process is repeated
after "n" attempts, if the external service is still out of
service, the message is stored in another store (garbage store)
All works fine when I try to test with one message, but when I try to test with more messages (> 20 but this number is variable ... ), the sampling processor hangs completely, nothing is shown in the logs. Looking in the console, sometimes (but not always ...), the processor is off, deactivate and in this case, to restore, I need to undeploy, stop and restart, and then deploy again my .car.
NOTE: I've to use the sampling processor and not the forwarding processor because this processor, after "n" attempts deactive itself and I can't use it for my goals.
I can't put here the complete code because is too long, but I can give you a sample .car that you can deploy and execute on your WSO2 installation (to simulate the external service I've used the echo service ...).
Here you are the sample car that you can download
Thank you very much in advance: all suggestions are appreciated!!!
Cesare
Message Forwarding Processor
Retrieves the messages stored in a message store and reliably forwards them to a specified endpoint. This processor attempts to send one message at a time and it does not dequeue a message from the store until it receives a response from the target endpoint. Therefore this processor is ideal for implementing in-order delivery scenarios and guaranteed delivery scenarios.
Sampling Processor
Retrieves the messages stored in a message store and injects them to a given sequence at specified intervals. This processor utilizes the Quartz scheduler framework for periodically processing messages. This can be used to implement message rate throttling scenarios.
--> You can use the forwarding processor and configure it so that it will never be deactivated, just add this parameter : <parameter name="max.delivery.attempts">-1</parameter>
I've been attempting to monitor the WSO2 MB 2.1.1 via JMX which appears to be a valid option for other WSO2 tools such as the ESB however the MB does not appear to be updating any of the MBeans for the Queue attributes.
Ex:
org.wso2.andes:type=VirtualHost.Queue,VirtualHost="carbon",name="testQueue"
This has a number of potentially useful attributes like the following:
ConsumerCount
ActiveConsumerCount
ReceivedMessageCount
MessageCount
However the counters always read "0" even when there are messages in the queue dropped in via the sample sender or manually.
Creating a new queue will create the following, but the issue is seen on the new queue as well.
org.wso2.andes:type=VirtualHost.Queue,VirtualHost="carbon",name="testQueue2"
Am I simply looking in the incorrect location or any other variety of user error?
Is this the intentional behavior of the application?
Do you have any suggestions which may assist in getting this data reported via JMX?
Any help would be appreciated.
Thanks
This does not appear to be functional in MB 2.1.1, it looks as if they may be moving to using the CEP or the BAM to provide metrics on some products.
I am using wso2esb4.7.0 and ActiveMQ 5.8.0 versions
i followed wso2esb docs they provided store and forward message store policy
But i dont want store i just want to consume those messages which is already has been store
by my client application i wish to pool that messages for every 5 sec
Is it possible in wso2esb jms using ActiveMq
will you write sample code of proxy
<messageProcessor name="Duplicate5" class="org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor" messageStore="Duplicate" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="interval">1000</parameter>
<parameter name="message.processor.reply.sequence">fault</parameter>
</messageProcessor>
i tried with this but its not working
To pull messages from the queue, you need to use JMS transport..Check JMS proxy for the samples..
The message store persists the messages in the form of serializable Java objects. This might contain certain underlying information(E.g.: Properties) which is not visible in the message when you view message details in the ActiveMQ console. The message processor uses such information stored by the store when processing the message. Therefore, as far as I understand, the message store and message processor should be used together if you want to make things functional.
If you're straightaway storing the received messages in ActiveMQ queue, you might have to configure the message consumer manually. Check this usecase [1].
And also look in to this blogpost example to get an idea [2].
[1] http://docs.wso2.org/wiki/display/ESB470/ESB+as+a+JMS+Consumer
[2] http://nuwanwimalasekara.blogspot.com/2013/04/jms-proxy-service-using-wso2-esb.html
Hope this helps.
You can not use the Message-processor alone, You must have combination of Message Store and message processor. if you want to understand the behavior of the Message Store and Message processor refer the blog some time back.
If you want to use listen/pull from the JMS using ESB you have to use ESB as a JMS Consumer. Please refer further on document to implementation detail.
WSO2 ESB is not loading all proxies (more than 20). Then we increased following two values in the startup script and it worked:
-Dsnd_t_core=120
-Dsnd_t_max=600
But then we encountered several fatal issues of the WSO2 ESB. Several JMS proxies were blocked and did not consume anymore messages. The worst thing of all: NO ERROR in the carbon.log!
In addition the CPU load on the server went up to 100%.
A restart did not solve the problem, only deactivating scheduled tasks or proxies solved the problem.
We now discovered, that a VFS proxy is creating exactly 120 threads (JConsole). With each transport.PollInterval it creates a new Thread.
Which values do you use for the -Dsnd_t_core and max?
Why is a VFS proxy creating a new thread (see jconsole) a t each PollInterval?
As long as I know, WSO2 ESB thread is based on java.util.concurrent ThreadPool.
In this link you can read about some ThreadPool characteristics like when will it create a new thread, the queue mechanism, and reject task policy.
Which values do you use for the -Dsnd_t_core and max?
-Dsnd_t_core is the minimum number of threads inside a ThreadPool. So WSO2 ESB will automatically create thread as much as you set the -Dsnd_t_core. The default value is 20. WSO2 ESB will create 20 vfs-worker if you don't specify the -Dsnd_t_core.
-Dsnd_t_max is the maximum number of threads inside a ThreadPool. WSO2 ESB will stop create a new thread if the maximum number is reached.
Why is a VFS proxy creating a new thread (see jconsole) a t each PollInterval?
WSO2 ESB will create a new thread in these conditions :
If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be
rejected.
So, as long as your queue is full and the maximum number of threads is not reached, WSO2 will create a new thread to handle a process. The PollInterval is set to specify the delay before your service start to poll the message or file from the source folder.
You can set the maxQueue number to unbound (-1) so the queue will never full and new thread will never be created.
I also found something from the JConsole. 1 service/1 proxy service will be handled only by 1 thread. I still trying to figure this out and make 1 service/1 proxy service is handled by 2 or more thread (multithread).
Hope this will help answer your question :)