I'm trying to do some load testing on the WSO2 ESB, and I have a SOAP message with a large payload that I'd like to send (10+MB, sometimes reaching 50 inbound, and sometimes even 300 outbound).
I'm using curl to send the messages, first to the backend SOPAUI service. This goes through fine. However when I send the message via a proxy service on the ESB, curl fails to finish uploading the file, and gets slower and slower as the upload progresses. The logs from the ESB do not seem to be reporting anything useful.
I have set the timeout on this to 20 minutes, and it still does not complete. Turning on MTOM has no effect, and I am using a simple passthrough proxy. The proxy service functions fine with small message sizes.
What else can I look into to get this working? If it is indeed possible.
You could try to configure Message Relay
in repository/conf/axis2.xml, you configure :
org.wso2.carbon.relay.BinaryRelayBuilder as the messageBuilder's class associated with your contentType (text/xml for exemple if soap11 of application/soap+xml with soap12)
org.wso2.carbon.relay.ExpandingMessageFormatter as the messageFormatter's class
Have a look there : https://docs.wso2.com/display/ESB481/Message+Relay
Turned out I had to switch to using the NHTTP transport rather than the Passthrough transport, thanks to this question here for prompting looking into this: WSO2 ESB unknown error code 102511
Requires an edit in carbon.xml, detailed in the accepted answer from the link above.
Related
I want to send a POST request with a body in WSO2. actually, I have a Django rest framework with some APIs and want to send requests in WSO2 API manager. it's all OK for GET, DELETE but when I want to POST some request I'll receive a bad request from Django server.
Django server is running on http://localhost:8000/user-api/users/
in wso2 api publisher api's URL is : http://localhost:8243/users/1.0.0
endpoint in wso2 is : http://localhost:8000/user-api/users/
the below picture is what I receive when I send GET request:
I send POST request with this json as body:
and here is what I receive when I send POST request:
Everything is ok when I check the server's URL in my browser.
UPDATE:
when I use my Django server as a gateway for WSO2 everything is OK but when I use default WSO2-Gateway everything goes wrong.
I'm not clear about what this question is really asking. But I feel this is generally asking how to debug this situation. I apologize if I got that wrong.
To start debugging a problem with a WSO2 server in the mix there are some common things to think about:
If this is a local or development region can I use http, instead of https, so that I can view things as plain text during debugging activities.
For the Enterprise Integrator products there is a test feature for back end endpoints in the admin Console. This can often be helpful to ensure communication can happen. It's like a simple ping.
Check inside the trace.log to see if there is anything helpful for you.
For APIM use the Test a REST API feature to try out the backend to see if that works before coming in the front door of the server.
For WSO2 products if reviewing logs aren't giving enough information then you can use the Wirelogs to enable you to view every piece of information that goes in and out of the WSO2 server. These are formatted roughly like Wireshark logs but the output is inside of the wso2carbon.log file. The key here is getting used to the >> and << to determine if the log entry is going out of or into the WSO2 server.
If you still can't find the issue then you might need to go down to reviewing the data as it goes across the wire. This is a bit complex at first but you'll quickly get the hang of it. (Hint: This is extremely helpful for trouble shooting JMS communication.)
On linux you can use tcpdump to write data to file and then open it with Wireshark. Windows has a similar system.
For tcpdump you need to use the interface name. It's often like eth0. You can find t using ifconfig. It's the value: before the flags.
Example tcpdump capture:
tcpdump -s 0 -I {interfacename} -W 30 -C 50 -w wsoei-01-07192022.pcap -Z root
tcpdump -s 0 -i ens192 -W 30 -C 50 -w wsoei-01-07192022.pcap -Z root
Once you have the pcap file captured you then use Wireshark to open it. And there you have it you can see everything that went across the wire. Once you have that you can start to see at what point things are breaking down.
Conclusion
If you zoom in close enough with your research you will eventually find where the breakdown is.
I have a BPS process that rapidly performs a lot of calls to an ESB proxy. The ESB proxy calls an ESB API, which calls the API of our own product.
Often, the BPS process pauses for exactly 3 minutes while waiting for a response from the ESB. After the 3 minutes, everything continues as if nothing happened. Using Wireshark and some additional Log mediators, I have discovered the following:
The hanging occurs in the ESB proxy, right before the out sequence.
According to Wireshark, the ESB API has already sent its response to the ESB proxy when the hanging occurs.
The fact that this hanging takes exactly 3 minutes should be a hint. I searched the config files for anything specifying 180 seconds, but the only ones are the http socket timeouts and the http transportReceiver's threadKeepAliveTime. If I reduce those, the BPS throws a p2p communication error instead of continuing. To me, this indicates that something somewhere in the background is still causing a 3 minute delay for some calls.
By the way, the hanging does not occur for some specific calls only. I have been testing several times, performing the exact same calls each time, and I cannot predict which call will hang or even if any call will hang at all.
I would suggest you add at least a <send /> mediator in your faultSequence.
Perhaps also add a simple makefault mediator (don't make it too complex just yet - just a static error response will do - at this stage you don't want faults originating within the faultSequence)
I suspect some error from the backend service or in the out sequence or maybe even faultSequence is not being handled - which means instead of sending a response back to your caller, the proxy hangs until time out instead
PS: It really helps if you post the source code of your proxy too
I had the same experience sometimes ago when I tried to call PUT or POST methods without a request body. But, by default, WSO2 ESB expects a message body to send the request to backend server when the http method is POST or PUT.
I had to either send the request with a request body (at least an empty JSON: {}) or use the property FORCE_POST_PUT_NOBODY in your APIs.
Read this post for more details on how to use FORCE_POST_PUT_NOBODY property.
If you enable the wire log in WSO2 you will realize NGIX is using HTTP 1.0 client in the request.
In order to fix that change your NGINX mapping to use "proxy_http_version 1.1;" and the problem will be sorted.
Cheers.
I ended up getting WSO2 support involved in this. Apparently, I was hitting some very rare edge case which was very hard for the WSO2 engineers to reproduce.
In the end, the fix turned out to be adding the following property just before the send mediator in the proxy service:
<property name="NO_KEEPALIVE" value="true" scope="axis2"/>
I've been working with WSO2 Message Broker in order to publish and consume messages through jms and other clients. I've gone through the official documentation including the samples.
What I want to know is that, is it possible for WSO2 MB to handle http requests directly instead of jms. Yes you can do something like this, by converting http to jms through WSO2 ESB which seems like a long process.
ie: I mean having http as the end point or the transport to publish and consume messages directly in WSO2 Message Broker.
In order to achieve this, you will have to use ESB + MB set up as explained at [1]. What you can do is to use ESB to wrap JMS functionality using a HTTP client. It is recommended to use ESB externally in a separate JVM when using this setup.
[1] http://wso2.com/library/articles/2013/03/configuring-wso2-esb-wso2-message-broker/
Regards,
Pubudu.
I am trying to setup very simple WSO2 ESB Proxy service. While using it, I am getting HTTP 202 response back and WSO2 ESB is not doing anything with the request beside logging it. Here is the background of my setup
My service implementation is using SOAP 1.2 over Http 1.1. When my client opens the connection to the server, it fires first request and asks for keep-alive connection. The ESB passes the request to the actual implementation and sends response back with transfer-encoding as chunked. So far it works as desired.
After the initial request response exchange, my client submits several requests in parallel and I get HTTP 202 responses for all of them. Looking at logs, it seems ESB is not sending the request to the actual implementation ever.
Is there something that I am doing wrong? How do I fix it?
What happens in this scenario is that your subsequent requests are hitting the main sequence of the WSO2 ESB. That is why you can only see a log for those requests. As you have already narrowed down this happens due to the jsessionId attached to the URL. To overcome this issue you can create a REST API with URL pattern to match the correct URL path. Please refer the following documentation.
https://docs.wso2.com/display/ESB481/Creating+APIs
I have a created web service that takes 2 complex types as parameters. I can use the auto-generated WSDL to create a new application that calls my web service and it works perfectly.
However, a 3rd party consumer is having trouble calling my web service. I see their incoming request in the IIS7 log file and a 500 error is being returned to them.
I suspect that their request is malformed. I would like to be able to see the entire <soap:envelope> that IIS7 is receiving so that I can troubleshoot the issue. I am using Windows Server 2008 and IIS7.
How can I do this?
I would open up a copy of WireShark while they are putting their request through. You can then see each packet which comes to your server. Wireshark is cool because it can put all of the packets back together so you can see the full HTTP request they're sending.