I am attempting to make a blocking call that retrieves a PDF file, but an error is thrown whenever I try to use the returned payload (including sending it).
The error is "Attempted read on closed stream", and I think it's happening because the response is being parsed into xml only when it's needed, but by then the call mediator has closed the connection.
Here is an example sequence:
<call blocking="true">
<endpoint>
<http uri-template="https://myserver.com/generatePDF?id=1234" method="GET"/>
</endpoint>
</call>
<log level="custom">
<property name="msg" value="Complete"/>
</log>
<log level="custom">
<property name="body" expression="$body"/>
</log>
<log level="full"/>
Watching the logs, the call is made and it waits for the response. Once the response is completed the "Completed" log message is printed. But when it tries to access the $body the following error occurs:
[2016-05-19 13:03:21,582] INFO - LogMediator msg = Complete
[2016-05-19 13:03:21,584] ERROR - SequenceMediator Error reading data handler
org.apache.axiom.om.OMException: Error reading data handler
at org.apache.axiom.om.impl.llom.OMTextImpl.internalSerialize(OMTextImpl.java:421)
at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:555)
at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:875)
at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:555)
at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:875)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.serialize(OMSerializableImpl.java:125)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.serialize(OMSerializableImpl.java:113)
at org.apache.axiom.om.impl.llom.OMElementImpl.toString(OMElementImpl.java:988)
at org.apache.synapse.util.xpath.SynapseXPath.stringValueOf(SynapseXPath.java:321)
at org.apache.synapse.mediators.MediatorProperty.getEvaluatedExpression(MediatorProperty.java:138)
at org.apache.synapse.mediators.builtin.LogMediator.setCustomProperties(LogMediator.java:216)
at org.apache.synapse.mediators.builtin.LogMediator.getCustomLogMessage(LogMediator.java:139)
at org.apache.synapse.mediators.builtin.LogMediator.getLogMessage(LogMediator.java:125)
at org.apache.synapse.mediators.builtin.LogMediator.mediate(LogMediator.java:95)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:81)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:48)
at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:149)
at org.apache.synapse.core.axis2.ProxyServiceMessageReceiver.receive(ProxyServiceMessageReceiver.java:185)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:395)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:142)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Attempted read on closed stream.
at org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:183)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:107)
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.wso2.carbon.relay.BinaryRelayBuilder.readAllFromInputSteam(BinaryRelayBuilder.java:39)
at org.wso2.carbon.relay.StreamingOnRequestDataSource.getInputStream(StreamingOnRequestDataSource.java:59)
at javax.activation.DataHandler.writeTo(DataHandler.java:305)
at org.apache.axiom.util.stax.XMLStreamWriterUtils.writeBase64(XMLStreamWriterUtils.java:62)
at org.apache.axiom.util.stax.XMLStreamWriterUtils.writeDataHandler(XMLStreamWriterUtils.java:138)
at org.apache.axiom.om.impl.llom.OMTextImpl.internalSerialize(OMTextImpl.java:418)
... 24 more
Stepping through the code I can see that the response is available on the http connection. An inputstream is built around that connection but after the call is complete the HTTP Connection is closed in the cleanup method.
When the body is first accessed, the stream is read but by then it's already closed, which gives the error below.
How can I retrieve a PDF using the call mediator with blocking="true"?
Details
WSO2 ESB 4.9.0
Added the following to the message builders section of axis2_blocking_client.xml
<messageBuilder contentType="application/pdf" class="org.wso2.carbon.relay.BinaryRelayBuilder" />
Binary-Relay uses these Formatters and Builders to relay messages---both well formed as well as ill-formed messages---through Apache Synapse without processing ( e.g. . XML Validation or processing) those messages. Binary-Relay includes a Builder and a Formatter. When a message is received, the Builder creates a fake SOAP message, attaches the content of the incoming SOAP message as a binary XML node, and sends it through the Apache Axis2 kernel, and when it is being sent out, the Binary-Relay Formatter expands the SOAP messages and writes the attached content to the output channel. Consequently, the message goes through Apache Synapse without processing its content, since the message is handled at the bytes level, even non-XML messages may go through Apache Synapse.
For more information please read.
Related
How do you add the IBM MQ Correlation ID header to a message in WSO2 Integrator?
I have a scenario where I receive a message from MQ, transform the message using a few mediators and calls to backend services, and need to reply with the transformed message to another (MQ) queue using WSO2 Integrator (version 6.1.1). The client receiving the message is expecting to find the same Correlation ID in my reply as in his original message.
IBM tells me the property is this one:
The CorrelationId to be included in the MQMD of a message when put on
a queue.
Defined in: MQMessage class
Data Type: String of 24 characters
Syntax: To get: correlid$ = MQMessage.CorrelationId To set: MQMessage.CorrelationId = correlid$
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q034650_.htm
It turns out it was an easy solution. The JMS_COORELATION_ID property must be set specifically in the axis2 scope.
So while this works:
<property name="JMS_COORELATION_ID" value="MyCoorelationId" scope="axis2" xmlns:ns="http://org.apache.synapse/xsd"/>
Using any other scope, such as default or transport, does not:
<property name="JMS_COORELATION_ID" value="MyCoorelationId" scope="transport" xmlns:ns="http://org.apache.synapse/xsd"/>
<property name="JMS_COORELATION_ID" value="MyCoorelationId" scope="default" xmlns:ns="http://org.apache.synapse/xsd"/>
Unfortunately the WSO2 documentation is dreadful and does not mention that other scopes do not work. Hope this saves someone time in the future.
I am very new in WSO2 and I have the following problem.
I am developing an ESB application that achieve the following tasks:
Perform some DSS queries on a local database.
Use the retrieved data to build some XML documents.
Send all these documents to a remote web service (implemented in PHP).
Obtain the web service responses (a specific response for all request that contains one of these generated XML document) and use its content to store a value on the previous local databse.
The web service receiving the previous request containin an XML document stores the document content on another database.
I have implemented this ESB application, and it seems to works (but I am finding some problem that I will explain later) using the following logic:
I have created an API containing an in sequence that perform the queries, build the XML documents, send these documents to the web service, something like this (I can't post the entire code because the XML creation logic is pretty big):
<?xml version="1.0" encoding="UTF-8"?>
<api context="/glisTest2" name="glisTest2" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="GET" outSequence="glisOutSequence">
<inSequence>
<payloadFactory media-type="xml">
<format>
<body/>
</format>
<args/>
</payloadFactory>
<header name="Action" scope="default" value="urn:FindNotProcessed"/>
<log level="full"/>
<callout endpointKey="prgfasEndpoint">
<source xmlns:ns="http://org.apache.synapse/xsd" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope" xpath="s11:Body/child::*[fn:position()=1] | s12:Body/child::*[fn:position()=1]"/>
<target xmlns:ns="http://org.apache.synapse/xsd" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope" xpath="s11:Body/child::*[fn:position()=1] | s12:Body/child::*[fn:position()=1]"/>
</callout>
<log level="full"/>
<property expression="count(//ds:Sample)" name="total_samples" scope="default" type="STRING" xmlns:ds="http://ws.wso2.org/dataservice"/>
<log level="custom">
<property expression="$ctx:total_samples" name="total samples: "/>
</log>
<!-- Iterate throug samples -->
<iterate expression="$body//ds:Sample" id="ITR_AGG" sequential="true" xmlns:ds="http://ws.wso2.org/dataservice">
<target sequence="sampleDataSequence"/>
<!-- This is my main sequence. It will call some other sequece in cascade: -->
</iterate>
</inSequence>
<faultSequence/>
</resource>
</api>
The callout mediator is used to obtain to call the DSS service that performs query to obtain data.
Now in this section of code:
<!-- Iterate throug samples -->
<iterate expression="$body//ds:Sample" id="ITR_AGG" xmlns:ds="http://ws.wso2.org/dataservice">
<target sequence="sampleDataSequence"/>
</iterate>
I am iterating on each retrieved element and for each element I will basically build an XML that will be send to the remote web service. All this logic (pretty big because the XML documents contains many fields) are contained into the sampleDataSequence sequence (this sequence will be performed at each iteration to build and send an XML document).
I am not attaching the entire code, at the end of the sampleDataSequence sequence I perform the web service call (a POST request) passing to it the current XML document (generated in the current iteration), in this way (previously I put the current XML document in the body request):
<property name="messageType" scope="axis2" type="STRING" value="application/xml"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="post"/>
<send>
<endpoint key="glisEndpoint"/>
</send>
So, for each iteration, I build a brand new XML document and I send to it to my web service (this works, the web service receive it).
So the in sequence end, then there is the out sequence that that receives and collects all the web service responses so then it can parse these response one by one and write a record into a result table on my local database.
I have done it in this way:
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="glisOutSequence" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
<aggregate id="ITR_AGG">
<completeCondition>
<messageCount max="-1" min="-1"/>
</completeCondition>
<onComplete expression="s11:Body/child::*[position()=1] | s12:Body/child::*[position()=1]" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope">
<property name="it_count" scope="operation" type="STRING" value="0"/>
<!-- Iterate over responses. For correct count calculations they should
be sequential -->
<iterate expression="$body//response" id="ITR_RES" sequential="true">
</aggregate>
<log level="custom">
<property expression="$body" name="AT THE END"/>
</log>
<send/>
</sequence>
So basically I am using the 2 Enterprise Integration Pattern provided by WSO2 ESB: Splitter and Aggregator to achieve this task. It seems to me that this should be the standard solution for this type of task (when you have to send n message to an external web service and then collect and operate on the n response from this web service). Is it my architecture the standard one for this type of task?
I am asking it because now I am finding the following problem:
I have deployed this application on a test environment and we are obtaining the following problem.
The test environment have both this WSO2 ESB application and the contacted web service installed on the same laptop computer (but in the production environment these system will be on different computer because there will be many WSO2 ESB application and a single central web service that will receive message from these ESB applications).
The problem is that the WSO2 application generate a lot of XML documents (something like 1000). So it will be perform 1000 consecutive POST requests to the web service.
The web service receives all these requests (containing the 1000 XML documents that have to be inserted into another database) but after a certain number of DB inserton, the DB (Postgree) give a too many clients exception.
So this exception could be related to: connection not closes, result set not closed or something like this.
My personal opinion (it could be wrong) is that it should be a problem related to the back end of the web service and not to my WSO2 ESB application. But the person that are testing it says to me that maybe the WSO2 application doesn't close the connections.
It seems strange to me baceusa it is an error on the DB and not on the web service but I really have no idea.
Another thing is that I fix this problem putting a delay of some milliseconds before send the POST request to the web service, in this way:
<!-- Inserted a short delay to prevent flooding the GLIS server -->
<script language="js">java.lang.Thread.sleep(200);</script>
<property name="messageType" scope="axis2" type="STRING" value="application/xml"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="post"/>
<property name="ClientApiNonBlocking" value="true" scope="axis2" action="remove"/>
<send>
<endpoint key="glisEndpoint"/>
</send
Putting a short delay before send the request it seems to works fine. My idea is that maybe the problem is that in this test environment both the ESB application and the web service application are installed on the same laptop so the comuncation is very very quick (there is not a net in the middle) so the ESB application shot 1000 request very quick, the web service can receive it but it can't write on the database.
Could be the problem?
Another doubt is: who are testing the application says to me that maybe I am using the wrong enterprise integration pattern to achieve this taks and that I should perform the web service call and elaborate the response one by one (for each XML document). In my personal opinion this should be impossible using ESB logic because if I send a single XML to the web service, when it give me a response ESB enter in the out sequence and so the ESB application will end after processing a single document (because I think that I can't come back to the in sequence after that the out sequence is completed). Is it my reasoning correct?
So, at the end: could be a problem of the test environment (both application on the same PC), or a problem of the PHP web service back end application (result set not closed or something like this, it seems strange because using the delay it works) or could be something related to my ESB application?
My personal opinion - I do not see how "not closing connections" from WSO2 side (acts as a client for PHP-service)
may product problems between PHP and database.
I do not see any problems with integration patterns, you used in your solution - everything looks ok.
So, I think problem in PHP web-service or it's specific configuration on local test laptop.
For example java-applications are always used connection-pool to get DB connections -
and settings of this connection-pool can control number of connections to DB.
Not sure - but probably PHP also has some kind of connection-pools.
Also, database usually has many options about possible number of connections at same time.
So - it may be everything is configured by properly in production system, but configured "by default" in test laptop,
so - as result we have this kind of problem.
Anyway - we have situation then called service has some specific SLA - for example number of connections it may accept at same time.
Also, it looks like, it may happens this service may be unavailable or return error (for example because of database connection problem).
your JS delay is really dirty solution - WSO2 ESB (as any other ESB) has much better solution -
called "Guaranteed Delivery".
In WSO2 it may be implemented by followed way: instead of sending message to glisEndpoint, you should put it into
Message Store.
Then you need configure
Message Processor :
it will get messages from message store and send to endpoint.
If calling endpoint will be failed (by some reason) - it will put message back to Message Store.
There are many configuration options in Message Processor - so, you can configure for example to send only 1 message per second, or any other pattern.
Please - keep in mind - there are many types of message stores: starting from simplest In Memory Message store and finishing using ActiveMQ or RabbitMQ.
You should choose proper message store type depending from your requirements.
I am newbie to WSO2 ESB, so far everything is OK.
I started exploring fire and forgot feature, looks like this can be used as below to invoke async calls to end points.
<property name="CHANNEL_COD" value="999"/>
<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
<property name="OUT_ONLY" value="true"/>
But when tested the API with above code, still ESB invoking the end point in syncronous. Tried this approach with call and send mediators.
Would anybody please help me is there any link I am missing here.
With ESB 4.8.1 :
If ClientApiNonBlocking property is not set, send mediator should be asynchronous.
Because of FORCE_SC_ACCEPTED, 202 http status code will be sended back at the end of IN mediation (ie : the message has been delivered to your endpoint, without waiting for the response)
If it takes several seconds to send the message to the endpoint, the caller will receive 202 status code AFTER that.
If you want this status code to be sent ASAP, you can use clone mediator and move send mediator inside clone's target
I want to proxy RemoteUserStoreManagerService (exposed on WSO2 IS) over WSO2 ESB.
When I call getUserClaimValuesForClaims operation I get the response. I tested the WSO2 IS (connected to my Active Directory) for performance. I sent requests in 100 threads (each thread with 250ms delay) and it gave me an average response of 250ms (which is ok for me).
So I proxied the getUserClaimValuesForClaims operation using the Transformation proxy and it also worked. But during the performace testing (with the same or smaller load) I have got following errors and not all the messages have returned (on average 1 of 100 messages).
[2014-01-06 19:28:21,047] INFO - LogMediator To: http://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:d2b85c03-beaf-409b-bf39-2e8143bd9e0b, Direction: response
[2014-01-06 19:28:38,441] ERROR - SourceHandler I/O error: An established connection was aborted by the software in your host machine
java.io.IOException: An established connection was aborted by the software in your host machine
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:25)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:202)
at sun.nio.ch.IOUtil.read(IOUtil.java:175)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243)
at org.apache.http.nio.reactor.ssl.SSLIOSession.receiveEncryptedData(SSLIOSession.java:348)
at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:376)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady
[2014-01-06 19:29:14,051] WARN - TargetHandler http-outgoing-66: Connection time out while in state: REQUEST_DONE
[2014-01-06 19:29:14,061] WARN - EndpointContext Endpoint : AnonymousEndpoint will be marked SUSPENDED as it failed
[2014-01-06 19:29:15,041] WARN - EndpointContext Suspending endpoint : AnonymousEndpoint - last suspend duration was : 30000ms and current suspend duration is : 30000ms - Next retry after : Mon
[2014-01-06 19:29:14,451] WARN - SourceHandler Connection time out after request is read: http-incoming-57
I thing that it says, it timeouts on backend; but I get the response logged in faultSeuence.
I noticed that the problem is caused by the XSLT mediator in proxy implementation, because when I removed the xslt mediators the log is empty. I tried xquery and it was the same. Even the log mediator in outSequence causes this issues.
Here is my final Proxy.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="GetUserProxy" transports="https http" startOnLoad="true" trace="disable">
<target>
<endpoint>
<address uri="https://localhost:9443/services/RemoteUserStoreManagerService.RemoteUserStoreManagerServiceHttpsSoap12Endpoint/"/>
</endpoint>
<inSequence/>
<outSequence>
<log/>
<send/>
</outSequence>
<faultSequence/>
</target>
</proxy>
When I remove the log mediator, the WARN messages disapears from log and ALL response messages are returned to SoapUI.
Could anybody tell me why it behaves this way?
Thank you very much.
David
RemoteUserStoreManagerService is an admin service. Therefore, you need to send Identity Server's user name/password in a basic authentication header or session cookie. I can not see setting of authorization headers in your proxy configuration such as following
<property expression="fn:concat('Basic ', base64Encode('username:password'))" name="Authorization" scope="transport"></property>
More details would be here. Also you need to create a message body to send to the "RemoteUserStoreManagerService" and read the response messages. These must be implemented using ESB configuration. However, you can write an custom ESB mediator to do this also. Sample java code to call this service can be found here. You can find the detail on writing custom mediator from here
We have figured out, that the problem is probably more general and doesn't relate only to WSO2 IS, because we have the same problem when similating the WSO2 IS service as SoapUI mock.
The problem is probably in HTTP-passthru transport in axis configuration, because when we switched to HTTP-NIO the problem disapears and it is even faster.
There is a post about performance tunnig that helped us.
Howevever, I would like to know what transport we should use in production or where the problem is. Because now it seems that we cannot use HTTP-passthru transport for production environment.
Is anyone experiencing this problem?
what are the versions of ESB and IS your working with..?
try to restart the ESB and call the Service again. Do remember call to Identity server is a secure call.
I am using WSO2 ESB 4.0.3 with Java 6 on MAC OSX 10.7.4. I have also installed Data Services Features.
In the typical scenario I have one proxy service which the client calls and I pass one the request to Data Service. Now if I have FAULT message from the Data service back to proxy how do I check in proxy service whether the response from Data Service is Fault or normal valid response?
Currently I am using following filter mediator logic in outsequence of proxy service
<filter xpath="get-property('FAULT')">
<then>
<log category="ERROR" level="custom" separator=",">
<property name="OWCHECK-faultMessage" value="TQS_OWCHECK - Received Fault From OWCHECK Data Service !!!!"/>
<property expression="$body" name="Fault-I-Got-Is"/>
<property name="OWCHECK-Forwading-Error" value="TQS_OWCHECK - Forwarding the Fault to Error Handler !!!!"/>
</log>
<else>
<xslt key="conf:/tqs/owcheck/proxy-output-transform.xslt"/>
<log category="INFO" level="custom" separator=",">
<property name="ValidResponse" value="TQS_OWCHECK - Sending Valid Response Back."/>
</log>
<send/>
</else>
</filter>
But this logic of checking the "FAULT" property works when axis2 has NIO senders & receivers in axis2.xml.
How ever if I switch the receivers & senders in axis2.xml from NIO to standard servlet one's (org.wso2.carbon.core.transports.http.HttpTransportListener / org.wso2.carbon.core.transports.http.HttpTransportListener) I do not get the "FAULT" property set and my error handling does not work.
Is there a standard way of checking if the response from one proxy service to another or response from data service to proxy is FAULT or not? I am looking for something which is independent of transport senders and receivers, at least HTTP ones.
Please help.
thanks
Abhijit
It is not good documentation I believe where none of the samples talk about how to handle the faults from Data service to Proxy service or am I missing something?
I would be thankful if I know the best practices to handle the errors from Data service as well as from one proxy to another proxy service.
Please help. This is big project at very prestigious company.
I think you need to check whether the soap body element has a soap:fault message or not. you can use the filter mediator with some xpath expression to check that.