WSO2 EI and WSO2 Developer - Setup an ESB Proxy with http basic authentication - wso2

I'm using WSO2 EI 6.3.0 and WSO2 Developer Studio 3.8.0.
I'm working with an ESB Project and a Proxy Service inside it.
The Proxy service is a simple pass through service;
it receives a request from a webapp and forward it to a
Soap Web Service, gets the response from the WS and gives
it back to the web app.
I realized this configuration following this tutorial:
https://youtu.be/3OsuGhEMQgc
The setup is very simple, everything worked fine until I found
a web service whose endpoint has http basic authentication.
I made it work thanks to this article:
https://medium.com/#Jenananthan/wso2-esb-how-to-invoke-basic-auth-protected-endpoint-service-5bcdd8290cb4
Here follows the xml that describes the proxy service:
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="ProxyWithBasicAuth" startOnLoad="true" trace="disable" transports="https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<endpoint name="endpoint_urn_uuid_xxxxxxxx-yyyy-zzzz-1111-234567892345">
<address trace="disable" uri="https://here.goes.the.uri.of.the.ws.endpoint"/>
</endpoint>
<inSequence>
<property value="this_is_the_username" name="username" scope="default" type="STRING"/>
<property value="this_is_the_password" name="password" scope="default" type="STRING"/>
<property expression="fn:concat($ctx:username,':',$ctx:password)" name="credentials" scope="default" type="STRING"/>
<property expression="fn:concat('Basic ', base64Encode($ctx:credentials))" name="Authorization" scope="transport" type="STRING" xmlns:ns="http://org.apache.synapse/xsd"/>
</inSequence>
<outSequence>
<send/>
</outSequence>
<faultSequence/>
</target>
</proxy>
So, this configuration (xml file) is working, but I wrote all
the section that enables basic auth without using any visual or drag and drop element of the gui.
My questions are:
1) is this the right way to setup basic auth on a proxy service?
2) is there any gui (drag and drop) element that allows to do this setup visually?
Note: I do not need to struggle with basic authentication between
"clients" and the ESB Proxy Service; this question is about basic
authentication between ESB Proxy Service and an "external" web service.

What you have done is correct. To invoke a service secured with Basic Auth, we just have to set the Authorization header to the message. You could use Header Mediator[1] or the Property mediator (as you have already used) to do that. There is no special Mediator (or a GUI element in the Tool) specifically designed for setting the Authorization header.
Btw, you could consider storing passwords in the secure-vault[2] rather than having it in plain text in the Synapse configuration.
[1] https://docs.wso2.com/display/EI640/Header+Mediator
[2] https://docs.wso2.com/display/EI640/Working+with+Passwords+in+the+ESB+profile

Related

Is there a way to call a WSO2 proxy service from a sequence?

I am trying to collect data every minute from a REST API and send data to an MQTT Broker.
To realized this I've create a task and some sequences.
I have a sequence that retrieve the API Token from the Restfull API with credentials then call another sequence that call the Restfull API with the token to make a request to the API.
The API then return my the result of the request.
I've manage to create a Proxy Service that send dummy data to an MQTT Broker, but I'm not able to call this Proxy Service after the Rest API return me the data.
How can I call the proxy service after the Token and the data is return to me?
I'm really new to WSO2 so if you know a better way to organize this process please feel free to share your knowledge.
I'm using WSO2 Entreprise Integrator 7.1.0, WSO2 Micro Integrator 1.2.0.
Thanks
The simplest way is to create sequence for send data to MQTT, like below, and call it.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="send.mqtt">
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
<property name="target.endpoint" value="StockReqEndPoint"/>
<send>
<endpoint>
<address uri="mqtt:/SampleProxy?mqtt.server.host.name=localhost&mqtt.server.port=1883&mqtt.client.id=esb.test.sender&mqtt.topic.name=esb.test2&mqtt.subscription.qos=2&mqtt.blocking.sender=true"/>
</endpoint>
</send>
</sequence>

Why I am incurring in these problems performing a lot of HTTP POST request (toward a web service) in a WSO2 ESB application?

I am very new in WSO2 and I have the following problem.
I am developing an ESB application that achieve the following tasks:
Perform some DSS queries on a local database.
Use the retrieved data to build some XML documents.
Send all these documents to a remote web service (implemented in PHP).
Obtain the web service responses (a specific response for all request that contains one of these generated XML document) and use its content to store a value on the previous local databse.
The web service receiving the previous request containin an XML document stores the document content on another database.
I have implemented this ESB application, and it seems to works (but I am finding some problem that I will explain later) using the following logic:
I have created an API containing an in sequence that perform the queries, build the XML documents, send these documents to the web service, something like this (I can't post the entire code because the XML creation logic is pretty big):
<?xml version="1.0" encoding="UTF-8"?>
<api context="/glisTest2" name="glisTest2" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="GET" outSequence="glisOutSequence">
<inSequence>
<payloadFactory media-type="xml">
<format>
<body/>
</format>
<args/>
</payloadFactory>
<header name="Action" scope="default" value="urn:FindNotProcessed"/>
<log level="full"/>
<callout endpointKey="prgfasEndpoint">
<source xmlns:ns="http://org.apache.synapse/xsd" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope" xpath="s11:Body/child::*[fn:position()=1] | s12:Body/child::*[fn:position()=1]"/>
<target xmlns:ns="http://org.apache.synapse/xsd" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope" xpath="s11:Body/child::*[fn:position()=1] | s12:Body/child::*[fn:position()=1]"/>
</callout>
<log level="full"/>
<property expression="count(//ds:Sample)" name="total_samples" scope="default" type="STRING" xmlns:ds="http://ws.wso2.org/dataservice"/>
<log level="custom">
<property expression="$ctx:total_samples" name="total samples: "/>
</log>
<!-- Iterate throug samples -->
<iterate expression="$body//ds:Sample" id="ITR_AGG" sequential="true" xmlns:ds="http://ws.wso2.org/dataservice">
<target sequence="sampleDataSequence"/>
<!-- This is my main sequence. It will call some other sequece in cascade: -->
</iterate>
</inSequence>
<faultSequence/>
</resource>
</api>
The callout mediator is used to obtain to call the DSS service that performs query to obtain data.
Now in this section of code:
<!-- Iterate throug samples -->
<iterate expression="$body//ds:Sample" id="ITR_AGG" xmlns:ds="http://ws.wso2.org/dataservice">
<target sequence="sampleDataSequence"/>
</iterate>
I am iterating on each retrieved element and for each element I will basically build an XML that will be send to the remote web service. All this logic (pretty big because the XML documents contains many fields) are contained into the sampleDataSequence sequence (this sequence will be performed at each iteration to build and send an XML document).
I am not attaching the entire code, at the end of the sampleDataSequence sequence I perform the web service call (a POST request) passing to it the current XML document (generated in the current iteration), in this way (previously I put the current XML document in the body request):
<property name="messageType" scope="axis2" type="STRING" value="application/xml"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="post"/>
<send>
<endpoint key="glisEndpoint"/>
</send>
So, for each iteration, I build a brand new XML document and I send to it to my web service (this works, the web service receive it).
So the in sequence end, then there is the out sequence that that receives and collects all the web service responses so then it can parse these response one by one and write a record into a result table on my local database.
I have done it in this way:
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="glisOutSequence" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
<aggregate id="ITR_AGG">
<completeCondition>
<messageCount max="-1" min="-1"/>
</completeCondition>
<onComplete expression="s11:Body/child::*[position()=1] | s12:Body/child::*[position()=1]" xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/" xmlns:s12="http://www.w3.org/2003/05/soap-envelope">
<property name="it_count" scope="operation" type="STRING" value="0"/>
<!-- Iterate over responses. For correct count calculations they should
be sequential -->
<iterate expression="$body//response" id="ITR_RES" sequential="true">
</aggregate>
<log level="custom">
<property expression="$body" name="AT THE END"/>
</log>
<send/>
</sequence>
So basically I am using the 2 Enterprise Integration Pattern provided by WSO2 ESB: Splitter and Aggregator to achieve this task. It seems to me that this should be the standard solution for this type of task (when you have to send n message to an external web service and then collect and operate on the n response from this web service). Is it my architecture the standard one for this type of task?
I am asking it because now I am finding the following problem:
I have deployed this application on a test environment and we are obtaining the following problem.
The test environment have both this WSO2 ESB application and the contacted web service installed on the same laptop computer (but in the production environment these system will be on different computer because there will be many WSO2 ESB application and a single central web service that will receive message from these ESB applications).
The problem is that the WSO2 application generate a lot of XML documents (something like 1000). So it will be perform 1000 consecutive POST requests to the web service.
The web service receives all these requests (containing the 1000 XML documents that have to be inserted into another database) but after a certain number of DB inserton, the DB (Postgree) give a too many clients exception.
So this exception could be related to: connection not closes, result set not closed or something like this.
My personal opinion (it could be wrong) is that it should be a problem related to the back end of the web service and not to my WSO2 ESB application. But the person that are testing it says to me that maybe the WSO2 application doesn't close the connections.
It seems strange to me baceusa it is an error on the DB and not on the web service but I really have no idea.
Another thing is that I fix this problem putting a delay of some milliseconds before send the POST request to the web service, in this way:
<!-- Inserted a short delay to prevent flooding the GLIS server -->
<script language="js">java.lang.Thread.sleep(200);</script>
<property name="messageType" scope="axis2" type="STRING" value="application/xml"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="post"/>
<property name="ClientApiNonBlocking" value="true" scope="axis2" action="remove"/>
<send>
<endpoint key="glisEndpoint"/>
</send
Putting a short delay before send the request it seems to works fine. My idea is that maybe the problem is that in this test environment both the ESB application and the web service application are installed on the same laptop so the comuncation is very very quick (there is not a net in the middle) so the ESB application shot 1000 request very quick, the web service can receive it but it can't write on the database.
Could be the problem?
Another doubt is: who are testing the application says to me that maybe I am using the wrong enterprise integration pattern to achieve this taks and that I should perform the web service call and elaborate the response one by one (for each XML document). In my personal opinion this should be impossible using ESB logic because if I send a single XML to the web service, when it give me a response ESB enter in the out sequence and so the ESB application will end after processing a single document (because I think that I can't come back to the in sequence after that the out sequence is completed). Is it my reasoning correct?
So, at the end: could be a problem of the test environment (both application on the same PC), or a problem of the PHP web service back end application (result set not closed or something like this, it seems strange because using the delay it works) or could be something related to my ESB application?
My personal opinion - I do not see how "not closing connections" from WSO2 side (acts as a client for PHP-service)
may product problems between PHP and database.
I do not see any problems with integration patterns, you used in your solution - everything looks ok.
So, I think problem in PHP web-service or it's specific configuration on local test laptop.
For example java-applications are always used connection-pool to get DB connections -
and settings of this connection-pool can control number of connections to DB.
Not sure - but probably PHP also has some kind of connection-pools.
Also, database usually has many options about possible number of connections at same time.
So - it may be everything is configured by properly in production system, but configured "by default" in test laptop,
so - as result we have this kind of problem.
Anyway - we have situation then called service has some specific SLA - for example number of connections it may accept at same time.
Also, it looks like, it may happens this service may be unavailable or return error (for example because of database connection problem).
your JS delay is really dirty solution - WSO2 ESB (as any other ESB) has much better solution -
called "Guaranteed Delivery".
In WSO2 it may be implemented by followed way: instead of sending message to glisEndpoint, you should put it into
Message Store.
Then you need configure
Message Processor :
it will get messages from message store and send to endpoint.
If calling endpoint will be failed (by some reason) - it will put message back to Message Store.
There are many configuration options in Message Processor - so, you can configure for example to send only 1 message per second, or any other pattern.
Please - keep in mind - there are many types of message stores: starting from simplest In Memory Message store and finishing using ActiveMQ or RabbitMQ.
You should choose proper message store type depending from your requirements.

proxy Services in WSO2

How to develope a web service in WSO2 using proxy services ?
Applying my Oracle Service Bus knowledge here to create a proxy service which would take a string as an Input and return the same as response, I find it difficult in WSO2 to create a proxy service using my XML Schema.
I found that we can do the same in WSO2 using Custom proxy template.
But I am not sure of defining an Input and output in proxy service ;
In a traditional Oracle Service Bus we can do it by defining our XMl Schemas.
Seems like you need some echo kind of a service. This kind of behavior can be achieved by configuring the wso2 esb as in following manner. However this is not really a web service. I hope you are aware of what is web service and what does a proxy service does. But for your requirement you can try something like follows.
<proxy name="loopBackProxy" startonload="true" statistics="disable" trace="disable" transports="https,http" xmlns="http://ws.apache.org/ns/synapse">
<target>
<insequence>
<log level="full"></log>
<header action="remove" name="To"></header>
<property name="RESPONSE" value="true"></property>
<!-- your esb modifications here -->
<send></send>
</insequence>
</target>
<description></description>
</proxy>
This doesnot contains a outsequence, what this does is redirects the input to the client.

Make flow between WSO2 EBS to WSO2 Appserver

I successfully installed WSO2 EBS and WSO2 Application server in same machine and also i resolved the port conflict issue. I installed backend services in app server mentioned in wso2 portal. My question is how to create a sequence between ebs to appserver in ebs sequence option for message flow between ebs - appserver - ebs.
note: Refer
http://wso2.com/library/articles/2011/01/wso2-esb-by-example-service-chaining.
If it is not possible the above condition means please tell how to make a message flow between ebs to app server mentioned in above URL.
Could you please send any document, video, screen shot regarding this issue as soon as possible.
Thank you.
In this service chaining scenario (in the link you mentioned) what happens is that the services are hosted in Application Server (AS). This hosted URL of the service is then defined as an endpoint in ESB. So when the proxy service in ESB receives a request, mediation is performed on the message and then sent to the endpoint that is defined in the send mediator. The response received from this endpoint, is then through the ESB to the consumer.
If you are looking at how to implement a service chaining scenario, this document will provide more detailed information.
http://wso2.com/library/articles/2012/11/enterprise-service-integration-wso2-esb
Hope this helps.....
In this example I had a proxy service that receives an incoming message from a client.
With this incoming message I extract the information I need from it, in this case the offset and limit using the property mediator as you can see in this line:
<property xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" name="offset" expression="//p:listarPersonas/p:offset"/>
I use the log mediator in this sequence because I need to see the values stored in the properties to debug stuff.
After that I create a new message using the payloadFactory because I need to send this message to the Application Server with that format.
<payloadFactory>
<format>
<p:listar_persona_paginado xmlns:p="http://ws.wso2.org/dataservice">
<p:offset>$1</p:offset>
<p:limit>$2</p:limit>
</p:listar_persona_paginado>
</format>
<args>
<arg xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" expression="//p:listarPersonas/p:offset"/>
<arg xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" expression="//p:listarPersonas/p:limit"/>
</args>
</payloadFactory>
Then I send the created message to an endpoint in the Application Server using the send mediator:
<send>
<endpoint>
<address uri="http://127.0.0.1:9765/services/buscarPersona"/>
</endpoint>
</send>
All the pieces together:
<sequence xmlns="http://ws.apache.org/ns/synapse">
<log>
<property name="SECUENCIA" value="ENTRADA DEL LISTADO DE PERSONAS"/>
<property xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" name="offset" expression="//p:listarPersonas/p:offset"/>
<property xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" name="limit" expression="//p:listarPersonas/p:limit"/>
</log>
<payloadFactory>
<format>
<p:listar_persona_paginado xmlns:p="http://ws.wso2.org/dataservice">
<p:offset>$1</p:offset>
<p:limit>$2</p:limit>
</p:listar_persona_paginado>
</format>
<args>
<arg xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" expression="//p:listarPersonas/p:offset"/>
<arg xmlns:ns="http://org.apache.synapse/xsd" xmlns:p="http://ejemplos.cuba.cu/Persona" expression="//p:listarPersonas/p:limit"/>
</args>
</payloadFactory>
<send>
<endpoint>
<address uri="http://127.0.0.1:9765/services/buscarPersona"/>
</endpoint>
</send>
</sequence>
You can define your own sequences within the ESB as much as you need. But when you access these sequences from a client, you need to implement a proxy service within the ESB. That proxy service allows you to call any number of sequences within that proxy service. In your case you need to access the backend services which are located in WSO2 application server. You can access these services from mediator. For a list of examples, please refer the below link
http://docs.wso2.org/wiki/display/ESB460/Proxy+Service+Samples

WSO2 soap12 endpoint to soap11 external service

I create a simple WSDL proxy in WSO2 to allow Soap 1.2 enabled applications to work with Soap 1.1 only external web service. Everything is fine except WSO2 doesn't transform soap11 replies back to soap12 when I send request via Soap12 WSO endpoint.
<proxy xmlns="http://ws.apache.org/ns/synapse" name="MySOAP" transports="http"
statistics="disable" trace="disable" startOnLoad="true">
<target>
<endpoint>
<wsdl service="ExtService" port="ExtPort" uri="https://my.local/wsdl/current
/ExtService.wsdl"/>
<property name="DISABLE_CHUNKING" value="true" scope="axis2"/>
</endpoint>
</target>
<publishWSDL uri="file:///home/sysadmin/MySOAP.wsdl"/>
<description></description>
</proxy>
Has anybody faced anything similar? WSO2 ESB version 4.6.0
You need to add the format as following.
<endpoint>
<address uri="http://localhost:9000/services/SimpleStockQuoteService" format="soap11" />
</endpoint>
Format - The message format for the endpoint. The available values are:
[format="soap11|soap12|pox|get"]
Leave As-Is - No transformation is done to the outgoing message.
SOAP 1.1 - Transforming message to SOAP 1.1.
SOAP 1.2 - Transforming message to SOAP 1.2.
Plain Old XML (POX) - Transforming to plain old XML format
Representational State Transfer (REST) - Transforming to HTTP Get
Request
GET
http://docs.wso2.org/wiki/display/ESB460/Default+Endpoint
This has been an issue for a while now. See jira https://wso2.org/jira/browse/ESBJAVA-1994.
If you don't want to manually transform your response from the backend, uncomment the relevant NHTTP transport entries in axis2.xml and comment out the HTTP PassThrough transport entries.