I have and scenario where I want to extract dynamically Issues from Jira in order to save them on ElasticSearch. I use a custom Proxy Service that connect with Jira (via Jira connector), get a project (first I want to save the project's information) and pass this information to Elastic. Following is the in Sequence of the proxy service:
<?xml version="1.0" encoding="UTF-8"?>
<inSequence xmlns="http://ws.apache.org/ns/synapse">
<property expression="json-eval($.username)" name="username"/>
<property expression="json-eval($.password)" name="password"/>
<property expression="json-eval($.uri)" name="uri"/>
<property expression="json-eval($.id)" name="id"/>
<property expression="json-eval($.uriEl)" name="uriEl"/>
<property expression="json-eval($.indexName)" name="indexName"/>
<property expression="json-eval($.type)" name="type"/>
<property expression="json-eval($.message)" name="message"/>
<jira.init>
<username>{$ctx:username}</username>
<password>{$ctx:password}</password>
<uri>{$ctx:uri}</uri>
</jira.init>
<jira.getProject>
<projectIdOrKey>{$ctx:id}</projectIdOrKey>
</jira.getProject>
<property expression="json-eval($.project)" name="project"
scope="default" type="STRING"/>
<elasticsearch.init>
<apiUrl>{$ctx:uriEl}</apiUrl>
</elasticsearch.init>
<elasticsearch.createDocumentWithIndex>
<indexName>{$ctx:indexName}</indexName>
<type>{$ctx:type}</type>
<message>{$ctx:message}</message>
<body>{$ctx:project}</body>
</elasticsearch.createDocumentWithIndex>
<respond/>
</inSequence>
And here is the sample request:
{
"username":"MyName",
"password":"AnStrongPassword",
"uri":"JiraURL",
"id":"MyProject",
"uriEl":"http://localhost:9200",
"indexName":"jira",
"type":"project",
"message":"Testing"
}
The other parameters (Endpoint, out Sequence and Fault Sequence) are by default.
The problem is that when I try to send this request via 'Try the Test service' option on WSO2 ESB, i get the following error:
Error connecting to the Tryit ajax proxy
I have tested the same code but only using the Jira connection (without get a project or connect to Elastic), and the same error occurs.
Looking to this error, I suppose that the problem is before to read the JSON request, but I don't know if it is related to the endpoint (I think that it is not necessary an endpoint for this scenario).
Thanks in advance.
This problem is totally independent of the jira connector etc. Problem is with the "try it" tool. This tool is not that much recommended for testing purposes. Instead of that, try invoking the proxy service via soap ui.
If you share a screenshot when the try it error occurs, I'll be able to comment on that as well.
After a few days I have found another solution to my problem without using proxy services. To connect ESB with Jira, I created this API on ESB:
<api xmlns="http://ws.apache.org/ns/synapse" name="transport_service" context="/transport_service">
<resource methods="GET">
<inSequence>
<jira.init>
<uri>MyURL</uri>
<username>MyUser</username>
<password>3MyPass</password>
</jira.init>
<jira.searchJira>
<query>project=MyProject</query>
<maxResults>50000</maxResults>
<fields>MyField1,MyField2,etc</fields>
</jira.searchJira>
<respond/>
</inSequence>
<outSequence>
<log level="full"/>
<property name="messageType" value="application/xml" scope="axis2"/>
<send/>
</outSequence>
</resource>
</api>
The command to call this API is simple:
curl -v -X GET http://localhost:8280/transport_service > jira-db.json
Finally, if you want to insert this json on elastic, you can follow the typical PUT via curl:
curl -XPUT 'http://localhost:9200/jira/jira/1' -d #jira-db.json
Additionally, you can write both commands on a daily bash script (using your scheduled task list), getting the same result that scheduling your proxy service on WSO2ESB:
#!/bin/bash
curl -v -X GET http://localhost:8280/transport_service > jira-db.json
curl -XPUT 'http://localhost:9200/jira-dev/jira/1' -d #jira-db.json
echo "Daily load completed"
Related
As you know, by default, WSO2 add a "activityid" Parameter to each response header which would be passed from API-Manager.
Reference:
https://docs.wso2.com/display/EMM210/Getting+Operation+Details+by+the+Activity+ID
For some reason, I need to remove this parameter from the response for client. As I searched, I find out the following link
https://apim.docs.wso2.com/en/4.0.0/deploy-and-publish/deploy-on-gateway/api-gateway/message-mediation/removing-specific-request-headers-from-response/
So I added following code into Main.XML and Fault.XML, However it doesn't work
<property name="activityid" scope="transport" action="remove"/>
I also add the following property to the data flow of a specific API through Integrator Studio. However it doesnt work
<resource methods="POST" uri-template="/attachment">
<inSequence>
<call>
<endpoint key="attachment"/>
</call>
<send/>
</inSequence>
<outSequence>
<property action="remove" name="TRANSPORT_HEADERS" scope="default"/>
<property action="remove" name="activityid" scope="default"/>
</outSequence>
<faultSequence/>
</resource>
Anybody can Give me some advice about this matter??
You can use the message mediation for API Manager. Add the above property mediator to the response path 1 of the API.
Under runtime configurations of the API, there is a message mediation section for both request and response paths. Here you can upload a mediation sequence which will be executed in the request flow as well as response flow. You can save the mediation you need to an xml file and upload the file for the response flow.
1 - https://apim.docs.wso2.com/en/4.0.0/deploy-and-publish/deploy-on-gateway/api-gateway/message-mediation/changing-the-default-mediation-flow-of-api-requests/
In wso2 ESB after calling an endpoint I am getting the response as number(ex: 78) with header application/json, if without processing the response if i send in out sequence it works fine i'll get the same response. But if I include any mediators for processing in between it'll throw exceptions like Could not save JSON payload. Invalid input stream found. A single string or number is not valid in some cases So, it may throwing the exception but this bug is resolved in wso2 EI 6.2.
So now I am able to process the response but if I use script mediator to get that value it shows {}. If i use json-eval($.) then also i am not able to get the value, also with xpath i am not able to get.
So how to get that response(the value in number) for further processing in wso2 ei, by using script mediator or by using json path.
If you are certain that the response only contains a number with the content-type header with application/json. you can take the value to a property as below.
<property name="RESPONSE_NUMBER" expression="//jsonValue" scope="default" type="INTEGER"/>
When you need this value somewhere else in the mediation flow you can take the value from the property(in this case RESPONSE_NUMBER) as below.
$ctx:RESPONSE_NUMBER
Here is a sample API which demonstrates how you can take the response value and use it in the mediation flow.
<api xmlns="http://ws.apache.org/ns/synapse" name="SampleAPI" context="/getNumber">
<resource methods="GET">
<inSequence>
<send>
<endpoint>
<http method="GET" uri-template="http://www.mocky.io/v2/5b02cc2c3000006600cee384"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<property name="RESPONSE_NUMBER" expression="//jsonValue" scope="default" type="INTEGER"/>
<payloadFactory media-type="json">
<format>{"Id": $1}</format>
<args>
<arg evaluator="xml" expression="$ctx:RESPONSE_NUMBER"/>
</args>
</payloadFactory>
<send/>
</outSequence>
</resource>
</api>
You can call the API with below curl command:
curl -v http://localhost:8280/getNumber
When I create proxy with send mediator with Rest service Post HTTP Method in HTTP endpoint url. Selected the endpoint as HTTP endpoint on proxy and post the request xml without soap envelop, this perfectly works and get the response in the response window.
But when I use the call mediator with the same HTTP end point url configuration, this does not works. I would like to know can we use call mediator for Post HTTP method? When I use Call mediator for the GET HTTP method which require only query parameters and does not require any request xml this works absolutely fine.
Here is the further information:
However issue is resloved by using the address endpoint in callmediator. When I Invoke the proxy from external Restt client ot Soap UI, it does works. If I use the Try this Service option in wso2 ESB will fail with the results 1. When Soap12 endpoint is selected and 2 when HTTP end point is selected as shown below.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="postIDMPCall"
transports="https http"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<property name="ContentType" value="text/xml" scope="axis2" type="STRING"/>
<property name="HTTP_METHOD" value="POST" scope="axis2" type="STRING"/>
<payloadFactory media-type="xml">
<format>
<organizationList xmlns="">
<xml content>
</organizationList>
</format>
<args/>
</payloadFactory>
<header name="_user" scope="transport" value="username"/>
<header name="_password" scope="transport" value="Password"/>
<call blocking="true">
<endpoint>
<address uri="http://<ip-address>:<port>/<resource-path>/UpdateOrganization"
format="rest"/>
</endpoint>
</call>
</inSequence>
</target>
</proxy>
Output: When soap12 endpoint is selected
Though posted the correct xml service does not recorgonize the correct xml format for soap12 endpoint.
FAILURE
Record is not processed successfully. Please Provide valid Request XML
When Http end point is selected
[2016-04-21 12:07:50,179] INFO - HTTPSender Unable to sendViaPost to url[http://://UpdateOrganization/mediate]
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
I think we can't use call mediator for this purpose because call mediator is context unaware mediator.
Your call should already perform a post out the box.
Did you try to set format="pox" if you expect simple xml as a response
I've used Smook Mediator in WSO2ESB to transform content in 'plain/text' to 'XML'. I don't know the output is pushed to sequence by default or not. So I tried to send output of Smook to a WS by Send Mediator like this, but nothing's happended. This is my PS's configuration:
<inSequence>
<log/>
<smooks config-key="conf:/smooksccconfig.xml">
<input type="text"/>
<output type="xml"/>
</smooks>
<log level="full"/>
<send receive="Testing">
<endpoint key="GeoEP"/>
</send>
</inSequence>
<outSequence>
<log level="full"/>
<send/>
</outSequence>
Is there any needed mediator to push result of transformation to the Sequence?
Can u give me some advises?
Thank,
Your approach seems fine. You don't need any mediator to push the results of Smooks transformation to a sequence. Anyway to figure out the error you can enable wirelogs in ESB. Please do the following steps.
Shotdown the ESB instance
Locate log4j.properties file under $ESB_HOME/repository/conf directory
Uncomment the line log4j.logger.org.apache.synapse.transport.http.wire=DEBUG
Then restart the ESB instance
symbol << represents contents going out of the ESB where as >> represents things comes into the ESB
Also please check whether your logfull mediator logs the correct output after the transformation. If that is the case, then you have to look into your endpoint config.
If you have any error messages in ESB console, then please attach them here so that we can have some more insight into your issue using them.
I am trying to access external web services (outside of intranet) using WSO2 ESB 4.0.3 but I do not know how to configure the proxy to let the request go outside. I can send SOAP messages over HTTP from my basic Java client app without issues; I could specify teh proxy details using Apache HttpComponents. So, connection details are OK.
I tried:
Editing <transportSender name="http" ... > and specifying parameters like "http.proxyHost" and "http.proxyPort".
Setting up a global parameter <parameter name="Proxy"> ...
Configuring the HTTP Transport Sender and specifying the appropriate parameters.
The best I got was:
ERROR_CODE = 101503, ERROR_MESSAGE = Connection refused or failed for : ...
So, what did I wrong and what should I do instead? Thanks.
Does your proxy server has any authentication? If yes and it uses Basic Auth you need to specify followoing properties before sending out the request..
<property name="Proxy-Authorization" expression="fn:concat('Basic', base64Encode('userName:password'))" scope="transport"/>
<property name="POST_TO_URI" value="true" scope="axis2"/>
The first property sets the Proxy-Authorization HTTP transport header with the base64 encoded user name and password as expected by the HTTP basic authentication.
The second property makes the out-going URL a complete URL understandable by the Proxy Server.
Following is an example proxy config...
<proxy name="StockQuoteProxy" startOnLoad="true">
<target>
<inSequence>
<property name="Proxy-Authorization" expression="fn:concat('Basic ', base64Encode('udayanga:test123'))" scope="transport"/>
<property name="POST_TO_URI" value="true" scope="axis2"/>
<send>
<endpoint>
<address uri="http://www.wso2.com:9000/services/SimpleStockQuoteService"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<publishWSDL uri="file:repository/samples/resources/proxy/sample_proxy_1.wsdl"/>
</proxy>