Related
We have a requirement where we need to feed data into an empty json array element from another array object using element comparison.
The sample payloads and required results are mentioned below for better understanding.
Payload1: (Input Payload)
{
"data": [
{
"name":"ram",
"eno":"100",
"dept":"Sales",
"sal":null
},
{
"name":"gopal",
"eno":"101",
"dept":"Sales",
"sal":null
},
{
"name":"hari",
"eno":"102",
"dept":"Sales",
"sal":null
},
{
"name":"pankaj",
"eno":"103",
"dept":"Sales",
"sal":null
},
{
"name":"raju",
"eno":"104",
"dept":"Sales",
"sal":null
}
]
}
Payload2: (Response From a third party webservice)
{
"data": [
{
"eno": "100",
"sal": 2000
},
{
"eno": "101",
"sal": 2300
},
{
"eno": "102",
"sal": 1800
},
{
"eno": "104",
"sal": 2500
}
]
}
Required Result:
{
"data": [
{
"name":"ram",
"eno":"100",
"dept":"Sales",
"sal":2000
},
{
"name":"gopal",
"eno":"101",
"dept":"Sales",
"sal":2300
},
{
"name":"hari",
"eno":"102",
"dept":"Sales",
"sal":1800
},
{
"name":"raju",
"eno":"104",
"dept":"Sales",
"sal":2500
}
]
}
.................................................................
Here is how you can do this by only using synapse. I hardcoded the second response.
<?xml version="1.0" encoding="UTF-8"?>
<api context="/HelloWorld" name="HelloWorld" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="POST">
<inSequence>
<enrich>
<source clone="true" type="body"/>
<target property="inputBody" type="property"/>
</enrich>
<payloadFactory media-type="json">
<format>
{
"data": [
{
"eno": "100",
"sal": 2000
},
{
"eno": "101",
"sal": 2300
},
{
"eno": "102",
"sal": 1800
},
{
"eno": "104",
"sal": 2500
}
]
}
</format>
<args/>
</payloadFactory>
<foreach expression="json-eval($.data)" id="foreach_1">
<sequence>
<property expression="json-eval($.eno)" name="eno" scope="default" type="STRING"/>
<property expression="json-eval($.sal)" name="sal" scope="default" type="STRING"/>
<enrich>
<source clone="true" property="inputBody" type="property"/>
<target type="body"/>
</enrich>
<enrich>
<source clone="true" property="sal" type="property"/>
<target xpath="//data[eno=$ctx:eno]/sal"/>
</enrich>
<enrich>
<source clone="true" type="body"/>
<target property="inputBody" type="property"/>
</enrich>
</sequence>
</foreach>
<enrich>
<source clone="true" property="inputBody" type="property"/>
<target type="body"/>
</enrich>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<respond/>
</inSequence>
<outSequence/>
<faultSequence/>
</resource>
</api>
I'm trying to validate message with json schema in WSO2 micro integrator 1.2.0.
<validate cache-schema="true">
<schema key="conf:schema/eip_dit_oko_jsonschema_stage_0_input_params.json"/>
<on-fail>
<payloadFactory media-type="json">
<format>{"Error":"$1","Error Details":"$2"}</format>
<args>
<arg evaluator="xml" expression="$ctx:ERROR_MESSAGE"/>
<arg evaluator="xml" expression="$ctx:ERROR_DETAIL"/>
</args>
</payloadFactory>
<property name="HTTP_SC" scope="axis2" type="STRING" value="500"/>
<respond/>
</on-fail>
</validate>
If the schema file is in registry
<item>
<file>eip_dit_oko_jsonschema_stage_0_input_params.json</file>
<path>/_system/config/schema</path>
<mediaType>application/json</mediaType>
<properties/>
</item>
then sequence fails with
[2022-11-22 21:14:33,492] ERROR {ValidateMediator} - {api:eip_dit_oko_api_stage_0} Error creating a new schema objects for schemas : [Value {name ='null', keyValue ='conf:schema/eip_dit_oko_jsonschema_stage_0_input_params.json'}] org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at com.sun.org.apache.xerces.internal.jaxp.validation.Util.toSAXParseException(Util.java:74)
at com.sun.org.apache.xerces.internal.jaxp.validation.Util.toSAXException(Util.java:62)
at com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory.newSchema(XMLSchemaFactory.java:258)
at org.apache.synapse.mediators.builtin.ValidateMediator.mediate(ValidateMediator.java:429)
...
Obviously, integrator tries to read json-schema file as xml.
If I try to follow this answer about using local entry instead of registry
<validate cache-schema="true">
<schema key="eip_dit_oko_jsonschema_stage_0_input_params"/>
...
</validate>
<?xml version="1.0" encoding="UTF-8"?>
<localEntry key="eip_dit_oko_jsonschema_stage_0_input_params" xmlns="http://ws.apache.org/ns/synapse"><![CDATA[{ "$schema": "http://json-schema.org/draft-04/schema", "id": "http://example.com/example.json", "type": "object", "title": "The root schema", "required": [ "getData" ], "properties": { "getData": { "id": "#getData", "type": "object", "title": "The getData schema", "required": [ "p_limit", "p_offset" ], "properties": { "p_limit": { "id": "#p_limit", "type": "integer", "title": "The p_limit schema" }, "p_offset": { "id": "#p_offset", "type": "integer", "title": "The p_offset schema" }, "p_defect_text": { "id": "#/properties/defect_text", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_district_code": { "id": "#p_district_code", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "integer" } }, "p_okrug_code": { "id": "#p_okrug_code", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "integer" } }, "p_status": { "id": "#p_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_sys_status": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_ticket": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_season": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_critical": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_owner_name": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_address": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_address_like": { "id": "#p_address_like", "type": "string" },"p_id_object": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "integer" } }, "p_id_300": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_type_object": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_id_systems": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_defect_el1": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_defect_el": { "id": "#p_sys_status", "type": "array", "items": {"id": "#/properties/defect_text/items","type": "string" } }, "p_sys_sla": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_sys_sla_from": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_sys_sla_to": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_data_creation_from": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_data_creation_to": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_view_date_from_from": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_view_date_to_to": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_view_date_from_to": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_view_date_to_from": { "id": "#p_sys_status", "type": "string", "format":"date-time", "title": "The p_sys_status schema" }, "p_deadline": { "id": "#p_sys_status", "type": "number", "title": "The p_sys_status schema" } } } }}]]></localEntry>
then mediation fails as this
[2022-11-22 16:13:17,517] WARN {SynapseConfigUtils} - Cannot convert object to a StreamSource
EDIT
Request:
curl -v http://localhost:8290/api/stage -H 'Content-Type: application/json' -d '{"getData": {"p_season": ["winter"], "p_limit": 10, "p_offset": 0}}'
One reason for the issue is that the Payload you are sending is not a JSON or you are not sending the Content-Type: application/json header. But I would assume if you are sending the incorrect content type it would fail before reaching the validate medaitor. So my guess is that you are not sending any Payload with the request at all. Are you trying to test this with a GET request? As per the code if you don't send a JSON payload it will go down the XML path which can cause the issue you are facing, hence make sure you send a valid JSON payload with your request.
I want to sent a request to my api like this:
{
"id": "22",
"code": "AXB23, ADR40, SMRS2"
}
And I want to use every code in a query like "SELECT ... WHERE ... IN(AXB23, ADR40, SMRS2)" in my datasource but I don't know how.
I found here a solution with <param type="IN" name="code" paramType="ARRAY" sqlType="STRING"/>
but it doesn't work.
Any suggestions?
EDIT*:
This is my api (inSequence - forget about the id from request in the original post, I just only have that 'code')
<inSequence>
<property expression="json-eval($..code)" name="code" scope="default" type="STRING"/>
<payloadFactory media-type="xml">
<format>
<selectTEST xmlns:xs="http://ws.wso2.org/dataservice/selectTEST">
<code>$1</code>
</selectTEST>
</format>
<args>
<arg evaluator="xml" expression="$ctx:code"/>
</args>
</payloadFactory>
<property name="messageType" scope="axis2" type="STRING" value="text/xml"/>
<header name="Action" scope="default" value="selectTEST"/>
<call>
<endpoint key="ep.select"/>
</call>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<respond/>
</inSequence>
and this is my dataservice:
<operation name="selectTEST" returnRequestStatus="true">
<call-query href="selectTEST">
<with-param name="code" query-param="code"/>
</call-query>
</operation>
<query id="selectTEST" useConfig="default">
<sql><![CDATA[
select x, y, z, w from mydatabase where code in (:code)
]]></sql>
<param type="IN" name="code" paramType="ARRAY" sqlType="STRING"/>
<result outputType="json">
{
"data": {
"records": [
{
"x": "$x",
"y": "$y",
"z": "$z",
"w": "$w"
}
]
}
}
</result>
</query>
My request:
{
"code": ["AXB23", "ADR40"]
}
If you are using the array type parameters you should pass the values as a proper JSON array. Something like this will work for you.
The Request:
{
"_poststudent": {
"code": ["AXB23","ADR40"]
}
}
Dataservice Query
<query id="CreateStudents" useConfig="default">
<sql>SELECT id, name, school, grade FROM students WHERE grade IN (:code)</sql>
<param name="code" paramType="ARRAY" sqlType="STRING" type="IN" optional="false" />
<result outputType="json">{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school", "grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"}]}}{"students":{"student":[{"name":"$name","school":"$school", "grade":"$grade"} ] }}{"students":{"student":[{"name":"$name","school":"$school","grade":"$grade"} ] } }</result>
</query>
Update
The Full Dataservice config.
<data name="RESTDataService" serviceNamespace="http://ws.wso2.org/dataservice/samples/json_sample" transports="http https">
<description>Exposing the data service as a REST service.</description>
<config id="default">
<property name="driverClassName">com.mysql.jdbc.Driver</property>
<property name="url">jdbc:mysql://localhost:3306/school_db</property>
<property name="org.wso2.ws.dataservice.user">xxxxx</property>
<property name="org.wso2.ws.dataservice.password">xxxxx</property>
</config>
<resource method="POST" path="student">
<call-query href="ReadStudents">
<with-param name="code" query-param="code" />
</call-query>
</resource>
<operation name="selectTEST" returnRequestStatus="true">
<call-query href="ReadStudents">
<with-param name="code" query-param="code" />
</call-query>
</operation>
<query id="ReadStudents" useConfig="default">
<sql>SELECT id, name, school, grade FROM students WHERE grade IN (:code)</sql>
<param name="code" paramType="ARRAY" sqlType="STRING" type="IN" optional="false" />
<result outputType="json">{ "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } { "students":{ "student":[ { "name":"$name", "school":"$school", "grade":"$grade" } ] } } </result>
</query>
</data>
Note that I have specified a <resource method="POST" path="student">. resources are for RESTfull invocations, and <operation name="selectTEST" returnRequestStatus="true"> are for SOAP calls. I have created both RestFul and SOAP services for your reference. Following is how you can invoke them.
Note: Here I'm directly calling the Dataservice.
RestFull Invocation.
curl --location --request POST 'http://localhost:8290/services/RESTDataService/student' \
--header 'Content-Type: application/json' \
--data-raw '{
"students": {
"code": ["AXB23", "ADR40"]
}
}'
SOAP Service Call
curl --location --request POST 'http://localhost:8290/services/RESTDataService/selectTEST' \
--header 'Action: selectTEST' \
--header 'Content-Type: text/xml' \
--header 'Accept: application/json' \
--data-raw '<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope/">
<soap:Body>
<code>AXB23</code>
<code>ADR40</code>
</soap:Body>
</soap:Envelope>'
Note: In a XML message an Array is depicted as repeating elements as shown above.
I was using steeltoe 2.4.3 in my .net core 3.1 application. Recently I updated steeltoe packages to v3.0.1 and it looks like logs are not coming in serilog format. I am not able to figure out what is wrong. I do have the serilog configuration in my appsettings like below.
{
"management": {
"endpoints": {
"path": "/myexample/cloudfoundryapplication",
"cloudfoundry": {
"validateCertificates": false
},
"actuator": {
"exposure": {
"include": [ "*" ],
"exclude": [ "env", "refresh" ]
}
}
}
},
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Information",
"Steeltoe": "Information"
}
},
"WriteTo": [
{
"Name": "Console",
"Args": {
"formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
}
},
{
"Name": "Trace",
"Args": {
"formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
}
}
],
"Enrich": [ "FromLogContext" ]
},
"AllowedHosts": "*"
}
Program.cs is like below
public static class Program
{
public static void Main(string[] args)
{
Activity.DefaultIdFormat = ActivityIdFormat.W3C;
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.AddCloudFoundryConfiguration()
.ConfigureLogging((builderContext, loggingBuilder) =>
{
// Add Serilog Dynamic Logger
loggingBuilder.AddDynamicSerilog();
})
.AddCloudFoundryActuators()
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
My .csproj looks iike below.
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>
<ItemGroup>
<None Update="wwwroot\**\*;*.yml">
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</None>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Serilog.Sinks.Trace" Version="2.1.0" />
<PackageReference Include="Serilog.Formatting.Compact" Version="1.1.0" />
<PackageReference Include="Steeltoe.Extensions.Logging.DynamicSerilogCore" Version="3.0.1" />
<PackageReference Include="Steeltoe.Management.CloudFoundryCore" Version="3.0.1" />
<PackageReference Include="Steeltoe.Extensions.Configuration.CloudFoundryCore" Version="3.0.1" />
</ItemGroup>
</Project>
AddDynamicSerilog doesn't (currently) add a console sink, try updating your code to look like this:
AddDynamicSerilog(new LoggerConfiguration().WriteTo.Console())
This PR will result in the Console sink being added automatically if no configuration is provided, and it will probably ship in Steeltoe 3.0.2 (no ETA yet)
I am very new in WSO2 API Manager and I have the following problem registering some API (implemented using WSO2 ESB component) on an old WSO2 API Manager version 1.9.0).
The problem is the following one:
I register an API on the API Manager and in the try it tool it generate and perform a request like this:
curl -X GET --header "Accept: application/json" --header "Authorization: Bearer c86b19dac9dfd7406ebe9013373c9de9" "https://api.MY-COMPANY.org/api/dsa2/v1.0.0/market/12?lang=1"
This request is sent to the endpoint representing the ESB API implementation (I see it in the log). This ESB implementation contains a script mediator that do some works on a JSON object that the API will return to the user.
I am attaching the code fo my API for clarity:
<?xml version="1.0" encoding="UTF-8"?>
<api context="/market" name="market_details" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="GET" protocol="http" uri-template="/{marketId}?lang={lang_id}">
<inSequence>
<log level="full"/>
<property expression="get-property('uri.var.marketId')" name="marketId" scope="default" type="STRING"/>
<property expression="get-property('uri.var.lang_id')" name="lang_id" scope="default" type="STRING"/>
<log level="custom">
<property expression="$ctx:marketId" name="Market ID"/>
<property expression="$ctx:lang_id" name="Lang ID"/>
</log>
<property name="messageType" scope="axis2" type="STRING" value="application/xml"/>
<payloadFactory media-type="xml">
<format>
<ds:GetMarketDetails xmlns:ds="http://ws.wso2.org/dataservice">
<ds:market_id>$1</ds:market_id>
<ds:language_id>$2</ds:language_id>
</ds:GetMarketDetails>
</format>
<args>
<arg evaluator="xml" expression="$ctx:marketId"/>
<arg evaluator="xml" expression="$ctx:lang_id"/>
</args>
</payloadFactory>
<header name="Action" scope="default" value="urn:GetMarketDetails"/>
<call>
<endpoint key="agrimarketprice_Endpoint"/>
</call>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<property expression="json-eval($.)" name="JSONPayload" scope="default" type="STRING"/>
<script language="js"><![CDATA[var log = mc.getServiceLog();
// Problem is: null values returned from DSS as #nil=true and converted as Object into JSON.
// See https://wso2.org/jira/browse/ESBJAVA-4467 as example of bug, reported into JIRA
// So, we need convert values, what may be nulls with using this function
function checkForNull(value) {
if (value instanceof Object && "#nil" in value) {
return null;
}
return value;
}
// stange workaround for getting JSON Payload. getPayloadJSON returned null.
var pl_string = mc.getProperty("JSONPayload");
log.info("PAYLOAD STRING: " + pl_string);
var payload = JSON.parse(pl_string);
// create new response
var response = payload.Markets.Market;
//log.info("RESPONSE: " + JSON.stringify(response));
//response.id = mc.getProperty("MarketId");
response.id = mc.getProperty("marketId");
// convert null values
response.id = checkForNull(response.id);
response.regione = checkForNull(response.regione);
response.province = checkForNull(response.province);
response.city = checkForNull(response.city);
response.district = checkForNull(response.district);
response.town = checkForNull(response.town);
response.village = checkForNull(response.village);
if(response.commodities && response.commodities.commoditiesList) {
// convert array of commodities into required HATEOS format
var commodity = new Array();
for (i = 0; i < response.commodities.commoditiesList.length; ++i) {
var el = response.commodities.commoditiesList[i];
var newEl = new Object();
newEl.commodity_details_id = checkForNull(el.commodity_details_id);
newEl.commodity_name_en = checkForNull(el.commodity_name_en);
newEl.commodity_name = checkForNull(el.commodity_name);
newEl.description = checkForNull(el.description);
newEl.image_link = checkForNull(el.image_link);
newEl.market_commodity_details_id = checkForNull(el.market_commodity_details_id);
newEl.price_series_id = checkForNull(el.price_series_id);
newEl.last_price_date = checkForNull(el.last_price_date);
newEl.last_avg_price = checkForNull(el.last_avg_price);
newEl.currency = checkForNull(el.currency);
newEl.measure_unit = checkForNull(el.measure_unit);
commodityDetailsLinks = [];
commodityDetailsRefObj = {};
commodityDetailsRefObj.rel = "commodity_details";
commodityDetailsRefObj.href = "http://5.249.148.180:8280/commodity_details/" + checkForNull(el.commodity_details_id);
commodityDetailsRefObj.type = "GET";
commodityDetailsLinks.push(commodityDetailsRefObj);
newEl.links = commodityDetailsLinks;
commodity.push(newEl);
}
response.commodities.commoditiesList = commodity;
}
log.info("PROCESSED RESPONSE:");
//var jsonStr = JSON.stringify(response, null, 2); // spacing level = 2
//log.info(jsonStr);
selfLinks = [];
selfsRefObj = {};
selfsRefObj.rel = "self";
selfsRefObj.href = "http://5.249.148.180:8280/market/" + checkForNull(response.id);
selfsRefObj.type = "GET";
selfLinks.push(selfsRefObj);
response.links = selfLinks;
delete response.id;
var jsonStr = JSON.stringify(response, null, 2); // spacing level = 2
log.info("RESPONSE PROCESSED:");
log.info(jsonStr);
// put payload back
mc.setPayloadJSON(response);
log.info("This is the end");]]></script>
<property name="RESPONSE" scope="default" type="STRING" value="true"/>
<header action="remove" name="To" scope="default"/>
<send/>
</inSequence>
<outSequence>
<send/>
</outSequence>
<faultSequence/>
</resource>
</api>
In the JavaScript mediator happens something very strange.
Calling my API as done by the API Manager try it tool (IMPORTANT: the problem seems to be the --header "Accept: application/json" header):
curl -X GET --header "Accept: application/json" --header "Authorization: Bearer c86b19dac9dfd7406ebe9013373c9de9" "https://api.MY-COMPANY.org/api/dsa2/v1.0.0/market/12?lang=1"
I obtain this wrong JSON as value of the payload value (the value of my pl_string):
{
"Markets": {
"Market": [{
"market_id": 12,
"market_name": "Kouthiaba",
"market_description": "Kouthiaba Market",
"localization_id": "2",
"long": null,
"lat": null,
"country": "Senegal",
"regione": null,
"province": null,
"city": null,
"district": null,
"town": null,
"village": null,
"commodities": {
"commoditiesList": [{
"commodity_details_id": 43,
"commodity_name_en": "Millet",
"commodity_name": "Millet",
"description": "Millet",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fmillet.png?alt=media&token=ff9c67ab-e07a-4097-95ae-826fe1aa49ed",
"market_commodity_details_id": 178,
"price_series_id": 2494,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "150.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 13,
"commodity_name_en": "Sorghum",
"commodity_name": "Sorghum",
"description": "Sorghum",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fsorghum.png?alt=media&token=ae33f8e8-50c4-4b8e-868f-1997d50d7ad4",
"market_commodity_details_id": 179,
"price_series_id": 2495,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "160.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 11,
"commodity_name_en": "Maize",
"commodity_name": "Maize",
"description": "Maize is a staple food in many parts of the world, with a total production of 1040M tonnes. However, not all of the maize produced is for human consumption but it is also utilised in bio fuel production. Some of the maize produced is used for corn ethanol, animal feed and other maize products such as corn-starch and corn syrup. Maize for human consumption is used in five different forms 1) Popcorn 2) Flint corn 3) Dent corn 4) Floury corn and 5) Sweet corn.",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fmaize.png?alt=media&token=34d5a149-1721-47e9-863b-c939c7fd7419",
"market_commodity_details_id": 180,
"price_series_id": 2496,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "125.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 38,
"commodity_name_en": "Rice Ordinary",
"commodity_name": "Rice Ordinary",
"description": "Rice Ordinary",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Friz.png?alt=media&token=c35e7648-1793-423b-acd2-52d8a1e58c53",
"market_commodity_details_id": 181,
"price_series_id": 2497,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "285.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 39,
"commodity_name_en": "Rice Perfumed",
"commodity_name": "Rice Perfumed",
"description": "Rice Perfumed",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Friz.png?alt=media&token=c35e7648-1793-423b-acd2-52d8a1e58c53",
"market_commodity_details_id": 182,
"price_series_id": 2498,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "450.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 40,
"commodity_name_en": "Black Eyed Pea",
"commodity_name": "Black Eyed Pea",
"description": "Black Eyed Pea",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fblack_eyed_pea.png?alt=media&token=ab397785-68da-413a-978b-e0ebab8407b4",
"market_commodity_details_id": 183,
"price_series_id": 2499,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "325.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 42,
"commodity_name_en": "Peanut with Shell",
"commodity_name": "Peanut with Shell",
"description": "Peanut with Shell",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fpeanut.png?alt=media&token=6d5ded68-3126-44df-b429-89fe32483d2d",
"market_commodity_details_id": 184,
"price_series_id": 2500,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "175.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 41,
"commodity_name_en": "Peanut without Shell",
"commodity_name": "Peanut without Shell",
"description": "Peanut without Shell",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fpeanut_open.png?alt=media&token=d5af0a5c-6327-418e-9fac-19d34fcedaf5",
"market_commodity_details_id": 185,
"price_series_id": 2501,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": "325.0000",
"currency": "XOF",
"measure_unit": "kilogram"
}
]
}
}
]
}
}
This JSON broke my script because I have:
{
"Markets": {
"Market": [
..................
]
}
}
instead (as expected):
{
"Markets": {
"Market": {
................
}
}
}
The strange thing is that if I perform the same call to my API manager (using cURL in my shell) without set the --header "Accept: application/json" header, in this way:
curl -X GET --header "Authorization: Bearer c86b19dac9dfd7406ebe9013373c9de9" "https://api.MY-COMPANY.org/api/dsa/v1.0.0/market/12?lang=1"
I am obtaining the correct JSON and my script is not broken, infact my JSON payload (the value of my pl_string) now is:
{
"Markets": {
"Market": {
"market_id": 12,
"market_name": "Kouthiaba",
"market_description": "Kouthiaba Market",
"localization_id": 2,
"long": {
"#nil": "true"
},
"lat": {
"#nil": "true"
},
"country": "Senegal",
"regione": {
"#nil": "true"
},
"province": {
"#nil": "true"
},
"city": {
"#nil": "true"
},
"district": {
"#nil": "true"
},
"town": {
"#nil": "true"
},
"village": {
"#nil": "true"
},
"commodities": {
"commoditiesList": [{
"commodity_details_id": 43,
"commodity_name_en": "Millet",
"commodity_name": "Millet",
"description": "Millet",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fmillet.png?alt=media&token=ff9c67ab-e07a-4097-95ae-826fe1aa49ed",
"market_commodity_details_id": 178,
"price_series_id": 2494,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 150.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 13,
"commodity_name_en": "Sorghum",
"commodity_name": "Sorghum",
"description": "Sorghum",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fsorghum.png?alt=media&token=ae33f8e8-50c4-4b8e-868f-1997d50d7ad4",
"market_commodity_details_id": 179,
"price_series_id": 2495,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 160.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 11,
"commodity_name_en": "Maize",
"commodity_name": "Maize",
"description": "Maize is a staple food in many parts of the world, with a total production of 1040M tonnes. However, not all of the maize produced is for human consumption but it is also utilised in bio fuel production. Some of the maize produced is used for corn ethanol, animal feed and other maize products such as corn-starch and corn syrup. Maize for human consumption is used in five different forms 1) Popcorn 2) Flint corn 3) Dent corn 4) Floury corn and 5) Sweet corn.",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fmaize.png?alt=media&token=34d5a149-1721-47e9-863b-c939c7fd7419",
"market_commodity_details_id": 180,
"price_series_id": 2496,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 125.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 38,
"commodity_name_en": "Rice Ordinary",
"commodity_name": "Rice Ordinary",
"description": "Rice Ordinary",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Friz.png?alt=media&token=c35e7648-1793-423b-acd2-52d8a1e58c53",
"market_commodity_details_id": 181,
"price_series_id": 2497,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 285.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 39,
"commodity_name_en": "Rice Perfumed",
"commodity_name": "Rice Perfumed",
"description": "Rice Perfumed",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Friz.png?alt=media&token=c35e7648-1793-423b-acd2-52d8a1e58c53",
"market_commodity_details_id": 182,
"price_series_id": 2498,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 450.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 40,
"commodity_name_en": "Black Eyed Pea",
"commodity_name": "Black Eyed Pea",
"description": "Black Eyed Pea",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fblack_eyed_pea.png?alt=media&token=ab397785-68da-413a-978b-e0ebab8407b4",
"market_commodity_details_id": 183,
"price_series_id": 2499,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 325.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 42,
"commodity_name_en": "Peanut with Shell",
"commodity_name": "Peanut with Shell",
"description": "Peanut with Shell",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fpeanut.png?alt=media&token=6d5ded68-3126-44df-b429-89fe32483d2d",
"market_commodity_details_id": 184,
"price_series_id": 2500,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 175.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}, {
"commodity_details_id": 41,
"commodity_name_en": "Peanut without Shell",
"commodity_name": "Peanut without Shell",
"description": "Peanut without Shell",
"image_link": "https://firebasestorage.googleapis.com/v0/b/my-company-digital-services-portfolio.appspot.com/o/img%2Ficons%2Fagrimarket%2Fcommodity%2Fpeanut_open.png?alt=media&token=d5af0a5c-6327-418e-9fac-19d34fcedaf5",
"market_commodity_details_id": 185,
"price_series_id": 2501,
"last_price_date": "2018-03-18+01:00",
"last_avg_price": 325.0000,
"currency": "XOF",
"measure_unit": "kilogram"
}
]
}
}
}
}
and this not broke my script.
So, my questions are:
1) Why setting this header to my call my JSON (that is obtained by the result of a DSS query call converted in JSON format) is different?
2) Can I say tO WSO2 API MANAGER to avoid so set this ****--header "Accept: application/json"** header in the call?
Practically the best thing to do is that my API manager perform a call like this:
curl -X GET --header "Authorization: Bearer c86b19dac9dfd7406ebe9013373c9de9" "https://api.MY-COMPANY.org/api/dsa/v1.0.0/market/12?lang=1"
3) At the moment my API manager is performing this call:
curl -X GET --header "Accept: application/json" --header "Authorization: Bearer c86b19dac9dfd7406ebe9013373c9de9" "https://api.MY-COMPANY.org/api/dsa/v1.0.0/market/12?lang=1"
I think that this --header "Accept: application/json" header is propagated to the call to the ESB implementation of my API (correct me if I am doing wrong assertion).
In case can I remove this header into my API definition at the beginning of my API flow? Could be an idea?
the problem seems to be the --header "Accept: application/json" header
I assume you are trying the APIs out from the API Store UI, where the json response type is default. You can change it default value (or set list of supported values) in the the API Publusher's definition filling the response types for each resource
1) Why setting this header to my call my JSON (that is obtained by the result of a DSS query call converted in JSON format) is different?
I believe DSS uses different library (or library version) to convert XML to JSON. According to my experience try to avoid this conversion if possible (as XML doesn't explicitly contain value type information)
you can even disable this automatic conversion somewhere in axis2. xml (for API, DSS and ESB)
2) Can I say tO WSO2 API MANAGER to avoid so set this ****--header "Accept: application/json"** header in the call? Practically the best thing to do is that my API manager
as already mentioned you can set the default or supported values in the Publisher. The set response types are used only by the API Store, the clients may send anything..
3) At the moment my API manager is performing this call:
... --header "Accept: application/json" header is propagated to the call to the ESB implementation of my API (correct me if I am doing wrong assertion).
all headers (except Authorization) are propagated