wso2 esb sequence DB update - wso2

I have a problem with the DBreport mediator which output a syntax error, but I can't see any error in the SQL statement
<sequence xmlns="http://ws.apache.org/ns/synapse" name="_dbcount">
<dblookup>
<connection>
<pool>
<password>1234</password>
<user>root</user>
<url>jdbc:mysql://localhost:3306/new_db</url>
<driver>com.mysql.jdbc.Driver</driver>
</pool>
</connection>
<statement>
<sql>
<![CDATA[ select * from consume where username= ? and id_api+ ?]]></sql>
<parameter value="riccardo" type="VARCHAR" />
<parameter value="1" type="INTEGER" />
<result name="result_use" column="use" />
<result name="result_user" column="username" />
</statement>
</dblookup>
<dbreport>
<connection>
<pool>
<password>1234</password>
<user>root</user>
<url>jdbc:mysql://localhost:3306/new_db</url>
<driver>com.mysql.jdbc.Driver</driver>
<property name="autocommit" value="false" />
</pool>
</connection>
<statement>
<sql>
<![CDATA[UPDATE consume SET use=21 WHERE username='riccardo' AND id_api='1']]></sql>
</statement>
</dbreport>
<log level="custom">
<property xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns3="http://org.apache.synapse/xsd" name="result for client" expression="get-property('result_user')" />
<property xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns3="http://org.apache.synapse/xsd" name="result for use" expression="get-property('result_use')" />
</log>
</sequence>
The first DB lookup works fine but the second SQL statement in the DBreport cannot update the value:
ERROR - DBReportMediator Error execuring insert statem
ent : UPDATE consume SET use='21' WHERE username='riccardo' AND id_api='1' again
st DataSource : jdbc:mysql://localhost:3306/new_db
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in
your SQL syntax; check the manual that corresponds to your MySQL server version
for the right syntax to use near 'use='21' WHERE username='riccardo' AND id_api=
'1'' at line 1
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)...
My tables are as follows:
consume (int id, varchar username, int id_api, int use)
users (int id, varchar username, varchar password)
many thanks
EDIT I think there are major issues with DBREPORT AND DBLOOKUP mediators when writing on a Mysql DB, reading is fine, but Update and Insert just don't work...
tell me if I am wrong

Nuvio,
I think this issue arises not because of something wrong in the DBReport mediator. While going through the reported stack trace and the sequence configuration I can see you have defined "use" as a column name. I'm not quite sure how you created a database column with the term "use" because it is a MySQL keyword and MySQL query compiler should throw a SQLException whenever you have a keyword as a column or any other user defined attribute in your SQL query. Having a non-keyword name for in place of the column "use" would fix your problem.
Cheers,
Prabath

Related

Error while building Passthrough stream First Element must contain the local name, Envelope , but found alerts

Scenario
I've created a proxy service which calls an endpoint. The endpoints gives XML response. I'm using Foreach Mediator to iterate through the XML response using the defined XPath expression. Finally, each record will be inserted to the database using the DB Report Mediator.
Proxy Service
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="alertsIoscoProxy" startOnLoad="true" statistics="enable" trace="enable" transports="http https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<send>
<endpoint>
<address uri="https://www.XXXXXX/investor_protection/investor_alerts/xml-feed">
<suspendOnFailure>
<initialDuration>-1</initialDuration>
<progressionFactor>1</progressionFactor>
</suspendOnFailure>
<markForSuspension>
<retriesBeforeSuspension>0</retriesBeforeSuspension>
</markForSuspension>
</address>
</endpoint>
</send>
</inSequence>
<outSequence>
<foreach expression="//alerts/alert" id="foreach_alert">
<sequence>
<dbreport>
<connection>
<pool>
<driver>org.postgresql.Driver</driver>
<url>jdbc:postgresql://localhost:5432/postgres</url>
<user>postgres</user>
<password>admin</password>
</pool>
</connection>
<statement>
<sql><![CDATA[insert into IOSCO_RESPONSE (ID, DATE_POSTED, COMPANY,REGULATOR, JURISDICTION, LINK, SUBJECT) values (?,?,?,?,?,?,?)]]></sql>
<parameter expression="//id" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//datePosted" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//company" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//regulator" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//jurisdiction" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//link" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
<parameter expression="//subject" type="VARCHAR" xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"/>
</statement>
</dbreport>
</sequence>
</foreach>
<log level="custom">
<property name="add" value="Records added!!!!"/>
</log>
<sequence key="Lookup"/>
<respond/>
</outSequence>
<faultSequence/>
</target>
</proxy>
Sample XML
<alerts>
<message>Results are capped at 500 records.</message>
<alert>
<id>23091</id>
<datePosted>20221209</datePosted>
<company>Trado Banco</company>
<regulator>Financial Conduct Authority</regulator>
<jurisdiction>United Kingdom</jurisdiction>
<link>https://www.fca.org.uk/news/warnings/trado-banco</link>
<subject>Regarding fraudulent or manipulative practices (insider dealing, market manipulation, misrepresentation of material information, etc.)<br /><br />Regarding registration of issuance, offer or sale of securities/derivatives, and reporting requirements<br /><br />Regarding market intermediaries (investment and trading advisers, collective investment schemes, brokers, dealers, and transfer agents)<br /><br />Regarding markets, exchanges, and clearing and settlement entities<br /><br />Miscellaneous</subject>
<comments/>
<attachments/>
</alert>
.
.
.
</alerts>
Error
[2022-12-13 23:42:55,323] INFO {TRACE_LOGGER} - {proxy:alertsIoscoProxy} Proxy Service alertsIoscoProxy received a new message...
[2022-12-13 23:42:55,323] INFO {TRACE_LOGGER} - {proxy:alertsIoscoProxy} Message To: null
[2022-12-13 23:42:55,323] INFO {TRACE_LOGGER} - {proxy:alertsIoscoProxy} SOAPAction: null
[2022-12-13 23:42:55,323] INFO {TRACE_LOGGER} - {proxy:alertsIoscoProxy} WSA-Action: null
[2022-12-13 23:42:55,324] INFO {TRACE_LOGGER} - Setting specified anonymous fault-sequence for proxy
[2022-12-13 23:42:55,324] INFO {TRACE_LOGGER} - {proxy:alertsIoscoProxy} Using the anonymous in-sequence of the proxy service for mediation
[2022-12-13 23:42:55,878] INFO {TRACE_LOGGER} - Setting specified anonymous fault-sequence for proxy
[2022-12-13 23:42:57,175] ERROR {RelayUtils} - Error while building Passthrough stream org.apache.axiom.soap.SOAPProcessingException: First Element must contain the local name, Envelope , but found alerts
at org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.constructNode(StAXSOAPModelBuilder.java:305)
at org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.createOMElement(StAXSOAPModelBuilder.java:252)
at org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.createNextOMElement(StAXSOAPModelBuilder.java:234)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:249)
at org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.getSOAPEnvelope(StAXSOAPModelBuilder.java:204)
at org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.<init>(StAXSOAPModelBuilder.java:154)
at org.apache.axiom.om.impl.AbstractOMMetaFactory.createStAXSOAPModelBuilder(AbstractOMMetaFactory.java:73)
at org.apache.axiom.om.impl.AbstractOMMetaFactory.createSOAPModelBuilder(AbstractOMMetaFactory.java:79)
at org.apache.axiom.om.OMXMLBuilderFactory.createSOAPModelBuilder(OMXMLBuilderFactory.java:196)
at org.apache.axis2.builder.SOAPBuilder.processDocument(SOAPBuilder.java:65)
at org.apache.synapse.transport.passthru.util.DeferredMessageBuilder.getDocument(DeferredMessageBuilder.java:153)
at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:169)
at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:122)
at org.apache.synapse.transport.util.PassThroughMessageHandler.buildMessage(PassThroughMessageHandler.java:103)
at org.apache.synapse.mediators.AbstractListMediator.buildMessage(AbstractListMediator.java:155)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:96)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:72)
at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:158)
at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:377)
at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:627)
at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:208)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:298)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Question
Why proxy Service is considering the response as Soap while its HTTP response with xml based formatting? How could I solve this issue?
One reason for this issue is, your response coming from https://www.XXXXXX/investor_protection/investor_alerts/xml-feed maybe having the Content-Type set to text/xml which is by default considered as a SOAP message and built with the SOAPMessageBuilder. If you can change the backend to send application/xml as the content type this would resolve the issue. Else you will have to change the Message Builder configurations to build text/xml messages with ApplicationXMLBuilder. For that you need to add the following to the deployment.toml.
[[custom_message_builders]]
content_type = "text/xml"
class="org.apache.axis2.builder.ApplicationXMLBuilder"
Note that this will affect all the SOAP 1.1 calls as well, which use text/xml content type. To avoid this you can create a custom message builder as well. More details are on the aforementioned doc.

save a dynamic value in wso2 esb for next use

i need to save result of web service login method as a property in wso2 esb. and use it whenever i want. token will be expired in 20 min so i want to save token in wso2 esb and when it expire regenerate it. how can i save a property dynamic in wso2 esb and retrieve it. my sequence to generate tokekn is :
<sequence name="x" xmlns="http://ws.apache.org/ns/synapse">
<payloadFactory media-type="xml">
<format>
<p:login xmlns:p="http://corebankingservice.endpoint.webservicegateway.core.channelmanager.caspian.com/">
<chUserInfoRequestBean xmlns="">
<password>****</password>
<username>user</username>
</chUserInfoRequestBean>
<channelServiceType xmlns="">***</channelServiceType>
</p:login>
</format>
</payloadFactory>
<send>
<endpoint>
<address uri="http://x.x.x.x:8280/services/..."/>
</endpoint>
</send>
</property name="SessionId" value = ...>
</sequence>
result of this sequence is SessionId so i need to save SessionId in memory or db or .... and use it for another method until it will be expired. how can i save such a this property in registery or cashe or database and use it later.
The only "official" way I know (which I used), is to use the dblookup mediator (retrieve the value later) and dbreport meditaor (to store the value) in a database.
Write value to a db:
<dbreport>
<connection>
<pool>
<dsName>datasource_name</dsName>
</pool>
</connection>
<statement>
<sql>
insert into table (attributName1, attributName2) values ('value1','value2')</sql>
</statement>
</dbreport>
The values can be passed from a property like in this example.
<sql>
insert into dbo.sais_config_db (serviceName, configKey, configValue) values ('staticVal1','staticVal2',?)</sql>
<parameter expression="get-property('myPropertyName')" type="CHAR"/>
Read value:
<dblookup>
<connection>
<pool>
<dsName>sais_config_db</dsName>
</pool>
</connection>
<statement>
<sql>
SELECT value1 FROM tableName where ....</sql>
<result name="resultFromDb" column="value1"/>
</statement>
</dblookup>
Hope that helps. There is another option, but I don't know if that works (haven't tried it).
Link
With WSO2 Enterprise Integrator 6.x.x and up, it's possible to set a property with scope "registry" that will persist the value between calls and being globally visible.
For example to persist the access token received in the JSON response:
<property description="propAccessToken" expression="json-eval($.access_token)" name="access_token" scope="registry" type="STRING"/>
Then to get the property use the following expression:
get-property('registry', 'access_token')

WSO2 guaranteed delivery message In memory store

Trying to make workable solution for guaranteed delivery In memory.
Create InMemoryStore InMemMessageStore, create and point InsertInvoice
create API so code looking like this :
Sequence:
<?xml version="1.0" encoding="UTF-8"?> <sequence name="InMMSsequence" xmlns="http://ws.apache.org/ns/synapse">
<log level="custom">
<property name="STATE" value="message is sent to InMemMessageStore"/>
</log>
<property name="FORCE_SC_ACCEPTED" scope="axis2" type="STRING" value="True"/>
<axis2ns12:store messageStore="InMemMessageStore" xmlns:axis2ns12="http://ws.apache.org/ns/synapse"/> </sequence>
my API looks like :
<resource methods="POST" uri-template="/sendMessage">
<inSequence>
<sequence key="InMMSsequence"/>
</inSequence>
<outSequence/>
<faultSequence/>
</resource>
Message Processor :
<messageProcessor name="MySMessageProcessor" class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor" targetEndpoint="InsertInvoice" messageStore="InMemMessageStore" xmlns="http://ws.apache.org/ns/synapse">
<parameter name="interval">1000</parameter>
<parameter name="client.retry.interval">1000</parameter>
<parameter name="max.delivery.attempts">4</parameter>
<parameter name="is.active">true</parameter>
<parameter name="max.delivery.drop">Disabled</parameter>
<parameter name="member.count">1</parameter>
</messageProcessor>
And point :
<endpoint xmlns="http://ws.apache.org/ns/synapse"
name="InsertInvoice">
<http uri-template="http://xxxx.xxx.xxx.xxx/InsertInvoiceVehicleList"
method="post">
<suspendOnFailure>
<progressionFactor>1.0</progressionFactor>
</suspendOnFailure>
<markForSuspension>
<retriesBeforeSuspension>0</retriesBeforeSuspension>
<retryDelay>0</retryDelay>
</markForSuspension> </http> </endpoint>
PROBLEMS:
over PostMan I'm sending the request in JSON format but when I open InMemMessageStore message is in XML ?!? Why?
Endpoint expecting message in JSON format. Probably this is the reason of failure BUT on log I see something like
Failed to send the message through the fault sequence. Sequence name does not Exist.
Is the fault sequence mandatory or it is complaining because I don't have any default error sequence defines at all ?
also
Unable to sendViaPost to url[http://xxx.xxx.xx.xx/InsertInvoiceVehicleList/sendMessage]
Why the 'sendMessage' (This is uritemplate that is defined in API is added on endpoint url ?!?!)
so : Biggest issue here is how to keep message in JSON format and how to keep endpoint url intact ...
I found the problem and I wanted to share it with others ...
1. call end point from API adds postfix on end gives the error
By default, URI template 'sendMessage' will get appended to the target URL when sending messages out. You need to use following property in your sequence to remove URI template from target URL
<property action="remove" name="REST_URL_POSTFIX" scope="axis2"/>.
2. message processor fails to send message to end point
My end point was api .net web service and issue was with HTTP 1.1 chunking,had to disable it since .NET service doesn’t support it.
This setting is in axis2_blocking_client.xml and axis2_client.xml
<parameter name="Transfer-Encoding">chunked</parameter>
3. message processor fails to send message to end point
my seccond End point was servicestack web service and I used end point like
http://xxxx.xxx.xxx.xxx/TMSALSvc/UpdateStatus
but problem was that I needed to specify the reponce format. Once when I put a EndPoint delaration like
http://xxx.xxx.xxx.xx/TMSALSvc/json/reply/UpdateStatus
everything forked
I hope that this can same some times for other facing the same issues

wso2 - using value from a property to retrive sequence name

I'm configuring a proxy-service and I have three sequences: s1,s2,s3
The value of 1,2,3 is stored in a local registry variable and it's get from the registry and stored in the property called 'myProp'
now, based on the value of this myProp I would like to call one of the three sequences.
I tried this:
<sequence key="s{concat(get-property('myProp'))}"/>
but doesn't works.
This is the property code:
<property name="myProp"
expression="get-property('registry','conf:repository/myVersion2.xml')"
scope="default"
type="STRING"/>
and this is what I'm trying to do:
<filter source="get-property('myProp')"
regex=".*>1<.*"
description="filter">
<then>
<log level="custom" separator=":">
<property name="TestVersion" value="LOG_S1_TRUE"/>
<property name="TestVersion" expression="get-property('myProp')"/>
</log>
<sequence key="s{concat(get-property('myProp'))}"/>
</then>
<else>
<log level="custom" separator=":">
<property name="TestVersion" value="LOG_S1_FALSE"/>
</log>
</else>
</filter>
I get this error from the log:
TID: [0] [ESB] [2015-07-03 12:47:25,340] ERROR {org.apache.synapse.mediators.base.SequenceMediator} - Sequence named Value {name ='null', keyValue ='s{concat(get-property('myProp'))}'} cannot be found {org.apache.synapse.mediators.base.SequenceMediator}
Thanks in advance to who know how to solve it.
Regards
Claudio
Assign the keyvalue to a property first to test it, you will notice that it cannot work :)
Try:
<sequence key="{concat('s', get-property('myProp'))}"/>

WSO2 ESB 4.5.0 vfs TransportSender as Endpoint Errors with 'Access is denied'

I am trying to put together a proof of concept for the use of wso2 esb. The proof of concept will rely on the ESB picking up a csv file dropped into a folder, converting the details to xml, posting them to a 3rd party web service, then converting the response, which should contain the binary for a pdf, into a pdf and dropping it into a folder.
The current problem with this is that when I configure a folder as an endpoint in wso2 esb 4.5.0, any file that I send to that end-point errors. A stripped down version of my configuration is as defined below: -
<proxy name="PDFPoller"
transports="vfs"
startOnLoad="true"
trace="enable"
statistics="enable">
<description/>
<target>
<inSequence>
<log level="custom">
<property name="status" value="PDF Receieved"/>
</log>
<log level="full"/>
<property name="transport.vfs.ReplyFileName"
expression="test1.pdf"
scope="transport"/>
<property name="OUT_ONLY" value="true"/>
<send>
<endpoint name="FileEpr">
<address uri="vfs:file:///c:/wso2/processed"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///C:/wso2/output</parameter>
<parameter name="transport.vfs.FileURI">file:///C:/wso2/PollFolder</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///C:/wso2/Failed</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.pdf</parameter>
<parameter name="transport.vfs.ContentType">application/pdf</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
The error that I get from the ESB is as follows: -
2012-10-29 09:46:00,642 [-] [Axis2 Task] ERROR VFSTransportSender IO Error while
org.apache.commons.vfs2.FileSystemException: Could not write to "file:///c:/wso2/processed".
at org.apache.commons.vfs2.provider.AbstractFileObject.getOutputStream(AbstractFileObject.java:1440)
at org.apache.commons.vfs2.provider.DefaultFileContent.getOutputStream(DefaultFileContent.java:462)
at org.apache.synapse.transport.vfs.VFSTransportSender.populateResponseFile(VFSTransportSender.java:232)
at org.apache.synapse.transport.vfs.VFSTransportSender.sendMessage(VFSTransportSender.java:173)
at org.apache.axis2.transport.base.AbstractTransportSender.invoke(AbstractTransportSender.java:112)
at org.apache.axis2.engine.AxisEngine$TransportNonBlockingInvocationWorker.run(AxisEngine.java:627)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException: c:\wso2\processed (Access is denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
at org.apache.commons.vfs2.provider.local.LocalFile.doGetOutputStream(LocalFile.java:220)
at org.apache.commons.vfs2.provider.AbstractFileObject.getOutputStream(AbstractFileObject.java:1432)
... 8 more
I don't believe that the problem is a local folder permissions problem, because I have a) checked the permissions on the folder manually and b) The ESB will move the file that I place in the 'pollfolder' to either 'Output' or 'Processed' if I set the transport.vfs.MoveAfterProcess property to that value, so it is possible to write to both these folders.
Any help would be appreciated.
You configuration seems fine. But I suspect the following section.
<property name="transport.vfs.ReplyFileName"
expression="test1.pdf"
scope="transport"/>
You can try without that property. Then it will create a file as same as the input file name inside the "processed" folder. If it works then try the above property as follows.
<property name="transport.vfs.ReplyFileName"
value="test1.pdf"
scope="transport"/>
As you can see, I have replaced "expression" attribute with "value". Expression should be used when you want to execute something, for example concatenating two strings to build the response file name etc.