Publish statistic from wso2 EI to wso2 Stream Processor - wso2

I need to know how is it possible to publish statistics via event publisher from Enteprise Integrator to Stream Processor.
I have following implementation of event publisher on my EI
<?xml version="1.0" encoding="UTF-8"?>
<eventPublisher name="MessageFlowStatisticsPublisher"
statistics="enable" trace="enable" xmlns="http://wso2.org/carbon/eventpublisher">
<from streamName="org.wso2.esb.analytics.stream.FlowEntry" version="1.0.0"/>
<mapping customMapping="disable" type="wso2event"/>
<to eventAdapterType="wso2event">
<property name="username">admin</property>
<property name="protocol">thrift</property>
<property name="publishingMode">non-blocking</property>
<property name="publishTimeout">0</property>
<property name="receiverURL">tcp://xxx:7611</property>
<property encrypted="true" name="password">xxx</property>
</to>
</eventPublisher>
On Stream Processor I have simple siddhi app for receive data and print them into log as is shown below
#App:name("FlowEntryApp")
#App:description("Plan of flow entry")
#source(type='wso2event', #map(type = 'wso2event'))
define stream FlowEntry(compressed bool, tenantId int, messageId string, flowData string);
#sink(type='log', prefix='My flowEntry:')
define stream TestOutputFlowEntry(messageId string, flowData string);
#info(name='FlowEntryOutput')
from FlowEntry
select messageId, flowData
group by messageId
insert into TestOutputFlowEntry;
Also I have setted all configuration for publishing statstics as "enable statistic" and "enable trace" for my proxy service. When I invoke my service, eventPublisher send wso2event to SP, this is working correctly. But on the SP side, SP handle error "No StreamDefinition for streamId org.wso2.esb.analytics.stream.FlowEntry:1.0.0 present in cache"
I know, that problem is in siddhi app, that I define stream "FlowEntry" instead of "org.wso2.esb.analytics.stream.FlowEntry" but siddhi language doesn't support characters like '.' in stream name.
So I tried to change stream name on EI site, change streamName in eventPublisher to 'FlowEntry' only, also I changed streamName in json file inside eventstream folder but now when I invoke my service, EI will not send any events to SP.
Have anybody idea how to publish org.wso2.esb.analytics.stream.FlowEntry stream to SP and then processed it by siddhi?

The stream name can be overridden by using wso2.stream.id element in the source annotation.
#source(type='wso2event', wso2.stream.id='org.wso2.esb.analytics.stream.FlowEntry', #map(type = 'wso2event'))<br>
define stream FlowEntry(compressed bool, tenantId int, messageId string, flowData string);
By using the above source definition, 'FlowEntry' can still be used inside the Siddhi App, while in the thrift server stream id will be defined as 'org.wso2.esb.analytics.stream.FlowEntry:1.0.0'.
Best Regards.

Related

Regd. getting empty response with data mapper in wso2 EI

I am trying to transform json to xml using datamapper and i am getting the empty xml structure but not getting any values in wso2. I have created data mapper dependencies and loaded both input and out structures and used AI to map json -> xml and they mapped correctly.
I tried on eclipse oxygen(esb-6.2.0) and integration studio(v8) also and deployed in EI(6.5.0) but still the behavior is same empty response structure. I kept a log in the in-sequence and it is logging the json request but not the xml response. I am not getting why the issue is happening.
Please provide your thoughts and attached the code to this post. Please do needful
The issue is with the API. In the API we need to specify the input and the output data type. In your case since the transformation is from JSON to XML, the input type needs to be JSON while the output type needs to be XML. But you have defined both as xml which resulted in this issue. Modify the input type to JSON and you will get the expected results
<api xmlns="http://ws.apache.org/ns/synapse" name="DataMapper" context="/data">
<resource methods="POST" url-mapping="/mapper">
<inSequence>
<log separator="/">
<property name="payload" expression="json-eval($.body)"/>
</log>
<datamapper config="gov:datamapper/Wso2DataMapper.dmc" inputSchema="gov:datamapper/Wso2DataMapper_inputSchema.json" outputSchema="gov:datamapper/Wso2DataMapper_outputSchema.json" xsltStyleSheet="gov:datamapper/Wso2DataMapper_xsltStyleSheet.xml" inputType="JSON" outputType="XML"/>
<respond/>
</inSequence>
<outSequence/>
<faultSequence/>
</resource>
</api>
input
{
"studNo":"Sample studNo",
"studName":"Sample studName",
"StudGroup":"Sample StudGroup",
"studAddress":{
"addrLine1":"Sample addrLine1",
"AddrLine2":"Sample AddrLine2",
"Mandal":"Sample Mandal",
"District":"Sample District",
"State":"Sample State",
"Country":"Sample Country",
"Zip":"Sample Zip"
}
}
output
<Student xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<studentNo>Sample studNo</studentNo>
<studentName>Sample studName</studentName>
<studentGroup>Sample StudGroup</studentGroup>
<studentAddress>
<AddressLine1>Sample AddrLine2</AddressLine1>
<AddressLine2>Sample addrLine1</AddressLine2>
<Mandal>Sample Mandal</Mandal>
<District>Sample District</District>
<State>Sample State</State>
<Country>Sample Country</Country>
<ZIP>Sample Zip</ZIP>
</studentAddress>
</Student>

How to get Request payload content at Response path class mediator using OMElement - WSO2 APIM ver 3.2.0

I am using WSO2 APIM version 3.2.0.
I have a POST request with the request payload.
In the response message mediation of WSO2 APIM I have added the policy that contains the class mediator that tries to get the payload sent during the request.
OMElement element = (OMElement) mc.getEnvelope().getBody().getFirstOMChild();
log.info("payload: " + element.toString());
The above code snippet prints the response payload content but I need the request payload content at the response path.
Response message mediation with a policy added
Below is the sequence with class mediator
sequence with class mediator
Code snippet inside class mediator
OMElement element = (OMElement) mc.getEnvelope().getBody().getFirstOMChild();
log.info("payload: " + element.toString());
Pls let me know what changes to be done, to get the request payload content.
First, we have to store the request payload in a custom property in the Message Context. Then, we can use that property to retrieve the Request Payload in the Response path of the execution.
For example: You are invoking an API with JSON Payload. So, we have to first capture the sent payload and store it in a custom property in the Message Context. Given below is a sample sequence to perform the same
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="admin--MockAPI:v1.0.0--In">
<property name="RequestPayload" expression="json-eval($)" />
<log level="custom">
<property name="RequestPayload" expression="$ctx:RequestPayload" />
</log>
</sequence>
Then, in the Response path, inside your custom class mediator, you have to access the RequestPayload property from the MessageContext to extract the stored payload. You can achieve this by using the following snippet
synapseContext.getProperty("RequestPayload");

WSO2 APIM-Analytic service (version:2.6.0) is not able to send the alert emails

In order to test this function, I chosed Abnormal Request Count option, which is in Alert Management menu of Api Store, and added a restful api. Then I called this api for many times by Postman.
The carbon.log, which is in <API-M_ANALYTICS_HOME>/wso2/worker/logs, recorded the following error message:
[2020-10-19 11:03:36,094] ERROR {org.wso2.siddhi.core.stream.output.sink.Sink} - Error on 'APIM_ALERT_EMAIL_NOTIFICATION'. Dropping event at Sink 'email' at 'EmailNotificationStream' as its still trying to reconnect!, events dropped '<strong>Message:</strong>A request from a new IP (10.9.16.77) detected by user:admin#carbon.super using application:devMap owned by admin#carbon.super. <br><br> <strong>Type:</strong>UnusualIPAccess <br><br> <strong>AlertTimestamp:</strong>2020-10-19 11:03:35'
[2020-10-19 11:06:10,307] ERROR {org.wso2.siddhi.core.stream.output.sink.Sink} - Error on 'APIM_ALERT_EMAIL_NOTIFICATION'. Dropping event at Sink 'email' at 'EmailNotificationStream' as its still trying to reconnect!, events dropped '<strong>Message:</strong>Abnormal request count detected during last minute using application devMap owned by admin#carbon.super for api :全球风向查询, abnormal request count:9. <br><br> <strong>Type:</strong>AbnormalRequestsPerMin <br><br> <strong>AlertTimestamp:</strong>2020-10-19 11:06:10'
About detailed configuration, I referred the following links:
Configuring Alerts:
https://docs.wso2.com/display/AM260/Configuring+Alerts#ConfiguringAlerts-ConfiguringalertsviatheStore
Enabling Notifications:
https://docs.wso2.com/display/AM260/Enabling+Notifications
According to these documents, I did the following things:
1.Open the <API-M_ANALYTICS_HOME>/conf/worker/deployment.yaml file to configure the sender email address. The sample code is shown below:
siddhi:
extensions:
...
-
extension:
name: email
namespace: sink
properties:
username: abcd#163.com
address: abcd#163.com
password: xxxx
...
2.Go to the <API-M_ANALYTICS_HOME>/resources/apim-analytics/ directory. Copy the APIM_ALERT_EMAIL_NOTIFICATION.siddhi file and paste it in the <API-M_ANALYTICS_HOME>/wso2/worker/deployment/siddhi-files directory.
3.Set the email server configurations in the <API-M_HOME>/repository/conf/output-event-adapters.xml file under the section.
<adapterConfig type="email">
<!-- Comment mail.smtp.user and mail.smtp.password properties to support connecting SMTP servers which use trust
based authentication rather username/password authentication -->
<property key="mail.smtp.from">abcd#163.com</property>
<property key="mail.smtp.user">abcd</property>
<property key="mail.smtp.password">xxxx</property>
<property key="mail.smtp.host">smtp.163.com</property>
<property key="mail.smtp.port">25</property>
<property key="mail.smtp.starttls.enable">true</property>
<property key="mail.smtp.auth">true</property>
<!-- Thread Pool Related Properties -->
<property key="minThread">8</property>
<property key="maxThread">100</property>
<property key="keepAliveTimeInMillis">20000</property>
<property key="jobQueueSize">10000</property>
</adapterConfig>
4.Log in to the Management Console and click Main > Resource > Browse. Browse to the /_system/config/apimgt/applicationdata/tenant-conf.json file and click Edit as Text.Set the NotificationsEnabled property to true as shown below:
"NotificationsEnabled":"true",
"Notifications":[{
"Type":"new_api_version",
"Notifiers" :[{
"Class":"org.wso2.carbon.apimgt.impl.notification.NewAPIVersionEmailNotifier",
"ClaimsRetrieverImplClass":"org.wso2.carbon.apimgt.impl.token.DefaultClaimsRetriever",
"Title": "Version $2 of $1 Released",
"Template": " <html> <body> <h3 style=\"color:Black;\">We’re happy to announce the arrival of the next major version $2 of $1 API which is now available in Our API Store.</h3>Click here to Visit WSO2 API Store</body></html>"
}]
}
]
I've also checked APIM_ALERT_EMAIL_NOTIFICATION.siddhi:
#App:name("APIM_ALERT_EMAIL_NOTIFICATION")
#App:description('Send email to all the subscribers of a particular alert')
#source(type="inMemory", topic="APIM_EMAIL_NOTIFICATION", #map(type='passThrough'))
define stream EmailAlertStream (
type string,
message string,
alertTimestamp string,
emails string);
#sink(type='email', content.type="text/html", #map(type ='text', #payload('<strong>Message:</strong>{{message}} <br><br> <strong>Type:</strong>{{type}} <br><br> <strong>AlertTimestamp:</strong>{{alertTimestamp}}')),subject='Alerts from WSO2 APIM Analytics',to='{{emails}}')
define stream EmailNotificationStream (
type string,
message string,
alertTimestamp string,
emails string);
from EmailAlertStream
select *
insert into EmailNotificationStream;
As you can see, there is nothing special.
Have I missed anything to do? I don't know what the problem is, please help me.
As per the exceptions you received. Looks like there are issues with connecting to the SMTP server. Can you please check the configurations and check there are any restrictions to use your email.
ex: in Gmail, you have to enable 'untrusted applications' to access Gmail.
I've found the reason of this problem.
By reading the source code (EmailSink.java), I know the information of SMTP Server should be configured in APIM_ALERT_EMAIL_NOTIFICATION.siddhi, otherwise the analytic server is going to use smtp.gmail.com by default.

How can we store payload in database using wso2 API manager?

I am publishing one custom API through rest HTTP endpoint and after publishing API I am subscribing the API. During subscription I am generating the production endpoint URL token and then trying to access the endpoint.I am able to get the api data on dashboard itself.
How can we store the corresponding api payload in database?
You can add a custom sequence and use DB Report mediator within that. Following is a DBreport mediator sample config.
<dbreport description="">
<connection>
<pool>
<password>regadmin</password>
<driver>com.mysql.jdbc.Driver</driver>
<url>jdbc:mysql://localhost:3306/regdb</url>
<user>regadmin</user>
</pool>
</connection>
<statement>
<sql>insert into tracker (`id`, `query`, `tracked`) values (NULL, ?, NOW())</sql>
<parameter expression="get-property('uri.var.id')" type="VARCHAR"/>
</statement>
</dbreport>
You can read about adding custom sequences from here and abount DBLookup mediator from here and DBReport Mediator from here.

WSO2 API Manager change http method

Does wso2 api manager v1.10.0 permit transforming the HTTP method of the request to the backend through custom in sequence?
I created an api with http GET resource through publisher web console. But since the endpoint support POST method only, i tried changing the HTTP Method by creating custom in sequence with property mediator :
<property name="HTTP_METHOD" value="POST" scope="axis2"/>
but the response showed a fault message :
{
"fault": {
"code": 403,
"type": "Status report",
"message": "Fault Call",
"description": "No matching resource found in the API for the given request"
}
}
The log files only showed these lines :
==> /opt/wso2am-gateway/repository/logs/wso2carbon.log <==
[2016-04-08 10:30:16,868] INFO - STATUS = Executing default 'fault' sequence, ERROR_CODE = 403, ERROR_MESSAGE = No matching resource found in the API for the given request {org.apache.synapse.mediators.builtin.LogMediator}
If i remove the property mediator, the request pass through and reach the backend.
Does anyone know how to solve this problem?
You can use the following custom inFlow mediation sequence to convert the HTTP_METHOD into POST from GET.
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="CustomIn" xmlns="http://ws.apache.org/ns/synapse">
<property action="remove" name="HTTP_METHOD" scope="axis2"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="POST"/>
</sequence>
Here, the request is the GET request from the client-side.
But, the above solution will be possible only when you are defining the GET and the POST resources similarly on your API (Although you don't use one of the both).
Otherwise, you will get the following error messages while you are invoking the API.
No matching resource found for given API Request
Method not allowed for given API resource
No matching resource found in the API for the given request
You need to also define POST resource on your API (although you don't use it)