API level rate limiting not happening in wso2 - wso2

I am using wso2-am 2.0.0 to achieve rate limiting for APIs. I have published an API with one user and it is then subscribed by another user. While subscribing, I am selecting a subscription tier of 20 requests per minute for that API. Also, the Application used to subscribe to the API, has a limit of 50 requests per minute. Now, when I am testing the API, only Application level rate limiting is happening after around 50 requests. The API level limit is not applied. Ideally the user should not be able to access the API after 20 requests. I am using windows machine to setup the wso2 server and Apache JMeter to send requests to the server. What am I missing here? Need some help...

Here is a 4MB/s policy.
#Plan:name('carbon.super_app_BandwidthPolicy')
#Plan:description('ExecutionPlan for app_BandwidthPolicy')
#Import('org.wso2.throttle.processed.request.stream:1.0.0')
define stream RequestStream (messageID string, appKey string, appTier string, subscriptionKey string, apiKey string, apiTier string, subscriptionTier string, resourceKey string, resourceTier string, userId string, apiContext string, apiVersion string, appTenant string, apiTenant string, appId string, apiName string, propertiesMap string);
#Export('org.wso2.throttle.globalThrottle.stream:1.0.0')
define stream GlobalThrottleStream (throttleKey string, isThrottled bool, expiryTimeStamp long);
FROM RequestStream
SELECT messageID, (appTenant == 'carbon.super' and appTier == 'BandwidthPolicy') AS isEligible, appKey AS throttleKey, propertiesMap
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min, 0)
select throttleKey, (sum(cast(map:get(propertiesMap,'messageSize'),'long')) >= 4194304) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;
from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
select *
insert into GlobalThrottleStream;

Related

REST API for monitoring undelivered message in google cloud pubsub

I want to implement a service to monitor an undelivered messages and send notification when it reach threshold or process further.
I already look through the Stackdriver. It provide me the monitoring and alert that It only provide the API to get the metricDescriptor but it does not provide an API to get the undelivered message as you can see in Stackdriver Monitoring API.
Is there actually an provided API to get the metrics value?
You can get the values via the projects.timeSeries.list method. You would set the name to projects/<your project>, filter to metric.type = "pubsub.googleapis.com/subscription/num_undelivered_messages", and end time (and if a range of values is desired, the start time as well) to a string representing a time in RFC3339 UTC "Zulu" format, e.g., 2018-10-04T14:00:00Z. If you want to look at a specific subscription, set the filter to metric.type = "pubsub.googleapis.com/subscription/num_undelivered_messages" AND resource.label.subscription_id = "<subscription name>".
The result will be one or more TimeSeries types (depending on whether or not you specified a specific subscription) with the points field including the data points for the specified time range, each of which will have the value's int64Value set to the number of messages that have have not been acknowledged by subscribers.

Google API quota usage conditions

I'm looking for ways to overcome API quotas without resorting to requesting a raise, and would like to know what is classified as .. per user.
For example:
Google People API has this quota: Read Requests per 100 seconds per user
I setup an OAuth Client ID: 123-5ampl3.apps.googleusercontent.com
And for whatever reason, my queries are going to exceed "100 seconds per user".
My question/concerns:
Can I create another client ID 123-an0th3r.apps.googleusercontent.com and have both call the same API so that I now essentially have requests per 200 seconds per user?
Or is per user not tied to the client IDs, but instead, to the project id.
Could I create another project and re-route extra API calls to there?
Or must I throttle my querying so it stays within the limit.
Thanks!

Amazon Lex get incoming SMS number in java lambda function handler from Twilio?

I've created a lex sms chatbot and published it to a Twilio SMS Channel. Within my java lambda fulfillment handleRequest function I receive 2 parameters: an Input object and a Context. Input has some type of system generated userId, but I need the phone number of the incoming sms message from Twilio.
I've configured Twilio to call Lex via webhook per these instructions:
http://docs.aws.amazon.com/lex/latest/dg/twilio-bot-association.html#twilio-bot-assoc-create-assoc
I'm using the system created webhook callback url between Lex and Twilio and would like to avoid writing my own Lex-Twilio go between with Gateway API and lambda if possible.
I think I'm missing a configuration in Twilio to send this to Lex maybe? I haven't setup any TwilML app or anything, just linked the SMS number to my webhook callback url for lex. Everything works fine except getting the incoming sms number.
UPDATE:
I exhausted all possibilities with my input object and context in my java lambda function. I guess there's no way to get at the payload from twilio in Java. I tried switching to an input stream handler but still only had the input format defined in the documentation here: http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
I had to bite the bullet and build my own gateway API and Node handler. It's just a layer that sits before lex and translates twilio into lex input format and vis versa on the response. It updates the userId with the incoming phone number.
This tutorial was very useful in doing this:
https://aws.amazon.com/blogs/ai/integrate-your-amazon-lex-bot-with-any-messaging-service/
It was a lot of work just to get one additional field, but It does provide me the flexibility to switch SMS providers in the future easier.
I am not sure as what exactly stopped you from getting the user phone number in your lambda function. You can always get sender phone number from 'userId' field from the input event in your lambda function. You can refer to the documentation to get more information.

AWS API Gateway - How do I get the date/timestamp/epoch in a body mapping template?

I need to include the request time in the body mapping template for an API Gateway method. Is there a date/time variable or function? I couldn't find anything in the template reference.
Example body mapping template:
Action=SendMessage&MessageBody=$util.urlEncode("{""timestamp"":""TIMESTAMP_HERE"",""body-json"":$input.json('$'),""params"":""$input.params()""}")
UPDATE:
API Gateway just added two new context variables http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html
$context.requestTime The CLF-formatted request time (dd/MMM/yyyy:HH:mm:ss +-hhmm).
$context.requestTimeEpoch The Epoch-formatted request time.
API gateway doesn't support this currently. Its been requested before on the forums - https://forums.aws.amazon.com/thread.jspa?messageID=697658&. We have this feature in our backlog, but unfortunately I can't commit to any timelines.
There appears to be a way to get a timestamp from the API Gateway request, but you need to look in the X-Ray documentation to find it:
Amazon API Gateway gateways add a trace ID to incoming HTTP requests in a header named X-Amzn-Trace-Id.
(...)
A trace_id consists of three numbers separated by hyphens. For example, 1-58406520-a006649127e371903a2de979. This includes:
The version number, that is, 1.
The time of the original request, in Unix epoch time, in 8 hexadecimal digits. For example, 10:00AM December 2nd, 2016 PST in epoch time is 1480615200 seconds, or 58406520 in hexadecimal.
(...)
Just set a time value at the top of your template for the date you want it to expire. Then add that as an attribute to your Item. For example, if you want the Item to expire in 60 days, you would add 5184000 seconds.
#set($expireDate = $context.requestTimeEpoch + 5184000)
{
"TableName": "your-table-name",
"Item": {
... other attributes
"expireDate": {"N": "$expireDate"},
}
}

Web Services in Siddhi

I have a web services (REST). I want to call the WS every time an event comes to my stream.
I have to create an extension of Siddhi and call the web service from java?
If I need java to invoke the Web service, should I extend "Aggregate Function" ?.
In Siddhi exists any instruction that I can use to call the web service?
The next is a sample of my WS:
http://72.37.125.125:3000/phishing?url=www.innerjoin.co
thanks for your comments
You will be able to use HTTP event publisher[1] to achieve this OOTB without writing custom code. Using this publisher you can send PUT/POST requests to pre-defined URL with custom payload. Create a HTTP publisher as mentioned in the shared documentation. Then you can use a custom json mapping to create the accepted payload for your web service. When you are creating custom mapping you can refer event attributes inside curly braces as follows.
{
"payload_name_for_webservice":{{attribute_name_in_event}},
}
[1]https://docs.wso2.com/display/CEP400/HTTP+Event+Publisher
Thank you.
but i have a receiver and other publisher.
i want call my WS from siddhi when arrive a new event to "org.wso2.event.sensor.stream:1.0.0", the input parameter to the web service is "meta_sensorName"
how i can make it?
My code is this:
/* Enter a unique ExecutionPlan */
#Plan:name('ExecutionPlan')
/* Enter a unique description for ExecutionPlan */
-- #Plan:description('ExecutionPlan')
/* define streams/tables and write queries here ... */
#Import('org.wso2.event.sensor.stream:1.0.0')
define stream sensorStream (meta_timestamp long, meta_isPowerSaverEnabled bool, meta_sensorId int, meta_sensorName string, correlation_longitude double, correlation_latitude double, humidity float, sensorValue double);