I was wondering about the performance of WSO2 throttling. Whilst checking it out, I did the following :
I created a new subscription tier (say Tier1)
I set a limit of 10 requests per hour on it
I created another subscription tier (say Tier2)
I set a limit of 20 requests per hour on it
I allowed my API to use those tiers and created a test application to use the API with Tier1
Using the API console, I started sending requests to my API
After the 10 calls(though it exceeded 10), I unsubscribed the test application and subscribed to the same API using the same application using Tier2
It took around 15 mins for the throttler to unlock. Which brings me to my first concern ...
How long does it take for a change in tier subscription to reflect on the same application????
In relation to the excess calls going through, I also set up tiers with limits on a per minute basis. Once again, I noticed excess calls going through.
What I also noticed was that this excess was in the initial number of calls and the throttling worked normally after those initial excess calls.
Which leads to my next ...
How long does it take for a newly set up tier to throttle properly? In the per-minute basis, I assume that some calls carry over to the next minute. Am I right in assuming that?
Any help in understanding this would be much appreciated. Thanks
Related
I have hosted my node app in Cloud Run and all of my requests served within 300 - 600ms time. But one endpoint that gets data from a 3rd party service so that request takes 1.2s - 2.5s to complete the request.
My doubts regarding this are
Is 1.2s - 2.5s requests suitable for cloud run? Or is there any rule that the requests should be completed within xx ms?
Also see the screenshot, I got a message along with the request in logs "The request caused a new container instance to be started and may thus take longer and use more CPU than a typical request"
What caused a new container instance to be started?
Is there any alternative or work around to handle long requests?
Any advice / suggestions would be greatly appreciated.
Thanks in advance.
I don't think that will be an issue unless you're worried about the cost of the CPU/memory time, which honestly should only matter if you're getting 10k+ requests/day. So, probably doesn't matter and cloud run can handle that just fine (my own app does requests longer than that with no problem)
It's possible that your service was "scaled to zero" meaning that there were no containers left running to serve requests. In that case, it would be necessary to start up a new instance and wait for whatever initializing/startup costs are associated with that process. It's also possible that it was auto-scaled due to all other instances being at their request limits. Make sure that your setting for max concurrent requests per instance is set greater than one - Node/Express can handle multiple requests at once. Plus, you'll only get charged for the total time spend, not per request:
In situations where you get very long (30 seconds, minutes+) operations, it may be a good idea to switch to some different data transfer method. You could use polling, where the client makes a request every 5 seconds and checks if the response is ready. You could also switch to some kind of push-based system like WebSockets, but Cloud Run doesn't have support for that.
TL;DR longer requests (~10-30 seconds) should be fine unless you're worried about the cost of the increased compute time they may occur at scale.
I have a path (mysite.com/myapiendpoint for sake of example) that is both resource intensive to service, and very prone to bot abuse. I need to rate limit access to that specific path to something like 10 requests per minute per client IP address. How can this be done?
I'm hosting off an EC2 instance with CloudFront and AWS WAF in front. I have the standard "Rate Based Rule" enabled, but its 2,000 requests per minute per IP address minimum is absolutely unusable for my application.
I was considering using API Gateway for this, and have used it in the past, but its rate limiting as I understand it is not based on IP address, so bots would simply use up the limit and legitimate users would constantly be denied usage of the endpoint.
My site does not use sessions of any sort, so I don't think I could do any sort of rate limiting in the server itself. Also please bear in mind my site is a one-man-operation and I'm somewhat new to AWS :)
How can I limit the usage per IP to something like 10 requests per minute, preferably in WAF?
[Edit]
After more research I'm wondering if I could enable header forwarding to the origin (running node/express) and use a rate-limiter package. Is this a viable solution?
I don't know if this is still useful to you - but I just got a tip from AWS support. If you add the rate limit rule multiple times, it effectively reduces the number of requests each time. Basically what happens is each time you add the rule, it counts an extra request for each IP. So say an IP makes a single request. If you have 2 rate limit rules applied, the request is counted twice. So basically, instead of 2000 requests, the IP only has to make 1000 before it gets blocked. If you add 3 rules, it will count each request 3 times - so the IP will be blocked at 667 requests.
The other thing they clarified is that the "window" is 5 minutes, but if the total is breached anywhere in that window, it will be blocked. I thought the WAF would only evaluate the requests after a 5 minute period. So for example. Say you have a single rule for 2000 requests in 5 minutes. Say an IP makes 2000 requests in the 1st minute, then only 10 requests after that for the next 4 minutes. I initially understood that the IP would only be blocked after minute 5 (because WAF evaluates a 5 minute window). But apparently, if the IP exceeds the limit anywhere in that window, it will be locked immediately. So if that IP makes 2000 requests in minute 1, it will actually be blocked from minute 2, 3, 4 and 5. But then will be allowed again from minute 6 onward.
This clarified a lot for me. Having said that, I haven't tested this yet. I assume the AWS support techie knows what he's talking about - but definitely worth testing first.
AWS have now finally released an update which allows the rate limit to go as low as 100 requests every 5 minutes.
Announcement post: https://aws.amazon.com/about-aws/whats-new/2019/08/lower-threshold-for-aws-waf-rate-based-rules/
Using rule twice will not work, because WAF rate based rule will count on cloud watch logs basis, both rule will count 2000 requests separately, so it would not work for you.
You can use AWS-WAF automation cloud front template, and choose lambda/Athena parser, this way, request count will perform on s3 logs basis, also you will be able to block SQL,XSS and bad bot requests.
I am trying to setup a function which will be working somewhere on the server. It is a simple GET request and I want to trigger it every second.
I tried google cloud functions and AWS. Both of them don't have a straightforward solution to run it every second. (every 1 minute only)
Could you please suggest me a service, or combination of services that will allow me to do it. (preferably not costly)
Here are some options on AWS ...
Launch a t2.nano EC2 instance to run a script that issues GET, then sleeps for 1 second, and repeats. You can't use cron (doesn't support every second). This costs about 13 cents per day.
If you are going to do this for months/years then reduce the cost by using Reserved Instances.
If you can tolerate periods where the GET requests don't happen then reduce the cost even further by using Spot instances.
That said, why do you need to issue a GET request every second? Perhaps there is a better solution here.
You can create a AWS Lambda function, which simply loops and issues the GET request every second, and exits after 240 requets (i.e. 4 minutes). Then create a CloudWatch event that fires every 4 minutes calling the Lambda function.
Every 4 minutes because the maximum timeout you can set for a Lambda function is 5 minutes.
This setup will likely incur only some trivial cost:
At 1 event per 4 minutes, it's $1/month for the CloudWatch events generated.
At 1 call per 4 minutes to a minimally configured (128MB) Lambda function, it's 324,000 GB-second worth of execution per month, just within the free tier of 400,000 GB-second.
Since network transfer into AWS is free, the response size of your GET request is irrelevant. And the first 1GB of transfer out to the Internet is free, which should cover all the GET requests themselves.
I'm looking for ways to overcome API quotas without resorting to requesting a raise, and would like to know what is classified as .. per user.
For example:
Google People API has this quota: Read Requests per 100 seconds per user
I setup an OAuth Client ID: 123-5ampl3.apps.googleusercontent.com
And for whatever reason, my queries are going to exceed "100 seconds per user".
My question/concerns:
Can I create another client ID 123-an0th3r.apps.googleusercontent.com and have both call the same API so that I now essentially have requests per 200 seconds per user?
Or is per user not tied to the client IDs, but instead, to the project id.
Could I create another project and re-route extra API calls to there?
Or must I throttle my querying so it stays within the limit.
Thanks!
We want to collect some metrics about our client public Facebook pages (~1-5K users) on a daily (or weekly) basis.
I'm talking about 3-5 typical metrics : "likes", "fan posts" etc.
I understand that according to the "Rate Limiting on the Graph API" documentation [1] it's possible to have 200 calls per 1 hour.
For now we don't have any FB public application that can help us to increase this limit. To generate application token we will create it to but I doubt it will have a lot of users.
Does anybody know will we have problems with rate limit exceptions while invoking Graph API more than 200 times per 60 min.?
I guess our expected rate is 5-10K calls per 60 min (once a day).
Phrase from the documentation [1] "Rate limiting in the FB Graph API is encountered only in rare circumstances" gives me hope that it won't be a problem.
Thank you!
[1] https://developers.facebook.com/docs/graph-api/advanced/rate-limiting
You won't have any problems initially. Facebook does not necessarily block apps immediately for being over the limits.
As per their documentation
If your app is making enough calls to be considered for rate limiting by our system, we return an X-App-Usage HTTP header
So, if you don't get any X-App-Usage header,Then your app hasn't be considered "worthy" of throttling by their automated systems yet.
So it would be best to check for this header, while making your api requests. Once you start receiving this Header, it would be best change your frequency of the API calls or give a timeout.