We are planning to use extensively Google APIs to transcribe audio in text from a single computer (about 10.000 requests approximately at the same time).
Except quota issues, are you aware of any other limitations on APIs calls coming from the same address ?
Thanks a lot !
If you look at the following article, you can see that there are some restrictions and limits when it comes to the speech-to-text api.
Other than the ones mentioned in the article, there are no restrictions or limits. When making a call to an API, it depends on the amount of requests, not where it comes from. The API does not care if the requests come from one or multiples addresses. As long as the requests do not go over the Quota that the API can handle, it will be okay.
Related
I've been trying for a while to figure out if Google Cloud has a mechanism for a "crowbar" limit on API usage, as a security measure.
The scenario I'm concerned about is, say I have an API keyfile on a server, and a hacker steals it and plugs it into their system that (for the purposes of some nebulous scam) is running an unbounded number of translate requests as quickly as possible. I then receive a $25,000 bill. I don't see any mechanisms to really prevent this.
I can limit the roles for that key, but if the hacker's interest is in the same API I use for my project, that doesn't help. I can set a billing limit, but that just sends me an email, and I am not a person who notices incoming email, so in practice it would take me days to realize this had happened.
I can set a quota, but all the quotas in the dashboard seem to be per-minute or per-day limits. I need a per-month quota, and I don't see any way to do it. Am I just missing something, or does Google simply provide no option for this?
I understand that "well just don't let your API key get compromised" is probably the common wisdom, but that's unacceptable to my use cases for a number of reasons, so I'm not interested in discussing it. Thanks!
Edit: I should note, Google's documentation says "you can set requests per day caps" - but there are no instructions on that page for how to do this, and I can see no mechanism for it. I don't want a per-day cap, I want a per-month cap, but I'd take a per-day cap if I could find one. These are the only quotas I see for Cloud Vision, for instance:
Quotas part 1
Quotas part 2
As per Link 1 there is no Hard quota limit for Vision API on a monthly basis . If you need to use this feature you can request for this feature using the link 2.
In the meantime or as workaround you can control your vision API budget by using the Cloud Billing Budget API by following the link 3.
I am implementing public API using Google Cloud Endpoints & Google Cloud Functions. This API will later be used in web application, and I need a way to throttle number of requests to prevent people from flooding API with huge number of requests, which could increase project maintenance costs. I don't care if API becomes unavailable due to throttling, protecting myself from those costs takes higher priority.
What should I do or which tools should I use to achieve this on Google Cloud?
If we look at the Google Cloud Endpoints documentation in the section called About Quotas we will find a description of a capability to limit the number of requests from calling applications. The article then goes into depth on how to set it all up which appears to be to add additional attributes to your API's exposed Open API spec.
There is also a great article called Rate-limiting strategies and techniques that provides a rich set of alternatives and thoughts. My suggestion would be to read this article in depth which will arm you with an overview of each of the choices at your disposal. There is also a rich set of additional references at the end of the article for further reading.
I have an App that consumes my own API (Google Cloud Functions) and my own Storage (there are images).
Now, I have a couple of clients, that wants to consume my API and my Storage (A Google Cloud Bucket).
The Cloud Storage is a bucket that contains a lot of photo that have Public Read Access.
I'm trying to define a tier pricing model, in which the price depends on 2 things:
The number of API calls,
The Cloud Storage Bandwidth
Meaning, I want to set some pricing in relation to the costs they are consuming on my Google Cloud account.
To give an example:
If a client does between 1 and 500.000 API calls, I'll change them 10 dollars. Between 500.001 and 1.000.000, I'll charge 18 dollars, etc, etc.
Same thing for the Cloud Storage Bandwidth, if they consume between 0GM and 10GB, it's going to cost 10 dollars. If they consume between 10GB and 100GB, it's going to cost 18 dollars, etc, etc.
How can I do it with Google Cloud? How can I know how my clients are consuming? And is there a way to share that information with them, so they are able to monitor the usage every day?
I'm thinking that measuring the API usage is not going to be THAT hard, because I can just save a value in the DB every time the user calls the API, but if there is a way to avoid it, will be good, due to Google Cloud is going to charge me for that DB write action (that I use to track the API usage).
On the other hand, for measuring the Cloud Storage, I was thinking something like this:
Let's suppose I have a Public Bucket with photos in the URL: buckets.google.com/photos.
If my client wants to get the /cats/ugly-cat.jpg photo, I can ask them to call A FUNCTION in /api/get-photo/?url=/cats/ugly-cat.jpg, so there in that Function a can track that the user just get a photo, and then I redirect the call to the real URL where the user is going to see the photo (buckets.google.com/photos/cats/ugly-cat.jpg). As you can see, this idea seems to be too ppor performant, due to it's going to charge the Function usage, the DB write, and also the Storage bandwidth usage. And even, that way doesn't track the Bandwidth. It only tracks the number of photos that the client wants to show.
As you can see, both ideas are a bit ugly, with poor performance.
There should be something already done that makes it beautiful.
Obviously, the API call (and also the photo link) may have the client API-KEY, to help to measure the usage. Something like:
functions.google.com/api/search-photos/?api-key=111, and
bucket.google.com/photos/cats/ugly-cat.jpg?api-key=111
Where 111 identifies the client 111.
So, the question: Do you know if there is a "best-known" way to do measure those usages?
I think Cloud Endpoints is the best solution for you because managing your API as you suggest might get unwieldy quickly.
Endpoints provides all the tools to control authentication, quota and cost management and a developer portal so your users can access documentation and interact with your API. It also integrates with all Cloud Platform products including Cloud Functions.
We run a website which heavily relies on the Amazon Product Advertising API (APAA). What happens is that when we experience a sudden spike in users it happens that we hit the rate-limit and all functions relying on the APAA shut down for a while. What can we do so that doesn't happen?
So, obviously we have some basic caching in place, but the APAA doesn't allow us to cache data for a very long time, and APAA queries can vary a lot so there may not be any cached data at all to query.
I think that your only option is to retry the API calls until they work — but do so in a smart way. Unfortunately, that's what everybody that gets throttled does and AWS expects people to handle that themselves.
You can implement an exponential backoff and add jitter to prevent cluster calls. AWS has a great blog post about solutions for this kind of problem: https://www.awsarchitectureblog.com/2015/03/backoff.html
I have a .Net application that uses list of names/email addresses and finds there match on Facebook using the graph API. During testing, my list had 900 names...I was checking facebook matches for each name in in a loop...The process completed...After that when I opened my Facebook page...it gave me message that my account has been suspended due to suspicious activities?
What am I doing wrong here? Doesn't facebook allow to search large number requests to their server? And 900 doesn't seem to be a big number either..
per the platform policies: https://developers.facebook.com/policy/ this may be the a suspected breach of their "Principals" section.
See Policies I.5
If you exceed, or plan to exceed, any of the following thresholds
please contact us by creating confidential bug report with the
"threshold policy" tag as you may be subject to additional terms: (>5M
MAU) or (>100M API calls per day) or (>50M impressions per day).
Also IV.5
Facebook messaging (i.e., email sent to an #facebook.com address) is
designed for communication between users, and not a channel for
applications to communicate directly with users.
Then the biggie, V. Enforcement. No surprise, it's both automated and also monitored by humans. So maybe seeing 900+ requests coming from your app.
What I'd recommend doing:
Storing what you can client side (in a cache or data store) so you make fewer calls to the API.
Put logging on your API calls so you, the developer, can see exactly what is happening. You might be surprise at what you find there.