I'm looking recommended ways to extract usage data by API key for a given set of Google Cloud Endpoints APIs. The project bills customers the usage of a certain API.
My goal is to know how many times each client calls each of the ESP apis in a given month so that they can be billed based on their usage.
Example
client_a:
api-key: 12345
client_b:
api-key: 67890
ESP1:
api_esp1
ESP2:
api_esp2_foo
api_esp2_bar
Ultimately, I want to know how many times client_a used each of the available ESP APIs, and how many times client_b used them.
One (part of the) solution is to create a GCP project per client as outlined here. I didn't figure out how to go from there though.
Related
I am new to Google Cloud Platform. I built an app that uses Google Sheets. I have several scripts for the sheet.
I am having an error - invoking too many times. URLfetch error., so I know I am calling the url too many times. I need help:
How do I find out exactly how many url fetches I am calling.
If i need to add more calls, what Google suite subscription do I
need to get?
Thanks
Looks like you are developing Google Apps Script
You can check the quota limits of App Script in the link,
It also points out the quota of URL Fetch calls is 20,000 / day which you already knew.
To your question:
To my acknowledgment, there's no way to check how much daily quota has been used.
You can consider at least G Suite basic for 100,000 / day quota.
If you need more than 100,000 / day, you'll need at least G Suite Business to apply for Early Access flexible quota.
I have an App that consumes my own API (Google Cloud Functions) and my own Storage (there are images).
Now, I have a couple of clients, that wants to consume my API and my Storage (A Google Cloud Bucket).
The Cloud Storage is a bucket that contains a lot of photo that have Public Read Access.
I'm trying to define a tier pricing model, in which the price depends on 2 things:
The number of API calls,
The Cloud Storage Bandwidth
Meaning, I want to set some pricing in relation to the costs they are consuming on my Google Cloud account.
To give an example:
If a client does between 1 and 500.000 API calls, I'll change them 10 dollars. Between 500.001 and 1.000.000, I'll charge 18 dollars, etc, etc.
Same thing for the Cloud Storage Bandwidth, if they consume between 0GM and 10GB, it's going to cost 10 dollars. If they consume between 10GB and 100GB, it's going to cost 18 dollars, etc, etc.
How can I do it with Google Cloud? How can I know how my clients are consuming? And is there a way to share that information with them, so they are able to monitor the usage every day?
I'm thinking that measuring the API usage is not going to be THAT hard, because I can just save a value in the DB every time the user calls the API, but if there is a way to avoid it, will be good, due to Google Cloud is going to charge me for that DB write action (that I use to track the API usage).
On the other hand, for measuring the Cloud Storage, I was thinking something like this:
Let's suppose I have a Public Bucket with photos in the URL: buckets.google.com/photos.
If my client wants to get the /cats/ugly-cat.jpg photo, I can ask them to call A FUNCTION in /api/get-photo/?url=/cats/ugly-cat.jpg, so there in that Function a can track that the user just get a photo, and then I redirect the call to the real URL where the user is going to see the photo (buckets.google.com/photos/cats/ugly-cat.jpg). As you can see, this idea seems to be too ppor performant, due to it's going to charge the Function usage, the DB write, and also the Storage bandwidth usage. And even, that way doesn't track the Bandwidth. It only tracks the number of photos that the client wants to show.
As you can see, both ideas are a bit ugly, with poor performance.
There should be something already done that makes it beautiful.
Obviously, the API call (and also the photo link) may have the client API-KEY, to help to measure the usage. Something like:
functions.google.com/api/search-photos/?api-key=111, and
bucket.google.com/photos/cats/ugly-cat.jpg?api-key=111
Where 111 identifies the client 111.
So, the question: Do you know if there is a "best-known" way to do measure those usages?
I think Cloud Endpoints is the best solution for you because managing your API as you suggest might get unwieldy quickly.
Endpoints provides all the tools to control authentication, quota and cost management and a developer portal so your users can access documentation and interact with your API. It also integrates with all Cloud Platform products including Cloud Functions.
This conceptual question has crept into my mind after becoming more familiar with AWS. In general, I’m curious if there is a best-practice and/or convention as to when an API provider should group endpoints into a new, separate API (vs. lumping the endpoints into an existing API).
To illustrate, let’s say a Service creates digital wallet coupons on behalf of Manufacturers, to be redeemed by Consumers at a bunch of Mom & pop stores — some of the activities the Service might engage in include:
Receiving data from the Manufacturers (in order to build the digital coupons)
Providing a mechanism for Consumers to find and download coupons
Providing a way for the Mom & pop stores’ payment terminals to validate the coupons
And, oh by the way, the Service might also be required to ...
Implement a variety of endpoints, based on technologies involved (e.g., PassKit with Apple Wallet)
So?
With AWS, it’s easy to modularize one’s backend (e.g., have an RDS instance for the database, run a few lambda functions for microservices, etc.) and load balance it all. API Gateway adds to this in that each endpoint can point to different things (lambda functions, EC2 instances via HTTP proxy, etc.).
Consequently, one approach might be to define one API in AWS API Gateway and have all the endpoints underneath it:
API: “Master”
/coupon
POST = create a new one (for Manufacturers)
PUT = update an existing one (for Manufacturers)
GET = retrieve one (for Consumers)
/coupon/validate
POST = verify it’s still valid (Mom & Pop store use-case)
/apple-wallet
/{version}
/passes
... per documentation
/devices
... per documentation
But would it make more sense for the Service to shave off the /apple-wallet endpoint and create an entirely new, separate API?
Alternatively, if the Service was going to publish documentation for public developers to use, would it make sense to move the Manufacturer-relevant endpoints into a separate API altogether?
Since AWS makes the effort of splitting endpoints so simple via API Gateway, are there any standard practices for when you should (or should not)?
Thank you for any insights / opinions!
My two cents. Think about your end-user for your APIs. You will have different developer end-users for each API set.
Your ideal situation will have each developer end-user only seeing the APIs that are relevant to them. So you should split your APIs into different Gateways according to the end-users
In the theoretical situation you describe:
Create an API for Manufacturers so they can integrate with you to create coupons. If you do the integration internally it will be the corporate sales and presales people who talk to the manufacturers
The users for the Service and End User coupons might end up being the
same app developers that create an interface for both stores and
users. So create a coupon API for them
Separating both should also give you security benefits as you will protect the knowledge of your Manufacturer API from the users who might try to hack it
I want to develop a system for finding possible routes for delivery drivers for a food delivery service and wanted to know what API options are out there. I am trying to find an API I can call from the backend that can do geocoding and find routes and times between points so I can calculate possible delivery address grouping for drivers. Most APIs I have seen have usage limits or say they need to be used with maps while I want to be able to make calls from a backend server without a map. I have seen googles and mapquests APIs but are there others maybe I am not aware that allow for a large number of calls?
I am working on a civics related project and I need to be able to display all the properties in the City of Philadelphia on a map, so I'll need to get the latitude & longitude for all 580,000 properties. (Only once)
Most APIs like Google/Yahoo have limits of 5,000 per day, and even BatchGeo has a similar limit.
Is there a way I can do a one-time geocoding of all these addresses?
You can find a list of free and paid geocoding services at USC site.
Also check Microsoft's Geocode Dataflow API, it allows up to 200,000 entries / 300 Mb and takes up to 14 days.
Another possibility to combine several services at once: use 4 services that allow 5,000 entries a day and you'll finish your task in a month.
You can use Map Quest of Cloud Made.
I have created a small utility to help compare these API's.
The utility is hosted at below url:
http://ankit-zalani.appspot.com/GeoCode/index.jsp
Tobias, I work for an address verification (and recently, geocoding) company called SmartyStreets.
Many services have usage restrictions based on volume and license agreements which prevent users from storing the results of geocoding queries. There are some vendors, however, which don't have limits or restrictions like that.
I would recommend something like LiveAddress which will not only geocode the addresses but also perform CASS-Certified verification to make sure your addresses are correct before giving you potentially faulty coordinates. You can run 580,000 or even millions at a time in a few minutes, and we allow you to store your results.
Hope this helps. If you have any more questions about addresses, I'll personally assist.
This thread is pretty old by now, but there have been some developments in recent years making bulk geocoding very cheap. My favorite option is to just obtain a geocoding server on AWS ( google: geocoding on aws), many options there, some free some with low hourly rates (total cost depends on the server you choose, of course.)