I have an application that uses Google Cloud Translation API for translating contents from a source language to the languages used by the different users.
Since there are several clients for the API, I would like to distinguish the request numbers for different clients, like making a distinction between Android and iOS clients.
There's a dashboard in Google Cloud Translation API Overview page that contains the Traffic chart with a By credential option, which should be able to distinguish the request numbers by their credentials (in our case, API keys.) Unfortunately, it doesn't, the only option available there being Unspecified.
Moreover, even if I change the filter of credentials to No selection, the Traffic map is still the same!
I have also attached the project's credential list, where only API keys are used.
Please help me how to know the request numbers by different API keys, thanks.
Update:
Here are how I send requests to Google Cloud Translation API.
I tried with Postman and Swift code in iOS (what I actually do in my project.) Sent GET/POST requests to the API with API keys and POST requests with service account token, but neither API keys nor service account shown in request logs as the first picture.
Postman
Request with API key in GET
Request with API key in POST
Request with service account token in POST
Swift code
Code
Response
I have also done some requests to Translation API using different credentials. After doing the requests, I have checked my dashboard and I encountered the same situation as you, with all requests marked as Unspecified when choosing the By credential option.
As it turns out, there is an issue related to this situation, as this is not the expected behavior. This has been notified and it will be sorted out by the Google team. You can keep track on any updates related to this issue here. If you click the star button on this site, you will get email notifications whenever any progress has been made. Please bear in mind that it may take some time for this issue to be resolved.
In the meantime, you may consider tracking the client information through the statistics of the applications that make requests to the API, if possible. Thanks for your help on finding this issue.
Related
I just want to make sure I've got the overall idea down and don't create an implementation that violates basic security best practices. Can somebody please check my understanding?
As I understand it, a user can log in to my application and the authentication server REST API can return a JWT that is signed, but NOT encrypted. Inside that token I can have claims inside the payload that my client application can access, such as features the user can use on the application. That way my client side website can change functionality based on the user privileges and roles. The JWT claims in the payload are NOT sensitive. They will be strings representing categories for images or documents, things like that.
When the user wants to get additional content (like a document, image, or video) from other REST API endpoints, they submit the JWT along with the GET request. My API can then verify the signature of the JWT and grant API access if appropriate.
This last part is what I'm most unsure about. My intent is to use another authorization server API endpoint which takes the JWT in a POST request and returns a simple "valid/invalid" response. My thought is that my Content Delivery Network (CDN) can use this API to verify that the JWT in possession is validly signed. I believe (and maybe here is where I'm goofing up) that the authorization server API can be publicly accessible to ease use by my other microservices. This seems fine because I'm just giving a boolean pass/fail on the validity of the token so I don't see any need to hide or obfuscate the API. I question this because I know AWS has backend stuff to validate and authorize for API calls but I like the simplicity of just using REST APIs for everything for my first implementation; to maintain simplicity.
So in summary:
1.) Signed, unencrypted JWT with non-sensitive user roles/privileges.
2.) Unencrypted so client side webpage can selectively render content based on user.
3.) Public authorization API that anybody could technically use so that my CDN (and other microservices) can validate JWTs.
Any major issue with this approach? Have I committed any technical sins?
Thank you so much in advance for your time on this matter.
Okay, I think I've sorted this out myself after finding a great video tutorial on this stuff. Below is the video I watched:
https://www.youtube.com/watch?v=_XbXkVdoG_0
I had some misconceptions and this video sorted them out. It appears that what I described in my question is precisely how JWT should be used.
I have a django application that puts a task in a queue. Another service is used to read that queue and process some files. At the end I need to save the processed files in the database managed by the django application.
I do not want to give the microservice access directly to the database, since I want the responsibility only to be to process the files.
So I wanted to post the changes to django using HTTP request. The problem is that I do not have any authorization at the time, even though I know that HTTP from this type of machine is to be accepted.
For the django application I use JWT as an authorization token. How is the best way to approach this type of problem? Maybe just send a token together to the queue? But how to make such token? It's not certain when the process will be executed..
When you really think about it, there is no need for your internal services to authenticate themselves if they are in the same network.
In that case - You can put Django behind an API gateway (don't write your own, find an open source highly rated project). Then you can control via this gateway which end point is allowed by which traffic source. Then you can easily control end points that are specifically for internal services and which end points need authentication by an external entity.
If they aren't in the same network (which means they are separated by the great gulf of the cloudy net) then the usual way two machines communicate is with an API key. In that case, you can configure your services with symmetric keys, or private/public pair, it doesn't really matter. Machines can be trusted with secret keys. Why would you need to send the token in the queue? If the service is allowed to post results to Django, its allowed to do so for all requests, so it needs to be configured with an API key that tells your API that it is allowed to post processed files.
Trying to understand how to use Cognito and API Gateway to secure an API.
Here is what I understand so far from AWS documentation and the Cognito user interface:
Clients
www-public - public facing website
www-admin - administrators website
Resource Servers
Prices - for this simple example the API will provide secured access to this resource.
Scopes
prices.read
prices.write
Again, very simple permissions on the API. Public www users can read prices, administrators can write them.
API Gateway
GET /prices - accessible to authenticated users that can read prices.
POST /prices - only accessible to administrators
Users
Administrators - can update prices via the POST method.
Non-administrators - cannot update prices.
Based on this...
Each client will request the scopes it is interested in. So for the public www site it will request prices.read and for the administration site both prices.read and prices.write.
The API Gateway will use two Cognito Authorisers, one for each HTTP Verb. So the GET method must check the user can read prices and the POST method that they can write prices.
The bit I don't see is how to put all of this together. I can make the clients request scopes but how do they now connect to user permissions?
When the token is generated, where is the functionality that says "Ok, you requested these scopes, now I'm going to check if this user has this permission and give you the right token?"
I understand that scopes ultimately related to the claims that will be returned in the token.For example, requesting the profile scope means that the token will contain certain claims e.g. email, surname etc.
I think based on this that my permissions will ultimately end up being claims that are returned when specific scopes are asked for. The fact that the two clients differ in what they request means that the prices write claim an never be returned to the public www client. It would never issue a token if the prices.write claim was requested.
What I can't see is where this fits in Cognito. There is the option to put users into groups but that is pretty much it. Likewise, there is nothing (that I could see) to relate scopes to claims.
I'm coming from a .Net and Identity Server background. Certainly in the last version of Identity Server I looked at there was a handler method where you would work out which claims to put into a token. I guess this would map into one of the custom handler lambda functions in Cognito. From there this would need to query Cognito and work out what claims to issue?
The final piece of the puzzle is how the API Gateway checks the claims. Can this be done in API Gateway or does the token need to be inspected in the Lambda function I will write to handle the API Gateway request?
Certainly using Identity Server and .Net there was a client library you would use in the API to inspect the claims and redact permissions accordingly. Guessing there is something similar in a Node JS Lambda function?
A few assumptions there as I'm basically in the dark. I think the basics are there but not sure how to connect everything together.
Hoping someone has figured this out.
I am certain I need to use an API Gateway, but I can't understand the main differences between the different tools in my use case scenario.
Currently, I have multiple services (DBs, Mobile App, Web App and some additional systems. Consider there are 15 different services) that communicate with each other through REST APIs. This is difficult to manage and test, so I would like to change the architecture into something more like what Netflix is doing with Zuul.
Ideally, the services don't know about the other services. They send a request to a specific endpoint (the API Gateway). Then, the API Gateway interacts with the necessary services and sends the response back.
Here is one example in practice: a service sends a request to a custom (endpoint) connector, the request is parsed, broken down into smaller requests that are sent to other services (that own the specific content requested), get the content back in the response, gather all the responses, create a final response with all the content gathered, send the response back to the first service that sent a request.
I need high availability, scalibility, fault tolerance, the ability to monitor and test all services in one place, ability to do canary testing, easy to add new services and manage the older services. I value open source software and mature software. Should run off premise.
The best solutions that I believe would solve my problem are: WSO2, Apigee, Zuul and Amazon API Gateway. I don't know which is more appropriate for my use case. I have looked at others, but I haven't found any advantages in features or cost against these 4.
Thank you for your feedback regarding advantages and disadvantages regarding these technologies! Other suggestions are also welcome!
Notes:
Not all of my services are on AWS, but some are.
The system needs to handle peaks with tens of thousands of requests per minute that happen regurarly, but never continuasly.
You can also consider the Forum Sentry API Security Gateway from Forum Systems (I work for Forum Systems).
Based on your example use case, if each of the "smaller request" services are using the same protocols (e.g. HTTPS), message formats (e.g. JSON) and security characteristics (TLS, authentication, etc.) then the solution should be relatively straight forward.
If each service is using different identity or message formats, say, then your API Gateway solution will also need to have strong capabilities around identity and message conversion. For example, one small request may require a BasicAuth header to authenticate to the service, whereas another small request may require a SAML assertion.
As you have landscape of different microservices, wherein each microservice has its own business context and is accessible by REST endponit. In this case, your client need not to aware about each microservice and thus API gateway comes into picture, using which you can have one entry point to all microservices landscape.
There are different API gateway solutions available like you said apigee, apiman etc. These framework gives some basic implementation of features required in API gateways like request throttling, monitization of request calls, authentication handle, centralized security etc.
Netflix's Zuul provides the filters which you need to implement yourself. So, if you are using Zuul, you have to implement all the features which you want to put in your API gateway by yourself.
I hope this explanation helps!!!
We're looking to replace an existing API manager with wso2 and one of the features of the other platform is that we can identify a 'user' of the API at run time and have the throttling work.
The 'user' is not OAuth identified, rather they authenticate via the API and a session id is returned (so they've never registered at a 'store'). This session id is then used to setup the throttling at the API Manager. Additionally the other tool has code to look for the user logging in and using that id in the throttling. so if a user tries to login too many times per hour the API manager blocks the request in addition to too many requests for a logged in user per hour. The combinations of login attempts, API calls etc. are summed into the throttle. (All this was implemented by their services team years ago)
The main reason we need this is we don't want to force our old clients to go to OAuth immediately but want much more visibility, reporting and throttling.
Thoughts on how to do this with wso2? I see where we can add our own Handlers to the API to figure out session ids, login ids etc. but I don't see where to create the logic to do the throttling.
Thanks,
Chris
Login to Carbon and edit the tiers.xml (/_system/governance/apimgt/applicationdata/tiers.xml)。You can found the section <throttle:ID throttle:type="ROLE">Unauthenticated</throttle:ID>