I am certain I need to use an API Gateway, but I can't understand the main differences between the different tools in my use case scenario.
Currently, I have multiple services (DBs, Mobile App, Web App and some additional systems. Consider there are 15 different services) that communicate with each other through REST APIs. This is difficult to manage and test, so I would like to change the architecture into something more like what Netflix is doing with Zuul.
Ideally, the services don't know about the other services. They send a request to a specific endpoint (the API Gateway). Then, the API Gateway interacts with the necessary services and sends the response back.
Here is one example in practice: a service sends a request to a custom (endpoint) connector, the request is parsed, broken down into smaller requests that are sent to other services (that own the specific content requested), get the content back in the response, gather all the responses, create a final response with all the content gathered, send the response back to the first service that sent a request.
I need high availability, scalibility, fault tolerance, the ability to monitor and test all services in one place, ability to do canary testing, easy to add new services and manage the older services. I value open source software and mature software. Should run off premise.
The best solutions that I believe would solve my problem are: WSO2, Apigee, Zuul and Amazon API Gateway. I don't know which is more appropriate for my use case. I have looked at others, but I haven't found any advantages in features or cost against these 4.
Thank you for your feedback regarding advantages and disadvantages regarding these technologies! Other suggestions are also welcome!
Notes:
Not all of my services are on AWS, but some are.
The system needs to handle peaks with tens of thousands of requests per minute that happen regurarly, but never continuasly.
You can also consider the Forum Sentry API Security Gateway from Forum Systems (I work for Forum Systems).
Based on your example use case, if each of the "smaller request" services are using the same protocols (e.g. HTTPS), message formats (e.g. JSON) and security characteristics (TLS, authentication, etc.) then the solution should be relatively straight forward.
If each service is using different identity or message formats, say, then your API Gateway solution will also need to have strong capabilities around identity and message conversion. For example, one small request may require a BasicAuth header to authenticate to the service, whereas another small request may require a SAML assertion.
As you have landscape of different microservices, wherein each microservice has its own business context and is accessible by REST endponit. In this case, your client need not to aware about each microservice and thus API gateway comes into picture, using which you can have one entry point to all microservices landscape.
There are different API gateway solutions available like you said apigee, apiman etc. These framework gives some basic implementation of features required in API gateways like request throttling, monitization of request calls, authentication handle, centralized security etc.
Netflix's Zuul provides the filters which you need to implement yourself. So, if you are using Zuul, you have to implement all the features which you want to put in your API gateway by yourself.
I hope this explanation helps!!!
Related
Most of our bespoke applications in AWS are containerized .NET Core apps and so far we haven't needed to build any SOAP APIs - virtually all of the APIs that we've delivered (whether exposed publicly via AWS API Gateway or internal only) are JSON RPC following a RESTful architecture/design principles.
We also have a third party COTS product deployed into our AWS account that produces SOAP webhooks and we need to receive and process these SOAP webhooks produced by the third party COTS product.
Ideally we don't want to build a bespoke SOAP API / translation layer - even if WCF is now supported in .NET Core / 5+, I'd much rather we just stick with HTTP/REST and .NET Core WebAPIs which is well understood by our delivery teams. Ideally there is a way e.g. a native AWS service, that would allow us to translate the SOAP request to a HTTP/REST request. Does anyone know if there's a native AWS service with these capabilities?
Cheers.
The translation would be something like the following
HTTP method GET/POST + SOAP operation verb endpoint
POST /soap/cancelOrder <message>{id}</message>
<=>
HTTP method GET/POST/PUT/PATCH/DELETE + REST resource noun endpoint
DELETE /rest/orders/{id}
Without a naming convention it is hard to translate. Another issue here, that real REST services return hypermedia with a flexible structure, while SOAP services return SOAP message with a ridig structure defined in WSDL. Yet another issue that SOAP is stateful, while REST is stateless.
Maybe in edge cases it is possible to automatically translate for example by services which do only basic CRUD without any real business logic.
I have an application that uses Google Cloud Translation API for translating contents from a source language to the languages used by the different users.
Since there are several clients for the API, I would like to distinguish the request numbers for different clients, like making a distinction between Android and iOS clients.
There's a dashboard in Google Cloud Translation API Overview page that contains the Traffic chart with a By credential option, which should be able to distinguish the request numbers by their credentials (in our case, API keys.) Unfortunately, it doesn't, the only option available there being Unspecified.
Moreover, even if I change the filter of credentials to No selection, the Traffic map is still the same!
I have also attached the project's credential list, where only API keys are used.
Please help me how to know the request numbers by different API keys, thanks.
Update:
Here are how I send requests to Google Cloud Translation API.
I tried with Postman and Swift code in iOS (what I actually do in my project.) Sent GET/POST requests to the API with API keys and POST requests with service account token, but neither API keys nor service account shown in request logs as the first picture.
Postman
Request with API key in GET
Request with API key in POST
Request with service account token in POST
Swift code
Code
Response
I have also done some requests to Translation API using different credentials. After doing the requests, I have checked my dashboard and I encountered the same situation as you, with all requests marked as Unspecified when choosing the By credential option.
As it turns out, there is an issue related to this situation, as this is not the expected behavior. This has been notified and it will be sorted out by the Google team. You can keep track on any updates related to this issue here. If you click the star button on this site, you will get email notifications whenever any progress has been made. Please bear in mind that it may take some time for this issue to be resolved.
In the meantime, you may consider tracking the client information through the statistics of the applications that make requests to the API, if possible. Thanks for your help on finding this issue.
I am implementing RESTful web services using NetBeans (JAX-RS) and Glassfish server to expose database resources.
I want to implement varying levels of security depending upon the users requirement (maybe choice provided to user). From various discussions regarding REST security, I understand there are different ways (given below) in which security can be implemented. How can I implement a security solution as a service which will be invoked depending upon the context.
TLS (HTTPS) +
1. HTTP Basic
2. HTTP Digest
3. two-legged OAuth
4. a Cookie-based approach
5. client certificates
6. Signed requests using HMAC and a limited lifetime
Thanks in advance
You can try to implement security by using Policies at Runtime.
Policy gives the flexibility to implement different security scenario (userauth, oauth,saml etc) depending upon the context.
We implemented the same in web service using Apache CXF and WS-SecurityPolicy.
For rest service I think similar concept can be used.
We have a web service that we want to protect against malicious attacks to make lots of requests to effectively get all the data out of it.
We have some level of protection with tokens that signed and exchanged, but a determined attacker could get these tokens and replay them on requests to our web service.
So I'm thinking the only protection is at the server level. One thought was to implement a request threshold in a specific time interval them block for duration of time, that grows if subsequent requests are made during the blackout period. After repeated attempts, completely blacklist.
However, I hate the idea of custom rolling our own solution and even using IPs at all since one bad user behind a proxy gets everyone else behind that proxy blocked.
What are the best practices for protecting a web service?
Update: To clarify, this is a general question about protection a web service against mass harvesting of data.
There are several solutions out there, but this one is nice and secure, IMO. Use X.509 certificate in SOAP headers for authentication at the message level. Here are a few links on the subject:
http://msdn.microsoft.com/en-us/library/ff648643.aspx
http://msdn.microsoft.com/en-us/library/ff648129.aspx
http://www.aspfree.com/c/a/VB.NET/Securing-Web-Services-with-X509-Certificates/
How can I configure WCF to use x509 certificates over the internet?
You don't tell us much about the data/service, but I once worked on the design of a system that would detect attacks by looking at patterns of the queries.
We currently have a SOAP based web service that our in house applications use to authenticate users. Basically, they send a SOAP request with the username and password. The web service authenticates their credentials against our data store and returns user information if the authentication is successful. The web service is secured using BASIC authentication and SSL.
We need to make modifications to this web service and I was considering re-writing it as a REST service. The REST services I have created in the past have been fairly simple and had no need for security. I have never created a REST service that used sensitive information, so I have a couple of questions / concerns:
First, is there a best practice for sending sensitive query parameters (user credentials) to a REST service securely? I can still use the BASIC authentication and SSL.
Second, if I send a query to a REST service using POST, is it still considered RESTful, or is GET required for REST queries?
You can use SSL and Basic authentication with REST web services as well.
HTTP GET is usually used for data retrieval (queries) but you can use HTTP POST as well. GET is especially useful if you can use any type of HTTP caching. POST is usefull if you need to transfer a lot of data to define your query or if your web service operation expects some complex data format instead of simple arguments.
Instead of doing the authentication via REST, you might also consider a networked authentication protocol to use in conjunction with web services. Technologies like Kerberos and OAuth were designed for these sorts of use cases.
To answer your questions, however:
REST encourages you to leverage HTTP and related protocols, so using SSL and BASIC authentication is quite appropriate.
REST encourages the use of not just GET and POST, but even other HTTP "verbs" such as PUT and DELETE. Use GET only for idempotent operations with no side-effects.
Going from SOAP to REST is taking a step backward as far as security goes.
As far as best practices:
Don't roll your own security. Use a framework or existing library that has been peer-reviewed and tested.
Don't pass unencrypted static keys. If you're using HTTP Basic and sending it across the wire, encrypt it.
Ideally, use hash-based message authentication code (HMAC) because it's the most secure.
Why REST security doesn't exist