Does API gateway support using consistent hash on a request parameter to select the upstream backend to route the request to?
Something like:
https://www.nginx.com/resources/wiki/modules/consistent_hash/
I want all requests matching a certain criteria, identified through the hash of request, to be sent to same backend server. For example, if I have 5 different backend servers and have 1 million end users send requests wit 5 different IDs in the url, then I want each backend server to process requests with just 1 ID.
I'm not aware of this option on API Gateway directly.
However, quite in complicated way, you can integrate API Gateway with the ALB. Then, in ALB, you can configure multiple listeners with rules requiring the desired hashes in request query parameter. For each of those listeners you will assign a target group with the backend server instance.
See documentation for details
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-private-integration.html
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-listener.html
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html
Related
I'm looking after solution where AWS Api Gateway changes method endpoint Url dynamically.
I am familiar with stage variables and in Integration request I can change endpoint per method like (https://${stageVariables.Url}/api/DoSomething).
What I need is that information how parse endpoint is included in requests.
https://${RequestData.Url}/api/DoSomething
I have same Api in different locations and to implement centralized Api keys and logging services I try to forward all traffic through this one Api Gateway.
After first request client gets its endpoint information, but I don't know how to solve that clients next requests to Gateway should forward to that endpoint which client get earlier.
I got an answer from AWS support. They told that I have to make a lambda function to process all requests or just use Stage variables.
I have a custom domain set up in AWS API Gateway. My intention is to use "API mappings" to send traffic for different API versions to their respective API Gateways, e.g.:
GET https://example.com/v1/foo is sent to an API gateway "APIv1" ($default stage) via an API mapping on the custom domain with path="v1".
GET https://example.com/v2/foo is sent to an API gateway "APIv2" ($default stage) via an API mapping on the custom domain with path="v2" (not shown)
The HTTP APIs themselves are configured with a single route /{proxy+} and an integration that sends requests to a private ALB:
This setup works fine as far as routing traffic goes, but the problem is that when the request makes it to the actual application, the routes the application receives are like /v1/foo instead of just /foo, which is what the app is expecting.
I've played around with different route matching and parameter mapping (of which I can find almost no examples for my use case) to no avail.
I could change my app code to match the routes that AWS is sending, but the entire point of this was to handle versioning using my AWS stack and not app code. Do I have another option?
If you create a resource called /foo and the proxy resource inside it, when you set integration you can define which path to pass and the {proxy} will have just the part after /foo, ignoring the v1 entirely.
See an example below.
In this case it is ignoring everything before v1 and it is also rewriting the integration to /api/{proxy}.
It will receive a request as GET https://example.com/abc/xyz/v1/foo and will forward to backend as GET https://example.com/api/foo.
Update
It can't be done via VPC Link, but we can use public ALB almost like private, like the explanation below.
It explain about CloudFront, but the same is valid for API Gateway.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/restrict-access-to-load-balancer.html
It's totally possible. You just need to use parameters mapping for this. Using the AWS UI it would be:
I am new to Lambda so I would like to understand how the following scenario can be deployed:
Lambda connected to API gateway ( which in turn connected to a reverse proxy)
The request from API gateway to lambda needs to be routed to 3 different ALBs each in different (VPCs) private subnets.
What configuration changes I need to bring in to achieve this apart from writing Lambda function (using python) to rewrite the urls?
It would be nice if someone can explain the message flow here.
Thanks
Abhijit
This to me seems to be multiple issues, I'll try to break down whats trying to be achieved.
Using ALBs with API Gateway
There are many options for how API Gateway can use load balancers to serve http traffic. The solution really depends on which type of API Gateway you are trying to use.
Assuming your API is either REST or WebSockets you are left with 2 choices for enabling HTTP traffic inbound to a load balancer:
Directly as a HTTP or HTTP_PROXY request, listing publicly accessible hostnames to which API Gateway will forward the traffic.
If you want to keep transit private then your only option is to create a network load balancer and make use of VPCLink to create a private connection between API Gateway and your Network resource.
If you're creating a HTTP API (sometimes referred to as API Gateway v2) then you can make use of direct connection to a private ALB, however be aware that at this time HTTP API does not support all the features of REST APIs so you would want to compare feature sets before doing this.
Using multiple load balancers to direct traffic
You determine the value per each resource/method combo, for example POST /example would be assigned its target endpoint, but only one.
My suggestion would be to make use of stage variables if you're using a REST API to specify any endpoints that you're forwarding traffic for the following reasons:
Prevents mistyping of domain names
Allows quick replacement of a hostname
Provides functionality for canary deployments to shift traffic proportionally between 2 variable names (these could be anything as long as the type is the same e.g. Lambda to another Lambda, not Lambda to a load balancer).
Using a Lambda to redirect
Technically a Lambda can perform a redirect by return a response using the below syntax
{
statusCode: 302,
headers: {
Location: 'https://api.example.com/new/path',
}
}
However be aware this will change the request to become a GET request, this will also remove the payload of the body request when the redirect occurs. Additionally you would need to set this up for every resource/method combo that you wanted to redirect.
There are 2 options that you have available to get around these issues, both involve using CloudFront combined with a Lambda#Edge function.
The first solution can act as a workaround for the request type changing, in the Origin Request event you could modify the Request URI property to match the new URI structure. By doing this your clients would be able to use the API still, whilst you would notify them of the depercations to certain paths that you were migrating.
The second solution acts as a workaround for the need to add redirects to each resource/method combo which can create a lot of mess of methods just for redirects. You could create a Lambda#Edge function to perform the same redirect on an Origin Response event. You could create mappings in your Lambda function to work out which URL it should redirect to.
There are many great examples on the Lambda#Edge example functions page
AWS comes with a service called Application Load Balancer and it could be a trigger to a lambda function. The way to call such a lambda function is by sending an HTTP/HTTPS request to ALB.
Now my question is how this is any different from using the API Gateway? And when should one use ALB over API Gateway (or the way around)?
One of the biggest reasons we use API gateway in front of our lambda functions instead of using an ALB is the native IAM (Identity and Access Management) integration that API GW has. We don't have to do any of the identity work ourselves, it's all delegated to IAM, and in addition to that, API GW has built-in request validation including validation of query string parameters and headers. In a nutshell, there are so many out of the box integrations what come with API GW, you wind up having to do a lot more work if you go the route of using an ALB.
It seems that the request/response limit is lower when using ALB, and WebSockets are not supported:
The maximum size of the request body that you can send to a Lambda
function is 1 MB. For related size limits, see HTTP Header Limits.
The maximum size of the response JSON that the Lambda function can
send is 1 MB.
WebSockets are not supported. Upgrade requests are rejected with an
HTTP 400 code.
See: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html
Payload limit with API Gateway is discussed here: Request payload limit with AWS API Gateway
Also the article already mentioned by #matesio provides information about additional things to consider when choosing between ALB and API Gateway.
Notable tweet referenced in the mentioned article:
If you are building an API and want to leverage AuthN/Z, request
validation, rate limiting, SDK generation, direct AWS service backend,
use #APIGateway. If you want to add Lambda to an existing web app
behind ALB you can now just add it to the needed route.
(From: Dougal Ballantyne, the Head of Product for Amazon API Gateway)
API gateways usually are richer in functionality than Load balancers. In addition to load balancing, API gateways often capable to do the following:
Content based based routing (some calls to v1 and some calls to v2 and so on, based on certain criteria)
IAM related functionality (eg: access validation )
Security (eg: SSL offloading, DDOS attack prevention, security credentials translation - eg: translating particular type of token to another, etc)
Payload translation (eg: XML to Json, etc)
Additionally, API gateways may be available in appliance form - and appliances are usually of low-latency, far more secure, etc.
I am not aware of specific features of AWS API gateway, but the above ones are general features of any API gateway. Nevertheless, when you have an option to use either LB or API gateway to offer a service on internet, API gateway is usually a better option, unless there are specific reasons to choose otherwise.
I'd like to add a default throttled API key for unauthenticated requests to prevent abuse.
How would I do this in API Gateway?
EDIT
To make it clearer what I need, how do I transform a request in API Gateway? Is this possible?
I would say using Cognito is the best way of authorizing API gateway.
If you want a default API key then you can go for custom API gateway authorizer. Please have a look on official documentation for the same here
You need to store the API Key in the Server Side of your application and shouldn't expose it to the Client Side (Although API Key is not considered as a security token, it can be used by malicious party to call your API).
There are couple of options you have based on the nature of your application consuming the API.
If it is a single page web application where front-end is hosted in S3, you can use AWS CloudFront to store the API Key in headers and forward it to the API Gateway, while also serving the frontend through the same CloudFront distribution. This will also remove the cross origin resource sharing problem between your web application and API Gateway.
If you have a web server, you can store the API Key at Web Server and use to proxy request to the API Gateway while setting the API Key header value.
Note: Don't use API Key for authentication which is not recommended.
This is how I would solve it.
Create Usage Plan with the throttle, burst and max limit on the request allowed.
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/APIGateway.html#createUsagePlan-property
API Key:
Create API Key (createApiKey) and associate it (createUsagePlanKey) with Usage Plan already defined. That will allow the limit defined for the requests received.
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/APIGateway.html#createApiKey-property
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/APIGateway.html#createUsagePlanKey-property
Have a separate lambda to monitor the Generated API-Keys and cleanup once it is expired, so you will not flood API-Gateway with unused keys.
If you take it to CloudFront, you can create Self Signed URL, that will be valid for a given period of time. After that time limit URL will be invalid. This is to keep yourself time-limited for the user, so within the given timelimit, what resource they can access.
One more usecase, we worked on, you can authenticate the user only on certain urls with custom Authorizer. Any other urls that get invokes, will return unauthorized without any additional code.
Hope it helps.