I am trying to setup a microservice architecture on AWS, each microservice is a REST API.
Some of the services are running on ECS using Fargate and some of the services are running as a set of lambdas.
I am trying to have each api route resolve to the correct service, whether it is a ECS or Lambda based service.
I can see how it would be possible using only ECS services (with Application Load Balancer and listeners) or using only Lambdas (with an API Gateway). But I just cant seem to figure out how to mix the two together.
I have been searching relentlessly all week and I cannot find any decent documentation or an example of how to implement something similar to this.
There appears to be a limit to the number of routes for ALB or API Gateway. If I have several lambda based services there will need to be a declared path for each Lambda function and they will use up the path limit very quickly.
Should there be an intermediary step between each service and the API Gateway? For instance, each Lambda service has its own API Gateway which 'groups' those functions together. Which would mean there will be a nested set of API Gateways that the parent API Gateway routes to. This doesn't feel correct though.
Any help in the right direction would be appreciated.
Thanks
Your AWS account's API Gateway REST and Websocket routes/resources limit can be increased with a request to AWS support.
Related
I deployed few microservices into Fargate. Each microservice will have around 30 API endpoints.
I have AWS ALB which will do the path based routing to the Fargate.
I created API Gateway APIs to externally expose the APIs. API Gateway integration method is HTTP and that points to the ALB endpoint.
Is this the proper way to setup the microservices? If not, please suggest the better approach.
And also, I want to automatically import the Swagger definition into API Gateway whenever there is a change in the Swagger definition. Swagger definition is exposed under /apidocs of each microservice. How do I automate the import of swagger definition into API gateway? Is there a commonly used approach?
Keep in mind you'll need to update both API GW and ALB paths. To import new swagger definition you'll need some kind of triggered event to:
Upload new swagger definition to S3.
Create an API gateway deployment.
Perform the path edition in ALB target group(s). This one could be done by triggering a CLI or API call (or sdk).
If you use CloudFormation, you could trigger an UPDATE containing all these new changes too. If so, also keep an eye on those resources parameters that may or not require a resource replacing.
Hope it helps.
I have developed a simple microservice, REST based using Java 8 and Spring Boot2.0. It has its own REST end points which I can call using Postman and I get the response very well. Now I have doubt in understanding the design & architecture if I want to deploy the same application on AWS cloud. I want my application to behave as serverless so I want to deploy on AWS using its Lambda service.
Please assist to clear my following doubts :-
1) First, can I upload my whole application code to AWS Lambda in order to make it serverless?
2) If yes, then do I need to use AWS API Gateway (compulsorily) to invoke my Lambda function when the request passes through it?
3) If yes (point 2), then end points which are there in my original microservice code will become ineffective and will be overridden by new API Gateway end points?
My whole doubt is about end points, which end point will be used to invoke the Lambda functions?
Please assist to clarify my doubt. If there is any sample reference material then it will be really great.
Cheers
Spring Boot and AWS Lambda don't naturally go together IMO.
Lambda is pure code, it does not present itself as a HTTP Server, it is just triggered by one of the other AWS services (API Gateway, CloudWatch, S3, DynamoDB, Kinesis, SDK, etc.). The handler receives a JSON request from the calling service, and processes the request. Here is an example.
API Gateway does much of what Spring Boot provides for you. API Gateway is always online waiting for HTTP requests to come in, for which you only pay for incoming requests, you do not pay for idle (the definition of serverless IMO).
Once a request comes in, API Gateway wraps the request payload with some additional environmental data and sends it to your Lambda handler, which processes the request and returns some response to the client.
Saying that, if you don't want to restructure your service, there are a couple of options open to you:
Wrap into a Docker image and use an AWS Container Service, either using ECS or ElasticBeanstalk, neither of these are considered to be serverless.
I have not tried this, but according to AWS:
You can use the aws-serverless-java-container library to run a Spring Boot application in AWS Lambda. You can use the library within your Lambda handler to load your Spring Boot application and proxy events to it.
See links to Convert your SpringBoot project and Deploy it to AWS Lambda.
Hope this helps.
I have a Django Project deployed in AWS EC2 and have created all the REST APIs with DRF. Everything is working great. When I point public.ip.of.ec2:8000/myapi, I get the correct response and these APIs are used in Android application.
Now, day by day the API hits will be increased. So in order to maintain API requests I want to integrate my APIs with APIGateway. After a lot of reading and searching I still could not find any definitive tutorial or steps as of how to integrate APIGateway with EC2.
I've followed this but still could not integrate APIGateway with AWS Service i.e., EC2.
Please suggest a way to do this.
I have a Django Project deployed in AWS EC2 and have created all the
REST APIs with DRF. Everything is working great. When I point
public.ip.of.ec2:8000/myapi, I get the correct response
Integration type you are looking for is HTTP integration
Now, day by day the API hits will be increased. So in order to
maintain API requests I want to integrate my APIs with APIGateway
Although you can configure API caching, throttling, logging and monitoring, AWS API Gateway won't scale your backend integration. You will need to configure autoscaling on backend service as well.
As my project is going to be deployed on AWS, we started thinking about AWS API Gateway as a way to have one main entry point for all of our microservices(frankly speaking, we also would like to use by some other reasons like security). I was playing with API Gateway REST API and I had feeling that it it a bit incovinient if we have to register there every REST service we have.
I found very good option of using AWS API Gateway and lambda function as a proxy. It is described here:
https://medium.com/wolox-driving-innovation/https-medium-com-wolox-driving-innovation-building-microservices-api-aws-e9a455cc3456
https://aws.amazon.com/blogs/compute/using-api-gateway-with-vpc-endpoints-via-aws-lambda
I would like to know your opinion about this approach. May be you could also share some other approaches that can simplify API Gateway configuration for REST API?
There are few considerations when you proxy your existing services through API Gateway.
If your backend is not publicly then you need to setup a VPC and a site to site VPN connection from the VPC to your backend Network and use Lambda's to proxy your services.
If you need do any data transformations or aggregations, you need to use Lambda's(Inside VPC is optional unless VPN connection is needed).
If you have complex integrations behind the API gateway for your services, you can look into having ESB or Messaging Middleware running in your on-premise or AWS then proxy to API Gateway.
You can move data model schema validations to API Gateway.
You can move service authentication to API Gateway by writing a Custom Authorizer Lambda.
If you happen to move your User pool and identity service to AWS, you can migrate to AWS Cognito Manage Service and use AWS Cognito Authorizer in API Gateway to authenticate.
For usecases when you adopt dumb pipes (as described on martinfowler.com) AWS API Gateway is a reasonable option.
For AWS API Gateway I'd suggest to describe/design your API first with RAML or OpenAPI/Swagger and then import into AWS using AWS API Importer.
As soon as you plan to move logic in there, such as dynamic routing, detailed monitoring, alerting, etc, I'd suggest considering other approaches, such as:
Apigee
Mulesoft
WSO2
You can also host them on an EC2 within your VPC or opt-in for the hosted version. (which does have a significant pricetag in some cases)
For describing APIs you can use RAML (for Mulesoft) or OpenAPI (ex-Swagger, for Apigee and WSO2). You can also convert between them using APIMATIC which enables you to migrate your specification across various API Gateways (even AWS).
I am fairly new to AWS Lambda but sure can see the benefits of it and stumbled upon the superb framework Serverless to help me built solutions on Lambda.
I started out building solutions using AWS API Gateway but really need "internal" VPC API's and not public Internet facing API's like API GW creates.
I found that Servless indeed can expose a HTTP endpoint but I can't figure out how this is done and how the URL is created.
When I deploy the Lambda from Serverless it gives me the URL, e.g.:
https://uxezd6ry8z.execute-api.eu-west-1.amazonaws.com/dev/ping
I would like to be able to find (or create) this same http listener for already existing Lambdas so my question is how is the URL created and where is teh actual HTTP listener deployed?
You might be looking for the invoke url,
1. go to https://console.aws.amazon.com/apigateway
2. select api link (which you have deployed on aws lambda).
3. select stages in left side panel and
see the invoke url.
Adding a http listener can be done by going to your lambda function, selecting the 'triggers' tab and 'add trigger', finally selecting API Gateway - but as others mentioned this does create a public facing url.
Duh, I was in the wrong AWS logon previously so the API GW was not showing any matching Serverless API and that was why I couldn't understand how they did it...
Once I logged into the AWS account that hosts the Serverless structure I can see the API GW GET API's for the Serverless HTTP listener.