I know application load balancers are new in AWS, and discussions (help) are scarce up-to now.
I have a few api containers (docker) running in EC2 Container Service (ECS). I can take advantage of application load balancers to manage routing on an application level rather than a network level. This is exactly what ECS has lacked up until now.
Getting to the point...
I'm trying to get to a point where the load balancer will detect the pattern in the request url and route the request to the correct container, but route the request without the pattern included.
For example:
http://elb.eu-west-1.elb.amazonaws.com/app1/ping
Should route request '/ping' to the app1 container
http://elb.eu-west-1.elb.amazonaws.com/app2/ping
Should route request '/ping' to the app2 container
etc...
Each app has it's own target group and corresponding pattern: /app1*, /app2*
the problem
I can successfully get the a request to '/app1/ping' to route to the app1 container however the request hits the container as '/app1/ping' (obviously) but I only need '/ping' to hit the container. '/app1' is irrelevant to the container.
Any ideas how I can achieve this?
Application Load Balancers do a couple of things very well, but there's an awfull lot they do not do. This is true for a lot of AWS services (e.g. SQS just recently, after almost a decade got FIFO support) and you can either love or hate this.
Your use case seems to fit the AWS API Gateway very well, which is a service that can be used to map certain external endpoints to certain internal endpoints (and a lot more...). There's even a blog post on the AWS blog about how to use Application Load Balancing with the EC2 Container Service and the API Gateway together.
Related
Need some serious help here, thanks a lot in advance !
I need to deploy a scalable 3 tier web application on AWS and I am having some doubts/trouble understanding the best practice to design the architecture.
NOTE: As per my understanding, all the backend requests are requested through the browser, after the Frontend server serves html/css/js to the user.
Let me show you what I have come up with till now :
Assuming the above 'note':
Cons (as per my understanding):
All the backend routes will be exposed to the outside world.
Even though backend servers are in private subnet, now that they're being accessed via external load balancer, the endpoints API could be accessed from the users.
How will we route a request from a Load balancer to another Load balancer. Because what I have seen is that you could only route a request to an EC2 instance added in the target group.
To overcome the cons as I think in the above approach, I came up with this architecture instead:
Pros (as per my understanding):
The backend routes are safe (in a way) because we have a way of internally connecting from the frontend to the backend servers(if required).
Cons:
If the request is made from the browser, the endpoints are again exposed.
Solution that I found online:
REAL BIG DOUBT IN THIS LAST ONE
This breaks all the logic of my understanding that : All the requests are made by the browser from the user to the backend because in this the requests to the backend are being routed FROM the frontend servers.
QUESTIONS
What if the backend request (say login) is made by the user from the browser?
How will this work out in such case?
seems like you have done some good work here.
Let me start by making things easy for you:
Users only interact with the Load Balancer: If you want to keep it simple and not break off your frontend asset serving to an external service like CloudFront, which you should if you are starting out, you will be hosting the application only via EC2 instances (application origin, or simply orgin). Your requests would look something like this:
Users <--> ALB <--> EC2
Notice how users never interact with EC2 instances directly, its always via Application Load Balancer (ALB).
If I can oversimply thing, this is how HTTP operates, a request is made to a resource at an IP and the response is sent back from the same resource or IP. So as in your diagram, a request will not be responded back by EC2 but rather be relayed via the ALB.
You don't need NAT gateway: NAT gateway are there to make it possible for resources in provate subnet access the internet. In this case, unless you want your application to access the internet, you don't need NAT gateway. Many large scale applications are actually locked down in part by not keeping this resource at all.
You are still protecting the origin: Given that only the ALB can be accessed over the internet and everything else internal you can structure things here in any way that you want to. you could have few internal microservices that can be used internally without ever being exposed to end users. Note that here request never leaves the VPN.
You can read more about this and build a sample application via the official docs here or access AWS tutorials here.
To me, #3 is the correct solution because it does not expose /api to end users (since you mention "I DO NOT want the users to directly access the /api"). In #1, I don't think you could limit access to /api to only the front-end servers, since security groups work on the whole load balancer, not per-target.
Also, being an Internet-facing load balancer, any requests from the front-end servers to the load balancer in #1 will be referencing the load balancer via public IP addresses. This will cause a 1c/GB charge to go "out of" the VPC and then back in again.
Only #3 correctly refers to back-end resources via private IP addresses. The internal load balancer will be referenced via private IP addresses.
Hello and thanks in advance.
First I want to provide some context to make answering my question easier.
We are using Google Cloud.
We got to a situation when our needs to be able to deploy updates easily for various parts of application bumped into our monolith architectures limitations.
Our app is not super big, but it already has 2 physical services - backend (scope being updated), and caching server which caches data and makes search in a mongo-like way over data from Google Datastore.
We have 2 options here.
"plugins" - like nanoservices running within same process which are developed in a way that these nanoservices do not know they are on the same process, all they know is a set of "plugins shell API" injected at activation of a nanoservice code. This shell gives the nanoservice access to a database, logging, configuration, routes registration, control events like refresh pages map and some metadata like website root url and root of static content deployed as supply stuff for a service version. Like https://static.server.com/deployments/foo/v2
standard microservices on kubernetes where same API mentioned exposed to each service via "shell client" package deployed as part of container image.
In short, this is a common "infrastructure vs library" dilemma, often mentioned in articles about microservices i read.
For library approach I have some vision already on how to implement all that including hot modules replacement without stopping server but the more i read about kubernetes the stronger I feel that I am inventing kubernetes (or similar) wheel.
How I imagine that:
1) there is a router service which is single service exposed outside from the cluster. When a page is requested, and this service attached to the load balancer we already have as backend.
It will handle authentication/authorization of outside requests, and pick the page to be rendered or API endpoint to be invoked. When a page requested, the related template is loaded, and data for pre-rendering is picked by calling related endpoint exposed by module service. When public API endpoint is picked, the matching service endpoint is called.
There are few services, including:
caching service (that service which deployed now at separate servers group, and what
updates service, which process the module services version switch and provides API to do so via some UI for admins.
modules services (one per modules). Each module exposes endpoints for providing preloaded pages data, endpoint to give list of pages routes to be registered, API endpoints implementations, and endpoint to list exposed API routes to be invoked through router service.
router service which process external requests and dispatch them across other services when appropriate using cached routes map, updated in case if one of internal services broadcasts pages map refresh event, e. g. updates service.
What is stopping me from starting to use kubernetes right away is the lack of knowledge about how to implement the following scenarios:
1) only 1 microservice must be exposed outside cluster, the "routing service".
2) reuse builtin services discovery etc to communicate with services within cluster like caching server.
3) Cluster's router service would be attached to cloud load balancer we already have as a backend.
In my opinion, you should have a look at NGINX Ingress Controller to build your routing scheme, more information you can find here and here.
EDIT In addition, you can try some other ingress controllers and among them Istio and Traefik definitely worth your attention as alternative solutions to NGINX Ingress Controller.
I have a Node-Express website running on a microservices based architecture. I deployed the microservices on Amazon ECS cluster with one EC2 instance. The microservices sit behind an Application Load Balancer that routes external traffic correctly to the services. This system is working as expected except for one problem: I need to make a POST request from one service to the other. I am trying to use axios for this but I don't know what url to post to in axios. When testing locally, I just used axios.post('http://localhost:3000/service2',...) inside service 1 but how should I do it here?
So There are various ways.
1. Use Application Load Balancer behind the service
In this method, you put your micro services behind the load balancer(s) and to send request, you give load balancer URL. You can have path based routing for same load balancer or you can use multiple load balancers.
2. Use Service Discovery
In this method, you let your requester discover it. Now Service discovery can be done in various way like using ALB or Route 53 or ECS or Key Value Store or Configuration Management or Third Party Software such as Consul
I have been working with spring and now would like to learn spring boot and microservices. I understand what microservice is all about and how it works. While going through docs i came across many things used to develop microservices along with spring boot which i am very much confused.
I have listed the systems below.and the questions:
Netflix Eureka - I understand this is service discovery platform.
All services will be registered to eureka server and all
microservices are eureka clients. Now my doubt is , without having
an API gateway is there any use with this service registry ? This is
to understand the actual use of service registry.
ZUULApi gateway- I understand ZUUL can be used as API gateway which is basically a load balancer , that calls appropriate
microservice corresponding to request URL. iS that assumption
correct? will the api gateway interact with Eureka for getting the
appropriate microservice?
NGINX - I have read NGINX can also be used as API gateway? Is that possible? Also i read some where else like NGINX can be used as a service registry , that is as an alternate for Eureka ! Thus which is right? Api gateway or service registry or both? I know nginx is a webserver and reverse proxies can be powerfully configured.
AWS api gateway - Is this can also be used as an alternate for ZUUL?
RIBBON - for what ribbon is used? I didn't understand !
AWS ALB- This can also be used for load balancing. Thus do we need ZUUL if we have AWS ALB?
Please help
without having an API gateway is there any use with this service registry ?
Yes. For example you can use it to locate (IP and port) of all your microservices. This comes in handy for devops type work. For example, at one project I worked on, we used Eureka to find all instances of our microservices and ping them for their status (/health, /info).
I understand ZUUL can be used as API gateway which is basically a load balancer , that calls appropriate microservice corresponding to request URL. iS that assumption correct?
Yes but it can do a lot more. Essentially because Zuul is more of a framework/library that you turn into a microservice, you can code it to implement any kind of routing logic you can come up with. It is very powerful in that sense. For example, lets say you want to change how you route based on time of day or any other external factors, with Zuul you can do it.
will the api gateway interact with Eureka for getting the appropriate microservice?
Yes. You configure Zuul to point to Eureka. It becomes a client to Eureka and even subscribes to Eureka for realtime updates (which instances have joined or left).
I have read NGINX can also be used as API gateway? Also i read some where else like NGINX can be used as a service registry , that is as an alternate for Eureka ! Thus which is right? Api gateway or service registry or both?
Nginx is pretty powerful and can do API gateway type work. But there are some major differences. AFAIK, microservices cannot dynamically register with Nginx, please correct me if I am wrong... as they can with Eureka. Second, while I know Nginx is highly (very highly) configurable, I suspect its configuration abilities do not come close to Zuul's routing capabilities (due to having the whole Java language at your disposal within Zuul to code your routing logic). It could be the case that there are service discovery solutions that work with Nginx. So Nginx will take care of the routing and such, but service discovery will still require a solution.
Is this can also be used as an alternate for ZUUL?
Yes AWS API Gateway can be used as a Zuul replacement of sorts. The issue here, just like Nginx, is service discovery. AWS API Gateway lets you apply logic to your routing... though not as open ended as Zuul.
for what ribbon is used?
While you can use the Ribbon library directly, for the most part consider it as an internal dependency of Zuul. It helps Zuul do the simple load balancing that it does. Please note that this project is in maintenance mode and not recommended any more.
This can also be used for load balancing. Thus do we need ZUUL if we have AWS ALB?
You can use ALB with ECS (elastic container service) to replace Eureka/Zuul. ECS will take care of the service discover for you and will map all instances of a particular service to a Target Group. Your ALB routing table can then route to Target Groups based on simple routing rules. The routing rules in ALB are very simple though, but improving over time.
Different systems which can be used for the working of microservices, that comes along with spring boot:
Eureka:
Probably the first microservice to be UP. Eureka is a service registry, means , it knows which ever microservices are running and in which port. Eureka is deploying as a sperate application and we can use #EnableEurekaServer annotation along with #SpringBootAPplication to make that app a eureka server. So our eureka service registery is UP and running. From now on all microservices will be registered in this eureka server by using #EnableDiscoveryClient annotation along with #SpringBootAPplication in all deployed microservices.
Zuul: ZUUL is a load balancer , routing application and reverse proxy server as well. That is before we were using apache for reverse proxy things , now , for microservices we can use ZUUL. Advantage is, in ZUUL we can programatically set configurations, like if /customer/* comes go to this microservice like that. Also ZUUL can act as a load balancer as well , which will pick the appropriate microservice in a round robin fashion. SO how does the ZUUL knows the details of microservices, the answer is eureka. It will work along with eureka to get microservice details. And in fact this ZUUL is also a Eureka client where we should mark using #EnableDiscoveryClient, thats how these 2 apps(Eureka and zuul) linked.
Ribbbon:
Ribbon use for load balancing. This is already available inside ZUUL, in which zuul is using Ribbon for load balancing stuff. Microservices are identified by service-name in properties file. IF we run 2 instances of one microservices in different port, this will be identified by Eureka and along with Ribbon(Inside zuul), requests will be redirected in a balanced way.
Aws ALB , NGINX , AWS Api gateway etc: There are alternatives for all the above mentioned things. Aws is having own load balancer, service discovery , api gateway etc . Not only AWS all cloud platofrms ,like Azure, have these. Its depends which one to use.
Adding a general question as well , How these microservices communicate each other: Using Resttemplate or Feignclient actual rest API can be called or Message queues like Rabbit MQ etc can be used .
Eureka can be used in conjunction with NGINX, which leads to very powerful combination.
I am using it on AWS EC2 environment. Previously instead of NGINX I was using Spring Cloud Gateway and before that Zuul. Depending of the load Spring Cloud Gateway was running on AWS t3.medium or t3.large instances. After moving to NGINX I am using t3.micro (8 times less memory) instance. I am almost sure that I can do the trick and with t3.nano (16 times less memory) instance, but I wanted to be sure that there will be no surprises.
Below are the high level steps what you have to do in order to plug NGINX in the Eureka ecosystem. More details you can find in NGINX With Eureka Instead of Spring Cloud Gateway or Zuul article.
Create a service which can read the configuration of all applications from Eureka and to 'translate' it to NGINX configuration.
Create a cronjob entry which at certain period will read the configuration from the above service and will call the NGINX hot reload
NGINX which will consume the configuration produced from the service and the cronjob and will work as API Gateway
I have a RESTful app deployed on a number of EC2 instances sitting behind a Load Balancer.
Authentication is handled in part by a custom request header called "X-App-Key".
I have just migrated my classic Load Balancers to Application Load Balancers and I'm starting to experience intermittent issues where some valid requests (via testing with CURL) are failing authentication for some users. It looks like the custom request header is only intermittently being passed through. Using apache bench approx 100 of 500 requests failed.
If I test with a classic Load Balancer all 500 succeed.
I looked into this a bit more and found that the users who this is failing for are using a slightly newer version of CURL and specifically the requests coming from these users are using HTTP2. If I add "--http1.1" to the CURL request they all pass fine.
So the issues seem to be specific to us using a custom request header with the new generation application load balancers and HTTP2.
Am I doing something wrong?!
I found the answer on this post...
AWS Application Load Balancer transforms all headers to lower case
It seems the headers come through from the ALB in lowercase. I needed to update my backend to support this
You probably have to enable Sticky sessions in your loadbalancer.
They are needed to keep the session open liked to the same instance.
But, it's at application level the need of having to keep a session active, and not really useful in some kind of services, (depending on the nature of your system, not really recommended) as it provides performance reduction in REST like systems.