Does anyone have any advice on how to minimize cross-az traffic for inter-pod communication when running Kubernetes in AWS? I want to keep my microservices pinned to the same availability zone so that microservice-a that resides in az-a will transmit it's payload to microservice-b also in az-a.
I know you can pin pods to a label and keep the traffic in the same AZ, but in addition to minimizing the cross az-traffic I also want to maintain HA by deploying to multiple AZs.
In case you're willing to use alpha features you could use inter-pod affinity or node affinity rules to implement such a behaviour without loosing high availability.
You'll find details in the official documentation
Without that you could just have one deployment pinned to one node and a second deployment pinned to another node and one service which selects pods from both deployments - example code can be found here
Related
Need a design advice. Need to run in AWS loadbalancer per packet (not per flow).
It's for unidirectional UDP based streaming.
That means that each packet received by the loadbalancer should be send to another target - so that all targets receive the same amount of packets.
I do not see any ready solution and considering using EC2 with iptables and "-m statistic --mode random" PREROUTING chain. Any comments on the performance of that module at 1 up to 10Gbit/s scale ? (how strong EC2 instance would i need?)
Any other advices / hints how to achieve it ?
Thanks,
AWS Network Loadbalancer can be configured to send to "random" Targets in the TargetGroup, but this behaviour is not documented and just stated (to be exact, it's not defined how this distribution is done). It's the general ELB behaviour that targets are chosen by some hidden algorithm. Maybe it's worth an experiment? Make sure that Stickiness is turned off, as this is enabled the exact opposite of your use case.
I couldn't find a hard definition of how many GBit/s a NLB will support. But there is the concept of LCU (Load Balancer Capacity Units) that determines also billing and needs to be taken into account. LCUs are exposed in CloudWatch
Custom EC2 Instances will work and also cost a lot as CPU scales (roughly) with network. Here is a general list of EC2 Instances that you can filter for your network requirements and also see the pricing for it.
Maybe you should generally go for Devices with Enhanced Networking and Nitro, as later have special hardware for fast networking.
We have many internet services, what are the considerations whether to use alb per service or single alb for all using listener rule pointing to target 🎯 group.
The services has its own clusters/target group with different functionality and different url.
Can one service spike impact other services?
Is it going to be a single point of failure ?
Cost perspective ?
Observability, monitoring, logs ?
Ease of management ?
Personally I would normally use a single ALB and use different listeners for different services.
For example, I have service1.domain.com and service2.domain.com. I would have two hostname listeners in the same ALB which route to the different services.
In my experience ALB is highly available and scales very nicely without any issues. I've never had a service become unreachable due to scaling issues. ALB's scale based on "Load Balancer Capacity Units" (LBCU). As your load balancer requires more capacity, AWS automatically assigns more LBCU's which allows it to handle more traffic.
Source: Own experience working on an international system consisting of monoliths and microservices which have a large degree of scaling between timezones.
You don't have impact on service B if service A has a spike, but the identification of which service is having bad times could be a little pain.
For monitoring perspective it's is a bit hard because is not that easy to have a fast identification of which service/target is suffering.
For management, as soon as different teams need to create/management its targets it can create some conflicts.
I wouldn't encourage you using that monolith architecture.
From cost perspective you can use one load balancer with multi forward rules, but using a single central load balancer for an entire application ecosystem essentially duplicates the standard monolith architecture, but increases the number of instances to be served by one load balancer enormously. In addition to being a single point of failure for the entire system should it go down, this single load balancer can very quickly become a major bottleneck, since all traffic to every microservice has to pass through it.
Using a separate load balancer per microservice type may add additional overhead but it make single point of failure per microservice in this model, incoming traffic for each type of microservice is sent to a different load balancer.
I have the following doubt relating how to correctly set up a WSO2 MB cluster respecting the requirements of high availability. I am following this official guide: https://docs.wso2.com/display/EI650/Clustering+the+Message+Broker+Profile#ClusteringtheMessageBrokerProfile-Testingthecluster
So I will have a two nodes WSO2 MB profile cluster. Now my doubt is related to the high availability concept (basically: if a single node is not working, the cluster should still works).
I have two nodes cluster, each node run on a specific server having a specific IP address, something like this
NODE 1 with the IP: XXX.XXX.XXX.1
NODE 2 with the IP: XXX.XXX.XXX.2
So lets suppose that I want publish a message into a queue defined on this cluster. I suppose that I can send the message indifferently to one of these 2 nodes (correct me if I am doing wrong assertion).
If this is the situation: how can I guarantee the high availability requirement? Can I simply put both my nodes under a load balancer? so that if one node doesn't work, the request is directed to the other
Is it a correct way to handle this situation?
Yes if you have both of the EI instances with MB profile clustering where the two servers have cluster coordination configured on JDBC or Hazelcast level, with the above-mentioned approach you will guarantee the high availability of the service.
You can make sure to follow the below provide additional precautions to make sure both servers do not go down for the same reason at ones.
Have the two servers in two separate instances rather than having
offset in the same instance.
You can configure the load balancer to
work in Active-Passive or Active-Active mode if you are deployed on a
cloud provider like AWS you can add configurations to redeploy the instances of the health check is failed for a given amount of time.
I have an application running on AWS EC2. In order to get high availability, we are planning to set up AWS Load balancer and host the application in a minimum of 2 EC2 instances.
We also heard about Docker swarm, in which we can create a service with 2 managers on 2 separate EC2 instances and the swarm will take care of the rest (instead of using ALB with 2 EC2). It will balance the load to all the containers inside the cluster and it also restarts the container if anything makes this go down.
So I want to know which is the best choice for my case. The application won't get heavy load/traffic. Why I choose load balancer is for high availability. If 1 instance goes down the other will be taken care of this.
If there are any other choices to fulfill my requirements, it will be highly appreciated.
Thanks.
I presume it's a stateless application.
A combination of both Swarm & ALB is what you can go for but you will need to incorporate autoscaling etc sooner or later which means you will need to manage the swarm cluster & maintain it.
With ALB you will get real good metrics which you will surely miss while using Swarm.
BUT, you do have a few better options which will manage the cluster for you. You just will have to manage & maintain the docker images -
Use ECS.
Use EKS (only in us-east-1 as if now)
Use ElasticBeanstalk MultiContainer.
Ref -
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
https://aws.amazon.com/eks/
Small stateful app on AWS. Statefullness is the issue and for sake of argument assume that the app must remain stateful. How can we create redundancy across multiple AZs?
Half Baked Ideas:
1) Mirrored setup in AZ1 and AZ2. Use ELB to route all traffic to AZ1. If there are health problems, stop routing to AZ1 and route to AZ2. Is this even possible? Isn't that like anti-load-balancing?
2) Use lambda to "Turn On" and instance already created in AZ2 when AZ1 has health issues. Would also turn off instance in AZ1. If so, could you point me towards some lambda documents?
3) Something way better and probably easier than 1 or 2
p.s. I know how to easily accomplish if the app was not stateful. Unfortunately the statefullness cannot be adjusted.
By statefullness I am assuming you need a central store for data.
Choosing technology to store state data:
1) If the nature of state retrieval is not chatty or the performance requirements are not extreme, i.e. not more than once to and fro's from the server to retrieve state you should choose DynamoDb to store the state, this is multi-az by default you dont have to do anything to make it HA.
2) In a high performance scenario or when multiple round trips are needed to retrieve state (try to avoid this) you can choose memcached which is available as an option in AWS elastiCache where you can deploy them in multiple AZ's for HA.
For HA of front end servers:
1) Just add servers into multiple AZ's and attach them to the load balancer, in order to do this:
a) Add two subnets using the VPC service into more than one AZ.
b) Create a load balancer and give it both of these subnets (this makes your LB HA), make sure to enable "Cross-Zone Load Balancing", this will distribute traffic evenly to the servers attached to it across zones.
c) Create two (minimum) app servers and add one to each subnet in different AZ, add both of them to the above created LB.
2) Make sure that your servers retrieve and store state from a central data layer (which is also HA now).
You are good to go, you can optionally setup auto scaling in AWS using launch configs and auto scaling groups if that's a requirement which I'm guessing it will be.