I have a few services in an AWS ECS cluster, using Fargate launch type. Those services are all in the same VPC and discover each other using Service Connect. The whole thing is behind an Application Load Balancer (ALB) which handles HTTP(S) traffic.
Now I'd like to access one of those services (a database) from an AWS Batch Job. The batch job is currently launched into the same VPC as well. Unfortunately (but not suprisingly) the batch job can't find the database service, because batch jobs seem to not use Service Connect.
How can I make this database service discoverable and accessible for the AWS Batch Job?
Related
I'm trying to see if I can use AWS Batch to perform scheduling so I don't have to deploy my own system. What I need the scheduler to do is perform a GRPC call at a given time. The GRPC would be an exposed port in my Docker Swarm inside VPC, so it can route to the appropriate service, but I can't find in the Job Definition page any notion of associating a VPC.
I have an ec2 cluster in which my ES is running. I want to use AWS app sync, can I connect that as a data source to it. If so how?
Or is it tightly coupled to use with Amazon OpenSearch Service?
Does ECS with EC2 machines can use service discovery?
The AWS documentation says all about Fargate,
have anyone done it with ALB, EC2 ECS cluster, service discovery.
ECS uses AWS Cloud Map for service discovery. You can use the AWS Cloud Map for other services as well:
You can also use AWS Cloud Map to register and locate any cloud resources, such as Amazon EC2 instances, Amazon DynamoDB tables, Amazon S3 buckets, Amazon Simple Queue Service (Amazon SQS) queues, or APIs deployed on top of Amazon API Gateway, among others.
Is there any way to do load balancing when deploying a series of microservices in AWS using Docker Swarm?
Basically, I'd like to attach a specific service to an elastic load balancer the same way that can be done using ECS. Using ECS or EKS directly is not an option.
A way to do it would be by configuring the health check of the elastic load balancer to attack an endpoint only available in the service to find but for what I understand this would be inefficient as it would basically make the ELB keep making health check requests to endpoints that don't even exist in some EC2 instances.
Is there any other way of doing this?
We got multiple k8s clusters on AWS (using EKS) each on its own VPC but they are VPC peered properly to communication to one central cluster which has an elasticsearch service that will collect logs from all clusters. We do not use AWS elasticsearch service but rather our own inside kubernetes.
We do use an ingress controller on each cluster and they have their own internal AWS load balancer.
I'm getting fluentd pods on each node of every cluster (through a daemonset) but it needs to be able to communicate to elasticsearch on the main cluster. Within the same cluster I can ship logs fine to elasticsearch but not from other clusters as they need to be able to access the service within that cluster.
What is some way or best way to achieve that?
This has been all breaking new ground for me so I wanted to make sure I'm not missing something obvious somewhere.