Cannot connect two ECS services via Service Discovery - amazon-web-services

I am new to AWS and I am trying to deploy simple app to AWS ECS. I have two simple docker containers, running in ECS Fargate:
‘Frontend’: Vue Js app, which makes a single request to backend;
‘Backend’: Django app, which serves the request;
Both services were launched within the same cluster, in default VPC and the same, single public subnet. For ‘Backend’ I configured Service Discovery: Namespace – test, Service Discovery Name – backend. Security group configured to allow All Traffic.
So, the problem is when frontend makes request:
axios.get('http://backend.test:8000/api/get-test/')
I got error: Failed to load resource: net::ERR_NAME_NOT_RESOLVED backend.test:8000/api/get-test/
However, executing in AWS Cloud9 command: dig +short backend.test returns correct private IP of the backend container.
When I change request to something like
axios.get('http://172.17.3.85:8000/api/get-test/')
where 172.17.3.85 is valid private IP of the backend container, I got following error:
GET http://172.17.3.85:8000/api/get-test/ net::ERR_CONNECTION_TIMED_OUT
However, if I spin out EC2 instance in the same VPC and subnet and SSH to it, I can ping backend container, and requests -
curl -v http://172.17.3.85:8000/api/get-test/
as well as
curl -v http://backend.test:8000/api/get-test/
return desired response.
The only case when everything is working as expected is when the request is like
axios.get('http://3.18.59.133:8000/api/get-test/'),
where 3.18.59.133 is valid Public IP of the backend container.
I would appreciate any suggestion where look further or how to connect two containers via service discovery as right now I am out of ideas.

Based on the discussion in comments and description of the problem, the reason is that the Frontend’: Vue Js app executes on the client side, for example, in the browser.
This explains all the issues described and discussed:
axios.get('http://backend.test:8000/api/get-test/') does not work as on the client side you can't resolve privte hosted zone.
axios.get('http://172.17.3.85:8000/api/get-test/') does not work because the 172.17.3.85 is valid only in the VPC, not on the client's network.
spin out EC2 instance in the same VPC and subnet and SSH works because private hosted zones can be resolved inside VPC.
axios.get('http://3.18.59.133:8000/api/get-test/') works because public IP can be used on the clinet side, unlike private IPs.

Related

Sending Requests from postman to AWS ec2 instance

I have my ec2 instance running a docker image and id like to test the functionality of my routes with postman, however whenever I ping the ec2 public dns, it says connection refused. I have the security group opened up for all traffic from my machine and am running the postman desktop app. The example route I'm trying to hit is
https://{IP address here}.compute-1.amazonaws.com:6000/register
and I'm sending a post request with some json in it. All of this works fine on my local machine. Please tell me what I'm missing?
You can't use https:. Instead you must use http: as default EC2 instance url does not support HTTPS. For valid https, the easiest way is to setup ALB or set it up on an instance directly.

How to allow calls to Cognito from an AWS ECS container instance?

I have a setup with an ALB and a target group created by ECS, I'm using Fargate and created a build pipeline by following this article. My app is built with NET core, I have an Angular frontend. Got all this working, I'm able to deploy my code changes, but I'm a bit stuck with the following issue.
I'm using Cognito for authentication and a custom domain that I set for the hosted UI. It seems that, from the browser, when I try to hit an endpoint that is secured, I get a 504 Gateway error, which somehow is not doing the redirect to Cognito in the browser. All this works fine when I run the application on localhost.
When I looked at the logs, I noticed the following exception:
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://cognito-idp.<region>.amazonaws.com/<region_and_a_code>/.well-known/openid-configuration'
Apparently, it can't establish a connection to Cognito. My containers are using only port 80, my target group instances are also using port 80, ALB uses HTTPS on 443 which directs the traffic to the target group, and for ALB port 80 I just redirect to 443.
I tried a few different things, like setting the authority value instead of the metadata address, tried using a BackChannelHttpHandler to execute the HTTPS call, tried updating the port mappings to allow communication on 443, but somehow it seems that it gets overridden by the task definition that I have created when I set up the build pipeline. The network mode in my task definition is now awsvpc, and if I try to set it to host, it will complain that I can't use it with Fargate.
What do I need to do to allow the HTTPS request from my Docker container instances to reach Cognito?
You are trying to set this up in a public ALB. This setup using a private NLB will work, might work with a private ALB as well. You can then set up vmc private links to get at the service's you need access too.

aws Internal load balancer net::ERR_NAME_NOT_RESOLVED when calling a service

I have a 3 tier ECS containers application. In presentation tier I have a public subnet where there's an angular app running on nginx server. For that I have application internet-facing load balancer. In the private subnet I have Java Spring REST API service that runs on tomcat server on port 8080, for that there's application internal load balancer. In the other private subnet I have RDS database.
Application client sends requests to internal load balancer url, and renders the response in the application.
While I am able to ssh to ec2 in public subnet and curl to rest service in private subnet and get response:
curl -X POST http://internal-qa-XXXXX-XXXXXXX.eu-west-2.elb.amazonaws.com:8080/api/products/all
I am not able to receive response when accessing the client in the browser. The application runs correctly, however when inspecting in the browser console I see:
POST http://internal-qa-XXXXX-XXXXXXX.eu-west-2.elb.amazonaws.com:8080/api/products/all net::ERR_NAME_NOT_RESOLVED.
I checked containers with Docker logs <container_id> and they run just fine.
Seurity groups and NACL are configured correctly, I even checked with all traffic allowed
Based on the comments.
The issue is most likely caused by the fact that the url endpoint of the internet load balancer is called from the client side, i.e. browser.
Url of internal load balancer isn't publicly callable.
To solve this, either the application has to be modified to use only publicly available endpoints, or the internal load balancer changed into internet facing.

AWS ECS: How to communicate between services within ECS Cluster without using ALB? Is there a way?

Suppose I have a service say auth(port:8080) which has 3 tasks running and let's say I have another service say config-server(port:8888), 2 tasks running, where auth will load the configuration properties from, similar to spring cloud config server.
Launch Type: EC2
auth service running on 8080 |
config-server service running on 8888
Now, in order to access config-server from auth, do I have to use ALB to call config-server or I can call using service name, like http://config-server:8888?
I tried but it's not working. Did I misunderstand any concept here?
I would like to get some insight on this.
This is how my Service Discovery Configuration looks like.
EDITS:
I created a private namespace test.lo and still not working..
curl http://config-server.test.lo
curl: (6) Could not resolve host: config-server.test.lo
These are general things to check.
Ensure that enableDnsHostnames and enableDnsSupport options for VPC are enabled.
Don't use local as a private namespace. It's a reserved name.
Check private hosted zone created in Route 53 and verify that it has all the A (and SRV if used) correctly set to the private IP address of the service's tasks.
Private hosted zone can be resolved only from the inside of the same VPC as the ECS service. Thus to check if they work, can create an instance in the VPC and inspect from there.
Use dig tool to check if the DNS actually resolves the private dns name into private IP addresses. It should return multiple addresses, one for each task in a service.
If using awsvpc network mode can using either A or SRV record types. Thus if SRV does not work, it could be worth checking with A record.

Public IP on service for AWS in Kubernetes fails

I started a cluster in aws following the guides and then went about following the guestbook. The problem I have is accessing it externally. I set the PublicIP to the ec2 publicIP and then use the ip to access it in the browser with port 8000 as specified in the guide.
Nothing showed. To make sure it was actually the service that wasn't showing anything I then removed the service and set a host port to be 8000. When I went to the ec2 instance IP I could access it correctly. So it seems there is a problem with my setup or something. The one thing I can think of is, I am inside a VPC with an internet gateway. I didn't add any of my json files I used, because they are almost exactly the same as the guestbook example with a few changes to allow my ec2 PublicIP, and a few changes for the VPC.
On AWS you have to use your PRIVATE ip address with Kubernetes' services, since your instance is not aware of its public ip. The NAT-ing on amazon's side is done in such a way that your service will be accessible using this configuration.
Update: please note that the possibility to set the public IP of a service explicitly was removed in the v1 API, so this issue is not relevant anymore.
Please check the following documentation page for workarounds: https://kubernetes.io/docs/user-guide/services/