I started a cluster in aws following the guides and then went about following the guestbook. The problem I have is accessing it externally. I set the PublicIP to the ec2 publicIP and then use the ip to access it in the browser with port 8000 as specified in the guide.
Nothing showed. To make sure it was actually the service that wasn't showing anything I then removed the service and set a host port to be 8000. When I went to the ec2 instance IP I could access it correctly. So it seems there is a problem with my setup or something. The one thing I can think of is, I am inside a VPC with an internet gateway. I didn't add any of my json files I used, because they are almost exactly the same as the guestbook example with a few changes to allow my ec2 PublicIP, and a few changes for the VPC.
On AWS you have to use your PRIVATE ip address with Kubernetes' services, since your instance is not aware of its public ip. The NAT-ing on amazon's side is done in such a way that your service will be accessible using this configuration.
Update: please note that the possibility to set the public IP of a service explicitly was removed in the v1 API, so this issue is not relevant anymore.
Please check the following documentation page for workarounds: https://kubernetes.io/docs/user-guide/services/
Related
I have a solution (AnzoGraph DB) deployed on my AWS Kubernetes Cluster (EC2 Instance), and was working totally fine.
Suddenly this solution stopped and i could not access it via the DNS anymore.
I tested the solution deployed on my cluster using kubectl port-forward command and they are working fine (the pods and services), thus i assume the problem is with AWS Loadbalancer.
To access the application we need to go through this path:
Request -> DNS -> AWS Load Balancer -> Services -> Pods.
The LoadBalancer is (classic) internal, so it's only accessible for me or the company using VPN.
Every time when I try to access the DNS , I got no response.
Any idea how i can fix it ? or where is the exact issue ? how can I troubleshoot this issue and follow the traffic on AWS ?
Thanks a lot for the help!
sorry I missed your post earlier.
lets start with a few questions...
You say you use k8s on AWS EC2, do you actually use EKS, or do you run a different k8s stack?
Also ... you mentioned that you access the LB from your (DB) client/ your software by DNS resolving the LB and then access AnzoGraph DB.
I want to make sure that the solution is actually DNS resolving the LB via DNS every time. if you have a long running service, and AWS changes the IP address of the LB, and your SW possibly had cached the IP, you would not be able to connect to the LB.
on the system you run your Software accessing AnzoGraph DB ... (I assume CentOS (7) )
make sure you have dig installed (yum install bind-utils)
dig {{ your DNS name of your LB }}
is that actually the IP address your SW is accessing?
has the IP address of the client changed? make sure the LB SG allows access
(https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-groups.html)
I assume you access the AnzoGraph DB frontend POD via 443?
as you write
"I tested the solution deployed on my cluster using kubectl port-forward command and they are working fine (the pods and services)"
we probably do not have to look for pod logs.
(if that was not the case, the LB would obviously block traffic as well.)
So I agree, that the most likely issue is (bad) DNS caching or SG due to different SRC IP being rejected by the classic LB SG.
also for completeness .. please tell us more about your env.
AnzoGraph DB image
EKS/k8s version
helm chart / AnzoGraph operator used.
Best - Frank
I have been looking for help with this problem, and the answers just say to add inbound rules to the security group. Well, I have done those and I am still unable to access my website from the public DNS (just putting that into the url box and navigating to it). There are multiple port 22 inbound rules for the people accessing my server, and the outbound rule is just "All traffic".
I've had this problem running Wordpress on EC2 instances. Things I'd try:
Access the instance via ssh. Check out https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html
If you're accessing through ssh, maybe it's because your disk is full. To check this you can run df -h on your Amazon EC2 server.
I tried accessing my newly setup AWS EC2 Instance and I had this same issue, I later realised I was accessing the public DNS via HTTPS which had not been set up. when I changed the url to use HTTP it worked. Ensure to configure HTTP in the security group.
I have created an EC2 (Windows) instance on AWS. I want to run a TCP server on this EC2 on TCP port 7889
and want to connect to it from outside world.
I have edited the inbound traffic rule in security group and added TCP rule for 7889 from anywhere.
But unfortunately my TCP client socket can not reach the server through the public ip of the EC2 instance.
Please help.
Note: The same thing worked for me few years ago. My little search has shown me few things like , I need to create VPC, public and private subnets, internet gateway and route tables.Is it true? All these were not required earlier.Is it a new addition?
Yes all you mentioned are absolutely needed. VPC, Subnet, Route table, Internet Gateway and EIP all are required.
The reason it worked for you previously because you might have used the default VPC with all other components already configured. Even now you will be having a default VPC in every AWS region which you can make use of it.
But it's really needed to have basic understanding of how things connect and work together because all these components are very basic and important. Refer AWS documents for more details.
If all the components are cofigured already, check the output of
netstat -nptl
command. The port 7999 should be available and it should be ready to accept any IP (i.e. point to 0.0.0.0)
Thank you for the responses. It was windows internal firewall as #Marcin suggested.
Everything works now.
I am new to AWS and I am trying to deploy simple app to AWS ECS. I have two simple docker containers, running in ECS Fargate:
‘Frontend’: Vue Js app, which makes a single request to backend;
‘Backend’: Django app, which serves the request;
Both services were launched within the same cluster, in default VPC and the same, single public subnet. For ‘Backend’ I configured Service Discovery: Namespace – test, Service Discovery Name – backend. Security group configured to allow All Traffic.
So, the problem is when frontend makes request:
axios.get('http://backend.test:8000/api/get-test/')
I got error: Failed to load resource: net::ERR_NAME_NOT_RESOLVED backend.test:8000/api/get-test/
However, executing in AWS Cloud9 command: dig +short backend.test returns correct private IP of the backend container.
When I change request to something like
axios.get('http://172.17.3.85:8000/api/get-test/')
where 172.17.3.85 is valid private IP of the backend container, I got following error:
GET http://172.17.3.85:8000/api/get-test/ net::ERR_CONNECTION_TIMED_OUT
However, if I spin out EC2 instance in the same VPC and subnet and SSH to it, I can ping backend container, and requests -
curl -v http://172.17.3.85:8000/api/get-test/
as well as
curl -v http://backend.test:8000/api/get-test/
return desired response.
The only case when everything is working as expected is when the request is like
axios.get('http://3.18.59.133:8000/api/get-test/'),
where 3.18.59.133 is valid Public IP of the backend container.
I would appreciate any suggestion where look further or how to connect two containers via service discovery as right now I am out of ideas.
Based on the discussion in comments and description of the problem, the reason is that the Frontend’: Vue Js app executes on the client side, for example, in the browser.
This explains all the issues described and discussed:
axios.get('http://backend.test:8000/api/get-test/') does not work as on the client side you can't resolve privte hosted zone.
axios.get('http://172.17.3.85:8000/api/get-test/') does not work because the 172.17.3.85 is valid only in the VPC, not on the client's network.
spin out EC2 instance in the same VPC and subnet and SSH works because private hosted zones can be resolved inside VPC.
axios.get('http://3.18.59.133:8000/api/get-test/') works because public IP can be used on the clinet side, unlike private IPs.
Suppose I have a service say auth(port:8080) which has 3 tasks running and let's say I have another service say config-server(port:8888), 2 tasks running, where auth will load the configuration properties from, similar to spring cloud config server.
Launch Type: EC2
auth service running on 8080 |
config-server service running on 8888
Now, in order to access config-server from auth, do I have to use ALB to call config-server or I can call using service name, like http://config-server:8888?
I tried but it's not working. Did I misunderstand any concept here?
I would like to get some insight on this.
This is how my Service Discovery Configuration looks like.
EDITS:
I created a private namespace test.lo and still not working..
curl http://config-server.test.lo
curl: (6) Could not resolve host: config-server.test.lo
These are general things to check.
Ensure that enableDnsHostnames and enableDnsSupport options for VPC are enabled.
Don't use local as a private namespace. It's a reserved name.
Check private hosted zone created in Route 53 and verify that it has all the A (and SRV if used) correctly set to the private IP address of the service's tasks.
Private hosted zone can be resolved only from the inside of the same VPC as the ECS service. Thus to check if they work, can create an instance in the VPC and inspect from there.
Use dig tool to check if the DNS actually resolves the private dns name into private IP addresses. It should return multiple addresses, one for each task in a service.
If using awsvpc network mode can using either A or SRV record types. Thus if SRV does not work, it could be worth checking with A record.