I am relatively new to AWS. I've been able to:
Setup edge-optimized API Gateway endpoints (w/a custom domain ... install an SSL cert, etc.)
Route requests from API Gateway to a server running on an EC2 instance via HTTP proxy
For learning, I'd like to put a Network Load Balancer (NLB) between API Gateway and the EC2 instance. After that, I want to experiment with Auto-Scaling Groups (in order to see how EC2 instances can be scaled out/in).
BUT I'm stuck on the NLB. After adding it in, requests made to API Gateway endpoints simply return internal server error.
Question
Should I be using an internal or internet-facing NLB to route the requests from API Gateway to the EC2 instances?
When I opt to use an internal NLB, I get a warning message You are creating an internal Load Balancer, but there is an Internet Gateway attached to these subnets ... and then I run into the internal server error problem.
When I setup an internet-facing NLB, API Gateway warns me something to the effect of, "Your NLB is already public-facing, so the use of a VPC Link might be unnecessary".
Can someone shed some light on what I should be doing?
I was able to create a NLB in a private subnet and create a VPC link. Private Subnets use network address translation (NAT) gateways, not internet gatways.
Link: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html
NLB
VPCLink
Related
The question is the same as this one but has screenshots of AWS resources' configurations.
Situation:
A backend is hosted as an App Runner service.
The backend needs to communicate with a private RDS instance. So, the App Runner service is add to a VPC.
Backend can now communicate with the RDS instance.
Expected:
Backend should also be able to make HTTP requests to the Internet.
Actual:
Any HTTP request from the backend to the Internet takes forever/times out.
Checklist:
An Internet gateway is connected.
Route table is set to route 0.0.0.0/0 to the Internet gateway.
Security group allows outbound access.
I've tried also putting a NAT gateway in place of the Internet gateway and set up the route table accordingly. But the behaviour was the same. Screenshots below are without the NAT gateway configuration.
Screenshots:
VPC Connector configuration on App Runner service
Security group outbound rule allowing all traffic
Route table routing outbound traffic to internet gateway
Route table association with all subnets (non-explicit. Default, didn't change)
How I know that my service has no outbound Internet access:
I'm making a request google.com
I've made a log before, after, and in catch of the request.
Log happens before, but then nothing happens. And my API that invokes this request keeps loading forever (until it Gateway Timeouts after 5 minutes).
So, what is wrong in my configuration above/How can I give outbound Internet access to the service?
And btw, I can access the service itself (i.e., inbound traffic) through the domain generated by App Runner.
A related discussion: https://github.com/aws/apprunner-roadmap/issues/109
According to the official App Runner documentation, you must use a NAT Gateway to provide Internet access to App Runner applications running in a VPC.
You mentioned you already tried to use a NAT Gateway in your question, but I think you configured it incorrectly. Please bear in mind the following:
Your VPC needs both public and private subnets configured in order to properly use a NAT Gateway. Public subnets are subnets that have a route to the Internet Gateway. Private subnets are subnets that have a route to the NAT Gateway.
The NAT Gateway itself must reside in a public subnet.
The App Runner application must be configured to run only in private subnets.
We have 10 instances which we deployed the app using the AWS ECS and ELB
Due to security reasons the API allows request only through specific IP whitelisted IP addresses.
So we are planning to pass the request through the proxy
How to route an API request go through a proxy
We are using nginx
Any specific way to route an API request go through a proxy will be helful
You won't need NGINX as a proxy for this use-case, I'd propose to consider looking into using AWS NAT Gateways. NAT Gateway is a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an Amazon Virtual Private Cloud (Amazon VPC). Its the perfect place to provide a static IP to all your subnet's outbound traffic.
In order to provide a NAT Gateway with static IP (Elastic IP) for your cluster's outbound traffic. This will allow your different tasks running inside your ECS cluster's private subnets to look like a single requesting entity from an outsider's POV (in your case, the 3rd party API is the outsider). To achieve this, you will have to:
Create 2 route tables (1 for private subnets, 1 for public subnets)
Internet gateway on the public subnet
Elastic IP address
Create a NAT Gateway and attach the elastic IP to it (This will be the IP whitelisted to the 3rd party API)
Ensure that all your tasks are running inside private subnets of the VPC
Add a rule in your route table for your private subnets that redirects outbound 0.0.0.0/0 to the NAT Gateway.
Add a rule in your route table for your public subnets that redirects outbound traffic 0.0.0.0/0 to the internet gateway.
You should consider using NAT Gateway instead. I am assuming you already would have all your containers in a VPC, so you can create a new NAT Gateway within this VPC itself.
You can refer to articles attached below to do this:
https://docs.aws.amazon.com/appstream2/latest/developerguide/add-nat-gateway-existing-vpc.html
https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html
Note: NAT Gateways have price associated with them.
If needed, you can use the elastic IP provided by NAT Gateways on your lambdas as well.
I would like to create a lambda(vpc) which would access resources in vpc and make a request to services(REST API) via public application load balancer. I found out that vpc end point is better solution than creating a nat gateway.
I have created a vpc endpoint for elasticloadbalancing(by following steps at https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) and given full access in the policy. I could not find how to access it from the lambda, what would be the URL to make the request?
Edit:
Thanks to John for the info that vpc endpoint is used to connect to ELB API. So Vpc endpoint would not solve our issue.
We have our infra in vpc which includes database(accessible within vpc only) and application servers running behind the ELB. For certain tasks we want to run lambda which will read database(for this reason lambda has to be inside vpc) and make API calls to our application using ELB. Since ELB is accessible from public dns only, lambda is not able to connect to ELB.
I have read that setting up NAT gateway is a solution. Are there other is simpler ways?
Yes, a NAT Gateway would allow the traffic from a private subnet to go out of the VPC and come back in to the Load Balancer's public IP addresses (via its Public DNS Name).
Alternatively, you could create an additional Internal Load Balancer that could accept traffic from within the VPC and send it to the Amazon EC2 instances.
I seem to be stuck with my architecture for ECS Microservices.
Current State :
ECS cluster with 2 EC2 instances in private subnets exposing a dockerized node express endpoint on port 3000 (1 service, 2 running tasks)
NAT gateway route out from VPC's default route table
internal NLB in front of those 2 EC2 instances.
VPC Link created in API Gateway for targeting the NLB.
API gateway integration with the VPC link (what do I enter as the endpoint URI? is it the DNS of the NLB? or custom domain name in step 6)
Created custom domain name for the API gateway
when I try to send a request to customdomainname:3000/task. I expected the task to be run However I do not get any response(There was an error connecting to customdomainname:3000/task).
Everything works perfectly fine if its an internet facing Application load balancer and instances in public subnet, but I need to use the VPC link approach to secure my application.
I understand my question might be vague, but I would appreciate any input and insights from anyone who has worked on this before
I've spent the majority of today reading google results and documentation on how to connect AWS API Gateway to EC2 instances (created by Elastic Beanstalk) in a private subnet. I know that API Gateway requires targets to be publicly addressable, so...
I manually created an Application Elastic Load Balancer that listens for (and terminates) HTTPS at a public IP address;
I created a VPC with two subnets: one public (holds load balancer) and one private (holds EC2 instances); and,
I believe I have to create security groups that allow everyone/everywhere to connect to the load balancer, but only entities in my public subnet to connect to my EC2 instances.
Unfortunately I'm unable to view the sample Beanstalk application via the load balancer's DNS name. The connection just times-out.
Can someone please confirm I've identified all the steps? Is there any way I can trace my requests to see where they're failing? Or (even better) why they're failing? Thanks!
Check your security groups to make sure that HTTPS traffic is allowed
Make sure your Network ACLs are allowing traffic from your load balance to your EC2 instances
Check your VPC routes to ensure there is a route from your load balancer to your EC2 instance