I have Public and Pvt Subnets in my VPC. I have some services running on EC2 in Pvt subnet, that needs to be accessed by external/mobile resources. How do I do this- is VPCLink and NLB the way to do it, or any other way, create some access point in Public subnet (??). Lambda seems to be the answer (for almost everything in AWS now) - not sure even how that access works for resources in Pvt Subnet.
Also the same Pvt Subnet has access external resources (outside of AWS) - how do I do this using the API Gateway?
Not quite understanding how the API-Gateway (and Lambda) is situated vis-a-vis - VPC and subnets- and how the network access control functions- can they access Pvt subnets directly or not. The documentation is somewhat silent on this, only talks about IAM - if someone can explain this. Found this on Lambda: AWS Lambda: How to setup a NAT gateway for a lambda function with VPC access.
The documentation says "API Gateway allows you to securely connect ... publicly addressable web services hosted inside or outside of AWS". My resources in Pvt subnet are not publicly addressable - I suppose.
Thanks
Are the services you have running on EC2 offering an API? API Gateway is meant to proxy API requests. It's commonly used in conjunction with Lambda to allow Lambda functions to process HTTP requests. An API Gateway is not necessary for your service. You can simply use an Application Load Balancer (ALB) or an Elastic Load Balancer (ELB). They can reside on a public subnet while your service remains in the private subnet. You can use security groups and VPC routing tables to allow communication from your public ALB/ELB to your private EC2 service.
With Ben’s help here is the answer
Introducing Amazon API Gateway Private Endpoints
Inbound: Accessing services hosted in Private Subnet via API Gateway
Endpoint integrations inside a private VPC. With this capability, you can now have your backend running on EC2 be private inside your VPC without the need for a publicly accessible IP address or load balancer.
So essentially API Gateway can access published endpoints, even in Private Subnets.
OutBound: Accessing externally hosted services from Private Subnet via API Gateway
API Gateway private endpoints are made possible via AWS PrivateLink interface VPC endpoints. Interface endpoints work by creating elastic network interfaces in subnets that you define inside your VPC. Those network interfaces then provide access to services running in other VPCs, or to AWS services such as API Gateway. When configuring your interface endpoints, you specify which service traffic should go through them. When using private DNS, all traffic to that service is directed to the interface endpoint instead of through a default route, such as through a NAT gateway or public IP address.
So you simply need to create a VPC endpoint in the Pvt Subnet for the API Gateway.
Related
Say I have an ec2 in a private subnet with access to the internet through a NAT Gateway and I have VPC endpoints (PrivateLink) for AWS services like DynamoDB or SNS. If my ec2 instance needs to interact with those services, it will use the private link or the NAT Gateway? In other words, the traffic will go on the public internet or will stay inside the VPC?
In other words, the traffic will go on the public internet or will stay inside the VPC?
It will go to interface endpoint. And the reason is that aws chooses more specific (longest prefix match) route when there is more than one choice where to direct traffic. So if VPC has to decide between 0.0.0.0/0 for NAT and interface endpoint address, interface endpoint will be prioritized.
You can easily check it yourself. Place interface endpoint in your private subnet which has route to NAT. Then black all incoming traffic to the interface endpoint using its security group. What you should see is that you are unable to access the service of the interface endpoint, even though there is NAT gateway.
When I configure an AWS Gateway VPC endpoint, a route table entry is created that points to the Gateway. Here, Gateway can be thought of performing the routing to AWS service (over private network).
However, for an AWS Inteface VPC endpoint, all that is visible is a Network interface that has a private IP address of the subnet. By default, a private IP can send traffic within the subnet or entire VPC provided Security Group and NACL allows the traffic. & it appears in this case there is no Route table entry to a Gateway or a Router for allowing traffic outside VPC.
How / Where is the interface routing the traffic to i.e. How does traffic leave the customer VPC?
Of course I understand that the traffic finally reaches the intended AWS service over private network but here I am trying to find out where is the Gateway or Router? Does AWS hide this implementation?
I cannot get my head around the fact that a simple Network Interface can accept traffic and route it to a service all by itself i.e. performing routing by itself? Clearly, in this case the traffic appears not flowing through the VPC router or another Gateway device.
I am aware this might be an AWS confidential implementation but any thoughts / idea on how they might have designed this feature?
It doesn't provide routing at all, by default a VPC interface endpoint when created will create an ENI per subnet in the VPC for you. It will also provide you a DNS name per each AZ and a global name that you can use within your applications.
In addition it supports the ability to have the AWS service domain name for the VPC interface endpoint be resolvable to the private IPs of the endpoint. As long as your VPC has DNS enabled it will first check the VPC private DNS resolver and then resolve it to the private IP rather than the public one.
This is done by adding an additional private hosted zone to your VPC which resolves service domains in your region such as ec2.us-east-1.amazonaws.com.
From the AWS side this is just an ENI created in your AWS VPC that is connected to one of AWS internal VPCs. It's actually possible to implement this for your own services too to share with another organisations VPCs, this is implemented using AWS PrivateLink.
For more information take a look at the Private DNS for interface endpoints page.
I am using few AWS Lambda functions, which are sitting inside private subnets,
These private subnets have VPC endpoints configured for the services for which the functions need access to,
The current setup does not use a NAT gateway, therefore all the traffic from the functions is going through the VPC endpoints.
I now have a use-case where we need to use a NAT gateway,
But would enabling NAT mean that the Functions would no longer use the VPC endpoints for external service access, and instead use the NAT?
I think this works as follows. For:
Gateway endpoints (S3, DynamoDB)
Routes to them are added automatically to our route tables when you create them. Docs says:
If you have an existing route in your route table for all internet
traffic (0.0.0.0/0) that points to an internet gateway, the endpoint
route takes precedence for all traffic destined for the service,
because the IP address range for the service is more specific than
0.0.0.0/0. All other internet traffic goes to your internet gateway, including traffic that's destined for the service in other Regions.
Interface VPC Endpoints
They work by modifying IP addresses in a DNS of a service. The IP address will be private addresses of the endpoint interfaces. Docs says:
The hosted zone contains a record set for the default DNS name for the
service (for example, ec2.us-east-1.amazonaws.com) that resolves to
the private IP addresses of the endpoint network interfaces in your
VPC. This enables you to make requests to the service using its
default DNS hostname instead of the endpoint-specific DNS hostnames.
To use private DNS, you must set the following VPC attributes to true:
enableDnsHostnames and enableDnsSupport.
Conclusion
So in both cases, priority is given to the interfaces, not the internet. I recommend checking the links provided. They have more info with examples to double check my conclusions.
VPC Endpoints or NAT Gateway?
AWS services like EC2, RDS, Lambda, and ElastiCache come with an Elastic Network Interface (ENI), which enables communication from within your VPCs via Private Endpoints. However, many AWS services provide a REST API, available via the Internet only. A few examples: S3, DynamoDB, CloudWatch, SQS, and Kinesis.
There are three options to make these services accessible from private subnets:
A VPC Endpoint type: Gateway Endpoints is free of charge, but are only available for S3 and DynamoDB.
A VPC Endpoint type: Interface Endpoint costs $7.20 per month and AZ plus $0.01 per GB and is available for most AWS services.
A NAT Gateway can be used to access AWS services or any other services with a public API. Costs are $32.40 per month and AZ plus $0.045 per GB.
Keep the following rules of thumb in mind when designing your network architecture.
Adding Gateway Endpoints for S3 and DynamoDB should be your default option.
Do you need to access non-AWS resources via the Internet, add a NAT Gateway. Do the math if traffic to AWS services justifies additional Interface Endpoints.
Are you only accessing AWS services from the private subnets? No more than four different services? Use Interface Endpoints. Otherwise, do the math to calculate costs for Interface Endpoints and NAT Gateway.
Ref Link: https://cloudonaut.io/advanved-aws-networking-pitfalls-that-you-should-avoid/
I would like to create a lambda(vpc) which would access resources in vpc and make a request to services(REST API) via public application load balancer. I found out that vpc end point is better solution than creating a nat gateway.
I have created a vpc endpoint for elasticloadbalancing(by following steps at https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) and given full access in the policy. I could not find how to access it from the lambda, what would be the URL to make the request?
Edit:
Thanks to John for the info that vpc endpoint is used to connect to ELB API. So Vpc endpoint would not solve our issue.
We have our infra in vpc which includes database(accessible within vpc only) and application servers running behind the ELB. For certain tasks we want to run lambda which will read database(for this reason lambda has to be inside vpc) and make API calls to our application using ELB. Since ELB is accessible from public dns only, lambda is not able to connect to ELB.
I have read that setting up NAT gateway is a solution. Are there other is simpler ways?
Yes, a NAT Gateway would allow the traffic from a private subnet to go out of the VPC and come back in to the Load Balancer's public IP addresses (via its Public DNS Name).
Alternatively, you could create an additional Internal Load Balancer that could accept traffic from within the VPC and send it to the Amazon EC2 instances.
I want to implement the scenario where AWS VPC I want connectivity to both Direct Connect and IGW, so some application I can access from data center and access aws resources from internet
Yes. You can provision an Internet Gateway on the VPC to provide direct access to the Internet, and you can also provision a Virtual Private Gateway that connects to Direct Connect for connectivity to your on-premises or data center networks.
The Route Table configuration will determine which traffic is routed to each gateway.
If your only need for using an Internet Gateway is to "access aws resources from internet" (rather than making your services public), then you should put your resources in private subnets and place a NAT Gateway in a public subnet. This way, your AWS resources will have outbound Internet connectivity via the NAT Gateway without being exposed to the Internet.