I've created a javascript lambda function that is triggered by an API gateway request. I've tested it from my frontend and it works fine (hosted outside AWS).
However, from my function I need to call an HTTP service from my on-prem network as an API call. I've tested the service through a local VPN connection in my machine and it works like a charm. Reading through the AWS documentation, I reached to the conclusion that I needed to configure my lambda to run inside a VPC and to connect that VPC to a site-to-site VPN.
I created the site-to-site VPN and the tunnels are up without issues. I created the client gateway for my on-prem network and also a Virtual Private Gateway for my AWS network. I checked the box to propagate to the routing table the on-prem IP prefixes I declared while creating the VPN. (I understand the IP prefixes are the on-prem IP endpoints I need to reach. For example let's say my services are hosted in 172.31.0.2 and 172.31.15.22, so I declared them as /32 since they're a single IP address and AWS asked me for a CIDR.)
I also created my VPC, associated it to the VPN and declared a subnet. The network admin gave me a CIDR range as routing option for our AWS services (let's say 172.31.50.160/29) so I declared the VPC CIDR as 172.21.0.0/16 and the subnet as 172.31.50.160/28. I also made sure lambda configurations specified my newly created VPC and subnet as well a security group.
While my frontend can still call the lambda without problems, I can't seem to reach the on-prem network. I always receive an ENOTFOUND error, so I assume this means the API call isn't going through the VPN tunnel, as the service is only reachable through the tunnel. So I'm guessing it's most likely a routing error. However, I'm not sure how to solve it.
My routing table shows the propagated IPs and the Virtual Private Gateway as their destination, as well as the 172.31.0.0/16 with the destination as Local. I imagine it could be a routing error, maybe I made a mistake setting that CIDR range as the VPC range.
I also tried adding cloudwatch logging to the VPC to check the traffic but nothing is logged, it always comes up empty. I made sure the IAM role I used for this had cloudwatch permissions, thinking that might be the issue, but even after that the logs are empty.
As you can see I have only very basic knowledge about networks, so any help is appreciated!
TL;DR
Goal: To allow my lambda function to access an on-prem service in a local machine.
Expected results: After connecting the lambda to a VPC that's associated to a running VPN, my lambda would be able to reach the local machine.
Actual results: The lambda is unable to locate the local machine (getaddrinfo ENOTFOUND error), seems as if the traffic is not going through the VPN tunnel.
Turns out that the same way that lambda cannot directly access the internet without a public NAT gateway and the right routing, lambda cannot access on-prem machines through the VPN without a private NAT gateway.
So I created a private subnet in my VPC, using the CIDR range the on-prem network administrator gave me. Then I routed my requests to the on-prem machines to the private NAT gateway. The routing table for this private subnet directed my traffic flow to the Virtual Private Gateway and in this way I was able to reach the on-prem machines.
Summary
Create a private subnet in your VPC (do not associate it to lambda)
Create a private NAT gateway and associate it with your new subnet
On the route table that manages the subnets associated with lambda, route your requests to the on-prem machines to the private NAT gateway you created on step 2
On the private NAT's routing table, route requests to your Virtual Private Gateway associated with your VPN
And done, you should be able to send HTTP requests from your lambda to your on-prem servers.
Related
My VPC is in eu-west-2. I have two subnets for an RDS instance, split across two different availability zones for reasons of high availability: eu-west-2a and eu-west-2b. I also have a Redshift cluster in its own subnet in eu-west-2c.
With this configuration, I have successfully configured an AWS Client VPN endpoint so that I can access RDS and Redshift from my local machine when connected to a VPN client with the appropriate configuration.
While following the same principles of using subnets for specific services, I would like my EC2 instances to live in private subnets that are also only accessible over a VPN connection. However, one of the limitations of the Client VPN service is:
You cannot associate multiple subnets from the same Availability Zone with a Client VPN endpoint.
This implies that I would need to create a separate endpoint for connecting to my private EC2 subnet—which feels like complete overkill for my modest networking architecture!
Is there a workaround?
By default, a subnet can reach the other subnets.
This means that you won't need to do anything. This will work out of the box. If not, check the route tables and see if there is a route from your VPN subnet to your private subnet.
When you associate the first subnet with the Client VPN endpoint, the following happens:
The state of the Client VPN endpoint changes to available. Clients can now establish a VPN connection, but they cannot access any resources in the VPC until you add the authorization rules.
The local route of the VPC is automatically added to the Client VPN endpoint route table. (This local route allows you to communicate with every subnet within the VPC that the subnet is in.)
The VPC's default security group is automatically applied for the Client VPN endpoint.
See documentation for details.
I read many posts about how to connect DynamoDB when using a lambda in VPC. They all talk about either I need to create VPC Endpoint and use it for Dynamo client configuration or I need to configure NAT gateway.
But, at the moment, in my current existing app, I don't use none of these. But, my lambda works ok. Only I see severe timeout when I execute benchmark tests which many people recommend using VPCE.
I just can't figure out how my lambda currently works (connect DDB and read/write there).
My lambda lives in VPC as I am also connecting Aurora Postgres.
In my account, I can't locate any single VPC endpoint nor NAT gateway. So, this just confuses me.
As I mentioned in my question, I re-confirmed that I can't locate either VPCE or NAT GW. I checked the correct region.
So, there is another case that, I believe, needs to be included as part of these documents or posting to avoid any confusion like this.
The scenario is that VPC can be connected to a TGW. Then, my VPC doesn't need to have the IGW/NAT to reach out to internet. All non local traffic will be routed to TGW and there it will be taken care of assuming there is IGW or NAT. They call it "Centralized outbound routing"
Centralized outbound routing to the internet using TGW
The following diagram shows the key components of the configuration for this scenario. You have applications in VPC A and VPC B that need outbound only internet access. You configure VPC C with a NAT gateway and an internet gateway. Connect all VPCs to a transit gateway. Configure routing so that outbound internet traffic from VPC A and VPC B traverses the transit gateway to VPC C. The NAT gateway in VPC C routes the traffic to the internet gateway
When I configure an AWS Gateway VPC endpoint, a route table entry is created that points to the Gateway. Here, Gateway can be thought of performing the routing to AWS service (over private network).
However, for an AWS Inteface VPC endpoint, all that is visible is a Network interface that has a private IP address of the subnet. By default, a private IP can send traffic within the subnet or entire VPC provided Security Group and NACL allows the traffic. & it appears in this case there is no Route table entry to a Gateway or a Router for allowing traffic outside VPC.
How / Where is the interface routing the traffic to i.e. How does traffic leave the customer VPC?
Of course I understand that the traffic finally reaches the intended AWS service over private network but here I am trying to find out where is the Gateway or Router? Does AWS hide this implementation?
I cannot get my head around the fact that a simple Network Interface can accept traffic and route it to a service all by itself i.e. performing routing by itself? Clearly, in this case the traffic appears not flowing through the VPC router or another Gateway device.
I am aware this might be an AWS confidential implementation but any thoughts / idea on how they might have designed this feature?
It doesn't provide routing at all, by default a VPC interface endpoint when created will create an ENI per subnet in the VPC for you. It will also provide you a DNS name per each AZ and a global name that you can use within your applications.
In addition it supports the ability to have the AWS service domain name for the VPC interface endpoint be resolvable to the private IPs of the endpoint. As long as your VPC has DNS enabled it will first check the VPC private DNS resolver and then resolve it to the private IP rather than the public one.
This is done by adding an additional private hosted zone to your VPC which resolves service domains in your region such as ec2.us-east-1.amazonaws.com.
From the AWS side this is just an ENI created in your AWS VPC that is connected to one of AWS internal VPCs. It's actually possible to implement this for your own services too to share with another organisations VPCs, this is implemented using AWS PrivateLink.
For more information take a look at the Private DNS for interface endpoints page.
I would like to create a lambda(vpc) which would access resources in vpc and make a request to services(REST API) via public application load balancer. I found out that vpc end point is better solution than creating a nat gateway.
I have created a vpc endpoint for elasticloadbalancing(by following steps at https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) and given full access in the policy. I could not find how to access it from the lambda, what would be the URL to make the request?
Edit:
Thanks to John for the info that vpc endpoint is used to connect to ELB API. So Vpc endpoint would not solve our issue.
We have our infra in vpc which includes database(accessible within vpc only) and application servers running behind the ELB. For certain tasks we want to run lambda which will read database(for this reason lambda has to be inside vpc) and make API calls to our application using ELB. Since ELB is accessible from public dns only, lambda is not able to connect to ELB.
I have read that setting up NAT gateway is a solution. Are there other is simpler ways?
Yes, a NAT Gateway would allow the traffic from a private subnet to go out of the VPC and come back in to the Load Balancer's public IP addresses (via its Public DNS Name).
Alternatively, you could create an additional Internal Load Balancer that could accept traffic from within the VPC and send it to the Amazon EC2 instances.
We're setting up an Amazon VPC in which we will provision (for now) a single EC2 instance and one RDS instance. This is to 'extend our data center', and should only be using private subnet(s).
So actually, we have this setup, and it is working well (insert smiley face icon). For all intents and purposes, we're mirroring the VPC scenario 4 outlined by Amazon here: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario4.html
tl;dr: A single VCP, with a VPN connecting to our corporate network. The VPN uses a Virtual Private Gateway (VPC end) and Customer Gateway (our end) to allow us access as necessary to the EC2, which contains a webserver connecting to the RDS instance as needed. Anyone on our network has access to the web server running on the EC2 via a URL. All this is working as expected.
The problem comes when the EC2 instance needs to access a resource on the Internet - The idea is for us to NOT have any public subnets, but to route all traffic from the EC2 instance through our VPN and out the 'standard' path of our corporate Internet access. However we're having trouble setting this up.
The fact that it can be done is hightlighted in Amazon's FAQ here:
https://aws.amazon.com/vpc/faqs/
Q. How do instances without public IP addresses access the Internet?
Instances without public IP addresses can access the Internet in one of two ways:
Instances without public IP addresses can route their traffic through a NAT gateway or a NAT instance to access the Internet. These instances use the public IP address of the NAT gateway or NAT instance to traverse the Internet. The NAT gateway or NAT instance allows outbound communication but doesn’t allow machines on the Internet to initiate a connection to the privately addressed instances.
For VPCs with a hardware VPN connection or Direct Connect connection, instances can route their Internet traffic down the virtual private gateway to your existing datacenter. From there, it can access the Internet via your existing egress points and network security/monitoring devices.
We are trying to avoid option #1 as there is a cost involved (along with complexity and security issues). #2 is the perfect resolution for us, but understanding the process to set it up has been eluding us for a while.
Can anyone walk us through what we need to do (or point us to the correct resources) to ensure the EC2 instance* can access the Internet by routing the traffic down the VPN, through our corporate datacenter, and our our existing Internet access point?
* and anything within the private subnet for that matter
If you are using scenario #2, then all there is to do on the AWS end is to ensure that traffic destined for the internet, 0.0.0.0/0 is routed to your Virtual Private Gateway.
Once traffic heads there, it will go to your Customer Gateway, and into your corporate datacenter. It's up to your local IT guys on that end to get Internet-destined traffic heading out, if it's even possible. But at that point, it's no longer an AWS issue.