Customer Gateway
A customer gateway is the anchor on your side of that connection. It
can be a physical or software appliance.
Customers can connect multiple customer gateways
(CGW) to a single VPC virtual private gateway (VGW) or can connect a
single router to multiple VPC VGWs.
Is there a difference between a router and customer gateway? From AWS faq section https://aws.amazon.com/vpc/faqs/#C9, it seems they are one and the same.
A Customer Gateway is the generic name describing a device that can speak IPSec and terminate the tunnels.
It could be a router, it could be a Linux server, but perhaps the most commonly used device is a hardware firewall device from a vendor like Cisco, Juniper, Sonicwall, etc.
Some routers can serve the purpose, but many routers do not have the capability of performing all of the things a customer gateway needs to do.
The section called Your Customer Gateway is written from the perspective of explaining this device to a technical person responsible for implementing one.
Your company has decided to use an optional Amazon VPC VPN connection that links your data center (or network) to your Amazon VPC virtual private cloud (VPC). A customer gateway is the anchor on your side of that connection. It can be a physical or software appliance.
It explains the necessary requirements in high-level, yet technical terms.
Related
I understand GCP Private Service Connect (PSC) is an effective solution for enabling service-centric private network connectivity for GCP APIs and other hosted services within and across VPC projects/organizations/on-prem setup based on Producer/Consumer project model.
GCP documentation on Private Service Connect explains the purpose, PSC Endpoint and Publish Service configuration however I don't find relevant details on what should be an ideal location (or best practice) for creating and configuring PSC Endpoint, Publish Service when you have Shared VPC based network setup.
IMO, PSC Endpoint & Publish Service are network resources so the ideal place is to create in 'Host Project' of Shared VPC Network as a 'Host Project' is meant for centralized network management.
Also, having PSC Endpoint and Publish Service in 'Host Project' will help in sharing a single 'PSC endpoint' for all the 'Service Project' resources (which otherwise require multiple PSC endpoints per Service Project). However I would like to understand from you if you have come across and/or implemented such scenario.
Update: I tried a Shared VPC setup wherein PSC creation is allowed in Service-Project which means GCP doesn't restrict the creation of PSC in Service-Project.
Host Project:
Service Project:
Service Project PSC setup:
I'm trying to build an AWS terraform IPSec VPN config. However, I can't remember where to find the AWS IPSec IP address; the terraform cgw documentation says the ip_address field is required.
The answer should assume the VPN will be attached to my AWS Transit Gateway.
My terraform:
resource "aws_customer_gateway" "cgw-abbv-for-local-and-remote" {
bgp_asn = 65001
ip_address = "A.B.C.D" #<-- I need this IP before terraform apply
type = "ipsec.1"
tags = {
Name = "insert-cgw-name-here"
}
}
resource "aws_vpn_connection" "vpn-abbv-for-local-and-remote" {
customer_gateway_id = aws_customer_gateway.cgw-abbv-for-local-and-remote.id
transit_gateway_id = aws_ec2_transit_gateway.my-tgw-name.id
type = aws_customer_gateway.cgw-abbv-for-local-and-remote.type
tags = {
Name = "insert-vpn-name-here"
}
}
Seems like OP already found the answer, but let me add my two cents since I spent a lot of time figuring things out when it comes to AWS VPN two years ago in order to pass the AWS Advanced Networking cert. This could potentially turn out useful for folks that are new to VPN - especially in the AWS ecosystem:
There is a fantastic book called AWS Certified Advanced Networking Official Study Guide which I would recommend everyone in a cloud network engineer role to read. [1]
It points out the following:
After you create a VPN connection, the VPN tunnel activates when traffic is generated
from your side of the VPN connection. The VGW is not the initiator; your customer gateway must initiate the tunnels. If your VPN connection experiences a period of idle time (usually
10 seconds, depending on your configuration), the tunnel may go down. This is because
AWS uses an on-demand DPD mechanism. If AWS receives no traffic from a VPN peer for
10 seconds, AWS sends a DPD “R-U-THERE” message. If the VPN peer does not respond
to three successive DPDs, the VPN peer is considered dead and AWS closes the tunnel. [pp. 100, 101]
At the non-AWS end of a VPN connection, the VPN is terminated on a customer gateway.
A customer gateway is the AWS term for the VPN termination device at the customer’s onpremises end. A customer gateway can also be hosted in AWS as an EC2 instance running
VPN software that meets the requirements given in the next section.
Most customers don’t require the purchase of an additional device and can reuse an
existing on-premises VPN termination device to create a tunnel to a VPC. [p. 110]
You can use any third-party VPN device that supports Layer 3 VPN technologies. AWS
does not support Layer 2 VPN technologies.
IPsec is used for the VGW at the AWS end of VPN termination, and so the IPsec protocol must be supported by your VPN device. You will set up two VPN tunnels per VGW.
Support for BGP routing protocol is optional but recommended for advanced routing capabilities. Other routing protocols like Open Shortest Path First (OSPF) are not supported by
AWS. You must ensure that you have opened the right ports in your on-premises firewall
for the IPsec traffic to flow. [p. 111]
That is in particular: both ends of the VPN connection must possess a public IP address!
If you didn't already, I really really recommend skipping through these pages to be aware of best-practices and the AWS-way of thinking when it comes to (hybrid) cloud architectures. You avoid getting confused afterwards if things didn't go the way you wanted to. IPSec (i.e. Layer-3) VPNs are harder to get right then most people think. One should be aware of all the routing and security relevant stuff such as: IKE, SAs, Policy-based routing, NAT-Traversal, ISAKMP etc. [see also p. 97: VPN Features -> Security & Routing sections].
Another good reference is the AWS Site-to-Site VPN guide (PDF). [2]
Also good to know: Many terraform attributes can also be found in the AWS CloudFormation docs. The docs for the AWS::EC2::CustomerGateway resource's IpAddress attribute state [3]:
The Internet-routable IP address for the customer gateway's outside interface. The address must be static.
[1] https://www.programmer-books.com/wp-content/uploads/2019/04/AWS-Certified-Advanced-Networking-Official-Study-Guide.pdf
[2] https://docs.aws.amazon.com/vpn/latest/s2svpn/s2s-vpn-user-guide.pdf
[3] https://docs.aws.amazon.com/de_de/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-customer-gateway.html#cfn-ec2-customergateway-ipaddress
This is not very clear in the terraform documentation, but I found an example on the internet that clarified this question.
In short, the aws_customer_gateway config is not on the AWS side of the IPSec tunnel... these resources are "remote" with respect to AWS:
So in this case, the ip_address will be the destination ip address of AWS IPSec packets leaving the AWS Transit Gateway; the aws_customer_gateway ip_address is not owned by AWS.
The ip_address for the Customer Gateway is the IP of the physical appliance/router sitting on-premise in the customer's data center. You need this ip_address to establish a VPN connection. The AWS docs help when you get lost in terraform as well.
I've been asked to look into an AWS setup for my organisation but this isn't my area of experience so it's a bit of a challenge. After doing some research, I'm hoping that API Gateway will work for us and I'd really appreciate it if someone could tell me if I'm along the right lines.
The plan is:
We create a VPC with several private subnets. The EC2 instances in the subnets will be hosting browser based applications like Apache Guacamole, Splunk etc.
We attach to the VPC an API Gateway with a REST API which will allow users access to only the applications on 'their' subnet
Users follow a link to the API Gateway from an external API which will provide Oauth2 credentials.
The API Gateway REST API verifies their credentials and serves them with a page with links to the private IP addresses for the services in 'their' subnet only. They can then click on the links and open the Splunk, Guacamole browser pages etc.
I've also looked at Client VPN as a possible solution but my organisation wants users to be able to connect directly to the individual subnets from an existing API without having to download any other tools (this is due to differing levels of expertise of users and the need to work remotely). If there is a better solution which would provide the same workflow then I'd be happy to implement that instead.
Thanks for any help
This sounds like it could work in theory. My main concern would be if Apache Guacomole, or any of the other services you are trying to expose, requires long lived HTTP connections. API Gateway has a hard requirement that all requests must take no longer than 29 seconds.
I would suggest also looking into exposing these services via a public Application Load Balancer, instead of API Gateway, which has OIDC authentication support. You'll need to look at the requirements of the specific services you are trying to expose to evaluate if API Gateway or ALB would be a better fit.
I would personally go about this by configuring each of these environments using an Infrastructure as Code, in such a way that you can create a new client environment by simply running your IaC tool with a few parameters like the client ID and the domain name or subdomain you want to use. I would actually spin each up in their own VPC since it sounds like you want each client's environment to be isolated from the others.
The official sample of AWS Advanced Networking Speciality questions contains a question about the most cost-effective connection between your
on-premises data centre and AWS ensuring confidentiality and integrity of the data in transit to your VPC (the question #7).
The correct answer implies establishing of the managed VPN connection between the customer gateway appliance and the virtual private gateway over the Direct Connect connection.
However one of the possible options in the list of answers offers a software VPN solution ("Set up an IPsec tunnel between your customer gateway and a software VPN on Amazon EC2 in the
VPC"). The explanation why this answer is incorrect says that:
it would not take
advantage of the already existing Direct Connect connection
My question is: why would not this software VPN connection take advantage of the already existing DC connection? What's the principal difference here?
Option 1: The question is flawed.
If you built a tunnel between a customer gateway device and an EC2 instance with traffic routing through the Direct Connect interconnection, then you are quite correct -- that traffic would use the existing Direct Connect connection.
If, on the other hand, you built a tunnel from the customer gateway to an EC2 instance over the Internet, then of course that traffic would not use the Direct Connect route.
There appears to be an implicit assumption that a tunnel between a device on the customer side and an EC2 instance would necessarily traverse the Internet, and that is a flawed assumption.
There are, of course, other reasons why the native solution might be preferable to a hand-rolled one with EC2 (e.g. survival of a the complete loss of an AZ or avoidance of downtime due to eventual instance hardware failures), but that isn't part of the scenario.
Option 2. The answer is wrong for a different reason than the explanation offered.
Having written and reflected on the above, I realized there might be a much simpler explanation: "it would not take advantage of the already existing Direct Connect connection" is simply the wrong justification for rejecting this answer.
It must be rejected on procedural grounds, because of the instruction to Choose 3. Here are the other two correct answers.
A) Set up a VPC with a virtual private gateway.
C) Configure a public virtual interface on your Direct Connect connection.
You don't need to have either of these things in order to implement a roll-your-own IPSec tunnel between on-premise and EC2 over Direct Connect. A Virtual Private Gateway is the AWS side of an AWS-managed VPN, and a Public Virtual Interface is necessary to make one of those accessible from inside Direct Connect (among other things, but it is not necessary in order to access VMs inside a VPC using private IPs over Direct Connect).
I would suggest that the answer you selected may simply be incorrect, because it doesn't belong with the other two, and the explanation that is offered misses the point entirely, and the explanation is itself incorrect.
I am working on a project were lots of machines/sensors will be sending messages to Kafka/Nifi cluster directly. This machine/sensors will be pushing messages from public internet not from the corporate network. We are using a Hortonworks distribution on the AWS cloud.
My question is: what is the best architectural practice to setup Kafka /Nifi cluster for such use cases, I don't want to put my cluster in the public subnet in order to receive messages from the public internet.
Can you please help me with this?
Obviously you shouldn't expose your Kafka to the world. Therefore "sensor data directly to Kafka" is the wrong approach, IMO. At least, without using some SSL channel
You could allow a specific subnet of your external devices to reach the internal subnet, assuming you know that range, however I think your better option here is to use either Minifi or Streamsets SDC which are event collectors sitting on the sensors, which can encrypt traffic to an open Nifi or Streamsets cluster, which can then forward events to the internal Kafka cluster. You already have Nifi apparently, and therefore Minifi was built for this purpose
Another option could be the Kafka REST proxy, but you'll still need to setup authentication / security layers around it
Use AWS IoT to receive the devices communication, this option gives you a security layer and isolates you HDF sandbox from the internet.
AWS IoT Core provides mutual authentication and encryption at all points of connection, so that data is never exchanged between devices and AWS IoT Core without a proven identity.
Then import the information with a NiFi processor.