What is the difference between Private Link and VPC endpoint? As per the documentation it seems like VPC endpoint is a gateway to access AWS services without exposing the data to internet. But the definition about AWS private link also looks similar.
Reference Link:
https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-services-overview.html
Does Private Link is the superset of VPC endpoint?
It would be really helpful if anyone provides the difference between these two with examples!
Thanks in Advance!
AWS defines them as:
VPC endpoint — The entry point in your VPC that enables you to connect privately to a service.
AWS PrivateLink — A technology that provides private connectivity between VPCs and services.
So PrivateLink is technology allowing you to privately (without Internet) access services in VPCs. These services can be your own, or provided by AWS.
Let's say that you've developed some application and you are hosting it in your VPC. You would like to enable access to this application to services in other VPCs and other AWS users/accounts. But you don't want to setup any VPC peering nor use Internet for that. This is where PrivateLink can be used. Using PrivateLink you can create your own VPC endpoint services which will enable other services to use your application.
In the above scenario, VPC interface endpoint is a resource that users of your application would have to create in their VPCs to connect to your application. This is same as when you create VPC interface endpoint to access AWS provided services privately (no Internet), such as Lambda, KMS or SMS.
There are also Gateway VPC endpoints which is older technology, replaced by PrivateLink. Gateways can only be used to access S3 and DynamoDB, nothing else.
To sum up, PrivateLink is general technology which can be used by you or AWS to allow private access to internal services. VPC interface endpoint is a resource that the users of such VPC services create in their own VPCs to interact with them.
Suppose there is a website xyz.com that I am hosting in a bunch of Ec2 instances, exposed to the outside world thru a Network load balancer.
Now, a client who has his/her own AWS account, wants to access this xyz.com from an Ec2 running in their aws account.
One approach is to go thru the Internet.
However the client wants to avoid the internet route.
He/she wants to use the AWS backbone to reach xyz.com.
The technology that enables that, is AWS Private link.
(note that if you search for Private Link in the AWS services, there will be none.
You will get "End point services" as the closest hit)
So, this is how to route traffic through the AWS backbone:
I, the owner of xyz.com, will create a VPC End Point Service (NOTE the keyword Service here)
The VPC End point service will point to my Network load balancer.
I will then give my VPC End point service name to the client.
The client will create a VPC End Point (NOTE.. this is different from #1).
While creating it, the client will specify the VPC End Point Service name (from #1) that he got from me.
I can choose to be prompted to accept the connection from the client to my VPC End point service.
As soon as I accept it, then the client can reach xyz.com from his/her EC2 instance.
There is no Internet, no direct connect or VPN.. this simply works; and its secure.
And which technology enabled it.. AWS Private link !!!
PRIVATE LINK IS THE ONLY TECHNOLOGY THAT ALLOWS 2 VPCS TO CONNECT THAT HAVE OVERLAPPING CIDR RANGES.
A useful way in understanding differences is in how they technically connect private resources to public services.
Gateway Endpoints route traffic by adding prefix lists within a VPC route table which targets the Gateway endpoint. It is a logical gateway object similar to a Internet Gateway.
In contrast, an Interface Endpoint uses Privatelink to inject into a VPC at the subnet level, via an Elastic Network Interface (ENI), giving network interface functionality, and therefore, DNS and private IP addressing as a means to connect to AWS public services, rather than simply being routed to it.
The differences in connections offer differing advantages and disadvantages (availability, resiliency, access, scalability, and etc), which then dictates how best to connect private resources to public services.
Privatelink is simply a very much abstracted technology to allow a more simplified connection by using DNS. The following AWS re:Invent offers a great overview of Privatelink: https://www.youtube.com/watch?v=abOFqytVqBU
As you correctly mentioned in the question that both VPC endpoint and AWS private link do not expose to internet. On AWS console under VPC, there is a clear option available to create an endpoint. But there is no option/label to create AWS private link. Actually, there is one more option/label called endpoint service. Creating endpoint service is one way to establish AWS private link. At one side of this AWS private link is your endpoint service and at the other side is your endpoint itself. And interestingly we create both these sides in two different VPCs. In other words, you are connecting two VPCs with this private link (instead of using internet or VPC peering).
understand like,
VPC1 got endpoint service ----> private link -----> VPC2 got endpoint
Here endpoint service side is service provider while endpoint is service consumer. So when you have some service (may be some application or s/w) that you think other VPC endpoints can consume you create endpoint service at your end and consumers will create endpoints at there end. When consumers create endpoints at their end they have to give/select your service name and thus private link will be established with your service.
Ultimately you can have multiple consumers of your service just like one to many relationship.
Related
A VPC endpoint enables connections between a virtual private cloud (VPC) and supported services, without requiring that you use an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection. Therefore, your VPC is not exposed to the public internet.
AWS PrivateLink is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services), and supported AWS Marketplace partner services. You do not need to use an internet gateway, NAT device, public IP address, AWS Direct Connect connection, or AWS Site-to-Site VPN connection to communicate with the service. Therefore, your VPC is not exposed to the public internet.
What's so insecure of using public internet if all my incoming(to AWS)/outgoing(from AWS) connections use https? Could this still be decrypted?
I'm not very knowledgeable in networking/security that's why I'm asking.
I think you are confusing two architectures.
Public applications - available over the internet. They require direct or peroxided internet access. One way to secure them is through HTTPS as you do. For them to work your VPC needs internet gateway and/or NAT as well.
Private applications - those applications are meant to be only accessible from within a AWS. Thus internet access to these applications is not required, and is bad practice. This is where AWS PrivateLink can be used. It allows you to expose your application to other AWS users, without them needing internet access or even access to your VPC. They can access your private applications from their own accounts and VPCs.
When I configure an AWS Gateway VPC endpoint, a route table entry is created that points to the Gateway. Here, Gateway can be thought of performing the routing to AWS service (over private network).
However, for an AWS Inteface VPC endpoint, all that is visible is a Network interface that has a private IP address of the subnet. By default, a private IP can send traffic within the subnet or entire VPC provided Security Group and NACL allows the traffic. & it appears in this case there is no Route table entry to a Gateway or a Router for allowing traffic outside VPC.
How / Where is the interface routing the traffic to i.e. How does traffic leave the customer VPC?
Of course I understand that the traffic finally reaches the intended AWS service over private network but here I am trying to find out where is the Gateway or Router? Does AWS hide this implementation?
I cannot get my head around the fact that a simple Network Interface can accept traffic and route it to a service all by itself i.e. performing routing by itself? Clearly, in this case the traffic appears not flowing through the VPC router or another Gateway device.
I am aware this might be an AWS confidential implementation but any thoughts / idea on how they might have designed this feature?
It doesn't provide routing at all, by default a VPC interface endpoint when created will create an ENI per subnet in the VPC for you. It will also provide you a DNS name per each AZ and a global name that you can use within your applications.
In addition it supports the ability to have the AWS service domain name for the VPC interface endpoint be resolvable to the private IPs of the endpoint. As long as your VPC has DNS enabled it will first check the VPC private DNS resolver and then resolve it to the private IP rather than the public one.
This is done by adding an additional private hosted zone to your VPC which resolves service domains in your region such as ec2.us-east-1.amazonaws.com.
From the AWS side this is just an ENI created in your AWS VPC that is connected to one of AWS internal VPCs. It's actually possible to implement this for your own services too to share with another organisations VPCs, this is implemented using AWS PrivateLink.
For more information take a look at the Private DNS for interface endpoints page.
I am using few AWS Lambda functions, which are sitting inside private subnets,
These private subnets have VPC endpoints configured for the services for which the functions need access to,
The current setup does not use a NAT gateway, therefore all the traffic from the functions is going through the VPC endpoints.
I now have a use-case where we need to use a NAT gateway,
But would enabling NAT mean that the Functions would no longer use the VPC endpoints for external service access, and instead use the NAT?
I think this works as follows. For:
Gateway endpoints (S3, DynamoDB)
Routes to them are added automatically to our route tables when you create them. Docs says:
If you have an existing route in your route table for all internet
traffic (0.0.0.0/0) that points to an internet gateway, the endpoint
route takes precedence for all traffic destined for the service,
because the IP address range for the service is more specific than
0.0.0.0/0. All other internet traffic goes to your internet gateway, including traffic that's destined for the service in other Regions.
Interface VPC Endpoints
They work by modifying IP addresses in a DNS of a service. The IP address will be private addresses of the endpoint interfaces. Docs says:
The hosted zone contains a record set for the default DNS name for the
service (for example, ec2.us-east-1.amazonaws.com) that resolves to
the private IP addresses of the endpoint network interfaces in your
VPC. This enables you to make requests to the service using its
default DNS hostname instead of the endpoint-specific DNS hostnames.
To use private DNS, you must set the following VPC attributes to true:
enableDnsHostnames and enableDnsSupport.
Conclusion
So in both cases, priority is given to the interfaces, not the internet. I recommend checking the links provided. They have more info with examples to double check my conclusions.
VPC Endpoints or NAT Gateway?
AWS services like EC2, RDS, Lambda, and ElastiCache come with an Elastic Network Interface (ENI), which enables communication from within your VPCs via Private Endpoints. However, many AWS services provide a REST API, available via the Internet only. A few examples: S3, DynamoDB, CloudWatch, SQS, and Kinesis.
There are three options to make these services accessible from private subnets:
A VPC Endpoint type: Gateway Endpoints is free of charge, but are only available for S3 and DynamoDB.
A VPC Endpoint type: Interface Endpoint costs $7.20 per month and AZ plus $0.01 per GB and is available for most AWS services.
A NAT Gateway can be used to access AWS services or any other services with a public API. Costs are $32.40 per month and AZ plus $0.045 per GB.
Keep the following rules of thumb in mind when designing your network architecture.
Adding Gateway Endpoints for S3 and DynamoDB should be your default option.
Do you need to access non-AWS resources via the Internet, add a NAT Gateway. Do the math if traffic to AWS services justifies additional Interface Endpoints.
Are you only accessing AWS services from the private subnets? No more than four different services? Use Interface Endpoints. Otherwise, do the math to calculate costs for Interface Endpoints and NAT Gateway.
Ref Link: https://cloudonaut.io/advanved-aws-networking-pitfalls-that-you-should-avoid/
We are having several microservices on AWS ECS. We have single ALB which has different target group for different microservices. We want to expose some endpoints externally while some endpoints just for internal communication.
The problem is that if we put our load balancer in public VPC than it means that we are exposing all register endpoints externally. If we move load balancer to private VPC, we have to use some sort of proxy in public VPC, which required additional infra/cost and custom implementation of all security concerns like D-DOS etc.
What possible approaches we can have or does AWS provide some sort of out of the box solution for this ?
I would strongly recommend running 2 albs for this. Sure, it will cost you more (not double because the traffic costs won't be doubled), but it's much more straight forward to have an internal load balancer and an external load balancer. Work hours cost money too! Running 2 albs will be the least admin and probably the cheapest overall.
Checkout WAF. It stands for web application firewall and is available as AWS service. Follow these steps as guidance:
Create a WAF ACL.
Add "String and regex matching" condition for your private endpoints.
Add "IP addresses" condition for your IP list/range that are allowed to access private endpoints.
Create a rule in your ACL to Allow access if both conditions above are met.
Assign ALB to your WAF ACL.
UPDATE:
In this case you have to use external facing ALB in a public subnet as mentioned by Dan Farrell in comment below.
I would suggest doing it like this:
one internal ALB
one target group per microservice, as limited by ECS.
one Network load balancer(NLB), with one ip based target group.
The Ip based target group will have the internal ALB ip addresses,as the private ip addresses for ALB are not static, you will need to setup cloudwatch cron rule with this lambda function(forked from aws documentation and modified to work on public endpoints as well):
https://github.com/talal-shobaita/populate-nlb-tg-withalb/
Both ALB and NLB are scalable and protected from DDOS by AWS, AWS WAF is another great tool that can be attached directly to your ALB listener for extended protection.
Alternatively, you can wait for AWS to support multiple target group registration per service, it is already in their roadmap:
https://github.com/aws/containers-roadmap/issues/104
This how we eventually solved.
Two LB one in private and one in public subnet.
Some APIs meant to be public, so directly exposed through public LB.
For some private APIs endpoints need to be exposed, added a proxy in public LB and routed those particular paths from public LB to private LB through this proxy.
These days API Gateway is the best way to do this. You can have your API serve a number of different endpoints while serving only the public ones via API Gateway and proxying back to the API.
I don't see it mentioned yet so I'll note that we use a CloudMap for internal routing and an ALB for "external" (in our case simply intra/inter-VPC) communication. I didn't read in depth, but I think this article describes it.
AWS Cloud Map is a managed solution that lets you map logical names to the components/resources for an application. It allows applications to discover the resources using one of the AWS SDKs, RESTful API calls, or DNS queries. AWS Cloud Map serves registered resources, which can be Amazon DynamoDB tables, Amazon Simple Queue Service (SQS) queues, any higher-level application services that are built using EC2 instances or ECS tasks, or using a serverless stack.
...
Amazon ECS is tightly integrated with AWS Cloud Map to enable service discovery for compute workloads running in ECS. When you enable service discovery for ECS services, it automatically keeps track of all task instances in AWS Cloud Map.
You want to look at AWS Security Groups.
A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.
For each security group, you add rules that control the inbound traffic to instances, and a separate set of rules that control the outbound traffic.
Even more specific to your use-case though might be their doc on ELB Security Groups. These are, as you may expect, security groups that are applied at the ELB level rather than the Instance level.
Using security groups, you can specify who has access to which endpoints.
I have an api that I deploy using elasticbeanstalk, and I wish to make it completely internal so that it cannot be accessed from the public Internet. I am doing this because I only want the service to be accessible via our other services (that each run in their own VPCs and are all deployed via elasticbeanstalk).
Is it possible to use an internal ELB with elasticbeanstalk? Is this even what I need to do? Can VPC peering help me here?
The ultimate aim is that the api needs to have some http resources public and some private. My approach was going to be to make the service private and expose any public resources via API Gateway, but perhaps this is not the right solution. Would it perhaps be better to expose everything through API Gateway, require IAM auth on the private resources and enforce in the api that requests come from API Gateway?
Yes, you can add an ELB to any VPC, including those on private address ranges that are not accessible to the internet.
When configuring your Beanstalk instance, choose an appropriate VPC, unselect "Associate Public IP Address", place an ELB in at least one private subnet and select "Internal" for the ELB visibility.
The VPC configuration page looks like: