I'd like to have a lambda expression that every hour makes a query on RDS database, pull some ARN (device tokens) and then sends these devices a notification via SNS. My desire is to remain inside the VPC and I'd like to avoid using NAT due to its cost. Should i create a VPC endpoint (is this called AWS PrivateLink?) that can reach out SNS+RDS? Is NAT and Endpoint similar in billing? Globally is this the right way to achieve a "cron sending notifications" on AWS?
RDS is reachable inside the VPC without the endpoint isn't it?
This is totally possibly
Your lambda can run in a private subnet in the VPC that communicates with your RDS database over the network on port 3306. Make sure your security group and NACL rules allow this.
You then need to create a VPC endpoint to your SNS service. Be sure your route table includes this route.
The main difference between VPC endpoints and AWS Privatelink is the following:
A VPC endpoint enables you to privately connect your VPC to supported AWS services. SNS, Kinesis, SQS, S3,....
While AWS Privatelink is more about creating your own application in your VPC and configuring it as an AWS PrivateLink-powered service (referred to as an endpoint service). Other AWS principals can create a connection from their VPC to your endpoint service using an interface VPC endpoint.
VPC endpoints are free. Nat Gateways cost per hour per GB.
Related
I have VPC with couple of subnets containing EC2 instances.
The EC2 instances have code that invokes various AWS services like dybamodb.
Is the connection from EC2 to AWS Service (like dynamodb) happening within the AWS Network, or via public internet?
Is there any way to control this?
Is the connection from EC2 to AWS Service (like dynamodb) happening within the AWS Network, or via public internet?
Technically the process on EC2 would be hitting the AWS DynamoDB public API which is on the Internet. The traffic would be routed through the Internet Gateway you have attached to the VPC. I think if it is all in the same region it may not actually leave the AWS data center, and you could try testing that via tools like traceroute, but I don't think there are any guarantees of that.
Is there any way to control this?
Yes, add a VPC Endpoint to your VPC for the service you want to connect to. Then the DNS server in your VPC will route all traffic to that service over the VPC Endpoint, instead of routing it to your VPC's Internet Gateway. The traffic will then be guaranteed to stay within the AWS network.
I have a VPC Endpoint Service exposing a MicroService deployed in a private VPC. There are multiple VPC Endpoints created in other AWS accounts and private VPCs that connect to my VPC Endpoint Service.
Is there a way to tell from with in the MicroSevice which VPC Endpoint called it?
You can use VPC flow log to check traffic flowing through your VPC or subnet level. These VPC flow logs while creating them has to be configured to be send logs to either S3 or CloudWatch logs.
We have 2 regions, primary and secondary where the VPC is configured so that the EC2 instances in that VPC would make requests to a private VPC endpoint that would serve up DynamoDB from that region. Our Amazon DynamoDB tables are global tables. The goal is to have our requests stay within the Amazon network for security reasons.
We have a scheduled task that would run on an EC2 instance in our primary region. We want to make it more resilient by having it failover DynamoDB requests to the secondary region in the event that the primary region DynamoDB service is degraded. This was recommend by AWS in the Availability and Durability section.
I've looked through these documentations: Endpoints for Amazon DynamoDB and Using Amazon VPC Endpoints to Access DynamoDB, but they don't seem to offer any solution. Is it even possible to make requests to a private VPC endpoint from another region?
The goal is to have multi-region resilient and good security by not having requests going out to the internet.
Unfortunately this isn't possible from the documentation at https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-ddb.html:
Endpoints currently do not support cross-region requests—ensure that you create your endpoint in the same Region as your DynamoDB tables.
Also documented here: https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-gateway.html#vpc-endpoints-limitations
Endpoint connections cannot be extended out of a VPC. Resources on the other side of a VPN connection, VPC peering connection, transit gateway, AWS Direct Connect connection, or ClassicLink connection in your VPC cannot use the endpoint to communicate with resources in the endpoint service.
I have a lambda function which must be placed in a VPC because it communicates over a VPC peering connection. I also need it to upload to my Elasticsearch Service domain (not in a VPC because it is publicly accessible). But currently when Lambda tries to talk to ES, it times out.
I ran into this problem with Secrets Manager, so I went into the lambda's VPC, clicked on Endpoints, and created a Secrets Manager endpoint. I would like to do the same with Elasticsearch Service, but it doesn't show up in my list of AWS Services, which is odd because it definitely is an AWS Service.
Any idea how I can add an endpoint for Elasticsearch Service in my VPC so my Lambda function can talk to it? Thanks!
There is no VPC endpoint for ES. The services that support the endpoints are listed here.
Regarding the timeout, you would have to provided detailed description of your VPC setup, subnets, route tables, lambda and explain how did you setup your ES and how do you try to access it from VPC. Its difficult to speculate why it times out without all the details.
But since your ES is public, one possibility could be because lambda in VPC will not be able to connect to it without the use of NAT gateway. By default, lambda in a VPC does not have internet connectivity.
I need to connect dynamoDb and elasticache from aws-lambda (otherthan using NAT Gateway).
ElastiCache provides essential caching methods along with help in making the Lambda state-ful. The concern is that for Lambda to work nice with DynamoDB it should be set to NoVPC.
If we have to use ElastiCache, Lambda and both have to be in the same VPC.TO use Both ElastiCache and DynamoDB together is quite a challenge specially with Lambda. Given the VPC challenges.Do you have any suggestions to make this easier?
A Lambda function would have to have VPC access to connect to ElastiCache, and it would have to have access to resources outside the VPC to access DynamoDB so it would require a NAT gateway. There is no way to provide access to both of those services to a single Lambda function without enabling VPC access and setting up a NAT gateway.
If you just need a Redis server and aren't required specifically to use ElasiCache, then you could use a RedisLabs instance which wouldn't require you to enable VPC access on your Lambda function.
There is now a relatively easy solution for DynamoDb access from a VPC: VPC Endpoints.
"Previously, if you wanted your EC2 (elroy: or lambda) instances in your VPC to be able to access DynamoDB, you had two options. You could use an Internet Gateway (with a NAT Gateway or assigning your instances public IPs) or you could route all of your traffic to your local infrastructure via VPN or AWS Direct Connect and then back to DynamoDB."
"A VPC endpoint for DynamoDB enables Amazon EC2 instances in your VPC to use their private IP addresses to access DynamoDB with no exposure to the public Internet...Your EC2 instances do not require public IP addresses, and you do not need an Internet gateway, a NAT device, or a virtual private gateway in your VPC. You use endpoint policies to control access to DynamoDB. Traffic between your VPC and the AWS service does not leave the Amazon network. "
The above quotes come from the links below. Note the the references to "EC2 instances" apply to lambda contexts as well.
See https://aws.amazon.com/blogs/aws/new-vpc-endpoints-for-dynamodb/
and
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/vpc-endpoints-dynamodb.html