I have a lambda job that works fine until I put it in a VPC, it seems to stop reading the kinesis stream as soon as that is done and works again when I put it in no VPC. Anyone have any advice on how to solve this?
For lambda to work properly in VPC, you need to add AWSLambdaVPCAccessExecutionRole managed policy to your function's execution role.
Also it needs to be remembered that lambda in your vpc does not have access to Internet:
When you connect a function to a VPC in your account, it does not have access to the internet unless your VPC provides access.
To enable access to public kinesis endpoints:
To give your function access to the internet, route outbound traffic to a NAT gateway in a public subnet. The NAT gateway has a public IP address and can connect to the internet through the VPC's internet gateway.
Alternatively, can setup VPC interface endpoints to access kinesis without going to the internet.
Hope this will be helpful.
Related
I read many posts about how to connect DynamoDB when using a lambda in VPC. They all talk about either I need to create VPC Endpoint and use it for Dynamo client configuration or I need to configure NAT gateway.
But, at the moment, in my current existing app, I don't use none of these. But, my lambda works ok. Only I see severe timeout when I execute benchmark tests which many people recommend using VPCE.
I just can't figure out how my lambda currently works (connect DDB and read/write there).
My lambda lives in VPC as I am also connecting Aurora Postgres.
In my account, I can't locate any single VPC endpoint nor NAT gateway. So, this just confuses me.
As I mentioned in my question, I re-confirmed that I can't locate either VPCE or NAT GW. I checked the correct region.
So, there is another case that, I believe, needs to be included as part of these documents or posting to avoid any confusion like this.
The scenario is that VPC can be connected to a TGW. Then, my VPC doesn't need to have the IGW/NAT to reach out to internet. All non local traffic will be routed to TGW and there it will be taken care of assuming there is IGW or NAT. They call it "Centralized outbound routing"
Centralized outbound routing to the internet using TGW
The following diagram shows the key components of the configuration for this scenario. You have applications in VPC A and VPC B that need outbound only internet access. You configure VPC C with a NAT gateway and an internet gateway. Connect all VPCs to a transit gateway. Configure routing so that outbound internet traffic from VPC A and VPC B traverses the transit gateway to VPC C. The NAT gateway in VPC C routes the traffic to the internet gateway
I would like my lambda function to be able to access the internet. I have followed the following AWS tutorial: AWS Knowledge Center Videos: How do I give internet access to my Lambda function in a VPC?
I have created a NAT.
Then I created a public-lambda network and created a route table that forwards default traffic to the NAT
Then on my lambda function I have attached it to the public-lambda network
Inside my lambda function I make a request to a server out on the web. It just times out. I don't get logs on why it timed out but I'm assuming it wasn't able to reach the internet.
Why I'm not being able to access the internet from my lambda function?
if your subnet has outbound access to internet(NAT/IGW) then definitely you lambda will get access to the internet.
first check with the subnet, try to spin one ec2 instance into that subnet and check the outbound connectivity using ping or curl.
second check the attached SG with lambda function.
just give a try, best of luck.
After watching AWS: How to Place Your Lambda Functions in a VPC - YouTube I was able to fix my issue.
The fix was that I needed to create a private subnet and create a routing table to route all default traffic to the NAT.
I have an AWS lambda function that makes a request to the internet. When it makes the request with NO VPC, it's ok, but when I add it to the VPC, it stops working. I've attached an Internet Gateway to the VPC and created a NAT Gateway with RT to use outbound 0.0.0.0/0, but it stills not working.
With the 15seconds timeout, it's always throwing TO.
Could you please help me?
I've already followed these sites:
https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7
http://derpturkey.com/lambda-vpc-and-internet-access-configuration/
Everything seems to be well configured.
Lambda configuration
Route Table configuration
IGW attached to VPC
There are three ways to grant Internet access to an AWS Lambda function:
Do not assign it a VPC, or
Assign it to a VPC and attach an Elastic IP Address to the ENI (Elastic Network Interface) that is created in the VPC, or
Assign it to a private subnet in a VPC and use a NAT Gateway to grant the private subnet access to the Internet
If you have already verified the network configurtions like VPC setup, Subnets, Route Table, IG/NAG gate way. And associated the Lambda with current subnet and security groups.
Then the last thing to verify would be:
For Lambda functions to work into custom or User Defined VPC, Adding Lambda to the VPC would require AWSLambdaVPCAccessExecutionRole in addition to AWSLambdaBasicExecutionRole.
Ensure that the associated role has the above permissions.
Source:
https://docs.aws.amazon.com/lambda/latest/dg/vpc-rds-create-iam-role.html
I am trying to decrypt a file that is stored inside an S3 bucket using KMS. Within an EC2 instance, I am able to retrieve this file from S3, but when I try to decrypt it using KMS, I get the following error:
HTTPSConnectionPool(host='kms.us-east-1.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(, 'Connection to kms.us-east-1.amazonaws.com timed out. (connect timeout=60)'))
I am however able to decrypt this file successfully from my local machine. Does anyone know why this may be?
If additional info is required, I would be happy to provide it :)
When you enable VPC for your Lambda AWS shows you following message -
When you enable VPC, your Lambda function will lose default internet access. If you require external internet access for your function, ensure that your security group allows outbound connections and that your VPC has a NAT gateway.
So make sure your lambda has access to internet with correct security groups and NAT gateway.
Also note having IG for your subnet is not sufficient. You need to have a NAT instance or gateway and a route from your subnet to that gateway in route table for lambda to have internet access associated to a custom VPC.
AWS Lambda uses the VPC information you provide to set up ENIs that allow your Lambda function to access VPC resources. Each ENI is assigned a private IP address from the IP address range within the Subnets you specify, but is not assigned any public IP addresses. Therefore, if your Lambda function requires Internet access (for example, to access AWS services that don't have VPC endpoints, such as Kinesis), you can configure a NAT instance inside your VPC or you can use the Amazon VPC NAT gateway. You cannot use an Internet gateway attached to your VPC, since that requires the ENI to have public IP addresses.
Important :
If your Lambda function needs Internet access, do not attach it to a public subnet or to a private subnet without Internet access. Instead, attach it only to private subnets with Internet access through a NAT instance or an Amazon VPC NAT gateway.
Source : http://docs.aws.amazon.com/lambda/latest/dg/vpc.html
One case can be that you are connecting to s3 bucket using VPC endpoint and not have an internet connection.
If this is the case then you have to enable internet connectivity either via NAT or putting your instance in a public subnet.
I need to connect dynamoDb and elasticache from aws-lambda (otherthan using NAT Gateway).
ElastiCache provides essential caching methods along with help in making the Lambda state-ful. The concern is that for Lambda to work nice with DynamoDB it should be set to NoVPC.
If we have to use ElastiCache, Lambda and both have to be in the same VPC.TO use Both ElastiCache and DynamoDB together is quite a challenge specially with Lambda. Given the VPC challenges.Do you have any suggestions to make this easier?
A Lambda function would have to have VPC access to connect to ElastiCache, and it would have to have access to resources outside the VPC to access DynamoDB so it would require a NAT gateway. There is no way to provide access to both of those services to a single Lambda function without enabling VPC access and setting up a NAT gateway.
If you just need a Redis server and aren't required specifically to use ElasiCache, then you could use a RedisLabs instance which wouldn't require you to enable VPC access on your Lambda function.
There is now a relatively easy solution for DynamoDb access from a VPC: VPC Endpoints.
"Previously, if you wanted your EC2 (elroy: or lambda) instances in your VPC to be able to access DynamoDB, you had two options. You could use an Internet Gateway (with a NAT Gateway or assigning your instances public IPs) or you could route all of your traffic to your local infrastructure via VPN or AWS Direct Connect and then back to DynamoDB."
"A VPC endpoint for DynamoDB enables Amazon EC2 instances in your VPC to use their private IP addresses to access DynamoDB with no exposure to the public Internet...Your EC2 instances do not require public IP addresses, and you do not need an Internet gateway, a NAT device, or a virtual private gateway in your VPC. You use endpoint policies to control access to DynamoDB. Traffic between your VPC and the AWS service does not leave the Amazon network. "
The above quotes come from the links below. Note the the references to "EC2 instances" apply to lambda contexts as well.
See https://aws.amazon.com/blogs/aws/new-vpc-endpoints-for-dynamodb/
and
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/vpc-endpoints-dynamodb.html