I have a lambda function which has the following logic in the handler:
log.info("about to get caller identity..")
caller_identity = boto3.client("sts").get_caller_identity()
log.info(caller_identity)
When I run this lambda function, it times out with the following error:
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "https://sts.amazonaws.com/"
Why is my lambda function not able to reach STS service?
Thanks!
This was the result of the Lambda being associated to a VPC in a private subnet with no way to communicate to the internet.
It is important that when using VPC configuration the Lambda is located in a subnet with the means to communicate with the internet such as a NAT. Without this your Lambda cannot communicate to the internet.
If you're trying to reach an AWS service you can check whether a VPC Endpoint is supported for the service to remove the need for internet connectivity.
Related
I have a Lambda function deployed into a public subnet in a VPC and I’m trying to connect to a Lambda function outside of a VPC and I’m running into connectivity issues.
I believe the security group settings and IAM policies will permit the connection, but I'm not sure if there's an issue with connecting to an out-of-VPC Lambda from an in-VPC one.
Is there a tool in AWS Console, AWS CLI or anywhere else that I can use to troubleshoot where the connection is failing? I’ve used the Reachability Analyzer before but it only works on a handful of resource types like EC2 instances.
I've tried invoking the out-of-VPC Lambda from inside my in-VPC Lambda, but the request doesn't work and I don't see any helpful information about what happened. I tried running the Reachability Analyzer, but it doesn't allow you to test if Lambda functions are reachable.
I was expecting the request to work, but I'm not sure if I need to configure a VPC interface endpoint because I'm connecting from an in-VPC Lambda to an out-of-VPC Lambda.
I’m new to networking and would appreciate any help.
I have a Lambda function deployed into a public subnet in a VPC and I’m trying to connect to a Lambda function outside of a VPC and I’m running into connectivity issues.
The Lambda function in the VPC never gets a public IP assigned to it. So it can't connect to anything outside of the VPC. It can't use the Internet Gateway attached to the public subnet because it doesn't have a public IP.
By "connect to a Lambda function outside of a VPC" what you are really doing is connecting to the AWS API outside of the VPC. You never "connect" to a Lambda function, because Lambda functions aren't running and just sitting around idle waiting for your request. Lambda functions don't really exist until a request comes in to the AWS Lambda Invoke API, at which point AWS spins up an instance of the Lambda function and passes it the invocation payload.
To fix this connectivity issue, you either need to create an AWS Lambda VPC Endpoint in your VPC, to handle requests to the Lambda API originating in your VPC. Or you need to move the VPC Lambda function to a private subnet, with a route to a NAT Gateway. Lambda functions in private subnets can access things outside the VPC by having their requests routed through the NAT Gateway.
I was expecting the request to work, but I'm not sure if I need to configure a VPC interface endpoint because I'm connecting from an in-VPC Lambda to an out-of-VPC Lambda.
That's not how VPC Interface Endpoints work. The entire purpose of VPC Interface Endpoints is to allow a resource inside your VPC to access part of the AWS API that exists outside the VPC. A VPC Interface Endpoint will absolutely allow your Lambda function running in the VPC to access the Lambda Invoke API, in order to trigger an execution of your out-of-VPC Lambda function.
I have a AWS Lambda function written in C# with a HTTP API Gateway to expose the lambda function.
When I try to invoke another endpoint via httpPost in c#, the lambda logs doesn't display any logs and the request via POSTMAN to the API Gateway returns Service Unavailable.
Should I enable CORS or anything else? I tried to enable CORS but the result still the same.
Can someone help me, please?
AWS Lambda functions running inside a VPC are never assigned a public IP address. So by default they can't connect to anything outside of the VPC. The only way to provide access to resources outside the VPC is to either place the Lambda functions in private VPC subnets with a route to a NAT gateway, or to create VPC endpoints for those services the Lambda function needs to connect to.
I have a ECS fargate container running inside a private VPC which doesn't have internet access. It needs to invoke a lambda via AWS SDK. Based on my understanding, AWS creates a default public endpoint for the lambda and when I call invokeLambda method the traffic will always go to internet. If my understanding is right, that means my Fargate container won't be able to call the lambda. Is it right?
If it is right, what is the alternative solution is? The goal is that the traffic won't go to internet in any chance.
Can I create a private link endpoint for my lambda?
Or create a API gateway with VPC endpoint which connects to lambda?
If my understanding is right, that means my Fargate container won't be able to call the lambda. Is it right?
Yes. Without NAT gateway or instance, you won't be able to directly invoke the lambda function from private subnet.
Can I create a private link endpoint for my lambda?
Sadly no. There are not VPC interface endpoints for lambda.
Or create a API gateway with VPC endpoint which connects to lambda?
Yes, this should be possible by creating private API gateway. The private API would be only accessible from within your VPC. But API gateway to lambda will still probably go over the internet.
The Security Overview of AWS Lambda whitepaper writes:
Invocations from Amazon Kinesis and DynamoDB streams, SQS queues, Application Load Balancer,and API Gateway follow the request-response path
For request-response invocations, the payload passes from the API caller—such as AWS API Gateway or the AWS SDK—to a load balancer, and then to the Lambda invoke service. This service identifies an execution environment for the function, and passes the payload to that execution environment to complete the invocation. Traffic to the load balancer passes over the internet, and is secured with TLS.
When a Lambda is attached to one (or more) VPC subnets, the post call to data exchange api times out. And when the Lambda is detached from all subnets, then this post call succeeds. This is happening consistently in golang Lambda environment.
In my use case, I am accessing Redis from Lambda, and Redis is accessible only from within the VPC.
Error message:
error=RequestError: send request failed
caused by: Post https://dataexchange.us-east-1.amazonaws.com/v1/data-sets: dial tcp 52.85.148.96:443: i/o timeout
An AWS Lambda function running in a VPC will never be assigned a public IP address. So in order for the Lambda function to access resources that exist outside the VPC, such as the AWS Data Exchange service, the VPC will need to be configured with a NAT Gateway that provides Internet access to the private subnet(s) the Lambda function is deployed to.
I have a Lambda function that I run every hour using a cloudwatch scheduled event.
When triggered the function makes an HTTP request to a service on the internet and then, having received the response, performs some SQL queries inside our VPC before making a couple more HTTP requests to the same service.
Because of the SQL database, the Lambda must run in our VPC. So in order to connect out to the internet, I've included our NAT gateway subnet among the Lambda's VPC subnets.
I've tried destroying and creating this Lambda function several times with the same result each time:
It runs as expected for a few/several hours. And then, after that, it never succeeds again because the attempt to make a TCP connection for the first HTTP request reliably fails. Only after I create a whole new Lambda function with the same code and configuration is it able to connect to the internet.
How can this be?
I've included our NAT gateway subnet among the Lambda's VPC subnets.
That is definitely a misconfiguration.
Lambda functions that need Internet access need to be on subnets whose default route points to a NAT Gateway -- none of which can also be a subnet that has an actual NAT Gateway on it. A NAT Gateway never provides NAT services to any instance on the same subnet as the gateway itself.