I'm getting the following error when trying to call create_topic() in Boto3. It works locally in sam running sam local invoke, but once deployed, it times out.
ConnectTimeoutError: Connect timeout on endpoint URL: "https://sns.us-east-2.amazonaws.com/"
Here's the code:
sns = boto3.client('sns')
topic_name = f'my-sns-topic-{ENVIRONMENT}'
topic = sns.create_topic(Name=topic_name)
notification_channel = {"SNSTopicArn": topic["TopicArn"], "RoleArn": "arn:aws:iam::my-role"}
My Lambda function is on private subnets. The function shouldn't require any access to the internet, so I think private subnets are ok (?). All my resources are on the same VPC.
Does the lambda function have to be on a public subnet to reach SNS? I tried adding a 0.0.0.0/0 route mapped to my internet gateway to the route table associated with the private subnet, but that didn't help.
What am I missing?
My Lambda function is on private subnets. The function shouldn't require any access to the internet,
If your lambda function is deployed in a VPC that does not have internet connectivity, then your lambda function will be unable to reach the service endpoint (sns.us-east-2) over the public internet, as you would expect.
If you want private connectivity to the service, then you need to provision a VPC interface endpoint for the service and deploy it in the same VPC as your lambda.
Related
I have a Lambda function deployed into a public subnet in a VPC and I’m trying to connect to a Lambda function outside of a VPC and I’m running into connectivity issues.
I believe the security group settings and IAM policies will permit the connection, but I'm not sure if there's an issue with connecting to an out-of-VPC Lambda from an in-VPC one.
Is there a tool in AWS Console, AWS CLI or anywhere else that I can use to troubleshoot where the connection is failing? I’ve used the Reachability Analyzer before but it only works on a handful of resource types like EC2 instances.
I've tried invoking the out-of-VPC Lambda from inside my in-VPC Lambda, but the request doesn't work and I don't see any helpful information about what happened. I tried running the Reachability Analyzer, but it doesn't allow you to test if Lambda functions are reachable.
I was expecting the request to work, but I'm not sure if I need to configure a VPC interface endpoint because I'm connecting from an in-VPC Lambda to an out-of-VPC Lambda.
I’m new to networking and would appreciate any help.
I have a Lambda function deployed into a public subnet in a VPC and I’m trying to connect to a Lambda function outside of a VPC and I’m running into connectivity issues.
The Lambda function in the VPC never gets a public IP assigned to it. So it can't connect to anything outside of the VPC. It can't use the Internet Gateway attached to the public subnet because it doesn't have a public IP.
By "connect to a Lambda function outside of a VPC" what you are really doing is connecting to the AWS API outside of the VPC. You never "connect" to a Lambda function, because Lambda functions aren't running and just sitting around idle waiting for your request. Lambda functions don't really exist until a request comes in to the AWS Lambda Invoke API, at which point AWS spins up an instance of the Lambda function and passes it the invocation payload.
To fix this connectivity issue, you either need to create an AWS Lambda VPC Endpoint in your VPC, to handle requests to the Lambda API originating in your VPC. Or you need to move the VPC Lambda function to a private subnet, with a route to a NAT Gateway. Lambda functions in private subnets can access things outside the VPC by having their requests routed through the NAT Gateway.
I was expecting the request to work, but I'm not sure if I need to configure a VPC interface endpoint because I'm connecting from an in-VPC Lambda to an out-of-VPC Lambda.
That's not how VPC Interface Endpoints work. The entire purpose of VPC Interface Endpoints is to allow a resource inside your VPC to access part of the AWS API that exists outside the VPC. A VPC Interface Endpoint will absolutely allow your Lambda function running in the VPC to access the Lambda Invoke API, in order to trigger an execution of your out-of-VPC Lambda function.
I have a lambda function that calls the parameter store to retrieve a credential. The code is as follows:
import boto3
ssm = boto3.client('ssm')
parameter = ssm.get_parameter(Name='credentials', WithDecryption=True)
print(parameter['Parameter']['Value'])
I have given AmazonSSMFullAccess to the lambda role. The lambda has a VPC which later I'll use it to connect to a Redshift database without public access. The inbound and outbound rules are as follows:
There is a post AWS Lambda cannot connect to Parameter Store which mentions that if the lambda requires VPC, then add a NAT gateway.
In the lambda subnet route table: , there seems to be already a route that goes to the internet?
But I am still getting lambda time-out errors :(
there seems to be already a route that goes to the internet?
Sadly, it does not. It seems you placed your lambda in a public subnet with route to internet gateway (IGW). However, you have to use private subnet with a route to NAT gateway. IGW and NAT are two different things. Have a look at this AWS guide how to make it work:
How do I give internet access to a Lambda function that's connected to an Amazon VPC?
Alternatively, you can setup VPC interface endpoint for Paramter store. Then you don't need internet access for your Lambda function.
I have a lambda function which has the following logic in the handler:
log.info("about to get caller identity..")
caller_identity = boto3.client("sts").get_caller_identity()
log.info(caller_identity)
When I run this lambda function, it times out with the following error:
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "https://sts.amazonaws.com/"
Why is my lambda function not able to reach STS service?
Thanks!
This was the result of the Lambda being associated to a VPC in a private subnet with no way to communicate to the internet.
It is important that when using VPC configuration the Lambda is located in a subnet with the means to communicate with the internet such as a NAT. Without this your Lambda cannot communicate to the internet.
If you're trying to reach an AWS service you can check whether a VPC Endpoint is supported for the service to remove the need for internet connectivity.
I am working on a project where my main lambda function is in a VPC in private subnet and some sister lambda functions in a different VPC which are in their own private subnets. How can I go about calling these sister lambdas's from the main lambda across VPC without giving internet access to each of them via a NAT gateway linking to a public subnet which has an internet gateway attached to it.
Other AWS services that my main lambda invokes are:
1. S3
2. Dynamodb
3. Autoscaling
4. ECS
5. RDS
This can be done, but there are some complex steps involved.
First of all, when you use aws-sdk, the calls are made through the internet. To avoid this situation and access the services within the AWS network, The AWS has introduced some private VPC endpoints. I have only used S3 and API gateway private endpoints to date. But there is more type of VPC endpoints.
This is how I would do today,
setup a private API gateway API to invoke lambda - The private API's are only accessible through a private VPC endpoint for API gateway.
create a private VPC endpoint for API gateway.
setup VPC peering between the VPCs
(from the sister lambda on other VPC) invoke the API through the VPC endpoints public DNS URL
The drawback of adding an API in front of the lambda is, the API has a hard timeout of 29 seconds.
hope this helps.
my lambda function can send message to SNS with publish method with "no VPC", but it will timeout when I put it in a VPC which has access to public internet with route and internet gateway.
Edited
I have my lambda in a public subnet which has 0.0.0.0/0 already routed to the internet gateway, so can not route it again to NAT. Is that possible to assign a EIP to lambda function
You have to add a NAT Gateway to your VPC in order for Lambda functions (and other things in your VPC that don't have a public IP) to access anything outside the VPC. You should read the Things to Know section of this AWS announcement.
Outside Internet is not accessible when VPC is enabled. So, when you send a request to publish to a SNS topic using libraries such as boto3, your requests will timeout.
NAT is one of the option but a more cost effective way would be to setup an interface endpoint to SNS from your VPN. You can go to AWS VPN console and create an endpoint and select SNS as the service.
Here is more details on how to setup your interface endpoint: https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint