Is it possible to initiate a temporary (10 mins) Websocket/TCP connection to a server on EC2 instance from AWS Lambda? So that they can communicate?
That's possible under a few conditions:
your Lambda Timeout is greater than 10 minutes
your Lambda Function is deployed into a VPC (only if the EC2 instance isn't accessible from the Internet)
The security groups on the instance and the Lambda function allow connectivity
The connection is initiated from your Lambda function (as Lambda execution contexts can't accept inbound connections)
Related
I have two lambdas. LambdaA is the parent lambda that invokes LambdaB in parallel using the Event InvocationType (boto3). In every invocation, LambdaA sends a payload of 5MB to LambdaB. Both the lambdas are in the same VPC and in the same two private subnets (and same security group).
Now, assuming that LambdaA invokes LambdaB 5000 times in parallel for further invocation, a total payload of 25GB would be transferred between LambdaA and LambdaB.
I am trying to find out if I would be charged for the 50GB of data transfer as a data transfer cost, given that the data transfer is within the same VNet and the same two private subnets (and same security group).
Would I also be charged if there are in the same VPC and in the same private subnet (only one and same security group)?
When an AWS Lambda function invokes another AWS Lambda function, it would be sending traffic to the endpoint of the AWS Lambda service (not to the other Lambda function itself). Since your first Lambda function is connected to a VPC and the AWS Lambda service endpoint is on the Internet, the request would need to exit the VPC to access the Internet.
From EC2 On-Demand Instance Pricing – Amazon Web Services:
Data transferred “in” to and “out” from public or Elastic IPv4 address is charged at $0.01/GB in each direction.
However, if your first Lambda function was not connected to a VPC, then there would be no such charge since the Lambda function would be directly connected to the Internet. Typically, you should only connect an AWS Lambda function to a VPC if it specifically needs to access resources in that VPC (eg an Amazon RDS database).
Alternatively, you could use a VPC Endpoint to directly connect to to the AWS Lambda service. From Configuring interface VPC endpoints for Lambda - AWS Lambda:
If you use Amazon Virtual Private Cloud (Amazon VPC) to host your AWS resources, you can establish a connection between your VPC and Lambda. You can use this connection to invoke your Lambda function without crossing the public internet.
This would allow your Lambda function to connect to the VPC, but also connect to the AWS Lambda service without 'exiting' the VPC, thereby avoiding the 1c/GB charge.
The main thing to realise is that the two Lambda functions are not directly communicating. Rather, the communication is to the AWS Lambda service, which is then responsible for provisioning and invoking the second Lambda function.
Yes, you will be charged the EC2 AZ to AZ ingress and egress cost.
If the data was downloaded via S3 there would be no cost.
I need this to restrict the ports in the security group to avoid opening all the traffic to all ports. Is it HTTPS or another protocol?
AWS Lambda doesn't have any open inbound ports. Lambda functions aren't continuously running, listening on ports for incoming traffic. When something invokes an AWS Lambda function it calls the AWS API, which then executes your function. If you have an AWS Lambda function configured to run in a VPC, inbound security group rules will have no affect. Assigning a security group to a Lambda function is primarily for having a security group ID you can reference in other security group rules, like when giving a Lambda function access to an RDS server.
I have a lambda function which has the following logic in the handler:
log.info("about to get caller identity..")
caller_identity = boto3.client("sts").get_caller_identity()
log.info(caller_identity)
When I run this lambda function, it times out with the following error:
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "https://sts.amazonaws.com/"
Why is my lambda function not able to reach STS service?
Thanks!
This was the result of the Lambda being associated to a VPC in a private subnet with no way to communicate to the internet.
It is important that when using VPC configuration the Lambda is located in a subnet with the means to communicate with the internet such as a NAT. Without this your Lambda cannot communicate to the internet.
If you're trying to reach an AWS service you can check whether a VPC Endpoint is supported for the service to remove the need for internet connectivity.
When a Lambda is attached to one (or more) VPC subnets, the post call to data exchange api times out. And when the Lambda is detached from all subnets, then this post call succeeds. This is happening consistently in golang Lambda environment.
In my use case, I am accessing Redis from Lambda, and Redis is accessible only from within the VPC.
Error message:
error=RequestError: send request failed
caused by: Post https://dataexchange.us-east-1.amazonaws.com/v1/data-sets: dial tcp 52.85.148.96:443: i/o timeout
An AWS Lambda function running in a VPC will never be assigned a public IP address. So in order for the Lambda function to access resources that exist outside the VPC, such as the AWS Data Exchange service, the VPC will need to be configured with a NAT Gateway that provides Internet access to the private subnet(s) the Lambda function is deployed to.
I have hosted a Lambda function using AWS Chalice inside a VPC since I want it to access a Serverless Aurora DB Instance. Now I also want this function to send_message() to an SQS.
I followed Tutorial: Sending a Message to an Amazon SQS Queue from Amazon Virtual Private Cloud and was able to call the SQS from inside my EC2. But even then I could not use my Lambda function to call the SQS.
It would be very helpful if someone could actually tell me how to do the whole thing manually rather than using the CloudFormation stack, or at least tell me how to get the SQS Endpoint working.
It appears that your situation is:
An Amazon VPC with an Amazon Aurora database
An AWS Lambda function that wants to communicate with the Aurora database AND an Amazon SQS queue
An AWS Lambda function can be configured as:
Connected to a subnet in a VPC, or
Not connected to a VPC, which means it is connected to the Internet
If you wish to have an AWS Lambda function communicate with resources inside a VPC AND the Internet, then you will need:
The Lambda function connected to a private subnet
A NAT Gateway in a public subnet
An Internet Gateway connected to the public subnet (it is most probably already in your VPC)
Alternatively, you can use a VPC Endpoint for SQS, which allows the Lambda function to access SQS without going to the Internet. If you are wanting to connect to multiple service (eg S3, SNS, SQS), it is probably easier just to use a NAT Gateway rather than creating VPC Endpoints for each service.
You either need to add a VPC Endpoint for SQS to your VPC, or place the Lambda function in subnets with a route to a NAT Gateway.