I'm creating a Square Client object like this:
const squareClient = new Client({
environment: Environment.Sandbox,
accessToken:
"The_correct_sandbox_token_goes_here",
});
const paymentsApi = squareClient.paymentsApi;
and calling the createPayment method from within a lambda function with a body like the one below:
{
"sourceId": "cnon:CBASEHY1uZmmlYRYagaqS7yd9Zo",
"amountMoney": {
"amount": "12500",
"currency": "USD"
},
"locationId": "Location_ID_here",
"idempotencyKey": "6a36e49c-914d-4934-bc34-c183ba0a08c5"
}
This works fine on my local machine (using serverless offline), but when deployed to AWS, the call to createPayment times out after six seconds. Is there something extra that needs to be done to call createPayment from a lambda function?
The timeout you experienced might be due to the function being attached to a VPC.
If an AWS Lambda function is configured to use a VPC, then the function only has access to the Internet if the following have been configured:
Lambda function is connected to a private subnet
A NAT Gateway or NAT Instance is running in a public subnet in the same VPC
A Route Table on the private subnet directs Internet-bound traffic to the NAT
If the Lambda function does not require access to any resources in the VPC, do not attach the function to a VPC. This will automatically grant direct access to the Internet.
Alternatively, you might try increasing the timeout of the Lambda function in case the external service simply needed more time.
Related
I gave my functions the exact same configurations like the images below.
But the functions in server-aws-development can access internet (tested it with GET call to google.com), but the authorizer function server-aws-token_authorizer can't. How is this possible?
Below the screenshot for the Route Table connected to the private subnets:
Below the screenshot for the Route Table for the public subnets:
In our system we have a flow of Lambdas calling other Lambdas based on results they get.
This flow will probably be refactor to use the AWS step function in the future but will probably stay as it is for the next month or two.
Recently a new lambda has been introduced, and we're encountering a problem:
if we test our Lambda function 1 locally, it properly calls function 2 on AWS.
However, after deploying the lambda and running the same test, function doesn't get called (there's no invocation in the monitoring chart)
here's a code example of Function 1:
'use strict';
const AWS = require("aws-sdk");
const lambda = new AWS.Lambda({ region: "eu-west-1" });
const myHandler = async (event, _, callback) => {
console.log(event)
return doSomething()
.then(something)
.then(somethingElse)
.catch(handleError)
function doSomething() {
return fetch('someurl.com');
}
function something(r) {
if(r.status === 204) {
let lambdaRes = callLambda( "LambdaB", { foo: bar } );
console.log(lambdaRes)
}
return r
}
somethingElse(r) {
if(r.status === 500) {
throw new Error(`bar`);
}
return r.json()
}
async function callLambda(fnName, payload ) {
let params = {
FunctionName: fnName,
InvocationType: 'Event',
Payload: JSON.stringify(payload)
}
return await lambda.invoke(params);
}
function handleError(e) {
if(e.message === 'bar') {
let lambdaRes = callLambda( "LambdaC", { bar: 'foo' } );
console.log(lambdaRes)
}
return callback(null, { message: `Completed with error`});
}
exports.handler = myHandler;
We tried to check if the problem was caused by authorization and tested by providing ALL authorization possible to function 1 but with no results.
Any suggestion?
UPDATE:
Both Function use the same
VPC: vpc-hashnumberVpc,
Subnets: [ subnet-hashnumber1 , subnet-hashnumber2 , subnet-hashnumber3 ]
Security Group: sg-hashnumberSg
When the first Lambda function calls lambda.invoke() it is not somehow connecting to the other Lambda function inside the VPC. At that point there is no running instance of the other Lambda function, only the definition of it exists. What is happening is lambda.invoke() makes a call to the AWS API which is on the public Internet, requesting that AWS create a new invocation of the second function.
Lambda functions running inside a VPC do not get public IP addresses assigned to them, even if they are in a public subnet. The only way to give a Lambda function inside a VPC access to the Internet is to place the Lambda function in a private subnet with a route to a NAT Gateway.
Alternatively, you can configure an AWS Lambda VPC Endpoint, which gives resources inside your VPC access to call the AWS Lambda API without going over the Internet.
So in order for a Lambda function running inside a VPC to make AWS API calls to invoke other Lambda functions, your options are to add a NAT Gateway to the VPC, or add an AWS Lambda VPC Endpoint to the VPC.
So if function 1 is able to call function 2 locally, I think the problem is with networking/AWS setup, here are a few things you can do :
Set up Log/Print statements at every level, so you can pinpoint which line is returning an error.
Is the function timing out? If so check the timeout and Memory Allocated to the function, if the function is memory intensive it will not complete.
Last but very important , check your network/Subnets, if you have configured lambda to be running on multiple subnets, choose one of them and try, look at the VPC, perhaps the Lambda is not attached to any VPC or check if the attached vpc is able to connect to the other lambda, have a look at Security Groups as well.
Most of my lambda issues are resolved with this, all the best.
I am trying to invoke an AWS Lambda function from another one. But after adding VPC with Lambda I receive this error:
send request failed caused by: Post "https://lambda.ap-southeast-1.amazonaws.com/2015-03-31/functions/seconda_lambda/invocations"
Here is the function:
_invoke_copier_lambda(map[string]string{
"parameter1" : "param_val",
})
func _invoke_copier_lambda(params map[string]string) error {
// Marshal the map into a JSON string.
bytes__data, err := json.Marshal(params)
if err != nil {
return err
}
jsonStr := string(bytes__data)
svci := sLambda.New(session.New())
input := &sLambda.InvokeInput{
FunctionName: aws.String(config.CopierLambdaName),
Payload: []byte(jsonStr),
}
_, err = svci.Invoke(input)
return err
}
What have I missed?
The API endpoint for AWS Lambda lives on the Internet. Therefore, invoking a Lambda function requires access to the Internet.
It appears that your Lambda functions are in a public subnet. This means that they do not have access to the Internet. To be able to communicate with the Internet an AWS Lambda function must either:
Be configured to not use a VPC, OR
Be configured to connect to a private subnet in a VPC, and the VPC has an NAT Gateway or NAT Instance in a public subnet. Also, the Route Table for the private subnet(s) point to the NAT Gateway/NAT Instance.
Lambda Destinations
There is actually an alternative way for one Lambda function to trigger another Lambda function by using the Lambda Destinations feature. When the first Lambda function completes, it can trigger another Lambda function as a Destination. This does not require Internet access because it is handled by the AWS Lambda service itself (rather than calling an API). Please note that Destinations is only available if the first function is triggered asynchronously.
See:
Introducing AWS Lambda Destinations | AWS Compute Blog
Trek10 | Lambda Destinations: What We Learned the Hard Way
I am running a Node(12.x) Lambda in AWS. The purpose of this lambda is to interact with Cloudformation stacks, and I'm doing that via the aws-sdk. When testing this lambda locally using lambda-local, it executes successfully and the stack can be seen in CREATING state in AWS console.
However, when I push and run this lambda in AWS, it fails after 15 seconds, and I get this error:
{"errorType":"TimeoutError","errorMessage":"Socket timed out without establishing a connection","code":"TimeoutError","message":"Socket timed out without establishing a connection","time":"2020-06-29T03:10:27.668Z","region":"us-east-1","hostname":"cloudformation.us-east-1.amazonaws.com","retryable":true,"stack":["TimeoutError: Socket timed out without establishing a connection"," at Timeout.connectTimeout [as _onTimeout] (/var/task/node_modules/aws-sdk/lib/http/node.js:69:15)"," at listOnTimeout (internal/timers.js:549:17)"," at processTimers (internal/timers.js:492:7)"]}
This lead me to investigate the lambda timeout and the possible configuration changes I could make found in https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/ and https://aws.amazon.com/premiumsupport/knowledge-center/lambda-vpc-troubleshoot-timeout/ but nothing worked.
I found a couple of similar issues such as AWS Lambda: Task timed out which include possible suggestions such as lambda timeout and lambda memory issues, but Ive set mine to 30 seconds and the logs show max memory used is 88MB out of possible 128MB, but I tried with an increase anyway, and no luck.
The curious part is that it fails without establishing a connection to hostname cloudformation.us-east-1.amazonaws.com. How is that possible when the role assigned to the lambda has full Cloudformation privileges? I'm completely out of ideas so any help would be greatly appreciated. Heres my code:
TEST EVENT:
{
"stackName": "mySuccessfulStack",
"app": "test"
}
Function my handler calls (createStack):
const AWS = require('aws-sdk');
const templates = {
"test": {
TemplateURL: "https://<bucket>.s3.amazonaws.com/<path_to_file>/test.template",
Capabilities: ["CAPABILITY_IAM"],
Parameters: {
"HostingBucket": "test-hosting-bucket"
}
}
}
async function createStack(event) {
AWS.config.update({
maxRetries: 2,
httpOptions: {
timeout: 30000,
connectTimeout: 5000
}
});
const cloudformation = new AWS.CloudFormation();
const { app, stackName } = event;
let stackParams = templates[app];
stackParams['StackName'] = app + "-" + stackName;
let formattedTemplateParams = [];
for (let [key, value] of Object.entries(stackParams.Parameters)) {
formattedTemplateParams.push({"ParameterKey":key, "ParameterValue": value})
}
stackParams['Parameters'] = formattedTemplateParams;
const result = await cloudformation.createStack(stackParams).promise();
return result;
}
Lambda function in a VPC does not public IP address nor internet access. From docs:
Connect your function to private subnets to access private resources. If your function needs internet access, use NAT. Connecting a function to a public subnet does not give it internet access or a public IP address.
There are two common solutions for that:
place lambda function in a private subnet and setup NAT gateway in public subnet. Then set route table from private subnet to the NAT device. This will enable the lambda to access the internet and subsequently CloudFormation service.
setup a VPC interface endpoint for CloudFormation. This will allow your lambda function in private subnet to access CloudFormation without the internet.
I'm using Terraform to build an API + corresponding lambda functions.
I have some other infrastructure, which I'd like to think is nicely set up (maybe I'm wrong?):
2 VPCs (let's just call'em test and prod)
Private & public subnets in each VPC
RDS DBs launched in the private subnets
All resources are identical on both VPCs; e.g. there's a test-private-subnet and a prod-private-subnet with exactly the same specs, same for DBs, etc.
Now, I'm working on the API and the lambdas that will power said API.
I don't feel like I need a test & prod API gateway and test & prod lambdas:
lambda code will be the same, just acting on different DB
you can use API stage_variables, with different ips, to achieve a test vs prod environment for the API
But when I try and setup a lambda with the vpc_config block (cause I need it to be associated with the security group that's allowed ingress on the DBs), I get the following error:
Error applying plan:
1 error(s) occurred:
* module.lambdas.aws_lambda_function.api-lambda-users: 1 error(s) occurred:
* aws_lambda_function.api-lambda-users: Error creating Lambda function: InvalidParameterValueException: Security Groups are required to be in the same VPC.
status code: 400, request id: xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx
My lambda config looks like this:
resource "aws_lambda_function" "api-lambda-users" {
provider = "PROVIDER"
function_name = "users"
s3_key = "users/${var.lambda-package-name}"
s3_bucket = "${var.api-lambdas-bucket}"
role = "${aws_iam_role.lambda-role.arn}"
handler = "${var.handler-name}"
runtime = "${var.lambda-runtime}"
vpc_config {
security_group_ids = [
//"${data.aws_security_group.prod-lambda.id}",
"${data.aws_security_group.test-lambda.id}"
]
subnet_ids = [
//"${data.aws_subnet.prod-primary.id}",
"${data.aws_subnet.test-primary.id}"
]
}
}
Notice I'd ideally like to just specify them, together, in their corresponding lists.
Am I missing something?
Suggestions?
Any help, related or not, is much appreciated.
Lambda running inside a vpc is subject to the same networking "rules" as ec2 instances. So it can't "exist" in two VPC's. If the lambda function needs to talk vpc resources in two separate VPC's you could use something like VPC peering or just run two copies of the function in the two different vpc's.
When you add VPC configuration to a Lambda function, it can only access resources in that VPC. If a Lambda function needs to access both VPC resources and the public Internet, the VPC needs to have a Network Address Translation (NAT) instance inside the VPC and a VPC Peering connection.