AWS Lambda destination Lambda not triggering - amazon-web-services

Background:
I'm developing a custom AWS github-webhook via Terraform. I'm using AWS API Gateway to trigger an AWS Lambda function that validates the GitHub webhook's sha256 signature from the request header. If the lambda function successfully validates the request, I want a child lambda function to be invoked via the async invocation destination feature provided by Lambda.
Problem:
Even though I've configured the async invocation with the target child Lambda function, the child function is not triggered when the parent Lambda function is successful. This is reflected in the fact that the child Lambda function's associated CloudWatch log group is empty.
Relevant Code:
Here's the Terraform configuration for the Lambda function destination:
resource "aws_lambda_function_event_invoke_config" "lambda" {
function_name = module.github_webhook.function_name
destination_config {
on_success {
destination = module.lambda.function_arn
}
}
}
If more code from the module is needed, feel free to ask in the comments. The entire source code for this module is here: https://github.com/marshall7m/terraform-aws-codebuild/tree/master/modules/dynamic-github-source
Attempts:
Made sure both parent/child Lambda functions have permission to create logs within their respective Cloudwatch log group (attached arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole policy to both)
Made sure the parent Lambda function has the correct permission to invoke the child function: "lambda:InvokeFunction", "lambda:InvokeAsync"
Setup async invocation for child lambda function for both success and failure parent Lambda runs (child function still not triggered)
Add API integration request parameter `{'X-Amz-Invocation-Type': 'Event'} as mentioned in: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-integration-async.html
For every attempt to fix this, I made sure to redeliver the request from the source (github webhook page) and not via the AWS Lambda console.

From your description it seems to me that you are invoking parent function synchronously. Lambda destinations are only for asynchronous invocations:
You can also configure Lambda to send an invocation record to another service. Lambda supports the following destinations for asynchronous invocation
So you have to execute your parent function asynchronously for your child function to be invoked.

Adding the API integration request parameter `{'X-Amz-Invocation-Type': 'Event'} as mentioned in: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-integration-async.html did the trick. I initially came to the conclusion that this solution doesn't work based on the fact that a new Cloudwatch log group stream wasn't created when I redelivered the github payload. As it turns out, when I took a closer look at the previous Cloudwatch log stream, I found out that Cloudwatch appends logs for retriggered invocations of the Lambda function to the previous associated Cloudwatch log stream.

Related

How get the content of SQS Message using Lambda in AWS?

everyone.
I'm interested in get the content of SQS Message using Lambda. Let me explain my infrastructure: I have an EC2 instance that has a script like that bellow. So, it script will send to SQS the message containing instance ID.
#!/bin/bash
INSTANCE_ID=$(curl http://*.*.*.*/latest/meta-data/instance-id)
REGION=$(curl http://*.*.*.*/latest/meta-data/placement/availability-zone | sed '$s/.$//')
QUEUE-URL=$(...)
aws sqs send-message --queue-url "${QUEUE-URL}" --message-body "${INSTANCE_ID}" --region "${REGION}"
The ideia is: when the SQS recieve the message, I would like to trigger a Lambda Function to modificate this instance. But, for that, I need the instance ID. I have searching a lot and unfortunately, I couldn't understand very well how I could get the instance ID, using AWS Lambda, from the SQS Message mentioned above.
I've been trying to solve this problem, but as I don't understand Lambda so much, I searched for many solutions and tested then. Unfortunately, I had no success. So, I interested to learn more about this service.
If someone's could help me with that, I'd be very greateful.
The AWS Lambda function can be configured with the Amazon SQS queue as a 'trigger'.
When a message is sent to the SQS queue, the Lambda function will be invoked. The message both will be available in the Lambda function body.
The code would look something like:
def lambda_handler(event, context):
for record in event['Records']:
body = record['body']
print("From SQS: " + body)
It is possible that multiple messages are passed to the Lambda function, so it first loops through each Record, then extracts the passed-in information in the body parameter.
The print() will show the contents of the message body in CloudWatch Logs. Check to make sure it contains what you expect. Then, add code that uses that value.
There is no need for your Lambda function to specifically call SQS -- this is handled automatically by the AWS Lambda service, which will then delete the message from the queue after your Lambda function successfully completes.
#John's answer Is all new you need.
Since you are new to AWS. I would tell you answer in a way that will help you to debug for future too.
You need to first make sure sns topic target is configured as trigger.
In your lambda function, there is parameter irrespective of any language event. This parameter contains the information about the event which triggered the lambda
Inside the event parameter there is not all the information, like source, event information, in this information you will get Your instance id.
Just try logging event -> then records and you will get your instance id.
Then you can play with your Instance ID
Tip : In the console there is lambda test event where you can generate a sample event form different aws services, sns is also there. You can visualise the sample event too before testing with real event form sns .
Docs for reference - https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-concepts.html#gettingstarted-concepts-event

AWS Lambda destination not triggering with DynamoDB Streams

I'm using AWS Dynamo Streams to trigger an AWS Lambda function. If the lambda function successfully invoked, I want a child lambda function to be invoked via the async invocation destination feature provided by Lambda.
Even though I've configured the async invocation with the target child Lambda function, the child function is not triggered when the parent Lambda function is successful, the child Lambda function's associated CloudWatch log group is empty.
My parent lambda has such policies: AWSLambdaInvocation-DynamoDB (Provides read access to DynamoDB Streams), AWSLambdaFullAccess, AWSLambdaBasicExecutionRole
Question:
Why don't onSuccess and onFailure destinations work with invoking parent lambda via AWS Dynamo Streams?
AWS support helped me resolve this issue.
If we configure an async destination for Lambda, the async destination would only work if your Lambda function is invoked asynchronously. So in this specific case, with a DynamoDB event source, DynamoDB does not invoke our function async, but rather Lambda reads from the stream. So there's another option to configure a destination for streams, but that would only be for an on-failure destination.
The async destination would only work with async event sources like SNS or S3.
The page that shows how different event sources works with Lambda
So, if you look under the heading "Services that invoke Lambda functions asynchronously" - those are the service integrations that invoke Lambda async and would work with async destinations. Either those services, or if you actually invoke your function VIA the CLI asynchronously.
So for example if you invoke your function from the CLI and pass the flag
--invocation-type Event. That would invoke your function async.
If you pass --invocation-type RequestResponse, that would be synchronous.

FInd out who invoked AWS lambda function

My AWS lambda function is getting invoked my multiple places like api-gateway, aws-SNS and cloud-watch event. Is there any way to figure out who invoked the lambda function? My lambda function's logic depends on the invoker.
Another way to achieve this is having three different lambda functions but I don't want to go that way if I can find invoker information in a single Lambda function itself.
I would look at the event object as all of the three services will have event of different structure.
For example, for CloudWatch Events I would check if there is a source field in the event. For SNS I would check for Records and API gateway for httpMethod.
But you can check for any other attribute that is unique to a given service. If you are not sure, just print out to logs example events from your function for the three services and check what would be the most suited attribute to look for.

Make cloudformation wait until an object in created in s3

I am trying a scenario where cloud formation has to wait until an object is created in the specified bucket (where the object creation happens outside the scope of cloud formation by an external application).
I tried enabling bucket event notifications and hook a lambda function (so whenever an object is created in the bucket, lambda function is triggered). But I am not sure how to make cloud formation wait until this hooked lambda function execution is invoked.
Kindly let me if there any ideas on how to achieve this scenario.
I think the following should work:
Create WaitConditionHandle
Create a lambda function and pass !Ref to the wait condition handle created as an environment variable. When you !Ref a wait condition you get an url address. The lambda has only one job - to call the url when invoked.
Create WaitCondition and associate it with the wait handle created in step 1.
Add DependsOn attribute to the WaitCondition so that the condition gets created after the last resource to be created before CFN should pause and wait.
Use the S3 notification (as you already wrote in your question) to invoke lambda created in Step 2 when you get your object. Lambda gets invoked, calls the url, wait conditions stops waiting, and CFN should continue.
With the above there are no loops or long running processes, such as calling a lambda every 2 minutes.
Max timeout for the WaitCondition is 12 hours. You should adjust it 40 minutes or 1h for instance.
Try using a wait condition to solve this: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-waitcondition.html
You could try using Custom CloudFormation resources: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-sns.html. This would require you can send a http request to a S3-url provided through an SNS notification.
You would create file based on the SNS-notification (using lambda?) and then send a request back to cloudformation.

AWS Lambda timeout when another (long) Lambda is invoked

I am using Step Functions that link together multiple Lambdas. In one of these lambdas I invoke another lambda. Both lambdas have 15 min timeout. However, after some mins the primary lambda that invokes the secondary lambda times out (the secondary lambda happily continues its work).
Error
com.amazonaws.SdkClientException
Cause
{"errorMessage": "Unable to execute HTTP request: Read timed out",
"errorType": "com.amazonaws.SdkClientException",
"stackTrace": [
cause": {
"errorMessage": "Read timed out",
"errorType": "java.net.SocketTimeoutException",
"stackTrace": [
"java.net.SocketInputStream.socketRead0(Native Method)",
"java.net.SocketInputStream.socketRead(SocketInputStream.java:116)",
"java.net.SocketInputStream.read(SocketInputStream.java:171)",
"java.net.SocketInputStream.read(SocketInputStream.java:141)",
"sun.security.ssl.InputRecord.readFully(InputRecord.java:465)",
"sun.security.ssl.InputRecord.read(InputRecord.java:503)",
"sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)",
"sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)",
"sun.security.ssl.AppInputStream.read(AppInputStream.java:105)",
This is the code, as you can see I try to raise the timeout using 4 different commands.
// (2) Instantiate AWSLambdaClientBuilder to build the Lambda client
AWSLambdaClientBuilder builder =
AWSLambdaClientBuilder.standard().withRegion(region);
// (3) Build the client, which will ultimately invoke
the function
AWSLambda client = builder.build();
// (4) Create an InvokeRequest with required parameters
InvokeRequest req = new
InvokeRequest().withFunctionName(random_arn).withPayload(jsonString);
// (5) Invoke the function and capture response
int timeout = (15*60*1000);
req.setSdkClientExecutionTimeout(timeout);
req.setSdkRequestTimeout(timeout);
req.withSdkClientExecutionTimeout(timeout);
req.withSdkRequestTimeout(timeout);
InvokeResult result = client.invoke(req);
Any idea how to tackle this timeout?
There is more than one timeout you might be interested when working with AWS Lambda. The one you are referring to is the function execution timeout - the timeout people usually mean while talking about Lambda. However, there are also other timeouts you want to configure. Take a look at AWS CLI docs:
https://docs.aws.amazon.com/cli/latest/reference/index.html
Here you have --cli-read-timeout and --cli-connect-timeout. If I'd create a Lambda function with say 5 mins function execution timeout, and invoke it using AWS CLI using RequestResponse (synchronous) invocation type, if it happens it takes more than 1 min to execute (default value for --cli-read-timeout), AWS Lambda will fail this execution and trigger automatic re-try, up to a max number of retries. From what you are describing, I believe you are facing that kind of trouble. Set all the timeouts and try again.
A few more links you may find helpful:
AWS Lambda using Java - setting the configuration connection/socket timeouts
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/section-client-configuration.html
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html#setSocketTimeout-int-
As you don't explicitly specify the invocation type, you invoke your second AWS Lambda function from your first AWS Lambda synchronously. So while your second AWS Lambda function is running, the first one will keep running as well, waiting for the response of the second function. That's why your first AWS Lambda function will time out if the sum of the run time of the first and the second AWS Lambda function is higher than the configured maximum timeout of the first AWS Lambda function (15 minutes in your case).
Invoking an AWS Lambda function synchronously from another AWS Lambda function is bad practice as you might run in timeout issues as you experience right now and spend money for running the first AWS Lambda function while the second AWS Lambda function is running.
What you could do is to invoke the second AWS Lambda function asynchronously, by setting the proper invocation type for your InvokeRequest:
InvokeRequest req = new InvokeRequest().withFunctionName(random_arn)
.withPayload(jsonString)
.withInvocationType(InvocationType.Event);
That of course only works if you don't depend on the output of the second AWS Lambda function to finish your first function.
The probably better option, as you're already using AWS Step functions, is to use a Step Functions state machine to orchestrate the invoking of the second AWS Lambda function, as described in the following tutorial: https://aws.amazon.com/de/getting-started/tutorials/create-a-serverless-workflow-step-functions-lambda/
Depending on how your AWS Lambda functions it might make sense to split the first AWS Lambda function, which is currently invoking the second AWS Lambda function into two parts running before and after the invoked Lambda function.