"module initialization " in AWS lambada - amazon-web-services

I have created a SQS queue and now i want to read the data from my lambada function.I have configured the trigger from SQS to this lambada function.
When i put the message in SQS Queue it shows me 1 message in Flight.But when i click test in lambda function it give me
Error
{
"errorMessage": "module initialization error"
}
I am using below boto3
import boto3
sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/944198216610/My-Queue'
response = sqs.receive_message(
QueueUrl='https://sqs.us-east-1.amazonaws.com/944198216610/My-Queue',
AttributeNames=[
'SentTimestamp'],
MaxNumberOfMessages=1,
MessageAttributeNames=[
'All'
],
VisibilityTimeout=0,
WaitTimeSeconds=0
)
message = response['Messages'][0]
receipt_handle = message['ReceiptHandle']
# Delete received message from queue
sqs.delete_message(
QueueUrl='https://sqs.us-east-1.amazonaws.com/944198216610/My-Queue',
ReceiptHandle=receipt_handle
)
print('Received and deleted message: %s' % message)

Your code calls sqs.receive_message, however to properly integrate Lambda with SQS you can use Lambda event triggers. This way, whenever an event goes into the queue, SQS will automatically invoke your Lambda function with that event (or multiple events based on specified batch size).
See here for setting up an SQS Lambda event trigger
Also see here for setting up a Lambda handler in python where the event parameter will be the event SQS triggers your lambda with (print it and see the format of the event so you can get an idea of how to use the event data).
This way, when you send a message to SQS, SQS will trigger your Lambda function, and if your Lambda returns successfully, SQS will remove the message from the queue. You don't need to do receive_message or delete_message manually.

Related

Lambda does not send message to SQS destination if invoked by an SQS source

I have the following set up:
I have a simple return in my Lambda function right now:
async function handler(event) {
return 'success';
}
If I invoke the lambda from the aws-cli using:
aws lambda invoke --function-name tt-yt-automation-production-1b2b-tiktok-download --invocation-type Event --cli-binary-format raw-in-base64-out /dev/null
It will return the message to the destination SQS (SQS#2), which will trigger another Lambda.
However, if SQS#1 has a message and invokes the Lambda, it does not send a message to SQS#2 and invokes the next Lambda in line.
My suspicion is that an SQS invocation doesn't count as an asynchronous invocation and won't send a message to SQS#2.
Is there a configuration I'm missing?
My suspicion is that an SQS invocation doesn't count as an asynchronous invocation and won't send a message to SQS#2.
You are correct. SQS invocation is synchronous. You have to "manually" (using AWS SDK) submit messages from your lambda to your SQS#2.

Destination only works when Lambda is invoked through AWS CLI

I have a hello-world test Lambda configured with:
trigger: API Gateway
destination: Amazon SQS. one queue for success, and another for failure.
import json
def lambda_handler(event, context):
print("Received event: " + json.dumps(event))
return {
"statusCode": 200,
"body": 'success'
}
When I invoke the Lambda via the CLI, the message gets enqueued to the success queue as expected:
aws lambda invoke --function-name event-destinations --invocation-type Event --payload '{}' response.json
However, when I invoke the Lambda via the API Gateway, no messages are enqueued to either destination queue. I have Lambda Proxy Integration enabled. Cloudwatch metrics confirm that the invocation is successful (Invocations count goes up, Errors count does not). The following returns a 200 and and the expected response body from my Lambda code:
curl 'https://REDACTED.execute-api.us-east-1.amazonaws.com/api_trigger' \
--header 'Content-Type: application/json' \
--data-raw '{}'
Similarly, no messages are enqueued to either destination queue when I use the Test button in the Lambda console. edit: this is expected behavior per https://www.trek10.com/blog/lambda-destinations-what-we-learned-the-hard-way
Why would the destination behavior differ between these 3 invocations? I have set retry attempts to 0 for this test.
It seems there is a set of valid {trigger, destination} pairs, and {API Gateway, SQS} is not one of them. Being able to invoke the lambda from a given trigger is not sufficient to get the event passed along to the destination. AWS console doesn't enforce these pairing or raise warnings.
I referenced the chart from: https://www.trek10.com/blog/lambda-destinations-what-we-learned-the-hard-way/
I added an S3 trigger to my lambda, and the S3 events are published to the destination SQS queues without issue.
Lambda Destinations are only triggered for asynchronous invocations. In Lambda non-proxy (custom) integration, the backend Lambda function is invoked synchronously by default.
You can configure the Lambda function for a Lambda non-proxy integration to be invoked asynchronously by specifying 'Event' as the Lambda invocation type. This is done as follows:
In Integration Request, add an X-Amz-Invocation-Type header with a static value of 'Event'.
Quoting from here.

Messages are not getting stored in DLQ when Lambda function gets failed

I have a DLQ configured to store messages when Lambda function gets failed.
Lambda Console Snippet -
Configuration in Lambda-
DLQ configuration -
Code Snippet:-
But message count is always 0 in DLQ, it's not increasing.
Where I am getting wrong?
Lamda failure messages are only put in DLQ if the lambda was invoked asynchronously
You can invoke your lambda asynchronously by specifying --invocation-type Event using aws sdk i.e.
$ aws lambda invoke --function-name my-function --invocation-type Event --payload '{ "key": "value" }' response.json
{
"StatusCode": 202
}
For more information you can read the documentation here

boto3 publish message in SNS

I want to run a script to publish message to my sns topic. I followed the documentation and my code is this :
import boto3
client = boto3.client('sns')
response = client.publish(
TopicArn='my topic arn',
Message='ptt message '
)
print("Response: {}".format(response))
It prints the response json, but I don't see any message in my AWS SNS console. I have a lambda function which is pretty basic as a subscriber to this sns.
The lambda code :
import json
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
How can I be sure that my message "ptt message" has been published successfully? How can I check it?
For testing of integration you can configure Delivery status logging for your SNS topic. There is a simple setting for this in GUI of SNS. This will export to CloudWatch the logs of your subscription triggers.
Secondly, your Lambda is not doing anything with the event, so your "ptt message stays in the "event" dictionary. You can for example log your event and see the structure where the actual message body comes.
import logging
def lambda_handler(event, context):
logging.info(event)
...
And simply to see that the Lambda was triggered by the subscription, you can open the CloudWatch Logs, find the LogGroup automatically created by AWS for your Lambda (has the name of the function in it) and search the Log Group. There are comfortable filters by "last X minutes".
In order to test this integration you can also publish test messages to SNS Topic from the web interface if this feels more convenient than running a script.

Lambda does not trigger SNS event. Chaining AWS lambdas with SNS

I am using AWS to chain lambdas together by having a second lambda subscribe to the SNS feed of the first lambda. The second lambda was not receiving anything from the first lambda and I also get no notification in CloudWatch (or my personal email that I subscribed with) that the SNS event has been triggered. So, I am wondering the question: How do I make sure my SNS receives an event from my lambda firing?
When I use the SNS web UI I get logs like these and my lambda logs show they have been touched, but only when I manually send a message.
{
"notification": {
"messageMD5Sum": "e9a72884afc5d9568f2748e34a4e50a4",
"messageId": "04f4a199-cd25-5a45-a877-f054df9b2adf",
"topicArn": "arn:aws:sns:us-east-1:xxxxxxxxxxxx:first-lambda",
"timestamp": "2017-06-28 02:12:14.098"
},
"delivery": {
"deliveryId": "173dca7a-7b31-55a2-8ee9-9bb7698f35f6",
"destination": "arn:aws:lambda:us-east-1:xxxxxxxxxx:function:second-labmda",
"providerResponse": "{\"lambdaRequestId\":\"33972992-5ba7-11e7-b363-09e0f6dc8594\"}",
"dwellTimeMs": 134,
"attempts": 1,
"statusCode": 202
},
"status": "SUCCESS"
}
I've given my lambda publish permissions in it's role, but that only shows up in IAM and when I view its policy inside of the lambda itself on the GUI I just see lambda firing permissions for the lambda itself. I also am sure that my SNS has permissions to write to the lambda because when you set up the lambda (as I did more than once yesterday) it asks for you to grant privileges to the SNS topic. However, even when testing the SNS topic the lambda is never reached, but I do get an email if I subscribe with my email account.
tl;dr
First lambda should be triggering an SNS event that a second lambda subscribes to. The SNS event is not receiving any data from my first lambda except for two logs in the middle of the night that never went through to my second lambda. How do I make sure my SNS receives an event from my lambda firing?
Solved this by adding a service to my lambda to send the message.
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.sns.AmazonSNSClient;
import com.amazonaws.services.sns.model.PublishRequest;
public class SNSPublishService {
private static final String ACCESS_KEY = "credential1";
private static final String SECRET_KEY = "credential2";
private static final String ARN = "arn:aws:sns:<region>:<id>:<topicname>";
public static void publish(String body) throws InterruptedException {
AmazonSNSClient service = new AmazonSNSClient(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
PublishRequest publishRequest = new PublishRequest()
.withTargetArn(ARN)
.withMessage(body);
service.publish(publishRequest);
}}