I am using fluent-ffmpeg nodejs package to run ffmpeg for audio conversion on AWS Lambda. I am using this FFmpeg layer for lambda.
Here is my code
const bitrate64 = ffmpeg("file.mp3").audioBitrate('64k');
bitrate64.outputOptions([
'-preset slow',
'-g 48',
"-map", "0:0",
'-hls_time 6',
'-master_pl_name master.m3u8',
'-hls_segment_filename 64k/fileSequence%d.ts'
])
.output('./64k/prog_index.m3u8')
.on('progress', function(progress) {
console.log('Processing 64k bitrate: ' + progress.percent + '% done')
})
.on('end', function(err, stdout, stderr) {
console.log('Finished processing 64k bitrate!')
})
.run()
after running it via AWS lambda I get following error message
ERROR Uncaught Exception
{
"errorType": "Error",
"errorMessage": "ffmpeg exited with code 1: Conversion failed!\n",
"stack": [
"Error: ffmpeg exited with code 1: Conversion failed!",
"",
" at ChildProcess.<anonymous> (/var/task/node_modules/fluent-ffmpeg/lib/processor.js:182:22)",
" at ChildProcess.emit (events.js:198:13)",
" at ChildProcess.EventEmitter.emit (domain.js:448:20)",
" at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)"
]
}
I don't get any more info so I am not sure what's going on. Can anyone tell me what's wrong here and how can I enable more detailed logs?
Added on error callback to get a detailed error and found that there are permissions issue on lambda
.on('error', function(err, stdout, stderr) {
if (err) {
console.log(err.message);
console.log("stdout:\n" + stdout);
console.log("stderr:\n" + stderr);
reject("Error");
}
})
Related
I have this python code inside Lambda:
#This script will run as a Lambda function on AWS.
import time, json
cmdStatus = "Failed"
message = ""
statusCode = 200
def lambda_handler(event, context):
time.sleep(2)
if(cmdStatus=="Failed"):
message = "Command execution failed"
statusCode = 400
elif(cmdStatus=="Success"):
message = "The script execution is successful"
statusCode = 200
else:
message = "The cmd status is: " + cmdStatus
statusCode = 500
return {
'statusCode': statusCode,
'body': json.dumps(message)
}
and I am invoking this Lambda from Azure DevOps Build Pipeline - AWS Lambda Invoke Function.
As you can see in the above code - have intentionally put that cmdStatus to Failed to make that Lambda fail but when executed from Azure DevOps Build Pipeline - the task succeeds. Strange.
How can I make the pipeline to fail in this case? Please help.
Thanks
I have been working with a similar issue myself and it looks like a bug in the task itself. It was reported in 2019 and nothing happened since so I wouldn't hold out much hope.
https://github.com/aws/aws-toolkit-azure-devops/issues/175
My workaround to this issue was to instead use the AWS CLI task with
Command: lambda
Subcommand: invoke
Options and Parameters: --function-name {nameOfYourFunction} response.json
Followed immediately by a bash task with an inline bash script
cat response.json
if grep -q "errorType" "response.json"; then
echo "An error was found"
exit 1
fi
echo "No error was found"
I am using AWS Lex and AWS Lambda for creating a chatbot. The request and response format are as follows
Event being passed to AWS Lambda
{
"alternativeIntents": [
{
"intentName": "AMAZON.FallbackIntent",
"nluIntentConfidence": null,
"slots": {}
}
],
"botVersion": "$LATEST",
"dialogState": "ConfirmIntent",
"intentName": "OrderBeverage",
"message": "you want to order 2 pints of beer",
"messageFormat": "PlainText",
"nluIntentConfidence": {
"score": 0.92
},
"responseCard": null,
"sentimentResponse": null,
"sessionAttributes": {},
"sessionId": "2021-05-10T09:13:06.841Z-bSWmdHVL",
"slotToElicit": null,
"slots": {
"Drink": "beer",
"Quantity": "2",
"Unit": "pints"
}
}
Response Format-
{
"statusCode": 200,
"dialogAction": {
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": "Message to convey to the user. For example, Thanks, your pizza has been ordered."
}
}
}
AWS LAMBDA PYTHON IMPLEMENTATION-
import json
def lambda_handler(event, context):
# TODO implement
slots= event["slots"];
drink,qty,unit= slots["Drink"], slots["Quantity"], slots["Unit"]
retStr= "your order of "+qty+" "+unit+ " of "+drink+ " is coming right up!";
return {"dialogAction": {
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": retStr
},
}
}
The formats are in accordance with the documentation, however still getting error in processing lambda response. What is the issue?
This error occurs when the execution of the Lambda function fails and throws an error back to Amazon Lex.
I have attempted to recreate your environment using the python code shared and the test input event.
The output format that you have specified in the original post is correct. Your problem appears to lie with the input test event. The input message that you are using differs from what Lex is actually sending to your Lambda function.
Try adding some additional debugging to your Lambda function to log the event that Lex passes into it and then use the logged event as your new test event.
Ensure that you have CloudWatch logging enabled for the Lambda function so that you can view the input message in the logs.
Here's how my Lambda function looks:
import json
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def dispatch(event):
# TODO implement
slots= event["slots"];
drink,qty,unit= slots["Drink"], slots["Quantity"], slots["Unit"]
retStr= "your order of "+qty+" "+unit+ " of "+drink+ " is coming right up!";
return {"dialogAction": {
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": retStr
},
}}
def lambda_handler(event, context):
logger.debug('event={}'.format(event))
response = dispatch(event)
logger.debug(response)
return response
Now if you test via the Lex console you will find your error in the CloudWatch logs.:
[ERROR] KeyError: 'slots' Traceback (most recent call last): File
"/var/task/lambda_function.py", line 33, in lambda_handler
response = dispatch(event) File "/var/task/lambda_function.py", line 19, in dispatch
slots= event["slots"];
Using this error trace and the logged event, you should see that slots is nested within currentIntent.
You will need to update your code to extract the slot values from the correct place.
Trust this helps you.
Setup
AWS Lambda (3s timeout)
NodeJS 12.x
mssql 2.6.1
tedious (dependency of mssql and so it installs tedious 6.7.0)
SQL Server DB in RDS (db.t3.small)
I'm also dealing with a fair bit of traffic, with roughly 1k invocations per minute.
Problem
Most of the time the Lambda executes just fine. Roughly 0.35% of the time the Lambda throws an error. The logs look like this:
In the screenshot you can see that the function STARTs, prints some debug info, then throws an error, ENDs, and REPORTs.
While this is a "timeout" error, the error message says,
RequestError: Timeout: Request failed to complete in 15000ms
This confuses me because as you see in the REPORT log, the invocation time was just 255.33ms total.
Question
The obvious question is how does something timeout after 15 seconds in just 255ms? Is this an issue with tedious, mssql, my code, or something else? If my code is relevant to the question please let me know and I can add it. I assume the code is basically functional because it works > 99% of the time.
Failed Theories:
The logs are interleaved and the errors are not from the 225ms invocation. That's wrong because in the screenshot the request ID's match up.
There's an intermittent error connecting to the DB, possibly a DNS issue. I have seen very rare EAIAGAIN DNS errors when resolving the DB host, but that doesn't seem to match with the steadiness of this error.
I didn't spot anything super helpful in the GitHub issues for Tedious.
Full Error
{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "RequestError: Timeout: Request failed to complete in 15000ms",
"reason": {
"errorType": "RequestError",
"errorMessage": "Timeout: Request failed to complete in 15000ms",
"code": "ETIMEOUT",
"originalError": {
"errorType": "RequestError",
"errorMessage": "Timeout: Request failed to complete in 15000ms",
"code": "ETIMEOUT",
"message": "Timeout: Request failed to complete in 15000ms",
"stack": [
"RequestError: Timeout: Request failed to complete in 15000ms",
" at RequestError (/var/task/node_modules/mssql/node_modules/tedious/lib/errors.js:32:12)",
" at Connection.requestTimeout (/var/task/node_modules/mssql/node_modules/tedious/lib/connection.js:1212:46)",
" at Timeout._onTimeout (/var/task/node_modules/mssql/node_modules/tedious/lib/connection.js:1180:14)",
" at listOnTimeout (internal/timers.js:549:17)",
" at processTimers (internal/timers.js:492:7)"
]
},
"name": "RequestError",
"number": "ETIMEOUT",
"precedingErrors": [],
"stack": [
"RequestError: Timeout: Request failed to complete in 15000ms",
" at Request.userCallback (/var/task/node_modules/mssql/lib/tedious/request.js:429:19)",
" at Request.callback (/var/task/node_modules/mssql/node_modules/tedious/lib/request.js:56:14)",
" at Connection.endOfMessageMarkerReceived (/var/task/node_modules/mssql/node_modules/tedious/lib/connection.js:2407:20)",
" at Connection.dispatchEvent (/var/task/node_modules/mssql/node_modules/tedious/lib/connection.js:1279:15)",
" at Parser.<anonymous> (/var/task/node_modules/mssql/node_modules/tedious/lib/connection.js:1072:14)",
" at Parser.emit (events.js:315:20)",
" at Parser.EventEmitter.emit (domain.js:482:12)",
" at Parser.<anonymous> (/var/task/node_modules/mssql/node_modules/tedious/lib/token/token-stream-parser.js:37:14)",
" at Parser.emit (events.js:315:20)",
" at Parser.EventEmitter.emit (domain.js:482:12)"
]
},
"promise": {},
"stack": [
"Runtime.UnhandledPromiseRejection: RequestError: Timeout: Request failed to complete in 15000ms",
" at process.<anonymous> (/var/runtime/index.js:35:15)",
" at process.emit (events.js:315:20)",
" at process.EventEmitter.emit (domain.js:482:12)",
" at processPromiseRejections (internal/process/promises.js:209:33)",
" at processTicksAndRejections (internal/process/task_queues.js:98:32)"
]
}
I built a native java AWS Lambda function using Graal and Micronaut as explained here
After deploying it to AWS Lambda (custom runtime), I can't successfully execute it.
The error that AWS shows is:
{
"errorType": "Runtime.ExitError",
"errorMessage": "RequestId: 9a231ad9-becc-49f7-832a-f9088f821fb2 Error: Runtime exited with error: exit status 1"
}
The AWS log output is:
START RequestId: 9a231ad9-becc-49f7-832a-f9088f821fb2 Version: $LATEST
01:13:08.015 [main] INFO i.m.context.env.DefaultEnvironment - Established active environments: [ec2, cloud, function]
Error executing function (Use -x for more information): Error decoding JSON stream for type [request]: No content to map due to end-of-input
at [Source: (BufferedInputStream); line: 1, column: 0]
END RequestId: 9a231ad9-becc-49f7-832a-f9088f821fb2
REPORT RequestId: 9a231ad9-becc-49f7-832a-f9088f821fb2 Duration: 698.31 ms Billed Duration: 700 ms Memory Size: 512 MB Max Memory Used: 54 MB
RequestId: 9a231ad9-becc-49f7-832a-f9088f821fb2 Error: Runtime exited with error: exit status 1
Runtime.ExitError
But when I test it locally using
echo '{"value":"testing"}' | ./server
I got
01:35:56.675 [main] INFO i.m.context.env.DefaultEnvironment - Established active environments: [function]
{"value":"New value: testing"}
The function code is:
#FunctionBean("user-data-function")
public class UserDataFunction implements Function<UserDataRequest, UserData> {
private static final Logger LOG = LoggerFactory.getLogger(UserDataFunction.class);
private final UserDataService userDataService;
public UserDataFunction(UserDataService userDataService) {
this.userDataService = userDataService;
}
#Override
public UserData apply(UserDataRequest request) {
if (LOG.isDebugEnabled()) {
LOG.debug("Request: {}", request.getValue());
}
return userDataService.get(request.getValue());
}
}
And the UserDataService is:
#Singleton
public class UserDataService {
public UserData get(String value) {
UserData userData = new UserData();
userData.setValue("New value: " + value);
return userData;
}
}
To test it on AWS console, I configured the following test event:
{ "value": "aws lambda test" }
PS.: I uploaded to AWS Lambda a zip file that contains the "server" and the "bootstrap" file to allow the "custom runtime" as explained before.
What I'm doing wrong?
Thanks in advance.
Tiago Peixoto.
EDIT: added the lambda test event used on AWS console.
Ok, I figured it out. I just changed the bootstrap file from this
#!/bin/sh
set -euo pipefail
./server
to this
#!/bin/sh
set -euo pipefail
# Processing
while true
do
HEADERS="$(mktemp)"
# Get an event
EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)
# Execute the handler function from the script
RESPONSE=$(echo "$EVENT_DATA" | ./server)
# Send the response
curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response" -d "$RESPONSE"
done
as explained here
I am trying the async/await approach in AWS lambda function with node v8.11.3 npm v5.10.0. When I run it gives me the following response:
{
"errorMessage": "Unexpected token function",
"errorType": "SyntaxError",
"stackTrace": [
" ^^^^^^^^",
"SyntaxError: Unexpected token function",
"createScript (vm.js:56:10)",
"Object.runInThisContext (vm.js:97:10)",
"Module._compile (module.js:542:28)",
"Object.Module._extensions..js (module.js:579:10)",
"Module.load (module.js:487:32)",
"tryModuleLoad (module.js:446:12)",
"Function.Module._load (module.js:438:3)",
"Module.require (module.js:497:17)",
"require (internal/module.js:20:19)"
]
}
The lambda function is:
const fetch = require('node-fetch')
exports.handler = async function(event,context)
{
console.log(event);
let img = await
fetch(`https://catappapi.herokuapp.com/users/${event.userId}`);
let parseddata = await img.json()
console.log(parseddata.imageUrl);
}
How to solve this issue?
You are getting this error because you are running an older version of nodejs in cloud9 environment process (can verify that by console.log(process.version) which would be different than node --version). Follow these steps to update the process node in your Cloud9 environment:
nvm install 11
nvm use 11
nvm alias default v11