I have a setup like this:
Browser -> API Gateway -> Lambda: 87 ms
Browser -> Node.js Script -> AWS SDK -> Lambda: 163 ms
If i call the Lambda function via API Gateway then then response time is ok.
If i call the Lambda function via AWS SDK then the response is very slow.
I would like to get rid of API Gateway. Because it is expensive. However, AWS-SDK works very slow.
I use the SDK like this:
var http = require('http');
var url = require('url');
var AWS = require('aws-sdk');
var lambda = new AWS.Lambda({region: "eu-central-1"});
http.createServer(function (req, res) {
var url = req.url.split('/');
var data1 = url[1];
var data2 = url[2];
if ( ! data1 || ! data2 ) {
res.writeHead(500, {'Content-Type': "application/json" });
res.end("");
return;
}
var params = {
FunctionName: 'myfunc',
InvocationType: "RequestResponse",
Payload: '{"data1": "' + data1 + '", "data2": "' + data2 + '"}',
LogType : 'None'
};
lambda.invoke(params, function(err, data) {
if (err) {
console.log(err, err.stack);
} else {
var result = JSON.parse(data.Payload);
res.writeHead(200, {'Content-Type': "application/json" });
res.end(result.body);
}
});
}).listen(8080);
How can I call the Lambda function fast?
API gateway comes with the CloudFront advantage that's why it will always be faster than invoking the lambda from an external host.
If you deploy the nodejs script to a lambda in the same region and in the same vpc/subnet (if you have configured), it should be faster compared to API Gateway.
Related
I have a Vue.js website on an S3 bucket, that uses API Gateway to run a Lambda function, all behind Cloudfront. Nothing special!
This was live and working circa 2019 and has been left on the backburner since then.
Jump forward to now, the Lambda function no longer works, it gets a referer that is different to what I used to get back in 2019.
2019: referer = https://xxxxxxxxxxxxxx.cloudfront.net/machine/12345
2022: referer = https://xxxxxxxxxxxxxx.cloudfront.net/
the "/machine/12345" is no longer part of the referer, is there any way to configure Cloudfront, Lambda or API Gateway to pass this through as before?
Edit:
My Lambda function is in node.js
Here's the code up until the failure point on 'split'
// Load the SDK for JavaScript
const AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'eu-west-1'});
const ddb = new AWS.DynamoDB.DocumentClient();
const cognitoClient = new AWS.CognitoIdentityServiceProvider();
exports.handler = (event, context, callback) => {
//Check for lambda invoked from pre-flight CORS OPTION request
console.log('event = ', event);
if(event.httpMethod == 'OPTIONS') {
callback(null, {
statusCode: 201,
headers: {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': 'X-Amz-Security-Token,Content-Type,X-Amz-Date,Authorization,X-Api-Key'
},
});
}
else
{
//Normal invocation
console.log('referer =', event.params.header.Referer);
//get machine name from event.params.header.Referer
//e.g. event.params.header.Referer = "http://localhost:8080/machine/12345/2";
var referer = event.params.header.Referer.split("machine/")[1].split("/");
//e.g. referer = [ '12345', '2'];
var selectedmachine = referer[0];
var pagenum = Number(referer[1]);
I want to get real-time data from a stock API (IEX API) and load it to DynamoDB. Most of the tutorials I've watched so far (like this around 15:10) show how to create Lambda functions and integrate with API gateway. However, they are still manually entering the data to load to the table. How do I use the API Gateway and Lambda to get data from the IEX API and load it to DynamoDB rather than writing the data myself?
Following some tutorials, my Lambda function for loading data to the table is:
const AWS = require('aws-sdk');
exports.handler = async (event, context) => {
const documentClient = new AWS.DynamoDB.DocumentClient();
let responseBody = "";
let statusCode = 0;
const { id, stocksymbol} = JSON.parse(event.body);
const params = {
TableName: "StockData",
Item: {
id: id,
stocksymbol: stocksymbol
}
};
try {
const data = await documentClient.put(params).promise();
responseBody = JSON.stringify(data);
statusCode = 201;
} catch(err) {
responseBody = `Unable to put item: ${err}`;
statusCode = 403;
}
const response = {
statusCode: statusCode,
headers: {
"Content-Type": "application/json"
},
body: responseBody
};
return response
};
I would be getting more data from the API (opening price, closing price etc.) but this is what I have for now.
Assuming you need to utilize pull mechanism (you need to get data from API yourself), you can use AWS EventBridge rule to trigger your lambda periodically with interval of your preference - https://docs.aws.amazon.com/eventbridge/latest/userguide/create-eventbridge-scheduled-rule.html. In lambda you download the API data and store them in DynamoDb.
If you can use push mechanism (you can get data pushed to you, e.g. https://iexcloud.io/docs/api/?gclid=CjwKCAiA_9r_BRBZEiwAHZ_v17o9kJuPyF5Do_E3_mwC0uHEh2yXqqOdtVgqvc34yEk5RR8W8028HRoC0HUQAvD_BwE#webhooks), you can set the your API gateway resource URL as a target path, while having your lambda function as the handler for the API Gateway resource URL, storing the pushed data to the DynamoDb.
I have an internal API that provides data in different formats by just passing the id + format. For Example if I want to get a PDF of a produc with ID = 1 I will just make a call to the app with apiurl/latest/1.pdf.
This works fine when I am in the internal network since the host is only available internally. To access it publicly, we have implemented an authorization using API gateway and Lambda. Lambda takes are of the authorization and return the result just fine :
When I request JSON data
When I request XML data
Here a sample version of the lambda:
var request = require('request');
exports.handler = function(event, context, callback) {
var fUrl = event.fUrl + event.pid;
if(event.fsUrl.indexOf('product') >-1){
fUrl = fUrl + '.' + event.format
}
request({
url: fUrl,
}, function(error, response, body) {
if(error){
return callback(error);
}else{
return callback(null, response.body);
}
});
}
but not PDF. Some screens from the postman. I used both Send and Download in Postman.
Any thoughts?
I have a NodeJS Lambda function that reads from a Kinesis stream, does some processing and writes it to another Kinesis stream. The writing part is causing an error on lambda. We use the aws-sdk npm module to write the data. When we call putRecord no callback is fired and lambda times out.
This writing code works well when run locally as a node app on a laptop.
Local config:
- The credentials of a programmatic user are with Kinesis Full Access policy are stored in the credentials file and the putRecord runs successfully returning the shardId, sequence number, etc.
Lambda function:
- The code for putRecord does not return and it hangs. No errors are thrown either. Lambda has a role with the Kinesis Full Access policy.
Code:
var AWS = require('aws-sdk');
var kinesis = new AWS.Kinesis({
region: 'us-east-1',
});
var randomNumber = Math.floor(Math.random() * 100000);
var data = 'data-' + randomNumber;
var partitionKey = 'pk-' + randomNumber;
var recordParams = {
Data: data,
PartitionKey: partitionKey,
StreamName: streamName
};
kinesis.putRecord(recordParams, function(err, data) {
console.log(data);
if (err) {
console.error(err);
}
});
Any idea what could be causing the issue. VPC or security group related maybe?
Thoughts and suggestions appereciated.
Thanks.
If you have uploaded the exact Node.js script code above to Lambda, it will definitely not work.
Lamda requires you to export a handler function that it will call.
So, your script should be written like this if you want it to be a Lambda function...
'use strict';
var AWS = require('aws-sdk');
var kinesis = new AWS.Kinesis({
region: 'us-east-1',
});
exports.handler = function (event, context, callback) {
var randomNumber = Math.floor(Math.random() * 100000);
var data = 'data-' + randomNumber;
var partitionKey = 'pk-' + randomNumber;
var recordParams = {
Data: data,
PartitionKey: partitionKey,
StreamName: streamName,
};
kinesis.putRecord(recordParams, function (err, data) {
callback(null, data);
if (err) {
callback(err);
}
});
};
Take note that instead of using console.log or console.error, you should call callback instead.
When you add a function to a VPC it only has access to resources inside that VPC. Any attempt to access resources outside the VPC will hang and eventually timeout. Since Kinesis doesn't exist inside your VPC, you can't access it.
The fix is to either run the Lambda function outside the VPC, or add a NAT Gateway to your VPC.
I have a AWS-Lambda function that gots triggered after a file was placed into an S3-Bucket.
This is my lambda-function:
var http = require('http');
exports.handler = function (event, context) {
var bucket = event.Records[0].s3.bucket.name;
var key = event.Records[0].s3.object.key;
var newKey = key.split('.')[0].slice(8);
var url = 'https://xxxxx.herokuapp.com/api/messages/'+newKey+'/processingFinished'
console.log("URL:" + url)
http.get(url, function (result) {
console.log('!!!! Success, with: ' + result.statusCode);
context.done(null);
}).on('error', function (err) {
console.log('Error, with: ' + err.message);
context.done("Failed");
});
};
In CloudWatch Logfiles i see complaints that https is not supported:
2017-07-27T10:38:04.735Z 8428136e-72b7-11e7-a5b9-bd0562c862a0 Error: Protocol "https:" not supported. Expected "http:"
at new ClientRequest (_http_client.js:54:11)
at Object.exports.request (http.js:31:10)
at Object.exports.get (http.js:35:21)
at exports.handler (/var/task/index.js:18:8)
But the listed URL can be opened with any webbrowser. The Server accepts SSL and the whole API is working per SSL.
Why is AWS denying SSL-Requests? How to solve this?
Per the node documentation: (https://nodejs.org/api/https.html)
HTTPS is the HTTP protocol over TLS/SSL. In Node.js this is implemented as a separate module.
Use
http = require('https');