I have a lambda function which fetches a file from s3 using the input key in event and needs to send the same to client. I am using the following function to get the file from s3
function getObject(key){
var params = {
Bucket: "my_bucket",
Key: key
}
return new Promise(function (resolve, reject){
s3.getObject(params, function (err, data){
if(err){
reject(err);
}
resolve(data.Body)
})
})
}
If I send the response of this promise (buffer) to context.succeed, it is displayed as a JSON array on front end. How can I send it as a file ? The files can be either ZIP or HTTP Archive (HAR) files. The s3 keys contain the appropriate extension. I am guessing it has got something to do with the "Integration Response" in API Gateway. But not able to figure out where to change
Good news, you can now handle binary input and output for API Gateway (announcement and documentation).
Basically, nothing changes in your Lambda Function, but you can now set the contentHandling API Gateway Integration property to CONVERT_TO_BINARY.
Unfortunately, the official AWS examples showcase only the HTTP API Gateway backend, as the AWS Lambda support seems not complete yet. For example, I haven't managed to return gzipped content from AWS Lambda yet, although it should be possible thanks to the new binary support and the $util.base64Decode() mapping utility.
Related
I have written a small cloud function in GCP which is subscribed to Pub/Sub event. When any cloud builds triggered function post message into the slack channel over webook.
In response, we get lots of details to trigger name, branch name, variables details but i am more interested in Build logs URL.
Currently getting build logs URL in response is like : logUrl: https://console.cloud.google.com/cloud-build/builds/899-08sdf-4412b-e3-bd52872?project=125205252525252
which requires GCP console access to check logs.
While in the console there an option View Raw. Is it possible to get that direct URL in the event response? so that i can directly sent it to slack and anyone can access direct logs without having GCP console access.
In your Cloud Build event message, you need to extract 2 values from the JSON message:
logsBucket
id
The raw file is stored here
<logsBucket>/log-<id>.txt
So, you can get it easily in your function with Cloud Storage client library (preferred solution) or with a simple HTTP Get call to the storage API.
If you need more guidance, let me know your dev language, I will send you a piece of code.
as #guillaume blaquiere helped.
Just sharing the piece of code used in cloud function to generate the singedURL of cloud build logs.
var filename ='log-' + build.id + '.txt';
var file = gcs.bucket(BUCKET_NAME).file(filename);
const getURL = async () => {
return new Promise((resolve, reject) => {
file.getSignedUrl({
action: 'read',
expires: Date.now() + 76000000
}, (err, url) => {
if (err) {
console.error(err);
reject(err);
}
console.log("URL");
resolve(url);
});
})
}
const singedUrl = await getURL();
if anyone looking for the whole code please follow this link : https://github.com/harsh4870/Cloud-build-slack-notification/blob/master/singedURL.js
I have a serverless project: AWS + Angular on the frontend. Currently, I get the data when page is initialized and refresh the data when press "update" button. However, I want to monitor changes in the table constantly. In Firebase there is onSnapShot() method, which sends the new data when a collection is updated.
I want to make something similar with AWS. However, in official documentation, I do not see how to correctly do it.
So here are 2 questions:
How can I connect to the WebSocket with aws-sdk? (Currently, I can connect only from the terminal with wscat -c myurl call. Or shall I simply send http.Post with websocket url?
is it possible to pass invoke in the callback URL? - I want to get data from DynamoDB when page initialize and then invoke it again and again (with a callback URL)
My Lambda function looks like this:
exports.handler = async (event, context) => {
let params = {
TableName: "documents"
}
let respond = await db.scan(params).promise();
return respond;
};
On the front-end I have:
ngOnInit(): void {
AWS.config.credentials = new AWS.Credentials({
accessKeyId: '//mykey', secretAccessKey: '//mysecretkey'
})
AWS.config.update({
region:'//myregion'
})
this.updateTable() // triggers post request to APi Gateway => lambda and receives a response with data.
}
From my understanding, you will need to set up a DynamoDB stream and a lambda function that respond to the database CRUD events, send the updated data to the WebSocket connection if the event data matches the criteria (document id for example), through AWS.ApiGatewayManagementApi. (FYI: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/ApiGatewayManagementApi.html)
I am struggling with getting Lambda + API gateway working well.
This is my code-
exports.handler = async (event) => {
console.log("EVENT -> ", event.body)
let buff = new Buffer(event.body, 'base64');
let text = buff.toString('UTF-8');
console.log("TEXT -> ",text)
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
return response;
};
I have referred to this answer, but, I don't see Integration Request in API Gateway.
What has to be done to get a proper JSON? I know I can use 3rd party npm libs. But, I prefer some fix at AWS end. I just want to use event.body which should return a JSON.
I will re-intepret your question the way I understand It.
1/ You created a Lambda function, connected to a API gateway
2/ You sent an some data to the Lambda function using API gateway
3/ API gateway invoke your function with some data
4/ The data you received inside your Lambda function is neither lost nor corrupted. It is received inside Lambda as the Base64-Encoded format
5/ You do not want to receive the data in Lambda as Base64-Encoded, You do not want to do a step of decoding like you did
let buff = new Buffer(event.body, 'base64');
let text = buff.toString('UTF-8');
If that is the case, you need to take a look at this
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-workflow.html
First, check what Content-Type header your request are sending.
Then, see if the data you sent to API gateway is actually textual data
Also, go to your API Gateway -> Setting -> Binary Media Types to see if any data type are specifically to be treated as Binary by API Gateway
I have a multi-endpoint webservice written in Flask and running on API Gateway and Lambda thanks to Zappa.
I have a second, very tiny, lambda, written in Node, that periodically hits one of the webservice endpoints. I do this by configuring the little lambda to have Internet access then use Node's https.request with these options:
const options = {
hostname: 'XXXXXXXXXX.execute-api.us-east-1.amazonaws.com',
port: 443,
path: '/path/to/my/endpoint',
method: 'POST',
headers: {
'Authorization': `Bearer ${s3cretN0tSt0r3d1nTheC0de}`,
}
};
and this works beautifully. But now I am wondering whether I should instead make the little lambda invoke the API endpoint directly using the AWS SDK. I have seen other S.O. questions on invoking lambdas from lambdas but I did not see any examples where the target lambda was a multi-endpoint webservice. All the examples I found used new AWS.Lambda({...}) and then called invokeFunction with params.
Is there a way to pass, say, an event to the target lambda which contained the path of the specific endpoint I want to call? (and the auth headers, etc.) * * * * OR * * * * is this just a really dumb idea, given that I have working code already? My thinking is that a direct SDK lambda invocation might (is this true?) bypass API Gateway and be cheaper, BUT, hitting the endpoint directly via API Gateway is better for logging. And since the periodic lambda runs once a day, it's probably free anyway.
If what I have now is best, that's a fine answer. A lambda invocation answer would be cool too, since I've not been able to find a good example in which the target lambda had multiple https endpoints.
You can invoke the Lambda function directly using the invoke method in AWS SDK.
var params = {
ClientContext: "MyApp",
FunctionName: "MyFunction",
InvocationType: "Event",
LogType: "Tail",
Payload: <Binary String>,
Qualifier: "1"
};
lambda.invoke(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
/*
data = {
FunctionError: "",
LogResult: "",
Payload: <Binary String>,
StatusCode: 123
}
*/
});
Refer the AWS JavaScript SDK lambda.invoke method for more details.
I'm using AWS API Gateway integrated with Lambda.
Note: I am NOT using Lambda Proxy.
I need to return a binary response from API Gateway. I have successfully set this up as follows:
Encoded my binary data as base64 UTF-8 string and returning ONLY that from my lambda function return "base64 encoded binary data"
Enabled CONVERT_TO_BINARY on the API Gateway Integration Response
Mapped the Content-Type header on the API Gateway Method Response to the binary media type of my binary content
Added the media type of my binary content to API Gateway's list of Binary Media Types
The issue is that as well as sending the binary data (which I can do successfully from the above steps), I need to include a x-my-header custom header in the API Response.
I know how to set-up header mapping in the API Gateway, but the header has to be calculated from database data, and therefore this value needs to also be returned from lambda.
My understanding of lambda integration (remember, I'm not using lambda proxy here) is that API gateway makes a HTTP request to trigger lambda. Lambda then returns a HTTP response to API Gateway, adding the functions output to the body, and also adding internal aws headers to the response.
Now it is possible to map a header to the Method Response using:
integration.response.header.header-name
My question is...
Can I tell lambda to add my custom header to a binary response, when I'm
using custom lambda integration (not proxy) ?
Note: IF i was using lambda proxy, I know that the return object looks as below, and then I would be able to send custom headers. But for reasons out of my control, I cant use lambda proxy.
Lambda return object solution IF i was using lambda proxy:
return {
'body': "base64 encoded binary data",
'headers': 'x-my-header': 'my-value',
'isBase64Encoded': True
}
For Lambda Integration (not proxy) I have tried modifying my lambda output...
return {
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
And setting up a mapping template in the integration response...
$input.json("$.base64-data")
And setting up a header mapping using...
integration.response.body.x-my-header
But API Gateway returns an error:
Execution failed due to configuration error: Unable to transform response
I believe this error occurs because there cannot be a mapping template when you have CONVERT_TO_BINARY enable. From AWS docs:
When converting a text payload to a binary blob, API Gateway assumes that the text data is a Base64-encoded string and outputs the binary data as a Base64-decoded blob. If the conversion fails, it returns a 500 response indicating an API configuration error. You do not provide a mapping template for such a conversion, although you must enable the passthrough behaviors on the API.
I realize this is an old question, but I ran into a similar header mapping issue recently, even though I'm not using binary data.
To my mind the return from Lambda could look like this
{
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
Based on that Lambda Response, you could apply the following mapping (which I modified based on an AWS example)
$input.json("$.base64-data")
#set($context.responseOverride.header.x-my-header = "$input.json('$.x-my-header')")
$input.json("$") references your response and $.[key] is the right way to reference your sub-keys.
You also have to preconfigure your header "x-my-header" in your method response.
In the integration Response the header mapping can just be an empty value - e.g.
""
The real value will be provided by the override ($context.responseOverride.header) which is set by the mapping template.