I have a GC Function that triggers cron-job. Almost everything works well, except I get error in Logs "Service unavailable" even though it works (runs API call).
This my code:
exports.helloPubSub = (event, context) => {
const message = event.data
? Buffer.from(event.data, 'base64').toString()
: 'Function';
console.log("message:", message);
const url = `example.com`
const targetAudience = '12345';
const {GoogleAuth} = require('google-auth-library');
const auth = new GoogleAuth();
async function request() {
const client = await auth.getIdTokenClient(targetAudience);
const res = await client.request({
'url': url,
'method': 'post',
'data': {
'token': 'myToken'
}});
console.info(res.data);
}
request().catch(err => {
console.error(err.message);
process.exitCode = 1;
});
};
The error "Service unavailable" appears after 5 minutes as the function starts. The function is performed for 7 minutes every 10 minutes, so there is time gap between next starting.
The question is, is it something with the function or with time-limit in GC? Any thoughts?
Updated:
This is my logs:
According to the official documentation:
HTTP status and error codes for JSON
503—Service Unavailable is a backendError (internal error), and you are encouraged to try again using truncated exponentioal backoff.
Related
I've been trying to solve this problem all day, looking everywhere on the web, even in the official AWS documentation, why does this error keep appearing when I try to send a message to the client through this code?
PS. i use SDK v3 with node.js 18
import {
ApiGatewayManagementApiClient,
PostToConnectionCommand,
} from "#aws-sdk/client-apigatewaymanagementapi";
export const handler = async (event) => {
const domain = event.requestContext.domainName;
const stage = event.requestContext.stage;
const connectionId = event.requestContext.connectionId;
const callbackUrl = `https://${domain}/${stage}`;
const client = new ApiGatewayManagementApiClient({ endpoint: callbackUrl });
const requestParams = {
ConnectionId: connectionId,
Data: "Hello!",
};
const command = new PostToConnectionCommand(requestParams);
try {
await client.send(command);
} catch (error) {
console.log(error);
}
return {
statusCode: 200,
};
};
fef17825-58ce-4ca7-8f38-85857f1aef0a Task timed out after 3.01 seconds
i tried any online guide or video, can anyone help me?
My question is this: how do I call a Firebase Cloud Function from a Cloud task and pass a payload through?
I tried following the tutorial here. The only difference is that I'm using Cloud functions for Firebase instead of regular Cloud Functions.
Here is my cloud function.
const functions = require("firebase-functions");
exports.myFunction = functions.https.onRequest((req, res) => {
console.log(req.query);
res.send('success');
});
When I query the url in the browser with parameters ?myparams=data I can log 'data' so I know the cloud function is basically working.
But when I try to call it from my queue (below) I get:
SyntaxError: Unexpected token o in JSON at position 1
at JSON.parse (<anonymous>)
My guess is that req is undefined.
I've been looking at this SO question and I am wondering if it has something to do with needing to use bodyParser for onRequest functions.
HTTP Event Cloud Function: request body value is undefined
I'm also seeing that some people have CORS issues with their cloud functions, which seems like it might be related.
Here is the task queue code that should be sending the payload.
const seconds = 5;
const project = 'xxxxx-xxxxxxx';
const queue = 'xxxxx';
const location = 'us-west2';
const url = 'https://us-central1-xxxxx-xxxxx.cloudfunctions.net/writeDB';
const payload = 'My data';
const parent = client.queuePath(project, location, queue);
const task = {
httpRequest: {
httpMethod: "POST",
url: url,
body: Buffer.from(JSON.stringify(payload)).toString("base64"),
headers: {
"Content-Type": "application/json"
},
oidcToken: {
serviceAccountEmail
}
}
};
task.scheduleTime = {
seconds: seconds + Date.now() / 1000,
};
const request = {parent: parent, task: task};
await client.createTask(request)
.then(response => {
const task = response[0].name;
console.log(`Created task ${task}`);
return {'Response': String(response)}
})
.catch(err => {
console.error(`Error in createTask: ${err.message || err}`);
next()
});
It calls the function, but for some reason it results in the error and the payload isn't logged.
Can anyone help?
As always, I'm happy to clarify the question if anything is unclear. Thanks!
I was able to replicate your error and I managed to fix it by changing the content type headers from "application/json" to "text/plain". I have also removed the JSON.stringify() function in the body value because your payload variable is a String type. Below is my modified sample of your code:
const {CloudTasksClient} = require('#google-cloud/tasks');
// Instantiates a client.
const client = new CloudTasksClient();
const seconds = 5;
const serviceAccountEmail = "xxxx-xxxxx-xxxxxx#appspot.gserviceaccount.com";
const project = 'xxxx-xxxxxx';
const queue = "xx-xxxxx";
const location = 'us-central1';
const url = "https://us-central1-xxxxx-xxxxx.cloudfunctions.net/myFunction";
const payload = 'My Data';
const parent = client.queuePath(project, location, queue);
async function quickstart() {
const task = {
httpRequest: {
httpMethod: "POST",
url: url,
body: Buffer.from(payload).toString("base64"), // your previous code: body: Buffer.from(JSON.stringify(payload)).toString("base64"),
headers: {
"Content-Type": "text/plain"
},
oidcToken: {
serviceAccountEmail
}
}
};
task.scheduleTime = {
seconds: seconds + Date.now() / 1000,
};
const request = {parent: parent, task: task};
await client.createTask(request)
.then(response => {
const task = response[0].name;
console.log(`Created task ${task}`);
return {'Response': String(response)}
})
.catch(err => {
console.error(`Error in createTask: ${err.message || err}`);
next()
});
}
quickstart();
In Cloud Functions, I changed req.query to req.body to get the result from Cloud Tasks
const functions = require("firebase-functions");
exports.myFunction = functions.https.onRequest((req, res) => {
console.log(req.body);
console.log('success')
res.send('success');
});
I tried every possible solution on the internet with no hope
What I am trying to do is simply use aws lambda functions (through the aws console) to fetch user fcm token from lets say DynamoDB (not included in the question), use that token to create endpointArn, send push to that specific device
I tested to send Using SNS console and the push gets to the device successfully but I failed to get it to the device using Lambda functions although it gives success status and message ID
Here is the code I used
// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set region
AWS.config.update({region: 'us-east-1'});
const sns = new AWS.SNS()
const sampleMessage = {
"GCM": {
"notification": {
"body": "Sample message for Android endpoints",
"title":"Title Test"
}
}
}
exports.handler = async (event) => {
const snsPayload = JSON.stringify(sampleMessage);
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
const params = {
PlatformApplicationArn: '<Platform Arn>',
Token: '<FCM Token>'
};
try {
const endpointData = await sns.createPlatformEndpoint(params).promise();
const paramsMessage = {
Message: snsPayload,
TargetArn: endpointData.EndpointArn
};
var publishTextPromise = await sns.publish(paramsMessage).promise();
response.MessageId = publishTextPromise.MessageId;
response.result = 'Success';
}
catch (e) {
console.log(e.stack)
response.result = 'Error'
}
return response;
};
After some trials and errors I figured out the solution for my own question
1- The GCM part of the payload should be a string not a json
2- The message parameter should have an attribute that explicitly sets the mime type of the payload to Json
Taking all that into consideration
const GCM_data = {
'notification': {
'body': 'Hellow from lambda function',
'title': 'Notification Title'
}
}
const data = {
"GCM": JSON.stringify(GCM_data)
}
const snsPayload = JSON.stringify(data)
and the params should look like
const paramsMessage = {
Message: snsPayload,
TargetArn: endpointData.EndpointArn,
MessageStructure: 'json'
};
and this will work :)
This is my setup.
Subscription A is a push subscription that POSTs messages to a cloud Run deployment.
That deployment exposes an HTTP endpoint, processes the message, posts the result to Topic B, and responds 200 to subscription A's POST request. The whole process takes ~1.5 seconds.
Therefore, for every message in subscription A, I should end up with 1 message in Topic B.
This is how my code looks like
My app started an Express server
const express = require('express');
const bodyParser = require('body-parser');
const _ = require('lodash');
const startBrowser = require('./startBrowser');
const tab = require('./tab');
const createMessage = require('./publishMessage');
const domain = 'https://example.com';
require('dotenv').config();
const app = express();
app.use(bodyParser.json());
const port = process.env.PORT || 8080;
app.listen(port, async () => {
console.log('Listening on port', port);
});
The endpoint where all the magic happens
app.post('/', async (req, res) => {
// Define the success and fail functions, that respond status 200 and 500 respectively
const failed = () => res.status(500).send();
const completed = async () => {
const response = await res.status(200).send();
if (response && res.writableEnded) {
console.log('successfully responded 200');
}
};
//Process the data coming from Subscription A
let pubsubMessage = decodeBase64Json(req.body.message.data);
let parsed = await processor(pubsubMessage);
//Post the processed data to topic B
let messageId = await postParsedData(parsed);
if (messageId) {
// ACK the message once the data has been processed and posted to topic B.
completed();
} else {
console.log('Didnt get a message id');
// failed();
}
});
//define the functions that post data to Topic B
const postParsedData = async (parsed) => {
if (!_.isEmpty(parsed)) {
const topicName = 'topic-B';
const messageIdInternal = await createMessage(parsed, topicName);
};
return messageId;
} else {
console.log('Parsed is Empty');
return null;
}
};
function decodeBase64Json(data) {
return JSON.parse(Buffer.from(data, 'base64').toString());
}
Execution time takes about ~1.5 seconds and I can see the successful responses logged on Cloud run every ~1.5seconds. That adds up to about ~2400messages/hour (per cloud run instance).
Topic B is getting new messages at ~2400messages/hour, Subscription A's acknowledgement rate is ~200messages/hour, which leads to the messages being re-delivered many times.
Subscription A's Acknowledgement deadline is 600 seconds.
The request timeout period in Cloud run is 300 seconds.
I've tried ACKing messages before they're published to topic-B or even before parsing, but I'm getting the same result.
Edit: added screenshot of the pending messages and processed messages. Many more messages processed than ACKed pending messages. Should be 1:1
Thanks for your help
Solution This error could not be reproduced by GCP support. It didn't happen with large amounts of Cloud Run VMs. The solution is just to increase the number of worker instances
You need to await your complete(); function call. like this
....
if (messageId) {
// ACK the message once the data has been processed and posted to topic B.
await completed();
} else {
console.log('Didnt get a message id');
// failed();
}
So when you create a new lambda from scracth you get the following default index.js inline code:
exports.handler = async (event) => {
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!')
};
return response;
};
This lambda is used by an api gateway endpoint. What is the proper way to return an error code if something isn't quite right? Would the gateway handle it or should I just change the status above to a new error code?
1xx (Informational): The request was received, continuing process
2xx (Successful): The request was successfully received, understood, and accepted
3xx (Redirection): Further action needs to be taken in order to complete the request
4xx (Client Error): The request contains bad syntax or cannot be fulfilled
5xx (Server Error): The server failed to fulfill an apparently valid request
This is just returning a 200 response which is a successful response. You simply just need to return the correct code depending on the implementation you have. I think That is what you are asking for.
For more information on status code you can read this Wiki
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes
But since you're asking about implementation I would do the following
exports.handler = async (event) => {
// TODO implement
let statusCode = 200;
let message = 'success';
... // do stuff
statusCode = 400 // something went wrong
const response = {
statusCode: statusCode,
body: JSON.stringify(message)
};
return response;
};
It's really all up to you based on how you want your application to scale. Just remember, never duplicate code. Keep it modular.
Additionally you can also declare this in a function
// Somewhere else
response = function (statusCode, message) {
return {
statusCode: statusCode,
body: JSON.stringify(message)
};
};
exports.handler = async (event) => {
// TODO implement
let statusCode = 200;
let message = 'success';
... // do stuff
statusCode = 400 // something went wrong
return response(statusCode, message);
};
Check out Error Handling Patterns from AWS.
It would be a good start unless you already read it.
https://aws.amazon.com/blogs/compute/error-handling-patterns-in-amazon-api-gateway-and-aws-lambda/
Essentially you need something of the following
exports.handler = (event, context, callback) => {
var myErrorObj = {
errorType : "InternalServerError",
httpStatus : 500,
requestId : context.awsRequestId,
message : "An unknown error has occurred. Please try again."
}
callback(JSON.stringify(myErrorObj));
};