How to be notified on Google Cloud Function timeout? - google-cloud-platform

Question

I've enabled the notifications in Stackdriver and I'm getting notification e-mails for exceptions just fine.
The problem is that I don't get any notification for timeouts.
Is there any way to be notified when a Google Cloud Function is killed by timeout?

Answer 1

Even though a timeout is not reported as an error, you can still set up a metric for timeout log entries, and then an alert on the metric exceeding a zero threshold.
From the GCP console, go to the Stackdriver Logging viewer (/logs/viewer), and build a filter like this:
resource.type="cloud_function"
resource.labels.function_name="[YOUR_FUNCTION_NAME_HERE]"
"finished with status: timeout"
The third line is a "contains" text filter. Timeout messages consistently contain this text. You can add other things or modify as needed.
Click Create Metric. Give the metric a name like "Function timeouts", and make sure the type is counter. You can leave the optional fields blank. Submit the form, and you should be redirected to /logs/metrics.
Under User-defined Metrics, you should see your new metric. Click the three-dot button on the right and select Create alert from metric.
Give the alert policy a meaningful name. Under target, you may also get some red text about being unable to produce a line plot. Click the helpful link to switch the aligner to mean and the aggregator to none. Then under Configuration, set the condition to "is above," threshold to "0", and for "most recent value."
Proceed with building the notification and documentation as desired. Make sure you add a notification channel so you get alerted. The UI should include hints on each field.
More detail is in the official documentation.

Answer 2

Navigate to "Create alerting policy" using the search box at the top of the dashboard.
Under "What do you want to track?" click "Add Condition."
Configure the new condition like so:
Click "Add."
Click "Next."
Select a notification channel or create a new one.
I unchecked "Notify on incident resolution."
Click "Next."
Provide a descriptive alert name and optional documentation.
Click "Save."
Ensure that at the top of the policy you see the word "Enabled" along with a green checkmark.

Answer 3

Came up with a workaround for this by forcing an error before Cloud Functions times out. In terms of workflow, I think this is much easier to control and be able to consolidate all the errors in one place, rather than having to configure settings elsewhere.
Basically something like the code snippet below:
exports.cloudFunction = async (event, context, callback) => {
try {
const timeout = setTimeout(function(){
throw new Error(`Timeout: ${event}`);
}, 58000); // 2sec buffer off the default 60s timeout
// DO SOMETHING
clearTimeout(timeout);
callback();
} catch(e) {
// HANDLE ERROR
callback(e);
}
}