AWS CloudWatch Synthetics Canary doesn't load the entire page - amazon-web-services

Question

I basically want to load the following page every minute and create a screenshot of it: https://www.amazon.com/b?ie=UTF8&node=24088939011
It takes about one minute to create an AWS CloudWatch canary: I open AWS CloudWatch, click on "Synthetics Canaries" on the left hand side, click on "Create canary", enter the URL to a webpage, and just use the default settings except that I change it from running every 5 minutes to running every minute.
Availability tab of the canary I created:
Configuration tab of the canary I created:
The canary runs and says it's 100% successful but when I look at the screenshots I see that the page never loads fully:
The screenshots should show what you would see when opening the page in your browser: https://www.amazon.com/b?ie=UTF8&node=24088939011
This is the default script that is used by the canary:
const { URL } = require('url');
const synthetics = require('Synthetics');
const log = require('SyntheticsLogger');
const syntheticsConfiguration = synthetics.getConfiguration();
const syntheticsLogHelper = require('SyntheticsLogHelper');
const loadBlueprint = async function () {
const urls = ['https://www.amazon.com/b?ie=UTF8&node=24088939011'];
// Set screenshot option
const takeScreenshot = true;
/* Disabling default step screen shots taken during Synthetics.executeStep() calls
* Step will be used to publish metrics on time taken to load dom content but
* Screenshots will be taken outside the executeStep to allow for page to completely load with domcontentloaded
* You can change it to load, networkidle0, networkidle2 depending on what works best for you.
*/
syntheticsConfiguration.disableStepScreenshots();
syntheticsConfiguration.setConfig({
continueOnStepFailure: true,
includeRequestHeaders: true, // Enable if headers should be displayed in HAR
includeResponseHeaders: true, // Enable if headers should be displayed in HAR
restrictedHeaders: [], // Value of these headers will be redacted from logs and reports
restrictedUrlParameters: [] // Values of these url parameters will be redacted from logs and reports
});
let page = await synthetics.getPage();
for (const url of urls) {
await loadUrl(page, url, takeScreenshot);
}
};
// Reset the page in-between
const resetPage = async function(page) {
try {
await page.goto('about:blank',{waitUntil: ['load', 'networkidle0'], timeout: 30000} );
} catch(ex) {
synthetics.addExecutionError('Unable to open a blank page ', ex);
}
}
const loadUrl = async function (page, url, takeScreenshot) {
let stepName = null;
let domcontentloaded = false;
try {
stepName = new URL(url).hostname;
} catch (error) {
const errorString = `Error parsing url: ${url}. ${error}`;
log.error(errorString);
/* If we fail to parse the URL, don't emit a metric with a stepName based on it.
It may not be a legal CloudWatch metric dimension name and we may not have an alarms
setup on the malformed URL stepName. Instead, fail this step which will
show up in the logs and will fail the overall canary and alarm on the overall canary
success rate.
*/
throw error;
}
await synthetics.executeStep(stepName, async function () {
const sanitizedUrl = syntheticsLogHelper.getSanitizedUrl(url);
/* You can customize the wait condition here. For instance, using 'networkidle2' or 'networkidle0' to load page completely.
networkidle0: Navigation is successful when the page has had no network requests for half a second. This might never happen if page is constantly loading multiple resources.
networkidle2: Navigation is successful when the page has no more then 2 network requests for half a second.
domcontentloaded: It's fired as soon as the page DOM has been loaded, without waiting for resources to finish loading. Can be used and then add explicit await page.waitFor(timeInMs)
*/
const response = await page.goto(url, { waitUntil: ['networkidle0'], timeout: 30000});
if (response) {
domcontentloaded = true;
const status = response.status();
const statusText = response.statusText();
logResponseString = `Response from url: ${sanitizedUrl} Status: ${status} Status Text: ${statusText}`;
//If the response status code is not a 2xx success code
if (response.status() < 200 || response.status() > 299) {
throw `Failed to load url: ${sanitizedUrl} ${response.status()} ${response.statusText()}`;
}
} else {
const logNoResponseString = `No response returned for url: ${sanitizedUrl}`;
log.error(logNoResponseString);
throw new Error(logNoResponseString);
}
});
// Wait for 15 seconds to let page load fully before taking screenshot.
if (domcontentloaded && takeScreenshot) {
await page.waitFor(15000);
await synthetics.takeScreenshot(stepName, 'loaded');
await resetPage(page);
}
};
const urls = [];
exports.handler = async () => {
return await loadBlueprint();
};
I tried creating a canary in exactly the same way but for another similar page (https://www.amazon.ca/b?ie=UTF8&node=6548466011) and it just works:
What am I doing wrong? Why aren't the screenshots that the canary takes showing the fully loaded page?
Why are parts of the page missing in the screenshot but they show up correctly when I open the page (https://www.amazon.com/b?ie=UTF8&node=24088939011) in my browser?