Puppeteer-core on the aws-lambda docker image Error: Target closed - amazon-web-services

I have been working with puppeteer-core on the aws-lambda docker image for a while but I have encountered recently a problem while running it locally.
It is a known old error "Protocol Error: Target closed", but I suddenly encountered the last few days.
I tried many solutions suggested on the threads like this one but so far I haven't been successful.
This is an easy to run simplified version of the code GitHub but I will write it down here too:
async function launchBrowser() {
try {
const puppeteer = require('puppeteer-core');
const browser = await puppeteer.launch({
args: [
'--no-sandbox',
'--no-zygote',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage',
'--single-process',
],
headless: true,
ignoreHTTPSErrors: true,
executablePath: '/usr/bin/chromium-browser',
});
return browser;
} catch (error) {
console.log("error",error);
return launchBrowser();
}
}
async function lambdaHandler(event, context) {
let result = null;
console.info(`EVENT ${JSON.stringify(event, null, 2)}`);
const browser = await launchBrowser();
console.info('browser launched');
try{
console.log(await browser.version());
const page = await browser.newPage();
await page.goto(event.url);
result = await page.title();
console.info("Title",result);
}
catch(error){
console.log(error)
}
return result;
}
module.exports = { handler: lambdaHandler };
The docker file:
FROM public.ecr.aws/lambda/nodejs:14
RUN yum install -y amazon-linux-extras
RUN amazon-linux-extras install epel -y
RUN yum install -y chromium
RUN npm install puppeteer-core
COPY src/* ${LAMBDA_TASK_ROOT}
CMD [ "app.handler" ]
It gets stuck on browser.version() and sends that error.
The confusing thing is that when I deployed the code on AWS I didn't have the same issue so I couldn't identify the origin of this sudden change and why the errors are very inconsistent.
I want to also specify that I am working on MacOS Monterey with apple M1.
So if anyone can enlighten that would be very helpful.

Related

AWS-Amplify DataStoreStateError: Tried to execute DataStore.query() while DataStore was "Stopping"

JavaScript Framework
React Native
Amplify APIs
DataStore
Amplify Categories
api
Environment information
Details
Describe the bug
DataStoreStateError: Tried to execute DataStore.query() while DataStore was "Stopping".
This can only be done while DataStore is "Started" or "Stopped". To remedy:
Ensure all calls to stop() and clear() have completed first.
If this is not poss...
Hi, I have an amplify project to host backend for react-native mobile app. Suddenly it stopped to work. Any call to the data store will receive the message i mentioned above.
Our work plan is to provide amplify backend as an NPM package. This package is used from app developer to make calls to AWS-Amplify. Since 6 months everything works fine until 4 days go. We didn't had breaking changes or something. Just added a few lambda function which i don't think it could cause this issue.
We are starting datastore when app is open. DataStore.start() and clear it on SignIng event and SignOut event. We didn't make any change in this flow since a long time (4 months).
We tried to reproduce the issue locally but it is not possible for somehow. Since we are using typescript to write our NPM package. We test functionality locally with node or tsx where everything works, then we publish it and downloaded in another project react-native where everything works good. Once it is deployed, no action from DataStore could be executed. The app crashed and Sentry show us the message above. And again that start to happen 4 days ago until now.
Expected behavior
To be able to execute DataStore manipulation functionality (query, save, delete... etc) / Get connection with data store.
Reproduction steps
Whenever we call datastore functions in the react-native app. It will show this message:
DataStoreStateError: Tried to execute DataStore.query() while DataStore was "Stopping".
This can only be done while DataStore is "Started" or "Stopped". To remedy:
Ensure all calls to stop() and clear() have completed first.
If this is not poss...
Code Snippet
// Put your code below this line.
// where i clear data store and start it, just using auth events
// Clear the local datastore when signing out.
// As advised in: https://docs.amplify.aws/lib/datastore/sync/q/platform/js/#clear-local-data
EventHandler.OnSignOut = async () => {
if (Application.hasBooted()) {
await Application.clear();
}
};
// Clear the local datastore when signing in.
// As advised in: https://docs.amplify.aws/lib/datastore/sync/q/platform/js/#clear-local-data
EventHandler.OnSignIn = async () => {
if (Application.hasBooted()) {
await Application.clear();
}
};
// Boot the application as soon as the DataSync is completed
// Check if it has not been booted already first
EventHandler.OnDataSynced = async () => {
if (!Application.hasBooted()) {
await Application.boot();
}
};
// Where i do datastore call
/**
* #name SaveAccount
* #description: Save or update an account.
* If the account already exists in the datastore based
* on its id, it is updates. Otherwise a
* #type {Function}
* #param {AccountDetails} accountDetails
* #param {AccountAttributes} params
* #returns {Promise<AccountDetails>}
*/
const SaveAccount = async (
accountDetails: AccountDetails,
params?: AccountAttributes,
): Promise<AccountDetails> => {
const current = await GetAccountDetails(accountDetails.id);
if (current instanceof AccountDetails) {
return await DataStore.save(
AccountDetails.copyOf(
current,
updated => {
if (params) {
for (const key in params) {
updated[key] = params[key];
}
}
},
),
);
}
accountDetails = accountDetails instanceof AccountDetails
? accountDetails
: new AccountDetails(accountDetails);
return await DataStore.save(accountDetails);
};
aws-exports.js
/* eslint-disable */
// WARNING: DO NOT EDIT. This file is automatically generated by AWS Amplify. It will be overwritten.
const awsmobile = {
"aws_project_region": "",
"aws_cognito_identity_pool_id": "",
"aws_cognito_region": "",
"aws_user_pools_id": "",
"aws_user_pools_web_client_id": "",
"oauth": {},
"aws_cognito_username_attributes": [
"EMAIL"
],
"aws_cognito_social_providers": [],
"aws_cognito_signup_attributes": [],
"aws_cognito_mfa_configuration": "OFF",
"aws_cognito_mfa_types": [
"SMS"
],
"aws_cognito_password_protection_settings": {
"passwordPolicyMinLength": 8,
"passwordPolicyCharacters": [
"REQUIRES_LOWERCASE",
"REQUIRES_NUMBERS",
"REQUIRES_UPPERCASE"
]
},
"aws_cognito_verification_mechanisms": [
"EMAIL"
],
"aws_appsync_graphqlEndpoint": "*****",
"aws_appsync_region": "",
"aws_appsync_authenticationType": "API_KEY",
"aws_appsync_apiKey": "",
"aws_user_files_s3_bucket": "",
"aws_user_files_s3_bucket_region": "*"
};
package.json
{
"name": "#financiallease/react-native-amplify",
"version": "7.1.0",
"description": "",
"author": "itsupport#financiallease.nl",
"main": "dist/financiallease.js",
"module": "dist/financiallease.js",
"browser": "dist/financiallease.js",
"typings": "dist/financiallease.d.ts",
"types": "dist/financiallease.d.ts",
"license": "LGPL",
"scripts": {
"__BUNDLING": null,
"build:clean": "rimraf dist/",
"build": "npm run build:clean && rollup -c",
"bundle-local": "npm run build && npm pack && mv -v financiallease-react-native-amplify-*.tgz /usr/local/npm/#financiallease/react-native-amplify.tgz",
"__LINTING": null,
"autoformat": "npm run lint-typescript -- --fix && npm run lint-nodejs -- --fix",
"lint-nodejs": "eslint --config amplify/.eslintrc.js 'amplify/backend/function/**/index.js'",
"lint-typescript": "eslint --config .eslintrc.js '{src,test}/**/*.ts'",
"lint": "npm run lint-typescript && npm run lint-nodejs",
"coverage": "jest -c jest.config.ts --collectCoverage --coverageDirectory=\"./coverage\" --ci --reporters=default --reporters=jest-junit --watchAll=false",
"test": "jest -c jest.config.ts",
"__DOC GENERATION": null,
"docs:generate": "npm run build:clean && sed '/[[_TOC_]]/d' README.md > README.sanitized.md && typedoc --readme README.sanitized.md --entryPoints src --entryPointStrategy expand --out docs --theme hierarchy --name \"React Native Amplify - docs\" --includeVersion",
"docs:serve": "node -r esm --inspect docker/server.js",
"__AMPLIFY BACKEND": null,
"amplify-modelgen": "node amplify/scripts/amplify-modelgen.js",
"amplify-push": "node amplify/scripts/amplify-push.js",
"scan": "npm run build && npm run lint && npm run test && npm run docs:generate",
"upgrade-amplify-deps": "npx npm-check-updates -i '/(#?aws-amplify|#react-native-community/netinfo)/' && npm update"
},
"publishConfig": {
"#financiallease:registry": "https://gitlab.com/api/v4/projects/35071033/packages/npm/"
},
"dependencies": {
"#algolia/client-search": "^4.14.2",
"#algolia/transporter": "^4.14.2",
"#aws-amplify/core": "^4.7.2",
"#aws-amplify/datastore": "^3.12.8",
"#react-native-async-storage/async-storage": "^1.17.4",
"#react-native-community/netinfo": "^9.3.0",
"#types/amplify": "^1.1.25",
"aws-amplify": "^4.3.33",
"aws-amplify-react-native": "^6.0.5",
"aws-sdk": "^2.1142.0",
"deep-equal": "^2.0.5"
},
"devDependencies": {
"#aws-amplify/cli-extensibility-helper": "^2.3.33",
"#babel/core": "^7.17.9",
"#babel/preset-env": "^7.16.11",
"#babel/preset-typescript": "^7.16.7",
"#rollup/plugin-alias": "^3.1.9",
"#rollup/plugin-babel": "^5.3.1",
"#rollup/plugin-commonjs": "^21.0.3",
"#rollup/plugin-json": "^4.1.0",
"#rollup/plugin-multi-entry": "^4.1.0",
"#rollup/plugin-node-resolve": "^13.1.3",
"#rollup/plugin-typescript": "^8.3.1",
"#types/jest": "^27.4.1",
"#types/jest-when": "^3.5.2",
"#types/node": "^17.0.30",
"#typescript-eslint/eslint-plugin": "^5.35.1",
"#typescript-eslint/parser": "^5.18.0",
"aws-sdk-mock": "^5.7.0",
"babel-jest": "^28.0.3",
"babel-plugin-module-resolver": "^4.1.0",
"base64-js": "^1.5.1",
"eslint": "^8.12.0",
"eslint-config-prettier": "^8.5.0",
"eslint-import-resolver-alias": "^1.1.2",
"eslint-plugin-jsdoc": "^39.2.9",
"eslint-plugin-react": "^7.29.4",
"eslint-plugin-sort-exports": "^0.6.0",
"esm": "^3.2.25",
"fetch-mock": "^9.11.0",
"isomorphic-unfetch": "^3.1.0",
"jest": "^27.5.1",
"jest-junit": "^13.2.0",
"jest-when": "^3.5.1",
"jsdoc": "^3.6.10",
"mustache": "^4.2.0",
"nodemon": "^2.0.16",
"prettier": "^2.7.1",
"rimraf": "^3.0.2",
"rollup": "^2.70.1",
"rollup-plugin-copy": "^3.4.0",
"rollup-plugin-dts": "^4.2.2",
"rollup-plugin-flat-dts": "^1.7.0",
"rollup-plugin-sourcemaps": "^0.6.3",
"rollup-plugin-terser": "^7.0.2",
"rollup-plugin-ts": "^3.0.2",
"ts-node": "^10.7.0",
"tsconfig-paths": "^3.14.1",
"tslib": "^2.3.1",
"typedoc": "^0.22.15",
"typedoc-theme-hierarchy": "^1.1.1",
"typescript": "^4.6.3",
"uuid": "^8.3.2"
}
}
Logs from Sentry
This change was introduced in this PR to explicitly handle internal race conditions between overlapping clears/stops and starts/queries/mutations. Previously, DataStore consumers would see random, erratic transaction/locking/corruption errors that were uninstructive, difficult to recover from, or silent. This change raises the conflicts so consuming code can be aware of and handle those conflicts.
With that said, you just need to ensure DataStore.clear() resolves first, after making sure the query you're trying to run should be on the post-clear() side of that boundary! This could be as simple as a retry loop or a flag — or even using the clear() promise as your "flag".
Initialize your flag:
let onReady = Promise.resolve();
let isReady = true;
When you need to clear:
let onReady = DataStore.clear();
isReady = false;
onReady.then(() => isReady = true);
Then, for operations that should operate once the clear is complete:
async getResults() {
await onReady;
return DataStore.query(...);
}
And for those that should be canceled (no longer make sense) if they interrupt a clear, something like:
async getResults() {
if (isReady) {
return DataStore.query(...);
} else {
throw new Error(
"Sorry. We're still clearing data. Try again shortly."
);
}
}
I've provided a little more detail in the cross-post of this question on GitHub.

detox[3338] DEBUG: [APP_STATUS] Failed to execute the current status query

I try to add detox to my expo application, everthing seams to work fine, but when I try to run the test I get following error message.
detox[3338] DEBUG: [APP_STATUS] Failed to execute the current status query.
firstTest.e2e.ts
const { reloadApp } = require('detox-expo-helpers');
describe('Example', () => {
beforeAll(async () => {
await reloadApp({
permissions: {
location: 'always',
userTracking: 'YES'
}
})
});
it('test', async () => {
await expect(element(by.id('settings'))).toBeVisible()
await element(by.id('settings')).tap()
console.log('success')
})
})
At first I start the expo server with yarn expo start --ios and then run the detox test with yarn detox test -c ios --loglevel trace. The app will be installed and loaded but stucks at the error message. Did any one has an solution?

How to merge Junit XML report in Cypress to integrate with AWS CB

I have initially used Mochaawesome report but cannot integrate with AWS. It turned out I need JUnit XML reporter in order to integrate with code build.
I've created Junit XML report but I don't know how to merge them into one xml file so that it can be used in AWS.
XML files got created (which I've been trying to merge them)
Cypress.json file
"reporter": "cypress-multi-reporters",
"reporterOptions": {
"reporterEnabled": "spec, mocha-junit-reporter",
"mochaJunitReporterReporterOptions": {
"mochaFile": "cypress/results/results-[hash].xml"
}
index.js file
"scripts": {
"delete:reports": "rm cypress/results/* || true",
"prereport": "delete:reports",
"report": "cypress run --reporter cypress-multi-reporters --reporter-options mochaFile=cypress/results/results-[hash].xml"
},
"dependencies": {
"cypress-multi-reporters": "^1.4.0",
"junit-report-merger": "^0.0.6",
"mocha": "^8.2.1",
"mocha-junit-reporter": "^2.0.0",
}
Command line (but it doesn't take the password so my tests all fail)
$ yarn report --env password=<password>
I've created a package specially for that purpose. It is called junit-report-merger.
You should write a Nodejs script which will use functions exported from that package:
merge.js
const path = require('path')
const {mergeFiles} = require('junit-report-merger')
const globby = require('globby')
const inputFiles = await globby(['results/report-*.xml'])
const outputFile = path.join(__dirname, 'results', 'combined-report.xml')
mergeFiles(
outputFile,
inputFiles,
err => {
if (err) {
console.error(err)
}
else {
console.log('successfully merged')
}
}
)
Once script is ready, you should run it after your tests. In your case, it will be something like this:
"scripts": {
"report": "cypress run --reporter cypress-multi-reporters --reporter-options mochaFile=cypress/results/results-[hash].xml",
"postreport": "node merge.js"
}
UPDATE
Just released version 1.0.0 of junit-report-merger, which has glob support, allows async/await and offers a CLI.
The code above still should work, but with that version, merge.js file from above can be written in a shorter way:
const path = require('path')
const {mergeFiles} = require('junit-report-merger')
const inputPattern = ['results/report-*.xml']
const outputFile = path.join(__dirname, 'results', 'combined-report.xml')
await mergeFiles(outputFile, inputPattern)
console.log('successfully merged')
But with version 1.0.0 you can avoid creating merge.js completely and use CLI instead.
Like this:
"scripts": {
"report": "cypress run --reporter cypress-multi-reporters --reporter-options mochaFile=cypress/results/results-[hash].xml",
"postreport": "jrm ./results/combined-report.xml \"./cypress/results/results-*.xml\""
}

How to set up pm2 in digital ocean cloud server for MERN Stack?

I able to set up the pm2 successfully with my MERN Stack application, but when I tried to run pm2 start server.js and it shows status online but I was not able to access my MERN stack application in the browser, but when I run my application without using pm2 npm run dev with Nodemon everything was working fine. Below is my server.js file
const express = require('express');
const mongoose = require('mongoose');
const morgan = require('morgan');
const bodyParser = require('body-parser');
const cors = require('cors');
require('dotenv').config();
const config = require('config');
const path = require('path');
const devPort = 8080;
// Setup express app
const app = express();
app.use(cors());
app.options('*', cors());
app.use(express.json());
app.use(morgan('combined'));
app.use(bodyParser.urlencoded({ extended: true }));
mongoose.Promise = global.Promise;
const db = config.get('MONGODB_URI');
mongoose.connect(db, {
useUnifiedTopology: true,
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false
});
mongoose.connection
.once('open', () => console.log('Database connected!'))
.on('error', error => console.log('Could not connect', error));
/*route/api/file is here*/
app.use('/api/user', require('./route/api/user'));
//server static assets in production
if (process.env.NODE_ENV === 'production') {
//set static folder
app.user(express.static('client/build'));
app.get('*', (req, res) => {
res.sendFile(path.resolve(__dirname, 'client', 'build', 'index.htm'));
});
}
app.listen(process.env.PORT || devPort, () =>
console.log(`Node JS is running on port ${devPort}`)
);
here is an image of pm2 server.js online in the terminal but I could not access it.
I wondered, is it the problem with the server.js file, please give suggestions.
Thank you for your help.
Setup pm2 config
cd to project folder
Create ecosystem.config.js file for pm2 with the following config
module.exports = {
apps : [{
name : 'APPNAME',
script : './index.js',
env: {
NODE_ENV: 'development'
},
env_production : {
NODE_ENV: 'production'
}
}],
};
Start app's process using pm2
For production : pm2 start --env production
For development : pm2 start --env development
Some basic pm2 commands
Stop App : pm2 stop APPNAME
Start App : pm2 start APPNAME
Monitor App : pm2 monit APPNAME
Delete App : pm2 delete APPNAME
Show list of running pm2 processes : pm2 list
Hope this helps!

newman execution throwing TypeError: newman.run is not a function

I have newman 3.9.3 version installed on my ubuntu box. Want to execute multiple collections from a folder but executing js file through me wired error saying
TypeError: newman.run is not a function.
Here is my execution script. Any help will be appreciated.
#!/usr/bin/env node
var newman = require(process.env.NVM_BIN+'/newman');
var fs = require('fs');
fs.readdir('./collections', function (err, files) {
if (err) { throw err; }
files = files.filter(function (file) {
return (file.substr(-5) === '.json');
});
// now wer iterate on each file name and call newman.run using each file name
files.forEach(function (file) {
newman.run({
environment: require(`${__dirname}/live.postmane_environment.json`),
collection: require(`${__dirname}/collections/${file}`),
reporters: ['cli']
}, function (err) {
console.info(`${file}: ${err ? err.name : 'ok'}!`);
});
});
});
Following is the exact error.
/app/postman/execute:15
newman.run({
^
TypeError: newman.run is not a function
at /app/postman/execute:15:16
at Array.forEach (native)
at /app/postman/execute:14:11
at FSReqWrap.oncomplete (fs.js:123:15)
For the time being, I've solved this problem by using bash script to run all of my collections available in a folder. that has done the job. but originally i could not understand why "run" is not available for "newman" object.
I was getting the same error message and solved installing the nodejs package:
npm install --save newman
Basically, when you make the bin as reference the nodejs doesnt know how to run:
Instead of:
var newman = require(process.env.NVM_BIN+'/newman');
Should be:
var newman = require('newman');