I'm making an Api that returns a pdf by puppeteer. I just instaled puppeteer with npm install chrome-Aws-lambda, install puppeteer --save-dev but when I run the Api, I get this exception.
I tried runing npm install but it doesn't work, how can I install chromium or make puppeteer works???
this is my code and package.json
let browser = await chromium.puppeteer.launch({ headless: true });
let page = await browser.newPage();
await page.goto("https://www.google.com");
const pdf = await page.pdf({
format: "A4",
printBackground: false,
preferCSSPageSize: true,
displayHeaderFooter: false,
headerTemplate: `<div class="header" style="font-size:20px; padding-left:15px;"><h1>Main Heading</h1></div> `,
footerTemplate: '<footer><h5>Page <span class="pageNumber"></span> of <span class="totalPages"></span></h5></footer>',
margin: { top: "200px", bottom: "150px", right: "20px", left: "20px" },
height: "200px",
width: "200px",
});
return {
statusCode: 200,
// Uncomment below to enable CORS requests
headers: {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Credentials": true
},
body: pdf
};
Pakage:
{
"name": "amplifysandboxpdf",
"version": "2.0.0",
"description": "Lambda function generated by Amplify",
"main": "index.js",
"license": "Apache-2.0",
"devDependencies": {
"#types/aws-lambda": "^8.10.92",
"#types/puppeteer": "^5.4.6",
"puppeteer": "^17.1.2"
},
"dependencies": {
"chrome-aws-lambda": "^10.1.0",
"puppeteer-core": "^10.0.0"
}
}
Usually, when you upload a zip to lambda, you need to provide both - your source code and node_modules (modules can also be added as a lambda layer). In your case error is because of missing package, so the first place I'd look at is the zip you provide to lambda and does it contain all the necessary packages
I see in your package.json you have both puppeteer and puppeteer-core. Puppeteer downloads chromium on installation (that's the bit that errors for you). First you need to decide if puppeteer is even necessary, maybe core package is enough, but if it is necessary - once again, it should be in your zip that you provide.
I'm not sure about how puppeteer does this, but if it downloads chromium into node_modules it should work once your zip file is correctly packaged. Otherwise, you might need to create your own docker container by using puppeteer base image Puppeteer base image.
If this is the case, these are the steps you need to take:
Create a dockerfile with puppeteer base image, build your own image and make sure your application is installed in there. Example of what this file could look like:
FROM ghcr.io/puppeteer/puppeteer:16.1.0
WORKDIR /home/pptruser/app
ADD ./package-lock.json /home/pptruser/app/package-lock.json
ADD ./package.json /home/pptruser/app/package.json
RUN npm ci
# Customize the line below - copy files that your application requires
ADD ./src/. /home/pptruser/app/src/
# Remove development dependencies (in your case puppeteer is not a devDependency)
RUN npm prune --production
CMD [ "index.js" ]
Test locally if it works in the container and later publish this image to ECR (AWS registry for docker containers)
Launch lambda by using the image instead of using NodeJS environment
Related
When I develop sample apps by using AWS CDK,I found app section in cdk.json.I have question about app. what is app ? it seems that app.js is executed.
When this command will be executed ? when we run cdk synth,is this command executed?
I don't know the purpose of this use.
Thanks
{
"app": "node packages/cdk/dist/app",
"output": "build/cdk.out",
"context": {
"#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"#aws-cdk/core:enableStackNameDuplicates": "true",
"aws-cdk:enableDiffNoFail": "true",
"#aws-cdk/core:stackRelativeExports": "true",
"#aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
"#aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
"#aws-cdk/aws-kms:defaultKeyPolicies": true,
"#aws-cdk/aws-s3:grantWriteWithoutAcl": true,
"#aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount": true,
"#aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"#aws-cdk/aws-efs:defaultEncryptionAtRest": true,
"systemName": "devio",
"envType": "stg"
}
}
Per the CDK documentation:
Since the AWS CDK supports programs written in a variety of languages, it uses a configuration option to specify the exact command necessary to run your app. This option can be specified in two ways.
First, and most commonly, it can be specified using the app key inside the file cdk.json, which is in the main directory of your AWS CDK project. The CDK Toolkit provides an appropriate command when creating a new project with cdk init.
Here is the cdk.json from a fresh TypeScript project, for instance:
{
"app": "npx ts-node bin/hello-cdk.ts"
}
I have started using Serverless framework with AWS. My source is in Typescript which would be built to JavaScript before deploying. This gets uploaded to S3 and then lambda function is created. I noticed that my lambda functions are over 70MB although I only have a few lines of code with operations that use just the aws-sdk, like querying DynamoDB or SecretsManager.
To investigate this, I downloaded the zipped file which gets uploaded to S3 by serverless framework and unzipped for its content. It has a folder named ${WORKSPACE} which accounts for the 70% of the package memory and it does not seem to have any relevant content for the lambda function.
My package.json looks like this
{
"name": "my-api",
"version": "0.0.1",
"description": "API to retrieve secrets from Secrets Manager",
"license": "UNLICENSED",
"engines": {
"node": ">= 14.0.0"
},
"scripts": {
"build": "tsc -p .",
"deploy": "sls deploy"
},
"dependencies": {
"#aws-sdk/client-secrets-manager": "^3.33.0"
},
"devDependencies": {
"serverless": "^2.59.0",
"typescript": "^4.4.3"
}
}
and the relevant part of the serverless.yml looks like this
service: sls-my-api
useDotenv: true
variablesResolutionMode: 20210326
frameworkVersion: '2'
package:
patterns:
# Includes
- 'dist/**.js'
# Excludes
- '!./**.md'
- '!./**.env*'
- '!.github/**'
- '!.npm/**'
- '!coverage/**'
- '!docker/**'
- '!docs/**'
- '!dist/tsconfig.build.tsbuildinfo'
- '!dist/**.d.ts'
- '!src/**'
- '!test/**'
- '!.eslintrc.js'
- '!.npmrc'
- '!.prettier*'
- '!./tsconfig*.json'
- '!Jenkinsfile'
- '!jest.json'
- '!nest-cli.json'
- '!run-sonar.sh'
- '!sonar-project.properties'
- '!tslint.json'
- '!.gitattributes'
- '!.npmignore'
- '!.s2iignore'
- '!database/**'
In the Jenkins stages, the following are run
npm run build
npm run deploy
Also because of the size limitation, inline editing is not available on the lambda console which usually is a very useful action for me, especially when testing and debugging quickly without having to wait for a redeployment.
Does the directory ${WORKSPACE} have any significance for this deployment? If not, how do I exclude it from the deployment package and make my lambda lean?
False Alarm!
The directory ${WORKSPACE} is generated because of Jenkins run and not because of the serverless framework. The frame although was picking it up while packing and deploying the application thus making the lambda function bulk.
Excluding it as follows did the trick.
package:
patterns:
# Includes
- 'dist/**.js'
# Excludes
- '!${WORKSPACE}'
Maybe something good to know for people using serverless cli on Jenkins.
Context
For a long time I was using Bootstrap and other libraries by sources. I mean I was including it as below:
- static
- vendor
- jquery
- popper
- bootstrap
- ...
But nowadays, I need to improve myself in managing my web dependencies. For some reasons :
Less sources in repositories and a more efficient team project
Greater support to choose version or test migrations
Saving time for later projects with same dependencies
I need to include a library which is only provided through NPM which is codemirror
So I am currently trying to manage dependencies via NPM, in a django project. But I'm stuck at a very basic step I guess. I don't know where to continue after npm install --save jquery popper.js bootstrap. I'm disturbed by all examples which are for node.js application...
Where I'm stuck
I have an index.js at the same level of my main package.json. I thought I had to import my scripts in this file, then include this script in my page. So I tried require Jquery then bootstrap, but I get errors on bootstrap required because jquery is undefined.
I don't have much code to show, the more I need is to understand how it works step by step rather than having it done to me.
Questions
My primary goal is to manage my main javascripts, I mean scripts that must appear through the whole site. Wrap them into one script, then include this script in my base.html assuming I'm in a django project.
I know that bootstrap < v5 depends on Jquery and popper.js, but it comes with its own package.json, isn't that enough ? Do we have to npm install jquery popper.js them anyway, as some people suggest in some SO thread or else ? Even after running npm install inside the node_module/bootstrap ? Additionally, the bootstrap.bundle.js contains popper.js, and node_modules inside bootstrap contains jquery. Why not use those ? See Bootstrap import with webpack Otherwise why are they there ?
How to bundle my javascript dependencies ? Via webpack ? For example, Jquery from bootstrap/node_modules/jquery/dist/jquery.js with bootstrap/dist/js/bootstrap.bundle.js ? Then I will have same versions used by bootstrap developers.
Then how to compile the wanted scripts into one script, and minify it for example ? Then I could get those scripts by collectstatic function into the folder where I group all my desired statics.
Perhaps I'm not clear, because all of this blurred my mind.
Webography
Use Sass/css within django, NPM way
Add react in django project, via NPM
Bootstrap package manager
Bootstrap import with webpack
Npm install jquery, poper, bootstrap
React with webpack
Editions with my progress
package.json
{
"name": "static_src",
"version": "1.0.0",
"description": "Static files from NPM.",
"private": true,
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"build": "npx webpack -c webpack-conf.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"bootstrap": "^4.5.3",
"codemirror": "^5.58.2",
"jquery": "^3.5.1",
"popper.js": "^1.16.1"
},
"devDependencies": {
"webpack": "^5.6.0",
"webpack-cli": "^4.2.0"
}
}
webpack-conf.js
var path = require('path');
var webpack = require('webpack');
module.exports = {
entry: './index.js',
output: { path: __dirname, filename: 'dist/bootstrap-bundle.js' }
};
index.js
import $ from 'jquery';
import {} from 'popper.js';
import 'bootstrap'
npm run build
> npx webpack -c webpack-conf.js
[webpack-cli] Compilation finished
asset dist/bootstrap-bundle.js 169 KiB [emitted] [minimized] (name: main) 1 related asset
runtime modules 1.13 KiB 5 modules
cacheable modules 508 KiB
./index.js 71 bytes [built] [code generated]
./node_modules/jquery/dist/jquery.js 281 KiB [built] [code generated]
./node_modules/bootstrap/dist/js/bootstrap.js 140 KiB [built] [code generated]
./node_modules/popper.js/dist/esm/popper.js 86.4 KiB [built] [code generated]
webpack 5.6.0 compiled successfully in 4541 ms
Result
Well for now I manage to include this bundle into my site, but an error occured, and it seems not including Jquery : ReferenceError: $ is not defined. Notably it occurs on this snippet for example :
$(function () {
$('[data-toggle="popover"]').popover()
$('[data-toggle="tooltip"]').tooltip()
})
Any idea ?
Answer to Q1, assuming you have run npm install bootstrap and you can find 'popper' and 'jquery' in the node_modules folder:
Add import 'bootstrap'; in your index.js (https://getbootstrap.com/docs/4.0/getting-started/webpack/)
NOTE:
IF you cannot find 'popper' and 'jquery' in the node_modules folder, THEN It's worthwhile to install them via npm using npm install #popperjs/core & npm install jquery
IF you had to install popper and jQuery manually for some reason, THEN Add import $ from 'jquery'; & import {} from '#popperjs/core'; in your index.js
I came to a great solution !
webpack-conf.js
var path = require('path');
var webpack = require('webpack');
module.exports = {
entry: './index.js',
output: {path: __dirname, filename: 'dist/bootstrap-bundle.js'},
plugins: [
new webpack.ProvidePlugin({
$: 'jquery',
jQuery: 'jquery'
})
]
};
index.js
import $ from 'jquery';
window.jQuery = $;
window.$ = $;
import {} from 'popper.js';
import 'bootstrap'
Then importing this script works well. Thanks to #Karanveer that unblurred my mind to put me on the right way. And this thread about webpack issues
I am new to AWS. I am exploring hosting my rest server on AWS but have not been able to solve the following problem:
Failed to find package.json. Node.js may have issues starting. Verify package.json is valid or place code in a file named server.js or app.js.
There were suggestions that I should zip only the files and sub-dir but not the root folder.
I tried the recommendation suggested to zip just the files and sub-folders but it still doesn't work.
The following are the files I zip:
bin
models
node_modules
public
routes
views
app.js
authenticate.js
ca.crt
ca.key
config.js
db.json
ia.crt
ia.csr
ia.key
ia.p12
package.json
The following are the content of my package.json file:
{
"name": "rest-server",
"version": "0.0.0",
"private": true,
"scripts": {
"start": "node ./bin/www"
},
"dependencies": {
"body-parser": "~1.15.2",
"cookie-parser": "~1.4.3",
"debug": "~2.2.0",
"express": "~4.14.0",
"jade": "~1.11.0",
"jsonwebtoken": "^7.1.9",
"mongoose": "^4.7.0",
"mongoose-currency": "^0.2.0",
"morgan": "~1.7.0",
"passport": "^0.3.2",
"passport-facebook": "^2.1.1",
"passport-local": "^1.0.0",
"passport-local-mongoose": "^4.0.0",
"serve-favicon": "~2.3.0"
}
}
I am using the Elastic Beanstalk Dashboard not the eb command line method.
What am I doing wrongly?
There is a possible solution from AWS developer forum regarding about the error in NodeJS deploy in elasticbeanstalk.It might due to the package.json was not zip in the correct location.
Node.js deploy fail in Elasticbeanstalk
https://forums.aws.amazon.com/thread.jspa?threadID=130140&tstart=0
You may try to download the sample application from AWS website and follow the folder structure from there:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/tutorials.html
I'm trying to get casperjs to work with my AWS Lambda function.
{
"errorMessage": "Cannot find module 'casper'",
"errorType": "Error",
"stackTrace": [
"Function.Module._load (module.js:276:25)",
"Module.require (module.js:353:17)",
"require (internal/module.js:12:17)",
"Object.<anonymous> (/var/task/index.js:3:14)",
"Module._compile (module.js:409:26)",
"Object.Module._extensions..js (module.js:416:10)",
"Module.load (module.js:343:32)",
"Function.Module._load (module.js:300:12)",
"Module.require (module.js:353:17)"
]
}
I keep getting this error where Lambda can't detect casperjs. I uploaded my zip file into Lambda, and installed the casperjs modules into my directory before I zipped the files up.
My package.json file says I have casperjs installed.
{
"name": "lambda",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"casperjs": "^1.1.3",
}
}
Would anyone know what I'm doing wrong? Thanks.
Since CasperJs relies on PhantomJs, you can set it up very similarly to this repo: https://github.com/TylerPachal/lambda-node-phantom.
The main difference being that you need to add and target CasperJs and you need to make sure that CasperJs can find and load PhantomJs.
Create a node_modules directory in your package directory.
Add a dependency for CasperJs to the packages.json file:
"dependencies": {
"casperjs": "latest"
}
In Terminal, navigate to your package directory and run 'npm update' to add the CasperJs package to the node_modules directory.
Assuming that you want to run CasperJs with the 'test' argument, the index.js file will need to be changed to look like this:
var childProcess = require('child_process');
var path = require('path');
exports.handler = function(event, context) {
// Set the path as described here: https://aws.amazon.com/blogs/compute/running-executables-in-aws-lambda/
process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'];
// Set the path to casperjs
var casperPath = path.join(__dirname, 'node_modules/casperjs/bin/casperjs');
// Arguments for the casper script
var processArgs = [
'test',
path.join(__dirname, 'casper_test_file.js')
];
// Launch the child process
childProcess.execFile(casperPath, processArgs, function(error, stdout, stderr) {
if (error) {
context.fail(error);
return;
}
if (stderr) {
context.fail(error);
return;
}
context.succeed(stdout);
});
}
If you don't want to run CasperJs with the 'test' argument, just remove it from the arguments list.
The PhantomJs binary in the root directory of your package needs to be renamed to phantomjs, so that CasperJs can find it. If you would like to get a new version of PhantomJs, you can get one here: https://bitbucket.org/ariya/phantomjs/downloads. Make sure to download a linux-x86_64.tar.bz2 type so that it can run in Lambda. Once downloaded, just pull a new binary out of the bin directory and place it in your root package directory.
In order for Lambda to have permission to access all the files, it's easiest to zip the package in a Unix-like operating system. Make sure that all the files in the package have read and execute permissions. From within the package directory: chmod -R o+rx *. Then zip it up with: zip -r my_package.zip *.
Upload the zipped package to your Lambda function.
According to Casper.js Docs, it is not a actually Node Module. So you cannot require it in Package.json and zip it up with node modules. You will need to find how to install it on the lambda instance or find a an actual node module that does what you want. I suspect installing casper on lambda might not be possible, but that's just my gut.
Warning
While CasperJS is installable via npm, it is not a NodeJS module and will not work with NodeJS out of the box. You cannot load casper by using require(‘casperjs’) in node. Note that CasperJS is not capable of using a vast majority of NodeJS modules out there. Experiment and use your best judgement.
http://docs.casperjs.org/en/latest/installation.html