Webpack 4 SplitChunks, grouping asynchronous chunks to certain size - webpack-4

We are trying to optimize our webpack bundling.Now we have a lot of small chunks generated, but we want to group them according to some rules:
Node modules and code should be separated, eg. A chunk is either all node_modules or all code
Each chunk size should be larger then 20KB
I kind of achieve the file size constraints, but the problem in the end is that on the first page load, all the chunks are downloaded at once, I split the file because I want them to be donwloaded only when it is needed, not all at once. The following are the steps I did to get to this point. If you can point me to the right way it would be really appreciated
1.The current situation
Settings:
optimization: {
splitChunks: {
chunks: 'all', // optimize both static and dynamic import
maxAsyncRequests: 5, // for each additional load no more than 5 files at a time
maxInitialRequests: 3, // each entrypoint should not request more then 3 js files
automaticNameDelimiter: '~', // delimeter for automatic naming
automaticNameMaxLength: 30, // length of name
name: true, // let webpack calculate name
cacheGroups: {
vendors: {
test: /[\\/]node_modules[\\/]/,
priority: -10,
enforce: true // seperate vendor from our code
},
default: {
minChunks: 2,
priority: -20,
reuseExistingChunk: true
}
}
},
},
You can see there are lots of small files
2.To understand where the small chunks come from, I force merge the asynchronous chunks into two, one from node_modules and one from code
Setting
optimization: {
splitChunks: {
... same as before ...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
},
async_code: {
chunks: "async",
priority: 10,
name: "async_code",
},
vendors: {
test: /[\\/]node_modules[\\/]/,
priority: -10,
enforce: true // seperate vendor from our code
},
default: {
minChunks: 2,
priority: -20,
reuseExistingChunk: true
}
}
},
},
No more small files, so the small files are asynchronous chunks
3.So now those small files are in my control of those 2 cachegroups, I attempt to break them into smaller files
Setting
optimization: {
splitChunks: {
...same as before...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
maxSize: 200000,
minSize: 200000,
},
async_code: {
chunks: "async",
priority: 10,
name: "async_code",
maxSize: 200000,
minSize: 200000,
},
...same as before...
}
},
},
This look exactly what I wanted. Only one problem is that all these files are loaded when I visit the first page. Which does not happen in the original scenario(1.). I suspect that it's because I force the name into cacheGroup. But If I don't force the name, then small chunks are generated
4.Here is what happen if I don't specify the cachegroup name :(
Setting
optimization: {
splitChunks: {
... same as before ...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
maxSize: 200000,
minSize: 200000,
},
async_code: {
chunks: "async",
priority: 10,
maxSize: 200000,
minSize: 200000,
},
... same as before ...
}
},
},
Is it possible to solve this problem in splitchunk? Thank you for all your help

First of all, I think you've got wrong understanding of splitChunks.chunks attribute. It helps to optimize the splitting of modules which are referenced in splitChunks.chunks type of chunks. For eg. chunks=async, it will consider the modules referenced from async chunks and split/optimize them.
Coming back to the original question,
Current Situation: All your modules above 20KB(default) are splitted into chunks, irrespective of whether they are referenced from async or initial chunks. There are other constraints as well, but let's hop over them for now.
To understand where the small chunks come from, I force merge the asynchronous chunks into two, one from node_modules and one from code: Now all your modules referenced from async chunks are bundled into vendor and code chunks. This might be because modules from all small chunks(larger than 20KB) were referenced from both async and initial chunks. Could try with chunks: initial, I would expect the same result.
Since entry/initial code has references in async_vendor and async_code chunks, they are correspondingly loaded.
To answer the original question, using 3rd config for initial chunks(chunks:initial) should help.

Related

Is there a way to exclude all files with 100% or at a threshold across the board from karma-coverage text reporter results?

This is very picky of me, I know, but I have a large Angular application where many files are covered at 100% or pretty darn close to it, and I want to exclude those results, so I can just deal with the ones I need to and don't have all the extra noise.
I don't want to exclude files by name because if they change they may go below the OK threshold.
My karma.conf.js (I'm aware I may not need some of the plugins, this is a shared file by the team):
// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '#angular-devkit/build-angular'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage'),
require('#angular-devkit/build-angular/plugins/karma')
],
reporters: ['progress', 'coverage'],
coverageReporter: {
reporters: [
{type: 'text'},
{type: 'text-summary'},
]
},
browsers: ['ChromeHeadless'],
preprocessors: {'**/*.ts': ['coverage']},
restartOnFileChange: true,
});
};
and I'm gonna try to attach a screenshot here of my sample output in the type: 'text' coverageReporter:
I've tried searching SO and karma / karma-coverage documentation, but I only can find excluding files/paths specifically by name, or updating thresholds, but the latter seems to be only for determining what colors show up. Thank you~
EDIT: And since I only have one file per directory, would also be useful to not duplicate the numbers by printing them for the directory AND the one file under it. Thinking I may just have to dig into the weeds and create a pull request or something.
For this you can generate coverage reports in HTML format. There you will get option to sort files based on code-coverage.
To enable such reports you will have to do update coverageReporter property in your karma configurations -
coverageReporter: {
includeAllSources: true,
dir: 'coverage/',
reporters: [
{type: 'text'},
{type: 'text-summary'},
{type: 'html', subdir: 'html/'}
]
},
Reports will be generated within coverage/html folder.

Using breakParent to force a linebreak

I'm working on a prettier plugin and am struggling to understand the breakParent command.
From the command documentation:
Include this anywhere to force all parent groups to break.
The documentation is quite sparse, so maybe I've just completely misunderstood what it's supposed to do, but my expectation would be that it will line break parent groups in exactly the same way they would have if the line exceeded the maximum width.
I would therefore have expected the following commands
{
type: 'group',
contents: {
type: 'concat',
parts: [
'<div>',
{
type: 'indent',
contents: {
type: 'concat',
parts: [{ type: 'line', soft: true }, 'Text', { type: 'break-parent' }],
},
},
{ type: 'line', soft: true },
'</div>',
],
},
}
to print
<div>
Text
</div>
but instead I get
<div>Text</div>
How do I achieve what I thought breakParent would do, i.e. force the surrounding code to line break from inside the group?
And if that's not possible, what does breakParent actually do?
(Obviously, it can be done by turning the soft lines into hard lines, but that's not the solution I'm looking for because only the code generating "Text" in the example above knows if a line break is needed or not)

Splitting chunks - problem with regex (js)?

I'm trying to split my build files in my webpack.config.js file, but my vendors file isn't being created at all. The remaining node_modules, which aren't react or moment files end up in the main.js. An example of a file that goes in main.js is ./node_modules/es-abstract. I put in my regex and filename in a regex checker, and it passes the test. I'm not sure what's going on; any help would be greatly appreciated it.
splitChunks: {
cacheGroups: {
moment: {
test: /[\\/]node_modules[\\/]((moment).*)[\\/]/,
name: 'moment',
chunks: 'all'
},
react: {
test: /[\\/]node_modules[\\/]((react).*)[\\/]/,
name: 'react',
chunks: 'all'
},
vendors: {
test: /[\\/]node_modules[\\/]((?!(moment|react)).*)[\\/]/,
name: 'vendors',
chunks: 'all'
}
}
}
According to David Gilbertson, SplitChunks default settings only allows for three chunks. To solve this, these settings will need to be added to split chunks: maxInitialRequests: Infinity,
minSize: 0,

Dojo build requesting already inlined templates

I'm hopelessly trying to make the Dijit template inlining functionality of Dojo builds for my AMD project work with no luck yet ...
The particular issue isn't the inlining of the HTML templates themselves, but the fact that they are still requested with Ajax (XHR) after being successfully inlined.
Templates are inlined the following way :
"url:app/widgets/Example/templates/example.html": '<div>\n\tHello World!</div>'
The Dijit widget itself, after building, defines templates like this :
define("dojo/_base/declare,dijit/_Widget,dojo/text!./templates/example.html".split(","), function (f, g, d) {
return f("MyApp.Example", [g], {
templateString: d,
});
});
I tried to build with :
the shrinksafe / closure optimiser
relative / absolute paths
using the old cache() method
using the templatePath property
but even after having run a successful build (0 errrors and a few warnings) where the templates were inlined, Dojo / Dijit still makes Ajax requests to these resources.
Here is my build profile :
var profile = {
basePath: '../src/',
action: 'release',
cssOptimize: 'comments',
mini: true,
optimize: 'closure',
layerOptimize: 'closure',
stripConsole: 'all',
selectorEngine: 'acme',
internStrings: true,
internStringsSkipList: false,
packages: [
'dojo',
'dijit',
'dojox',
'app'
],
layers: {
'dojo/dojo': {
include: [
'app/run'
],
boot: true,
customBase: true
},
},
staticHasFeatures: {
'dojo-trace-api': 0,
'dojo-log-api': 0,
'dojo-publish-privates': 0,
'dojo-sync-loader': 0,
'dojo-xhr-factory': 0,
'dojo-test-sniff': 0
}
};
Due to this issue my application is completely unusable because there are so many files to download separately (browsers have a limit on the number of parallel connections).
Thank you very much in advance !
UPDATE :
The two lines loading dojo.js and the run.js in my index.html :
<script data-dojo-config='async: 1, tlmSiblingOfDojo: 0, isDebug: 1' src='/public/dojo/dojo.js'></script>
<script src='/public/app-desktop/run.js'></script>
Here is the new build-profile :
var profile = {
basePath: '../src/',
action: 'release',
cssOptimize: 'comments',
mini: true,
internStrings: true,
optimize: 'closure',
layerOptimize: 'closure',
stripConsole: 'all',
selectorEngine: 'acme',
packages : [
'dojo',
'dijit',
'app-desktop'
],
layers: {
'dojo/dojo': {
include: [
'dojo/request/xhr',
'dojo/i18n',
'dojo/domReady',
'app-desktop/main'
],
boot: true,
customBase: true
}
},
staticHasFeatures: {
'dojo-trace-api': 0,
'dojo-log-api': 0,
'dojo-publish-privates': 0,
'dojo-sync-loader': 0,
'dojo-xhr-factory': 0,
'dojo-test-sniff': 0
}
};
My new run.js file :
require({
async: 1,
isDebug: 1,
baseUrl: '/public',
packages: [
'dojo',
'dijit',
'dojox',
'saga',
'historyjs',
'wysihtml5',
'app-shared',
'jquery',
'jcrop',
'introjs',
'app-desktop'
],
deps: [
'app-desktop/main',
'dojo/domReady!'
],
callback: function (Main) {
debugger;
var main = new Main();
debugger;
main.init();
}
});
and my main.js file looks like this :
define([
'dojo/_base/declare',
'app-desktop/widgets/Application',
'app-desktop/config/Config',
'saga/utils/Prototyping',
'dojo/window',
'dojo/domReady!'
], function (declare, Application, ConfigClass, Prototyping, win) {
return declare([], {
init: function() {
// ... other stuff
application = new Application();
application.placeAt(document.body);
// ... some more stuff
}
});
});
In build-mode, I get the following error :
GET http://localhost:4000/app-desktop/run.js 404 (Not Found)
which is weird because it means that the build process made it so that dojo has an external dependency rather than an a already inlined dojoConfig variable in the builded file.
In normal-mode, files get requested, but the app is never created.
In both cases none of the two debuggers set in the run.js file were run which means that the callback method was never called for some reason.
Thank you for your help !
I've printed values of requireCacheUrl and require.cache to console in the method load() of dojo/text.js. At least in my case, keys of my templates in the cache differs from lookup keys on one leading slash.
For example, I have "dojo/text!./templates/Address.html" in my widget. It present with key url:/app/view/templates/Address.html in the cache but is searched like url:app/view/templates/Address.html, causing cache miss and xhr request.
With additional slash in dojo/text.js (line 183 for version 1.9.1) it seemingly works (line would looks like requireCacheUrl = "url:/" + url).
Not sure what kind of bugs this "fix" could introduce. So, it probably worth to report this issue to dojo folks.
UPD: Well, I see you've already reported this issue. Here is the link: https://bugs.dojotoolkit.org/ticket/17458.
UPD: Don't use hack described above. It was only attempt to narrow issue. Real issue in my project was with packages and baseUrl settings. Initially I created my project based on https://github.com/csnover/dojo-boilerplate. Then fixed it as in neonstalwart's sample.
This sounds like https://bugs.dojotoolkit.org/ticket/17141. If it is, you just need to update to Dojo 1.9.1.

Custom build for Dojo 1.7.1 with Layer and CSS Optimization

With Dojo 1.6.x it was quite easy to create a custom build. In the end I only needed to include a dojo.js file, my application layer file and an optimized css file with all the styles. Simple and easy.
But with Dojo 1.7.x I don't get it. My goal is just to include an opmtimized dojo.js file, my application layer file with all my widgets and stuff and an optmized css file.
Here is my profile.js
var profile = {
releaseDir: "./release",
basePath: "..",
action: "release",
cssOptimize: "comments",
mini: true,
optimize: "closure",
layerOptimize: "closure",
stripConsole: "all",
selectorEngine: "acme",
packages:[
{
name: "dojo",
location: "./sources/dojo"
},
{
name: "dijit",
location: "./sources/dijit"
},
{
name: "dojox",
location: "./sources/dojox"
}
],
layers: {
"dojo/dojo": {
name: "myDojo.js",
include: [ "dojo/dojo" ],
boot: true,
dependencies: [ "dojo/parser", "dojo/data/ItemFileReadStore", "dijit/themes/tundra", "dijit/Dialog", "dijit/form/Form", "dijit/form/Button", "dijit/form/CheckBox", "dijit/form/ComboBox", "dijit/form/DateTextBox", "dijit/form/FilteringSelect", "dijit/form/NumberSpinner", "dijit/form/Textarea", "dijit/form/TextBox", "dijit/form/TimeTextBox", "dijit/form/ValidationTextBox", "dijit/layout/ContentPane", "dijit/layout/TabContainer", "dijit/Tooltip", "dojox/widget/ColorPicker" ]
}
},
resourceTags: {
amd: function (filename, mid) {
return /\.js$/.test(filename);
}
}
};
When I run the build a release is created. I found the dojo.js which has the size of about 580 KB uncompressed. But I did not fond my application file and the compressed css file with all styles.
What am I doing wrong?
Thanks, Ralf
Your layer specification seems to be incorrect. Try this instead:
layers: {
"dojo/myDojo": {
include: [ "dojo/parser", "dojo/data/ItemFileReadStore",
"dijit/themes/tundra", "dijit/Dialog", "dijit/form/Form",
"dijit/form/Button", "dijit/form/CheckBox",
"dijit/form/ComboBox", "dijit/form/DateTextBox",
"dijit/form/FilteringSelect", "dijit/form/NumberSpinner",
"dijit/form/Textarea", "dijit/form/TextBox",
"dijit/form/TimeTextBox", "dijit/form/ValidationTextBox",
"dijit/layout/ContentPane", "dijit/layout/TabContainer",
"dijit/Tooltip", "dojox/widget/ColorPicker"
],
boot: true
}
},
References
http://dojotoolkit.org/reference-guide/1.7/build/
http://dojotoolkit.org/documentation/tutorials/1.7/build/