I'm trying to split my build files in my webpack.config.js file, but my vendors file isn't being created at all. The remaining node_modules, which aren't react or moment files end up in the main.js. An example of a file that goes in main.js is ./node_modules/es-abstract. I put in my regex and filename in a regex checker, and it passes the test. I'm not sure what's going on; any help would be greatly appreciated it.
splitChunks: {
cacheGroups: {
moment: {
test: /[\\/]node_modules[\\/]((moment).*)[\\/]/,
name: 'moment',
chunks: 'all'
},
react: {
test: /[\\/]node_modules[\\/]((react).*)[\\/]/,
name: 'react',
chunks: 'all'
},
vendors: {
test: /[\\/]node_modules[\\/]((?!(moment|react)).*)[\\/]/,
name: 'vendors',
chunks: 'all'
}
}
}
According to David Gilbertson, SplitChunks default settings only allows for three chunks. To solve this, these settings will need to be added to split chunks: maxInitialRequests: Infinity,
minSize: 0,
Related
We are trying to optimize our webpack bundling.Now we have a lot of small chunks generated, but we want to group them according to some rules:
Node modules and code should be separated, eg. A chunk is either all node_modules or all code
Each chunk size should be larger then 20KB
I kind of achieve the file size constraints, but the problem in the end is that on the first page load, all the chunks are downloaded at once, I split the file because I want them to be donwloaded only when it is needed, not all at once. The following are the steps I did to get to this point. If you can point me to the right way it would be really appreciated
1.The current situation
Settings:
optimization: {
splitChunks: {
chunks: 'all', // optimize both static and dynamic import
maxAsyncRequests: 5, // for each additional load no more than 5 files at a time
maxInitialRequests: 3, // each entrypoint should not request more then 3 js files
automaticNameDelimiter: '~', // delimeter for automatic naming
automaticNameMaxLength: 30, // length of name
name: true, // let webpack calculate name
cacheGroups: {
vendors: {
test: /[\\/]node_modules[\\/]/,
priority: -10,
enforce: true // seperate vendor from our code
},
default: {
minChunks: 2,
priority: -20,
reuseExistingChunk: true
}
}
},
},
You can see there are lots of small files
2.To understand where the small chunks come from, I force merge the asynchronous chunks into two, one from node_modules and one from code
Setting
optimization: {
splitChunks: {
... same as before ...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
},
async_code: {
chunks: "async",
priority: 10,
name: "async_code",
},
vendors: {
test: /[\\/]node_modules[\\/]/,
priority: -10,
enforce: true // seperate vendor from our code
},
default: {
minChunks: 2,
priority: -20,
reuseExistingChunk: true
}
}
},
},
No more small files, so the small files are asynchronous chunks
3.So now those small files are in my control of those 2 cachegroups, I attempt to break them into smaller files
Setting
optimization: {
splitChunks: {
...same as before...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
maxSize: 200000,
minSize: 200000,
},
async_code: {
chunks: "async",
priority: 10,
name: "async_code",
maxSize: 200000,
minSize: 200000,
},
...same as before...
}
},
},
This look exactly what I wanted. Only one problem is that all these files are loaded when I visit the first page. Which does not happen in the original scenario(1.). I suspect that it's because I force the name into cacheGroup. But If I don't force the name, then small chunks are generated
4.Here is what happen if I don't specify the cachegroup name :(
Setting
optimization: {
splitChunks: {
... same as before ...
cacheGroups: {
async_vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: "async",
priority: 20,
name:"async_vendor",
maxSize: 200000,
minSize: 200000,
},
async_code: {
chunks: "async",
priority: 10,
maxSize: 200000,
minSize: 200000,
},
... same as before ...
}
},
},
Is it possible to solve this problem in splitchunk? Thank you for all your help
First of all, I think you've got wrong understanding of splitChunks.chunks attribute. It helps to optimize the splitting of modules which are referenced in splitChunks.chunks type of chunks. For eg. chunks=async, it will consider the modules referenced from async chunks and split/optimize them.
Coming back to the original question,
Current Situation: All your modules above 20KB(default) are splitted into chunks, irrespective of whether they are referenced from async or initial chunks. There are other constraints as well, but let's hop over them for now.
To understand where the small chunks come from, I force merge the asynchronous chunks into two, one from node_modules and one from code: Now all your modules referenced from async chunks are bundled into vendor and code chunks. This might be because modules from all small chunks(larger than 20KB) were referenced from both async and initial chunks. Could try with chunks: initial, I would expect the same result.
Since entry/initial code has references in async_vendor and async_code chunks, they are correspondingly loaded.
To answer the original question, using 3rd config for initial chunks(chunks:initial) should help.
I am currently using webpack and running a test that goes through node_modules and excludes one specific module.
However, i am struggling with the regular expression as i am trying it on the regex checker, and it works fine. However, on the webpack it fails.
Example: https://regex101.com/r/rO4jD0/34
^node_modules\/(?!example).*
On the webpack itself, i am writing the following:
vendor: {
test: /^[\\/]node_modules[\\/](?!example).*[\\/]/,
// test: /^[\\/]node_modules[\\/](?!example)[\\/].*/,
// test: /^node_modules\/(?!example).*/,
name: 'vendor',
chunks: 'all',
}
The tests are examples of the tests i've run so far but unfortunately all of them failed.
As an FYI - this works fine but it doesnt exclude that one example folder.
test: /[\\/]node_modules[\\/]/,
Is this to split out a large vendor module into its own bundle? Here is how I did that:
vendor: {
name: 'vendor',
test: /[\\/]node_modules[\\/]((?!(example.js|slick-carousel)).*)[\\/]/,
name: 'vendors',
chunks: 'all'
},
v_example: {
test: /[\\/]node_modules[\\/]((example.js).*)[\\/]/,
name: 'v_example',
chunks: 'all'
},
v_slick: {
test: /[\\/]node_modules[\\/]((slick-carousel).*)[\\/]/,
name: 'v_slick',
chunks: 'all'
}
I have file tree like app/components/forms/components/formError/components
I need to write a rule that tests all .scss files that are only inside root components or inside scene folder. How can I do this?
The current rule is next:
{
test: /.*\/(components|scenes)\/.*\.scss$/,
use: [{
loader: 'style-loader',
}, {
loader: 'css-loader',
options: {
modules: true,
localIdentName: '[local]___[hash:base64:5]',
},
}, {
loader: 'sass-loader',
}],
}
but it didn't find required files
You may use
/(components|scenes).*\.scss$/
Details
(components|scenes) - matches the first components or scenes substring
.* - any 0+ chars, as many as possible, up to the
\.scss$ - .scss at the end of the string.
See this regex demo.
Running Karma + Jasmine Tests with RequireJS -- Getting off the ground
Help! . . . _ _ _ . . . SOS!
Currently, I have an exercise project up for getting comfortable with KarmaJS -- and Unit Testing, at large. The broad issue is that I really have no transparent view of what Karma is doing behind the scenes, and I can't seem to find adequate documentation in relevant areas. Without further delay...
Here is my folder structure:
root
|-/lib
|-/[dependencies] (/angular, /angular-mocks, /bootstrap, /etc) # from bower
|-/src
|-/[unreferenced directories] (/js, /css, /views) # not referenced anywhere
|-app.js # sets up angular.module('app', ...)
|-globals.js # may be referenced in RequireJS main file; not used.
|-index.html # loads bootstrap.css and RequireJS main file
|-main.js # .config + require(['app', 'etc'])
|-routeMap.js # sets up a single route
|-test-file.js # *** simple define(function(){ return {...}; })
|-/test
|-/spec
|-test-test-file.js # *** require || define(['test-file'])
|-.bowerrc # { "directory": "lib" }
|-bower.json # standard format
|-karma.conf.js # *** HELP!
|-test-main.js # *** Save Our Souls!!!
karma.conf.js
// Karma configuration
// Generated on Wed Nov 19 2014 15:16:56 GMT-0700 (Mountain Standard Time)
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['jasmine', 'requirejs'],
// list of files / patterns to load in the browser
files: [
//'test/spec/test-test-file.js',
//'lib/**/*.js',
//'src/**/*.js',
//'test/spec/**/*.js',
'test-main.js',
{pattern: 'lib/**/*.js', included: false},
{pattern: 'src/**/*.js', included: false},
{pattern: 'test/spec/*.js', included: true}
],
// list of files to exclude
exclude: [
'lib/**/!(angular|angular-mocks|angular-resource|angular-route|require|text).js',
'lib/**/**/!(jquery|bootstrap).js',
'src/app.js'
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['Chrome'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false
});
};
test-main.js
var allTestFiles = [];
var TEST_REGEXP = /(spec|test)\.js$/i;
var pathToModule = function(path) {
return path.replace(/^\/base\//, '').replace(/\.js$/, '');
};
Object.keys(window.__karma__.files).forEach(function(file) {
if (TEST_REGEXP.test(file)) {
// Normalize paths to RequireJS module names.
allTestFiles.push(pathToModule(file));
}
});
require.config({
// Karma serves files under /base, which is the basePath from your config file
baseUrl: '/base/src',
paths: {
angular: '../lib/angular/angular',
ngRoute: '../lib/angular-route/angular-route',
jquery: '../lib/jQuery/dist/jquery',
bootstrap: '../lib/bootstrap/dist/js/bootstrap',
models: 'models',
controllers: 'controllers',
globals: 'globals',
routeMap: 'routeMap'
},
shim: {
angular: {
exports: 'angular'
},
ngRoute: {
deps: ['angular']
},
jquery: {
exports: '$'
},
bootstrap: {
deps: ['jquery']
}
},
// dynamically load all test files
deps: allTestFiles,
// we have to kickoff jasmine, as it is asynchronous
callback: window.__karma__.start
});
test-test-file.js
console.log('....................');
define(function(){
//console.log('testing test-file', testFile);
describe('Testing testing', function(){
it('should work', function(){
expect(true).toEqual(true);
});
});
});
test-file.js
define('testFile', [], function(){
return function init(sandbox){
var app, application = app = sandbox.app
, globals = sandbox.globals;
return {
some: 'module'
};
};
});
Questions & Descriptions
Key points I would love to hear answers for are
what does { pattern: '...', include: true|false } do?
best way to exclude all the extra stuff inside the bower directories.
what files do I need to include in the test-main.js file?
what files do I need to include in the karma.conf.js file?
what does test-main.js actually do; what's it for?
The times I receive errors & issues is as soon as I wrap my spec in a define(...) call -- event when I give the module an ID -- define('someId', function(){ ... }) -- do I need to return something out of this module, as it is a define call?
Other times, I receive the 'ol ERROR: 'There is no timestamp for /base/src/app.js!'. "Timestamp, of course! How silly of me..." -- what in the world does this mean?! Sometimes I get the infamous "Executed 0 of 0 ERROR" -- I could also use some clarity here, please. Really, I get plenty of ERROR: '...no timestamp...' errors -- and even 404s when it seems I should be pulling that library in with the karma.conf.js files config...???
It even seems that usually when I explicitly tell karma to excludesrc/app.js I still get 404s and errors.
tl;dr
Obviously, I'm a bit of a confused novice about Karma and *DD at large...
I can run test-test-file.js fine when my karma.conf.js files array looks like [ 'test-main.js', 'test/spec/test-test-file.js' ] -- but, still, if I wrap my test in a RequireJS define call I get the "Mismatching anonymous define()" error mentioned above.
It seems that when I add { pattern: '...', include: false } then karma just doesn't add any of my files for the given pattern whatsoever (???).
If someone can even simply direct me toward how to use RequireJS with Karma -- namely so that I can just wrap my tests in a define/require call and pull in the module I want to test... That would be greatly appreciated.
As its somewhat difficult to keep these types of questions short and still provide adequate information, I hope I didn't make it too long.
EDIT
After reading the answer from glepretre and some fiddling on my own, I reconfigured my project as follows:
Moved test-main.js to test/test-main.js,
renamed test-test-file.js to testFileSpec.js -- moved it from test/spec to test/,
karma.conf.js:
...
// list of files / patterns to load in the browser
files: [
{pattern: 'lib/**/*.js', included: false},
{pattern: 'src/**/*.js', included: false},
{pattern: 'test/**/*Spec.js', included: false},
'test/test-main.js'
],
....
test/test-main.js:
/* **************** HOW COME THE DEFAULT (Karma-generated) CONFIGURATION DOES ***NOT WORK???
var allTestFiles = [];
var TEST_REGEXP = /(spec|test)\.js$/i;
var pathToModule = function(path) {
return path.replace(/^\/base\//, '').replace(/\.js$/, '');
};
Object.keys(window.__karma__.files).forEach(function(file) {
if (TEST_REGEXP.test(file)) {
// Normalize paths to RequireJS module names.
allTestFiles.push(pathToModule(file));
}
});
*/
var tests = [];
for (var file in window.__karma__.files) {
if (/Spec\.js$/.test(file)) {
tests.push(file);
}
}
require.config({
// Karma serves files under /base, which is the basePath from your config file
baseUrl: '/base/src',
paths: {},
shim: {},
// dynamically load all test files
//deps: allTestFiles,
//
deps: tests,
// we have to kickoff jasmine, as it is asynchronous
callback: window.__karma__.start
});
I am now running Unit Tests successfully! Special Thanks to glepretre & all other contributors.
Thanks for any insight at all :)
OK, I will try to address each question at a time:
Question 1
what does { pattern: '...', included: true|false } do?
Karma's default behavior is to:
find all files matching the pattern (mandatory property)
watch them for changes (watched option) in order to restart your unit tests to give you live result when you are editing your code (works if only you leave the default autoWatch default value to true).
serve them using its own webserver (served option)
include them in the browser using <script> (included option)
So, in the files array of the karma config you can use default behavior by adding only string patterns:
files: [
// this will match all your JS files
// in the src/ directory and subdirectories
'src/**/*.js'
]
Or use the full object syntax to customize each option:
files: [
{pattern: 'src/**/*.js', watched: true, served: true, included: false}
]
Using requireJS, you DO NOT want them to be included because it will be in conflict with requireJS behavior!
Included. Description: Should the files be included in the browser using <script> tag? Use false if you want to load them manually, eg. using Require.js.
from karma/config/files docs
NB: Pay attention to the order in which you add the files/patterns in the array. It matters! For more understanding, set logLevel: config.LOG_DEBUG in your karma config.
Question 2
what files do I need to include in the karma.conf.js file?
At least all required files for the proper functioning of your components for unit testing.
Basically, all files listed in your define([]) and require() blocks.
Question 3
best way to exclude all the extra stuff inside the bower directories.
What are you trying to do exactly?
Based on what I wrote before, you can see that you can add selectively the files you will need in your tests.
I use to add the pattern '/bower_components/**/*.js' and even '/bower_components/**/*.html' when my bower packages are using templates. I never noticed any significant performance issue if that's what you are worried about... Up to you to define the file patterns you will need.
Question 4 & 5
what does test-main.js actually do; what's it for?
what files do I need to include in the test-main.js file?
The purpose of the test-main.js file is to find and load your test files before starting Karma. It connects the dots between Karma and requireJS
You must choose a convention to name your test files and then define the TEST_REGEXP to match all of them.
The "official" angular style guide and best practices for app structure recommends using the suffix *_test.js.
EDIT: Your regexp is not working because it is defined to catch "spec.js" || "test.js" at the end or your spec file name is ending by "file.js" ;) Please see http://regex101.com/r/bE9tV9/1
One more thing
I hope I was clear enough. You can have a look at our starter app structure for our projects using Angular + Require: angular-requirejs-ready. It's already set up and tested with both Karma and Protractor.
Despite using the Dojo build system, my app is still including a large number of javascript files which I would have hoped to be covered by the build.
Here's my build profile:
var profile = (function(){
return {
basePath: "./",
releaseDir: "release",
action: "release",
selectorEngine: "acme",
cssOptimize: "comments.keepLines",
packages:[{
name: "dojo",
location: "dojo"
},{
name: "dijit",
location: "dijit"
},{
name: "dojox",
location: "dojox"
},{
name: "my",
location: "my"
}],
layers: {
"my/admin": {
include: ['dojo/ready', 'dojo/dom', 'dojo/query', 'dojo/request/xhr', 'my/Form', 'my/Tree/Radio']
}
}
};
})();
The app is still including the following JS files on each request: my/Form.js (even though this is listed in the profile), dojo/fx/Toggler.js, dijit/_base.js, dijit/WidgetSet.js, dijit/_base/focus.js, dijit/_base/place.js, dijit/place.js, dijit/_base/popup.js, dijit/popup.js, dijit/BackgroundIframe.js, dijit/_base/scroll.js, dijit/_base/sniff.js, dijit/_base/typematic.js, dijit/typematic.js, dijit/_base/wai.js, dijit/_base/window.js.
my/Tree/Radio extends dijit/Tree, so I'm assuming a lot of the files above are dijit base files that are being loaded automatically by dijit.Tree. But surely the build tool should resolve dependencies like this and include them in the 'built' file?
I am using Dojo 1.8.3.
In dojo/fx, it dynamically looks up the Toggler with the comment
use indirection so modules not rolled into a build
Not sure why, but if you add dojo/fx/Toggler to the include of your build script, it should not make the additional xhr requests.
EDIT: Apparently dijit/Widget does something similar with dijit/_base, so you will want to add that to the includes as well.
http://trac.dojotoolkit.org/ticket/14262