How to make a NodeJS package build C++/WebAssembly upon install? - c++

How do you structure a NodeJS package, containing wrapped C++ code that's compiled into web assembly, so that when you run npm install <package name>, the compilation step happens?
I have a package mypackage configured so that when I run npm run build in its project directory, C++ code is compiled into web assembly, and this is then bundled with other Javascript for the package.
I'm now trying to use this package from another project, and if I run npm install --save mypackage, it installs the package's Javascript, but doesn't run it's build process, so none of the web assembly is created, resulting in a broken package.

How you do it
In the scriptis section in package.json file, you can add a postinstall script, while will be run each time after a package is installed. You can find more about npm scripts here https://docs.npmjs.com/misc/scripts
inside pacakge.json
...
"scripts": {
...
"postinstall": "npm run build"
}
Should you do it
The only valid reason to make build process happen at the target (consumer) system is if the build is dependent on the operating system or architecture of the target system, or is dependent on some configuration/properties of the target system. If the build is not related then it should be made at publishing time, this way it will be done once for all consumers, saving bandwidth, and time. usually also you save space because the bundled package is less than the source.
If you decide to package the built artifact (your bundled package) then its a good idea to use the prepublish script
"scripts": {
...
// this will make sure that you are always publishing the most updated built artifact, instead of having to manually run build each time you want to publish
"prepublish": "npm run build"
}
Another good idea, is to also exclude the source files from the published package using .npmignore, to actually save space.

Related

Yarn script fails to execute when building in Visual Studio Team Services

I managed to setup Yarn in Angular app and when I run: yarn install
it successfully installs all the packages which makes the app run with no issues.
Inside package.json I have written some scripts to help me out run the specific task (I want node_modules folder to be inside wwwroot).
"scripts": {
"install-dev": "yarn install --modules-folder ./wwwroot/node_modules",
"install-ci": "install --modules-folder ./wwwroot/node_modules",
}
When I have to install packages locally I use yarn run install-dev, which works perfectly.
However, when I need to build the project using Visual Studio Team Services, using Yarn Build and Release Tasks by Geek Learning I pass in run install-ci inside VS Team Services Arguments which fails with the following error:
VS Console.png
What I find weird is that it says Version: 1.1.1001, but under it writes yarn run v0.27.5. It seems like that version v0.27.5 does not support --modules-folder (and anyway why is it running the wrong version?).
Here is how Yarn task is configured in Visual Studio Team Services - VS Yarn task configuration.png.
I am making sure that I am in the correct branch (bower-to-yarn) as well as pointing to the Project Directory - src/MyApp where I have package.json file. Also, it says that the task should run with Version: 1.*.
Any help would be very useful. Thank you!
The reason why I have "install-ci" without yarn is because inside vsts console yarn gets loaded and does not need to be specified again. The same applies to Arguments field inside Visual Studio Team Services run install-ci.

Can't run 'ng' from VSTS agent

I'm trying to run unit tests against our AngularCLI project using our hosted VSTS build agents however it keeps running into trouble when it tries to run 'ng test'.
To resolve this I have tried to make the agent use the ng tool directly by providing the path to the tool. This hasn't worked as it looks like it's trying to run 'ng test' where the tool is rather than in the specified current working directory:
I've also tried to add it as an environment variable in Windows (we're using Windows Server 2012 to host the VSTS agent) and setting the tool in the VSTS agent as just ng however it doesn't appear to be finding the ng tool:
How can I get the VSTS agent to make use of the ng tool to run tests? We have got #angular/cli installed on the server hosting the agent.
The thing is that you won't get angular cli installed on VSTS globally as its build server is not supporting that. But the good thing you not even need cli globally installed on your agent.
All you need is npm run ng build -- prod - this way it will always run the local version. Also this way you won't need to take care of updating your global package at all.
Use npm run ng test to run tests, npm run ng e2e to run protractor. If you need to pass any more params to any of these just use --
As mentioned by #Kuncevic, to use the Angular CLI without installing it globally, you will need to use the npm run command.
To run an Angular build using Azure Devops:
Add an npm task to install dependencies (choose install for the command)
Add another npm task, but choose custom for the command. Then add your command and arguments:
run ng -- build --output-path=dist --configuration=prod
Note how npm is not a part of the command and arguments since this will be provided by the task. Also note how -- separates the command to be run and the arguments to be passed to the command.

WebPack on VSTS Hosted Build

We're using the hosted build agent on VSTS to build and release our ASP.NET Core code to Azure App service.
My question is: can we run WebPack to handle front-end tasks on this hosted build on VSTS or do we have to do it manually before checking the code into our repository?
Update:
I'm utilizing the new ASP.NET Core Build (Preview) template that's available on VSTS -- see below:
Here are the steps -- out of the box:
For VSTS we're working on an extension, currently it's in beta phase, you can ask for a share.
Check the VSTS marketplace.
Check this github repo.
Webpack is definitively not a first class citizen for VS2015 and VSTS. Streamlining webpack for CI/CD has been a real headache in my case, especially as webpack was introduced hastily to solve dreadful performance issues with a large monolithic SPA (ASP.NET 4.6, Kendo, 15,000 files, 2000 folders). To cut short, after trying many scenarios to make sure that freshly rebuilt bundles would end up in IIS and Azure webapp, I did a 2-pass build. The sequence of VSTS tasks is as follows: npm install global, npm install local, npm webpack install local, npm webpack install global, build pass 1, webpack, build pass 2, etc... This works with hosted and private agents, providing you supply the proper path for webpack as webpack is installed in a different location in host and in private (did not find a way to chose the webpack install location for consistency). I scorch everything before starting the build. Also need to do these in VS2015 solution : (1) unload "built" folder, and (2) Add Content Include="Built\StarStar" in project file. The "built" folder contains the bundles and should appear greyed, otherwise more bad surprises and instabilities to deal with...
Build-Pass #2 task in VSTS BUILD allows to collect the fresh bundles generated by Build-Pass #1 and includes them automatically in the package to be published.
Without a second build-pass, collecting the bundles and merging them in the zip package is a nightmare, especially when you have 15,000 files to unzip then rezip (300 ms per file!!). Did not find file-merging capability that I could readily use in VSTS.
I have my hears to the ground listening for someone coming up with a more efficient CI/CD scheme for webpack. In the meanwhile, my 2-pass-build workaround is working flawlessly, but slow indeed.
I anticipate that the advances with ASP.NET core, Angular 2 and webpack will look into solving this elegantly.

build wso2 apim 1.10.0 from source

So far I installed deployment version of wso2 AM. Now I would like to build it from source and try running it instead of the binaries I downloaded from the site.
Based on WSO2 documentation, I understand the steps are:
1) Download the carbon kernel source:
git clone -b 4.4.x https://github.com/wso2/carbon-kernel.git
2) Download the APIM source:
git clone https://github.com/wso2/product-apim
3) Build APIM from source
cd <SOURCE-DIR>\product-apim
mvn clean install
Are these steps sufficient, or am I missing something?
Should I build carbon-kernel in addition to building apim-manager?
On previous stackoverflow question, I read that carbon-kernel is not really necessary, and instead i should download and build carbon-apimgmt. Is this correct?
After I build the sources, how do I "package" all the compiled binaries along with all other necessary artifacts, in order to form an equivalent package to the wso2am-1.10.0.zip which I download from the site? Or is there another way to install and run the built code?
Github projects related to API manger can be found in following locations
apimgt component repo:
https://github.com/wso2/carbon-apimgt
This repository contains org.wso2.carbon.apimgt component related source code.
product repo:
https://github.com/wso2/product-apim
This repository contains all the resources needed to build the product package and intergration tests for the product.
master branch of these repositories are used for current development. (if you open parent pom.xml file you would find SNAPSHOT versions). If you build the default branches you would build the current development version of the api manager. (at this time, 1.10.1-SNAPSHOT). To build already released product you need to build released tag.
Steps to Build API manager 1.10.0
clone product:
git clone https://github.com/wso2/product-apim
Checkout release tag v1.10.0:
git checkout v1.10.0
Build the product:
mvn clean install (or mvn clean install -Dmaven.test.skip=true to skip integration tests)
get the product from
product-apim\modules\distribution\product\target
You do not have to build the 'carbon-apimgt' repository because the component build using that is already released and can be found in the nexus repo.
If you want to build the component (say need to provide a fix for a bug) build the 'v5.0.3' tag from the 'carbon-apimgt' repo.
git clone https://github.com/wso2/product-apim
git checkout v5.0.3
I'm posting the steps I did:
git clone https://github.com/wso2/carbon-appmgt
git clone https://github.com/wso2/product-apim
cd <SRC>/carbon-appmgt
mvn clean install
cd <SRC>/product-apim
mvn clean install
The ZIP file was found in
<SRC>\product-apim\modules\distribution\product\target
It is similar to the ZIP file that you download from the site.

Managing npm package installation for non-web developers

I work on a project where I am the only front-end developer amongst a team of C++ developers. When we build our release variant, I want the C++ developers to run the web build process (npm install, grunt/gulp build which does concat/minification/etc...). In order for that to happen, they have to npm install all the devDependencies.
Is there a way to allow them to quickly install the necessary npm modules without having to re-download them ever time npm install is called? Or make the npm install only go through installation once?
npm link doesn't work since that links to the web application and not the node modules that the web application depends upon.
tar.gz would be possible but that means updating the tar.gz every time a node module gets updated.
Curious what development process others suggest for tacking working in a mixed language environment.
You can checkout node_modules to your git or whatever version control you're using, so they won't be downloaded every time.
Yes, someone will have to update modules once in a while, but some people (including npm itself) do just that.
You can also put a caching proxy server (i.e. sinopia) to download packages from, so downloading would be a bit faster.