We are using Visual Studio Team Services for build and deployment of several sites. Some .NET and some are not. All working fine when using the hosted agent. Due to performance issues and long queue times for the hosted agent we need to use our own build agent.
Here comes the problem:
When running the VSTS builds we get an error when running bower:
******************************************************************************
Starting: bower install
******************************************************************************
C:\Program Files (x86)\nodejs\npm.cmd install -g bower
C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\bower -> C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\bower\bin\bower
bower#1.7.9 C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\bower
Not found bower: null
******************************************************************************
Finishing: bower install
******************************************************************************
The problem is that bower is actually in that location:
And here are the actual bower build step:
How can we fix this Not found bower: null error?
There are known issues with tools installed into profile folders it has to do with permissions. It's easier to install the tool from an administrative console with the -g parameter. And ensure that the central NPM version is added to the service or system's %path% environment variable.
Or pass in a specific location by adding additional parameters to the call to npm: npm install --prefix "$(Agent.WorkFolder)" Bower Then specify the same location in the Advanced section of the Bower task.
Bower CLI location: $(Agent.WorkFolder)\node_modules\
(You'd need to check the exact location the package is installed to, I'm slightly guessing at the moment ;)).
Related
I am using expo#43.0.3 (and expo-cli#5.0.3) to manage my react native project and I have to install an npm package from local source:
$ npm install /path/to/mypackage
In my package.json the package is successfully linked via
"dependencies": {
...
"myPackage": "file:../../mypackage",
...
}
I can also confirm the package works when installing to a new plain node project (same node version 14.8.2)
Now when I start expo via expo start and navigate to the app it does not throw any error but only a warning:
› Reloading apps
warn No apps connected. Sending "reload" to all React Native apps failed. Make sure your app is running in the simulator or on a phone connected via USB.
When using the package from registry everything builds, however.
I tried to use the private packages section form the expo docs, but they only describe how to use private packages from registry but not local.
Anything I'm missing here?
edit:
After resetting the expo network adapters it loads the bundle but it now says it can't find the package:
Unable to resolve module myPackage from /home/user/path/to/myPackage/file.js: myPackage could not be found within the project or in these directories:
node_modules
If you are sure the module exists, try these steps:
1. Clear watchman watches: watchman watch-del-all
2. Delete node_modules and run yarn install
3. Reset Metro's cache: yarn start --reset-cache
4. Remove the cache: rm -rf /tmp/metro-*
However, I'm not using watchman and I'm not using yarn and rmoving metro- folders from /tmp did not make a difference.
As it turned out in this issue on GitHub it can be solved via npm pack:
run npm pack inside of your library and then npm install path/to/the/packed/file.tgz from your project
Which worked fine for the setup I described in the question.
Background
I have a CI pipeline for a C++ library I've been developing. So far, I can distribute this lib to Linux and Windows systems. Since I use GitLab to build, test and package my lib, I'd like to have my Windows builds running faster and I have no clue on how to do that.
Currently, I use the following script for my Windows builds:
.windows_template:
tags:
- windows
before_script:
- choco install cmake.install -y --installargs '"ADD_CMAKE_TO_PATH=System"'
- choco install python --pre -y
- choco install git -y
- $env:ChocolateyInstall = Convert-Path "$((Get-Command choco).Path)\..\.."; Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1"; refreshenv
- python -m pip install --upgrade pip
- pip install conan monotonic
The problem
Any build with the script above can take up to 10 minutes; worse: I have two stages, each one taking the same amount of time. This means that my whole CI pipeline will take 20 minutes to finish because of slowness in Windows builds.
Ideal solution
EVERYTHING in my before_script can be cached or stored as an image. I only need some hints on how to do it properly.
Additional information
I use the following tools for my builds:
CMake: to support my building process;
Python3: to test and build packages;
Conan (requires Python3): to support the creation of packages with several features, as well as distribute them;
Git: to download Googletest in CMake configuration step This is already provided in the cookbooks - I might just remove this extra installation step in my before_script;
Googletest (requires Python3): testing library;
Visual Studio DEV Tools: to compile the library This is already in the cookbooks.
Installing packages like this (whether it's OS packages though apt-get install... or pip, or anything else) is generally against best practices for CI/CD jobs because every job that runs will have to do the same thing, costing a lot of time as you run more pipelines, as you've seen already.
A few alternatives are to search for an existing image that has everything you need (possible but not likely with more dependencies), split up your job into pieces that might be solved by an image with just one or two dependencies, or create a custom docker image to use in your jobs. I answered a similar question with an example a few weeks ago here: "Unable to locate package git" when running GitLab CI/CD pipeline
But here's an example Dockerfile with Windows:
# Dockerfile
FROM mcr.microsoft.com/windows
RUN ./install_chocolatey.sh
RUN choco install cmake.install -y --installargs '"ADD_CMAKE_TO_PATH=System"'
RUN choco install python --pre -y
RUN choco install git -y
...
The FROM line says that our new image extends the mcr.microsoft.com/windows base image. You can extend any image you have access to, even if it already extends another image (in fact, that's how most images work: they start with something small, like a base OS installation, then add things needed for that package. PHP for example starts on an Ubuntu image, then installs the necessary PHP packages).
The first RUN line is just an example. I'm not a Windows user and don't have experience installing Chocolatey, but you'd do here whatever you'd normally do to install it locally. The rest are for installing whatever else you need.
Then run
docker build /path/to/dockerfile-dir -t mygroup/mytag:version
The path you supply needs to be the directory that contains the Dockerfile, not the Dockerfile itself. The -t flag sets the image's tag after it's built (though you can do that with a separate command, docker tag too).
Next, you'll have to log into whichever registry you're using (Docker Hub (https://docs.docker.com/docker-hub/repos/), Gitlab Container Registry (https://docs.gitlab.com/ee/user/packages/container_registry/), a private registry your employer may support, or any other option.
docker login my.docker.hub.com
Now you can push the image to the registry:
docker push my.docker.hub.com/mygroup/mytag:version
You'll have to review the information in the docs about telling your Gitlab runner or pipelines how to authenticate with the registry (unless it's Public on Docker Hub or you use the Gitlab Container Registry) https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-an-image-from-a-private-container-registry
Once all that's done, you can use your new image in your CI jobs, and everything we put into the image will be ready to use:
.windows_template:
image: my.docker.hub.com/mygroup/mytag:version
tags:
- windows
...
I'm looking to set up automated builds with Visual Studio Team Services but I keep running into trouble. My build definition:
npm install gulp
npm install --save-dev jshint gulp-jshint ruby
npm install gulp-ruby-sass gulp-autoprefixer gulp-cssnano gulp-concat gulp-uglify gulp-imagemin gulp-notify gulp-rename gulp-livereload gulp-cache del --save-dev
gem install sass
gulp
The build fails when attempting to install the sass gem with "'gem' is not recognized as an internal or external command". If I build without installing the sass gem first, gulp will fail with "'sass' is not recognized as an internal or external command". Anyone have experience with getting sass to work in Visual Studio Team Services?
There seem to be several issue here. First you might need to make you familiar how npm works, whats the meaning of --save-dev and whats the difference between local and globally installed modules.
--save-dev is used to save the package for development purpose, while --save is used to save the package required for the application to run. Both are commands which you run on your development machine and you put the resulting package.json under version control.
On the build server you will just run an npm install which will restore all the packages listed in the package.json.
These is for local modules. You can also install modules globally using the -g flag. This will store them outside of your current project, and binaries will be available in your PATH variable. Modules which you need inside your project (using require) need to be installed locally. Modules you'll call from the shell (eg gulp-cli) need to be installed globally.
Therefore what you need to do:
On your development machine add all local npm modules using npm install with either the --save or --save-dev flag.
Put the resulting package.json file under version control.
On the build server you need to make sure that all required global npm modules are installed.
Call npm install using the VSTS npm task to restore the local npm modules. You won't need to specify which modules need to be installed, since they're already listed in the package.json file.
Call gulp using the VSTS gulp task with the appropriate arguments.
I recently upgraded my app from Ember 1.3 to Ember 2.4.2. However after deploying it to my production environment with ember build --env production I noticed a couple of issues.
The first issue was that each script/stylesheet tag had a integrity attribute attached to it which made it impossible to load those resources. After some investigation I decided to just replace the value for those attributes to be empty. This allowed the app to download the resources but I then noticed an error in the console that said define is not defined on the minified website.js file that Ember creates. I'm unable to replicate this issue locally because the app runs fine. What steps can I additionally take to investigate this issue? Could this be related to some addon that I have installed, or maybe the SRI integrity attribute that I removed?
Edit: I just tried building and deploying a vanilla app with Ember (e.g. ember new testApp and ember build --env production and encountered the same issue. Is there an issue with Ember that I don't know about? I don't think it's my server since I'm able to render a plain index.html just fine.
I prefer to use the latest NodeJS (v5.8) and the latest npm.
You can update your npm with npm install -g npm.
The best way to install NodeJS
I guess, you have the latest ember-cli. You can update it if you are outside of your project:
$ cd ~
$ npm install -g ember-cli
I just cloned your repository.
$ git clone https://github.com/uioporqwerty/website.git
$ cd website
$ npm install && bower install
$ ember server
Got the following error:
➜ website git:(master) ember s
version: 2.4.2
The Broccoli Plugin: [ConfigLoader] failed with:
Error: Attempting to watch missing directory: config
at EventEmitter.Watcher_addWatchDir [as addWatchDir] (/Users/szines/projects/temp/website/node_modules/broccoli-sane-watcher/index.js:90:11)
at /Users/szines/projects/temp/website/node_modules/ember-cli-broccoli/lib/builder.js:95:35
...
Because of a couple of important files are missing from your ember-cli project, I just run ember init
$ ember init
During this process you see a question. You can check suggested changes with 'd', but it would just remove bootstrap and font-awesome, which is fine, because we will reinstall them with ember install.
[?] Overwrite bower.json? (Yndh) Y
Now we remove all the previous bower and npm package, temp folder and dist folder.
$ rm -rf bower_components node_modules tmp dist
$ npm install && bower install
Install ember-bootstrap and ember-font-awesome:
$ ember install ember-bootstrap
$ ember install ember-font-awesome
Launch the server:
$ ember server
Your app working like a charm. Open localhost:4200 in your browser.
You can build the production:
$ ember build --prod
And all the files will be in /dist folder.
Or run the production version with the server:
$ ember server --prod
Enjoy your new Ember! :)
Update:
You can use ember-cli-update for updating your Ember app:
$ npx ember-cli-update
$ npx ember-cli-update --run-codemods
Source: Update Ember.js app
Zoltan's answer was very helpful in being a part of the solution to my problem. After fixing my local website according to Zoltan's answer I noticed some issues.
First, there was the following error with Ember-Cli 2.4.2:
Could not find module ember-data/-private\system\references\record imported from ember-data/-private/system/references
This issue has been resolved in the master branch of the Github repo for ember-cli but it hasn't made its way to the npm repo. So what I did was uninstall ember-cli with npm remove -g ember-cli and then followed the development instructions on the github repo https://github.com/ember-cli/ember-cli. This fixed that issue.
Additionally, I noticed that my nginx server was not serving js and css files; I tested this by creating a index.html with test.css and test.js and noticed that the server wasn't dishing out those files. Instead of trying to find the issue I noticed my nginx was a bit out of date on my RPI, it was at version 1.6.x, so I thought this would be a good time to update it; I updated it to 1.9.7 by first removing the old version then running this script https://gist.github.com/MattWilcox/402e2e8aa2e1c132ee24. After that I had nginx 1.9.7 but I did somehow have apache2 installed which caused its own headaches with recognizing the default website so I removed it.
Finally, since my settings were overridden I restored /etc/nginx from an rsync backup I have of my pi by just copying over that directory.
Then boom, everything worked. It took a while but everything seems to be working great at https://www.uioporqwerty.com and the qualsys lab report is working fine too https://www.ssllabs.com/ssltest/analyze.html?d=www.uioporqwerty.com so all my settings carried over correctly :)
Glad to have Ember-Cli working correctly.
I get the below error message while I switch between my ember applications:
version: 2.4.2
Could not start watchman; falling back to NodeWatcher for file system events.
Visit ember-cli.com for more info.
Livereload server on http://localhost:49153
Could not serve on http://localhost:4200. It is either in use or you do not have permission.
Here are the packages that I have setup on my machine:
npm --version
2.14.20
bower --version
1.7.7
ember --version
2.4.2
node: 4.4.0
os: linux x64
Ember-cli (starting from release 0.1.3) requires watchman or some other service (e.g. NodeWatcher) for file watching.
You can find this info in the ember-cli release 0.1.3 changelog:
https://github.com/ember-cli/ember-cli/releases/tag/v0.1.3
Although I believe you could use your Ember app without watchman, could you confirm that you already have watchman installed?
If not, here's what works fine for me in Ubuntu 64bit:
// installing watchman from source
git clone https://github.com/facebook/watchman.git
cd watchman
./autogen.sh
./configure
make
sudo make install
if you still have problems after installing try also resetting your watchman configuration:
watchman watch-del-all
watchman watch-del /home/myproject // your project folder goes here...
watchman watch-project /home/myproject // your project folder goes
full installation instructions regarding watchman can also be found here
https://facebook.github.io/watchman/docs/install.html