I'm a little confused about what Travis CI does. In an OpenSource project I'm contributing to we have as a problem - amongst others - that Travis is currently not passing. Problem is: In development locally every test of the suite passes but one (of two) Travis builds seems to fail and I don't quite understand why. Here's the PR where you'll find Travis' reports: https://github.com/magdalena19/lberg-map/pull/228
This is what .travis.yml looks like:
before_script:
- psql -c 'create database "lberg-map_test";' -U postgres
language: ruby
rvm:
- 2.3.1
addons:
code_climate:
repo_token: <secret>
after_success:
- bundle exec codeclimate-test-reporter
test:
adapter: postgresql
database: travis_ci_test
env:
global:
- secure:
...
Secondly we've observed multiple times Travis CI not passing - where local runs of the test suites did. What we did sometimes was to implement neglectable changes to some part within the code, try again and then it worked. What are possible explanations for this behavior?
I'd be glad if anyone could give me a clue!
thx and regards,
Andi
Related
My team has a gitlab repo. It has two parts: an NPM package under projects folder and an angular application under src folder. So there are 2 projects in the angular.json file.
We currently have unit tests with coverage setup in our gitlab pipes. The issue is, since we have 2 projects in this repo, we really need to show the coverage for each project.
I noticed in demo image of the gitlab badges documentation (https://docs.gitlab.com/ee/user/project/badges.html), they have a 'JS Coverage' badge. This seems to be a custom badge (I can't find a list of given badges, but I'm not finding anything for 'JS coverage', so I'm assuming it's custom).
So I think I can do something like that to create 2 custom badges that has the code coverage of each project (1 for 'Pkg Coverage' and 1 for 'App Coverage'). But (TBH) the documentation around creating custom badges isn't great. I need to know how to store this custom value to use in the badge, and how to update in the gitlab pipe.
Does anyone know how to achieve this? If I could just figure out how that example is using 'JS Coverage' (and how to update the value in the pipe), then I could figure out what I need to do for my 2 custom badges. Any tips?
Some details, right now we have a gitlab job like this (it runs unit tests and updates the coverage values. Since 'ng test' runs the tests of both projects 1 by 1, the code coverage of the 1st project is saved to the 'coverage' value):
unit-tests:
stage: test
rules:
# Run unit tests, including when merge requests are merged to default branch (so coverage % is updated)
- when: on_success
image: trion/ng-cli-karma:$ANGULAR_VERSION
before_script:
- *angular-env-setup-script
coverage: '/Statements \W+: (\d+\.\d+)%.*/'
script:
- npm run build:ds-prod
- npm install dist/ds
- ng test --code-coverage --progress false --watch false
artifacts:
expose_as: "Coverage Report"
paths:
- coverage/
tags:
- kubernetes-runner
I'm trying to understand the GitLab Pipelines and after a few tries I was able to successfully automate my unit tests. Now I'm trying to add the code coverage badge into my project and/or readme file but it always seems to show unknown.
Files:
+ application
+ system
- unit-tests
- tests
UtilTest.php
autoload.php
phpunit
.gitignore
.gitlab-ci.yml
.htaccess
index.php
readme.md
.gitlab-ci.yml:
image: php:5.6
stages:
- test
app:unit-tests:
stage: test
script:
- php ./unit-tests/phpunit --bootstrap ./unit-tests/autoload.php ./unit-tests/tests
coverage: '/Code Coverage: \d+\.\d+/'
On the project's Test coverage parsing section I have this set up:
So I was able to fix this by using PHP 7.2 as the Docker image and installing xdebug on the before_script call.
.gitlab-ci.yml:
image: php:7.2
stages:
- test
before_script:
- pecl install xdebug
- docker-php-ext-enable xdebug
app:unit-tests:
stage: test
script:
- php ./unit-tests/phpunit --bootstrap ./unit-tests/autoload.php ./unit-tests/tests --coverage-text --colors=never
coverage: '/^\s*Lines:\s*\d+.\d+\%/'
I had to use PHP 7.2 because when I tried running pecl install xdebug it said it requires PHP 7. Ideally I would like to use PHP 5.6 because that's what our current server has just so the tests are on similar versions but I'll leave it as it is for now.
I had to add --coverage-text --colors=never on the script call for it to output the numbers. Then on the coverage call I changed it to '/^\s*Lines:\s*\d+.\d+\%/' which I also used under the Test coverage parsing section on the project settings.
And now the code coverage properly shows me my expected values.
Is there any way to only run a matrix build in travis on deploy? Right now we use the same .travis.yml file for test and deploy, and a matrix build (and thus two workers) is triggered in both cases. I can't find a way to only run the build as a matrix in the case in which we are deploying and not when we are running tests (or perhaps to only use a matrix during the deploy process). The main reason I'd like to do this is so that I don't trigger extra builds when PRs are created and I just need the test build to run.
I also couldn't find a simple way we could run a single build for npm install/npm test and then spin off two separate workers/a matrix for the "deploy" process, which would also solve the problem.
Here's a snip of my current .travis.yml file:
language: node_js
node_js: 4.2.1
env:
global:
- APP_NAME=example
matrix:
- CF_DOMAIN=example1.net CF_TARGET=https://target1.com APP_NAME=${APP_NAME}-1
- CF_DOMAIN=example2.net CF_TARGET=https://target2.com APP_NAME=${APP_NAME}-2
branches:
only:
- master
deploy:
- provider: script
skip_cleanup: true
script: node_modules/.bin/deploy.sh
on:
branch: master
It might also work for us to only run a matrix build on a push hook, but not on a pr.
There was a similar issue posted in GitHub for Travis. It was suggested to use two separate .travis.yml files.
https://github.com/travis-ci/travis-ci/issues/2778
I try to connect together our cicleCI with browserstack and run our integration_test and unit tests not only with PhantomJS but on real Firefox and Internet Explorer as well, using Browserstack service.
I try to configure browserstack-cli. I can run the test from circleci via tunnel on Browserstack, but never report back to circleci server.
Could you please share your experience if you already played with this stack? Thank you very much!
The solution is to use BrowserStackLocal and browserstack-cli tools together. The 64bit linux version of BrowserStackLocal builds up the tunnel from circleCI server to Browserstack server. After that we can use browserstack-cli to launch browsers and running tests from testem.
Download BrowserStackLocal
and insert in .browserstack folder in your project.
64bit linux version of BrowserStackLocal: http://www.browserstack.com/local-testing (Binnaries)
Create a script,
which will run and create settings for browserstack-cli. You have to setup global variables in circleCI, and you can keep your access details there secretly. Let's call this file runthis.sh and save in .browserstack folder. This script will run your BrowserStackLocal binary as well, so the tunnel will be exist.
#!/bin/bash
echo "{\"username\":\"`echo $BS_USER`\", \"password\":\"`echo $BS_PASSWORD`\", \"privateKey\": \"`echo $BS_KEY`\", \"apiKey\":\"`echo $BS_KEY`\"}" >> ~/.browserstack/browserstack.json
./.browserstack/BrowserStackLocal $BS_KEY &
CircleCI config
(circle.yml) file mainly depend of your project. We have to copy .browserstack folder in home folder, install bower, browserstack-cli and testem.
An example:
machine:
timezone:
Pacific/Auckland
node:
version: v0.10.28
dependencies:
pre:
- mv ./.browserstack ~/
- sh ~/.browserstack/runthis.sh
post:
- bower install
- npm install browserstack-cli -g
- npm install testem -g
test:
override:
- PATH=$PATH:bin grunt integration_tests_cli; testem ci
- PATH=$PATH:bin grunt tests_cli; testem ci
Testem config:
testem.yml - Most of the part is depend of your project. Important in our case is launchers section.
framework: "qunit"
test_page: "tmp/index.html"
src_files:
- "tmp/assets/application.js"
- "tmp/tests.js"
- "tmp/integration_tests.js"
launchers:
bs_chrome:
command: browserstack launch chrome --attach
protocol: browser
timeout: 300
launch_in_ci:
- "PhantomJS"
- "bs_chrome"
launch_in_dev:
- "Chrome"
- "Firefox"
- "PhantomJS"
parallel: 2
So, if you update your project on github, circleci will launch your test and connect to browserstack and gonna use browsers there...
How can one get the build_number (and other build metadata) from within the after_script command in Travis-CI?
What have been tried already:
The documentation on build configuration says this, in the IRC notification section:
You also have the possibility to customize the message that will be
sent to the channel(s) with a template:
notifications:
irc:
channels:
- "irc.freenode.org#travis"
- "irc.freenode.org#some-other-channel"
template:
- "%{repository} (%{commit}) : %{message} %{foo} "
- "Build details: %{build_url}"
You can interpolate the following variables:
repository: your GitHub repo URL.
build_number: build number.
branch: branch build name.
commit: shorten commit SHA
author: commit author name.
message: travis message to the build.
compare_url: commit change view URL.
build_url: URL of the build detail.
Trying to get this to work within an after_script command as below, did not work at all:
language: java
after_script:
- git commit -a -m "Committed by Travis-CI build number: %{build_number}"
It behaved as if .travis.yml file was absent/invalid (even though it did pass the Travis-CI YAML validation here).
It seems as though this should be doable, but could not find any sample that does this.
Could someone point me in the right direction?
The string replacements you can do for IRC output only work there unfortunately. They're only meant to be used for notifications in general, to customize the output, but are currently only available for IRC.
There's still a way to get the current build number, by accessing the TRAVIS_JOB_ID environment variable. If you change your script to the following line, things should work as expected:
after_success:
- git commit -a -m "Committed by Travis-CI build number: $TRAVIS_JOB_ID"
I use this in my deploy script:
git commit -am "Auto deploy from Travis CI build $TRAVIS_BUILD_ID"
More on Travis CI Documentation.