Code coverage is always unknown in GitLab - unit-testing

I'm trying to understand the GitLab Pipelines and after a few tries I was able to successfully automate my unit tests. Now I'm trying to add the code coverage badge into my project and/or readme file but it always seems to show unknown.
Files:
+ application
+ system
- unit-tests
- tests
UtilTest.php
autoload.php
phpunit
.gitignore
.gitlab-ci.yml
.htaccess
index.php
readme.md
.gitlab-ci.yml:
image: php:5.6
stages:
- test
app:unit-tests:
stage: test
script:
- php ./unit-tests/phpunit --bootstrap ./unit-tests/autoload.php ./unit-tests/tests
coverage: '/Code Coverage: \d+\.\d+/'
On the project's Test coverage parsing section I have this set up:

So I was able to fix this by using PHP 7.2 as the Docker image and installing xdebug on the before_script call.
.gitlab-ci.yml:
image: php:7.2
stages:
- test
before_script:
- pecl install xdebug
- docker-php-ext-enable xdebug
app:unit-tests:
stage: test
script:
- php ./unit-tests/phpunit --bootstrap ./unit-tests/autoload.php ./unit-tests/tests --coverage-text --colors=never
coverage: '/^\s*Lines:\s*\d+.\d+\%/'
I had to use PHP 7.2 because when I tried running pecl install xdebug it said it requires PHP 7. Ideally I would like to use PHP 5.6 because that's what our current server has just so the tests are on similar versions but I'll leave it as it is for now.
I had to add --coverage-text --colors=never on the script call for it to output the numbers. Then on the coverage call I changed it to '/^\s*Lines:\s*\d+.\d+\%/' which I also used under the Test coverage parsing section on the project settings.
And now the code coverage properly shows me my expected values.

Related

How to Have 2 Code Coverages in Gitlab Repo Badges

My team has a gitlab repo. It has two parts: an NPM package under projects folder and an angular application under src folder. So there are 2 projects in the angular.json file.
We currently have unit tests with coverage setup in our gitlab pipes. The issue is, since we have 2 projects in this repo, we really need to show the coverage for each project.
I noticed in demo image of the gitlab badges documentation (https://docs.gitlab.com/ee/user/project/badges.html), they have a 'JS Coverage' badge. This seems to be a custom badge (I can't find a list of given badges, but I'm not finding anything for 'JS coverage', so I'm assuming it's custom).
So I think I can do something like that to create 2 custom badges that has the code coverage of each project (1 for 'Pkg Coverage' and 1 for 'App Coverage'). But (TBH) the documentation around creating custom badges isn't great. I need to know how to store this custom value to use in the badge, and how to update in the gitlab pipe.
Does anyone know how to achieve this? If I could just figure out how that example is using 'JS Coverage' (and how to update the value in the pipe), then I could figure out what I need to do for my 2 custom badges. Any tips?
Some details, right now we have a gitlab job like this (it runs unit tests and updates the coverage values. Since 'ng test' runs the tests of both projects 1 by 1, the code coverage of the 1st project is saved to the 'coverage' value):
unit-tests:
stage: test
rules:
# Run unit tests, including when merge requests are merged to default branch (so coverage % is updated)
- when: on_success
image: trion/ng-cli-karma:$ANGULAR_VERSION
before_script:
- *angular-env-setup-script
coverage: '/Statements \W+: (\d+\.\d+)%.*/'
script:
- npm run build:ds-prod
- npm install dist/ds
- ng test --code-coverage --progress false --watch false
artifacts:
expose_as: "Coverage Report"
paths:
- coverage/
tags:
- kubernetes-runner

Integrating Selenium with Gitlab CI

I have created an automated selenium test script which works perfectly fine.
My task now is to set up Gitlab CI and try to automatically run this selenium script when I make a push to git.
Is it possible to make the selenium script automatically execute and inform the user if the script runs successfully or it fails?
Thank you
How to automatically run Automation Tests on Gitlab Ci with Selenium and specflow with a .net Project ?
If this is something ,you are looking for then .
Here is the core part which is to setup the gitlab-ci.yml file :
Here is how the sample gitlab-ci.yml should look :
image: please give your own docker which can download .net stuff
variables:
DOCKER_DRIVER: overlay2
SOURCE_CODE_DIRECTORY: 'src'
BINARIES_DIRECTORY: 'bin'
OBJECTS_DIRECTORY: 'obj'
NUGET_PACKAGES_DIRECTORY: '.nuget'
stages:
- Build
- Test
before_script:
- 'dotnet restore ${SOURCE_CODE_DIRECTORY}/TestProject.sln --packages ${NUGET_PACKAGES_DIRECTORY}'
Build:
stage: Build
script:
- 'dotnet build $SOURCE_CODE_DIRECTORY/TestProject.sln --no-restore'
except:
- tags
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
expire_in: 2 hr
Test:
stage: Test
services:
- selenium/standalone-chrome:latest
script:
- 'export MSBUILDSINGLELOADCONTEXT=1'
- 'export selenium_remote_url=http://selenium__standalone-chrome:4444/wd/hub/'
- 'export PATH=$PATH:${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- 'dotnet test $SOURCE_CODE_DIRECTORY/ExpressTestProject.sln --no-restore'
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
Thats it .When you set up your project with this .git-lab-ci.yml ,90 % of your job is done .
The tests will run automatically in Gitlab ,whenever you commit something in your source tree or Tfs.
Thanks

One of two Travis CI builds failing: how can that be?

I'm a little confused about what Travis CI does. In an OpenSource project I'm contributing to we have as a problem - amongst others - that Travis is currently not passing. Problem is: In development locally every test of the suite passes but one (of two) Travis builds seems to fail and I don't quite understand why. Here's the PR where you'll find Travis' reports: https://github.com/magdalena19/lberg-map/pull/228
This is what .travis.yml looks like:
before_script:
- psql -c 'create database "lberg-map_test";' -U postgres
language: ruby
rvm:
- 2.3.1
addons:
code_climate:
repo_token: <secret>
after_success:
- bundle exec codeclimate-test-reporter
test:
adapter: postgresql
database: travis_ci_test
env:
global:
- secure:
...
Secondly we've observed multiple times Travis CI not passing - where local runs of the test suites did. What we did sometimes was to implement neglectable changes to some part within the code, try again and then it worked. What are possible explanations for this behavior?
I'd be glad if anyone could give me a clue!
thx and regards,
Andi

How do i configure .gitlab-ci.yml to build my django project

I have succesfully installed and configured gitlab and gitlab-ci-multirunners. What i want to do now is configure the .gitlab-ci.yml file so that it runs python manage.py test and succeed if the tests pass and fail otherwise.
What would be the best approach to achieve this?
test_app:
script: python manage.py test
Something like the above should do it. Note, the exit code of the script command determines if the build passes or fails. If you need multiple lines of shell scripts you can use a yaml list:
test_app:
script:
- python dosetup.py
- python manage.py test
test_app is the name of the build job while the script property defines the shell commands to run for the given build job. When using multiple script lines, each line is run as a separate command. If any of the lines return exit code != 0 the build will fail.
By default a build job in .gitlab-ci.yml runs as test. If you need multiple types of build steps you can define them as such:
types:
- build
- test
build_app:
type: build
script: echo Building!
test_app:
type: test
script: python manage.py test
More info in the official documentation: https://docs.gitlab.com/ce/ci/yaml/

Ember.js - CircleCI - BrowserStack

I try to connect together our cicleCI with browserstack and run our integration_test and unit tests not only with PhantomJS but on real Firefox and Internet Explorer as well, using Browserstack service.
I try to configure browserstack-cli. I can run the test from circleci via tunnel on Browserstack, but never report back to circleci server.
Could you please share your experience if you already played with this stack? Thank you very much!
The solution is to use BrowserStackLocal and browserstack-cli tools together. The 64bit linux version of BrowserStackLocal builds up the tunnel from circleCI server to Browserstack server. After that we can use browserstack-cli to launch browsers and running tests from testem.
Download BrowserStackLocal
and insert in .browserstack folder in your project.
64bit linux version of BrowserStackLocal: http://www.browserstack.com/local-testing (Binnaries)
Create a script,
which will run and create settings for browserstack-cli. You have to setup global variables in circleCI, and you can keep your access details there secretly. Let's call this file runthis.sh and save in .browserstack folder. This script will run your BrowserStackLocal binary as well, so the tunnel will be exist.
#!/bin/bash
echo "{\"username\":\"`echo $BS_USER`\", \"password\":\"`echo $BS_PASSWORD`\", \"privateKey\": \"`echo $BS_KEY`\", \"apiKey\":\"`echo $BS_KEY`\"}" >> ~/.browserstack/browserstack.json
./.browserstack/BrowserStackLocal $BS_KEY &
CircleCI config
(circle.yml) file mainly depend of your project. We have to copy .browserstack folder in home folder, install bower, browserstack-cli and testem.
An example:
machine:
timezone:
Pacific/Auckland
node:
version: v0.10.28
dependencies:
pre:
- mv ./.browserstack ~/
- sh ~/.browserstack/runthis.sh
post:
- bower install
- npm install browserstack-cli -g
- npm install testem -g
test:
override:
- PATH=$PATH:bin grunt integration_tests_cli; testem ci
- PATH=$PATH:bin grunt tests_cli; testem ci
Testem config:
testem.yml - Most of the part is depend of your project. Important in our case is launchers section.
framework: "qunit"
test_page: "tmp/index.html"
src_files:
- "tmp/assets/application.js"
- "tmp/tests.js"
- "tmp/integration_tests.js"
launchers:
bs_chrome:
command: browserstack launch chrome --attach
protocol: browser
timeout: 300
launch_in_ci:
- "PhantomJS"
- "bs_chrome"
launch_in_dev:
- "Chrome"
- "Firefox"
- "PhantomJS"
parallel: 2
So, if you update your project on github, circleci will launch your test and connect to browserstack and gonna use browsers there...