Azure Pipelines build all projects individually before running tests - unit-testing

We have an application that has a number of projects isolated in their own solutions, each with their own UnitTest and IntegrationTest projects within those solutions. What happens on our locally hosted Azure DevOps applications is that the following code forces Azure DevOps to build each project in the solution before running tests. What I'd like to do is to run all tests sequentially on an initial build or at least cut the build time down because on the build server each build takes about a minute or 2 which is the bulk of the time. Since we have XUnit running the tests in say Rider it processes all tests across a solution from multiple projects well within a minute.
Is there a way to cut the build time or is this as good as it gets?
- task: DotNetCoreCLI#2
displayName: Unit Tests
inputs:
command: test
projects: '**/*UnitTest*/*.csproj'
arguments: '--configuration $(BuildConfiguration)'
# run integration tests
- task: DotNetCoreCLI#2
displayName: Integration Tests
inputs:
command: test
projects: '**/*IntegrationTest*/*.csproj'
arguments: '--configuration $(BuildConfiguration)'

What happens on our locally hosted azure devops application is that
the following code below will cause Azure Devops to build each project
in the solution before running tests.
For this issue , you can add --no-build argument to skip the project build on test run.
--no-build:
Doesn't build the test project before running it. This is listed in the Options part of document.
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: test
projects: '**/*UnitTest*/*.csproj'
arguments: '--configuration $(BuildConfiguration) --no-build'
Here is a case with similar issue , you can refer to it.

Related

How to Have 2 Code Coverages in Gitlab Repo Badges

My team has a gitlab repo. It has two parts: an NPM package under projects folder and an angular application under src folder. So there are 2 projects in the angular.json file.
We currently have unit tests with coverage setup in our gitlab pipes. The issue is, since we have 2 projects in this repo, we really need to show the coverage for each project.
I noticed in demo image of the gitlab badges documentation (https://docs.gitlab.com/ee/user/project/badges.html), they have a 'JS Coverage' badge. This seems to be a custom badge (I can't find a list of given badges, but I'm not finding anything for 'JS coverage', so I'm assuming it's custom).
So I think I can do something like that to create 2 custom badges that has the code coverage of each project (1 for 'Pkg Coverage' and 1 for 'App Coverage'). But (TBH) the documentation around creating custom badges isn't great. I need to know how to store this custom value to use in the badge, and how to update in the gitlab pipe.
Does anyone know how to achieve this? If I could just figure out how that example is using 'JS Coverage' (and how to update the value in the pipe), then I could figure out what I need to do for my 2 custom badges. Any tips?
Some details, right now we have a gitlab job like this (it runs unit tests and updates the coverage values. Since 'ng test' runs the tests of both projects 1 by 1, the code coverage of the 1st project is saved to the 'coverage' value):
unit-tests:
stage: test
rules:
# Run unit tests, including when merge requests are merged to default branch (so coverage % is updated)
- when: on_success
image: trion/ng-cli-karma:$ANGULAR_VERSION
before_script:
- *angular-env-setup-script
coverage: '/Statements \W+: (\d+\.\d+)%.*/'
script:
- npm run build:ds-prod
- npm install dist/ds
- ng test --code-coverage --progress false --watch false
artifacts:
expose_as: "Coverage Report"
paths:
- coverage/
tags:
- kubernetes-runner

Pipeline: .NetCoreCLI Test Task throws non-zero error

I have used the following yml command for my .Net 5 API project and xUnit Test Project but it throws error and my pipeline is not getting succeeded. Where did I go wrong?
Note: The pipeline is not getting succeded even if the task executed the test cases and showing 15 test cases passed and 2 test cases are filed.
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
projects: '**/GeniusData.Test/GeniusData.Test.csproj'
displayName: 'Restore Projects'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Test/*.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
displayName: 'Test Project'
You're using the DotNetCoreCLI#2 task which will always fail when tests fail. That's by design: failing tests should break the build.

How to publish 2 sets of tests results on Azure Devop?

We have a git repository that contains .Net code(backend) and since recently typescript code(angular, front-end).
When we added the angular test execution, it appears that the initial .net tests are not correctly published. From my test, it seems that only the last publish is conserved.
How can we keep both of them? It's important because it's used in a PR and we don't want ton miss anything.
Here is how we publish the .Net tests:
- task: PublishTestResults#2
inputs:
testResultsFormat: 'NUnit'
testResultsFiles: '**/TEST-*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)/testResults'
failTaskOnFailedTests: true
testRunTitle: '.Net tests'
buildPlatform: 'Any CPU'
buildConfiguration: 'Debug'
and here how we publish the Angular tests:
- task: PublishTestResults#2
displayName: 'Publish Angular test results'
condition: succeededOrFailed()
inputs:
searchFolder: $(System.DefaultWorkingDirectory)/angular-test/results
testRunTitle: Angular
testResultsFormat: JUnit
testResultsFiles: '**/TESTS*.xml'
How can we ensure both tests results are considered in Azure Devop:
(here you see only the angular tests and its (few) tests. You also see there has been two tests run).
I taught about doing only once the PublishTestResults task, but since they have different format(.net is NUnit, while angular is JUnit), it will not work.

Can I build app in CodeBuild only once, and then run parallel Cypress tests on it using a build-matrix?

I have been following this official documentation on how to get parallel builds running in AWS CodeBuild using a batch matrix. Right now my buildspec.yml is structured like this:
## buildspec.yml
version: 0.2
batch:
fast-fail: false
build-matrix:
dynamic:
env:
variables:
INSTANCES:
- A
WORKERS:
- 1
- 2
phases:
install:
commands:
- npm ci
build:
commands:
- npx cypress run <params>
In this example we run two parallel workers, though IRL we run 11.
This works well for one use case, where we check out the code and run the Cypress tests against the pre-defined URL of one of our test environments. However, we have another use-case where we need to build the application within the CodeBuild container, start a server on localhost, and then run the Cypress tests against that.
One option, of course, is just to build the app 11 times. However, since CodeBuild pricing is by the machine minute, I'd rather build once instead of 11 times. I also don't like the idea of technically testing 11 different builds (albeit all built off the same commit).
What I'm looking for is behavior similar to Docker's multi-stage build functionality, where you can build the app once in one environment, and then copy that artifact to 11 separate envs, where the parallel tests will then run. Is functionality like this going to be possible within CodeBuild itself, or will I have to do something like have two CodeBuild builds and upload the artifact to S3? Any and all ideas welcome.

Use matrix build in Travis only on deploy

Is there any way to only run a matrix build in travis on deploy? Right now we use the same .travis.yml file for test and deploy, and a matrix build (and thus two workers) is triggered in both cases. I can't find a way to only run the build as a matrix in the case in which we are deploying and not when we are running tests (or perhaps to only use a matrix during the deploy process). The main reason I'd like to do this is so that I don't trigger extra builds when PRs are created and I just need the test build to run.
I also couldn't find a simple way we could run a single build for npm install/npm test and then spin off two separate workers/a matrix for the "deploy" process, which would also solve the problem.
Here's a snip of my current .travis.yml file:
language: node_js
node_js: 4.2.1
env:
global:
- APP_NAME=example
matrix:
- CF_DOMAIN=example1.net CF_TARGET=https://target1.com APP_NAME=${APP_NAME}-1
- CF_DOMAIN=example2.net CF_TARGET=https://target2.com APP_NAME=${APP_NAME}-2
branches:
only:
- master
deploy:
- provider: script
skip_cleanup: true
script: node_modules/.bin/deploy.sh
on:
branch: master
It might also work for us to only run a matrix build on a push hook, but not on a pr.
There was a similar issue posted in GitHub for Travis. It was suggested to use two separate .travis.yml files.
https://github.com/travis-ci/travis-ci/issues/2778