I'm trying to configure a GitHub action:
My action contains the job for running the unit test by collecting code coverage. As I see in the log:
Test Run Successful.
Total tests: 336
Passed: 336
Total time: 14.0930 Seconds
Calculating coverage result...
Generating report 'TestResults/coverage.netcoreapp2.1.info'
Nonetheless, after these lines log contains an error message:
/home/runner/.nuget/packages/coverlet.msbuild/2.9.0/build/coverlet.msbuild.targets(31,5): error : Module test path not found [/home/runner/work/ObservableComputations/ObservableComputations/src/ObservableComputations.Test/ObservableComputations.Test.csproj]
The job is failed.
I tried to run
dotnet test --no-build --filter Name~Casting --verbosity normal /p:CollectCoverage=true /p:CoverletOutput=TestResults/ /p:CoverletOutputFormat=lcov
at my local machine (MS Windows) and didn't get this error.
Any help is greatly appreciated.
The reason was in -no-build parameter for dotnet test. It seems coverage collecting requires dotnet test to build itself.
Related
I have a .NET Framework project that is being built on a self hosted Windows build agent.
There is a step to run tests, and that step needs to provide code coverage reports and stats.
When i try using "dotnet test" the step runs and the tests complete, the .coverage files are also generated. When i check the build summary after it's complete i see the standard test results and report, and also the code coverage tab. The code coverage tab has a download link to get the file. There is no code coverage report. There is also a link to "Setup Code Coverage" on the initial build summary screen.
Why is there no code coverage report? and why is the "Setup Code Coverage" link still visible?
This is incredibly frustrating! I must be missing something incredibly obvious, but the docs suggest what i have done is correct.
Using VSTest task rather that dotnet tests results in the same outcome, but runs far slower.
displayName: dotnet test
inputs:
command: test
arguments: '--configuration $(BuildConfiguration) --collect:"Code Coverage"'
workingDirectory: '$(Build.SourcesDirectory)\src'```
I eventually achieved this by using Hugh Lin's answer for help and modifying for my own purposes.
We have Coverlet as a reference in the project, and ReportGenerator installed into Azure DevOps, so that made this a little easier.
I found that we had an issue with a SOAP API reference that was causing the huge performance issues with generating a report. Once I filtered that out with a "classfilter"the process became more manageable. I also found that without the "disable.coverage.autogenerate" variable the "PublishCodeCoverageResults" task will take forever and likely fail as it tries to do the "ReportGenerator" step itself by without the "classfilters". It does this because the "ReportGenerator" is built into the "PublishCodeCoverageResults" step now, but due to having no filters it doesn't work for this scenario.
This is running against a .NET Framework project so there were a few adjustments to the projects needed to ensure "dotnet test" works successfully.
variables:
disable.coverage.autogenerate: 'true'
- task: DotNetCoreCLI#2
displayName: dotnet test
inputs:
command: test
publishTestResults: true
arguments: '/p:CollectCoverage=true /p:CoverletOutputFormat=cobertura --no-restore'
workingDirectory: '$(Build.SourcesDirectory)\src'
configuration: "$(buildConfiguration)"
- task: reportgenerator#4
inputs:
reports: '$(Build.SourcesDirectory)\src\*.UnitTests\coverage.cobertura.xml'
targetdir: '$(Common.TestResultsDirectory)/CoverageReport/'
classfilters: '-NAMESPACE*'
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)\src\*.UnitTests\coverage.cobertura.xml'
reportDirectory: '$(Common.TestResultsDirectory)/CoverageReport/'
By default, the code coverage for the dotnet test task is output to a .codecoverage file, which Azure DevOps does not know how to interpret and only provides as a downloadable file. Code coverage Tab only supports code coverage data in Jacoco or Cobertura formats. So the result of the *.coverage file can not be shown by tables and graphs.
If you want more detailed code coverage report, you need to use coverlet in .Net framework by install the tool during the pipeline and then generate the report. For example in PowerShell script:
dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12
dotnet tool install coverlet.console --tool-path . --version 1.4.1
mkdir .\reports
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*test*.dll" }
$coverlet = "$pwd\coverlet.exe"
& $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml"} |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }
Then add Publish code coverage task:
For details, you can refer to the case mentioned in the comment and this ticket.
Currently, I am using pants to build and test our python2.7 packages. We started noticing the following error during run goal after pytests goal is completed. I tried adding constraints.txt file to exclude the dependencies its complaining about, but the error wouldn't stop. Is there a way to exclude these dependencies during the pants build step?
./pants test.pytest --coverage=auto tests/python/abcd/test/xyz
[pytest]
[cache]
No cached artifacts for 1 target.
Invalidated 1 target.
[run]
Failed to execute PEX file, missing compatible dependencies for:
importlib-metadata
zipp
No .coverage file was found! Skipping coverage reporting.
tests/python/abcd/test/xyz ..... SUCCESS
FAILURE
Can you try increasing the verbosity with:
PEX_VERBOSE=9 ./pants test.pytest --coverage=auto tests/python/abcd/test/xyz
and then respond here?
This error can be caused by a couple of things. Are you executing the PEX on the same operating system that the PEX was built on? Do you also include platforms=[...] in your python_binary function in your BUILD file?
I'm trying to run a single test class in a Java Play project but fails misserably.
If I try to run
testOnly my.app.TheClassTest
from within sbt (as described in JavaTest and sbt test) I get this result:
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for test:testOnly
My only suspicion is that the message "No tests to run for test:testOnly" does not include the name of the class I try to test.
If I try to run it from command line
sbt testOnly "my.app.TheClassTest"
It runs all the tests and then I get the following error:
[error] Expected ID character
[error] Not a valid command: net (similar: set, new, inspect)
[error] Expected project ID
[error] Expected configuration
[error] Expected ':' (if selecting a configuration)
[error] Expected key
[error] Not a valid key: my (similar: test, name, assets)
[error] my.app.TheClassTest
I've tried all kinds of variations such as testOnly TheClassTest, test-only my.app.TheClassTest, test:testOnly etc. with only minor variations in result. Using testOnly within sbt I can write whatever I feel like and still always get the same response.
Running all tests work fine.
Is there at least a way to get a more
From sbt you can try to get the autocompletion for the command:
sbt:my-project> testOnly <tab>
When running from command line (so outside the sbt prompt), for example in continuous integration, or other scripts etc., then you take the command in quotes including the argument, like this:
sbt "testOnly my.package.TheClassTest"
Otherwise testOnly does not seem to receive the argument.
From sbt you can try to get the autocompletion for the command:
sbt:my-project> testOnly <tab>
It should display the lists of tests classes that are available. This may not work with the oldest versions of sbt.
If you see no classes, try to run test:compile before to compile your test classes.
For sbt 1.5.2, this worked:
sbt "project dbm; testOnly my.package.TestClass"
I am trying to get gitlab code coverage parsing working. The server is a local instance of Gitlab 10.4.1-ee. The code coverage tool is lcov via a slightly modified version of this cmake file.
I've entered the regex into the CI Settings as well as in the gitlab ci file to no avail. From what I understand the code coverage will not even attempt to parse if this is not supplied. It did work on one job (out of hundreds) and never again (not sure why). I have supplied both the output and the regex as inputs into http://rubular.com and it seems to parse correctly. I've also fooled around with various iterations of including the single quotes or not or the slashes or not in the regex also to no avail. I can't see where we can get any debugging output or something to show that this step is actually performed.
The project's CI/CD Settings > Test coverage parsing entry:
\bOverall\D+(\d+[.]\d\%)
The job's relevant .gitlab-ci.yml
coverage:
stage: build
image: <redacted>:stable
script:
- mkdir build
- cd build
- cmake -DCMAKE_BUILD_TYPE=Debug .. && make coverage
coverage: '/(?m)\bOverall\D+(\d+[.]\d\%)/'
artifacts:
paths:
- build/coverage/
The job's relevant output:
Overall coverage rate:
lines......: 95.2% (749 of 787 lines)
functions..: 96.5% (110 of 114 functions)
Open ./coverage/index.html in your browser to view the coverage report.
[100%] Built target coverage
Uploading artifacts...
build/coverage/: found 63 matching files
Uploading artifacts to coordinator... ok id=20671 responseStatus=201 Created token=kRnB--qX
Job succeeded
Turns out gitlab's coverage parser is not multi-line. The following regex ended up working lines[\.]+\: (\d+\.\d+)\%. My ci file coverage line ended up being:
coverage: '/lines[\.]+\: (\d+\.\d+)\%/'
I'm trying to use coverage with Django, but I seem to be getting incorrect results. My app is named "stats" and I have this test:
class ListSchoolsTest(TestCase):
def test_initial_list(self):
self.client.login(username='skeezy', password='skeezy')
resp = self.client.get("/stats/list_schools/")
self.assertEqual(resp.status_code, 200)
On the command line, I run:
coverage run --source="." manage.py test stats
And the test passes. All my views are currently in stats/views.py
But when I run "coverage report", I get this line:
Name Stmts Miss Cover
----------------------------------------
<snip>
stats/views 110 110 0%
Any idea what I am (not) doing that would cause coverage to report all lines missed in stats/views.py, even though it would have to be hit in order for the test to pass? (just as a belt-and-suspenders, I put a print statement in my view, and it's definitely getting hit.)
Maybe you have pip installed your app without the -e flag? Then the modules are not imported from your project directory but the path they were installed to and coverage thinks those are different files.