I am struggling to get it to run my tests. In the Execute Windows batch command on Jenkins, I have put the following command:
C:\Program Files (x86)\Gallio\bin\Gallio.Echo.exe" /report-type:Html /verbosity:quiet “Project.Tests\bin\Release\*.Tests.dll
It doesn't like the *.Tests.dll bit, in that it says:
Cannot find directory containing file pattern Project.Tests\bin\Release\*.Tests.dll.
My next question would be if I change the report-type to xml, is it straight forward to get my unit test reports published in hudson?
many thanks.
The following extract runs Gallio tests in a Windows Batch command build step (Download Gallio from)
"c:\Program Files\Gallio\bin\Gallio.Echo.exe" %WORKSPACE%\YourTestPro\bin\Debug\YourTestPro.dll /report-directory:%WORKSPACE%\TestResults /report-type:Xml /working-directory:%WORKSPACE%
To setup Jenkins report generation
Add the Gallio Jenkins plugin to your Jenkins installation to publish the report
In your project under the Add post project action build steps add the xUnit test result report
Select the Add button that appears in the xUnit action step
Select Gallio - N/A
Add TestResults/*.xml to the Gallio N/A Pattern
You can use Maven plugin for .Net: http://docs.codehaus.org/display/SONAR/.Net+plugin
It takes care of Gallio command line generation: you can provide a filter :
<visual.test.project.pattern>*.Tests</visual.test.project.pattern>
<gallio.filter>Category:UnitTests</gallio.filter>
Then, the Maven plugin generates the command line.
You also have a lot of advantages using maven: integration with Partcover/NCover, stylecop/fxcop/gendarme, etc.
OR you can make a MSBuild script for that:
<itemGroup>
<TestsDll Include="**\bin\$(Configuration)\*.Tests.dll" />
</itemGroup>
<Exec Command="Gallio.Echo.exe #(TestsDll, ' ')"/>
I am pretty sure you want to look into using the Gallio Plugin instead of using the execute windows batch command.
Related
I want to execute the tests from this R#/Rider plugin within a GitHub Action and therefore I need to run the tests from the CLI. Simply runnning dotnet test does exactly nothing but building the project.
How can I run tests of a R#/Rider plugin from the CLI so that I can embed this step into a GitHub Action?
Thanks in advance!
Kind of a bug in our template :) It was missing some references to work with the .NET SDK, more specifically these:
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
Also in this commit: https://github.com/matkoch/resharper-fluentassertions/commit/42fdd20f5c6168402d09b47a7e9e3baa1cc9852c
One remark here: dotnet test must be invoked on the solution file (not the test project itself). Otherwise, there was an error about wrong paths. For now, I think that's acceptable.
I haven't tried with dotnet test, but they're just nunit tests, so can be run with the nunit command line.
It is not really an issue I would like to ask today but I search best practices to unit testing a UWP application with Jenkins.
First, I created a Unit Test App for my main application and, on my development machine, I generated appx package using MSBuild command line tool. It works well and I can also execute unit tests with the VS Test console.
Then I tried to automate it with Jenkins on my build server. But the MSBuild command does not work. I have no issue but it produces no output. After lot of research (without any success), my question is the following:
What is the best way to make UWP unit testing with a result dashboard using Jenkins?
Thanks
In order to make MSBuild work correctly, I had to create two subsequent build steps, one with command line argument:
-t:restore
in order to make it restore all the nuget dependencies, and a second one with command line arguments:
/t:Rebuild /p:Configuration=Release
/p:UapAppxPackageBuildMode=StoreUpload
for the real release compilation and the creation of both sideload and storeUpload msix files
I have some specflow test and I have configured it to Run in Teamcity using NUnit Runner. But currently
I have a requiremenet to run all the test using SpecRun Runner in TeamCity.
I am quite clueless as how to configure to run test using specrunner in Teamcity as I didnt find a option
in the Runner dropdown in Teamcity to select SpecRun.
Can anyone please help me in giving some info on configuring specrun in Teamcity.
Thanks
I suspect that you will have to use a generic command line step in order to run SpecRun.exe to run the tests. As long as you use the command line switch /buildserver:teamcity then SpecRun should output the correct messages to allow integration with TeamCity so that it shows the tests as passed or failed.
Timothy, Teamcity Runner Dropdown does not provide a Specrun Option.
What can be done is write a .bat file to run runtests.cmd, i.e
"C:\Users\username\Documents\Visual Studio 2015\Projects\Blah_Blah_Project\Project_Name\runtests.cmd".
Now, save this .bat file in any C drive location. Select "Command Line" from TeamCity Dropdown, click on Advanced Option. Mention the Working Directory(path to .bat file) and Command Executable (name of the bat file).
I am assuming here, you have downloaded Specrun.Specflow plugin using nuget manager in Visual Studio and configured Default.srprofile file and successfully fired the specflow testcases from command line using runtests.cmd.
I have written some automated test cases in java (selenium IDE)for a project.The project is using ruby on rails. The project was configured in TeamCity. Now I am planning to add these test cases as a build step. How can I achieve this. Which build step should I use.
How do you run them on your local machine?
If you run them from command line you can create build configuation with Command Line runner.If you run them by running JUnit test you can create a build configuration that runs JUnit tests (using some build tools like Ant, Maven, Gradle).
I have a maven2 project in hudson and when the cobertura reporting plugin runs, it causes the unit tests to show that they have run twice. I don't mind them running multiple times, but the trend graph shows twice as many tests as we actually are running. Is there a way to make sure the graph only shows them once?
thanks,
Jeff
This is a known bug. Just wait for it to be fixed.
The workaround I use (works in Hudson 1.391) is to configure cobertura in separate Maven profile and run it in a Hudson job as a post-build step.
Mode detailed instructions:
Add cobertura to your project pom in a special profile (so it won't run while default lifecycle) and configure it to create report in xml format.
Install "Hudson M2 Extra Steps Plugin"
Configure your Hudson job as Maven 2 project
In your job configuration in the "Build" section configure usual clean/install goals
In "Build Environment" section select "Configure M2 Extra Build Steps" and add Maven post-build step. Configure it to run "cobertura:cobertura -P your_cobertura_profile_name"
In "Post-build Actions" select "Publish Cobertura Coverage Report" and configure proper xml report pattern (default should work just fine)
I had the same problem recently when I was running maven goals test and emma:emma on the same job. emma seems to have rerun all tests thus doubling the results. When I removed goal test my unit tests still got executed but test results went back to normal. Could be the same with cobertura.