No test report files were found in Hudson with NUnit - unit-testing

I could successfully run NUnit in Hudson using batch command and it generates the TestResult.xml in the NUnit installation directory. But Hudson generates the error that
No NUnit test report files were found. Configuration error?
after the correct path is given to the xml file under Publish NUnit test result report. Should the xml file be in the project's workspace or is there any other configuration to make this work???
I referred to the Problem Publishing NUnit Testing Result Reports with Hudson
too but no answer was found.

After trying in different ways to solve the above problem, I could solve it by placing the TestResult.xml file in the workspace folder for the project in Hudson s jobs folder and giving only TestResult.xml under Publish NUnit test result report instead of giving the absolute path to the xml file.
Hope this may help someone...

I found that using double quotes is not working.
Example:
Test report XMLs = nunit-result.xml (works!)
Test report XMLs = "nunit-result.xml" (do NOT work)
There is a SMALL chance that this is your problem...

Related

UWP unit testing with Jenkins

It is not really an issue I would like to ask today but I search best practices to unit testing a UWP application with Jenkins.
First, I created a Unit Test App for my main application and, on my development machine, I generated appx package using MSBuild command line tool. It works well and I can also execute unit tests with the VS Test console.
Then I tried to automate it with Jenkins on my build server. But the MSBuild command does not work. I have no issue but it produces no output. After lot of research (without any success), my question is the following:
What is the best way to make UWP unit testing with a result dashboard using Jenkins?
Thanks
In order to make MSBuild work correctly, I had to create two subsequent build steps, one with command line argument:
-t:restore
in order to make it restore all the nuget dependencies, and a second one with command line arguments:
/t:Rebuild /p:Configuration=Release
/p:UapAppxPackageBuildMode=StoreUpload
for the real release compilation and the creation of both sideload and storeUpload msix files

How to get JaCoCo instrumentation code coverage while executing webApplication on server

I want to get a code coverage report (or tracking controller flow of an application) when I execute the some scenario on live application i.e. while application running in server, so I have refereed the official Jacoco documentation and did as fallows
As a first step, I tried to bind the jacoco agent to the application running port
export MAVEN_OPTS=-javaagent:/home/user/.m2/repository/org/jacoco/org.jacoco.agent/0.7.4.201502262128/org.jacoco.agent-0.7.4.201502262128-runtime.jar=includes=*,output=tcpserver,port=6300,address=*,dumponexit=true
then I have started the server and executed some test scenarios by that time I have also requested jacoco to get the dump by executing following command
mvn jacoco:dump
when I executed the above command first it showed like java.net.BindException: Address already in use then it printed as fallows
[INFO] Connecting to localhost/127.0.0.1:6300
When I stopped the server jacoco.exec has been generated to a known location there after I have requested to generate a jacoco report by executing the following command
mvn jacoco:report
Finally I got the report but when I open the index.html page it showed 0% coverage.
you can reproduce my difficulty with following sample Spring MVC project
please download Spring MVC project from https://github.com/mkyong/spring4-mvc-ajax-example from the Github
In my case I am using WildFly 8.x server to run the project and add the following dependencies to downloaded projects pom.xml
maven-surefire-plugin of 2.19.1 version
jacoco-maven-plugin of 0.7.4.201502262128 version
junit of 4.12 version
Then clean and build the project by using following commands.
mvn clean install
once it is done get the .war file from target folder of the project and keep inside the wildfly serverwildfly-9.0.0.CR2/standalone/deployments/ folder
Once the .war is deployed go to /wildfly-9.0.0.CR2/bin/ folder and execute/run the ./standalone.sh file or depending on platform you can run the.dat/.sh file
once project is deployed and server is started execute the following command
mvn jacoco:dump
it displayed like listening/connected to localhost/127.0.0.1:6300
Then execute some test scenarios and execute following maven command
mvn jaococ:report
Finally report will be generated with 0% coverage!.
Downloaded project can be found in MKyong website
Please help me to get the code coverage report and any help will be appreciated.
JaCoCo requires the exact same class files for report generation that were used at execution time, so
if report is completely empty, then classes were not provided
if report contains classes but their coverage is 0%, then they don't match classes that were used at runtime - this is described along with other related information in JaCoCo documentation on page http://www.jacoco.org/jacoco/trunk/doc/classids.html
and in either case check existence of warnings in log.
Update for updated question
Here is what I did:
downloaded and unpacked JaCoCo 0.7.9 into /tmp/jacoco/jacoco-0.7.9
downloaded and unpacked Wildfly 9.0.0.CR2 into /tmp/jacoco/wildfly-9.0.0.CR2
cloned https://github.com/mkyong/spring4-mvc-ajax-example into /tmp/jacoco/spring4-mvc-ajax-example and built as mvn verify
copied /tmp/jacoco/spring4-mvc-ajax-example/spring4-mvc-maven-ajax-example-1.0-SNAPSHOT.war into /tmp/jacoco/wildfly-9.0.0.CR2/standalone/deployments
Wildfly started as JAVA_OPTS=-javaagent:/tmp/jacoco/jacoco-0.7.9/lib/jacocoagent.jar=output=tcpserver ./standalone.sh and got enough time to deploy application
in directory /tmp/jacoco/spring4-mvc-ajax-example executed mvn org.jacoco:jacoco-maven-plugin:0.7.9:dump org.jacoco:jacoco-maven-plugin:0.7.9:report (note that version of used agent matches version of jacoco-maven-plugin) so that it created /tmp/jacoco/spring4-mvc-ajax-example/jacoco.exec and report /tmp/jacoco/spring4-mvc-ajax-example/site/jacoco:
opened http://localhost:8080/spring4-mvc-maven-ajax-example-1.0-SNAPSHOT/ and did some actions
executed mvn org.jacoco:jacoco-maven-plugin:0.7.9:dump org.jacoco:jacoco-maven-plugin:0.7.9:report again to get an updated report:

Configure SpecRun in TeamCity

I have some specflow test and I have configured it to Run in Teamcity using NUnit Runner. But currently
I have a requiremenet to run all the test using SpecRun Runner in TeamCity.
I am quite clueless as how to configure to run test using specrunner in Teamcity as I didnt find a option
in the Runner dropdown in Teamcity to select SpecRun.
Can anyone please help me in giving some info on configuring specrun in Teamcity.
Thanks
I suspect that you will have to use a generic command line step in order to run SpecRun.exe to run the tests. As long as you use the command line switch /buildserver:teamcity then SpecRun should output the correct messages to allow integration with TeamCity so that it shows the tests as passed or failed.
Timothy, Teamcity Runner Dropdown does not provide a Specrun Option.
What can be done is write a .bat file to run runtests.cmd, i.e
"C:\Users\username\Documents\Visual Studio 2015\Projects\Blah_Blah_Project\Project_Name\runtests.cmd".
Now, save this .bat file in any C drive location. Select "Command Line" from TeamCity Dropdown, click on Advanced Option. Mention the Working Directory(path to .bat file) and Command Executable (name of the bat file).
I am assuming here, you have downloaded Specrun.Specflow plugin using nuget manager in Visual Studio and configured Default.srprofile file and successfully fired the specflow testcases from command line using runtests.cmd.

Jenkins/Hudson and Gallio Unit-Testing Integration

I am struggling to get it to run my tests. In the Execute Windows batch command on Jenkins, I have put the following command:
C:\Program Files (x86)\Gallio\bin\Gallio.Echo.exe" /report-type:Html /verbosity:quiet “Project.Tests\bin\Release\*.Tests.dll
It doesn't like the *.Tests.dll bit, in that it says:
Cannot find directory containing file pattern Project.Tests\bin\Release\*.Tests.dll.
My next question would be if I change the report-type to xml, is it straight forward to get my unit test reports published in hudson?
many thanks.
The following extract runs Gallio tests in a Windows Batch command build step (Download Gallio from)
"c:\Program Files\Gallio\bin\Gallio.Echo.exe" %WORKSPACE%\YourTestPro\bin\Debug\YourTestPro.dll /report-directory:%WORKSPACE%\TestResults /report-type:Xml /working-directory:%WORKSPACE%
To setup Jenkins report generation
Add the Gallio Jenkins plugin to your Jenkins installation to publish the report
In your project under the Add post project action build steps add the xUnit test result report
Select the Add button that appears in the xUnit action step
Select Gallio - N/A
Add TestResults/*.xml to the Gallio N/A Pattern
You can use Maven plugin for .Net: http://docs.codehaus.org/display/SONAR/.Net+plugin
It takes care of Gallio command line generation: you can provide a filter :
<visual.test.project.pattern>*.Tests</visual.test.project.pattern>
<gallio.filter>Category:UnitTests</gallio.filter>
Then, the Maven plugin generates the command line.
You also have a lot of advantages using maven: integration with Partcover/NCover, stylecop/fxcop/gendarme, etc.
OR you can make a MSBuild script for that:
<itemGroup>
<TestsDll Include="**\bin\$(Configuration)\*.Tests.dll" />
</itemGroup>
<Exec Command="Gallio.Echo.exe #(TestsDll, ' ')"/>
I am pretty sure you want to look into using the Gallio Plugin instead of using the execute windows batch command.

double unit test reporting with hudson and maven

I have a maven2 project in hudson and when the cobertura reporting plugin runs, it causes the unit tests to show that they have run twice. I don't mind them running multiple times, but the trend graph shows twice as many tests as we actually are running. Is there a way to make sure the graph only shows them once?
thanks,
Jeff
This is a known bug. Just wait for it to be fixed.
The workaround I use (works in Hudson 1.391) is to configure cobertura in separate Maven profile and run it in a Hudson job as a post-build step.
Mode detailed instructions:
Add cobertura to your project pom in a special profile (so it won't run while default lifecycle) and configure it to create report in xml format.
Install "Hudson M2 Extra Steps Plugin"
Configure your Hudson job as Maven 2 project
In your job configuration in the "Build" section configure usual clean/install goals
In "Build Environment" section select "Configure M2 Extra Build Steps" and add Maven post-build step. Configure it to run "cobertura:cobertura -P your_cobertura_profile_name"
In "Post-build Actions" select "Publish Cobertura Coverage Report" and configure proper xml report pattern (default should work just fine)
I had the same problem recently when I was running maven goals test and emma:emma on the same job. emma seems to have rerun all tests thus doubling the results. When I removed goal test my unit tests still got executed but test results went back to normal. Could be the same with cobertura.