I am using TFS 2015 (update 2), C++, Google test and Sonarqube 5.6 (with Cxx community plugin).
I am able to import the coverage, compute duplication, create issues using cppcheck but the number of tests is not importing in sonarqube.
I need to generate a Junit-like XML file using <test executable> --gtest_output=xml:<filename> but in TFS (vNext), I use the VSTestTask which uses vstest.console.exe to run my *Test.exe and there seems to be no way to output as xml (it defaults to .trx).
Has anyone managed to correctly import GTest test metrics into sonarqube? Is a XSLT to transform from trx to xunit the only way...?
May be i need to properly fill in the sonar.cxx.vstest.reportsPaths but the filename of the trx is dynamically set by the vstest.console.exe...
Thanks,
Jon
I know this topic is a little bit old but I got the same problem than you to import gtest test report into SonarQube.
I've finally found a converter that transforms a gtest test report to the Generic Format. It supports also junit xml format but I did not test by myself. The original script was made by another guy, bloodle, but I forked his repository to migrate to Python 3. All the thanks go to him.
The simplest way is converting the test result to XML format. After that you just used the default import functionality.To achieve this, use CoverageCoverter.exe with below code.
class Program
{
static int Main(string[] args)
{
if ( args.Length != 2)
{
Console.WriteLine("Coverage Convert - reads VStest binary code coverage data, and outputs it in XML format.");
Console.WriteLine("Usage: ConverageConvert <sourcefile> <destinationfile>");
return 1;
}
CoverageInfo info;
string path;
try
{
path = System.IO.Path.GetDirectoryName(args[0]);
info = CoverageInfo.CreateFromFile(args[0], new string[] { path }, new string[] { });
}
catch (Exception e)
{
Console.WriteLine("Error opening coverage data: {0}",e.Message);
return 1;
}
CoverageDS data = info.BuildDataSet();
try
{
data.WriteXml(args[1]);
}
catch (Exception e)
{
Console.WriteLine("Error writing to output file: {0}", e.Message);
return 1;
}
return 0;
}
}
More detail info and ways please refer Publishing vstest results? & MSTest Plugin
I just put **/TestResults/*.trx in Visual Studio Test Reports Paths (sonar.cxx.vstest.reportsPaths) and now it is being loaded correctly... go figure.
Related
I want to test logic of my build.gradle script.
Excerpt of the script would be:
(...other tasks and methods...)
def readCustomerFile(File file) {
def schema = <prepare schema>
def report = schema.validate(JsonLoader.fromFile(file))
if (!report.success) {
throw new GradleException("File is not valid! " + report.toString())
}
return new groovy.json.JsonSlurper().parse(file)
}
task readFiles {
mustRunAfter 'prepareCustomerProject'
doLast {
if (System.env.CUSTOMER_FILE_OVERRIDE) {
project.ext.customerFileData = readCustomerFile(System.env.CUSTOMER_FILE_OVERRIDE)
}
else if (customerFile.exists()) {
project.ext.customerFileData = readCustomerFile(customerFile)
}
else {
throw new GradleException("Customer File is not provided! It is expected to be in CUSTOMER_FILE_OVERRIDE variable or in ${customerFile}")
}
}
}
(...other tasks and methods...)
I would like to test both method and task itself.
The 'prepareProject' task is quite lengthy in execution, but in 'real' setup it does magic necessary to set properties necessary for not only task above.
For testing I only want to e.g. set run readFiles task and validate results, making sure that either property on project was correctly set or exception was thrown.
I have looked into gradle test kit, but it is not what I need, as I was unable to find anything that would allow me to e.g. inspect project.
I have seen Guide for Testing Gradle Scripts, but this post is quite old and does not address my need / problem. I have also had a look at gradle docs Testing Build Logic with TestKit, but looking GradleRunner does not seem to offer any real inspection or project preparing abilities.
Plus, it would make us use jUnit, effectively adding whole classes structure only for testing purposes. Not clean and hard to maintain.
Googling gradle + test + task and other variations finds tons of ways of running xUnit tests, but that's not what I need here.
Summarizing, what I need is:
test gradle tasks and methods from build.gradle in separation (test kit will run task with all its dependencies, I don't want this)
prepare project before test run (test kit does not seem to allow this)
verify task / method output
Has anyone successfully done this?
Or am I approaching this in a wrong way?
I'm fairly new to gradle, searching for good options to test my build scripts.
I've recently started using testlink test management tool in my project and I'm facing a new problem on bulk updation of test cases in testlink.
This is not a problem for manual test cases but for automation it is tedious to update the result (Pass or Fail) of every single test case that you executed.
I have around 5000+ test cases and 50% of them are automated so when the automated.
So, when the automation script is finished executing 2500+ test cases for a particular release, I need to update the result of all these test cases manuall in testlink as Pass or Fail.
I also tried to link the automation tool with testlink but that didn't worked out.
So, I just want to know if there's any easy way to bulk update the test cases in testlink. May be using some DB queries and all?
I think it's difficult because you should know several data in order to insert the correct information for your project, test plan, build, testsuite, testcase and testcase version. So, you will need all this information.
Anyway, the table to insert the information is "executions" with the following query you can check the needed information:
select * from executions;
Regards, David.
1) you can use the XML-RPC API
2) generate a XML file with Format for importing results, then upload it manually
avoid using direct access to DB via SQL
iam also updating the test link using Selenium webdriver
the code is below :-
public class appFunctions extends Keywords {
// Substitute your Dev Key Here
public static String DEV_KEY= "1eab09b6158d9df31e76142b85253243";
public static String SERVER_URL = "https://testlink.fondsdepotbank.de/testlink/lib/api/xmlrpc/v1/xmlrpc.php";
public static void clearXLResults() throws IOException
{
try
{
int testStepsRow=DriverScript.xlObj.getRowCount("TestSteps");
int controllerRow=DriverScript.xlObj.getRowCount("Controller");
//Clear previous results
for(int i=2;i<=testStepsRow;i++)
{
DriverScript.xlObj.setCellData("TestSteps",DriverScript.testStepsStatusCol, i, "");
}
for(int j=2;j<=controllerRow;j++)
{
DriverScript.xlObj.setCellData("Controller", DriverScript.controllerStatusCol, j, "");
}
}catch(Exception e)
{
e.printStackTrace();
log.writeLog("Unable to clear previous test results in excel");
}
}
public static void updateResultsTestLink() throws IOException
{
try
{
TestLinkAPIClient api=new TestLinkAPIClient(DEV_KEY, SERVER_URL);
String result;
//read controller status
int controllerRow=DriverScript.xlObj.getRowCount("Controller");
for(int k=2;k<=controllerRow;k++)
{
String currentRowStatus=DriverScript.xlObj.getCellData("Controller",DriverScript.controllerStatusCol,k);
String currentRowProject=DriverScript.xlObj.getCellData("Controller",DriverScript.controllerProjectCol,k);
String currentRowPlan=DriverScript.xlObj.getCellData("Controller",DriverScript.controllerPlanCol,k);
String currentRowBuild=DriverScript.xlObj.getCellData("Controller",DriverScript.controllerBuildCol,k);
String currentRowTCID=DriverScript.xlObj.getCellData("Controller",DriverScript.controllerTCIDCol,k);
if(currentRowStatus.equalsIgnoreCase("pass"))
{
result= TestLinkAPIResults.TEST_PASSED;
api.reportTestCaseResult(currentRowProject, currentRowPlan, currentRowTCID, currentRowBuild, null, result);
}
if(currentRowStatus.equalsIgnoreCase("fail"))
{
result= TestLinkAPIResults.TEST_FAILED;
api.reportTestCaseResult(currentRowProject, currentRowPlan, currentRowTCID, currentRowBuild, null, result);
}
}
}catch(Exception e)
{
e.printStackTrace();
log.writeLog("Unable to update results in Testlink");
}
}
I can't seem to figure out how to configure log output and results folders for Gradle TestNG runs.
First, Gradle picks $project_dir/build/reports/tests by default for HTML report output. How do I change this default location? I tried setting testReportDir in build.gradle to no avail. Specifying a map in call to useTestNG(), e.g.
test {
if (runTest) {
// enable TestNG support (default is JUnit)
useTestNG {
outputDirectory = file("$buildDir/test-output")
}
}
}
does not work as well.
Second, I tried using TestNG's Reporter for some additional logging, e.g:
#Test
public void testMethod() {
parseFile();
Reporter.log("File parsed");
Assert.assertTrue(...);
}
But I can't seem to find the logs! Can someone please help?
The testReportDir property is read-only. You'll need to set Project.testReportDirName. You should be able to enable test logging like so.
test {
testLogging.showStandardStreams = true
}
Is it possible to retrieve the results of the Unit tests run using NUnit programmatically? If Yes then How? I have read that we can generate XML file using Nunit Console. Is there any other way around apart from generating and then parsing XML?
Yes, it is possible. You can use current test context to get name of test, status, working directory, etc
[TearDown]
public void TearDown()
{
var context = TestContext.CurrentContext;
// context.Test.Name
// context.Result.Status
}
How do I setup logging in a grails unit-test?
When I try log.info or log.debug, the .txt output files are empty, even after I tried adding a console appender. What's going on here?
This might help, it's taken from the 1.2 release notes
By default, grails does not show the
output from your tests. You can make
it do so by passing -echoOut and/or
-echoErr to test-app:
grails test-app -echoOut -echoErr
If you extend GrailsUnitTestCase, you ought to be able to use mockLogging(), but the appenders you set up in Grails config won't be applied in a unit test, which works in isolation from the real framework. They'd only be available in integration tests.
I wasn't able to use mockLogging in grails 1.3.7 with GrailsUnitTestCase either. I think there is probably a bug and it may work in Grails 2.0. Here is what I did to work around it:
class Foo {
String name
Long invokeLogTest(String key) {
if (key.empty) {
log.error("key was sent as empty string")
return 10
}
}
}
void testErrorCase() {
def f = new Foo(name:'jp')
f.metaClass.log = [error:{}]
assert 10 == f.invokeLogTest("")
}