I can't seem to figure out how to configure log output and results folders for Gradle TestNG runs.
First, Gradle picks $project_dir/build/reports/tests by default for HTML report output. How do I change this default location? I tried setting testReportDir in build.gradle to no avail. Specifying a map in call to useTestNG(), e.g.
test {
if (runTest) {
// enable TestNG support (default is JUnit)
useTestNG {
outputDirectory = file("$buildDir/test-output")
}
}
}
does not work as well.
Second, I tried using TestNG's Reporter for some additional logging, e.g:
#Test
public void testMethod() {
parseFile();
Reporter.log("File parsed");
Assert.assertTrue(...);
}
But I can't seem to find the logs! Can someone please help?
The testReportDir property is read-only. You'll need to set Project.testReportDirName. You should be able to enable test logging like so.
test {
testLogging.showStandardStreams = true
}
Related
I am using TFS 2015 (update 2), C++, Google test and Sonarqube 5.6 (with Cxx community plugin).
I am able to import the coverage, compute duplication, create issues using cppcheck but the number of tests is not importing in sonarqube.
I need to generate a Junit-like XML file using <test executable> --gtest_output=xml:<filename> but in TFS (vNext), I use the VSTestTask which uses vstest.console.exe to run my *Test.exe and there seems to be no way to output as xml (it defaults to .trx).
Has anyone managed to correctly import GTest test metrics into sonarqube? Is a XSLT to transform from trx to xunit the only way...?
May be i need to properly fill in the sonar.cxx.vstest.reportsPaths but the filename of the trx is dynamically set by the vstest.console.exe...
Thanks,
Jon
I know this topic is a little bit old but I got the same problem than you to import gtest test report into SonarQube.
I've finally found a converter that transforms a gtest test report to the Generic Format. It supports also junit xml format but I did not test by myself. The original script was made by another guy, bloodle, but I forked his repository to migrate to Python 3. All the thanks go to him.
The simplest way is converting the test result to XML format. After that you just used the default import functionality.To achieve this, use CoverageCoverter.exe with below code.
class Program
{
static int Main(string[] args)
{
if ( args.Length != 2)
{
Console.WriteLine("Coverage Convert - reads VStest binary code coverage data, and outputs it in XML format.");
Console.WriteLine("Usage: ConverageConvert <sourcefile> <destinationfile>");
return 1;
}
CoverageInfo info;
string path;
try
{
path = System.IO.Path.GetDirectoryName(args[0]);
info = CoverageInfo.CreateFromFile(args[0], new string[] { path }, new string[] { });
}
catch (Exception e)
{
Console.WriteLine("Error opening coverage data: {0}",e.Message);
return 1;
}
CoverageDS data = info.BuildDataSet();
try
{
data.WriteXml(args[1]);
}
catch (Exception e)
{
Console.WriteLine("Error writing to output file: {0}", e.Message);
return 1;
}
return 0;
}
}
More detail info and ways please refer Publishing vstest results? & MSTest Plugin
I just put **/TestResults/*.trx in Visual Studio Test Reports Paths (sonar.cxx.vstest.reportsPaths) and now it is being loaded correctly... go figure.
I am using log4j in my application.
If I want to turn on logging for a package I can simple do the below [in my log4j.properties file]:
log4j.logger.com.myorg.somepackage= DEBUG
This will cause log4j to log any messages from "com.myorg.somepackage" to my root logger.
My problem is, how do I stop logging from a package if I use plugins like maven shade?
For example, let's say you have package "com.myorg.somepackage" which is relocated (via maven shade plugin) to "com.someotherorg.dependency.com.myorg.somepackage".
If I wanted to set the level to warn I know I could do the below:
log4j.logger.com.someotherorg.dependency.com.myorg.somepackage= WARN
However, in my case, the dependency is shaded for multiple projects and I dont want to have to:
log4j.logger.com.someotherorg.dependency.com.myorg.somepackage= WARN
log4j.logger.com.someotherorg1.dependency.com.myorg.somepackage= WARN
log4j.logger.com.someotherorg2.dependency.com.myorg.somepackage= WARN
...etc
So how can I have log4j ignore "com.myorg.somepackage" regardless of where it lies in the package name? Is there some sort of REGEX I don't know about for this?
I would like to do something along the lines of:
log4j.logger.*.com.myorg.somepackage= WARN
but that doesn't work.
I can't find default functionality to do this in log4j, however log4j does allow you to create your own filters.
Here is what I did to resolve it:
import org.apache.log4j.Level;
import org.apache.log4j.spi.Filter;
import org.apache.log4j.spi.LoggingEvent;
public class CustomFilterer extends Filter {
#Override
public int decide(LoggingEvent event) {
if (!event.getLoggerName().contains("com.myorg.somepackage")) {
return Filter.NEUTRAL;
} else {
if (event.getLevel().isGreaterOrEqual(Level.WARN)) {
return Filter.NEUTRAL;
} else {
return Filter.DENY;
}
}
}
}
`
Is it possible to configure system properties dynamically for a Gradle test task? I haven't found a way to make it work.
I am working within my own Gradle plugin, GwtPlugin. The apply() method looks like this:
/** Apply the plugin. */
void apply(Project project) {
project.plugins.apply(JavaPlugin)
project.plugins.apply(WarPlugin)
project.extensions.create("gwt", GwtPluginExtension, project)
project.extensions.create("testSuite", TestSuitePluginExtension, project)
project.convention.plugins.gwt = new GwtPluginConvention(project)
applyGwt(project)
applyTestSuite(project)
}
In the applyTestSuite() method, I create a tasks for my test suites. The definition for the integrationtest task looks like this:
// Run integration tests, assumed to be found in a class suites/IntegrationTestSuite.
project.task("integrationtest", type: org.gradle.api.tasks.testing.Test, dependsOn: project.tasks.buildApplication) {
workingDir = { project.testSuite.getWorkingDir() == null ? project.projectDir : project.testSuite.getWorkingDir() }
scanForTestClasses = false
scanForTestClasses = false
enableAssertions = false
outputs.upToDateWhen { false }
include "suites/IntegrationTestSuite.class"
systemProperty "integration.test.server.wait", project.gwt.getServerWait()
beforeSuite { descriptor ->
if (descriptor.className == "suites.IntegrationTestSuite") {
project.convention.plugins.gwt.rebootDevmode()
}
}
afterSuite { descriptor ->
if (descriptor.className == "suites.IntegrationTestSuite") {
project.convention.plugins.gwt.killDevmode()
}
}
}
I want to get configuration for the integration.test.server.wait system property from project.gwt.getServerWait(). I can't figure out how to make this work, and I'm beginning to think it's not possible.
If I hardcode the system property, everything works as expected:
systemProperty "integration.test.server.wait", 10
The problem seems to be that the system property is set when the task is defined, but my extension doesn't have any values at that point. I can't figure out how to work around this.
For instance, I tried putting the project.gwt.getServerWait() call in a closure, but in that case the system property gets set to a string like:
com.me.gradle.GwtPlugin$_applyTestSuite_closure10_closure42#320de756
I also tried moving the systemProperty line to a doFirst block. In that case, the doFirst block gets a sensible value from my extension (I can print it), but the assignment is apparently too late to influence the test runner.
Is there any way for me to accomplish this? If not, is there another way to pass dynamic configuration to my test runner?
I have found a way to make this work. The trick is to set the system property on the test task sometime later, when the property is certain to be available. The simplest way to make that happen seems to be via a dummy dependency:
project.task("integrationtestconfig") << {
outputs.upToDateWhen { false }
project.tasks.integrationtest.systemProperty("integration.test.server.wait", project.gwt.getServerWait())
}
project.task("integrationtest", type: org.gradle.api.tasks.testing.Test,
dependsOn: project.tasks.buildApplication, project.tasks.integrationtestconfig)
...
It's not the elegant solution I was hoping for, but it does work and it's not too difficult to follow.
What I do is:
ext {
// Sets a sensible default
myProperty project.properties.myProperty
}
task preTest << {
// Computes the property...
project.ext.myProperty = ...
}
task myTest {
...
doFirst {
systemProperty 'myProperty', project.ext.myProperty
}
}
I define a sensible default value in the gradle.properties file:
myProperty = A sensible default value
In a multi module environment, it can be trickier. I then use rootProject.ext.myProperty from the test task.
Best regards,
Damien.
Dunno what version you were using to do this, but this seems the most convenient way to me:
task doSomethingBeforeTest {
doLast {
// some stuff to do
test {
systemProperties['some.property'] = 'prop'
}
}
}
basically, just put that test block in your task and set the property. (this works as of gradle 4.0 - not sure about previous versions, but I imagine it would).
I'm not sure if this works, but have you tried to do it in this way
doLast {
systemProperty "integration.test.server.wait", project.gwt.getServerWait()
}
within your plugin script?
May be this has to do with the phase (configuration, ... ) when the things are resolved in gradle.
Is there an MSTest equivalent to NUnit's Explicit Attribute?
No, the closest you will get is with the [Ignore] attribute.
However, MSTest offers other ways of disabling or enabling tests using Test Lists. Whether you like them or not, Test Lists are the recommended way to select tests in MSTest.
When you want the test only to assert when ran with the debugger (implicitly run manually I assume) then you may find this useful:
if (!System.Diagnostics.Debugger.IsAttached) return;
Add the line above at the beginning of the method marked with [TestMethod].
Then the test is always ran, but nothing is asserted when there is no debugger attached.
So when you want to run it manually, do it in debug mode.
I am using this helper:
public static class TestUtilities
{
public static void CheckDeveloper()
{
var _ =
Environment.GetEnvironmentVariable("DEVELOPER") ??
throw new AssertInconclusiveException("DEVELOPER environment variable is not found.");
}
}
Use it at the beginning of the tests you want. The test will only run if the DEVELOPER environment variable is set. In this case, the rest of the tests will be executed correctly and the dotnet test command will return a successful result.
How do I setup logging in a grails unit-test?
When I try log.info or log.debug, the .txt output files are empty, even after I tried adding a console appender. What's going on here?
This might help, it's taken from the 1.2 release notes
By default, grails does not show the
output from your tests. You can make
it do so by passing -echoOut and/or
-echoErr to test-app:
grails test-app -echoOut -echoErr
If you extend GrailsUnitTestCase, you ought to be able to use mockLogging(), but the appenders you set up in Grails config won't be applied in a unit test, which works in isolation from the real framework. They'd only be available in integration tests.
I wasn't able to use mockLogging in grails 1.3.7 with GrailsUnitTestCase either. I think there is probably a bug and it may work in Grails 2.0. Here is what I did to work around it:
class Foo {
String name
Long invokeLogTest(String key) {
if (key.empty) {
log.error("key was sent as empty string")
return 10
}
}
}
void testErrorCase() {
def f = new Foo(name:'jp')
f.metaClass.log = [error:{}]
assert 10 == f.invokeLogTest("")
}